shinys000114/linux
1
0
Fork 0
mirror of https://github.com/hardkernel/linux.git synced 2026-06-07 03:15:31 +09:00
Code Issues Packages Projects Releases Wiki Activity

Merge 148842c98a ("Merge tag 'x86-apic-2020-12-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip") into android-mainline

Browse Source 
Steps on the way to 5.11-rc1

Resolves merge issue in :
	include/linux/sched.h

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I9d5af6793ed59d1c4fd23e91cc41864c4a6cf277
This commit is contained in:
Greg Kroah-Hartman
2020-12-16 20:42:42 +01:00
parent 03c69872d6 148842c98a
commit cbc135c55e
136 changed files with 2037 additions and 2389 deletions
Download Patch File Download Diff File Expand all files Collapse all files

96
Documentation/driver-api/io-mapping.rst
View File

@@ -20,78 +20,72 @@ A mapping object is created during driver initialization using::
mappable, while 'size' indicates how large a mapping region to
enable. Both are in bytes.
This _wc variant provides a mapping which may only be used
with the io_mapping_map_atomic_wc or io_mapping_map_wc.
This _wc variant provides a mapping which may only be used with
io_mapping_map_atomic_wc(), io_mapping_map_local_wc() or
io_mapping_map_wc().
With this mapping object, individual pages can be mapped either atomically
or not, depending on the necessary scheduling environment. Of course, atomic
maps are more efficient::
With this mapping object, individual pages can be mapped either temporarily
or long term, depending on the requirements. Of course, temporary maps are
more efficient. They come in two flavours::
void *io_mapping_map_local_wc(struct io_mapping *mapping,
unsigned long offset)
void *io_mapping_map_atomic_wc(struct io_mapping *mapping,
unsigned long offset)
'offset' is the offset within the defined mapping region.
Accessing addresses beyond the region specified in the
creation function yields undefined results. Using an offset
which is not page aligned yields an undefined result. The
return value points to a single page in CPU address space.
'offset' is the offset within the defined mapping region. Accessing
addresses beyond the region specified in the creation function yields
undefined results. Using an offset which is not page aligned yields an
undefined result. The return value points to a single page in CPU address
space.
This _wc variant returns a write-combining map to the
page and may only be used with mappings created by
io_mapping_create_wc
This _wc variant returns a write-combining map to the page and may only be
used with mappings created by io_mapping_create_wc()
Note that the task may not sleep while holding this page
mapped.
Temporary mappings are only valid in the context of the caller. The mapping
is not guaranteed to be globaly visible.
::
io_mapping_map_local_wc() has a side effect on X86 32bit as it disables
migration to make the mapping code work. No caller can rely on this side
effect.
io_mapping_map_atomic_wc() has the side effect of disabling preemption and
pagefaults. Don't use in new code. Use io_mapping_map_local_wc() instead.
Nested mappings need to be undone in reverse order because the mapping
code uses a stack for keeping track of them::
addr1 = io_mapping_map_local_wc(map1, offset1);
addr2 = io_mapping_map_local_wc(map2, offset2);
...
io_mapping_unmap_local(addr2);
io_mapping_unmap_local(addr1);
The mappings are released with::
void io_mapping_unmap_local(void *vaddr)
void io_mapping_unmap_atomic(void *vaddr)
'vaddr' must be the value returned by the last
io_mapping_map_atomic_wc call. This unmaps the specified
page and allows the task to sleep once again.
'vaddr' must be the value returned by the last io_mapping_map_local_wc() or
io_mapping_map_atomic_wc() call. This unmaps the specified mapping and
undoes the side effects of the mapping functions.
If you need to sleep while holding the lock, you can use the non-atomic
variant, although they may be significantly slower.
::
If you need to sleep while holding a mapping, you can use the regular
variant, although this may be significantly slower::
void *io_mapping_map_wc(struct io_mapping *mapping,
unsigned long offset)
This works like io_mapping_map_atomic_wc except it allows
the task to sleep while holding the page mapped.
This works like io_mapping_map_atomic/local_wc() except it has no side
effects and the pointer is globaly visible.
::
The mappings are released with::
void io_mapping_unmap(void *vaddr)
This works like io_mapping_unmap_atomic, except it is used
for pages mapped with io_mapping_map_wc.
Use for pages mapped with io_mapping_map_wc().
At driver close time, the io_mapping object must be freed::
void io_mapping_free(struct io_mapping *mapping)
Current Implementation
======================
The initial implementation of these functions uses existing mapping
mechanisms and so provides only an abstraction layer and no new
functionality.
On 64-bit processors, io_mapping_create_wc calls ioremap_wc for the whole
range, creating a permanent kernel-visible mapping to the resource. The
map_atomic and map functions add the requested offset to the base of the
virtual address returned by ioremap_wc.
On 32-bit processors with HIGHMEM defined, io_mapping_map_atomic_wc uses
kmap_atomic_pfn to map the specified page in an atomic fashion;
kmap_atomic_pfn isn't really supposed to be used with device pages, but it
provides an efficient mapping for this usage.
On 32-bit processors without HIGHMEM defined, io_mapping_map_atomic_wc and
io_mapping_map_wc both use ioremap_wc, a terribly inefficient function which
performs an IPI to inform all processors about the new mapping. This results
in a significant performance penalty.

15
arch/alpha/include/asm/kmap_types.h
View File

@@ -1,15 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_KMAP_TYPES_H
#define _ASM_KMAP_TYPES_H
/* Dummy header just to define km_type. */
#ifdef CONFIG_DEBUG_HIGHMEM
#define __WITH_KM_FENCE
#endif
#include <asm-generic/kmap_types.h>
#undef __WITH_KM_FENCE
#endif

1
arch/arc/Kconfig
View File

@@ -507,6 +507,7 @@ config LINUX_RAM_BASE
config HIGHMEM
bool "High Memory Support"
select ARCH_DISCONTIGMEM_ENABLE
select KMAP_LOCAL
help
With ARC 2G:2G address split, only upper 2G is directly addressable by
kernel. Enable this to potentially allow access to rest of 2G and PAE

26
arch/arc/include/asm/highmem.h
View File

@@ -9,17 +9,29 @@
#ifdef CONFIG_HIGHMEM
#include <uapi/asm/page.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#define FIXMAP_SIZE PGDIR_SIZE
#define PKMAP_SIZE PGDIR_SIZE
/* start after vmalloc area */
#define FIXMAP_BASE (PAGE_OFFSET - FIXMAP_SIZE - PKMAP_SIZE)
#define FIXMAP_SIZE PGDIR_SIZE /* only 1 PGD worth */
#define KM_TYPE_NR ((FIXMAP_SIZE >> PAGE_SHIFT)/NR_CPUS)
#define FIXMAP_ADDR(nr) (FIXMAP_BASE + ((nr) << PAGE_SHIFT))
#define FIX_KMAP_SLOTS (KM_MAX_IDX * NR_CPUS)
#define FIX_KMAP_BEGIN (0UL)
#define FIX_KMAP_END ((FIX_KMAP_BEGIN + FIX_KMAP_SLOTS) - 1)
#define FIXADDR_TOP (FIXMAP_BASE + (FIX_KMAP_END << PAGE_SHIFT))
/*
* This should be converted to the asm-generic version, but of course this
* is needlessly different from all other architectures. Sigh - tglx
*/
#define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT))
#define __virt_to_fix(x) (((FIXADDR_TOP - ((x) & PAGE_MASK))) >> PAGE_SHIFT)
/* start after fixmap area */
#define PKMAP_BASE (FIXMAP_BASE + FIXMAP_SIZE)
#define PKMAP_SIZE PGDIR_SIZE
#define LAST_PKMAP (PKMAP_SIZE >> PAGE_SHIFT)
#define LAST_PKMAP_MASK (LAST_PKMAP - 1)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
@@ -29,11 +41,13 @@
extern void kmap_init(void);
#define arch_kmap_local_post_unmap(vaddr) \
local_flush_tlb_kernel_range(vaddr, vaddr + PAGE_SIZE)
static inline void flush_cache_kmaps(void)
{
flush_cache_all();
}
#endif
#endif

14
arch/arc/include/asm/kmap_types.h
View File

@@ -1,14 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2015 Synopsys, Inc. (www.synopsys.com)
*/
#ifndef _ASM_KMAP_TYPES_H
#define _ASM_KMAP_TYPES_H
/*
* We primarily need to define KM_TYPE_NR here but that in turn
* is a function of PGDIR_SIZE etc.
* To avoid circular deps issue, put everything in asm/highmem.h
*/
#endif

56
arch/arc/mm/highmem.c
View File

@@ -36,9 +36,8 @@
* This means each only has 1 PGDIR_SIZE worth of kvaddr mappings, which means
* 2M of kvaddr space for typical config (8K page and 11:8:13 traversal split)
*
* - fixmap anyhow needs a limited number of mappings. So 2M kvaddr == 256 PTE
* slots across NR_CPUS would be more than sufficient (generic code defines
* KM_TYPE_NR as 20).
* - The fixed KMAP slots for kmap_local/atomic() require KM_MAX_IDX slots per
* CPU. So the number of CPUs sharing a single PTE page is limited.
*
* - pkmap being preemptible, in theory could do with more than 256 concurrent
* mappings. However, generic pkmap code: map_new_virtual(), doesn't traverse
@@ -47,48 +46,6 @@
*/
extern pte_t * pkmap_page_table;
static pte_t * fixmap_page_table;
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
int idx, cpu_idx;
unsigned long vaddr;
cpu_idx = kmap_atomic_idx_push();
idx = cpu_idx + KM_TYPE_NR * smp_processor_id();
vaddr = FIXMAP_ADDR(idx);
set_pte_at(&init_mm, vaddr, fixmap_page_table + idx,
mk_pte(page, prot));
return (void *)vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kv)
{
unsigned long kvaddr = (unsigned long)kv;
if (kvaddr >= FIXMAP_BASE && kvaddr < (FIXMAP_BASE + FIXMAP_SIZE)) {
/*
* Because preemption is disabled, this vaddr can be associated
* with the current allocated index.
* But in case of multiple live kmap_atomic(), it still relies on
* callers to unmap in right order.
*/
int cpu_idx = kmap_atomic_idx();
int idx = cpu_idx + KM_TYPE_NR * smp_processor_id();
WARN_ON(kvaddr != FIXMAP_ADDR(idx));
pte_clear(&init_mm, kvaddr, fixmap_page_table + idx);
local_flush_tlb_kernel_range(kvaddr, kvaddr + PAGE_SIZE);
kmap_atomic_idx_pop();
}
}
EXPORT_SYMBOL(kunmap_atomic_high);
static noinline pte_t * __init alloc_kmap_pgtable(unsigned long kvaddr)
{
@@ -108,10 +65,9 @@ void __init kmap_init(void)
{
/* Due to recursive include hell, we can't do this in processor.h */
BUILD_BUG_ON(PAGE_OFFSET < (VMALLOC_END + FIXMAP_SIZE + PKMAP_SIZE));
BUILD_BUG_ON(KM_TYPE_NR > PTRS_PER_PTE);
pkmap_page_table = alloc_kmap_pgtable(PKMAP_BASE);
BUILD_BUG_ON(LAST_PKMAP > PTRS_PER_PTE);
fixmap_page_table = alloc_kmap_pgtable(FIXMAP_BASE);
BUILD_BUG_ON(FIX_KMAP_SLOTS > PTRS_PER_PTE);
pkmap_page_table = alloc_kmap_pgtable(PKMAP_BASE);
alloc_kmap_pgtable(FIXMAP_BASE);
}

1
arch/arm/Kconfig
View File

@@ -1499,6 +1499,7 @@ config HAVE_ARCH_PFN_VALID
config HIGHMEM
bool "High Memory Support"
depends on MMU
select KMAP_LOCAL
help
The address space of ARM processors is only 4 Gigabytes large
and it has to accommodate user address space, kernel address

4
arch/arm/include/asm/fixmap.h
View File

@@ -7,14 +7,14 @@
#define FIXADDR_TOP (FIXADDR_END - PAGE_SIZE)
#include <linux/pgtable.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
enum fixed_addresses {
FIX_EARLYCON_MEM_BASE,
__end_of_permanent_fixed_addresses,
FIX_KMAP_BEGIN = __end_of_permanent_fixed_addresses,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * NR_CPUS) - 1,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * NR_CPUS) - 1,
/* Support writing RO kernel text via kprobes, jump labels, etc. */
FIX_TEXT_POKE0,

34
arch/arm/include/asm/highmem.h
View File

@@ -2,7 +2,8 @@
#ifndef _ASM_HIGHMEM_H
#define _ASM_HIGHMEM_H
#include <asm/kmap_types.h>
#include <asm/cachetype.h>
#include <asm/fixmap.h>
#define PKMAP_BASE (PAGE_OFFSET - PMD_SIZE)
#define LAST_PKMAP PTRS_PER_PTE
@@ -46,19 +47,32 @@ extern pte_t *pkmap_page_table;
#ifdef ARCH_NEEDS_KMAP_HIGH_GET
extern void *kmap_high_get(struct page *page);
#else
static inline void *arch_kmap_local_high_get(struct page *page)
{
if (IS_ENABLED(CONFIG_DEBUG_HIGHMEM) && !cache_is_vivt())
return NULL;
return kmap_high_get(page);
}
#define arch_kmap_local_high_get arch_kmap_local_high_get
#else /* ARCH_NEEDS_KMAP_HIGH_GET */
static inline void *kmap_high_get(struct page *page)
{
return NULL;
}
#endif
#endif /* !ARCH_NEEDS_KMAP_HIGH_GET */
/*
* The following functions are already defined by <linux/highmem.h>
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
extern void *kmap_atomic_pfn(unsigned long pfn);
#endif
#define arch_kmap_local_post_map(vaddr, pteval) \
local_flush_tlb_kernel_page(vaddr)
#define arch_kmap_local_pre_unmap(vaddr) \
do { \
if (cache_is_vivt()) \
__cpuc_flush_dcache_area((void *)vaddr, PAGE_SIZE); \
} while (0)
#define arch_kmap_local_post_unmap(vaddr) \
local_flush_tlb_kernel_page(vaddr)
#endif

10
arch/arm/include/asm/kmap_types.h
View File

@@ -1,10 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ARM_KMAP_TYPES_H
#define __ARM_KMAP_TYPES_H
/*
* This is the "bare minimum". AIO seems to require this.
*/
#define KM_TYPE_NR 16
#endif

1
arch/arm/mm/Makefile
View File

@@ -19,7 +19,6 @@ obj-$(CONFIG_MODULES) += proc-syms.o
obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o
obj-$(CONFIG_ALIGNMENT_TRAP) += alignment.o
obj-$(CONFIG_HIGHMEM) += highmem.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
obj-$(CONFIG_ARM_PV_FIXUP) += pv-fixup-asm.o

121
arch/arm/mm/highmem.c
View File

@@ -1,121 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* arch/arm/mm/highmem.c -- ARM highmem support
*
* Author: Nicolas Pitre
* Created: september 8, 2008
* Copyright: Marvell Semiconductors Inc.
*/
#include <linux/module.h>
#include <linux/highmem.h>
#include <linux/interrupt.h>
#include <asm/fixmap.h>
#include <asm/cacheflush.h>
#include <asm/tlbflush.h>
#include "mm.h"
static inline void set_fixmap_pte(int idx, pte_t pte)
{
unsigned long vaddr = __fix_to_virt(idx);
pte_t *ptep = virt_to_kpte(vaddr);
set_pte_ext(ptep, pte, 0);
local_flush_tlb_kernel_page(vaddr);
}
static inline pte_t get_fixmap_pte(unsigned long vaddr)
{
pte_t *ptep = virt_to_kpte(vaddr);
return *ptep;
}
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned int idx;
unsigned long vaddr;
void *kmap;
int type;
#ifdef CONFIG_DEBUG_HIGHMEM
/*
* There is no cache coherency issue when non VIVT, so force the
* dedicated kmap usage for better debugging purposes in that case.
*/
if (!cache_is_vivt())
kmap = NULL;
else
#endif
kmap = kmap_high_get(page);
if (kmap)
return kmap;
type = kmap_atomic_idx_push();
idx = FIX_KMAP_BEGIN + type + KM_TYPE_NR * smp_processor_id();
vaddr = __fix_to_virt(idx);
#ifdef CONFIG_DEBUG_HIGHMEM
/*
* With debugging enabled, kunmap_atomic forces that entry to 0.
* Make sure it was indeed properly unmapped.
*/
BUG_ON(!pte_none(get_fixmap_pte(vaddr)));
#endif
/*
* When debugging is off, kunmap_atomic leaves the previous mapping
* in place, so the contained TLB flush ensures the TLB is updated
* with the new mapping.
*/
set_fixmap_pte(idx, mk_pte(page, prot));
return (void *)vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int idx, type;
if (kvaddr >= (void *)FIXADDR_START) {
type = kmap_atomic_idx();
idx = FIX_KMAP_BEGIN + type + KM_TYPE_NR * smp_processor_id();
if (cache_is_vivt())
__cpuc_flush_dcache_area((void *)vaddr, PAGE_SIZE);
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(vaddr != __fix_to_virt(idx));
set_fixmap_pte(idx, __pte(0));
#else
(void) idx; /* to kill a warning */
#endif
kmap_atomic_idx_pop();
} else if (vaddr >= PKMAP_ADDR(0) && vaddr < PKMAP_ADDR(LAST_PKMAP)) {
/* this address was obtained through kmap_high_get() */
kunmap_high(pte_page(pkmap_page_table[PKMAP_NR(vaddr)]));
}
}
EXPORT_SYMBOL(kunmap_atomic_high);
void *kmap_atomic_pfn(unsigned long pfn)
{
unsigned long vaddr;
int idx, type;
struct page *page = pfn_to_page(pfn);
preempt_disable();
pagefault_disable();
if (!PageHighMem(page))
return page_address(page);
type = kmap_atomic_idx_push();
idx = FIX_KMAP_BEGIN + type + KM_TYPE_NR * smp_processor_id();
vaddr = __fix_to_virt(idx);
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(get_fixmap_pte(vaddr)));
#endif
set_fixmap_pte(idx, pfn_pte(pfn, kmap_prot));
return (void *)vaddr;
}

1
arch/csky/Kconfig
View File

@@ -286,6 +286,7 @@ config NR_CPUS
config HIGHMEM
bool "High Memory Support"
depends on !CPU_CK610
select KMAP_LOCAL
default y
config FORCE_MAX_ZONEORDER

4
arch/csky/include/asm/fixmap.h
View File

@@ -8,7 +8,7 @@
#include <asm/memory.h>
#ifdef CONFIG_HIGHMEM
#include <linux/threads.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#endif
enum fixed_addresses {
@@ -17,7 +17,7 @@ enum fixed_addresses {
#endif
#ifdef CONFIG_HIGHMEM
FIX_KMAP_BEGIN,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * NR_CPUS) - 1,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * NR_CPUS) - 1,
#endif
__end_of_fixed_addresses
};

6
arch/csky/include/asm/highmem.h
View File

@@ -9,7 +9,7 @@
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/uaccess.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#include <asm/cache.h>
/* undef for production */
@@ -32,10 +32,12 @@ extern pte_t *pkmap_page_table;
#define ARCH_HAS_KMAP_FLUSH_TLB
extern void kmap_flush_tlb(unsigned long addr);
extern void *kmap_atomic_pfn(unsigned long pfn);
#define flush_cache_kmaps() do {} while (0)
#define arch_kmap_local_post_map(vaddr, pteval) kmap_flush_tlb(vaddr)
#define arch_kmap_local_post_unmap(vaddr) kmap_flush_tlb(vaddr)
extern void kmap_init(void);
#endif /* __KERNEL__ */

75
arch/csky/mm/highmem.c
View File

@@ -9,8 +9,6 @@
#include <asm/tlbflush.h>
#include <asm/cacheflush.h>
static pte_t *kmap_pte;
unsigned long highstart_pfn, highend_pfn;
void kmap_flush_tlb(unsigned long addr)
@@ -19,67 +17,7 @@ void kmap_flush_tlb(unsigned long addr)
}
EXPORT_SYMBOL(kmap_flush_tlb);
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte - idx)));
#endif
set_pte(kmap_pte-idx, mk_pte(page, prot));
flush_tlb_one((unsigned long)vaddr);
return (void *)vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int idx;
if (vaddr < FIXADDR_START)
return;
#ifdef CONFIG_DEBUG_HIGHMEM
idx = KM_TYPE_NR*smp_processor_id() + kmap_atomic_idx();
BUG_ON(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
pte_clear(&init_mm, vaddr, kmap_pte - idx);
flush_tlb_one(vaddr);
#else
(void) idx; /* to kill a warning */
#endif
kmap_atomic_idx_pop();
}
EXPORT_SYMBOL(kunmap_atomic_high);
/*
* This is the same as kmap_atomic() but can map memory that doesn't
* have a struct page associated with it.
*/
void *kmap_atomic_pfn(unsigned long pfn)
{
unsigned long vaddr;
int idx, type;
pagefault_disable();
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
set_pte(kmap_pte-idx, pfn_pte(pfn, PAGE_KERNEL));
flush_tlb_one(vaddr);
return (void *) vaddr;
}
static void __init kmap_pages_init(void)
void __init kmap_init(void)
{
unsigned long vaddr;
pgd_t *pgd;
@@ -96,14 +34,3 @@ static void __init kmap_pages_init(void)
pte = pte_offset_kernel(pmd, vaddr);
pkmap_page_table = pte;
}
void __init kmap_init(void)
{
unsigned long vaddr;
kmap_pages_init();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN);
kmap_pte = pte_offset_kernel((pmd_t *)pgd_offset_k(vaddr), vaddr);
}

13
arch/ia64/include/asm/kmap_types.h
View File

@@ -1,13 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_IA64_KMAP_TYPES_H
#define _ASM_IA64_KMAP_TYPES_H
#ifdef CONFIG_DEBUG_HIGHMEM
#define __WITH_KM_FENCE
#endif
#include <asm-generic/kmap_types.h>
#undef __WITH_KM_FENCE
#endif /* _ASM_IA64_KMAP_TYPES_H */

1
arch/microblaze/Kconfig
View File

@@ -155,6 +155,7 @@ config XILINX_UNCACHED_SHADOW
config HIGHMEM
bool "High memory support"
depends on MMU
select KMAP_LOCAL
help
The address space of Microblaze processors is only 4 Gigabytes large
and it has to accommodate user address space, kernel address

4
arch/microblaze/include/asm/fixmap.h
View File

@@ -20,7 +20,7 @@
#include <asm/page.h>
#ifdef CONFIG_HIGHMEM
#include <linux/threads.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#endif
#define FIXADDR_TOP ((unsigned long)(-PAGE_SIZE))
@@ -47,7 +47,7 @@ enum fixed_addresses {
FIX_HOLE,
#ifdef CONFIG_HIGHMEM
FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * num_possible_cpus()) - 1,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * num_possible_cpus()) - 1,
#endif
__end_of_fixed_addresses
};

6
arch/microblaze/include/asm/highmem.h
View File

@@ -25,7 +25,6 @@
#include <linux/uaccess.h>
#include <asm/fixmap.h>
extern pte_t *kmap_pte;
extern pte_t *pkmap_page_table;
/*
@@ -52,6 +51,11 @@ extern pte_t *pkmap_page_table;
#define flush_cache_kmaps() { flush_icache(); flush_dcache(); }
#define arch_kmap_local_post_map(vaddr, pteval) \
local_flush_tlb_page(NULL, vaddr);
#define arch_kmap_local_post_unmap(vaddr) \
local_flush_tlb_page(NULL, vaddr);
#endif /* __KERNEL__ */
#endif /* _ASM_HIGHMEM_H */

1
arch/microblaze/mm/Makefile
View File

@@ -6,4 +6,3 @@
obj-y := consistent.o init.o
obj-$(CONFIG_MMU) += pgtable.o mmu_context.o fault.o
obj-$(CONFIG_HIGHMEM) += highmem.o

78
arch/microblaze/mm/highmem.c
View File

@@ -1,78 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* highmem.c: virtual kernel memory mappings for high memory
*
* PowerPC version, stolen from the i386 version.
*
* Used in CONFIG_HIGHMEM systems for memory pages which
* are not addressable by direct kernel virtual addresses.
*
* Copyright (C) 1999 Gerhard Wichert, Siemens AG
* Gerhard.Wichert@pdb.siemens.de
*
*
* Redesigned the x86 32-bit VM architecture to deal with
* up to 16 Terrabyte physical memory. With current x86 CPUs
* we now support up to 64 Gigabytes physical RAM.
*
* Copyright (C) 1999 Ingo Molnar <mingo@redhat.com>
*
* Reworked for PowerPC by various contributors. Moved from
* highmem.h by Benjamin Herrenschmidt (c) 2009 IBM Corp.
*/
#include <linux/export.h>
#include <linux/highmem.h>
/*
* The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
* gives a more generic (and caching) interface. But kmap_atomic can
* be used in IRQ contexts, so in some (very limited) cases we need
* it.
*/
#include <asm/tlbflush.h>
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte-idx)));
#endif
set_pte_at(&init_mm, vaddr, kmap_pte-idx, mk_pte(page, prot));
local_flush_tlb_page(NULL, vaddr);
return (void *) vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int type;
unsigned int idx;
if (vaddr < __fix_to_virt(FIX_KMAP_END))
return;
type = kmap_atomic_idx();
idx = type + KM_TYPE_NR * smp_processor_id();
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
#endif
/*
* force other mappings to Oops if they'll try to access
* this pte without first remap it
*/
pte_clear(&init_mm, vaddr, kmap_pte-idx);
local_flush_tlb_page(NULL, vaddr);
kmap_atomic_idx_pop();
}
EXPORT_SYMBOL(kunmap_atomic_high);

5
arch/microblaze/mm/init.c
View File

@@ -50,16 +50,11 @@ EXPORT_SYMBOL(min_low_pfn);
EXPORT_SYMBOL(max_low_pfn);
#ifdef CONFIG_HIGHMEM
pte_t *kmap_pte;
EXPORT_SYMBOL(kmap_pte);
static void __init highmem_init(void)
{
pr_debug("%x\n", (u32)PKMAP_BASE);
map_page(PKMAP_BASE, 0, 0); /* XXX gross */
pkmap_page_table = virt_to_kpte(PKMAP_BASE);
kmap_pte = virt_to_kpte(__fix_to_virt(FIX_KMAP_BEGIN));
}
static void highmem_setup(void)

1
arch/mips/Kconfig
View File

@@ -2719,6 +2719,7 @@ config WAR_MIPS34K_MISSED_ITLB
config HIGHMEM
bool "High Memory Support"
depends on 32BIT && CPU_SUPPORTS_HIGHMEM && SYS_SUPPORTS_HIGHMEM && !CPU_MIPS32_3_5_EVA
select KMAP_LOCAL
config CPU_SUPPORTS_HIGHMEM
bool

4
arch/mips/include/asm/fixmap.h
View File

@@ -17,7 +17,7 @@
#include <spaces.h>
#ifdef CONFIG_HIGHMEM
#include <linux/threads.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#endif
/*
@@ -52,7 +52,7 @@ enum fixed_addresses {
#ifdef CONFIG_HIGHMEM
/* reserved pte's for temporary kernel mappings */
FIX_KMAP_BEGIN = FIX_CMAP_END + 1,
FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * NR_CPUS) - 1,
#endif
__end_of_fixed_addresses
};

6
arch/mips/include/asm/highmem.h
View File

@@ -24,7 +24,7 @@
#include <linux/interrupt.h>
#include <linux/uaccess.h>
#include <asm/cpu-features.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
/* declarations for highmem.c */
extern unsigned long highstart_pfn, highend_pfn;
@@ -48,11 +48,11 @@ extern pte_t *pkmap_page_table;
#define ARCH_HAS_KMAP_FLUSH_TLB
extern void kmap_flush_tlb(unsigned long addr);
extern void *kmap_atomic_pfn(unsigned long pfn);
#define flush_cache_kmaps() BUG_ON(cpu_has_dc_aliases)
extern void kmap_init(void);
#define arch_kmap_local_post_map(vaddr, pteval) local_flush_tlb_one(vaddr)
#define arch_kmap_local_post_unmap(vaddr) local_flush_tlb_one(vaddr)
#endif /* __KERNEL__ */

13
arch/mips/include/asm/kmap_types.h
View File

@@ -1,13 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_KMAP_TYPES_H
#define _ASM_KMAP_TYPES_H
#ifdef CONFIG_DEBUG_HIGHMEM
#define __WITH_KM_FENCE
#endif
#include <asm-generic/kmap_types.h>
#undef __WITH_KM_FENCE
#endif

77
arch/mips/mm/highmem.c
View File

@@ -8,8 +8,6 @@
#include <asm/fixmap.h>
#include <asm/tlbflush.h>
static pte_t *kmap_pte;
unsigned long highstart_pfn, highend_pfn;
void kmap_flush_tlb(unsigned long addr)
@@ -17,78 +15,3 @@ void kmap_flush_tlb(unsigned long addr)
flush_tlb_one(addr);
}
EXPORT_SYMBOL(kmap_flush_tlb);
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte - idx)));
#endif
set_pte(kmap_pte-idx, mk_pte(page, prot));
local_flush_tlb_one((unsigned long)vaddr);
return (void*) vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int type __maybe_unused;
if (vaddr < FIXADDR_START)
return;
type = kmap_atomic_idx();
#ifdef CONFIG_DEBUG_HIGHMEM
{
int idx = type + KM_TYPE_NR * smp_processor_id();
BUG_ON(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
/*
* force other mappings to Oops if they'll try to access
* this pte without first remap it
*/
pte_clear(&init_mm, vaddr, kmap_pte-idx);
local_flush_tlb_one(vaddr);
}
#endif
kmap_atomic_idx_pop();
}
EXPORT_SYMBOL(kunmap_atomic_high);
/*
* This is the same as kmap_atomic() but can map memory that doesn't
* have a struct page associated with it.
*/
void *kmap_atomic_pfn(unsigned long pfn)
{
unsigned long vaddr;
int idx, type;
preempt_disable();
pagefault_disable();
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
set_pte(kmap_pte-idx, pfn_pte(pfn, PAGE_KERNEL));
flush_tlb_one(vaddr);
return (void*) vaddr;
}
void __init kmap_init(void)
{
unsigned long kmap_vstart;
/* cache the first kmap pte */
kmap_vstart = __fix_to_virt(FIX_KMAP_BEGIN);
kmap_pte = virt_to_kpte(kmap_vstart);
}

4
arch/mips/mm/init.c
View File

@@ -36,7 +36,6 @@
#include <asm/cachectl.h>
#include <asm/cpu.h>
#include <asm/dma.h>
#include <asm/kmap_types.h>
#include <asm/maar.h>
#include <asm/mmu_context.h>
#include <asm/sections.h>
@@ -402,9 +401,6 @@ void __init paging_init(void)
pagetable_init();
#ifdef CONFIG_HIGHMEM
kmap_init();
#endif
#ifdef CONFIG_ZONE_DMA
max_zone_pfns[ZONE_DMA] = MAX_DMA_PFN;
#endif

1
arch/nds32/Kconfig.cpu
View File

@@ -157,6 +157,7 @@ config HW_SUPPORT_UNALIGNMENT_ACCESS
config HIGHMEM
bool "High Memory Support"
depends on MMU && !CPU_CACHE_ALIASING
select KMAP_LOCAL
help
The address space of Andes processors is only 4 Gigabytes large
and it has to accommodate user address space, kernel address

4
arch/nds32/include/asm/fixmap.h
View File

@@ -6,7 +6,7 @@
#ifdef CONFIG_HIGHMEM
#include <linux/threads.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#endif
enum fixed_addresses {
@@ -14,7 +14,7 @@ enum fixed_addresses {
FIX_KMAP_RESERVED,
FIX_KMAP_BEGIN,
#ifdef CONFIG_HIGHMEM
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * NR_CPUS),
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * NR_CPUS) - 1,
#endif
FIX_EARLYCON_MEM_BASE,
__end_of_fixed_addresses

22
arch/nds32/include/asm/highmem.h
View File

@@ -5,7 +5,6 @@
#define _ASM_HIGHMEM_H
#include <asm/proc-fns.h>
#include <asm/kmap_types.h>
#include <asm/fixmap.h>
/*
@@ -45,11 +44,22 @@ extern pte_t *pkmap_page_table;
extern void kmap_init(void);
/*
* The following functions are already defined by <linux/highmem.h>
* when CONFIG_HIGHMEM is not set.
* FIXME: The below looks broken vs. a kmap_atomic() in task context which
* is interupted and another kmap_atomic() happens in interrupt context.
* But what do I know about nds32. -- tglx
*/
#ifdef CONFIG_HIGHMEM
extern void *kmap_atomic_pfn(unsigned long pfn);
#endif
#define arch_kmap_local_post_map(vaddr, pteval) \
do { \
__nds32__tlbop_inv(vaddr); \
__nds32__mtsr_dsb(vaddr, NDS32_SR_TLB_VPN); \
__nds32__tlbop_rwr(pteval); \
__nds32__isb(); \
} while (0)
#define arch_kmap_local_pre_unmap(vaddr) \
do { \
__nds32__tlbop_inv(vaddr); \
__nds32__isb(); \
} while (0)
#endif

1
arch/nds32/mm/Makefile
View File

@@ -3,7 +3,6 @@ obj-y := extable.o tlb.o fault.o init.o mmap.o \
mm-nds32.o cacheflush.o proc.o
obj-$(CONFIG_ALIGNMENT_TRAP) += alignment.o
obj-$(CONFIG_HIGHMEM) += highmem.o
ifdef CONFIG_FUNCTION_TRACER
CFLAGS_REMOVE_proc.o = $(CC_FLAGS_FTRACE)

48
arch/nds32/mm/highmem.c
View File

@@ -1,48 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
// Copyright (C) 2005-2017 Andes Technology Corporation
#include <linux/export.h>
#include <linux/highmem.h>
#include <linux/sched.h>
#include <linux/smp.h>
#include <linux/interrupt.h>
#include <linux/memblock.h>
#include <asm/fixmap.h>
#include <asm/tlbflush.h>
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned int idx;
unsigned long vaddr, pte;
int type;
pte_t *ptep;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR * smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
pte = (page_to_pfn(page) << PAGE_SHIFT) | prot;
ptep = pte_offset_kernel(pmd_off_k(vaddr), vaddr);
set_pte(ptep, pte);
__nds32__tlbop_inv(vaddr);
__nds32__mtsr_dsb(vaddr, NDS32_SR_TLB_VPN);
__nds32__tlbop_rwr(pte);
__nds32__isb();
return (void *)vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
if (kvaddr >= (void *)FIXADDR_START) {
unsigned long vaddr = (unsigned long)kvaddr;
pte_t *ptep;
kmap_atomic_idx_pop();
__nds32__tlbop_inv(vaddr);
__nds32__isb();
ptep = pte_offset_kernel(pmd_off_k(vaddr), vaddr);
set_pte(ptep, 0);
}
}
EXPORT_SYMBOL(kunmap_atomic_high);

1
arch/openrisc/mm/init.c
View File

@@ -33,7 +33,6 @@
#include <asm/io.h>
#include <asm/tlb.h>
#include <asm/mmu_context.h>
#include <asm/kmap_types.h>
#include <asm/fixmap.h>
#include <asm/tlbflush.h>
#include <asm/sections.h>

1
arch/openrisc/mm/ioremap.c
View File

@@ -15,7 +15,6 @@
#include <linux/io.h>
#include <linux/pgtable.h>
#include <asm/pgalloc.h>
#include <asm/kmap_types.h>
#include <asm/fixmap.h>
#include <asm/bug.h>
#include <linux/sched.h>

13
arch/parisc/include/asm/kmap_types.h
View File

@@ -1,13 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_KMAP_TYPES_H
#define _ASM_KMAP_TYPES_H
#ifdef CONFIG_DEBUG_HIGHMEM
#define __WITH_KM_FENCE
#endif
#include <asm-generic/kmap_types.h>
#undef __WITH_KM_FENCE
#endif

1
arch/powerpc/Kconfig
View File

@@ -410,6 +410,7 @@ menu "Kernel options"
config HIGHMEM
bool "High memory support"
depends on PPC32
select KMAP_LOCAL
source "kernel/Kconfig.hz"

4
arch/powerpc/include/asm/fixmap.h
View File

@@ -20,7 +20,7 @@
#include <asm/page.h>
#ifdef CONFIG_HIGHMEM
#include <linux/threads.h>
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
#endif
#ifdef CONFIG_KASAN
@@ -55,7 +55,7 @@ enum fixed_addresses {
FIX_EARLY_DEBUG_BASE = FIX_EARLY_DEBUG_TOP+(ALIGN(SZ_128K, PAGE_SIZE)/PAGE_SIZE)-1,
#ifdef CONFIG_HIGHMEM
FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */
FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * NR_CPUS) - 1,
#endif
#ifdef CONFIG_PPC_8xx
/* For IMMR we need an aligned 512K area */

7
arch/powerpc/include/asm/highmem.h
View File

@@ -24,12 +24,10 @@
#ifdef __KERNEL__
#include <linux/interrupt.h>
#include <asm/kmap_types.h>
#include <asm/cacheflush.h>
#include <asm/page.h>
#include <asm/fixmap.h>
extern pte_t *kmap_pte;
extern pte_t *pkmap_page_table;
/*
@@ -60,6 +58,11 @@ extern pte_t *pkmap_page_table;
#define flush_cache_kmaps() flush_cache_all()
#define arch_kmap_local_post_map(vaddr, pteval) \
local_flush_tlb_page(NULL, vaddr)
#define arch_kmap_local_post_unmap(vaddr) \
local_flush_tlb_page(NULL, vaddr)
#endif /* __KERNEL__ */
#endif /* _ASM_HIGHMEM_H */

13
arch/powerpc/include/asm/kmap_types.h
View File

@@ -1,13 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
#ifndef _ASM_POWERPC_KMAP_TYPES_H
#define _ASM_POWERPC_KMAP_TYPES_H
#ifdef __KERNEL__
/*
*/
#define KM_TYPE_NR 16
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_KMAP_TYPES_H */

1
arch/powerpc/mm/Makefile
View File

@@ -16,7 +16,6 @@ obj-$(CONFIG_NEED_MULTIPLE_NODES) += numa.o
obj-$(CONFIG_PPC_MM_SLICES) += slice.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
obj-$(CONFIG_NOT_COHERENT_CACHE) += dma-noncoherent.o
obj-$(CONFIG_HIGHMEM) += highmem.o
obj-$(CONFIG_PPC_COPRO_BASE) += copro_fault.o
obj-$(CONFIG_PPC_PTDUMP) += ptdump/
obj-$(CONFIG_KASAN) += kasan/

67
arch/powerpc/mm/highmem.c
View File

@@ -1,67 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* highmem.c: virtual kernel memory mappings for high memory
*
* PowerPC version, stolen from the i386 version.
*
* Used in CONFIG_HIGHMEM systems for memory pages which
* are not addressable by direct kernel virtual addresses.
*
* Copyright (C) 1999 Gerhard Wichert, Siemens AG
* Gerhard.Wichert@pdb.siemens.de
*
*
* Redesigned the x86 32-bit VM architecture to deal with
* up to 16 Terrabyte physical memory. With current x86 CPUs
* we now support up to 64 Gigabytes physical RAM.
*
* Copyright (C) 1999 Ingo Molnar <mingo@redhat.com>
*
* Reworked for PowerPC by various contributors. Moved from
* highmem.h by Benjamin Herrenschmidt (c) 2009 IBM Corp.
*/
#include <linux/highmem.h>
#include <linux/module.h>
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
WARN_ON(IS_ENABLED(CONFIG_DEBUG_HIGHMEM) && !pte_none(*(kmap_pte - idx)));
__set_pte_at(&init_mm, vaddr, kmap_pte-idx, mk_pte(page, prot), 1);
local_flush_tlb_page(NULL, vaddr);
return (void*) vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
if (vaddr < __fix_to_virt(FIX_KMAP_END))
return;
if (IS_ENABLED(CONFIG_DEBUG_HIGHMEM)) {
int type = kmap_atomic_idx();
unsigned int idx;
idx = type + KM_TYPE_NR * smp_processor_id();
WARN_ON(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
/*
* force other mappings to Oops if they'll try to access
* this pte without first remap it
*/
pte_clear(&init_mm, vaddr, kmap_pte-idx);
local_flush_tlb_page(NULL, vaddr);
}
kmap_atomic_idx_pop();
}
EXPORT_SYMBOL(kunmap_atomic_high);

7
arch/powerpc/mm/mem.c
View File

@@ -62,11 +62,6 @@
unsigned long long memory_limit;
bool init_mem_is_free;
#ifdef CONFIG_HIGHMEM
pte_t *kmap_pte;
EXPORT_SYMBOL(kmap_pte);
#endif
pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t vma_prot)
{
@@ -236,8 +231,6 @@ void __init paging_init(void)
map_kernel_page(PKMAP_BASE, 0, __pgprot(0)); /* XXX gross */
pkmap_page_table = virt_to_kpte(PKMAP_BASE);
kmap_pte = virt_to_kpte(__fix_to_virt(FIX_KMAP_BEGIN));
#endif /* CONFIG_HIGHMEM */
printk(KERN_DEBUG "Top of RAM: 0x%llx, Total RAM: 0x%llx\n",

8
arch/sh/include/asm/fixmap.h
View File

@@ -13,9 +13,6 @@
#include <linux/kernel.h>
#include <linux/threads.h>
#include <asm/page.h>
#ifdef CONFIG_HIGHMEM
#include <asm/kmap_types.h>
#endif
/*
* Here we define all the compile-time 'special' virtual
@@ -53,11 +50,6 @@ enum fixed_addresses {
FIX_CMAP_BEGIN,
FIX_CMAP_END = FIX_CMAP_BEGIN + (FIX_N_COLOURS * NR_CPUS) - 1,
#ifdef CONFIG_HIGHMEM
FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * NR_CPUS) - 1,
#endif
#ifdef CONFIG_IOREMAP_FIXED
/*
* FIX_IOREMAP entries are useful for mapping physical address

15
arch/sh/include/asm/kmap_types.h
View File

@@ -1,15 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __SH_KMAP_TYPES_H
#define __SH_KMAP_TYPES_H
/* Dummy header just to define km_type. */
#ifdef CONFIG_DEBUG_HIGHMEM
#define __WITH_KM_FENCE
#endif
#include <asm-generic/kmap_types.h>
#undef __WITH_KM_FENCE
#endif

8
arch/sh/mm/init.c
View File

@@ -362,9 +362,6 @@ void __init mem_init(void)
mem_init_print_info(NULL);
pr_info("virtual kernel memory layout:\n"
" fixmap : 0x%08lx - 0x%08lx (%4ld kB)\n"
#ifdef CONFIG_HIGHMEM
" pkmap : 0x%08lx - 0x%08lx (%4ld kB)\n"
#endif
" vmalloc : 0x%08lx - 0x%08lx (%4ld MB)\n"
" lowmem : 0x%08lx - 0x%08lx (%4ld MB) (cached)\n"
#ifdef CONFIG_UNCACHED_MAPPING
@@ -376,11 +373,6 @@ void __init mem_init(void)
FIXADDR_START, FIXADDR_TOP,
(FIXADDR_TOP - FIXADDR_START) >> 10,
#ifdef CONFIG_HIGHMEM
PKMAP_BASE, PKMAP_BASE+LAST_PKMAP*PAGE_SIZE,
(LAST_PKMAP*PAGE_SIZE) >> 10,
#endif
(unsigned long)VMALLOC_START, VMALLOC_END,
(VMALLOC_END - VMALLOC_START) >> 20,

1
arch/sparc/Kconfig
View File

@@ -139,6 +139,7 @@ config MMU
config HIGHMEM
bool
default y if SPARC32
select KMAP_LOCAL
config ZONE_DMA
bool

8
arch/sparc/include/asm/highmem.h
View File

@@ -24,7 +24,6 @@
#include <linux/interrupt.h>
#include <linux/pgtable.h>
#include <asm/vaddrs.h>
#include <asm/kmap_types.h>
#include <asm/pgtsrmmu.h>
/* declarations for highmem.c */
@@ -33,8 +32,6 @@ extern unsigned long highstart_pfn, highend_pfn;
#define kmap_prot __pgprot(SRMMU_ET_PTE | SRMMU_PRIV | SRMMU_CACHE)
extern pte_t *pkmap_page_table;
void kmap_init(void) __init;
/*
* Right now we initialize only a single pte table. It can be extended
* easily, subsequent pte tables have to be allocated in one physical
@@ -53,6 +50,11 @@ void kmap_init(void) __init;
#define flush_cache_kmaps() flush_cache_all()
/* FIXME: Use __flush_tlb_one(vaddr) instead of flush_cache_all() -- Anton */
#define arch_kmap_local_post_map(vaddr, pteval) flush_cache_all()
#define arch_kmap_local_post_unmap(vaddr) flush_cache_all()
#endif /* __KERNEL__ */
#endif /* _ASM_HIGHMEM_H */

11
arch/sparc/include/asm/kmap_types.h
View File

@@ -1,11 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_KMAP_TYPES_H
#define _ASM_KMAP_TYPES_H
/* Dummy header just to define km_type. None of this
* is actually used on sparc. -DaveM
*/
#include <asm-generic/kmap_types.h>
#endif

4
arch/sparc/include/asm/vaddrs.h
View File

@@ -32,13 +32,13 @@
#define SRMMU_NOCACHE_ALCRATIO 64 /* 256 pages per 64MB of system RAM */
#ifndef __ASSEMBLY__
#include <asm/kmap_types.h>
#include <asm/kmap_size.h>
enum fixed_addresses {
FIX_HOLE,
#ifdef CONFIG_HIGHMEM
FIX_KMAP_BEGIN,
FIX_KMAP_END = (KM_TYPE_NR * NR_CPUS),
FIX_KMAP_END = (KM_MAX_IDX * NR_CPUS),
#endif
__end_of_fixed_addresses
};

3
arch/sparc/mm/Makefile
View File

@@ -15,6 +15,3 @@ obj-$(CONFIG_SPARC32) += leon_mm.o
# Only used by sparc64
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
# Only used by sparc32
obj-$(CONFIG_HIGHMEM) += highmem.o

115
arch/sparc/mm/highmem.c
View File

@@ -1,115 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* highmem.c: virtual kernel memory mappings for high memory
*
* Provides kernel-static versions of atomic kmap functions originally
* found as inlines in include/asm-sparc/highmem.h. These became
* needed as kmap_atomic() and kunmap_atomic() started getting
* called from within modules.
* -- Tomas Szepe <szepe@pinerecords.com>, September 2002
*
* But kmap_atomic() and kunmap_atomic() cannot be inlined in
* modules because they are loaded with btfixup-ped functions.
*/
/*
* The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
* gives a more generic (and caching) interface. But kmap_atomic can
* be used in IRQ contexts, so in some (very limited) cases we need it.
*
* XXX This is an old text. Actually, it's good to use atomic kmaps,
* provided you remember that they are atomic and not try to sleep
* with a kmap taken, much like a spinlock. Non-atomic kmaps are
* shared by CPUs, and so precious, and establishing them requires IPI.
* Atomic kmaps are lightweight and we may have NCPUS more of them.
*/
#include <linux/highmem.h>
#include <linux/export.h>
#include <linux/mm.h>
#include <asm/cacheflush.h>
#include <asm/tlbflush.h>
#include <asm/vaddrs.h>
static pte_t *kmap_pte;
void __init kmap_init(void)
{
unsigned long address = __fix_to_virt(FIX_KMAP_BEGIN);
/* cache the first kmap pte */
kmap_pte = virt_to_kpte(address);
}
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
long idx, type;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
/* XXX Fix - Anton */
#if 0
__flush_cache_one(vaddr);
#else
flush_cache_all();
#endif
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte-idx)));
#endif
set_pte(kmap_pte-idx, mk_pte(page, prot));
/* XXX Fix - Anton */
#if 0
__flush_tlb_one(vaddr);
#else
flush_tlb_all();
#endif
return (void*) vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int type;
if (vaddr < FIXADDR_START)
return;
type = kmap_atomic_idx();
#ifdef CONFIG_DEBUG_HIGHMEM
{
unsigned long idx;
idx = type + KM_TYPE_NR * smp_processor_id();
BUG_ON(vaddr != __fix_to_virt(FIX_KMAP_BEGIN+idx));
/* XXX Fix - Anton */
#if 0
__flush_cache_one(vaddr);
#else
flush_cache_all();
#endif
/*
* force other mappings to Oops if they'll try to access
* this pte without first remap it
*/
pte_clear(&init_mm, vaddr, kmap_pte-idx);
/* XXX Fix - Anton */
#if 0
__flush_tlb_one(vaddr);
#else
flush_tlb_all();
#endif
}
#endif
kmap_atomic_idx_pop();
}
EXPORT_SYMBOL(kunmap_atomic_high);

2
arch/sparc/mm/srmmu.c
View File

@@ -971,8 +971,6 @@ void __init srmmu_paging_init(void)
sparc_context_init(num_contexts);
kmap_init();
{
unsigned long max_zone_pfn[MAX_NR_ZONES] = { 0 };

1
arch/um/include/asm/fixmap.h
View File

@@ -3,7 +3,6 @@
#define __UM_FIXMAP_H
#include <asm/processor.h>
#include <asm/kmap_types.h>
#include <asm/archparam.h>
#include <asm/page.h>
#include <linux/threads.h>

13
arch/um/include/asm/kmap_types.h
View File

@@ -1,13 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2002 Jeff Dike (jdike@karaya.com)
*/
#ifndef __UM_KMAP_TYPES_H
#define __UM_KMAP_TYPES_H
/* No more #include "asm/arch/kmap_types.h" ! */
#define KM_TYPE_NR 14
#endif

4
arch/x86/Kconfig
View File

@@ -14,10 +14,11 @@ config X86_32
select ARCH_WANT_IPC_PARSE_VERSION
select CLKSRC_I8253
select CLONE_BACKWARDS
select GENERIC_VDSO_32
select HAVE_DEBUG_STACKOVERFLOW
select KMAP_LOCAL
select MODULES_USE_ELF_REL
select OLD_SIGACTION
select GENERIC_VDSO_32
config X86_64
def_bool y
@@ -92,6 +93,7 @@ config X86
select ARCH_SUPPORTS_ACPI
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_NUMA_BALANCING if X86_64
select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096
select ARCH_USE_BUILTIN_BSWAP
select ARCH_USE_QUEUED_RWLOCKS
select ARCH_USE_QUEUED_SPINLOCKS

16
arch/x86/include/asm/apic.h
View File

@@ -259,6 +259,7 @@ static inline u64 native_x2apic_icr_read(void)
extern int x2apic_mode;
extern int x2apic_phys;
extern void __init x2apic_set_max_apicid(u32 apicid);
extern void __init check_x2apic(void);
extern void x2apic_setup(void);
static inline int x2apic_enabled(void)
@@ -305,11 +306,10 @@ struct apic {
void (*send_IPI_all)(int vector);
void (*send_IPI_self)(int vector);
/* dest_logical is used by the IPI functions */
u32 dest_logical;
u32 disable_esr;
u32 irq_delivery_mode;
u32 irq_dest_mode;
enum apic_delivery_modes delivery_mode;
bool dest_mode_logical;
u32 (*calc_dest_apicid)(unsigned int cpu);
@@ -520,12 +520,10 @@ static inline void apic_smt_update(void) { }
#endif
struct msi_msg;
struct irq_cfg;
#ifdef CONFIG_PCI_MSI
void x86_vector_msi_compose_msg(struct irq_data *data, struct msi_msg *msg);
#else
# define x86_vector_msi_compose_msg NULL
#endif
extern void __irq_msi_compose_msg(struct irq_cfg *cfg, struct msi_msg *msg,
bool dmar);
extern void ioapic_zap_locks(void);

16
arch/x86/include/asm/apicdef.h
View File

@@ -432,15 +432,13 @@ struct local_apic {
#define BAD_APICID 0xFFFFu
#endif
enum ioapic_irq_destination_types {
dest_Fixed = 0,
dest_LowestPrio = 1,
dest_SMI = 2,
dest__reserved_1 = 3,
dest_NMI = 4,
dest_INIT = 5,
dest__reserved_2 = 6,
dest_ExtINT = 7
enum apic_delivery_modes {
APIC_DELIVERY_MODE_FIXED = 0,
APIC_DELIVERY_MODE_LOWESTPRIO = 1,
APIC_DELIVERY_MODE_SMI = 2,
APIC_DELIVERY_MODE_NMI = 4,
APIC_DELIVERY_MODE_INIT = 5,
APIC_DELIVERY_MODE_EXTINT = 7,
};
#endif /* _ASM_X86_APICDEF_H */

15
arch/x86/include/asm/fixmap.h
View File

@@ -14,13 +14,20 @@
#ifndef _ASM_X86_FIXMAP_H
#define _ASM_X86_FIXMAP_H
#include <asm/kmap_size.h>
/*
* Exposed to assembly code for setting up initial page tables. Cannot be
* calculated in assembly code (fixmap entries are an enum), but is sanity
* checked in the actual fixmap C code to make sure that the fixmap is
* covered fully.
*/
#define FIXMAP_PMD_NUM 2
#ifndef CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP
# define FIXMAP_PMD_NUM 2
#else
# define KM_PMDS (KM_MAX_IDX * ((CONFIG_NR_CPUS + 511) / 512))
# define FIXMAP_PMD_NUM (KM_PMDS + 2)
#endif
/* fixmap starts downwards from the 507th entry in level2_fixmap_pgt */
#define FIXMAP_PMD_TOP 507
@@ -31,7 +38,6 @@
#include <asm/pgtable_types.h>
#ifdef CONFIG_X86_32
#include <linux/threads.h>
#include <asm/kmap_types.h>
#else
#include <uapi/asm/vsyscall.h>
#endif
@@ -92,9 +98,9 @@ enum fixed_addresses {
FIX_IO_APIC_BASE_0,
FIX_IO_APIC_BASE_END = FIX_IO_APIC_BASE_0 + MAX_IO_APICS - 1,
#endif
#ifdef CONFIG_X86_32
#ifdef CONFIG_KMAP_LOCAL
FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */
FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1,
FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_MAX_IDX * NR_CPUS) - 1,
#ifdef CONFIG_PCI_MMCONFIG
FIX_PCIE_MCFG,
#endif
@@ -151,7 +157,6 @@ extern void reserve_top_address(unsigned long reserve);
extern int fixmaps_set;
extern pte_t *kmap_pte;
extern pte_t *pkmap_page_table;
void __native_set_fixmap(enum fixed_addresses idx, pte_t pte);

23
arch/x86/include/asm/fpu/api.h
View File

@@ -29,17 +29,32 @@ extern void fpregs_mark_activate(void);
* A context switch will (and softirq might) save CPU's FPU registers to
* fpu->state and set TIF_NEED_FPU_LOAD leaving CPU's FPU registers in
* a random state.
*
* local_bh_disable() protects against both preemption and soft interrupts
* on !RT kernels.
*
* On RT kernels local_bh_disable() is not sufficient because it only
* serializes soft interrupt related sections via a local lock, but stays
* preemptible. Disabling preemption is the right choice here as bottom
* half processing is always in thread context on RT kernels so it
* implicitly prevents bottom half processing as well.
*
* Disabling preemption also serializes against kernel_fpu_begin().
*/
static inline void fpregs_lock(void)
{
preempt_disable();
local_bh_disable();
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
local_bh_disable();
else
preempt_disable();
}
static inline void fpregs_unlock(void)
{
local_bh_enable();
preempt_enable();
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
local_bh_enable();
else
preempt_enable();
}
#ifdef CONFIG_X86_DEBUG_FPU

13
arch/x86/include/asm/highmem.h
View File

@@ -23,7 +23,6 @@
#include <linux/interrupt.h>
#include <linux/threads.h>
#include <asm/kmap_types.h>
#include <asm/tlbflush.h>
#include <asm/paravirt.h>
#include <asm/fixmap.h>
@@ -58,11 +57,17 @@ extern unsigned long highstart_pfn, highend_pfn;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
void *kmap_atomic_pfn(unsigned long pfn);
void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot);
#define flush_cache_kmaps() do { } while (0)
#define arch_kmap_local_post_map(vaddr, pteval) \
arch_flush_lazy_mmu_mode()
#define arch_kmap_local_post_unmap(vaddr) \
do { \
flush_tlb_one_kernel((vaddr)); \
arch_flush_lazy_mmu_mode(); \
} while (0)
extern void add_highpages_with_active_regions(int nid, unsigned long start_pfn,
unsigned long end_pfn);

11
arch/x86/include/asm/hpet.h
View File

@@ -74,17 +74,6 @@ extern void hpet_disable(void);
extern unsigned int hpet_readl(unsigned int a);
extern void force_hpet_resume(void);
struct irq_data;
struct hpet_channel;
struct irq_domain;
extern void hpet_msi_unmask(struct irq_data *data);
extern void hpet_msi_mask(struct irq_data *data);
extern void hpet_msi_write(struct hpet_channel *hc, struct msi_msg *msg);
extern struct irq_domain *hpet_create_irq_domain(int hpet_id);
extern int hpet_assign_irq(struct irq_domain *domain,
struct hpet_channel *hc, int dev_num);
#ifdef CONFIG_HPET_EMULATE_RTC
#include <linux/interrupt.h>

14
arch/x86/include/asm/hw_irq.h
View File

@@ -39,18 +39,16 @@ enum irq_alloc_type {
X86_IRQ_ALLOC_TYPE_PCI_MSI,
X86_IRQ_ALLOC_TYPE_PCI_MSIX,
X86_IRQ_ALLOC_TYPE_DMAR,
X86_IRQ_ALLOC_TYPE_AMDVI,
X86_IRQ_ALLOC_TYPE_UV,
X86_IRQ_ALLOC_TYPE_IOAPIC_GET_PARENT,
X86_IRQ_ALLOC_TYPE_HPET_GET_PARENT,
};
struct ioapic_alloc_info {
int pin;
int node;
u32 trigger : 1;
u32 polarity : 1;
u32 valid : 1;
struct IO_APIC_route_entry *entry;
int pin;
int node;
u32 is_level : 1;
u32 active_low : 1;
u32 valid : 1;
};
struct uv_alloc_info {

7
arch/x86/include/asm/hyperv-tlfs.h
View File

@@ -23,6 +23,13 @@
#define HYPERV_CPUID_IMPLEMENT_LIMITS 0x40000005
#define HYPERV_CPUID_NESTED_FEATURES 0x4000000A
#define HYPERV_CPUID_VIRT_STACK_INTERFACE 0x40000081
#define HYPERV_VS_INTERFACE_EAX_SIGNATURE 0x31235356 /* "VS#1" */
#define HYPERV_CPUID_VIRT_STACK_PROPERTIES 0x40000082
/* Support for the extended IOAPIC RTE format */
#define HYPERV_VS_PROPERTIES_EAX_EXTENDED_IOAPIC_RTE BIT(2)
#define HYPERV_HYPERVISOR_PRESENT_BIT 0x80000000
#define HYPERV_CPUID_MIN 0x40000005
#define HYPERV_CPUID_MAX 0x4000ffff

79
arch/x86/include/asm/io_apic.h
View File

@@ -13,15 +13,6 @@
* Copyright (C) 1997, 1998, 1999, 2000 Ingo Molnar
*/
/* I/O Unit Redirection Table */
#define IO_APIC_REDIR_VECTOR_MASK 0x000FF
#define IO_APIC_REDIR_DEST_LOGICAL 0x00800
#define IO_APIC_REDIR_DEST_PHYSICAL 0x00000
#define IO_APIC_REDIR_SEND_PENDING (1 << 12)
#define IO_APIC_REDIR_REMOTE_IRR (1 << 14)
#define IO_APIC_REDIR_LEVEL_TRIGGER (1 << 15)
#define IO_APIC_REDIR_MASKED (1 << 16)
/*
* The structure of the IO-APIC:
*/
@@ -65,52 +56,40 @@ union IO_APIC_reg_03 {
};
struct IO_APIC_route_entry {
__u32 vector : 8,
delivery_mode : 3, /* 000: FIXED
* 001: lowest prio
* 111: ExtINT
*/
dest_mode : 1, /* 0: physical, 1: logical */
delivery_status : 1,
polarity : 1,
irr : 1,
trigger : 1, /* 0: edge, 1: level */
mask : 1, /* 0: enabled, 1: disabled */
__reserved_2 : 15;
__u32 __reserved_3 : 24,
dest : 8;
} __attribute__ ((packed));
struct IR_IO_APIC_route_entry {
__u64 vector : 8,
zero : 3,
index2 : 1,
delivery_status : 1,
polarity : 1,
irr : 1,
trigger : 1,
mask : 1,
reserved : 31,
format : 1,
index : 15;
union {
struct {
u64 vector : 8,
delivery_mode : 3,
dest_mode_logical : 1,
delivery_status : 1,
active_low : 1,
irr : 1,
is_level : 1,
masked : 1,
reserved_0 : 15,
reserved_1 : 17,
virt_destid_8_14 : 7,
destid_0_7 : 8;
};
struct {
u64 ir_shared_0 : 8,
ir_zero : 3,
ir_index_15 : 1,
ir_shared_1 : 5,
ir_reserved_0 : 31,
ir_format : 1,
ir_index_0_14 : 15;
};
struct {
u64 w1 : 32,
w2 : 32;
};
};
} __attribute__ ((packed));
struct irq_alloc_info;
struct ioapic_domain_cfg;
#define IOAPIC_EDGE 0
#define IOAPIC_LEVEL 1
#define IOAPIC_MASKED 1
#define IOAPIC_UNMASKED 0
#define IOAPIC_POL_HIGH 0
#define IOAPIC_POL_LOW 1
#define IOAPIC_DEST_MODE_PHYSICAL 0
#define IOAPIC_DEST_MODE_LOGICAL 1
#define IOAPIC_MAP_ALLOC 0x1
#define IOAPIC_MAP_CHECK 0x2

13
arch/x86/include/asm/iomap.h
View File

@@ -9,19 +9,14 @@
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/uaccess.h>
#include <linux/highmem.h>
#include <asm/cacheflush.h>
#include <asm/tlbflush.h>
void __iomem *
iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot);
void __iomem *__iomap_local_pfn_prot(unsigned long pfn, pgprot_t prot);
void
iounmap_atomic(void __iomem *kvaddr);
int iomap_create_wc(resource_size_t base, unsigned long size, pgprot_t *prot);
int
iomap_create_wc(resource_size_t base, unsigned long size, pgprot_t *prot);
void
iomap_free(resource_size_t base, unsigned long size);
void iomap_free(resource_size_t base, unsigned long size);
#endif /* _ASM_X86_IOMAP_H */

9
arch/x86/include/asm/irq_remapping.h
View File

@@ -44,9 +44,6 @@ extern int irq_remapping_reenable(int);
extern int irq_remap_enable_fault_handling(void);
extern void panic_if_irq_remap(const char *msg);
extern struct irq_domain *
irq_remapping_get_irq_domain(struct irq_alloc_info *info);
/* Create PCI MSI/MSIx irqdomain, use @parent as the parent irqdomain. */
extern struct irq_domain *
arch_create_remap_msi_irq_domain(struct irq_domain *par, const char *n, int id);
@@ -71,11 +68,5 @@ static inline void panic_if_irq_remap(const char *msg)
{
}
static inline struct irq_domain *
irq_remapping_get_irq_domain(struct irq_alloc_info *info)
{
return NULL;
}
#endif /* CONFIG_IRQ_REMAP */
#endif /* __X86_IRQ_REMAPPING_H */

3
arch/x86/include/asm/irqdomain.h
View File

@@ -12,6 +12,9 @@ enum {
X86_IRQ_ALLOC_LEGACY = 0x2,
};
extern int x86_fwspec_is_ioapic(struct irq_fwspec *fwspec);
extern int x86_fwspec_is_hpet(struct irq_fwspec *fwspec);
extern struct irq_domain *x86_vector_domain;
extern void init_irq_alloc_info(struct irq_alloc_info *info,

13
arch/x86/include/asm/kmap_types.h
View File

@@ -1,13 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_X86_KMAP_TYPES_H
#define _ASM_X86_KMAP_TYPES_H
#if defined(CONFIG_X86_32) && defined(CONFIG_DEBUG_HIGHMEM)
#define __WITH_KM_FENCE
#endif
#include <asm-generic/kmap_types.h>
#undef __WITH_KM_FENCE
#endif /* _ASM_X86_KMAP_TYPES_H */

50
arch/x86/include/asm/msi.h
View File

@@ -9,4 +9,54 @@ typedef struct irq_alloc_info msi_alloc_info_t;
int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec,
msi_alloc_info_t *arg);
/* Structs and defines for the X86 specific MSI message format */
typedef struct x86_msi_data {
u32 vector : 8,
delivery_mode : 3,
dest_mode_logical : 1,
reserved : 2,
active_low : 1,
is_level : 1;
u32 dmar_subhandle;
} __attribute__ ((packed)) arch_msi_msg_data_t;
#define arch_msi_msg_data x86_msi_data
typedef struct x86_msi_addr_lo {
union {
struct {
u32 reserved_0 : 2,
dest_mode_logical : 1,
redirect_hint : 1,
reserved_1 : 1,
virt_destid_8_14 : 7,
destid_0_7 : 8,
base_address : 12;
};
struct {
u32 dmar_reserved_0 : 2,
dmar_index_15 : 1,
dmar_subhandle_valid : 1,
dmar_format : 1,
dmar_index_0_14 : 15,
dmar_base_address : 12;
};
};
} __attribute__ ((packed)) arch_msi_msg_addr_lo_t;
#define arch_msi_msg_addr_lo x86_msi_addr_lo
#define X86_MSI_BASE_ADDRESS_LOW (0xfee00000 >> 20)
typedef struct x86_msi_addr_hi {
u32 reserved : 8,
destid_8_31 : 24;
} __attribute__ ((packed)) arch_msi_msg_addr_hi_t;
#define arch_msi_msg_addr_hi x86_msi_addr_hi
#define X86_MSI_BASE_ADDRESS_HIGH (0)
struct msi_msg;
u32 x86_msi_msg_get_destid(struct msi_msg *msg, bool extid);
#endif /* _ASM_X86_MSI_H */

57
arch/x86/include/asm/msidef.h
View File

@@ -1,57 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_X86_MSIDEF_H
#define _ASM_X86_MSIDEF_H
/*
* Constants for Intel APIC based MSI messages.
*/
/*
* Shifts for MSI data
*/
#define MSI_DATA_VECTOR_SHIFT 0
#define MSI_DATA_VECTOR_MASK 0x000000ff
#define MSI_DATA_VECTOR(v) (((v) << MSI_DATA_VECTOR_SHIFT) & \
MSI_DATA_VECTOR_MASK)
#define MSI_DATA_DELIVERY_MODE_SHIFT 8
#define MSI_DATA_DELIVERY_FIXED (0 << MSI_DATA_DELIVERY_MODE_SHIFT)
#define MSI_DATA_DELIVERY_LOWPRI (1 << MSI_DATA_DELIVERY_MODE_SHIFT)
#define MSI_DATA_LEVEL_SHIFT 14
#define MSI_DATA_LEVEL_DEASSERT (0 << MSI_DATA_LEVEL_SHIFT)
#define MSI_DATA_LEVEL_ASSERT (1 << MSI_DATA_LEVEL_SHIFT)
#define MSI_DATA_TRIGGER_SHIFT 15
#define MSI_DATA_TRIGGER_EDGE (0 << MSI_DATA_TRIGGER_SHIFT)
#define MSI_DATA_TRIGGER_LEVEL (1 << MSI_DATA_TRIGGER_SHIFT)
/*
* Shift/mask fields for msi address
*/
#define MSI_ADDR_BASE_HI 0
#define MSI_ADDR_BASE_LO 0xfee00000
#define MSI_ADDR_DEST_MODE_SHIFT 2
#define MSI_ADDR_DEST_MODE_PHYSICAL (0 << MSI_ADDR_DEST_MODE_SHIFT)
#define MSI_ADDR_DEST_MODE_LOGICAL (1 << MSI_ADDR_DEST_MODE_SHIFT)
#define MSI_ADDR_REDIRECTION_SHIFT 3
#define MSI_ADDR_REDIRECTION_CPU (0 << MSI_ADDR_REDIRECTION_SHIFT)
/* dedicated cpu */
#define MSI_ADDR_REDIRECTION_LOWPRI (1 << MSI_ADDR_REDIRECTION_SHIFT)
/* lowest priority */
#define MSI_ADDR_DEST_ID_SHIFT 12
#define MSI_ADDR_DEST_ID_MASK 0x00ffff0
#define MSI_ADDR_DEST_ID(dest) (((dest) << MSI_ADDR_DEST_ID_SHIFT) & \
MSI_ADDR_DEST_ID_MASK)
#define MSI_ADDR_EXT_DEST_ID(dest) ((dest) & 0xffffff00)
#define MSI_ADDR_IR_EXT_INT (1 << 4)
#define MSI_ADDR_IR_SHV (1 << 3)
#define MSI_ADDR_IR_INDEX1(index) ((index & 0x8000) >> 13)
#define MSI_ADDR_IR_INDEX2(index) ((index & 0x7fff) << 5)
#endif /* _ASM_X86_MSIDEF_H */

1
arch/x86/include/asm/paravirt_types.h
View File

@@ -41,7 +41,6 @@
#ifndef __ASSEMBLY__
#include <asm/desc_defs.h>
#include <asm/kmap_types.h>
#include <asm/pgtable_types.h>
#include <asm/nospec-branch.h>

6
arch/x86/include/asm/pgtable_64_types.h
View File

@@ -143,7 +143,11 @@ extern unsigned int ptrs_per_p4d;
#define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE)
/* The module sections ends with the start of the fixmap */
#define MODULES_END _AC(0xffffffffff000000, UL)
#ifndef CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP
# define MODULES_END _AC(0xffffffffff000000, UL)
#else
# define MODULES_END _AC(0xfffffffffe000000, UL)
#endif
#define MODULES_LEN (MODULES_END - MODULES_VADDR)
#define ESPFIX_PGD_ENTRY _AC(-2, UL)

2
arch/x86/include/asm/x86_init.h
View File

@@ -116,6 +116,7 @@ struct x86_init_pci {
* @init_platform: platform setup
* @guest_late_init: guest late init
* @x2apic_available: X2APIC detection
* @msi_ext_dest_id: MSI supports 15-bit APIC IDs
* @init_mem_mapping: setup early mappings during init_mem_mapping()
* @init_after_bootmem: guest init after boot allocator is finished
*/
@@ -123,6 +124,7 @@ struct x86_hyper_init {
void (*init_platform)(void);
void (*guest_late_init)(void);
bool (*x2apic_available)(void);
bool (*msi_ext_dest_id)(void);
void (*init_mem_mapping)(void);
void (*init_after_bootmem)(void);
};

73
arch/x86/kernel/apic/apic.c
View File

@@ -93,6 +93,11 @@ static unsigned int disabled_cpu_apicid __ro_after_init = BAD_APICID;
*/
static int apic_extnmi __ro_after_init = APIC_EXTNMI_BSP;
/*
* Hypervisor supports 15 bits of APIC ID in MSI Extended Destination ID
*/
static bool virt_ext_dest_id __ro_after_init;
/*
* Map cpu index to physical APIC ID
*/
@@ -1591,7 +1596,7 @@ static void setup_local_APIC(void)
apic->init_apic_ldr();
#ifdef CONFIG_X86_32
if (apic->dest_logical) {
if (apic->dest_mode_logical) {
int logical_apicid, ldr_apicid;
/*
@@ -1841,20 +1846,34 @@ static __init void try_to_enable_x2apic(int remap_mode)
return;
if (remap_mode != IRQ_REMAP_X2APIC_MODE) {
/* IR is required if there is APIC ID > 255 even when running
* under KVM
u32 apic_limit = 255;
/*
* Using X2APIC without IR is not architecturally supported
* on bare metal but may be supported in guests.
*/
if (max_physical_apicid > 255 ||
!x86_init.hyper.x2apic_available()) {
if (!x86_init.hyper.x2apic_available()) {
pr_info("x2apic: IRQ remapping doesn't support X2APIC mode\n");
x2apic_disable();
return;
}
/*
* without IR all CPUs can be addressed by IOAPIC/MSI
* only in physical mode
* If the hypervisor supports extended destination ID in
* MSI, that increases the maximum APIC ID that can be
* used for non-remapped IRQ domains.
*/
if (x86_init.hyper.msi_ext_dest_id()) {
virt_ext_dest_id = 1;
apic_limit = 32767;
}
/*
* Without IR, all CPUs can be addressed by IOAPIC/MSI only
* in physical mode, and CPUs with an APIC ID that cannnot
* be addressed must not be brought online.
*/
x2apic_set_max_apicid(apic_limit);
x2apic_phys = 1;
}
x2apic_enable();
@@ -2478,6 +2497,46 @@ int hard_smp_processor_id(void)
return read_apic_id();
}
void __irq_msi_compose_msg(struct irq_cfg *cfg, struct msi_msg *msg,
bool dmar)
{
memset(msg, 0, sizeof(*msg));
msg->arch_addr_lo.base_address = X86_MSI_BASE_ADDRESS_LOW;
msg->arch_addr_lo.dest_mode_logical = apic->dest_mode_logical;
msg->arch_addr_lo.destid_0_7 = cfg->dest_apicid & 0xFF;
msg->arch_data.delivery_mode = APIC_DELIVERY_MODE_FIXED;
msg->arch_data.vector = cfg->vector;
msg->address_hi = X86_MSI_BASE_ADDRESS_HIGH;
/*
* Only the IOMMU itself can use the trick of putting destination
* APIC ID into the high bits of the address. Anything else would
* just be writing to memory if it tried that, and needs IR to
* address APICs which can't be addressed in the normal 32-bit
* address range at 0xFFExxxxx. That is typically just 8 bits, but
* some hypervisors allow the extended destination ID field in bits
* 5-11 to be used, giving support for 15 bits of APIC IDs in total.
*/
if (dmar)
msg->arch_addr_hi.destid_8_31 = cfg->dest_apicid >> 8;
else if (virt_ext_dest_id && cfg->dest_apicid < 0x8000)
msg->arch_addr_lo.virt_destid_8_14 = cfg->dest_apicid >> 8;
else
WARN_ON_ONCE(cfg->dest_apicid > 0xFF);
}
u32 x86_msi_msg_get_destid(struct msi_msg *msg, bool extid)
{
u32 dest = msg->arch_addr_lo.destid_0_7;
if (extid)
dest |= msg->arch_addr_hi.destid_8_31 << 8;
return dest;
}
EXPORT_SYMBOL_GPL(x86_msi_msg_get_destid);
/*
* Override the generic EOI implementation with an optimized version.
* Only called during early boot when only one CPU is active and with

18
arch/x86/kernel/apic/apic_flat_64.c
View File

@@ -53,7 +53,7 @@ static void _flat_send_IPI_mask(unsigned long mask, int vector)
unsigned long flags;
local_irq_save(flags);
__default_send_IPI_dest_field(mask, vector, apic->dest_logical);
__default_send_IPI_dest_field(mask, vector, APIC_DEST_LOGICAL);
local_irq_restore(flags);
}
@@ -113,15 +113,13 @@ static struct apic apic_flat __ro_after_init = {
.apic_id_valid = default_apic_id_valid,
.apic_id_registered = flat_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 1, /* logical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = true,
.disable_esr = 0,
.dest_logical = APIC_DEST_LOGICAL,
.check_apicid_used = NULL,
.init_apic_ldr = flat_init_apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,
@@ -206,15 +204,13 @@ static struct apic apic_physflat __ro_after_init = {
.apic_id_valid = default_apic_id_valid,
.apic_id_registered = flat_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 0, /* physical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = false,
.disable_esr = 0,
.dest_logical = 0,
.check_apicid_used = NULL,
.init_apic_ldr = physflat_init_apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,

10
arch/x86/kernel/apic/apic_noop.c
View File

@@ -95,19 +95,15 @@ struct apic apic_noop __ro_after_init = {
.apic_id_valid = default_apic_id_valid,
.apic_id_registered = noop_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
/* logical delivery broadcast to all CPUs: */
.irq_dest_mode = 1,
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = true,
.disable_esr = 0,
.dest_logical = APIC_DEST_LOGICAL,
.check_apicid_used = default_check_apicid_used,
.init_apic_ldr = noop_init_apic_ldr,
.ioapic_phys_id_map = default_ioapic_phys_id_map,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,
.apicid_to_cpu_present = physid_set_mask_of_physid,

16
arch/x86/kernel/apic/apic_numachip.c
View File

@@ -246,15 +246,13 @@ static const struct apic apic_numachip1 __refconst = {
.apic_id_valid = numachip_apic_id_valid,
.apic_id_registered = numachip_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 0, /* physical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = false,
.disable_esr = 0,
.dest_logical = 0,
.check_apicid_used = NULL,
.init_apic_ldr = flat_init_apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,
@@ -295,15 +293,13 @@ static const struct apic apic_numachip2 __refconst = {
.apic_id_valid = numachip_apic_id_valid,
.apic_id_registered = numachip_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 0, /* physical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = false,
.disable_esr = 0,
.dest_logical = 0,
.check_apicid_used = NULL,
.init_apic_ldr = flat_init_apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,

9
arch/x86/kernel/apic/bigsmp_32.c
View File

@@ -127,16 +127,13 @@ static struct apic apic_bigsmp __ro_after_init = {
.apic_id_valid = default_apic_id_valid,
.apic_id_registered = bigsmp_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
/* phys delivery to target CPU: */
.irq_dest_mode = 0,
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = false,
.disable_esr = 1,
.dest_logical = 0,
.check_apicid_used = bigsmp_check_apicid_used,
.init_apic_ldr = bigsmp_init_apic_ldr,
.ioapic_phys_id_map = bigsmp_ioapic_phys_id_map,
.setup_apic_routing = bigsmp_setup_apic_routing,
.cpu_present_to_apicid = bigsmp_cpu_present_to_apicid,

525
arch/x86/kernel/apic/io_apic.c
View File

@@ -48,6 +48,7 @@
#include <linux/jiffies.h> /* time_after() */
#include <linux/slab.h>
#include <linux/memblock.h>
#include <linux/msi.h>
#include <asm/irqdomain.h>
#include <asm/io.h>
@@ -63,7 +64,6 @@
#include <asm/setup.h>
#include <asm/irq_remapping.h>
#include <asm/hw_irq.h>
#include <asm/apic.h>
#define for_each_ioapic(idx) \
@@ -89,12 +89,12 @@ struct irq_pin_list {
};
struct mp_chip_data {
struct list_head irq_2_pin;
struct IO_APIC_route_entry entry;
int trigger;
int polarity;
struct list_head irq_2_pin;
struct IO_APIC_route_entry entry;
bool is_level;
bool active_low;
bool isa_irq;
u32 count;
bool isa_irq;
};
struct mp_ioapic_gsi {
@@ -286,31 +286,26 @@ static void io_apic_write(unsigned int apic, unsigned int reg,
writel(value, &io_apic->data);
}
union entry_union {
struct { u32 w1, w2; };
struct IO_APIC_route_entry entry;
};
static struct IO_APIC_route_entry __ioapic_read_entry(int apic, int pin)
{
union entry_union eu;
struct IO_APIC_route_entry entry;
eu.w1 = io_apic_read(apic, 0x10 + 2 * pin);
eu.w2 = io_apic_read(apic, 0x11 + 2 * pin);
entry.w1 = io_apic_read(apic, 0x10 + 2 * pin);
entry.w2 = io_apic_read(apic, 0x11 + 2 * pin);
return eu.entry;
return entry;
}
static struct IO_APIC_route_entry ioapic_read_entry(int apic, int pin)
{
union entry_union eu;
struct IO_APIC_route_entry entry;
unsigned long flags;
raw_spin_lock_irqsave(&ioapic_lock, flags);
eu.entry = __ioapic_read_entry(apic, pin);
entry = __ioapic_read_entry(apic, pin);
raw_spin_unlock_irqrestore(&ioapic_lock, flags);
return eu.entry;
return entry;
}
/*
@@ -321,11 +316,8 @@ static struct IO_APIC_route_entry ioapic_read_entry(int apic, int pin)
*/
static void __ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e)
{
union entry_union eu = {{0, 0}};
eu.entry = e;
io_apic_write(apic, 0x11 + 2*pin, eu.w2);
io_apic_write(apic, 0x10 + 2*pin, eu.w1);
io_apic_write(apic, 0x11 + 2*pin, e.w2);
io_apic_write(apic, 0x10 + 2*pin, e.w1);
}
static void ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e)
@@ -344,12 +336,12 @@ static void ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e)
*/
static void ioapic_mask_entry(int apic, int pin)
{
struct IO_APIC_route_entry e = { .masked = true };
unsigned long flags;
union entry_union eu = { .entry.mask = IOAPIC_MASKED };
raw_spin_lock_irqsave(&ioapic_lock, flags);
io_apic_write(apic, 0x10 + 2*pin, eu.w1);
io_apic_write(apic, 0x11 + 2*pin, eu.w2);
io_apic_write(apic, 0x10 + 2*pin, e.w1);
io_apic_write(apic, 0x11 + 2*pin, e.w2);
raw_spin_unlock_irqrestore(&ioapic_lock, flags);
}
@@ -422,20 +414,15 @@ static void __init replace_pin_at_irq_node(struct mp_chip_data *data, int node,
add_pin_to_irq_node(data, node, newapic, newpin);
}
static void io_apic_modify_irq(struct mp_chip_data *data,
int mask_and, int mask_or,
static void io_apic_modify_irq(struct mp_chip_data *data, bool masked,
void (*final)(struct irq_pin_list *entry))
{
union entry_union eu;
struct irq_pin_list *entry;
eu.entry = data->entry;
eu.w1 &= mask_and;
eu.w1 |= mask_or;
data->entry = eu.entry;
data->entry.masked = masked;
for_each_irq_pin(entry, data->irq_2_pin) {
io_apic_write(entry->apic, 0x10 + 2 * entry->pin, eu.w1);
io_apic_write(entry->apic, 0x10 + 2 * entry->pin, data->entry.w1);
if (final)
final(entry);
}
@@ -459,13 +446,13 @@ static void mask_ioapic_irq(struct irq_data *irq_data)
unsigned long flags;
raw_spin_lock_irqsave(&ioapic_lock, flags);
io_apic_modify_irq(data, ~0, IO_APIC_REDIR_MASKED, &io_apic_sync);
io_apic_modify_irq(data, true, &io_apic_sync);
raw_spin_unlock_irqrestore(&ioapic_lock, flags);
}
static void __unmask_ioapic(struct mp_chip_data *data)
{
io_apic_modify_irq(data, ~IO_APIC_REDIR_MASKED, 0, NULL);
io_apic_modify_irq(data, false, NULL);
}
static void unmask_ioapic_irq(struct irq_data *irq_data)
@@ -506,8 +493,8 @@ static void __eoi_ioapic_pin(int apic, int pin, int vector)
/*
* Mask the entry and change the trigger mode to edge.
*/
entry1.mask = IOAPIC_MASKED;
entry1.trigger = IOAPIC_EDGE;
entry1.masked = true;
entry1.is_level = false;
__ioapic_write_entry(apic, pin, entry1);
@@ -535,15 +522,15 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
/* Check delivery_mode to be sure we're not clearing an SMI pin */
entry = ioapic_read_entry(apic, pin);
if (entry.delivery_mode == dest_SMI)
if (entry.delivery_mode == APIC_DELIVERY_MODE_SMI)
return;
/*
* Make sure the entry is masked and re-read the contents to check
* if it is a level triggered pin and if the remote-IRR is set.
*/
if (entry.mask == IOAPIC_UNMASKED) {
entry.mask = IOAPIC_MASKED;
if (!entry.masked) {
entry.masked = true;
ioapic_write_entry(apic, pin, entry);
entry = ioapic_read_entry(apic, pin);
}
@@ -556,8 +543,8 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
* doesn't clear the remote-IRR if the trigger mode is not
* set to level.
*/
if (entry.trigger == IOAPIC_EDGE) {
entry.trigger = IOAPIC_LEVEL;
if (!entry.is_level) {
entry.is_level = true;
ioapic_write_entry(apic, pin, entry);
}
raw_spin_lock_irqsave(&ioapic_lock, flags);
@@ -659,8 +646,8 @@ void mask_ioapic_entries(void)
struct IO_APIC_route_entry entry;
entry = ioapics[apic].saved_registers[pin];
if (entry.mask == IOAPIC_UNMASKED) {
entry.mask = IOAPIC_MASKED;
if (!entry.masked) {
entry.masked = true;
ioapic_write_entry(apic, pin, entry);
}
}
@@ -745,44 +732,7 @@ static int __init find_isa_irq_apic(int irq, int type)
return -1;
}
#ifdef CONFIG_EISA
/*
* EISA Edge/Level control register, ELCR
*/
static int EISA_ELCR(unsigned int irq)
{
if (irq < nr_legacy_irqs()) {
unsigned int port = 0x4d0 + (irq >> 3);
return (inb(port) >> (irq & 7)) & 1;
}
apic_printk(APIC_VERBOSE, KERN_INFO
"Broken MPtable reports ISA irq %d\n", irq);
return 0;
}
#endif
/* ISA interrupts are always active high edge triggered,
* when listed as conforming in the MP table. */
#define default_ISA_trigger(idx) (IOAPIC_EDGE)
#define default_ISA_polarity(idx) (IOAPIC_POL_HIGH)
/* EISA interrupts are always polarity zero and can be edge or level
* trigger depending on the ELCR value. If an interrupt is listed as
* EISA conforming in the MP table, that means its trigger type must
* be read in from the ELCR */
#define default_EISA_trigger(idx) (EISA_ELCR(mp_irqs[idx].srcbusirq))
#define default_EISA_polarity(idx) default_ISA_polarity(idx)
/* PCI interrupts are always active low level triggered,
* when listed as conforming in the MP table. */
#define default_PCI_trigger(idx) (IOAPIC_LEVEL)
#define default_PCI_polarity(idx) (IOAPIC_POL_LOW)
static int irq_polarity(int idx)
static bool irq_active_low(int idx)
{
int bus = mp_irqs[idx].srcbus;
@@ -791,90 +741,139 @@ static int irq_polarity(int idx)
*/
switch (mp_irqs[idx].irqflag & MP_IRQPOL_MASK) {
case MP_IRQPOL_DEFAULT:
/* conforms to spec, ie. bus-type dependent polarity */
if (test_bit(bus, mp_bus_not_pci))
return default_ISA_polarity(idx);
else
return default_PCI_polarity(idx);
/*
* Conforms to spec, ie. bus-type dependent polarity. PCI
* defaults to low active. [E]ISA defaults to high active.
*/
return !test_bit(bus, mp_bus_not_pci);
case MP_IRQPOL_ACTIVE_HIGH:
return IOAPIC_POL_HIGH;
return false;
case MP_IRQPOL_RESERVED:
pr_warn("IOAPIC: Invalid polarity: 2, defaulting to low\n");
fallthrough;
case MP_IRQPOL_ACTIVE_LOW:
default: /* Pointless default required due to do gcc stupidity */
return IOAPIC_POL_LOW;
return true;
}
}
#ifdef CONFIG_EISA
static int eisa_irq_trigger(int idx, int bus, int trigger)
/*
* EISA Edge/Level control register, ELCR
*/
static bool EISA_ELCR(unsigned int irq)
{
if (irq < nr_legacy_irqs()) {
unsigned int port = 0x4d0 + (irq >> 3);
return (inb(port) >> (irq & 7)) & 1;
}
apic_printk(APIC_VERBOSE, KERN_INFO
"Broken MPtable reports ISA irq %d\n", irq);
return false;
}
/*
* EISA interrupts are always active high and can be edge or level
* triggered depending on the ELCR value. If an interrupt is listed as
* EISA conforming in the MP table, that means its trigger type must be
* read in from the ELCR.
*/
static bool eisa_irq_is_level(int idx, int bus, bool level)
{
switch (mp_bus_id_to_type[bus]) {
case MP_BUS_PCI:
case MP_BUS_ISA:
return trigger;
return level;
case MP_BUS_EISA:
return default_EISA_trigger(idx);
return EISA_ELCR(mp_irqs[idx].srcbusirq);
}
pr_warn("IOAPIC: Invalid srcbus: %d defaulting to level\n", bus);
return IOAPIC_LEVEL;
return true;
}
#else
static inline int eisa_irq_trigger(int idx, int bus, int trigger)
static inline int eisa_irq_is_level(int idx, int bus, bool level)
{
return trigger;
return level;
}
#endif
static int irq_trigger(int idx)
static bool irq_is_level(int idx)
{
int bus = mp_irqs[idx].srcbus;
int trigger;
bool level;
/*
* Determine IRQ trigger mode (edge or level sensitive):
*/
switch (mp_irqs[idx].irqflag & MP_IRQTRIG_MASK) {
case MP_IRQTRIG_DEFAULT:
/* conforms to spec, ie. bus-type dependent trigger mode */
if (test_bit(bus, mp_bus_not_pci))
trigger = default_ISA_trigger(idx);
else
trigger = default_PCI_trigger(idx);
/*
* Conforms to spec, ie. bus-type dependent trigger
* mode. PCI defaults to level, ISA to edge.
*/
level = !test_bit(bus, mp_bus_not_pci);
/* Take EISA into account */
return eisa_irq_trigger(idx, bus, trigger);
return eisa_irq_is_level(idx, bus, level);
case MP_IRQTRIG_EDGE:
return IOAPIC_EDGE;
return false;
case MP_IRQTRIG_RESERVED:
pr_warn("IOAPIC: Invalid trigger mode 2 defaulting to level\n");
fallthrough;
case MP_IRQTRIG_LEVEL:
default: /* Pointless default required due to do gcc stupidity */
return IOAPIC_LEVEL;
return true;
}
}
static int __acpi_get_override_irq(u32 gsi, bool *trigger, bool *polarity)
{
int ioapic, pin, idx;
if (skip_ioapic_setup)
return -1;
ioapic = mp_find_ioapic(gsi);
if (ioapic < 0)
return -1;
pin = mp_find_ioapic_pin(ioapic, gsi);
if (pin < 0)
return -1;
idx = find_irq_entry(ioapic, pin, mp_INT);
if (idx < 0)
return -1;
*trigger = irq_is_level(idx);
*polarity = irq_active_low(idx);
return 0;
}
#ifdef CONFIG_ACPI
int acpi_get_override_irq(u32 gsi, int *is_level, int *active_low)
{
*is_level = *active_low = 0;
return __acpi_get_override_irq(gsi, (bool *)is_level,
(bool *)active_low);
}
#endif
void ioapic_set_alloc_attr(struct irq_alloc_info *info, int node,
int trigger, int polarity)
{
init_irq_alloc_info(info, NULL);
info->type = X86_IRQ_ALLOC_TYPE_IOAPIC;
info->ioapic.node = node;
info->ioapic.trigger = trigger;
info->ioapic.polarity = polarity;
info->ioapic.is_level = trigger;
info->ioapic.active_low = polarity;
info->ioapic.valid = 1;
}
#ifndef CONFIG_ACPI
int acpi_get_override_irq(u32 gsi, int *trigger, int *polarity);
#endif
static void ioapic_copy_alloc_attr(struct irq_alloc_info *dst,
struct irq_alloc_info *src,
u32 gsi, int ioapic_idx, int pin)
{
int trigger, polarity;
bool level, pol_low;
copy_irq_alloc_info(dst, src);
dst->type = X86_IRQ_ALLOC_TYPE_IOAPIC;
@@ -883,20 +882,20 @@ static void ioapic_copy_alloc_attr(struct irq_alloc_info *dst,
dst->ioapic.valid = 1;
if (src && src->ioapic.valid) {
dst->ioapic.node = src->ioapic.node;
dst->ioapic.trigger = src->ioapic.trigger;
dst->ioapic.polarity = src->ioapic.polarity;
dst->ioapic.is_level = src->ioapic.is_level;
dst->ioapic.active_low = src->ioapic.active_low;
} else {
dst->ioapic.node = NUMA_NO_NODE;
if (acpi_get_override_irq(gsi, &trigger, &polarity) >= 0) {
dst->ioapic.trigger = trigger;
dst->ioapic.polarity = polarity;
if (__acpi_get_override_irq(gsi, &level, &pol_low) >= 0) {
dst->ioapic.is_level = level;
dst->ioapic.active_low = pol_low;
} else {
/*
* PCI interrupts are always active low level
* triggered.
*/
dst->ioapic.trigger = IOAPIC_LEVEL;
dst->ioapic.polarity = IOAPIC_POL_LOW;
dst->ioapic.is_level = true;
dst->ioapic.active_low = true;
}
}
}
@@ -906,12 +905,12 @@ static int ioapic_alloc_attr_node(struct irq_alloc_info *info)
return (info && info->ioapic.valid) ? info->ioapic.node : NUMA_NO_NODE;
}
static void mp_register_handler(unsigned int irq, unsigned long trigger)
static void mp_register_handler(unsigned int irq, bool level)
{
irq_flow_handler_t hdl;
bool fasteoi;
if (trigger) {
if (level) {
irq_set_status_flags(irq, IRQ_LEVEL);
fasteoi = true;
} else {
@@ -933,14 +932,14 @@ static bool mp_check_pin_attr(int irq, struct irq_alloc_info *info)
* pin with real trigger and polarity attributes.
*/
if (irq < nr_legacy_irqs() && data->count == 1) {
if (info->ioapic.trigger != data->trigger)
mp_register_handler(irq, info->ioapic.trigger);
data->entry.trigger = data->trigger = info->ioapic.trigger;
data->entry.polarity = data->polarity = info->ioapic.polarity;
if (info->ioapic.is_level != data->is_level)
mp_register_handler(irq, info->ioapic.is_level);
data->entry.is_level = data->is_level = info->ioapic.is_level;
data->entry.active_low = data->active_low = info->ioapic.active_low;
}
return data->trigger == info->ioapic.trigger &&
data->polarity == info->ioapic.polarity;
return data->is_level == info->ioapic.is_level &&
data->active_low == info->ioapic.active_low;
}
static int alloc_irq_from_domain(struct irq_domain *domain, int ioapic, u32 gsi,
@@ -1219,10 +1218,9 @@ void ioapic_zap_locks(void)
static void io_apic_print_entries(unsigned int apic, unsigned int nr_entries)
{
int i;
char buf[256];
struct IO_APIC_route_entry entry;
struct IR_IO_APIC_route_entry *ir_entry = (void *)&entry;
char buf[256];
int i;
printk(KERN_DEBUG "IOAPIC %d:\n", apic);
for (i = 0; i <= nr_entries; i++) {
@@ -1230,20 +1228,21 @@ static void io_apic_print_entries(unsigned int apic, unsigned int nr_entries)
snprintf(buf, sizeof(buf),
" pin%02x, %s, %s, %s, V(%02X), IRR(%1d), S(%1d)",
i,
entry.mask == IOAPIC_MASKED ? "disabled" : "enabled ",
entry.trigger == IOAPIC_LEVEL ? "level" : "edge ",
entry.polarity == IOAPIC_POL_LOW ? "low " : "high",
entry.masked ? "disabled" : "enabled ",
entry.is_level ? "level" : "edge ",
entry.active_low ? "low " : "high",
entry.vector, entry.irr, entry.delivery_status);
if (ir_entry->format)
if (entry.ir_format) {
printk(KERN_DEBUG "%s, remapped, I(%04X), Z(%X)\n",
buf, (ir_entry->index2 << 15) | ir_entry->index,
ir_entry->zero);
else
printk(KERN_DEBUG "%s, %s, D(%02X), M(%1d)\n",
buf,
entry.dest_mode == IOAPIC_DEST_MODE_LOGICAL ?
"logical " : "physical",
entry.dest, entry.delivery_mode);
(entry.ir_index_15 << 15) | entry.ir_index_0_14,
entry.ir_zero);
} else {
printk(KERN_DEBUG "%s, %s, D(%02X%02X), M(%1d)\n", buf,
entry.dest_mode_logical ? "logical " : "physical",
entry.virt_destid_8_14, entry.destid_0_7,
entry.delivery_mode);
}
}
}
@@ -1368,7 +1367,8 @@ void __init enable_IO_APIC(void)
/* If the interrupt line is enabled and in ExtInt mode
* I have found the pin where the i8259 is connected.
*/
if ((entry.mask == 0) && (entry.delivery_mode == dest_ExtINT)) {
if (!entry.masked &&
entry.delivery_mode == APIC_DELIVERY_MODE_EXTINT) {
ioapic_i8259.apic = apic;
ioapic_i8259.pin = pin;
goto found_i8259;
@@ -1410,14 +1410,16 @@ void native_restore_boot_irq_mode(void)
*/
if (ioapic_i8259.pin != -1) {
struct IO_APIC_route_entry entry;
u32 apic_id = read_apic_id();
memset(&entry, 0, sizeof(entry));
entry.mask = IOAPIC_UNMASKED;
entry.trigger = IOAPIC_EDGE;
entry.polarity = IOAPIC_POL_HIGH;
entry.dest_mode = IOAPIC_DEST_MODE_PHYSICAL;
entry.delivery_mode = dest_ExtINT;
entry.dest = read_apic_id();
entry.masked = false;
entry.is_level = false;
entry.active_low = false;
entry.dest_mode_logical = false;
entry.delivery_mode = APIC_DELIVERY_MODE_EXTINT;
entry.destid_0_7 = apic_id & 0xFF;
entry.virt_destid_8_14 = apic_id >> 8;
/*
* Add it to the IO-APIC irq-routing table:
@@ -1618,21 +1620,16 @@ static void __init delay_without_tsc(void)
static int __init timer_irq_works(void)
{
unsigned long t1 = jiffies;
unsigned long flags;
if (no_timer_check)
return 1;
local_save_flags(flags);
local_irq_enable();
if (boot_cpu_has(X86_FEATURE_TSC))
delay_with_tsc();
else
delay_without_tsc();
local_irq_restore(flags);
/*
* Expect a few ticks at least, to be sure some possible
* glue logic does not lock up after one or two first
@@ -1641,10 +1638,10 @@ static int __init timer_irq_works(void)
* least one tick may be lost due to delays.
*/
/* jiffies wrap? */
if (time_after(jiffies, t1 + 4))
return 1;
return 0;
local_irq_disable();
/* Did jiffies advance? */
return time_after(jiffies, t1 + 4);
}
/*
@@ -1696,13 +1693,13 @@ static bool io_apic_level_ack_pending(struct mp_chip_data *data)
raw_spin_lock_irqsave(&ioapic_lock, flags);
for_each_irq_pin(entry, data->irq_2_pin) {
unsigned int reg;
struct IO_APIC_route_entry e;
int pin;
pin = entry->pin;
reg = io_apic_read(entry->apic, 0x10 + pin*2);
e.w1 = io_apic_read(entry->apic, 0x10 + pin*2);
/* Is the remote IRR bit set? */
if (reg & IO_APIC_REDIR_REMOTE_IRR) {
if (e.irr) {
raw_spin_unlock_irqrestore(&ioapic_lock, flags);
return true;
}
@@ -1849,21 +1846,62 @@ static void ioapic_ir_ack_level(struct irq_data *irq_data)
eoi_ioapic_pin(data->entry.vector, data);
}
/*
* The I/OAPIC is just a device for generating MSI messages from legacy
* interrupt pins. Various fields of the RTE translate into bits of the
* resulting MSI which had a historical meaning.
*
* With interrupt remapping, many of those bits have different meanings
* in the underlying MSI, but the way that the I/OAPIC transforms them
* from its RTE to the MSI message is the same. This function allows
* the parent IRQ domain to compose the MSI message, then takes the
* relevant bits to put them in the appropriate places in the RTE in
* order to generate that message when the IRQ happens.
*
* The setup here relies on a preconfigured route entry (is_level,
* active_low, masked) because the parent domain is merely composing the
* generic message routing information which is used for the MSI.
*/
static void ioapic_setup_msg_from_msi(struct irq_data *irq_data,
struct IO_APIC_route_entry *entry)
{
struct msi_msg msg;
/* Let the parent domain compose the MSI message */
irq_chip_compose_msi_msg(irq_data, &msg);
/*
* - Real vector
* - DMAR/IR: 8bit subhandle (ioapic.pin)
* - AMD/IR: 8bit IRTE index
*/
entry->vector = msg.arch_data.vector;
/* Delivery mode (for DMAR/IR all 0) */
entry->delivery_mode = msg.arch_data.delivery_mode;
/* Destination mode or DMAR/IR index bit 15 */
entry->dest_mode_logical = msg.arch_addr_lo.dest_mode_logical;
/* DMAR/IR: 1, 0 for all other modes */
entry->ir_format = msg.arch_addr_lo.dmar_format;
/*
* - DMAR/IR: index bit 0-14.
*
* - Virt: If the host supports x2apic without a virtualized IR
* unit then bit 0-6 of dmar_index_0_14 are providing bit
* 8-14 of the destination id.
*
* All other modes have bit 0-6 of dmar_index_0_14 cleared and the
* topmost 8 bits are destination id bit 0-7 (entry::destid_0_7).
*/
entry->ir_index_0_14 = msg.arch_addr_lo.dmar_index_0_14;
}
static void ioapic_configure_entry(struct irq_data *irqd)
{
struct mp_chip_data *mpd = irqd->chip_data;
struct irq_cfg *cfg = irqd_cfg(irqd);
struct irq_pin_list *entry;
/*
* Only update when the parent is the vector domain, don't touch it
* if the parent is the remapping domain. Check the installed
* ioapic chip to verify that.
*/
if (irqd->chip == &ioapic_chip) {
mpd->entry.dest = cfg->dest_apicid;
mpd->entry.vector = cfg->vector;
}
ioapic_setup_msg_from_msi(irqd, &mpd->entry);
for_each_irq_pin(entry, mpd->irq_2_pin)
__ioapic_write_entry(entry->apic, entry->pin, mpd->entry);
}
@@ -1919,7 +1957,7 @@ static int ioapic_irq_get_chip_state(struct irq_data *irqd,
* irrelevant because the IO-APIC treats them as fire and
* forget.
*/
if (rentry.irr && rentry.trigger) {
if (rentry.irr && rentry.is_level) {
*state = true;
break;
}
@@ -2027,6 +2065,7 @@ static inline void __init unlock_ExtINT_logic(void)
int apic, pin, i;
struct IO_APIC_route_entry entry0, entry1;
unsigned char save_control, save_freq_select;
u32 apic_id;
pin = find_isa_irq_pin(8, mp_INT);
if (pin == -1) {
@@ -2042,14 +2081,16 @@ static inline void __init unlock_ExtINT_logic(void)
entry0 = ioapic_read_entry(apic, pin);
clear_IO_APIC_pin(apic, pin);
apic_id = hard_smp_processor_id();
memset(&entry1, 0, sizeof(entry1));
entry1.dest_mode = IOAPIC_DEST_MODE_PHYSICAL;
entry1.mask = IOAPIC_UNMASKED;
entry1.dest = hard_smp_processor_id();
entry1.delivery_mode = dest_ExtINT;
entry1.polarity = entry0.polarity;
entry1.trigger = IOAPIC_EDGE;
entry1.dest_mode_logical = true;
entry1.masked = false;
entry1.destid_0_7 = apic_id & 0xFF;
entry1.virt_destid_8_14 = apic_id >> 8;
entry1.delivery_mode = APIC_DELIVERY_MODE_EXTINT;
entry1.active_low = entry0.active_low;
entry1.is_level = false;
entry1.vector = 0;
ioapic_write_entry(apic, pin, entry1);
@@ -2117,13 +2158,12 @@ static inline void __init check_timer(void)
struct irq_cfg *cfg = irqd_cfg(irq_data);
int node = cpu_to_node(0);
int apic1, pin1, apic2, pin2;
unsigned long flags;
int no_pin1 = 0;
if (!global_clock_event)
return;
local_irq_save(flags);
local_irq_disable();
/*
* get/set the timer IRQ vector:
@@ -2178,9 +2218,9 @@ static inline void __init check_timer(void)
* so only need to unmask if it is level-trigger
* do we really have level trigger timer?
*/
int idx;
idx = find_irq_entry(apic1, pin1, mp_INT);
if (idx != -1 && irq_trigger(idx))
int idx = find_irq_entry(apic1, pin1, mp_INT);
if (idx != -1 && irq_is_level(idx))
unmask_ioapic_irq(irq_get_irq_data(0));
}
irq_domain_deactivate_irq(irq_data);
@@ -2191,7 +2231,6 @@ static inline void __init check_timer(void)
goto out;
}
panic_if_irq_remap("timer doesn't work through Interrupt-remapped IO-APIC");
local_irq_disable();
clear_IO_APIC_pin(apic1, pin1);
if (!no_pin1)
apic_printk(APIC_QUIET, KERN_ERR "..MP-BIOS bug: "
@@ -2215,7 +2254,6 @@ static inline void __init check_timer(void)
/*
* Cleanup, just in case ...
*/
local_irq_disable();
legacy_pic->mask(0);
clear_IO_APIC_pin(apic2, pin2);
apic_printk(APIC_QUIET, KERN_INFO "....... failed.\n");
@@ -2232,7 +2270,6 @@ static inline void __init check_timer(void)
apic_printk(APIC_QUIET, KERN_INFO "..... works.\n");
goto out;
}
local_irq_disable();
legacy_pic->mask(0);
apic_write(APIC_LVT0, APIC_LVT_MASKED | APIC_DM_FIXED | cfg->vector);
apic_printk(APIC_QUIET, KERN_INFO "..... failed.\n");
@@ -2251,7 +2288,6 @@ static inline void __init check_timer(void)
apic_printk(APIC_QUIET, KERN_INFO "..... works.\n");
goto out;
}
local_irq_disable();
apic_printk(APIC_QUIET, KERN_INFO "..... failed :(.\n");
if (apic_is_x2apic_enabled())
apic_printk(APIC_QUIET, KERN_INFO
@@ -2260,7 +2296,7 @@ static inline void __init check_timer(void)
panic("IO-APIC + timer doesn't work! Boot with apic=debug and send a "
"report. Then try booting with the 'noapic' option.\n");
out:
local_irq_restore(flags);
local_irq_enable();
}
/*
@@ -2284,36 +2320,37 @@ out:
static int mp_irqdomain_create(int ioapic)
{
struct irq_alloc_info info;
struct irq_domain *parent;
int hwirqs = mp_ioapic_pin_count(ioapic);
struct ioapic *ip = &ioapics[ioapic];
struct ioapic_domain_cfg *cfg = &ip->irqdomain_cfg;
struct mp_ioapic_gsi *gsi_cfg = mp_ioapic_gsi_routing(ioapic);
struct fwnode_handle *fn;
char *name = "IO-APIC";
struct irq_fwspec fwspec;
if (cfg->type == IOAPIC_DOMAIN_INVALID)
return 0;
init_irq_alloc_info(&info, NULL);
info.type = X86_IRQ_ALLOC_TYPE_IOAPIC_GET_PARENT;
info.devid = mpc_ioapic_id(ioapic);
parent = irq_remapping_get_irq_domain(&info);
if (!parent)
parent = x86_vector_domain;
else
name = "IO-APIC-IR";
/* Handle device tree enumerated APICs proper */
if (cfg->dev) {
fn = of_node_to_fwnode(cfg->dev);
} else {
fn = irq_domain_alloc_named_id_fwnode(name, ioapic);
fn = irq_domain_alloc_named_id_fwnode("IO-APIC", mpc_ioapic_id(ioapic));
if (!fn)
return -ENOMEM;
}
fwspec.fwnode = fn;
fwspec.param_count = 1;
fwspec.param[0] = mpc_ioapic_id(ioapic);
parent = irq_find_matching_fwspec(&fwspec, DOMAIN_BUS_ANY);
if (!parent) {
if (!cfg->dev)
irq_domain_free_fwnode(fn);
return -ENODEV;
}
ip->irqdomain = irq_domain_create_linear(fn, hwirqs, cfg->ops,
(void *)(long)ioapic);
@@ -2587,30 +2624,6 @@ static int io_apic_get_version(int ioapic)
return reg_01.bits.version;
}
int acpi_get_override_irq(u32 gsi, int *trigger, int *polarity)
{
int ioapic, pin, idx;
if (skip_ioapic_setup)
return -1;
ioapic = mp_find_ioapic(gsi);
if (ioapic < 0)
return -1;
pin = mp_find_ioapic_pin(ioapic, gsi);
if (pin < 0)
return -1;
idx = find_irq_entry(ioapic, pin, mp_INT);
if (idx < 0)
return -1;
*trigger = irq_trigger(idx);
*polarity = irq_polarity(idx);
return 0;
}
/*
* This function updates target affinity of IOAPIC interrupts to include
* the CPUs which came online during SMP bringup.
@@ -2934,44 +2947,49 @@ static void mp_irqdomain_get_attr(u32 gsi, struct mp_chip_data *data,
struct irq_alloc_info *info)
{
if (info && info->ioapic.valid) {
data->trigger = info->ioapic.trigger;
data->polarity = info->ioapic.polarity;
} else if (acpi_get_override_irq(gsi, &data->trigger,
&data->polarity) < 0) {
data->is_level = info->ioapic.is_level;
data->active_low = info->ioapic.active_low;
} else if (__acpi_get_override_irq(gsi, &data->is_level,
&data->active_low) < 0) {
/* PCI interrupts are always active low level triggered. */
data->trigger = IOAPIC_LEVEL;
data->polarity = IOAPIC_POL_LOW;
data->is_level = true;
data->active_low = true;
}
}
static void mp_setup_entry(struct irq_cfg *cfg, struct mp_chip_data *data,
struct IO_APIC_route_entry *entry)
/*
* Configure the I/O-APIC specific fields in the routing entry.
*
* This is important to setup the I/O-APIC specific bits (is_level,
* active_low, masked) because the underlying parent domain will only
* provide the routing information and is oblivious of the I/O-APIC
* specific bits.
*
* The entry is just preconfigured at this point and not written into the
* RTE. This happens later during activation which will fill in the actual
* routing information.
*/
static void mp_preconfigure_entry(struct mp_chip_data *data)
{
struct IO_APIC_route_entry *entry = &data->entry;
memset(entry, 0, sizeof(*entry));
entry->delivery_mode = apic->irq_delivery_mode;
entry->dest_mode = apic->irq_dest_mode;
entry->dest = cfg->dest_apicid;
entry->vector = cfg->vector;
entry->trigger = data->trigger;
entry->polarity = data->polarity;
entry->is_level = data->is_level;
entry->active_low = data->active_low;
/*
* Mask level triggered irqs. Edge triggered irqs are masked
* by the irq core code in case they fire.
*/
if (data->trigger == IOAPIC_LEVEL)
entry->mask = IOAPIC_MASKED;
else
entry->mask = IOAPIC_UNMASKED;
entry->masked = data->is_level;
}
int mp_irqdomain_alloc(struct irq_domain *domain, unsigned int virq,
unsigned int nr_irqs, void *arg)
{
int ret, ioapic, pin;
struct irq_cfg *cfg;
struct irq_data *irq_data;
struct mp_chip_data *data;
struct irq_alloc_info *info = arg;
struct mp_chip_data *data;
struct irq_data *irq_data;
int ret, ioapic, pin;
unsigned long flags;
if (!info || nr_irqs > 1)
@@ -2989,7 +3007,6 @@ int mp_irqdomain_alloc(struct irq_domain *domain, unsigned int virq,
if (!data)
return -ENOMEM;
info->ioapic.entry = &data->entry;
ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, info);
if (ret < 0) {
kfree(data);
@@ -3003,22 +3020,20 @@ int mp_irqdomain_alloc(struct irq_domain *domain, unsigned int virq,
irq_data->chip_data = data;
mp_irqdomain_get_attr(mp_pin_to_gsi(ioapic, pin), data, info);
cfg = irqd_cfg(irq_data);
add_pin_to_irq_node(data, ioapic_alloc_attr_node(info), ioapic, pin);
mp_preconfigure_entry(data);
mp_register_handler(virq, data->is_level);
local_irq_save(flags);
if (info->ioapic.entry)
mp_setup_entry(cfg, data, info->ioapic.entry);
mp_register_handler(virq, data->trigger);
if (virq < nr_legacy_irqs())
legacy_pic->mask(virq);
local_irq_restore(flags);
apic_printk(APIC_VERBOSE, KERN_DEBUG
"IOAPIC[%d]: Set routing entry (%d-%d -> 0x%x -> IRQ %d Mode:%i Active:%i Dest:%d)\n",
ioapic, mpc_ioapic_id(ioapic), pin, cfg->vector,
virq, data->trigger, data->polarity, cfg->dest_apicid);
"IOAPIC[%d]: Preconfigured routing entry (%d-%d -> IRQ %d Level:%i ActiveLow:%i)\n",
ioapic, mpc_ioapic_id(ioapic), pin, virq,
data->is_level, data->active_low);
return 0;
}

6
arch/x86/kernel/apic/ipi.c
View File

@@ -260,7 +260,7 @@ void default_send_IPI_mask_sequence_logical(const struct cpumask *mask,
for_each_cpu(query_cpu, mask)
__default_send_IPI_dest_field(
early_per_cpu(x86_cpu_to_logical_apicid, query_cpu),
vector, apic->dest_logical);
vector, APIC_DEST_LOGICAL);
local_irq_restore(flags);
}
@@ -279,7 +279,7 @@ void default_send_IPI_mask_allbutself_logical(const struct cpumask *mask,
continue;
__default_send_IPI_dest_field(
early_per_cpu(x86_cpu_to_logical_apicid, query_cpu),
vector, apic->dest_logical);
vector, APIC_DEST_LOGICAL);
}
local_irq_restore(flags);
}
@@ -297,7 +297,7 @@ void default_send_IPI_mask_logical(const struct cpumask *cpumask, int vector)
local_irq_save(flags);
WARN_ON(mask & ~cpumask_bits(cpu_online_mask)[0]);
__default_send_IPI_dest_field(mask, vector, apic->dest_logical);
__default_send_IPI_dest_field(mask, vector, APIC_DEST_LOGICAL);
local_irq_restore(flags);
}

153
arch/x86/kernel/apic/msi.c
View File

@@ -15,7 +15,6 @@
#include <linux/hpet.h>
#include <linux/msi.h>
#include <asm/irqdomain.h>
#include <asm/msidef.h>
#include <asm/hpet.h>
#include <asm/hw_irq.h>
#include <asm/apic.h>
@@ -23,38 +22,11 @@
struct irq_domain *x86_pci_msi_default_domain __ro_after_init;
static void __irq_msi_compose_msg(struct irq_cfg *cfg, struct msi_msg *msg)
{
msg->address_hi = MSI_ADDR_BASE_HI;
if (x2apic_enabled())
msg->address_hi |= MSI_ADDR_EXT_DEST_ID(cfg->dest_apicid);
msg->address_lo =
MSI_ADDR_BASE_LO |
((apic->irq_dest_mode == 0) ?
MSI_ADDR_DEST_MODE_PHYSICAL :
MSI_ADDR_DEST_MODE_LOGICAL) |
MSI_ADDR_REDIRECTION_CPU |
MSI_ADDR_DEST_ID(cfg->dest_apicid);
msg->data =
MSI_DATA_TRIGGER_EDGE |
MSI_DATA_LEVEL_ASSERT |
MSI_DATA_DELIVERY_FIXED |
MSI_DATA_VECTOR(cfg->vector);
}
void x86_vector_msi_compose_msg(struct irq_data *data, struct msi_msg *msg)
{
__irq_msi_compose_msg(irqd_cfg(data), msg);
}
static void irq_msi_update_msg(struct irq_data *irqd, struct irq_cfg *cfg)
{
struct msi_msg msg[2] = { [1] = { }, };
__irq_msi_compose_msg(cfg, msg);
__irq_msi_compose_msg(cfg, msg, false);
irq_data_get_irq_chip(irqd)->irq_write_msi_msg(irqd, msg);
}
@@ -276,6 +248,17 @@ struct irq_domain *arch_create_remap_msi_irq_domain(struct irq_domain *parent,
#endif
#ifdef CONFIG_DMAR_TABLE
/*
* The Intel IOMMU (ab)uses the high bits of the MSI address to contain the
* high bits of the destination APIC ID. This can't be done in the general
* case for MSIs as it would be targeting real memory above 4GiB not the
* APIC.
*/
static void dmar_msi_compose_msg(struct irq_data *data, struct msi_msg *msg)
{
__irq_msi_compose_msg(irqd_cfg(data), msg, true);
}
static void dmar_msi_write_msg(struct irq_data *data, struct msi_msg *msg)
{
dmar_msi_write(data->irq, msg);
@@ -288,6 +271,7 @@ static struct irq_chip dmar_msi_controller = {
.irq_ack = irq_chip_ack_parent,
.irq_set_affinity = msi_domain_set_affinity,
.irq_retrigger = irq_chip_retrigger_hierarchy,
.irq_compose_msi_msg = dmar_msi_compose_msg,
.irq_write_msi_msg = dmar_msi_write_msg,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
@@ -356,114 +340,3 @@ void dmar_free_hwirq(int irq)
irq_domain_free_irqs(irq, 1);
}
#endif
/*
* MSI message composition
*/
#ifdef CONFIG_HPET_TIMER
static inline int hpet_dev_id(struct irq_domain *domain)
{
struct msi_domain_info *info = msi_get_domain_info(domain);
return (int)(long)info->data;
}
static void hpet_msi_write_msg(struct irq_data *data, struct msi_msg *msg)
{
hpet_msi_write(irq_data_get_irq_handler_data(data), msg);
}
static struct irq_chip hpet_msi_controller __ro_after_init = {
.name = "HPET-MSI",
.irq_unmask = hpet_msi_unmask,
.irq_mask = hpet_msi_mask,
.irq_ack = irq_chip_ack_parent,
.irq_set_affinity = msi_domain_set_affinity,
.irq_retrigger = irq_chip_retrigger_hierarchy,
.irq_write_msi_msg = hpet_msi_write_msg,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
static int hpet_msi_init(struct irq_domain *domain,
struct msi_domain_info *info, unsigned int virq,
irq_hw_number_t hwirq, msi_alloc_info_t *arg)
{
irq_set_status_flags(virq, IRQ_MOVE_PCNTXT);
irq_domain_set_info(domain, virq, arg->hwirq, info->chip, NULL,
handle_edge_irq, arg->data, "edge");
return 0;
}
static void hpet_msi_free(struct irq_domain *domain,
struct msi_domain_info *info, unsigned int virq)
{
irq_clear_status_flags(virq, IRQ_MOVE_PCNTXT);
}
static struct msi_domain_ops hpet_msi_domain_ops = {
.msi_init = hpet_msi_init,
.msi_free = hpet_msi_free,
};
static struct msi_domain_info hpet_msi_domain_info = {
.ops = &hpet_msi_domain_ops,
.chip = &hpet_msi_controller,
.flags = MSI_FLAG_USE_DEF_DOM_OPS,
};
struct irq_domain *hpet_create_irq_domain(int hpet_id)
{
struct msi_domain_info *domain_info;
struct irq_domain *parent, *d;
struct irq_alloc_info info;
struct fwnode_handle *fn;
if (x86_vector_domain == NULL)
return NULL;
domain_info = kzalloc(sizeof(*domain_info), GFP_KERNEL);
if (!domain_info)
return NULL;
*domain_info = hpet_msi_domain_info;
domain_info->data = (void *)(long)hpet_id;
init_irq_alloc_info(&info, NULL);
info.type = X86_IRQ_ALLOC_TYPE_HPET_GET_PARENT;
info.devid = hpet_id;
parent = irq_remapping_get_irq_domain(&info);
if (parent == NULL)
parent = x86_vector_domain;
else
hpet_msi_controller.name = "IR-HPET-MSI";
fn = irq_domain_alloc_named_id_fwnode(hpet_msi_controller.name,
hpet_id);
if (!fn) {
kfree(domain_info);
return NULL;
}
d = msi_create_irq_domain(fn, domain_info, parent);
if (!d) {
irq_domain_free_fwnode(fn);
kfree(domain_info);
}
return d;
}
int hpet_assign_irq(struct irq_domain *domain, struct hpet_channel *hc,
int dev_num)
{
struct irq_alloc_info info;
init_irq_alloc_info(&info, NULL);
info.type = X86_IRQ_ALLOC_TYPE_HPET;
info.data = hc;
info.devid = hpet_dev_id(domain);
info.hwirq = dev_num;
return irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, &info);
}
#endif

9
arch/x86/kernel/apic/probe_32.c
View File

@@ -69,16 +69,13 @@ static struct apic apic_default __ro_after_init = {
.apic_id_valid = default_apic_id_valid,
.apic_id_registered = default_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
/* logical delivery broadcast to all CPUs: */
.irq_dest_mode = 1,
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = true,
.disable_esr = 0,
.dest_logical = APIC_DEST_LOGICAL,
.check_apicid_used = default_check_apicid_used,
.init_apic_ldr = default_init_apic_ldr,
.ioapic_phys_id_map = default_ioapic_phys_id_map,
.setup_apic_routing = setup_apic_flat_routing,
.cpu_present_to_apicid = default_cpu_present_to_apicid,

49
arch/x86/kernel/apic/vector.c
View File

@@ -640,7 +640,50 @@ static void x86_vector_debug_show(struct seq_file *m, struct irq_domain *d,
}
#endif
int x86_fwspec_is_ioapic(struct irq_fwspec *fwspec)
{
if (fwspec->param_count != 1)
return 0;
if (is_fwnode_irqchip(fwspec->fwnode)) {
const char *fwname = fwnode_get_name(fwspec->fwnode);
return fwname && !strncmp(fwname, "IO-APIC-", 8) &&
simple_strtol(fwname+8, NULL, 10) == fwspec->param[0];
}
return to_of_node(fwspec->fwnode) &&
of_device_is_compatible(to_of_node(fwspec->fwnode),
"intel,ce4100-ioapic");
}
int x86_fwspec_is_hpet(struct irq_fwspec *fwspec)
{
if (fwspec->param_count != 1)
return 0;
if (is_fwnode_irqchip(fwspec->fwnode)) {
const char *fwname = fwnode_get_name(fwspec->fwnode);
return fwname && !strncmp(fwname, "HPET-MSI-", 9) &&
simple_strtol(fwname+9, NULL, 10) == fwspec->param[0];
}
return 0;
}
static int x86_vector_select(struct irq_domain *d, struct irq_fwspec *fwspec,
enum irq_domain_bus_token bus_token)
{
/*
* HPET and I/OAPIC cannot be parented in the vector domain
* if IRQ remapping is enabled. APIC IDs above 15 bits are
* only permitted if IRQ remapping is enabled, so check that.
*/
if (apic->apic_id_valid(32768))
return 0;
return x86_fwspec_is_ioapic(fwspec) || x86_fwspec_is_hpet(fwspec);
}
static const struct irq_domain_ops x86_vector_domain_ops = {
.select = x86_vector_select,
.alloc = x86_vector_alloc_irqs,
.free = x86_vector_free_irqs,
.activate = x86_vector_activate,
@@ -822,6 +865,12 @@ void apic_ack_edge(struct irq_data *irqd)
apic_ack_irq(irqd);
}
static void x86_vector_msi_compose_msg(struct irq_data *data,
struct msi_msg *msg)
{
__irq_msi_compose_msg(irqd_cfg(data), msg, false);
}
static struct irq_chip lapic_controller = {
.name = "APIC",
.irq_ack = apic_ack_edge,

10
arch/x86/kernel/apic/x2apic_cluster.c
View File

@@ -61,7 +61,7 @@ __x2apic_send_IPI_mask(const struct cpumask *mask, int vector, int apic_dest)
if (!dest)
continue;
__x2apic_send_IPI_dest(dest, vector, apic->dest_logical);
__x2apic_send_IPI_dest(dest, vector, APIC_DEST_LOGICAL);
/* Remove cluster CPUs from tmpmask */
cpumask_andnot(tmpmsk, tmpmsk, &cmsk->mask);
}
@@ -184,15 +184,13 @@ static struct apic apic_x2apic_cluster __ro_after_init = {
.apic_id_valid = x2apic_apic_id_valid,
.apic_id_registered = x2apic_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 1, /* logical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = true,
.disable_esr = 0,
.dest_logical = APIC_DEST_LOGICAL,
.check_apicid_used = NULL,
.init_apic_ldr = init_x2apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,

17
arch/x86/kernel/apic/x2apic_phys.c
View File

@@ -8,6 +8,12 @@
int x2apic_phys;
static struct apic apic_x2apic_phys;
static u32 x2apic_max_apicid __ro_after_init;
void __init x2apic_set_max_apicid(u32 apicid)
{
x2apic_max_apicid = apicid;
}
static int __init set_x2apic_phys_mode(char *arg)
{
@@ -98,6 +104,9 @@ static int x2apic_phys_probe(void)
/* Common x2apic functions, also used by x2apic_cluster */
int x2apic_apic_id_valid(u32 apicid)
{
if (x2apic_max_apicid && apicid > x2apic_max_apicid)
return 0;
return 1;
}
@@ -148,15 +157,13 @@ static struct apic apic_x2apic_phys __ro_after_init = {
.apic_id_valid = x2apic_apic_id_valid,
.apic_id_registered = x2apic_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 0, /* physical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = false,
.disable_esr = 0,
.dest_logical = 0,
.check_apicid_used = NULL,
.init_apic_ldr = init_x2apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,

12
arch/x86/kernel/apic/x2apic_uv_x.c
View File

@@ -728,9 +728,9 @@ static void uv_send_IPI_one(int cpu, int vector)
unsigned long dmode, val;
if (vector == NMI_VECTOR)
dmode = dest_NMI;
dmode = APIC_DELIVERY_MODE_NMI;
else
dmode = dest_Fixed;
dmode = APIC_DELIVERY_MODE_FIXED;
val = (1UL << UVH_IPI_INT_SEND_SHFT) |
(apicid << UVH_IPI_INT_APIC_ID_SHFT) |
@@ -832,15 +832,13 @@ static struct apic apic_x2apic_uv_x __ro_after_init = {
.apic_id_valid = uv_apic_id_valid,
.apic_id_registered = uv_apic_id_registered,
.irq_delivery_mode = dest_Fixed,
.irq_dest_mode = 0, /* Physical */
.delivery_mode = APIC_DELIVERY_MODE_FIXED,
.dest_mode_logical = false,
.disable_esr = 0,
.dest_logical = APIC_DEST_LOGICAL,
.check_apicid_used = NULL,
.init_apic_ldr = uv_init_apic_ldr,
.ioapic_phys_id_map = NULL,
.setup_apic_routing = NULL,
.cpu_present_to_apicid = default_cpu_present_to_apicid,

29
arch/x86/kernel/cpu/mshyperv.c
View File

@@ -366,9 +366,38 @@ static void __init ms_hyperv_init_platform(void)
#endif
}
static bool __init ms_hyperv_x2apic_available(void)
{
return x2apic_supported();
}
/*
* If ms_hyperv_msi_ext_dest_id() returns true, hyperv_prepare_irq_remapping()
* returns -ENODEV and the Hyper-V IOMMU driver is not used; instead, the
* generic support of the 15-bit APIC ID is used: see __irq_msi_compose_msg().
*
* Note: for a VM on Hyper-V, the I/O-APIC is the only device which
* (logically) generates MSIs directly to the system APIC irq domain.
* There is no HPET, and PCI MSI/MSI-X interrupts are remapped by the
* pci-hyperv host bridge.
*/
static bool __init ms_hyperv_msi_ext_dest_id(void)
{
u32 eax;
eax = cpuid_eax(HYPERV_CPUID_VIRT_STACK_INTERFACE);
if (eax != HYPERV_VS_INTERFACE_EAX_SIGNATURE)
return false;
eax = cpuid_eax(HYPERV_CPUID_VIRT_STACK_PROPERTIES);
return eax & HYPERV_VS_PROPERTIES_EAX_EXTENDED_IOAPIC_RTE;
}
const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
.name = "Microsoft Hyper-V",
.detect = ms_hyperv_platform,
.type = X86_HYPER_MS_HYPERV,
.init.x2apic_available = ms_hyperv_x2apic_available,
.init.msi_ext_dest_id = ms_hyperv_msi_ext_dest_id,
.init.init_platform = ms_hyperv_init_platform,
};

48
arch/x86/kernel/crash_dump_32.c
View File

@@ -13,8 +13,6 @@
#include <linux/uaccess.h>
static void *kdump_buf_page;
static inline bool is_crashed_pfn_valid(unsigned long pfn)
{
#ifndef CONFIG_X86_PAE
@@ -41,15 +39,11 @@ static inline bool is_crashed_pfn_valid(unsigned long pfn)
* @userbuf: if set, @buf is in user address space, use copy_to_user(),
* otherwise @buf is in kernel address space, use memcpy().
*
* Copy a page from "oldmem". For this page, there is no pte mapped
* in the current kernel. We stitch up a pte, similar to kmap_atomic.
*
* Calling copy_to_user() in atomic context is not desirable. Hence first
* copying the data to a pre-allocated kernel page and then copying to user
* space in non-atomic context.
* Copy a page from "oldmem". For this page, there might be no pte mapped
* in the current kernel.
*/
ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
size_t csize, unsigned long offset, int userbuf)
ssize_t copy_oldmem_page(unsigned long pfn, char *buf, size_t csize,
unsigned long offset, int userbuf)
{
void *vaddr;
@@ -59,38 +53,16 @@ ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
if (!is_crashed_pfn_valid(pfn))
return -EFAULT;
vaddr = kmap_atomic_pfn(pfn);
vaddr = kmap_local_pfn(pfn);
if (!userbuf) {
memcpy(buf, (vaddr + offset), csize);
kunmap_atomic(vaddr);
memcpy(buf, vaddr + offset, csize);
} else {
if (!kdump_buf_page) {
printk(KERN_WARNING "Kdump: Kdump buffer page not"
" allocated\n");
kunmap_atomic(vaddr);
return -EFAULT;
}
copy_page(kdump_buf_page, vaddr);
kunmap_atomic(vaddr);
if (copy_to_user(buf, (kdump_buf_page + offset), csize))
return -EFAULT;
if (copy_to_user(buf, vaddr + offset, csize))
csize = -EFAULT;
}
kunmap_local(vaddr);
return csize;
}
static int __init kdump_buf_page_init(void)
{
int ret = 0;
kdump_buf_page = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!kdump_buf_page) {
printk(KERN_WARNING "Kdump: Failed to allocate kdump buffer"
" page\n");
ret = -ENOMEM;
}
return ret;
}
arch_initcall(kdump_buf_page_init);

32
arch/x86/kernel/devicetree.c
View File

@@ -184,31 +184,31 @@ static unsigned int ioapic_id;
struct of_ioapic_type {
u32 out_type;
u32 trigger;
u32 polarity;
u32 is_level;
u32 active_low;
};
static struct of_ioapic_type of_ioapic_type[] =
{
{
.out_type = IRQ_TYPE_EDGE_RISING,
.trigger = IOAPIC_EDGE,
.polarity = 1,
},
{
.out_type = IRQ_TYPE_LEVEL_LOW,
.trigger = IOAPIC_LEVEL,
.polarity = 0,
.out_type = IRQ_TYPE_EDGE_FALLING,
.is_level = 0,
.active_low = 1,
},
{
.out_type = IRQ_TYPE_LEVEL_HIGH,
.trigger = IOAPIC_LEVEL,
.polarity = 1,
.is_level = 1,
.active_low = 0,
},
{
.out_type = IRQ_TYPE_EDGE_FALLING,
.trigger = IOAPIC_EDGE,
.polarity = 0,
.out_type = IRQ_TYPE_LEVEL_LOW,
.is_level = 1,
.active_low = 1,
},
{
.out_type = IRQ_TYPE_EDGE_RISING,
.is_level = 0,
.active_low = 0,
},
};
@@ -228,7 +228,7 @@ static int dt_irqdomain_alloc(struct irq_domain *domain, unsigned int virq,
return -EINVAL;
it = &of_ioapic_type[type_index];
ioapic_set_alloc_attr(&tmp, NUMA_NO_NODE, it->trigger, it->polarity);
ioapic_set_alloc_attr(&tmp, NUMA_NO_NODE, it->is_level, it->active_low);
tmp.devid = mpc_ioapic_id(mp_irqdomain_ioapic_idx(domain));
tmp.ioapic.pin = fwspec->param[0];

122
arch/x86/kernel/hpet.c
View File

@@ -7,6 +7,7 @@
#include <linux/cpu.h>
#include <linux/irq.h>
#include <asm/irq_remapping.h>
#include <asm/hpet.h>
#include <asm/time.h>
@@ -50,7 +51,7 @@ unsigned long hpet_address;
u8 hpet_blockid; /* OS timer block num */
bool hpet_msi_disable;
#ifdef CONFIG_PCI_MSI
#ifdef CONFIG_GENERIC_MSI_IRQ
static DEFINE_PER_CPU(struct hpet_channel *, cpu_hpet_channel);
static struct irq_domain *hpet_domain;
#endif
@@ -467,9 +468,8 @@ static void __init hpet_legacy_clockevent_register(struct hpet_channel *hc)
/*
* HPET MSI Support
*/
#ifdef CONFIG_PCI_MSI
void hpet_msi_unmask(struct irq_data *data)
#ifdef CONFIG_GENERIC_MSI_IRQ
static void hpet_msi_unmask(struct irq_data *data)
{
struct hpet_channel *hc = irq_data_get_irq_handler_data(data);
unsigned int cfg;
@@ -479,7 +479,7 @@ void hpet_msi_unmask(struct irq_data *data)
hpet_writel(cfg, HPET_Tn_CFG(hc->num));
}
void hpet_msi_mask(struct irq_data *data)
static void hpet_msi_mask(struct irq_data *data)
{
struct hpet_channel *hc = irq_data_get_irq_handler_data(data);
unsigned int cfg;
@@ -489,12 +489,122 @@ void hpet_msi_mask(struct irq_data *data)
hpet_writel(cfg, HPET_Tn_CFG(hc->num));
}
void hpet_msi_write(struct hpet_channel *hc, struct msi_msg *msg)
static void hpet_msi_write(struct hpet_channel *hc, struct msi_msg *msg)
{
hpet_writel(msg->data, HPET_Tn_ROUTE(hc->num));
hpet_writel(msg->address_lo, HPET_Tn_ROUTE(hc->num) + 4);
}
static void hpet_msi_write_msg(struct irq_data *data, struct msi_msg *msg)
{
hpet_msi_write(irq_data_get_irq_handler_data(data), msg);
}
static struct irq_chip hpet_msi_controller __ro_after_init = {
.name = "HPET-MSI",
.irq_unmask = hpet_msi_unmask,
.irq_mask = hpet_msi_mask,
.irq_ack = irq_chip_ack_parent,
.irq_set_affinity = msi_domain_set_affinity,
.irq_retrigger = irq_chip_retrigger_hierarchy,
.irq_write_msi_msg = hpet_msi_write_msg,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
static int hpet_msi_init(struct irq_domain *domain,
struct msi_domain_info *info, unsigned int virq,
irq_hw_number_t hwirq, msi_alloc_info_t *arg)
{
irq_set_status_flags(virq, IRQ_MOVE_PCNTXT);
irq_domain_set_info(domain, virq, arg->hwirq, info->chip, NULL,
handle_edge_irq, arg->data, "edge");
return 0;
}
static void hpet_msi_free(struct irq_domain *domain,
struct msi_domain_info *info, unsigned int virq)
{
irq_clear_status_flags(virq, IRQ_MOVE_PCNTXT);
}
static struct msi_domain_ops hpet_msi_domain_ops = {
.msi_init = hpet_msi_init,
.msi_free = hpet_msi_free,
};
static struct msi_domain_info hpet_msi_domain_info = {
.ops = &hpet_msi_domain_ops,
.chip = &hpet_msi_controller,
.flags = MSI_FLAG_USE_DEF_DOM_OPS,
};
static struct irq_domain *hpet_create_irq_domain(int hpet_id)
{
struct msi_domain_info *domain_info;
struct irq_domain *parent, *d;
struct fwnode_handle *fn;
struct irq_fwspec fwspec;
if (x86_vector_domain == NULL)
return NULL;
domain_info = kzalloc(sizeof(*domain_info), GFP_KERNEL);
if (!domain_info)
return NULL;
*domain_info = hpet_msi_domain_info;
domain_info->data = (void *)(long)hpet_id;
fn = irq_domain_alloc_named_id_fwnode(hpet_msi_controller.name,
hpet_id);
if (!fn) {
kfree(domain_info);
return NULL;
}
fwspec.fwnode = fn;
fwspec.param_count = 1;
fwspec.param[0] = hpet_id;
parent = irq_find_matching_fwspec(&fwspec, DOMAIN_BUS_ANY);
if (!parent) {
irq_domain_free_fwnode(fn);
kfree(domain_info);
return NULL;
}
if (parent != x86_vector_domain)
hpet_msi_controller.name = "IR-HPET-MSI";
d = msi_create_irq_domain(fn, domain_info, parent);
if (!d) {
irq_domain_free_fwnode(fn);
kfree(domain_info);
}
return d;
}
static inline int hpet_dev_id(struct irq_domain *domain)
{
struct msi_domain_info *info = msi_get_domain_info(domain);
return (int)(long)info->data;
}
static int hpet_assign_irq(struct irq_domain *domain, struct hpet_channel *hc,
int dev_num)
{
struct irq_alloc_info info;
init_irq_alloc_info(&info, NULL);
info.type = X86_IRQ_ALLOC_TYPE_HPET;
info.data = hc;
info.devid = hpet_dev_id(domain);
info.hwirq = dev_num;
return irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, &info);
}
static int hpet_clkevt_msi_resume(struct clock_event_device *evt)
{
struct hpet_channel *hc = clockevent_to_channel(evt);

6
arch/x86/kernel/kvm.c
View File

@@ -740,6 +740,11 @@ static void __init kvm_apic_init(void)
#endif
}
static bool __init kvm_msi_ext_dest_id(void)
{
return kvm_para_has_feature(KVM_FEATURE_MSI_EXT_DEST_ID);
}
static void __init kvm_init_platform(void)
{
kvmclock_init();
@@ -769,6 +774,7 @@ const __initconst struct hypervisor_x86 x86_hyper_kvm = {
.type = X86_HYPER_KVM,
.init.guest_late_init = kvm_guest_init,
.init.x2apic_available = kvm_para_available,
.init.msi_ext_dest_id = kvm_msi_ext_dest_id,
.init.init_platform = kvm_init_platform,
#if defined(CONFIG_AMD_MEM_ENCRYPT)
.runtime.sev_es_hcall_prepare = kvm_sev_es_hcall_prepare,

8
arch/x86/kernel/smpboot.c
View File

@@ -752,13 +752,14 @@ static void __init smp_quirk_init_udelay(void)
int
wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip)
{
u32 dm = apic->dest_mode_logical ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
unsigned long send_status, accept_status = 0;
int maxlvt;
/* Target chip */
/* Boot on the stack */
/* Kick the second */
apic_icr_write(APIC_DM_NMI | apic->dest_logical, apicid);
apic_icr_write(APIC_DM_NMI | dm, apicid);
pr_debug("Waiting for send to finish...\n");
send_status = safe_apic_wait_icr_idle();
@@ -985,10 +986,7 @@ wakeup_cpu_via_init_nmi(int cpu, unsigned long start_ip, int apicid,
if (!boot_error) {
enable_start_cpu0 = 1;
*cpu0_nmi_registered = 1;
if (apic->dest_logical == APIC_DEST_LOGICAL)
id = cpu0_logical_apicid;
else
id = apicid;
id = apic->dest_mode_logical ? cpu0_logical_apicid : apicid;
boot_error = wakeup_secondary_cpu_via_nmi(id, start_ip);
}

1
arch/x86/kernel/x86_init.c
View File

@@ -110,6 +110,7 @@ struct x86_init_ops x86_init __initdata = {
.init_platform = x86_init_noop,
.guest_late_init = x86_init_noop,
.x2apic_available = bool_x86_init_noop,
.msi_ext_dest_id = bool_x86_init_noop,
.init_mem_mapping = x86_init_noop,
.init_after_bootmem = x86_init_noop,
},

29
arch/x86/kvm/irq_comm.c
View File

@@ -16,8 +16,6 @@
#include <trace/events/kvm.h>
#include <asm/msidef.h>
#include "irq.h"
#include "ioapic.h"
@@ -104,22 +102,19 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
struct kvm_lapic_irq *irq)
{
trace_kvm_msi_set_irq(e->msi.address_lo | (kvm->arch.x2apic_format ?
(u64)e->msi.address_hi << 32 : 0),
e->msi.data);
struct msi_msg msg = { .address_lo = e->msi.address_lo,
.address_hi = e->msi.address_hi,
.data = e->msi.data };
irq->dest_id = (e->msi.address_lo &
MSI_ADDR_DEST_ID_MASK) >> MSI_ADDR_DEST_ID_SHIFT;
if (kvm->arch.x2apic_format)
irq->dest_id |= MSI_ADDR_EXT_DEST_ID(e->msi.address_hi);
irq->vector = (e->msi.data &
MSI_DATA_VECTOR_MASK) >> MSI_DATA_VECTOR_SHIFT;
irq->dest_mode = kvm_lapic_irq_dest_mode(
!!((1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo));
irq->trig_mode = (1 << MSI_DATA_TRIGGER_SHIFT) & e->msi.data;
irq->delivery_mode = e->msi.data & 0x700;
irq->msi_redir_hint = ((e->msi.address_lo
& MSI_ADDR_REDIRECTION_LOWPRI) > 0);
trace_kvm_msi_set_irq(msg.address_lo | (kvm->arch.x2apic_format ?
(u64)msg.address_hi << 32 : 0), msg.data);
irq->dest_id = x86_msi_msg_get_destid(&msg, kvm->arch.x2apic_format);
irq->vector = msg.arch_data.vector;
irq->dest_mode = kvm_lapic_irq_dest_mode(msg.arch_addr_lo.dest_mode_logical);
irq->trig_mode = msg.arch_data.is_level;
irq->delivery_mode = msg.arch_data.delivery_mode << 8;
irq->msi_redir_hint = msg.arch_addr_lo.redirect_hint;
irq->level = 1;
irq->shorthand = APIC_DEST_NOSHORT;
}

59
arch/x86/mm/highmem_32.c
View File

@@ -4,65 +4,6 @@
#include <linux/swap.h> /* for totalram_pages */
#include <linux/memblock.h>
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
BUG_ON(!pte_none(*(kmap_pte-idx)));
set_pte(kmap_pte-idx, mk_pte(page, prot));
arch_flush_lazy_mmu_mode();
return (void *)vaddr;
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
/*
* This is the same as kmap_atomic() but can map memory that doesn't
* have a struct page associated with it.
*/
void *kmap_atomic_pfn(unsigned long pfn)
{
return kmap_atomic_prot_pfn(pfn, kmap_prot);
}
EXPORT_SYMBOL_GPL(kmap_atomic_pfn);
void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
if (vaddr >= __fix_to_virt(FIX_KMAP_END) &&
vaddr <= __fix_to_virt(FIX_KMAP_BEGIN)) {
int idx, type;
type = kmap_atomic_idx();
idx = type + KM_TYPE_NR * smp_processor_id();
#ifdef CONFIG_DEBUG_HIGHMEM
WARN_ON_ONCE(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
#endif
/*
* Force other mappings to Oops if they'll try to access this
* pte without first remap it. Keeping stale mappings around
* is a bad idea also, in case the page changes cacheability
* attributes or becomes a protected page in a hypervisor.
*/
kpte_clear_flush(kmap_pte-idx, vaddr);
kmap_atomic_idx_pop();
arch_flush_lazy_mmu_mode();
}
#ifdef CONFIG_DEBUG_HIGHMEM
else {
BUG_ON(vaddr < PAGE_OFFSET);
BUG_ON(vaddr >= (unsigned long)high_memory);
}
#endif
}
EXPORT_SYMBOL(kunmap_atomic_high);
void __init set_highmem_pages_init(void)
{
struct zone *zone;

15
arch/x86/mm/init_32.c
View File

@@ -394,19 +394,6 @@ repeat:
return last_map_addr;
}
pte_t *kmap_pte;
static void __init kmap_init(void)
{
unsigned long kmap_vstart;
/*
* Cache the first kmap pte:
*/
kmap_vstart = __fix_to_virt(FIX_KMAP_BEGIN);
kmap_pte = virt_to_kpte(kmap_vstart);
}
#ifdef CONFIG_HIGHMEM
static void __init permanent_kmaps_init(pgd_t *pgd_base)
{
@@ -712,8 +699,6 @@ void __init paging_init(void)
__flush_tlb_all();
kmap_init();
/*
* NOTE: at this point the bootmem allocator is fully available.
*/

57
arch/x86/mm/iomap_32.c
View File

@@ -44,28 +44,7 @@ void iomap_free(resource_size_t base, unsigned long size)
}
EXPORT_SYMBOL_GPL(iomap_free);
void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
preempt_disable();
pagefault_disable();
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR * smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
set_pte(kmap_pte - idx, pfn_pte(pfn, prot));
arch_flush_lazy_mmu_mode();
return (void *)vaddr;
}
/*
* Map 'pfn' using protections 'prot'
*/
void __iomem *
iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
void __iomem *__iomap_local_pfn_prot(unsigned long pfn, pgprot_t prot)
{
/*
* For non-PAT systems, translate non-WB request to UC- just in
@@ -81,36 +60,6 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
/* Filter out unsupported __PAGE_KERNEL* bits: */
pgprot_val(prot) &= __default_kernel_pte_mask;
return (void __force __iomem *) kmap_atomic_prot_pfn(pfn, prot);
return (void __force __iomem *)__kmap_local_pfn_prot(pfn, prot);
}
EXPORT_SYMBOL_GPL(iomap_atomic_prot_pfn);
void
iounmap_atomic(void __iomem *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
if (vaddr >= __fix_to_virt(FIX_KMAP_END) &&
vaddr <= __fix_to_virt(FIX_KMAP_BEGIN)) {
int idx, type;
type = kmap_atomic_idx();
idx = type + KM_TYPE_NR * smp_processor_id();
#ifdef CONFIG_DEBUG_HIGHMEM
WARN_ON_ONCE(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
#endif
/*
* Force other mappings to Oops if they'll try to access this
* pte without first remap it. Keeping stale mappings around
* is a bad idea also, in case the page changes cacheability
* attributes or becomes a protected page in a hypervisor.
*/
kpte_clear_flush(kmap_pte-idx, vaddr);
kmap_atomic_idx_pop();
}
pagefault_enable();
preempt_enable();
}
EXPORT_SYMBOL_GPL(iounmap_atomic);
EXPORT_SYMBOL_GPL(__iomap_local_pfn_prot);

8
arch/x86/pci/intel_mid_pci.c
View File

@@ -215,7 +215,7 @@ static int pci_write(struct pci_bus *bus, unsigned int devfn, int where,
static int intel_mid_pci_irq_enable(struct pci_dev *dev)
{
struct irq_alloc_info info;
int polarity;
bool polarity_low;
int ret;
u8 gsi;
@@ -230,7 +230,7 @@ static int intel_mid_pci_irq_enable(struct pci_dev *dev)
switch (intel_mid_identify_cpu()) {
case INTEL_MID_CPU_CHIP_TANGIER:
polarity = IOAPIC_POL_HIGH;
polarity_low = false;
/* Special treatment for IRQ0 */
if (gsi == 0) {
@@ -252,11 +252,11 @@ static int intel_mid_pci_irq_enable(struct pci_dev *dev)
}
break;
default:
polarity = IOAPIC_POL_LOW;
polarity_low = true;
break;
}
ioapic_set_alloc_attr(&info, dev_to_node(&dev->dev), 1, polarity);
ioapic_set_alloc_attr(&info, dev_to_node(&dev->dev), 1, polarity_low);
/*
* MRST only have IOAPIC, the PCI irq lines are 1:1 mapped to

Some files were not shown because too many files have changed in this diff Show More