From c05d042fda889f1e591b9d306482d9c16072dd60 Mon Sep 17 00:00:00 2001 From: "Gustavo A. R. Silva" Date: Fri, 19 Jun 2020 20:14:06 -0500 Subject: [PATCH 01/91] sparc64: viohs: Use struct_size() helper Make use of the struct_size() helper instead of an open-coded version in order to avoid any potential type mistakes. Also, remove unnecessary variable _len_. This code was detected with the help of Coccinelle and, audited and fixed manually. Addresses-KSPP-ID: https://github.com/KSPP/linux/issues/83 Signed-off-by: Gustavo A. R. Silva Signed-off-by: David S. Miller --- arch/sparc/kernel/viohs.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/sparc/kernel/viohs.c b/arch/sparc/kernel/viohs.c index 7db5aabe9708..e27afd233bf5 100644 --- a/arch/sparc/kernel/viohs.c +++ b/arch/sparc/kernel/viohs.c @@ -428,7 +428,7 @@ static int process_dreg_info(struct vio_driver_state *vio, struct vio_dring_register *pkt) { struct vio_dring_state *dr; - int i, len; + int i; viodbg(HS, "GOT DRING_REG INFO ident[%llx] " "ndesc[%u] dsz[%u] opt[0x%x] ncookies[%u]\n", @@ -482,9 +482,7 @@ static int process_dreg_info(struct vio_driver_state *vio, pkt->num_descr, pkt->descr_size, pkt->options, pkt->num_cookies); - len = (sizeof(*pkt) + - (dr->ncookies * sizeof(struct ldc_trans_cookie))); - if (send_ctrl(vio, &pkt->tag, len) < 0) + if (send_ctrl(vio, &pkt->tag, struct_size(pkt, cookies, dr->ncookies)) < 0) goto send_nack; vio->dr_state |= VIO_DR_STATE_RXREG; From 5124b31c1e90797e26710377d04eb005759494a0 Mon Sep 17 00:00:00 2001 From: Corentin Labbe Date: Wed, 10 Jun 2020 07:55:19 +0000 Subject: [PATCH 02/91] sparc: piggyback: handle invalid image With an old elftoaout, the generation of tftpboot.img fail with "lseek: invalid argument". This is due to offset being negative. Instead of printing this error message, let's print a better one. Signed-off-by: Corentin Labbe Signed-off-by: David S. Miller --- arch/sparc/boot/piggyback.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/sparc/boot/piggyback.c b/arch/sparc/boot/piggyback.c index a7a38fb4ece0..6d74064add0a 100644 --- a/arch/sparc/boot/piggyback.c +++ b/arch/sparc/boot/piggyback.c @@ -154,6 +154,10 @@ static off_t get_hdrs_offset(int kernelfd, const char *filename) offset -= LOOKBACK; /* skip a.out header */ offset += AOUT_TEXT_OFFSET; + if (offset < 0) { + errno = -EINVAL; + die("Calculated a negative offset, probably elftoaout generated an invalid image. Did you use a recent elftoaout ?"); + } if (lseek(kernelfd, offset, SEEK_SET) < 0) die("lseek"); if (read(kernelfd, buffer, BUFSIZE) != BUFSIZE) From 37719576046b553092d5a88de90459306dfdec6b Mon Sep 17 00:00:00 2001 From: "Alexander A. Klimov" Date: Mon, 13 Jul 2020 11:53:51 +0200 Subject: [PATCH 03/91] arch/sparc: Replace HTTP links with HTTPS ones Rationale: Reduces attack surface on kernel devs opening the links for MITM as HTTPS traffic is much harder to manipulate. Deterministic algorithm: For each file: If not .svg: For each line: If doesn't contain `\bxmlns\b`: For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`: If neither `\bgnu\.org/license`, nor `\bmozilla\.org/MPL\b`: If both the HTTP and HTTPS versions return 200 OK and serve the same content: Replace HTTP with HTTPS. Signed-off-by: Alexander A. Klimov Signed-off-by: David S. Miller --- arch/sparc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 5bf2dc163540..5f724e53ce9b 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -175,7 +175,7 @@ config SMP Management" code will be disabled if you say Y here. See also and the SMP-HOWTO - available at . + available at . If you don't know what to do here, say N. From e0602848213587d02d146ede8ad31c269982c839 Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Tue, 21 Jul 2020 14:00:33 -0700 Subject: [PATCH 04/91] SPARC: backoff.h: delete a duplicated word Drop the repeated word "other". Signed-off-by: Randy Dunlap Cc: "David S. Miller" Cc: sparclinux@vger.kernel.org Signed-off-by: David S. Miller --- arch/sparc/include/asm/backoff.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/include/asm/backoff.h b/arch/sparc/include/asm/backoff.h index 8625946d8d00..597a22953bc5 100644 --- a/arch/sparc/include/asm/backoff.h +++ b/arch/sparc/include/asm/backoff.h @@ -18,7 +18,7 @@ * * When we spin, we try to use an operation that will cause the * current cpu strand to block, and therefore make the core fully - * available to any other other runnable strands. There are two + * available to any other runnable strands. There are two * options, based upon cpu capabilities. * * On all cpus prior to SPARC-T4 we do three dummy reads of the From 7780918b36489f0b2f9a3749d7be00c2ceaec513 Mon Sep 17 00:00:00 2001 From: Al Viro Date: Mon, 20 Jul 2020 02:21:51 +0100 Subject: [PATCH 05/91] sparc32: fix a user-triggerable oops in clear_user() Back in 2.1.29 the clear_user() guts (__bzero()) had been merged with memset(). Unfortunately, while all exception handlers had been copied, one of the exception table entries got lost. As the result, clear_user() starting at 128*n bytes before the end of page and spanning between 8 and 127 bytes into the next page would oops when the second page is unmapped. It's trivial to reproduce - all it takes is main() { int fd = open("/dev/zero", O_RDONLY); char *p = mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0); munmap(p + 8192, 8192); read(fd, p + 8192 - 128, 192); } which had been oopsing since March 1997. Says something about the quality of test coverage... ;-/ And while today sparc32 port is nearly dead, back in '97 it had been very much alive; in fact, sparc64 had only been in mainline for 3 months by that point... Cc: stable@kernel.org Fixes: v2.1.29 Signed-off-by: Al Viro Signed-off-by: David S. Miller --- arch/sparc/lib/memset.S | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/sparc/lib/memset.S b/arch/sparc/lib/memset.S index b89d42b29e34..f427f34b8b79 100644 --- a/arch/sparc/lib/memset.S +++ b/arch/sparc/lib/memset.S @@ -142,6 +142,7 @@ __bzero: ZERO_LAST_BLOCKS(%o0, 0x48, %g2) ZERO_LAST_BLOCKS(%o0, 0x08, %g2) 13: + EXT(12b, 13b, 21f) be 8f andcc %o1, 4, %g0 From 72a7af33e08083c118b57c93c1456cc9a5045dc6 Mon Sep 17 00:00:00 2001 From: Mike Rapoport Date: Sat, 27 Jun 2020 11:16:52 +0300 Subject: [PATCH 06/91] sparc32: use PUD rather than PGD to get PMD in srmmu_inherit_prom_mappings() This is a misprint in the page table traversal in srmmu_inherit_prom_mappings`() function which accessed a PMD entry using PGD rather than PUD. Since sparc32 has only 3 page table levels, the PGD and PUD are essentially the same and usage of __nocache_fix() removed the type checking. Use PUD for the consistency and to avoid breakage because of upcoming addition of type checking into __nocache_fix(). Fixes: 7235db268a2777bc38 ("sparc32: use pgtable-nopud instead of 4level-fixup") Signed-off-by: Mike Rapoport Signed-off-by: David S. Miller --- arch/sparc/mm/srmmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index 0070f8b9a753..f5fa07958f34 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -822,7 +822,7 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, memset(__nocache_fix(pmdp), 0, SRMMU_PMD_TABLE_SIZE); pud_set(__nocache_fix(pudp), pmdp); } - pmdp = pmd_offset(__nocache_fix(pgdp), start); + pmdp = pmd_offset(__nocache_fix(pudp), start); if (what == 1) { *(pmd_t *)__nocache_fix(pmdp) = __pmd(probed); start += PMD_SIZE; From c0d5b0c721b67d13796914031cc8b9bbf2c7f453 Mon Sep 17 00:00:00 2001 From: Mike Rapoport Date: Sat, 27 Jun 2020 11:16:53 +0300 Subject: [PATCH 07/91] sparc32: srmmu: improve type safety of __nocache_fix() The __nocache_fix(VADDR) macro is used to add an offset for pointers and its "return type" is 'void *'. We can do better and keep the type information with simply by casting the return value to (__typeof__(VADDR)). This will ".. show when those pgd/p4d/pud pointers get mis-used because they don't end up dropping the type info.." The addition of the casting to __nocache_fix() also allows to remove explicit casts at its call sites. Suggested-by: Linus Torvalds Signed-off-by: Mike Rapoport Link: https://lkml.kernel.org/r/CAHk-=wisORTa7QVPnFqNw9pFs62UiwgsD4C4d=MtYy1o4JPyGQ@mail.gmail.com Signed-off-by: David S. Miller --- arch/sparc/include/asm/pgtsrmmu.h | 2 +- arch/sparc/mm/srmmu.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/sparc/include/asm/pgtsrmmu.h b/arch/sparc/include/asm/pgtsrmmu.h index 7708d015712b..6067925972d9 100644 --- a/arch/sparc/include/asm/pgtsrmmu.h +++ b/arch/sparc/include/asm/pgtsrmmu.h @@ -113,7 +113,7 @@ extern unsigned long last_valid_pfn; extern void *srmmu_nocache_pool; #define __nocache_pa(VADDR) (((unsigned long)VADDR) - SRMMU_NOCACHE_VADDR + __pa((unsigned long)srmmu_nocache_pool)) #define __nocache_va(PADDR) (__va((unsigned long)PADDR) - (unsigned long)srmmu_nocache_pool + SRMMU_NOCACHE_VADDR) -#define __nocache_fix(VADDR) __va(__nocache_pa(VADDR)) +#define __nocache_fix(VADDR) ((__typeof__(VADDR))__va(__nocache_pa(VADDR))) /* Accessing the MMU control register. */ unsigned int srmmu_get_mmureg(void); diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index f5fa07958f34..da4e10b4cb37 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -689,7 +689,7 @@ static void __init srmmu_early_allocate_ptable_skeleton(unsigned long start, pgdp = pgd_offset_k(start); p4dp = p4d_offset(pgdp, start); pudp = pud_offset(p4dp, start); - if (pud_none(*(pud_t *)__nocache_fix(pudp))) { + if (pud_none(*__nocache_fix(pudp))) { pmdp = __srmmu_get_nocache( SRMMU_PMD_TABLE_SIZE, SRMMU_PMD_TABLE_SIZE); if (pmdp == NULL) @@ -698,7 +698,7 @@ static void __init srmmu_early_allocate_ptable_skeleton(unsigned long start, pud_set(__nocache_fix(pudp), pmdp); } pmdp = pmd_offset(__nocache_fix(pudp), start); - if (srmmu_pmd_none(*(pmd_t *)__nocache_fix(pmdp))) { + if (srmmu_pmd_none(*__nocache_fix(pmdp))) { ptep = __srmmu_get_nocache(PTE_SIZE, PTE_SIZE); if (ptep == NULL) early_pgtable_allocfail("pte"); @@ -810,11 +810,11 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, p4dp = p4d_offset(pgdp, start); pudp = pud_offset(p4dp, start); if (what == 2) { - *(pgd_t *)__nocache_fix(pgdp) = __pgd(probed); + *__nocache_fix(pgdp) = __pgd(probed); start += PGDIR_SIZE; continue; } - if (pud_none(*(pud_t *)__nocache_fix(pudp))) { + if (pud_none(*__nocache_fix(pudp))) { pmdp = __srmmu_get_nocache(SRMMU_PMD_TABLE_SIZE, SRMMU_PMD_TABLE_SIZE); if (pmdp == NULL) @@ -828,7 +828,7 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, start += PMD_SIZE; continue; } - if (srmmu_pmd_none(*(pmd_t *)__nocache_fix(pmdp))) { + if (srmmu_pmd_none(*__nocache_fix(pmdp))) { ptep = __srmmu_get_nocache(PTE_SIZE, PTE_SIZE); if (ptep == NULL) early_pgtable_allocfail("pte"); @@ -836,7 +836,7 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, pmd_set(__nocache_fix(pmdp), ptep); } ptep = pte_offset_kernel(__nocache_fix(pmdp), start); - *(pte_t *)__nocache_fix(ptep) = __pte(probed); + *__nocache_fix(ptep) = __pte(probed); start += PAGE_SIZE; } } @@ -850,7 +850,7 @@ static void __init do_large_mapping(unsigned long vaddr, unsigned long phys_base unsigned long big_pte; big_pte = KERNEL_PTE(phys_base >> 4); - *(pgd_t *)__nocache_fix(pgdp) = __pgd(big_pte); + *__nocache_fix(pgdp) = __pgd(big_pte); } /* Map sp_bank entry SP_ENTRY, starting at virtual address VBASE. */ @@ -940,7 +940,7 @@ void __init srmmu_paging_init(void) srmmu_ctx_table_phys = (ctxd_t *)__nocache_pa(srmmu_context_table); for (i = 0; i < num_contexts; i++) - srmmu_ctxd_set((ctxd_t *)__nocache_fix(&srmmu_context_table[i]), srmmu_swapper_pg_dir); + srmmu_ctxd_set(__nocache_fix(&srmmu_context_table[i]), srmmu_swapper_pg_dir); flush_cache_all(); srmmu_set_ctable_ptr((unsigned long)srmmu_ctx_table_phys); From 17ec0a17e90f4b761025a6b80c8d458b9d5f9ddd Mon Sep 17 00:00:00 2001 From: "Gustavo A. R. Silva" Date: Tue, 7 Jul 2020 12:24:36 -0500 Subject: [PATCH 08/91] sparc: Use fallthrough pseudo-keyword Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/latest/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva Signed-off-by: David S. Miller --- arch/sparc/kernel/auxio_64.c | 1 - arch/sparc/kernel/central.c | 2 +- arch/sparc/kernel/kgdb_32.c | 2 +- arch/sparc/kernel/kgdb_64.c | 2 +- arch/sparc/kernel/pcr.c | 2 +- arch/sparc/kernel/prom_32.c | 2 +- arch/sparc/kernel/signal32.c | 4 ++-- arch/sparc/kernel/signal_32.c | 4 ++-- arch/sparc/kernel/signal_64.c | 4 ++-- arch/sparc/math-emu/math_32.c | 8 ++++---- 10 files changed, 15 insertions(+), 16 deletions(-) diff --git a/arch/sparc/kernel/auxio_64.c b/arch/sparc/kernel/auxio_64.c index 4843f48bfe85..774a82b0c649 100644 --- a/arch/sparc/kernel/auxio_64.c +++ b/arch/sparc/kernel/auxio_64.c @@ -87,7 +87,6 @@ void auxio_set_lte(int on) __auxio_sbus_set_lte(on); break; case AUXIO_TYPE_EBUS: - /* FALL-THROUGH */ default: break; } diff --git a/arch/sparc/kernel/central.c b/arch/sparc/kernel/central.c index bfae98ab8638..23f8838dd96e 100644 --- a/arch/sparc/kernel/central.c +++ b/arch/sparc/kernel/central.c @@ -55,7 +55,7 @@ static int clock_board_calc_nslots(struct clock_board *p) else return 5; } - /* Fallthrough */ + fallthrough; default: return 4; } diff --git a/arch/sparc/kernel/kgdb_32.c b/arch/sparc/kernel/kgdb_32.c index 7580775a14b9..58ad3f7de1fb 100644 --- a/arch/sparc/kernel/kgdb_32.c +++ b/arch/sparc/kernel/kgdb_32.c @@ -122,7 +122,7 @@ int kgdb_arch_handle_exception(int e_vector, int signo, int err_code, linux_regs->pc = addr; linux_regs->npc = addr + 4; } - /* fall through */ + fallthrough; case 'D': case 'k': diff --git a/arch/sparc/kernel/kgdb_64.c b/arch/sparc/kernel/kgdb_64.c index 5d6c2d287e85..177746ae2c81 100644 --- a/arch/sparc/kernel/kgdb_64.c +++ b/arch/sparc/kernel/kgdb_64.c @@ -148,7 +148,7 @@ int kgdb_arch_handle_exception(int e_vector, int signo, int err_code, linux_regs->tpc = addr; linux_regs->tnpc = addr + 4; } - /* fall through */ + fallthrough; case 'D': case 'k': diff --git a/arch/sparc/kernel/pcr.c b/arch/sparc/kernel/pcr.c index c0886b400dad..2a12c86af956 100644 --- a/arch/sparc/kernel/pcr.c +++ b/arch/sparc/kernel/pcr.c @@ -359,7 +359,7 @@ int __init pcr_arch_init(void) * counter overflow interrupt so we can't make use of * their hardware currently. */ - /* fallthrough */ + fallthrough; default: err = -ENODEV; goto out_unregister; diff --git a/arch/sparc/kernel/prom_32.c b/arch/sparc/kernel/prom_32.c index da8902295c8c..3df960c137f7 100644 --- a/arch/sparc/kernel/prom_32.c +++ b/arch/sparc/kernel/prom_32.c @@ -224,7 +224,7 @@ void __init of_console_init(void) case PROMDEV_TTYB: skip = 1; - /* FALLTHRU */ + fallthrough; case PROMDEV_TTYA: type = "serial"; diff --git a/arch/sparc/kernel/signal32.c b/arch/sparc/kernel/signal32.c index e2c6f0abda00..e9695a06492f 100644 --- a/arch/sparc/kernel/signal32.c +++ b/arch/sparc/kernel/signal32.c @@ -646,7 +646,7 @@ static inline void syscall_restart32(unsigned long orig_i0, struct pt_regs *regs case ERESTARTSYS: if (!(sa->sa_flags & SA_RESTART)) goto no_system_call_restart; - /* fallthrough */ + fallthrough; case ERESTARTNOINTR: regs->u_regs[UREG_I0] = orig_i0; regs->tpc -= 4; @@ -686,7 +686,7 @@ void do_signal32(struct pt_regs * regs) regs->tpc -= 4; regs->tnpc -= 4; pt_regs_clear_syscall(regs); - /* fall through */ + fallthrough; case ERESTART_RESTARTBLOCK: regs->u_regs[UREG_G1] = __NR_restart_syscall; regs->tpc -= 4; diff --git a/arch/sparc/kernel/signal_32.c b/arch/sparc/kernel/signal_32.c index 3b005b6c3e0f..e08d294d63e4 100644 --- a/arch/sparc/kernel/signal_32.c +++ b/arch/sparc/kernel/signal_32.c @@ -441,7 +441,7 @@ static inline void syscall_restart(unsigned long orig_i0, struct pt_regs *regs, case ERESTARTSYS: if (!(sa->sa_flags & SA_RESTART)) goto no_system_call_restart; - /* fallthrough */ + fallthrough; case ERESTARTNOINTR: regs->u_regs[UREG_I0] = orig_i0; regs->pc -= 4; @@ -507,7 +507,7 @@ static void do_signal(struct pt_regs *regs, unsigned long orig_i0) regs->pc -= 4; regs->npc -= 4; pt_regs_clear_syscall(regs); - /* fall through */ + fallthrough; case ERESTART_RESTARTBLOCK: regs->u_regs[UREG_G1] = __NR_restart_syscall; regs->pc -= 4; diff --git a/arch/sparc/kernel/signal_64.c b/arch/sparc/kernel/signal_64.c index 6937339a272c..255264bcb46a 100644 --- a/arch/sparc/kernel/signal_64.c +++ b/arch/sparc/kernel/signal_64.c @@ -461,7 +461,7 @@ static inline void syscall_restart(unsigned long orig_i0, struct pt_regs *regs, case ERESTARTSYS: if (!(sa->sa_flags & SA_RESTART)) goto no_system_call_restart; - /* fallthrough */ + fallthrough; case ERESTARTNOINTR: regs->u_regs[UREG_I0] = orig_i0; regs->tpc -= 4; @@ -532,7 +532,7 @@ static void do_signal(struct pt_regs *regs, unsigned long orig_i0) regs->tpc -= 4; regs->tnpc -= 4; pt_regs_clear_syscall(regs); - /* fall through */ + fallthrough; case ERESTART_RESTARTBLOCK: regs->u_regs[UREG_G1] = __NR_restart_syscall; regs->tpc -= 4; diff --git a/arch/sparc/math-emu/math_32.c b/arch/sparc/math-emu/math_32.c index 72e560ef4a09..d5beec856146 100644 --- a/arch/sparc/math-emu/math_32.c +++ b/arch/sparc/math-emu/math_32.c @@ -359,7 +359,7 @@ static int do_one_mathemu(u32 insn, unsigned long *pfsr, unsigned long *fregs) *pfsr |= (6 << 14); return 0; /* simulate invalid_fp_register exception */ } - /* fall through */ + fallthrough; case 2: if (freg & 1) { /* doublewords must have bit 5 zeroed */ *pfsr |= (6 << 14); @@ -380,7 +380,7 @@ static int do_one_mathemu(u32 insn, unsigned long *pfsr, unsigned long *fregs) *pfsr |= (6 << 14); return 0; /* simulate invalid_fp_register exception */ } - /* fall through */ + fallthrough; case 2: if (freg & 1) { /* doublewords must have bit 5 zeroed */ *pfsr |= (6 << 14); @@ -408,13 +408,13 @@ static int do_one_mathemu(u32 insn, unsigned long *pfsr, unsigned long *fregs) *pfsr |= (6 << 14); return 0; /* simulate invalid_fp_register exception */ } - /* fall through */ + fallthrough; case 2: if (freg & 1) { /* doublewords must have bit 5 zeroed */ *pfsr |= (6 << 14); return 0; } - /* fall through */ + fallthrough; case 1: rd = (void *)&fregs[freg]; break; From 0a95a6d1a4cd6e10e70f8a6c93dc2ecd02e7d6ab Mon Sep 17 00:00:00 2001 From: Qinglang Miao Date: Tue, 28 Jul 2020 10:28:17 +0800 Subject: [PATCH 09/91] sparc: use for_each_child_of_node() macro Use for_each_child_of_node() macro instead of open coding it. Signed-off-by: Qinglang Miao Signed-off-by: David S. Miller --- arch/sparc/kernel/pci.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c index 5d45b6d766d6..9c2b720bfd20 100644 --- a/arch/sparc/kernel/pci.c +++ b/arch/sparc/kernel/pci.c @@ -552,9 +552,8 @@ static void pci_of_scan_bus(struct pci_pbm_info *pbm, pci_info(bus, "scan_bus[%pOF] bus no %d\n", node, bus->number); - child = NULL; prev_devfn = -1; - while ((child = of_get_next_child(node, child)) != NULL) { + for_each_child_of_node(node, child) { if (ofpci_verbose) pci_info(bus, " * %pOF\n", child); reg = of_get_property(child, "reg", ®len); From 4cc8ca08ed0934cfcb9a4f3998119ea954780b02 Mon Sep 17 00:00:00 2001 From: Peter Collingbourne Date: Mon, 17 Aug 2020 20:17:45 -0700 Subject: [PATCH 10/91] sparc: remove SA_STATIC_ALLOC macro definition This macro is not exposed to uapi and is unreferenced in the kernel. The last reference to it was removed in commit 4c6773c3954c ("sparc32,leon: add support for extended interrupt controller"), so remove the definition. Signed-off-by: Peter Collingbourne Signed-off-by: David S. Miller --- arch/sparc/include/asm/signal.h | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/arch/sparc/include/asm/signal.h b/arch/sparc/include/asm/signal.h index 827b73a97f8a..28f81081e37d 100644 --- a/arch/sparc/include/asm/signal.h +++ b/arch/sparc/include/asm/signal.h @@ -9,18 +9,6 @@ #include #ifndef __ASSEMBLY__ -/* - * DJHR - * SA_STATIC_ALLOC is used for the sparc32 system to indicate that this - * interrupt handler's irq structure should be statically allocated - * by the request_irq routine. - * The alternative is that arch/sparc/kernel/irq.c has carnal knowledge - * of interrupt usage and that sucks. Also without a flag like this - * it may be possible for the free_irq routine to attempt to free - * statically allocated data.. which is NOT GOOD. - * - */ -#define SA_STATIC_ALLOC 0x8000 #define __ARCH_HAS_KA_RESTORER #define __ARCH_HAS_SA_RESTORER From b6b9b67d67f0962e9e976c2d3623a992ef7f30b0 Mon Sep 17 00:00:00 2001 From: Ben Hutchings Date: Thu, 27 Aug 2020 17:47:36 +0100 Subject: [PATCH 11/91] sparc32: signal: Fix stack trampoline for RT signals The stack trampoline generated by the sparc32 native version of setup_rt_frame() calls sigreturn(), not rt_sigreturn(). This will crash the task if it's ever used. (glibc sets its own restorer, so was not affected.) The sparc64 compat implementation has the right syscall number. This is untested; I have no way to run a sparc32 kernel. Signed-off-by: Ben Hutchings Signed-off-by: David S. Miller --- arch/sparc/kernel/signal_32.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/sparc/kernel/signal_32.c b/arch/sparc/kernel/signal_32.c index e08d294d63e4..76b934241e09 100644 --- a/arch/sparc/kernel/signal_32.c +++ b/arch/sparc/kernel/signal_32.c @@ -401,8 +401,8 @@ static int setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs, else { regs->u_regs[UREG_I7] = (unsigned long)(&(sf->insns[0]) - 2); - /* mov __NR_sigreturn, %g1 */ - err |= __put_user(0x821020d8, &sf->insns[0]); + /* mov __NR_rt_sigreturn, %g1 */ + err |= __put_user(0x82102065, &sf->insns[0]); /* t 0x10 */ err |= __put_user(0x91d02010, &sf->insns[1]); From bf8c5541df535d1c0e122c0bcc839894961984bc Mon Sep 17 00:00:00 2001 From: Jing Xiangfeng Date: Fri, 11 Sep 2020 15:13:41 +0800 Subject: [PATCH 12/91] sbus: char: Remove meaningless jump label out_free After commit 57a4a3d7f756 ("display7seg: Introduce the use of the managed version of kzalloc"), The out_free jump label has nothing to do but goto out. So remove it. Signed-off-by: Jing Xiangfeng Signed-off-by: David S. Miller --- drivers/sbus/char/display7seg.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/sbus/char/display7seg.c b/drivers/sbus/char/display7seg.c index fad936eb845f..00e72b97d0b6 100644 --- a/drivers/sbus/char/display7seg.c +++ b/drivers/sbus/char/display7seg.c @@ -186,7 +186,7 @@ static int d7s_probe(struct platform_device *op) p->regs = of_ioremap(&op->resource[0], 0, sizeof(u8), "d7s"); if (!p->regs) { printk(KERN_ERR PFX "Cannot map chip registers\n"); - goto out_free; + goto out; } err = misc_register(&d7s_miscdev); @@ -228,8 +228,6 @@ out: out_iounmap: of_iounmap(&op->resource[0], p->regs, sizeof(u8)); - -out_free: goto out; } From 634c21bb9867e06221ee1527c5e157e01cd7712c Mon Sep 17 00:00:00 2001 From: "Gustavo A. R. Silva" Date: Fri, 20 Nov 2020 12:32:20 -0600 Subject: [PATCH 13/91] security: keys: Fix fall-through warnings for Clang In preparation to enable -Wimplicit-fallthrough for Clang, fix a warning by explicitly adding a break statement instead of letting the code fall through to the next case. Link: https://github.com/KSPP/linux/issues/115 Signed-off-by: Gustavo A. R. Silva Signed-off-by: David Howells Reviewed-by: Jarkko Sakkinen Reviewed-by: Ben Boeckel --- security/keys/process_keys.c | 1 + 1 file changed, 1 insertion(+) diff --git a/security/keys/process_keys.c b/security/keys/process_keys.c index 1fe8b934f656..e3d79a7b6db6 100644 --- a/security/keys/process_keys.c +++ b/security/keys/process_keys.c @@ -783,6 +783,7 @@ try_again: if (need_perm != KEY_AUTHTOKEN_OVERRIDE && need_perm != KEY_DEFER_PERM_CHECK) goto invalid_key; + break; case 0: break; } From 796e46f9e2cb2d823578044598ee8fe77f86e3f7 Mon Sep 17 00:00:00 2001 From: Jann Horn Date: Tue, 24 Nov 2020 00:54:00 +0100 Subject: [PATCH 14/91] keys: Remove outdated __user annotations When the semantics of the ->read() handlers were changed such that "buffer" is a kernel pointer, some __user annotations survived. Since they're wrong now, get rid of them. Fixes: d3ec10aa9581 ("KEYS: Don't write out to userspace while holding key semaphore") Signed-off-by: Jann Horn Signed-off-by: David Howells Reviewed-by: Ben Boeckel --- security/keys/keyring.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/security/keys/keyring.c b/security/keys/keyring.c index 14abfe765b7e..977066208387 100644 --- a/security/keys/keyring.c +++ b/security/keys/keyring.c @@ -452,7 +452,7 @@ static void keyring_describe(const struct key *keyring, struct seq_file *m) struct keyring_read_iterator_context { size_t buflen; size_t count; - key_serial_t __user *buffer; + key_serial_t *buffer; }; static int keyring_read_iterator(const void *object, void *data) @@ -479,7 +479,7 @@ static int keyring_read_iterator(const void *object, void *data) * times. */ static long keyring_read(const struct key *keyring, - char __user *buffer, size_t buflen) + char *buffer, size_t buflen) { struct keyring_read_iterator_context ctx; long ret; @@ -491,7 +491,7 @@ static long keyring_read(const struct key *keyring, /* Copy as many key IDs as fit into the buffer */ if (buffer && buflen) { - ctx.buffer = (key_serial_t __user *)buffer; + ctx.buffer = (key_serial_t *)buffer; ctx.buflen = buflen; ctx.count = 0; ret = assoc_array_iterate(&keyring->keys, From 8fe62e0c0e2efa5437f3ee81b65d69e70a45ecd2 Mon Sep 17 00:00:00 2001 From: Gabriel Krisman Bertazi Date: Tue, 24 Nov 2020 15:28:02 -0500 Subject: [PATCH 15/91] watch_queue: Drop references to /dev/watch_queue The merged API doesn't use a watch_queue device, but instead relies on pipes, so let the documentation reflect that. Fixes: f7e47677e39a ("watch_queue: Add a key/keyring notification facility") Signed-off-by: Gabriel Krisman Bertazi Signed-off-by: David Howells Acked-by: Jarkko Sakkinen Reviewed-by: Ben Boeckel --- Documentation/security/keys/core.rst | 4 ++-- samples/Kconfig | 2 +- samples/watch_queue/watch_test.c | 2 +- security/keys/Kconfig | 8 ++++---- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Documentation/security/keys/core.rst b/Documentation/security/keys/core.rst index aa0081685ee1..b3ed5c581034 100644 --- a/Documentation/security/keys/core.rst +++ b/Documentation/security/keys/core.rst @@ -1040,8 +1040,8 @@ The keyctl syscall functions are: "key" is the ID of the key to be watched. - "queue_fd" is a file descriptor referring to an open "/dev/watch_queue" - which manages the buffer into which notifications will be delivered. + "queue_fd" is a file descriptor referring to an open pipe which + manages the buffer into which notifications will be delivered. "filter" is either NULL to remove a watch or a filter specification to indicate what events are required from the key. diff --git a/samples/Kconfig b/samples/Kconfig index 0ed6e4d71d87..e76cdfc50e25 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -210,7 +210,7 @@ config SAMPLE_WATCHDOG depends on CC_CAN_LINK config SAMPLE_WATCH_QUEUE - bool "Build example /dev/watch_queue notification consumer" + bool "Build example watch_queue notification API consumer" depends on CC_CAN_LINK && HEADERS_INSTALL help Build example userspace program to use the new mount_notify(), diff --git a/samples/watch_queue/watch_test.c b/samples/watch_queue/watch_test.c index 46e618a897fe..8c6cb57d5cfc 100644 --- a/samples/watch_queue/watch_test.c +++ b/samples/watch_queue/watch_test.c @@ -1,5 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 -/* Use /dev/watch_queue to watch for notifications. +/* Use watch_queue API to watch for notifications. * * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved. * Written by David Howells (dhowells@redhat.com) diff --git a/security/keys/Kconfig b/security/keys/Kconfig index 83bc23409164..c161642a8484 100644 --- a/security/keys/Kconfig +++ b/security/keys/Kconfig @@ -119,7 +119,7 @@ config KEY_NOTIFICATIONS bool "Provide key/keyring change notifications" depends on KEYS && WATCH_QUEUE help - This option provides support for getting change notifications on keys - and keyrings on which the caller has View permission. This makes use - of the /dev/watch_queue misc device to handle the notification - buffer and provides KEYCTL_WATCH_KEY to enable/disable watches. + This option provides support for getting change notifications + on keys and keyrings on which the caller has View permission. + This makes use of pipes to handle the notification buffer and + provides KEYCTL_WATCH_KEY to enable/disable watches. From 272a121940a286d7abaf7ac3ec5a37c5dbfa7b89 Mon Sep 17 00:00:00 2001 From: Denis Efremov Date: Thu, 27 Aug 2020 10:29:23 +0300 Subject: [PATCH 16/91] security/keys: use kvfree_sensitive() Use kvfree_sensitive() instead of open-coding it. Signed-off-by: Denis Efremov Signed-off-by: David Howells Reviewed-by: Jarkko Sakkinen Reviewed-by: Ben Boeckel --- security/keys/big_key.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/security/keys/big_key.c b/security/keys/big_key.c index 691347dea3c1..d17e5f09eeb8 100644 --- a/security/keys/big_key.c +++ b/security/keys/big_key.c @@ -121,8 +121,7 @@ int big_key_preparse(struct key_preparsed_payload *prep) *path = file->f_path; path_get(path); fput(file); - memzero_explicit(buf, enclen); - kvfree(buf); + kvfree_sensitive(buf, enclen); } else { /* Just store the data in a buffer */ void *data = kmalloc(datalen, GFP_KERNEL); @@ -140,8 +139,7 @@ err_fput: err_enckey: kfree_sensitive(enckey); error: - memzero_explicit(buf, enclen); - kvfree(buf); + kvfree_sensitive(buf, enclen); return ret; } @@ -273,8 +271,7 @@ long big_key_read(const struct key *key, char *buffer, size_t buflen) err_fput: fput(file); error: - memzero_explicit(buf, enclen); - kvfree(buf); + kvfree_sensitive(buf, enclen); } else { ret = datalen; memcpy(buffer, key->payload.data[big_key_data], datalen); From 60f0f0b3cdfda667a8d1897b3004173a582bcd72 Mon Sep 17 00:00:00 2001 From: Krzysztof Kozlowski Date: Thu, 29 Oct 2020 16:48:30 +0100 Subject: [PATCH 17/91] KEYS: asymmetric: Fix kerneldoc Fix W=1 compile warnings (invalid kerneldoc): crypto/asymmetric_keys/asymmetric_type.c:160: warning: Function parameter or member 'kid1' not described in 'asymmetric_key_id_same' crypto/asymmetric_keys/asymmetric_type.c:160: warning: Function parameter or member 'kid2' not described in 'asymmetric_key_id_same' crypto/asymmetric_keys/asymmetric_type.c:160: warning: Excess function parameter 'kid_1' description in 'asymmetric_key_id_same' crypto/asymmetric_keys/asymmetric_type.c:160: warning: Excess function parameter 'kid_2' description in 'asymmetric_key_id_same' Signed-off-by: Krzysztof Kozlowski Signed-off-by: David Howells Acked-by: Randy Dunlap Reviewed-by: Ben Boeckel Reviewed-by: Jarkko Sakkinen --- crypto/asymmetric_keys/asymmetric_type.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/crypto/asymmetric_keys/asymmetric_type.c b/crypto/asymmetric_keys/asymmetric_type.c index 33e77d846caa..ad8af3d70ac0 100644 --- a/crypto/asymmetric_keys/asymmetric_type.c +++ b/crypto/asymmetric_keys/asymmetric_type.c @@ -152,7 +152,8 @@ EXPORT_SYMBOL_GPL(asymmetric_key_generate_id); /** * asymmetric_key_id_same - Return true if two asymmetric keys IDs are the same. - * @kid_1, @kid_2: The key IDs to compare + * @kid1: The key ID to compare + * @kid2: The key ID to compare */ bool asymmetric_key_id_same(const struct asymmetric_key_id *kid1, const struct asymmetric_key_id *kid2) @@ -168,7 +169,8 @@ EXPORT_SYMBOL_GPL(asymmetric_key_id_same); /** * asymmetric_key_id_partial - Return true if two asymmetric keys IDs * partially match - * @kid_1, @kid_2: The key IDs to compare + * @kid1: The key ID to compare + * @kid2: The key ID to compare */ bool asymmetric_key_id_partial(const struct asymmetric_key_id *kid1, const struct asymmetric_key_id *kid2) From 328c95db01df9d8875f77e49ee4322e60e1337cd Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Fri, 7 Aug 2020 09:51:23 -0700 Subject: [PATCH 18/91] security: keys: delete repeated words in comments Drop repeated words in comments. {to, will, the} Signed-off-by: Randy Dunlap Signed-off-by: David Howells Reviewed-by: Jarkko Sakkinen Reviewed-by: Ben Boeckel Cc: keyrings@vger.kernel.org Cc: James Morris Cc: "Serge E. Hallyn" Cc: linux-security-module@vger.kernel.org --- security/keys/keyctl.c | 2 +- security/keys/keyring.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c index 61a614c21b9b..96a92a645216 100644 --- a/security/keys/keyctl.c +++ b/security/keys/keyctl.c @@ -506,7 +506,7 @@ error: * keyring, otherwise replace the link to the matching key with a link to the * new key. * - * The key must grant the caller Link permission and the the keyring must grant + * The key must grant the caller Link permission and the keyring must grant * the caller Write permission. Furthermore, if an additional link is created, * the keyring's quota will be extended. * diff --git a/security/keys/keyring.c b/security/keys/keyring.c index 977066208387..5e6a90760753 100644 --- a/security/keys/keyring.c +++ b/security/keys/keyring.c @@ -881,7 +881,7 @@ found: * * Keys are matched to the type provided and are then filtered by the match * function, which is given the description to use in any way it sees fit. The - * match function may use any attributes of a key that it wishes to to + * match function may use any attributes of a key that it wishes to * determine the match. Normally the match function from the key type would be * used. * @@ -1204,7 +1204,7 @@ static int keyring_detect_cycle_iterator(const void *object, } /* - * See if a cycle will will be created by inserting acyclic tree B in acyclic + * See if a cycle will be created by inserting acyclic tree B in acyclic * tree A at the topmost level (ie: as a direct child of A). * * Since we are adding B to A at the top level, checking for cycles should just From c224926edfc2f774df6aefa865e31a0a00e24dde Mon Sep 17 00:00:00 2001 From: Tom Rix Date: Wed, 22 Jul 2020 06:46:10 -0700 Subject: [PATCH 19/91] KEYS: remove redundant memset Reviewing use of memset in keyctl_pkey.c keyctl_pkey_params_get prologue code to set params up memset(params, 0, sizeof(*params)); params->encoding = "raw"; keyctl_pkey_query has the same prologue and calls keyctl_pkey_params_get. So remove the prologue. Signed-off-by: Tom Rix Signed-off-by: David Howells Reviewed-by: Ben Boeckel --- security/keys/keyctl_pkey.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/security/keys/keyctl_pkey.c b/security/keys/keyctl_pkey.c index 931d8dfb4a7f..5de0d599a274 100644 --- a/security/keys/keyctl_pkey.c +++ b/security/keys/keyctl_pkey.c @@ -166,8 +166,6 @@ long keyctl_pkey_query(key_serial_t id, struct kernel_pkey_query res; long ret; - memset(¶ms, 0, sizeof(params)); - ret = keyctl_pkey_params_get(id, _info, ¶ms); if (ret < 0) goto error; From 1539dd785a1c7be294fcdfaafc3137dab8321806 Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Sun, 19 Jul 2020 11:07:34 -0700 Subject: [PATCH 20/91] crypto: asymmetric_keys: fix some comments in pkcs7_parser.h Drop the doubled word "the" in a comment. Change "THis" to "This". Signed-off-by: Randy Dunlap Signed-off-by: David Howells Reviewed-by: Ben Boeckel Cc: keyrings@vger.kernel.org --- crypto/asymmetric_keys/pkcs7_parser.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/crypto/asymmetric_keys/pkcs7_parser.h b/crypto/asymmetric_keys/pkcs7_parser.h index 6565fdc2d4ca..e17f7ce4fb43 100644 --- a/crypto/asymmetric_keys/pkcs7_parser.h +++ b/crypto/asymmetric_keys/pkcs7_parser.h @@ -41,10 +41,9 @@ struct pkcs7_signed_info { * * This contains the generated digest of _either_ the Content Data or * the Authenticated Attributes [RFC2315 9.3]. If the latter, one of - * the attributes contains the digest of the the Content Data within - * it. + * the attributes contains the digest of the Content Data within it. * - * THis also contains the issuing cert serial number and issuer's name + * This also contains the issuing cert serial number and issuer's name * [PKCS#7 or CMS ver 1] or issuing cert's SKID [CMS ver 3]. */ struct public_key_signature *sig; From c52b7c807b0a6ae26582208a0b07c2a6a796b50f Mon Sep 17 00:00:00 2001 From: "Alexander A. Klimov" Date: Thu, 16 Jul 2020 21:52:27 +0200 Subject: [PATCH 21/91] encrypted-keys: Replace HTTP links with HTTPS ones Rationale: Reduces attack surface on kernel devs opening the links for MITM as HTTPS traffic is much harder to manipulate. Deterministic algorithm: For each file: If not .svg: For each line: If doesn't contain `\bxmlns\b`: For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`: If neither `\bgnu\.org/license`, nor `\bmozilla\.org/MPL\b`: If both the HTTP and HTTPS versions return 200 OK and serve the same content: Replace HTTP with HTTPS. Signed-off-by: Alexander A. Klimov Signed-off-by: David Howells Reviewed-by: Ben Boeckel --- include/keys/encrypted-type.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/keys/encrypted-type.h b/include/keys/encrypted-type.h index 38afb341c3f2..abfcbe02001a 100644 --- a/include/keys/encrypted-type.h +++ b/include/keys/encrypted-type.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2010 IBM Corporation * Copyright (C) 2010 Politecnico di Torino, Italy - * TORSEC group -- http://security.polito.it + * TORSEC group -- https://security.polito.it * * Authors: * Mimi Zohar From d13fc8747218c1a5c7bdf69c54a4c64ea52f0d81 Mon Sep 17 00:00:00 2001 From: Alex Shi Date: Fri, 13 Nov 2020 16:58:15 +0800 Subject: [PATCH 22/91] PKCS#7: drop function from kernel-doc pkcs7_validate_trust_one The function is a static function, so no needs add into kernel-doc. and we could avoid warning: crypto/asymmetric_keys/pkcs7_trust.c:25: warning: Function parameter or member 'pkcs7' not described in 'pkcs7_validate_trust_one' crypto/asymmetric_keys/pkcs7_trust.c:25: warning: Function parameter or member 'sinfo' not described in 'pkcs7_validate_trust_one' crypto/asymmetric_keys/pkcs7_trust.c:25: warning: Function parameter or member 'trust_keyring' not described in 'pkcs7_validate_trust_one' Signed-off-by: Alex Shi Signed-off-by: David Howells Reviewed-by: Ben Boeckel Cc: Herbert Xu Cc: "David S. Miller" Cc: keyrings@vger.kernel.org Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- crypto/asymmetric_keys/pkcs7_trust.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crypto/asymmetric_keys/pkcs7_trust.c b/crypto/asymmetric_keys/pkcs7_trust.c index 61af3c4d82cc..b531df2013c4 100644 --- a/crypto/asymmetric_keys/pkcs7_trust.c +++ b/crypto/asymmetric_keys/pkcs7_trust.c @@ -16,7 +16,7 @@ #include #include "pkcs7_parser.h" -/** +/* * Check the trust on one PKCS#7 SignedInfo block. */ static int pkcs7_validate_trust_one(struct pkcs7_message *pkcs7, From 3c0940c4ff078064b9e67f52a18cd543ad467fb3 Mon Sep 17 00:00:00 2001 From: YueHaibing Date: Wed, 2 Sep 2020 22:07:17 +0800 Subject: [PATCH 23/91] crypto: pkcs7: Use match_string() helper to simplify the code match_string() returns the array index of a matching string. Use it instead of the open-coded implementation. Signed-off-by: YueHaibing Signed-off-by: David Howells Reviewed-by: Ben Boeckel --- crypto/asymmetric_keys/pkcs7_verify.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/crypto/asymmetric_keys/pkcs7_verify.c b/crypto/asymmetric_keys/pkcs7_verify.c index ce49820caa97..0b4d07aa8811 100644 --- a/crypto/asymmetric_keys/pkcs7_verify.c +++ b/crypto/asymmetric_keys/pkcs7_verify.c @@ -141,11 +141,10 @@ int pkcs7_get_digest(struct pkcs7_message *pkcs7, const u8 **buf, u32 *len, *buf = sinfo->sig->digest; *len = sinfo->sig->digest_size; - for (i = 0; i < HASH_ALGO__LAST; i++) - if (!strcmp(hash_algo_name[i], sinfo->sig->hash_algo)) { - *hash_algo = i; - break; - } + i = match_string(hash_algo_name, HASH_ALGO__LAST, + sinfo->sig->hash_algo); + if (i >= 0) + *hash_algo = i; return 0; } From 464e96aeb16ab4e071d353f69ebbddfa08c8d731 Mon Sep 17 00:00:00 2001 From: Tom Rix Date: Fri, 27 Nov 2020 11:15:43 -0800 Subject: [PATCH 24/91] keys: remove trailing semicolon in macro definition The macro use will already have a semicolon. Signed-off-by: Tom Rix Signed-off-by: David Howells Acked-by: Jarkko Sakkinen Reviewed-by: Ben Boeckel --- include/linux/key.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/key.h b/include/linux/key.h index 0f2e24f13c2b..1b0837c975b9 100644 --- a/include/linux/key.h +++ b/include/linux/key.h @@ -360,7 +360,7 @@ static inline struct key *request_key(struct key_type *type, * completion of keys undergoing construction with a non-interruptible wait. */ #define request_key_net(type, description, net, callout_info) \ - request_key_tag(type, description, net->key_domain, callout_info); + request_key_tag(type, description, net->key_domain, callout_info) /** * request_key_net_rcu - Request a key for a net namespace under RCU conditions @@ -372,7 +372,7 @@ static inline struct key *request_key(struct key_type *type, * network namespace are used. */ #define request_key_net_rcu(type, description, net) \ - request_key_rcu(type, description, net->key_domain); + request_key_rcu(type, description, net->key_domain) #endif /* CONFIG_NET */ extern int wait_for_key_construction(struct key *key, bool intr); From 09315b2d0d6944b9e249003c04abb88b5594a683 Mon Sep 17 00:00:00 2001 From: Tianjia Zhang Date: Wed, 18 Nov 2020 20:30:31 +0800 Subject: [PATCH 25/91] crypto: public_key: Remove redundant header file from public_key.h The akcipher.h header file was originally introduced in SM2, and then the definition of SM2 was moved to the existing code. This header file is left and should be removed. Signed-off-by: Tianjia Zhang Signed-off-by: David Howells Reviewed-by: Ben Boeckel --- include/crypto/public_key.h | 1 - 1 file changed, 1 deletion(-) diff --git a/include/crypto/public_key.h b/include/crypto/public_key.h index 948c5203ca9c..47accec68cb0 100644 --- a/include/crypto/public_key.h +++ b/include/crypto/public_key.h @@ -12,7 +12,6 @@ #include #include -#include /* * Cryptographic data for the public-key subtype of the asymmetric key type. From 0b2d443bf52756a9c364a41492dae537bc62683f Mon Sep 17 00:00:00 2001 From: Alex Shi Date: Fri, 6 Nov 2020 22:38:33 +0800 Subject: [PATCH 26/91] certs/blacklist: fix kernel doc interface issue certs/blacklist.c:84: warning: Function parameter or member 'hash' not described in 'mark_hash_blacklisted' Signed-off-by: Alex Shi Signed-off-by: David Howells Reviewed-by: Ben Boeckel Cc: David Woodhouse Cc: keyrings@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- certs/blacklist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certs/blacklist.c b/certs/blacklist.c index 6514f9ebc943..2719fb2fbc1c 100644 --- a/certs/blacklist.c +++ b/certs/blacklist.c @@ -78,7 +78,7 @@ static struct key_type key_type_blacklist = { /** * mark_hash_blacklisted - Add a hash to the system blacklist - * @hash - The hash as a hex string with a type prefix (eg. "tbs:23aa429783") + * @hash: The hash as a hex string with a type prefix (eg. "tbs:23aa429783") */ int mark_hash_blacklisted(const char *hash) { From 84ffbefd657b25dbca0dbd7772226fb83b8213b8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= Date: Fri, 20 Nov 2020 19:04:18 +0100 Subject: [PATCH 27/91] certs: Fix blacklisted hexadecimal hash string check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When looking for a blacklisted hash, bin2hex() is used to transform a binary hash to an ascii (lowercase) hexadecimal string. This string is then search for in the description of the keys from the blacklist keyring. When adding a key to the blacklist keyring, blacklist_vet_description() checks the hash prefix and the hexadecimal string, but not that this string is lowercase. It is then valid to set hashes with uppercase hexadecimal, which will be silently ignored by the kernel. Add an additional check to blacklist_vet_description() to check that hexadecimal strings are in lowercase. Signed-off-by: Mickaël Salaün Signed-off-by: David Howells Reviewed-by: Ben Boeckel Cc: David Woodhouse --- certs/blacklist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certs/blacklist.c b/certs/blacklist.c index 2719fb2fbc1c..a888b934a1cd 100644 --- a/certs/blacklist.c +++ b/certs/blacklist.c @@ -37,7 +37,7 @@ static int blacklist_vet_description(const char *desc) found_colon: desc++; for (; *desc; desc++) { - if (!isxdigit(*desc)) + if (!isxdigit(*desc) || isupper(*desc)) return -EINVAL; n++; } From f14602caf4faef18999985bc87a414b552844ad2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= Date: Fri, 20 Nov 2020 19:04:22 +0100 Subject: [PATCH 28/91] PKCS#7: Fix missing include MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add missing linux/types.h for size_t. [DH: Changed from stddef.h] Signed-off-by: Mickaël Salaün Signed-off-by: David Howells Reviewed-by: Ben Boeckel --- include/linux/verification.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/verification.h b/include/linux/verification.h index 911ab7c2b1ab..a655923335ae 100644 --- a/include/linux/verification.h +++ b/include/linux/verification.h @@ -8,6 +8,8 @@ #ifndef _LINUX_VERIFICATION_H #define _LINUX_VERIFICATION_H +#include + /* * Indicate that both builtin trusted keys and secondary trusted keys * should be used. From 4993e1f9479a4161fd7d93e2b8b30b438f00cb0f Mon Sep 17 00:00:00 2001 From: David Howells Date: Fri, 20 Nov 2020 19:04:23 +0100 Subject: [PATCH 29/91] certs: Fix blacklist flag type confusion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit KEY_FLAG_KEEP is not meant to be passed to keyring_alloc() or key_alloc(), as these only take KEY_ALLOC_* flags. KEY_FLAG_KEEP has the same value as KEY_ALLOC_BYPASS_RESTRICTION, but fortunately only key_create_or_update() uses it. LSMs using the key_alloc hook don't check that flag. KEY_FLAG_KEEP is then ignored but fortunately (again) the root user cannot write to the blacklist keyring, so it is not possible to remove a key/hash from it. Fix this by adding a KEY_ALLOC_SET_KEEP flag that tells key_alloc() to set KEY_FLAG_KEEP on the new key. blacklist_init() can then, correctly, pass this to keyring_alloc(). We can also use this in ima_mok_init() rather than setting the flag manually. Note that this doesn't fix an observable bug with the current implementation but it is required to allow addition of new hashes to the blacklist in the future without making it possible for them to be removed. Fixes: 734114f8782f ("KEYS: Add a system blacklist keyring") Reported-by: Mickaël Salaün Signed-off-by: David Howells cc: Mickaël Salaün cc: Mimi Zohar Cc: David Woodhouse --- certs/blacklist.c | 2 +- include/linux/key.h | 1 + security/integrity/ima/ima_mok.c | 5 ++--- security/keys/key.c | 2 ++ 4 files changed, 6 insertions(+), 4 deletions(-) diff --git a/certs/blacklist.c b/certs/blacklist.c index a888b934a1cd..029471947838 100644 --- a/certs/blacklist.c +++ b/certs/blacklist.c @@ -162,7 +162,7 @@ static int __init blacklist_init(void) KEY_USR_VIEW | KEY_USR_READ | KEY_USR_SEARCH, KEY_ALLOC_NOT_IN_QUOTA | - KEY_FLAG_KEEP, + KEY_ALLOC_SET_KEEP, NULL, NULL); if (IS_ERR(blacklist_keyring)) panic("Can't allocate system blacklist keyring\n"); diff --git a/include/linux/key.h b/include/linux/key.h index 1b0837c975b9..7febc4881363 100644 --- a/include/linux/key.h +++ b/include/linux/key.h @@ -289,6 +289,7 @@ extern struct key *key_alloc(struct key_type *type, #define KEY_ALLOC_BUILT_IN 0x0004 /* Key is built into kernel */ #define KEY_ALLOC_BYPASS_RESTRICTION 0x0008 /* Override the check on restricted keyrings */ #define KEY_ALLOC_UID_KEYRING 0x0010 /* allocating a user or user session keyring */ +#define KEY_ALLOC_SET_KEEP 0x0020 /* Set the KEEP flag on the key/keyring */ extern void key_revoke(struct key *key); extern void key_invalidate(struct key *key); diff --git a/security/integrity/ima/ima_mok.c b/security/integrity/ima/ima_mok.c index 36cadadbfba4..1e5c01916173 100644 --- a/security/integrity/ima/ima_mok.c +++ b/security/integrity/ima/ima_mok.c @@ -38,13 +38,12 @@ __init int ima_mok_init(void) (KEY_POS_ALL & ~KEY_POS_SETATTR) | KEY_USR_VIEW | KEY_USR_READ | KEY_USR_WRITE | KEY_USR_SEARCH, - KEY_ALLOC_NOT_IN_QUOTA, + KEY_ALLOC_NOT_IN_QUOTA | + KEY_ALLOC_SET_KEEP, restriction, NULL); if (IS_ERR(ima_blacklist_keyring)) panic("Can't allocate IMA blacklist keyring."); - - set_bit(KEY_FLAG_KEEP, &ima_blacklist_keyring->flags); return 0; } device_initcall(ima_mok_init); diff --git a/security/keys/key.c b/security/keys/key.c index ebe752b137aa..c45afdd1dfbb 100644 --- a/security/keys/key.c +++ b/security/keys/key.c @@ -303,6 +303,8 @@ struct key *key_alloc(struct key_type *type, const char *desc, key->flags |= 1 << KEY_FLAG_BUILTIN; if (flags & KEY_ALLOC_UID_KEYRING) key->flags |= 1 << KEY_FLAG_UID_KEYRING; + if (flags & KEY_ALLOC_SET_KEEP) + key->flags |= 1 << KEY_FLAG_KEEP; #ifdef KEY_DEBUGGING key->magic = KEY_DEBUG_MAGIC; From a6cb0ab7daf78ce87d70212dfdb01a622d833500 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= Date: Fri, 20 Nov 2020 19:04:25 +0100 Subject: [PATCH 30/91] certs: Replace K{U,G}IDT_INIT() with GLOBAL_ROOT_{U,G}ID MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Align with the new macros and add appropriate include files. Signed-off-by: Mickaël Salaün Signed-off-by: David Howells Cc: David Woodhouse --- certs/blacklist.c | 4 ++-- certs/system_keyring.c | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/certs/blacklist.c b/certs/blacklist.c index 029471947838..bffe4c6f4a9e 100644 --- a/certs/blacklist.c +++ b/certs/blacklist.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include "blacklist.h" @@ -156,8 +157,7 @@ static int __init blacklist_init(void) blacklist_keyring = keyring_alloc(".blacklist", - KUIDT_INIT(0), KGIDT_INIT(0), - current_cred(), + GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, current_cred(), (KEY_POS_ALL & ~KEY_POS_SETATTR) | KEY_USR_VIEW | KEY_USR_READ | KEY_USR_SEARCH, diff --git a/certs/system_keyring.c b/certs/system_keyring.c index 798291177186..4b693da488f1 100644 --- a/certs/system_keyring.c +++ b/certs/system_keyring.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -98,7 +99,7 @@ static __init int system_trusted_keyring_init(void) builtin_trusted_keys = keyring_alloc(".builtin_trusted_keys", - KUIDT_INIT(0), KGIDT_INIT(0), current_cred(), + GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, current_cred(), ((KEY_POS_ALL & ~KEY_POS_SETATTR) | KEY_USR_VIEW | KEY_USR_READ | KEY_USR_SEARCH), KEY_ALLOC_NOT_IN_QUOTA, @@ -109,7 +110,7 @@ static __init int system_trusted_keyring_init(void) #ifdef CONFIG_SECONDARY_TRUSTED_KEYRING secondary_trusted_keys = keyring_alloc(".secondary_trusted_keys", - KUIDT_INIT(0), KGIDT_INIT(0), current_cred(), + GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, current_cred(), ((KEY_POS_ALL & ~KEY_POS_SETATTR) | KEY_USR_VIEW | KEY_USR_READ | KEY_USR_SEARCH | KEY_USR_WRITE), From 8f0bfc25c907f38e7f9dc498e8f43000d77327ef Mon Sep 17 00:00:00 2001 From: Lukas Bulwahn Date: Mon, 25 Jan 2021 17:14:09 +0100 Subject: [PATCH 31/91] watch_queue: rectify kernel-doc for init_watch() The command './scripts/kernel-doc -none kernel/watch_queue.c' reported a mismatch in the kernel-doc of init_watch(). Rectify the kernel-doc, such that no issues remain for watch_queue.c. Signed-off-by: Lukas Bulwahn Signed-off-by: David Howells --- kernel/watch_queue.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/watch_queue.c b/kernel/watch_queue.c index 0ef8f65bd2d7..9c9eb20dd2c5 100644 --- a/kernel/watch_queue.c +++ b/kernel/watch_queue.c @@ -413,7 +413,7 @@ static void put_watch(struct watch *watch) } /** - * init_watch_queue - Initialise a watch + * init_watch - Initialise a watch * @watch: The watch to initialise. * @wqueue: The queue to assign. * From 9dc00b25eadf2908ae76ac0607b55a9f4e0e0cdc Mon Sep 17 00:00:00 2001 From: Barry Song Date: Sat, 6 Feb 2021 00:33:25 +1300 Subject: [PATCH 32/91] dma-mapping: benchmark: pretend DMA is transmitting In a real dma mapping user case, after dma_map is done, data will be transmit. Thus, in multi-threaded user scenario, IOMMU contention should not be that severe. For example, if users enable multiple threads to send network packets through 1G/10G/100Gbps NIC, usually the steps will be: map -> transmission -> unmap. Transmission delay reduces the contention of IOMMU. Here a delay is added to simulate the transmission between map and unmap so that the tested result could be more accurate for TX and simple RX. A typical TX transmission for NIC would be like: map -> TX -> unmap since the socket buffers come from OS. Simple RX model eg. disk driver, is also map -> RX -> unmap, but real RX model in a NIC could be more complicated considering packets can come spontaneously and many drivers are using pre-mapped buffers pool. This is in the TBD list. Signed-off-by: Barry Song Signed-off-by: Christoph Hellwig --- kernel/dma/map_benchmark.c | 12 ++++++++++- .../testing/selftests/dma/dma_map_benchmark.c | 21 ++++++++++++++++--- 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/kernel/dma/map_benchmark.c b/kernel/dma/map_benchmark.c index da95df381483..e0e64f8b0739 100644 --- a/kernel/dma/map_benchmark.c +++ b/kernel/dma/map_benchmark.c @@ -21,6 +21,7 @@ #define DMA_MAP_BENCHMARK _IOWR('d', 1, struct map_benchmark) #define DMA_MAP_MAX_THREADS 1024 #define DMA_MAP_MAX_SECONDS 300 +#define DMA_MAP_MAX_TRANS_DELAY (10 * NSEC_PER_MSEC) #define DMA_MAP_BIDIRECTIONAL 0 #define DMA_MAP_TO_DEVICE 1 @@ -36,7 +37,8 @@ struct map_benchmark { __s32 node; /* which numa node this benchmark will run on */ __u32 dma_bits; /* DMA addressing capability */ __u32 dma_dir; /* DMA data direction */ - __u8 expansion[84]; /* For future use */ + __u32 dma_trans_ns; /* time for DMA transmission in ns */ + __u8 expansion[80]; /* For future use */ }; struct map_benchmark_data { @@ -87,6 +89,9 @@ static int map_benchmark_thread(void *data) map_etime = ktime_get(); map_delta = ktime_sub(map_etime, map_stime); + /* Pretend DMA is transmitting */ + ndelay(map->bparam.dma_trans_ns); + unmap_stime = ktime_get(); dma_unmap_single(map->dev, dma_addr, PAGE_SIZE, map->dir); unmap_etime = ktime_get(); @@ -218,6 +223,11 @@ static long map_benchmark_ioctl(struct file *file, unsigned int cmd, return -EINVAL; } + if (map->bparam.dma_trans_ns > DMA_MAP_MAX_TRANS_DELAY) { + pr_err("invalid transmission delay\n"); + return -EINVAL; + } + if (map->bparam.node != NUMA_NO_NODE && !node_possible(map->bparam.node)) { pr_err("invalid numa node\n"); diff --git a/tools/testing/selftests/dma/dma_map_benchmark.c b/tools/testing/selftests/dma/dma_map_benchmark.c index 537d65968c48..fb23ce9617ea 100644 --- a/tools/testing/selftests/dma/dma_map_benchmark.c +++ b/tools/testing/selftests/dma/dma_map_benchmark.c @@ -12,9 +12,12 @@ #include #include +#define NSEC_PER_MSEC 1000000L + #define DMA_MAP_BENCHMARK _IOWR('d', 1, struct map_benchmark) #define DMA_MAP_MAX_THREADS 1024 #define DMA_MAP_MAX_SECONDS 300 +#define DMA_MAP_MAX_TRANS_DELAY (10 * NSEC_PER_MSEC) #define DMA_MAP_BIDIRECTIONAL 0 #define DMA_MAP_TO_DEVICE 1 @@ -36,7 +39,8 @@ struct map_benchmark { __s32 node; /* which numa node this benchmark will run on */ __u32 dma_bits; /* DMA addressing capability */ __u32 dma_dir; /* DMA data direction */ - __u8 expansion[84]; /* For future use */ + __u32 dma_trans_ns; /* time for DMA transmission in ns */ + __u8 expansion[80]; /* For future use */ }; int main(int argc, char **argv) @@ -46,12 +50,12 @@ int main(int argc, char **argv) /* default single thread, run 20 seconds on NUMA_NO_NODE */ int threads = 1, seconds = 20, node = -1; /* default dma mask 32bit, bidirectional DMA */ - int bits = 32, dir = DMA_MAP_BIDIRECTIONAL; + int bits = 32, xdelay = 0, dir = DMA_MAP_BIDIRECTIONAL; int cmd = DMA_MAP_BENCHMARK; char *p; - while ((opt = getopt(argc, argv, "t:s:n:b:d:")) != -1) { + while ((opt = getopt(argc, argv, "t:s:n:b:d:x:")) != -1) { switch (opt) { case 't': threads = atoi(optarg); @@ -68,6 +72,9 @@ int main(int argc, char **argv) case 'd': dir = atoi(optarg); break; + case 'x': + xdelay = atoi(optarg); + break; default: return -1; } @@ -85,6 +92,12 @@ int main(int argc, char **argv) exit(1); } + if (xdelay < 0 || xdelay > DMA_MAP_MAX_TRANS_DELAY) { + fprintf(stderr, "invalid transmit delay, must be in 0-%ld\n", + DMA_MAP_MAX_TRANS_DELAY); + exit(1); + } + /* suppose the mininum DMA zone is 1MB in the world */ if (bits < 20 || bits > 64) { fprintf(stderr, "invalid dma mask bit, must be in 20-64\n"); @@ -109,6 +122,8 @@ int main(int argc, char **argv) map.node = node; map.dma_bits = bits; map.dma_dir = dir; + map.dma_trans_ns = xdelay; + if (ioctl(fd, cmd, &map)) { perror("ioctl"); exit(1); From 81d88ce55092edf1a1f928efb373f289c6b90efd Mon Sep 17 00:00:00 2001 From: Christoph Hellwig Date: Tue, 24 Nov 2020 16:38:40 +0100 Subject: [PATCH 33/91] dma-mapping: remove the {alloc,free}_noncoherent methods It turns out allowing non-contigous allocations here was a rather bad idea, as we'll now need to define ways to get the pages for mmaping or dma_buf sharing. Revert this change and stick to the original concept. A different API for the use case of non-contigous allocations will be added back later. Signed-off-by: Christoph Hellwig Reviewed-by: Tomasz Figa Tested-by: Ricardo Ribalda :wq --- Documentation/core-api/dma-api.rst | 66 +++++++++++------------------- drivers/iommu/dma-iommu.c | 30 -------------- include/linux/dma-map-ops.h | 5 --- include/linux/dma-mapping.h | 17 ++++++-- kernel/dma/mapping.c | 40 ------------------ 5 files changed, 36 insertions(+), 122 deletions(-) diff --git a/Documentation/core-api/dma-api.rst b/Documentation/core-api/dma-api.rst index 75cb757bbff0..e6d23f117308 100644 --- a/Documentation/core-api/dma-api.rst +++ b/Documentation/core-api/dma-api.rst @@ -526,46 +526,6 @@ for the kernel vs the device. If you don't understand how cache line coherency works between a processor and an I/O device, you should not be using this part of the API. -:: - - void * - dma_alloc_noncoherent(struct device *dev, size_t size, - dma_addr_t *dma_handle, enum dma_data_direction dir, - gfp_t gfp) - -This routine allocates a region of bytes of consistent memory. It -returns a pointer to the allocated region (in the processor's virtual address -space) or NULL if the allocation failed. The returned memory may or may not -be in the kernel direct mapping. Drivers must not call virt_to_page on -the returned memory region. - -It also returns a which may be cast to an unsigned integer the -same width as the bus and given to the device as the DMA address base of -the region. - -The dir parameter specified if data is read and/or written by the device, -see dma_map_single() for details. - -The gfp parameter allows the caller to specify the ``GFP_`` flags (see -kmalloc()) for the allocation, but rejects flags used to specify a memory -zone such as GFP_DMA or GFP_HIGHMEM. - -Before giving the memory to the device, dma_sync_single_for_device() needs -to be called, and before reading memory written by the device, -dma_sync_single_for_cpu(), just like for streaming DMA mappings that are -reused. - -:: - - void - dma_free_noncoherent(struct device *dev, size_t size, void *cpu_addr, - dma_addr_t dma_handle, enum dma_data_direction dir) - -Free a region of memory previously allocated using dma_alloc_noncoherent(). -dev, size and dma_handle and dir must all be the same as those passed into -dma_alloc_noncoherent(). cpu_addr must be the virtual address returned by -dma_alloc_noncoherent(). - :: struct page * @@ -600,9 +560,29 @@ reused. dma_addr_t dma_handle, enum dma_data_direction dir) Free a region of memory previously allocated using dma_alloc_pages(). -dev, size and dma_handle and dir must all be the same as those passed into -dma_alloc_noncoherent(). page must be the pointer returned by -dma_alloc_pages(). +dev, size, dma_handle and dir must all be the same as those passed into +dma_alloc_pages(). page must be the pointer returned by dma_alloc_pages(). + +:: + + void * + dma_alloc_noncoherent(struct device *dev, size_t size, + dma_addr_t *dma_handle, enum dma_data_direction dir, + gfp_t gfp) + +This routine is a convenient wrapper around dma_alloc_pages that returns the +kernel virtual address for the allocated memory instead of the page structure. + +:: + + void + dma_free_noncoherent(struct device *dev, size_t size, void *cpu_addr, + dma_addr_t dma_handle, enum dma_data_direction dir) + +Free a region of memory previously allocated using dma_alloc_noncoherent(). +dev, size, dma_handle and dir must all be the same as those passed into +dma_alloc_noncoherent(). cpu_addr must be the virtual address returned by +dma_alloc_noncoherent(). :: diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 4078358ed66e..255533faf905 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1197,34 +1197,6 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, return cpu_addr; } -#ifdef CONFIG_DMA_REMAP -static void *iommu_dma_alloc_noncoherent(struct device *dev, size_t size, - dma_addr_t *handle, enum dma_data_direction dir, gfp_t gfp) -{ - if (!gfpflags_allow_blocking(gfp)) { - struct page *page; - - page = dma_common_alloc_pages(dev, size, handle, dir, gfp); - if (!page) - return NULL; - return page_address(page); - } - - return iommu_dma_alloc_remap(dev, size, handle, gfp | __GFP_ZERO, - PAGE_KERNEL, 0); -} - -static void iommu_dma_free_noncoherent(struct device *dev, size_t size, - void *cpu_addr, dma_addr_t handle, enum dma_data_direction dir) -{ - __iommu_dma_unmap(dev, handle, size); - __iommu_dma_free(dev, size, cpu_addr); -} -#else -#define iommu_dma_alloc_noncoherent NULL -#define iommu_dma_free_noncoherent NULL -#endif /* CONFIG_DMA_REMAP */ - static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma, void *cpu_addr, dma_addr_t dma_addr, size_t size, unsigned long attrs) @@ -1295,8 +1267,6 @@ static const struct dma_map_ops iommu_dma_ops = { .free = iommu_dma_free, .alloc_pages = dma_common_alloc_pages, .free_pages = dma_common_free_pages, - .alloc_noncoherent = iommu_dma_alloc_noncoherent, - .free_noncoherent = iommu_dma_free_noncoherent, .mmap = iommu_dma_mmap, .get_sgtable = iommu_dma_get_sgtable, .map_page = iommu_dma_map_page, diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 70fcd0f610ea..11e02537b9e0 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -22,11 +22,6 @@ struct dma_map_ops { gfp_t gfp); void (*free_pages)(struct device *dev, size_t size, struct page *vaddr, dma_addr_t dma_handle, enum dma_data_direction dir); - void *(*alloc_noncoherent)(struct device *dev, size_t size, - dma_addr_t *dma_handle, enum dma_data_direction dir, - gfp_t gfp); - void (*free_noncoherent)(struct device *dev, size_t size, void *vaddr, - dma_addr_t dma_handle, enum dma_data_direction dir); int (*mmap)(struct device *, struct vm_area_struct *, void *, dma_addr_t, size_t, unsigned long attrs); diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 2e49996a8f39..fbfa3f5abd94 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -263,10 +263,19 @@ struct page *dma_alloc_pages(struct device *dev, size_t size, dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp); void dma_free_pages(struct device *dev, size_t size, struct page *page, dma_addr_t dma_handle, enum dma_data_direction dir); -void *dma_alloc_noncoherent(struct device *dev, size_t size, - dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp); -void dma_free_noncoherent(struct device *dev, size_t size, void *vaddr, - dma_addr_t dma_handle, enum dma_data_direction dir); + +static inline void *dma_alloc_noncoherent(struct device *dev, size_t size, + dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp) +{ + struct page *page = dma_alloc_pages(dev, size, dma_handle, dir, gfp); + return page ? page_address(page) : NULL; +} + +static inline void dma_free_noncoherent(struct device *dev, size_t size, + void *vaddr, dma_addr_t dma_handle, enum dma_data_direction dir) +{ + dma_free_pages(dev, size, virt_to_page(vaddr), dma_handle, dir); +} static inline dma_addr_t dma_map_single_attrs(struct device *dev, void *ptr, size_t size, enum dma_data_direction dir, unsigned long attrs) diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index f87a89d08654..68992e35c8c3 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -515,46 +515,6 @@ void dma_free_pages(struct device *dev, size_t size, struct page *page, } EXPORT_SYMBOL_GPL(dma_free_pages); -void *dma_alloc_noncoherent(struct device *dev, size_t size, - dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp) -{ - const struct dma_map_ops *ops = get_dma_ops(dev); - void *vaddr; - - if (!ops || !ops->alloc_noncoherent) { - struct page *page; - - page = dma_alloc_pages(dev, size, dma_handle, dir, gfp); - if (!page) - return NULL; - return page_address(page); - } - - size = PAGE_ALIGN(size); - vaddr = ops->alloc_noncoherent(dev, size, dma_handle, dir, gfp); - if (vaddr) - debug_dma_map_page(dev, virt_to_page(vaddr), 0, size, dir, - *dma_handle); - return vaddr; -} -EXPORT_SYMBOL_GPL(dma_alloc_noncoherent); - -void dma_free_noncoherent(struct device *dev, size_t size, void *vaddr, - dma_addr_t dma_handle, enum dma_data_direction dir) -{ - const struct dma_map_ops *ops = get_dma_ops(dev); - - if (!ops || !ops->free_noncoherent) { - dma_free_pages(dev, size, virt_to_page(vaddr), dma_handle, dir); - return; - } - - size = PAGE_ALIGN(size); - debug_dma_unmap_page(dev, dma_handle, size, dir); - ops->free_noncoherent(dev, size, vaddr, dma_handle, dir); -} -EXPORT_SYMBOL_GPL(dma_free_noncoherent); - int dma_supported(struct device *dev, u64 mask) { const struct dma_map_ops *ops = get_dma_ops(dev); From 114c93bf2859550f1b63ddab14eb0601be57ceea Mon Sep 17 00:00:00 2001 From: Bjorn Andersson Date: Wed, 20 Jan 2021 17:41:29 -0800 Subject: [PATCH 34/91] dt-bindings: mailbox: qcom: Add SC8180X APCS compatible Add compatible for the Qualcomm SC8180x APCS block to the Qualcomm APCS binding. Signed-off-by: Bjorn Andersson Acked-by: Rob Herring Signed-off-by: Jassi Brar --- .../devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml index ffd09b664ff5..ec4054087bc1 100644 --- a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml +++ b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml @@ -24,6 +24,7 @@ properties: - qcom,msm8998-apcs-hmss-global - qcom,qcs404-apcs-apps-global - qcom,sc7180-apss-shared + - qcom,sc8180x-apss-shared - qcom,sdm660-apcs-hmss-global - qcom,sdm845-apss-shared - qcom,sm8150-apss-shared From d10c851f25f328237deb61350052c5d73d4dd19b Mon Sep 17 00:00:00 2001 From: Bjorn Andersson Date: Wed, 20 Jan 2021 17:41:30 -0800 Subject: [PATCH 35/91] mailbox: qcom: Add SC8180X apcs compatible The Qualcomm SC8180X platform has a APSS block exposing the usual IPC bits, add a compatible for this. Signed-off-by: Bjorn Andersson Signed-off-by: Jassi Brar --- drivers/mailbox/qcom-apcs-ipc-mailbox.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/mailbox/qcom-apcs-ipc-mailbox.c b/drivers/mailbox/qcom-apcs-ipc-mailbox.c index 077e5c6a9ef7..0fb0ab028b62 100644 --- a/drivers/mailbox/qcom-apcs-ipc-mailbox.c +++ b/drivers/mailbox/qcom-apcs-ipc-mailbox.c @@ -159,6 +159,7 @@ static const struct of_device_id qcom_apcs_ipc_of_match[] = { { .compatible = "qcom,msm8998-apcs-hmss-global", .data = &msm8998_apcs_data }, { .compatible = "qcom,qcs404-apcs-apps-global", .data = &msm8916_apcs_data }, { .compatible = "qcom,sc7180-apss-shared", .data = &apps_shared_apcs_data }, + { .compatible = "qcom,sc8180x-apss-shared", .data = &apps_shared_apcs_data }, { .compatible = "qcom,sdm660-apcs-hmss-global", .data = &sdm660_apcs_data }, { .compatible = "qcom,sdm845-apss-shared", .data = &apps_shared_apcs_data }, { .compatible = "qcom,sm8150-apss-shared", .data = &apps_shared_apcs_data }, From 7a4647bdc242256d7f70ba562d19055bf11e54a6 Mon Sep 17 00:00:00 2001 From: Suman Anna Date: Tue, 9 Feb 2021 13:36:42 -0600 Subject: [PATCH 36/91] dt-bindings: mailbox: omap: Update binding for AM64x SoCs Update the existing OMAP Mailbox binding to include the info for AM64x SoCs. There are some minor IP integration differences between the AM64x SoCs and the previous AM65x and J721E SoC families. Signed-off-by: Suman Anna Acked-by: Rob Herring Signed-off-by: Jassi Brar --- Documentation/devicetree/bindings/mailbox/omap-mailbox.txt | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt b/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt index 5fe80c1c19fc..12371f5c6cd9 100644 --- a/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt +++ b/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt @@ -28,6 +28,9 @@ SoCs has each of these instances form a cluster and combine multiple clusters into a single IP block present within the Main NavSS. The interrupt lines from all these clusters are multiplexed and routed to different processor subsystems over a limited number of common interrupt output lines of an Interrupt Router. +The AM64x SoCS also uses a single IP block comprising of multiple clusters, +but the number of clusters are smaller, and the interrupt output lines are +connected directly to various processors. Mailbox Device Node: ==================== @@ -42,6 +45,7 @@ Required properties: "ti,omap4-mailbox" for OMAP44xx, OMAP54xx, AM33xx, AM43xx and DRA7xx SoCs "ti,am654-mailbox" for K3 AM65x and J721E SoCs + "ti,am64-mailbox" for K3 AM64x SoCs - reg: Contains the mailbox register address range (base address and length) - interrupts: Contains the interrupt information for the mailbox From 595be654b0c23b64268b3f49934588e7be42f762 Mon Sep 17 00:00:00 2001 From: Suman Anna Date: Tue, 9 Feb 2021 13:36:43 -0600 Subject: [PATCH 37/91] mailbox: omap: Add support for K3 AM64x SoCs The AM64x SoC contains a Mailbox IP instance with multiple clusters in the MAIN domain, and is a variant of the IP on current AM65x and J721E SoCs. The AM64x SoC has only 8 clusters with no interrupts routed to the A53 core on the first 2 clusters. The interrupt outputs from the IP do not go through any Interrupt Routers and are hard-wired to each processor, with only couple of interrupts from each cluster reaching the A53 core. The IP is also not built with the K3 safety feature in hardware. Add the support for this IP through a new compatible. Signed-off-by: Suman Anna Signed-off-by: Jassi Brar --- drivers/mailbox/omap-mailbox.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/mailbox/omap-mailbox.c b/drivers/mailbox/omap-mailbox.c index 93fe08aef3ca..7295e3835e30 100644 --- a/drivers/mailbox/omap-mailbox.c +++ b/drivers/mailbox/omap-mailbox.c @@ -3,7 +3,7 @@ * OMAP mailbox driver * * Copyright (C) 2006-2009 Nokia Corporation. All rights reserved. - * Copyright (C) 2013-2019 Texas Instruments Incorporated - https://www.ti.com + * Copyright (C) 2013-2021 Texas Instruments Incorporated - https://www.ti.com * * Contact: Hiroshi DOYU * Suman Anna @@ -663,6 +663,10 @@ static const struct of_device_id omap_mailbox_of_match[] = { .compatible = "ti,am654-mailbox", .data = &omap4_data, }, + { + .compatible = "ti,am64-mailbox", + .data = &omap4_data, + }, { /* end */ }, From 0d17014e91898b20fb5010a03470f11fee3c6391 Mon Sep 17 00:00:00 2001 From: Manivannan Sadhasivam Date: Mon, 18 Jan 2021 09:41:52 +0530 Subject: [PATCH 38/91] dt-bindings: mailbox: Add binding for SDX55 APCS Add devicetree YAML binding for SDX55 APCS GCC block. The APCS block acts as the mailbox controller and also provides a clock output and takes 3 clock sources (pll, aux, ref) as input. Signed-off-by: Manivannan Sadhasivam Reviewed-by: Rob Herring Signed-off-by: Jassi Brar --- .../mailbox/qcom,apcs-kpss-global.yaml | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml index ec4054087bc1..5dc1173d03fd 100644 --- a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml +++ b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml @@ -34,9 +34,11 @@ properties: clocks: description: phandles to the parent clocks of the clock driver + minItems: 2 items: - description: primary pll parent of the clock driver - description: auxiliary parent + - description: reference clock '#mbox-cells': const: 1 @@ -45,9 +47,11 @@ properties: const: 0 clock-names: + minItems: 2 items: - const: pll - const: aux + - const: ref required: - compatible @@ -56,6 +60,35 @@ required: additionalProperties: false +allOf: + - if: + properties: + compatible: + enum: + - qcom,ipq6018-apcs-apps-global + - qcom,ipq8074-apcs-apps-global + - qcom,msm8916-apcs-kpss-global + - qcom,msm8994-apcs-kpss-global + - qcom,msm8996-apcs-hmss-global + - qcom,msm8998-apcs-hmss-global + - qcom,qcs404-apcs-apps-global + - qcom,sc7180-apss-shared + - qcom,sdm660-apcs-hmss-global + - qcom,sdm845-apss-shared + - qcom,sm8150-apss-shared + then: + properties: + clocks: + maxItems: 2 + - if: + properties: + compatible: + enum: + - qcom,sdx55-apcs-gcc + then: + properties: + clocks: + maxItems: 3 examples: # Example apcs with msm8996 From c319f78cba83bb0475cbcbf221492c651bd35b2b Mon Sep 17 00:00:00 2001 From: Manivannan Sadhasivam Date: Mon, 18 Jan 2021 09:41:53 +0530 Subject: [PATCH 39/91] mailbox: qcom: Add support for SDX55 APCS IPC In SDX55, the IPC bits are located in the APCS GCC block. Also, this block can provide clock functionality. Hence, add support for IPC with correct offset and name of the clock provider. Signed-off-by: Manivannan Sadhasivam Signed-off-by: Jassi Brar --- drivers/mailbox/qcom-apcs-ipc-mailbox.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/mailbox/qcom-apcs-ipc-mailbox.c b/drivers/mailbox/qcom-apcs-ipc-mailbox.c index 0fb0ab028b62..f25324d03842 100644 --- a/drivers/mailbox/qcom-apcs-ipc-mailbox.c +++ b/drivers/mailbox/qcom-apcs-ipc-mailbox.c @@ -61,11 +61,15 @@ static const struct qcom_apcs_ipc_data apps_shared_apcs_data = { .offset = 12, .clk_name = NULL }; +static const struct qcom_apcs_ipc_data sdx55_apcs_data = { + .offset = 0x1008, .clk_name = "qcom-sdx55-acps-clk" +}; + static const struct regmap_config apcs_regmap_config = { .reg_bits = 32, .reg_stride = 4, .val_bits = 32, - .max_register = 0xFFC, + .max_register = 0x1008, .fast_io = true, }; @@ -163,6 +167,7 @@ static const struct of_device_id qcom_apcs_ipc_of_match[] = { { .compatible = "qcom,sdm660-apcs-hmss-global", .data = &sdm660_apcs_data }, { .compatible = "qcom,sdm845-apss-shared", .data = &apps_shared_apcs_data }, { .compatible = "qcom,sm8150-apss-shared", .data = &apps_shared_apcs_data }, + { .compatible = "qcom,sdx55-apcs-gcc", .data = &sdx55_apcs_data }, {} }; MODULE_DEVICE_TABLE(of, qcom_apcs_ipc_of_match); From af59bcff8a031bb495469aaecc386addad4cade9 Mon Sep 17 00:00:00 2001 From: Viresh Kumar Date: Wed, 30 Dec 2020 10:12:04 +0530 Subject: [PATCH 40/91] mailbox: arm_mhuv2: Fix sparse warnings This patch fixes a bunch of sparse warnings in the newly added arm_mhuv2 driver. drivers/mailbox/arm_mhuv2.c:506:24: warning: incorrect type in argument 1 (different address spaces) drivers/mailbox/arm_mhuv2.c:506:24: expected void const volatile [noderef] __iomem *addr drivers/mailbox/arm_mhuv2.c:506:24: got unsigned int [usertype] * drivers/mailbox/arm_mhuv2.c:547:42: warning: incorrect type in argument 2 (different address spaces) drivers/mailbox/arm_mhuv2.c:547:42: expected unsigned int [usertype] *reg drivers/mailbox/arm_mhuv2.c:547:42: got unsigned int [noderef] __iomem * drivers/mailbox/arm_mhuv2.c:625:42: warning: incorrect type in argument 2 (different address spaces) drivers/mailbox/arm_mhuv2.c:625:42: expected unsigned int [usertype] *reg drivers/mailbox/arm_mhuv2.c:625:42: got unsigned int [noderef] __iomem * drivers/mailbox/arm_mhuv2.c:972:24: warning: dereference of noderef expression drivers/mailbox/arm_mhuv2.c:973:22: warning: dereference of noderef expression drivers/mailbox/arm_mhuv2.c:993:25: warning: dereference of noderef expression drivers/mailbox/arm_mhuv2.c:1026:24: warning: dereference of noderef expression drivers/mailbox/arm_mhuv2.c:1027:22: warning: dereference of noderef expression drivers/mailbox/arm_mhuv2.c:1048:17: warning: dereference of noderef expression Reported-by: kernel test robot Signed-off-by: Viresh Kumar Signed-off-by: Jassi Brar --- drivers/mailbox/arm_mhuv2.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/mailbox/arm_mhuv2.c b/drivers/mailbox/arm_mhuv2.c index 67fb10885bb4..8223c1005254 100644 --- a/drivers/mailbox/arm_mhuv2.c +++ b/drivers/mailbox/arm_mhuv2.c @@ -238,19 +238,19 @@ struct mhuv2_mbox_chan_priv { }; /* Macro for reading a bitfield within a physically mapped packed struct */ -#define readl_relaxed_bitfield(_regptr, _field) \ +#define readl_relaxed_bitfield(_regptr, _type, _field) \ ({ \ u32 _regval; \ _regval = readl_relaxed((_regptr)); \ - (*(typeof((_regptr)))(&_regval))._field; \ + (*(_type *)(&_regval))._field; \ }) /* Macro for writing a bitfield within a physically mapped packed struct */ -#define writel_relaxed_bitfield(_value, _regptr, _field) \ +#define writel_relaxed_bitfield(_value, _regptr, _type, _field) \ ({ \ u32 _regval; \ _regval = readl_relaxed(_regptr); \ - (*(typeof(_regptr))(&_regval))._field = _value; \ + (*(_type *)(&_regval))._field = _value; \ writel_relaxed(_regval, _regptr); \ }) @@ -496,7 +496,7 @@ static const struct mhuv2_protocol_ops mhuv2_data_transfer_ops = { /* Interrupt handlers */ -static struct mbox_chan *get_irq_chan_comb(struct mhuv2 *mhu, u32 *reg) +static struct mbox_chan *get_irq_chan_comb(struct mhuv2 *mhu, u32 __iomem *reg) { struct mbox_chan *chans = mhu->mbox.chans; int channel = 0, i, offset = 0, windows, protocol, ch_wn; @@ -969,8 +969,8 @@ static int mhuv2_tx_init(struct amba_device *adev, struct mhuv2 *mhu, mhu->mbox.ops = &mhuv2_sender_ops; mhu->send = reg; - mhu->windows = readl_relaxed_bitfield(&mhu->send->mhu_cfg, num_ch); - mhu->minor = readl_relaxed_bitfield(&mhu->send->aidr, arch_minor_rev); + mhu->windows = readl_relaxed_bitfield(&mhu->send->mhu_cfg, struct mhu_cfg_t, num_ch); + mhu->minor = readl_relaxed_bitfield(&mhu->send->aidr, struct aidr_t, arch_minor_rev); spin_lock_init(&mhu->doorbell_pending_lock); @@ -990,7 +990,7 @@ static int mhuv2_tx_init(struct amba_device *adev, struct mhuv2 *mhu, mhu->mbox.txdone_poll = false; mhu->irq = adev->irq[0]; - writel_relaxed_bitfield(1, &mhu->send->int_en, chcomb); + writel_relaxed_bitfield(1, &mhu->send->int_en, struct int_en_t, chcomb); /* Disable all channel interrupts */ for (i = 0; i < mhu->windows; i++) @@ -1023,8 +1023,8 @@ static int mhuv2_rx_init(struct amba_device *adev, struct mhuv2 *mhu, mhu->mbox.ops = &mhuv2_receiver_ops; mhu->recv = reg; - mhu->windows = readl_relaxed_bitfield(&mhu->recv->mhu_cfg, num_ch); - mhu->minor = readl_relaxed_bitfield(&mhu->recv->aidr, arch_minor_rev); + mhu->windows = readl_relaxed_bitfield(&mhu->recv->mhu_cfg, struct mhu_cfg_t, num_ch); + mhu->minor = readl_relaxed_bitfield(&mhu->recv->aidr, struct aidr_t, arch_minor_rev); mhu->irq = adev->irq[0]; if (!mhu->irq) { @@ -1045,7 +1045,7 @@ static int mhuv2_rx_init(struct amba_device *adev, struct mhuv2 *mhu, writel_relaxed(0xFFFFFFFF, &mhu->recv->ch_wn[i].mask_set); if (mhu->minor) - writel_relaxed_bitfield(1, &mhu->recv->int_en, chcomb); + writel_relaxed_bitfield(1, &mhu->recv->int_en, struct int_en_t, chcomb); return 0; } From 09d12157b3bf3b42f74b7871930064eee3e080a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Tue, 2 Feb 2021 20:43:08 +0100 Subject: [PATCH 41/91] mailbox: arm_mhuv2: make remove callback return void MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit My build tests failed to catch that amba driver that would have needed adaption in commit 3fd269e74f2f ("amba: Make the remove callback return void"). Change the remove function to make the driver build again. Reported-by: kernel test robot Fixes: 3fd269e74f2f ("amba: Make the remove callback return void") Signed-off-by: Uwe Kleine-König Acked-by: Viresh Kumar Signed-off-by: Jassi Brar --- drivers/mailbox/arm_mhuv2.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/mailbox/arm_mhuv2.c b/drivers/mailbox/arm_mhuv2.c index 8223c1005254..cdfb1939fabf 100644 --- a/drivers/mailbox/arm_mhuv2.c +++ b/drivers/mailbox/arm_mhuv2.c @@ -1095,14 +1095,12 @@ static int mhuv2_probe(struct amba_device *adev, const struct amba_id *id) return ret; } -static int mhuv2_remove(struct amba_device *adev) +static void mhuv2_remove(struct amba_device *adev) { struct mhuv2 *mhu = amba_get_drvdata(adev); if (mhu->frame == SENDER_FRAME) writel_relaxed(0x0, &mhu->send->access_request); - - return 0; } static struct amba_id mhuv2_ids[] = { From 4450f128c51160bfded6b483eba37d0628d7adb2 Mon Sep 17 00:00:00 2001 From: Magnum Shan Date: Thu, 4 Feb 2021 16:18:19 +0800 Subject: [PATCH 42/91] mailbox: sprd: correct definition of SPRD_OUTBOX_FIFO_FULL According to the specification, bit[2] represents SPRD_OUTBOX_FIFO_FULL, not bit[0], so correct it. Fixes: ca27fc26cd22 ("mailbox: sprd: Add Spreadtrum mailbox driver") Signed-off-by: Magnum Shan Signed-off-by: Chunyan Zhang Reviewed-by: Baolin Wang Signed-off-by: Jassi Brar --- drivers/mailbox/sprd-mailbox.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/mailbox/sprd-mailbox.c b/drivers/mailbox/sprd-mailbox.c index f6fab24ae8a9..4c325301a2fe 100644 --- a/drivers/mailbox/sprd-mailbox.c +++ b/drivers/mailbox/sprd-mailbox.c @@ -35,7 +35,7 @@ #define SPRD_MBOX_IRQ_CLR BIT(0) /* Bit and mask definiation for outbox's SPRD_MBOX_FIFO_STS register */ -#define SPRD_OUTBOX_FIFO_FULL BIT(0) +#define SPRD_OUTBOX_FIFO_FULL BIT(2) #define SPRD_OUTBOX_FIFO_WR_SHIFT 16 #define SPRD_OUTBOX_FIFO_RD_SHIFT 24 #define SPRD_OUTBOX_FIFO_POS_MASK GENMASK(7, 0) From d1e6bc0ca01c0b0b1a2d543223024cf2c8f52fb7 Mon Sep 17 00:00:00 2001 From: Mikko Perttunen Date: Wed, 10 Feb 2021 15:49:45 +0200 Subject: [PATCH 43/91] mailbox: tegra-hsp: Set lockdep class dynamically On Tegra194, due to both BPMP and TCU using mailboxes, we get a lockdep spew at boot. Both are using different instances of HSP, so this is harmless. As such give each HSP instance a different lockdep class. Signed-off-by: Mikko Perttunen Signed-off-by: Jassi Brar --- drivers/mailbox/tegra-hsp.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/mailbox/tegra-hsp.c b/drivers/mailbox/tegra-hsp.c index e07091d71986..acd0675da681 100644 --- a/drivers/mailbox/tegra-hsp.c +++ b/drivers/mailbox/tegra-hsp.c @@ -98,7 +98,9 @@ struct tegra_hsp { unsigned int num_ss; unsigned int num_db; unsigned int num_si; + spinlock_t lock; + struct lock_class_key lock_key; struct list_head doorbells; struct tegra_hsp_mailbox *mailboxes; @@ -775,6 +777,18 @@ static int tegra_hsp_probe(struct platform_device *pdev) return err; } + lockdep_register_key(&hsp->lock_key); + lockdep_set_class(&hsp->lock, &hsp->lock_key); + + return 0; +} + +static int tegra_hsp_remove(struct platform_device *pdev) +{ + struct tegra_hsp *hsp = platform_get_drvdata(pdev); + + lockdep_unregister_key(&hsp->lock_key); + return 0; } @@ -834,6 +848,7 @@ static struct platform_driver tegra_hsp_driver = { .pm = &tegra_hsp_pm_ops, }, .probe = tegra_hsp_probe, + .remove = tegra_hsp_remove, }; static int __init tegra_hsp_init(void) From 7323fb22f05ff1d20498d267828870a5fbbaebd6 Mon Sep 17 00:00:00 2001 From: Shiyang Ruan Date: Tue, 26 Jan 2021 10:13:31 +0800 Subject: [PATCH 44/91] device-dax: Fix default return code of range_parse() The return value of range_parse() indicates the size when it is positive. The error code should be negative. Signed-off-by: Shiyang Ruan Reviewed-by: Joao Martins Link: https://lore.kernel.org/r/20210126021331.1059933-1-ruansy.fnst@cn.fujitsu.com Reported-by: Zhang Qilong Fixes: 8490e2e25b5a ("device-dax: add a range mapping allocation attribute") Signed-off-by: Dan Williams --- drivers/dax/bus.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 737b207c9e30..3003558c1a8b 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -1038,7 +1038,7 @@ static ssize_t range_parse(const char *opt, size_t len, struct range *range) { unsigned long long addr = 0; char *start, *end, *str; - ssize_t rc = EINVAL; + ssize_t rc = -EINVAL; str = kstrdup(opt, GFP_KERNEL); if (!str) From 8409f942a1f8c28296910b7b1d60f27dd3e03162 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 12 Feb 2021 18:10:42 +0100 Subject: [PATCH 45/91] libnvdimm/dimm: Simplify nvdimm_remove() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit nvdimm_remove is only ever called after nvdimm_probe() returned successfully. In this case driver data is always set to a non-NULL value so the check for driver data being NULL can go away as it's always false. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210212171043.2136580-1-u.kleine-koenig@pengutronix.de Signed-off-by: Dan Williams --- drivers/nvdimm/dimm.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c index 7d4ddc4d9322..94be3ae1d29f 100644 --- a/drivers/nvdimm/dimm.c +++ b/drivers/nvdimm/dimm.c @@ -117,9 +117,6 @@ static int nvdimm_remove(struct device *dev) { struct nvdimm_drvdata *ndd = dev_get_drvdata(dev); - if (!ndd) - return 0; - nvdimm_bus_lock(dev); dev_set_drvdata(dev, NULL); nvdimm_bus_unlock(dev); From 1f975074634a63f014e2b7e76852ee6d6005a91d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 12 Feb 2021 18:10:43 +0100 Subject: [PATCH 46/91] libnvdimm: Make remove callback return void MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All drivers return 0 in their remove callback and the driver core ignores the return value of nvdimm_bus_remove() anyhow. So simplify by changing the driver remove callback to return void and return 0 unconditionally to the upper layer. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210212171043.2136580-2-u.kleine-koenig@pengutronix.de Signed-off-by: Dan Williams --- drivers/dax/pmem/compat.c | 3 +-- drivers/nvdimm/blk.c | 3 +-- drivers/nvdimm/bus.c | 13 +++++-------- drivers/nvdimm/dimm.c | 4 +--- drivers/nvdimm/pmem.c | 4 +--- drivers/nvdimm/region.c | 4 +--- include/linux/nd.h | 2 +- 7 files changed, 11 insertions(+), 22 deletions(-) diff --git a/drivers/dax/pmem/compat.c b/drivers/dax/pmem/compat.c index 863c114fd88c..d81dc35fd65d 100644 --- a/drivers/dax/pmem/compat.c +++ b/drivers/dax/pmem/compat.c @@ -41,10 +41,9 @@ static int dax_pmem_compat_release(struct device *dev, void *data) return 0; } -static int dax_pmem_compat_remove(struct device *dev) +static void dax_pmem_compat_remove(struct device *dev) { device_for_each_child(dev, NULL, dax_pmem_compat_release); - return 0; } static struct nd_device_driver dax_pmem_compat_driver = { diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c index 22e5617b2cea..8a53728e13e6 100644 --- a/drivers/nvdimm/blk.c +++ b/drivers/nvdimm/blk.c @@ -310,11 +310,10 @@ static int nd_blk_probe(struct device *dev) return nsblk_attach_disk(nsblk); } -static int nd_blk_remove(struct device *dev) +static void nd_blk_remove(struct device *dev) { if (is_nd_btt(dev)) nvdimm_namespace_detach_btt(to_nd_btt(dev)); - return 0; } static struct nd_device_driver nd_blk_driver = { diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c index 2304c6183822..48f0985ca8a0 100644 --- a/drivers/nvdimm/bus.c +++ b/drivers/nvdimm/bus.c @@ -113,18 +113,17 @@ static int nvdimm_bus_remove(struct device *dev) struct nd_device_driver *nd_drv = to_nd_device_driver(dev->driver); struct module *provider = to_bus_provider(dev); struct nvdimm_bus *nvdimm_bus = walk_to_nvdimm_bus(dev); - int rc = 0; if (nd_drv->remove) { debug_nvdimm_lock(dev); - rc = nd_drv->remove(dev); + nd_drv->remove(dev); debug_nvdimm_unlock(dev); } - dev_dbg(&nvdimm_bus->dev, "%s.remove(%s) = %d\n", dev->driver->name, - dev_name(dev), rc); + dev_dbg(&nvdimm_bus->dev, "%s.remove(%s)\n", dev->driver->name, + dev_name(dev)); module_put(provider); - return rc; + return 0; } static void nvdimm_bus_shutdown(struct device *dev) @@ -427,7 +426,7 @@ static void free_badrange_list(struct list_head *badrange_list) list_del_init(badrange_list); } -static int nd_bus_remove(struct device *dev) +static void nd_bus_remove(struct device *dev) { struct nvdimm_bus *nvdimm_bus = to_nvdimm_bus(dev); @@ -446,8 +445,6 @@ static int nd_bus_remove(struct device *dev) spin_unlock(&nvdimm_bus->badrange.lock); nvdimm_bus_destroy_ndctl(nvdimm_bus); - - return 0; } static int nd_bus_probe(struct device *dev) diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c index 94be3ae1d29f..91d9163ee303 100644 --- a/drivers/nvdimm/dimm.c +++ b/drivers/nvdimm/dimm.c @@ -113,7 +113,7 @@ static int nvdimm_probe(struct device *dev) return rc; } -static int nvdimm_remove(struct device *dev) +static void nvdimm_remove(struct device *dev) { struct nvdimm_drvdata *ndd = dev_get_drvdata(dev); @@ -121,8 +121,6 @@ static int nvdimm_remove(struct device *dev) dev_set_drvdata(dev, NULL); nvdimm_bus_unlock(dev); put_ndd(ndd); - - return 0; } static struct nd_device_driver nvdimm_driver = { diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 875076b0ea6c..062f0f22bac9 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -564,7 +564,7 @@ static int nd_pmem_probe(struct device *dev) return pmem_attach_disk(dev, ndns); } -static int nd_pmem_remove(struct device *dev) +static void nd_pmem_remove(struct device *dev) { struct pmem_device *pmem = dev_get_drvdata(dev); @@ -579,8 +579,6 @@ static int nd_pmem_remove(struct device *dev) pmem->bb_state = NULL; } nvdimm_flush(to_nd_region(dev->parent), NULL); - - return 0; } static void nd_pmem_shutdown(struct device *dev) diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c index bfce87ed72ab..e0c34120df37 100644 --- a/drivers/nvdimm/region.c +++ b/drivers/nvdimm/region.c @@ -87,7 +87,7 @@ static int child_unregister(struct device *dev, void *data) return 0; } -static int nd_region_remove(struct device *dev) +static void nd_region_remove(struct device *dev) { struct nd_region *nd_region = to_nd_region(dev); @@ -108,8 +108,6 @@ static int nd_region_remove(struct device *dev) */ sysfs_put(nd_region->bb_state); nd_region->bb_state = NULL; - - return 0; } static int child_notify(struct device *dev, void *data) diff --git a/include/linux/nd.h b/include/linux/nd.h index 55c735997805..cec526c8043d 100644 --- a/include/linux/nd.h +++ b/include/linux/nd.h @@ -26,7 +26,7 @@ struct nd_device_driver { struct device_driver drv; unsigned long type; int (*probe)(struct device *dev); - int (*remove)(struct device *dev); + void (*remove)(struct device *dev); void (*shutdown)(struct device *dev); void (*notify)(struct device *dev, enum nvdimm_event event); }; From 5b8e64f1ada37574b9ab124e1414af2adf688a19 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 5 Feb 2021 23:28:38 +0100 Subject: [PATCH 47/91] device-dax: Prevent registering drivers without probe callback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The bus probe function dax_bus_probe() calls the probe callback without checking it to be non-NULL. Prevent a NULL pointer exception if a driver without a probe function is registered by refusing to register this driver. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210205222842.34896-2-uwe@kleine-koenig.org Signed-off-by: Dan Williams --- drivers/dax/bus.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 3003558c1a8b..2b29728ec2fd 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -1392,6 +1392,13 @@ int __dax_driver_register(struct dax_device_driver *dax_drv, struct device_driver *drv = &dax_drv->drv; int rc = 0; + /* + * dax_bus_probe() calls dax_drv->probe() unconditionally. + * So better be safe than sorry and ensure it is provided. + */ + if (!dax_drv->probe) + return -EINVAL; + INIT_LIST_HEAD(&dax_drv->ids); drv->owner = module; drv->name = mod_name; From 8029968e2ae02361f376751459dc644b45970b40 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 5 Feb 2021 23:28:39 +0100 Subject: [PATCH 48/91] device-dax: Properly handle drivers without remove callback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit If all resources are allocated in .probe() using devm_ functions it might make sense to not provide a .remove() callback. Then the right thing is to just return success. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210205222842.34896-3-uwe@kleine-koenig.org Signed-off-by: Dan Williams --- drivers/dax/bus.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 2b29728ec2fd..ef53da441c5d 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -178,8 +178,12 @@ static int dax_bus_remove(struct device *dev) { struct dax_device_driver *dax_drv = to_dax_drv(dev->driver); struct dev_dax *dev_dax = to_dev_dax(dev); + int ret = 0; - return dax_drv->remove(dev_dax); + if (dax_drv->remove) + ret = dax_drv->remove(dev_dax); + + return ret; } static struct bus_type dax_bus_type = { From e307bf11c5198dbc0c9de0694c3e85c681648df6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 5 Feb 2021 23:28:40 +0100 Subject: [PATCH 49/91] device-dax: Fix error path in dax_driver_register MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The static variable match_always_count is supposed to track if there is a driver registered that has .match_always set. If driver_register() fails, the previous increment must be undone. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210205222842.34896-4-uwe@kleine-koenig.org Signed-off-by: Dan Williams --- drivers/dax/bus.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index ef53da441c5d..d2419e88073a 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -1420,7 +1420,15 @@ int __dax_driver_register(struct dax_device_driver *dax_drv, mutex_unlock(&dax_bus_lock); if (rc) return rc; - return driver_register(drv); + + rc = driver_register(drv); + if (rc && dax_drv->match_always) { + mutex_lock(&dax_bus_lock); + match_always_count -= dax_drv->match_always; + mutex_unlock(&dax_bus_lock); + } + + return rc; } EXPORT_SYMBOL_GPL(__dax_driver_register); From c80b53204d6ee8f70e5f4e846bc0e62dda64aee0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 5 Feb 2021 23:28:41 +0100 Subject: [PATCH 50/91] device-dax: Drop an empty .remove callback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The dax core properly handles a dax driver not having a remove callback. So drop it without changing the effective behaviour. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210205222842.34896-5-uwe@kleine-koenig.org Signed-off-by: Dan Williams --- drivers/dax/device.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 5da2980bb16b..db92573c94e8 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -452,15 +452,9 @@ int dev_dax_probe(struct dev_dax *dev_dax) } EXPORT_SYMBOL_GPL(dev_dax_probe); -static int dev_dax_remove(struct dev_dax *dev_dax) -{ - /* all probe actions are unwound by devm */ - return 0; -} - static struct dax_device_driver device_dax_driver = { .probe = dev_dax_probe, - .remove = dev_dax_remove, + /* all probe actions are unwound by devm, so .remove isn't necessary */ .match_always = 1, }; From 0d519e0d52ee7c532d4018b90cd0b042d374c06d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Fri, 5 Feb 2021 23:28:42 +0100 Subject: [PATCH 51/91] dax-device: Make remove callback return void MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The driver core ignores the return value of struct bus_type::remove() because there is only little that can be done. To simplify the quest to make this function return void, let struct dax_device_driver::remove() return void, too. All users already unconditionally return 0, this commit makes it obvious that returning an error code isn't intended. Signed-off-by: Uwe Kleine-König Link: https://lore.kernel.org/r/20210205222842.34896-6-uwe@kleine-koenig.org Signed-off-by: Dan Williams --- drivers/dax/bus.c | 5 ++--- drivers/dax/bus.h | 2 +- drivers/dax/kmem.c | 7 ++----- 3 files changed, 5 insertions(+), 9 deletions(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index d2419e88073a..452e85ae87a8 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -178,12 +178,11 @@ static int dax_bus_remove(struct device *dev) { struct dax_device_driver *dax_drv = to_dax_drv(dev->driver); struct dev_dax *dev_dax = to_dev_dax(dev); - int ret = 0; if (dax_drv->remove) - ret = dax_drv->remove(dev_dax); + dax_drv->remove(dev_dax); - return ret; + return 0; } static struct bus_type dax_bus_type = { diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index 72b92f95509f..1e946ad7780a 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -39,7 +39,7 @@ struct dax_device_driver { struct list_head ids; int match_always; int (*probe)(struct dev_dax *dev); - int (*remove)(struct dev_dax *dev); + void (*remove)(struct dev_dax *dev); }; int __dax_driver_register(struct dax_device_driver *dax_drv, diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index 403ec42472d1..ac231cc36359 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -136,7 +136,7 @@ err_res_name: } #ifdef CONFIG_MEMORY_HOTREMOVE -static int dev_dax_kmem_remove(struct dev_dax *dev_dax) +static void dev_dax_kmem_remove(struct dev_dax *dev_dax) { int i, success = 0; struct device *dev = &dev_dax->dev; @@ -176,11 +176,9 @@ static int dev_dax_kmem_remove(struct dev_dax *dev_dax) kfree(data); dev_set_drvdata(dev, NULL); } - - return 0; } #else -static int dev_dax_kmem_remove(struct dev_dax *dev_dax) +static void dev_dax_kmem_remove(struct dev_dax *dev_dax) { /* * Without hotremove purposely leak the request_mem_region() for the @@ -190,7 +188,6 @@ static int dev_dax_kmem_remove(struct dev_dax *dev_dax) * request_mem_region(). */ any_hotremove_failed = true; - return 0; } #endif /* CONFIG_MEMORY_HOTREMOVE */ From 4cdadfd5e0a70017fec735b7b6d7f2f731842dc6 Mon Sep 17 00:00:00 2001 From: Dan Williams Date: Tue, 16 Feb 2021 20:09:50 -0800 Subject: [PATCH 52/91] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints The CXL.mem protocol allows a device to act as a provider of "System RAM" and/or "Persistent Memory" that is fully coherent as if the memory was attached to the typical CPU memory controller. With the CXL-2.0 specification a PCI endpoint can implement a "Type-3" device interface and give the operating system control over "Host Managed Device Memory". See section 2.3 Type 3 CXL Device. The memory range exported by the device may optionally be described by the platform firmware memory map, or by infrastructure like LIBNVDIMM to provision persistent memory capacity from one, or more, CXL.mem devices. A pre-requisite for Linux-managed memory-capacity provisioning is this cxl_mem driver that can speak the mailbox protocol defined in section 8.2.8.4 Mailbox Registers. For now just land the initial driver boiler-plate and Documentation/ infrastructure. Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron Acked-by: David Rientjes (v1) Cc: Jonathan Corbet Link: https://www.computeexpresslink.org/download-the-specification Link: https://lore.kernel.org/r/20210217040958.1354670-2-ben.widawsky@intel.com Signed-off-by: Dan Williams --- Documentation/driver-api/cxl/index.rst | 12 ++++ .../driver-api/cxl/memory-devices.rst | 15 +++++ Documentation/driver-api/index.rst | 1 + drivers/Kconfig | 1 + drivers/Makefile | 1 + drivers/cxl/Kconfig | 35 +++++++++++ drivers/cxl/Makefile | 4 ++ drivers/cxl/mem.c | 62 +++++++++++++++++++ drivers/cxl/pci.h | 17 +++++ include/linux/pci_ids.h | 1 + 10 files changed, 149 insertions(+) create mode 100644 Documentation/driver-api/cxl/index.rst create mode 100644 Documentation/driver-api/cxl/memory-devices.rst create mode 100644 drivers/cxl/Kconfig create mode 100644 drivers/cxl/Makefile create mode 100644 drivers/cxl/mem.c create mode 100644 drivers/cxl/pci.h diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst new file mode 100644 index 000000000000..036e49553542 --- /dev/null +++ b/Documentation/driver-api/cxl/index.rst @@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0 + +==================== +Compute Express Link +==================== + +.. toctree:: + :maxdepth: 1 + + memory-devices + +.. only:: subproject and html diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst new file mode 100644 index 000000000000..71617e9002f1 --- /dev/null +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -0,0 +1,15 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. include:: + +=================================== +Compute Express Link Memory Devices +=================================== + +A Compute Express Link Memory Device is a CXL component that implements the +CXL.mem protocol. It contains some amount of volatile memory, persistent memory, +or both. It is enumerated as a PCI device for configuration and passing +messages over an MMIO mailbox. Its contribution to the System Physical +Address space is handled via HDM (Host Managed Device Memory) decoders +that optionally define a device's contribution to an interleaved address +range across multiple devices underneath a host-bridge or interleaved +across host-bridges. diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index 2456d0a97ed8..d246a18fd78f 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -35,6 +35,7 @@ available subsections can be seen below. usb/index firewire pci/index + cxl/index spi i2c ipmb diff --git a/drivers/Kconfig b/drivers/Kconfig index dcecc9f6e33f..62c753a73651 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -6,6 +6,7 @@ menu "Device Drivers" source "drivers/amba/Kconfig" source "drivers/eisa/Kconfig" source "drivers/pci/Kconfig" +source "drivers/cxl/Kconfig" source "drivers/pcmcia/Kconfig" source "drivers/rapidio/Kconfig" diff --git a/drivers/Makefile b/drivers/Makefile index fd11b9ac4cc3..678ea810410f 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/ obj-y += base/ block/ misc/ mfd/ nfc/ obj-$(CONFIG_LIBNVDIMM) += nvdimm/ obj-$(CONFIG_DAX) += dax/ +obj-$(CONFIG_CXL_BUS) += cxl/ obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/ obj-$(CONFIG_NUBUS) += nubus/ obj-y += macintosh/ diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig new file mode 100644 index 000000000000..9e80b311e928 --- /dev/null +++ b/drivers/cxl/Kconfig @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0-only +menuconfig CXL_BUS + tristate "CXL (Compute Express Link) Devices Support" + depends on PCI + help + CXL is a bus that is electrically compatible with PCI Express, but + layers three protocols on that signalling (CXL.io, CXL.cache, and + CXL.mem). The CXL.cache protocol allows devices to hold cachelines + locally, the CXL.mem protocol allows devices to be fully coherent + memory targets, the CXL.io protocol is equivalent to PCI Express. + Say 'y' to enable support for the configuration and management of + devices supporting these protocols. + +if CXL_BUS + +config CXL_MEM + tristate "CXL.mem: Memory Devices" + help + The CXL.mem protocol allows a device to act as a provider of + "System RAM" and/or "Persistent Memory" that is fully coherent + as if the memory was attached to the typical CPU memory + controller. + + Say 'y/m' to enable a driver (named "cxl_mem.ko" when built as + a module) that will attach to CXL.mem devices for + configuration, provisioning, and health monitoring. This + driver is required for dynamic provisioning of CXL.mem + attached memory which is a prerequisite for persistent memory + support. Typically volatile memory is mapped by platform + firmware and included in the platform memory map, but in some + cases the OS is responsible for mapping that memory. See + Chapter 2.3 Type 3 CXL Device in the CXL 2.0 specification. + + If unsure say 'm'. +endif diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile new file mode 100644 index 000000000000..4a30f7c3fc4a --- /dev/null +++ b/drivers/cxl/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_CXL_MEM) += cxl_mem.o + +cxl_mem-y := mem.o diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c new file mode 100644 index 000000000000..ce33c5ee77c9 --- /dev/null +++ b/drivers/cxl/mem.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include "pci.h" + +static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) +{ + int pos; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC); + if (!pos) + return 0; + + while (pos) { + u16 vendor, id; + + pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER1, &vendor); + pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER2, &id); + if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id) + return pos; + + pos = pci_find_next_ext_capability(pdev, pos, + PCI_EXT_CAP_ID_DVSEC); + } + + return 0; +} + +static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) +{ + struct device *dev = &pdev->dev; + int regloc; + + regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); + if (!regloc) { + dev_err(dev, "register location dvsec not found\n"); + return -ENXIO; + } + + return 0; +} + +static const struct pci_device_id cxl_mem_pci_tbl[] = { + /* PCI class code for CXL.mem Type-3 Devices */ + { PCI_DEVICE_CLASS((PCI_CLASS_MEMORY_CXL << 8 | CXL_MEMORY_PROGIF), ~0)}, + { /* terminate list */ }, +}; +MODULE_DEVICE_TABLE(pci, cxl_mem_pci_tbl); + +static struct pci_driver cxl_mem_driver = { + .name = KBUILD_MODNAME, + .id_table = cxl_mem_pci_tbl, + .probe = cxl_mem_probe, + .driver = { + .probe_type = PROBE_PREFER_ASYNCHRONOUS, + }, +}; + +MODULE_LICENSE("GPL v2"); +module_pci_driver(cxl_mem_driver); diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h new file mode 100644 index 000000000000..e464bea3f4d3 --- /dev/null +++ b/drivers/cxl/pci.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ +#ifndef __CXL_PCI_H__ +#define __CXL_PCI_H__ + +#define CXL_MEMORY_PROGIF 0x10 + +/* + * See section 8.1 Configuration Space Registers in the CXL 2.0 + * Specification + */ +#define PCI_DVSEC_VENDOR_ID_CXL 0x1E98 +#define PCI_DVSEC_ID_CXL 0x0 + +#define PCI_DVSEC_ID_CXL_REGLOC_OFFSET 0x8 + +#endif /* __CXL_PCI_H__ */ diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index d8156a5dbee8..766260a9b247 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -51,6 +51,7 @@ #define PCI_BASE_CLASS_MEMORY 0x05 #define PCI_CLASS_MEMORY_RAM 0x0500 #define PCI_CLASS_MEMORY_FLASH 0x0501 +#define PCI_CLASS_MEMORY_CXL 0x0502 #define PCI_CLASS_MEMORY_OTHER 0x0580 #define PCI_BASE_CLASS_BRIDGE 0x06 From 8adaf747c9f0b470aea1b0c88583aa0a344e1540 Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Tue, 16 Feb 2021 20:09:51 -0800 Subject: [PATCH 53/91] cxl/mem: Find device capabilities Provide enough functionality to utilize the mailbox of a memory device. The mailbox is used to interact with the firmware running on the memory device. The flow is proven with one implemented command, "identify". Because the class code has already told the driver this is a memory device and the identify command is mandatory. CXL devices contain an array of capabilities that describe the interactions software can have with the device or firmware running on the device. A CXL compliant device must implement the device status and the mailbox capability. Additionally, a CXL compliant memory device must implement the memory device capability. Each of the capabilities can [will] provide an offset within the MMIO region for interacting with the CXL device. The capabilities tell the driver how to find and map the register space for CXL Memory Devices. The registers are required to utilize the CXL spec defined mailbox interface. The spec outlines two mailboxes, primary and secondary. The secondary mailbox is earmarked for system firmware, and not handled in this driver. Primary mailboxes are capable of generating an interrupt when submitting a background command. That implementation is saved for a later time. Reported-by: Colin Ian King (coverity) Reported-by: Dan Carpenter (smatch) Signed-off-by: Ben Widawsky Reviewed-by: Dan Williams (v2) Link: https://www.computeexpresslink.org/download-the-specification Link: https://lore.kernel.org/r/20210217040958.1354670-3-ben.widawsky@intel.com Signed-off-by: Dan Williams --- .../driver-api/cxl/memory-devices.rst | 14 + drivers/cxl/cxl.h | 90 +++ drivers/cxl/mem.c | 577 +++++++++++++++++- drivers/cxl/pci.h | 14 + 4 files changed, 693 insertions(+), 2 deletions(-) create mode 100644 drivers/cxl/cxl.h diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst index 71617e9002f1..43177e700d62 100644 --- a/Documentation/driver-api/cxl/memory-devices.rst +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -13,3 +13,17 @@ Address space is handled via HDM (Host Managed Device Memory) decoders that optionally define a device's contribution to an interleaved address range across multiple devices underneath a host-bridge or interleaved across host-bridges. + +Driver Infrastructure +===================== + +This section covers the driver infrastructure for a CXL memory device. + +CXL Memory Device +----------------- + +.. kernel-doc:: drivers/cxl/mem.c + :doc: cxl mem + +.. kernel-doc:: drivers/cxl/mem.c + :internal: diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h new file mode 100644 index 000000000000..baac26d9e63b --- /dev/null +++ b/drivers/cxl/cxl.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2020 Intel Corporation. */ + +#ifndef __CXL_H__ +#define __CXL_H__ + +#include +#include +#include + +/* CXL 2.0 8.2.8.1 Device Capabilities Array Register */ +#define CXLDEV_CAP_ARRAY_OFFSET 0x0 +#define CXLDEV_CAP_ARRAY_CAP_ID 0 +#define CXLDEV_CAP_ARRAY_ID_MASK GENMASK_ULL(15, 0) +#define CXLDEV_CAP_ARRAY_COUNT_MASK GENMASK_ULL(47, 32) +/* CXL 2.0 8.2.8.2 CXL Device Capability Header Register */ +#define CXLDEV_CAP_HDR_CAP_ID_MASK GENMASK(15, 0) +/* CXL 2.0 8.2.8.2.1 CXL Device Capabilities */ +#define CXLDEV_CAP_CAP_ID_DEVICE_STATUS 0x1 +#define CXLDEV_CAP_CAP_ID_PRIMARY_MAILBOX 0x2 +#define CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX 0x3 +#define CXLDEV_CAP_CAP_ID_MEMDEV 0x4000 + +/* CXL 2.0 8.2.8.4 Mailbox Registers */ +#define CXLDEV_MBOX_CAPS_OFFSET 0x00 +#define CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0) +#define CXLDEV_MBOX_CTRL_OFFSET 0x04 +#define CXLDEV_MBOX_CTRL_DOORBELL BIT(0) +#define CXLDEV_MBOX_CMD_OFFSET 0x08 +#define CXLDEV_MBOX_CMD_COMMAND_OPCODE_MASK GENMASK_ULL(15, 0) +#define CXLDEV_MBOX_CMD_PAYLOAD_LENGTH_MASK GENMASK_ULL(36, 16) +#define CXLDEV_MBOX_STATUS_OFFSET 0x10 +#define CXLDEV_MBOX_STATUS_RET_CODE_MASK GENMASK_ULL(47, 32) +#define CXLDEV_MBOX_BG_CMD_STATUS_OFFSET 0x18 +#define CXLDEV_MBOX_PAYLOAD_OFFSET 0x20 + +/* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */ +#define CXLMDEV_STATUS_OFFSET 0x0 +#define CXLMDEV_DEV_FATAL BIT(0) +#define CXLMDEV_FW_HALT BIT(1) +#define CXLMDEV_STATUS_MEDIA_STATUS_MASK GENMASK(3, 2) +#define CXLMDEV_MS_NOT_READY 0 +#define CXLMDEV_MS_READY 1 +#define CXLMDEV_MS_ERROR 2 +#define CXLMDEV_MS_DISABLED 3 +#define CXLMDEV_READY(status) \ + (FIELD_GET(CXLMDEV_STATUS_MEDIA_STATUS_MASK, status) == \ + CXLMDEV_MS_READY) +#define CXLMDEV_MBOX_IF_READY BIT(4) +#define CXLMDEV_RESET_NEEDED_MASK GENMASK(7, 5) +#define CXLMDEV_RESET_NEEDED_NOT 0 +#define CXLMDEV_RESET_NEEDED_COLD 1 +#define CXLMDEV_RESET_NEEDED_WARM 2 +#define CXLMDEV_RESET_NEEDED_HOT 3 +#define CXLMDEV_RESET_NEEDED_CXL 4 +#define CXLMDEV_RESET_NEEDED(status) \ + (FIELD_GET(CXLMDEV_RESET_NEEDED_MASK, status) != \ + CXLMDEV_RESET_NEEDED_NOT) + +/** + * struct cxl_mem - A CXL memory device + * @pdev: The PCI device associated with this CXL device. + * @regs: IO mappings to the device's MMIO + * @status_regs: CXL 2.0 8.2.8.3 Device Status Registers + * @mbox_regs: CXL 2.0 8.2.8.4 Mailbox Registers + * @memdev_regs: CXL 2.0 8.2.8.5 Memory Device Registers + * @payload_size: Size of space for payload + * (CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register) + * @mbox_mutex: Mutex to synchronize mailbox access. + * @firmware_version: Firmware version for the memory device. + * @pmem_range: Persistent memory capacity information. + * @ram_range: Volatile memory capacity information. + */ +struct cxl_mem { + struct pci_dev *pdev; + void __iomem *regs; + + void __iomem *status_regs; + void __iomem *mbox_regs; + void __iomem *memdev_regs; + + size_t payload_size; + struct mutex mbox_mutex; /* Protects device mailbox and firmware */ + char firmware_version[0x10]; + + struct range pmem_range; + struct range ram_range; +}; + +#endif /* __CXL_H__ */ diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index ce33c5ee77c9..04e15759bc9b 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -3,7 +3,491 @@ #include #include #include +#include #include "pci.h" +#include "cxl.h" + +/** + * DOC: cxl mem + * + * This implements a CXL memory device ("type-3") as it is defined by the + * Compute Express Link specification. + * + * The driver has several responsibilities, mainly: + * - Create the memX device and register on the CXL bus. + * - Enumerate device's register interface and map them. + * - Probe the device attributes to establish sysfs interface. + * - Provide an IOCTL interface to userspace to communicate with the device for + * things like firmware update. + * - Support management of interleave sets. + * - Handle and manage error conditions. + */ + +#define cxl_doorbell_busy(cxlm) \ + (readl((cxlm)->mbox_regs + CXLDEV_MBOX_CTRL_OFFSET) & \ + CXLDEV_MBOX_CTRL_DOORBELL) + +/* CXL 2.0 - 8.2.8.4 */ +#define CXL_MAILBOX_TIMEOUT_MS (2 * HZ) + +enum opcode { + CXL_MBOX_OP_IDENTIFY = 0x4000, + CXL_MBOX_OP_MAX = 0x10000 +}; + +/** + * struct mbox_cmd - A command to be submitted to hardware. + * @opcode: (input) The command set and command submitted to hardware. + * @payload_in: (input) Pointer to the input payload. + * @payload_out: (output) Pointer to the output payload. Must be allocated by + * the caller. + * @size_in: (input) Number of bytes to load from @payload_in. + * @size_out: (input) Max number of bytes loaded into @payload_out. + * (output) Number of bytes generated by the device. For fixed size + * outputs commands this is always expected to be deterministic. For + * variable sized output commands, it tells the exact number of bytes + * written. + * @return_code: (output) Error code returned from hardware. + * + * This is the primary mechanism used to send commands to the hardware. + * All the fields except @payload_* correspond exactly to the fields described in + * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and + * @payload_out are written to, and read from the Command Payload Registers + * defined in CXL 2.0 8.2.8.4.8. + */ +struct mbox_cmd { + u16 opcode; + void *payload_in; + void *payload_out; + size_t size_in; + size_t size_out; + u16 return_code; +#define CXL_MBOX_SUCCESS 0 +}; + +static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm) +{ + const unsigned long start = jiffies; + unsigned long end = start; + + while (cxl_doorbell_busy(cxlm)) { + end = jiffies; + + if (time_after(end, start + CXL_MAILBOX_TIMEOUT_MS)) { + /* Check again in case preempted before timeout test */ + if (!cxl_doorbell_busy(cxlm)) + break; + return -ETIMEDOUT; + } + cpu_relax(); + } + + dev_dbg(&cxlm->pdev->dev, "Doorbell wait took %dms", + jiffies_to_msecs(end) - jiffies_to_msecs(start)); + return 0; +} + +static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm, + struct mbox_cmd *mbox_cmd) +{ + struct device *dev = &cxlm->pdev->dev; + + dev_dbg(dev, "Mailbox command (opcode: %#x size: %zub) timed out\n", + mbox_cmd->opcode, mbox_cmd->size_in); +} + +/** + * __cxl_mem_mbox_send_cmd() - Execute a mailbox command + * @cxlm: The CXL memory device to communicate with. + * @mbox_cmd: Command to send to the memory device. + * + * Context: Any context. Expects mbox_mutex to be held. + * Return: -ETIMEDOUT if timeout occurred waiting for completion. 0 on success. + * Caller should check the return code in @mbox_cmd to make sure it + * succeeded. + * + * This is a generic form of the CXL mailbox send command thus only using the + * registers defined by the mailbox capability ID - CXL 2.0 8.2.8.4. Memory + * devices, and perhaps other types of CXL devices may have further information + * available upon error conditions. Driver facilities wishing to send mailbox + * commands should use the wrapper command. + * + * The CXL spec allows for up to two mailboxes. The intention is for the primary + * mailbox to be OS controlled and the secondary mailbox to be used by system + * firmware. This allows the OS and firmware to communicate with the device and + * not need to coordinate with each other. The driver only uses the primary + * mailbox. + */ +static int __cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, + struct mbox_cmd *mbox_cmd) +{ + void __iomem *payload = cxlm->mbox_regs + CXLDEV_MBOX_PAYLOAD_OFFSET; + u64 cmd_reg, status_reg; + size_t out_len; + int rc; + + lockdep_assert_held(&cxlm->mbox_mutex); + + /* + * Here are the steps from 8.2.8.4 of the CXL 2.0 spec. + * 1. Caller reads MB Control Register to verify doorbell is clear + * 2. Caller writes Command Register + * 3. Caller writes Command Payload Registers if input payload is non-empty + * 4. Caller writes MB Control Register to set doorbell + * 5. Caller either polls for doorbell to be clear or waits for interrupt if configured + * 6. Caller reads MB Status Register to fetch Return code + * 7. If command successful, Caller reads Command Register to get Payload Length + * 8. If output payload is non-empty, host reads Command Payload Registers + * + * Hardware is free to do whatever it wants before the doorbell is rung, + * and isn't allowed to change anything after it clears the doorbell. As + * such, steps 2 and 3 can happen in any order, and steps 6, 7, 8 can + * also happen in any order (though some orders might not make sense). + */ + + /* #1 */ + if (cxl_doorbell_busy(cxlm)) { + dev_err_ratelimited(&cxlm->pdev->dev, + "Mailbox re-busy after acquiring\n"); + return -EBUSY; + } + + cmd_reg = FIELD_PREP(CXLDEV_MBOX_CMD_COMMAND_OPCODE_MASK, + mbox_cmd->opcode); + if (mbox_cmd->size_in) { + if (WARN_ON(!mbox_cmd->payload_in)) + return -EINVAL; + + cmd_reg |= FIELD_PREP(CXLDEV_MBOX_CMD_PAYLOAD_LENGTH_MASK, + mbox_cmd->size_in); + memcpy_toio(payload, mbox_cmd->payload_in, mbox_cmd->size_in); + } + + /* #2, #3 */ + writeq(cmd_reg, cxlm->mbox_regs + CXLDEV_MBOX_CMD_OFFSET); + + /* #4 */ + dev_dbg(&cxlm->pdev->dev, "Sending command\n"); + writel(CXLDEV_MBOX_CTRL_DOORBELL, + cxlm->mbox_regs + CXLDEV_MBOX_CTRL_OFFSET); + + /* #5 */ + rc = cxl_mem_wait_for_doorbell(cxlm); + if (rc == -ETIMEDOUT) { + cxl_mem_mbox_timeout(cxlm, mbox_cmd); + return rc; + } + + /* #6 */ + status_reg = readq(cxlm->mbox_regs + CXLDEV_MBOX_STATUS_OFFSET); + mbox_cmd->return_code = + FIELD_GET(CXLDEV_MBOX_STATUS_RET_CODE_MASK, status_reg); + + if (mbox_cmd->return_code != 0) { + dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n"); + return 0; + } + + /* #7 */ + cmd_reg = readq(cxlm->mbox_regs + CXLDEV_MBOX_CMD_OFFSET); + out_len = FIELD_GET(CXLDEV_MBOX_CMD_PAYLOAD_LENGTH_MASK, cmd_reg); + + /* #8 */ + if (out_len && mbox_cmd->payload_out) { + /* + * Sanitize the copy. If hardware misbehaves, out_len per the + * spec can actually be greater than the max allowed size (21 + * bits available but spec defined 1M max). The caller also may + * have requested less data than the hardware supplied even + * within spec. + */ + size_t n = min3(mbox_cmd->size_out, cxlm->payload_size, out_len); + + memcpy_fromio(mbox_cmd->payload_out, payload, n); + mbox_cmd->size_out = n; + } else { + mbox_cmd->size_out = 0; + } + + return 0; +} + +/** + * cxl_mem_mbox_get() - Acquire exclusive access to the mailbox. + * @cxlm: The memory device to gain access to. + * + * Context: Any context. Takes the mbox_mutex. + * Return: 0 if exclusive access was acquired. + */ +static int cxl_mem_mbox_get(struct cxl_mem *cxlm) +{ + struct device *dev = &cxlm->pdev->dev; + u64 md_status; + int rc; + + mutex_lock_io(&cxlm->mbox_mutex); + + /* + * XXX: There is some amount of ambiguity in the 2.0 version of the spec + * around the mailbox interface ready (8.2.8.5.1.1). The purpose of the + * bit is to allow firmware running on the device to notify the driver + * that it's ready to receive commands. It is unclear if the bit needs + * to be read for each transaction mailbox, ie. the firmware can switch + * it on and off as needed. Second, there is no defined timeout for + * mailbox ready, like there is for the doorbell interface. + * + * Assumptions: + * 1. The firmware might toggle the Mailbox Interface Ready bit, check + * it for every command. + * + * 2. If the doorbell is clear, the firmware should have first set the + * Mailbox Interface Ready bit. Therefore, waiting for the doorbell + * to be ready is sufficient. + */ + rc = cxl_mem_wait_for_doorbell(cxlm); + if (rc) { + dev_warn(dev, "Mailbox interface not ready\n"); + goto out; + } + + md_status = readq(cxlm->memdev_regs + CXLMDEV_STATUS_OFFSET); + if (!(md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status))) { + dev_err(dev, "mbox: reported doorbell ready, but not mbox ready\n"); + rc = -EBUSY; + goto out; + } + + /* + * Hardware shouldn't allow a ready status but also have failure bits + * set. Spit out an error, this should be a bug report + */ + rc = -EFAULT; + if (md_status & CXLMDEV_DEV_FATAL) { + dev_err(dev, "mbox: reported ready, but fatal\n"); + goto out; + } + if (md_status & CXLMDEV_FW_HALT) { + dev_err(dev, "mbox: reported ready, but halted\n"); + goto out; + } + if (CXLMDEV_RESET_NEEDED(md_status)) { + dev_err(dev, "mbox: reported ready, but reset needed\n"); + goto out; + } + + /* with lock held */ + return 0; + +out: + mutex_unlock(&cxlm->mbox_mutex); + return rc; +} + +/** + * cxl_mem_mbox_put() - Release exclusive access to the mailbox. + * @cxlm: The CXL memory device to communicate with. + * + * Context: Any context. Expects mbox_mutex to be held. + */ +static void cxl_mem_mbox_put(struct cxl_mem *cxlm) +{ + mutex_unlock(&cxlm->mbox_mutex); +} + +/** + * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device. + * @cxlm: The CXL memory device to communicate with. + * @opcode: Opcode for the mailbox command. + * @in: The input payload for the mailbox command. + * @in_size: The length of the input payload + * @out: Caller allocated buffer for the output. + * @out_size: Expected size of output. + * + * Context: Any context. Will acquire and release mbox_mutex. + * Return: + * * %>=0 - Number of bytes returned in @out. + * * %-E2BIG - Payload is too large for hardware. + * * %-EBUSY - Couldn't acquire exclusive mailbox access. + * * %-EFAULT - Hardware error occurred. + * * %-ENXIO - Command completed, but device reported an error. + * * %-EIO - Unexpected output size. + * + * Mailbox commands may execute successfully yet the device itself reported an + * error. While this distinction can be useful for commands from userspace, the + * kernel will only be able to use results when both are successful. It's + * expected that all callers of this function know exactly the size of the data + * they will consume from the hardware. + * + * See __cxl_mem_mbox_send_cmd() + */ +static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode, + void *in, size_t in_size, + void *out, size_t out_size) +{ + struct mbox_cmd mbox_cmd = { + .opcode = opcode, + .payload_in = in, + .size_in = in_size, + .size_out = out_size, + .payload_out = out, + }; + int rc; + + if (out_size > cxlm->payload_size) + return -E2BIG; + + rc = cxl_mem_mbox_get(cxlm); + if (rc) + return rc; + + rc = __cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd); + cxl_mem_mbox_put(cxlm); + if (rc) + return rc; + + /* TODO: Map return code to proper kernel style errno */ + if (mbox_cmd.return_code != CXL_MBOX_SUCCESS) + return -ENXIO; + + if (mbox_cmd.size_out != out_size) + return -EIO; + + return 0; +} + +/** + * cxl_mem_setup_regs() - Setup necessary MMIO. + * @cxlm: The CXL memory device to communicate with. + * + * Return: 0 if all necessary registers mapped. + * + * A memory device is required by spec to implement a certain set of MMIO + * regions. The purpose of this function is to enumerate and map those + * registers. + */ +static int cxl_mem_setup_regs(struct cxl_mem *cxlm) +{ + struct device *dev = &cxlm->pdev->dev; + int cap, cap_count; + u64 cap_array; + + cap_array = readq(cxlm->regs + CXLDEV_CAP_ARRAY_OFFSET); + if (FIELD_GET(CXLDEV_CAP_ARRAY_ID_MASK, cap_array) != + CXLDEV_CAP_ARRAY_CAP_ID) + return -ENODEV; + + cap_count = FIELD_GET(CXLDEV_CAP_ARRAY_COUNT_MASK, cap_array); + + for (cap = 1; cap <= cap_count; cap++) { + void __iomem *register_block; + u32 offset; + u16 cap_id; + + cap_id = FIELD_GET(CXLDEV_CAP_HDR_CAP_ID_MASK, + readl(cxlm->regs + cap * 0x10)); + offset = readl(cxlm->regs + cap * 0x10 + 0x4); + register_block = cxlm->regs + offset; + + switch (cap_id) { + case CXLDEV_CAP_CAP_ID_DEVICE_STATUS: + dev_dbg(dev, "found Status capability (0x%x)\n", offset); + cxlm->status_regs = register_block; + break; + case CXLDEV_CAP_CAP_ID_PRIMARY_MAILBOX: + dev_dbg(dev, "found Mailbox capability (0x%x)\n", offset); + cxlm->mbox_regs = register_block; + break; + case CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX: + dev_dbg(dev, "found Secondary Mailbox capability (0x%x)\n", offset); + break; + case CXLDEV_CAP_CAP_ID_MEMDEV: + dev_dbg(dev, "found Memory Device capability (0x%x)\n", offset); + cxlm->memdev_regs = register_block; + break; + default: + dev_dbg(dev, "Unknown cap ID: %d (0x%x)\n", cap_id, offset); + break; + } + } + + if (!cxlm->status_regs || !cxlm->mbox_regs || !cxlm->memdev_regs) { + dev_err(dev, "registers not found: %s%s%s\n", + !cxlm->status_regs ? "status " : "", + !cxlm->mbox_regs ? "mbox " : "", + !cxlm->memdev_regs ? "memdev" : ""); + return -ENXIO; + } + + return 0; +} + +static int cxl_mem_setup_mailbox(struct cxl_mem *cxlm) +{ + const int cap = readl(cxlm->mbox_regs + CXLDEV_MBOX_CAPS_OFFSET); + + cxlm->payload_size = + 1 << FIELD_GET(CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK, cap); + + /* + * CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register + * + * If the size is too small, mandatory commands will not work and so + * there's no point in going forward. If the size is too large, there's + * no harm is soft limiting it. + */ + cxlm->payload_size = min_t(size_t, cxlm->payload_size, SZ_1M); + if (cxlm->payload_size < 256) { + dev_err(&cxlm->pdev->dev, "Mailbox is too small (%zub)", + cxlm->payload_size); + return -ENXIO; + } + + dev_dbg(&cxlm->pdev->dev, "Mailbox payload sized %zu", + cxlm->payload_size); + + return 0; +} + +static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, + u32 reg_hi) +{ + struct device *dev = &pdev->dev; + struct cxl_mem *cxlm; + void __iomem *regs; + u64 offset; + u8 bar; + int rc; + + cxlm = devm_kzalloc(&pdev->dev, sizeof(*cxlm), GFP_KERNEL); + if (!cxlm) { + dev_err(dev, "No memory available\n"); + return NULL; + } + + offset = ((u64)reg_hi << 32) | FIELD_GET(CXL_REGLOC_ADDR_MASK, reg_lo); + bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo); + + /* Basic sanity check that BAR is big enough */ + if (pci_resource_len(pdev, bar) < offset) { + dev_err(dev, "BAR%d: %pr: too small (offset: %#llx)\n", bar, + &pdev->resource[bar], (unsigned long long)offset); + return NULL; + } + + rc = pcim_iomap_regions(pdev, BIT(bar), pci_name(pdev)); + if (rc) { + dev_err(dev, "failed to map registers\n"); + return NULL; + } + regs = pcim_iomap_table(pdev)[bar]; + + mutex_init(&cxlm->mbox_mutex); + cxlm->pdev = pdev; + cxlm->regs = regs + offset; + + dev_dbg(dev, "Mapped CXL Memory Device resource\n"); + return cxlm; +} static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) { @@ -28,10 +512,65 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) return 0; } +/** + * cxl_mem_identify() - Send the IDENTIFY command to the device. + * @cxlm: The device to identify. + * + * Return: 0 if identify was executed successfully. + * + * This will dispatch the identify command to the device and on success populate + * structures to be exported to sysfs. + */ +static int cxl_mem_identify(struct cxl_mem *cxlm) +{ + struct cxl_mbox_identify { + char fw_revision[0x10]; + __le64 total_capacity; + __le64 volatile_capacity; + __le64 persistent_capacity; + __le64 partition_align; + __le16 info_event_log_size; + __le16 warning_event_log_size; + __le16 failure_event_log_size; + __le16 fatal_event_log_size; + __le32 lsa_size; + u8 poison_list_max_mer[3]; + __le16 inject_poison_limit; + u8 poison_caps; + u8 qos_telemetry_caps; + } __packed id; + int rc; + + rc = cxl_mem_mbox_send_cmd(cxlm, CXL_MBOX_OP_IDENTIFY, NULL, 0, &id, + sizeof(id)); + if (rc < 0) + return rc; + + /* + * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias. + * For now, only the capacity is exported in sysfs + */ + cxlm->ram_range.start = 0; + cxlm->ram_range.end = le64_to_cpu(id.volatile_capacity) - 1; + + cxlm->pmem_range.start = 0; + cxlm->pmem_range.end = le64_to_cpu(id.persistent_capacity) - 1; + + memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision)); + + return 0; +} + static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct device *dev = &pdev->dev; - int regloc; + struct cxl_mem *cxlm = NULL; + u32 regloc_size, regblocks; + int rc, regloc, i; + + rc = pcim_enable_device(pdev); + if (rc) + return rc; regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); if (!regloc) { @@ -39,7 +578,41 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) return -ENXIO; } - return 0; + /* Get the size of the Register Locator DVSEC */ + pci_read_config_dword(pdev, regloc + PCI_DVSEC_HEADER1, ®loc_size); + regloc_size = FIELD_GET(PCI_DVSEC_HEADER1_LENGTH_MASK, regloc_size); + + regloc += PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET; + regblocks = (regloc_size - PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET) / 8; + + for (i = 0; i < regblocks; i++, regloc += 8) { + u32 reg_lo, reg_hi; + u8 reg_type; + + /* "register low and high" contain other bits */ + pci_read_config_dword(pdev, regloc, ®_lo); + pci_read_config_dword(pdev, regloc + 4, ®_hi); + + reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo); + + if (reg_type == CXL_REGLOC_RBI_MEMDEV) { + cxlm = cxl_mem_create(pdev, reg_lo, reg_hi); + break; + } + } + + if (!cxlm) + return -ENODEV; + + rc = cxl_mem_setup_regs(cxlm); + if (rc) + return rc; + + rc = cxl_mem_setup_mailbox(cxlm); + if (rc) + return rc; + + return cxl_mem_identify(cxlm); } static const struct pci_device_id cxl_mem_pci_tbl[] = { diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h index e464bea3f4d3..af3ec078cf6c 100644 --- a/drivers/cxl/pci.h +++ b/drivers/cxl/pci.h @@ -9,9 +9,23 @@ * See section 8.1 Configuration Space Registers in the CXL 2.0 * Specification */ +#define PCI_DVSEC_HEADER1_LENGTH_MASK GENMASK(31, 20) #define PCI_DVSEC_VENDOR_ID_CXL 0x1E98 #define PCI_DVSEC_ID_CXL 0x0 #define PCI_DVSEC_ID_CXL_REGLOC_OFFSET 0x8 +#define PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET 0xC + +/* BAR Indicator Register (BIR) */ +#define CXL_REGLOC_BIR_MASK GENMASK(2, 0) + +/* Register Block Identifier (RBI) */ +#define CXL_REGLOC_RBI_MASK GENMASK(15, 8) +#define CXL_REGLOC_RBI_EMPTY 0 +#define CXL_REGLOC_RBI_COMPONENT 1 +#define CXL_REGLOC_RBI_VIRT 2 +#define CXL_REGLOC_RBI_MEMDEV 3 + +#define CXL_REGLOC_ADDR_MASK GENMASK(31, 16) #endif /* __CXL_PCI_H__ */ From b39cb1052a5cf41bc12201ec1c0ddae5cb8be868 Mon Sep 17 00:00:00 2001 From: Dan Williams Date: Tue, 16 Feb 2021 20:09:52 -0800 Subject: [PATCH 54/91] cxl/mem: Register CXL memX devices Create the /sys/bus/cxl hierarchy to enumerate: * Memory Devices (per-endpoint control devices) * Memory Address Space Devices (platform address ranges with interleaving, performance, and persistence attributes) * Memory Regions (active provisioned memory from an address space device that is in use as System RAM or delegated to libnvdimm as Persistent Memory regions). For now, only the per-endpoint control devices are registered on the 'cxl' bus. However, going forward it will provide a mechanism to coordinate cross-device interleave. Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron (v2) Link: https://lore.kernel.org/r/20210217040958.1354670-4-ben.widawsky@intel.com Signed-off-by: Dan Williams --- Documentation/ABI/testing/sysfs-bus-cxl | 26 ++ .../driver-api/cxl/memory-devices.rst | 5 + drivers/cxl/Makefile | 3 + drivers/cxl/bus.c | 29 ++ drivers/cxl/cxl.h | 3 + drivers/cxl/mem.c | 285 +++++++++++++++++- 6 files changed, 349 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-bus-cxl create mode 100644 drivers/cxl/bus.c diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl new file mode 100644 index 000000000000..2fe7490ad6a8 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -0,0 +1,26 @@ +What: /sys/bus/cxl/devices/memX/firmware_version +Date: December, 2020 +KernelVersion: v5.12 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) "FW Revision" string as reported by the Identify + Memory Device Output Payload in the CXL-2.0 + specification. + +What: /sys/bus/cxl/devices/memX/ram/size +Date: December, 2020 +KernelVersion: v5.12 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) "Volatile Only Capacity" as bytes. Represents the + identically named field in the Identify Memory Device Output + Payload in the CXL-2.0 specification. + +What: /sys/bus/cxl/devices/memX/pmem/size +Date: December, 2020 +KernelVersion: v5.12 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) "Persistent Only Capacity" as bytes. Represents the + identically named field in the Identify Memory Device Output + Payload in the CXL-2.0 specification. diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst index 43177e700d62..1fef2c0a167d 100644 --- a/Documentation/driver-api/cxl/memory-devices.rst +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -27,3 +27,8 @@ CXL Memory Device .. kernel-doc:: drivers/cxl/mem.c :internal: + +CXL Bus +------- +.. kernel-doc:: drivers/cxl/bus.c + :doc: cxl bus diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile index 4a30f7c3fc4a..a314a1891f4d 100644 --- a/drivers/cxl/Makefile +++ b/drivers/cxl/Makefile @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_CXL_BUS) += cxl_bus.o obj-$(CONFIG_CXL_MEM) += cxl_mem.o +ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL +cxl_bus-y := bus.o cxl_mem-y := mem.o diff --git a/drivers/cxl/bus.c b/drivers/cxl/bus.c new file mode 100644 index 000000000000..58f74796d525 --- /dev/null +++ b/drivers/cxl/bus.c @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ +#include +#include + +/** + * DOC: cxl bus + * + * The CXL bus provides namespace for control devices and a rendezvous + * point for cross-device interleave coordination. + */ +struct bus_type cxl_bus_type = { + .name = "cxl", +}; +EXPORT_SYMBOL_GPL(cxl_bus_type); + +static __init int cxl_bus_init(void) +{ + return bus_register(&cxl_bus_type); +} + +static void cxl_bus_exit(void) +{ + bus_unregister(&cxl_bus_type); +} + +module_init(cxl_bus_init); +module_exit(cxl_bus_exit); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index baac26d9e63b..8fd4a177fe25 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -57,6 +57,7 @@ (FIELD_GET(CXLMDEV_RESET_NEEDED_MASK, status) != \ CXLMDEV_RESET_NEEDED_NOT) +struct cxl_memdev; /** * struct cxl_mem - A CXL memory device * @pdev: The PCI device associated with this CXL device. @@ -74,6 +75,7 @@ struct cxl_mem { struct pci_dev *pdev; void __iomem *regs; + struct cxl_memdev *cxlmd; void __iomem *status_regs; void __iomem *mbox_regs; @@ -87,4 +89,5 @@ struct cxl_mem { struct range ram_range; }; +extern struct bus_type cxl_bus_type; #endif /* __CXL_H__ */ diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 04e15759bc9b..1c0195b07063 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -1,6 +1,9 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ #include +#include +#include +#include #include #include #include @@ -23,6 +26,12 @@ * - Handle and manage error conditions. */ +/* + * An entire PCI topology full of devices should be enough for any + * config + */ +#define CXL_MEM_MAX_DEVS 65536 + #define cxl_doorbell_busy(cxlm) \ (readl((cxlm)->mbox_regs + CXLDEV_MBOX_CTRL_OFFSET) & \ CXLDEV_MBOX_CTRL_DOORBELL) @@ -65,6 +74,27 @@ struct mbox_cmd { #define CXL_MBOX_SUCCESS 0 }; +/** + * struct cxl_memdev - CXL bus object representing a Type-3 Memory Device + * @dev: driver core device object + * @cdev: char dev core object for ioctl operations + * @cxlm: pointer to the parent device driver data + * @ops_active: active user of @cxlm in ops handlers + * @ops_dead: completion when all @cxlm ops users have exited + * @id: id number of this memdev instance. + */ +struct cxl_memdev { + struct device dev; + struct cdev cdev; + struct cxl_mem *cxlm; + struct percpu_ref ops_active; + struct completion ops_dead; + int id; +}; + +static int cxl_mem_major; +static DEFINE_IDA(cxl_memdev_ida); + static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm) { const unsigned long start = jiffies; @@ -294,6 +324,33 @@ static void cxl_mem_mbox_put(struct cxl_mem *cxlm) mutex_unlock(&cxlm->mbox_mutex); } +static long cxl_memdev_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + struct cxl_memdev *cxlmd; + struct inode *inode; + int rc = -ENOTTY; + + inode = file_inode(file); + cxlmd = container_of(inode->i_cdev, typeof(*cxlmd), cdev); + + if (!percpu_ref_tryget_live(&cxlmd->ops_active)) + return -ENXIO; + + /* TODO: ioctl body */ + + percpu_ref_put(&cxlmd->ops_active); + + return rc; +} + +static const struct file_operations cxl_memdev_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = cxl_memdev_ioctl, + .compat_ioctl = compat_ptr_ioctl, + .llseek = noop_llseek, +}; + /** * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device. * @cxlm: The CXL memory device to communicate with. @@ -512,6 +569,197 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) return 0; } +static struct cxl_memdev *to_cxl_memdev(struct device *dev) +{ + return container_of(dev, struct cxl_memdev, dev); +} + +static void cxl_memdev_release(struct device *dev) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + + percpu_ref_exit(&cxlmd->ops_active); + ida_free(&cxl_memdev_ida, cxlmd->id); + kfree(cxlmd); +} + +static char *cxl_memdev_devnode(struct device *dev, umode_t *mode, kuid_t *uid, + kgid_t *gid) +{ + return kasprintf(GFP_KERNEL, "cxl/%s", dev_name(dev)); +} + +static ssize_t firmware_version_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + + return sprintf(buf, "%.16s\n", cxlm->firmware_version); +} +static DEVICE_ATTR_RO(firmware_version); + +static ssize_t payload_max_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + + return sprintf(buf, "%zu\n", cxlm->payload_size); +} +static DEVICE_ATTR_RO(payload_max); + +static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + unsigned long long len = range_len(&cxlm->ram_range); + + return sprintf(buf, "%#llx\n", len); +} + +static struct device_attribute dev_attr_ram_size = + __ATTR(size, 0444, ram_size_show, NULL); + +static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + unsigned long long len = range_len(&cxlm->pmem_range); + + return sprintf(buf, "%#llx\n", len); +} + +static struct device_attribute dev_attr_pmem_size = + __ATTR(size, 0444, pmem_size_show, NULL); + +static struct attribute *cxl_memdev_attributes[] = { + &dev_attr_firmware_version.attr, + &dev_attr_payload_max.attr, + NULL, +}; + +static struct attribute *cxl_memdev_pmem_attributes[] = { + &dev_attr_pmem_size.attr, + NULL, +}; + +static struct attribute *cxl_memdev_ram_attributes[] = { + &dev_attr_ram_size.attr, + NULL, +}; + +static struct attribute_group cxl_memdev_attribute_group = { + .attrs = cxl_memdev_attributes, +}; + +static struct attribute_group cxl_memdev_ram_attribute_group = { + .name = "ram", + .attrs = cxl_memdev_ram_attributes, +}; + +static struct attribute_group cxl_memdev_pmem_attribute_group = { + .name = "pmem", + .attrs = cxl_memdev_pmem_attributes, +}; + +static const struct attribute_group *cxl_memdev_attribute_groups[] = { + &cxl_memdev_attribute_group, + &cxl_memdev_ram_attribute_group, + &cxl_memdev_pmem_attribute_group, + NULL, +}; + +static const struct device_type cxl_memdev_type = { + .name = "cxl_memdev", + .release = cxl_memdev_release, + .devnode = cxl_memdev_devnode, + .groups = cxl_memdev_attribute_groups, +}; + +static void cxlmdev_unregister(void *_cxlmd) +{ + struct cxl_memdev *cxlmd = _cxlmd; + struct device *dev = &cxlmd->dev; + + percpu_ref_kill(&cxlmd->ops_active); + cdev_device_del(&cxlmd->cdev, dev); + wait_for_completion(&cxlmd->ops_dead); + cxlmd->cxlm = NULL; + put_device(dev); +} + +static void cxlmdev_ops_active_release(struct percpu_ref *ref) +{ + struct cxl_memdev *cxlmd = + container_of(ref, typeof(*cxlmd), ops_active); + + complete(&cxlmd->ops_dead); +} + +static int cxl_mem_add_memdev(struct cxl_mem *cxlm) +{ + struct pci_dev *pdev = cxlm->pdev; + struct cxl_memdev *cxlmd; + struct device *dev; + struct cdev *cdev; + int rc; + + cxlmd = kzalloc(sizeof(*cxlmd), GFP_KERNEL); + if (!cxlmd) + return -ENOMEM; + init_completion(&cxlmd->ops_dead); + + /* + * @cxlm is deallocated when the driver unbinds so operations + * that are using it need to hold a live reference. + */ + cxlmd->cxlm = cxlm; + rc = percpu_ref_init(&cxlmd->ops_active, cxlmdev_ops_active_release, 0, + GFP_KERNEL); + if (rc) + goto err_ref; + + rc = ida_alloc_range(&cxl_memdev_ida, 0, CXL_MEM_MAX_DEVS, GFP_KERNEL); + if (rc < 0) + goto err_id; + cxlmd->id = rc; + + dev = &cxlmd->dev; + device_initialize(dev); + dev->parent = &pdev->dev; + dev->bus = &cxl_bus_type; + dev->devt = MKDEV(cxl_mem_major, cxlmd->id); + dev->type = &cxl_memdev_type; + dev_set_name(dev, "mem%d", cxlmd->id); + + cdev = &cxlmd->cdev; + cdev_init(cdev, &cxl_memdev_fops); + + rc = cdev_device_add(cdev, dev); + if (rc) + goto err_add; + + return devm_add_action_or_reset(dev->parent, cxlmdev_unregister, cxlmd); + +err_add: + ida_free(&cxl_memdev_ida, cxlmd->id); +err_id: + /* + * Theoretically userspace could have already entered the fops, + * so flush ops_active. + */ + percpu_ref_kill(&cxlmd->ops_active); + wait_for_completion(&cxlmd->ops_dead); + percpu_ref_exit(&cxlmd->ops_active); +err_ref: + kfree(cxlmd); + + return rc; +} + /** * cxl_mem_identify() - Send the IDENTIFY command to the device. * @cxlm: The device to identify. @@ -612,7 +860,11 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; - return cxl_mem_identify(cxlm); + rc = cxl_mem_identify(cxlm); + if (rc) + return rc; + + return cxl_mem_add_memdev(cxlm); } static const struct pci_device_id cxl_mem_pci_tbl[] = { @@ -631,5 +883,34 @@ static struct pci_driver cxl_mem_driver = { }, }; +static __init int cxl_mem_init(void) +{ + dev_t devt; + int rc; + + rc = alloc_chrdev_region(&devt, 0, CXL_MEM_MAX_DEVS, "cxl"); + if (rc) + return rc; + + cxl_mem_major = MAJOR(devt); + + rc = pci_register_driver(&cxl_mem_driver); + if (rc) { + unregister_chrdev_region(MKDEV(cxl_mem_major, 0), + CXL_MEM_MAX_DEVS); + return rc; + } + + return 0; +} + +static __exit void cxl_mem_exit(void) +{ + pci_unregister_driver(&cxl_mem_driver); + unregister_chrdev_region(MKDEV(cxl_mem_major, 0), CXL_MEM_MAX_DEVS); +} + MODULE_LICENSE("GPL v2"); -module_pci_driver(cxl_mem_driver); +module_init(cxl_mem_init); +module_exit(cxl_mem_exit); +MODULE_IMPORT_NS(CXL); From 583fa5e71caeb79e04e477e9837e2f7fa53b71e4 Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Tue, 16 Feb 2021 20:09:53 -0800 Subject: [PATCH 55/91] cxl/mem: Add basic IOCTL interface Add a straightforward IOCTL that provides a mechanism for userspace to query the supported memory device commands. CXL commands as they appear to userspace are described as part of the UAPI kerneldoc. The command list returned via this IOCTL will contain the full set of commands that the driver supports, however, some of those commands may not be available for use by userspace. Memory device commands first appear in the CXL 2.0 specification. They are submitted through a mailbox mechanism specified in the CXL 2.0 specification. The send command allows userspace to issue mailbox commands directly to the hardware. The list of available commands to send are the output of the query command. The driver verifies basic properties of the command and possibly inspect the input (or output) payload to determine whether or not the command is allowed (or might taint the kernel). Reported-by: kernel test robot # bug in earlier revision Reported-by: Stephen Rothwell Signed-off-by: Ben Widawsky Reviewed-by: Dan Williams (v2) Cc: Al Viro Link: https://lore.kernel.org/r/20210217040958.1354670-5-ben.widawsky@intel.com Signed-off-by: Dan Williams --- .clang-format | 1 + .../driver-api/cxl/memory-devices.rst | 12 + .../userspace-api/ioctl/ioctl-number.rst | 1 + drivers/cxl/mem.c | 283 +++++++++++++++++- include/uapi/linux/cxl_mem.h | 156 ++++++++++ 5 files changed, 452 insertions(+), 1 deletion(-) create mode 100644 include/uapi/linux/cxl_mem.h diff --git a/.clang-format b/.clang-format index 10dc5a9a61b3..3f11c8901b43 100644 --- a/.clang-format +++ b/.clang-format @@ -109,6 +109,7 @@ ForEachMacros: - 'css_for_each_child' - 'css_for_each_descendant_post' - 'css_for_each_descendant_pre' + - 'cxl_for_each_cmd' - 'device_for_each_child_node' - 'dma_fence_chain_for_each' - 'do_for_each_ftrace_op' diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst index 1fef2c0a167d..1bad466f9167 100644 --- a/Documentation/driver-api/cxl/memory-devices.rst +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -32,3 +32,15 @@ CXL Bus ------- .. kernel-doc:: drivers/cxl/bus.c :doc: cxl bus + +External Interfaces +=================== + +CXL IOCTL Interface +------------------- + +.. kernel-doc:: include/uapi/linux/cxl_mem.h + :doc: UAPI + +.. kernel-doc:: include/uapi/linux/cxl_mem.h + :internal: diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index a4c75a28c839..6eb8e634664d 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -352,6 +352,7 @@ Code Seq# Include File Comments 0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver 0xCD 01 linux/reiserfs_fs.h +0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices 0xCF 02 fs/cifs/ioctl.c 0xDB 00-0F drivers/char/mwave/mwavepub.h 0xDD 00-3F ZFCP device driver see drivers/s390/scsi/ diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 1c0195b07063..aa8f843fcca1 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ +#include #include #include #include @@ -40,6 +41,7 @@ #define CXL_MAILBOX_TIMEOUT_MS (2 * HZ) enum opcode { + CXL_MBOX_OP_INVALID = 0x0000, CXL_MBOX_OP_IDENTIFY = 0x4000, CXL_MBOX_OP_MAX = 0x10000 }; @@ -95,6 +97,49 @@ struct cxl_memdev { static int cxl_mem_major; static DEFINE_IDA(cxl_memdev_ida); +/** + * struct cxl_mem_command - Driver representation of a memory device command + * @info: Command information as it exists for the UAPI + * @opcode: The actual bits used for the mailbox protocol + * + * The cxl_mem_command is the driver's internal representation of commands that + * are supported by the driver. Some of these commands may not be supported by + * the hardware. The driver will use @info to validate the fields passed in by + * the user then submit the @opcode to the hardware. + * + * See struct cxl_command_info. + */ +struct cxl_mem_command { + struct cxl_command_info info; + enum opcode opcode; +}; + +#define CXL_CMD(_id, sin, sout) \ + [CXL_MEM_COMMAND_ID_##_id] = { \ + .info = { \ + .id = CXL_MEM_COMMAND_ID_##_id, \ + .size_in = sin, \ + .size_out = sout, \ + }, \ + .opcode = CXL_MBOX_OP_##_id, \ + } + +/* + * This table defines the supported mailbox commands for the driver. This table + * is made up of a UAPI structure. Non-negative values as parameters in the + * table will be validated against the user's input. For example, if size_in is + * 0, and the user passed in 1, it is an error. + */ +static struct cxl_mem_command mem_commands[] = { + CXL_CMD(IDENTIFY, 0, 0x43), +}; + +#define cxl_for_each_cmd(cmd) \ + for ((cmd) = &mem_commands[0]; \ + ((cmd) - mem_commands) < ARRAY_SIZE(mem_commands); (cmd)++) + +#define cxl_cmd_count ARRAY_SIZE(mem_commands) + static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm) { const unsigned long start = jiffies; @@ -324,6 +369,242 @@ static void cxl_mem_mbox_put(struct cxl_mem *cxlm) mutex_unlock(&cxlm->mbox_mutex); } +/** + * handle_mailbox_cmd_from_user() - Dispatch a mailbox command for userspace. + * @cxlm: The CXL memory device to communicate with. + * @cmd: The validated command. + * @in_payload: Pointer to userspace's input payload. + * @out_payload: Pointer to userspace's output payload. + * @size_out: (Input) Max payload size to copy out. + * (Output) Payload size hardware generated. + * @retval: Hardware generated return code from the operation. + * + * Return: + * * %0 - Mailbox transaction succeeded. This implies the mailbox + * protocol completed successfully not that the operation itself + * was successful. + * * %-ENOMEM - Couldn't allocate a bounce buffer. + * * %-EFAULT - Something happened with copy_to/from_user. + * * %-EINTR - Mailbox acquisition interrupted. + * * %-EXXX - Transaction level failures. + * + * Creates the appropriate mailbox command and dispatches it on behalf of a + * userspace request. The input and output payloads are copied between + * userspace. + * + * See cxl_send_cmd(). + */ +static int handle_mailbox_cmd_from_user(struct cxl_mem *cxlm, + const struct cxl_mem_command *cmd, + u64 in_payload, u64 out_payload, + s32 *size_out, u32 *retval) +{ + struct device *dev = &cxlm->pdev->dev; + struct mbox_cmd mbox_cmd = { + .opcode = cmd->opcode, + .size_in = cmd->info.size_in, + .size_out = cmd->info.size_out, + }; + int rc; + + if (cmd->info.size_out) { + mbox_cmd.payload_out = kvzalloc(cmd->info.size_out, GFP_KERNEL); + if (!mbox_cmd.payload_out) + return -ENOMEM; + } + + if (cmd->info.size_in) { + mbox_cmd.payload_in = vmemdup_user(u64_to_user_ptr(in_payload), + cmd->info.size_in); + if (IS_ERR(mbox_cmd.payload_in)) + return PTR_ERR(mbox_cmd.payload_in); + } + + rc = cxl_mem_mbox_get(cxlm); + if (rc) + goto out; + + dev_dbg(dev, + "Submitting %s command for user\n" + "\topcode: %x\n" + "\tsize: %ub\n", + cxl_command_names[cmd->info.id].name, mbox_cmd.opcode, + cmd->info.size_in); + + rc = __cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd); + cxl_mem_mbox_put(cxlm); + if (rc) + goto out; + + /* + * @size_out contains the max size that's allowed to be written back out + * to userspace. While the payload may have written more output than + * this it will have to be ignored. + */ + if (mbox_cmd.size_out) { + dev_WARN_ONCE(dev, mbox_cmd.size_out > *size_out, + "Invalid return size\n"); + if (copy_to_user(u64_to_user_ptr(out_payload), + mbox_cmd.payload_out, mbox_cmd.size_out)) { + rc = -EFAULT; + goto out; + } + } + + *size_out = mbox_cmd.size_out; + *retval = mbox_cmd.return_code; + +out: + kvfree(mbox_cmd.payload_in); + kvfree(mbox_cmd.payload_out); + return rc; +} + +/** + * cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND. + * @cxlm: &struct cxl_mem device whose mailbox will be used. + * @send_cmd: &struct cxl_send_command copied in from userspace. + * @out_cmd: Sanitized and populated &struct cxl_mem_command. + * + * Return: + * * %0 - @out_cmd is ready to send. + * * %-ENOTTY - Invalid command specified. + * * %-EINVAL - Reserved fields or invalid values were used. + * * %-ENOMEM - Input or output buffer wasn't sized properly. + * + * The result of this command is a fully validated command in @out_cmd that is + * safe to send to the hardware. + * + * See handle_mailbox_cmd_from_user() + */ +static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm, + const struct cxl_send_command *send_cmd, + struct cxl_mem_command *out_cmd) +{ + const struct cxl_command_info *info; + struct cxl_mem_command *c; + + if (send_cmd->id == 0 || send_cmd->id >= CXL_MEM_COMMAND_ID_MAX) + return -ENOTTY; + + /* + * The user can never specify an input payload larger than what hardware + * supports, but output can be arbitrarily large (simply write out as + * much data as the hardware provides). + */ + if (send_cmd->in.size > cxlm->payload_size) + return -EINVAL; + + if (send_cmd->flags & ~CXL_MEM_COMMAND_FLAG_MASK) + return -EINVAL; + + if (send_cmd->rsvd) + return -EINVAL; + + if (send_cmd->in.rsvd || send_cmd->out.rsvd) + return -EINVAL; + + /* Convert user's command into the internal representation */ + c = &mem_commands[send_cmd->id]; + info = &c->info; + + /* Check the input buffer is the expected size */ + if (info->size_in >= 0 && info->size_in != send_cmd->in.size) + return -ENOMEM; + + /* Check the output buffer is at least large enough */ + if (info->size_out >= 0 && send_cmd->out.size < info->size_out) + return -ENOMEM; + + memcpy(out_cmd, c, sizeof(*c)); + out_cmd->info.size_in = send_cmd->in.size; + /* + * XXX: out_cmd->info.size_out will be controlled by the driver, and the + * specified number of bytes @send_cmd->out.size will be copied back out + * to userspace. + */ + + return 0; +} + +static int cxl_query_cmd(struct cxl_memdev *cxlmd, + struct cxl_mem_query_commands __user *q) +{ + struct device *dev = &cxlmd->dev; + struct cxl_mem_command *cmd; + u32 n_commands; + int j = 0; + + dev_dbg(dev, "Query IOCTL\n"); + + if (get_user(n_commands, &q->n_commands)) + return -EFAULT; + + /* returns the total number if 0 elements are requested. */ + if (n_commands == 0) + return put_user(cxl_cmd_count, &q->n_commands); + + /* + * otherwise, return max(n_commands, total commands) cxl_command_info + * structures. + */ + cxl_for_each_cmd(cmd) { + const struct cxl_command_info *info = &cmd->info; + + if (copy_to_user(&q->commands[j++], info, sizeof(*info))) + return -EFAULT; + + if (j == n_commands) + break; + } + + return 0; +} + +static int cxl_send_cmd(struct cxl_memdev *cxlmd, + struct cxl_send_command __user *s) +{ + struct cxl_mem *cxlm = cxlmd->cxlm; + struct device *dev = &cxlmd->dev; + struct cxl_send_command send; + struct cxl_mem_command c; + int rc; + + dev_dbg(dev, "Send IOCTL\n"); + + if (copy_from_user(&send, s, sizeof(send))) + return -EFAULT; + + rc = cxl_validate_cmd_from_user(cxlmd->cxlm, &send, &c); + if (rc) + return rc; + + /* Prepare to handle a full payload for variable sized output */ + if (c.info.size_out < 0) + c.info.size_out = cxlm->payload_size; + + rc = handle_mailbox_cmd_from_user(cxlm, &c, send.in.payload, + send.out.payload, &send.out.size, + &send.retval); + if (rc) + return rc; + + return copy_to_user(s, &send, sizeof(send)); +} + +static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd, + unsigned long arg) +{ + switch (cmd) { + case CXL_MEM_QUERY_COMMANDS: + return cxl_query_cmd(cxlmd, (void __user *)arg); + case CXL_MEM_SEND_COMMAND: + return cxl_send_cmd(cxlmd, (void __user *)arg); + default: + return -ENOTTY; + } +} + static long cxl_memdev_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -337,7 +618,7 @@ static long cxl_memdev_ioctl(struct file *file, unsigned int cmd, if (!percpu_ref_tryget_live(&cxlmd->ops_active)) return -ENXIO; - /* TODO: ioctl body */ + rc = __cxl_memdev_ioctl(cxlmd, cmd, arg); percpu_ref_put(&cxlmd->ops_active); diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h new file mode 100644 index 000000000000..887781dc3b6c --- /dev/null +++ b/include/uapi/linux/cxl_mem.h @@ -0,0 +1,156 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * CXL IOCTLs for Memory Devices + */ + +#ifndef _UAPI_CXL_MEM_H_ +#define _UAPI_CXL_MEM_H_ + +#include + +/** + * DOC: UAPI + * + * Not all of all commands that the driver supports are always available for use + * by userspace. Userspace must check the results from the QUERY command in + * order to determine the live set of commands. + */ + +#define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands) +#define CXL_MEM_SEND_COMMAND _IOWR(0xCE, 2, struct cxl_send_command) + +#define CXL_CMDS \ + ___C(INVALID, "Invalid Command"), \ + ___C(IDENTIFY, "Identify Command"), \ + ___C(MAX, "invalid / last command") + +#define ___C(a, b) CXL_MEM_COMMAND_ID_##a +enum { CXL_CMDS }; + +#undef ___C +#define ___C(a, b) { b } +static const struct { + const char *name; +} cxl_command_names[] = { CXL_CMDS }; + +/* + * Here's how this actually breaks out: + * cxl_command_names[] = { + * [CXL_MEM_COMMAND_ID_INVALID] = { "Invalid Command" }, + * [CXL_MEM_COMMAND_ID_IDENTIFY] = { "Identify Command" }, + * ... + * [CXL_MEM_COMMAND_ID_MAX] = { "invalid / last command" }, + * }; + */ + +#undef ___C + +/** + * struct cxl_command_info - Command information returned from a query. + * @id: ID number for the command. + * @flags: Flags that specify command behavior. + * @size_in: Expected input size, or -1 if variable length. + * @size_out: Expected output size, or -1 if variable length. + * + * Represents a single command that is supported by both the driver and the + * hardware. This is returned as part of an array from the query ioctl. The + * following would be a command that takes a variable length input and returns 0 + * bytes of output. + * + * - @id = 10 + * - @flags = 0 + * - @size_in = -1 + * - @size_out = 0 + * + * See struct cxl_mem_query_commands. + */ +struct cxl_command_info { + __u32 id; + + __u32 flags; +#define CXL_MEM_COMMAND_FLAG_MASK GENMASK(0, 0) + + __s32 size_in; + __s32 size_out; +}; + +/** + * struct cxl_mem_query_commands - Query supported commands. + * @n_commands: In/out parameter. When @n_commands is > 0, the driver will + * return min(num_support_commands, n_commands). When @n_commands + * is 0, driver will return the number of total supported commands. + * @rsvd: Reserved for future use. + * @commands: Output array of supported commands. This array must be allocated + * by userspace to be at least min(num_support_commands, @n_commands) + * + * Allow userspace to query the available commands supported by both the driver, + * and the hardware. Commands that aren't supported by either the driver, or the + * hardware are not returned in the query. + * + * Examples: + * + * - { .n_commands = 0 } // Get number of supported commands + * - { .n_commands = 15, .commands = buf } // Return first 15 (or less) + * supported commands + * + * See struct cxl_command_info. + */ +struct cxl_mem_query_commands { + /* + * Input: Number of commands to return (space allocated by user) + * Output: Number of commands supported by the driver/hardware + * + * If n_commands is 0, kernel will only return number of commands and + * not try to populate commands[], thus allowing userspace to know how + * much space to allocate + */ + __u32 n_commands; + __u32 rsvd; + + struct cxl_command_info __user commands[]; /* out: supported commands */ +}; + +/** + * struct cxl_send_command - Send a command to a memory device. + * @id: The command to send to the memory device. This must be one of the + * commands returned by the query command. + * @flags: Flags for the command (input). + * @rsvd: Must be zero. + * @retval: Return value from the memory device (output). + * @in: Parameters associated with input payload. + * @in.size: Size of the payload to provide to the device (input). + * @in.rsvd: Must be zero. + * @in.payload: Pointer to memory for payload input, payload is little endian. + * @out: Parameters associated with output payload. + * @out.size: Size of the payload received from the device (input/output). This + * field is filled in by userspace to let the driver know how much + * space was allocated for output. It is populated by the driver to + * let userspace know how large the output payload actually was. + * @out.rsvd: Must be zero. + * @out.payload: Pointer to memory for payload output, payload is little endian. + * + * Mechanism for userspace to send a command to the hardware for processing. The + * driver will do basic validation on the command sizes. In some cases even the + * payload may be introspected. Userspace is required to allocate large enough + * buffers for size_out which can be variable length in certain situations. + */ +struct cxl_send_command { + __u32 id; + __u32 flags; + __u32 rsvd; + __u32 retval; + + struct { + __s32 size; + __u32 rsvd; + __u64 payload; + } in; + + struct { + __s32 size; + __u32 rsvd; + __u64 payload; + } out; +}; + +#endif From 13237183c735f5cba4ae26bc782c613ae0d4e4d3 Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Tue, 16 Feb 2021 20:09:54 -0800 Subject: [PATCH 56/91] cxl/mem: Add a "RAW" send command The CXL memory device send interface will have a number of supported commands. The raw command is not such a command. Raw commands allow userspace to send a specified opcode to the underlying hardware and bypass all driver checks on the command. The primary use for this command is to [begrudgingly] allow undocumented vendor specific hardware commands. While not the main motivation, it also allows prototyping new hardware commands without a driver patch and rebuild. While this all sounds very powerful it comes with a couple of caveats: 1. Bug reports using raw commands will not get the same level of attention as bug reports using supported commands (via taint). 2. Supported commands will be rejected by the RAW command. With this comes new debugfs knob to allow full access to your toes with your weapon of choice. Signed-off-by: Ben Widawsky Reviewed-by: Dan Williams (v2) Reviewed-by: Jonathan Cameron Cc: Ariel Sibley Link: https://lore.kernel.org/r/20210217040958.1354670-6-ben.widawsky@intel.com Signed-off-by: Dan Williams --- drivers/cxl/Kconfig | 18 +++++ drivers/cxl/mem.c | 132 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 12 +++- 3 files changed, 161 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 9e80b311e928..97dc4d751651 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -32,4 +32,22 @@ config CXL_MEM Chapter 2.3 Type 3 CXL Device in the CXL 2.0 specification. If unsure say 'm'. + +config CXL_MEM_RAW_COMMANDS + bool "RAW Command Interface for Memory Devices" + depends on CXL_MEM + help + Enable CXL RAW command interface. + + The CXL driver ioctl interface may assign a kernel ioctl command + number for each specification defined opcode. At any given point in + time the number of opcodes that the specification defines and a device + may implement may exceed the kernel's set of associated ioctl function + numbers. The mismatch is either by omission, specification is too new, + or by design. When prototyping new hardware, or developing / debugging + the driver it is useful to be able to submit any possible command to + the hardware, even commands that may crash the kernel due to their + potential impact to memory currently in use by the kernel. + + If developing CXL hardware or the driver say Y, otherwise say N. endif diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index aa8f843fcca1..5319412e245c 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -1,6 +1,8 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ #include +#include +#include #include #include #include @@ -42,7 +44,14 @@ enum opcode { CXL_MBOX_OP_INVALID = 0x0000, + CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID, + CXL_MBOX_OP_ACTIVATE_FW = 0x0202, CXL_MBOX_OP_IDENTIFY = 0x4000, + CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, + CXL_MBOX_OP_SET_LSA = 0x4103, + CXL_MBOX_OP_SET_SHUTDOWN_STATE = 0x4204, + CXL_MBOX_OP_SCAN_MEDIA = 0x4304, + CXL_MBOX_OP_GET_SCAN_MEDIA = 0x4305, CXL_MBOX_OP_MAX = 0x10000 }; @@ -96,6 +105,8 @@ struct cxl_memdev { static int cxl_mem_major; static DEFINE_IDA(cxl_memdev_ida); +static struct dentry *cxl_debugfs; +static bool cxl_raw_allow_all; /** * struct cxl_mem_command - Driver representation of a memory device command @@ -132,6 +143,49 @@ struct cxl_mem_command { */ static struct cxl_mem_command mem_commands[] = { CXL_CMD(IDENTIFY, 0, 0x43), +#ifdef CONFIG_CXL_MEM_RAW_COMMANDS + CXL_CMD(RAW, ~0, ~0), +#endif +}; + +/* + * Commands that RAW doesn't permit. The rationale for each: + * + * CXL_MBOX_OP_ACTIVATE_FW: Firmware activation requires adjustment / + * coordination of transaction timeout values at the root bridge level. + * + * CXL_MBOX_OP_SET_PARTITION_INFO: The device memory map may change live + * and needs to be coordinated with HDM updates. + * + * CXL_MBOX_OP_SET_LSA: The label storage area may be cached by the + * driver and any writes from userspace invalidates those contents. + * + * CXL_MBOX_OP_SET_SHUTDOWN_STATE: Set shutdown state assumes no writes + * to the device after it is marked clean, userspace can not make that + * assertion. + * + * CXL_MBOX_OP_[GET_]SCAN_MEDIA: The kernel provides a native error list that + * is kept up to date with patrol notifications and error management. + */ +static u16 cxl_disabled_raw_commands[] = { + CXL_MBOX_OP_ACTIVATE_FW, + CXL_MBOX_OP_SET_PARTITION_INFO, + CXL_MBOX_OP_SET_LSA, + CXL_MBOX_OP_SET_SHUTDOWN_STATE, + CXL_MBOX_OP_SCAN_MEDIA, + CXL_MBOX_OP_GET_SCAN_MEDIA, +}; + +/* + * Command sets that RAW doesn't permit. All opcodes in this set are + * disabled because they pass plain text security payloads over the + * user/kernel boundary. This functionality is intended to be wrapped + * behind the keys ABI which allows for encrypted payloads in the UAPI + */ +static u8 security_command_sets[] = { + 0x44, /* Sanitize */ + 0x45, /* Persistent Memory Data-at-rest Security */ + 0x46, /* Security Passthrough */ }; #define cxl_for_each_cmd(cmd) \ @@ -162,6 +216,16 @@ static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm) return 0; } +static bool cxl_is_security_command(u16 opcode) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(security_command_sets); i++) + if (security_command_sets[i] == (opcode >> 8)) + return true; + return false; +} + static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm, struct mbox_cmd *mbox_cmd) { @@ -431,6 +495,9 @@ static int handle_mailbox_cmd_from_user(struct cxl_mem *cxlm, cxl_command_names[cmd->info.id].name, mbox_cmd.opcode, cmd->info.size_in); + dev_WARN_ONCE(dev, cmd->info.id == CXL_MEM_COMMAND_ID_RAW, + "raw command path used\n"); + rc = __cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd); cxl_mem_mbox_put(cxlm); if (rc) @@ -460,6 +527,29 @@ out: return rc; } +static bool cxl_mem_raw_command_allowed(u16 opcode) +{ + int i; + + if (!IS_ENABLED(CONFIG_CXL_MEM_RAW_COMMANDS)) + return false; + + if (security_locked_down(LOCKDOWN_NONE)) + return false; + + if (cxl_raw_allow_all) + return true; + + if (cxl_is_security_command(opcode)) + return false; + + for (i = 0; i < ARRAY_SIZE(cxl_disabled_raw_commands); i++) + if (cxl_disabled_raw_commands[i] == opcode) + return false; + + return true; +} + /** * cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND. * @cxlm: &struct cxl_mem device whose mailbox will be used. @@ -471,6 +561,7 @@ out: * * %-ENOTTY - Invalid command specified. * * %-EINVAL - Reserved fields or invalid values were used. * * %-ENOMEM - Input or output buffer wasn't sized properly. + * * %-EPERM - Attempted to use a protected command. * * The result of this command is a fully validated command in @out_cmd that is * safe to send to the hardware. @@ -495,6 +586,40 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm, if (send_cmd->in.size > cxlm->payload_size) return -EINVAL; + /* + * Checks are bypassed for raw commands but a WARN/taint will occur + * later in the callchain + */ + if (send_cmd->id == CXL_MEM_COMMAND_ID_RAW) { + const struct cxl_mem_command temp = { + .info = { + .id = CXL_MEM_COMMAND_ID_RAW, + .flags = 0, + .size_in = send_cmd->in.size, + .size_out = send_cmd->out.size, + }, + .opcode = send_cmd->raw.opcode + }; + + if (send_cmd->raw.rsvd) + return -EINVAL; + + /* + * Unlike supported commands, the output size of RAW commands + * gets passed along without further checking, so it must be + * validated here. + */ + if (send_cmd->out.size > cxlm->payload_size) + return -EINVAL; + + if (!cxl_mem_raw_command_allowed(send_cmd->raw.opcode)) + return -EPERM; + + memcpy(out_cmd, &temp, sizeof(temp)); + + return 0; + } + if (send_cmd->flags & ~CXL_MEM_COMMAND_FLAG_MASK) return -EINVAL; @@ -1166,6 +1291,7 @@ static struct pci_driver cxl_mem_driver = { static __init int cxl_mem_init(void) { + struct dentry *mbox_debugfs; dev_t devt; int rc; @@ -1182,11 +1308,17 @@ static __init int cxl_mem_init(void) return rc; } + cxl_debugfs = debugfs_create_dir("cxl", NULL); + mbox_debugfs = debugfs_create_dir("mbox", cxl_debugfs); + debugfs_create_bool("raw_allow_all", 0600, mbox_debugfs, + &cxl_raw_allow_all); + return 0; } static __exit void cxl_mem_exit(void) { + debugfs_remove_recursive(cxl_debugfs); pci_unregister_driver(&cxl_mem_driver); unregister_chrdev_region(MKDEV(cxl_mem_major, 0), CXL_MEM_MAX_DEVS); } diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index 887781dc3b6c..c316028730e7 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -22,6 +22,7 @@ #define CXL_CMDS \ ___C(INVALID, "Invalid Command"), \ ___C(IDENTIFY, "Identify Command"), \ + ___C(RAW, "Raw device command"), \ ___C(MAX, "invalid / last command") #define ___C(a, b) CXL_MEM_COMMAND_ID_##a @@ -115,6 +116,9 @@ struct cxl_mem_query_commands { * @id: The command to send to the memory device. This must be one of the * commands returned by the query command. * @flags: Flags for the command (input). + * @raw: Special fields for raw commands + * @raw.opcode: Opcode passed to hardware when using the RAW command. + * @raw.rsvd: Must be zero. * @rsvd: Must be zero. * @retval: Return value from the memory device (output). * @in: Parameters associated with input payload. @@ -137,7 +141,13 @@ struct cxl_mem_query_commands { struct cxl_send_command { __u32 id; __u32 flags; - __u32 rsvd; + union { + struct { + __u16 opcode; + __u16 rsvd; + } raw; + __u32 rsvd; + }; __u32 retval; struct { From 472b1ce6e9d6396ab3f11fc5101c6b63b934a018 Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Tue, 16 Feb 2021 20:09:55 -0800 Subject: [PATCH 57/91] cxl/mem: Enable commands via CEL CXL devices identified by the memory-device class code must implement the Device Command Interface (described in 8.2.9 of the CXL 2.0 spec). While the driver already maintains a list of commands it supports, there is still a need to be able to distinguish between commands that the driver knows about from commands that are optionally supported by the hardware. The Command Effects Log (CEL) is specified in the CXL 2.0 specification. The CEL is one of two types of logs, the other being vendor specific. They are distinguished in hardware/spec via UUID. The CEL is useful for 2 things: 1. Determine which optional commands are supported by the CXL device. 2. Enumerate any vendor specific commands The CEL is used by the driver to determine which commands are available in the hardware and therefore which commands userspace is allowed to execute. The set of enabled commands might be a subset of commands which are advertised in UAPI via CXL_MEM_SEND_COMMAND IOCTL. With the CEL enabling comes a internal flag to indicate a base set of commands that are enabled regardless of CEL. Such commands are required for basic interaction with the hardware and thus can be useful in debug cases, for example if the CEL is corrupted. The implementation leaves the statically defined table of commands and supplements it with a bitmap to determine commands that are enabled. This organization was chosen for the following reasons: - Smaller memory footprint. Doesn't need a table per device. - Reduce memory allocation complexity. - Fixed command IDs to opcode mapping for all devices makes development and debugging easier. - Certain helpers are easily achievable, like cxl_for_each_cmd(). Signed-off-by: Ben Widawsky Reviewed-by: Dan Williams (v2) Reviewed-by: Jonathan Cameron (v3) Link: https://lore.kernel.org/r/20210217040958.1354670-7-ben.widawsky@intel.com Signed-off-by: Dan Williams --- drivers/cxl/cxl.h | 2 + drivers/cxl/mem.c | 223 +++++++++++++++++++++++++++++++++-- include/uapi/linux/cxl_mem.h | 1 + 3 files changed, 219 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 8fd4a177fe25..6f14838c2d25 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -69,6 +69,7 @@ struct cxl_memdev; * (CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register) * @mbox_mutex: Mutex to synchronize mailbox access. * @firmware_version: Firmware version for the memory device. + * @enabled_commands: Hardware commands found enabled in CEL. * @pmem_range: Persistent memory capacity information. * @ram_range: Volatile memory capacity information. */ @@ -84,6 +85,7 @@ struct cxl_mem { size_t payload_size; struct mutex mbox_mutex; /* Protects device mailbox and firmware */ char firmware_version[0x10]; + unsigned long *enabled_cmds; struct range pmem_range; struct range ram_range; diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 5319412e245c..e31b3045e231 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -46,6 +46,8 @@ enum opcode { CXL_MBOX_OP_INVALID = 0x0000, CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID, CXL_MBOX_OP_ACTIVATE_FW = 0x0202, + CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400, + CXL_MBOX_OP_GET_LOG = 0x0401, CXL_MBOX_OP_IDENTIFY = 0x4000, CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, CXL_MBOX_OP_SET_LSA = 0x4103, @@ -108,10 +110,28 @@ static DEFINE_IDA(cxl_memdev_ida); static struct dentry *cxl_debugfs; static bool cxl_raw_allow_all; +enum { + CEL_UUID, + VENDOR_DEBUG_UUID, +}; + +/* See CXL 2.0 Table 170. Get Log Input Payload */ +static const uuid_t log_uuid[] = { + [CEL_UUID] = UUID_INIT(0xda9c0b5, 0xbf41, 0x4b78, 0x8f, 0x79, 0x96, + 0xb1, 0x62, 0x3b, 0x3f, 0x17), + [VENDOR_DEBUG_UUID] = UUID_INIT(0xe1819d9, 0x11a9, 0x400c, 0x81, 0x1f, + 0xd6, 0x07, 0x19, 0x40, 0x3d, 0x86), +}; + /** * struct cxl_mem_command - Driver representation of a memory device command * @info: Command information as it exists for the UAPI * @opcode: The actual bits used for the mailbox protocol + * @flags: Set of flags effecting driver behavior. + * + * * %CXL_CMD_FLAG_FORCE_ENABLE: In cases of error, commands with this flag + * will be enabled by the driver regardless of what hardware may have + * advertised. * * The cxl_mem_command is the driver's internal representation of commands that * are supported by the driver. Some of these commands may not be supported by @@ -123,9 +143,12 @@ static bool cxl_raw_allow_all; struct cxl_mem_command { struct cxl_command_info info; enum opcode opcode; + u32 flags; +#define CXL_CMD_FLAG_NONE 0 +#define CXL_CMD_FLAG_FORCE_ENABLE BIT(0) }; -#define CXL_CMD(_id, sin, sout) \ +#define CXL_CMD(_id, sin, sout, _flags) \ [CXL_MEM_COMMAND_ID_##_id] = { \ .info = { \ .id = CXL_MEM_COMMAND_ID_##_id, \ @@ -133,6 +156,7 @@ struct cxl_mem_command { .size_out = sout, \ }, \ .opcode = CXL_MBOX_OP_##_id, \ + .flags = _flags, \ } /* @@ -142,10 +166,11 @@ struct cxl_mem_command { * 0, and the user passed in 1, it is an error. */ static struct cxl_mem_command mem_commands[] = { - CXL_CMD(IDENTIFY, 0, 0x43), + CXL_CMD(IDENTIFY, 0, 0x43, CXL_CMD_FLAG_FORCE_ENABLE), #ifdef CONFIG_CXL_MEM_RAW_COMMANDS - CXL_CMD(RAW, ~0, ~0), + CXL_CMD(RAW, ~0, ~0, 0), #endif + CXL_CMD(GET_SUPPORTED_LOGS, 0, ~0, CXL_CMD_FLAG_FORCE_ENABLE), }; /* @@ -633,6 +658,10 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm, c = &mem_commands[send_cmd->id]; info = &c->info; + /* Check that the command is enabled for hardware */ + if (!test_bit(info->id, cxlm->enabled_cmds)) + return -ENOTTY; + /* Check the input buffer is the expected size */ if (info->size_in >= 0 && info->size_in != send_cmd->in.size) return -ENOMEM; @@ -757,6 +786,17 @@ static const struct file_operations cxl_memdev_fops = { .llseek = noop_llseek, }; +static inline struct cxl_mem_command *cxl_mem_find_command(u16 opcode) +{ + struct cxl_mem_command *c; + + cxl_for_each_cmd(c) + if (c->opcode == opcode) + return c; + + return NULL; +} + /** * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device. * @cxlm: The CXL memory device to communicate with. @@ -777,9 +817,7 @@ static const struct file_operations cxl_memdev_fops = { * * Mailbox commands may execute successfully yet the device itself reported an * error. While this distinction can be useful for commands from userspace, the - * kernel will only be able to use results when both are successful. It's - * expected that all callers of this function know exactly the size of the data - * they will consume from the hardware. + * kernel will only be able to use results when both are successful. * * See __cxl_mem_mbox_send_cmd() */ @@ -787,6 +825,7 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode, void *in, size_t in_size, void *out, size_t out_size) { + const struct cxl_mem_command *cmd = cxl_mem_find_command(opcode); struct mbox_cmd mbox_cmd = { .opcode = opcode, .payload_in = in, @@ -812,7 +851,11 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode, if (mbox_cmd.return_code != CXL_MBOX_SUCCESS) return -ENXIO; - if (mbox_cmd.size_out != out_size) + /* + * Variable sized commands can't be validated and so it's up to the + * caller to do that if they wish. + */ + if (cmd->info.size_out >= 0 && mbox_cmd.size_out != out_size) return -EIO; return 0; @@ -947,6 +990,14 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, mutex_init(&cxlm->mbox_mutex); cxlm->pdev = pdev; cxlm->regs = regs + offset; + cxlm->enabled_cmds = + devm_kmalloc_array(dev, BITS_TO_LONGS(cxl_cmd_count), + sizeof(unsigned long), + GFP_KERNEL | __GFP_ZERO); + if (!cxlm->enabled_cmds) { + dev_err(dev, "No memory available for bitmap\n"); + return NULL; + } dev_dbg(dev, "Mapped CXL Memory Device resource\n"); return cxlm; @@ -1166,6 +1217,160 @@ err_ref: return rc; } +static int cxl_xfer_log(struct cxl_mem *cxlm, uuid_t *uuid, u32 size, u8 *out) +{ + u32 remaining = size; + u32 offset = 0; + + while (remaining) { + u32 xfer_size = min_t(u32, remaining, cxlm->payload_size); + struct cxl_mbox_get_log { + uuid_t uuid; + __le32 offset; + __le32 length; + } __packed log = { + .uuid = *uuid, + .offset = cpu_to_le32(offset), + .length = cpu_to_le32(xfer_size) + }; + int rc; + + rc = cxl_mem_mbox_send_cmd(cxlm, CXL_MBOX_OP_GET_LOG, &log, + sizeof(log), out, xfer_size); + if (rc < 0) + return rc; + + out += xfer_size; + remaining -= xfer_size; + offset += xfer_size; + } + + return 0; +} + +/** + * cxl_walk_cel() - Walk through the Command Effects Log. + * @cxlm: Device. + * @size: Length of the Command Effects Log. + * @cel: CEL + * + * Iterate over each entry in the CEL and determine if the driver supports the + * command. If so, the command is enabled for the device and can be used later. + */ +static void cxl_walk_cel(struct cxl_mem *cxlm, size_t size, u8 *cel) +{ + struct cel_entry { + __le16 opcode; + __le16 effect; + } __packed * cel_entry; + const int cel_entries = size / sizeof(*cel_entry); + int i; + + cel_entry = (struct cel_entry *)cel; + + for (i = 0; i < cel_entries; i++) { + u16 opcode = le16_to_cpu(cel_entry[i].opcode); + struct cxl_mem_command *cmd = cxl_mem_find_command(opcode); + + if (!cmd) { + dev_dbg(&cxlm->pdev->dev, + "Opcode 0x%04x unsupported by driver", opcode); + continue; + } + + set_bit(cmd->info.id, cxlm->enabled_cmds); + } +} + +struct cxl_mbox_get_supported_logs { + __le16 entries; + u8 rsvd[6]; + struct gsl_entry { + uuid_t uuid; + __le32 size; + } __packed entry[]; +} __packed; + +static struct cxl_mbox_get_supported_logs *cxl_get_gsl(struct cxl_mem *cxlm) +{ + struct cxl_mbox_get_supported_logs *ret; + int rc; + + ret = kvmalloc(cxlm->payload_size, GFP_KERNEL); + if (!ret) + return ERR_PTR(-ENOMEM); + + rc = cxl_mem_mbox_send_cmd(cxlm, CXL_MBOX_OP_GET_SUPPORTED_LOGS, NULL, + 0, ret, cxlm->payload_size); + if (rc < 0) { + kvfree(ret); + return ERR_PTR(rc); + } + + return ret; +} + +/** + * cxl_mem_enumerate_cmds() - Enumerate commands for a device. + * @cxlm: The device. + * + * Returns 0 if enumerate completed successfully. + * + * CXL devices have optional support for certain commands. This function will + * determine the set of supported commands for the hardware and update the + * enabled_cmds bitmap in the @cxlm. + */ +static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm) +{ + struct cxl_mbox_get_supported_logs *gsl; + struct device *dev = &cxlm->pdev->dev; + struct cxl_mem_command *cmd; + int i, rc; + + gsl = cxl_get_gsl(cxlm); + if (IS_ERR(gsl)) + return PTR_ERR(gsl); + + rc = -ENOENT; + for (i = 0; i < le16_to_cpu(gsl->entries); i++) { + u32 size = le32_to_cpu(gsl->entry[i].size); + uuid_t uuid = gsl->entry[i].uuid; + u8 *log; + + dev_dbg(dev, "Found LOG type %pU of size %d", &uuid, size); + + if (!uuid_equal(&uuid, &log_uuid[CEL_UUID])) + continue; + + log = kvmalloc(size, GFP_KERNEL); + if (!log) { + rc = -ENOMEM; + goto out; + } + + rc = cxl_xfer_log(cxlm, &uuid, size, log); + if (rc) { + kvfree(log); + goto out; + } + + cxl_walk_cel(cxlm, size, log); + kvfree(log); + + /* In case CEL was bogus, enable some default commands. */ + cxl_for_each_cmd(cmd) + if (cmd->flags & CXL_CMD_FLAG_FORCE_ENABLE) + set_bit(cmd->info.id, cxlm->enabled_cmds); + + /* Found the required CEL */ + rc = 0; + } + +out: + kvfree(gsl); + return rc; +} + /** * cxl_mem_identify() - Send the IDENTIFY command to the device. * @cxlm: The device to identify. @@ -1266,6 +1471,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + rc = cxl_mem_enumerate_cmds(cxlm); + if (rc) + return rc; + rc = cxl_mem_identify(cxlm); if (rc) return rc; diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index c316028730e7..59227f82a4c1 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -23,6 +23,7 @@ ___C(INVALID, "Invalid Command"), \ ___C(IDENTIFY, "Identify Command"), \ ___C(RAW, "Raw device command"), \ + ___C(GET_SUPPORTED_LOGS, "Get Supported Logs"), \ ___C(MAX, "invalid / last command") #define ___C(a, b) CXL_MEM_COMMAND_ID_##a From 57ee605b976c30a86613648935d255bbe704aeab Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Tue, 16 Feb 2021 20:09:56 -0800 Subject: [PATCH 58/91] cxl/mem: Add set of informational commands Add initial set of formal commands beyond basic identify and command enumeration. Signed-off-by: Ben Widawsky Reviewed-by: Dan Williams Reviewed-by: Jonathan Cameron (v2) Link: https://lore.kernel.org/r/20210217040958.1354670-8-ben.widawsky@intel.com Signed-off-by: Dan Williams --- drivers/cxl/mem.c | 9 +++++++++ include/uapi/linux/cxl_mem.h | 5 +++++ 2 files changed, 14 insertions(+) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index e31b3045e231..6d7d3870b5da 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -45,12 +45,16 @@ enum opcode { CXL_MBOX_OP_INVALID = 0x0000, CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID, + CXL_MBOX_OP_GET_FW_INFO = 0x0200, CXL_MBOX_OP_ACTIVATE_FW = 0x0202, CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400, CXL_MBOX_OP_GET_LOG = 0x0401, CXL_MBOX_OP_IDENTIFY = 0x4000, + CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, + CXL_MBOX_OP_GET_LSA = 0x4102, CXL_MBOX_OP_SET_LSA = 0x4103, + CXL_MBOX_OP_GET_HEALTH_INFO = 0x4200, CXL_MBOX_OP_SET_SHUTDOWN_STATE = 0x4204, CXL_MBOX_OP_SCAN_MEDIA = 0x4304, CXL_MBOX_OP_GET_SCAN_MEDIA = 0x4305, @@ -171,6 +175,11 @@ static struct cxl_mem_command mem_commands[] = { CXL_CMD(RAW, ~0, ~0, 0), #endif CXL_CMD(GET_SUPPORTED_LOGS, 0, ~0, CXL_CMD_FLAG_FORCE_ENABLE), + CXL_CMD(GET_FW_INFO, 0, 0x50, 0), + CXL_CMD(GET_PARTITION_INFO, 0, 0x20, 0), + CXL_CMD(GET_LSA, 0x8, ~0, 0), + CXL_CMD(GET_HEALTH_INFO, 0, 0x12, 0), + CXL_CMD(GET_LOG, 0x18, ~0, CXL_CMD_FLAG_FORCE_ENABLE), }; /* diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index 59227f82a4c1..3155382dfc9b 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -24,6 +24,11 @@ ___C(IDENTIFY, "Identify Command"), \ ___C(RAW, "Raw device command"), \ ___C(GET_SUPPORTED_LOGS, "Get Supported Logs"), \ + ___C(GET_FW_INFO, "Get FW Info"), \ + ___C(GET_PARTITION_INFO, "Get Partition Information"), \ + ___C(GET_LSA, "Get Label Storage Area"), \ + ___C(GET_HEALTH_INFO, "Get Health Info"), \ + ___C(GET_LOG, "Get Log"), \ ___C(MAX, "invalid / last command") #define ___C(a, b) CXL_MEM_COMMAND_ID_##a From 68a5a9a243354ed52f7b37b057bd5e98cba870c8 Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Tue, 16 Feb 2021 20:09:57 -0800 Subject: [PATCH 59/91] MAINTAINERS: Add maintainers of the CXL driver Signed-off-by: Ben Widawsky Cc: Dan Williams Cc: Vishal Verma Cc: Ira Weiny Cc: Alison Schofield Link: https://lore.kernel.org/r/20210217040958.1354670-9-ben.widawsky@intel.com Signed-off-by: Dan Williams --- MAINTAINERS | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index d3e847f7f3dc..f60e658a7c6e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4445,6 +4445,17 @@ M: Miguel Ojeda S: Maintained F: include/linux/compiler_attributes.h +COMPUTE EXPRESS LINK (CXL) +M: Alison Schofield +M: Vishal Verma +M: Ira Weiny +M: Ben Widawsky +M: Dan Williams +L: linux-cxl@vger.kernel.org +S: Maintained +F: drivers/cxl/ +F: include/uapi/linux/cxl_mem.h + CONEXANT ACCESSRUNNER USB DRIVER L: accessrunner-general@lists.sourceforge.net S: Orphan From 8befe0280de4b15832d37e45fef443b7afdd5a5f Mon Sep 17 00:00:00 2001 From: Tiezhu Yang Date: Wed, 3 Feb 2021 13:36:28 +0800 Subject: [PATCH 60/91] sparc: Replace test_ti_thread_flag() with test_tsk_thread_flag() Use test_tsk_thread_flag() directly instead of test_ti_thread_flag() to improve readability when the argument type is struct task_struct, it is similar with commit 5afc78551bf5 ("arm64: Use test_tsk_thread_flag() for checking TIF_SINGLESTEP"). Signed-off-by: Tiezhu Yang Signed-off-by: David S. Miller --- arch/sparc/kernel/process_32.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/kernel/process_32.c b/arch/sparc/kernel/process_32.c index 13cb5638fab8..33660c395d9d 100644 --- a/arch/sparc/kernel/process_32.c +++ b/arch/sparc/kernel/process_32.c @@ -184,7 +184,7 @@ void exit_thread(struct task_struct *tsk) #ifndef CONFIG_SMP if (last_task_used_math == tsk) { #else - if (test_ti_thread_flag(task_thread_info(tsk), TIF_USEDFPU)) { + if (test_tsk_thread_flag(tsk, TIF_USEDFPU)) { #endif /* Keep process from leaving FPU in a bogon state. */ put_psr(get_psr() | PSR_EF); From bda166930c37604ffa93f2425426af6921ec575a Mon Sep 17 00:00:00 2001 From: Andreas Larsson Date: Fri, 5 Feb 2021 14:20:31 +0100 Subject: [PATCH 61/91] sparc32: Limit memblock allocation to low memory Commit cca079ef8ac29a7c02192d2bad2ffe4c0c5ffdd0 changed sparc32 to use memblocks instead of bootmem, but also made high memory available via memblock allocation which does not work together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller --- arch/sparc/mm/init_32.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/sparc/mm/init_32.c b/arch/sparc/mm/init_32.c index eb2946b1df8a..6139c5700ccc 100644 --- a/arch/sparc/mm/init_32.c +++ b/arch/sparc/mm/init_32.c @@ -197,6 +197,9 @@ unsigned long __init bootmem_init(unsigned long *pages_avail) size = memblock_phys_mem_size() - memblock_reserved_size(); *pages_avail = (size >> PAGE_SHIFT) - high_pages; + /* Only allow low memory to be allocated via memblock allocation */ + memblock_set_current_limit(max_low_pfn << PAGE_SHIFT); + return max_pfn; } From c599a7821be7952033727a30e93bd2d053073c78 Mon Sep 17 00:00:00 2001 From: Andreas Larsson Date: Thu, 4 Feb 2021 17:23:41 +0100 Subject: [PATCH 62/91] sparc32: Preserve clone syscall flags argument for restarts due to signals This fixes a bug where a clone syscall that is restarted due to a pending signal is restarted with garbage in the register %o0 that holds the clone flags. This keep the original %i0 of a syscall (as seen from the trap handler) in %l6 rather than %l5. This is done because for clone (and also qfork) %l5 is used as a temporary variable in the same register window. Before this, that temporary value would be the value that was then incorrectly used as the orig_i0 argument to do_notify_resume. In order to preserve %l6, the temporary usage of %l6 in ret_sys_call is changed to use %l5 instead and the setting %l6 to 0 or 1 was removed. The use of that 0 or 1 value in %l6 was removed in commit 28e6103665301ce60634e8a77f0b657c6cc099de. Signed-off-by: Andreas Larsson Signed-off-by: David S. Miller --- arch/sparc/kernel/entry.S | 8 +++----- arch/sparc/kernel/rtrap_32.S | 2 +- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/sparc/kernel/entry.S b/arch/sparc/kernel/entry.S index f636acf3312f..c385c93b39bc 100644 --- a/arch/sparc/kernel/entry.S +++ b/arch/sparc/kernel/entry.S @@ -1009,7 +1009,7 @@ do_syscall: andcc %l5, _TIF_SYSCALL_TRACE, %g0 mov %i4, %o4 bne linux_syscall_trace - mov %i0, %l5 + mov %i0, %l6 2: call %l7 mov %i5, %o5 @@ -1018,16 +1018,15 @@ do_syscall: st %o0, [%sp + STACKFRAME_SZ + PT_I0] ret_sys_call: - ld [%curptr + TI_FLAGS], %l6 + ld [%curptr + TI_FLAGS], %l5 cmp %o0, -ERESTART_RESTARTBLOCK ld [%sp + STACKFRAME_SZ + PT_PSR], %g3 set PSR_C, %g2 bgeu 1f - andcc %l6, _TIF_SYSCALL_TRACE, %g0 + andcc %l5, _TIF_SYSCALL_TRACE, %g0 /* System call success, clear Carry condition code. */ andn %g3, %g2, %g3 - clr %l6 st %g3, [%sp + STACKFRAME_SZ + PT_PSR] bne linux_syscall_trace2 ld [%sp + STACKFRAME_SZ + PT_NPC], %l1 /* pc = npc */ @@ -1042,7 +1041,6 @@ ret_sys_call: sub %g0, %o0, %o0 or %g3, %g2, %g3 st %o0, [%sp + STACKFRAME_SZ + PT_I0] - mov 1, %l6 st %g3, [%sp + STACKFRAME_SZ + PT_PSR] bne linux_syscall_trace2 ld [%sp + STACKFRAME_SZ + PT_NPC], %l1 /* pc = npc */ diff --git a/arch/sparc/kernel/rtrap_32.S b/arch/sparc/kernel/rtrap_32.S index dca8ed810046..8931fe266346 100644 --- a/arch/sparc/kernel/rtrap_32.S +++ b/arch/sparc/kernel/rtrap_32.S @@ -75,7 +75,7 @@ signal_p: ld [%sp + STACKFRAME_SZ + PT_PSR], %t_psr mov %g2, %o2 - mov %l5, %o1 + mov %l6, %o1 call do_notify_resume add %sp, STACKFRAME_SZ, %o0 ! pt_regs ptr From 520615e1f5b2e617845238c650b58b43592fa923 Mon Sep 17 00:00:00 2001 From: Christoph Hellwig Date: Thu, 24 Sep 2020 06:15:23 +0200 Subject: [PATCH 63/91] sparc64: switch defconfig from the legacy ide driver to libata Replace the ide options with the equivalent libata options. This has been carried by various downstreams like the linux-build-test repo for years already. Signed-off-by: Christoph Hellwig Signed-off-by: David S. Miller --- arch/sparc/configs/sparc64_defconfig | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/sparc/configs/sparc64_defconfig b/arch/sparc/configs/sparc64_defconfig index bde4d21a8ac8..35a32adbbc59 100644 --- a/arch/sparc/configs/sparc64_defconfig +++ b/arch/sparc/configs/sparc64_defconfig @@ -66,9 +66,8 @@ CONFIG_CDROM_PKTCDVD=m CONFIG_CDROM_PKTCDVD_WCACHE=y CONFIG_ATA_OVER_ETH=m CONFIG_SUNVDC=m -CONFIG_IDE=y -CONFIG_BLK_DEV_IDECD=y -CONFIG_BLK_DEV_ALI15X3=y +CONFIG_ATA=y +CONFIG_PATA_ALI=y CONFIG_RAID_ATTRS=m CONFIG_SCSI=y CONFIG_BLK_DEV_SD=y From a57cdeb369ef73066937e06df5bcbb34624b13e8 Mon Sep 17 00:00:00 2001 From: Corentin Labbe Date: Mon, 18 Jan 2021 08:23:45 +0000 Subject: [PATCH 64/91] sparc: sparc64_defconfig: add necessary configs for qemu The sparc64 qemu machines uses sunhme network hardware by default, so for simple NFS boot testing using qemu, having CONFIG_HAPPYMEAL is useful. And so we need also IP_PNP_DHCP for NFS boot. For the same reason we need to enable its storage which is a PATA_CMD64. And finally, we need CONFIG_DEVTMPFS for handling recent udev/systemd. All those options will permit to enable boot testing in both kernelCI and gentoo's kernelCI. Signed-off-by: Corentin Labbe Signed-off-by: David S. Miller --- arch/sparc/configs/sparc64_defconfig | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/sparc/configs/sparc64_defconfig b/arch/sparc/configs/sparc64_defconfig index 35a32adbbc59..e2c0d5ecfef4 100644 --- a/arch/sparc/configs/sparc64_defconfig +++ b/arch/sparc/configs/sparc64_defconfig @@ -235,3 +235,9 @@ CONFIG_CRYPTO_TWOFISH=m CONFIG_CRC16=m CONFIG_LIBCRC32C=m CONFIG_VCC=m +CONFIG_ATA=y +CONFIG_PATA_CMD64X=y +CONFIG_HAPPYMEAL=y +CONFIG_IP_PNP=y +CONFIG_IP_PNP_DHCP=y +CONFIG_DEVTMPFS=y From da5447265434045410f579e4257dd64bf64de6c0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Thu, 14 Jan 2021 18:57:18 +0100 Subject: [PATCH 65/91] tty: vcc: Drop impossible to hit WARN_ON MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit vcc_get() returns the port that has provided port->index. As the port that is about to be removed isn't removed yet this trivially will find this port. So simplify the call to not assign an identical value to the port pointer and drop the warning that is never hit. Signed-off-by: Uwe Kleine-König Reviewed-by: Jiri Slaby Signed-off-by: David S. Miller --- drivers/tty/vcc.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/drivers/tty/vcc.c b/drivers/tty/vcc.c index 9ffd42e333b8..a4388a8d6bc7 100644 --- a/drivers/tty/vcc.c +++ b/drivers/tty/vcc.c @@ -695,12 +695,9 @@ static int vcc_remove(struct vio_dev *vdev) tty_vhangup(port->tty); /* Get exclusive reference to VCC, ensures that there are no other - * clients to this port + * clients to this port. This cannot fail. */ - port = vcc_get(port->index, true); - - if (WARN_ON(!port)) - return -ENODEV; + vcc_get(port->index, true); tty_unregister_device(vcc_tty_driver, port->index); From e1d830ab32808dcb63861b63d44d96768f9cf7d3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Thu, 14 Jan 2021 18:57:17 +0100 Subject: [PATCH 66/91] tty: vcc: Drop unnecessary if block MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit If vcc_probe() succeeded dev_set_drvdata() is called with a non-NULL value, and if vcc_probe() failed vcc_remove() isn't called. So there is no way dev_get_drvdata() can return NULL in vcc_remove() and the check can just go away. Signed-off-by: Uwe Kleine-König Reviewed-by: Jiri Slaby Signed-off-by: David S. Miller --- drivers/tty/vcc.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/tty/vcc.c b/drivers/tty/vcc.c index a4388a8d6bc7..e2d6205f83ce 100644 --- a/drivers/tty/vcc.c +++ b/drivers/tty/vcc.c @@ -681,9 +681,6 @@ static int vcc_remove(struct vio_dev *vdev) { struct vcc_port *port = dev_get_drvdata(&vdev->dev); - if (!port) - return -ENODEV; - del_timer_sync(&port->rx_timer); del_timer_sync(&port->tx_timer); From 60cb8a9073ba81490afe0e5b1e8e9394528de10b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= Date: Thu, 14 Jan 2021 18:57:16 +0100 Subject: [PATCH 67/91] tty: hvcs: Drop unnecessary if block MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit If hvcs_probe() succeeded dev_set_drvdata() is called with a non-NULL value, and if hvcs_probe() failed hvcs_remove() isn't called. So there is no way dev_get_drvdata() can return NULL in hvcs_remove() and the check can just go away. Signed-off-by: Uwe Kleine-König Reviewed-by: Jiri Slaby Reviewed-by: Tyrel Datwyler Signed-off-by: David S. Miller --- drivers/tty/hvc/hvcs.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/tty/hvc/hvcs.c b/drivers/tty/hvc/hvcs.c index 55105ac38f89..c6d1e6589edf 100644 --- a/drivers/tty/hvc/hvcs.c +++ b/drivers/tty/hvc/hvcs.c @@ -825,9 +825,6 @@ static int hvcs_remove(struct vio_dev *dev) unsigned long flags; struct tty_struct *tty; - if (!hvcsd) - return -ENODEV; - /* By this time the vty-server won't be getting any more interrupts */ spin_lock_irqsave(&hvcsd->lock, flags); From 80bddf5c93a99e11fc9faf7e4b575d01cecd45d3 Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Wed, 25 Nov 2020 16:40:11 -0800 Subject: [PATCH 68/91] sparc64: only select COMPAT_BINFMT_ELF if BINFMT_ELF is set Currently COMPAT on SPARC64 selects COMPAT_BINFMT_ELF unconditionally, even when BINFMT_ELF is not enabled. This causes a kconfig warning. Instead, just select COMPAT_BINFMT_ELF if BINFMT_ELF is enabled. This builds cleanly with no kconfig warnings. WARNING: unmet direct dependencies detected for COMPAT_BINFMT_ELF Depends on [n]: COMPAT [=y] && BINFMT_ELF [=n] Selected by [y]: - COMPAT [=y] && SPARC64 [=y] Fixes: 26b4c912185a ("sparc,sparc64: unify Kconfig files") Signed-off-by: Randy Dunlap Cc: "David S. Miller" Cc: sparclinux@vger.kernel.org Cc: Sam Ravnborg Signed-off-by: David S. Miller --- arch/sparc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 5f724e53ce9b..ff2b610830aa 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -510,7 +510,7 @@ config COMPAT bool depends on SPARC64 default y - select COMPAT_BINFMT_ELF + select COMPAT_BINFMT_ELF if BINFMT_ELF select HAVE_UID16 select ARCH_WANT_OLD_COMPAT_IPC select COMPAT_OLD_SIGACTION From a970a9764c773ae6daa94db934dfe3d790bfc977 Mon Sep 17 00:00:00 2001 From: "Matthew Wilcox (Oracle)" Date: Wed, 25 Nov 2020 03:46:54 +0000 Subject: [PATCH 69/91] sparc: Fix handling of page table constructor failure The page has just been allocated, so its refcount is 1. free_unref_page() is for use on pages which have a zero refcount. Use __free_page() like the other implementations of pte_alloc_one(). Fixes: 1ae9ae5f7df7 ("sparc: handle pgtable_page_ctor() fail") Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Reviewed-by: Mike Rapoport Acked-by: Vlastimil Babka Signed-off-by: David S. Miller --- arch/sparc/mm/init_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 02e6e5e0f106..81c963ba416e 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2899,7 +2899,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) if (!page) return NULL; if (!pgtable_pte_page_ctor(page)) { - free_unref_page(page); + __free_page(page); return NULL; } return (pte_t *) page_address(page); From b3554aa2470b5db1222c31e08ec9c29ab33eabc7 Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Mon, 16 Nov 2020 16:48:00 -0800 Subject: [PATCH 70/91] sparc: fix led.c driver when PROC_FS is not enabled Fix Sparc build when CONFIG_PROC_FS is not enabled. Fixes this build error: arch/sparc/kernel/led.c:107:30: error: 'led_proc_ops' defined but not used [-Werror=unused-const-variable=] 107 | static const struct proc_ops led_proc_ops = { | ^~~~~~~~~~~~ cc1: all warnings being treated as errors Fixes: 97a32539b956 ("proc: convert everything to "struct proc_ops"") Reported-by: kernel test robot Signed-off-by: Randy Dunlap Cc: Alexey Dobriyan Cc: Andrew Morton Cc: Lars Kotthoff Cc: "David S. Miller" Cc: sparclinux@vger.kernel.org Signed-off-by: David S. Miller --- arch/sparc/kernel/led.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/sparc/kernel/led.c b/arch/sparc/kernel/led.c index bd48575172c3..3a66e62eb2a0 100644 --- a/arch/sparc/kernel/led.c +++ b/arch/sparc/kernel/led.c @@ -50,6 +50,7 @@ static void led_blink(struct timer_list *unused) add_timer(&led_blink_timer); } +#ifdef CONFIG_PROC_FS static int led_proc_show(struct seq_file *m, void *v) { if (get_auxio() & AUXIO_LED) @@ -111,6 +112,7 @@ static const struct proc_ops led_proc_ops = { .proc_release = single_release, .proc_write = led_proc_write, }; +#endif static struct proc_dir_entry *led; From 76962e03934e1a77795852c1d64bd8491a00fb52 Mon Sep 17 00:00:00 2001 From: Kaixu Xia Date: Mon, 9 Nov 2020 16:06:56 +0800 Subject: [PATCH 71/91] sparc32: Fix comparing pointer to 0 coccicheck warning Fixes coccicheck warning: /arch/sparc/mm/srmmu.c:354:42-43: WARNING comparing pointer to 0 Avoid pointer type value compared to 0. Reported-by: Tosk Robot Signed-off-by: Kaixu Xia Signed-off-by: David S. Miller --- arch/sparc/mm/srmmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index da4e10b4cb37..6ceea577a39e 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -351,7 +351,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) pte_t *ptep; struct page *page; - if ((ptep = pte_alloc_one_kernel(mm)) == 0) + if (!(ptep = pte_alloc_one_kernel(mm))) return NULL; page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); spin_lock(&mm->page_table_lock); From 147d8622f2a26ef34beacc60e1ed8b66c2fa457f Mon Sep 17 00:00:00 2001 From: Khalid Aziz Date: Fri, 23 Oct 2020 11:56:11 -0600 Subject: [PATCH 72/91] sparc64: Use arch_validate_flags() to validate ADI flag When userspace calls mprotect() to enable ADI on an address range, do_mprotect_pkey() calls arch_validate_prot() to validate new protection flags. arch_validate_prot() for sparc looks at the first VMA associated with address range to verify if ADI can indeed be enabled on this address range. This has two issues - (1) Address range might cover multiple VMAs while arch_validate_prot() looks at only the first VMA, (2) arch_validate_prot() peeks at VMA without holding mmap lock which can result in race condition. arch_validate_flags() from commit c462ac288f2c ("mm: Introduce arch_validate_flags()") allows for VMA flags to be validated for all VMAs that cover the address range given by user while holding mmap lock. This patch updates sparc code to move the VMA check from arch_validate_prot() to arch_validate_flags() to fix above two issues. Suggested-by: Jann Horn Suggested-by: Christoph Hellwig Suggested-by: Catalin Marinas Signed-off-by: Khalid Aziz Reviewed-by: Catalin Marinas Signed-off-by: David S. Miller --- arch/sparc/include/asm/mman.h | 60 +++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 28 deletions(-) diff --git a/arch/sparc/include/asm/mman.h b/arch/sparc/include/asm/mman.h index f94532f25db1..274217e7ed70 100644 --- a/arch/sparc/include/asm/mman.h +++ b/arch/sparc/include/asm/mman.h @@ -57,36 +57,40 @@ static inline int sparc_validate_prot(unsigned long prot, unsigned long addr) { if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM | PROT_ADI)) return 0; - if (prot & PROT_ADI) { - if (!adi_capable()) - return 0; - - if (addr) { - struct vm_area_struct *vma; - - vma = find_vma(current->mm, addr); - if (vma) { - /* ADI can not be enabled on PFN - * mapped pages - */ - if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) - return 0; - - /* Mergeable pages can become unmergeable - * if ADI is enabled on them even if they - * have identical data on them. This can be - * because ADI enabled pages with identical - * data may still not have identical ADI - * tags on them. Disallow ADI on mergeable - * pages. - */ - if (vma->vm_flags & VM_MERGEABLE) - return 0; - } - } - } return 1; } + +#define arch_validate_flags(vm_flags) arch_validate_flags(vm_flags) +/* arch_validate_flags() - Ensure combination of flags is valid for a + * VMA. + */ +static inline bool arch_validate_flags(unsigned long vm_flags) +{ + /* If ADI is being enabled on this VMA, check for ADI + * capability on the platform and ensure VMA is suitable + * for ADI + */ + if (vm_flags & VM_SPARC_ADI) { + if (!adi_capable()) + return false; + + /* ADI can not be enabled on PFN mapped pages */ + if (vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) + return false; + + /* Mergeable pages can become unmergeable + * if ADI is enabled on them even if they + * have identical data on them. This can be + * because ADI enabled pages with identical + * data may still not have identical ADI + * tags on them. Disallow ADI on mergeable + * pages. + */ + if (vm_flags & VM_MERGEABLE) + return false; + } + return true; +} #endif /* CONFIG_SPARC64 */ #endif /* __ASSEMBLY__ */ From 356184fb6d67770d7788f316226be810a430937b Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Mon, 19 Oct 2020 22:33:28 -0700 Subject: [PATCH 73/91] sparc: make xchg() into a statement expression Fix several of the same warning by using a GCC statement expression (extension): ../arch/sparc/include/asm/cmpxchg_32.h:28:22: warning: value computed is not used [-Wunused-value] 28 | #define xchg(ptr,x) ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)))) Passes sparc32 allmodconfig with no xchg warnings. Signed-off-by: Randy Dunlap Cc: Arnd Bergmann Cc: "David S. Miller" Cc: sparclinux@vger.kernel.org Signed-off-by: David S. Miller --- arch/sparc/include/asm/cmpxchg_32.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sparc/include/asm/cmpxchg_32.h b/arch/sparc/include/asm/cmpxchg_32.h index c73b5a3ab7b9..a53d744d4212 100644 --- a/arch/sparc/include/asm/cmpxchg_32.h +++ b/arch/sparc/include/asm/cmpxchg_32.h @@ -25,7 +25,7 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, int return x; } -#define xchg(ptr,x) ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)))) +#define xchg(ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) /* Emulate cmpxchg() the same way we emulate atomics, * by hashing the object address and indexing into an array From c26958cb5a0d9053d1358258827638773f3d36ed Mon Sep 17 00:00:00 2001 From: Jann Horn Date: Fri, 19 Feb 2021 14:59:35 +0800 Subject: [PATCH 74/91] Take mmap lock in cacheflush syscall We need to take the mmap lock around find_vma() and subsequent use of the VMA. Otherwise, we can race with concurrent operations like munmap(), which can lead to use-after-free accesses to freed VMAs. Fixes: 1000197d8013 ("nios2: System calls handling") Signed-off-by: Jann Horn Signed-off-by: Ley Foon Tan --- arch/nios2/kernel/sys_nios2.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/nios2/kernel/sys_nios2.c b/arch/nios2/kernel/sys_nios2.c index cd390ec4f88b..b1ca85699952 100644 --- a/arch/nios2/kernel/sys_nios2.c +++ b/arch/nios2/kernel/sys_nios2.c @@ -22,6 +22,7 @@ asmlinkage int sys_cacheflush(unsigned long addr, unsigned long len, unsigned int op) { struct vm_area_struct *vma; + struct mm_struct *mm = current->mm; if (len == 0) return 0; @@ -34,16 +35,22 @@ asmlinkage int sys_cacheflush(unsigned long addr, unsigned long len, if (addr + len < addr) return -EFAULT; + if (mmap_read_lock_killable(mm)) + return -EINTR; + /* * Verify that the specified address region actually belongs * to this process. */ - vma = find_vma(current->mm, addr); - if (vma == NULL || addr < vma->vm_start || addr + len > vma->vm_end) + vma = find_vma(mm, addr); + if (vma == NULL || addr < vma->vm_start || addr + len > vma->vm_end) { + mmap_read_unlock(mm); return -EFAULT; + } flush_cache_range(vma, addr, addr + len); + mmap_read_unlock(mm); return 0; } From 9abcfcb20320e8f693e89d86573b58e6289931cb Mon Sep 17 00:00:00 2001 From: Andreas Oetken Date: Fri, 19 Feb 2021 14:41:03 +0800 Subject: [PATCH 75/91] nios2: fixed broken sys_clone syscall The tls pointer must be pushed on the stack prior to calling nios2_clone as it is the 5th function argument. Prior handling of the tls pointer was done inside former called function copy_thread_tls using the r8 register from the current_pt_regs directly. This was a bad design and resulted in the current bug introduced in 04bd52fb. Fixes: 04bd52fb ("nios2: enable HAVE_COPY_THREAD_TLS, switch to kernel_clone_args") Signed-off-by: Andreas Oetken Signed-off-by: Ley Foon Tan Acked-by: Christian Brauner --- arch/nios2/kernel/entry.S | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/nios2/kernel/entry.S b/arch/nios2/kernel/entry.S index da8442450e46..0794cd7803df 100644 --- a/arch/nios2/kernel/entry.S +++ b/arch/nios2/kernel/entry.S @@ -389,7 +389,10 @@ ENTRY(ret_from_interrupt) */ ENTRY(sys_clone) SAVE_SWITCH_STACK + subi sp, sp, 4 /* make space for tls pointer */ + stw r8, 0(sp) /* pass tls pointer (r8) via stack (5th argument) */ call nios2_clone + addi sp, sp, 4 RESTORE_SWITCH_STACK ret From 7f7bc20bc41a4fbcd2db75b375ac95e5faf958ae Mon Sep 17 00:00:00 2001 From: Andreas Oetken Date: Fri, 19 Feb 2021 14:41:50 +0800 Subject: [PATCH 76/91] nios2: Don't use _end for calculating min_low_pfn If there is a initramfs linked into the kernel which will be freed later on there is free memory somewhere between _etext and _end, thus using _end for min_low_pfn is not correct. This may lead to issues in dma_capable when checking 'min(addr, end) < phys_to_dma(dev, PFN_PHYS(min_low_pfn)))' as the address addr might be below min_low_pfn. Picked find_limits from architecture arm for applying min_low_pfn and max_low_pfn. Maybe using _etext for min_low_pfn would be fine too. Signed-off-by: Andreas Oetken Signed-off-by: Ley Foon Tan --- arch/nios2/kernel/setup.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/nios2/kernel/setup.c b/arch/nios2/kernel/setup.c index 3c6e3c813a0b..d2f21957e99c 100644 --- a/arch/nios2/kernel/setup.c +++ b/arch/nios2/kernel/setup.c @@ -32,8 +32,6 @@ EXPORT_SYMBOL(memory_start); unsigned long memory_end; EXPORT_SYMBOL(memory_end); -unsigned long memory_size; - static struct pt_regs fake_regs = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}; @@ -141,16 +139,22 @@ asmlinkage void __init nios2_boot_init(unsigned r4, unsigned r5, unsigned r6, parse_early_param(); } +static void __init find_limits(unsigned long *min, unsigned long *max_low, + unsigned long *max_high) +{ + *max_low = PFN_DOWN(memblock_get_current_limit()); + *min = PFN_UP(memblock_start_of_DRAM()); + *max_high = PFN_DOWN(memblock_end_of_DRAM()); +} + void __init setup_arch(char **cmdline_p) { int dram_start; console_verbose(); - dram_start = memblock_start_of_DRAM(); - memory_size = memblock_phys_mem_size(); - memory_start = PAGE_ALIGN((unsigned long)__pa(_end)); - memory_end = (unsigned long) CONFIG_NIOS2_MEM_BASE + memory_size; + memory_start = memblock_start_of_DRAM(); + memory_end = memblock_end_of_DRAM(); init_mm.start_code = (unsigned long) _stext; init_mm.end_code = (unsigned long) _etext; @@ -161,11 +165,10 @@ void __init setup_arch(char **cmdline_p) /* Keep a copy of command line */ *cmdline_p = boot_command_line; - min_low_pfn = PFN_UP(memory_start); - max_low_pfn = PFN_DOWN(memory_end); + find_limits(&min_low_pfn, &max_low_pfn, &max_pfn); max_mapnr = max_low_pfn; - memblock_reserve(dram_start, memory_start - dram_start); + memblock_reserve(__pa_symbol(_stext), _end - _stext); #ifdef CONFIG_BLK_DEV_INITRD if (initrd_start) { memblock_reserve(virt_to_phys((void *)initrd_start), From 58294927f468a428c7a1ba09cf986fdca14a2747 Mon Sep 17 00:00:00 2001 From: Dan Carpenter Date: Fri, 19 Feb 2021 12:54:38 +0300 Subject: [PATCH 77/91] cxl/mem: Return -EFAULT if copy_to_user() fails The copy_to_user() function returns the number of bytes remaining to be copied, but we want to return -EFAULT if the copy doesn't complete. Fixes: b754ffbbc0ee ("cxl/mem: Add basic IOCTL interface") Signed-off-by: Dan Carpenter Link: https://lore.kernel.org/r/YC+K3kgzqm20zCWY@mwanda Signed-off-by: Dan Williams --- drivers/cxl/mem.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 6d7d3870b5da..df895bcca63a 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -752,7 +752,10 @@ static int cxl_send_cmd(struct cxl_memdev *cxlmd, if (rc) return rc; - return copy_to_user(s, &send, sizeof(send)); + if (copy_to_user(s, &send, sizeof(send))) + return -EFAULT; + + return 0; } static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd, From 6b50df2b8c208a04d44b8df5b7baaf668ceb8fc3 Mon Sep 17 00:00:00 2001 From: Viresh Kumar Date: Mon, 22 Feb 2021 12:48:06 +0530 Subject: [PATCH 78/91] mailbox: arm_mhuv2: Skip calling kfree() with invalid pointer It is possible that 'data' passed to kfree() is set to a error value instead of allocated space. Make sure it doesn't get called with invalid pointer. Fixes: 5a6338cce9f4 ("mailbox: arm_mhuv2: Add driver") Cc: v5.11 # v5.11 Reported-by: kernel test robot Reported-by: Dan Carpenter Signed-off-by: Viresh Kumar Signed-off-by: Jassi Brar --- drivers/mailbox/arm_mhuv2.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/mailbox/arm_mhuv2.c b/drivers/mailbox/arm_mhuv2.c index cdfb1939fabf..d997f8ebfa98 100644 --- a/drivers/mailbox/arm_mhuv2.c +++ b/drivers/mailbox/arm_mhuv2.c @@ -699,7 +699,9 @@ static irqreturn_t mhuv2_receiver_interrupt(int irq, void *arg) ret = IRQ_HANDLED; } - kfree(data); + if (!IS_ERR(data)) + kfree(data); + return ret; } From 88ff5d466c0250259818f3153dbdc4af1f8615dd Mon Sep 17 00:00:00 2001 From: Ben Widawsky Date: Sat, 20 Feb 2021 19:58:46 -0800 Subject: [PATCH 79/91] cxl/mem: Fix potential memory leak When submitting a command for userspace, input and output payload bounce buffers are allocated. For a given command, both input and output buffers may exist and so when allocation of the input buffer fails, the output buffer must be freed too. As far as I can tell, userspace can't easily exploit the leak to OOM a machine unless the machine was already near OOM state. Fixes: 583fa5e71cae ("cxl/mem: Add basic IOCTL interface") Reported-by: Konrad Rzeszutek Wilk Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron Reviewed-by: Konrad Rzeszutek Wilk Link: https://lore.kernel.org/r/20210221035846.680145-1-ben.widawsky@intel.com Signed-off-by: Dan Williams --- drivers/cxl/mem.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index df895bcca63a..244cb7d89678 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -514,8 +514,10 @@ static int handle_mailbox_cmd_from_user(struct cxl_mem *cxlm, if (cmd->info.size_in) { mbox_cmd.payload_in = vmemdup_user(u64_to_user_ptr(in_payload), cmd->info.size_in); - if (IS_ERR(mbox_cmd.payload_in)) + if (IS_ERR(mbox_cmd.payload_in)) { + kvfree(mbox_cmd.payload_out); return PTR_ERR(mbox_cmd.payload_in); + } } rc = cxl_mem_mbox_get(cxlm); From 99d0021569c71c325f41a7dd0a08a380010ce95c Mon Sep 17 00:00:00 2001 From: Peter Zijlstra Date: Thu, 6 Aug 2020 15:14:09 -0700 Subject: [PATCH 80/91] objtool: Add a pass for generating __mcount_loc Add the --mcount option for generating __mcount_loc sections needed for dynamic ftrace. Using this pass requires the kernel to be compiled with -mfentry and CC_USING_NOP_MCOUNT to be defined in Makefile. Link: https://lore.kernel.org/lkml/20200625200235.GQ4781@hirez.programming.kicks-ass.net/ Signed-off-by: Peter Zijlstra [Sami: rebased, dropped config changes, fixed to actually use --mcount, and wrote a commit message.] Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- tools/objtool/builtin-check.c | 3 +- tools/objtool/builtin.h | 2 +- tools/objtool/check.c | 82 +++++++++++++++++++++++++++++++++++ tools/objtool/check.h | 1 + tools/objtool/objtool.c | 1 + tools/objtool/objtool.h | 1 + 6 files changed, 88 insertions(+), 2 deletions(-) diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c index c6d199bfd0ae..e92e76f69176 100644 --- a/tools/objtool/builtin-check.c +++ b/tools/objtool/builtin-check.c @@ -18,7 +18,7 @@ #include "builtin.h" #include "objtool.h" -bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux; +bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux, mcount; static const char * const check_usage[] = { "objtool check [] file.o", @@ -35,6 +35,7 @@ const struct option check_options[] = { OPT_BOOLEAN('s', "stats", &stats, "print statistics"), OPT_BOOLEAN('d', "duplicate", &validate_dup, "duplicate validation for vmlinux.o"), OPT_BOOLEAN('l', "vmlinux", &vmlinux, "vmlinux.o validation"), + OPT_BOOLEAN('M', "mcount", &mcount, "generate __mcount_loc"), OPT_END(), }; diff --git a/tools/objtool/builtin.h b/tools/objtool/builtin.h index 85c979caa367..94565a72b701 100644 --- a/tools/objtool/builtin.h +++ b/tools/objtool/builtin.h @@ -8,7 +8,7 @@ #include extern const struct option check_options[]; -extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux; +extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux, mcount; extern int cmd_check(int argc, const char **argv); extern int cmd_orc(int argc, const char **argv); diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 5f8d3eed78a1..44e3ee1425cd 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -530,6 +530,65 @@ static int create_static_call_sections(struct objtool_file *file) return 0; } +static int create_mcount_loc_sections(struct objtool_file *file) +{ + struct section *sec, *reloc_sec; + struct reloc *reloc; + unsigned long *loc; + struct instruction *insn; + int idx; + + sec = find_section_by_name(file->elf, "__mcount_loc"); + if (sec) { + INIT_LIST_HEAD(&file->mcount_loc_list); + WARN("file already has __mcount_loc section, skipping"); + return 0; + } + + if (list_empty(&file->mcount_loc_list)) + return 0; + + idx = 0; + list_for_each_entry(insn, &file->mcount_loc_list, mcount_loc_node) + idx++; + + sec = elf_create_section(file->elf, "__mcount_loc", 0, sizeof(unsigned long), idx); + if (!sec) + return -1; + + reloc_sec = elf_create_reloc_section(file->elf, sec, SHT_RELA); + if (!reloc_sec) + return -1; + + idx = 0; + list_for_each_entry(insn, &file->mcount_loc_list, mcount_loc_node) { + + loc = (unsigned long *)sec->data->d_buf + idx; + memset(loc, 0, sizeof(unsigned long)); + + reloc = malloc(sizeof(*reloc)); + if (!reloc) { + perror("malloc"); + return -1; + } + memset(reloc, 0, sizeof(*reloc)); + + reloc->sym = insn->sec->sym; + reloc->addend = insn->offset; + reloc->type = R_X86_64_64; + reloc->offset = idx * sizeof(unsigned long); + reloc->sec = reloc_sec; + elf_add_reloc(file->elf, reloc); + + idx++; + } + + if (elf_rebuild_reloc_section(file->elf, reloc_sec)) + return -1; + + return 0; +} + /* * Warnings shouldn't be reported for ignored functions. */ @@ -956,6 +1015,22 @@ static int add_call_destinations(struct objtool_file *file) insn->type = INSN_NOP; } + if (mcount && !strcmp(insn->call_dest->name, "__fentry__")) { + if (reloc) { + reloc->type = R_NONE; + elf_write_reloc(file->elf, reloc); + } + + elf_write_insn(file->elf, insn->sec, + insn->offset, insn->len, + arch_nop_insn(insn->len)); + + insn->type = INSN_NOP; + + list_add_tail(&insn->mcount_loc_node, + &file->mcount_loc_list); + } + /* * Whatever stack impact regular CALLs have, should be undone * by the RETURN of the called function. @@ -2927,6 +3002,13 @@ int check(struct objtool_file *file) goto out; warnings += ret; + if (mcount) { + ret = create_mcount_loc_sections(file); + if (ret < 0) + goto out; + warnings += ret; + } + out: if (ret < 0) { /* diff --git a/tools/objtool/check.h b/tools/objtool/check.h index 5ec00a4b891b..070baec2050a 100644 --- a/tools/objtool/check.h +++ b/tools/objtool/check.h @@ -23,6 +23,7 @@ struct instruction { struct list_head list; struct hlist_node hash; struct list_head static_call_node; + struct list_head mcount_loc_node; struct section *sec; unsigned long offset; unsigned int len; diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c index 9df0cd86d310..c1819a6f2a18 100644 --- a/tools/objtool/objtool.c +++ b/tools/objtool/objtool.c @@ -62,6 +62,7 @@ struct objtool_file *objtool_open_read(const char *_objname) INIT_LIST_HEAD(&file.insn_list); hash_init(file.insn_hash); INIT_LIST_HEAD(&file.static_call_list); + INIT_LIST_HEAD(&file.mcount_loc_list); file.c_file = !vmlinux && find_section_by_name(file.elf, ".comment"); file.ignore_unreachables = no_unreachable; file.hints = false; diff --git a/tools/objtool/objtool.h b/tools/objtool/objtool.h index 4125d4578b23..cf004dd60c2b 100644 --- a/tools/objtool/objtool.h +++ b/tools/objtool/objtool.h @@ -19,6 +19,7 @@ struct objtool_file { struct list_head insn_list; DECLARE_HASHTABLE(insn_hash, 20); struct list_head static_call_list; + struct list_head mcount_loc_list; bool ignore_unreachables, c_file, hints, rodata; }; From 18a14575ae31c5a97a5e87e961932a5016d369be Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Wed, 28 Oct 2020 10:16:26 -0700 Subject: [PATCH 81/91] objtool: Fix __mcount_loc generation with Clang's assembler When objtool generates relocations for the __mcount_loc section, it tries to reference __fentry__ calls by their section symbol offset. However, this fails with Clang's integrated assembler as it may not generate section symbols for every section. This patch looks up a function symbol instead if the section symbol is missing, similarly to commit e81e07244325 ("objtool: Support Clang non-section symbols in ORC generation"). Signed-off-by: Sami Tolvanen --- tools/objtool/check.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 44e3ee1425cd..ab87fb92793c 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -573,8 +573,21 @@ static int create_mcount_loc_sections(struct objtool_file *file) } memset(reloc, 0, sizeof(*reloc)); - reloc->sym = insn->sec->sym; - reloc->addend = insn->offset; + if (insn->sec->sym) { + reloc->sym = insn->sec->sym; + reloc->addend = insn->offset; + } else { + reloc->sym = find_symbol_containing(insn->sec, insn->offset); + + if (!reloc->sym) { + WARN("missing symbol for insn at offset 0x%lx\n", + insn->offset); + return -1; + } + + reloc->addend = insn->offset - reloc->sym->offset; + } + reloc->type = R_X86_64_64; reloc->offset = idx * sizeof(unsigned long); reloc->sec = reloc_sec; From 0e731dbc18241d68318e0a7d2c2c0087c9073fb9 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Fri, 17 Jul 2020 12:04:27 -0700 Subject: [PATCH 82/91] objtool: Don't autodetect vmlinux.o With LTO, we run objtool on vmlinux.o, but don't want noinstr validation. This change requires --vmlinux to be passed to objtool explicitly. Suggested-by: Peter Zijlstra Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- scripts/link-vmlinux.sh | 2 +- tools/objtool/builtin-check.c | 6 +----- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index c5919d5a0b4f..423a4106f0dd 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -106,7 +106,7 @@ objtool_link() local objtoolopt; if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then - objtoolopt="check" + objtoolopt="check --vmlinux" if [ -z "${CONFIG_FRAME_POINTER}" ]; then objtoolopt="${objtoolopt} --no-fp" fi diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c index e92e76f69176..facfc10bc5dc 100644 --- a/tools/objtool/builtin-check.c +++ b/tools/objtool/builtin-check.c @@ -41,7 +41,7 @@ const struct option check_options[] = { int cmd_check(int argc, const char **argv) { - const char *objname, *s; + const char *objname; struct objtool_file *file; int ret; @@ -52,10 +52,6 @@ int cmd_check(int argc, const char **argv) objname = argv[0]; - s = strstr(objname, "vmlinux.o"); - if (s && !s[9]) - vmlinux = true; - file = objtool_open_read(objname); if (!file) return 1; From 22c8542d7b220ffc53816d47b371b1fe34341d4a Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Fri, 25 Sep 2020 16:43:53 -0700 Subject: [PATCH 83/91] tracing: add support for objtool mcount This change adds build support for using objtool to generate __mcount_loc sections. Signed-off-by: Sami Tolvanen --- Makefile | 7 +++++++ kernel/trace/Kconfig | 13 +++++++++++++ scripts/Makefile.build | 3 +++ 3 files changed, 23 insertions(+) diff --git a/Makefile b/Makefile index 2233951666f7..3189eacf3185 100644 --- a/Makefile +++ b/Makefile @@ -860,6 +860,9 @@ ifdef CONFIG_FTRACE_MCOUNT_USE_CC endif endif endif +ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL + CC_FLAGS_USING += -DCC_USING_NOP_MCOUNT +endif ifdef CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT ifdef CONFIG_HAVE_C_RECORDMCOUNT BUILD_C_RECORDMCOUNT := y @@ -1236,6 +1239,10 @@ uapi-asm-generic: PHONY += prepare-objtool prepare-resolve_btfids prepare-objtool: $(objtool_target) ifeq ($(SKIP_STACK_VALIDATION),1) +ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL + @echo "error: Cannot generate __mcount_loc for CONFIG_DYNAMIC_FTRACE=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel" >&2 + @false +endif ifdef CONFIG_UNWINDER_ORC @echo "error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel" >&2 @false diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 3985a31a49b7..936ccab1b938 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -60,6 +60,11 @@ config HAVE_NOP_MCOUNT help Arch supports the gcc options -pg with -mrecord-mcount and -nop-mcount +config HAVE_OBJTOOL_MCOUNT + bool + help + Arch supports objtool --mcount + config HAVE_C_RECORDMCOUNT bool help @@ -612,10 +617,18 @@ config FTRACE_MCOUNT_USE_CC depends on !FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY depends on FTRACE_MCOUNT_RECORD +config FTRACE_MCOUNT_USE_OBJTOOL + def_bool y + depends on HAVE_OBJTOOL_MCOUNT + depends on !FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY + depends on !FTRACE_MCOUNT_USE_CC + depends on FTRACE_MCOUNT_RECORD + config FTRACE_MCOUNT_USE_RECORDMCOUNT def_bool y depends on !FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY depends on !FTRACE_MCOUNT_USE_CC + depends on !FTRACE_MCOUNT_USE_OBJTOOL depends on FTRACE_MCOUNT_RECORD config TRACING_MAP diff --git a/scripts/Makefile.build b/scripts/Makefile.build index d94fc9ab819d..59fabd9685ee 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -238,6 +238,9 @@ endif ifdef CONFIG_X86_SMAP objtool_args += --uaccess endif +ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL + objtool_args += --mcount +endif # 'OBJECT_FILES_NON_STANDARD := y': skip objtool checking for a directory # 'OBJECT_FILES_NON_STANDARD_foo.o := 'y': skip objtool checking for a file From 6dafca97803309c3cb5148d449bfa711e41ddef2 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Thu, 6 Aug 2020 15:15:02 -0700 Subject: [PATCH 84/91] x86, build: use objtool mcount Select HAVE_OBJTOOL_MCOUNT if STACK_VALIDATION is selected to use objtool to generate __mcount_loc sections for dynamic ftrace with Clang and gcc <5 (later versions of gcc use -mrecord-mcount). Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7b6dd10b162a..9cd9bd6562e3 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -167,6 +167,7 @@ config X86 select HAVE_CONTEXT_TRACKING if X86_64 select HAVE_CONTEXT_TRACKING_OFFSTACK if HAVE_CONTEXT_TRACKING select HAVE_C_RECORDMCOUNT + select HAVE_OBJTOOL_MCOUNT if STACK_VALIDATION select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE From 41425ebe20245c99b44d6ba0f017be9bfc28414f Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Wed, 30 Sep 2020 14:36:59 -0700 Subject: [PATCH 85/91] objtool: Split noinstr validation from --vmlinux This change adds a --noinstr flag to objtool to allow us to specify that we're processing vmlinux.o without also enabling noinstr validation. This is needed to avoid false positives with LTO when we run objtool on vmlinux.o without CONFIG_DEBUG_ENTRY. Signed-off-by: Sami Tolvanen --- scripts/link-vmlinux.sh | 2 +- tools/objtool/builtin-check.c | 3 ++- tools/objtool/builtin.h | 2 +- tools/objtool/check.c | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 423a4106f0dd..18aed761f9c0 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -106,7 +106,7 @@ objtool_link() local objtoolopt; if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then - objtoolopt="check --vmlinux" + objtoolopt="check --vmlinux --noinstr" if [ -z "${CONFIG_FRAME_POINTER}" ]; then objtoolopt="${objtoolopt} --no-fp" fi diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c index facfc10bc5dc..b84cdc72b51f 100644 --- a/tools/objtool/builtin-check.c +++ b/tools/objtool/builtin-check.c @@ -18,7 +18,7 @@ #include "builtin.h" #include "objtool.h" -bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux, mcount; +bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux, mcount, noinstr; static const char * const check_usage[] = { "objtool check [] file.o", @@ -34,6 +34,7 @@ const struct option check_options[] = { OPT_BOOLEAN('a', "uaccess", &uaccess, "enable uaccess checking"), OPT_BOOLEAN('s', "stats", &stats, "print statistics"), OPT_BOOLEAN('d', "duplicate", &validate_dup, "duplicate validation for vmlinux.o"), + OPT_BOOLEAN('n', "noinstr", &noinstr, "noinstr validation for vmlinux.o"), OPT_BOOLEAN('l', "vmlinux", &vmlinux, "vmlinux.o validation"), OPT_BOOLEAN('M', "mcount", &mcount, "generate __mcount_loc"), OPT_END(), diff --git a/tools/objtool/builtin.h b/tools/objtool/builtin.h index 94565a72b701..2502bb27de17 100644 --- a/tools/objtool/builtin.h +++ b/tools/objtool/builtin.h @@ -8,7 +8,7 @@ #include extern const struct option check_options[]; -extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux, mcount; +extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux, mcount, noinstr; extern int cmd_check(int argc, const char **argv); extern int cmd_orc(int argc, const char **argv); diff --git a/tools/objtool/check.c b/tools/objtool/check.c index ab87fb92793c..85993e606782 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -244,7 +244,7 @@ static void init_insn_state(struct insn_state *state, struct section *sec) * not correctly determine insn->call_dest->sec (external symbols do * not have a section). */ - if (vmlinux && sec) + if (vmlinux && noinstr && sec) state->noinstr = sec->noinstr; } From b1a1a1a09b4606d41723a426110382d2077c26fb Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Mon, 13 Apr 2020 16:10:13 -0700 Subject: [PATCH 86/91] kbuild: lto: postpone objtool With LTO, LLVM bitcode won't be compiled into native code until modpost_link, or modfinal for modules. This change postpones calls to objtool until after these steps, and moves objtool_args to Makefile.lib, so the arguments can be reused in Makefile.modfinal. As we didn't have objects to process earlier, we use --duplicate when processing vmlinux.o. This change also disables unreachable instruction warnings with LTO to avoid warnings about the int3 padding between functions. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- scripts/Makefile.build | 22 ++-------------------- scripts/Makefile.lib | 12 ++++++++++++ scripts/Makefile.modfinal | 19 ++++++++++++++++--- scripts/link-vmlinux.sh | 28 +++++++++++++++++++++++++--- 4 files changed, 55 insertions(+), 26 deletions(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 59fabd9685ee..3f6bf0ea7c0e 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -218,30 +218,11 @@ cmd_record_mcount = $(if $(findstring $(strip $(CC_FLAGS_FTRACE)),$(_c_flags)), endif # CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT ifdef CONFIG_STACK_VALIDATION +ifndef CONFIG_LTO_CLANG ifneq ($(SKIP_STACK_VALIDATION),1) __objtool_obj := $(objtree)/tools/objtool/objtool -objtool_args = $(if $(CONFIG_UNWINDER_ORC),orc generate,check) - -objtool_args += $(if $(part-of-module), --module,) - -ifndef CONFIG_FRAME_POINTER -objtool_args += --no-fp -endif -ifdef CONFIG_GCOV_KERNEL -objtool_args += --no-unreachable -endif -ifdef CONFIG_RETPOLINE - objtool_args += --retpoline -endif -ifdef CONFIG_X86_SMAP - objtool_args += --uaccess -endif -ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL - objtool_args += --mcount -endif - # 'OBJECT_FILES_NON_STANDARD := y': skip objtool checking for a directory # 'OBJECT_FILES_NON_STANDARD_foo.o := 'y': skip objtool checking for a file # 'OBJECT_FILES_NON_STANDARD_foo.o := 'n': override directory skip for a file @@ -253,6 +234,7 @@ objtool_obj = $(if $(patsubst y%,, \ $(__objtool_obj)) endif # SKIP_STACK_VALIDATION +endif # CONFIG_LTO_CLANG endif # CONFIG_STACK_VALIDATION # Rebuild all objects when objtool changes, or is enabled/disabled. diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 3ff3dbb3a830..6d004143c4d7 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -220,6 +220,18 @@ dtc_cpp_flags = -Wp,-MMD,$(depfile).pre.tmp -nostdinc \ $(addprefix -I,$(DTC_INCLUDE)) \ -undef -D__DTS__ +# Objtool arguments are also needed for modfinal with LTO, so we define +# then here to avoid duplication. +objtool_args = \ + $(if $(CONFIG_UNWINDER_ORC),orc generate,check) \ + $(if $(part-of-module), --module,) \ + $(if $(CONFIG_FRAME_POINTER),, --no-fp) \ + $(if $(or $(CONFIG_GCOV_KERNEL),$(CONFIG_LTO_CLANG)), \ + --no-unreachable,) \ + $(if $(CONFIG_RETPOLINE), --retpoline,) \ + $(if $(CONFIG_X86_SMAP), --uaccess,) \ + $(if $(CONFIG_FTRACE_MCOUNT_USE_OBJTOOL), --mcount,) + # Useful for describing the dependency of composite objects # Usage: # $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 6de2c35b64e8..735e11e9041b 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -9,7 +9,7 @@ __modfinal: include include/config/auto.conf include $(srctree)/scripts/Kbuild.include -# for c_flags +# for c_flags and objtool_args include $(srctree)/scripts/Makefile.lib # find all modules listed in modules.order @@ -34,10 +34,23 @@ ifdef CONFIG_LTO_CLANG # With CONFIG_LTO_CLANG, reuse the object file we compiled for modpost to # avoid a second slow LTO link prelink-ext := .lto -endif + +# ELF processing was skipped earlier because we didn't have native code, +# so let's now process the prelinked binary before we link the module. + +ifdef CONFIG_STACK_VALIDATION +ifneq ($(SKIP_STACK_VALIDATION),1) +cmd_ld_ko_o += \ + $(objtree)/tools/objtool/objtool $(objtool_args) \ + $(@:.ko=$(prelink-ext).o); + +endif # SKIP_STACK_VALIDATION +endif # CONFIG_STACK_VALIDATION + +endif # CONFIG_LTO_CLANG quiet_cmd_ld_ko_o = LD [M] $@ - cmd_ld_ko_o = \ + cmd_ld_ko_o += \ $(LD) -r $(KBUILD_LDFLAGS) \ $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) \ -T scripts/module.lds -o $@ $(filter %.o, $^); \ diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 18aed761f9c0..3b261b0f74f0 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -103,14 +103,36 @@ modpost_link() objtool_link() { + local objtoolcmd; local objtoolopt; + if [ "${CONFIG_LTO_CLANG} ${CONFIG_STACK_VALIDATION}" = "y y" ]; then + # Don't perform vmlinux validation unless explicitly requested, + # but run objtool on vmlinux.o now that we have an object file. + if [ -n "${CONFIG_UNWINDER_ORC}" ]; then + objtoolcmd="orc generate" + fi + + objtoolopt="${objtoolopt} --duplicate" + + if [ -n "${CONFIG_FTRACE_MCOUNT_USE_OBJTOOL}" ]; then + objtoolopt="${objtoolopt} --mcount" + fi + fi + if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then - objtoolopt="check --vmlinux --noinstr" + objtoolopt="${objtoolopt} --noinstr" + fi + + if [ -n "${objtoolopt}" ]; then + if [ -z "${objtoolcmd}" ]; then + objtoolcmd="check" + fi + objtoolopt="${objtoolopt} --vmlinux" if [ -z "${CONFIG_FRAME_POINTER}" ]; then objtoolopt="${objtoolopt} --no-fp" fi - if [ -n "${CONFIG_GCOV_KERNEL}" ]; then + if [ -n "${CONFIG_GCOV_KERNEL}" ] || [ -n "${CONFIG_LTO_CLANG}" ]; then objtoolopt="${objtoolopt} --no-unreachable" fi if [ -n "${CONFIG_RETPOLINE}" ]; then @@ -120,7 +142,7 @@ objtool_link() objtoolopt="${objtoolopt} --uaccess" fi info OBJTOOL ${1} - tools/objtool/objtool ${objtoolopt} ${1} + tools/objtool/objtool ${objtoolcmd} ${objtoolopt} ${1} fi } From e242db40be271793b248bf0ad1b7d7bd6359dc76 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Tue, 16 Apr 2019 14:40:00 -0700 Subject: [PATCH 87/91] x86, vdso: disable LTO only for vDSO Disable LTO for the vDSO. Note that while we could use Clang's LTO for the 64-bit vDSO, it won't add noticeable benefit for the small amount of C code. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/x86/entry/vdso/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile index 02e3e42f380b..05c4abc2fdfd 100644 --- a/arch/x86/entry/vdso/Makefile +++ b/arch/x86/entry/vdso/Makefile @@ -91,7 +91,7 @@ ifneq ($(RETPOLINE_VDSO_CFLAGS),) endif endif -$(vobjs): KBUILD_CFLAGS := $(filter-out $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL) +$(vobjs): KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO) $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL) # # vDSO code runs in userspace and -pg doesn't help with profiling anyway. @@ -150,6 +150,7 @@ KBUILD_CFLAGS_32 := $(filter-out -fno-pic,$(KBUILD_CFLAGS_32)) KBUILD_CFLAGS_32 := $(filter-out -mfentry,$(KBUILD_CFLAGS_32)) KBUILD_CFLAGS_32 := $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS_32)) KBUILD_CFLAGS_32 := $(filter-out $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS_32)) +KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS_32)) KBUILD_CFLAGS_32 += -m32 -msoft-float -mregparm=0 -fpic KBUILD_CFLAGS_32 += -fno-stack-protector KBUILD_CFLAGS_32 += $(call cc-option, -foptimize-sibling-calls) From d2dcd3e37475a66012f9702c4f9f0863ccdae716 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Wed, 16 Sep 2020 12:57:56 -0700 Subject: [PATCH 88/91] x86, cpu: disable LTO for cpu.c Clang incorrectly inlines functions with differing stack protector attributes, which breaks __restore_processor_state() that relies on stack protector being disabled. This change disables LTO for cpu.c to work aroung the bug. Link: https://bugs.llvm.org/show_bug.cgi?id=47479 Suggested-by: Nick Desaulniers Signed-off-by: Sami Tolvanen --- arch/x86/power/Makefile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/power/Makefile b/arch/x86/power/Makefile index 6907b523e856..5f711a441623 100644 --- a/arch/x86/power/Makefile +++ b/arch/x86/power/Makefile @@ -5,5 +5,9 @@ OBJECT_FILES_NON_STANDARD_hibernate_asm_$(BITS).o := y # itself be stack-protected CFLAGS_cpu.o := -fno-stack-protector +# Clang may incorrectly inline functions with stack protector enabled into +# __restore_processor_state(): https://bugs.llvm.org/show_bug.cgi?id=47479 +CFLAGS_REMOVE_cpu.o := $(CC_FLAGS_LTO) + obj-$(CONFIG_PM_SLEEP) += cpu.o obj-$(CONFIG_HIBERNATION) += hibernate_$(BITS).o hibernate_asm_$(BITS).o hibernate.o From b33fff07e3e3817d94dbec7bf2040070ecd96d16 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Wed, 10 Apr 2019 14:17:20 -0700 Subject: [PATCH 89/91] x86, build: allow LTO to be selected Pass code model and stack alignment to the linker as these are not stored in LLVM bitcode, and allow CONFIG_LTO_CLANG* to be enabled. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/x86/Kconfig | 2 ++ arch/x86/Makefile | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 9cd9bd6562e3..c15d16f1a971 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -95,6 +95,8 @@ config X86 select ARCH_SUPPORTS_DEBUG_PAGEALLOC select ARCH_SUPPORTS_NUMA_BALANCING if X86_64 select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096 + select ARCH_SUPPORTS_LTO_CLANG if X86_64 + select ARCH_SUPPORTS_LTO_CLANG_THIN if X86_64 select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_QUEUED_RWLOCKS select ARCH_USE_QUEUED_SPINLOCKS diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 7116da3980be..845baf58d147 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -166,6 +166,11 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif +ifdef CONFIG_LTO_CLANG +KBUILD_LDFLAGS += -plugin-opt=-code-model=kernel \ + -plugin-opt=-stack-alignment=$(if $(CONFIG_X86_32),4,8) +endif + # Workaround for a gcc prelease that unfortunately was shipped in a suse release KBUILD_CFLAGS += -Wno-sign-compare # From 5e95325fbbbdea96de91e7bdb05fe95a3031e37d Mon Sep 17 00:00:00 2001 From: Sami Tolvanen Date: Tue, 23 Feb 2021 13:59:52 -0800 Subject: [PATCH 90/91] kbuild: lto: force rebuilds when switching CONFIG_LTO When doing non-clean builds and switching between CONFIG_LTO=n and CONFIG_LTO=y, the build system (correctly) didn't notice that assembly and LTO-excluded C object files were rewritten in place by objtool (to add the .orc_unwind* sections), since their build command lines were the same between CONFIG_LTO=y and CONFIG_LTO=n. The objtool step would fail: vmlinux.o: warning: objtool: file already has .orc_unwind section, skipping make: *** [Makefile:1194: vmlinux] Error 255 Avoid this by making sure the build will see a difference between an LTO and non-LTO build (by including "-fno-lto" in KBUILD_*FLAGS). This will get ignored when CC_FLAGS_LTO is present, and will not be included at all when CONFIG_LTO=n. Signed-off-by: Sami Tolvanen Signed-off-by: Kees Cook --- Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 3189eacf3185..639873def1fd 100644 --- a/Makefile +++ b/Makefile @@ -910,7 +910,8 @@ KBUILD_LDFLAGS += -mllvm -import-instr-limit=5 endif ifdef CONFIG_LTO -KBUILD_CFLAGS += $(CC_FLAGS_LTO) +KBUILD_CFLAGS += -fno-lto $(CC_FLAGS_LTO) +KBUILD_AFLAGS += -fno-lto export CC_FLAGS_LTO endif From a555bdd0c58ce368fb1324f0e93d17f7c4babece Mon Sep 17 00:00:00 2001 From: Linus Torvalds Date: Wed, 24 Feb 2021 08:57:06 -0800 Subject: [PATCH 91/91] Kbuild: enable TRIM_UNUSED_KSYMS again, with some guarding In commit 5cf0fd591f2e ("Kbuild: disable TRIM_UNUSED_KSYMS option") I disabled this option because it's hugely expensive at build time, and I questioned how much use it gets. Several people piped up and convinced me it's actually useful, so instead of disabling it entirely, it now depends on EXPERT and gets disabled by COMPILE_TEST builds so that 'allmodconfig' style things don't enable it. I still hope somebody will take a look at the build time issue, because as Arnd also noted: "However, the combination of thinlto and trim indeed has a steep cost in compile time, taking almost twice as long as a normal defconfig (gc-sections makes it slightly faster)" Cc: Masahiro Yamada Cc: Arnd Bergmann Cc: Jessica Yu Cc: Cristoph Hellwig , Cc: Miroslav Benes Cc: Emil Velikov Signed-off-by: Linus Torvalds --- init/Kconfig | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/init/Kconfig b/init/Kconfig index ba8bd5256980..c2a18f56e944 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2273,8 +2273,8 @@ config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS If unsure, say N. config TRIM_UNUSED_KSYMS - bool "Trim unused exported kernel symbols" - depends on BROKEN + bool "Trim unused exported kernel symbols" if EXPERT + depends on !COMPILE_TEST help The kernel and some modules make many symbols available for other modules to use via EXPORT_SYMBOL() and variants. Depending