From 4ea18cd059a4986a6a6f94a7f6d019b750bece65 Mon Sep 17 00:00:00 2001 From: Suren Baghdasaryan Date: Mon, 21 Nov 2022 14:23:00 -0800 Subject: [PATCH] ANDROID: mm: fix speculative walk which is unsafe under RCU Speculative page fault handling expects MMU_GATHER_RCU_TABLE_FREE to guarantee that page tables are stable, however tlb_remove_table() has a slow-path fall-back case when __get_free_page() returns NULL and tlb_remove_table_one() gets called. The way synchronization is implemented in that function is not RCU-safe and require IRQs to be disabled (see the comment in tlb_remove_table_sync_one()). Fix the invalid assumption to disable IRQs even when MMU_GATHER_RCU_TABLE_FREE=y. Bug: 257443051 Change-Id: I227f351607cf73022cb31f6f7a232cab41cf6a5a Signed-off-by: Suren Baghdasaryan --- mm/memory.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 317267711873..f18e08d114f7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2767,16 +2767,13 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range); * This is similar to what fast GUP does, but fast GUP also needs to * protect against races with THP page splitting, so it always needs * to disable interrupts. - * Speculative page faults only need to protect against page table reclamation, - * so rcu_read_lock() is sufficient in the MMU_GATHER_RCU_TABLE_FREE case. + * Speculative page faults need to protect against page table reclamation, + * even with MMU_GATHER_RCU_TABLE_FREE case page table removal slow-path is + * not RCU-safe (see comment inside tlb_remove_table_sync_one), therefore + * we still have to disable IRQs. */ -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE -#define speculative_page_walk_begin() rcu_read_lock() -#define speculative_page_walk_end() rcu_read_unlock() -#else #define speculative_page_walk_begin() local_irq_disable() #define speculative_page_walk_end() local_irq_enable() -#endif bool __pte_map_lock(struct vm_fault *vmf) {