From 14e1028389b8641d034d0d6978cc1e7e0e58d54b Mon Sep 17 00:00:00 2001 From: John Stultz Date: Mon, 8 Aug 2022 16:56:00 +0000 Subject: [PATCH] ANDROID: sched: Fix off-by-one with cpupri MAX_RT_PRIO evaluation This patch addresses an issue seen where SCHED_FIFO prio 99 tasks were being woken up on a cpu where a long-running softirq was executing, and the RT task was not being migrated, causing long (10+ms wakeup latencies). Prior to upstream commit 934fc3314b39 ("sched/cpupri: Remap CPUPRI_NORMAL to MAX_RT_PRIO-1") the task->prio -> cpupri mapping is a little ackward. For RT tasks, its calculated as: cpupri = MAX_RT_PRIO - prio + 1; See: https://android.googlesource.com/kernel/common/+/refs/heads/android13-5.10/kernel/sched/cpupri.c#39 This is added ontop of the also ackward detail that the task->prio is inverted (RT prio99 -> 0), means the cpupri mapping for RT tasks goes from 2->101. This makes it easy to evaluate the cpupri incorrectly. Which it turns out happened In commit 3adfd8e344ac ("ANDROID: sched: avoid placing RT threads on cores handling softirqs"): https://android.googlesource.com/kernel/common/+/3adfd8e344ac32845eb3c15743f90a7e3b49dcf3%5E%21/ With the lines: int task_pri = convert_prio(p->prio); bool drop_nopreempts = task_pri <= MAX_RT_PRIO; Where the added logic to decide to migrate a rt task off of a cpu depended on this drop_nopreempts being true. This works properly for rt tasks from prio 99 to 1, but for the case of task->prio == 0 (userland rt prio 99 tasks) it breaks, as the cpupri will be MAX_RT_PRIO - 0 + 1, which then gets checked as <= MAX_RT_PRIO. This prevents the cpu from being dropped from the scheduling set and prevents the rt user prio 99 task from migrating, which results in high priority rt tasks being left on cpus where long running softirqs are executing, causing long latencies. This patch fixes the off by one by changing the evaulation to MAX_RT_PRIO + 1, so that user-prio 99 tasks will also be migrated if appropriate. Luckilly this odd cpupri mapping has been fixed upstream, making this patch no longer necessary in 5.15 and newer kernels. Fixes: 3adfd8e344ac ("ANDROID: sched: avoid placing RT threads on cores handling softirqs") Signed-off-by: John Stultz Change-Id: Ia2db7cd461eb4c90f5850b791de1ae95582f7530 --- kernel/sched/cpupri.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/cpupri.c b/kernel/sched/cpupri.c index 450a724b4542..2896860c8333 100644 --- a/kernel/sched/cpupri.c +++ b/kernel/sched/cpupri.c @@ -150,7 +150,7 @@ int cpupri_find_fitness(struct cpupri *cp, struct task_struct *p, { int task_pri = convert_prio(p->prio); int idx, cpu; - bool drop_nopreempts = task_pri <= MAX_RT_PRIO; + bool drop_nopreempts = task_pri <= MAX_RT_PRIO + 1; BUG_ON(task_pri >= CPUPRI_NR_PRIORITIES);