Commit Graph

38808 Commits

Author SHA1 Message Date
Ramji Jiyani
46ee6c3cdd ANDROID: GKI: Create symbol files in include/config
Create input symbol files to generate GKI modules header
under include/config. By placing files in this generated
directory, the default filters that ignore certain files
will work without any special handling required, and they
will also be available to inspect after the build to inspect
for the debugging purposes.

abi_gki_protected_exports: Input for gki_module_protected_exports.h
From :- ${objtree}/abi_gki_protected_exports
To :- include/config/abi_gki_protected_exports

all_kmi_symbols: Input for gki_module_unprotected.h
- Rename to abi_gki_kmi_symbols
From :- all_kmi_symbols
To :- include/config/abi_gki_kmi_symbols

Bug: 286529877
Test: TH
Test: Manual verification of the generated files
Change-Id: Iafa10631e7712a8e1e87a2f56cfd614de6b1053a
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
2023-08-01 21:21:47 +00:00
Ramji Jiyani
48fffa48b0 ANDROID: GKI: Move GKI module headers to generated includes
Change build time generated GKI module headers location
From :- kernel/module/gki_module_*.h
To :- include/generated/gki_module_*.h

This prevents the kernel source from being contaminated.
By placing the header files in a generated directory,
the default filters that ignore certain files will work
without any special handling required.

Bug: 286529877
Test: Manual verification & TH
Change-Id: Ie247d1c132ddae54906de2e2850e95d7ae9edd50
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
(cherry picked from commit e9cba885543fc50a5b59ff7234d02b74a380573c)
2023-07-24 17:57:22 +00:00
Zhang Zhengming
4d634bb7be UPSTREAM: relayfs: fix out-of-bounds access in relay_file_read
commit 43ec16f145 upstream.

There is a crash in relay_file_read, as the var from
point to the end of last subbuf.

The oops looks something like:
pc : __arch_copy_to_user+0x180/0x310
lr : relay_file_read+0x20c/0x2c8
Call trace:
 __arch_copy_to_user+0x180/0x310
 full_proxy_read+0x68/0x98
 vfs_read+0xb0/0x1d0
 ksys_read+0x6c/0xf0
 __arm64_sys_read+0x20/0x28
 el0_svc_common.constprop.3+0x84/0x108
 do_el0_svc+0x74/0x90
 el0_svc+0x1c/0x28
 el0_sync_handler+0x88/0xb0
 el0_sync+0x148/0x180

We get the condition by analyzing the vmcore:

1). The last produced byte and last consumed byte
    both at the end of the last subbuf

2). A softirq calls function(e.g __blk_add_trace)
    to write relay buffer occurs when an program is calling
    relay_file_read_avail().

        relay_file_read
                relay_file_read_avail
                        relay_file_read_consume(buf, 0, 0);
                        //interrupted by softirq who will write subbuf
                        ....
                        return 1;
                //read_start point to the end of the last subbuf
                read_start = relay_file_read_start_pos
                //avail is equal to subsize
                avail = relay_file_read_subbuf_avail
                //from  points to an invalid memory address
                from = buf->start + read_start
                //system is crashed
                copy_to_user(buffer, from, avail)

Bug: 288957094
Link: https://lkml.kernel.org/r/20230419040203.37676-1-zhang.zhengming@h3c.com
Fixes: 8d62fdebda ("relay file read: start-pos fix")
Signed-off-by: Zhang Zhengming <zhang.zhengming@h3c.com>
Reviewed-by: Zhao Lei <zhao_lei1@hoperun.com>
Reviewed-by: Zhou Kete <zhou.kete@h3c.com>
Reviewed-by: Pengcheng Yang <yangpc@wangsu.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit f6ee841ff2)
Signed-off-by: Lee Jones <joneslee@google.com>
Change-Id: Ibbdf65d8bf2268c3e8c09520f595167a2ed41e8b
2023-07-04 08:30:26 +01:00
Yifan Hong
d40f3254b6 FROMLIST: kheaders: dereferences the source tree
When the kernel is built inside a sandbox container,
a forest of symlinks to the source files may be
created in the container. In this case, the generated
kheaders.tar.xz should follow these symlinks
to access the source files, instead of packing
the symlinks themselves.

Test: manual (add kheaders_data.tar.xz to the output,
  then examine the contents)
Bug: 276339429
Fixes: b0acbba3f489 ("Revert "Revert "Revert "FROMLIST: kheaders: Follow symlinks to source files."""")
Link: https://lore.kernel.org/lkml/20230420010029.2702543-1-elsk@google.com/
Signed-off-by: Yifan Hong <elsk@google.com>
(cherry picked from https://android-review.googlesource.com/q/commit:28fa7afc424f3dc53358c0e9b080433d78f0cd54)
Merged-In: Ie4db22dfa13d05fdccb3ad8f4fae2fe3fead994e
Change-Id: Ie4db22dfa13d05fdccb3ad8f4fae2fe3fead994e
2023-06-23 09:24:00 +00:00
Qais Yousef
5af00d8531 ANDROID: Export cpu_push_stop
To enable handling RT tasks that are stuck on wrong CPU after changing
uclamp_min value.

Bug: 286099809
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: Ie223d34df6f21640e38b123d2dc3e674ee7c5e79
2023-06-22 21:29:55 +00:00
Dietmar Eggemann
78f4244dc9 BACKPORT: cgroup/cpuset: Free DL BW in case can_attach() fails
cpuset_can_attach() can fail. Postpone DL BW allocation until all tasks
have been checked. DL BW is not allocated per-task but as a sum over
all DL tasks migrating.

If multiple controllers are attached to the cgroup next to the cpuset
controller a non-cpuset can_attach() can fail. In this case free DL BW
in cpuset_cancel_attach().

Finally, update cpuset DL task count (nr_deadline_tasks) only in
cpuset_attach().

Bug: 238390134
Suggested-by: Waiman Long <longman@redhat.com>
Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
(cherry picked from commit 2ef269ef1a
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup/for-6.5)
[Conflict where we try to pull extra new code that is not applicable for
5.15, reject the new code]
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I54944463706cfb91cd441f003d54f78539d15316
2023-06-15 21:03:11 +00:00
Dietmar Eggemann
72c8100467 BACKPORT: sched/deadline: Create DL BW alloc, free & check overflow interface
While moving a set of tasks between exclusive cpusets,
cpuset_can_attach() -> task_can_attach() calls dl_cpu_busy(..., p) for
DL BW overflow checking and per-task DL BW allocation on the destination
root_domain for the DL tasks in this set.

This approach has the issue of not freeing already allocated DL BW in
the following error cases:

(1) The set of tasks includes multiple DL tasks and DL BW overflow
    checking fails for one of the subsequent DL tasks.

(2) Another controller next to the cpuset controller which is attached
    to the same cgroup fails in its can_attach().

To address this problem rework dl_cpu_busy():

(1) Split it into dl_bw_check_overflow() & dl_bw_alloc() and add a
    dedicated dl_bw_free().

(2) dl_bw_alloc() & dl_bw_free() take a `u64 dl_bw` parameter instead of
    a `struct task_struct *p` used in dl_cpu_busy(). This allows to
    allocate DL BW for a set of tasks too rather than only for a single
    task.

Bug: 238390134
Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
(cherry picked from commit 85989106fe
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup/for-6.5)
[Trivial conflict due to clash with GKI specific code added in the same
location in the file]
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I569877547ff7c6d20309f006382b7a416d656fb8
2023-06-15 21:03:11 +00:00
Juri Lelli
71cf567de1 FROMGIT: cgroup/cpuset: Iterate only if DEADLINE tasks are present
update_tasks_root_domain currently iterates over all tasks even if no
DEADLINE task is present on the cpuset/root domain for which bandwidth
accounting is being rebuilt. This has been reported to introduce 10+ ms
delays on suspend-resume operations.

Skip the costly iteration for cpusets that don't contain DEADLINE tasks.

Bug: 238390134
Reported-by: Qais Yousef <qyousef@layalina.io>
Link: https://lore.kernel.org/lkml/20230206221428.2125324-1-qyousef@layalina.io/
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
(cherry picked from commit c0f78fd5ed
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup/for-6.5)
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: Idc3f348d90b1024be2408fe8a855e09c354409c8
2023-06-15 21:03:11 +00:00
Juri Lelli
16a812c05e BACKPORT: sched/cpuset: Keep track of SCHED_DEADLINE task in cpusets
Qais reported that iterating over all tasks when rebuilding root domains
for finding out which ones are DEADLINE and need their bandwidth
correctly restored on such root domains can be a costly operation (10+
ms delays on suspend-resume).

To fix the problem keep track of the number of DEADLINE tasks belonging
to each cpuset and then use this information (followup patch) to only
perform the above iteration if DEADLINE tasks are actually present in
the cpuset for which a corresponding root domain is being rebuilt.

Bug: 238390134
Reported-by: Qais Yousef <qyousef@layalina.io>
Link: https://lore.kernel.org/lkml/20230206221428.2125324-1-qyousef@layalina.io/
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
(cherry picked from commit 6c24849f55
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup/for-6.5)
[Trivial conflicts in cpuset.c and dealdine.c. Reject new code/fields
that are in upstream but not in 5.15]
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I8b991e2b25d66adf73a91c656c1a22cc9dbeea7b
2023-06-15 21:03:11 +00:00
Juri Lelli
0dfe87dff2 BACKPORT: sched/cpuset: Bring back cpuset_mutex
Turns out percpu_cpuset_rwsem - commit 1243dc518c ("cgroup/cpuset:
Convert cpuset_mutex to percpu_rwsem") - wasn't such a brilliant idea,
as it has been reported to cause slowdowns in workloads that need to
change cpuset configuration frequently and it is also not implementing
priority inheritance (which causes troubles with realtime workloads).

Convert percpu_cpuset_rwsem back to regular cpuset_mutex. Also grab it
only for SCHED_DEADLINE tasks (other policies don't care about stable
cpusets anyway).

Bug: 238390134
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
(cherry picked from commit 111cd11bbc
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup/for-6.5)
[Conflicts in all the files. Mostly trivial. cpuset.c had new functions
added on mainline and we rejected lots of hunks related to this new
code. Removed ANDROID specific BUG check that uses cpuset_rwsem which
was a left over from a revert of CPU PAUSE feature that should no longer
exist on 5.15]
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: Ibc77724cdd16a98053b277a79f724bbeb49621de
2023-06-15 21:03:11 +00:00
Juri Lelli
2a32b2b9e8 FROMGIT: cgroup/cpuset: Rename functions dealing with DEADLINE accounting
rebuild_root_domains() and update_tasks_root_domain() have neutral
names, but actually deal with DEADLINE bandwidth accounting.

Rename them to use 'dl_' prefix so that intent is more clear.

No functional change.

Bug: 238390134
Suggested-by: Qais Yousef <qyousef@layalina.io>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
(cherry picked from commit ad3a557daf
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup/for-6.5)
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I6211b92c2c377ff44603966bce276550476d9bca
2023-06-15 21:03:11 +00:00
Matthias Maennich
faaabc7aa2 ANDROID: Remove all but top-level OWNERS
Now that the branch is used to create production GKI
images, need to institute ACK DrNo for all commits.

The DrNo approvers are in the android-mainline branch
at /OWNERS_DrNo.

Bug: 287162457
Signed-off-by: Matthias Maennich <maennich@google.com>
Change-Id: Id5bb83d7add5f314df6816c1c51b4bf2d8018e79
2023-06-15 09:54:56 +01:00
Peter Zijlstra
18bae38a20 UPSTREAM: sched: Fix DEBUG && !SCHEDSTATS warn
commit 769fdf83df upstream.

When !SCHEDSTATS schedstat_enabled() is an unconditional 0 and the
whole block doesn't exist, however GCC figures the scoped variable
'stats' is unused and complains about it.

Upgrade the warning from -Wunused-variable to -Wunused-but-set-variable
by writing it in two statements. This fixes the build because the new
warning is in W=1.

Given that whole if(0) {} thing, I don't feel motivated to change
things overly much and quite strongly feel this is the compiler being
daft.

Fixes: cb3e971c435d ("sched: Make struct sched_statistics independent of fair sched class")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Change-Id: I3b1f6cc605ae53a43f4a75a8d1a6cf2a947998ea
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 0a008c5098)
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-06-14 12:35:34 +00:00
Yafang Shao
8b4a04dce2 UPSTREAM: sched: Make struct sched_statistics independent of fair sched class
[ Upstream commit ceeadb83ae ]

If we want to use the schedstats facility to trace other sched classes, we
should make it independent of fair sched class. The struct sched_statistics
is the schedular statistics of a task_struct or a task_group. So we can
move it into struct task_struct and struct task_group to achieve the goal.

After the patch, schestats are orgnized as follows,

    struct task_struct {
       ...
       struct sched_entity se;
       struct sched_rt_entity rt;
       struct sched_dl_entity dl;
       ...
       struct sched_statistics stats;
       ...
   };

Regarding the task group, schedstats is only supported for fair group
sched, and a new struct sched_entity_stats is introduced, suggested by
Peter -

    struct sched_entity_stats {
        struct sched_entity     se;
        struct sched_statistics stats;
    } __no_randomize_layout;

Then with the se in a task_group, we can easily get the stats.

The sched_statistics members may be frequently modified when schedstats is
enabled, in order to avoid impacting on random data which may in the same
cacheline with them, the struct sched_statistics is defined as cacheline
aligned.

As this patch changes the core struct of scheduler, so I verified the
performance it may impact on the scheduler with 'perf bench sched
pipe', suggested by Mel. Below is the result, in which all the values
are in usecs/op.
                                  Before               After
      kernel.sched_schedstats=0  5.2~5.4               5.2~5.4
      kernel.sched_schedstats=1  5.3~5.5               5.3~5.5
[These data is a little difference with the earlier version, that is
 because my old test machine is destroyed so I have to use a new
 different test machine.]

Almost no impact on the sched performance.

No functional change.

[lkp@intel.com: reported build failure in earlier version]

Change-Id: I3df219ae37b431796057e380098afa7f6bb2bc63
Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Mel Gorman <mgorman@suse.de>
Link: https://lore.kernel.org/r/20210905143547.4668-3-laoar.shao@gmail.com
Stable-dep-of: 39afe5d6fc ("sched/fair: Fix inaccurate tally of ttwu_move_affine")
Signed-off-by: Sasha Levin <sashal@kernel.org>
(cherry picked from commit c3b9f95598)
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-06-14 12:35:34 +00:00
Kefeng Wang
4742f48a5a BACKPORT: arm64: Enable KCSAN
This patch enables KCSAN for arm64, with updates to build rules
to not use KCSAN for several incompatible compilation units.

Recent GCC version(at least GCC10) made outline-atomics as the
default option(unlike Clang), which will cause linker errors
for kernel/kcsan/core.o. Disables the out-of-line atomics by
no-outline-atomics to fix the linker errors.

Meanwhile, as Mark said[1], some latent issues are needed to be
fixed which isn't just a KCSAN problem, we make the KCSAN depends
on EXPERT for now.

Tested selftest and kcsan_test(built with GCC11 and Clang 13),
and all passed.

[1] https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N

Acked-by: Marco Elver <elver@google.com> # kernel/kcsan
Tested-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Link: https://lore.kernel.org/r/20211211131734.126874-1-wangkefeng.wang@huawei.com
[catalin.marinas@arm.com: added comment to justify EXPERT]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

Bug: 286152957
Change-Id: I369e13fdd1a86c5c5dca0e72c60eae5fb1595299
(cherry picked from commit dd03762ab6)
[joey: Resolve minor conflicts in arch/arm64/Kconfig,
arch/arm64/kvm/hyp/nvhe/Makefile,
kernel/kcsan/Makefile]
Signed-off-by: Joey Jiao <quic_jiangenj@quicinc.com>
2023-06-09 08:39:36 +00:00
Qais Yousef
858f6f1541 ANDROID: sched/fair: unlink misfit from cpu overutilized
Partially applies patch from Vincent Guittot

	e5ed0550c0: ("sched/fair: unlink misfit task from cpu overutilized")

The full patch improves the search logic under thermal pressure, but
it's a bit intrusive to fully backport. We are interested in the part
that unlinks misfit from overutilized only.

Bug: 283975667
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I2ae21499b846d13d3889f8cacdb70652817b066f
2023-06-06 19:51:56 +00:00
Stephen Dickey
7624f5afdf ANDROID: kernel/sched: rebuild_sched_domains export
Vendor module needs to rebuild sched domains at boot, in the
event that cpufreq initializes the energy model too late.

Bug: 242898038
Change-Id: Ifaf1223366ac81c3f3c382dd0f61110fce9c1b20
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
(cherry picked from commit 20ba031cf8)
2023-06-06 18:15:52 +00:00
Ard Biesheuvel
ee09e21c01 UPSTREAM: scs: add support for dynamic shadow call stacks
In order to allow arches to use code patching to conditionally emit the
shadow stack pushes and pops, rather than always taking the performance
hit even on CPUs that implement alternatives such as stack pointer
authentication on arm64, add a Kconfig symbol that can be set by the
arch to omit the SCS codegen itself, without otherwise affecting how
support code for SCS and compiler options (for register reservation, for
instance) are emitted.

Also, add a static key and some plumbing to omit the allocation of
shadow call stack for dynamic SCS configurations if SCS is disabled at
runtime.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Tested-by: Sami Tolvanen <samitolvanen@google.com>
Link: https://lore.kernel.org/r/20221027155908.1940624-3-ardb@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>

(cherry picked from commit 9beccca098)
Bug: 283954062
Change-Id: I71ed23533124b071bd6bf5ab91b2af3bbf03b42b
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
2023-05-25 16:09:30 -07:00
Qais Yousef
10bca2e53c ANDROID: Add new hook to enable overriding uclamp_validate()
We want to add more special values, specifically for uclamp_max so that
it can be set automatically to the most efficient value based on the
core it's running on.

Bug: 283761535
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I57343c4544f6cac621c855cbb94de0b8d80c51fa
2023-05-23 20:40:47 +00:00
heshuai1
ca57d52a21 ANDROID: user: Add vendor hook to user for GKI purpose
In order to implement our scheduling tuning policy in certain cases, we need to initialize the variables that we have defined in the user_struct. To achieve this, we will add a vendor hook to user.c at alloc_uid, which will ensure that our own logic is executed during the initialization of the user_struct.

Bug: 187458531
Bug: 283222331

Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: I078484aac2c3d396aba5971d6d0f491652f3781c
(cherry picked from commit c9b8fa644f)
(cherry picked from commit 9ac0923ef565e4de4e1f35edcba6fcb7e45948c9)
2023-05-18 13:30:37 +00:00
lijianzhong
dc3f95c564 ANDROID: export find_user() & free_uid()for GKI purpose.
Exporting the symbols find_user() & free_uid() to access user task information in ko module for monitoring and optimization purposes. This is a necessary component of our scheduling policy.

Bug: 183674818
Bug: 283222331

Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I12135c0af312904dd21b6f074beda086ad5ece98
(cherry picked from commit 16350016d8)
(cherry picked from commit eec2cd3df3aa2d92136658d3619dc5142155c7d4)
2023-05-18 13:30:37 +00:00
lijianzhong
555f3fe563 ANDROID: sched: Add vendor hooks for cpu affinity.
Add vendor hooks for CPU affinity to support OEM's tuning policy, where
we can block or unblock a certain type of application's CPU affinity.

Bug: 183674818
Bug: 283222331

Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I3402abec4d9faa08f564409bfb8db8d7902f3aa2
(cherry picked from commit 7cf9646c245fdc63e2a3c9fad457c11fabdd2dfc)
2023-05-18 13:30:37 +00:00
lijianzhong
53a5908faa ANDROID: cgroup: Add vendor hook for cpuset.
This hook allows us to capture information when a process is forked so
that we can stat and set some key task's CPU affinity in the ko module
later. This patch, along with aosp/2565450, is necessary for our
affinity settings.

Bug: 183674818
Bug: 283222331

Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: Ib93e05e5f6c338c5f7ada56bfebdd705f87f1f66
(cherry picked from commit a188361628461c58a4dfc72869d9acb1dfa2542f)
(cherry picked from commit 8947e06ff7)
2023-05-18 13:30:37 +00:00
lijianzhong
7d9c8f70d7 ANDROID: export cpuset_cpus_allowed()for GKI purpose.
Exporting the symbol cpuset_cpus_allowed() so that we can adjust a
certain type of application's CPU affinity in vendor hooks according
to our tuning policy.
Related commit: aosp/2565450

Bug: 189725786
Bug: 283222331

Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I7919a893ab64bb441ab43cbb0b16825ed76d802d
(cherry picked from commit 5a7d01ed73e4fc812fda1d7288086dc73a283405)
2023-05-18 13:30:37 +00:00
heshuai1
e47455c5dd ANDROID: freezer: Add vendor hook to freezer for GKI purpose.
Add the vendor hook to freezer.c so that OEM's logic can be executed
when the process is about to be frozen. We need to clear the flag for
some tasks and rebind task dependencies for optimization purposes.

Bug: 187458531
Bug: 283222331

Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: Iea42fd9604d6b33ccd6502425416f0dd28eecebb
(cherry picked from commit a1580311c3)
2023-05-18 13:30:37 +00:00
Zhuguangqing
4219a3afd4 ANDROID: freezer: export the freezer_cgrp_subsys for GKI purpose.
Exporting the symbol freezer_cgrp_subsys, in that vendor module can
add can_attach & cancel_attach member function. It is vendor-specific
tuning.

Bug: 182496370
Bug: 283222331

Signed-off-by: Zhuguangqing <zhuguangqing@xiaomi.com>
Change-Id: I153682b9d1015eed3f048b45ea6495ebb8f3c261
(cherry picked from commit 8a90e4d4e555dd5484213c6fec5061958016a194)
2023-05-18 13:30:37 +00:00
heshuai1
2d3899353a ANDROID: power: Add vendor hook to qos for GKI purpose.
Add vendor hooks in add/update/remove frequency QoS request process to
ensure that we can access the OEM's "frequency watchdog" logic for
abnormal frequency monitoring. This is necessary for our power tuning
policy.

Bug: 187458531
Bug: 283222331

Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: I1fb8fd6134432ecfb44ad242c66ccd8280ab9b43
(cherry picked from commit c445fe4dc67ad74dacfa548bc78876a7ce057086)
(cherry picked from commit 0ea0d6a7a2)
2023-05-18 13:30:37 +00:00
John Stultz
9bb6bf15fc FROMGIT: locking/rwsem: Add __always_inline annotation to __down_read_common() and inlined callers
Apparently despite it being marked inline, the compiler
may not inline __down_read_common() which makes it difficult
to identify the cause of lock contention, as the blocked
function in traceevents will always be listed as
__down_read_common().

So this patch adds __always_inline annotation to the common
function (as well as the inlined helper callers) to force it to
be inlined so the blocking function will be listed (via Wchan)
in traceevents.

Fixes: c995e638cc ("locking/rwsem: Fold __down_{read,write}*()")
Reported-by: Tim Murray <timmurray@google.com>
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Waiman Long <longman@redhat.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20230503023351.2832796-1-jstultz@google.com
Bug: 277817995
(cherry picked from commit 92cc5d00a4
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking/urgent)
Signed-off-by: John Stultz <jstultz@google.com>
(cherry picked from https://android-review.googlesource.com/q/commit:a6c75b2e64573cb9f49f6b89808207856fc0309b)
Merged-In: Ifad7ed7fe9f2d5a9eb0cfe7c35e45c0e86bc3ad4
Change-Id: Ifad7ed7fe9f2d5a9eb0cfe7c35e45c0e86bc3ad4
2023-05-12 16:47:54 +00:00
xieliujie
e5d032fb33 ANDROID: vendor_hooks: Add hooks for oem futex optimization
If an important task is going to sleep through do_futex(),
find out it's futex-owner by the pid comes from userspace,
and boost the owner by some means to shorten the sleep time.
How to boost? Depends on these hooks:
commit 53e8099784 ("ANDROID: vendor_hooks: Add hooks for scheduler")

Bug: 243110112
Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: I9a315cfb414fd34e0ef7a2cf9d57df50d4dd984f
(cherry picked from commit 548da5d23d)
2023-05-11 02:31:28 +00:00
Greg Kroah-Hartman
dad20f7a85 Revert "Revert "sched/fair: Fixes for capacity inversion detection""
This reverts commit 937164e07b8e56fa2e34bf05ee956ac4fe6a6b47.

It was perserving the ABI, but that is not needed anymore at this point
in time.

Change-Id: Ie7aaed3a99ce91258f7eae8683ea27b4d807b7b4
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-05-11 02:31:28 +00:00
Greg Kroah-Hartman
60e138c123 Revert "Revert "sched/uclamp: Fix a uninitialized variable warnings""
This reverts commit be0a0feb8e627f2f2f5324c65e5f91f921d4c1b8.

It was perserving the ABI, but that is not needed anymore at this point
in time.

Change-Id: I8913de3e6e83876a2b2ed4460c935c3adb2a0721
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-05-11 02:31:28 +00:00
Greg Kroah-Hartman
745bc5edc0 Revert "Revert "sched/fair: Consider capacity inversion in util_fits_cpu()""
This reverts commit 16ed1a1dc35911c07b5978781cc4dd54d86a73b1.

It was perserving the ABI, but that is not needed anymore at this point
in time.

Change-Id: If9b195f46c800a362857e91a349ed3aad4606030
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-05-11 02:31:27 +00:00
Greg Kroah-Hartman
96fa320d7c Revert "Revert "sched/fair: Detect capacity inversion""
This reverts commit 4c150ff0cc193b3b1fc6f0f68a4d79f9aac4f338.

It was perserving the ABI, but that is not needed anymore at this point
in time.

Change-Id: Ibae2a63d0d9287f4cbcae1e34b467bdf5c82576b
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-05-11 02:31:27 +00:00
Liujie Xie
456a8d4c1f ANDROID: vendor_hooks: Add hooks for rwsem and mutex
Add hooks to apply oem's optimization of rwsem and mutex

Bug: 182237112
Signed-off-by: xieliujie <xieliujie@oppo.com>
(cherry picked from commit 80b4341d05)

Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: I36895c432e5b6d6bff8781b4a7872badb693284c
Signed-off-by: Carlos Llamas <cmllamas@google.com>
[cmllamas: completes the cherry-pick of original commit 80b4341d05
since commit 0902cc73b793 was only partial]
(cherry picked from commit d4528a28cb5be0c322031f333a6230fa3042931f)
2023-05-06 00:25:54 +00:00
Liujie Xie
a945254842 ANDROID: vendor_hooks: Add hooks for mutex and rwsem optimistic spin
These hooks help us do the following things:
a) Record the number of mutex and rwsem optimistic spin.
b) Monitor the time of mutex and rwsem optimistic spin.
c) Make it possible if oems don't want mutex and rwsem to optimistic spin
for a long time.

Bug: 267565260
Change-Id: I2bee30fb17946be85e026213b481aeaeaee2459f
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
(cherry picked from commit d01f7e1269)
(cherry picked from commit 05b5ff11ad98c5896b352b4c376a84b63684e06c)
2023-05-06 00:25:54 +00:00
Rick Yiu
f9688670ca ANDROID: sched: Add trace_android_rvh_setscheduler
Sync to android13-5.10. This vendor hook is declared already.

Bug: 245675204
Change-Id: Ib081b52542380d22317f225a50b553cda5f2634c
Signed-off-by: Rick Yiu <rickyiu@google.com>
2023-05-05 20:45:38 +00:00
Peifeng Li
3568391d31 ANDROID: vendor_hook: add hooks to protect locking-tsk in cpu scheduler
Providing vendor hooks to record the start time of holding the lock, which
protects rwsem/mutex locking-process from being preemptedfor a short time
in some cases.

- android_vh_record_mutex_lock_starttime
- android_vh_record_rtmutex_lock_starttime
- android_vh_record_rwsem_lock_starttime
- android_vh_record_pcpu_rwsem_starttime

Bug: 241191475

Signed-off-by: Peifeng Li <lipeifeng@oppo.com>
Change-Id: I0e967a1e8b77c32a1ad588acd54028fae2f90c4e
(cherry picked from commit f7294947672eb6b786f3c16b49e71e6a239101ad)
2023-05-05 20:26:24 +00:00
Liujie Xie
e1f430a487 ANDROID: vendor_hooks: Add hooks to record the time of the process in various states
These hooks will do the following works:
a) record the time of the process in various states
b) Make corresponding optimization strategies in different hooks

Bug: 205938967

Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Change-Id: Ia3c47bbf0aadd17337ce18fd910343b1b8c3ef93
(cherry picked from commit a61d61bab7)
Signed-off-by: Carlos Llamas <cmllamas@google.com>
(cherry picked from commit b7a1174cc5dfc68cf769547b91958df407e55981)
2023-04-28 23:05:18 +00:00
xieliujie
5491cac560 ANDROID: sched: Add vendor hooks to compute new cpu freq.
add vendor hooks to compute new cpu freq for oem feature.

Bug: 280021175

Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: I2b8e1f76f3b9792148f153190b862face679ebbd
2023-04-28 23:02:59 +00:00
Liujie Xie
08d7a5f89e ANDROID: vendor_hooks: Add hooks for futex
We want to use this hook to record the sleeping time due to Futex

Bug: 210947226

Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Change-Id: I637f889dce42937116d10979e0c40fddf96cd1a2
(cherry picked from commit a7ab784f60)
2023-04-28 22:58:52 +00:00
Greg Kroah-Hartman
bad74575a1 Revert "sched/fair: Detect capacity inversion"
This reverts commit 99b704ae7a.

It breaks the current Android kernel abi.  It will be brought back at
the next KABI break update.

Bug: 161946584
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: Ie9a141414435aa17dc97ef1e1ec8c23dbe7816e8
2023-04-28 10:37:41 +00:00
Greg Kroah-Hartman
114ae28faa Revert "sched/fair: Consider capacity inversion in util_fits_cpu()"
This reverts commit 98762616db.

It breaks the current Android kernel abi.  It will be brought back at
the next KABI break update.

Bug: 161946584
Change-Id: I4ed9d6760b8d2e26bad66d9af39d7819e7b464d9
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-04-28 10:37:41 +00:00
Greg Kroah-Hartman
b06a054a03 Revert "sched/uclamp: Fix a uninitialized variable warnings"
This reverts commit 4ee882e0e1.

It breaks the current Android kernel abi.  It will be brought back at
the next KABI break update.

Bug: 161946584
Change-Id: I6f6c769ebcd31248b16f792add4d206c8c1b5c19
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-04-28 10:37:41 +00:00
Greg Kroah-Hartman
8ea8ecbd12 Revert "sched/fair: Fixes for capacity inversion detection"
This reverts commit e779884c71.

It breaks the current Android kernel abi.  It will be brought back at
the next KABI break update.

Bug: 161946584
Change-Id: Ic6ccde7b57e525a742602d783457c810a1ca0930
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-04-28 10:37:41 +00:00
Greg Kroah-Hartman
66858c87b1 Merge 5.15.109 into android14-5.15
Changes in 5.15.109
	ARM: dts: rockchip: fix a typo error for rk3288 spdif node
	arm64: dts: qcom: ipq8074-hk01: enable QMP device, not the PHY node
	arm64: dts: meson-g12-common: specify full DMC range
	arm64: dts: imx8mm-evk: correct pmic clock source
	netfilter: br_netfilter: fix recent physdev match breakage
	regulator: fan53555: Explicitly include bits header
	regulator: fan53555: Fix wrong TCS_SLEW_MASK
	net: sched: sch_qfq: prevent slab-out-of-bounds in qfq_activate_agg
	virtio_net: bugfix overflow inside xdp_linearize_page()
	sfc: Split STATE_READY in to STATE_NET_DOWN and STATE_NET_UP.
	sfc: Fix use-after-free due to selftest_work
	netfilter: nf_tables: fix ifdef to also consider nf_tables=m
	i40e: fix accessing vsi->active_filters without holding lock
	i40e: fix i40e_setup_misc_vector() error handling
	netfilter: nf_tables: validate catch-all set elements
	netfilter: nf_tables: tighten netlink attribute requirements for catch-all elements
	bnxt_en: Do not initialize PTP on older P3/P4 chips
	mlxfw: fix null-ptr-deref in mlxfw_mfa2_tlv_next()
	bonding: Fix memory leak when changing bond type to Ethernet
	net: rpl: fix rpl header size calculation
	mlxsw: pci: Fix possible crash during initialization
	spi: spi-rockchip: Fix missing unwind goto in rockchip_sfc_probe()
	bpf: Fix incorrect verifier pruning due to missing register precision taints
	e1000e: Disable TSO on i219-LM card to increase speed
	f2fs: Fix f2fs_truncate_partial_nodes ftrace event
	Input: i8042 - add quirk for Fujitsu Lifebook A574/H
	platform/x86 (gigabyte-wmi): Add support for A320M-S2H V2
	selftests: sigaltstack: fix -Wuninitialized
	scsi: megaraid_sas: Fix fw_crash_buffer_show()
	scsi: core: Improve scsi_vpd_inquiry() checks
	net: dsa: b53: mmap: add phy ops
	s390/ptrace: fix PTRACE_GET_LAST_BREAK error handling
	nvme-tcp: fix a possible UAF when failing to allocate an io queue
	xen/netback: use same error messages for same errors
	platform/x86: gigabyte-wmi: add support for X570S AORUS ELITE
	rtmutex: Add acquire semantics for rtmutex lock acquisition slow path
	iio: light: tsl2772: fix reading proximity-diodes from device tree
	nilfs2: initialize unused bytes in segment summary blocks
	memstick: fix memory leak if card device is never registered
	kernel/sys.c: fix and improve control flow in __sys_setres[ug]id()
	mmc: sdhci_am654: Set HIGH_SPEED_ENA for SDR12 and SDR25
	drm/i915: Fix fast wake AUX sync len
	mm/khugepaged: check again on anon uffd-wp during isolation
	mm: page_alloc: skip regions with hugetlbfs pages when allocating 1G pages
	sched/uclamp: Fix fits_capacity() check in feec()
	sched/uclamp: Make cpu_overutilized() use util_fits_cpu()
	sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition
	sched/fair: Detect capacity inversion
	sched/fair: Consider capacity inversion in util_fits_cpu()
	sched/uclamp: Fix a uninitialized variable warnings
	sched/fair: Fixes for capacity inversion detection
	MIPS: Define RUNTIME_DISCARD_EXIT in LD script
	docs: futex: Fix kernel-doc references after code split-up preparation
	purgatory: fix disabling debug info
	fuse: fix attr version comparison in fuse_read_update_size()
	fuse: always revalidate rename target dentry
	fuse: fix deadlock between atomic O_TRUNC and page invalidation
	udp: Call inet6_destroy_sock() in setsockopt(IPV6_ADDRFORM).
	tcp/udp: Call inet6_destroy_sock() in IPv6 sk->sk_destruct().
	inet6: Remove inet6_destroy_sock() in sk->sk_prot->destroy().
	dccp: Call inet6_destroy_sock() via sk->sk_destruct().
	sctp: Call inet6_destroy_sock() via sk->sk_destruct().
	pwm: meson: Explicitly set .polarity in .get_state()
	pwm: iqs620a: Explicitly set .polarity in .get_state()
	pwm: hibvt: Explicitly set .polarity in .get_state()
	counter: 104-quad-8: Fix race condition between FLAG and CNTR reads
	iio: adc: at91-sama5d2_adc: fix an error code in at91_adc_allocate_trigger()
	mm/page_alloc: fix potential deadlock on zonelist_update_seq seqlock
	ASoC: fsl_asrc_dma: fix potential null-ptr-deref
	ASN.1: Fix check for strdup() success
	soc: sifive: l2_cache: fix missing iounmap() in error path in sifive_l2_init()
	soc: sifive: l2_cache: fix missing free_irq() in error path in sifive_l2_init()
	soc: sifive: l2_cache: fix missing of_node_put() in sifive_l2_init()
	Linux 5.15.109

Change-Id: I2165f73cd4b1056ffb268e8b6a12e71588309188
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-04-26 13:37:41 +00:00
Qais Yousef
e779884c71 sched/fair: Fixes for capacity inversion detection
commit da07d2f9c1 upstream.

Traversing the Perf Domains requires rcu_read_lock() to be held and is
conditional on sched_energy_enabled(). Ensure right protections applied.

Also skip capacity inversion detection for our own pd; which was an
error.

Fixes: 44c7b80bff ("sched/fair: Detect capacity inversion")
Reported-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://lore.kernel.org/r/20230112122708.330667-3-qyousef@layalina.io
(cherry picked from commit da07d2f9c1)
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-04-26 13:51:53 +02:00
Qais Yousef
4ee882e0e1 sched/uclamp: Fix a uninitialized variable warnings
commit e26fd28db8 upstream.

Addresses the following warnings:

> config: riscv-randconfig-m031-20221111
> compiler: riscv64-linux-gcc (GCC) 12.1.0
>
> smatch warnings:
> kernel/sched/fair.c:7263 find_energy_efficient_cpu() error: uninitialized symbol 'util_min'.
> kernel/sched/fair.c:7263 find_energy_efficient_cpu() error: uninitialized symbol 'util_max'.

Fixes: 244226035a ("sched/uclamp: Fix fits_capacity() check in feec()")
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://lore.kernel.org/r/20230112122708.330667-2-qyousef@layalina.io
(cherry picked from commit e26fd28db8)
[Conflict in kernel/sched/fair.c due to new automatic variables being
added on master vs 5.15]
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-04-26 13:51:53 +02:00
Qais Yousef
98762616db sched/fair: Consider capacity inversion in util_fits_cpu()
commit aa69c36f31 upstream.

We do consider thermal pressure in util_fits_cpu() for uclamp_min only.
With the exception of the biggest cores which by definition are the max
performance point of the system and all tasks by definition should fit.

Even under thermal pressure, the capacity of the biggest CPU is the
highest in the system and should still fit every task. Except when it
reaches capacity inversion point, then this is no longer true.

We can handle this by using the inverted capacity as capacity_orig in
util_fits_cpu(). Which not only addresses the problem above, but also
ensure uclamp_max now considers the inverted capacity. Force fitting
a task when a CPU is in this adverse state will contribute to making the
thermal throttling last longer.

Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20220804143609.515789-10-qais.yousef@arm.com
(cherry picked from commit aa69c36f31)
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-04-26 13:51:53 +02:00
Qais Yousef
99b704ae7a sched/fair: Detect capacity inversion
commit 44c7b80bff upstream.

Check each performance domain to see if thermal pressure is causing its
capacity to be lower than another performance domain.

We assume that each performance domain has CPUs with the same
capacities, which is similar to an assumption made in energy_model.c

We also assume that thermal pressure impacts all CPUs in a performance
domain equally.

If there're multiple performance domains with the same capacity_orig, we
will trigger a capacity inversion if the domain is under thermal
pressure.

The new cpu_in_capacity_inversion() should help users to know when
information about capacity_orig are not reliable and can opt in to use
the inverted capacity as the 'actual' capacity_orig.

Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20220804143609.515789-9-qais.yousef@arm.com
(cherry picked from commit 44c7b80bff)
[fix trivial conflict in kernel/sched/sched.h due to code shuffling]
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-04-26 13:51:53 +02:00
Qais Yousef
1de6ee9d81 sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition
commit d81304bc61 upstream.

If the utilization of the woken up task is 0, we skip the energy
calculation because it has no impact.

But if the task is boosted (uclamp_min != 0) will have an impact on task
placement and frequency selection. Only skip if the util is truly
0 after applying uclamp values.

Change uclamp_task_cpu() signature to avoid unnecessary additional calls
to uclamp_eff_get(). feec() is the only user now.

Fixes: 732cd75b8c ("sched/fair: Select an energy-efficient CPU on task wake-up")
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20220804143609.515789-8-qais.yousef@arm.com
(cherry picked from commit d81304bc61)
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-04-26 13:51:52 +02:00