If a queue is run before all requeued requests have been sent to the I/O
scheduler, the I/O scheduler may dispatch the wrong request. Fix this by
making __blk_mq_run_hw_queue() process the requeue_list instead of
blk_mq_requeue_work().
Bug: 275581839
Change-Id: Id72c0787607f90857ae8e75998d532b96b46b8c2
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Requeue requests instead of sending these to the dispatch list if a CPU
is unplugged to prevent reordering of zoned writes.
Bug: 275581839
Change-Id: I2f03ba0832fc9609472c2bfa19d2cdd8abeffd8d
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Make sure that the I/O scheduler has control over which requests are
dispatched.
Bug: 275581839
Change-Id: If8c70df11584b023c452fbba28c67b092ddac850
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Send requests with the REQ_FUA or REQ_PREFLUSH flag set to the I/O
scheduler. Do not change the behavior for REQ_OP_FLUSH requests.
This patch prevents that zoned writes with the REQ_FUA and/or
REQ_PREFLUSH flag set are reordered against each other.
Bug: 275581839
Change-Id: Ib7f18932a066c40f5b76dc69c1864a080350251d
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Combine two return statements into a single return statement.
Bug: 275581839
Change-Id: I543e3dad88766157c0c334def4d525c10334e476
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Remove the 'hctx' argument because it is not used.
Bug: 275581839
Change-Id: I1f632931bc6c3d1ad89c4e6e93417c7301699559
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Set KMI_GENERATION=7 for 5/24 KMI update
type 'struct binder_transaction' changed
byte size changed from 176 to 192
member 'pid_t from_pid' was added
member 'pid_t from_tid' was added
13 members ('struct binder_transaction* from_parent' .. 'kuid_t sender_euid') changed
offset changed by 64
member 'ktime_t start_time' was added
4 members ('struct list_head fd_fixups' .. 'u64 android_vendor_data1') changed
offset changed by 128
Bug: 283322303
Change-Id: Ic9eaacc3707e86614dfb9fc6abde30ba722ed3bf
Signed-off-by: Carlos Llamas <cmllamas@google.com>
This commit adds support for getting the pid and tid information of
the sender for asynchronous transfers in binderfs transfer records.
In previous versions, it was not possible to obtain this information
from the transfer records. While this information may not be necessary
for all use cases, it can be useful in some scenarios.
Signed-off-by: Chuang Zhang <zhangchuang3@xiaomi.com>
Acked-by: Carlos Llamas <cmllamas@google.com>
Link: https://lore.kernel.org/r/0c1e8bd37c68dd1518bb737b06b768cde9659386.1682333709.git.zhangchuang3@xiaomi.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Bug: 235773151
(cherry picked from commit c21c0f9a20
git: //git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git
char-misc-next)
Change-Id: I7e729853353522164c4a3dd1094145dfd19af961
Signed-off-by: Carlos Llamas <cmllamas@google.com>
This reverts commit ad2bd67e07.
Qualcomm is reporting that internal debugging tools in addition to
Lauterbach Trace32 (T32) debugger (vLT_20230509_159359) are having
issues with DWARFv5.
Since the change to the DWARF debug info format was made after the
Android Feature Complete deadline, revert it from android14-* kernel
branches. We'll leave DWARFv5 for android-mainline and try again next
release in android15-*.
Bug: 281863640
Reported-by: Satya Durga Srinivasu Prabhala <quic_satyap@quicinc.com>
Change-Id: I5a1657bdf6785d23f3492dc12e6f3df0954b8deb
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Modern interactive systems, such as recent Android phones, tend to have
power efficient shallow idle states. Selecting deeper idle states on a
device while a latency-sensitive workload is running can adversely
impact performance due to increased latency. Additionally, if the CPU
wakes up from a deeper sleep before its target residency as is often the
case, it results in a waste of energy on top of that.
At the moment, none of the available idle governors take any scheduling
information into account. They also tend to overestimate the idle
duration quite often, which causes them to select excessively deep idle
states, thus leading to increased wakeup latency and lower performance
with no power saving. For 'menu' while web browsing on Android for
instance, those types of wakeups ('too deep') account for over 24% of
all wakeups.
At the same time, on some platforms idle state 0 can be power efficient
enough to warrant wanting to prefer it over idle state 1. This is
because the power usage of the two states can be so close that
sufficient amounts of too deep state 1 sleeps can completely offset the
state 1 power saving to the point where it would've been more power
efficient to just use state 0 instead. This is, of course, for systems
where state 0 is not a polling state, such as arm-based devices.
Sleeps that happened in state 0 while they could have used state 1 ('too
shallow') only save less power than they otherwise could have. Too deep
sleeps, on the other hand, harm performance and nullify the potential
power saving from using state 1 in the first place. While taking this
into account, it is clear that on balance it is preferable for an idle
governor to have more too shallow sleeps instead of more too deep sleeps
on those kinds of platforms.
This patch specifically tunes TEO to prefer shallower idle states in
order to reduce wakeup latency and achieve better performance.
To this end, before selecting the next idle state it uses the avg_util
signal of a CPU's runqueue in order to determine to what extent the CPU
is being utilized. This util value is then compared to a threshold
defined as a percentage of the CPU's capacity (capacity >> 6 ie. ~1.5%
in the current implementation). If the util is above the threshold, the
index of the idle state selected by TEO metrics will be reduced by 1,
thus selecting a shallower state. If the util is below the threshold,
the governor defaults to the TEO metrics mechanism to try to select the
deepest available idle state based on the closest timer event and its
own correctness.
The main goal of this is to reduce latency and increase performance for
some workloads. Under some workloads it will result in an increase in
power usage (Geekbench 5) while for other workloads it will also result
in a decrease in power usage compared to TEO (PCMark Web, Jankbench,
Speedometer).
It can provide drastically decreased latency and performance benefits in
certain types of workloads that are sensitive to latency.
Example test results:
1. GB5 (better score, latency & more power usage)
| metric | menu | teo | teo-util-aware |
| ------------------------------------- | -------------- | ----------------- | ----------------- |
| gmean score | 2826.5 (0.0%) | 2764.8 (-2.18%) | 2865 (1.36%) |
| gmean power usage [mW] | 2551.4 (0.0%) | 2606.8 (2.17%) | 2722.3 (6.7%) |
| gmean too deep % | 14.99% | 9.65% | 4.02% |
| gmean too shallow % | 2.5% | 5.96% | 14.59% |
| gmean task wakeup latency (asynctask) | 78.16μs (0.0%) | 61.60μs (-21.19%) | 54.45μs (-30.34%) |
2. Jankbench (better score, latency & less power usage)
| metric | menu | teo | teo-util-aware |
| ------------------------------------- | -------------- | ----------------- | ----------------- |
| gmean frame duration | 13.9 (0.0%) | 14.7 (6.0%) | 12.6 (-9.0%) |
| gmean jank percentage | 1.5 (0.0%) | 2.1 (36.99%) | 1.3 (-17.37%) |
| gmean power usage [mW] | 144.6 (0.0%) | 136.9 (-5.27%) | 121.3 (-16.08%) |
| gmean too deep % | 26.00% | 11.00% | 2.54% |
| gmean too shallow % | 4.74% | 11.89% | 21.93% |
| gmean wakeup latency (RenderThread) | 139.5μs (0.0%) | 116.5μs (-16.49%) | 91.11μs (-34.7%) |
| gmean wakeup latency (surfaceflinger) | 124.0μs (0.0%) | 151.9μs (22.47%) | 87.65μs (-29.33%) |
Bug: 258723112
Change-Id: Ifea46051fb4b0554349aa50145130e38efa5f163
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
[ rjw: Comment edits and white space adjustments ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit 9ce0f7c4bc)
[Build error due to different signature of sched_cpu_util()]
Signed-off-by: Qais Yousef <qyousef@google.com>
Add a no_poll flag to teo_find_shallower_state() that will let the
function optionally not consider polling states.
This allows the caller to guard against the function inadvertently
resulting in TEO putting the CPU in a polling state when that
behaviour is undesirable.
Bug: 258723112
Change-Id: Ic3879fd4428dd4c7b19caaee6f9fd847cdb58483
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit 27f8508801)
Signed-off-by: Qais Yousef <qyousef@google.com>
We want to add more special values, specifically for uclamp_max so that
it can be set automatically to the most efficient value based on the
core it's running on.
Bug: 283761535
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: I57343c4544f6cac621c855cbb94de0b8d80c51fa
Kernel IOMMU drivers can report system misconfiguration through
pkvm_iommu_finalize().
Then EL2 can take the proper response, for example if there are missing
IOMMUs, and DMA isolation can't be ensured, it would clear pvmfw so
PVMs can't be launched.
However, this is not clean as userspace can still query pvmfw info
and launch PVMs that loops in undefined instruction aborts as pvmfw
is cleared.
To fix this, before deprivilege, the kernel will erase pvmfw if the
IOMMUs are not finalised.
Bug: 268607700
Test: Launch PVM with missing S2MPU => Fail immediately with -8
Test: Launch PVM with all S2MPU => Pass
Change-Id: I9fd2440805f6b2f2ad4395ce61df5b272ed84fef
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Rename this function to align the function name with the function name
used in the upstream Linux kernel.
Bug: 275581839
Change-Id: I085ffacf535e65d8afbb9935a2cca26c4fec0d80
Signed-off-by: Bart Van Assche <bvanassche@google.com>
In bfq_pd_alloc(), the function bfqg_stats_init() init bfqg. If
blkg_rwstat_init() init bfqg_stats->bytes successful and init
bfqg_stats->ios failed, bfqg_stats_init() return failed, bfqg will
be freed. But blkg_rwstat->cpu_cnt is not deleted from the list of
percpu_counters. If we traverse the list of percpu_counters, It will
have UAF problem.
we should use blkg_rwstat_exit() to cleanup bfqg_stats bytes in the
above scenario.
Fixes: commit fd41e60331 ("bfq-iosched: stop using blkg->stat_bytes and ->stat_ios")
Change-Id: I133bae6a318a8ddc4f10a8ee8e63595a1219ea1e
Signed-off-by: Zheng Liang <zhengliang6@huawei.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/r/20211018024225.1493938-1-zhengliang6@huawei.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
(cherry picked from commit 2fc428f6b7)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Logging callback symbolic name is generating too many different
messages making Abort analysis miss big trends.
Stick to console reported message providing sufficient information.
Bug: 280678242
Signed-off-by: Thierry Strudel <tstrudel@google.com>
(cherry picked from https://android-review.googlesource.com/q/commit:9f8f2ea03ede21d8928793c4be87eeb76d81bc7f)
Merged-In: Ic0ea662a60919454060e3a085aeabd8a4099e0b4
Change-Id: Ic0ea662a60919454060e3a085aeabd8a4099e0b4
For devices with no cache it can make sense to use cache only mode as a
mechanism for trapping writes to hardware which is inaccessible but since
no cache is equivalent to cache bypass we force such devices into bypass
mode. This means that our check that bypass and cache only mode aren't both
enabled simultanously is less sensible for devices without a cache so relax
it.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20220622171723.1235749-1-broonie@kernel.org
Signed-off-by: Mark Brown <broonie@kernel.org>
Bug: 274730214
Change-Id: I85f9ba5343f28b24c2b0588e8a1487d31b654283
(cherry picked from commit 3d0afe9cf1)
Signed-off-by: Chunyan Zhang <chunyan.zhang@unisoc.com>
(cherry picked from commit b099f1eb0b330487fcc77e8a991919608fd07189)
Adding the following symbols:
- thermal_zone_get_zone_by_name
Bug: 269389888
Change-Id: I5f06af2b802f36369f2392320a7fe17e16fc406c
Signed-off-by: Chris Lu <luchris@google.com>
Add ANDROID_OEM_DATA_ARRAY to the struct user so that we can record the necessary data to implement the OEM's scheduling policy.
Bug: 183674818
Bug: 283222331
Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I418293391bc62a68207eae51a79bd45a7f6cd2ed
(cherry picked from commit 77e791866b)
In order to implement our scheduling tuning policy in certain cases, we need to initialize the variables that we have defined in the user_struct. To achieve this, we will add a vendor hook to user.c at alloc_uid, which will ensure that our own logic is executed during the initialization of the user_struct.
Bug: 187458531
Bug: 283222331
Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: I078484aac2c3d396aba5971d6d0f491652f3781c
(cherry picked from commit c9b8fa644f)
(cherry picked from commit 9ac0923ef565e4de4e1f35edcba6fcb7e45948c9)
Exporting the symbols find_user() & free_uid() to access user task information in ko module for monitoring and optimization purposes. This is a necessary component of our scheduling policy.
Bug: 183674818
Bug: 283222331
Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I12135c0af312904dd21b6f074beda086ad5ece98
(cherry picked from commit 16350016d8)
(cherry picked from commit eec2cd3df3aa2d92136658d3619dc5142155c7d4)
Add vendor hooks for CPU affinity to support OEM's tuning policy, where
we can block or unblock a certain type of application's CPU affinity.
Bug: 183674818
Bug: 283222331
Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I3402abec4d9faa08f564409bfb8db8d7902f3aa2
(cherry picked from commit 7cf9646c245fdc63e2a3c9fad457c11fabdd2dfc)
This hook allows us to capture information when a process is forked so
that we can stat and set some key task's CPU affinity in the ko module
later. This patch, along with aosp/2565450, is necessary for our
affinity settings.
Bug: 183674818
Bug: 283222331
Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: Ib93e05e5f6c338c5f7ada56bfebdd705f87f1f66
(cherry picked from commit a188361628461c58a4dfc72869d9acb1dfa2542f)
(cherry picked from commit 8947e06ff7)
Exporting the symbol cpuset_cpus_allowed() so that we can adjust a
certain type of application's CPU affinity in vendor hooks according
to our tuning policy.
Related commit: aosp/2565450
Bug: 189725786
Bug: 283222331
Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: I7919a893ab64bb441ab43cbb0b16825ed76d802d
(cherry picked from commit 5a7d01ed73e4fc812fda1d7288086dc73a283405)
Add some hooks in the binder module so that we can do task dependency analysis and statistical work in OEM's module for further optimization.
Bug: 235925535
Bug: 283222331
Signed-off-by: Zhuguangqing <zhuguangqing@xiaomi.com>
Change-Id: Id47e59c4e3ccd07b26eef758ada147b98cd1964e
Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
[ cmllamas: don't export complete private definition struct binder_alloc
in vendor hooks, instead just pass member alloc->free_async_space as
implemented by heshuai1 and squashed here ]
(cherry picked from commit 254fb1f4034fde523378ee58a501d212a59047b7)
Add the vendor hook to freezer.c so that OEM's logic can be executed
when the process is about to be frozen. We need to clear the flag for
some tasks and rebind task dependencies for optimization purposes.
Bug: 187458531
Bug: 283222331
Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: Iea42fd9604d6b33ccd6502425416f0dd28eecebb
(cherry picked from commit a1580311c3)
Exporting the symbol freezer_cgrp_subsys, in that vendor module can
add can_attach & cancel_attach member function. It is vendor-specific
tuning.
Bug: 182496370
Bug: 283222331
Signed-off-by: Zhuguangqing <zhuguangqing@xiaomi.com>
Change-Id: I153682b9d1015eed3f048b45ea6495ebb8f3c261
(cherry picked from commit 8a90e4d4e555dd5484213c6fec5061958016a194)
Add vendor hooks in add/update/remove frequency QoS request process to
ensure that we can access the OEM's "frequency watchdog" logic for
abnormal frequency monitoring. This is necessary for our power tuning
policy.
Bug: 187458531
Bug: 283222331
Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: I1fb8fd6134432ecfb44ad242c66ccd8280ab9b43
(cherry picked from commit c445fe4dc67ad74dacfa548bc78876a7ce057086)
(cherry picked from commit 0ea0d6a7a2)
Writing to HFGWTR_EL2 is undefined on CPUs without the capability,
so make sure not to touch the register when that is the case.
Bug: 282917063
Bug: 282993310
Change-Id: I85211105d6b1fbee854b4f13f037b8573c5a211c
Signed-off-by: Quentin Perret <qperret@google.com>
This reverts commit 6cf9fb4579. It was a
temporary workaround which will be replaced a proper fix shortly.
Bug: 282917063
Bug: 282993310
Change-Id: I0caec4a63aabaecbf3fa7ec5d5b4c6930ea6efcd
Signed-off-by: Quentin Perret <qperret@google.com>
Export the tracepoint task_rename to identify specific new task，
to customize task's util for power and performance, or optimize
task schedule parameters.
Bug: 189985971
Change-Id: I3bb71eae316e3096d361e7b47012ba46ea4be509
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
(cherry picked from commit 016d3f7b69)
(cherry picked from commit cfc14a391adb4d44d8186694b4884815bd85be6c)
[ Upstream commit 3aa1e96a2b ]
A BE connected to more than one FE, e.g. in a mixer case, can go
through the following transitions.
play FE1 -> BE state is START
pause FE1 -> BE state is PAUSED
play FE2 -> BE state is START
stop FE2 -> BE state is STOP (see note [1] below)
release FE1 -> BE state is START
stop FE1 -> BE state is STOP
play FE1 -> BE state is START
pause FE1 -> BE state is PAUSED
play FE2 -> BE state is START
release FE1 -> BE state is START
stop FE2 -> BE state is START
stop FE1 -> BE state is STOP
play FE1 -> BE state is START
play FE2 -> BE state is START (no change)
pause FE1 -> BE state is START (no change)
pause FE2 -> BE state is PAUSED
release FE1 -> BE state is START
release FE2 -> BE state is START (no change)
stop FE1 -> BE state is START (no change)
stop FE2 -> BE state is STOP
The existing code for PAUSE_RELEASE only allows for the case where the
BE is paused, which clearly would not work in the sequences above.
Extend the allowed states to restart the BE when PAUSE_RELEASE is
received, and increase the refcount if the BE is already in START.
[1] the existing logic does not move the BE state back to PAUSED when
the FE2 is stopped. This patch does not change the logic; it would be
painful to keep a history of changes on the FE side, the state machine
is already rather complicated with transitions based on the last BE
state and the trigger type.
Reported-by: Bard Liao <bard.liao@intel.com>
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Reviewed-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Reviewed-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Reviewed-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Link: https://lore.kernel.org/r/20211207173745.15850-7-pierre-louis.bossart@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
Fixes: aa9ff6a495 ("ASoC: soc-compress: Reposition and add pcm_mutex")
Signed-off-by: Yixuan Jiang <yixuanjiang@google.com>
Cc: stable@vger.kernel.org # 5.15+
Bug: 277241308
Link: https://lore.kernel.org/all/20230511120841.2096524-7-yixuanjiang@google.com
Change-Id: I2b5af9448df27082b75365c45357235d2887f33c
Signed-off-by: Yixuan Jiang <yixuanjiang@google.com>
