Doing a Exponential moving average per nr_running++/-- does not
guarantee a fixed sample rate which induces errors if there are lots of
threads being enqueued/dequeued from the rq (Linpack mt). Instead of
keeping track of the avg, the scheduler now keeps track of the integral
of nr_running and allows the readers to perform filtering on top.
Implemented a proper exponential moving average for the runnables
governor and a straight 100ms average for the balanced governor. Tweaked
the thresholds for the runnables governor to minimize latency. Also,
decreased sample_rate for the runnables governor to the absolute minimum
of 10msecs.
Signed-off-by: Huang, Tao <huangtao@rock-chips.com>
On device_busy, the runnable governor stops its sample timer and cancels
any queued runnable_work. However, it is possible for the cpuquiet
platform driver to call device_busy on cpuquiet_quiesence/wake_cpu calls
which can be invoked from the queued runnable_worker thread. This will
lead to the runnable_worker thread waiting infinitely on it self
finishing in cancel_work_sync.
Fix this by removing the cancel_work_sync call as it isn't necessary
because the worker doesn't requeue it self. Stopping the timer is
sufficient.
Bug 200002511
Change-Id: Ic69061b12d4dbb96b259ed9b62a0475271bd36d0
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
(cherry picked from commit b9a2a2cfa91d14935fdef626517d738399af4e4d)
Reviewed-on: http://git-master/r/415315
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Reviewed-by: Kerwin Wan <kerwinw@nvidia.com>
Added sysfs node that exposes the nr_runnable threshold at which cores
are quiesenced/woken up. To write into this node:
echo 1core# 2core# ... n_core# > nr_run_thresholds
Bug 1427140
Change-Id: I7cabd63692cf92f7fe8d1064437b1d73002d4ba5
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/369017
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Runnables govenror will now work properly for any number of cpus as
defined by NR_CPUS. Switched the ordering of nr_run_threshold init to
ensure that array is initialized correctly to UINT_MAX for NR_CPUS-1 cpu.
Bug 1427140
Change-Id: Ic612406ceaf80dd7e0b6693906df339f8b6512aa
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/369016
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Runnable sysfs node wasn't being free'd in stop via a kobject_put call
because the sysfs node wasn't allocated with kobject_create. Now do an
explicity kfree on stop after the kobject_put.
Change-Id: I5ba554312757380dfb2e814f75832b28d4c7edf8
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/346645
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
cpuquiet_switch_governors needs to be protected with the cpuquiet_lock.
Change-Id: I948d1190a270b9e8b732eaad80a303560fe9f282
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/345013
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Moved cpuquiet_lock to the back of cpuquiet_switch_governor().
cpuquiet_switch_governor() has to be protected by cpuquiet_lock.
In the runnable cpuquiet_governor case, it can occur a kernel panic or BUG_ON
because schedule_work() and INIT_WORK() can be performed at the same time.
Bug 1410816
Change-Id: I6d3d69bd8896e5425a762cbc4a58343a319edf29
Signed-off-by: Jubeom Kim <jubeomk@nvidia.com>
(cherry picked from commit 8502d0b7c057f96b7cd0495b49e37fc968d4bf8a)
Reviewed-on: http://git-master/r/337335
Reviewed-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Tested-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
- tegra_cpuquiet 'no_lp' node now accepts '-1' to force LP cluster on
- when a cluster is forced the switches take place immediately without
any delay
Bug 1247636
Change-Id: I1ec07acc3cf51419a4770542c6a1c90b92e0bf66
Signed-off-by: Simo Melenius <smelenius@nvidia.com>
Reviewed-on: http://git-master/r/208772
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
(cherry picked from commit ab16bbb6f581da538448d2a64e03dd272b8fb985)
Reviewed-on: http://git-master/r/210358
Userspace expects changes to happen synchronously. Implement this by waiting
with a (configureable) timeout for the action to happen.
Bug 1220065
Change-Id: I81301719707e4baf2b3aea62c38fc771ffe1205d
Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-on: http://git-master/r/200013
Reviewed-by: Simone Willett <swillett@nvidia.com>
Tested-by: Simone Willett <swillett@nvidia.com>
runnables_stop() can deadlock when cancel_work_sync() waits for the
work function to end and the work function blocks on the same lock
held by runnables_stop().
Removing the locks from runnables_work_func() fixes this. This should
be safe because runnables_lock protects runnables_state and
runnables_work_func() only reads runnables_state. Also, the functions
that change state to DISABLED do a cancel_work_sync() to guarantee the
work function stopped running.
Bug 1215668
Change-Id: I70617b3b0fc81db8555869e67e3b11652af8d94c
Signed-off-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-on: http://git-master/r/193881
Reviewed-by: Mandar Padmawar <mpadmawar@nvidia.com>
Tested-by: Mandar Padmawar <mpadmawar@nvidia.com>
Include cpuquiet in GCOV profiling when enabled by defconfig.
Change-Id: Iadac806d4b7efd9e34afa765dbec59f0491b96a8
Signed-off-by: Juha Tukkinen <jtukkinen@nvidia.com>
Reviewed-on: http://git-master/r/192980
Reviewed-by: Automatic_Commit_Validation_User
Fixed a race where the cpuquiet driver would call device_free() before
the governor was started, which would lead to a kernel panic because
runnables_timer was not initialized.
Introduced RUNNING state, so states are:
- DISABLED: truly disabled
- IDLE: enabled, but not polling upon request from the driver
- RUNNING: actively polling
Bug 1189042
Change-Id: I45b9ce40e61e1cfddde74ff7b2691722204045bb
Signed-off-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-on: http://git-master/r/168803
Reviewed-by: Sang-Hun Lee <sanlee@nvidia.com>
Tested-by: Sang-Hun Lee <sanlee@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Add the necessary code to initialize the runnable threads governor first when
selected as the default governor.
Change-Id: Iebbf304f3934b429d8a217011b3cb45ecddfd558
Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-on: http://git-master/r/131438
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Juha Tukkinen <jtukkinen@nvidia.com>
Rebase-Id: Rb5009930e554648a17d78edf9d8bdd17199c9c6a
The balanced governor relies on cpufreq for some of its decisions. Fail
initialization if there are no cpufreq OPP tables available.
Change-Id: I5d64215d56f50d5df52c1433208b1eb40440c812
Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-on: http://git-master/r/131437
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-by: Juha Tukkinen <jtukkinen@nvidia.com>
Rebase-Id: R03bb471218a088a816a67232861759cba2b91f38
Add options to enable/disable individual governors and adjust the Makefile
to only build the selected governors.
Change-Id: Ia3d36afd0f1d2e8cc622956fa0dc16c23cc9904d
Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-on: http://git-master/r/131436
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-by: Juha Tukkinen <jtukkinen@nvidia.com>
Rebase-Id: R0353ccd8d52227302fe7a890b9805b6c9170f58c
Doing a Exponential moving average per nr_running++/-- does not
guarantee a fixed sample rate which induces errors if there are lots of
threads being enqueued/dequeued from the rq (Linpack mt). Instead of
keeping track of the avg, the scheduler now keeps track of the integral
of nr_running and allows the readers to perform filtering on top.
Implemented a proper exponential moving average for the runnables
governor and a straight 100ms average for the balanced governor. Tweaked
the thresholds for the runnables governor to minimize latency. Also,
decreased sample_rate for the runnables governor to the absolute minimum
of 10msecs.
Updated to K3.4
Change-Id: Ia25bf8baf2a1a015ba188b2c06e551e89b16c5f8
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-on: http://git-master/r/131147
Reviewed-by: Juha Tukkinen <jtukkinen@nvidia.com>
The governor stops sampling whenever the device is busy and starts its
sampling loop when the device is free.
Change-Id: Ifdae4a84e56c2734be48e49f6112b537602a02da
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/124680
Reviewed-by: Varun Colbert <vcolbert@nvidia.com>
Tested-by: Varun Colbert <vcolbert@nvidia.com>
[perf] The runnable threads governor only looks at the average number of
runnables in the system to make a decision when bringing cores
offline/online. First pass; tweaks thresholds and delays to reduce
decision latency to about ~50-70ms per core (from ~100-150ms per core)
Change-Id: Idd3b268a74a8f56ad3fc0e5c7f388174d1b6611f
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/124679
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Enable cpuquiet by default if autohotplug is enabled. Cpuquiet will now
replace autohotplug as the hotplugging infrastructure. The down_delay in
the balanced governor has also been increased to 2s from 500ms to match
a similar patch for autohotplug.
Change-Id: I36747af67c55547b3dee95f7d435f9ebc075d393
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/123851
Reviewed-by: Varun Colbert <vcolbert@nvidia.com>
Tested-by: Varun Colbert <vcolbert@nvidia.com>
Added generic busy/free notifiers that the driver can invoke to let the
governor know that it cannot process further core online/offline
requests (invoked in our case whenever we switch to the LP cluster).
Change-Id: I5e3f7f28f38806a7f87050e8d0c8d2f2cf7521aa
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/114807
Reviewed-by: Lokesh Pathak <lpathak@nvidia.com>
Tested-by: Lokesh Pathak <lpathak@nvidia.com>
Now consider frequency greater than or equal to idle_top_freq as UP
Change-Id: I1332d46d1e42a00b3b31897b158eaf4ccfbaf8f5
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/113678
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Aleksandr Frid <afrid@nvidia.com>
Reviewed-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
GVS: Gerrit_Virtual_Submit
Rebase-Id: Rd67b2870e4916d1e616adf61f4bd4f3ef8916d1f
The balanced governor now looks at the average number of runnable
threads when bringing cores online and offline. The balanced governor
parameters have also been updated to reflect a similar patch for
autohotplug.
Change-Id: I8dac26659ba43d95a68830c6cc268591a7f03f80
Signed-off-by: Sai Charan Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/111282
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Aleksandr Frid <afrid@nvidia.com>
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Rebase-Id: Raeb86f51e0e21475ee9b96f97ce95f00883dac11
This is a port of the existing governor logic in cpu-tegra3.c.
Change-Id: Id79e6fc697dd0de85242fb2471bbed3d10101693
Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Reviewed-on: http://git-master/r/105271
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Tested-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Rebase-Id: Rf6addc1b20f47ccee2180380d021b54c41adff57
