This reverts commit cae118b6ac.
Upstream removed the support for CMDLINE_EXTEND on arm64 due to some
inconsistencies in the way command-line was assembled together. However,
the patch inadvertently deleted same config option from the android tree
which was first introduced by commit 056f12819b ("ANDROID: arm64: copy
CONFIG_CMDLINE_EXTEND from ARM") back in 2014.
Upstream is currently working on a solution to fix the divergence by
allowing prepend and append functionalities. While we wait for the new
series to land let's bring back CMDLINE_EXTEND option for android, so
arm64 can actually use of the functionality provided by d9a2a3f2c2
("ANDROID: of: Support CONFIG_CMDLINE_EXTEND config option").
Bug: 120440972
Bug: 261894961
Change-Id: I3b7fd55e95af49b64f284582dee40392074d32f1
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
The old logic assumes CMDLINE_FROM_BOOTLOADER vs. CMDLINE_FORCE and
ignores CMDLINE_EXTEND. Here's the old logic:
- CONFIG_CMDLINE_FORCE=true
CONFIG_CMDLINE
- dt bootargs=non-empty:
dt bootargs
- dt bootargs=empty, @data is non-empty string
@data is left unchanged
- dt bootargs=empty, @data is empty string
CONFIG_CMDLINE (or "" if that's not defined)
The new logic is now documented in of_fdt.h and is copied here for
reference:
- CONFIG_CMDLINE_FORCE=true
CONFIG_CMDLINE
- CONFIG_CMDLINE_EXTEND=true, @data is non-empty string
@data + dt bootargs (even if dt bootargs are empty)
- CONFIG_CMDLINE_EXTEND=true, @data is empty string
CONFIG_CMDLINE + dt bootargs (even if dt bootargs are empty)
- CMDLINE_FROM_BOOTLOADER=true, dt bootargs=non-empty:
dt bootargs
- CMDLINE_FROM_BOOTLOADER=true, dt bootargs=empty, @data is non-empty string
@data is left unchanged
- CMDLINE_FROM_BOOTLOADER=true, dt bootargs=empty, @data is empty string
CONFIG_CMDLINE (or "" if that's not defined)
Signed-off-by: Doug Anderson <dianders@chromium.org>
CC: devicetree-discuss@lists.ozlabs.org
CC: Grant Likely <grant.likely@secretlab.ca>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Rob Herring <rob.herring@calxeda.com>
Bug: 120440972
Bug: 261894961
Change-Id: I40ace250847f813358125dfcaa8998fd32cf7ea3
Signed-off-by: Colin Cross <ccross@android.com>
[AmitP: Folded following android-4.9 commit changes into this patch
e820270abb ("ANDROID: of: fix CONFIG_CMDLINE_EXTEND")
9a4a740554 ("ANDROID: of: Fix build warnings")]
Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
[eberman: Resolve conflicts due to
commit 60f20d84dc ("of/fdt: Rework early_init_dt_scan_chosen() to call directly")
commit 7a12dd077e ("of: move from strlcpy with unused retval to strscpy")]
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
Due to compiler optimizations like inlining, there are cases where
MMIO traces using _THIS_IP_ for caller information might not be
sufficient to provide accurate debug traces.
1) With optimizations (Seen with GCC):
In this case, _THIS_IP_ works fine and prints the caller information
since it will be inlined into the caller and we get the debug traces
on who made the MMIO access, for ex:
rwmmio_read: qcom_smmu_tlb_sync+0xe0/0x1b0 width=32 addr=0xffff8000087447f4
rwmmio_post_read: qcom_smmu_tlb_sync+0xe0/0x1b0 width=32 val=0x0 addr=0xffff8000087447f4
2) Without optimizations (Seen with Clang):
_THIS_IP_ will not be sufficient in this case as it will print only
the MMIO accessors itself which is of not much use since it is not
inlined as below for example:
rwmmio_read: readl+0x4/0x80 width=32 addr=0xffff8000087447f4
rwmmio_post_read: readl+0x48/0x80 width=32 val=0x4 addr=0xffff8000087447f4
So in order to handle this second case as well irrespective of the compiler
optimizations, add _RET_IP_ to MMIO trace to make it provide more accurate
debug information in all these scenarios.
Before:
rwmmio_read: readl+0x4/0x80 width=32 addr=0xffff8000087447f4
rwmmio_post_read: readl+0x48/0x80 width=32 val=0x4 addr=0xffff8000087447f4
After:
rwmmio_read: qcom_smmu_tlb_sync+0xe0/0x1b0 -> readl+0x4/0x80 width=32 addr=0xffff8000087447f4
rwmmio_post_read: qcom_smmu_tlb_sync+0xe0/0x1b0 -> readl+0x4/0x80 width=32 val=0x0 addr=0xffff8000087447f4
Fixes: 210031971c ("asm-generic/io: Add logging support for MMIO accessors")
Signed-off-by: Sai Prakash Ranjan <quic_saipraka@quicinc.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Bug: 262467838
Change-Id: I04e50dfbc59574a06ed5def8027fc4f48f422b1d
(cherry picked from commit 5e5ff73c2ehttps://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic.git master)
Signed-off-by: Naman Jain <quic_namajain@quicinc.com>
Currently firmware_class.patch commandline can take a single path for
loading firmwares on custom paths. SoC vendors and oems can have
firmwares in multiple file system paths. So add support for paassing
multiple paths through command line for firmware loader.
Add a getter function to read the class path.
For example - firmware_class.path="/vendor,/vendor/firmware_mnt,
/oem/firmware". firmware_class.path can take upto 10 file system
paths with ',' separation.
Bug: 259298396
Change-Id: I31d1470d7dd0255c7aefd856f3c129bdb4b7f2e8
Signed-off-by: Prasad Sodagudi <quic_psodagud@quicinc.com>
Signed-off-by: Vamsi Krishna Lanka <quic_vamslank@quicinc.com>
This config helps address audio buffer underrun issues on arm64 targets.
Bug: 168521633
Signed-off-by: J. Avila <elavila@google.com>
[jstultz: Rebased to android-mainline, also enabled on x86]
Signed-off-by: John Stultz <jstultz@google.com>
Change-Id: Iffb39b2c1d55f5d88d1475f68d5ed5a3bba90a2b
Tasklets are supposed to finish their work quickly and
should not block the current running process, but it is not
guaranteed that. Currently softirq_entry/exit can be used to
know total tasklets execution time, but not helpful to track
individual tasklet's execution time. With that we can't find
any culprit tasklet function, which is taking more time.
Add tasklet_entry/exit trace point support to track
individual tasklet execution.
This patch has been carried in the Android tree for awhile
so I wanted to submit it for review upstream. Feedback would
be appreciated!
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Connor O'Brien <connoro@google.com>
Cc: kernel-team@android.com
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
[elavila: Port to android-mainline]
Signed-off-by: J. Avila <elavila@google.com>
[jstultz: Rebased to upstream, cut unused trace points, added
comments for the tracepoints, reworded commit]
Signed-off-by: John Stultz <jstultz@google.com>
Link: https://lore.kernel.org/lkml/20221213185310.1315794-1-jstultz@google.com/
Bug: 168521633
Change-Id: I3944fcedffae54a5f761d0b18ff1c41d2c3e4aeb
Defer the softirq processing to ksoftirqd if a RT task is
running or queued on the current CPU. This complements the RT
task placement algorithm which tries to find a CPU that is not
currently busy with softirqs.
Currently NET_TX, NET_RX, BLOCK and IRQ_POLL softirqs are only
deferred as they can potentially run for long time.
Additionally, this patch stubs out ksoftirqd_running() logic,
in the CONFIG_RT_SOFTIRQ_AWARE_SCHED case, as deferring
potentially long-running softirqs will cause the logic to not
process shorter-running softirqs immediately. By stubbing it out
the potentially long running softirqs are deferred, but the
shorter running ones can still run immediately.
This patch includes folded-in fixes by:
Lingutla Chandrasekhar <clingutla@codeaurora.org>
Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
J. Avila <elavila@google.com>
Cc: John Dias <joaodias@google.com>
Cc: Connor O'Brien <connoro@google.com>
Cc: Rick Yiu <rickyiu@google.com>
Cc: John Kacur <jkacur@redhat.com>
Cc: Qais Yousef <qyousef@google.com>
Cc: Chris Redpath <chris.redpath@arm.com>
Cc: Abhijeet Dharmapurikar <adharmap@quicinc.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: kernel-team@android.com
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
[satyap@codeaurora.org: trivial merge conflict resolution.]
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
[elavila: Port to mainline, squash with bugfix]
Signed-off-by: J. Avila <elavila@google.com>
[jstultz: Rebase to linus/HEAD, minor rearranging of code,
included bug fix Reported-by: Qais Yousef <qais.yousef@arm.com> ]
Signed-off-by: John Stultz <jstultz@google.com>
Link: https://lore.kernel.org/lkml/20221116075929.453876-4-jstultz@google.com/
Change-Id: Iedb590216e373788950243165ff83f4c43015f31
Bug: 168521633
---
v4:
* Fix commit message to accurately note long-running softirqs
(suggested by Qais)
* Switch to using rt_task(current) (suggested by Qais)
v5:
* Switch to using CONFIG_RT_SOFTIRQ_AWARE_SCHED (suggested by
Joel Fernandes <joel@joelfernandes.org>)
In certain audio use cases, scheduling RT threads on cores that
are handling softirqs can lead to glitches. Prevent this
behavior in cases where the softirq is likely to take a long
time. To avoid unnecessary migrations, the old behavior is
preserved for RCU, SCHED and TIMER irqs which are expected to be
relatively quick.
This patch reworks and combines two related changes originally
by John Dias <joaodias@google.com>
Cc: John Dias <joaodias@google.com>
Cc: Connor O'Brien <connoro@google.com>
Cc: Rick Yiu <rickyiu@google.com>
Cc: John Kacur <jkacur@redhat.com>
Cc: Qais Yousef <qyousef@google.com>
Cc: Chris Redpath <chris.redpath@arm.com>
Cc: Abhijeet Dharmapurikar <adharmap@quicinc.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: kernel-team@android.com
Signed-off-by: John Dias <joaodias@google.com>
[elavila: Port to mainline, amend commit text]
Signed-off-by: J. Avila <elavila@google.com>
[connoro: Reworked, simplified, and merged two patches together]
Signed-off-by: Connor O'Brien <connoro@google.com>
[jstultz: Further simplified and fixed issues, reworded commit
message, removed arm64-isms]
Signed-off-by: John Stultz <jstultz@google.com>
Link: https://lore.kernel.org/lkml/20221116075929.453876-3-jstultz@google.com/
Change-Id: Iea6719e1d900be4a3492949685f1218f299f29e8
Bug: 168521633
---
v2:
* Reformatted Kconfig entry to match coding style
(Reported-by: Randy Dunlap <rdunlap@infradead.org>)
* Made rt_task_fits_capacity_and_may_preempt static to
avoid warnings (Reported-by: kernel test robot <lkp@intel.com>)
* Rework to use preempt_count and drop kconfig dependency on ARM64
v3:
* Use introduced __cpu_softirq_pending() to avoid s390 build
issues (Reported-by: kernel test robot <lkp@intel.com>)
v4:
* Drop TASKLET_SOFTIRQ from LONG_SOFTIRQS (suggested by Qais)
* Depend on !PREEMPT_RT (Suggested by Qais)
* Larger simplification of logic (suggested by Qais)
* Rework LONG_SOFTIRQS to use BIT() macros
* Rename task_may_preempt() to cpu_busy_with_softirqs()
v5:
* Conditionalize active_softirqs handling (suggested by Alexander
Gordeev <agordeev@linux.ibm.com>)
* Reorder rt_task_fits_cpu to have the "fast" function first
(Suggested by Alexander Gordeev <agordeev@linux.ibm.com>)
* Fix bug I introduced in v2 condensing
task_thread_info(task)->preempt_count to preempt_count()
(Reported-by: Alexander Gordeev <agordeev@linux.ibm.com>)
* Tweak comment discription to remove the vauge "slow"
descriptor of softirqs being run by ksoftirqd
(Suggested by Alexander Gordeev <agordeev@linux.ibm.com>)
* Switch to using CONFIG_RT_SOFTIRQ_AWARE_SCHED (suggested by
Joel Fernandes <joel@joelfernandes.org>)
* Simplify cpu_busy_with_softirqs() logic as pointed out by
Alexander Gordeev <agordeev@linux.ibm.com>
* Switch to using IS_ENABLED rather then defining my own macro
(suggsted by Joel Fernandes <joel@joelfernandes.org>)
Vendors might want to change tasks affinity settings when they are
moving from one cpuset into the other. Add vendor hook to give control
to vendor to implement what they need.
This feature is necessary to control hotplug behaviour within Qualcomm's
proprietary load tracking scheme, WALT.
This reverts commit 034ddf86f7 ("Revert "ANDROID: sched/cpuset: Add
vendor hook to change tasks affinity"") to effectively bring back the
original change.
Bug: 174125747
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Change-Id: Id4e9c3e47e3b4e041804bdf10cbd9e36179bc172
This deliberately changes the behavior of the per-cpuset
cpus file to not be effected by hotplug. When a cpu is offlined, it will
be removed from the cpuset/cpus file. When a cpu is onlined, if the cpuset
originally requested that cpu be a part of the cpuset, that cpu will be
restored to the cpuset. The cpus files still have to be hierachical, but
the ranges no longer have to be out of the currently online cpus, just the
physically present cpus.
This reverts commit 3fc3fe757f ("Revert "ANDROID: cpuset: Make cpusets
restore on hotplug""). Reverting the revert effectively bringing back
the original change.
Bug: 174125747
Bug: 120444281
Signed-off-by: Dmitry Shmidt <dimitrysh@google.com>
[AmitP: Refactored original changes to align with upstream commit
201af4c0fa ("cgroup: move cgroup files under kernel/cgroup/")]
Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
[satyap@codeaurora.org: port to android-mainline kernel]
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
[SaiHarshiniN: Resolved merge conflict post 6.1-rc1]
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Change-Id: I588f6172c15b48ecadb85f161dae948ce9aeca93
UPDATED: Cherry pick updated with exports for the
protected modules for kernel 6.1
Update protected export symbols list with exports
from list of protected modules at
android/gki_protected_modules.
It includes symbols from every GKI modules except
zram & zsmalloc; and serves as a baseline.
Bug: 232430739
Test: TH
Change-Id: Iec33dfe093b4e9e0281b910b2b3bf998cef55394
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
(cherry picked from commit 16c63232db)
android/gki_protected_modules serves as a running
list of protected GKI modules. This list is being
used as an input to generate list of protected
GKI modules exports at android/abi_gki_protected_exports
All GKI modules are protected except zram.ko & zsmalloc.ko
as baseline in this list.
Bug: 232430739
Test: TH
Change-Id: I0c993769b9d07543755fd056199b0e4d10d27f77
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
(cherry picked from commit 90ff743687)
Only enforce export protection if there are symbols in the
unprotected list for the Kernel Module Interface (KMI).
This is only relevant for targets like arm64 that have
defined ABI symbol lists. This allows non-GKI targets
like arm and x86 to continue using GKI source code
without disabling the feature for those targets.
Bug: 232430739
Test: TH
Fixes: fd1e768866 ("ANDROID: GKI: Protect exports of protected GKI modules")
Change-Id: Ie89e8f63eda99d9b7aacd1bb76d036b3ff4ba37c
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
(cherry picked from commit a6eaf3db80)
Implement support for protecting the exported symbols of
protected GKI modules.
Only signed GKI modules are permitted to export symbols
listed in the android/abi_gki_protected_exports file.
Attempting to export these symbols from an unsigned module
will result in the module failing to load, with a
'Permission denied' error message.
Bug: 232430739
Test: TH
Change-Id: I3e8b330938e116bb2e022d356ac0d55108a84a01
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
(cherry picked from commit fd1e768866)
When CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is set, the code in algboss.c
that handles CRYPTO_MSG_ALG_REGISTER is unnecessary, so make it be
compiled out.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bug: 256875295
(cherry picked from commit 441cb1b730)
Change-Id: I11ebf60e1915ad5d13bd16a26d6c2c0944b4c401
Signed-off-by: Eric Biggers <ebiggers@google.com>
Make the kdf_sp800108 self-test only print a message on success when
fips_enabled, so that it's consistent with testmgr.c and doesn't spam
the kernel log with a message that isn't really important.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bug: 256875295
(cherry picked from commit 790c4c9f53)
Change-Id: Icb2c93c0ed51789fd425d36ebcaebf01af6b4c78
Signed-off-by: Eric Biggers <ebiggers@google.com>
Make kdf_sp800108 honor the CONFIG_CRYPTO_MANAGER_DISABLE_TESTS kconfig
option, so that it doesn't always waste time running its self-test.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bug: 256875295
(cherry picked from commit 0bf365c0ef)
Change-Id: Ib189f727a7bb1a231cc8ec4ca6450e685bd678f7
Signed-off-by: Eric Biggers <ebiggers@google.com>
The crypto_boot_test_finished static key is unnecessary when self-tests
are disabled in the kconfig, so optimize it out accordingly, along with
the entirety of crypto_start_tests(). This mainly avoids the overhead
of an unnecessary static_branch_enable() on every boot.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bug: 256875295
(cherry picked from commit 06bd9c967e)
Change-Id: I68eff9772dc219a8786bf410cb4e946052ea7811
Signed-off-by: Eric Biggers <ebiggers@google.com>
Since algboss always skips testing of algorithms with the
CRYPTO_ALG_INTERNAL flag, there is no need to go through the dance of
creating the test kthread, which creates a lot of overhead. Instead, we
can just directly finish the algorithm registration, like is now done
when self-tests are disabled entirely.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bug: 256875295
(cherry picked from commit 9cadd73ade)
Change-Id: I10f814cd6903d41265f69297d8568b43ec30012e
Signed-off-by: Eric Biggers <ebiggers@google.com>
Currently, registering an algorithm with the crypto API always causes a
notification to be posted to the "cryptomgr", which then creates a
kthread to self-test the algorithm. However, if self-tests are disabled
in the kconfig (as is the default option), then this kthread just
notifies waiters that the algorithm has been tested, then exits.
This causes a significant amount of overhead, especially in the kthread
creation and destruction, which is not necessary at all. For example,
in a quick test I found that booting a "minimum" x86_64 kernel with all
the crypto options enabled (except for the self-tests) takes about 400ms
until PID 1 can start. Of that, a full 13ms is spent just doing this
pointless dance, involving a kthread being created, run, and destroyed
over 200 times. That's over 3% of the entire kernel start time.
Fix this by just skipping the creation of the test larval and the
posting of the registration notification entirely, when self-tests are
disabled.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bug: 256875295
(cherry picked from commit a7008584ab)
Change-Id: Ia6be068618e9286c1be01415a6766ba2fa94fc0d
Signed-off-by: Eric Biggers <ebiggers@google.com>
Hypervisor vendor modules may need to create non-cacheable mappings in
the hypervisor stage-1 for interacting with devices such as IOMMUs.
Add support for this memory type to the KVM pgtable API and implement
it for both stage-1 and stage-2.
Bug: 244373730
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I2f88db7fe47e16366018e3e48f30d09b299ae6e4
If a KVM_FUNC_MMIO_GUARD_MAP hypercall from a protected guest fails at
EL2 due to running out of page-table memory, the call is forwarded to
the host so that additional memory can be donated using the vCPU's
memcache.
Unfortunately, the host filters out these calls the hypervisor will
replay the guest's HVC instruction forever, making no progress because
it will fail each time.
Avoid filtering out KVM_FUNC_MMIO_GUARD_MAP, in the same way as we
handle the SHARE and UNSHARE hypercalls.
Bug: 262700476
Cc: Keir Fraser <keirf@google.com>
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Idd14c6bc08a4232939676e3566b79cbc7c927a3a
This optimization allows us to re-create higher order block mappings in
the host stage2 pagetables after we teardown a guest VM.
When the host reclaims ownership during guest teardown, the page table
walker drops the refcount of the counted entries and clears out
unreferenced entries (refcount == 1). Clearing out the entry installs a
zero PTE. When the host stage2 receives a data abort because there is no
mapping associated, it will try to create the largest possible block
mapping from the founded leaf entry.
With the current patch, we increase the chances of finding a leaf entry
that has level < 3 if the requested region comes from a reclaimed torned
down VM memory. This has the advantage of reducing the TLB pressure at
host stage2.
To increase the coalescing chances, we modify the way we refcount page
table descriptors for host stage2:
- non-zero invalid PTEs
- any of the reserved-high bits(58-55) toogled
- non-default attribute mappings
- page table descriptors
Bug: 222044487
Test: dump the host stage2 pagetables and view the mapping
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Change-Id: I90ff4ec2185e9a76d7ad17e77ef9bdd8ce3e8698
Signed-off-by: Quentin Perret <qperret@google.com>
In preparation for the coalescing algorithm implementation, move the
function which verifies if a page table entry is a tabel to the common
header.
Bug: 222044487
Change-Id: I4124b7727e91f61b8f0a7e44cd91403d09d83c3c
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Move the host specific code for PTE reference counting out of the
pagetable code and define a new structure that wraps all the PTE
manipulation callbacks. This structure will be passed during the
pagetable code initialization and it allows to register different
callback for [guest|host].
Bug: 222044487
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Change-Id: I116e8322935762df2f2be6e8d51a3f0c140b3d36
Signed-off-by: Quentin Perret <qperret@google.com>
Make PTE attribute definitions available from kvm_pgtable.h and take
them out of the pagetable code. These attributes will be used later in
mem_protect.c to construct different masks during the PTE manipulation
callbacks.
Bug: 222044487
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Change-Id: I2f7108815ef0fa536e7f3314762a412119400fe9
Signed-off-by: Quentin Perret <qperret@google.com>
Refactor the code and add stage2_clear_pte(..) which removes the PTE
without dropping the refcount for an entry.
Bug: 222044487
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Change-Id: Ia2cb47f2ffad6faa5c6b4ec8a37bcbe61be0bc2f
Signed-off-by: Quentin Perret <qperret@google.com>
Extend the scope of the stage2_freewalker by passing the pgt instead of
the mm_ops callbacks. This will later be used by the stage2_pte_is_counted
function.
Bug: 222044487
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Change-Id: I390661eb106cbdb863cbb1832e39ec155c439091
Signed-off-by: Quentin Perret <qperret@google.com>
The recently introduced support for pKVM modules has clearly lacked a
bit of testing with CONFIG_MODULES=n.
Make sure to not access __pkvm_modules_enabled when CONFIG_MODULES=n to
fix the link.
Bug: 244543039
Bug: 244373730
Reported-by: kernel test robot <lkp@intel.com>
Change-Id: I65c6ad7c07c2812b85dcd435b24b3444c0ce7f1e
Signed-off-by: Quentin Perret <qperret@google.com>
The recently introduced support for pKVM modules has clearly lacked a
bit of testing with CONFIG_MODULES=n.
Fix the broken __pkvm_init_module() stub to make the bots happy.
Bug: 244543039
Bug: 244373730
Reported-by: kernel test robot <lkp@intel.com>
Change-Id: Ie59cbf46442721de78ef51523debadc4c156530e
Signed-off-by: Quentin Perret <qperret@google.com>
Disable module loading, based on the cmdline option
kvm-arm.protected_modules. If enabled, the feature can later be
irreversibly disabled with the HVC __pkvm_close_module_registration().
Bug: 244543039
Bug: 244373730
Change-Id: I693a1db350ae15bcaab59103a68b60f087dcc6b4
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Return the fixmap VA with the page offset, instead of the page base
address.
Bug: 244543039
Bug: 229972309
Change-Id: I40c342f84e3cd395156ef846f0434c028d4e3fa3
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When pKVM is in use, allow pKVM modules to register custom hypercall
handlers:
* pkvm_register_el2_call(): Give a handler to the hypervisor and gets in
return the newly registered hypercall number.
* pkvm_el2_mod_call(): Call the previously registered hypercall handler.
There is a limit of 128 hypercalls that can be registered.
Bug: 244543039
Bug: 244373730
Change-Id: I3d6c89675efe5f65f6b53c3b45ae155d1a00164c
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Later introduction of custom HVCs will require to store some data.
Conviniently, the only thing we need is the start addr (in the hyp VA) of
the text section. So use this value as a token to avoid having to store
it anywhere.
Bug: 244543039
Bug: 244373730
Change-Id: Idd0bcbbb36d189aa4932833ca5b40382c2cddb08
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
In preparation for allowing to restrict host permissions at stage-2 for
certain pages, introduce some infrastructure allowing a pKVM module to
register a permission fault handler.
Bug: 244543039
Bug: 244373730
Change-Id: I8035c64969cc0ebb01c8936b1974b3bc103ba84f
Signed-off-by: Quentin Perret <qperret@google.com>
pKVM currently doesn't map pages with reduced permissions in stage2
page-tables, for both host and guests. But in preparation for allowing
that to happen, introduce a new page 'meta-state' that is set whenever a
page has non-default permissions. Introducing it ensures that e.g.
host-to-guest donations or shares will fail for a page that the host has
read-only, as the page will no longer be PKVM_PAGE_OWNED only.
Bug: 244543039
Bug: 244373730
Change-Id: Ib3b20c7edabafb9a305f000a37bbdb4bcc8fdbbb
Signed-off-by: Quentin Perret <qperret@google.com>
Expose the hyp_fixmap helpers in the module_ops struct to allow dynamic
mapping and unmapping of pages from pKVM modules.
Bug: 244543039
Bug: 244373730
Change-Id: I201db6044ed5eb4c2821a64a6b650b931dd2e389
Signed-off-by: Quentin Perret <qperret@google.com>
Expose the hyp_puts() and hyp_putx64() helpers in the module_ops struct
to allow logging messages on the UART from pKVM modules.
Bug: 244543039
Bug: 244373730
Change-Id: Ica578667297e5a1f94c370603c29482be89982a9
Signed-off-by: Quentin Perret <qperret@google.com>
Debugging a hypervisor tends to be trickier than normal system code
such as the kernel. The lack of availability of a UART framework is a
significant contributor to that. In order to address this, introduce a
framework allowing to load serial drivers into the hypervisor.
Bug: 244543039
Bug: 244373730
Change-Id: I2e7a1fd9abc9d5aa9d95f1d271a997d54a8fd582
Signed-off-by: Quentin Perret <qperret@google.com>
pKVM has an internal API allowing to create mappings in the 'private'
range of the hypervisor VA space for which there are no rules
constraining the VA-PA relation (hence comparable to the vmalloc area in
the kernel). This will be a useful API for hypervisor modules, so expose
it in the recently introduced module ops struct.
Bug: 244543039
Bug: 244373730
Change-Id: I2a8f958a02c3c3b9871224b65b00b207820a507a
Signed-off-by: Quentin Perret <qperret@google.com>
Since .hyp.rodata sections of pKVM modules are emitted with SHT_MERGE,
ld.ldd feels free to attempt merging it with other sections.
Unfortunately, the pKVM module linker script doesn't always place them
in output sections, hence causing link failures:
ld.lld: error: drivers/misc/pkvm-pl011/hyp/kvm_nvhe.tmp.o:(.hyp.rodata):
offset is outside the section
In practice, ld.ldd only seems to attempt merging .note.gnu.property with
.hyp.rodata. To work around the problem, make sure to explicitely place
the .note.gnu.property in .hyp.rodata from the start, hence preventing
ld.ldd from trying to optimize further.
A preferable solution would be to teach ld.lld that merging pKVM modules
sections is a bad idea, or to make sure the sections are not emitted
with SHT_MERGE to begin with, but we couldn't find an obvious way to make
that happen. This workaround is nothing more than a pratical compromise.
Bug: 244543039
Reported-by: Will Deacon <will@kernel.org>
Suggested-by: Will Deacon <will@kernel.org>
Change-Id: Iae902bdfd21915f552e218515cd77881a95fef2d
Signed-off-by: Quentin Perret <qperret@google.com>
All nVHE hypervisor code is currently required to be statically linked
into the kernel image. Sadly, scaling pKVM will inevitably require
running _some_ hardware-specific code in the hypervisor due to the
absence of architecture requirements regarding IOMMU implementations or
power management, for example.
In order to address this issue, introduce the ability to load modules
in the pKVM hypervisor at run-time. pKVM modules are expected to be
embedded inside kernel modules, and to be loaded into pKVM when their
kernel counterpart is loaded at EL1. pKVM module loading is defined as a
privileged operation -- all of them must be loaded while the host kernel
is still part of the trusted computing base.
Bug: 244543039
Bug: 244373730
Co-authored-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Change-Id: If8e5d3ac0a2893c892aff09e5b51d3b8e14693f8
Signed-off-by: Quentin Perret <qperret@google.com>
Building and linking hypervisor modules into kernel modules involves
very similar problems to building and linking the KVM nVHE hypervisor
object into the kernel image. In order to re-use the same building
procedure, let's factor out and generalize the nVHE makefile in a way
that allows it to be re-used for module builds.
Bug: 244543039
Bug: 244373730
Change-Id: I89b2630f8cfd3ce5b624300fc277749be4fc9e04
Signed-off-by: Quentin Perret <qperret@google.com>
[vdonnefort@: Merge with updated nvhe/Makefile]
Signed-off-by: Vincent Donnefort <vdonnefort@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
The start and end pointers of the hyp relocs section are currently
specified in the vmlinux linker script. In order to ease re-using the
same relocation procedure for hypervisor modules, emit the delimiters
from the generated hyp-reloc.S file directly.
Bug: 244543039
Bug: 244373730
Change-Id: I845af2d40e1dd13301069537c6325f6a6f381ce4
Signed-off-by: Quentin Perret <qperret@google.com>
In order to allow re-use of the gen-hyprel tool to build hypervisor
modules in the future, move it up to the arm64 tools folder.
Bug: 244543039
Bug: 244373730
Change-Id: I188a2dac1acf4974213499970cc29552807497eb
Signed-off-by: Quentin Perret <qperret@google.com>
The new pkvm_remove_mappings() allows the caller to unmap a memory range
from the hypervisor.
This is to allow later introduction of in-hypervisor tracing.
Bug: 244543039
Bug: 229972309
Change-Id: I9edc7c2ae55c4b7f5d464d26ce3351b5fd4bf9f3
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The process of applying hypervisor relocations involves patching
addresses in the hypervisor object. In the existing KVM nVHE relocation
procedure, the relocations are applied early enough for write-permission
not to be a problem when touching e.g. the rodata section. But applying
relocations on hypervisor modules embedded in a kernel module proves
more challenging, as the kernel module loader will actively map text and
rodata sections read-only. In order to allow patching in those sections,
let's introduce a new helper function using the text fixmap to
temporarily map the relevant pages writable.
Bug: 244543039
Bug: 244373730
Change-Id: I9dcdade1927e5bc121db87bc950fb70a374c44cd
Signed-off-by: Quentin Perret <qperret@google.com>
