Add logging support for MMIO high level accessors such as read{b,w,l,q}
and their relaxed versions to aid in debugging unexpected crashes/hangs
caused by the corresponding MMIO operation. Also add a generic flag
(__DISABLE_TRACE_MMIO__) which is used to disable MMIO tracing in nVHE KVM
and if required can be used to disable MMIO tracing for specific drivers.
Bug: 169045115
Link: https://lore.kernel.org/lkml/e64a532ce10e88269b1e8550293e07ea669c5f73.1638858747.git.quic_saipraka@quicinc.com/
Change-Id: I861e9d351cc2660051a40f36b391609c06aabb99
Signed-off-by: Sai Prakash Ranjan <quic_saipraka@quicinc.com>
Signed-off-by: Vamsi Krishna Lanka <quic_vamslank@quicinc.com>
Generic MMIO read/write i.e., __raw_{read,write}{b,l,w,q} accessors
are typically used to read/write from/to memory mapped registers
and can cause hangs or some undefined behaviour in following few
cases,
* If the access to the register space is unclocked, for example: if
there is an access to multimedia(MM) block registers without MM
clocks.
* If the register space is protected and not set to be accessible from
non-secure world, for example: only EL3 (EL: Exception level) access
is allowed and any EL2/EL1 access is forbidden.
* If xPU(memory/register protection units) is controlling access to
certain memory/register space for specific clients.
and more...
Such cases usually results in instant reboot/SErrors/NOC or interconnect
hangs and tracing these register accesses can be very helpful to debug
such issues during initial development stages and also in later stages.
So use ftrace trace events to log such MMIO register accesses which
provides rich feature set such as early enablement of trace events,
filtering capability, dumping ftrace logs on console and many more.
Sample output:
rwmmio_write: __qcom_geni_serial_console_write+0x160/0x1e0 width=32 val=0xa0d5d addr=0xfffffbfffdbff700
rwmmio_post_write: __qcom_geni_serial_console_write+0x160/0x1e0 width=32 val=0xa0d5d addr=0xfffffbfffdbff700
rwmmio_read: qcom_geni_serial_poll_bit+0x94/0x138 width=32 addr=0xfffffbfffdbff610
rwmmio_post_read: qcom_geni_serial_poll_bit+0x94/0x138 width=32 val=0x0 addr=0xfffffbfffdbff610
Bug: 169045115
Link: https://lore.kernel.org/lkml/76983c26d889df7252a17017a48754163fb6b0d5.1638858747.git.quic_saipraka@quicinc.com/
Change-Id: Ia21f54f8ce8f11a5613c7218dc7c9f7248766273
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
Co-developed-by: Sai Prakash Ranjan <quic_saipraka@quicinc.com>
Signed-off-by: Sai Prakash Ranjan <quic_saipraka@quicinc.com>
Signed-off-by: Vamsi Krishna Lanka <quic_vamslank@quicinc.com>
Add vendor hook for affinity setting to GIC irqchip driver.
GIC irqchip driver support only single CPU for affinity setting.
But GIC HW can support multi target IRQ action, any available CPU can
service IRQ when requested irq arised. This feature is very useful
for some fast irq action required device.
So Add vendor hook to support multi target affinity setting.
Bug: 210559155
Change-Id: If44b2244bb623bf8d4755109a6dca32ac35df795
Signed-off-by: Hajun Sung <hajun.sung@samsung.com>
Samsung ExynosAuto v9 SoC has two types of host controller interface to
support the virtualization of UFS Device. One is the physical host (PH)
that is the same as conventional UFSHCI, and the other is the virtual host
(VH) that supports data transfer function only.
In this configuration the virtual host does not support UIC commands. Add a
quirk to return 0 when the UIC command send function is called.
Link: https://lore.kernel.org/r/20211018124216.153072-2-chanho61.park@samsung.com
Cc: Alim Akhtar <alim.akhtar@samsung.com>
Cc: James E.J. Bottomley <jejb@linux.ibm.com>
Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: jongmin jeong <jjmin.jeong@samsung.com>
Signed-off-by: Chanho Park <chanho61.park@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 206186055
(cherry picked from commit a22bcfdbf1)
[Chanho: Resolved cherry-pick conflict in drivers/scsi/ufs/ufshcd.h due
to below ANDROID keyslot patch.
74e7fc0492 ("ANDROID: scsi: ufs: split up UFSHCD_QUIRK_NO_KEYSLOTS")
No code change from original patch]
Change-Id: Iff59b08ebf1494542c548678772cfc0230c2f91e
By default with SELinux enabled behavior for unsigned
module loading is same as sig_enforce=1. This causes
loading of unsigned modules fail. All modules in Android
GKI are unsigned except GKI modules. Do not prevent
module loading in case of CONFIG_SIG_MODULE_PROTECT; which
was introduced to change behavior of sig_enforce to allow
unsigned modules but not access to protected symbols.
Bug: 200082547
Bug: 214445388
Fixes: 9ab6a24225 ("ANDROID: GKI: Add module load time protected symbol lookup")
Test: TreeHugger
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: Iab3113d706cbd7db7a5684897bcafd5671a6d424
CONFIG_MODULE_SIG_ALL needs to be set for gki_defconig,
but will require an override via device fragments
to avoid signing the vendor modules at build-time.
It defaults to 'y' so no need to explicitly set for
gki_defconfig.
Bug: 200082547
Bug: 214445388
Fixes: 9ab6a24225 ("ANDROID: GKI: Add module load time protected symbol lookup")
Test: TH, manual builds including P21 mainline
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: Iafc0936b5e7bfb781b28642d1ec233a7fcf85f09
Open Profile for DICE is an open protocol for measured boot compatible
with the Trusted Computing Group's Device Identifier Composition
Engine (DICE) specification. The generated Compound Device Identifier
(CDI) certificates represent the hardware/software combination measured
by DICE, and can be used for remote attestation and sealing.
Add a driver that exposes reserved memory regions populated by firmware
with DICE CDIs and exposes them to userspace via a character device.
Userspace obtains the memory region's size from read() and calls mmap()
to create a mapping of the memory region in its address space. The
mapping is not allowed to be write+shared, giving userspace a guarantee
that the data were not overwritten by another process.
Userspace can also call write(), which triggers a wipe of the DICE data
by the driver. Because both the kernel and userspace mappings use
write-combine semantics, all clients observe the memory as zeroed after
the syscall has returned.
Acked-by: Rob Herring <robh@kernel.org>
Cc: Andrew Scull <ascull@google.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: David Brazdil <dbrazdil@google.com>
Link: https://lore.kernel.org/r/20220104100645.1810028-3-dbrazdil@google.com
Bug: 198197082
[willdeacon@: Fixed context conflict in Makefile]
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 209580772
Change-Id: If1160c8cc3a39ea822e089d1b80c837aec8075fa
Signed-off-by: Will Deacon <willdeacon@google.com>
Add DeviceTree bindings for Open Profile for DICE, an open protocol for
measured boot. Firmware uses DICE to measure the hardware/software
combination and generates Compound Device Identifier (CDI) certificates.
These are stored in memory and the buffer is described in the DT as
a reserved memory region compatible with 'google,open-dice'.
Signed-off-by: David Brazdil <dbrazdil@google.com>
Link: https://lore.kernel.org/r/20220104100645.1810028-2-dbrazdil@google.com
Bug: 198197082
Bug: 209580772
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: If318ad91ef1ae26ff639f99a4349e8c737d286b6
Signed-off-by: Will Deacon <willdeacon@google.com>
As pKVM does not trust the host, it should not be involved in the
handling of, or be able to observe the response to entropy requests
issues by protected guests.
When an SMC-based implementation of the ARM SMCCC TRNG interface is
present, pass any HVC-based requests directly on to the secure firmware.
Co-developed-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: Ica492ce49fd059a62ecc31bb7ac13c9adb773a08
Signed-off-by: Will Deacon <willdeacon@google.com>
Using an alias of the host's `__icache_flags` variable at EL2 for pKVM
is risky, as it provides the host with a mechanism to elide cache
maintenance of guest pages by causing functions such as icache_is_vpipt()
to erroneously return false.
Create a private copy of the __icache_flags variable at EL2 and
initialise it using the host's version during pKVM init.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: I595f11d1e336dadae0eb82222e4da79a1069012a
Signed-off-by: Will Deacon <willdeacon@google.com>
On initialising the MMIO guard infrastructure, register the
earlycon mapping if present.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I379387253d08e2414fa386a3360a45391da7d90d
Signed-off-by: Will Deacon <willdeacon@google.com>
In order to transfer the early mapping state into KVM's MMIO
guard infrastucture, provide a small helper that will retrieve
the associated PTE.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Iefc1c57d5e9476b718a8a68f60e562a57b09fb6a
Signed-off-by: Will Deacon <willdeacon@google.com>
Should a guest desire to enroll into the MMIO guard, allow it to
do so with a command-line option.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Ia9a77f693531740500739693c52b4959abacafd4
Signed-off-by: Will Deacon <willdeacon@google.com>
Implement the previously defined ioremap/iounmap hooks for arm64,
calling into KVM's MMIO guard if available.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I86a78f8941fb60078fb873a34c5eb32830a00259
Signed-off-by: Will Deacon <willdeacon@google.com>
Add a pair of hooks (ioremap_phys_range_hook/iounmap_phys_range_hook)
that can be implemented by an architecture. Contrary to the existing
arch_sync_kernel_mappings(), this one tracks things at the physical
address level.
This is specially useful in these virtualised environments where
the guest has to tell the host whether (and how) it intends to use
a MMIO device.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I970c2e632cb2b01060d5e66e4194fa9248188f43
Signed-off-by: Will Deacon <willdeacon@google.com>
Document the hypercalls user for the MMIO guard infrastructure.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I927bcd6c5e3ef932265d817288ff2b46b0e0db66
Signed-off-by: Will Deacon <willdeacon@google.com>
Plumb the MMIO checking code into the MMIO fault handling code.
Any fault hitting outside of an MMIO region will now report
an invalid syndrome, and won't leak any data from the guest.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I68bef2d0211a804aa1e598aeaa0c85dc4098f61e
Signed-off-by: Will Deacon <willdeacon@google.com>
Plumb in the hypercall interface to allow a guest to discover,
enroll, map and unmap MMIO regions.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I0390456ffde8ceca351d3d8e82fd1dddeb747fac
Signed-off-by: Will Deacon <willdeacon@google.com>
Introduce the infrastructure required to identify an IPA region
that is expected to be used as an MMIO window.
This include mapping, unmapping and checking the regions. Nothing
calls into it yet, so no expected functional change.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I227eaa28b98e067e3daae4f9e1071eb37a6761cc
Signed-off-by: Will Deacon <willdeacon@google.com>
Add a per-VM flag indicating that the guest has bought into the
MMIO guard enforcement framework.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: If60b2b38a419a9f44ebe9029f55dd016fd2444b5
Signed-off-by: Will Deacon <willdeacon@google.com>
In order to simplify the implementation of an EL2-only version of
MMIO guard, expose topup_hyp_memcache() and simplify its usage
by only requiring a vcpu.
While we're at it, make free_hyp_memcache() visible in kvm_host.h
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I4f54c57a9693cf7a3450f99fedc15ae32af09a31
Signed-off-by: Will Deacon <willdeacon@google.com>
Define the handful of hypercalls that MMIO guard will require.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Iac312b2327c31a1532fdb38e8fa8066291d9f611
Signed-off-by: Will Deacon <willdeacon@google.com>
Don't blindly assume that the PTE is valid when checking whether
it describes an executable or cacheable mapping.
This makes sure that we don't issue CMOs for invalid mappings.
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I5b271c91aa6ceb23f7b1e6a571e30d080866d5c9
Signed-off-by: Will Deacon <willdeacon@google.com>
We currently deal with a set of booleans for VM features,
while they could be better represented as set of flags
contained in an unsigned long, similarily to what we are
doing on the CPU side.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I86be6bab12287c3eb21bbe03f255e2899edbdffb
Signed-off-by: Will Deacon <willdeacon@google.com>
Since we must still support the dreaded set/way CMOs for non-protected
VMs (as well as the equivalent operation when vcpus switch their MMU
on), perform an invalidation that will iterate over all the pages
that have been donated to the guest, one after the other.
This requires a minor change to the locking used for donation so
that all donated pages can be seen by a concurrent invalidation.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I1780127722bda7bdc884bb4e68db6ae47d042822
There is no difference between protected and non-protected guests
when it comes to shadow structures, and we want these shadow
structures to have the same life cycle.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I7e9bf366aae6bd0542d0038d24e2350a9dd23cd0
Signed-off-by: Will Deacon <willdeacon@google.com>
We want the host to handle everything as usual.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Icf8ee146917e886bca258815cf948a1b12540353
Signed-off-by: Will Deacon <willdeacon@google.com>
Instead of donating memory to non-pVMs, share the memory, which
gives us a good enough approximation of the usual behaviour.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I47213754613110a6fb8157806eb96ddf92ead346
Signed-off-by: Will Deacon <willdeacon@google.com>
In order to deal with state synchronisation between EL1 and EL2,
we use the following setup:
- On exit from EL2, the state is forcefully marked clean.
- Should a trap be handled, the state is synchronised and immediately
marked dirty
- On vcpu_put(), the state is also marked dirty, since it can be
modified by userspace
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I47a889ca5432566f236de4630d81753348632f8a
Signed-off-by: Will Deacon <willdeacon@google.com>
In order for a non-protected guest to be functionnal, userspace
has to be able to query its state, which means that the host view
of the vcpu has to be kept up to date.
In order to achieve this, we establish the following scheme for EL2:
- On entering vcpu_run(), we check for the KVM_ARM64_PKVM_STATE_DIRTY
flag in the host vcpu. If set, we sync the state *from* the host
to the shadow version.
- On exiting vcpu_run(), we don't do anything, but let the host
issue a synch hypercall if required.
- On vcpu_put(), we force a synchronisation *to* the host.
The El1 host will have a complementary approach in the following
patches.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I42811a25d2e176d6c7d9a66ade6e9149a96e9256
Signed-off-by: Will Deacon <willdeacon@google.com>
A non-protected guest requires a lot less handling than a protected
one when dealing with entries/exits from/to EL2.
Since we already indiredct those, introduce new entry/exit tables
for non-pVMs.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I66602bc491a4a87d6482b12e4eaf7aa53a7dbfd9
Signed-off-by: Will Deacon <willdeacon@google.com>
As we're about to need to copy some state back and forth for
non0-protected guests, pass the full loaded state to the flush/sync
functions.
No functionnal change.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I7ad6a00a7500e91237fcc0981261c819b2224ee0
Signed-off-by: Will Deacon <willdeacon@google.com>
When pKVM is enabled, all the vcpus must have a shadow structure
managed by the hypervisor, irrespective of theur protection status.
This field thus represents the wrong abstraction. Replace it with
'pkvm_loaded_state.is_protected', which tracks whether a vcpu is
part of a protected VM.
pkvm_loaded_state gets also moved around for convenience with the
following patches.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Ic9876fde543abb350fe8969d5b4661e30092f553
Signed-off-by: Will Deacon <willdeacon@google.com>
A number of KVM definitions are keyd on _KVM_NVHE_HYPERVISOR__
being defined or not. Make sure we advertise this #define when
compiling hyp-constants.o, so that we get the right stuff.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Ied191c0a18274258cffede72b06b0fb5bba5604e
Signed-off-by: Will Deacon <willdeacon@google.com>
Instead of poking into the internals of the host KVM structure,
stick to the shadow structures when trying to work out whether
a vcpu is part of a protected VM or not.
Take this opportunity to sprinkle a couple of unlikely(), just
because.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I22a096e1e3cfe34cd2658684b02d8bac486416c4
Signed-off-by: Will Deacon <willdeacon@google.com>
As we can't really rely on the host side for the protection status,
snapshot the expected status at VM creation time.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I0943eadba25e6c9fe718f29e749b9fcc8fbb79ba
Signed-off-by: Will Deacon <willdeacon@google.com>
As KVM is moving to using an xarray to hold the vcpus instead of
the fixed size array that has been the norm so far, we are faced
with two options: either teach the EL2 code to parse an xarray
when building the shadow structures, or find an alternative way
of communicating the vcpus to the EL2 code.
An easy way to deal with the second approach is to use the page
that EL1 donates to HYP to hold the VM S2 PDG. Instead of just
giving the memory, let's copy the pointers to the vcpus in this
page. The overhead is acceptable (it happens only at VM creation
time), and in most cases we only have a handful of vcpus.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: Id0264f0960821563c4b3c0dfcbc43598b85a1f3b
Signed-off-by: Will Deacon <willdeacon@google.com>
There really isn't much point in keeping these separate.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I78b5c2d33bd4178415d51b2bccabfb5a7590d2c2
Signed-off-by: Will Deacon <willdeacon@google.com>
Move the vcpu memcache topup into its own helper, as we will
eventually need it for the MMIO guard page table updates
(which uses the exact same mechanism).
No functionnal change.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Bug: 209580772
Change-Id: I72bac5e8be91acbb696a1428fc5cc6cc84d2df66
Signed-off-by: Will Deacon <willdeacon@google.com>
Expose a new capability, KVM_CAP_ARM_PROTECTED_VM, for protected VMs
which allows the size of the PVM firmware region to be discovered from
userspace and for the firmware load address to be specified if it is
required.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: I819b9b2cfa227f1a0607a8f683aa01d4ae50704f
Signed-off-by: Will Deacon <willdeacon@google.com>
Introduce a new virtual machine type, KVM_VM_TYPE_ARM_PROTECTED, which
specifies that the guest memory pages are to be unmapped from the host
stage-2 by the hypervisor.
Signed-off-by: Will Deacon <will@kernel.org>
[willdeacon@: Changed UAPI constants to reduce change of upstream collisions]
Bug: 209580772
Change-Id: I9de1ad96fec4f62434a81101749435f8b0596162
Signed-off-by: Will Deacon <willdeacon@google.com>
When a PVM firmware image is present for a protected VM, treat the first
running vCPU as the "primary" vCPU and reset its registers accordingly,
in particular by initialising its PC to enter the firmware at startup.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: I26676637145c7d809c5dc5ac0ad0e1fadaf275d2
Signed-off-by: Will Deacon <willdeacon@google.com>
When the host donates a page to a protected guest at an IPA which
coincides with the PVM firmware load address, copy-in the relevant
firmware page after unmapping it from the host but before mapping it
into the guest.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: I8cec813fa52938945f3122655deb785523a96ec8
Signed-off-by: Will Deacon <willdeacon@google.com>
Unmap the PVM firmware memory from the pKVM host by transferring
ownership of the pages to the hypervisor when the host deprivileges
itself during boot.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: I311642f543c0c73d0e0cf2ec051e8e2d9759c5d1
Signed-off-by: Will Deacon <willdeacon@google.com>
Add support for a "linux,pkvm-guest-firmware-memory" reserved memory
region, which can be used to identify a firmware image for protected
VMs. If pKVM fails to initialise and a firmware region is advertised,
then the memory is cleared during boot.
Signed-off-by: Will Deacon <will@kernel.org>
[willdeacon@: Include linux/io.h for memremap() and friends]
Bug: 209580772
Change-Id: Ibfcc0ff00d4b8a42747452047856cb9ba8def4c4
Signed-off-by: Will Deacon <willdeacon@google.com>
has_vhe() expands to a compile-time constant when evaluated from the VHE
or nVHE code, alternatively checking a static key when called from
elsewhere in the kernel. On face value, this looks like a case of
premature optimization, but in fact this allows symbol references on
VHE-specific code paths to be dropped from the nVHE object.
Expand the comment in has_vhe() to make this clearer, hopefully
discouraging anybody from simplifying the code.
Cc: David Brazdil <dbrazdil@google.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 209580772
Change-Id: Icce36e192cafa14d388cb1d0a24585b6fcf6e46e
Signed-off-by: Will Deacon <willdeacon@google.com>
