Add '__pkvm_iommu_driver_init' hypcall and 'struct pkvm_iommu_ops' with
an 'init' callback implemented by an EL2 driver. Driver-specific data
can be passed to 'init' from the host. The memory is pinned while
the callback processed it.
Bug: 190463801
Change-Id: I1185350bb46d41ff060a207af8e6d1f2f8a3d32d
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 1d9ae14c92)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The data abort fault IPA obtained from HFAR_EL2 has the bottom 12 bits
zeroed out. This broke the host MMIO DABT handler because the offsets
of accessed MMIO registers were rounded down to the nearest page.
Include FAR_EL2 in the address to fix the issue.
Bug: 220194478
Change-Id: I6473e2dfbe189c58c15c0e5647d695d07f88c5e0
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 346987baf5)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The S2MPU must wait for a v9 device to finish invalidation before
accessing its SFRs. Failure to do so can result in memory transaction
timeouts.
Add a loop that polls the STATUS register while the return value has
the BUSY and ON_INVALIDATING bits set.
Test: builds, boots
Bug: 190463801
Bug: 206761586
Change-Id: Ie8755bd3466b2c76ca05d6f3f2dd6e8e7bce592c
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit e149939df2)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Comments in S2MPU driver code were mistakenly prefixed with /**,
denoting a kernel-doc comment. Since these do not match kernel-doc
syntax, replace them with regular /* comments.
Test: n/a
Bug: 190463801
Change-Id: I0c68bb5d1c843caeb4d535430bdfc866ba8d119c
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 4377d9dea9)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The `init` callback of an IOMMU driver is called just before
`finalize_host_mappings` so that EL2 mappings created by drivers are
subsequently unmapped from host stage-2. However, at this point hyp has
already switched to the buddy allocator, having reserved pages allocated
by the early allocator, but `pkvm_pgtable.mm_ops` have not been switched
to buddy allocator callbacks. As a result, pages allocated for EL2
mappings of the IOMMU driver are allocated by the obsoleted early
allocator and remain treated as free by the buddy allocator. This likely
leads to a corruption in the free page lists and a later hyp panic.
Move the initialization of `pkvm_pgtable.mm_ops` before
`finalize_host_mappings` and the call to IOMMU's `init`.
Test: run a VM
Test: adb shell cmd jobscheduler run -f android 5132250
Bug: 190463801
Bug: 209004831
Change-Id: I1f6e00bca087d889b0cad4bd43d044895e37006c
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 395d045123)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The function is only used in the compilation unit where it is defined.
Silence a warning by marking it static.
Test: builds
Bug: 190463801
Change-Id: I296cffefdef4639ef2bab644d42f1374ee1a2f60
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 91abc8ece2)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The S2MPU driver needs to protect its MMIO registers from the host.
Implement the host_stage2_adjust_mmio_range callback and restrict
the address range that is about to be mapped in to avoid the known
S2MPU MMIO regions.
Test: builds, boots
Bug: 190463801
Change-Id: Ib46f5dd651b9368c31940035e4c28a7324fc4160
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 8f23406153)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The host should not have access to the vast majority of S2MPU MMIO
registers. Currently it only needs access to fault information, in
the future maybe also performance registers.
Implement an MMIO trap handler for the S2MPU, allowing read-only
access to FAULT_* registers, and a write-only access to
INTERRUPT_CLEAR.
Test: builds, boots
Bug: 190463801
Change-Id: Ia482cc65642ba9ec303f443591e8f0fe192d4d27
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 81e70911d6)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The 'host_stage2_set_owner' callback indicates that a range of
PA-contiguous pages changed owner. With all devices owned by the host,
the driver sets the protection bits in the corresponding FMPT/SMPT to
either MPT_PROT_RW if owned by the host or MPT_PROT_NONE otherwise.
For each gigabyte region, the implementation will select between 1G and
4K/64K (depending on PAGE_SIZE) mappings and populate the L1ENTRY_ATTR
register or SMPT bitmap, respectivelly.
The driver never dynamically switches between two granularities which
both require a SMPT. This is because the L1ENTRY_ATTR and
L1ENTRY_L2TABLE_ADDR registers would need to be set atomically.
Test: builds, boots
Bug: 190463801
Change-Id: Ifb0bdcaa143ef8eb213ba4133ac86d8b610a4bcf
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 4475d993aa)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
S2MPU Second-level Memory Protection Table is a PA-contiguous buffer
containing an array of 2-bit read/write entries at given granularity
for a given gigabyte physical address space region. The size of SMPT
varies per granularity but at the finest 4K granularity it is 64KB
PA-contiguous, aligned to 64KB.
Allocate sufficient number of SMPT buffers for the S2MPU driver assuming
4K granularity for 4K/16K PAGE_SIZE, and 64K granularity for 64K
PAGE_SIZE. We also assume that all S2MPUs share SMPTs for a given
gigabyte region. There are 34 gigabyte regions that can be set by the
driver (GBs 4-33 always block all traffic).
Hyp takes ownership of the memory in s2mpu_init and assigns pointers to
the buffers to L1ENTRY_L2TABLE_ADDR registers on init and power-on
events. The pointers remain static as the driver will only change
granularity between 1G and 4K/64K (depending on PAGE_SIZE).
Test: builds, boots
Bug: 190463801
Change-Id: I3fcad8b3ce5d194a987b09d042bd56d59bb35e5e
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit f0e1de52ef)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Intercept SMCs known to be used by the host to inform EL3 about power
events, either powering SoC blocks on or off.
Test: builds, boots
Bug: 190463801
Change-Id: I306433c8c1b712df24569cbd4dc346f72b4c9650
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 8ca0b34fe4)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Initialize the S2MPU driver in __pkvm_init_stage2_iommu if requested by
the host. The driver sets kvm_iommu_ops and configures all S2MPUs which
are powered on at that point (ie. all S2MPUs on currently supported
devices).
The S2MPU L1ENTRY registers are set to 1G granularity and R/W access.
CTRL0/CTRL1/CFG as set to reasonable defaults, though the code relies on
the reset state blocking all traffic as well.
On fault the S2MPUs are configured to return SLVERR/DECERR (v8/9) to the
master. Interrupts are enabled for all VIDs and trigger an IRQ handler
if EL1 init registered a handler as a result of a DT interrupts entry.
Because the host can configure the SSMTs freely, all permission bits are
configured for all VIDs. For v9 CONTEXT_CFG_VALID_VIDS is set to the
value precomputed at EL1, allocating a context ID to each VID.
Test: builds, boots
Bug: 190463801
Change-Id: I4a824e90b5d474dd83c97ef53e4df3c8b68da6ba
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 8aa6c440da)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Create variables in hyp that will hold the DT information about S2MPUs
to use by hyp at runtime. Copy the information from EL1 to EL2.
The EL1 code computes the size of the data and allocates a sufficient
number of pages, which hyp will later take ownership of.
Test: builds, boots
Bug: 190463801
Change-Id: Ic3d4bfa3ec11f7c2e1b4474910e2f57a62139a75
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit bc80f81582)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The S2MPU can be configured to trigger an interrupt on faults: access
permission (both regular and during page table walks) and if no matching
context ID is found for request's VID (v9 only).
When interrupt information is provided in the S2MPU's DT node, parse the
information and enable an IRQ handler. Later patch will enable the
functionality in the S2MPU.
Test: builds, boots
Bug: 190463801
Change-Id: I11d1a896406011cff1506ee1bd124bfc66ffa914
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 2517c4e5f0)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
S2MPU_CONTEXT_CFG_VALID_VID register must be configured on v9,
allocating a context ID in range 0 to S2MPU_NUM_CONTEXT to each valid
VID. For now assume that all 8 VIDs are valid. This will change once
the hypervisor takes control over SSMT configuration as well.
If there are more VIDs than available context IDs, the driver prints
a warning that DMA may be blocked and continues.
Test: builds, boots
Bug: 190463801
Change-Id: I0c9e0a5c9470b27debaade2c4e02e16c6577fbfe
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 923353be1e)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Read S2MPU_VERSION during driver init and check it against list of
supported versions. The register fields are as follows:
- MAJOR_ARCH_VER,
- MINOR_ARCH_VER,
- REV_ARCH_VER,
- RTL_VER.
Their exact use is not documented. For now, we mask out RTL_VER and
expect a match on MAJOR_, MINOR_ and REV_ARCH_VER. This may be tweaked
in the future.
Test: builds, boots
Bug: 190463801
Change-Id: I9709fde5f4d3ca4c23f84919c37b081302846917
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 4a7da93bdb)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Start EL1 portion of the S2MPU driver with an init function which
probes the Device tree for nodes compatible with 'google,s2mpu'.
Parse and check the base, size and power domain ID.
Test: builds, boots
Bug: 190463801
Change-Id: I5f0b32febb4e922fdfdfe10a9a9c823e20b8e26f
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 4e91a00153)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Create a skeleton driver for the S2MPU - an EL1 portion called during
KVM init which will parse the DT and configure the kernel, and an EL2
portion which will program the S2MPUs later at runtime. The code is
behind CONFIG_KVM_S2MPU.
Test: builds, boots
Bug: 190463801
Change-Id: I58206535f3493e1d989576a9db2112d370a1cb4d
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit b2de5483b7)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add a new kvm_iommu_ops hook to the lower-EL instruction/data abort
handler, which allows the IOMMU driver to restrict the region of device
memory that is about to be mapped in the host stage-2.
This can be used by the IOMMU driver to restrict access to the MMIO
registers of the IOMMU itself.
Test: builds, boots
Bug: 190463801
Change-Id: I51cf3cfd84c889627e290d74579657447964ca16
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit cc1ad46fb2)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add a new kvm_iommu_ops hook which allows the IOMMU driver to handle
data aborts in unmapped device memory regions. If the abort is handled
by the driver, the global abort handler will not attempt to map in the
page.
For example, this enables the IOMMU driver to virtualize access to
the underlying IOMMU hardware, or to allow access to a subset of the
functionality, eg. performance counters.
Test: builds, boots
Bug: 190463801
Change-Id: I84adbc992e577ac6ceb09f4856e1c648df580f76
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 25f81ec77b)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add a new hook to kvm_iommu_ops that is invoked whenever a range of
pages changes their owner in the host stage2. This is currently limited
to finalize_host_mappings, which changes the owner of EL2-mapped pages
from host to hyp.
The driver is expected to apply corresponding changes in the IOMMU it
controls, so that only the new owner can access the page range.
Test: builds, boots
Bug: 190463801
Change-Id: I0809f4859a9117d1a37506b7aa9e19c6bd25ffdb
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 3cd8b5b00b)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
IOMMU drivers need to intercept power management SMCs between the host
and EL3. Add a hook to hyp's 'handle_host_smc'.
Test: builds, boots
Bug: 190463801
Change-Id: Ied34b60d4bb0e5ae0fbf03f8ce1dc22a09679e37
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit d2efcdcb2b)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Bootstrap infrastructure for IOMMU drivers by introducing kvm_iommu_ops
struct in EL2 that is populated based on a iommu_driver parameter to
__pkvm_init hypercall and selected in EL1 early init.
An 'init' operation is called in __pkvm_init_finalise, giving the driver
an opportunity to initialize itself in EL2 and create any EL2 mappings
that it will need. 'init' is specifically called before
'finalize_host_mappings' so that:
(a) pages mapped by the driver change owner to hyp,
(b) ownership changes in 'finalize_host_mappings' get reflected in
IOMMU mappings (added in a future patch).
Test: builds, boots
Bug: 190463801
Change-Id: I04c9f32c6eda846e6e377cb3d23330eb143b6242
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 79775d0225)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
aosp/2257747 merged v5 of the pKVM hypervisor state series as FROMLIST.
Since then, version 6 was posted and queued by the upstream maintainer:
https://lore.kernel.org/r/166819337067.3836113.13147674500457473286.b4-ty@kernel.org
Rather than revert v5 from android (and the dozens of dependent patches),
snap to v6 so that we're in-sync with upstream.
Bug: 233587962
[willdeacon@: Fix conflicts with 'stage2_mc' introduced by accounting work]
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I137bbd611c180cbe03e63a55705150f8f9c2ae31
Signed-off-by: Quentin Perret <qperret@google.com>
When using nVHE in protected mode, protected memory can be between
host and a guest. Tracking this value is interesting from a debug
perspective, to identify potential leaks.
Keeping the count of memory sharing is easy, each share/unshare will return
to the host where the accounting will take place.
Bug: 222044477
Change-Id: I43dcd258789f79dbfe489e5bf721e606c5e6e022
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When using the nVHE protected mode, the stage-2 page tables are handled by
the hypervisor, but are backed by memory donated by the host. That memory
is accounted during the donation (add to the vCPUs hyp_memcache) under
secondary pagetable stats.
On VM teardown, those pages are mixed with others in the teardown_mc, so use
a separated teardown_stage2_mc to deduct them from accounting after
reclaim.
Bug: 222044477
Change-Id: I2a45ce65c5ce9cf96aabd1b66d6f83ffe4808a0c
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When using nVHE in protected mode, the host allocates memory for the
hypervisor to store shadow structures and the stage-2 page tables. This has
been proven to be an interesting value to follow, for debug and health
purpose. Account for those allocations in bytes, in a newly created VM
statistic "protected_hyp_mem".
It is expected, on VM teardown to reclaim all that memory. Raise a warning
if not all the donations are recovered.
Bug: 222044477
Change-Id: I18657d275f2ced67ceb6d0e4bd5ce41cf1d41dc8
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Force the cast to silence the warning.
We don't have a proper way to dynamically allocate memory at EL2,
and hence no proper way to dynamically allocate percpu fields.
Instead, we rely on memory donated from the host and index it by
hyp_smp_processor_id().
Bug: 258616809
Reported-by: Todd Kjos <tkjos@google.com>
Change-Id: I3f7efd4a41294a6696992ce0a49270c4f9468bb5
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Although FF-A claims to require version v1.2 of SMCCC, in reality the
current set of calls work just fine with v1.1 and some devices ship with
EL3 firmware that advertises this configuration.
Allow pKVM to proxy FF-A calls for these devices by relaxing our SMCCC
version check to permit SMCCC v1.1+
Reported-by: Alan Stokes <alanstokes@google.com>
Bug: 222663556
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I41e9ff35f169df3609acee7bbc67999c1d11c9d1
Signed-off-by: Quentin Perret <qperret@google.com>
As it turns out, the kernel's DMA code doesn't enforce the
SG_MAX_SEGMENTS limit on the number of elements in an sglist, which can
confuse the pKVM FF-A proxy which has a buffer sized to contain a
descriptor of at most SG_MAX_SEGMENTS constituents.
As the number of elements in an sglist doesn't seem to have an actual
upper bound, let's paper over the issue for now by increasing the size
of the pKVM buffer based on empirical 'measurements'. Longer term we
might need to make this value configurable on the kernel's cmdline, or
to rework the FF-A proxy to sanely handle large descriptors, although
this is not clear how at the time of writing.
Bug: 221256863
Signed-off-by: Quentin Perret <qperret@google.com>
Change-Id: If252f01bec8ae71c0fe1f7007a3ca7b037924c84
Signed-off-by: Quentin Perret <qperret@google.com>
FF-A memory descriptors may need to be sent in fragments when they don't
fit in the mailboxes. Doing so involves using the FRAG_TX and FRAG_RX
primitives defined in the FF-A protocol.
Add support in the pKVM FF-A relayer for fragmented descriptors by
monitoring outgoing FRAG_TX transactions and by buffering large
descriptors on the reclaim path.
[ qperret: BACKPORT because I removed the erroneous ANDROID tag from the
patch title posted upstream ]
Bug: 254811097
Co-developed-by: Andrew Walbran <qwandor@google.com>
Change-Id: I701f279cd4820abb0b6d7c2572ee28e0f943edad
Signed-off-by: Andrew Walbran <qwandor@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-13-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
Intecept FFA_MEM_RECLAIM calls from the host and transition the host
stage-2 page-table entries from the SHARED_OWNED state back to the OWNED
state once EL3 has confirmed that the secure mapping has been reclaimed.
Bug: 254811097
Change-Id: I58365e1b3fafa47f290a292fe57f6d2ed7f9091b
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-11-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
Extend pKVM's memory protection code so that we can update the host's
stage-2 page-table to track pages shared with secure world by the host
using FF-A and prevent those pages from being mapped into a guest.
[ qperret: BACKPORT due to context conflicts in mem_protect.c caused by
the presense of guest-related memory transition in the android kernel
(host_donate_guest and friends) ]
Bug: 254811097
Co-developed-by: Andrew Walbran <qwandor@google.com>
Change-Id: Ib4d404cd1d4fa11d7bf8c1d0b8ec00838a8038a0
Signed-off-by: Andrew Walbran <qwandor@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-9-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
The FF-A proxy code needs to allocate its own buffer pair for
communication with EL3 and for forwarding calls from the host at EL1.
Reserve a couple of pages for this purpose and use them to initialise
the hypervisor's FF-A buffer structure.
Bug: 254811097
Co-developed-by: Andrew Walbran <qwandor@google.com>
Change-Id: Id72cd7f59be20eb6d1faa6f1c5e64ecc8debf929
Signed-off-by: Andrew Walbran <qwandor@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-7-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
Probe FF-A during pKVM initialisation so that we can detect any
inconsistencies in the version or partition ID early on.
[ qperret: BACKPORT due to trivial conflict with header includes in
setup.c ]
Bug: 254811097
Change-Id: I7def4c2c497017ba86621bc98298bc65ffdeefae
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-5-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
When KVM is initialised in protected mode, we must take care to filter
certain FFA calls from the host kernel so that the integrity of guest
and hypervisor memory is maintained and is not made available to the
secure world.
As a first step, intercept and block all memory-related FF-A SMC calls
from the host to EL3. This puts the framework in place for handling them
properly.
Bug: 254811097
Co-developed-by: Andrew Walbran <qwandor@google.com>
Change-Id: I5279bce56956c590862a68e8c4803dd2205e3f81
Signed-off-by: Andrew Walbran <qwandor@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-4-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
FF-A function IDs and error codes will be needed in the hypervisor too,
so move to them to the header file where they can be shared. Rename the
version constants with an "FFA_" prefix so that they are less likely
to clash with other code in the tree.
Bug: 254811097
Co-developed-by: Andrew Walbran <qwandor@google.com>
Change-Id: I00ed487279fdfb61ea34ae99140c6fac8ee89187
Signed-off-by: Andrew Walbran <qwandor@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20221116170335.2341003-2-qperret@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
On the guest teardown path, pKVM will zero the pages used to back the
guest shadow data structures before returning them to the host as they
may contain secrets (e.g. in the vCPU registers). However, the zeroing
is done using a cacheable alias, and CMOs are missing, hence giving the
host a potential opportunity to read the original content of the shadow
structs from memory.
Fix this by issuing CMOs after zeroing the pages.
[ qperret@: moved the CMOs to __unmap_donated_memory() to cover all
callers, including the __pkvm_init_vm() error path ]
Bug: 259551298
Change-Id: Id696d47d16e4c3fd870cb70b792eeb7f2282fc78
Signed-off-by: Quentin Perret <qperret@google.com>
If a malicious/compromised host issues a PSCI SYSTEM_RESET call in the
presence of guest-owned pages then the contents of those pages may be
susceptible to cold-reboot attacks.
Use the PSCI MEM_PROTECT call to ensure that volatile memory is wiped by
the firmware if a SYSTEM_RESET occurs while unpoisoned guest pages exist
in the system. Since this call does not offer protection for a "warm"
reset initiated by SYSTEM_RESET2, detect this case in the PSCI relay and
repaint the call to a standard SYSTEM_RESET instead.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 254821051
Change-Id: I5c3dd93bc83ebcd0b6cea2ec734f6e3a77f0064e
Signed-off-by: Will Deacon <willdeacon@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When donating pages to the guest, we only check the first IPA in the
range against the pvmfw loading range. Although this is fine for the
page-at-a-time faulting path, it doesn't fit with the rest of the mem
protection logic, which deals with the possibility of an arbitrarily
sized contiguous address range.
Rework the logic so that we check the whole IPA range during guest
donation and trigger the pvmfw loading path if any of the pages
intersect with the pvmfw region.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 254819795
Change-Id: I6fef9f1898e65a95cab7f6a0ffa8aa422a8d5a91
Signed-off-by: Will Deacon <willdeacon@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When poisoning the pvmfw pages during system reset at EL2, ensure that we
use a writable fixmap mapping rather than the persistent read-only mapping
of the region.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 254819795
Change-Id: I4c8be092d3c822695afd7d03d0d64163664a9f64
Signed-off-by: Will Deacon <willdeacon@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
pkvm_clear_pvmfw_pages() is used to poison the pvmfw pages during reset,
so rename it to pkvm_poison_pvmfw_pages() instead.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 254819795
Change-Id: Ie5b9c90f0707fa81d9099425cff35383bfb0d009
Signed-off-by: Will Deacon <willdeacon@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
hyp_zero_page() is used for poisoning memory, so rename it to
hyp_poison_page() to avoid confusing with the concept of a "zero page"
and make it available outside of mem_protect.c as it will be used to
poison the pvmfw memory in a subsequent patch.
Signed-off-by: Will Deacon <will@kernel.org>
Bug: 254819795
Change-Id: Ia4aec46437db3ffe466ae09bd180392fa06c0b46
Signed-off-by: Will Deacon <willdeacon@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
hyp_fixmap_map() never returns NULL, so remove the redundant checks for
it and simplify the error handling in the callers.
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 254819795
Change-Id: Ie73a97cc3d9bded3750abe6e243003827393ee5e
Signed-off-by: Quentin Perret <qperret@google.com>
