Extend the scope of the stage2_freewalker by passing the pgt instead of
the mm_ops callbacks. This will later be used by the stage2_pte_is_counted
function.
Bug: 222044487
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Change-Id: I390661eb106cbdb863cbb1832e39ec155c439091
Signed-off-by: Quentin Perret <qperret@google.com>
The recently introduced support for pKVM modules has clearly lacked a
bit of testing with CONFIG_MODULES=n.
Make sure to not access __pkvm_modules_enabled when CONFIG_MODULES=n to
fix the link.
Bug: 244543039
Bug: 244373730
Reported-by: kernel test robot <lkp@intel.com>
Change-Id: I65c6ad7c07c2812b85dcd435b24b3444c0ce7f1e
Signed-off-by: Quentin Perret <qperret@google.com>
The recently introduced support for pKVM modules has clearly lacked a
bit of testing with CONFIG_MODULES=n.
Fix the broken __pkvm_init_module() stub to make the bots happy.
Bug: 244543039
Bug: 244373730
Reported-by: kernel test robot <lkp@intel.com>
Change-Id: Ie59cbf46442721de78ef51523debadc4c156530e
Signed-off-by: Quentin Perret <qperret@google.com>
Disable module loading, based on the cmdline option
kvm-arm.protected_modules. If enabled, the feature can later be
irreversibly disabled with the HVC __pkvm_close_module_registration().
Bug: 244543039
Bug: 244373730
Change-Id: I693a1db350ae15bcaab59103a68b60f087dcc6b4
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Return the fixmap VA with the page offset, instead of the page base
address.
Bug: 244543039
Bug: 229972309
Change-Id: I40c342f84e3cd395156ef846f0434c028d4e3fa3
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When pKVM is in use, allow pKVM modules to register custom hypercall
handlers:
* pkvm_register_el2_call(): Give a handler to the hypervisor and gets in
return the newly registered hypercall number.
* pkvm_el2_mod_call(): Call the previously registered hypercall handler.
There is a limit of 128 hypercalls that can be registered.
Bug: 244543039
Bug: 244373730
Change-Id: I3d6c89675efe5f65f6b53c3b45ae155d1a00164c
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Later introduction of custom HVCs will require to store some data.
Conviniently, the only thing we need is the start addr (in the hyp VA) of
the text section. So use this value as a token to avoid having to store
it anywhere.
Bug: 244543039
Bug: 244373730
Change-Id: Idd0bcbbb36d189aa4932833ca5b40382c2cddb08
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
In preparation for allowing to restrict host permissions at stage-2 for
certain pages, introduce some infrastructure allowing a pKVM module to
register a permission fault handler.
Bug: 244543039
Bug: 244373730
Change-Id: I8035c64969cc0ebb01c8936b1974b3bc103ba84f
Signed-off-by: Quentin Perret <qperret@google.com>
pKVM currently doesn't map pages with reduced permissions in stage2
page-tables, for both host and guests. But in preparation for allowing
that to happen, introduce a new page 'meta-state' that is set whenever a
page has non-default permissions. Introducing it ensures that e.g.
host-to-guest donations or shares will fail for a page that the host has
read-only, as the page will no longer be PKVM_PAGE_OWNED only.
Bug: 244543039
Bug: 244373730
Change-Id: Ib3b20c7edabafb9a305f000a37bbdb4bcc8fdbbb
Signed-off-by: Quentin Perret <qperret@google.com>
Expose the hyp_fixmap helpers in the module_ops struct to allow dynamic
mapping and unmapping of pages from pKVM modules.
Bug: 244543039
Bug: 244373730
Change-Id: I201db6044ed5eb4c2821a64a6b650b931dd2e389
Signed-off-by: Quentin Perret <qperret@google.com>
Expose the hyp_puts() and hyp_putx64() helpers in the module_ops struct
to allow logging messages on the UART from pKVM modules.
Bug: 244543039
Bug: 244373730
Change-Id: Ica578667297e5a1f94c370603c29482be89982a9
Signed-off-by: Quentin Perret <qperret@google.com>
Debugging a hypervisor tends to be trickier than normal system code
such as the kernel. The lack of availability of a UART framework is a
significant contributor to that. In order to address this, introduce a
framework allowing to load serial drivers into the hypervisor.
Bug: 244543039
Bug: 244373730
Change-Id: I2e7a1fd9abc9d5aa9d95f1d271a997d54a8fd582
Signed-off-by: Quentin Perret <qperret@google.com>
pKVM has an internal API allowing to create mappings in the 'private'
range of the hypervisor VA space for which there are no rules
constraining the VA-PA relation (hence comparable to the vmalloc area in
the kernel). This will be a useful API for hypervisor modules, so expose
it in the recently introduced module ops struct.
Bug: 244543039
Bug: 244373730
Change-Id: I2a8f958a02c3c3b9871224b65b00b207820a507a
Signed-off-by: Quentin Perret <qperret@google.com>
Since .hyp.rodata sections of pKVM modules are emitted with SHT_MERGE,
ld.ldd feels free to attempt merging it with other sections.
Unfortunately, the pKVM module linker script doesn't always place them
in output sections, hence causing link failures:
ld.lld: error: drivers/misc/pkvm-pl011/hyp/kvm_nvhe.tmp.o:(.hyp.rodata):
offset is outside the section
In practice, ld.ldd only seems to attempt merging .note.gnu.property with
.hyp.rodata. To work around the problem, make sure to explicitely place
the .note.gnu.property in .hyp.rodata from the start, hence preventing
ld.ldd from trying to optimize further.
A preferable solution would be to teach ld.lld that merging pKVM modules
sections is a bad idea, or to make sure the sections are not emitted
with SHT_MERGE to begin with, but we couldn't find an obvious way to make
that happen. This workaround is nothing more than a pratical compromise.
Bug: 244543039
Reported-by: Will Deacon <will@kernel.org>
Suggested-by: Will Deacon <will@kernel.org>
Change-Id: Iae902bdfd21915f552e218515cd77881a95fef2d
Signed-off-by: Quentin Perret <qperret@google.com>
All nVHE hypervisor code is currently required to be statically linked
into the kernel image. Sadly, scaling pKVM will inevitably require
running _some_ hardware-specific code in the hypervisor due to the
absence of architecture requirements regarding IOMMU implementations or
power management, for example.
In order to address this issue, introduce the ability to load modules
in the pKVM hypervisor at run-time. pKVM modules are expected to be
embedded inside kernel modules, and to be loaded into pKVM when their
kernel counterpart is loaded at EL1. pKVM module loading is defined as a
privileged operation -- all of them must be loaded while the host kernel
is still part of the trusted computing base.
Bug: 244543039
Bug: 244373730
Co-authored-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Change-Id: If8e5d3ac0a2893c892aff09e5b51d3b8e14693f8
Signed-off-by: Quentin Perret <qperret@google.com>
Building and linking hypervisor modules into kernel modules involves
very similar problems to building and linking the KVM nVHE hypervisor
object into the kernel image. In order to re-use the same building
procedure, let's factor out and generalize the nVHE makefile in a way
that allows it to be re-used for module builds.
Bug: 244543039
Bug: 244373730
Change-Id: I89b2630f8cfd3ce5b624300fc277749be4fc9e04
Signed-off-by: Quentin Perret <qperret@google.com>
[vdonnefort@: Merge with updated nvhe/Makefile]
Signed-off-by: Vincent Donnefort <vdonnefort@google.com
Signed-off-by: Quentin Perret <qperret@google.com>
The start and end pointers of the hyp relocs section are currently
specified in the vmlinux linker script. In order to ease re-using the
same relocation procedure for hypervisor modules, emit the delimiters
from the generated hyp-reloc.S file directly.
Bug: 244543039
Bug: 244373730
Change-Id: I845af2d40e1dd13301069537c6325f6a6f381ce4
Signed-off-by: Quentin Perret <qperret@google.com>
In order to allow re-use of the gen-hyprel tool to build hypervisor
modules in the future, move it up to the arm64 tools folder.
Bug: 244543039
Bug: 244373730
Change-Id: I188a2dac1acf4974213499970cc29552807497eb
Signed-off-by: Quentin Perret <qperret@google.com>
The new pkvm_remove_mappings() allows the caller to unmap a memory range
from the hypervisor.
This is to allow later introduction of in-hypervisor tracing.
Bug: 244543039
Bug: 229972309
Change-Id: I9edc7c2ae55c4b7f5d464d26ce3351b5fd4bf9f3
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The process of applying hypervisor relocations involves patching
addresses in the hypervisor object. In the existing KVM nVHE relocation
procedure, the relocations are applied early enough for write-permission
not to be a problem when touching e.g. the rodata section. But applying
relocations on hypervisor modules embedded in a kernel module proves
more challenging, as the kernel module loader will actively map text and
rodata sections read-only. In order to allow patching in those sections,
let's introduce a new helper function using the text fixmap to
temporarily map the relevant pages writable.
Bug: 244543039
Bug: 244373730
Change-Id: I9dcdade1927e5bc121db87bc950fb70a374c44cd
Signed-off-by: Quentin Perret <qperret@google.com>
In order to allow the re-use of __arch64_insn_write() to fixup
relocations in pKVM modules, refactor it into a more abstract
__arch64_text_write() function which also takes a size as parameter.
No functional changes intended.
Bug: 244543039
Bug: 244373730
Change-Id: I60d789d8a4b1271deeb2f6ac6d3b7fc55bdbb465
Signed-off-by: Quentin Perret <qperret@google.com>
Update host_stage2_set_owner_locked() and caller functions
to use enum pkvm_component_id instead of u8, as IDs are not
defined as const variables anymore. and all other functions
and structs use this enum consistently.
Use the correct max number of IDs as it is not calculated from
invalid PTEs anymore.
Test: MicrodroidHostTestCases
Bug: 255731794
Change-Id: Ib45dc34ff9ff20716adb79920f32875034587343
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add S2MPU V9 code with current page table ops and version ops.
Most SMPT_* macros are now function of protection bits
To keep logic modification minimal and avoid duplicate code
SMPT and FMPT function are kept the same and the values that
changed between S2MPU versions are used as variables instead of
macros
Bug: 255731794
Change-Id: I2a1b8bab630032d8c923c23e96e1182ce5f734ff
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
No functional change
This commit defines the new registers, fields and constants of V9 S2MPU
Bug: 255731794
Change-Id: Ic2e0f276d83604efd7534f74de862df05d4d5696
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
No functional change
Rename versions to match MAJOR_ARCH_VER[31:28] in user manual
0x11000000: is v1
0x20000000: is v2
Bug: 255731794
Change-Id: I30762293a3fab8194b616b4520b047baf3a94790
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
As initialization sequence differ between versions, add a struct
version_ops to abstract init operations
This is just a clean before pushing new S2MPU code.
No functional changes.
Bug: 255731794
Change-Id: I67cc2fb351981280e23860d6f83bdc5632f3abc1
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add new a function to explicitly init S2MPU driver from EL1
pkvm_iommu_s2mpu_init. Instead of being called implicitly from
register function, it now should be called from EL1 driver register
function, and it should pass the expected version of S2MPUs.
EL2 driver will work only on a set of compatible S2MPUs at run time
matching the version at init.
Bug: 255731794
Change-Id: I01a39aa357e368b636fe3c9347651e92f3c62fc2
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
No functional change
Move SMPT and MPT functions to io-pgtable-s2mpu.c as they will need to
change later when new version of S2MPU is added
Bug: 255731794
Change-Id: Ie890bd4e085c1e23a0d033147f955ba8789b8a28
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Create a seprate file where S2MPU page table operations are allocated
based on configuration
This will be helpful when supporting S2MPU versions that have
different SMPT and FMPT formats
Bug: 255731794
Change-Id: I0b80a329d4357511baf6847baf84dc4644bb835f
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
This will allow IOMMU drivers to be loaded as EL2 modules
To achieve that, EL2 module init should register and init
the driver using pkvm_iommu_driver_init.
To avoid having to hardcode S2MPU drivers in IOMMU EL2 code,
pkvm_iommu_driver_init will have a ptr to driver struct instead of
id.
Id is removed from pkvm_iommu_driver and the address of the struct
is treated as the id of the driver, the kernel is expected to pass
linear mapping address of the driver and the hypervisor is expected
to use hyp address.
Bug: 249481474
Change-Id: I49db5b5d88a691d66fa2d5302198785da94d70d2
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The PSCI specification defines only a 32-bit function ID for the
MEM_PROTECT call used to protect against cold reboot attacks.
Fix the pKVM hypervisor invocation of MEM_PROTECT to use the 32-bit
function ID instead of the unallocated 64-bit flavour.
[ qperret: dropped the change to include/uapi/linux/psci.h as the 32 bit
variant of the call have been introduced upstream by 3137f2e600
("firmware/psci: Add debugfs support to ease debugging") ]
Bug: 260316363
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I675a57419064f7f006960ca5370e9dc2d5279a90
Signed-off-by: Quentin Perret <qperret@google.com>
This reverts commit 87bcd3edf3.
Reason for revert: Memory reclaim capability will be checked by the
host before configuring the virtio_balloon device.
Bug: 240239989
Change-Id: I03e7c39ec6d671babeace4040138b416c7e201cf
Signed-off-by: Keir Fraser <keirf@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
SysMMU_SYNCs provide an invalidation-complete signal to the S2MPU
driver but the latency can be quite high. Improve this by waiting for
all the SYNCs in parallel - separate the initiation of invalidation
barrier from waiting for completion. This way we initiate invalidation
on all SYNCs first, then wait for all of them to complete.
The previously introduced exponential-backoff only kicks in if the
SYNC_COMP_COMPLETE bit is not set after the parallel invalidation.
Bug: 249161451
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I9d544bc65f8633d376c7ccd65ea23195ca432964
(cherry picked from commit 1da102d4e3d54285f1d2071a7bee79fbb4c9ce71)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add a new callback to pkvm_iommu_ops called after
host_stage2_idmap_apply on all IOMMU devices. This allows the drivers to
complete operations like invalidation in two stages.
Bug: 249161451
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I9c077fd2b18ce54ad67eb34ef16bc94428797419
(cherry picked from commit ce39549d9283e32cee6a476e4603e7b586957ecf)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
When handling host stage-2 faults the hypervisor currently updates the
CPU _and_ IOMMUs page-tables. However, since we currently proactively
map accessible PA ranges into IOMMUs, updating them during stage-2
faults is unnecessary -- it only needs to be done during ownership
transitions. Optimize this by skipping the IOMMU updates from the host
memory abort path, which also reduces contention on the host stage-2
lock during boot and saves up to 1.1 sec of boot time on Pixel 6.
Bug: 232879742
Change-Id: Icb6f1b2d10de8cc2f976bbc94b9b79ad7aaa6135
Signed-off-by: Quentin Perret <qperret@google.com>
(cherry picked from commit 20c6e1ba55)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The SysMMU_SYNC provides an invalidation-complete signal to the
hypervisor. Currently the hypervisor will wait indefinitely for the SYNC
to set the SYNC_COMP_COMPLETE bit. In practice, this case deadlock as
the hypervisor holds the host lock while waiting for the SYNC.
To avoid deadlock, adjust the algorithm to time out after a given number
of reads of the SYNC_COMP register (new constant SYNC_TIMEOUT_BASE).
This can be a small number as most attempts succeed after a single read
of the SFR.
If the wait-loop times out, the hypervisor will try again, multiplying
the maximum number of SFR reads with SYNC_TIMEOUT_MULTIPLIER each time.
This number was selected to grow quickly, in case there is a lot of DMA
traffic that would be slowing down the SYNC request.
Finally, if the hardware does not set the bit even after
SYNC_MAX_RETRIES, the algorithm will give up to avoid deadlock. The
value was selected so that the worst-case time spent in
__wait_for_invalidation_complete() remains tolerable.
Bug: 250727777
Change-Id: I00230e6dd71de12bac223e4fe118806bdc3872f4
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 992b5f98ca)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
To ease debugging, allow the host to read the state of S2MPU's control
registers. These values do not need to be kept secret from the host.
Bug: 190463801
Change-Id: I5df7de81caceaddfaf2bfb0948ff2d167de487c3
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 5d6831add7)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The state of the S2MPU does not need to be kept secret from the host as
it merely reflects the permissions that the host has and knows about.
To make debugging DMA issues easier, allow the host to query entries
from the MPTC cache. This involves writing the set and way IDs of the
query to the READ_MPTC register and then reading the MPTC entry
information from READ_MPTC_TAG_PPN/TAG_OTHERS/DATA. Modify the S2MPU
DABT handler to allow this register access pattern.
Bug: 190463801
Bug: 229793579
Change-Id: I5661c09a284a4796e18b346f9e294ba64d7d8c0c
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit d5c0f0f937)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Allow read-only access to L1ENTRY_ATTR and L1ENTRY_L2TABLE S2MPU
registers. This allows the host to dump the register state for debugging
purposes. It is safe because the state of the S2MPU is known to the host
anyway.
Bug: 190463801
Change-Id: I44b3633dbad3c122ce521c37813dbf2ae690a678
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit e56d9603a6)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
In preparation for adding more entries to the list of S2MPU registers
accessible to the host, refactor the code to use a switch instead of
a series of ifs. No functional change intended.
Bug: 190463801
Change-Id: I9075a7f080b3391c98fa10f26a943bce0b183394
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 96767ad7be)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Extract the L1ENTRY_ATTR_{PRON,GRAN}_MASK constants out of macros that
create the corresponding constants. This will allow EL1 users to use the
masks to get the fields out of register values. Also extract
L1ENTRY_L2TABLE_ADDR_SHIFT for adjusting the L2 table address.
Bug: 190463801
Change-Id: I5ad6657c7b12d10b81217ee80583eade1e17aeb8
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit c43dfe89fe)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Change the permissions that MPTs are initialized with from PROT_NONE to
PROT_RW. No functional change intended as the generic IOMMU code
sets permissions for the entire address space later. This will allow to
optimize boot time by only unmapping pages not available to host.
Bug: 190463801
Bug: 218012133
Change-Id: Id3891f9de3afe88a0008e8488b3172b73fa1bb69
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 174ac5b7c5)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Currently the generic IOMMU code lets the driver initialize its PT and
then invokes callbacks to set the permissions across the entire PA
range. Optimize this by making it a requirement on the driver to
initialize its PTs to all memory owned by the host. snapshot_host_stage2
then only calls the driver's callback for memory regions not owned by
the host.
Bug: 190463801
Bug: 218012133
Change-Id: I99a826d921d494269078c3a84d90323455a0b769
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 4e56697b42)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The second argument of the kvm_pgtable_walker callback was
misinterpreted as the end of the current entry, where in fact it is
the end of the walked memory region. Fix this by computing the end of
the current entry from the start and the level.
This did not affect correctness, as the code iterates linarly over
the entire address space, but it did affect boot time.
Bug: 190463801
Bug: 218012133
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I6d189b87645f47cd215a783c1bc9e1f032ff8c62
(cherry picked from commit 58f8121600)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Add new hypercall that the host can use to inform the hypervisor that
all hypervisor-controlled IOMMUs have been registered and no new
registrations should be allowed. This will typically be called at the
end of kernel module initialization phase.
Bug: 190463801
Change-Id: I124e6693a3ac0b1a81988640fcb9e09f08bf4ccf
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 8fd93b0ef9)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
The IOMMU DABT handler currently checks if the device is considered
powered by hyp before resolving the request. If the power tracking does
not reflect reality, the IOMMU may trigger issues in the host but the
incorrect state prevents it from diagnosing the issue.
Drop the powered check from the generic IOMMU code. The host accessing
the device's SFR means that it assumes it is powered, and individual
drivers can choose to reject that DABT request.
Bug: 224891559
Bug: 190463801
Change-Id: I3418a29c7deb7e3e866f89d933b9dad0aaa06365
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 798c4ea545)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
SysMMU_SYNC devices expose an interface to start a sync counter and
poll its SFR until the device signals that all memory transactions in
flight at the start have drained. This gives the hypervisor a reliable
indicator that S2MPU invalidation has fully completed and all new
transactions will use the new MPTs.
Add a new pKVM IOMMU driver that the host can use to register
SysMMU_SYNCs. Each device is expected to be a supplier to exactly one
S2MPU (parent), but multiple SYNCs can supply a single S2MPU.
To keep things simple, the SYNCs do not implement suspend/resume and are
assumed to follow the power transitions of their parent.
Following an invalidation, the S2MPU driver iterates over its children
and waits for each SYNC to signal that its transactions have drained.
The algorithm currently waits on each SYNC in turn. If latency proves to
be an issue, this could be optimized to initiate a SYNC on all powered
devices before starting to poll.
Bug: 190463801
Change-Id: I0006602bb5a683d39a6542b61b5ece13ebc28c3f
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit 57381d548d)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
In preparation for adding new IOMMU devices that act as suppliers to
others, add the notion of a parent IOMMU device. Such device must be
registered after its parent and the driver of the parent device must
validate the addition.
The relation has no generic implications, it is up to drivers to make
use of it.
Bug: 190463801
Change-Id: I1e4be18a5ad826f84b0ea895129d2ef54ee17f85
Signed-off-by: David Brazdil <dbrazdil@google.com>
(cherry picked from commit e69c61cf4e)
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
