In the situations where the DWC3 gadget stops active transfers, once
calling the dwc3_gadget_giveback(), there is a chance where a function
driver can queue a new USB request in between the time where the dwc3
lock has been released and re-aquired. This occurs after we've already
issued an ENDXFER command. When the stop active transfers continues
to remove USB requests from all dep lists, the newly added request will
also be removed, while controller still has an active TRB for it.
This can lead to the controller accessing an unmapped memory address.
Fix this by ensuring parameters to prevent EP queuing are set before
calling the stop active transfers API.
Fixes: ae7e86108b ("usb: dwc3: Stop active transfers before halting the controller")
Signed-off-by: Wesley Cheng <wcheng@codeaurora.org>
Link: https://lore.kernel.org/r/1615507142-23097-1-git-send-email-wcheng@codeaurora.org
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit f09ddcfcb8)
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I7b12af154c9c4a0768a30767f4c1abe5b9c27bc7
This reverts commit 5f85626b13.
Temporarily reverting to let partner stabilize. Planning to
re-enable by end of March.
Bug: 182884953
Change-Id: Iffa933d7065d206a73750bd546adc71c637e1581
Signed-off-by: Todd Kjos <tkjos@google.com>
Temporarily reverting to let partner stabilize. Planning to
re-enable by end of March.
Bug: 182884953
Signed-off-by: Todd Kjos <tkjos@google.com>
Change-Id: I83dec946fcb9b6872d3f91aa33afd2ae71a4fe97
This reverts commit 2df558b6f0 ("f2fs: fix to tag FIEMAP_EXTENT_MERGED
in f2fs_fiemap()") as this breaks older Android userspaces without
aosp/1590474.
Bug: 182956687
Change-Id: Iaff9f0f39e03201f198af87c01e6faf38da1b0f0
Signed-off-by: Elliot Berman <eberman@codeaurora.org>
In order to further configure stage-2 page-tables, pass flags to the
init function using a new enum.
The first of these flags allows to disable FWB even if the hardware
supports it as we will need to do so for the host stage-2.
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210317141714.383046-2-qperret@google.com
Bug: 178098380
Change-Id: I19441d59f5bd31f6ffbefbab06c7377e29ad3246
When KVM runs in nVHE protected mode, use the host stage 2 to unmap the
hypervisor sections by marking them as owned by the hypervisor itself.
The long-term goal is to ensure the EL2 code can remain robust
regardless of the host's state, so this starts by making sure the host
cannot e.g. write to the .hyp sections directly.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-37-qperret@google.com
Bug: 178098380
Change-Id: I747a37058f65b9cd5b9ce8dc72a8e385cf68f5d2
The host currently writes directly in EL2 per-CPU data sections from
the PMU code when running in nVHE. In preparation for unmapping the EL2
sections from the host stage 2, disable PMU support in protected mode as
we currently do not have a use-case for it.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-36-qperret@google.com
Bug: 178098380
Change-Id: I4e8e2205484f0375a9a9ea15491f099832a03118
We will soon unmap the .hyp sections from the host stage 2 in Protected
nVHE mode, which obviously works with at least page granularity, so make
sure to align them correctly.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-35-qperret@google.com
Bug: 178098380
Change-Id: Ia631ed37a9cc66d83604fa84f901706800529a94
When KVM runs in protected nVHE mode, make use of a stage 2 page-table
to give the hypervisor some control over the host memory accesses. The
host stage 2 is created lazily using large block mappings if possible,
and will default to page mappings in absence of a better solution.
>From this point on, memory accesses from the host to protected memory
regions (e.g. not 'owned' by the host) are fatal and lead to hyp_panic().
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-34-qperret@google.com
Bug: 178098380
Change-Id: Id663fd6d2c183cf3a44c5677df29b8cab94f5e4e
Since the host stage 2 will be identity mapped, and since it will own
most of memory, it would preferable for performance to try and use large
block mappings whenever that is possible. To ease this, introduce a new
helper in the KVM page-table code which allows to search for large
ranges of available IPA space. This will be used in the host memory
abort path to greedily idmap large portion of the PA space.
Signed-off-by: Quentin Perret <qperret@google.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210315143536.214621-32-qperret@google.com
Bug: 178098380
Change-Id: Ib5a9470a4e7f86e8adfa25d273c56279657c9a22
As the host stage 2 will be identity mapped, all the .hyp memory regions
and/or memory pages donated to protected guestis will have to marked
invalid in the host stage 2 page-table. At the same time, the hypervisor
will need a way to track the ownership of each physical page to ensure
memory sharing or donation between entities (host, guests, hypervisor) is
legal.
In order to enable this tracking at EL2, let's use the host stage 2
page-table itself. The idea is to use the top bits of invalid mappings
to store the unique identifier of the page owner. The page-table owner
(the host) gets identifier 0 such that, at boot time, it owns the entire
IPA space as the pgd starts zeroed.
Provide kvm_pgtable_stage2_set_owner() which allows to modify the
ownership of pages in the host stage 2. It re-uses most of the map()
logic, but ends up creating invalid mappings instead. This impacts
how we do refcount as we now need to count invalid mappings when they
are used for ownership tracking.
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-30-qperret@google.com
Bug: 178098380
Change-Id: I0ade3e55896ec77631f0decb92e1ce494fc936e2
kvm_set_invalid_pte() currently only clears bit 0 from a PTE because
stage2_map_walk_table_post() needs to be able to follow the anchor. In
preparation for re-using bits 63-01 from invalid PTEs, make sure to zero
it entirely by ensuring to cache the anchor's child upfront.
Acked-by: Will Deacon <will@kernel.org>
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-29-qperret@google.com
Bug: 178098380
Change-Id: Ia370eb397c69af690d82ab408d4076c9a5b95382
Extend the memory pool allocated for the hypervisor to include enough
pages to map all of memory at page granularity for the host stage 2.
While at it, also reserve some memory for device mappings.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-27-qperret@google.com
Bug: 178098380
Change-Id: I53188d32b68741e7413fbb8b99ea7b8eb42d454b
The current stage2 page-table allocator uses a memcache to get
pre-allocated pages when it needs any. To allow re-using this code at
EL2 which uses a concept of memory pools, make the memcache argument of
kvm_pgtable_stage2_map() anonymous, and let the mm_ops zalloc_page()
callbacks use it the way they need to.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-26-qperret@google.com
Bug: 178098380
Change-Id: I33c2c84ac77872433bfd9d5bee864d913ba64080
In order to make use of the stage 2 pgtable code for the host stage 2,
change kvm_s2_mmu to use a kvm_arch pointer in lieu of the kvm pointer,
as the host will have the former but not the latter.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-21-qperret@google.com
Bug: 178098380
Change-Id: I1fa990ed0729f9475573bc711dda331dffc76fbb
Previous commits have introduced infrastructure to enable the EL2 code
to manage its own stage 1 mappings. However, this was preliminary work,
and none of it is currently in use.
Put all of this together by elevating the mapping creation at EL2 when
memory protection is enabled. In this case, the host kernel running
at EL1 still creates _temporary_ EL2 mappings, only used while
initializing the hypervisor, but frees them right after.
As such, all calls to create_hyp_mappings() after kvm init has finished
turn into hypercalls, as the host now has no 'legal' way to modify the
hypevisor page tables directly.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-19-qperret@google.com
Bug: 178098380
Change-Id: I0c93c298bc16fc913c6e0faf51c395aa0215c444
When memory protection is enabled, the EL2 code needs the ability to
create and manage its own page-table. To do so, introduce a new set of
hypercalls to bootstrap a memory management system at EL2.
This leads to the following boot flow in nVHE Protected mode:
1. the host allocates memory for the hypervisor very early on, using
the memblock API;
2. the host creates a set of stage 1 page-table for EL2, installs the
EL2 vectors, and issues the __pkvm_init hypercall;
3. during __pkvm_init, the hypervisor re-creates its stage 1 page-table
and stores it in the memory pool provided by the host;
4. the hypervisor then extends its stage 1 mappings to include a
vmemmap in the EL2 VA space, hence allowing to use the buddy
allocator introduced in a previous patch;
5. the hypervisor jumps back in the idmap page, switches from the
host-provided page-table to the new one, and wraps up its
initialization by enabling the new allocator, before returning to
the host.
6. the host can free the now unused page-table created for EL2, and
will now need to issue hypercalls to make changes to the EL2 stage 1
mappings instead of modifying them directly.
Note that for the sake of simplifying the review, this patch focuses on
the hypervisor side of things. In other words, this only implements the
new hypercalls, but does not make use of them from the host yet. The
host-side changes will follow in a subsequent patch.
Credits to Will for __pkvm_init_switch_pgd.
Acked-by: Will Deacon <will@kernel.org>
Co-authored-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-18-qperret@google.com
Bug: 178098380
Change-Id: I039096f049ad3fa083f56e19fb66ea09645d749a
In order to re-map the guest vectors at EL2 when pKVM is enabled,
refactor __kvm_vector_slot2idx() and kvm_init_vector_slot() to move all
the address calculation logic in a static inline function.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-16-qperret@google.com
Bug: 178098380
Change-Id: I52442e4b873cb881b2306dc2aacd3060812f5520
We will need to do cache maintenance at EL2 soon, so compile a copy of
__flush_dcache_area at EL2, and provide a copy of arm64_ftr_reg_ctrel0
as it is needed by the read_ctr macro.
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-15-qperret@google.com
Bug: 178098380
Change-Id: Icdf042dc83a4486b615514a9bd6e27af0859ce75
Introduce the infrastructure in KVM enabling to copy CPU feature
registers into EL2-owned data-structures, to allow reading sanitised
values directly at EL2 in nVHE.
Given that only a subset of these features are being read by the
hypervisor, the ones that need to be copied are to be listed under
<asm/kvm_cpufeature.h> together with the name of the nVHE variable that
will hold the copy. This introduces only the infrastructure enabling
this copy. The first users will follow shortly.
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-14-qperret@google.com
Bug: 178098380
Change-Id: I20b08439434b1f6f1c48230b60ce113063c0757f
When memory protection is enabled, the hyp code will require a basic
form of memory management in order to allocate and free memory pages at
EL2. This is needed for various use-cases, including the creation of hyp
mappings or the allocation of stage 2 page tables.
To address these use-case, introduce a simple memory allocator in the
hyp code. The allocator is designed as a conventional 'buddy allocator',
working with a page granularity. It allows to allocate and free
physically contiguous pages from memory 'pools', with a guaranteed order
alignment in the PA space. Each page in a memory pool is associated
with a struct hyp_page which holds the page's metadata, including its
refcount, as well as its current order, hence mimicking the kernel's
buddy system in the GFP infrastructure. The hyp_page metadata are made
accessible through a hyp_vmemmap, following the concept of
SPARSE_VMEMMAP in the kernel.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-13-qperret@google.com
Bug: 178098380
Change-Id: Id06e90de959a4e23df6d62f9072eab954f5b783f
With nVHE, the host currently creates all stage 1 hypervisor mappings at
EL1 during boot, installs them at EL2, and extends them as required
(e.g. when creating a new VM). But in a world where the host is no
longer trusted, it cannot have full control over the code mapped in the
hypervisor.
In preparation for enabling the hypervisor to create its own stage 1
mappings during boot, introduce an early page allocator, with minimal
functionality. This allocator is designed to be used only during early
bootstrap of the hyp code when memory protection is enabled, which will
then switch to using a full-fledged page allocator after init.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-11-qperret@google.com
Bug: 178098380
Change-Id: Ibe4c77634a293565171dc22b196065699a2a1f06
In order to allow the usage of code shared by the host and the hyp in
static inline library functions, allow the usage of kvm_nvhe_sym() at
EL2 by defaulting to the raw symbol name.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-10-qperret@google.com
Bug: 178098380
Change-Id: If82eb4fadb95772646d2215028dbb70a05a6a671
kvm_call_hyp() has some logic to issue a function call or a hypercall
depending on the EL at which the kernel is running. However, all the
code compiled under __KVM_NVHE_HYPERVISOR__ is guaranteed to only run
at EL2 which allows us to simplify.
Add ifdefery to kvm_host.h to simplify kvm_call_hyp() in .hyp.text.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-9-qperret@google.com
Bug: 178098380
Change-Id: I8bfe2e5f8febeb7f9e75cfe60e18f24af7b9797c
Currently, the hyp code cannot make full use of a bss, as the kernel
section is mapped read-only.
While this mapping could simply be changed to read-write, it would
intermingle even more the hyp and kernel state than they currently are.
Instead, introduce a __hyp_bss section, that uses reserved pages, and
create the appropriate RW hyp mappings during KVM init.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-8-qperret@google.com
Bug: 178098380
Change-Id: Iec51e75deebf8db9feb7358a7c6e4892c2624c5d
In preparation for enabling the creation of page-tables at EL2, factor
all memory allocation out of the page-table code, hence making it
re-usable with any compatible memory allocator.
No functional changes intended.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-7-qperret@google.com
Bug: 178098380
Change-Id: Ic72f50d58498435d3490931a3e1d695cd19d3b9d
Currently, the KVM page-table allocator uses a mix of put_page() and
free_page() calls depending on the context even though page-allocation
is always achieved using variants of __get_free_page().
Make the code consistent by using put_page() throughout, and reduce the
memory management API surface used by the page-table code. This will
ease factoring out page-allocation from pgtable.c, which is a
pre-requisite to creating page-tables at EL2.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-6-qperret@google.com
Bug: 178098380
Change-Id: I998e88dd31decc7dd1141af9dbda5e60c8370baa
Move the initialization of kvm_nvhe_init_params in a dedicated function
that is run early, and only once during KVM init, rather than every time
the KVM vectors are set and reset.
This also opens the opportunity for the hypervisor to change the init
structs during boot, hence simplifying the replacement of host-provided
page-table by the one the hypervisor will create for itself.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-5-qperret@google.com
Bug: 178098380
Change-Id: I83ee2ac4889ce39f8d02da1d9682f813713a1540
We will soon need to synchronise multiple CPUs in the hyp text at EL2.
The qspinlock-based locking used by the host is overkill for this purpose
and relies on the kernel's "percpu" implementation for the MCS nodes.
Implement a simple ticket locking scheme based heavily on the code removed
by commit c11090474d ("arm64: locking: Replace ticket lock implementation
with qspinlock").
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-4-qperret@google.com
Bug: 178098380
Change-Id: Iecc108a89c71689ba281bf3e15dd0b1015595f1a
Pull clear_page(), copy_page(), memcpy() and memset() into the nVHE hyp
code and ensure that we always execute the '__pi_' entry point on the
offchance that it changes in future.
[ qperret: Commit title nits and added linker script alias ]
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210315143536.214621-3-qperret@google.com
Bug: 178098380
Change-Id: Ie4a82f71d87f3151f8d5e1054b0fd1f8ed44d583
Remove the set_sugov_sched_attr hook which is no longer needed with a
modular governor. The IOWait hook must stay, however.
Bug: 171598214
Signed-off-by: Quentin Perret <qperret@google.com>
Change-Id: Ie68df673bc78ca76c90ba1e6c32ecaa4bba10c89
Add logic in enable_thermal_genl_check to avoid
thermal-hal being woken up all the time by on-die sensor.
When the temperature of the on-die sensor thermal_zone updates,
the thermal_zone_device_update() will be called to handle thermal
and then the netlink events will be sent
through thermal_notify_tz_trip_().
But there is currently no way to filter thermal_zone
in the thermal_netlink, so many netlink events will be sent
in a short amount of time. Because the on-die sensors
are very sensitive. Their temperature may jump up and down
quickly in a few ms.
Bug: 170682696
Test: boot and thermal-hal can receive thermal genl events from kernel
Change-Id: Idb3f4b07a2a2740c01d8785910878bfe6edc832d
Signed-off-by: davidchao <davidchao@google.com>
Currently when an MMC host driver declares crypto support, the MMC core
passes a keyslot and 32-bit DUN down to the driver in each request.
This is enough for the "standard" eMMC v5.2 crypto (cqhci-crypto).
However some SoCs don't follow this standard. They don't have keyslots
but rather take keys directly, and they support longer DUNs.
To allow such hardware to be supported, modify the MMC core to pass a
pointer to the bio_crypt_ctx along with each request, replacing the
crypto_enabled and data_unit_num fields. This way is more flexible.
Also update cqhci-crypto accordingly to keep it working.
(Not being sent upstream yet because this isn't really useful by itself;
it would need support for hardware that needs it to be upstreamed too.)
Bug: 180886435
Bug: 182283899
Change-Id: I8faf3135f570ca3f2798cf812b4245a5df5515ba
Signed-off-by: Eric Biggers <ebiggers@google.com>
