The introduction of ufshcd_dme_configure_adapt() refactored out duplication
from the Mediatek and Qualcomm drivers.
Both these implementations had the logic of:
gear_tx == UFS_HS_G4 => PA_INITIAL_ADAPT
gear_tx != UFS_HS_G4 => PA_NO_ADAPT
but now both implementations pass PA_INITIAL_ADAPT as "adapt_val" and if
gear_tx is not UFS_HS_G4 that is replaced with PA_INITIAL_ADAPT. In other
words, it's PA_INITIAL_ADAPT in both above cases.
The result is that e.g. Qualcomm SM8150 has no longer functional UFS, so
adjust the logic to match the previous implementation.
Link: https://lore.kernel.org/r/20201121044810.507288-1-bjorn.andersson@linaro.org
Fixes: fc85a74e28 ("scsi: ufs: Refactor ADAPT configuration function")
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit 66df79ccbc)
Bug: 204438323
Change-Id: Ibb643d41f8d27a6982fac7ce301b6debdc72dbd2
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Nowadays many vendors initialize their device parameters in their own
vendor drivers. The initialization code is almost the same as well as the
pre-defined definitions. Introduce a common device parameter initialization
function which assign the most common initial values. With this function,
we could remove those duplicated codes in vendor drivers.
Link: https://lore.kernel.org/r/20201116065054.7658-3-stanley.chu@mediatek.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit 65858014ee)
Bug: 204438323
Change-Id: Ib64ea3e3e45e9702c8416807884442eefef4f486
Signed-off-by: Bart Van Assche <bvanassche@google.com>
DeepSleep is a UFS v3.1 feature that achieves the lowest power consumption
of the device, apart from power off.
In DeepSleep mode, no commands are accepted, and the only way to exit is
using a hardware reset or power cycle.
This patch assumes that if a power cycle was an option, then power off
would be preferable, so only exit via a hardware reset is supported.
Drivers that wish to support DeepSleep need to set a new capability flag
UFSHCD_CAP_DEEPSLEEP and provide a hardware reset via the existing
->device_reset() callback.
It is assumed that UFS devices with wspecversion >= 0x310 support
DeepSleep.
[mkp: dropped sysfs ABI doc due to conflicts]
Link: https://lore.kernel.org/r/20201103141403.2142-2-adrian.hunter@intel.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Can Guo <cang@codeaurora.org>
Acked-by: Jonathan Corbet <corbet@lwn.net>
Acked-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit fe1d4c2ebc)
Bug: 204438323
Change-Id: I054995d84b1d92b696ab9d8dda0946c1bbe47f92
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Android commit d9ade6f2d9 ("FROMGIT: ufs: ufs-exynos: use
UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE") differs from the upstream
commit f1ef9047aa ("scsi: ufs: ufs-exynos: Use
UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE"). Additionally, that upstream
commit has already been backported. See also Android commit
e74b237ef9 ("scsi: ufs: ufs-exynos: Use
UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE"). Hence this revert.
Bug: 204438323
Change-Id: I61242407e621e7f963a32d83daf63b30efddbfd9
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Align a structure definition with the upstream code. See also upstream
commit 638e6271ca ("scsi: ufs-mediatek: Add HS-G4 support").
Bug: 204438323
Change-Id: Ie4eac4b7088950fe813ab7236718ea872a91c085
Signed-off-by: Bart Van Assche <bvanassche@google.com>
This patch only changes whitespace. See also commit 4db7a23605 ("scsi:
ufs: Fix concurrency of error handler and other error recovery paths").
Bug: 204438323
Change-Id: I18e1bf734f3f2b2a3451c62184d5177a2cb28a6c
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Move the ufshcd_uic_hibern8_enter() and ufshcd_uic_hibern8_exit()
declarations to where these occur in the upstream code.
Bug: 204438323
Change-Id: Ia7fbee3828baefe53e11f7189522abe129479f43
Signed-off-by: Bart Van Assche <bvanassche@google.com>
.... so it can be referenced in mixed builds.
Test: build cuttlefish
Bug: 202075496
Change-Id: I8d79847c54c639fa619edf3280c021f02ba76645
Signed-off-by: Yifan Hong <elsk@google.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
kvm_fixed_config.h is pkvm specific, and would be better placed
near its users. At the same time, include/nvhe/sys_regs.h is now
almost empty.
Merge the two into arch/arm64/kvm/hyp/include/nvhe/fixed_config.h.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-9-maz@kernel.org
(cherry picked from commit 3061725d16)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ic49cfff8fd36a7b3b59d398ff1a17d32ccef9966
Rather than exposing a whole set of helper functions to retrieve
individual ID registers, use the existing decoding tree and expose
a single helper instead.
This allow a number of functions to be made static, and we now
have a single entry point to maintain.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-3-maz@kernel.org
(cherry picked from commit ce75916749)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I157fad2bc3ed6153f8adf88c99fcc0970fdff449
The previous rework of the early exit code to provide an EC-based
decoding tree missed the fact that we have two trap paths for
ptrauth: the instructions (EC_PAC) and the sysregs (EC_SYS64).
Rework the handlers to call the ptrauth handling code on both
paths.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-2-maz@kernel.org
(cherry picked from commit 8a049862c3)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I4c0637e413db9a5798e7739289b246553b1c1c52
Inspired by commit 254272ce65 ("kvm: x86: Add memcg accounting to KVM
allocations"), it would be better to make arm64 KVM consistent with
common kvm codes.
The memory allocations of VM scope should be charged into VM process
cgroup, hence change GFP_KERNEL to GFP_KERNEL_ACCOUNT.
There remain a few cases since these allocations are global, not in VM
scope.
Signed-off-by: Jia He <justin.he@arm.com>
Reviewed-by: Oliver Upton <oupton@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210907123112.10232-3-justin.he@arm.com
(cherry picked from commit 115bae923a)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I54f9a681446a07e0f0319ed8a07fb641ce1c7dcc
Inspired by commit 254272ce65 ("kvm: x86: Add memcg accounting to KVM
allocations"), it would be better to make arm64 vgic consistent with
common kvm codes.
The memory allocations of VM scope should be charged into VM process
cgroup, hence change GFP_KERNEL to GFP_KERNEL_ACCOUNT.
There remain a few cases since these allocations are global, not in VM
scope.
Signed-off-by: Jia He <justin.he@arm.com>
Reviewed-by: Oliver Upton <oupton@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210907123112.10232-2-justin.he@arm.com
(cherry picked from commit 3ef231670b)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Id635a38791f61c7c04a9a1d255f0418e7da2bac8
Having realised that a virtual LPI does transition through an active
state that does not exist on bare metal, align the CPU interface
emulation with the behaviour specified in the architecture pseudocode.
The LPIs now transition to active on IAR read, and to inactive on
EOI write. Special care is taken not to increment the EOIcount for
an LPI that isn't present in the LRs.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-6-maz@kernel.org
(cherry picked from commit 9d449c71bd)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I93a517888cbcd56560920cb2cb9d671542591d7e
On systems that advertise ICH_VTR_EL2.SEIS, we trap all GICv3 sysreg
accesses from the guest. From a performance perspective, this is OK
as long as the guest doesn't hammer the GICv3 CPU interface.
In most cases, this is fine, unless the guest actively uses
priorities and switches PMR_EL1 very often. Which is exactly what
happens when a Linux guest runs with irqchip.gicv3_pseudo_nmi=1.
In these condition, the performance plumets as we hit PMR each time
we mask/unmask interrupts. Not good.
There is however an opportunity for improvement. Careful reading
of the architecture specification indicates that the only GICv3
sysreg belonging to the common group (which contains the SGI
registers, PMR, DIR, CTLR and RPR) that is allowed to generate
a SError is DIR. Everything else is safe.
It is thus possible to substitute the trapping of all the common
group with just that of DIR if it supported by the implementation.
Yes, that's yet another optional bit of the architecture.
So let's just do that, as it leads to some impressive result on
the M1:
Without this change:
bash-5.1# /host/home/maz/hackbench 100 process 1000
Running with 100*40 (== 4000) tasks.
Time: 56.596
With this change:
bash-5.1# /host/home/maz/hackbench 100 process 1000
Running with 100*40 (== 4000) tasks.
Time: 8.649
which is a pretty convincing result.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Link: https://lore.kernel.org/r/20211010150910.2911495-4-maz@kernel.org
(cherry picked from commit 0924729b21)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I93eeac29ea47f1f03d0dc4dd97efe9e2ec18db58
The infamous M1 has a feature nobody else ever implemented,
in the form of the "GIC locally generated SError interrupts",
also known as SEIS for short.
These SErrors are generated when a guest does something that violates
the GIC state machine. It would have been simpler to just *ignore*
the damned thing, but that's not what this HW does. Oh well.
This part of of the architecture is also amazingly under-specified.
There is a whole 10 lines that describe the feature in a spec that
is 930 pages long, and some of these lines are factually wrong.
Oh, and it is deprecated, so the insentive to clarify it is low.
Now, the spec says that this should be a *virtual* SError when
HCR_EL2.AMO is set. As it turns out, that's not always the case
on this CPU, and the SError sometimes fires on the host as a
physical SError. Goodbye, cruel world. This clearly is a HW bug,
and it means that a guest can easily take the host down, on demand.
Thankfully, we have seen systems that were just as broken in the
past, and we have the perfect vaccine for it.
Apple M1, please meet the Cavium ThunderX workaround. All your
GIC accesses will be trapped, sanitised, and emulated. Only the
signalling aspect of the HW will be used. It won't be super speedy,
but it will at least be safe. You're most welcome.
Given that this has only ever been seen on this single implementation,
that the spec is unclear at best and that we cannot trust it to ever
be implemented correctly, gate the workaround solely on ICH_VTR_EL2.SEIS
being set.
Tested-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-3-maz@kernel.org
(cherry picked from commit df652bcf11)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Icee1ac199b1e6850d6d487d386f2e6e6929c19ca
Until now, we always let ID_AA64PFR0_EL1.GIC reflect the value
visible on the host, even if we were running a GICv2-enabled VM
on a GICv3+compat host.
That's fine, but we also now have the case of a host that does not
expose ID_AA64PFR0_EL1.GIC==1 despite having a vGIC. Yes, this is
confusing. Thank you M1.
Let's go back to first principles and expose ID_AA64PFR0_EL1.GIC=1
when a GICv3 is exposed to the guest. This also hides a GICv4.1
CPU interface from the guest which has no business knowing about
the v4.1 extension.
Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-2-maz@kernel.org
(cherry picked from commit 562e530fd7)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Iffb0a1fcd962413e7048f48efb5129c723ebaddc
We currently check SCTLR_EL1.EE when computing the address of
a faulting guest access. However, the fault could have occured at
EL0, in which case the right bit to check would be SCTLR_EL1.E0E.
This is pretty unlikely to cause any issue in practice: You'd have
to have a guest with a LE EL1 and a BE EL0 (or the other way around),
and have mapped a device into the EL0 page tables.
Good luck with that!
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Link: https://lore.kernel.org/r/20211012112312.1247467-1-maz@kernel.org
(cherry picked from commit 69adec18e9)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I63c040ceb6aeabb9bc70a51f43f2136ebf381a41
Recent changes to KVM for arm64 has made it impossible for the host to
hibernate or use kexec when protected mode is enabled via the kernel
command line.
There are people who rely on kexec (for example, developers who use kexec
as a quick way to test a new kernel), let's document this change in
behaviour, so it doesn't catch them by surprise and we have a place to
point people to if it does.
Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211011153835.291147-1-alexandru.elisei@arm.com
(cherry picked from commit 53e8ce137f)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I674520b73423e9a7c705788853ad7d9e8f1be2e2
Protected KVM does not support protected AArch32 guests. However,
it is possible for the guest to force run AArch32, potentially
causing problems. Add an extra check so that if the hypervisor
catches the guest doing that, it can prevent the guest from
running again by resetting vcpu->arch.target and returning
ARM_EXCEPTION_IL.
If this were to happen, The VMM can try and fix it by re-
initializing the vcpu with KVM_ARM_VCPU_INIT, however, this is
likely not possible for protected VMs.
Adapted from commit 22f553842b ("KVM: arm64: Handle Asymmetric
AArch32 systems")
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-12-tabba@google.com
(cherry picked from commit 5f39efc420)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I0bf557b32ef0a6bae510e4a97fbdb2ed0359355d
Trap accesses to restricted features for VMs running in protected
mode.
Access to feature registers are emulated, and only supported
features are exposed to protected VMs.
Accesses to restricted registers as well as restricted
instructions are trapped, and an undefined exception is injected
into the protected guests, i.e., with EC = 0x0 (unknown reason).
This EC is the one used, according to the Arm Architecture
Reference Manual, for unallocated or undefined system registers
or instructions.
Only affects the functionality of protected VMs. Otherwise,
should not affect non-protected VMs when KVM is running in
protected mode.
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-11-tabba@google.com
(cherry picked from commit 1423afcb41)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I380d0b748414254a2ae25eb3a10a5c2a34a7c6a9
Protected VMs have more restricted features that need to be
trapped. Moreover, the host should not be trusted to set the
appropriate trapping registers and their values.
Initialize the trapping registers, i.e., hcr_el2, mdcr_el2, and
cptr_el2 at EL2 for protected guests, based on the values of the
guest's feature id registers.
No functional change intended as trap handlers introduced in the
previous patch are still not hooked in to the guest exit
handlers.
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-9-tabba@google.com
(cherry picked from commit 2a0c343386)
[willdeacon@: Resolve conflict with hypercall definitions moving to an enum]
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I16390fa00dbbd8c08e4c84a6a718f8b743212b95
Add system register handlers for protected VMs. These cover Sys64
registers (including feature id registers), and debug.
No functional change intended as these are not hooked in yet to
the guest exit handlers introduced earlier. So when trapping is
triggered, the exit handlers let the host handle it, as before.
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-8-tabba@google.com
(cherry picked from commit 6c30bfb18d)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I6b1f89f181739383c2354dab88f7f8044551febf
Simplify the early exception handling by slicing the gigantic decoding
tree into a more manageable set of functions, similar to what we have
in handle_exit.c.
This will also make the structure reusable for pKVM's own early exit
handling.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211010145636.1950948-4-tabba@google.com
(cherry picked from commit 8fb2046180)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I73dea0f33208fa7ac41a945b46a09b5d4dfcf969
