Commit Graph

983942 Commits

Author SHA1 Message Date
Adrian Hunter
c098fbf4cf UPSTREAM: scsi: ufs: Add DeepSleep feature
DeepSleep is a UFS v3.1 feature that achieves the lowest power consumption
of the device, apart from power off.

In DeepSleep mode, no commands are accepted, and the only way to exit is
using a hardware reset or power cycle.

This patch assumes that if a power cycle was an option, then power off
would be preferable, so only exit via a hardware reset is supported.

Drivers that wish to support DeepSleep need to set a new capability flag
UFSHCD_CAP_DEEPSLEEP and provide a hardware reset via the existing
 ->device_reset() callback.

It is assumed that UFS devices with wspecversion >= 0x310 support
DeepSleep.

[mkp: dropped sysfs ABI doc due to conflicts]

Link: https://lore.kernel.org/r/20201103141403.2142-2-adrian.hunter@intel.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Can Guo <cang@codeaurora.org>
Acked-by: Jonathan Corbet <corbet@lwn.net>
Acked-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit fe1d4c2ebc)
Bug: 204438323
Change-Id: I054995d84b1d92b696ab9d8dda0946c1bbe47f92
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:21 -07:00
Asutosh Das
260ba85526 UPSTREAM: scsi: ufs: qcom: Enable aggressive power collapse for ufs HBA
Enabling this capability to let HBA power-collapse more often to save
power.

Link: https://lore.kernel.org/r/1306284ab2215425ca0a3d9c802574cbd6d35ea7.1603825776.git.asutoshd@codeaurora.org
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit 61906fd465)
Bug: 204438323
Change-Id: I03e54c651d7735d9eaeb15dabea1dc6c5c54b413
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:21 -07:00
Bart Van Assche
43031c379c ANDROID: Revert "FROMGIT: ufs: ufs-exynos: use UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE"
Android commit d9ade6f2d9 ("FROMGIT: ufs: ufs-exynos: use
UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE") differs from the upstream
commit f1ef9047aa ("scsi: ufs: ufs-exynos: Use
UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE"). Additionally, that upstream
commit has already been backported. See also Android commit
e74b237ef9 ("scsi: ufs: ufs-exynos: Use
UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE"). Hence this revert.

Bug: 204438323
Change-Id: I61242407e621e7f963a32d83daf63b30efddbfd9
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:21 -07:00
Bart Van Assche
daa0085d45 ANDROID: scsi: ufs: Move the ufs_mtk_host.hw_ver member variable
Align a structure definition with the upstream code. See also upstream
commit 638e6271ca ("scsi: ufs-mediatek: Add HS-G4 support").

Bug: 204438323
Change-Id: Ie4eac4b7088950fe813ab7236718ea872a91c085
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:21 -07:00
Bart Van Assche
27a75f9751 ANDROID: scsi: ufs: Align ufshcd_init() with its upstream equivalent
This patch only changes whitespace. See also commit 4db7a23605 ("scsi:
ufs: Fix concurrency of error handler and other error recovery paths").

Bug: 204438323
Change-Id: I18e1bf734f3f2b2a3451c62184d5177a2cb28a6c
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:21 -07:00
Bart Van Assche
e555992723 ANDROID: scsi: ufs: Adjust ufshcd_uic_hibern8_*() declarations
Move the ufshcd_uic_hibern8_enter() and ufshcd_uic_hibern8_exit()
declarations to where these occur in the upstream code.

Bug: 204438323
Change-Id: Ia7fbee3828baefe53e11f7189522abe129479f43
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:20 -07:00
Yifan Hong
b2a2f767ac ANDROID: Kleaf: common kernel public
.... so it can be referenced in mixed builds.

Test: build cuttlefish
Bug: 202075496
Change-Id: I8d79847c54c639fa619edf3280c021f02ba76645
Signed-off-by: Yifan Hong <elsk@google.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
2021-11-03 18:52:11 +00:00
Marc Zyngier
437b0bbd10 UPSTREAM: KVM: arm64: pkvm: Give priority to standard traps over pvm handling
Checking for pvm handling first means that we cannot handle ptrauth
traps or apply any of the workarounds (GICv3 or TX2 #219).

Flip the order around.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-12-maz@kernel.org
(cherry picked from commit 0730559011)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I89b2cf3dd330ddcde01c8da3dc42aac83be05c0b
2021-11-03 18:31:55 +00:00
Marc Zyngier
a6ee337e90 UPSTREAM: KVM: arm64: pkvm: Pass vpcu instead of kvm to kvm_get_exit_handler_array()
Passing a VM pointer around is odd, and results in extra work on
VHE. Follow the rest of the design that uses the vcpu instead, and
let the nVHE code look into the struct kvm as required.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-11-maz@kernel.org
(cherry picked from commit 0c7639cc83)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I729249b737cdfa9044a062623ebc38a151b65be9
2021-11-03 18:31:55 +00:00
Marc Zyngier
728d7deaaf UPSTREAM: KVM: arm64: pkvm: Move kvm_handle_pvm_restricted around
Place kvm_handle_pvm_restricted() next to its little friends such
as kvm_handle_pvm_sysreg().

This allows to make inject_undef64() static.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-10-maz@kernel.org
(cherry picked from commit 746bdeadc5)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I685939f25878d46c954233763ee5c040b793ab8b
2021-11-03 18:31:55 +00:00
Marc Zyngier
a978296a57 UPSTREAM: KVM: arm64: pkvm: Consolidate include files
kvm_fixed_config.h is pkvm specific, and would be better placed
near its users. At the same time, include/nvhe/sys_regs.h is now
almost empty.

Merge the two into arch/arm64/kvm/hyp/include/nvhe/fixed_config.h.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-9-maz@kernel.org
(cherry picked from commit 3061725d16)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ic49cfff8fd36a7b3b59d398ff1a17d32ccef9966
2021-11-03 18:31:55 +00:00
Marc Zyngier
7a1762a57e UPSTREAM: KVM: arm64: pkvm: Preserve pending SError on exit from AArch32
Don't drop a potential SError when a guest gets caught red-handed
running AArch32 code.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-8-maz@kernel.org
(cherry picked from commit 271b728605)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I088567169eed5d1fc3083e9356bebaa9cef8b120
2021-11-03 18:31:54 +00:00
Marc Zyngier
1d3fbee1fd UPSTREAM: KVM: arm64: pkvm: Handle GICv3 traps as required
Forward accesses to the ICV_*SGI*_EL1 registers to EL1, and
emulate ICV_SRE_EL1 by returning a fixed value.

This should be enough to support GICv3 in a protected guest.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-7-maz@kernel.org
(cherry picked from commit cbca197384)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I558b29e59e2eeb2a5f67ec195a11e938688aaf77
2021-11-03 18:31:54 +00:00
Marc Zyngier
4843dc6299 UPSTREAM: KVM: arm64: pkvm: Drop sysregs that should never be routed to the host
A bunch of system registers (most of them MM related) should never
trap to the host under any circumstance. Keep them close to our chest.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-6-maz@kernel.org
(cherry picked from commit f3d5ccabab)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: If57c2ad25f496e74fdee2e81369e4c430ca78410
2021-11-03 18:31:54 +00:00
Marc Zyngier
be59fc27a6 UPSTREAM: KVM: arm64: pkvm: Drop AArch32-specific registers
All the SYS_*32_EL2 registers are AArch32-specific. Since we forbid
AArch32, there is no need to handle those in any way.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-5-maz@kernel.org
(cherry picked from commit 3c90cb15e2)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I21b0eeb7d9558ccb1711db540988ba45df83a19f
2021-11-03 18:31:54 +00:00
Marc Zyngier
f84594d89a UPSTREAM: KVM: arm64: pkvm: Make the ERR/ERX*_EL1 registers RAZ/WI
The ERR*/ERX* registers should be handled as RAZ/WI, and there
should be no need to involve EL1 for that.

Add a helper that handles such registers, and repaint the sysreg
table to declare these registers as RAZ/WI.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-4-maz@kernel.org
(cherry picked from commit 8ffb418883)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I05e1542f2f6f4aa4fb4f7f273503ddd1799488c2
2021-11-03 18:31:54 +00:00
Marc Zyngier
1d5ab00d4a UPSTREAM: KVM: arm64: pkvm: Use a single function to expose all id-regs
Rather than exposing a whole set of helper functions to retrieve
individual ID registers, use the existing decoding tree and expose
a single helper instead.

This allow a number of functions to be made static, and we now
have a single entry point to maintain.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-3-maz@kernel.org
(cherry picked from commit ce75916749)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I157fad2bc3ed6153f8adf88c99fcc0970fdff449
2021-11-03 18:31:54 +00:00
Marc Zyngier
c450ffa042 UPSTREAM: KVM: arm64: Fix early exit ptrauth handling
The previous rework of the early exit code to provide an EC-based
decoding tree missed the fact that we have two trap paths for
ptrauth: the instructions (EC_PAC) and the sysregs (EC_SYS64).

Rework the handlers to call the ptrauth handling code on both
paths.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211013120346.2926621-2-maz@kernel.org
(cherry picked from commit 8a049862c3)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I4c0637e413db9a5798e7739289b246553b1c1c52
2021-11-03 18:31:54 +00:00
Jia He
e91c432900 UPSTREAM: KVM: arm64: Add memcg accounting to KVM allocations
Inspired by commit 254272ce65 ("kvm: x86: Add memcg accounting to KVM
allocations"), it would be better to make arm64 KVM consistent with
common kvm codes.

The memory allocations of VM scope should be charged into VM process
cgroup, hence change GFP_KERNEL to GFP_KERNEL_ACCOUNT.

There remain a few cases since these allocations are global, not in VM
scope.

Signed-off-by: Jia He <justin.he@arm.com>
Reviewed-by: Oliver Upton <oupton@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210907123112.10232-3-justin.he@arm.com
(cherry picked from commit 115bae923a)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I54f9a681446a07e0f0319ed8a07fb641ce1c7dcc
2021-11-03 18:31:54 +00:00
Jia He
19b11377f8 UPSTREAM: KVM: arm64: vgic: Add memcg accounting to vgic allocations
Inspired by commit 254272ce65 ("kvm: x86: Add memcg accounting to KVM
allocations"), it would be better to make arm64 vgic consistent with
common kvm codes.

The memory allocations of VM scope should be charged into VM process
cgroup, hence change GFP_KERNEL to GFP_KERNEL_ACCOUNT.

There remain a few cases since these allocations are global, not in VM
scope.

Signed-off-by: Jia He <justin.he@arm.com>
Reviewed-by: Oliver Upton <oupton@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210907123112.10232-2-justin.he@arm.com
(cherry picked from commit 3ef231670b)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Id635a38791f61c7c04a9a1d255f0418e7da2bac8
2021-11-03 18:31:53 +00:00
Marc Zyngier
2ae16afb10 UPSTREAM: KVM: arm64: vgic-v3: Align emulated cpuif LPI state machine with the pseudocode
Having realised that a virtual LPI does transition through an active
state that does not exist on bare metal, align the CPU interface
emulation with the behaviour specified in the architecture pseudocode.

The LPIs now transition to active on IAR read, and to inactive on
EOI write. Special care is taken not to increment the EOIcount for
an LPI that isn't present in the LRs.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-6-maz@kernel.org
(cherry picked from commit 9d449c71bd)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I93a517888cbcd56560920cb2cb9d671542591d7e
2021-11-03 18:31:53 +00:00
Marc Zyngier
b092b90a21 UPSTREAM: KVM: arm64: vgic-v3: Don't advertise ICC_CTLR_EL1.SEIS
Since we are trapping all sysreg accesses when ICH_VTR_EL2.SEIS
is set, and that we never deliver an SError when emulating
any of the GICv3 sysregs, don't advertise ICC_CTLR_EL1.SEIS.

Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-5-maz@kernel.org
(cherry picked from commit f87ab68272)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I6b4876d925b148a36b492fc23243d8f36a529fbb
2021-11-03 18:31:53 +00:00
Marc Zyngier
0b95abfaca UPSTREAM: KVM: arm64: vgic-v3: Reduce common group trapping to ICV_DIR_EL1 when possible
On systems that advertise ICH_VTR_EL2.SEIS, we trap all GICv3 sysreg
accesses from the guest. From a performance perspective, this is OK
as long as the guest doesn't hammer the GICv3 CPU interface.

In most cases, this is fine, unless the guest actively uses
priorities and switches PMR_EL1 very often. Which is exactly what
happens when a Linux guest runs with irqchip.gicv3_pseudo_nmi=1.
In these condition, the performance plumets as we hit PMR each time
we mask/unmask interrupts. Not good.

There is however an opportunity for improvement. Careful reading
of the architecture specification indicates that the only GICv3
sysreg belonging to the common group (which contains the SGI
registers, PMR, DIR, CTLR and RPR) that is allowed to generate
a SError is DIR. Everything else is safe.

It is thus possible to substitute the trapping of all the common
group with just that of DIR if it supported by the implementation.
Yes, that's yet another optional bit of the architecture.
So let's just do that, as it leads to some impressive result on
the M1:

Without this change:
	bash-5.1# /host/home/maz/hackbench 100 process 1000
	Running with 100*40 (== 4000) tasks.
	Time: 56.596

With this change:
	bash-5.1# /host/home/maz/hackbench 100 process 1000
	Running with 100*40 (== 4000) tasks.
	Time: 8.649

which is a pretty convincing result.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Link: https://lore.kernel.org/r/20211010150910.2911495-4-maz@kernel.org
(cherry picked from commit 0924729b21)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I93eeac29ea47f1f03d0dc4dd97efe9e2ec18db58
2021-11-03 18:31:53 +00:00
Marc Zyngier
70b935e851 UPSTREAM: KVM: arm64: vgic-v3: Work around GICv3 locally generated SErrors
The infamous M1 has a feature nobody else ever implemented,
in the form of the "GIC locally generated SError interrupts",
also known as SEIS for short.

These SErrors are generated when a guest does something that violates
the GIC state machine. It would have been simpler to just *ignore*
the damned thing, but that's not what this HW does. Oh well.

This part of of the architecture is also amazingly under-specified.
There is a whole 10 lines that describe the feature in a spec that
is 930 pages long, and some of these lines are factually wrong.
Oh, and it is deprecated, so the insentive to clarify it is low.

Now, the spec says that this should be a *virtual* SError when
HCR_EL2.AMO is set. As it turns out, that's not always the case
on this CPU, and the SError sometimes fires on the host as a
physical SError. Goodbye, cruel world. This clearly is a HW bug,
and it means that a guest can easily take the host down, on demand.

Thankfully, we have seen systems that were just as broken in the
past, and we have the perfect vaccine for it.

Apple M1, please meet the Cavium ThunderX workaround. All your
GIC accesses will be trapped, sanitised, and emulated. Only the
signalling aspect of the HW will be used. It won't be super speedy,
but it will at least be safe. You're most welcome.

Given that this has only ever been seen on this single implementation,
that the spec is unclear at best and that we cannot trust it to ever
be implemented correctly, gate the workaround solely on ICH_VTR_EL2.SEIS
being set.

Tested-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-3-maz@kernel.org
(cherry picked from commit df652bcf11)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Icee1ac199b1e6850d6d487d386f2e6e6929c19ca
2021-11-03 18:31:53 +00:00
Marc Zyngier
f4e2ec7389 UPSTREAM: KVM: arm64: Force ID_AA64PFR0_EL1.GIC=1 when exposing a virtual GICv3
Until now, we always let ID_AA64PFR0_EL1.GIC reflect the value
visible on the host, even if we were running a GICv2-enabled VM
on a GICv3+compat host.

That's fine, but we also now have the case of a host that does not
expose ID_AA64PFR0_EL1.GIC==1 despite having a vGIC. Yes, this is
confusing. Thank you M1.

Let's go back to first principles and expose ID_AA64PFR0_EL1.GIC=1
when a GICv3 is exposed to the guest. This also hides a GICv4.1
CPU interface from the guest which has no business knowing about
the v4.1 extension.

Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010150910.2911495-2-maz@kernel.org
(cherry picked from commit 562e530fd7)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Iffb0a1fcd962413e7048f48efb5129c723ebaddc
2021-11-03 18:31:53 +00:00
Marc Zyngier
f7c9b45b28 UPSTREAM: KVM: arm64: Fix reporting of endianess when the access originates at EL0
We currently check SCTLR_EL1.EE when computing the address of
a faulting guest access. However, the fault could have occured at
EL0, in which case the right bit to check would be SCTLR_EL1.E0E.

This is pretty unlikely to cause any issue in practice: You'd have
to have a guest with a LE EL1 and a BE EL0 (or the other way around),
and have mapped a device into the EL0 page tables.

Good luck with that!

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Link: https://lore.kernel.org/r/20211012112312.1247467-1-maz@kernel.org
(cherry picked from commit 69adec18e9)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I63c040ceb6aeabb9bc70a51f43f2136ebf381a41
2021-11-03 18:31:53 +00:00
Alexandru Elisei
54dcb1a625 UPSTREAM: Documentation: admin-guide: Document side effects when pKVM is enabled
Recent changes to KVM for arm64 has made it impossible for the host to
hibernate or use kexec when protected mode is enabled via the kernel
command line.

There are people who rely on kexec (for example, developers who use kexec
as a quick way to test a new kernel), let's document this change in
behaviour, so it doesn't catch them by surprise and we have a place to
point people to if it does.

Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211011153835.291147-1-alexandru.elisei@arm.com
(cherry picked from commit 53e8ce137f)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I674520b73423e9a7c705788853ad7d9e8f1be2e2
2021-11-03 18:31:53 +00:00
Fuad Tabba
e57b02181e UPSTREAM: KVM: arm64: Handle protected guests at 32 bits
Protected KVM does not support protected AArch32 guests. However,
it is possible for the guest to force run AArch32, potentially
causing problems. Add an extra check so that if the hypervisor
catches the guest doing that, it can prevent the guest from
running again by resetting vcpu->arch.target and returning
ARM_EXCEPTION_IL.

If this were to happen, The VMM can try and fix it by re-
initializing the vcpu with KVM_ARM_VCPU_INIT, however, this is
likely not possible for protected VMs.

Adapted from commit 22f553842b ("KVM: arm64: Handle Asymmetric
AArch32 systems")

Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-12-tabba@google.com
(cherry picked from commit 5f39efc420)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I0bf557b32ef0a6bae510e4a97fbdb2ed0359355d
2021-11-03 18:31:52 +00:00
Fuad Tabba
70d78e70b9 UPSTREAM: KVM: arm64: Trap access to pVM restricted features
Trap accesses to restricted features for VMs running in protected
mode.

Access to feature registers are emulated, and only supported
features are exposed to protected VMs.

Accesses to restricted registers as well as restricted
instructions are trapped, and an undefined exception is injected
into the protected guests, i.e., with EC = 0x0 (unknown reason).
This EC is the one used, according to the Arm Architecture
Reference Manual, for unallocated or undefined system registers
or instructions.

Only affects the functionality of protected VMs. Otherwise,
should not affect non-protected VMs when KVM is running in
protected mode.

Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-11-tabba@google.com
(cherry picked from commit 1423afcb41)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I380d0b748414254a2ae25eb3a10a5c2a34a7c6a9
2021-11-03 18:31:52 +00:00
Fuad Tabba
72201fd6ba UPSTREAM: KVM: arm64: Move sanitized copies of CPU features
Move the sanitized copies of the CPU feature registers to the
recently created sys_regs.c. This consolidates all copies in a
more relevant file.

No functional change intended.

Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-10-tabba@google.com
(cherry picked from commit 72e1be120e)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I4138d2c02c7f10427dea8c50b4e205a26d88b99b
2021-11-03 18:31:52 +00:00
Fuad Tabba
055490c66d BACKPORT: KVM: arm64: Initialize trap registers for protected VMs
Protected VMs have more restricted features that need to be
trapped. Moreover, the host should not be trusted to set the
appropriate trapping registers and their values.

Initialize the trapping registers, i.e., hcr_el2, mdcr_el2, and
cptr_el2 at EL2 for protected guests, based on the values of the
guest's feature id registers.

No functional change intended as trap handlers introduced in the
previous patch are still not hooked in to the guest exit
handlers.

Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-9-tabba@google.com
(cherry picked from commit 2a0c343386)
[willdeacon@: Resolve conflict with hypercall definitions moving to an enum]
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I16390fa00dbbd8c08e4c84a6a718f8b743212b95
2021-11-03 18:31:52 +00:00
Fuad Tabba
325eedfbd9 UPSTREAM: KVM: arm64: Add handlers for protected VM System Registers
Add system register handlers for protected VMs. These cover Sys64
registers (including feature id registers), and debug.

No functional change intended as these are not hooked in yet to
the guest exit handlers introduced earlier. So when trapping is
triggered, the exit handlers let the host handle it, as before.

Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-8-tabba@google.com
(cherry picked from commit 6c30bfb18d)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I6b1f89f181739383c2354dab88f7f8044551febf
2021-11-03 18:31:52 +00:00
Fuad Tabba
943e2e3312 UPSTREAM: KVM: arm64: Simplify masking out MTE in feature id reg
Simplify code for hiding MTE support in feature id register when
MTE is not enabled/supported by KVM.

No functional change intended.

Signed-off-by: Fuad Tabba <tabba@google.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-7-tabba@google.com
(cherry picked from commit 16dd1fbb12)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ida53f2393ec89e1f6bb89f6bdffdc6461fadbf5f
2021-11-03 18:31:52 +00:00
Fuad Tabba
6374824968 UPSTREAM: KVM: arm64: Add missing field descriptor for MDCR_EL2
It's not currently used. Added for completeness.

No functional change intended.

Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-6-tabba@google.com
(cherry picked from commit 5386839077)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I5285c895d38977e85311f61d12085d7c04608707
2021-11-03 18:31:52 +00:00
Fuad Tabba
a72e44fd80 UPSTREAM: KVM: arm64: Pass struct kvm to per-EC handlers
We need struct kvm to check for protected VMs to be able to pick
the right handlers for them in subsequent patches.

Signed-off-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211010145636.1950948-5-tabba@google.com
(cherry picked from commit 3b1a690eda)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I806c6b96f894b22da284f3864cd1820c74a59275
2021-11-03 18:31:51 +00:00
Marc Zyngier
23ca5095d5 UPSTREAM: KVM: arm64: Move early handlers to per-EC handlers
Simplify the early exception handling by slicing the gigantic decoding
tree into a more manageable set of functions, similar to what we have
in handle_exit.c.

This will also make the structure reusable for pKVM's own early exit
handling.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211010145636.1950948-4-tabba@google.com
(cherry picked from commit 8fb2046180)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I73dea0f33208fa7ac41a945b46a09b5d4dfcf969
2021-11-03 18:31:51 +00:00
Marc Zyngier
1b22c8d87e UPSTREAM: KVM: arm64: Don't include switch.h into nvhe/kvm-main.c
hyp-main.c includes switch.h while it only requires adjust-pc.h.
Fix it to remove an unnecessary dependency.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211010145636.1950948-3-tabba@google.com
(cherry picked from commit cc1e6fdfa9)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I4ebb380d0938c9c52a28290600834ca564524ba5
2021-11-03 18:31:51 +00:00
Marc Zyngier
307468fab8 UPSTREAM: KVM: arm64: Move __get_fault_info() and co into their own include file
In order to avoid including the whole of the switching helpers
in unrelated files, move the __get_fault_info() and related helpers
into their own include file.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://lore.kernel.org/r/20211010145636.1950948-2-tabba@google.com
(cherry picked from commit 7dd9b5a157)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ibe44e3c8ad84233cbbd4c5d66f42f0e8578d7b86
2021-11-03 18:31:51 +00:00
Alexandru Elisei
43594fbf0b UPSTREAM: KVM: arm64: Replace get_raz_id_reg() with get_raz_reg()
Reading a RAZ ID register isn't different from reading any other RAZ
register, so get rid of get_raz_id_reg() and replace it with get_raz_reg(),
which does the same thing, but does it without going through two layers of
indirection.

No functional change.

Suggested-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211011105840.155815-4-alexandru.elisei@arm.com
(cherry picked from commit ebf6aa8c04)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Idd8725ff57571a1cb1c95a0ddcf35059e4a41ccc
2021-11-03 18:31:51 +00:00
Alexandru Elisei
fcc366aff2 UPSTREAM: KVM: arm64: Use get_raz_reg() for userspace reads of PMSWINC_EL0
PMSWINC_EL0 is a write-only register and was initially part of the VCPU
register state, but was later removed in commit 7a3ba3095a ("KVM:
arm64: Remove PMSWINC_EL0 shadow register"). To prevent regressions, the
register was kept accessible from userspace as Read-As-Zero (RAZ).

The read function that is used to handle userspace reads of this
register is get_raz_id_reg(), which, while technically correct, as it
returns 0, it is not semantically correct, as PMSWINC_EL0 is not an ID
register as the function name suggests.

Add a new function, get_raz_reg(), to use it as the accessor for
PMSWINC_EL0, as to not conflate get_raz_id_reg() to handle other types
of registers.

No functional change intended.

Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211011105840.155815-3-alexandru.elisei@arm.com
(cherry picked from commit 5a43097623)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I6e64f10097c1f1e6094ea0c4c48b03c69fa7053b
2021-11-03 18:31:51 +00:00
Alexandru Elisei
d6905346b5 UPSTREAM: KVM: arm64: Return early from read_id_reg() if register is RAZ
If read_id_reg() is called for an ID register which is Read-As-Zero (RAZ),
it initializes the return value to zero, then goes through a list of
registers which require special handling before returning the final value.

By not returning as soon as it checks that the register should be RAZ, the
function creates the opportunity for bugs, if, for example, a patch changes
a register to RAZ (like has happened with PMSWINC_EL0 in commit
11663111cd), but doesn't remove the special handling from read_id_reg();
or if a register is RAZ in certain situations, but readable in others.

Return early to make it impossible for a RAZ register to be anything other
than zero.

Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211011105840.155815-2-alexandru.elisei@arm.com
(cherry picked from commit 00d5101b25)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I048ae4c92f29e617326807536de760a4e3536f83
2021-11-03 18:31:51 +00:00
Sean Christopherson
a9684b06cd BACKPORT: KVM: arm64: Depend on HAVE_KVM instead of OF
Select HAVE_KVM at all times on arm64, as the OF requirement is
always there (even in the case of an ACPI system, we still depend
on some of the OF infrastructure), and won't fo away.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Acked-by: Will Deacon <will@kernel.org>
[maz: Drop the "HAVE_KVM if OF" dependency, as OF is always there on arm64,
 new commit message]
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210921222231.518092-3-seanjc@google.com
(cherry picked from commit e26bb75aa2)
[willdeacon@: Fixed context conflict in arm64 KVM Kconfig]
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I64236d409454463bf0f2b0a2efb2e5a00d8f3deb
2021-11-03 18:31:51 +00:00
Sean Christopherson
ddd24bfa83 UPSTREAM: KVM: arm64: Unconditionally include generic KVM's Kconfig
Unconditionally "source" the generic KVM Kconfig instead of wrapping it
with KVM=y.  A future patch will select HAVE_KVM so that referencing
HAVE_KVM in common kernel code doesn't break, and because KVM=y and
HAVE_KVM=n is weird.  Source the generic KVM Kconfig unconditionally so
that HAVE_KVM and KVM don't end up with a circular dependency.

Note, all but one of generic KVM's "configs" are of the HAVE_XYZ nature,
and the one outlier correctly takes a dependency on CONFIG_KVM, i.e. the
generic Kconfig is intended to be included unconditionally.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
[maz: made NVHE_EL2_DEBUG depend on KVM]
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210921222231.518092-2-seanjc@google.com
(cherry picked from commit c8f1e96734)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I94e4a0666f5da3ff17ebfa0068eec5a9358556f3
2021-11-03 18:31:50 +00:00
Marc Zyngier
a621a4e6cb UPSTREAM: KVM: arm64: Allow KVM to be disabled from the command line
Although KVM can be compiled out of the kernel, it cannot be disabled
at runtime. Allow this possibility by introducing a new mode that
will prevent KVM from initialising.

This is useful in the (limited) circumstances where you don't want
KVM to be available (what is wrong with you?), or when you want
to install another hypervisor instead (good luck with that).

Reviewed-by: David Brazdil <dbrazdil@google.com>
Acked-by: Will Deacon <will@kernel.org>
Acked-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Andrew Scull <ascull@google.com>
Link: https://lore.kernel.org/r/20211001170553.3062988-1-maz@kernel.org
(cherry picked from commit b6a68b97af)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ie796c716fc7cece906a8cded0ae4652a828988bb
2021-11-03 18:31:50 +00:00
Ricardo Koller
27402d5cc9 UPSTREAM: KVM: arm64: vgic: Drop vgic_check_ioaddr()
There are no more users of vgic_check_ioaddr(). Move its checks to
vgic_check_iorange() and then remove it.

Signed-off-by: Ricardo Koller <ricarkol@google.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211005011921.437353-6-ricarkol@google.com
(cherry picked from commit 96e9038969)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ib6716c451f76bd80d7596f2c9837df4f2ecfcf7a
2021-11-03 18:31:50 +00:00
Ricardo Koller
cb9c7ad480 UPSTREAM: KVM: arm64: vgic-v3: Check ITS region is not above the VM IPA size
Verify that the ITS region does not extend beyond the VM-specified IPA
range (phys_size).

  base + size > phys_size AND base < phys_size

Add the missing check into vgic_its_set_attr() which is called when
setting the region.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211005011921.437353-5-ricarkol@google.com
(cherry picked from commit 2ec02f6c64)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I859d3e9e702b84579e67f2e0e0b8e13e0759b921
2021-11-03 18:31:50 +00:00
Ricardo Koller
42319f7a6e UPSTREAM: KVM: arm64: vgic-v2: Check cpu interface region is not above the VM IPA size
Verify that the GICv2 CPU interface does not extend beyond the
VM-specified IPA range (phys_size).

  base + size > phys_size AND base < phys_size

Add the missing check into kvm_vgic_addr() which is called when setting
the region. This patch also enables some superfluous checks for the
distributor (vgic_check_ioaddr was enough as alignment == size for the
distributors).

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211005011921.437353-4-ricarkol@google.com
(cherry picked from commit c56a87da0a)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ib462da73c1f819152ee799507f037126db02ebbe
2021-11-03 18:31:50 +00:00
Ricardo Koller
de006431c7 UPSTREAM: KVM: arm64: vgic-v3: Check redist region is not above the VM IPA size
Verify that the redistributor regions do not extend beyond the
VM-specified IPA range (phys_size). This can happen when using
KVM_VGIC_V3_ADDR_TYPE_REDIST or KVM_VGIC_V3_ADDR_TYPE_REDIST_REGIONS
with:

  base + size > phys_size AND base < phys_size

Add the missing check into vgic_v3_alloc_redist_region() which is called
when setting the regions, and into vgic_v3_check_base() which is called
when attempting the first vcpu-run. The vcpu-run check does not apply to
KVM_VGIC_V3_ADDR_TYPE_REDIST_REGIONS because the regions size is known
before the first vcpu-run. Note that using the REDIST_REGIONS API
results in a different check, which already exists, at first vcpu run:
that the number of redist regions is enough for all vcpus.

Finally, this patch also enables some extra tests in
vgic_v3_alloc_redist_region() by calculating "size" early for the legacy
redist api: like checking that the REDIST region can fit all the already
created vcpus.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211005011921.437353-3-ricarkol@google.com
(cherry picked from commit 4612d98f58)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I9c5877cb305621d206e36250aa970a55926d4b5e
2021-11-03 18:31:50 +00:00
Ricardo Koller
fdd4c3ca37 UPSTREAM: kvm: arm64: vgic: Introduce vgic_check_iorange
Add the new vgic_check_iorange helper that checks that an iorange is
sane: the start address and size have valid alignments, the range is
within the addressable PA range, start+size doesn't overflow, and the
start wasn't already defined.

No functional change.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Ricardo Koller <ricarkol@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211005011921.437353-2-ricarkol@google.com
(cherry picked from commit f25c5e4daf)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Ic702fe5c28dc1c7a7bc5975703acee084b8a7b6b
2021-11-03 18:31:50 +00:00
Will Deacon
975154d170 UPSTREAM: KVM: arm64: Disable privileged hypercalls after pKVM finalisation
After pKVM has been 'finalised' using the __pkvm_prot_finalize hypercall,
the calling CPU will have a Stage-2 translation enabled to prevent access
to memory pages owned by EL2.

Although this forms a significant part of the process to deprivilege the
host kernel, we also need to ensure that the hypercall interface is
reduced so that the EL2 code cannot, for example, be re-initialised using
a new set of vectors.

Re-order the hypercalls so that only a suffix remains available after
finalisation of pKVM.

Cc: Marc Zyngier <maz@kernel.org>
Cc: Quentin Perret <qperret@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211008135839.1193-7-will@kernel.org
(cherry picked from commit 057bed206f)
Bug: 204960018
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I61e57c7bff456d40866acd01475b2e9ace1e1bd6
2021-11-03 18:31:50 +00:00