This reverts commit 453a82127f as it
breaks the KABI. It will be reverted the next KABI gate in a week.
Fixes: 8993e6067f ("Linux 5.15.26")
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I77fe1425a713d080dcdec1cea148c25c3783b982
Commit da51406344 ("usb: gadget: rndis: add spinlock for rndis
response list") broke the rndis abi by adding a lock to the device
structure. This is the correct thing to do, to resolve an issue, but
work around this by moving the lock to be a one-lock-per-driver instead
of a per-device lock. This matches the first submission of this commit,
so it still resolves the same problem, while preserving the ABI for now.
Bug: 161946584
Fixes: da51406344 ("usb: gadget: rndis: add spinlock for rndis response list")
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I7c4d32524b7b10f23a15be35f762b1ed9f345b95
Changes in 5.15.26
mm/filemap: Fix handling of THPs in generic_file_buffered_read()
cgroup/cpuset: Fix a race between cpuset_attach() and cpu hotplug
cgroup-v1: Correct privileges check in release_agent writes
x86/ptrace: Fix xfpregs_set()'s incorrect xmm clearing
btrfs: tree-checker: check item_size for inode_item
btrfs: tree-checker: check item_size for dev_item
clk: jz4725b: fix mmc0 clock gating
io_uring: don't convert to jiffies for waiting on timeouts
io_uring: disallow modification of rsrc_data during quiesce
selinux: fix misuse of mutex_is_locked()
vhost/vsock: don't check owner in vhost_vsock_stop() while releasing
parisc/unaligned: Fix fldd and fstd unaligned handlers on 32-bit kernel
parisc/unaligned: Fix ldw() and stw() unalignment handlers
KVM: x86/mmu: make apf token non-zero to fix bug
drm/amd/display: Protect update_bw_bounding_box FPU code.
drm/amd/pm: fix some OEM SKU specific stability issues
drm/amd: Check if ASPM is enabled from PCIe subsystem
drm/amdgpu: disable MMHUB PG for Picasso
drm/amdgpu: do not enable asic reset for raven2
drm/i915: Widen the QGV point mask
drm/i915: Correctly populate use_sagv_wm for all pipes
drm/i915: Fix bw atomic check when switching between SAGV vs. no SAGV
sr9700: sanity check for packet length
USB: zaurus: support another broken Zaurus
CDC-NCM: avoid overflow in sanity checking
netfilter: xt_socket: fix a typo in socket_mt_destroy()
netfilter: xt_socket: missing ifdef CONFIG_IP6_NF_IPTABLES dependency
netfilter: nf_tables_offload: incorrect flow offload action array size
tee: export teedev_open() and teedev_close_context()
optee: use driver internal tee_context for some rpc
ping: remove pr_err from ping_lookup
Revert "i40e: Fix reset bw limit when DCB enabled with 1 TC"
gpu: host1x: Always return syncpoint value when waiting
perf evlist: Fix failed to use cpu list for uncore events
perf data: Fix double free in perf_session__delete()
mptcp: fix race in incoming ADD_ADDR option processing
mptcp: add mibs counter for ignored incoming options
selftests: mptcp: fix diag instability
selftests: mptcp: be more conservative with cookie MPJ limits
bnx2x: fix driver load from initrd
bnxt_en: Fix active FEC reporting to ethtool
bnxt_en: Fix offline ethtool selftest with RDMA enabled
bnxt_en: Fix incorrect multicast rx mask setting when not requested
hwmon: Handle failure to register sensor with thermal zone correctly
net/mlx5: Fix tc max supported prio for nic mode
ice: check the return of ice_ptp_gettimex64
ice: initialize local variable 'tlv'
net/mlx5: Update the list of the PCI supported devices
bpf: Fix crash due to incorrect copy_map_value
bpf: Do not try bpf_msg_push_data with len 0
selftests: bpf: Check bpf_msg_push_data return value
bpf: Fix a bpf_timer initialization issue
bpf: Add schedule points in batch ops
io_uring: add a schedule point in io_add_buffers()
net: __pskb_pull_tail() & pskb_carve_frag_list() drop_monitor friends
nvme: also mark passthrough-only namespaces ready in nvme_update_ns_info
tipc: Fix end of loop tests for list_for_each_entry()
gso: do not skip outer ip header in case of ipip and net_failover
net: mv643xx_eth: process retval from of_get_mac_address
openvswitch: Fix setting ipv6 fields causing hw csum failure
drm/edid: Always set RGB444
net/mlx5e: Fix wrong return value on ioctl EEPROM query failure
drm/vc4: crtc: Fix runtime_pm reference counting
drm/i915/dg2: Print PHY name properly on calibration error
net/sched: act_ct: Fix flow table lookup after ct clear or switching zones
net: ll_temac: check the return value of devm_kmalloc()
net: Force inlining of checksum functions in net/checksum.h
netfilter: nf_tables: unregister flowtable hooks on netns exit
nfp: flower: Fix a potential leak in nfp_tunnel_add_shared_mac()
net: mdio-ipq4019: add delay after clock enable
netfilter: nf_tables: fix memory leak during stateful obj update
net/smc: Use a mutex for locking "struct smc_pnettable"
surface: surface3_power: Fix battery readings on batteries without a serial number
udp_tunnel: Fix end of loop test in udp_tunnel_nic_unregister()
net/mlx5: DR, Cache STE shadow memory
ibmvnic: schedule failover only if vioctl fails
net/mlx5: DR, Don't allow match on IP w/o matching on full ethertype/ip_version
net/mlx5: Fix possible deadlock on rule deletion
net/mlx5: Fix wrong limitation of metadata match on ecpf
net/mlx5: DR, Fix the threshold that defines when pool sync is initiated
net/mlx5e: MPLSoUDP decap, fix check for unsupported matches
net/mlx5e: kTLS, Use CHECKSUM_UNNECESSARY for device-offloaded packets
net/mlx5: Update log_max_qp value to be 17 at most
spi: spi-zynq-qspi: Fix a NULL pointer dereference in zynq_qspi_exec_mem_op()
gpio: rockchip: Reset int_bothedge when changing trigger
regmap-irq: Update interrupt clear register for proper reset
net-timestamp: convert sk->sk_tskey to atomic_t
RDMA/rtrs-clt: Fix possible double free in error case
RDMA/rtrs-clt: Move free_permit from free_clt to rtrs_clt_close
bnxt_en: Increase firmware message response DMA wait time
configfs: fix a race in configfs_{,un}register_subsystem()
RDMA/ib_srp: Fix a deadlock
tracing: Dump stacktrace trigger to the corresponding instance
tracing: Have traceon and traceoff trigger honor the instance
iio:imu:adis16480: fix buffering for devices with no burst mode
iio: adc: men_z188_adc: Fix a resource leak in an error handling path
iio: adc: tsc2046: fix memory corruption by preventing array overflow
iio: adc: ad7124: fix mask used for setting AIN_BUFP & AIN_BUFM bits
iio: accel: fxls8962af: add padding to regmap for SPI
iio: imu: st_lsm6dsx: wait for settling time in st_lsm6dsx_read_oneshot
iio: Fix error handling for PM
sc16is7xx: Fix for incorrect data being transmitted
ata: pata_hpt37x: disable primary channel on HPT371
Revert "USB: serial: ch341: add new Product ID for CH341A"
usb: gadget: rndis: add spinlock for rndis response list
USB: gadget: validate endpoint index for xilinx udc
tracefs: Set the group ownership in apply_options() not parse_options()
USB: serial: option: add support for DW5829e
USB: serial: option: add Telit LE910R1 compositions
usb: dwc2: drd: fix soft connect when gadget is unconfigured
usb: dwc3: pci: Add "snps,dis_u2_susphy_quirk" for Intel Bay Trail
usb: dwc3: pci: Fix Bay Trail phy GPIO mappings
usb: dwc3: gadget: Let the interrupt handler disable bottom halves.
xhci: re-initialize the HC during resume if HCE was set
xhci: Prevent futile URB re-submissions due to incorrect return value.
nvmem: core: Fix a conflict between MTD and NVMEM on wp-gpios property
mtd: core: Fix a conflict between MTD and NVMEM on wp-gpios property
driver core: Free DMA range map when device is released
btrfs: prevent copying too big compressed lzo segment
RDMA/cma: Do not change route.addr.src_addr outside state checks
thermal: int340x: fix memory leak in int3400_notify()
staging: fbtft: fb_st7789v: reset display before initialization
tps6598x: clear int mask on probe failure
IB/qib: Fix duplicate sysfs directory name
riscv: fix nommu_k210_sdcard_defconfig
riscv: fix oops caused by irqsoff latency tracer
tty: n_gsm: fix encoding of control signal octet bit DV
tty: n_gsm: fix proper link termination after failed open
tty: n_gsm: fix NULL pointer access due to DLCI release
tty: n_gsm: fix wrong tty control line for flow control
tty: n_gsm: fix wrong modem processing in convergence layer type 2
tty: n_gsm: fix deadlock in gsmtty_open()
pinctrl: fix loop in k210_pinconf_get_drive()
pinctrl: k210: Fix bias-pull-up
gpio: tegra186: Fix chip_data type confusion
memblock: use kfree() to release kmalloced memblock regions
ice: Fix race conditions between virtchnl handling and VF ndo ops
ice: fix concurrent reset and removal of VFs
Linux 5.15.26
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: Ied0cc9bd48b7af71a064107676f37b0dd39ce3cf
Add a sample program that supports various tests that the FIPS
certification lab is required to do on fips140.ko. To do its work it
uses AF_ALG, as well as the /dev/fips140 device node provided by a build
of fips140.ko with CONFIG_CRYPTO_FIPS140_MOD_EVAL_TESTING enabled.
Original commits from android12-5.10:
* 109f31ac23 ("ANDROID: fips140: add userspace interface for evaluation testing")
* a481d43521 ("ANDROID: fips140: refactor and rename fips140_lab_test")
* 3a624c9ccd ("ANDROID: fips140: add show_invalid_inputs command to fips140_lab_util")
Bug: 153614920
Bug: 188620248
Change-Id: Ide1875f39d439c3955d03a5f41160382544d47bd
Signed-off-by: Eric Biggers <ebiggers@google.com>
To meet FIPS 140 requirements, add support for building a kernel module
"fips140.ko" that contains various cryptographic algorithms built from
existing kernel source files. At load time, the module checks its own
integrity and self-tests its algorithms, then registers the algorithms
with the crypto API to supersede the original algorithms provided by the
kernel itself.
[ebiggers: this commit originated from "ANDROID: crypto: fips140 -
perform load time integrity check", but I've folded many later commits
into it to make forward porting easier. See below]
Original commits from android12-5.10:
* 6be141eb36 ("ANDROID: crypto: fips140 - perform load time integrity check")
* 868be244bb ("ANDROID: inject correct HMAC digest into fips140.ko at build time")
* 091338cb39 ("ANDROID: fips140: add missing static keyword to fips140_init()")
* c799c6644b ("ANDROID: fips140: adjust some log messages")
* 92de53472e ("ANDROID: fips140: log already-live algorithms")
* 0af06624ea ("ANDROID: fips140: check for errors from initcalls")
* 634445a640 ("ANDROID: fips140: fix deadlock in unregister_existing_fips140_algos()")
* e886dd4c33 ("ANDROID: fips140: unregister existing DRBG algorithms")
* b7397e89db ("ANDROID: fips140: add power-up cryptographic self-tests")
* 50661975be ("ANDROID: fips140: add/update module help text")
* b397a0387c ("ANDROID: fips140: test all implementations")
* 17ccefe140 ("ANDROID: fips140: use full 16-byte IV")
* 1be58af077 ("ANDROID: fips140: remove non-prediction-resistant DRBG test")
* 2b5843ae2d ("ANDROID: fips140: add AES-CBC-CTS")
* 2ee56aad31 ("ANDROID: fips140: add AES-CMAC")
* 960ebb2b56 ("ANDROID: fips140: add jitterentropy to fips140 module")
* e5b14396f9 ("ANDROID: fips140: take into account AES-GCM not being approvable")
* 52b70d491b ("ANDROID: fips140: use FIPS140_CFLAGS when compiling fips140-selftests.c")
* 6b995f5a54 ("ANDROID: fips140: preserve RELA sections without relying on the module loader")
* e45108ecff ("ANDROID: fips140: block crypto operations until tests complete")
* ecf9341134 ("ANDROID: fips140: remove in-place updating of live algorithms")
* 482b0323cf ("ANDROID: fips140: zeroize temporary values from integrity check")
* 64d769e53f ("ANDROID: fips140: add service indicators")
* 8d7f609cda ("ANDROID: fips140: add name and version, and a function to retrieve them")
* 6b7c37f6c4 ("ANDROID: fips140: use UTS_RELEASE as FIPS version")
* 903e97a0ca ("ANDROID: fips140: refactor evaluation testing support")
* 97fb2104fe ("ANDROID: fips140: add support for injecting integrity error")
* 109f31ac23 ("ANDROID: fips140: add userspace interface for evaluation testing")
Bug: 153614920
Bug: 188620248
Test: tested that the module builds and can be loaded on raven.
Change-Id: I3fde49dbc3d16b149b072a27ba5b4c6219015c94
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
The FIPS140 module incorporates copies of builtin code, which gets
integrity checked at module load time, and registered in a way that
ensures that the integrity checked versions supersede the builtin ones.
These objects are compiled as builtin code, and so their init hooks will
be exported from the binary in the same way as builtin initcalls are,
i.e., annotated with a level that defines the order in which the hooks
are expected to be invoked.
[ebiggers: separated this out from the original commit 6be141eb36
("ANDROID: crypto: fips140 - perform load time integrity check")
from android12-5.10, since this changes an existing file]
Bug: 153614920
Bug: 188620248
Change-Id: Iac5c3d1a8aa031c236e2c78a5b40f3ceb0b77f83
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
may_use_simd() should only be called by code that may use FP/SIMD when
it is available, and so checking whether the system supports FP/SIMD in
the first place should be redundant - the caller in question (e.g., a
SIMD crypto algorithm) should never be initialized in the first place.
Checking the system capability involves jump labels and therefore code
patching, which interferes with our ability to perform an integrity
check on some of the crypto code. So let's get rid of the capability
check altogether.
Bug: 153614920
Bug: 188620248
Change-Id: Ia8df624f4648cc980a12a44eeb82e8f186d5f961
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
The arm64 LSE atomics implementation uses both alternatives patching and
jump label patching, both of which need to be selectively disabled when
building the FIPS140 module, or the hashing of the .text section no
longer works.
We already disable jump labels in generic code, but this uncovers a
rather nasty circular include dependency, as the jump label fallback
code uses atomics, which are provided by the LSE code if enabled.
So let's disable LSE as well when building the FIPS140 module: this does
not have any impact on the code, as no code patching goes on in this
module anyway, but it avoids #include hell.
Bug: 153614920
Bug: 188620248
Change-Id: Ia3d823fa3a309777f0c955d619ae8b139dc74061
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
The FIPS140 crypto module takes a HMAC digest of its own .text and
.rodata section in its module_init() hook. This digest is compared to a
digest taken at build time, which means that we need to take some extra
care to ensure that the build time and runtime versions line up.
One thing we cannot tolerate in this case is alternatives patching. In
the general case, we cannot simply ignore alternatives, but fortunately,
there is only a small subset that actually gets instantiated in the
FIPS140 module, and all of these can be ignored if we are willing to
accept that the FIPS140 module does not support VHE hardware, and does
not work when running with pseudo-NMI support enabled. None of this is
important for the use case targeted by the FIPS140 module, so this is
something we should be able to live with.
Bug: 153614920
Bug: 188620248
Change-Id: Ie6666e01d5524a3c33aa451609bab2f29b612f8c
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
The fips140 module doesn't support jump labels, as they would invalidate
the hash of the .text section. So when building the module, switch to
the generic implementation that does not rely on arch-specific code
patching support.
This fixes a failure in check_fips140_module_hmac() caused by the module
containing a call to crypto_alg_put(), which is an inline function that
calls refcount_dec_and_test(), which on arm64 uses a jump label.
Note that the optimized definition of struct static_key is retained, to
ensure ABI compatibility across the FIPS140 module boundary. To ensure
that static keys and their associated jump labels remain in a consistent
state, the fips140 module will not be able to manipulate static keys,
but only to check their state.
Bug: 153614920
Bug: 188620248
Change-Id: Ie834bbf2eed5d09bfae7f387b711a934bedf390d
Signed-off-by: Eric Biggers <ebiggers@google.com>
[ardb: disable jump labels in generic code not in arm64 arch code]
Signed-off-by: Ard Biesheuvel <ardb@google.com>
In fips140.ko, enable the behavior that the upstream fips_enabled flag
controls, such as the XTS weak key check which apparently is required.
Note that some of this behavior, such as the DRBG continuity check, is
allegedly not required. But to ensure we don't miss anything that was
already handled upstream, it seems best to define fips_enabled to 1. We
can still disable anything that turns out to be problematic.
Bug: 153614920
Bug: 188620248
Change-Id: Idcded9e69e7d7cdf7f2937009af209857b0c08e2
Signed-off-by: Eric Biggers <ebiggers@google.com>
Add vendor hooks that will allow the FIPS140 kernel module to override
the implementations of the AES library routines. The FIPS 140 versions
are identical to the normal ones, but their code and rodata will have been
integrity checked at module load time.
Bug: 153614920
Bug: 188620248
Change-Id: I5711fc42eced903565fd3c8d41ca7cdd82641148
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Add a vendor hook that will allow the FIPS140 kernel module to override
the implementation of the sha256() library routine. The FIPS 140 version
is identical to the normal one, but its code and rodata will have been
integrity checked at module load time.
Bug: 153614920
Bug: 188620248
Change-Id: I8ccc4f0cc8206af39fa922134b438dacac2a614a
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Permit the use of AR archives in .a format as input to the partial link
that produces a kernel module. This permits a set of builtin objects to
be bundled with a module object, to create a single module carrying the
payload of several modules. This is used by the FIPS 140 module.
Bug: 153614920
Bug: 188620248
Change-Id: I7183e6922a03aed498f947062bf0d36709371294
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
To meet FIPS requirements, fips140.ko must check its own integrity at
load time. This requires that it know where its .text and .rodata
sections are. To allow this, make the module linker script support
defining symbols that enclose these sections.
[ebiggers: Separated this out from the original commit
"ANDROID: crypto: fips140 - perform load time integrity check" and
folded in two later changes to this script. See below.]
Original commits from android12-5.10:
* 6be141eb36 ("ANDROID: crypto: fips140 - perform load time integrity check")
* e8d56bd78b ("ANDROID: module: apply special LTO treatment to .text even if CFI is disabled")
* 109f31ac23 ("ANDROID: fips140: add userspace interface for evaluation testing")
Bug: 153614920
Bug: 188620248
Change-Id: I22209ff4e6444f9115eca6909bcb653fd5d14aec
Signed-off-by: Ard Biesheuvel <ardb@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Calls into the skcipher API can only occur from contexts where the SIMD
unit is available, so there is no need for the SIMD helper.
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
(cherry picked from commit 676e508122)
Bug: 153614920
Bug: 188620248
Change-Id: Ie243ad1065273bca7583713ed47af184331ea976
Signed-off-by: Eric Biggers <ebiggers@google.com>
Calls into the skcipher API can only occur from contexts where the SIMD
unit is available, so there is no need for the SIMD helper.
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
(cherry picked from commit 96c34e1436)
Bug: 153614920
Bug: 188620248
Change-Id: Id2466cd3ba7e3d3f6fdcf18a08de82823be7d5d0
Signed-off-by: Eric Biggers <ebiggers@google.com>
As pointed out by Evgenii Stepanov one potential issue with the new ABI for
enabling asymmetric is that if there are multiple places where MTE is
configured in a process, some of which were compiled with the old prctl.h
and some of which were compiled with the new prctl.h, there may be problems
keeping track of which MTE modes are requested. For example some code may
disable only sync and async modes leaving asymmetric mode enabled when it
intended to fully disable MTE.
In order to avoid such mishaps remove asymmetric mode from the prctl(),
instead implicitly allowing it if both sync and async modes are requested.
This should not disrupt userspace since a process requesting both may
already see a mix of sync and async modes due to differing defaults between
CPUs or changes in default while the process is running but it does mean
that userspace is unable to explicitly request asymmetric mode without
changing the system default for CPUs.
Reported-by: Evgenii Stepanov <eugenis@google.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Evgenii Stepanov <eugenis@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Joey Gouly <joey.gouly@arm.com>
Cc: Branislav Rankov <branislav.rankov@arm.com>
Link: https://lore.kernel.org/r/20220309131200.112637-1-broonie@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit cf220ad674
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/mte)
Bug: 217221156
Change-Id: I1170cb5f345243843fdebbc9d6c54c6443d782ae
Signed-off-by: Evgenii Stepanov <eugenis@google.com>
MTE3 adds a new mode which is synchronous for reads but asynchronous for
writes. Document the userspace ABI for this feature, we call the new
mode ASYMM and add a new prctl flag and mte_tcf_preferred value for it.
Signed-off-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20220216173224.2342152-2-broonie@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 3f9ab2a698
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/mte)
Bug: 217221156
Change-Id: I729022914fc51579e8434ada97517672c8bb5a4f
Signed-off-by: Evgenii Stepanov <eugenis@google.com>
This FROMLIST change has been updated. Reverting to be replaced with the
final version FROMGIT.
This reverts commit 926ce98105.
Bug: 217221156
Change-Id: Ieeb144fb99a8d3b82c4d1d0d54cd084a22649878
Signed-off-by: Evgenii Stepanov <eugenis@google.com>
In for_each_object_track we go through meta data of the slab
object in function(fn), and as a result false postive out-of-bound
access is reported by kasan. Fix this by wrapping that function call
with metadata_access_enable/disable.
Bug: 222651868
Fixes: ee8d2c7884 ("ANDROID: mm: add get_each_object_track function")
Change-Id: Ifb4241a9c3e397a52759d467aa267d1297e297dd
Signed-off-by: Vijayanand Jitta <quic_vjitta@quicinc.com>
(cherry picked from commit cd6e5d5d7d)
Add three new symbols to the aarch64 kernel ABI. These are to be
called from vendor modules to register an IOMMU with pKVM and
notify the hypervisor about its PM events.
New symbols:
- pkvm_iommu_s2mpu_register
- pkvm_iommu_suspend
- pkvm_iommu_resume
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I9797326a54cba6abd1b233682379de10139c2303
With new generic IOMMU code in place, and with all S2MPU code
having been migrated to the new pkvm_iommu_ops callbacks, remove
all the now unused code.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I6abc7ef0f400250cbb38a673feb1db35116c3f69
Remove the existing 's2mpu_host_stage2_set_owner' hook implementation
and refactor the code to match the prepare/apply split of the generic
IOMMU callbacks for updating host stage-2 mappings.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: If550fe2c41198c320559c8125ec9ecc0479eb249
Core SMPT manipulation code returns mpt_update_flags, signalling whether
the caller should flush the dcache (MPT_UPDATE_L2) or write new L1ATTR
values to S2MPU MMIO registers (MPT_UPDATE_L1).
In preparation for splitting the code into a driver-global and
per-device portions, store the value in the corresponding FMPT.
As long as the two code portions are called from a single critical
section, the FMPT value is guaranteed to not change.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Iec06697e8826b0dba682476b39cf64acd6337166
Previously the S2MPU DABT handler would be called directly from the host
DABT handler and it would look up the corresponding S2MPU device. Now the
lookup is done in the generic IOMMU DABT handler and only the actual
S2MPU register access is left to the driver itself.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I5236cf01b9e1dcc65a00081797a13ee92a4e263b
The host is now expected to notify EL2 about PM state changes of
individual IOMMU devices. Remove the old code that intercepted SMCs
and instead rely on callbacks from the core IOMMU code.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I2dd49836c01e405562ba62c00efc711f084d5963
Create 'struct pkvm_iommu_ops' for the S2MPU and a new driver ID to the
list of IOMMU drivers. Implement the 'init' callback, accepting donated
memory from the host to back SMPTs. If the donation is successful,
the SMPTs are assigned to 'host_mpt'.
Export 'pkvm_iommu_s2mpu_register' for a kernel module to call to
register an S2MPU device. First call to this function will also
run the global S2MPU driver initializer.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I3d1aaf8535114beae956993674a2b436414f07c4
The function is superseded by the generic
pkvm_iommu_host_stage2_adjust_range, remove it.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I7d138a3c2e2497bdc19e6e6e95b7870ac48d890e
Replace all uses of 'struct s2mpu' with the generic 'struct pkvm_iommu'.
'struct s2mpu_drv_data' is created to accommodate driver-specific values
associated with 'struct pkvm_iommu' and allocated by the generic code.
These changes are safe because the S2MPU code is currently unused.
The EL1 code that initialized it had been removed.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ib12b64ffa3281be83440e33cdd032d80df6e4868
EL2 S2MPU driver relied on EL1 code which parsed the DT and populated
EL2 driver data before deprivileging of the host. The driver is now
moving to later initialization from kernel modules, which will take over
the role of parsing the DT and power management. Remove the unused code.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I96542ceeec4fcf1040658779a922363b1e41e976
S2MPU code previously assumed that all S2MPUs were powered on at boot
and would check the version register and precompute the value of
S2MPU.CONTEXT_CFG_VALID_VID.
With EL1 S2MPU code being removed, and to allow for S2MPUs not powered
at boot, move the code to EL2 and run it on resume.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ib3c3926c1ed3b78fe39769758a8d66490963d4c5
IOMMU drivers may need to keep their own state of the host stage-2
mappings, eg. because they cannot share the PTs with the CPU. To this
end, walk the host stage-2 at driver init time and pass the current
state of host stage-2 mappings to the driver.
The driver initialization lock is released together with host_kvm
lock. That was the driver starts receiving stage-2 updates immediately
after the snapshot is taken.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I62c54c43d2e165d4abab5efbe14e6ea2589c9ed0
Add IOMMU callbacks for host stage-2 idmap changes.
'host_stage2_idmap_prepare' is called first and is expected to apply
the changes on the driver level, eg. update driver-specific page table
information. If successful, the generic code invokes
'host_stage2_idmap_apply' on each currently powered IOMMU device
associated with the driver to apply the changes.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Icf0d7b9c4b5b7219074b54c961db2fe85561d114
Replace the 'host_mmio_dabt_handler' hook in kvm_iommu_ops with
an equivalent callback in the new pkvm_iommu_ops. The generic portion
of the code finds the IOMMU device at the faulted address and invokes
the callback on it.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I44147ceb7877dc1999fd10f4db55659bbbec5bb7
Add suspend/resume callbacks for IOMMU devices. The EL1 kernel driver
is expected to call these when the IOMMU device is powered on but is
about to be used or about to stop being used.
pkvm_iommu_suspend/resume are exported for use by kernel modules.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I5cd38aaeb685bcdae0368453138cc099055adb27
Add '__pkvm_iommu_register' hypcall for registering a new IOMMU device.
The handler allocates a linked-list entry for the device from a memory
pool provided by the host. If the pool has run out, the handler returns
-ENOMEM and expects the host to call it again with a fresh mem pool.
The inputs are validated, eg. ID is unique and memory region does not
overlap with existing IOMMUs. The driver can also implement a 'validate'
callback for driver-specific input validation.
If successful, the handler creates a private EL2 mapping for the device,
forces the memory region is unmapped from host stage-2 and inserts the
device into the linked list. Future attempts to map the MMIO region will
fail because of pkvm_iommu_host_stage2_adjust_range.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: If54ba41cd0b219c6e63508b542d526703ab5b97e
Introduce a linked list of IOMMU devices and
'pkvm_iommu_host_stage2_adjust_range' called from host DABT handler.
The function will adjust the memory range that is about to be mapped
to avoid MMIO regions of all devices in the linked list. If the host
tried to access a device MMIO region, the access is declined.
The function replaces the existing call to
'kvm_iommu.ops.host_stage2_adjust_mmio_range' callback.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ib38256f0005588810a4400efd9a85380d354be59
Add '__pkvm_iommu_driver_init' hypcall and 'struct pkvm_iommu_ops' with
an 'init' callback implemented by an EL2 driver. Driver-specific data
can be passed to 'init' from the host. The memory is pinned while
the callback processed it.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I7cfe51de553e07083747467e1e3ca8bc51737035
The hypervisor has not needed its own .data section because all globals
were either .rodata or .bss. Linked lists are initialized with the head
pointing to itself. To avoid having to work around this by initializing
at runtime, add a .hyp.data section.
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I7a56dc4c93e05bbef53c66837164d17c6103b6b8
The pKVM hypervisor currently zeroes all the pages mapped into guests
when tearing them down for confidentiality reasons. However, for pages
that are shared with the host this is unecessary at best as the content
of memory is already visible. This is particularly bad for non-protected
guests as all their memory is shared with the host by definition.
Add a new flag to distingish pages that solely need to be updated from
an ownership perspective and those that need to be zeroed.
NOTE: We should probably overhaul the teardown procedure at some point
to avoid the proliferation of those flags, but that would require
significant changes so we might not want that in Android 13.
Bug: 223678931
Change-Id: Icefc85a0bdcdf9958e9eb6871c794f68b06a007f
Signed-off-by: Quentin Perret <qperret@google.com>
The pKVM shadow table is protected by 'shadow_lock', however this lock
is only taken across relatively fine-grained calls when inserting and
removing entries from the table. This poses a problem for higher-level
functions such as __pkvm_init_shadow(), where a partially-initialised
shadow entry is made transiently visibly to get_shadow_vcpu() and could
potentially be loaded in an inconsistent state by another CPU.
Push the locking out of the insert/remove functions and up into
__pkvm_{init,teardown}_shadow() so that the shadow state always appears
to be consistent as long as the lock is held.
Bug: 216808671
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: I74c563a539c1ce35f5da86a8281e47c7d435bd27
There's no reason to make the internal shadow table data directly
accessible outside of pkvm.c, so make it all static and provide an
initialisation function to install the initial pages.
Bug: 216808671
Signed-off-by: Will Deacon <willdeacon@google.com>
Change-Id: Idc0908796ebbd2b620494f5d4d6b6055455c8013
Add hooks to gather data of unsual aborts and summarize it with
other information.
Bug: 222638752
Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I74eb36b8551ed9a5e6dc87507939a7f4d81c9c18
Add hooks to gather data of kernel fault and summarize it with
other information.
Bug: 222638752
Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I7d6a66837f2e896a413bd8d878f26928669d96e6
Add hooks to gather data of unfrozen tasks and summarize it
with other information.
Bug: 222638752
Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I61da3d253bd9959c6f06e09c9a35c4b242cedafe
Add hook to gather data of softlockup and summarize it with
other information.
Bug: 222638752
Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I5263bbd573c3fa4b4c981ac26c943721ce09506d
