S2MPU Second-level Memory Protection Table is a PA-contiguous buffer
containing an array of 2-bit read/write entries at given granularity
for a given gigabyte physical address space region. The size of SMPT
varies per granularity but at the finest 4K granularity it is 64KB
PA-contiguous, aligned to 64KB.
Allocate sufficient number of SMPT buffers for the S2MPU driver assuming
4K granularity for 4K/16K PAGE_SIZE, and 64K granularity for 64K
PAGE_SIZE. We also assume that all S2MPUs share SMPTs for a given
gigabyte region. There are 34 gigabyte regions that can be set by the
driver (GBs 4-33 always block all traffic).
Hyp takes ownership of the memory in s2mpu_init and assigns pointers to
the buffers to L1ENTRY_L2TABLE_ADDR registers on init and power-on
events. The pointers remain static as the driver will only change
granularity between 1G and 4K/64K (depending on PAGE_SIZE).
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: If27436087ebba1dd0a977960d960d5eaff4279fd
Intercept SMCs known to be used by the host to inform EL3 about power
events, either powering SoC blocks on or off.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I936f7aaecf3ec77ec252f067a3c1571aadba4cdb
Initialize the S2MPU driver in __pkvm_init_stage2_iommu if requested by
the host. The driver sets kvm_iommu_ops and configures all S2MPUs which
are powered on at that point (ie. all S2MPUs on currently supported
devices).
The S2MPU L1ENTRY registers are set to 1G granularity and R/W access.
CTRL0/CTRL1/CFG as set to reasonable defaults, though the code relies on
the reset state blocking all traffic as well.
On fault the S2MPUs are configured to return SLVERR/DECERR (v8/9) to the
master. Interrupts are enabled for all VIDs and trigger an IRQ handler
if EL1 init registered a handler as a result of a DT interrupts entry.
Because the host can configure the SSMTs freely, all permission bits are
configured for all VIDs. For v9 CONTEXT_CFG_VALID_VIDS is set to the
value precomputed at EL1, allocating a context ID to each VID.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I20e658a77c93fa18a07e388d13a639dc67e600d8
Create variables in hyp that will hold the DT information about S2MPUs
to use by hyp at runtime. Copy the information from EL1 to EL2.
The EL1 code computes the size of the data and allocates a sufficient
number of pages, which hyp will later take ownership of.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I45ff22ea2049dd285bfcedbad3390baba9fc59a0
The S2MPU can be configured to trigger an interrupt on faults: access
permission (both regular and during page table walks) and if no matching
context ID is found for request's VID (v9 only).
When interrupt information is provided in the S2MPU's DT node, parse the
information and enable an IRQ handler. Later patch will enable the
functionality in the S2MPU.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I5e26a64fc2f09f96f36a93ea9bc5bf3035a71077
S2MPU_CONTEXT_CFG_VALID_VID register must be configured on v9,
allocating a context ID in range 0 to S2MPU_NUM_CONTEXT to each valid
VID. For now assume that all 8 VIDs are valid. This will change once
the hypervisor takes control over SSMT configuration as well.
If there are more VIDs than available context IDs, the driver prints
a warning that DMA may be blocked and continues.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I7a78999b73c4ddf4ff78900a7c91eabe57aad572
Read S2MPU_VERSION during driver init and check it against list of
supported versions. The register fields are as follows:
- MAJOR_ARCH_VER,
- MINOR_ARCH_VER,
- REV_ARCH_VER,
- RTL_VER.
Their exact use is not documented. For now, we mask out RTL_VER and
expect a match on MAJOR_, MINOR_ and REV_ARCH_VER. This may be tweaked
in the future.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I2fbd20ab78a992c8bdb3574a6d480012260c9ded
Start EL1 portion of the S2MPU driver with an init function which
probes the Device tree for nodes compatible with 'google,s2mpu'.
Parse and check the base, size and power domain ID.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ic8b421e20b40b4a9fc5fb268dece00a11e35e3eb
Create a skeleton driver for the S2MPU - an EL1 portion called during
KVM init which will parse the DT and configure the kernel, and an EL2
portion which will program the S2MPUs later at runtime. The code is
behind CONFIG_KVM_S2MPU.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ic6a3460cad69fba673754cc1926a5bac88f1fa17
Add a new kvm_iommu_ops hook to the lower-EL instruction/data abort
handler, which allows the IOMMU driver to restrict the region of device
memory that is about to be mapped in the host stage-2.
This can be used by the IOMMU driver to restrict access to the MMIO
registers of the IOMMU itself.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I58f7b7a78795a3bfc4d77f7128e4fdc1110fde58
Add a new kvm_iommu_ops hook which allows the IOMMU driver to handle
data aborts in unmapped device memory regions. If the abort is handled
by the driver, the global abort handler will not attempt to map in the
page.
For example, this enables the IOMMU driver to virtualize access to
the underlying IOMMU hardware, or to allow access to a subset of the
functionality, eg. performance counters.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ie2e38652f0568c27d30190fcc3879592872863ae
Add a new hook to kvm_iommu_ops that is invoked whenever a range of
pages changes their owner in the host stage2. This is currently limited
to finalize_host_mappings, which changes the owner of EL2-mapped pages
from host to hyp.
The driver is expected to apply corresponding changes in the IOMMU it
controls, so that only the new owner can access the page range.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ic5ca01a56344cd0253bf7b71560f057ba0e54d6b
IOMMU drivers need to intercept power management SMCs between the host
and EL3. Add a hook to hyp's 'handle_host_smc'.
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ief42370ac371f6b3b87a0d46c00acfffece47052
Bootstrap infrastructure for IOMMU drivers by introducing kvm_iommu_ops
struct in EL2 that is populated based on a iommu_driver parameter to
__pkvm_init hypercall and selected in EL1 early init.
An 'init' operation is called in __pkvm_init_finalise, giving the driver
an opportunity to initialize itself in EL2 and create any EL2 mappings
that it will need. 'init' is specifically called before
'finalize_host_mappings' so that:
(a) pages mapped by the driver change owner to hyp,
(b) ownership changes in 'finalize_host_mappings' get reflected in
IOMMU mappings (added in a future patch).
Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Icd496d193c0bbf811406c3a8d4b94610e2a03197
Add show_mem symbol which will be used by the hard-lockup
debugging module to debug_symbols driver.
Bug: 199478662
Signed-off-by: Woody Lin <woodylin@google.com>
Change-Id: I479700e9f1428b4e1192881b4e3b67c9e43afbeb
The Android UFS driver calls the HPB functions in a different way than
the upstream UFS driver. Align the Android UFS driver with the upstream
driver.
Bug: 204438323
Change-Id: Ia98f96c86768db8f995ba31cbac20e76e7201e2b
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Android commit e7d848c921 ("FROMLIST: scsi: ufs: Fix deadlock while
suspending ufs host") disabled runtime power management. Re-enable
runtime power management since runtime power management support has
been fixed after that commit was merged.
Bug: 204438323
Change-Id: I4d473262e90f242bc55eaa07b8a3bf7ab47c1a65
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The upstream commit edc0596cc0 ("scsi: ufs: core: Stop clearing UNIT
ATTENTIONS") removed the ufs_rpmb_wlun_template but its Android backport
not. Hence this patch that removes the ufs_rpmb_wlun_template.
Bug: 204438323
Change-Id: I86206399a9745e26ad5f84c2cac2179a37ba80e7
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Bring the Android code in sync with the upstream code. This patch does
not change any functionality.
Bug: 204438323
Change-Id: I8fe9f9f3c7d39feb1e211a6b6dc00d6e638ef1b4
Signed-off-by: Bart Van Assche <bvanassche@google.com>
There is no blank line in the upstream code above the luns_avail
declaration. Hence this patch.
Bug: 204438323
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I79927cc14cc011c87eb96ca8847f0bd6a046bb40
Callers of ufshcd_reset_and_restore() expect it to return in an operational
state. However, the code only checks direct errors and so the ufshcd_state
may not be UFSHCD_STATE_OPERATIONAL due to error interrupts.
Fix by also checking ufshcd_state, still allowing non-fatal errors which
are left for the error handler to deal with.
Link: https://lore.kernel.org/r/20211002154550.128511-2-adrian.hunter@intel.com
Reviewed-by: Avri altman <avri.altman@wdc.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 54a4045342)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: Ibfa8f43aaa256c96efbfcb8e185067beb655d603
When building an allmodconfig kernel, the following build error shows up:
aarch64-linux-gnu-ld: drivers/scsi/ufs/ufs-hwmon.o: in function `ufs_hwmon_probe':
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:177: undefined reference to `hwmon_device_register_with_info'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:177:(.text+0x510): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_device_register_with_info'
aarch64-linux-gnu-ld: drivers/scsi/ufs/ufs-hwmon.o: in function `ufs_hwmon_remove':
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:195: undefined reference to `hwmon_device_unregister'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:195:(.text+0x5c8): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_device_unregister'
aarch64-linux-gnu-ld: drivers/scsi/ufs/ufs-hwmon.o: in function `ufs_hwmon_notify_event':
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:206: undefined reference to `hwmon_notify_event'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:206:(.text+0x64c): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_notify_event'
aarch64-linux-gnu-ld: /home/anders/src/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:209: undefined reference to `hwmon_notify_event'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:209:(.text+0x66c): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_notify_event'
Since SCSI_UFS_HWMON can't be built as a module, SCSI_UFS_HWMON has to
depend on HWMON=y.
Link: https://lore.kernel.org/r/20210927084615.1938432-1-anders.roxell@linaro.org
Fixes: e88e2d3220 ("scsi: ufs: core: Probe for temperature notification support")
Also-reported-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org> # build-tested
Acked-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 60c98a87fc)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I233fa50a3b5ba54b609c05b3b271276db6339afb
The device may notify the host of an extreme temperature by using the
exception event mechanism. The exception can be raised when the device’s
Tcase temperature is either too high or too low.
It is essentially up to the platform to decide what further actions need to
be taken. leave a placeholder for a designated vop for that.
Link: https://lore.kernel.org/r/20210915060407.40-3-avri.altman@wdc.com
Reviewed-by: Guenter Roeck <linux@roeck-us.net>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Daejun Park <daejun7.park@samsung.com>
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 322c4b29ee)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: Iaf049a274fb4be5c3bffd92fa2651ed50b16ca29
The 'request' member of struct scsi_cmnd is superfluous. The struct request
and struct scsi_cmnd data structures are adjacent and hence the request
pointer can be derived easily from a scsi_cmnd pointer. Introduce a helper
function that performs that conversion in a type-safe way. This patch is
the first step towards removing the request member from struct
scsi_cmnd. Making that change has the following advantages:
- This is a performance optimization since adding an offset to a pointer
takes less time than dereferencing a pointer.
- struct scsi_cmnd becomes smaller.
Link: https://lore.kernel.org/r/20210809230355.8186-2-bvanassche@acm.org
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 51f3a47889)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: Iba5551cbb5d80daaab10dbbdb302aac7892ea442
Managed device links are deleted by device_del(). However it is possible to
add a device link to a consumer before device_add(), and then discovering
an error prevents the device from being used. In that case normally
references to the device would be dropped and the device would be deleted.
However the device link holds a reference to the device, so the device link
and device remain indefinitely (unless the supplier is deleted).
For UFSHCD, if a LUN fails to probe (e.g. absent BOOT WLUN), the device
will not have been registered but can still have a device link holding a
reference to the device. The unwanted device link will prevent runtime
suspend indefinitely.
Amend device link removal to accept removal of a link with an unregistered
consumer device (suggested by Rafael), and fix UFSHCD by explicitly
deleting the device link when SCSI destroys the SCSI device.
Link: https://lore.kernel.org/r/a1c9bac8-b560-b662-f0aa-58c7e000cbbd@intel.com
Fixes: b294ff3e34 ("scsi: ufs: core: Enable power management for wlun")
Reviewed-by: Rafael J. Wysocki <rafael@kernel.org>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit bf25967ac5)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I68f785335dfd4b762130552cdf492a6ae974bc38
From arch/arm/include/asm/io.h
#define __iowmb() wmb()
[ ... ]
#define writel(v,c) ({ __iowmb(); writel_relaxed(v,c); })
From Documentation/memory-barriers.txt: "Note that, when using writel(), a
prior wmb() is not needed to guarantee that the cache coherent memory
writes have completed before writing to the MMIO region."
In other words, calling wmb() before writel() is not necessary. Hence
remove the wmb() calls that precede a writel() call. Remove the wmb() calls
that precede a ufshcd_send_command() call since the latter function uses
writel(). Remove the wmb() call from ufshcd_wait_for_dev_cmd() since the
following chain of events guarantees that the CPU will see up-to-date LRB
values:
- UFS controller writes to host memory.
- UFS controller posts completion interrupt after the memory writes from
the previous step are visible to the CPU.
- complete(hba->dev_cmd.complete) is called from the UFS interrupt handler.
- The wait_for_completion(hba->dev_cmd.complete) call in
ufshcd_wait_for_dev_cmd() returns.
Link: https://lore.kernel.org/r/20210722033439.26550-10-bvanassche@acm.org
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Stanley Chu <stanley.chu@mediatek.com>
Cc: Can Guo <cang@codeaurora.org>
Cc: Asutosh Das <asutoshd@codeaurora.org>
Cc: Avri Altman <avri.altman@wdc.com>
Tested-by: Avri altman <avri.altman@wdc.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Daejun Park <daejun7.park@samsung.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit 3d2ac73d13)
Bug: 204438323
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: Ica6352b01e27b4a0fcee7dc9df44beab9139cc2c
