Reason for revert:
MMIO guard is initialised too late for earlycon, so the fixup is
a waste of time.
Signed-off-by: Keir Fraser <keirf@google.com>
Bug: 259645922
Change-Id: Iace214f7140baf073cfa5cef1ee65fca9e4a76d0
Add kasan.page_alloc.sample=10 to CONFIG_CMDLINE in gki_defconfig to make
Hardware Tag-Based (MTE) KASAN tag only one out of every 10 page_alloc
allocations with the order equal or larger than 3, which the omitted
default value for the kasan.page_alloc.sample.order parameter.
As Hardware Tag-Based KASAN is intended to be used in production, its
performance impact is crucial. As page_alloc allocations tend to be big,
tagging and checking all such allocations can introduce a significant
slowdown.
When running a local loopback test on a testing MTE-enabled device in sync
mode, enabling Hardware Tag-Based KASAN introduces a ~50% slowdown.
Setting kasan.page_alloc.sampling to a value higher than 1 allows to lower
the slowdown. The performance improvement saturates around the sampling
interval value of 10 with the default sampling page order of 3, see
b/238286329. This lowers the slowdown to ~20%. The slowdown in real
scenarios involving the network will likely be better.
Enabling page_alloc sampling has a downside: KASAN misses bad accesses to
a page_alloc allocation that has not been tagged. This lowers the value
of KASAN as a security mitigation.
However, based on measuring the number of page_alloc allocations of
different orders during boot in a test build, sampling with the default
kasan.page_alloc.sample.order value affects only ~7% of allocations. The
rest ~93% of allocations are still checked deterministically.
Bug: 238286329
Bug: 264310057
Change-Id: Id361822b8bbf929378cabbe0350b658d6120e840
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
[The patch is in the mm-unstable tree.]
The implementation of page_alloc poisoning sampling assumed that
tag_clear_highpage resets page tags for __GFP_ZEROTAGS allocations.
However, this is no longer the case since commit 70c248aca9
("mm: kasan: Skip unpoisoning of user pages").
This leads to kernel crashes when MTE-enabled userspace mappings are
used with Hardware Tag-Based KASAN enabled.
Reset page tags for __GFP_ZEROTAGS allocations in post_alloc_hook().
Also clarify and fix related comments.
Fixes: 44383cef54 ("kasan: allow sampling page_alloc allocations for HW_TAGS")
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reported-by: Peter Collingbourne <pcc@google.com>
Tested-by: Peter Collingbourne <pcc@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Marco Elver <elver@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Bug: 238286329
Bug: 264310057
Link: https://lore.kernel.org/all/5dbd866714b4839069e2d8469ac45b60953db290.1674592780.git.andreyknvl@google.com/
Change-Id: Iea4234bcf7e35337c8063827b07039583bca9c66
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
[The patch is in mm-stable tree.]
As Hardware Tag-Based KASAN is intended to be used in production, its
performance impact is crucial. As page_alloc allocations tend to be big,
tagging and checking all such allocations can introduce a significant
slowdown.
Add two new boot parameters that allow to alleviate that slowdown:
- kasan.page_alloc.sample, which makes Hardware Tag-Based KASAN tag only
every Nth page_alloc allocation with the order configured by the second
added parameter (default: tag every such allocation).
- kasan.page_alloc.sample.order, which makes sampling enabled by the first
parameter only affect page_alloc allocations with the order equal or
greater than the specified value (default: 3, see below).
The exact performance improvement caused by using the new parameters
depends on their values and the applied workload.
The chosen default value for kasan.page_alloc.sample.order is 3, which
matches both PAGE_ALLOC_COSTLY_ORDER and SKB_FRAG_PAGE_ORDER. This is
done for two reasons:
1. PAGE_ALLOC_COSTLY_ORDER is "the order at which allocations are deemed
costly to service", which corresponds to the idea that only large and
thus costly allocations are supposed to sampled.
2. One of the workloads targeted by this patch is a benchmark that sends
a large amount of data over a local loopback connection. Most multi-page
data allocations in the networking subsystem have the order of
SKB_FRAG_PAGE_ORDER (or PAGE_ALLOC_COSTLY_ORDER).
When running a local loopback test on a testing MTE-enabled device in sync
mode, enabling Hardware Tag-Based KASAN introduces a ~50% slowdown.
Applying this patch and setting kasan.page_alloc.sampling to a value
higher than 1 allows to lower the slowdown. The performance improvement
saturates around the sampling interval value of 10 with the default
sampling page order of 3. This lowers the slowdown to ~20%. The slowdown
in real scenarios involving the network will likely be better.
Enabling page_alloc sampling has a downside: KASAN misses bad accesses to
a page_alloc allocation that has not been tagged. This lowers the value
of KASAN as a security mitigation.
However, based on measuring the number of page_alloc allocations of
different orders during boot in a test build, sampling with the default
kasan.page_alloc.sample.order value affects only ~7% of allocations. The
rest ~93% of allocations are still checked deterministically.
Link: https://lkml.kernel.org/r/129da0614123bb85ed4dd61ae30842b2dd7c903f.1671471846.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Brand <markbrand@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Bug: 238286329
Bug: 264310057
(cherry picked from commit 44383cef54 git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-stable)
Change-Id: I85f9eb4e93eeddff8f8d06238f433226affca177
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
* This is the initial symbol list for
rockpi4 target.
Bug: 261782070
Change-Id: I76f8924f5e40c6ecfb91a5291b86a9d4fd9badb0
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>
This introduces a description of the MEM_RELINQUISH hypercall. MMIO
guard hypercalls are already described in another file, which we now
link to.
Signed-off-by: Keir Fraser <keirf@google.com>
Bug: 265943840
Change-Id: Iaffde3419f6432d76598e48c9bab53f672430b7a
Firstly, the hypercall IDs have been renumbered but that was not
reflected in the documentation. Secondly, the argument registers r1-r3
are checked to be zero when calling
ARM_SMCCC_KVM_FUNC_MMIO_GUARD_INFO.
Signed-off-by: Keir Fraser <keirf@google.com>
Bug: 265943840
Change-Id: I9684d1e71af7d8627d079cfd89d437cfc28be09f
Change a few uses of ARM_SMCCC_VENDOR_HYP_KVM_FOO_FUNC_ID to
ARM_SMCCC_KVM_FUNC_FOO.
Signed-off-by: Keir Fraser <keirf@google.com>
Bug: 265943840
Change-Id: Id1642243ca4105a23e808cf28d13bfb81d9a5ac2
When communicating with accessory devices via USBFS, the initialisation
call-stack looks like:
ConfigFS > Gadget ConfigFS > UDC > Gadget ConfigFS > Composite
Eventually ending up in composite_dev_prepare() where memory for the
data buffer is allocated and initialised. The default size used for the
allocation is USB_COMP_EP0_BUFSIZ (4k). When handling bulk transfers,
acc_ctrlrequest() needs to be able to handle buffers up to
BULK_BUFFER_SIZE (16k). Instead of adding new generic attributes to
'struct usb_request' to track the size of the allocated buffer, we can
simply split off the affected thread of execution to travel via a
knowledgeable abstracted function acc_ctrlrequest_composite() where we
can complete the necessary specific checks.
Bug: 264029575
Signed-off-by: Lee Jones <joneslee@google.com>
Change-Id: Ia1280f85499621d3fa57f7262b4a2c80f4be7773
Signed-off-by: Lee Jones <joneslee@google.com>
This reverts commit fe19dff7e6.
Reason for revert:
Observed frequent boot crashes on a device with sampling KASAN enabled.
Bug: 265863271
Change-Id: Ib7860295065ed7aaa36d9e47d8aaa97918c7bc57
Signed-off-by: Peter Collingbourne <pcc@google.com>
This reverts commit 0352e06246.
Reason for revert:
Observed frequent boot crashes on a device with sampling KASAN enabled.
Bug: 265863271
Change-Id: Ie4d9c2ffc39288a9f9a941784bf661eed14503e9
Signed-off-by: Peter Collingbourne <pcc@google.com>
Add and export two functions to enable ESI and config ESI base addresses.
The calls to these exported functions will be added by the next patch in
this series.
Change-Id: Icccf24b3a0275f76af63c81f1899c48f8c739cbd
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit e02288e026 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Remove dependency on kleaf intermediate abi_symbollist.raw
and use vendor symbol lists as dependencies for header
generation targets. This prevents the feature to break in
case kleaf path and/or sandboxes changes and rebuilds the
header files if any vendor symbol list changes.
Update the header generation script to process the symbol lists
before generating headers to make sure symbols are in byte order,
any symbol list marker, whitespaces and emptylines are removed
for kernel binary search API.
Bug: 232430739
Test: TH
Change-Id: Ib5783fb21543844dac7faf1fb0fcf3e7bd2bf608
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
As Event Specific Interrupt message format is not defined in UFSHCI JEDEC
specs, and the ESI handling highly depends on how the format is designed,
hence add a vendor specific ops such that SoC vendors can configure their
own ESI handlers. If ESI vops is not provided or returning error, go with
the legacy (central) interrupt way.
Change-Id: I568e14b01e98868e5149c95cf3f6ee815c72868a
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit edb0db0560 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Complete CQE requests in poll. Assumption is that several poll completion
may happen in different CPUs for the same completion queue. Hence a spin
lock protection is added.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I99e7ab96fea562904a572e29f28ecfd10c3ab6cc
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit ed975065c3 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Add support for completing requests from Completion Queue. Some host
controllers support vendor specific registers that provide a bitmap of all
CQs which have at least one completed CQE. Add this support. The MCQ
specification doesn't provide the Task Tag or its equivalent in the
Completion Queue Entry. So use an indirect method to find the Task Tag
from the Completion Queue Entry.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: Iecd678ef71f5c64925723db198c020fe4cfe36e4
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit f87b2c4182 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Add support to find the hardware queue on which the request would be
queued. Since the very first queue is to serve device commands, an offset
of 1 is added to the index of the hardware queue.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I8ea638482b7c9b9528867284855d142ca9bbeb58
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 854f84e7fe git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Add support to send commands using multiple submission queues in MCQ mode.
Modify the functions that use ufshcd_send_command().
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: Ib570322ed4456e2328349df4c38f9b46f9f423b6
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 22a2d563de git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Enable shared tags for MCQ. For UFS, this should not have a huge
performance impact. It however simplifies the MCQ implementation and reuses
most of the existing code in the issue and completion path. Also add
multiple queue mapping to map_queue().
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: Ida770b88a94957d51f9e564ffa1f4cfe4dfae4ed
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 0d33728fc0 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Bug: 241479010
Test: incfs_test passes, play confirm behavior in bug is fixed
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ie51f2b76d0873057f54fecf7fcc793c66df20969
Bug: 261875471
Test: New tests fail initially, pass with aosp/2225243
Change-Id: Ia5f2073dc4710306bd348cb2aab6d2599fd06500
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Also fix race whereby multiple providers writinig the same block would
actually write out the same block.
Note that multiple_providers_test started failing when incfs was ported
to 5.15, and these fixes are needed to make the test reliable
Bug: 264703896
Test: incfs-test passes, specifically multiple_providers_test. Ran 100
times
Change-Id: I05ad5b2b2f62cf218256222cecb79bbe9953bd97
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Also fix test build, test memory leaks
Test: run various range options
Change-Id: I747b3c02f0eb7da440c9d2c2a67e19164b8c9908
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Bug: 264703896
Runtime and operation registers are defined per Submission and Completion
queue. The location of these registers is not defined in the spec; meaning
the offsets and stride may vary for different HC vendors. Establish the
stride, base address, and doorbell address offsets from vendor host driver
and program it.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I047ebe88dcca27e9ec2ac422b6b9ce28634c595d
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 2468da61ea git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
To read the bqueuedepth, the device descriptor is fetched in Single
Doorbell Mode. This allocated memory may not be enough for MCQ mode because
the number of tags supported in MCQ mode may be larger than in SDB mode.
Hence, release the memory allocated in SDB mode and allocate memory for MCQ
mode operation. Define the UFS hardware queue and Completion Queue Entry.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: Ib11a63c7f9014dbd88669c2f3b6675b12e44f528
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 4682abfae2 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The UFS device defines the supported queuedepth by bqueuedepth which has a
max value of 256. The HC defines MAC (Max Active Commands) that defines
the max number of commands that in flight to the UFS device. Calculate and
configure the nutrs based on both these values.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I7b1d3d33e14c144129a80f8f46ac29a8e82a0251
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 7224c80687 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Multi-circular queue (MCQ) has been added in UFSHC v4.0 standard in
addition to the Single Doorbell mode. The MCQ mode supports multiple
submission and completion queues. Add support to allocate and configure
the queues. Add module parameters support to configure the queues.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I4d8e0f058394d11966f910fe26a84f94bc0340b8
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 57b1c0ef89 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
If MCQ support is present, enabling it after MCQ support has been
configured would require reallocating tags and memory. It would also free
up the already allocated memory in Single Doorbell Mode. So defer invoking
scsi_add_host() until MCQ is configured.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I87153313994b32a675a9dd0f78ea41bfdaaa0d55
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 0cab4023ec git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Task Tag is limited to 8 bits and this restricts the number of active I/Os
to 255. In multi-circular queue mode, this may not be enough. The
specification provides EXT_IID which can be used to increase the number of
I/Os if the UFS device and UFSHC support it. This patch adds support to
probe for EXT_IID support in UFS device and UFSHC.
Co-developed-by: Can Guo <quic_cang@quicinc.com>
Change-Id: I6ad6c2d4f360626bfa8e9a94373d8a5c6893d9ec
Signed-off-by: Can Guo <quic_cang@quicinc.com>
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 6e1d850acf git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
In 'include/ufs/ufshcd.h' file, 'enum dma_data_direction' will be used,
which is defined in linux/dma-direction.h, however, this header file is not
included in ufshcd.h, thus causing the following compilation warning:
"warning: ‘enum dma_data_direction’ declared inside parameter list will not
be visible outside of this definition or declaration"
Fix this warning by including 'linux/dma-direction.h'.
Fixes: 6ff265fc5e ("scsi: ufs: core: bsg: Add advanced RPMB support in ufs_bsg")
Reported-by: Xiaosen He <quic_xiaosenh@quicinc.com>
Reported-by: Bart Van Assche <bvanassche@acm.org>
Change-Id: If56d3463377fd4fc157a52807e1a3eccfc5209a2
Signed-off-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit f3e57da528 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Starting from Qcom UFS version 4.0, vendor specific REG_UFS_PARAM0 register
can be used to determine the maximum gear supported by the controller.
Suggested-by: Can Guo <quic_cang@quicinc.com>
Reviewed-by: Andrew Halaney <ahalaney@redhat.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I44a1a54788b77b6072d8514e873515789e60deaf
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 2c407fe970 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Starting from Qualcomm UFS version 4, the UFS device needs to be
reinitialized after switching to maximum gear by the UFS core. Hence, add
support for it by enabling the UFSHCD_QUIRK_REINIT_AFTER_MAX_GEAR_SWITCH
quirk, implementing reinit_notify() callback and using the agreed gear
speed for setting the PHY mode.
Suggested-by: Can Guo <quic_cang@quicinc.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I2caf0076bca06385e53e6f6cd4e05e81b1f016ea
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit baf5ddac90 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
In the preparation of adding support for new gears, move the logic that
finds the gear for each platform to a new function. This helps with code
readability and also allows the logic to be used in other places of the
driver in future.
While at it, make it clear that this driver only supports symmetric gear
setting (hs_tx_gear == hs_rx_gear).
Reviewed-by: Andrew Halaney <ahalaney@redhat.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I518bf29d1c77f2368b922c6c272378cadf2bff98
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit c270986527 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Some platforms like Qcom, requires the UFS device to be reinitialized after
switching to maximum gear speed. So add support for that in UFS core by
introducing a new quirk (UFSHCD_CAP_REINIT_AFTER_MAX_GEAR_SWITCH) and doing
the reinitialization, if the quirk is enabled by the controller driver.
Suggested-by: Can Guo <quic_cang@quicinc.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I838fde7f3987257f0611bb37ffc0a1683e8a5c58
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 96a7141da3 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
reinit_notify() callback can be used by the UFS controller drivers to
perform changes required for UFSHCD reinit that can happen during max gear
switch.
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: Ieafb979fb0006f7bf6e0a4f357822a15d558c181
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit c2c38c573a git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
On newer UFS revisions, the register at offset 0xD0 is called,
REG_UFS_PARAM0. Since the existing register, RETRY_TIMER_REG is not used
anywhere, it is safe to use the new name.
Reviewed-by: Andrew Halaney <ahalaney@redhat.com>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I85d0bb15ef0c1af3be9fd3e8d9aa1e53225a64d7
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 7959587f32 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
In the reset assert and deassert callbacks, the supplied "id" is not used
at all and only the HBA reset is performed all the time. So there is no
reason to use a WARN_ON() on the "id".
Reviewed-by: Andrew Halaney <ahalaney@redhat.com>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I498403cd74ca787acc69ec852e4e3e8087f24dc8
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit a98c2183e8 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
A goto statement in an error path is useful if the function needs to do
cleanup other than returning the error code. But in this driver, goto
statements are used for just returning the error code in many places. This
really makes it hard to read the code.
Get rid of those goto statements and just return the error code directly.
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Tested-by: Andrew Halaney <ahalaney@redhat.com> # Qdrive3/sa8540p-ride
Change-Id: I311b5dc3a38a408c121ef315e39319fb4aea4027
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 031312dbc6 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Add driver code for Unisoc UFS host controller, along with UFS
initialization.
Change-Id: If768bbc760201c1c7f5acbf88d1d765cc8aa9352
Signed-off-by: Zhe Wang <zhe.wang1@unisoc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit df7320bac3 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Commit 0fd420005c ("Merge 5.15.89 into android14-5.15") added an
rcu_read_lock() call in ufshcd_queuecommand() that is not present in the
upstream kernel. Since this makes locking in ufshcd_queuecommand()
unbalanced, remove that rcu_read_lock() call again.
Bug: 258234315
Change-Id: I555d91fe0aaabe95c9b762848b05af6c1ffc2be4
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Modify the UFSHCD core to allow 'struct ufshcd_sg_entry' to be
variable-length. The default is the standard length, but variants can
override ufs_hba::sg_entry_size with a larger value if there are
vendor-specific fields following the standard ones.
This is needed to support inline encryption with ufs-exynos (FMP).
Cc: Eric Biggers <ebiggers@google.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Change-Id: I4fa3303b0ed70951ea4ec345bb3d58c6a008917d
Signed-off-by: Eric Biggers <ebiggers@google.com>
[ bvanassche: edited commit message and introduced CONFIG_SCSI_UFS_VARIABLE_SG_ENTRY_SIZE ]
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit ada1e653a5 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
[ bvanassche: backported this patch to the Android kernel tree ]
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Host-managed zoned block devices report the sense code 21h/04h
(UNALIGNED WRITE COMMAND) when attempting to write data at another LBA
than the write pointer. Retry unaligned writes after all other pending
commands have completed instead of retrying these immediately. The
change in this patch makes the following function decide to queue the
failed command to the error handler instead of retrying it immediately:
static void scsi_complete(struct request *rq)
{
[ ... ]
switch (disposition) {
case SUCCESS:
scsi_finish_command(cmd);
break;
case NEEDS_RETRY:
scsi_queue_insert(cmd, SCSI_MLQUEUE_EH_RETRY);
break;
case ADD_TO_MLQUEUE:
scsi_queue_insert(cmd, SCSI_MLQUEUE_DEVICE_BUSY);
break;
default:
scsi_eh_scmd_add(cmd);
break;
}
}
This patch has not been posted on any Linux kernel mailing list since
there is agreement that another approach will be taken to preserve the
order of WRITE commands for zoned devices. However, the implementation
of that new approach is not yet available.
Bug: 197782466
Change-Id: I3529ce4f01ff166a0c8ac321eb31b0e7771906ac
Signed-off-by: Bart Van Assche <bvanassche@google.com>
