Because WB performs writes in SLC mode, it is not possible to use
WriteBooster indefinitely. Vendors can set a lifetime limit in the device.
If the lifetime exceeds this limit, the device ican disable the WB feature.
The feature is defined in the "bWriteBoosterBufferLifeTimeEst (IDN = 1E)"
attribute.
With lifetime exceeding the limit value, the current driver continuously
performs the following query:
- Write Flag: WB_ENABLE / DISABLE
- Read attr: Available Buffer Size
- Read attr: Current Buffer Size
This patch recognizes that WriteBooster is no longer supported by the
device, and prevents unnecessary queries.
Link: https://lore.kernel.org/r/1891546521.01643252701746.JavaMail.epsvc@epcpadp3
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Acked-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Jinyoung Choi <j-young.choi@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit f681d1078d)
Change-Id: I5d5f61725c0d7ec2c5166122da37c1f1ffc1c233
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Call shost_for_each_device() with holding host->host_lock will cause a
deadlock situation, which will cause the system to stall (the log as
follow). Fix this issue by using __shost_for_each_device() in
ufshcd_pending_cmds().
stalls on CPUs/tasks:
all trace:
__switch_to+0x120/0x170
0xffff800011643998
ask dump for CPU 5:
ask:kworker/u16:2 state:R running task stack: 0 pid: 80 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
0x0
ask dump for CPU 6:
ask:kworker/u16:6 state:R running task stack: 0 pid: 164 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
0xffff54e7c4429f80
ask dump for CPU 7:
ask:kworker/u16:4 state:R running task stack: 0 pid: 153 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
blk_mq_run_hw_queue+0x34/0x110
blk_mq_sched_insert_request+0xb0/0x120
blk_execute_rq_nowait+0x68/0x88
blk_execute_rq+0x4c/0xd8
__scsi_execute+0xec/0x1d0
scsi_vpd_inquiry+0x84/0xf0
scsi_get_vpd_buf+0x34/0xb8
scsi_attach_vpd+0x34/0x140
scsi_probe_and_add_lun+0xa6c/0xab8
__scsi_scan_target+0x438/0x4f8
scsi_scan_channel+0x6c/0xa8
scsi_scan_host_selected+0xf0/0x150
do_scsi_scan_host+0x88/0x90
scsi_scan_host+0x1b4/0x1d0
ufshcd_async_scan+0x248/0x310
async_run_entry_fn+0x30/0x178
process_one_work+0x1e8/0x368
worker_thread+0x40/0x478
kthread+0x174/0x180
ret_from_fork+0x10/0x20
Link: https://lore.kernel.org/r/20211214120537.531628-1-huobean@gmail.com
Fixes: 8d077ede48 ("scsi: ufs: Optimize the command queueing code")
Reported-by: YongQin Liu <yongqin.liu@linaro.org>
Reported-by: Amit Pundir <amit.pundir@linaro.org>
Tested-by: John Stultz <john.stultz@linaro.org>
Tested-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Co-developed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit 99c66a8868)
Change-Id: I32ef0b93a17ed59c9011c5104b9843a8f9e38e8e
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The time spent in io_schedule() and also the interrupt latency are
significant when submitting direct I/O to a UFS device. Hence this patch
that implements polling support. User space software can enable polling by
passing the RWF_HIPRI flag to the preadv2() system call or the
IORING_SETUP_IOPOLL flag to the io_uring interface.
Although the block layer supports to partition the tag space for
interrupt-based completions (HCTX_TYPE_DEFAULT) purposes and polling
(HCTX_TYPE_POLL), the choice has been made to use the same hardware queue
for both hctx types because partitioning the tag space would negatively
affect performance.
On my test setup this patch increases IOPS from 2736 to 22000 (8x) for the
following test:
for hipri in 0 1; do
fio --ioengine=io_uring --iodepth=1 --rw=randread \
--runtime=60 --time_based=1 --direct=1 --name=qd1 \
--filename=/dev/block/sda --ioscheduler=none --gtod_reduce=1 \
--norandommap --hipri=$hipri
done
Link: https://lore.kernel.org/r/20211203231950.193369-18-bvanassche@acm.org
Tested-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit eaab9b5730)
Change-Id: I18905c9948fea32d26e11b36b9a292cb5e9569dd
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Remove the clock scaling lock from ufshcd_queuecommand() since it is a
performance bottleneck. Instead check the SCSI device budget bitmaps in the
code that waits for ongoing ufshcd_queuecommand() calls. A bit is set in
sdev->budget_map just before scsi_queue_rq() is called and a bit is cleared
from that bitmap if scsi_queue_rq() does not submit the request or after
the request has finished. See also the blk_mq_{get,put}_dispatch_budget()
calls in the block layer.
There is no risk for a livelock since the block layer delays queue reruns
if queueing a request fails because the SCSI host has been blocked.
Link: https://lore.kernel.org/r/20211203231950.193369-17-bvanassche@acm.org
Cc: Asutosh Das (asd) <asutoshd@codeaurora.org>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit 8d077ede48)
Change-Id: I8f9e5c07bf7effd8a8df13de187c4a3190acffd7
Signed-off-by: Bart Van Assche <bvanassche@google.com>
If the UFS Device WLUN is runtime suspended and is in the same power mode,
link state, and b_rpm_dev_flush_capable (BKOP or WB buffer flush etc)
state, then it can remain runtime suspended instead of being runtime
resumed and then system suspended.
The following patch has cleared the way for that to happen:
scsi: core: pm: Only runtime resume if necessary
So amend the logic accordingly.
Note, the ufs-hisi driver uses different RPM and SPM, but it is made
explicit by a new parameter to suspend prepare.
Link: https://lore.kernel.org/r/20211027130614.406985-2-adrian.hunter@intel.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit ddba1cf7a5)
Change-Id: Ib10b7a5bfb804fa743233869d9c8f0f4acc7a02c
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The build only descends into drivers/scsi/ufs/ if SCSI_UFSHCD is enabled.
Hence all later config symbols should depend on SCSI_UFSHCD to prevent
asking the user about config symbols for driver code that won't be built
anyway. Unfortunately not all symbols have that dependency.
Fix this by wrapping them all into a big if/endif block. Remove the now
superfluous explicit dependencies on SCSI_UFSHCD from all symbols that
already had it.
Link: https://lore.kernel.org/r/20211106164650.1571068-1-geert@linux-m68k.org
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit d28a78537d)
Change-Id: Ia35afb501a8131aa15e164255e95a52321cddadf
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Fix the following sparse warnings in ufshpb_set_hpb_read_to_upiu():
sparse warnings: (new ones prefixed by >>)
drivers/scsi/ufs/ufshpb.c:335:27: sparse: sparse: cast from restricted __be64
drivers/scsi/ufs/ufshpb.c:335:25: sparse: expected restricted __be64 [usertype] ppn_tmp
drivers/scsi/ufs/ufshpb.c:335:25: sparse: got unsigned long long [usertype]
Link: https://lore.kernel.org/r/20211111222452.384089-1-huobean@gmail.com
Reported-by: kernel test robot <lkp@intel.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit 73185a1377)
Change-Id: I23a4b4ec1abe0f24a7e592a0415c45608a95510e
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The UFS controller of the ExynosAuto v9 SoC supports a multi-host interface
for I/O virtualization. In general, we're using para-virtualized driver to
support a block device by several virtual machines.
Multi-host functionality extends the host controller by providing register
interfaces that can be used by each VM's UFS drivers respectively. This way
we can provide direct access to the UFS device for multiple VMs similar to
PCIe SR-IOV.
We divide this M-HCI as PH (Physical Host) and VHs (Virtual Host). The PH
supports all UFSHCI functions (all SAPs) like a conventional UFSHCI but the
VH only supports data transfer functions. Thus, except UTP_CMD_SAP and
UTP_TMPSAP, the PH should handle all the physical features.
Provide an initial implementation of PH part. M-HCI can support up to four
interfaces (1 for a PH and 3 for VHs) but this patch initially supports
only 1 PH and 1 VH. For this, we uses TASK_TAG[7:5] field so TASK_TAG[4:0]
for 32 doorbell will be supported. After the PH is initiated, this will
send a ready message to VHs through a mailbox register. The message handler
is not fully implemented yet such as supporting reset / abort cases.
Link: https://lore.kernel.org/r/20211018124216.153072-14-chanho61.park@samsung.com
Cc: Alim Akhtar <alim.akhtar@samsung.com>
Cc: Kiwoong Kim <kwmad.kim@samsung.com>
Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
Reviewed-by: Inki Dae <inki.dae@samsung.com>
Signed-off-by: Chanho Park <chanho61.park@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
(cherry picked from commit b52aea54b6)
Bug: 234653003
Change-Id: I2497687801045c859538faa90a7e68d86686ba72
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Casting an int explicitly to u16 when passed as an argument to a function
is not necessary.
Since prd_table and ucd_prdt_ptr both have type struct ufshcd_sg_entry *,
remove the casts from assignments of these two to each other.
This patch does not change any functionality.
Link: https://lore.kernel.org/r/20211020214024.2007615-9-bvanassche@acm.org
Acked-by: Avri Altman <Avri.Altman@wdc.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit 3ad317a1f9)
Change-Id: I3e93b004ddd44a3a58e70cdcd74d98af317a740d
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Introduce an array for debugfs attributes to make it easier to add new
debugfs attributes. Change the value of the inode.i_private pointer for
debugfs attributes from a pointer to the HBA data structure to a pointer to
the attribute description for the stats attribute. Store the HBA pointer in
the private data of the parent inode instead.
Link: https://lore.kernel.org/r/20211020214024.2007615-7-bvanassche@acm.org
Acked-by: Avri Altman <Avri.Altman@wdc.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit e0022c6c29)
Change-Id: Ic1cea27282c79d25fa394f9a5d0e454542c29085
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Commit 73dc3c4ac7 ("scsi: ufs: Retry aborted SCSI commands instead of
completing these successfully") is not necessary. If a SCSI command is
aborted successfully the UFS controller has not modified the command status
and the command status still has the value assigned by
ufshcd_prepare_req_desc_hdr(), namely OCS_INVALID_COMMAND_STATUS. The
function ufshcd_transfer_rsp_status() requeues commands that have an
invalid command status. Hence revert commit 73dc3c4ac7.
Link: https://lore.kernel.org/r/20211020214024.2007615-2-bvanassche@acm.org
Acked-by: Avri Altman <Avri.Altman@wdc.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit 1168252357)
Change-Id: I7f0754b60cf5281c92983f1469aa8129fa02104d
Signed-off-by: Bart Van Assche <bvanassche@google.com>
sched_clock() is not meant to be used in portable driver code, and assuming
a particular clock frequency is not how this is meant to be used. It also
causes a build failure because of a missing header inclusion:
drivers/scsi/ufs/ufs-mediatek.c:321:12: error: implicit declaration of function 'sched_clock' [-Werror,-Wimplicit-function-declaration]
timeout = sched_clock() + retry_ms * 1000000UL;
A better interface to use here ktime_get_mono_fast_ns(), which works mostly
like ktime_get() but is safe to use inside of a suspend callback.
Link: https://lore.kernel.org/r/20211018132022.2281589-1-arnd@kernel.org
Fixes: 9561f58442 ("scsi: ufs: mediatek: Support vops pre suspend to disable auto-hibern8")
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit bb4a8dcb4e)
Change-Id: I22a07dbb7116d0a73253a5551aaf98697e41bc5e
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The SCSI error handler calls scsi_unjam_host() which can call the queue
function ufshcd_queuecommand() indirectly. The error handler changes the
state to UFSHCD_STATE_RESET while running, but error interrupts that
happen while the error handler is running could change the state to
UFSHCD_STATE_EH_SCHEDULED_NON_FATAL which would allow requests to go
through ufshcd_queuecommand() even though the error handler is running.
Block that hole by checking whether the error handler is in progress.
Link: https://lore.kernel.org/r/20211008084048.257498-1-adrian.hunter@intel.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 234653003
(cherry picked from commit d489f18ad1)
Change-Id: Iab73fed401a9cdd442804e747c4370305ebd3692
Signed-off-by: Bart Van Assche <bvanassche@google.com>
