rpcrdma_rep objects are removed from their free list by only a
single thread: the Receive completion handler. Thus that free list
can be converted to an llist, where a single-threaded consumer and
a multi-threaded producer (rpcrdma_buffer_put) can both access the
llist without the need for any serialization.
This eliminates spin lock contention between the Receive completion
handler and rpcrdma_buffer_get, and makes the rep consumer wait-
free.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Clean up: Now that the free list is used sparingly, get rid of the
separate spin lock protecting it.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Instead of a globally-contended MR free list, cache MRs in each
rpcrdma_req as they are released. This means acquiring and releasing
an MR will be lock-free in the common case, even outside the
transport send lock.
The original idea of per-rpcrdma_req MR free lists was suggested by
Shirley Ma <shirley.ma@oracle.com> several years ago. I just now
figured out how to make that idea work with on-demand MR allocation.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Refactor: Retrieve an MR and handle error recovery entirely in
rpc_rdma.c, as this is not a device-specific function.
Note that since commit 89f90fe1ad ("SUNRPC: Allow calls to
xprt_transmit() to drain the entire transmit queue"), the
xprt_transmit function handles the cond_resched. The transport no
longer has to do this itself.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Clean up. There is only one remaining rpcrdma_mr_put call site, and
it can be directly replaced with unmap_and_put because mr->mr_dir is
set to DMA_NONE just before the call.
Now all the call sites do a DMA unmap, and we can just rename
mr_unmap_and_put to mr_put, which nicely matches mr_get.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Clean up: rpcrdma_mr_pop call sites check if the list is empty
first. Let's replace the list_empty with less costly logic.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Commit 48be539dd4 ("xprtrdma: Introduce ->alloc_slot call-out for
xprtrdma") added a separate alloc_slot and free_slot to the RPC/RDMA
transport. Later, commit 75891f502f ("SUNRPC: Support for
congestion control when queuing is enabled") modified the generic
alloc/free_slot methods, but neglected the methods in xprtrdma.
Found via code review.
Fixes: 75891f502f ("SUNRPC: Support for congestion control ... ")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Clean up: There are other "all" list heads. For code clarity
distinguish this one as for use only for MRs by renaming it.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Make the field name the same for all trace points that handle
pointers to struct rpcrdma_rep. That makes it easy to grep for
matching rep points in trace output.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
I've heard rumors of an NFS/RDMA server implementation that has a
default credit limit of 1024. The client's default setting remains
at 128.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Although I haven't seen any performance results that justify it,
I've received several complaints that NFS/RDMA no longer supports
a maximum rsize and wsize of 1MB. These days it is somewhat smaller.
To simplify the logic that determines whether a chunk list is
necessary, the implementation uses a fixed maximum size of the
transport header. Currently that maximum size is 256 bytes, one
quarter of the default inline threshold size for RPC/RDMA v1.
Since commit a78868497c ("xprtrdma: Reduce max_frwr_depth"), the
size of chunks is also smaller to take advantage of inline page
lists in device internal MR data structures.
The combination of these two design choices has reduced the maximum
NFS rsize and wsize that can be used for most RNIC/HCAs. Increasing
the maximum transport header size and the maximum number of RDMA
segments it can contain increases the negotiated maximum rsize/wsize
on common RNIC/HCAs.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Commit 302d3deb20 ("xprtrdma: Prevent inline overflow") added this
calculation back in 2016, but got it wrong. I tested only the lower
bound, which is why there is a max_t there. The upper bound should be
rounded up too.
Now, when using DIV_ROUND_UP, that takes care of the lower bound as
well.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Things have changed since this comment was written. In particular,
the reworking of connection closing, on-demand creation of MRs, and
the removal of fr_state all mean that deferring MR recovery to
frwr_map is no longer needed. The description is obsolete.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Micro-optimization: For xdr_commit_encode call sites in
net/sunrpc/xdr.c, eliminate the extra calling sequence. On my
client, this change saves about a microsecond for every 30 calls
to xdr_reserve_space().
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Clean up: commit c544577dad ("SUNRPC: Clean up transport write
space handling") appears to have removed the last caller of
rpc_wake_up_queued_task_on_wq().
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
In encode_attrs(), there is an if statement on line 1145 to check
whether label is NULL:
if (label && (attrmask[2] & FATTR4_WORD2_SECURITY_LABEL))
When label is NULL, it is used on lines 1178-1181:
*p++ = cpu_to_be32(label->lfs);
*p++ = cpu_to_be32(label->pi);
*p++ = cpu_to_be32(label->len);
p = xdr_encode_opaque_fixed(p, label->label, label->len);
To fix these bugs, label is checked before being used.
These bugs are found by a static analysis tool STCheck written by us.
Signed-off-by: Jia-Ju Bai <baijiaju1990@gmail.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Pull MTD fix from Richard Weinberger:
"A single fix for MTD to correctly set the spi-nor WP pin"
* tag 'fixes-for-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux:
mtd: spi-nor: Fix the disabling of write protection at init
Pull btrfs fixes from David Sterba:
"Two fixes that popped up during testing:
- fix for sysfs-related code that adds/removes block groups, warnings
appear during several fstests in connection with sysfs updates in
5.3, the fix essentially replaces a workaround with scope NOFS and
applies to 5.2-based branch too
- add sanity check of trim range"
* tag 'for-5.3-rc4-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
btrfs: trim: Check the range passed into to prevent overflow
Btrfs: fix sysfs warning and missing raid sysfs directories
Pull x86 fixes from Thomas Gleixner:
"A set of fixes for x86:
- Fix the inconsistent error handling in the umwait init code
- Rework the boot param zeroing so gcc9 stops complaining about out
of bound memset. The resulting source code is actually more sane to
read than the smart solution we had
- Maintainers update so Tony gets involved when Intel models are
added
- Some more fallthrough fixes"
* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/boot: Save fields explicitly, zero out everything else
MAINTAINERS, x86/CPU: Tony Luck will maintain asm/intel-family.h
x86/fpu/math-emu: Address fallthrough warnings
x86/apic/32: Fix yet another implicit fallthrough warning
x86/umwait: Fix error handling in umwait_init()
Pull EFI fix from Thomas Gleixner:
"A single fix for a EFI mixed mode regression caused by recent rework
which did not take the firmware bitwidth into account"
* 'efi-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
efi-stub: Fix get_efi_config_table on mixed-mode setups
Pull SPDX fixes from Greg KH:
"Here are four small SPDX fixes for 5.3-rc5.
A few style fixes for some SPDX comments, added an SPDX tag for one
file, and fix up some GPL boilerplate for another file.
All of these have been in linux-next for a few weeks with no reported
issues (they are comment changes only, so that's to be expected...)"
* tag 'spdx-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx:
i2c: stm32: Use the correct style for SPDX License Identifier
intel_th: Use the correct style for SPDX License Identifier
coccinelle: api/atomic_as_refcounter: add SPDX License Identifier
kernel/configs: Replace GPL boilerplate code with SPDX identifier
Pull char/misc driver fixes from Greg KH:
"Here are some small char and misc driver fixes for 5.3-rc5.
These are two different subsystems needing some fixes, the habanalabs
driver which is has some more big endian fixes for problems found. The
other are some small soundwire fixes, including some Kconfig
dependencies needed to resolve reported build errors.
All of these have been in linux-next this week with no reported
issues"
* tag 'char-misc-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
misc: xilinx-sdfec: fix dependency and build error
habanalabs: fix device IRQ unmasking for BE host
habanalabs: fix endianness handling for internal QMAN submission
habanalabs: fix completion queue handling when host is BE
habanalabs: fix endianness handling for packets from user
habanalabs: fix DRAM usage accounting on context tear down
habanalabs: Avoid double free in error flow
soundwire: fix regmap dependencies and align with other serial links
soundwire: cadence_master: fix definitions for INTSTAT0/1
soundwire: cadence_master: fix register definition for SLAVE_STATE
Pull staging/IIO fixes from Greg KH:
"Here are four small staging and iio driver fixes for 5.3-rc5
Two are for the dt3000 comedi driver for some reported problems found
in that codebase, and two are some small iio fixes.
All of these have been in linux-next this week with no reported
issues"
* tag 'staging-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging:
staging: comedi: dt3000: Fix rounding up of timer divisor
staging: comedi: dt3000: Fix signed integer overflow 'divider * base'
iio: adc: max9611: Fix temperature reading in probe
iio: frequency: adf4371: Fix output frequency setting
Pull USB fixes from Greg KH:
"Here are number of small USB fixes for 5.3-rc5.
Syzbot has been on a tear recently now that it has some good USB
debugging hooks integrated, so there's a number of fixes in here found
by those tools for some _very_ old bugs. Also a handful of gadget
driver fixes for reported issues, some hopefully-final dma fixes for
host controller drivers, and some new USB serial gadget driver ids.
All of these have been in linux-next this week with no reported issues
(the usb-serial ones were in linux-next in its own branch, but merged
into mine on Friday)"
* tag 'usb-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
usb: add a hcd_uses_dma helper
usb: don't create dma pools for HCDs with a localmem_pool
usb: chipidea: imx: fix EPROBE_DEFER support during driver probe
usb: host: fotg2: restart hcd after port reset
USB: CDC: fix sanity checks in CDC union parser
usb: cdc-acm: make sure a refcount is taken early enough
USB: serial: option: add the BroadMobi BM818 card
USB: serial: option: Add Motorola modem UARTs
USB: core: Fix races in character device registration and deregistraion
usb: gadget: mass_storage: Fix races between fsg_disable and fsg_set_alt
usb: gadget: composite: Clear "suspended" on reset/disconnect
usb: gadget: udc: renesas_usb3: Fix sysfs interface of "role"
USB: serial: option: add D-Link DWM-222 device ID
USB: serial: option: Add support for ZTE MF871A
Pull block fixes from Jens Axboe:
"A collection of fixes that should go into this series. This contains:
- Revert of the REQ_NOWAIT_INLINE and associated dio changes. There
were still corner cases there, and even though I had a solution for
it, it's too involved for this stage. (me)
- Set of NVMe fixes (via Sagi)
- io_uring fix for fixed buffers (Anthony)
- io_uring defer issue fix (Jackie)
- Regression fix for queue sync at exit time (zhengbin)
- xen blk-back memory leak fix (Wenwen)"
* tag 'for-linus-2019-08-17' of git://git.kernel.dk/linux-block:
io_uring: fix an issue when IOSQE_IO_LINK is inserted into defer list
block: remove REQ_NOWAIT_INLINE
io_uring: fix manual setup of iov_iter for fixed buffers
xen/blkback: fix memory leaks
blk-mq: move cancel of requeue_work to the front of blk_exit_queue
nvme-pci: Fix async probe remove race
nvme: fix controller removal race with scan work
nvme-rdma: fix possible use-after-free in connect error flow
nvme: fix a possible deadlock when passthru commands sent to a multipath device
nvme-core: Fix extra device_put() call on error path
nvmet-file: fix nvmet_file_flush() always returning an error
nvmet-loop: Flush nvme_delete_wq when removing the port
nvmet: Fix use-after-free bug when a port is removed
nvme-multipath: revalidate nvme_ns_head gendisk in nvme_validate_ns
Pull Hyper-V fixes from Sasha Levin:
- A few fixes for the userspace hyper-v tools from Adrian Vladu.
- A fix for the hyper-v MAINTAINERs entry from Lan Tianyu.
- Fix for SPDX license identifier in the userspace tools from Nishad
Kamdar.
* tag 'hyperv-fixes-signed' of git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux:
MAINTAINERS: Fix Hyperv vIOMMU driver file name
tools: hv: Use the correct style for SPDX License Identifier
tools: hv: fix typos in toolchain
tools: hv: fix KVP and VSS daemons exit code
tools: hv: fixed Python pep8/flake8 warnings for lsvmbus
The Hyperv vIOMMU file name should be "hyperv-iommu.c" rather
than "hyperv_iommu.c". This patch is to fix it.
Signed-off-by: Lan Tianyu <Tianyu.Lan@microsoft.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
This patch corrects the SPDX License Identifier style
in the trace header file related to Microsoft Hyper-V
client drivers.
For C header files Documentation/process/license-rules.rst
mandates C-like comments (opposed to C source files where
C++ style should be used)
Changes made by using a script provided by Joe Perches here:
https://lkml.org/lkml/2019/2/7/46
Suggested-by: Joe Perches <joe@perches.com>
Signed-off-by: Nishad Kamdar <nishadkamdar@gmail.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Pull i2c fixes from Wolfram Sang:
"I2C has one revert because of a regression, two fixes for tiny race
windows (which we were not able to trigger), a MAINTAINERS addition,
and a SPDX fix"
* 'i2c/for-current' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux:
i2c: stm32: Use the correct style for SPDX License Identifier
i2c: emev2: avoid race when unregistering slave client
i2c: rcar: avoid race when unregistering slave client
MAINTAINERS: i2c-imx: take over maintainership
Revert "i2c: imx: improve the error handling in i2c_imx_dma_request()"
Pull RISC-V fixes from Paul Walmsley:
- Two patches to fix significant bugs in floating point register
context handling
- A minor fix in RISC-V flush_tlb_page(), to supply a valid end address
to flush_tlb_range()
- Two minor defconfig additions: to build the virtio hwrng driver by
default (for QEMU targets), and to partially synchronize the 32-bit
defconfig with the 64-bit defconfig
* tag 'riscv/for-v5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux:
riscv: Make __fstate_clean() work correctly.
riscv: Correct the initialized flow of FP register
riscv: defconfig: Update the defconfig
riscv: rv32_defconfig: Update the defconfig
riscv: fix flush_tlb_range() end address for flush_tlb_page()
Johan writes:
USB-serial fixes for 5.3-rc5
Here are some new modem device ids.
All have been in linux-next with no reported issues.
Signed-off-by: Johan Hovold <johan@kernel.org>
* tag 'usb-serial-5.3-rc5' of https://git.kernel.org/pub/scm/linux/kernel/git/johan/usb-serial:
USB: serial: option: add the BroadMobi BM818 card
USB: serial: option: Add Motorola modem UARTs
USB: serial: option: add D-Link DWM-222 device ID
USB: serial: option: Add support for ZTE MF871A
Pull Xtensa fix from Max Filippov:
"Add missing isync into cpu_reset to make sure ITLB changes are
effective"
* tag 'xtensa-20190816' of git://github.com/jcmvbkbc/linux-xtensa:
xtensa: add missing isync to the cpu_reset TLB code
Pull arm64 fixes from Catalin Marinas:
- Don't taint the kernel if CPUs have different sets of page sizes
supported (other than the one in use).
- Issue I-cache maintenance for module ftrace trampoline.
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64: ftrace: Ensure module ftrace trampoline is coherent with I-side
arm64: cpufeature: Don't treat granule sizes as strict
The initial support for dynamic ftrace trampolines in modules made use
of an indirect branch which loaded its target from the beginning of
a special section (e71a4e1beb ("arm64: ftrace: add support for far
branches to dynamic ftrace")). Since no instructions were being patched,
no cache maintenance was needed. However, later in be0f272bfc ("arm64:
ftrace: emit ftrace-mod.o contents through code") this code was reworked
to output the trampoline instructions directly into the PLT entry but,
unfortunately, the necessary cache maintenance was overlooked.
Add a call to __flush_icache_range() after writing the new trampoline
instructions but before patching in the branch to the trampoline.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: James Morse <james.morse@arm.com>
Cc: <stable@vger.kernel.org>
Fixes: be0f272bfc ("arm64: ftrace: emit ftrace-mod.o contents through code")
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Pull power management fixes from Rafael Wysocki:
"These add a check to avoid recent suspend-to-idle power regression on
systems with NVMe drives where the PCIe ASPM policy is "performance"
(or when the kernel is built without ASPM support), fix an issue
related to frequency limits in the schedutil cpufreq governor and fix
a mistake related to the PM QoS usage in the cpufreq core introduced
recently.
Specifics:
- Disable NVMe power optimization related to suspend-to-idle added
recently on systems where PCIe ASPM is not able to put PCIe links
into low-power states to prevent excess power from being drawn by
the system while suspended (Rafael Wysocki).
- Make the schedutil governor handle frequency limits changes
properly in all cases (Viresh Kumar).
- Prevent the cpufreq core from treating positive values returned by
dev_pm_qos_update_request() as errors (Viresh Kumar)"
* tag 'pm-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled
PCI/ASPM: Add pcie_aspm_enabled()
cpufreq: schedutil: Don't skip freq update when limits change
cpufreq: dev_pm_qos_update_request() can return 1 on success
Pull sound fixes from Takashi Iwai:
"All small fixes targeted for stable:
- Two fixes for USB-audio with malformed descriptor, spotted by
fuzzers
- Two fixes Conexant HD-audio codec wrt power management
- Quirks for HD-audio AMD platform and HP laptop
- HD-audio memory leak fix"
* tag 'sound-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound:
ALSA: usb-audio: Fix a stack buffer overflow bug in check_input_term
ALSA: usb-audio: Fix an OOB bug in parse_audio_mixer_unit
ALSA: hda - Add a generic reboot_notify
ALSA: hda - Let all conexant codec enter D3 when rebooting
ALSA: hda/realtek - Add quirk for HP Envy x360
ALSA: hda - Fix a memory leak bug
ALSA: hda - Apply workaround for another AMD chip 1022:1487
Pull drm fixes from Dave Airlie:
"Nothing too crazy this week, one amdgpu fix to use vmalloc for a
struct that grew in size, and another MST fix for nouveau, and some
other misc fixes:
i915:
- single GVT use after free fix
scheduler:
- entity destruction race fix
amdgpu:
- struct allocation fix
- gfx9 soft recovery fix
nouveau:
- followup MST fix
ast:
- vga register race fix"
* tag 'drm-fixes-2019-08-16' of git://anongit.freedesktop.org/drm/drm:
drm/nouveau: Only recalculate PBN/VCPI on mode/connector changes
drm/ast: Fixed reboot test may cause system hanged
drm/scheduler: use job count instead of peek
drm/amd/display: use kvmalloc for dc_state (v2)
drm/amdgpu: fix gfx9 soft recovery
drm/i915: Use after free in error path in intel_vgpu_create_workload()
Recent gcc compilers (gcc 9.1) generate warnings about an out of bounds
memset, if the memset goes accross several fields of a struct. This
generated a couple of warnings on x86_64 builds in sanitize_boot_params().
Fix this by explicitly saving the fields in struct boot_params
that are intended to be preserved, and zeroing all the rest.
[ tglx: Tagged for stable as it breaks the warning free build there as well ]
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Suggested-by: H. Peter Anvin <hpa@zytor.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20190731054627.5627-2-jhubbard@nvidia.com
Vinod writes:
soundwire fixes for v5.3-rc5
Pierre sent fixes which are queued now for v5.3-rc5 are:
- regmap dependecy
- cadence register definitions
* tag 'soundwire-5.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/soundwire:
soundwire: fix regmap dependencies and align with other serial links
soundwire: cadence_master: fix definitions for INTSTAT0/1
soundwire: cadence_master: fix register definition for SLAVE_STATE
`check_input_term` recursively calls itself with input from
device side (e.g., uac_input_terminal_descriptor.bCSourceID)
as argument (id). In `check_input_term`, if `check_input_term`
is called with the same `id` argument as the caller, it triggers
endless recursive call, resulting kernel space stack overflow.
This patch fixes the bug by adding a bitmap to `struct mixer_build`
to keep track of the checked ids and stop the execution if some id
has been checked (similar to how parse_audio_unit handles unitid
argument).
Reported-by: Hui Peng <benquike@gmail.com>
Reported-by: Mathias Payer <mathias.payer@nebelwelt.net>
Signed-off-by: Hui Peng <benquike@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
