[ Upstream commit aaa3149bbe ]
We try to hold TX virtqueue mutex in vhost_net_rx_peek_head_len()
after RX virtqueue mutex is held in handle_rx(). This requires an
appropriate lock nesting notation to calm down deadlock detector.
Fixes: 0308813724 ("vhost_net: basic polling support")
Reported-by: syzbot+7f073540b1384a614e09@syzkaller.appspotmail.com
Signed-off-by: Jason Wang <jasowang@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 1cb81756b7)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Ia72badca8f5f92095cfad91051fe47f76e6b03b7
commit e9cb423913 upstream.
We used to call mutex_lock() in vhost_dev_lock_vqs() which tries to
hold mutexes of all virtqueues. This may confuse lockdep to report a
possible deadlock because of trying to hold locks belong to same
class. Switch to use mutex_lock_nested() to avoid false positive.
Fixes: 6b1e6cc785 ("vhost: new device IOTLB API")
Reported-by: syzbot+dbb7c1161485e61b0241@syzkaller.appspotmail.com
Signed-off-by: Jason Wang <jasowang@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit bd3ccdc6f9)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I20829e721706c10ef1696de0d78dd99110e40a6b
commit a72b69dc08 upstream.
The vhost_vsock->guest_cid field is uninitialized when /dev/vhost-vsock
is opened until the VHOST_VSOCK_SET_GUEST_CID ioctl is called.
kvmalloc(..., GFP_KERNEL | __GFP_RETRY_MAYFAIL) does not zero memory.
All other vhost_vsock fields are initialized explicitly so just
initialize this field too.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Cc: Daniel Verkamp <dverkamp@chromium.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 258d8549b5)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I230006942ceb015f6549449160205cbb501b8fa6
[ Upstream commit 8b949bef91 ]
We check tx avail through vhost_enable_notify() in the past which is
wrong since it only checks whether or not guest has filled more
available buffer since last avail idx synchronization which was just
done by vhost_vq_avail_empty() before. What we really want is checking
pending buffers in the avail ring. Fix this by calling
vhost_vq_avail_empty() instead.
This issue could be noticed by doing netperf TCP_RR benchmark as
client from guest (but not host). With this fix, TCP_RR from guest to
localhost restores from 1375.91 trans per sec to 55235.28 trans per
sec on my laptop (Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz).
Fixes: 0308813724 ("vhost_net: basic polling support")
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit f5755c0e87)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Ia61d229fb5c82e64fd1c30aff06a9f938d20e467
commit 499fde662f upstream.
As reported by Michal, vsock_stream_sendmsg() could still
sleep at vsock_stream_has_space() after prepare_to_wait():
vsock_stream_has_space
vmci_transport_stream_has_space
vmci_qpair_produce_free_space
qp_lock
qp_acquire_queue_mutex
mutex_lock
Just switch to the new wait API like we did for commit
d9dc8b0f8b ("net: fix sleeping for sk_wait_event()").
Reported-by: Michal Kubecek <mkubecek@suse.cz>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Jorgen Hansen <jhansen@vmware.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Claudio Imbrenda <imbrenda@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Cc: "Jorgen S. Hansen" <jhansen@vmware.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 6be6e48daa)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ie0ac0685071c2034c9565e2554ff5e41f6cfd619
[ Upstream commit 380feae0de ]
Otherwise we'll leave the packets queued until releasing vsock device.
E.g., if guest is slow to start up, resulting ETIMEDOUT on connect, guest
will get the connect requests from failed host sockets.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
Signed-off-by: Peng Tao <bergwolf@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 98d20e5902)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I0685ec15925c662d4a51cdd24fc4f26a8c7ff723
commit cda8bba0f9 upstream.
Currently, under certain circumstances vhost_init_is_le does just a part
of the initialization job, and depends on vhost_reset_is_le being called
too. For this reason vhost_vq_init_access used to call vhost_reset_is_le
when vq->private_data is NULL. This is not only counter intuitive, but
also real a problem because it breaks vhost_net. The bug was introduced to
vhost_net with commit 2751c9882b ("vhost: cross-endian support for
legacy devices"). The symptom is corruption of the vq's used.idx field
(virtio) after VHOST_NET_SET_BACKEND was issued as a part of the vhost
shutdown on a vq with pending descriptors.
Let us make sure the outcome of vhost_init_is_le never depend on the state
it is actually supposed to initialize, and fix virtio_net by removing the
reset from vhost_vq_init_access.
With the above, there is no reason for vhost_reset_is_le to do just half
of the job. Let us make vhost_reset_is_le reinitialize is_le.
Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Reported-by: Michael A. Tebolt <miket@us.ibm.com>
Reported-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Fixes: commit 2751c9882b ("vhost: cross-endian support for legacy devices")
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Tested-by: Michael A. Tebolt <miket@us.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 1594edd9ea)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I5436f3410dc32fe059c904c8001fbe05feae756d
[ Upstream commit 0516ffd88f ]
Propagate the error when vhost_vq_init_access() fails and set
vq->private_data to NULL.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit ae36f6a65a)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I1e062ae003b92d000967921014a31bfc574632d8
[ Upstream commit 6c083c2b8a ]
Multi vsocks may setup the same cid at the same time.
Signed-off-by: Gao feng <omarapazanadi@gmail.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
(cherry picked from commit 2d5a1b3179)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ic2fa852a2080a4df1cb3b823c02d65a354c15b34
local_addr.svm_cid is host cid. We should check guest cid instead,
which is remote_addr.svm_cid. Otherwise we end up resetting all
connections to all guests.
Cc: stable@vger.kernel.org [4.8+]
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peng Tao <bergwolf@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit c4587631c7)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ib9bf9e1f0befa71f4ebc0ff0cdcef7115a41b110
commit f83f12d660 upstream.
These fields are 64 bit, using le32_to_cpu and friends
on these will not do the right thing.
Fix this up.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit e80ceb2da5)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ieccae61162243040f9ad29dc465a50be7f45ee62
If a pending socket is marked as rejected, we will decrease the
sk_ack_backlog twice. So don't decrement it for rejected sockets
in vsock_pending_work().
Testing of the rejected socket path was done through code
modifications.
Reported-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jorgen Hansen <jhansen@vmware.com>
Reviewed-by: Adit Ranadive <aditr@vmware.com>
Reviewed-by: Aditya Sarwade <asarwade@vmware.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 1190cfdb1a)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ic610a13e81f53fc570e037d0685c1263dc646e70
Remove unnecessary use of enable/disable callback notifications
and the incorrect more space available check.
The virtio_transport_tx_work handles when the TX virtqueue
has more buffers available.
Signed-off-by: Gerard Garcia <ggarcia@deic.uab.cat>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 21bc54fc0c)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Id76bb00fb4393add39d876e29b0e20617c6033e7
Stash the packet length in a local variable before handing over
ownership of the packet to virtio_transport_recv_pkt() or
virtio_transport_free_pkt().
This patch solves the use-after-free since pkt is no longer guaranteed
to be alive.
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 3fda5d6e58)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I2a6a8b2eb1b647645ff7c76a37f61dce3b0fab9f
Use kvfree() instead of open-coding it.
Signed-off-by: Wei Yongjun <weiyj.lk@gmail.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit b226acab2f)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ie270449a727baf74ee547137928e166e772b62ad
vringh is pulled in by caif and mic, but the other
vhost config does not need to be there.
In particular, it makes no sense to have vhost net/scsi/sock
under caif/mic.
Create a separate Kconfig file and put vringh bits there.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 4d93824561)
[astrachan: Backported around no mic driver on 4.4]
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I6e630228bf91a8569bc665e9ad5d399eca8f7384
vringh isn't used by vhost net or scsi - it's used
by CAIF only at the moment. Drop the dependency.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 6190efb08c)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I9db24d3ced664637cffcf27fde8a1c08962bbebe
vringh isn't used by vhost net or scsi - it's used
by CAIF only at the moment. Drop the dependency.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 6190efb08c)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I2373bf9a5cd21d350af3aca957df811c6aaeae63
Detect and fail early if long wrap around is triggered.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit ec33d031a1)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Id71c1ea0355ce3e403bb5865dc3056d197fe218b
Enable virtio-vsock and vhost-vsock.
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 304ba62fd4)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I0b0f08cd28a94516903dbf3452e5999375e0f85a
VM sockets vhost transport implementation. This driver runs on the
host.
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 433fc58e6b)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Id90d852ffd498a7d89075cddb6d8ed0b9af5e69f
VM sockets virtio transport implementation. This driver runs in the
guest.
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 0ea9e1d3a9)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ib12e1e4d21183ac3d917316566694758717596bd
This module contains the common code and header files for the following
virtio_transporto and vhost_vsock kernel modules.
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 06a8fc7836)
[astrachan: Backported around stable backport 62209d1 ("vsock: split
dwork to avoid reinitializations")]
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I723c073db804663ad4bf83b657c72b16cbdb220a
The virtio transport will implement graceful shutdown and the related
SO_LINGER socket option. This requires orphaning the sock but keeping
it in the table of connections after .release().
This patch adds the vsock_remove_sock() function and leaves it up to the
transport when to remove the sock.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 6773b7dc39)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I889cdbc0b1de8d2ff54a70ab7a6b4623edb3de06
struct vsock_transport contains function pointers called by AF_VSOCK
core code. The transport may want its own transport-specific function
pointers and they can be added after struct vsock_transport.
Allow the transport to fetch vsock_transport. It can downcast it to
access transport-specific function pointers.
The virtio transport will use this.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 0b01aeb3d2)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I442706ae71dc14c70fc4033d9719134c2d034509
There are several places where the listener and pending or accept queue
child sockets are accessed at the same time. Lockdep is unhappy that
two locks from the same class are held.
Tell lockdep that it is safe and document the lock ordering.
Originally Claudio Imbrenda <imbrenda@linux.vnet.ibm.com> sent a similar
patch asking whether this is safe. I have audited the code and also
covered the vsock_pending_work() function.
Suggested-by: Claudio Imbrenda <imbrenda@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 4192f672fa)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I0cb7ee964057e9338971e1a2043ae17557feaec7
This patch tries to implement an device IOTLB for vhost. This could be
used with userspace(qemu) implementation of DMA remapping
to emulate an IOMMU for the guest.
The idea is simple, cache the translation in a software device IOTLB
(which is implemented as an interval tree) in vhost and use vhost_net
file descriptor for reporting IOTLB miss and IOTLB
update/invalidation. When vhost meets an IOTLB miss, the fault
address, size and access can be read from the file. After userspace
finishes the translation, it writes the translated address to the
vhost_net file to update the device IOTLB.
When device IOTLB is enabled by setting VIRTIO_F_IOMMU_PLATFORM all vq
addresses set by ioctl are treated as iova instead of virtual address and
the accessing can only be done through IOTLB instead of direct userspace
memory access. Before each round or vq processing, all vq metadata is
prefetched in device IOTLB to make sure no translation fault happens
during vq processing.
In most cases, virtqueues are contiguous even in virtual address space.
The IOTLB translation for virtqueue itself may make it a little
slower. We might add fast path cache on top of this patch.
Signed-off-by: Jason Wang <jasowang@redhat.com>
[mst: use virtio feature bit: VHOST_F_DEVICE_IOTLB -> VIRTIO_F_IOMMU_PLATFORM ]
[mst: fix build warnings ]
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
[ weiyj.lk: missing unlock on error ]
Signed-off-by: Wei Yongjun <weiyj.lk@gmail.com>
(cherry picked from commit 6b1e6cc785)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + vsock adb tunnel
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I10e4e64d6bc9b36a0d9b444c2319e290921c63c6
Current pre-sorted memory region array has some limitations for future
device IOTLB conversion:
1) need extra work for adding and removing a single region, and it's
expected to be slow because of sorting or memory re-allocation.
2) need extra work of removing a large range which may intersect
several regions with different size.
3) need trick for a replacement policy like LRU
To overcome the above shortcomings, this patch convert it to interval
tree which can easily address the above issue with almost no extra
work.
The patch could be used for:
- Extend the current API and only let the userspace to send diffs of
memory table.
- Simplify Device IOTLB implementation.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit a9709d6874)
[astrachan: Backported around stable backport 711df71 ("vhost_net: stop
device during reset owner")]
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I51c7c7229908a5ce1f082a80eeda5a01e85c2234
This patch introduces vhost memory accessors which were just wrappers
for userspace address access helpers. This is a requirement for vhost
device iotlb implementation which will add iotlb translations in those
accessors.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit bfe2bc5128)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Ia67c171384109e646f55027a71dd8df9c6b9c61a
We don't stop rx polling socket during rx processing, this will lead
unnecessary wakeups from under layer net devices (E.g
sock_def_readable() form tun). Rx will be slowed down in this
way. This patch avoids this by stop polling socket during rx
processing. A small drawback is that this introduces some overheads in
light load case because of the extra start/stop polling, but single
netperf TCP_RR does not notice any change. In a super heavy load case,
e.g using pktgen to inject packet to guest, we get about ~8.8%
improvement on pps:
before: ~1240000 pkt/s
after: ~1350000 pkt/s
Signed-off-by: Jason Wang <jasowang@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 8241a1e466)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Ia51c61a6c1976b6f6406342f369ffc365d964078
The vsock_transport structure is never modified, so declare it as const.
Done with the help of Coccinelle.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 56130915bb)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I59ffded185fbf22aaeb39753870ef9c866ab1a2a
We use spinlock to synchronize the work list now which may cause
unnecessary contentions. So this patch switch to use llist to remove
this contention. Pktgen tests shows about 5% improvement:
Before:
~1300000 pps
After:
~1370000 pps
Signed-off-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 04b96e5528)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Icf032cf2010eaedd92dc5906d454274709b4a9b4
We used to implement the work flushing through tracking queued seq,
done seq, and the number of flushing. This patch simplify this by just
implement work flushing through another kind of vhost work with
completion. This will be used by lockless enqueuing patch.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 7235acdb11)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: Id3903050706dd734a0217c56ad8ca99b2b22471e
If skb_recv_datagram returns an skb, we should ignore the err
value returned. Otherwise, datagram receives will return EAGAIN
when they have to wait for a datagram.
Acked-by: Adit Ranadive <aditr@vmware.com>
Signed-off-by: Jorgen Hansen <jhansen@vmware.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 9c995cc9a2)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I69d487529656cb2fe8be9c2cef0db440d4db5cac
When a thread is prepared for waiting by calling prepare_to_wait, sleeping
is not allowed until either the wait has taken place or finish_wait has
been called. The existing code in af_vsock imposed unnecessary no-sleep
assumptions to a broad list of backend functions.
This patch shrinks the influence of prepare_to_wait to the area where it
is strictly needed, therefore relaxing the no-sleep restriction there.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.vnet.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit f7f9b5e7f8)
[astrachan: Backported around stable backport 3223ea1 ("vsock: use new
wait API for vsock_stream_sendmsg()")]
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: Ic1d7bae4b1187ad48194bf4d0b1dd09ab0275734
This patch tries to poll for new added tx buffer or socket receive
queue for a while at the end of tx/rx processing. The maximum time
spent on polling were specified through a new kind of vring ioctl.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 0308813724)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I93db6c14e39b2db04e047ee2e0dce3af5436d95e
This patch introduces a helper which will return true if we're sure
that the available ring is empty for a specific vq. When we're not
sure, e.g vq access failure, return false instead. This could be used
for busy polling code to exit the busy loop.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit d4a60603fa)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I1d4e972543470530a5bc37e19f4199b1f345e042
This path introduces a helper which can give a hint for whether or not
there's a work queued in the work list. This could be used for busy
polling code to exit the busy loop.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 526d3e7ff5)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I2717a212f888b0d3ea78f1ce0bcc21ec9dee5050
Looking at how callers use this, maybe we should just rename init_used
to vhost_vq_init_access. The _used suffix was a hint that we
access the vq used ring. But maybe what callers care about is
that it must be called after access_ok.
Also, this function manipulates the vq->is_le field which isn't related
to the vq used ring.
This patch simply renames vhost_init_used() to vhost_vq_init_access() as
suggested by Michael.
No behaviour change.
Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit 80f7d0301e)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: If8ef33d7f00e515afbff3adc7b00607c4a58fdf6
The default use case for vhost is when the host and the vring have the
same endianness (default native endianness). But there are cases where
they differ and vhost should byteswap when accessing the vring.
The first case is when the host is big endian and the vring belongs to
a virtio 1.0 device, which is always little endian.
This is covered by the vq->is_le field. This field is initialized when
userspace calls the VHOST_SET_FEATURES ioctl. It is reset when the device
stops.
We already have a vhost_init_is_le() helper, but the reset operation is
opencoded as follows:
vq->is_le = virtio_legacy_is_little_endian();
It isn't clear that we are resetting vq->is_le here.
This patch moves the code to a helper with a more explicit name.
The other case where we may have to byteswap is when the architecture can
switch endianness at runtime (bi-endian). If endianness differs in the host
and in the guest, then legacy devices need to be used in cross-endian mode.
This mode is available with CONFIG_VHOST_CROSS_ENDIAN_LEGACY=y, which
introduces a vq->user_be field. Userspace may enable cross-endian mode
by calling the SET_VRING_ENDIAN ioctl before the device is started. The
cross-endian mode is disabled when the device is stopped.
The current names of the helpers that manipulate vq->user_be are unclear.
This patch renames those helpers to clearly show that this is cross-endian
stuff and with explicit enable/disable semantics.
No behaviour change.
Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit c507203756)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I42271a7cbc4ecc3826116943805bc59d4f8bd192
We don't want side effects. If something fails, we rollback vq->is_le to
its previous value.
Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
(cherry picked from commit e1f33be918)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I18c57e5c78aa3e89d267213770f915a5a5c76100
checkpatch.pl wants arrays of strings declared as follows:
static const char * const names[] = { "vq-1", "vq-2", "vq-3" };
Currently the find_vqs() function takes a const char *names[] argument
so passing checkpatch.pl's const char * const names[] results in a
compiler error due to losing the second const.
This patch adjusts the find_vqs() prototype and updates all virtio
transports. This makes it possible for virtio_balloon.c, virtio_input.c,
virtgpu_kms.c, and virtio_rpmsg_bus.c to use the checkpatch.pl-friendly
type.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Bjorn Andersson <bjorn.andersson@sonymobile.com>
(cherry picked from commit f7ad26ff95)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Change-Id: I23513ea85e7a43efd0c604fc4445b301b4f610ba
We do not often add/delete a napi context.
Moving napi_hash[] into read mostly section avoids potential false sharing.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 6180d9de61)
Bug: 121166534
Test: Ran cuttlefish with android-4.4 + VSOCKETS, VMWARE_VMCI_VSOCKETS
Signed-off-by: Alistair Strachan <astrachan@google.com>
Change-Id: I891b9dfef88c44f6480842754cd7d81e7a250792
This option is reverted in change Iebcb0cd9982f36c4bd2552811f9147325a291db0.
Bug: 71728490
Change-Id: If25dfd56abed16bfe579840e4a52d6bedbae69ca
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
It's only in DIO before, complement for BIO.
Bug: 120445624
Change-Id: I90b6fb15e355978da8805ed6306c595819be989d
Signed-off-by: Peng Zhou <peng.zhou@mediatek.com>
Changes in 4.4.170
USB: hso: Fix OOB memory access in hso_probe/hso_get_config_data
xhci: Don't prevent USB2 bus suspend in state check intended for USB3 only
USB: serial: option: add GosunCn ZTE WeLink ME3630
USB: serial: option: add HP lt4132
USB: serial: option: add Simcom SIM7500/SIM7600 (MBIM mode)
USB: serial: option: add Fibocom NL668 series
USB: serial: option: add Telit LN940 series
mmc: core: Reset HPI enabled state during re-init and in case of errors
mmc: omap_hsmmc: fix DMA API warning
gpio: max7301: fix driver for use with CONFIG_VMAP_STACK
Drivers: hv: vmbus: Return -EINVAL for the sys files for unopened channels
x86/mtrr: Don't copy uninitialized gentry fields back to userspace
drm/ioctl: Fix Spectre v1 vulnerabilities
ip6mr: Fix potential Spectre v1 vulnerability
ipv4: Fix potential Spectre v1 vulnerability
ax25: fix a use-after-free in ax25_fillin_cb()
ibmveth: fix DMA unmap error in ibmveth_xmit_start error path
ieee802154: lowpan_header_create check must check daddr
ipv6: explicitly initialize udp6_addr in udp_sock_create6()
isdn: fix kernel-infoleak in capi_unlocked_ioctl
netrom: fix locking in nr_find_socket()
packet: validate address length
packet: validate address length if non-zero
sctp: initialize sin6_flowinfo for ipv6 addrs in sctp_inet6addr_event
vhost: make sure used idx is seen before log in vhost_add_used_n()
VSOCK: Send reset control packet when socket is partially bound
xen/netfront: tolerate frags with no data
gro_cell: add napi_disable in gro_cells_destroy
sock: Make sock->sk_stamp thread-safe
ALSA: rme9652: Fix potential Spectre v1 vulnerability
ALSA: emu10k1: Fix potential Spectre v1 vulnerabilities
ALSA: pcm: Fix potential Spectre v1 vulnerability
ALSA: emux: Fix potential Spectre v1 vulnerabilities
ALSA: hda: add mute LED support for HP EliteBook 840 G4
ALSA: hda/tegra: clear pending irq handlers
USB: serial: pl2303: add ids for Hewlett-Packard HP POS pole displays
USB: serial: option: add Fibocom NL678 series
usb: r8a66597: Fix a possible concurrency use-after-free bug in r8a66597_endpoint_disable()
Input: elan_i2c - add ACPI ID for touchpad in ASUS Aspire F5-573G
KVM: x86: Use jmp to invoke kvm_spurious_fault() from .fixup
perf pmu: Suppress potential format-truncation warning
ext4: fix possible use after free in ext4_quota_enable
ext4: missing unlock/put_page() in ext4_try_to_write_inline_data()
ext4: fix EXT4_IOC_GROUP_ADD ioctl
ext4: force inode writes when nfsd calls commit_metadata()
spi: bcm2835: Fix race on DMA termination
spi: bcm2835: Fix book-keeping of DMA termination
spi: bcm2835: Avoid finishing transfer prematurely in IRQ mode
cdc-acm: fix abnormal DATA RX issue for Mediatek Preloader.
media: vivid: free bitmap_cap when updating std/timings/etc.
MIPS: Ensure pmd_present() returns false after pmd_mknotpresent()
MIPS: Align kernel load address to 64KB
CIFS: Fix error mapping for SMB2_LOCK command which caused OFD lock problem
x86/kvm/vmx: do not use vm-exit instruction length for fast MMIO when running nested
spi: bcm2835: Unbreak the build of esoteric configs
powerpc: Fix COFF zImage booting on old powermacs
ARM: imx: update the cpu power up timing setting on i.mx6sx
Input: restore EV_ABS ABS_RESERVED
checkstack.pl: fix for aarch64
xfrm: Fix bucket count reported to userspace
scsi: bnx2fc: Fix NULL dereference in error handling
Input: omap-keypad - fix idle configuration to not block SoC idle states
scsi: zfcp: fix posting too many status read buffers leading to adapter shutdown
fork: record start_time late
hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined
mm, devm_memremap_pages: mark devm_memremap_pages() EXPORT_SYMBOL_GPL
mm, devm_memremap_pages: kill mapping "System RAM" support
sunrpc: fix cache_head leak due to queued request
sunrpc: use SVC_NET() in svcauth_gss_* functions
crypto: x86/chacha20 - avoid sleeping with preemption disabled
ALSA: cs46xx: Potential NULL dereference in probe
ALSA: usb-audio: Avoid access before bLength check in build_audio_procunit()
ALSA: usb-audio: Fix an out-of-bound read in create_composite_quirks
dlm: fixed memory leaks after failed ls_remove_names allocation
dlm: possible memory leak on error path in create_lkb()
dlm: lost put_lkb on error path in receive_convert() and receive_unlock()
dlm: memory leaks on error path in dlm_user_request()
gfs2: Fix loop in gfs2_rbm_find
b43: Fix error in cordic routine
9p/net: put a lower bound on msize
iommu/vt-d: Handle domain agaw being less than iommu agaw
ceph: don't update importing cap's mseq when handing cap export
genwqe: Fix size check
intel_th: msu: Fix an off-by-one in attribute store
power: supply: olpc_battery: correct the temperature units
Linux 4.4.170
Change-Id: I1b2927583f8853bfeb3ad11d045c2cf5c5c926f3
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
