Michel Lespinasse 38a76013ad mm: avoid taking rmap locks in move_ptes() ...

During mremap(), the destination VMA is generally placed after the
original vma in rmap traversal order: in move_vma(), we always have
new_pgoff >= vma->vm_pgoff, and as a result new_vma->vm_pgoff >=
vma->vm_pgoff unless vma_merge() merged the new vma with an adjacent one.

When the destination VMA is placed after the original in rmap traversal
order, we can avoid taking the rmap locks in move_ptes().

Essentially, this reintroduces the optimization that had been disabled in
"mm anon rmap: remove anon_vma_moveto_tail".  The difference is that we
don't try to impose the rmap traversal order; instead we just rely on
things being in the desired order in the common case and fall back to
taking locks in the uncommon case.  Also we skip the i_mmap_mutex in
addition to the anon_vma lock: in both cases, the vmas are traversed in
increasing vm_pgoff order with ties resolved in tree insertion order.

Signed-off-by: Michel Lespinasse <walken@google.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Daniel Santos <daniel.santos@pobox.com>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2012-10-09 16:22:42 +09:00

..

9p

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

adfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

affs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

afs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

autofs4

make get_file() return its argument

2012-09-26 21:10:25 -04:00

befs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

bfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

btrfs

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

cachefiles

fs: cachefiles: add support for large files in filesystem caching

2012-07-30 17:25:21 -07:00

ceph

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

cifs

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

coda

fs: push rcu_barrier() from deactivate_locked_super() to filesystems

2012-10-02 21:35:55 -04:00

configfs

userns: Convert configfs to use kuid and kgid where appropriate

2012-09-18 01:01:37 -07:00

cramfs

userns: Convert cramfs to use kuid/kgid where appropriate

2012-09-21 03:13:08 -07:00

debugfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace

2012-10-02 11:11:09 -07:00

devpts

VFS: Pass mount flags to sget()

2012-07-14 16:38:34 +04:00

dlm

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next

2012-10-02 13:38:27 -07:00

ecryptfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

efs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

exofs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

exportfs

switch dentry_open() to struct path, make it grab references itself

2012-07-23 00:01:29 +04:00

ext2

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

ext3

Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2012-10-04 09:14:01 -07:00

ext4

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

fat

fat: simplify writeback_inode()

2012-10-06 03:05:12 +09:00

freevxfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

fscache

FS-Cache: Fix __fscache_uncache_all_inode_pages()'s outer loop

2011-07-21 10:59:16 -07:00

fuse

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

gfs2

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

hfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

hfsplus

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

hostfs

userns: Convert hostfs to use kuid and kgid where appropriate

2012-09-21 03:13:23 -07:00

hpfs

hpfs: convert to use leXX_add_cpu()

2012-10-06 03:05:08 +09:00

hppfs

switch dentry_open() to struct path, make it grab references itself

2012-07-23 00:01:29 +04:00

hugetlbfs

mm: replace vma prio_tree with an interval tree

2012-10-09 16:22:39 +09:00

isofs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

jbd

jbd: Fix assertion failure in commit code due to lacking transaction credits

2012-09-12 15:52:03 +02:00

jbd2

Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4

2012-10-08 06:36:39 +09:00

jffs2

rbtree: move some implementation details from rbtree.h to rbtree.c

2012-10-09 16:22:32 +09:00

jfs

Merge tag 'jfs-3.7' of git://github.com/kleikamp/linux-shaggy

2012-10-03 08:48:21 -07:00

lockd

close the race in nlmsvc_free_block()

2012-09-22 20:48:20 -04:00

logfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

minix

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

ncpfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

nfs

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

nfs_common

Fix common misspellings

2011-03-31 11:26:23 -03:00

nfsd

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

nilfs2

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

nls

nls: fix (and rename) mac NLS table files and config options

2012-06-01 19:51:22 -07:00

notify

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

ntfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

ocfs2

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

omfs

omfs: convert to use beXX_add_cpu()

2012-10-06 03:05:31 +09:00

openpromfs

fs: push rcu_barrier() from deactivate_locked_super() to filesystems

2012-10-02 21:35:55 -04:00

proc

rbtree: fix incorrect rbtree node insertion in fs/proc/proc_sysctl.c

2012-10-09 16:22:32 +09:00

pstore

pstore: Avoid recursive spinlocks in the oops_in_progress case

2012-09-20 17:04:50 -07:00

qnx4

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

qnx6

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

quota

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace

2012-10-02 11:11:09 -07:00

ramfs

don't pass nameidata to ->create()

2012-07-14 16:34:47 +04:00

reiserfs

Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2012-10-04 09:14:01 -07:00

romfs

fs: push rcu_barrier() from deactivate_locked_super() to filesystems

2012-10-02 21:35:55 -04:00

squashfs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

sysfs

sysfs: Fix comment typo "sysf_create_link".

2012-09-04 16:11:31 -07:00

sysv

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

ubifs

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

udf

Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2012-10-04 09:14:01 -07:00

ufs

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

xfs

mm: kill vma flag VM_CAN_NONLINEAR

2012-10-09 16:22:17 +09:00

aio.c

aio: now fput() is OK from interrupt context; get rid of manual delayed __fput()

2012-07-22 23:57:59 +04:00

anon_inodes.c

anon_inodes: move allocation of anon_inode into ->mount()

2012-03-20 21:29:45 -04:00

attr.c

ima: add inode_post_setattr call

2012-09-07 14:57:46 -04:00

bad_inode.c

don't pass nameidata to ->create()

2012-07-14 16:34:47 +04:00

binfmt_aout.c

coredump: pass siginfo_t* to do_coredump() and below, not merely signr

2012-10-06 03:05:16 +09:00

binfmt_elf_fdpic.c

mm: kill vma flag VM_RESERVED and mm->reserved_vm counter

2012-10-09 16:22:19 +09:00

binfmt_elf.c

mm: kill vma flag VM_RESERVED and mm->reserved_vm counter

2012-10-09 16:22:19 +09:00

binfmt_em86.c

__register_binfmt() made void

2012-03-20 21:29:46 -04:00

binfmt_flat.c

coredump: pass siginfo_t* to do_coredump() and below, not merely signr

2012-10-06 03:05:16 +09:00

binfmt_misc.c

vfs: Rename end_writeback() to clear_inode()

2012-05-06 13:43:41 +08:00

binfmt_script.c

__register_binfmt() made void

2012-03-20 21:29:46 -04:00

binfmt_som.c

VM: add "vm_mmap()" helper function

2012-04-20 17:29:13 -07:00

bio-integrity.c

fs: remove the second argument of k[un]map_atomic()

2012-03-20 21:48:21 +08:00

bio.c

Merge branch 'for-linus' of git://git.kernel.dk/linux-block

2012-08-25 11:36:43 -07:00

block_dev.c

fs/block-dev.c:fix performance regression in O_DIRECT writes to md block devices

2012-08-02 09:50:39 +02:00

buffer.c

Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4

2012-10-08 06:36:39 +09:00

char_dev.c

char_dev.c: fix up some whitespace errors

2011-12-13 11:18:17 -08:00

compat_binfmt_elf.c

coredump: extend core dump note section to contain file names of mapped files

2012-10-06 03:05:17 +09:00

compat_ioctl.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

compat.c

compat: fs: Generic compat_sys_sendfile implementation

2012-10-02 21:35:55 -04:00

coredump.c

coredump: pass siginfo_t* to do_coredump() and below, not merely signr

2012-10-06 03:05:16 +09:00

coredump.h

coredump: update coredump-related headers

2012-10-06 03:05:15 +09:00

dcache.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

dcookies.c

fs: reduce the use of module.h wherever possible

2012-02-28 19:31:58 -05:00

direct-io.c

block: move down direct IO plugging

2012-08-09 15:23:09 +02:00

drop_caches.c

vmscan: change shrinker API by passing shrink_control struct

2011-05-25 08:39:26 -07:00

eventfd.c

eventfd: change int to __u64 in eventfd_signal()

2012-05-31 17:49:32 -07:00

eventpoll.c

epoll: support for disabling items, and a self-test app

2012-10-06 03:05:00 +09:00

exec.c

mm: avoid taking rmap locks in move_ptes()

2012-10-09 16:22:42 +09:00

fcntl.c

fcntl: fix misannotations

2012-09-26 22:20:20 -04:00

fhandle.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

fifo.c

fifo: Do not restart open() if it already found a partner

2012-07-16 08:33:14 -07:00

file_table.c

Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security

2012-10-02 21:38:48 -07:00

file.c

export fget_light

2012-09-26 21:10:06 -04:00

filesystems.c

vfs: convert fs_supers to hlist

2012-01-03 22:52:39 -05:00

fs_struct.c

get rid of ->mnt_longterm

2012-07-14 16:32:47 +04:00

fs-writeback.c

Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4

2012-10-08 06:36:39 +09:00

generic_acl.c

userns: Pass a userns parameter into posix_acl_to_xattr and posix_acl_from_xattr

2012-09-18 01:01:35 -07:00

inode.c

mm: replace vma prio_tree with an interval tree

2012-10-09 16:22:39 +09:00

internal.h

fs: Add freezing handling to mnt_want_write() / mnt_drop_write()

2012-07-31 09:40:38 +04:00

ioctl.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

ioprio.c

Merge branch 'for-3.5/core' of git://git.kernel.dk/linux-block

2012-05-30 08:52:42 -07:00

Kconfig

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-03-21 13:36:41 -07:00

Kconfig.binfmt

coredump: make core dump functionality optional

2012-10-06 03:05:15 +09:00

libfs.c

vfs: fix kerneldoc for generic_fh_to_parent()

2012-09-05 10:59:30 +02:00

locks.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

Makefile

coredump: make core dump functionality optional

2012-10-06 03:05:15 +09:00

mbcache.c

vmscan: change shrinker API by passing shrink_control struct

2011-05-25 08:39:26 -07:00

mount.h

get rid of magic in proc_namespace.c

2012-07-14 16:32:48 +04:00

mpage.c

fs: reduce the use of module.h wherever possible

2012-02-28 19:31:58 -05:00

namei.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

namespace.c

do_add_mount()/umount -l races

2012-09-22 20:48:18 -04:00

no-block.c

…

open.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

pipe.c

pipe(2) - race-free error recovery

2012-09-26 21:08:52 -04:00

pnode.c

VFS: Make clone_mnt()/copy_tree()/collect_mounts() return errors

2012-07-14 16:37:27 +04:00

pnode.h

vfs: switch pnode.h macros to struct mount *

2012-01-03 22:57:11 -05:00

posix_acl.c

userns: Convert vfs posix_acl support to use kuids and kgids

2012-09-18 01:01:35 -07:00

proc_namespace.c

get rid of magic in proc_namespace.c

2012-07-14 16:32:48 +04:00

read_write.c

compat: fs: Generic compat_sys_sendfile implementation

2012-10-02 21:35:55 -04:00

read_write.h

compat: fs: Generic compat_sys_sendfile implementation

2012-10-02 21:35:55 -04:00

readdir.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

select.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

seq_file.c

userns: Make seq_file's user namespace accessible

2012-08-14 21:47:55 -07:00

signalfd.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

splice.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

stack.c

fs: reduce the use of module.h wherever possible

2012-02-28 19:31:58 -05:00

stat.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-10-02 20:25:04 -07:00

statfs.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

super.c

idr: rename MAX_LEVEL to MAX_IDR_LEVEL

2012-10-06 03:04:56 +09:00

sync.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

timerfd.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

utimes.c

switch simple cases of fget_light to fdget

2012-09-26 22:20:08 -04:00

xattr_acl.c

userns: Pass a userns parameter into posix_acl_to_xattr and posix_acl_from_xattr

2012-09-18 01:01:35 -07:00

xattr.c

Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security

2012-10-02 21:38:48 -07:00