Files
linux/include/linux
Matt Fleming a55c7454a8 sched/topology: Improve load balancing on AMD EPYC systems
SD_BALANCE_{FORK,EXEC} and SD_WAKE_AFFINE are stripped in sd_init()
for any sched domains with a NUMA distance greater than 2 hops
(RECLAIM_DISTANCE). The idea being that it's expensive to balance
across domains that far apart.

However, as is rather unfortunately explained in:

  commit 32e45ff43e ("mm: increase RECLAIM_DISTANCE to 30")

the value for RECLAIM_DISTANCE is based on node distance tables from
2011-era hardware.

Current AMD EPYC machines have the following NUMA node distances:

 node distances:
 node   0   1   2   3   4   5   6   7
   0:  10  16  16  16  32  32  32  32
   1:  16  10  16  16  32  32  32  32
   2:  16  16  10  16  32  32  32  32
   3:  16  16  16  10  32  32  32  32
   4:  32  32  32  32  10  16  16  16
   5:  32  32  32  32  16  10  16  16
   6:  32  32  32  32  16  16  10  16
   7:  32  32  32  32  16  16  16  10

where 2 hops is 32.

The result is that the scheduler fails to load balance properly across
NUMA nodes on different sockets -- 2 hops apart.

For example, pinning 16 busy threads to NUMA nodes 0 (CPUs 0-7) and 4
(CPUs 32-39) like so,

  $ numactl -C 0-7,32-39 ./spinner 16

causes all threads to fork and remain on node 0 until the active
balancer kicks in after a few seconds and forcibly moves some threads
to node 4.

Override node_reclaim_distance for AMD Zen.

Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Suravee.Suthikulpanit@amd.com
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas.Lendacky@amd.com
Cc: Tony Luck <tony.luck@intel.com>
Link: https://lkml.kernel.org/r/20190808195301.13222-3-matt@codeblueprint.co.uk
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-09-03 09:17:37 +02:00
..
2019-06-20 09:23:22 +02:00
2019-05-31 11:13:10 +02:00
2018-05-26 09:16:44 +02:00
2019-07-10 09:00:57 -06:00
2019-05-14 19:52:50 -07:00
2019-07-01 08:18:54 -06:00
2019-07-10 09:00:57 -06:00
2019-07-10 09:00:57 -06:00
2018-11-07 13:44:59 -07:00
2018-11-07 13:44:59 -07:00
2019-04-09 17:05:46 -07:00
2019-05-07 08:39:02 -06:00
2018-06-15 18:10:01 -03:00
2019-06-12 20:27:13 +02:00
2019-07-18 09:49:30 +02:00
2018-03-26 15:09:38 +02:00
2018-08-22 10:52:48 -07:00
2018-11-13 21:55:24 +01:00
2019-02-28 03:28:53 -05:00
2019-06-12 20:30:39 +02:00
2019-02-28 08:24:23 -07:00
2018-12-19 10:42:08 +01:00
2019-04-22 09:48:12 -06:00
2019-02-15 16:54:38 +01:00
2019-07-05 21:34:50 +02:00
2019-06-26 13:19:46 -07:00
2018-11-19 19:03:46 -07:00
2018-06-22 13:43:27 +09:00
2019-06-10 13:00:24 +02:00
2019-04-08 22:56:14 +02:00
2018-07-12 21:35:28 +02:00
2019-07-05 15:19:10 -07:00
2018-11-30 13:29:04 +00:00
2018-04-12 09:41:19 -07:00
2019-03-07 18:32:03 -08:00
2018-06-07 17:34:35 -07:00
2018-03-26 13:14:43 -04:00
2018-03-26 13:14:43 -04:00
2019-01-30 20:51:47 -05:00
2018-10-17 13:56:58 -07:00
2019-07-08 19:25:19 -07:00
2019-06-13 09:02:33 -04:00
2019-02-20 07:22:17 -07:00
2019-02-20 07:22:10 -07:00
2019-07-09 14:15:37 -07:00
2018-12-06 15:45:46 +01:00
2019-02-08 15:02:49 -08:00
2018-07-10 17:22:35 +02:00
2018-06-07 17:34:36 -07:00
2018-10-21 10:46:39 -04:00
2019-07-16 19:23:25 -07:00
2018-10-08 22:53:10 +11:00
2019-05-14 19:52:51 -07:00
2019-06-12 11:42:13 +02:00
2019-06-15 12:25:49 +02:00
2019-05-14 19:52:48 -07:00
2018-07-20 01:11:45 +02:00
2018-05-11 17:28:45 -07:00
2019-04-02 17:57:35 +02:00
2019-05-08 22:14:36 +02:00
2018-09-25 20:17:35 -07:00
2019-05-31 12:37:46 -07:00
2019-05-16 15:51:55 -07:00
2018-07-07 17:25:23 +02:00
2019-07-09 14:32:14 -06:00
2019-02-07 16:38:35 +01:00
2019-07-08 10:51:25 +02:00
2018-06-20 11:35:56 +02:00
2019-07-12 11:05:43 -07:00
2018-10-11 09:16:44 -07:00
2019-06-27 21:22:15 +08:00
2019-02-07 00:13:27 +01:00
2019-07-07 11:50:03 +02:00
2018-12-10 10:17:45 +01:00
2019-01-11 18:05:40 -08:00
2019-06-05 11:54:38 +02:00
2018-09-11 14:11:51 +02:00
2019-05-15 17:35:54 +01:00
2019-04-09 15:14:49 -06:00
2018-12-22 12:15:29 +01:00
2019-07-10 09:00:57 -06:00
2018-05-14 09:51:34 -04:00