Peter Zijlstra
69d927bba3
x86/atomic: Fix smp_mb__{before,after}_atomic()
...
Recent probing at the Linux Kernel Memory Model uncovered a
'surprise'. Strongly ordered architectures where the atomic RmW
primitive implies full memory ordering and
smp_mb__{before,after}_atomic() are a simple barrier() (such as x86)
fail for:
*x = 1;
atomic_inc(u);
smp_mb__after_atomic();
r0 = *y;
Because, while the atomic_inc() implies memory order, it
(surprisingly) does not provide a compiler barrier. This then allows
the compiler to re-order like so:
atomic_inc(u);
*x = 1;
smp_mb__after_atomic();
r0 = *y;
Which the CPU is then allowed to re-order (under TSO rules) like:
atomic_inc(u);
r0 = *y;
*x = 1;
And this very much was not intended. Therefore strengthen the atomic
RmW ops to include a compiler barrier.
NOTE: atomic_{or,and,xor} and the bitops already had the compiler
barrier.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2019-06-17 12:09:59 +02:00
..
2019-05-28 17:26:44 -07:00
2018-06-03 20:40:33 +10:00
2019-05-14 19:52:48 -07:00
2019-05-07 18:02:51 -07:00
2019-06-01 15:51:31 -07:00
2019-02-12 15:20:09 +00:00
2019-06-13 10:07:19 +01:00
2018-04-12 16:08:02 +02:00
2019-06-13 03:00:30 -06:00
2019-01-08 17:15:10 -08:00
2019-05-14 10:05:18 +02:00
2018-09-09 15:08:58 -06:00
2019-06-14 17:46:14 -10:00
2018-08-10 16:09:40 -06:00
2019-05-14 19:52:49 -07:00
2018-11-07 13:29:04 +01:00
2019-04-25 15:38:12 +08:00
2019-05-14 19:52:51 -07:00
2019-05-08 13:41:58 -04:00
2019-05-22 09:08:20 -05:00
2019-04-19 12:46:27 -06:00
2019-05-20 13:18:45 -06:00
2019-05-17 19:29:40 +02:00
2018-11-25 12:25:53 -07:00
2018-11-06 07:36:22 -07:00
2017-04-06 10:55:24 +09:00
2019-01-07 15:36:11 -07:00
2019-01-16 17:42:35 +01:00
2019-05-07 19:34:17 -07:00
2019-05-31 11:27:25 +02:00
2017-01-11 09:42:59 +01:00
2019-05-23 09:27:39 -06:00
2018-09-09 15:08:58 -06:00
2018-07-15 13:55:44 +02:00
2019-04-23 23:30:07 +02:00
2019-04-17 09:39:22 +02:00
2018-12-18 14:55:22 +01:00
2019-05-03 13:16:18 -07:00
2019-05-03 16:47:54 +02:00
2018-03-05 14:41:38 -08:00
2018-09-09 15:08:58 -06:00
2017-07-25 19:56:23 +01:00
2019-02-07 12:56:23 -07:00
2019-02-01 16:05:03 -07:00
2019-01-22 13:37:25 +01:00
2018-12-11 16:28:41 -08:00
2018-09-09 15:08:58 -06:00
2019-05-21 00:02:59 +09:00
2019-05-21 09:31:28 -06:00
2018-07-26 16:21:09 -06:00
2019-02-11 08:27:47 -07:00
2018-11-20 09:30:43 -07:00
2017-04-16 10:06:33 -06:00
2019-05-07 16:06:28 -06:00
2019-06-03 11:55:48 +02:00
2018-09-09 15:08:58 -06:00
2018-06-15 18:11:26 -03:00
2018-01-15 14:29:42 -08:00
2019-05-16 11:57:16 -07:00
2016-09-20 12:38:48 -06:00
2018-09-09 15:08:58 -06:00
2019-01-14 08:37:17 -07:00
2018-09-09 15:08:58 -06:00
2018-09-04 23:37:38 +02:00
2018-09-09 15:08:58 -06:00
2019-05-22 12:18:20 -07:00
2018-12-21 12:44:41 -08:00
2018-09-28 15:14:54 +02:00
2017-10-30 21:37:53 +09:00
2018-09-09 15:08:58 -06:00
2018-10-25 06:50:48 -07:00
2018-08-18 12:30:42 -07:00
2018-12-06 12:29:47 +00:00
2019-01-27 12:29:37 +01:00
2019-05-03 02:54:58 +10:00
2017-09-08 18:26:51 -07:00
2019-05-08 12:42:50 -07:00
2018-03-26 12:13:21 -04:00
2018-02-23 08:25:45 -07:00
2019-03-26 14:37:06 -07:00
2018-06-04 14:02:11 -07:00
2018-11-20 09:30:43 -07:00
2019-01-27 12:29:37 +01:00
2019-02-27 09:00:02 -05:00
2019-02-22 08:54:09 -07:00
2019-04-25 11:37:42 +02:00
2018-12-18 16:13:04 +01:00
2019-04-03 11:55:47 +02:00
2019-05-08 17:13:35 -07:00
2019-05-23 09:23:11 -06:00
2018-10-07 09:16:50 -06:00
2019-04-08 14:13:43 +07:00
2019-05-14 09:47:45 -07:00
2019-02-04 21:23:59 -05:00
2019-04-17 10:37:23 -07:00
2019-02-22 08:50:17 -07:00
2019-05-15 16:05:47 -07:00
2019-05-10 13:24:53 -04:00
2019-05-21 10:11:19 +02:00
2019-03-18 12:00:28 -06:00
2019-05-17 10:33:30 -07:00
2019-06-01 15:51:31 -07:00
2018-09-09 15:08:58 -06:00
2019-03-02 15:28:20 +01:00
2019-05-16 09:05:12 +02:00
2019-03-18 13:45:27 +01:00
2019-02-03 18:06:19 -08:00
2016-06-30 13:07:33 -06:00
2019-04-11 12:37:02 -06:00
2019-06-17 12:09:59 +02:00
2017-07-14 13:51:27 -06:00
2017-07-14 13:51:27 -06:00
2017-07-14 13:51:28 -06:00
2016-12-14 16:30:12 -08:00
2019-05-08 12:42:50 -07:00
2016-10-24 08:12:35 -02:00
2019-05-24 09:09:32 -06:00
2017-07-14 13:51:30 -06:00
2019-04-19 10:56:04 +02:00
2017-07-14 13:51:30 -06:00
2017-07-14 13:51:31 -06:00
2017-07-14 13:51:34 -06:00
2018-05-14 16:44:41 +02:00
2017-07-14 13:51:31 -06:00
2019-05-09 08:40:55 -07:00
2019-03-10 12:47:57 -07:00
2017-07-14 13:51:33 -06:00
2019-02-11 08:23:07 -07:00
2016-08-14 11:52:40 -06:00
2019-05-08 12:42:50 -07:00
2018-09-09 14:46:44 -06:00
2017-07-14 13:51:34 -06:00
2017-07-14 13:51:35 -06:00
2017-07-14 13:51:36 -06:00
2017-07-14 13:51:36 -06:00
2017-07-14 13:51:37 -06:00
2017-07-14 13:51:37 -06:00
2019-05-14 07:57:29 -07:00
2017-07-14 13:51:38 -06:00
2017-07-14 13:51:38 -06:00
2017-07-14 13:51:39 -06:00
2017-07-14 13:51:38 -06:00
2018-07-18 08:44:22 -06:00
2017-09-27 16:03:45 -05:00
2017-07-14 13:51:41 -06:00
2018-01-24 12:32:58 +01:00
2017-07-14 13:51:42 -06:00
2017-07-14 13:51:42 -06:00
2017-07-14 13:51:43 -06:00
2017-07-14 13:51:43 -06:00
2018-08-30 10:59:48 -07:00
2018-12-06 13:57:03 +01:00
2019-05-07 19:34:17 -07:00
2017-07-14 13:51:45 -06:00
2017-07-14 13:51:45 -06:00
2017-07-14 13:51:46 -06:00
2017-05-16 08:44:19 -03:00
2019-04-05 16:02:30 -10:00
2017-07-14 13:51:47 -06:00
2019-04-01 14:33:42 -06:00
2019-04-23 13:34:17 +01:00
2017-07-14 13:57:54 -06:00
2018-06-26 09:01:27 -06:00
2019-04-11 12:37:03 -06:00
2017-07-14 13:57:56 -06:00
2019-05-03 10:49:17 -04:00
2017-07-14 13:57:56 -06:00
2017-07-14 13:57:57 -06:00
2017-07-14 13:57:58 -06:00
2017-07-14 13:57:58 -06:00
2017-10-19 12:56:44 -06:00
2017-07-14 13:57:59 -06:00
2019-04-09 19:27:42 +02:00
2017-07-06 08:23:30 +02:00
2017-09-08 18:26:48 -07:00
2017-07-14 13:58:02 -06:00
2018-06-15 13:36:08 +02:00
2017-07-14 13:58:03 -06:00
2019-04-26 13:57:55 +01:00
2017-07-14 13:58:04 -06:00
2019-03-25 10:34:55 -06:00
2017-07-14 13:58:04 -06:00
2017-07-14 13:58:05 -06:00
2017-07-14 13:58:06 -06:00
2017-07-14 13:58:06 -06:00
2017-07-14 13:58:07 -06:00
2019-04-11 12:37:03 -06:00
2019-02-06 16:44:16 -07:00
2016-10-24 08:12:35 -02:00
2017-11-20 10:45:50 -07:00
2018-10-11 11:28:53 -05:00
2017-05-24 13:01:27 -03:00
2017-07-14 13:58:14 -06:00
2017-07-14 13:58:08 -06:00
2019-04-11 12:37:03 -06:00
2018-06-08 10:24:27 -06:00
2018-05-08 09:16:41 -06:00
2019-04-11 12:37:03 -06:00
2017-07-14 13:58:11 -06:00
2017-07-14 13:58:11 -06:00
2017-07-14 13:58:12 -06:00