ARM: 7955/1: spinlock: ensure we have a compiler barrier before sev
authorWill Deacon <will.deacon@arm.com>
Fri, 7 Feb 2014 18:12:32 +0000 (19:12 +0100)
committerRussell King <rmk+kernel@arm.linux.org.uk>
Mon, 10 Feb 2014 11:44:50 +0000 (11:44 +0000)
When unlocking a spinlock, we require the following, strictly ordered
sequence of events:

<barrier> /* dmb */
<unlock>
<barrier> /* dsb */
<sev>

Whilst the code does indeed reflect this in terms of the architecture,
the final <barrier> + <sev> have been contracted into a single inline
asm without a "memory" clobber, therefore the compiler is at liberty to
reorder the unlock to the end of the above sequence. In such a case,
a waiting CPU may be woken up before the lock has been unlocked, leading
to extremely poor performance.

This patch reworks the dsb_sev() function to make use of the dsb()
macro and ensure ordering against the unlock.

Cc: <stable@vger.kernel.org>
Reported-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
arch/arm/include/asm/spinlock.h

index ef3c6072aa45345ae4594f22aebbe9a9ebc538f1..ac4bfae26702b0be3333c1184f1ac552f6c0eddd 100644 (file)
 
 static inline void dsb_sev(void)
 {
-#if __LINUX_ARM_ARCH__ >= 7
-       __asm__ __volatile__ (
-               "dsb ishst\n"
-               SEV
-       );
-#else
-       __asm__ __volatile__ (
-               "mcr p15, 0, %0, c7, c10, 4\n"
-               SEV
-               : : "r" (0)
-       );
-#endif
+
+       dsb(ishst);
+       __asm__(SEV);
 }
 
 /*