arm64: fix KASLR boot-time I-cache maintenance

Commit f80fb3a3d50843a4 ("arm64: add support for kernel ASLR") missed a
DSB necessary to complete I-cache maintenance in the primary boot path,
and hence stale instructions may still be present in the I-cache and may
be executed until the I-cache maintenance naturally completes.

Since commit 8ec41987436d566f ("arm64: mm: ensure patched kernel text is
fetched from PoU"), all CPUs invalidate their I-caches after their MMU
is enabled. Prior a CPU's MMU having been enabled, arbitrary lines may
have been fetched from the PoC into I-caches. We never patch text
expected to be executed with the MMU off. Thus, it is unnecessary to
perform broadcast I-cache maintenance in the primary boot path.

This patch reduces the scope of the I-cache maintenance to the local
CPU, and adds the missing DSB with similar scope, matching prior
maintenance in the primary boot path.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Ard Biesehvuel <ard.biesheuvel@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit b90b4a608ea2401cc491828f7a385edd2e236e37)
Signed-off-by: Alex Shi <alex.shi@linaro.org>

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 319f896c6e744a3ee47f4fede17a75bee4d7d9e2..a88a15447c3bb382970a940b39fb9d6e0e6b9529 100644 (file)
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -740,8 +740,9 @@ __enable_mmu:
 
        msr     sctlr_el1, x19                  // re-enable the MMU
        isb
-       ic      ialluis                         // flush instructions fetched
-       isb                                     // via old mapping
+       ic      iallu                           // flush instructions fetched
+       dsb     nsh                             // via old mapping
+       isb
        add     x27, x27, x23                   // relocated __mmap_switched
 #endif
        br      x27