Add retpoline support

[theLOICofFRANCE]

Hi,

I could see that you are at version 4.9.92 without the support of retpoline [Linux 4.9.77].
However, I have created a port for version 4.9.74 which is available here.

My version is a port from "patch-4.9.76-77" without the following files :

a/arch/x86/mm/kaiser.c
a/arch/x86/mm/pgtable.c
a/arch/x86/kernel/irq_32.c
a/arch/x86/kernel/mcount_64.S
a/arch/x86/kernel/cpu/bugs_64.c
a/arch/x86/crypto/aesni-intel_asm.S
a/arch/x86/crypto/camellia-aesni-avx2-asm_64.S
a/arch/x86/crypto/camellia-aesni-avx-asm_64.S
a/arch/x86/crypto/crc32c-pcl-intel-asm_64.S
a/arch/x86/entry/entry_32.S
a/arch/x86/entry/entry_64.S

With the addition (BPF_JIT_ALWAYS_ON) :

a/net/socket.c
a/net/core/sysctl_net_core.c
a/net/core/filter.c
a/lib/test_bpf.c
a/kernel/bpf/core.c
a/init/Kconfig
a/arch/x86/lib/cmdline.c
a/arch/x86/include/asm/cmdline.h

Retpoline is not a blocking point like PAX_UDEREF so the longer you wait, the harder it will be for you to catch up with its integration.

I saw otherwise that you catch up the delay via the official patches, have you tried to do this as a commit for the blocking points? See this.

[matthewruffell]

So. We all want Retpoline support, and yeah, it looks like it is compatible and can be made to work. Problem is, with security in general and most importantly this patchset, the devil is in the details.

Sure, you can merge most of the patchset and probably get more protection than you get without it, but does it make it "done" or not likely to have holes? Its a tradeoff of potentially offering a false sense of security of having full Retpoline support, when it could be broken. So for now, I'm leaving it out until I've spent some time reviewing it.

I've been playing with Retpoline for a few weeks now, and I want to try and support everything, but I keep running into issues where the kernel oops because of RAP.

Lets have a look. You say you omit a/arch/x86/crypto/camellia-aesni-avx2-asm_64.S patches from your patchset. Lets have a look at the patch:

@@ -1337,7 +1338,7 @@ camellia_xts_crypt_32way:
 	vpxor 14 * 32(%rax), %ymm15, %ymm14;
 	vpxor 15 * 32(%rax), %ymm15, %ymm15;
 
-	call *%r9;
+	CALL_NOSPEC %r9;
 
 	addq $(16 * 32), %rsp;

Now, most of the early spectre mitigations is replacing all assembly "call" instructions with a macro for "CALL_NOSPEC" which performs some sanity checks and then performs a call later.

If we peek into the dappersec patchset, we see that all "call" instructions are annotated similarly with RAP:

@@ -1337,7 +1338,7 @@ camellia_xts_crypt_32way:
 	vpxor 14 * 32(%rax), %ymm15, %ymm14;
 	vpxor 15 * 32(%rax), %ymm15, %ymm15;
 
-	call *%r9;
+	pax_indirect_call "%r9", __camellia_enc_blk32;
 
 	addq $(16 * 32), %rsp;

If we go and have a look at the definition of pax_indirect_call in arch/x86/include/asm/alternative-asm.h

+.macro pax_indirect_call target extra
+#ifdef CONFIG_PAX_RAP
+	rap_call "*\target" hash=\extra
+#else
+	call *\target
+#endif
+.endm

and the macro for rap_call

+#ifdef CONFIG_PAX_RAP
+.macro rap_call target hash="" sym=""
+
+	jmp 2001f
+	.ifb \hash
+	__ASM_RAP_RET_HASH(__rap_hash_ret_\target)
+	.else
+	__ASM_RAP_RET_HASH(__rap_hash_ret_\hash)
+	.endif
+	.skip 8-(2002f-2001f),0xcc
+
+	.ifnb \sym
+	.globl \sym
+\sym :
+	.endif
+
+2001:	call \target
+2002:
+.endm

They also have "call" instructions that need to be changed to "CALL_NOSPEC", but only if CONFIG_RETPOLINE is enabled anyway.

I've been playing around with this, and have duplicated the marcos into separate standard and nospec versions. I can get it all compiling, but I still oops, which I haven't had time to debug. It could be something entirely different from the above though. There is still a lot of moving parts, and its still a really invasive change to the kernel.

I can't claim to be an expert, I'm just trying to do my best, so I'll keep looking into the bigger picture for now.

Saying that, I'll start slowly merging some Retpoline support in over the coming weeks and we can see how effective it is.

Let me know if you understand the specifics more than I do, any input is helpful.

And for anyone else reading this, like always, if you need security, then use an upstream kernel, or go and pay for a subscription of grsecurity from Open Source Security. My forward port will never match the effort and dedication from Spender or pipacs, and they understand all of this far more than I do.

I'm simply maintaining this patchset because I think it contains some interesting mitigations, and those mitigations are valuable to keep around for as long as possible. Since minipli has stopped maintaining his fork, I seem to be the only one keeping this patchset from bitrot. I'll keep at it for a little longer, but it is getting harder and harder to keep it going.