drcpusim complains about CET instructions emitted by recent gcc
Pasting from https://github.com/DynamoRIO/dynamorio/issues/4953#issuecomment-922071343
For the drcpusim failures:
<Invalid Klamath instruction "nop" @ tool.cpuid+0x1190. Aborting.>
It's not actually a nop: DR doesn't decode CET instructions yet (#4040):
bruening@ubuntu:~/dr/git/build_x86_dbg_tests$ objdump -d suite/tests/bin/tool.cpuid | grep -A 4 'main>:'
00001747 <main>:
1747: f3 0f 1e fb endbr32
174b: 8d 4c 24 04 lea 0x4(%esp),%ecx
Compiling -march=pentium2
does not remove it which seems crazy.
Compiling -fcf-protection=none
does remove it from the .o.
There are some in the test tools library and drlibc too. If I build them with -fcf-protection=none then we just have instances in _start, __libc_csu_init, etc.:
bruening@ubuntu:~/dr/git/build_x86_dbg_tests$ objdump -d suite/tests/bin/tool.cpuid | grep -B 1 endbr
00001000 <.init>:
1000: f3 0f 1e fb endbr32
--
00001190 <_start>:
1190: f3 0f 1e fb endbr32
--
126e: 66 90 xchg %ax,%ax
1270: f3 0f 1e fb endbr32
--
12bf: 90 nop
12c0: f3 0f 1e fb endbr32
--
00002630 <__libc_csu_init>:
2630: f3 0f 1e fb endbr32
--
000026a0 <__libc_csu_fini>:
26a0: f3 0f 1e fb endbr32
--
000026ac <.fini>:
26ac: f3 0f 1e fb endbr32
Adding -mno-shstk (the other option at https://man7.org/linux/man-pages/man1/gcc.1.html that mentions CET) to all 3 targets: no difference.
Linker options:
- -z cet-report=none
- -no-shstk
Tried manually:
bruening@ubuntu:~/dr/git/build_x86_dbg_tests$ /usr/bin/cc -m32 -mno-shstk -fcf-protection=none -Wl,-z -Wl,cet-report=none -Wl,--hash-style=both -m32 -rdynamic suite/tests/CMakeFiles/tool.cpuid.dir/__/__/clients/drcpusim/tests/cpuid.c.o -o suite/tests/bin/tool.cpuid /usr/lib32/libm.so /usr/lib32/libdl.so suite/tests/bin/libtools.a lib32/libdrlibc.a
bruening@ubuntu:~/dr/git/build_x86_dbg_tests$ objdump -d suite/tests/bin/tool.cpuid | grep -B 1 endbr
00001000 <_init>:
1000: f3 0f 1e fb endbr32
--
<...>
bruening@ubuntu:~/dr/git/build_x86_dbg_tests$ /usr/bin/cc -m32 -mno-shstk -fcf-protection=none -Wl,--hash-style=both -m32 -rdynamic suite/tests/CMakeFiles/tool.cpuid.dir/__/__/clients/drcpusim/tests/cpuid.c.o -o suite/tests/bin/tool.cpuid /usr/lib32/libm.so /usr/lib32/libdl.so suite/tests/bin/libtools.a lib32/libdrlibc.a -Wl,-no-shstk
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
collect2: error: ld returned 1 exit status
bruening@ubuntu:~/dr/git/build_x86_dbg_tests$ /usr/bin/cc -m32 -mno-shstk -fcf-protection=none -Wl,-no-shstk -Wl,--hash-style=both -m32 -rdynamic suite/tests/CMakeFiles/tool.cpuid.dir/__/__/clients/drcpusim/tests/cpuid.c.o -o suite/tests/bin/tool.cpuid /usr/lib32/libm.so /usr/lib32/libdl.so suite/tests/bin/libtools.a lib32/libdrlibc.a
/usr/bin/ld: attempted static link of dynamic object `/usr/lib32/libm.so'
collect2: error: ld returned 1 exit status
I can't even figure out why the Ubuntu20 VM gcc 9.3.0 has this enabled by default while my laptop 10.3.0 does not: gcc -v
doesn't show a flag for the VM that's not there for my laptop that seems at all related to CET.
So stumped at how to even disable for a CET toolchain. The easiest solution is to remove these 2 tests are just too hard to run. But once we add #4040 I think many more drcpusim tests will fail when they realize these aren't multi-byte nops from SSE but are much later. Should we add an option to drcpusim to ignore these instructions? We'd need #4040 though.