AVX-512 handling breaks DynamoRIO execution on Windows with full AVX-512 support
Describe the bug Execution of any application under DynamoRIO is impossible in the following scenario:
- Windows
- Full AVX-512 Support (i.e.
avx512_enabled
is set to true in proc.c) https://github.com/DynamoRIO/dynamorio/blob/cf1ec32e9b89c1d8a28e0f355b8338d4c05c0f14/core/arch/x86/proc.c#L437
Note: This behavior appears with and w/o client and is deterministic.
To Reproduce Steps to reproduce the behavior:
- Take Windows machine with AVX-512 (full) support (No VM), check flag
avx512_enabled = true
inproc.c
. - Execute any Program like
notepad.exe
3.1 In release builds, DR immediately exits 3.2 In debug builds, assertion fires:rip-relative reference out of 32-bit reach
Access Violation, as reported by WinDBG
(3aa8.3a44): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
dynamorio!encode_cti+0x3b [inlined in dynamorio!instr_encode_arch+0x102]:
00000000`7107e042 8803 mov byte ptr [rbx],al ds:00000000`00000000=??
0:000> kb
# RetAddr : Args to Child : Call Site
00 (Inline Function) : --------`-------- --------`-------- --------`-------- --------`-------- : dynamorio!encode_cti+0x3b [c:\users\felix\source\repos\dynamorio\core\arch\x86\encode.c @ 2598]
01 00000000`7107b13a : 00000000`00000000 00000000`71070c00 00000237`69b0e780 00000000`00000000 : dynamorio!instr_encode_arch+0x102 [c:\users\felix\source\repos\dynamorio\core\arch\x86\encode.c @ 2785]
02 (Inline Function) : --------`-------- --------`-------- --------`-------- --------`-------- : dynamorio!instr_encode_to_copy+0x16 [c:\users\felix\source\repos\dynamorio\core\arch\encode_shared.c @ 132]
03 00000000`710b0a51 : 00000000`00000000 00000237`69b0e780 00000000`00000000 00000000`00000002 : dynamorio!instr_encode+0x1a [c:\users\felix\source\repos\dynamorio\core\arch\encode_shared.c @ 140]
04 00000000`710b1eaa : 00000000`00000007 00000237`69b35b30 00000000`710b32e0 00000000`00000000 : dynamorio!emit_intercept_code+0x1eb1 [c:\users\felix\source\repos\dynamorio\core\win32\callback.c @ 1328]
05 00000000`710adaf9 : 00000000`7118427f 00000237`69ac1a01 00000000`710b32e0 00000000`00000000 : dynamorio!intercept_call+0x26a [c:\users\felix\source\repos\dynamorio\core\win32\callback.c @ 1743]
06 00000000`7100594a : 00000000`7118c4b8 00000000`00000218 00000000`00000000 00000000`00000000 : dynamorio!callback_interception_init_finish+0x59 [c:\users\felix\source\repos\dynamorio\core\win32\callback.c @ 7747]
07 00000000`7109b40c : 000000a1`0510f700 00000000`00000000 00000000`00000000 00000237`67ea0031 : dynamorio!dynamorio_app_init+0x23a [c:\users\felix\source\repos\dynamorio\core\dynamo.c @ 692]
08 00000000`710d28e0 : 00007ff9`1c1d9770 00000000`710d28d0 00000000`00000000 00000000`00000000 : dynamorio!auto_setup+0x1c [c:\users\felix\source\repos\dynamorio\core\arch\x86_code.c @ 169]
09 00007ff9`1c1d9770 : 00000000`710d28d0 00000000`00000000 00000000`00000000 00000000`00000000 : dynamorio!dynamo_auto_start+0x10 [C:\Users\felix\CMakeBuilds\1ac36d74-80db-f93a-9dba-3c80b8e1e38c\build\x64-Release\core\x86.asm_core.s @ 1599]
0a 00000000`710d28d0 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!GetProcAddressStub
0b 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : dynamorio!dynamo_auto_start [C:\Users\felix\CMakeBuilds\1ac36d74-80db-f93a-9dba-3c80b8e1e38c\build\x64-Release\core\x86.asm_core.s @ 1591]
Debug Output with loglevel 3
Versions
- All versions newer than (including) cronbuild-7.91.18109
- Prior Versions are not affected
- When manually disabling AVX512 support by setting
avx512_enabled = false
(inproc.c
). Dynamorio is executed correctly
cc @TomSie, @jones718