CRASH (any WOW64 app) on AMD Ryzen in drlibc_x86.asm:566 WOW64 mode switch

Created by: justintaft

Describe the bug

bin32\ddrun.exe crashes most of the time when operating on 32 bit applications. Maybe one out of 100 times notepad.exe launches. In my setup, Windows 10 is being virtualized by KVM. AMD 3900x Processor is being used. Older versions of DynamoRio has the same issue. bin64\drrun.exe works fine for 64 bit applications.

C:\Users\justin\Downloads\DynamoRIO-Windows-7.91.18301-1\DynamoRIO-Windows-7.91.18301-1>bin32\drrun.exe -- C:\windows\syswow64\notepad.exe

To Reproduce Steps to reproduce the behavior:

Download Window's cronbuild-7.91.18301A from https://github.com/DynamoRIO/dynamorio/releases .
Extract and cd to DynamoRIO folder.
Run the command bin32\drrun.exe -- C:\windows\syswow64\notepad.exe
Note the application likely crashes.
Repeat step 3 around 100 times, and notice notepad launches.

Please also answer these questions:

What happens when you run without any client?

No client is being used. Clients do work, but again need to run 100 times or so to get the client to run. I

What happens when you run with debug build ("-debug" flag to drrun/drconfig/drinject)?

Same issue occurs, but with more detailed error message. When notepad manages to start, it takes a longer time for the process to open.

Image showing the error message:

Image showing notepad.exe does launch once in a while:

Expected behavior

Notepad to open and run.

Screenshots or Pasted Text

Following is StackTrace in WinDBG. The above reproduction steps is for starting in windows without

ModLoad: 75ab0000 75ad5000   C:\Windows\SysWOW64\IMM32.DLL
ModLoad: 14460000 14909000   C:\Users\justin\Downloads\DynamoRIO-Windows-7.91.18301-1\DynamoRIO-Windows-7.91.18301-1\lib32\debug\dynamorio.dll
(1348.2a8): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=00fbf490 ecx=b58d14d4 edx=00000000 esi=2190b860 edi=00fbf540
eip=14775eab esp=00fbf4b0 ebp=00fbf4e4 iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010212
dynamorio!switch_modes_and_call+0x4e:
14775eab 5b              pop     ebx
1:002> k
 # ChildEBP RetAddr  
00 00fbf4e4 146fe909 dynamorio!switch_modes_and_call+0x4e [C:\projects\dynamorio\build\build_debug-32\core\CMakeFiles\drlibc.dir\drlibc\drlibc_x86.asm.obj.s @ 1961] 
01 00fbf510 146fed98 dynamorio!os_take_over_wow64_extra+0x49 [c:\projects\dynamorio\core\win32\os.c @ 2433] 
02 00fbfa30 146f1928 dynamorio!os_take_over_thread+0x2b8 [c:\projects\dynamorio\core\win32\os.c @ 2571] 
03 00fbfa70 1446fc58 dynamorio!os_take_over_all_unknown_threads+0x1f8 [c:\projects\dynamorio\core\win32\os.c @ 2707] 
04 00fbfaa4 146d7322 dynamorio!dynamorio_take_over_threads+0xf8 [c:\projects\dynamorio\core\dynamo.c @ 2904] 
05 00fbfb10 146d7aa8 dynamorio!auto_setup+0x202 [c:\projects\dynamorio\core\arch\x86_code.c @ 198] 
06 00000000 00000000 dynamorio!dynamo_auto_start+0x8 [C:\projects\dynamorio\build\build_debug-32\core\CMakeFiles\dynamorio.dir\arch\x86\x86.asm.obj.s @ 1600]

1:003> .exr -1
ExceptionAddress: 70afe90b (dynamorio!switch_modes_and_call+0x0000004e)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000003
   Parameter[1]: 00000000
Attempt to write to address 00000000

0:000> !address esp

                                     
Mapping file section regions...
Mapping module regions...
Mapping PEB regions...
Mapping TEB and stack regions...
Mapping heap regions...
Mapping page heap regions...
Mapping other regions...
Mapping stack trace database regions...
Mapping activation context regions...

Usage:                  Stack
Base Address:           00fff000
End Address:            01010000
Region Size:            00011000 (  68.000 kB)
State:                  00001000          MEM_COMMIT
Protect:                00000004          PAGE_READWRITE
Type:                   00020000          MEM_PRIVATE
Allocation Base:        00fd0000
Allocation Protect:     00000004          PAGE_READWRITE
More info:              ~0k


Content source: 1 (target), length: bd0

Versions

What version of DynamoRIO are you using? Windows-7.91.18301
Does the latest build from https://github.com/DynamoRIO/dynamorio/wiki/Latest-Build solve the problem? No
What operating system version are you running on? ("Windows 10" is not sufficient: give the release number.)

Windows 10 1809 (Build 17763.973) is being used. I am virtualizing Windows through KVM.

Is your application 32-bit or 64-bit?

32 bit

Additional context None