i#3753: Fix incorrect decode assertion firing on invalid user input.

Removes an assertion and returns an invalid instruction instead if invalid 3-byte VEX.mm bits are observed in the fast decoder.

Fixes #3753 (closed) Issue: #3713