Add a light x86-64 decoder; back code-xref with it

The reversing keystone: a length-disassembly decoder with control-flow and
RIP-relative target extraction (x86dec.h), pure over a byte buffer - no vmie_mem,
no cr3, no Windows. Table-driven length over the 1-byte / 0F / 0F38 / 0F3A maps,
legacy + REX + VEX prefixes, ModRM/SIB, displacements and immediates (66 and
REX.W operand-size aware). It reports the instruction length plus the rel and
RIP-relative targets of near call/jmp/jcc and any RIP-relative memory operand.
EVEX is a documented gap (decodes as length 0). This is the primitive the rest
of the static-reversing layer builds on (function inventory, call graph, xref).

gva_code_xref now brute-scans with the decoder instead of its own ad-hoc E8/E9
and REX.W-lea heuristic, which is removed - one decoder in the tree. Because a
brute scan can re-enter a prefixed instruction one byte in and decode a shorter
aliased form with the same target, the scan drops a match that starts inside the
extent of an already-accepted one; real, non-overlapping instructions are
unaffected.
This commit is contained in:
2026-06-16 18:11:29 +03:00
parent c36ffe295d
commit 3199fbf258
5 changed files with 560 additions and 52 deletions
+1
View File
@@ -21,6 +21,7 @@ add_library(vmie STATIC
src/handlers/sigscan.c
src/handlers/sigset.c
src/handlers/codescan.c
src/handlers/x86dec.c
src/handlers/pmap.c
src/handlers/snapdiff.c)
target_include_directories(vmie