New semsig_hash(mem_view_t) primitive (include/semsig.h): a position-,
register-, immediate- and instruction-order-invariant function fingerprint.
It folds per-instruction operand-canonical tokens, splitting the body into
basic blocks via cfg_blocks and using a within-block order-insensitive
sort-then-fold (between-block order preserved), so it survives compiler
register reallocation and instruction scheduling that the byte-mask
func_hash/sig_generate cannot. Distinct hash domain from func_hash.
The operand decode is delegated to an OPTIONAL external disassembler behind
a private adapter seam (src/handlers/semsig_backend.h); semsig.c stays
backend-neutral and includes no backend header. VMIE_DISASM={OFF|zydis|
capstone} gates the build: OFF (default) compiles only semsig_stub.c and
keeps the zero-dependency build (VMIE_HAVE_DISASM=0, packages unchanged).
The backend is brought from VMIE_DISASM_SRC or find_package - never
vendored. Adds the vmie_win32_func_semsig wrapper.
The CI deb job builds with the Zydis backend (libzydis-dev); the runtime
package dependency on libzydis is derived automatically by dpkg-shlibdeps.
Under KVA shadow (KPTI) the kernel CR3 marks user-half PML4Es NX as
hardening, so OR-ing that top-level NX down the walk made every user
region non-executable - even live code (ntdll .text read back r--u).
Ring 3 executes those pages via the user/shadow CR3, which reaches the
same shared lower tables through an NX-clear PML4E, so the kernel
table's user-half PML4E NX is not what enforces user execution. Ignore
it on the user half; sub-PML4E NX (shared between both CR3s) stays
authoritative.
vmie_win32_open_ro_fd now takes a kcr3 and builds the struct-offset
profile read-only from the image (find ntoskrnl, resolve
PsInitialSystemProcess, build the profile, derive the System _EPROCESS),
so a read-only context has the full read surface - including
proc_list/proc_modules, not just the cr3-parameterised PE walks.
The read-only bring-up tail is factored out of host_bootstrap into a
shared helper; host_bootstrap keeps the beacon find / cr3 recovery / ACK
around it, so its behaviour and return codes are unchanged.
vmie_win32_open_ro_fd maps the RAM backing file PROT_READ (via
gpa_from_ro_fd) and does NOT bootstrap; the caller passes a cr3 to each
read call. The read surfaces work this way - sections / functions /
imports / exports / callgraph / hooks, gva_read_text, and the generic
gva_* and scanner surfaces via vmie_win32_mem() - while any write returns
-1 because the mapping is read-only. The RW twin vmie_win32_open_fd and
host_bootstrap are unchanged.
vmie_mem could only be built from a path. Add a second input: an
already-open file descriptor, dup()'d internally (borrowed - the
caller's fd stays valid, core owns and closes its copy).
- vmie_mem_from_fd / vmie_mem_from_fd_segs (read-write, as the path
constructors: PROT_RW, MAP_SHARED)
- vmie_mem_from_ro_fd / vmie_mem_from_ro_fd_segs (read-only: map
PROT_READ, mark the source ro; gpa_write/gva_write return -1, every
read path is unchanged; accepts an O_RDONLY fd)
- vmie_win32_open_fd (the win32 context over an fd backing file)
Factor the mmap/validate tail and the single-low segment map out of the
path constructors into shared helpers, so the path and fd inputs go
through one mmap site and one map builder each.
Wave 2 of the code-analysis layer:
- vmie_win32_imports resolves the import directory (INT/IAT) to {iat_rva, dll,
name, ordinal} - named APIs, walking the name and slot thunks in lockstep so
every import carries the IAT slot a call lands on.
- vmie_win32_inline_hooks decodes each .pdata function's entry and reports any
whose first instruction is a direct jmp/call leaving the module image - the
detour/trampoline shape.
- vmie_win32_func_imports records, in order, the IAT slots a function calls
through (call qword [rip+disp] onto an import slot): the function's API-call
sequence, named by correlating with vmie_win32_imports.
- func_hash (codeanalysis.h) hashes a function position-independently, zeroing
the displacement bytes the decoder locates - one primitive for fingerprinting
known code and for detecting a changed body across snapshots.
Devirtualization needs no new call and is documented as a composition: a
vtable's methods are gva_jumptable(vtable_va), its instances are
pmap_referrers(vtable_va), and func_hash names each method. Imports reuse the
shared data-directory accessor; the analyses reuse the function/section/decode
primitives - no second PE or instruction parser.
Wave 1 of the code-analysis layer, built on the x86-64 decoder:
- vmie_win32_callgraph walks each .pdata function with the decoder and emits an
edge for every direct call/jmp whose target lands in the module - the
intra-module call graph. Indirect edges are left to the IAT and jump tables.
- gva_jumptable recovers a switch's case targets from an indirect jump's table:
consecutive pointer entries that land in an executable region.
- cfg_blocks splits one function view into basic blocks (a generic handler:
leaders from intra-function branch targets, cut after jmp/jcc/ret).
- gva_imm_xref finds the instructions whose immediate operand equals a constant
- the dual of code-xref for magic values, error codes, syscall numbers.
The decoder now also reports imm_off/imm_len so a caller can read or match the
immediate operand. The generic primitives live in the new codeanalysis.h
(jump tables, basic blocks) and scan.h (constant xref); the .pdata-bound call
graph stays on the win32 surface and reuses the existing function/section/decode
primitives - no second PE or instruction parser.
Three reversing capabilities on the win32 surface plus a pure sig-gen handler:
- vmie_win32_functions enumerates a module's functions from the exception
directory (.pdata RUNTIME_FUNCTION), folding unwind chain continuations into
their primary - authoritative non-leaf boundaries, not prologue heuristics.
- vmie_win32_exports resolves the export table to {name, rva, ordinal,
forwarded}: named functions with no PDB or network. vmie_win32_pdb_ref pulls
the CodeView/RSDS {guid, age, pdb} from the debug directory - the symbol-server
key for any module (full PDB parsing stays out of scope).
- sig_generate (siggen.h) builds a unique masked signature for a code span,
wildcarding the rel/RIP-relative displacement bytes the x86 decoder locates and
growing until it matches the scope exactly once - the dual of sigscan.
The decoder now also reports disp_off/disp_len so a caller can mask the floating
bytes. The MZ/PE walk gains one shared data-directory accessor and one shared
CodeView/RSDS parser; the kernel bootstrap is moved onto both, removing its
private copies - one PE parser in the tree.
vmie_win32_sections lists a module's PE sections (name, RVA, virtual size,
VR_* protection) for any image base in a process address space - including a
base found by scanning, not only loader-list modules. vmie_win32_section_view
gathers a section's bytes into a caller buffer and returns a mem_view_t whose
base_va is chosen by view_base: SECTION_LOCAL (0, section-relative offsets),
MODULE_RVA (ASLR-stable module RVAs), or ABSOLUTE_VA (live VA). Because the pure
scanners report base_va + offset, the mode directly selects the coordinate space
of every hit - feeding a view to sig_all or x86_decode yields section-relative,
RVA, or absolute results with no extra work.
The MZ/PE header walk is factored into one helper that both pe_find_section and
the new enumerator share - no second parser. The whole public surface is
documented with the operational nuances (coordinate stability, borrowed-buffer
lifetime, truncation, residency) and worked examples.
The generic address-space layer no longer names win32 in its comments: the
khalf_score and gva_translate doc-comments described themselves in terms of
their current Windows consumer, a downward coupling from the stable layer to a
specific, swappable one. Reworded to describe what each primitive does, not who
calls it. Also drop a dangling reference to the renamed engine handle.
State the contract's real scope in memmodel.h: OS-agnostic but architecture-
bound. The address-space key is the x86-64 CR3 (the PML4 base), shared by any
guest OS on x86-64 - CR3 is an ISA register, not a Windows concept; only its
per-process storage (DirectoryTableBase) is win32-specific and stays in the
win32 engine.
The agent no longer self-terminates on a 120s deadline: a VMI host may attach
at any time, so the beacon now polls its ack flag indefinitely and exits only
once the host sets it. The fixed lifetime was an artifact, not a requirement.
The contract drops from three constants to one. The companion magic is derived
as the byte-reversed primary (__builtin_bswap64, folded to an immediate at -O2),
giving the same 16-byte beacon signature from a single source of truth; the host
acknowledges by echoing that value into the ack slot instead of carrying a
separate ack constant.
Name and isolate the Windows engine as one of potentially several. The
public surface moves to include/win32.h with an opaque vmie_win32 handle
(vmie_win32_open/close/mem); the engine's Windows internals — host bring-up,
the struct-offset profile, process/module/PE/text decode — live under
src/engine/win32. The generic address-space layer stays in src/engine
(gva.c + engine-arch.h, carrying no offset table): gva.c is de-profiled, and
CR3 bring-up reaches the hot translator through a cold gva_translate bridge
so the zero-copy hot path stays private and inlinable.
A memory source is now first-class and public: vmie_mem_open/_open_segs/
_close open a flat dump (or an explicit segment map) as a vmie_mem, with
gpa_seg promoted to the public contract. The physical signature scan is
exposed source-agnostically: sig_scan_mem returns GPAs for any vmie_mem,
sig_scan_sources scans several sources with per-source attribution, and
sig_from_bytes builds an exact needle from a byte span. The pure matcher is
unchanged; dumps and the live engine image are scanned uniformly, neither
needing the other.
CORE (src/core): vmie_mem — guest-physical substrate with a data-driven
segment map (replaces the hardcoded 4 GiB PCI-hole topology). ENGINE
(src/engine): x86-64 paging + Windows bring-up; produces the generic memory
model. HANDLERS (src/handlers): the signature/value/pointer scanners, which
now consume an OS-agnostic contract.
Keystone: gva_ctx is split into vmie_mem (core) + vmie (engine); the generic
access functions take vmie_mem* + cr3 and no longer compile in the Windows
offset table. New public contract include/memmodel.h (vmie_mem, mem_view_t,
vregion, task, range, the gva_* access); win32 surface in include/vmie.h.
Leak relocations: the PE parser, UTF-16 decode and CR3-recovery heuristics
move engine-side; the matcher stays a pure, source-agnostic handler, and the
pointer scanner takes a generic range[] instead of reaching into the process
enumerator.
lirent
