mirror of
https://dev.lirent.ru/Vatrog/vm-introspection-engine.git
synced 2026-07-08 23:46:36 +03:00
Add operand-normalized semantic code signatures
New semsig_hash(mem_view_t) primitive (include/semsig.h): a position-,
register-, immediate- and instruction-order-invariant function fingerprint.
It folds per-instruction operand-canonical tokens, splitting the body into
basic blocks via cfg_blocks and using a within-block order-insensitive
sort-then-fold (between-block order preserved), so it survives compiler
register reallocation and instruction scheduling that the byte-mask
func_hash/sig_generate cannot. Distinct hash domain from func_hash.
The operand decode is delegated to an OPTIONAL external disassembler behind
a private adapter seam (src/handlers/semsig_backend.h); semsig.c stays
backend-neutral and includes no backend header. VMIE_DISASM={OFF|zydis|
capstone} gates the build: OFF (default) compiles only semsig_stub.c and
keeps the zero-dependency build (VMIE_HAVE_DISASM=0, packages unchanged).
The backend is brought from VMIE_DISASM_SRC or find_package - never
vendored. Adds the vmie_win32_func_semsig wrapper.
The CI deb job builds with the Zydis backend (libzydis-dev); the runtime
package dependency on libzydis is derived automatically by dpkg-shlibdeps.
This commit is contained in:
@@ -0,0 +1,126 @@
|
||||
/* semsig.h - generic (OS-agnostic) semantic / normalized code signature.
|
||||
*
|
||||
* Handler layer: a NORMALIZED function fingerprint that survives what the byte
|
||||
* world cannot. func_hash (codeanalysis.h) and sig_generate (siggen.h) are
|
||||
* BYTE fingerprints - both only neutralize the rel/RIP-relative displacement
|
||||
* bytes; they still change when the compiler shuffles registers, picks a
|
||||
* different equivalent register allocation, alters an immediate/displacement
|
||||
* VALUE, or reschedules independent instructions. semsig_hash instead decodes
|
||||
* OPERANDS (registers, operand classes, sizes, mnemonic group) and folds a
|
||||
* canonical token stream, so it answers "semantically the same function" rather
|
||||
* than "byte-for-byte the same function" - a FLIRT-like recognition primitive.
|
||||
*
|
||||
* Operand decode is NOT in the light decoder (x86dec.h is deliberately pure and
|
||||
* length-only). It is supplied by an OPTIONAL external disassembler (Zydis or
|
||||
* Capstone) selected at configure time, behind a private adapter seam. So this
|
||||
* primitive is FEATURE-GATED: with no backend the symbol still exists (ABI
|
||||
* stable) but returns the "no hash" sentinel 0, and VMIE_HAVE_DISASM is 0 (see
|
||||
* below). The byte world (func_hash / sig_generate) is untouched either way.
|
||||
*
|
||||
* NOT COMPARABLE TO func_hash. semsig_hash uses a different algorithm over a
|
||||
* different (normalized) input, so its values live in their OWN domain: never
|
||||
* compare a semsig_hash to a func_hash. Compare semsig_hash to semsig_hash only
|
||||
* (equality / a known-hash table), exactly as func_hash is used for library-ID.
|
||||
*/
|
||||
#ifndef VMIE_SEMSIG_H
|
||||
#define VMIE_SEMSIG_H
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
#include "memmodel.h" /* mem_view_t (the single owner of the view type) */
|
||||
|
||||
/* Compile-time feature availability. Pushed from CMake as a PUBLIC compile
|
||||
* definition so consumers of this header see the SAME value the library was
|
||||
* built with:
|
||||
* VMIE_HAVE_DISASM == 1 built with a disassembler backend; semsig_hash is
|
||||
* live, semsig_backend_name() names the backend.
|
||||
* VMIE_HAVE_DISASM == 0 built without a backend (the default); semsig_hash
|
||||
* always returns 0 and semsig_backend_name() == "none".
|
||||
* Test it BEFORE calling to tell "feature off" apart from "empty/undecodable
|
||||
* function" - both return 0, but only the latter is a real input. The default
|
||||
* here keeps the header valid when compiled outside the project build. */
|
||||
#ifndef VMIE_HAVE_DISASM
|
||||
#define VMIE_HAVE_DISASM 0
|
||||
#endif
|
||||
|
||||
/* Reorder-/register-canonical semantic hash of one function view. `fn` is a view
|
||||
* spanning EXACTLY one function (e.g. a section-view sub-range covering a
|
||||
* func_range from vmie_win32_functions): fn.data[0] is the function's first
|
||||
* byte, fn.size its length. Reported in the view's own coordinate space, but the
|
||||
* hash is position-INDEPENDENT (like func_hash), so SECTION_LOCAL is enough for
|
||||
* a stable value.
|
||||
*
|
||||
* What it normalizes (the canonicalization contract):
|
||||
* - mnemonic CLASS is kept (the semantic backbone of each instruction);
|
||||
* - the concrete REGISTER identity is erased to its register CLASS, so a
|
||||
* rax<->rcx reshuffle does not change the hash;
|
||||
* - operand WIDTH is kept (mov al,_ differs from mov rax,_);
|
||||
* - immediate and displacement VALUES are erased (only "an imm of this width
|
||||
* is present" survives), as func_hash/sig_generate already do for rel/RIP;
|
||||
* - memory-operand SHAPE is kept (has_base / has_index / is_riprel flags, not
|
||||
* the register ids);
|
||||
* - operand ORDER is kept (mov dst,src differs from mov src,dst).
|
||||
*
|
||||
* REORDER INVARIANCE (v1 default behavior) AND ITS COST. The fold is two-level
|
||||
* and tied to the CFG. The function is split into basic blocks (via cfg_blocks),
|
||||
* and:
|
||||
* - WITHIN a block the per-instruction token hashes are folded ORDER-
|
||||
* INSENSITIVELY (sorted, then folded), so the compiler's intra-block
|
||||
* instruction scheduling - reordering independent instructions - does not
|
||||
* change the hash;
|
||||
* - BETWEEN blocks the order is PRESERVED (blocks are folded in cfg_blocks'
|
||||
* ascending-start order), because block order is the control-flow structure
|
||||
* and must stay significant.
|
||||
* The cost, stated plainly (this is a PROPERTY of the hash, not a free win): the
|
||||
* within-block order-insensitive fold LOWERS discrimination - two blocks with
|
||||
* the same MULTISET of normalized instructions but a different internal order
|
||||
* yield the SAME block hash, even when that order is semantically meaningful
|
||||
* (a data dependency this primitive does not model). That RAISES the false-match
|
||||
* (collision) probability versus an order-sensitive hash. It is a deliberate
|
||||
* trade for robustness against the scheduler; zero collisions are NOT promised.
|
||||
*
|
||||
* Returns the 64-bit semantic hash, or 0 on any of: `fn` empty (no data / size
|
||||
* 0), a decode desync (cfg_blocks could not split the bytes, or the backend
|
||||
* could not decode an instruction inside a block), or a build with no
|
||||
* disassembler backend (VMIE_HAVE_DISASM == 0). 0 is therefore "no hash", never
|
||||
* a valid fingerprint - the SAME sentinel as func_hash, so callers handle it the
|
||||
* same way.
|
||||
*
|
||||
* Token layout (stability contract). The per-instruction token folded by this
|
||||
* function is a fixed little-endian tuple; its layout is part of the hash's
|
||||
* stability contract (changing it changes every value). Per instruction:
|
||||
* byte 0 : mnemonic_class (the stable SEM_MN_* class id, low 8 bits)
|
||||
* byte 1 : mnemonic_class high (the SEM_MN_* class id, high 8 bits)
|
||||
* byte 2 : noperands (0..4)
|
||||
* byte 3 : flags (bit0 = is_control_flow)
|
||||
* bytes 4.. : per operand i in [0, noperands), 2 bytes each:
|
||||
* +0 : (kind & 0x0f) | ((reg_class & 0x0f) << 4)
|
||||
* +1 : (width_log2 & 0x07)
|
||||
* | (mem_has_base ? 0x08 : 0)
|
||||
* | (mem_has_index ? 0x10 : 0)
|
||||
* | (mem_is_riprel ? 0x20 : 0)
|
||||
* trailing : for the OTHER mnemonic class only, the raw opcode byte(s) so
|
||||
* unclassified instructions still differ from one another.
|
||||
* Operand order in the tuple follows the instruction's operand order (kept).
|
||||
*
|
||||
* Example - recognize a library function semantically (register-shuffle stable):
|
||||
* mem_view_t fn; // a SECTION_LOCAL sub-view of one function body
|
||||
* uint64_t h = semsig_hash(fn);
|
||||
* if (h && h == known_crt_memcpy_semsig) puts("looks like memcpy");
|
||||
*
|
||||
* Example - guard on availability before relying on the feature:
|
||||
* #if VMIE_HAVE_DISASM
|
||||
* uint64_t h = semsig_hash(fn); // live; 0 only on empty/desync
|
||||
* #else
|
||||
* // feature not built (semsig_hash would return 0)
|
||||
* #endif */
|
||||
uint64_t semsig_hash(mem_view_t fn) __attribute__((cold));
|
||||
|
||||
/* Stable identity of the compiled disassembler backend, for diagnostics and for
|
||||
* tagging which decoder produced a stored hash. Returns a static, never-NULL
|
||||
* string: the backend name (e.g. "zydis", "capstone") when VMIE_HAVE_DISASM==1,
|
||||
* or "none" when built without a backend. The normalized hash is designed to be
|
||||
* domain-stable ACROSS backends (the mnemonic classes are backend-neutral), so
|
||||
* this name is informational, not a hash-comparison key. */
|
||||
const char* semsig_backend_name(void);
|
||||
|
||||
#endif /* VMIE_SEMSIG_H */
|
||||
Reference in New Issue
Block a user