Skip to content
Reverse Engineering
Obfuscationadvanced

Code Virtualization

Native instructions are translated into a custom bytecode executed by an embedded interpreter, so a reverser sees a fetch-decode-dispatch loop instead of the original machine code.

Code virtualization is the strongest commercial software-protection primitive. The protector lifts a region of native code into a custom instruction set (the bytecode) and ships an embedded interpreter — a virtual machine — that executes that bytecode at runtime. The original x86/x64 is gone; the reverser instead faces an opcode handler table and a dispatch loop. It is used by VMProtect, Themida/WinLicense, Code Virtualizer and the academic Tigress tool.

How it works

The VM keeps its own virtual context (a struct or a region of the native stack) holding virtual registers, a virtual instruction pointer (VIP) and a virtual stack pointer. The core is a fetch → decode → dispatch → next loop: read the next bytecode opcode, index a handler table, run the handler, advance VIP.

text
; native (before virtualization)
add eax, ebx
xor ecx, eax

; after virtualization the same logic becomes bytecode + a dispatcher:
vm_loop:
    movzx  eax, byte [VIP]        ; fetch opcode
    inc    VIP
    jmp    [handler_table + eax*8] ; dispatch

handler_VADD:                      ; one handler per virtual opcode
    pop_vreg r0
    pop_vreg r1
    add      r0, r1
    push_vreg r0
    jmp      vm_loop               ; next

Detection & bypass

  • Static — In IDA/Ghidra look for an indirect jmp/call through a table inside a tight loop (the dispatcher), plus a cluster of short, structurally similar functions (the handlers) that all jump back to the same loop head. A large opaque "VM entry" stub that pushes all registers and a context pointer is a strong marker. VMProtect handlers are heavily MBA-obfuscated and chained through push/ret.
  • Dynamic — Trace one execution to recover the real opcode sequence: log VIP and the dispatched handler address at every loop iteration. Emulation (Unicorn/Qiling) or DBI lets you record the bytecode stream and the side-effects of each handler, which reveals the semantics of each virtual opcode without manual reading. Symbolic execution (Triton, Miasm) of a single handler yields its transfer function.
  • Patch / simplify — Map each virtual opcode to its native semantics, then lift the recorded bytecode into an IR and re-emit simplified native code (devirtualization). The output replaces the VM region.
  • Tools — VTIL for lifting/optimization, NoVmp for VMProtect 2.x, Triton and Miasm for symbolic handler analysis, Tigress's own devirtualization research passes, and manual tracing in x64dbg.
Votes

Comments(0)