Runtime Decryption Stub

A runtime decryption stub is the minimal loader that almost every custom packer ships. The packed file contains only the stub as real code; the actual malware lives as an encrypted blob in a section or overlay. When the process starts, the stub allocates a new region, decrypts the blob into it, and jumps to the recovered OEP (original entry point). Until that jump, a static disassembly sees only the stub and a wall of high-entropy bytes.

The defining trait is the freshly allocated buffer: the plaintext payload is never present on disk, only materialised in memory for the lifetime of the run.

How it works

The stub requests memory that is writable so it can decrypt into it and executable so it can run the result — classically a single PAGE_EXECUTE_READWRITE (RWX) allocation, which is itself a red flag:

SIZE_T n = payload_size;
BYTE *mem = VirtualAlloc(NULL, n, MEM_COMMIT | MEM_RESERVE,
                         PAGE_EXECUTE_READWRITE);   /* RWX — telltale */

for (SIZE_T i = 0; i < n; i++)                       /* in-place decrypt */
    mem[i] = encrypted[i] ^ key[i % keylen];

/* control transfer to the reconstructed original entry point */
void (*oep)(void) = (void (*)(void))(mem + oep_off);
oep();

More careful packers split the steps — allocate PAGE_READWRITE, decrypt, then flip to PAGE_EXECUTE_READ with VirtualProtect — to avoid an RWX region, but the write-then-execute sequence on the same page remains. On Linux the same shape uses mmap/mprotect with PROT_WRITE|PROT_EXEC.

Detection & analysis

Static analysis: The stub is small and self-contained: a decrypt loop (xor, add/sub, or an RC4/AES setup), an allocation call, and an indirect jump or call into the allocated buffer. The encrypted payload appears as a high-entropy section or overlay. There are no meaningful imports beyond the memory and crypto primitives, and no readable strings from the payload.

Dynamic analysis: Set breakpoints on VirtualAlloc/VirtualProtect (or mmap/mprotect) requesting executable permission, let the decrypt loop run, then dump the allocated region — that buffer is the unpacked payload. The tail jump into the buffer marks the OEP; stop there and dump with Scylla or pe-sieve, then rebuild the IAT. Watch for the write-then-execute transition on the same page (a W^X violation) as the reliable trigger.

Detection rule hint: Flag images that allocate RWX memory (or write-then-mark-executable the same page) and immediately call/jmp into it, especially when paired with a high-entropy section and an import list limited to VirtualAlloc, VirtualProtect, and GetProcAddress — a profile that distinguishes a runtime unpacking stub from ordinary JIT or plugin loaders.