Skip to content
Reverse Engineering
Packing & Cryptersbeginner

PE Overlay Payload

An encrypted or compressed payload appended after the last section of a PE file — the overlay — that the loader never maps but the stub reads from disk and unpacks at runtime.

An overlay is any data that sits in a PE file past the end of the last section. The Windows loader maps sections according to the section table and stops there, so overlay bytes are never loaded into memory by the OS. That makes the overlay a convenient hiding place: the stub opens its own file on disk, seeks past the mapped image, and reads the appended payload itself.

Because the overlay is just raw file content, packers store the real malware there encrypted or compressed. Static tools that only parse mapped sections overlook it entirely, and the visible code is just a small benign-looking loader.

How it works

The end of the mapped image is PointerToRawData + SizeOfRawData of the last section; everything after that, up to the file size, is overlay:

text
File layout on disk
+---------------------------+  offset 0
| DOS header + stub         |
| PE header / Optional hdr  |
| Section table             |
+---------------------------+
| .text   (loader stub)     |  mapped by the OS loader
| .rdata  (strings, IAT)    |
| .data                     |
+---------------------------+  <- end of last section (raw)
| OVERLAY                   |  NOT mapped — read by the stub itself
|   [4-byte length]         |
|   [encrypted payload ...] |
+---------------------------+  end of file

The stub locates itself, computes the overlay offset, reads the blob, and decrypts it into memory before executing it:

c
HANDLE f = CreateFileW(self_path, GENERIC_READ, FILE_SHARE_READ,
                       NULL, OPEN_EXISTING, 0, NULL);
DWORD  fsize = GetFileSize(f, NULL);
SetFilePointer(f, overlay_off, NULL, FILE_BEGIN);   /* skip mapped image */
DWORD  n = fsize - overlay_off;
BYTE  *buf = VirtualAlloc(NULL, n, MEM_COMMIT, PAGE_READWRITE);
ReadFile(f, buf, n, &n, NULL);
rc4_decrypt(buf, n, key);                            /* recover real payload */

Detection & analysis

Static analysis: Compare the file size to the end of the last section (PointerToRawData + SizeOfRawData); a large gap means an overlay. Tools like PE-bear, CFF Explorer, and pecheck report overlay size directly. Run an entropy scan on the trailing bytes — a high, flat ~7.9 bits/byte region is encrypted or compressed data, not legitimate appended resources or a digital signature (which lives in the certificate directory, not loose overlay).

Dynamic analysis: Breakpoint on CreateFileW/ReadFile against the sample's own path, and on VirtualAlloc, then dump the buffer after the decryption routine runs to recover the plaintext payload. The stub almost always re-reads itself, so a file-handle to its own image is a strong runtime signal.

Detection rule hint: Alert when overlay size exceeds a few KB and trailing entropy is above ~7.2, especially combined with a tiny .text and a stub that calls GetModuleFileName then CreateFile/ReadFile on its own path — a pattern far more typical of overlay-stashed payloads than of benign installers.

Votes

Comments(0)