Skip to content
Reverse Engineering
Anti-Disassemblyintermediate

FPU Instruction Pointer Abuse

Malware recovers its own runtime address through the FPU's saved instruction pointer via fnstenv, hiding the GetPC step from static xref analysis.

Decoder stubs need their own runtime address before they can touch embedded strings, keys, or import data, and the obvious way to get it — a call $+5/pop GetPC thunk — leaves a loud, recognisable signature. A quieter alternative exploits the x87 FPU: when any floating-point instruction executes, the CPU records the address of the last non-control FPU instruction in an internal register. The fnstenv (or fstenv) instruction dumps that saved environment, including the FPU instruction pointer, to memory, where the stub can read it back as its own program counter. This is the heart of the Shikata Ga Nai encoder's GetPC step.

The anti-disassembly effect is the absence of a signal. Unlike a call, the fnstenv path creates no cross-reference, no phantom sub-function, and no suspicious self-referential branch for a disassembler to flag. The address recovery happens silently inside the FPU state, so an analyst skimming the call graph sees an innocuous floating-point op followed by a stack read, with no obvious link to position-independent code. The dependency on a previous FPU instruction is easy to miss because that instruction is often a harmless fldz or fnop planted several bytes earlier.

How it works

A throwaway FPU instruction sets the saved IP; fnstenv then writes a 28-byte environment block to the stack, and offset +0x0C of that block is the address of the FPU instruction. The stub pops that field straight into a register:

asm
; Bytes:  D9 EE  D9 74 24 F4  5B  81 C3 ...
00: D9 EE            fldz                  ; harmless FPU op -> saved FPU IP = 0x00
02: D9 74 24 F4      fnstenv [esp-0x0C]    ; dump 28-byte env; +0x0C field = 0x00
06: 5B               pop  ebx              ; EBX = saved FPU IP (0x00), i.e. our PC
07: 81 C3 1B 10 00 00 add ebx, 0x101B      ; rebase EBX onto the embedded blob

The fnstenv [esp-0x0C] deliberately writes the environment so that its instruction-pointer field lands exactly at the new top of stack; the immediate pop ebx then lifts that pointer — the address of the fldz at 0x00 — into EBX with no call ever executed. A linear sweep decodes all four instructions cleanly and records no xref, so nothing in the listing hints that EBX now holds a runtime PC. The connection between fldz and the value popped is purely a CPU-state side channel the disassembler cannot model.

Packers vary the seed instruction to dodge naive byte signatures:

asm
; Bytes:  D9 D0  D9 74 24 F4  59 ...
00: D9 D0            fnop                  ; alternate seed; saved FPU IP = 0x00
02: D9 74 24 F4      fnstenv [esp-0x0C]    ; same env dump, +0x0C = 0x00
06: 59               pop  ecx              ; ECX = recovered program counter

Any non-control FPU instruction (fldz, fld1, fnop, fldpi) works as the seed, so the only invariant is the fnstenv/pop pair — and even the [esp-0x0C] displacement shifts when the stub reserves a different stack frame.

Detection & analysis

Static analysis:

  • The fingerprint is a non-control FPU instruction (D9 EE fldz, D9 D0 fnop, D9 E8 fld1) shortly followed by fnstenv/fstenv and then a pop reg. IDA/Ghidra show no xref, so search the bytes, not the call graph.
  • Treat the pop reg after fnstenv as the GetPC result: trace the subsequent add reg, imm to compute the file offset of the blob the stub will decode.
  • Disassemblers rarely annotate the +0x0C environment field, so manually note that the popped register now equals the address of the seed FPU instruction.

Dynamic analysis:

  • Single-step to the pop; the register then holds the live address of the seed instruction. Subtract the image base to map it to a file offset and follow the decoder loop to plaintext.
  • Emulators (Unicorn, speakeasy) model the FPU environment block, so they expose the recovered base register without needing real FPU hardware semantics.

Detection rule hint:

Flag any fnstenv/fstenv whose destination is [esp-0x0C] (or an equivalent stack slot landing the IP field at the top of stack) when immediately followed by a pop r32 — especially when a seed FPU instruction (fldz/fnop/fld1) precedes it. Legitimate code saves the full FPU environment around context switches and never pops a single field straight into a general register, making the fnstenv+pop couple a high-confidence FPU-GetPC indicator.

Votes

Comments(0)