Lesson 4.3 · Disassembly & Code Analysis· 50 min
Recognising Compiler Idioms
Map optimised x86-64 compiler output back to C: frames, magic-number division, branchless code, loops, jump tables, structs and vtables.
Objectives
- Read Windows x64 and System V function frames, including shadow space, alignment and stack canaries
- Translate arithmetic idioms back to source, and recover the divisor hidden in a magic-number multiplication
- Recognise conditionals, loops, switch jump tables, struct accesses and virtual calls at -O0 and -O2
- Decide when a function is library code to identify rather than reverse
A compiler does not translate C line by line. It picks from a fairly small repertoire of instruction patterns, the same few dozen shapes, over and over. Once you know those shapes, a page of disassembly stops being a list of instructions and becomes a list of constructs: "a bounds-checked jump table", "an unsigned divide by 7", "a loop over an array of 32-byte structs".
Malware passes through the same optimisers as any other code, so this skill transfers to every sample. Functions and Control-Flow Graphs showed how a tool groups instructions into functions; here we read what is inside them, on x86-64 with the Windows x64 ABI first and System V noted. Every listing was produced with GCC 15 (mingw-w64 for PE, a Linux cross-compiler for ELF), and you will reproduce them in the lab.
Optimisation level changes everything
The same C function looks very different at different optimisation levels:
| Level | What you see | What it means for you |
|---|---|---|
-O0 / MSVC /Od | Frame pointer, every variable in memory, every argument spilled to the stack, one C statement per block | Easy to map to source, but verbose |
-O2 / MSVC /O2 | Frameless functions, values in registers, loops rewritten, calls inlined or turned into jumps | Dense; idioms matter most here |
-O3 | -O2 plus aggressive vectorisation and unrolling | SIMD loops with scalar "tail" code |
-Os / MSVC /O1 | Size-optimised: short sequences, rep string instructions | Compact but sometimes odd choices |
Most malware is optimised, but debug builds do leak out. push rbp /
mov rbp,rsp in every function, with arguments stored to memory right after
the prologue, points to -O0 or /Od code.
Function frames
Prologue and epilogue
An unoptimised function builds a classic frame (see the stack):
<sum_array>: ; -O0, mingw-w64
push rbp
mov rbp,rsp
sub rsp,0x10 ; room for two locals
mov QWORD PTR [rbp+0x10],rcx ; home argument 1 (a)
mov DWORD PTR [rbp+0x18],edx ; home argument 2 (n)
...
add rsp,0x10
pop rbp
retAt -O2 the frame pointer disappears. The function addresses everything
relative to rsp, or does not touch the stack at all. A function that calls
others must still keep the stack aligned and, on Windows, reserve space for its
callees:
<count_down>: ; -O2, mingw-w64
push rsi ; save non-volatile registers it will use
push rbx
sub rsp,0x28 ; 0x20 shadow space + 8 bytes of alignment
...
add rsp,0x28
pop rbx
pop rsi
retTwo Windows x64 rules explain the numbers:
- Shadow space. The caller always reserves 32 bytes (
0x20) directly above the return address, one 8-byte slot for each register argument. The callee may storercx,rdx,r8andr9there. That is why the-O0listing above writes its arguments to[rbp+0x10]and[rbp+0x18]: withrbppointing at the savedrbpand the return address at[rbp+8], those are the first two shadow slots. - Alignment.
rspmust be 16-byte aligned at everycall. The call pushes 8 bytes, so on entryrspis 8 bytes off. Two pushes plus0x28brings the total to8 + 16 + 40 = 64, a multiple of 16. When you see an odd-lookingsub rsp,0x28or0x38, it is almost always this arithmetic.
The pushed registers are callee-saved ones (rbx, rbp, rdi, rsi,
r12 to r15 on Windows), pushed only if used; see
Calling conventions.
Arguments and locals
| Windows x64 | System V AMD64 (Linux, macOS) | |
|---|---|---|
| Integer and pointer arguments | rcx, rdx, r8, r9, then stack | rdi, rsi, rdx, rcx, r8, r9, then stack |
| Floating-point arguments | xmm0 to xmm3 (positional, shared with integer slots) | xmm0 to xmm7 (independent count) |
| Return value | rax / xmm0 | rax (and rdx) / xmm0 |
| Caller-reserved space | 32-byte shadow space | none, but leaf functions may use a 128-byte red zone below rsp |
| Fifth argument | [rsp+0x28] at entry | seventh integer argument at [rsp+8] at entry |
In a frameless -O2 function, locals live at [rsp+x] and arguments that do
not fit in registers live above the shadow space. In -O0 code, locals are at
negative offsets [rbp-x] and homed arguments at positive offsets [rbp+x].
The fastest way to tell which ABI a snippet uses: the first thing it touches is
rcx/edx on Windows, rdi/esi on System V. Compare the same function
compiled both ways:
<get_size>: ; PE, Windows x64 <get_size>: ; ELF, System V
movsxd rdx,edx movsxd rsi,esi
shl rdx,0x5 shl rsi,0x5
mov rax,QWORD PTR [rcx+rdx*1+0x10] mov rax,QWORD PTR [rdi+rsi*1+0x10]
ret retThe movsxd sign-extends a 32-bit int index before it can take part in an
address (Sign extension). Remember that Windows
is LLP64: long is 32 bits there.
Stack canaries
A function with a local buffer usually carries a stack cookie: a secret
value written below the saved registers in the prologue and checked before
ret. MSVC's /GS (on by default) produces this shape. The listing was compiled with
clang-cl /GS /O2, which follows the MSVC conventions:
sub rsp,0x68
mov rax,QWORD PTR [rip+0x0] ; __security_cookie
xor rax,rsp ; cookie mixed with the frame address
mov QWORD PTR [rsp+0x60],rax
lea rcx,[rsp+0x20] ; the buffer, just above shadow space
call g
mov rcx,QWORD PTR [rsp+0x60]
xor rcx,rsp
...
call __security_check_cookieGCC's equivalents compare against __stack_chk_guard (mingw-w64) or
fs:0x28 (Linux) and call __stack_chk_fail. Treat the pattern as
boilerplate, with one useful hint: the function has a buffer on its stack, and
the cookie slot marks where it ends.
Arithmetic idioms
Compilers avoid slow instructions and long encodings. The results are predictable once you know the substitutions:
| You see | Source construct | Why |
|---|---|---|
xor eax,eax | x = 0 | Shorter than mov eax,0, breaks dependencies (XOR) |
lea eax,[rcx+rcx*4] then add eax,eax | x * 10 | Two cheap operations beat imul (LEA) |
shl rdx,0x5 | x * 32 | Power-of-two multiply is a shift (shifts) |
lea eax,[rdi+0x7] | a + 7 into a new register | Three-operand add without touching flags |
imul by a large odd constant, then shr/sar | x / constant | Multiply by reciprocal instead of div |
quotient * d, then sub from x | x % constant | x - (x / d) * d |
and eax,0x7 | x % 8 on an unsigned value | Power-of-two modulo is a mask |
sar ecx,0x1f | sign of x as 0 or -1 | Used to correct signed division and in branchless abs |
cdq / cqo before idiv | signed division by a runtime value | Sign-extends the dividend into edx/rdx (DIV / IDIV) |
xor edx,edx before div | unsigned division by a runtime value | Clears the high half of the dividend |
Our times10 function shows the multiply idiom exactly:
<times10>:
lea eax,[rcx+rcx*4] ; x*5
add eax,eax ; *2
retDivision by a constant
div and idiv take tens of cycles, so a compiler that knows the divisor in
advance replaces the division with a multiplication by a scaled reciprocal. The
idea: x / d equals (x * m) >> s when m is roughly 2^s / d, rounded up,
and s is large enough that the rounding error never changes the integer
result. GCC does this even at -O0. Here is unsigned mod10(unsigned x):
<mod10>:
mov edx,0xcccccccd
mov eax,ecx
imul rax,rdx ; 64-bit product x * m
shr rax,0x23 ; >> 35 -> quotient q = x / 10
lea edx,[rax+rax*4] ; q*5
mov eax,ecx
add edx,edx ; q*10
sub eax,edx ; x - q*10 -> remainder
retRecovering the divisor. Add up every right shift applied to the product
(here, 35) and compute 2^s / m. The result is always slightly below an
integer, because m was rounded up. Round it up:
2**35 / 0xcccccccd = 9.9999999994 -> divisor 10The trailing lea/add/sub is the modulo idiom: rebuild q * 10 and
subtract it from x.
Some divisors, 7 among them, need a magic number one bit wider than the
register. The compiler then stores only the low 32 bits of the multiplier and
adds a fix-up sequence. unsigned div7(unsigned x):
<div7>:
mov eax,ecx
imul rax,rax,0x24924925
shr rax,0x20 ; t = high 32 bits of x*m
sub ecx,eax ; x - t
shr ecx,1 ; (x - t) >> 1
add eax,ecx ; t + (x - t)/2
shr eax,0x2
retRecognise the sub / shr 1 / add triple. It computes the same result as
multiplying by the 33-bit value 2^32 + m. The total shift is 32 + 1 + 2 = 35:
2**35 / (2**32 + 0x24924925) = 6.9999999994 -> divisor 7The signed version, int sdiv7(int x), adds two more idioms:
<sdiv7>:
movsxd rax,ecx
imul rax,rax,0xffffffff92492493
shr rax,0x20
add eax,ecx ; magic is "negative", so add x back
sar ecx,0x1f ; 0 if x >= 0, -1 if x < 0
sar eax,0x2 ; arithmetic shift keeps the sign
sub eax,ecx ; +1 for negative x: C rounds toward zero
retHere the total shift is 32 + 2 = 34, and treating the magic as the unsigned
32-bit value 0x92492493 gives 2^34 / 0x92492493 = 6.99999999..., so the
divisor is again 7. The sar 0x1f / sub pair is the signature of signed
division: it makes negative results round toward zero, as C requires.
Tip: You do not need to memorise magic numbers. Two you will see constantly are
0xcccccccd(divide by 10, or by 5 with a smaller shift) and0xaaaaaaab(divide by 3). For anything else, apply the formula and confirm by testing the instruction sequence in Python, as in the lab. Ghidra and IDA decompilers usually fold these sequences back into/and%for you.
Conditionals and booleans
An if compiles to a cmp or test that sets flags,
then a conditional jump. The jump usually goes to the code that skips the
then block, so its condition is the inverse of the source condition:
if (n > 0) { ... } becomes test edx,edx / jle skip. test reg,reg is the
idiom for comparing with zero.
Optimisers remove short branches when both outcomes are cheap to compute:
<max_branchless>: <is_nonzero>:
cmp edx,ecx xor eax,eax
mov eax,ecx test ecx,ecx
cmovge eax,edx setne al
ret retThe first is return a > b ? a : b; with a conditional move.
The second is boolean normalisation: return x != 0; must return exactly 0
or 1, and setcc writes a single byte, so the compiler
zeroes eax first. At -O0 you see setne al followed by movzx eax,al
instead. Both mean "convert a condition into an int". When you see setcc
feeding an add, the source was counting how many conditions are true.
Loops
At -O0, GCC lays out a for loop as: initialise, jump to the test at the
bottom, body, increment, test, jump back if true.
mov DWORD PTR [rbp-0x8],0x0 ; i = 0
jmp test
body:
... ; s += a[i]
add DWORD PTR [rbp-0x8],0x1 ; i++
test:
mov eax,DWORD PTR [rbp-0x8]
cmp eax,DWORD PTR [rbp+0x18] ; i < n ?
jl bodyAt -O2, the same loop is rotated. A guard checks once whether the loop
runs at all, and the body becomes a do/while with the test at the bottom.
The index often disappears too. Here the compiler turned a[i] into a pointer
walking from a to a + n:
<sum_array>:
test edx,edx
jle empty ; guard: n <= 0 -> return 0
movsxd rdx,edx
xor eax,eax ; s = 0
lea rdx,[rcx+rdx*4] ; end = a + n
loop:
add eax,DWORD PTR [rcx] ; s += *p
add rcx,0x4 ; p++
cmp rcx,rdx
jne loop
retSo a guard plus a bottom-tested loop is simply how a for or while loop
looks after optimisation. It is not evidence that the author wrote do/while.
Loops whose counter does not matter are often turned around to count down.
while (n-- > 0) calls += puts("tick"); became a loop over ebx = n - 1 that
ends with sub ebx,0x1 / jae loop: the carry flag signals when the counter
wraps below zero.
At -O3 (and in hot loops of modern -O2 builds) the loop may be
vectorised. Look for xmm or ymm registers and packed instructions
(see SIMD registers):
loop:
movdqu xmm2,XMMWORD PTR [rax] ; load 4 ints
add rax,0x10
paddd xmm0,xmm2 ; 4 running sums at once
cmp r8,rax
jne loop
... ; psrldq/paddd: add the 4 lanes together
movd eax,xmm0
... ; scalar tail: up to 3 leftover elementsA vectorised loop has a set-up, a vector body and a scalar tail. Only the body
carries the real operation (paddd: add packed 32-bit integers); skip the
rest. Unrolling looks similar without
SIMD: the body is repeated with offsets +0x4, +0x8, +0xc and the counter
advances by several elements per iteration.
switch statements
A switch becomes one of three shapes, depending on how dense its case values
are.
Jump table. For dense cases, the compiler bounds-checks the value and jumps
through a table. Our dispatch(int cmd) has cases 0 to 6:
<dispatch>:
cmp ecx,0x6
ja default ; unsigned: also catches cmd < 0
lea rdx,[rip+0x2afc] ; table at 0x14000402c (.rdata)
mov ecx,ecx
movsxd rax,DWORD PTR [rdx+rcx*4] ; 32-bit signed entry
add rax,rdx ; entry is relative to the table
jmp raxNote the unsigned ja: one comparison rejects both values above 6 and
negative values, which become huge when read as unsigned. If the lowest case
is not zero, a sub ecx,<lowest> comes first. With cases 10 to 15 you get
sub ecx,0xa / cmp ecx,0x5. The lowest case and the bound together give
you the case range.
x64 tables rarely hold absolute addresses. GCC stores 32-bit offsets relative
to the table, as above; MSVC typically stores 32-bit RVAs and adds the image
base (lea reg,[__ImageBase]). When a disassembler fails to decode a table,
common in obfuscated code, read the entries and add each to the base yourself.
Lookup table. When every case only produces a value, GCC skips the code
entirely and indexes an array of results. classify (cases 0 to 6 returning
11, 23, 37...) compiles to cmp ecx,0x6 / ja / mov eax,[rax+rcx*4] over
a table named CSWTCH.5, whose bytes in .rdata are exactly those return
values. The -O0 build of the same functions uses neither kind of table, just a
chain of cmp/je/jg.
Compare tree. Sparse cases (3, 40, 200, 1000, 4096, 9999) cannot use a table without wasting space, so the compiler builds a binary search:
<sparse>:
cmp ecx,0x3e8 ; 1000: the pivot
je case_1000
jg upper_half
cmp ecx,0x28 ; 40
je case_40
...
upper_half:
cmp ecx,0x1000 ; 4096
...A je then jg on the same constant marks a split point. The constants
compared against the register are the case list; in a backdoor's command
dispatcher, they are the command IDs.
Structs and arrays
x86 addressing, [base + index*scale + disp] (see
addressing modes), maps directly onto C
data access:
| Operand | Typical source |
|---|---|
[rcx+0x10] | p->field at offset 0x10 |
[rcx+rdx*4] | a[i] for 4-byte elements |
[rcx+rdx*8+0x10] | p->arr[i] for 8-byte elements, starting at offset 0x10 |
shl rdx,0x5 then [rcx+rdx*1+0x10] | p[i].field where sizeof(*p) == 32 |
lea rax,[rcx+0x4] | &p->field or p->array, an address not a load |
get_size returns r[i].size. The shl rdx,0x5 gives the element size,
32 bytes, and the displacement 0x10 gives the field offset. The load width
(QWORD PTR) gives the field type: 8 bytes. Scales are limited to 1, 2, 4 and
8, so larger or odd element sizes appear as a shift or imul on the index
first.
To recover a struct, collect every offset and access width used with the same base pointer across the functions that receive it, and sort by offset; gaps are unknown fields or padding. Ghidra's "Auto Create Structure" drafts this for you, and Cross-References and Data Flow shows how to follow the pointer between functions.
Inlined memory and string operations
Small memset, memcpy and struct copies with a known size are expanded
inline, so there is no call to find. Depending on size and flags:
| Case | What GCC emitted in our builds |
|---|---|
memset of a 32-byte struct, -O2 | pxor xmm0,xmm0 then movups [rcx],xmm0 / movups [rcx+0x10],xmm0 |
200-byte struct copy, -O2 | a run of movdqu loads and movups stores, 16 bytes each |
200-byte memset or copy, -Os | mov ecx,0x32 then rep stos DWORD PTR [rdi],eax or rep movs |
The rep forms (see string operations)
spell out their arguments: rdi is the destination, rsi the source, rcx the
count in units of the element size. 0x32 4-byte units is 200 bytes. A
pxor followed by a string of movups to one base with increasing offsets is
a zeroed struct or buffer.
String scanning has its own shapes. A hand-written strlen compiles to a byte
loop: cmp BYTE PTR [rax],0x0 / jne, then sub of the start pointer. Older
32-bit code uses repne scasb with ecx = -1. The optimised C runtime
implementations test 16 bytes at a time with pcmpeqb and pmovmskb, or use
bit tricks that check a whole word for a zero byte. When you see those
constructs in a big function with no author logic, you are almost certainly in
library code.
C++: this and virtual calls
C++ methods take the object pointer, this, as a hidden first argument: rcx
on Windows, rdi on System V. A virtual call loads the object's first
quadword (the vtable pointer) and calls through a slot:
<describe(Shape const*)>:
mov rax,QWORD PTR [rcx] ; vtable = s->__vptr
mov rbx,rcx ; keep this across calls
call QWORD PTR [rax+0x18] ; slot 3: area()
mov rcx,rbx ; this again for the next call
mov esi,eax
mov rax,QWORD PTR [rbx]
call QWORD PTR [rax+0x10] ; slot 2: sides()Slot index is offset divided by 8. With GCC's (Itanium) ABI, a virtual
destructor takes two slots, so sides() and area() land at 0x10 and
0x18. MSVC uses one destructor slot, so the same class has different offsets.
To resolve the call, find where objects of this class are constructed: the
constructor writes the vtable address to [rcx], and that vtable in .rdata
lists the function pointers in slot order. Run-time type information (RTTI),
when present, sits next to the vtable and often names the class.
Library code: do not reverse memcpy
A statically linked sample can contain hundreds of C runtime, compression or crypto functions. Reversing them by hand wastes hours. Identify them instead:
IDA FLIRT matches byte patterns (relocated bytes masked out) against
signature libraries; Ghidra Function ID does the same with hash
databases; Binary Ninja ships its own signatures. Pick the ones matching the
compiler you identified in Identifying and Hashing Files,
or build your own from a library you compile. Without signatures, library
functions still stand out: large, many callers, no author strings, SIMD or
bit tricks. Name them by behaviour (maybe_memcpy) and move on.
Warning: Obfuscators deliberately break idiom recognition. Instruction substitution replaces
x + ywith longer equivalent expressions, mixed boolean-arithmetic hides constants in polynomial identities, and dead code insertion pads real idioms with instructions that do nothing. If familiar shapes vanish from an otherwise normal binary, that absence is a finding in itself.
Idiom cheat sheet
| Pattern | Source construct |
|---|---|
push rbp / mov rbp,rsp in every function | Unoptimised build or frame pointers kept |
sub rsp,0x28 / 0x38 with no locals used | Shadow space plus alignment (Windows x64) |
Store of rcx/rdx to [rbp+0x10]/[rbp+0x18] | -O0 homing of arguments |
__security_cookie xor rsp, check before ret | /GS stack buffer present |
imul by odd constant + shifts | Division by a constant |
sub, shr 1, add after a high-half multiply | Division needing a 33-bit magic |
sar reg,0x1f + sub | Signed division rounding toward zero |
test/cmp + setcc (+movzx) | Boolean result (==, !=, < as a value) |
cmovcc | ?:, min, max, simple if assignment |
| Guard + bottom-tested loop | Any for/while at -O2 |
sub reg,1 / jae or dec / jnz | Counted loop running down |
movdqu/paddd loop + scalar tail | Vectorised array loop |
cmp + ja + indexed movsxd + jmp reg | Dense switch (jump table) |
cmp + ja + indexed load of a constant | switch that only returns values |
je/jg pairs on one register | Sparse switch (binary search) |
[reg+disp] with several fixed disp on one base | Struct field access |
shl/imul on index then [base+idx+disp] | Array of structs |
pxor + movups run / rep stos | Inlined memset |
mov rax,[rcx] then call [rax+N] | C++ virtual call, slot N/8 |
Lab: match the idioms
You will compile a file of small functions at -O0 and -O2, find each idiom
from this lesson, decode a jump table by hand and recover a divisor from its
magic number. Everything runs on your analysis machine; nothing is executed.
-
Save the source as
idioms.c:c // idioms.c - small benign functions, one compiler idiom each #include <stdio.h> #include <string.h> struct record { int id; /* +0x00 */ char tag[12]; /* +0x04 */ long long size; /* +0x10 */ int flags; /* +0x18 */ }; __attribute__((noinline)) unsigned div7(unsigned x) { return x / 7; } __attribute__((noinline)) int sdiv7(int x) { return x / 7; } __attribute__((noinline)) unsigned mod10(unsigned x) { return x % 10; } __attribute__((noinline)) int times10(int x) { return x * 10; } __attribute__((noinline)) int classify(int op) { switch (op) { case 0: return 11; case 1: return 23; case 2: return 37; case 3: return 41; case 4: return 59; case 5: return 67; case 6: return 71; default: return -1; } } __attribute__((noinline)) const char *dispatch(int cmd) { switch (cmd) { case 0: return "ping"; case 1: return "info"; case 2: return "list"; case 3: return "get"; case 4: return "put"; case 5: return "sleep"; case 6: return "exit"; default: return "unknown"; } } __attribute__((noinline)) long long get_size(struct record *r, int i) { return r[i].size; } __attribute__((noinline)) int sum_array(const int *a, int n) { int s = 0; for (int i = 0; i < n; i++) s += a[i]; return s; } __attribute__((noinline)) int max_branchless(int a, int b) { return a > b ? a : b; } __attribute__((noinline)) int is_nonzero(int x) { return x != 0; } __attribute__((noinline)) void clear_record(struct record *r) { memset(r, 0, sizeof *r); } int main(void) { struct record recs[2] = { {1, "alpha", 100, 0}, {2, "beta", 200, 1} }; int arr[64]; for (int i = 0; i < 64; i++) arr[i] = i; printf("%u %d %u %d\n", div7(100), sdiv7(-100), mod10(1234), times10(9)); printf("%d %s\n", classify(3), dispatch(5)); printf("%lld %d\n", get_size(recs, 1), sum_array(arr, 64)); printf("%d %d\n", max_branchless(3, 9), is_nonzero(5)); clear_record(&recs[0]); printf("%d\n", recs[0].id); return 0; }noinlinekeeps each function separate; in real samples most would be inlined into their callers. -
Build both levels, and dump each to a text file:
bash x86_64-w64-mingw32-gcc -O0 -o idioms_O0.exe idioms.c x86_64-w64-mingw32-gcc -O2 -o idioms_O2.exe idioms.c x86_64-w64-mingw32-objdump -d -M intel --no-show-raw-insn idioms_O0.exe > O0.txt x86_64-w64-mingw32-objdump -d -M intel --no-show-raw-insn idioms_O2.exe > O2.txt awk '/^[0-9a-f]+ <div7>:/,/^$/' O2.txt # print one functionThe binaries are not stripped, so names give you an answer key. Your addresses may differ slightly.
-
Frames. Compare
sum_arrayinO0.txtandO2.txt. Find the homed arguments at[rbp+0x10]and[rbp+0x18], the two locals at[rbp-0x4](s) and[rbp-0x8](i), and confirm that the-O2version has no frame at all. Then look atmaininO0.txt: how big is itssub rsp, and why is it not a multiple of 16 on its own? (Count the pushes.) -
Arithmetic. Match
times10,div7,sdiv7andmod10to the listings in this lesson. Note thatdiv7inO0.txtalready usesimul rax,rax,0x24924925: GCC applies the magic-number rewrite even at-O0. -
Recover the divisors in Python. Save and run
magic.py. It computes the divisor from each constant, then models the exact instruction sequences and checks them against Python's own//and%:python # magic.py - recover the divisor behind a multiply-by-reciprocal sequence M10 = 0xcccccccd # mod10: imul by this, then shr rax,0x23 print("mod10 :", 2**0x23 / M10) M7 = 0x24924925 # div7: imul, shr 32, then the sub/shr 1/add/shr 2 fix-up print("div7 :", 2**35 / (2**32 + M7)) def div7_asm(x): # the exact instruction sequence, in Python t = (x * M7) >> 32 # imul rax,rax,0x24924925 ; shr rax,0x20 q = ((x - t) >> 1) + t # sub ecx,eax ; shr ecx,1 ; add eax,ecx return q >> 2 # shr eax,0x2 def mod10_asm(x): q = (x * M10) >> 0x23 # imul rax,rdx ; shr rax,0x23 return x - (q * 5) * 2 # lea edx,[rax+rax*4] ; add edx,edx ; sub eax,edx import random xs = [0, 1, 6, 7, 2**32 - 1] + [random.getrandbits(32) for _ in range(1_000_000)] assert all(div7_asm(x) == x // 7 for x in xs) assert all(mod10_asm(x) == x % 10 for x in xs) print("div7 and mod10 sequences match x//7 and x%10 on", len(xs), "inputs")text mod10 : 9.999999999417923 div7 : 6.99999999938882 div7 and mod10 sequences match x//7 and x%10 on 1000005 inputsBoth ratios sit just below an integer, so the divisors are 10 and 7, and the assertions confirm the sequences over a million inputs.
-
Conditionals. Find
max_branchlessandis_nonzeroat both levels. Which instruction does the-O0build ofis_nonzeroadd, and why? -
Loops. Identify the guard, the rotated loop and the pointer increment in
-O2sum_array. Then rebuild just that function at-O3and find thepadddbody and the scalar tail:bash x86_64-w64-mingw32-gcc -O3 -c -o idioms_O3.o idioms.c x86_64-w64-mingw32-objdump -d -M intel --no-show-raw-insn idioms_O3.o | awk '/<sum_array>:/,/^$/' -
Decode the jump table. In
O2.txt,dispatchloads the table address withlea rdx,[rip+...]; objdump prints the resolved address in the comment (0x14000402cin our build). Dump the raw bytes:bash x86_64-w64-mingw32-objdump -s -j .rdata idioms_O2.exe | head -9text 140004000 70696e67 00696e66 6f006c69 73740067 ping.info.list.g 140004010 65740070 75740073 6c656570 00657869 et.put.sleep.exi 140004020 7400756e 6b6e6f77 6e000000 24d5ffff t.unknown...$... 140004030 14d5ffff 34d5ffff 44d5ffff 64d5ffff ....4...D...d... 140004040 74d5ffff 54d5ffff ...The first entry,
24d5ffff, is little-endian0xffffd524, which is-0x2adc. Adding it to0x14000402cgives0x140001550, alea rax,[rip+...]that points atping. Automate it withpefile(pip install pefile):python # jumptable.py - decode a relative jump table in a PE file import struct, sys import pefile path, table_va, count = sys.argv[1], int(sys.argv[2], 16), int(sys.argv[3]) pe = pefile.PE(path) rva = table_va - pe.OPTIONAL_HEADER.ImageBase raw = pe.get_data(rva, 4 * count) for i, (off,) in enumerate(struct.iter_unpack("<i", raw)): print(f"case {i}: offset {off:#x} -> target {table_va + off:#x}")text $ python3 jumptable.py idioms_O2.exe 0x14000402c 7 case 0: offset -0x2adc -> target 0x140001550 case 1: offset -0x2aec -> target 0x140001540 case 2: offset -0x2acc -> target 0x140001560 ... case 6: offset -0x2aac -> target 0x140001580The count, 7, comes from the bounds check
cmp ecx,0x6. Check that each target loads the right string. Then findCSWTCHin.rdataforclassifyand read the return values straight out of the table. -
Structs. From
get_sizealone, write down the struct size and the offset and width of the field it reads. Compare with the comments in the source. What doesclear_recordtell you about the size ofstruct record? -
System V comparison. If you have a Linux cross-compiler (or run this on Linux with plain
gcc), build an object file and compare the argument registers:bash x86_64-linux-gnu-gcc -O2 -c -o idioms_O2.o idioms.c x86_64-linux-gnu-objdump -d -M intel --no-show-raw-insn idioms_O2.o | awk '/<get_size>:/,/^$/' -
Optional: compare Ghidra's decompiler output for
div7,mod10anddispatchwith the source, and pasteidioms.cinto Compiler Explorer to compare GCC, Clang and MSVC.
Questions to answer: Which functions look nearly identical at -O0 and
-O2, and which change completely? Why does the dispatch bounds check use
ja rather than jg? If you saw imul by 0xaaaaaaab followed by shr by
33, what would the source have been? Why can classify use a table of values
while dispatch needs a table of code offsets? In a stripped binary, what
would tell you that clear_record is an inlined memset rather than author
logic?
Key takeaways
- Compilers reuse a small set of patterns; reading them as source constructs is the real speed-up in code analysis.
- Frames tell you the ABI and optimisation level:
rcx/rdx/r8/r9and shadow space on Windows x64,rdi/rsiand no shadow space on System V,push rbpeverywhere at-O0. - A multiply by an odd constant followed by shifts is a division. Sum the
shifts, divide
2^sby the magic (adding2^32in the 33-bit fix-up form) and round up to recover the divisor, then verify in Python. - Dense
switchstatements become bounds-checked jump or value tables, sparse ones become compare trees. On x64 the tables hold 32-bit offsets, not absolute addresses. - Scaled addressing reveals element sizes and field offsets, and
call [rax+N]after loading[rcx]is a virtual call through slotN/8. - Identify library code with FLIRT or Function ID rather than reversing it, and treat missing idioms as a hint that an obfuscator has been at work.