Skip to content

Leçon 1.4 · Fondations· 35 min

How a Binary Is Loaded and Run

What happens between double-clicking an executable and main(): process creation, memory mapping, the loader, imports, ASLR, CRT startup and TLS callbacks.

Cette leçon n’est disponible qu’en anglais pour le moment.

Objectifs

  • Describe how Windows and Linux turn a file on disk into a running process
  • Explain how the loader maps an image, resolves imports and applies relocations under ASLR
  • Distinguish the entry point from main and identify code that runs before either
  • Locate the PEB and loader data structures that malware walks to find modules

A binary on disk is a blueprint, not a program. Before a single instruction of the author's code runs, the operating system builds an address space, maps the file into it, loads every library it depends on, patches addresses, and runs a surprising amount of code that is not main. Malware hides in each of those steps, and debuggers show you the process at different points along the way. If you do not know the sequence, you will not know what you are looking at.

This lesson follows Windows first, since most samples are PE files, then shows the Linux equivalent.

Processes and virtual address spaces

Each process gets its own virtual address space: a private range of addresses that the CPU and kernel translate, page by page (usually 4 KB), into physical memory. Two processes can both use address 0x140001000 and see completely different bytes. Each page also carries protection bits — readable, writable, executable — which is why code sections are typically R-X and data sections RW- (see memory protection).

A typical 64-bit Windows process looks roughly like this:

text
  low addresses
  ┌───────────────────────────────┐
  │ (unmapped, catches NULL)      │
  ├───────────────────────────────┤
  │ thread stacks                 │  RW-
  │ process heap(s)               │  RW-
  ├───────────────────────────────┤
  │ sample.exe image              │  headers R--, .text R-X,
  │   (ImageBase, e.g. 0x7FF6…)   │  .rdata R--, .data RW-
  ├───────────────────────────────┤
  │ other allocations, mapped     │
  │ files, PEB/TEB                │
  ├───────────────────────────────┤
  │ kernel32.dll, KernelBase.dll, │  each mapped as an image
  │ ntdll.dll  (0x7FFx…)          │
  └───────────────────────────────┘
  high addresses (kernel space above, not accessible)

The exact order varies with ASLR. What matters is that the executable, each DLL, the stack and the heap are separate regions, each with its own base and protections. A memory map viewer such as VMMap or System Informer lists them.

Mapping an image

The loader does not read the file into memory as a flat blob. Section headers describe where each piece of the file goes in memory:

text
        file on disk                          image in memory
  ┌─────────────────────┐ 0x000       ┌─────────────────────┐ ImageBase + 0x0000
  │ headers             │             │ headers             │
  ├─────────────────────┤ 0x400       ├─────────────────────┤ ImageBase + 0x1000
  │ .text  (raw data)   │ ──────────► │ .text               │
  ├─────────────────────┤             ├─────────────────────┤ ImageBase + 0x3000
  │ .rdata              │ ──────────► │ .rdata              │
  ├─────────────────────┤             ├─────────────────────┤ ImageBase + 0x5000
  │ .data               │ ──────────► │ .data  (+ zero-fill │
  └─────────────────────┘             │        for .bss)    │
   packed tightly,                    └─────────────────────┘
   FileAlignment (0x200)               page-aligned, SectionAlignment (0x1000)

Each PE section has a raw offset and size in the file and a virtual address and size in memory. Because sections are aligned differently on disk and in memory, file offsets and virtual addresses differ, and you will convert between them constantly — a topic for PE Sections. A section whose virtual size exceeds its raw size is zero-filled; packers use this to reserve an empty region they will later fill with unpacked code.

ELF describes the same idea with program headers (segments): each PT_LOAD entry says "map this file range at this address with these permissions." Sections are a linker's view; segments are the loader's view.

Windows: from CreateProcess to main

When a program calls CreateProcess, the work is split between the kernel and ntdll.dll inside the new process:

text
  parent process                        kernel
  CreateProcessW ──► NtCreateUserProcess ──► open file, create image section
                                             create process + address space
                                             map sample.exe and ntdll.dll
                                             create PEB, TEB, initial thread
                                                        │
  ─────────────── new process, user mode ───────────────┼──────────────────
                                                        ▼
  ntdll!LdrInitializeThunk
     └─ loader initialisation
          • load imported DLLs (recursively)
          • resolve imports → write addresses into the IAT
          • apply base relocations if needed
          • run TLS callbacks and DllMain(DLL_PROCESS_ATTACH)
  ntdll!RtlUserThreadStart
     └─ AddressOfEntryPoint  = CRT startup (e.g. mainCRTStartup)
          └─ initialise C runtime, globals, argv
               └─ main / WinMain

Resolving imports

The PE's import directory lists DLL names and, for each, the functions needed (by name or by ordinal). For every DLL the loader either finds it already loaded or searches for it, maps it, and recursively processes its imports. Then it looks up each function in the DLL's export table and writes the resulting address into the corresponding slot of the Import Address Table (IAT). Compiled code calls imports indirectly through these slots:

text
  call qword ptr [rip + IAT_slot_for_CreateFileW]

This is why a static tool can list a sample's imports: they are data the loader must be able to read. It is also why malware that wants to hide its capabilities skips the import table and finds functions itself at runtime — see dynamic import resolution and API hashing.

Relocations and ASLR

Every PE records a preferred ImageBase. Code that contains absolute addresses is only correct at that base. With ASLR, Windows deliberately loads images at randomised bases, so the loader must fix those absolute addresses using the base relocation table (.reloc): a list of locations in the image that hold absolute addresses and must be adjusted by the difference between the actual and preferred base.

An image opts in to ASLR with the IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE flag (MSVC's /DYNAMICBASE, the default for years). Some details worth knowing:

  • x64 code mostly uses RIP-relative addressing, so it needs far fewer relocations than 32-bit code.
  • Windows randomises DLL bases once per boot, which lets every process share one copy of each system DLL. Stacks and heaps, by contrast, are randomised per process.
  • An image without a relocation table cannot be moved, so it loads at its preferred base. Old or hand-crafted malware sometimes shows up this way.

On Linux the same role is played by position-independent executables (PIE), now the default on most distributions; see PIE and ASLR.

Entry point is not main

AddressOfEntryPoint in the optional header points to the CRT startup routine, not to main. For MSVC programs that is mainCRTStartup, WinMainCRTStartup or _DllMainCRTStartup, which set up security cookies, initialise the C runtime, run global constructors, parse the command line and only then call main. When you open a sample in a debugger and it stops at the "entry breakpoint", you are in compiler boilerplate, often several calls away from the author's code.

That distinction matters twice over in malware:

  • In a normal binary, finding main means stepping past CRT startup to the call that passes argc/argv (or hInstance for WinMain).
  • In a packed binary, the entry point is the unpacking stub. The real program's entry point, once unpacked, is the original entry point (OEP) — the target of manual unpacking later in this path.

TLS callbacks: code before the entry point

The PE TLS directory can list callback functions that the loader invokes during process initialisation — before the entry point — and again on thread creation and exit. They exist for legitimate thread-local-storage setup, but malware uses them to run anti-debugging checks or decryption before an analyst's debugger reaches its entry breakpoint. See TLS callbacks.

Tip: In x64dbg, open Options → Preferences → Events and enable breaking on TLS Callbacks (and the System Breakpoint) so you stop before any sample code runs. Check the TLS directory in PE-bear during triage; a TLS directory with callbacks in a small, non-CRT program deserves attention.

The PEB and loader data

Each process has a Process Environment Block (PEB), a user-mode structure the loader fills in. Each thread has a Thread Environment Block (TEB). On x64, the gs segment register points to the current TEB, and the TEB holds a pointer to the PEB at offset 0x60; on 32-bit x86, fs:[0x30] holds the PEB pointer directly (see segment registers).

x64 PEB fieldOffsetWhy analysts care
BeingDebugged0x02Read by IsDebuggerPresent — and by malware directly
ImageBaseAddress0x10Where the main image was actually loaded
Ldr0x18Pointer to PEB_LDR_DATA, the list of loaded modules
NtGlobalFlag0xBCHeap-debug flags set when started under a debugger

PEB_LDR_DATA holds doubly linked lists (in load order, memory order and initialisation order) of LDR_DATA_TABLE_ENTRY records, one per module, with each module's base address, size and name. Shellcode and import-hiding malware walk exactly this list to find kernel32.dll without calling any API, so the instruction sequence "read gs:[0x60], then offset 0x18" is a strong signal in disassembly. The BeingDebugged and NtGlobalFlag fields are the basis of IsDebuggerPresent and NtGlobalFlag checks.

Linux: from execve to main

The Linux sequence has the same shape with different names:

  1. execve asks the kernel to replace the current process image. The kernel reads the ELF header and program headers and maps each PT_LOAD segment.
  2. If the binary has a PT_INTERP header — every dynamically linked binary does — the kernel also maps the named interpreter, typically /lib64/ld-linux-x86-64.so.2, and starts execution in the interpreter, not in the program. It passes an auxiliary vector (AT_ENTRY, AT_PHDR, AT_BASE and others) on the stack to say where everything is.
  3. The dynamic loader maps every library listed as DT_NEEDED, applies relocations, and prepares the GOT. Function imports may be resolved eagerly or lazily on first call through the PLT (see GOT and PLT), depending on linker flags such as -z now.
  4. It runs library initialisers, then jumps to the program's entry point _start, which calls __libc_start_main, which runs constructors (from .init_array) and finally calls main.

Just as with TLS callbacks, constructors run before main: a function marked __attribute__((constructor)) executes before main. Linux malware uses this, and so does LD_PRELOAD-based hooking, which injects a library the loader maps before the program's own dependencies.

For more ELF detail, see ELF for Malware Analysts.

Lab: watch ASLR and the loader

Use a Linux or WSL shell with gcc and gdb. The program is benign: it prints some addresses and waits.

  1. Write and build the program:

    c
    // where.c — prints where things live, then waits for Enter
    #include <stdio.h>
    #include <stdlib.h>
    
    int counter = 7;                       /* global in .data */
    
    int main(void) {
        int local = 0;                     /* on the stack    */
        char *buf = malloc(64);            /* on the heap     */
        printf("main    %p\n", (void *)main);
        printf("counter %p\n", (void *)&counter);
        printf("local   %p\n", (void *)&local);
        printf("heap    %p\n", (void *)buf);
        printf("printf  %p\n", (void *)printf);
        getchar();                         /* pause so we can inspect */
        free(buf);
        return 0;
    }
    bash
    gcc -O0 -o where where.c
    gcc -O0 -no-pie -o where-nopie where.c
  2. Observe ASLR. Run ./where three times (press Enter each time). All five addresses change between runs. Check the system setting:

    bash
    cat /proc/sys/kernel/randomize_va_space   # 2 = full randomisation
  3. Compare a non-PIE build. Run ./where-nopie several times. main and counter now stay fixed (typically around 0x401000/0x404000), while the stack, heap and printf in libc still move. Only the executable itself lost its randomisation.

  4. Read the memory map. Run ./where and leave it waiting. In a second terminal:

    bash
    cat /proc/$(pgrep -n where)/maps

    Match each printed address to a line: main falls in the r-xp mapping of where, counter in its rw-p mapping, the heap in [heap], local in [stack], and printf inside libc.so.6. Note the separate mappings with different permissions for one file — those are its PT_LOAD segments.

  5. Stop before any program code runs. Start the binary under gdb:

    bash
    gdb -q ./where
    (gdb) starti
    (gdb) info proc mappings
    (gdb) x/i $pc

    starti stops at the very first user-mode instruction. x/i $pc shows you are inside ld-linux-x86-64.so.2, and the mapping list contains the program and the loader but no libc yet — the loader has not loaded it.

  6. Let the loader finish. Continue to main and look again:

    bash
    (gdb) break main
    (gdb) continue
    (gdb) info proc mappings
    (gdb) info sharedlibrary

    libc is now mapped. Run the program twice inside gdb and notice the addresses are identical: gdb disables ASLR by default (set disable-randomization on). Turn it off with set disable-randomization off to see randomisation again. Remember this when comparing debugger addresses with a live process.

  7. (Optional, Windows) Build the same program with cl /Od where.c and run it a few times. The stack and heap addresses change on every run; the executable's own addresses often stay the same until reboot. Then open it in x64dbg: execution first pauses at the system breakpoint inside ntdll, then at the entry breakpoint on mainCRTStartup. Use the Memory Map tab to find the image, its sections and their protections, and follow main from the CRT startup code.

Questions to answer: Which addresses did the non-PIE build fix, and which did it not — and why? Why does libc not appear in the mapping list at starti? In x64dbg, how many instructions and calls separate the entry breakpoint from main, and what would you expect to see there in a packed sample?

Key takeaways

  • Each process has a private virtual address space; the executable, each library, the stack and the heap are separate regions with their own permissions.
  • The loader maps sections or segments at their virtual addresses, so file offsets and memory addresses differ.
  • Imports are resolved at load time into the IAT (Windows) or GOT (Linux); malware that hides imports resolves them itself, often by walking the PEB.
  • ASLR moves images, stacks and heaps; base relocations or position-independent code make that possible.
  • The entry point is CRT startup, not main — and TLS callbacks and constructors run even earlier, which malware exploits.