Skip to content

Lesson 5.1 · Windows Internals for Analysts· 40 min

The Windows API for Analysts

Read the Win32 API the way a malware author uses it — layers, naming conventions, handles and the calling convention — to predict capability from imports alone.

Objectives

  • Describe the Win32 API as a layered system: subsystem DLLs, ntdll, the syscall boundary and the kernel
  • Read an A/W-suffixed function name and a handle-returning signature the way Microsoft Learn documents it
  • Follow the x64 calling convention through a disassembly listing to recover a call's real arguments
  • Explain why malware resolves APIs dynamically instead of importing them, and recognise the patterns that do it
  • Predict a sample's capability from its import table before running a single instruction

Cross-References and Data Flow gave you the habit of following a call to where it leads. On Windows, most of those calls eventually lead somewhere specific: a function in the Win32 API, the enormous surface of kernel32.dll, advapi32.dll, user32.dll, ws2_32.dll and dozens of siblings that every Windows program — legitimate or not — is built on. Reading a sample's imports is reading its stated intentions, before you have run a single instruction. This lesson is about reading that surface the way it was designed to be read: what a function's name tells you, what its parameters mean, and how to find both when a sample refuses to import them normally.

A layered API, not a flat one

Win32 is not the bottom of the stack. Every subsystem DLL you see in an import table is itself a thin wrapper:

text
  kernel32.dll / advapi32.dll / user32.dll / ws2_32.dll / wininet.dll / crypt32.dll
                          │  (Win32 API — what samples import)
                          ▼
                     ntdll.dll
                          │  (Native API — Nt*/Zw* functions, undocumented structures)
                          ▼
                      syscall
                          │  (mode transition, see below)
                          ▼
                    the kernel (ntoskrnl.exe)

kernel32!CreateFileW, for example, validates its arguments, builds the right structures, and calls ntdll!NtCreateFile, which executes the syscall instruction to cross into kernel mode. Almost every capability a sample needs — files, the registry, processes, threads, memory, sockets — is exposed at this Win32 layer, so that is where most of your reading happens. The native layer underneath it, and the small set of samples that talk to it directly to dodge user-mode monitoring, gets its own lesson later in this module: User Mode, Kernel Mode and the Native API. For now, treat ntdll as a name that appears in stack traces and move on — you do not need its internals to read an import table.

Reading a function name

Microsoft's naming conventions are consistent enough to read without a reference open, most of the time.

The A/W suffix

Most string-taking functions exist twice: an ANSI version taking char*, and a Wide version taking wchar_t* (UTF-16). CreateFileA and CreateFileW do the same thing; the header macro CreateFile resolves to whichever one your program was compiled for. In practice:

  • Modern, non-legacy malware overwhelmingly calls the W forms, because that is what current toolchains default to. A binary that imports mostly A functions is often either old, cross-compiled with an older toolchain default, or deliberately mimicking legacy software.
  • A function present in only one form (GetModuleHandleW has no meaningful A/W distinction beyond the string argument; some newer APIs, like most of bcrypt.dll, take UTF-16 unconditionally and have no suffix at all) is not a red flag by itself — check the specific function on Microsoft Learn before reading anything into it.

Handles are the common currency

Nearly every kernel-managed object — a file, a process, a thread, a registry key, a mutex, a memory section — is represented in user mode as an opaque HANDLE: a process-local number with no meaning outside that process (see Processes, Threads and DLLs for how the handle table fits into a process). The pattern repeats everywhere in the API:

text
  HANDLE CreateFileW(...)                 -> a handle to a file
  HANDLE OpenProcess(...)                 -> a handle to another process
  HANDLE CreateMutexW(...)                -> a handle to a named mutex
  BOOL   WriteProcessMemory(HANDLE hProc, ...)   -> consumes a process handle
  BOOL   CloseHandle(HANDLE)              -> releases any of the above

Once you recognise this, an unfamiliar function's shape tells you its role before you read a single line of documentation: something that returns a HANDLE acquires a resource; something that takes one operates on a resource acquired earlier, usually by a different call you should go and find.

Reading a Microsoft Learn page

Every Win32 function's documentation page follows the same layout, and four parts of it matter for capability-reading:

SectionWhat it tells you
SyntaxParameter types and order — read this to match arguments against a disassembly, covered below
ParametersWhat each argument controls — flags worth knowing by name (dwDesiredAccess, dwCreationDisposition)
Return valueWhat success and failure look like — usually a HANDLE, a BOOL, or a count
Requirements → DLLWhich import library the function actually lives in — the name in a sample's import table

Take CreateFileW: its signature is

c
HANDLE CreateFileW(
  LPCWSTR               lpFileName,
  DWORD                  dwDesiredAccess,
  DWORD                  dwShareMode,
  LPSECURITY_ATTRIBUTES  lpSecurityAttributes,
  DWORD                  dwCreationDisposition,
  DWORD                  dwFlagsAndAttributes,
  HANDLE                 hTemplateFile
);

You do not need to memorise every flag. You need to know that argument two (dwDesiredAccess) is usually GENERIC_READ, GENERIC_WRITE, or both, and argument five (dwCreationDisposition) is one of CREATE_NEW, CREATE_ALWAYS, OPEN_EXISTING and similar — so that when you recover those two values from a disassembly, you can say "opens or creates a file, for writing" without opening the reference again.

Now take WriteProcessMemory:

c
BOOL WriteProcessMemory(
  HANDLE  hProcess,
  LPVOID  lpBaseAddress,
  LPCVOID lpBuffer,
  SIZE_T  nSize,
  SIZE_T  *lpNumberOfBytesWritten
);

The first parameter is a HANDLE to a process other than the caller's own in the overwhelming majority of malicious uses — legitimate self-writes are rare. An import table containing OpenProcess (acquires that handle), VirtualAllocEx (reserves memory in the target) and WriteProcessMemory (writes into it), in that order, is close to a definition of process injection before you have looked at a single byte of code — a pattern Processes, Threads and DLLs and the later injection lesson build on directly.

The calling convention, read from disassembly

Reading imports gets you the name; reading a disassembly gets you the arguments actually passed. On x64 Windows, every function uses one calling convention: the first four integer or pointer arguments go in rcx, rdx, r8, r9 in that order (see general-purpose registers), and any further arguments spill to the stack. The caller also reserves 32 bytes of shadow space above the return address — stack room the callee may use to spill those four register arguments, whether or not it actually does.

text
  rcx -> arg 1        r8  -> arg 3            [rsp+0x28] -> arg 5
  rdx -> arg 2        r9  -> arg 4            [rsp+0x30] -> arg 6, ...

rax carries the return value. rbx, rbp, rdi, rsi, r12–r15 are callee-saved — a called function must restore them before returning, so you can trust their value across a call you have not stepped into; rcx, rdx, r8–r11 are caller-saved and may be clobbered by any call.

Reading a real CreateFileW call site puts this together:

text
  lea  rcx, [rip+0x2A10]      ; arg1 lpFileName -> a UTF-16 string, read it in the dump
  mov  edx, 0xC0000000        ; arg2 dwDesiredAccess = GENERIC_READ | GENERIC_WRITE
  xor  r8d, r8d               ; arg3 dwShareMode = 0 (no sharing)
  xor  r9d, r9d               ; arg4 lpSecurityAttributes = NULL
  mov  dword ptr [rsp+0x20], 3   ; arg5 dwCreationDisposition = OPEN_EXISTING
  mov  dword ptr [rsp+0x28], 0x80   ; arg6 dwFlagsAndAttributes = FILE_ATTRIBUTE_NORMAL
  mov  qword ptr [rsp+0x30], 0      ; arg7 hTemplateFile = NULL
  call qword ptr [rip+0x3F12]       ; -> CreateFileW (via the IAT)

The pattern to build is mechanical: find the call (or, for an imported function, the jmp/call through the IAT), then walk backwards from it collecting the last write to each of rcx, rdx, r8, r9 and the shadow-space stack slots, in that order, ignoring writes that a later instruction overwrites before the call. lea loading a RIP-relative address is almost always a pointer argument (a string, a buffer, a structure); an immediate mov into a register is a flag or a size you can look up directly against the parameter list from Microsoft Learn.

Why malware avoids the import table

Everything above assumes the function you care about is sitting in the import table with its real name — which is exactly what many samples try to avoid, because Reading Capabilities from Imports works just as well for a defender as it does for you here. Three patterns recur:

  • Dynamic resolution. Instead of a static import, the sample calls kernel32!LoadLibraryA with a DLL name it may have built or decrypted at runtime, then GetProcAddress with the function name as a string, and calls through the returned pointer. The DLL and function names still exist as strings somewhere in the binary — often encoded — which is why Strings and Obfuscated Strings and Scripting String Decryption matter as much as the import table itself. See dynamic import resolution.
  • Hashed names. To avoid even the plaintext function name appearing as a string, the sample hashes each export name it walks and compares against a hard-coded constant, resolving the address without ever writing "WriteProcessMemory" anywhere in the file. See API hashing.
  • Manual export walking. The most self-contained form skips GetProcAddress too: the sample reads its own or ntdll's PEB to find the loaded-module list, walks a module's export directory in memory, and resolves the address by hand — leaving no LoadLibrary/GetProcAddress calls to break on at all. See PEB-walk API resolution and, for the opposite defensive trick of a benign-looking but scrambled import table, import table obfuscation.

The consequence for triage: a short or unremarkable import list is not evidence of a simple program. LoadLibraryA and GetProcAddress present together, with little else beyond them, is itself a signal that the interesting imports are being resolved somewhere you cannot see without running the sample or decoding its strings.

Predicting capability from imports alone

Put the naming conventions and the resolution patterns together and an import table becomes a checklist you can read in under a minute:

Imports seen togetherCapability implied
InternetOpenW, InternetConnectW, HttpSendRequestW (or WinHttp* equivalents)HTTP-based network communication
RegOpenKeyExW, RegSetValueExW under \Run pathsRegistry-based persistence
OpenProcess, VirtualAllocEx, WriteProcessMemory, CreateRemoteThreadClassic process injection
CryptEncrypt/BCryptEncrypt, FindFirstFileW, FindNextFileWBulk file encryption
GetAsyncKeyState or SetWindowsHookExW with WH_KEYBOARD_LLKeylogging
LoadLibraryA + GetProcAddress, little elseCapability hidden behind dynamic resolution — read the strings next

Treat this table as a starting hypothesis, not a verdict — the later Triage Report and behaviour-specific lessons in Malware Behaviours confirm it against actual code and runtime evidence. But an import table read this way, in the first few minutes with a sample, tells you where to point every tool that comes after it.

Lab: predict, then verify, from an import table alone

You need mingw-w64, pefile (pip install pefile), and PE-bear or another PE viewer. No sample runs in this lab — everything happens statically, on a program you compile yourself.

  1. Save apilab.c. It calls a handful of Win32 functions typical of a dropper's early behaviour, without actually doing anything harmful:

    c
    /* apilab.c: exercises a small set of Win32 APIs for import-table reading. */
    #include <windows.h>
    #include <stdio.h>
    
    int main(void) {
        wchar_t path[MAX_PATH];
        GetModuleFileNameW(NULL, path, MAX_PATH);
    
        HANDLE f = CreateFileW(L"apilab_marker.txt", GENERIC_WRITE, 0, NULL,
                                CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
        if (f != INVALID_HANDLE_VALUE) {
            const char msg[] = "apilab ran";
            DWORD written;
            WriteFile(f, msg, sizeof(msg) - 1, &written, NULL);
            CloseHandle(f);
        }
    
        HKEY key;
        if (RegCreateKeyExW(HKEY_CURRENT_USER,
                             L"Software\\ApiLabDemo", 0, NULL, 0,
                             KEY_WRITE, NULL, &key, NULL) == ERROR_SUCCESS) {
            RegSetValueExW(key, L"Marker", 0, REG_SZ,
                           (const BYTE *)L"demo", sizeof(L"demo"));
            RegCloseKey(key);
        }
    
        printf("done: %ls\n", path);
        return 0;
    }
    bash
    x86_64-w64-mingw32-gcc -O2 -s -o apilab.exe apilab.c -ladvapi32
  2. Before running it anywhere, list its imports with pefile:

    python
    # dumpimports.py
    import sys, pefile
    pe = pefile.PE(sys.argv[1])
    for entry in pe.DIRECTORY_ENTRY_IMPORT:
        print(entry.dll.decode())
        for imp in entry.imports:
            if imp.name:
                print("   ", imp.name.decode())
    bash
    python3 dumpimports.py apilab.exe
  3. Write down a prediction before opening the disassembler: for each imported function, its A/W suffix, whether it returns or consumes a HANDLE, and one sentence of capability — using only the table above and Microsoft Learn's parameter lists, not the source you already have.

  4. Open apilab.exe in PE-bear, jump to the RegSetValueExW and CreateFileW call sites in the disassembly view, and walk backwards from each call to recover rcx/rdx/r8/r9 and any shadow-space arguments, exactly as in the worked example above. Compare the recovered dwDesiredAccess and dwCreationDisposition values against the constants on the CreateFileW reference page.

  5. Check your prediction against the source you already have — did the import list alone correctly predict "creates or overwrites a file" and "writes a registry value under HKCU\Software"? Which single import, if it had been the only interesting one present, would have told you nothing on its own until you also saw LoadLibraryA and GetProcAddress sitting next to it?

Questions to answer: If apilab.exe had called RegSetValueExW on a key under ...\Run instead of a demo path, which single word would your written prediction have needed to change? Which of the six imports you dumped exists in both an A and a W form, and which does not — and does Microsoft Learn's "Requirements" section change which DLL you would expect to see in the import table for each? If you replaced the static RegCreateKeyExW/RegSetValueExW calls with LoadLibraryA("advapi32.dll") + two GetProcAddress calls, what would dumpimports.py show instead, and what would you have to read next to recover the same capability?

Key takeaways

  • Win32 is a layer over ntdll and the syscall boundary, not the bottom of the stack — but it is where nearly every capability a sample needs is exposed, and where most of your reading happens.
  • The A/W suffix, the handle-in/handle-out shape of a signature, and the "Requirements → DLL" line on a Microsoft Learn page let you read an unfamiliar function almost as fast as a familiar one.
  • The x64 calling convention (rcx, rdx, r8, r9, then the stack, with 32 bytes of shadow space) lets you recover a call's real arguments by walking backwards from the call instruction.
  • A short import table is not evidence of a simple sample: LoadLibraryA and GetProcAddress, API hashing, and PEB export-walking all exist specifically to keep the real capability out of the import table.
  • Read imports as a hypothesis about capability, then confirm it with strings, disassembly and — later in this path — dynamic behaviour, rather than trusting the import list on its own.