Skip to content
Reverse Engineering

Concepts

general

Memory protection (NX / DEP / W^X)

Page permissions — read, write, execute — and the W^X rule that no page is both writable and executable. The mitigation that turned simple shellcode injection into ROP.

The CPU's paging unit attaches permission bits to every page of memory: Read, Write and eXecute. The OS sets these per region, and the hardware faults on any access that violates them (a SIGSEGV on Linux, an access-violation on Windows).

The NX bit and W^X

The NX bit (No-eXecute; marketed as DEP, Data Execution Prevention, on Windows) lets a page be readable/writable without being executable. Modern systems enforce W^X — write XOR execute — the policy that no page is both writable and executable at the same time:

RegionTypical permsWhy
.text (code)r-xexecutable, not writable
.rodatar--constants, read-only
.data / .bssrw-mutable globals, not executable
stack / heaprw-data only — NX

You can read these live from /proc/<pid>/maps (the rwxp column) or with a debugger's vmmap.

Why this matters for exploitation

Before NX, the classic attack was: overflow a buffer, write shellcode onto the stack, and redirect execution into it. W^X kills that directly — the stack is writable but not executable, so the injected bytes fault the moment the CPU tries to run them.

The response was code reuse: instead of injecting new code, chain together fragments of existing executable code.

  • ret2libc — return into an existing library function (e.g. system).
  • ROP (Return-Oriented Programming) — chain short "gadgets" ending in ret, driving the flow entirely through values placed on the stack.

These need no writable+executable page, so they sidestep W^X — which is why NX shifted exploitation from shellcode to ROP rather than ending it.

Reverse-engineering & security notes

  • A region that is rwx is a red flag: JITs (JavaScript, .NET, Lua), packers and self-modifying / unpacking malware allocate rwx (or flip permissions with mprotect / VirtualProtect) so they can write code and then run it. A call to mprotect(..., PROT_EXEC) or VirtualProtect(..., PAGE_EXECUTE_READWRITE) is a strong unpacking/codegen signal.
  • Combined with ASLR and stack canaries, NX is one leg of the standard mitigation stack; checksec reports all of them at once.
  • Permission faults are how you recognise the bug class: jumping to data → exec fault; writing to .text/GOT → write fault (a sign RELRO or r-x is doing its job).

See also: PIE & ASLR · GOT & PLT · The stack · CALL / RET.