Patching Binaries to Defeat Checks

Debugging Malware with x64dbg ended by flipping a flag to take the branch a sample refused, and promised that persistent patching would get its own lesson. This is it. Patching means changing a program's bytes — in a debugger for one run, or on the file for good — so that it does something it would not do on its own.

For a malware analyst, patching is not about cracking software. It is about making a stubborn sample talk: a binary gates its interesting behaviour behind a check for a debugger, a sandbox, a command-line argument, a mutex or a C2 response, and rather than satisfy every check you edit the check away and watch what happens next. The capability you uncover is real; the sequence of events is forced, so you report it as such. Everything here runs in the isolated lab from Building a Safe Analysis Lab.

What a patch buys you

Most analysis patches are one of four small moves. Each defeats a category of check you have already met in this path.

Concretely, these are how you get past the evasion techniques in the rest of this module. A IsDebuggerPresent or CheckRemoteDebuggerPresent call whose result drives a jcc — flip the jump or force the return to zero. A CPUID hypervisor check or a user-activity sandbox check — same treatment. An RDTSC timing trap or a delayed-execution stall — NOP the delay or the comparison. In every case you found the check first with the static and dynamic methods from Cross-References and Data Flow and Reading Capabilities from Imports; patching is only the last step.

In memory or on the file

There are two places to patch, and choosing well saves time.

An in-memory patch happens in the debugger. In x64dbg you select the instruction, press Space, and type the replacement; the change lives in the process and vanishes on restart unless you save it. This is the right tool while you are still exploring: it is instant, trivially reversible, and it works on code that only exists after unpacking or decryption — code that is not in the file at all. The whole of How Packers Work is a reminder that the bytes you care about often live only in memory; you cannot file- patch what is compressed on disk. Patch such code in memory, or patch it after you dump and repair it as in Dumping Memory and Extracting Payloads.

A file patch edits the executable on disk, so the change persists across runs and can be shared with a teammate or fed to a sandbox. The cost is that you must locate the byte yourself and you risk tripping an integrity check. Reach for a file patch when the check runs many times (flipping a flag by hand on every hit is misery), when you want a reusable "declawed" sample, or when another tool — not a debugger — will run the binary.

From RVA to file offset

Your disassembler reports an instruction's address as an RVA (or a virtual address you subtract the image base from). The file has no RVAs; it has raw offsets. Every PE section records both, so the conversion is:

file_offset = RVA − section.VirtualAddress + section.PointerToRawData

for the section whose virtual range contains the RVA. pefile does this for you with pe.get_offset_from_rva(rva), which is what the lab uses. Get it wrong and you overwrite an unrelated byte — always disassemble the bytes at the offset afterward to confirm you hit the instruction you meant.

Warning: A file patch can break things beyond the instruction. The PE header carries an optional CheckSum; the Windows loader verifies it for drivers and some system images, so those will fail to load until you recompute it (PE-bear and many tools can). Any byte change invalidates an Authenticode signature — fine for lab analysis, but the file will show as tampered. And the malware may checksum itself: a self-verification routine that hashes its own .text will notice your patch and change behaviour. When a file patch misbehaves, suspect self-integrity first.

Encoding a patch by hand

A patch is legal only if it leaves the surrounding code disassembling correctly. The cardinal rule: preserve instruction boundaries and total length. If your replacement is shorter than what it replaces, pad the remainder so the next instruction still starts where it started. If it is longer, you cannot fit it in place — you need a code cave or a different approach. See x86 instruction encoding for why lengths are fixed by the bytes, not the mnemonic.

The building blocks:

Two encodings of the same jump matter. A short conditional jump is two bytes: an opcode 0x70–0x7F plus a one-byte signed offset (74 16 is je +0x16). A near conditional jump is six bytes: 0F then 0x80–0x8F then a four-byte offset. The unconditional jmp is EB rel8 (2 bytes) or E9 rel32 (5 bytes). Because a short je and a short jmp are both two bytes, turning one into the other is a clean one-byte opcode swap with the offset left alone — the safest patch there is. Flipping je to jne is likewise one byte and keeps the jump conditional, which is what you want when you only mean to invert a single decision. Consult an opcode reference rather than trusting memory for anything rarer.

NOPing a call is the other everyday patch. A near call is E8 plus a four-byte displacement — five bytes — so you replace all five with 90. Leave four and the stray displacement byte becomes a bogus instruction that derails the disassembler and, worse, the CPU. To neutralise a check and control its result, it is often cleaner to keep the call and instead patch the check function itself to 31 c0 c3, or to patch the jcc that consumes its result.

Tip: Prefer the smallest, most local patch that answers your question. Flipping one jcc is easier to reason about, document and undo than NOPing a block. If you find yourself NOPing many instructions, step back — a hook or an environment change (below) may be the right tool.

Document every patch

A patched sample is no longer the sample. If you do not record what you changed, you will confuse yourself an hour later and mislead anyone who reads your notes. For each patch, write down:

Hash the patched file, keep the original, and — as the debugging lesson insisted — report any behaviour you reached through a patch as forced. The capability is genuine; the path to it was not one a victim would take.

When patching is the wrong tool

Patching is surgical and permanent, which makes it the wrong choice as often as the right one.

• The check runs constantly, or you need normal behaviour otherwise. If a sample calls its anti-debug routine in a hot loop but you want everything else intact, hooking the API (as in the anti-debug plugin ScyllaHide) is cleaner than carving the binary — it changes what the function returns without touching the call sites. See Anti-Debugging.
• The problem is the environment, not the code. When a sample bails because the machine looks like a VM, patching each of a dozen sandbox checks is fragile and easy to miss one. Making the VM look like a real workstation — more RAM, real filenames, user activity — defeats them all at once and keeps the sample's behaviour faithful.
• Later code depends on what you skipped. NOP a decryption call and the code that reads the decrypted buffer will crash or read garbage. If a check has side effects, forcing its result is safer than removing it.
• The code is packed or self-modifying. You cannot meaningfully patch bytes on disk that are compressed or rewritten at runtime; work in memory after they are restored (manual unpacking is covered in an upcoming lesson).

A good rule: flip a flag while exploring, hook when a check is noisy, change the environment when the environment is the tell, and file-patch when you want a durable, shareable result and have confirmed the sample does not check itself.

Lab: patch a gate to reveal hidden behaviour

You will build a program that hides its behaviour behind an environment-variable check, then write a patcher that finds the gate with capstone, converts its RVA to a file offset with pefile, flips one byte, and produces a patched copy — which you verify with wine. You need mingw-w64, a virtual environment with pefile and capstone, and wine (if it is unavailable, verify by disassembling the patched bytes instead). Listings come from MinGW-w64 GCC, pefile 2024.8.26, Capstone 5.0.7 and Wine 11.0; your addresses will differ.

1. Save gate.c. The is_unlocked helper is kept out of main so a single conditional jump in main gates the two paths:

   #include <stdio.h>
   #include <stdlib.h>
   #include <string.h>
   
   /* 1 only when LAB_UNLOCK holds the expected value */
   __attribute__((noinline))
   static int is_unlocked(void) {
       const char *v = getenv("LAB_UNLOCK");
       return v != NULL && strcmp(v, "open-sesame") == 0;
   }
   
   int main(void) {
       if (is_unlocked())
           puts("unlocked - real behaviour would run here");
       else
           puts("check failed - hiding behaviour");
       return 0;
   }

   python3 -m venv venv && ./venv/bin/pip install pefile capstone
   x86_64-w64-mingw32-gcc -O1 -s -o gate.exe gate.c
   wine gate.exe                       # no variable set
   LAB_UNLOCK=nope wine gate.exe       # wrong value
   LAB_UNLOCK=open-sesame wine gate.exe # correct value

   check failed - hiding behaviour
   check failed - hiding behaviour
   unlocked - real behaviour would run here

   The behaviour is hidden unless the check passes — exactly the shape of a sample that refuses to act outside its expected environment.

2. Save patch.py. It finds main as the function that references the "hiding behaviour" string, takes the first short jcc in it as the gate, maps its RVA to a file offset, flips the opcode byte, and writes a patched copy:

   # patch.py — flip the conditional jump that gates the hidden branch
   import pefile
   from capstone import Cs, CS_ARCH_X86, CS_MODE_64
   from capstone.x86 import X86_OP_MEM, X86_REG_RIP, X86_GRP_JUMP
   
   SRC, DST = "gate.exe", "gate_patched.exe"
   JCC = {0x74: ("je", 0x75, "jne"), 0x75: ("jne", 0x74, "je")}
   
   pe = pefile.PE(SRC)
   base = pe.OPTIONAL_HEADER.ImageBase
   img = pe.get_memory_mapped_image()
   md = Cs(CS_ARCH_X86, CS_MODE_64); md.detail = True
   
   def refs_string(insns, needle):
       for i in insns:
           for op in i.operands:
               if op.type == X86_OP_MEM and op.mem.base == X86_REG_RIP:
                   ea = i.address + i.size + op.mem.disp - base
                   s = img[ea:img.find(b"\0", ea)]
                   if needle in s:
                       return True
       return False
   
   # x64 PE files list every non-leaf function's bounds in .pdata
   funcs = [(base + e.struct.BeginAddress, base + e.struct.EndAddress)
            for e in pe.DIRECTORY_ENTRY_EXCEPTION]
   start, end = next((s, e) for s, e in funcs
                     if refs_string(list(md.disasm(img[s-base:e-base], s)), b"hiding behaviour"))
   insns = list(md.disasm(img[start-base:end-base], start))
   
   gate = next(i for i in insns if X86_GRP_JUMP in i.groups and i.bytes[0] in JCC)
   rva = gate.address - base
   was, opp, opp_name = JCC[gate.bytes[0]]
   print(f"gate: {gate.mnemonic} at VA {gate.address:#x} (RVA {rva:#x}) bytes {gate.bytes.hex(' ')}")
   
   off = pe.get_offset_from_rva(rva)
   print(f"RVA {rva:#x} -> file offset {off:#x}")
   data = bytearray(open(SRC, "rb").read())
   assert data[off] == gate.bytes[0], "opcode at file offset does not match"
   print(f"file byte at {off:#x}: {data[off]:#04x} ({was})  ->  {opp:#04x} ({opp_name})")
   data[off] = opp
   open(DST, "wb").write(data)
   print(f"wrote {DST} ({len(data)} bytes)")
   
   patched = next(md.disasm(bytes(data[off:off+2]), gate.address))
   print(f"patched instruction: {patched.mnemonic} {patched.op_str} (bytes {patched.bytes.hex(' ')})")

   ./venv/bin/python patch.py

   gate: je at VA 0x1400014d6 (RVA 0x14d6) bytes 74 16
   RVA 0x14d6 -> file offset 0x8d6
   file byte at 0x8d6: 0x74 (je)  ->  0x75 (jne)
   wrote gate_patched.exe (16896 bytes)
   patched instruction: jne 0x1400014ee (bytes 75 16)

   One byte changed: the je that jumped to the "check failed" block became a jne, so the sense of the decision is inverted while its target and length stay identical. The re-disassembly confirms the boundary survived.

3. Confirm exactly one byte differs, then run both copies:

   cmp -l gate.exe gate_patched.exe
   wine gate.exe            # unpatched, no variable
   wine gate_patched.exe    # patched, no variable

   2263 164 165
   check failed - hiding behaviour
   unlocked - real behaviour would run here

   cmp -l reports a single differing byte (offset 2263 decimal is 0x8d6 plus one, in cmp's 1-based count; 164/165 octal are 0x74/0x75). Same input, opposite behaviour: the patched binary reveals the branch the original hid. If wine is unavailable, rerun patch.py's final step on the patched file to show jne in place of je — the disassembly is proof enough that the gate is inverted.

4. Compare the two byte-level approaches. In x64dbg you would reach the same result by stopping at the je and pressing Space to assemble jne — an in-memory patch that vanishes on restart. The file patch you just made persists. Try also flipping the gate to an unconditional jump instead (0x74 → 0xeb) and predict how the behaviour differs from the jne version before you test it.

Questions to answer: With the gate patched to jne, what does LAB_UNLOCK=open-sesame wine gate_patched.exe now print, and why? Why is flipping je to jne here safer than NOPing the test and je together? If gate.exe recomputed a hash of its own .text at startup and refused to run on a mismatch, which of your two patches (in-memory vs file) would survive, and how would you handle the other? What file offset would the gate map to if its RVA fell in a section whose PointerToRawData differed from its VirtualAddress?

Key takeaways

• Patching is an analysis tool: it forces a sample to reveal behaviour hidden behind a check, and any path reached this way is reported as forced.
• Patch in memory (debugger, reversible, works on unpacked code) while exploring; patch the file when you need a durable, shareable result.
• Convert an RVA to a file offset with RVA − VirtualAddress + PointerToRawData (or pefile.get_offset_from_rva), and always re-disassemble to confirm the hit.
• Preserve instruction boundaries and length: flip a jcc opcode (74↔75), force a branch (74→eb), NOP a whole call with 90s, or stub a function with 31 c0 c3.
• File patches can trip PE checksums, invalidate signatures and be caught by a sample's own integrity check — suspect self-verification when a patch misfires.
• Hook instead of patch when a check is noisy, change the environment when the environment is the tell, and never file-patch code that is packed or self-modifying — work on the restored bytes in memory.