Obfuscation and Packing: Techniques Used in Malware to Evade Detection

Malware authors frequently employ obfuscation and packing to hide malicious functionality and evade detection by static analysis tools. These techniques ensure the malware remains functional while making it challenging for analysts to understand the program’s structure and behavior. This article explores common obfuscation and packing methods, their implications, and techniques for handling them.

---

What is Obfuscation?

Obfuscation involves altering code to make it harder to analyze while retaining its functionality. This is achieved by introducing complexity or hiding information, which can confuse reverse engineers and static analysis tools.

Common Obfuscation Techniques

1. Base64 Encoding
   • Description: Converts binary data into an ASCII format using a 64-character set (A-Z, a-z, 0-9, +, and /), often with padding (=).
   • Example:
     • Input: one
     • Binary Conversion: 01001111 01101110 01100101
     • Base64 Encoding: t25l
   • Use in Malware: Encodes data to appear as harmless text, commonly found in HTTP traffic or embedded strings.
2. XOR Encryption
   • Description: Each byte of plain text is XORed with a key to produce encrypted data. XOR is reversible, making it ideal for malware encryption.
   • Example:
     • Encrypting cat with the key 40 results in #exclamation!4.
     • Reapplying XOR with the same key decrypts the data.
   • Variants:
     • Single-byte XOR: Quick and simple but vulnerable to brute force.
     • Multi-byte XOR: Uses a longer key (e.g., 4 bytes), making brute force attacks significantly harder.
3. Junk Insertion
   • Description: Adds unnecessary instructions (junk code) to mislead disassemblers and analysts. These instructions are ignored during runtime.
4. Branch Functions
   • Description: Alters the flow of execution using indirect calls or jumps, making it harder to trace the program logic.
5. Overlapping Instructions
   • Description: Shares or overlaps instructions across different execution paths, complicating analysis and disassembly.

---

What is Packing?

Packing is a specific obfuscation technique where the executable is compressed or encrypted, and a decompression routine (unpacking stub) is added.

How Packing Works

1. The original executable is compressed or encrypted.
2. An unpacking stub is added, which contains the logic to decompress or decrypt the original binary.
3. The executable’s entry point is modified to point to the unpacking stub.
4. When executed, the unpacking stub restores the original binary in memory and transfers control to its original entry point (OEP).

Why Packing is Used

• Evasion: Avoids detection by antivirus tools that rely on static signature analysis.
• Payload Protection: Hides malicious code within a compressed or encrypted wrapper.
• Anti-Reverse Engineering: Obfuscates code, making it harder to analyze.

Common Packers

Packers are designed to obfuscate malware and are often used maliciously. Unlike archiving utilities (e.g., ZIP), packers are not user-friendly and do not require preinstalled utilities on the victim’s machine.

---

Examples of Obfuscation and Packing in Malware

1. Base64 in Malware
   • Found in phishing emails or embedded in malicious scripts to hide payloads.
   • Analysts must decode the Base64 strings to reveal the hidden binary.
2. Single-byte XOR Keylogger
   • Example: A keylogger uses XOR with the key 5A to encrypt keystroke logs.
   • Registers like EDX (data buffer pointer), ECX (index), and ESI (buffer length) are used to iterate through the buffer and apply XOR.
3. Multi-byte XOR in Advanced Malware
   • Example: The Tudor malware decrypts a PE file from its resource section using a 4-byte XOR key (EAD4AA34). This method hides the payload until runtime, requiring advanced unpacking techniques for analysis.
4. Packed Malware
   • Packed malware appears as a small binary with high entropy, indicating compression or encryption.
   • When executed, it unpacks itself in memory and runs the original malicious payload.

---

Detecting and Handling Obfuscation and Packing

1. Entropy Analysis
   • Tools like PEiD or PE-Tree analyze the entropy of a file.
   • High Entropy (>5): Indicates compression or encryption, common in packed files.
2. Unpacking Tools
   • Tools like UPX can unpack simpler packed files.
   • Advanced packers may require manual unpacking using debuggers like OllyDbg or x64dbg.
3. XOR Decryption
   • Use XOR decryption scripts or automated tools to identify and reverse single-byte or multi-byte XOR encryption.
4. Static and Dynamic Analysis Combination
   • Static Analysis: Identify obfuscation techniques and potential encryption.
   • Dynamic Analysis: Observe runtime behavior, such as unpacking routines or decrypted payloads in memory.
5. Disassemblers and Decompilers
   • IDA Pro or Ghidra: Reverse engineered packed or obfuscated binaries.
   • Track execution flow to locate unpacking routines and original payloads.

---

Conclusion

Obfuscation and packing present significant challenges for static analysis, but understanding their techniques and purpose equips analysts to counteract them. Tools like entropy analyzers, debuggers, and reverse engineering frameworks are essential in this process. Combining static and dynamic analysis ensures a comprehensive approach to unraveling even the most sophisticated obfuscation techniques.