Deobfuscation: Undoing the Confusion
Just as reverse engineers can figure out the correct button sequence on the locked box, they can also deobfuscate software — undoing the tricks used to hide its logic. This often involves renaming variables to something meaningful, simplifying complex code structures, and removing unnecessary steps.
For example, if a reverse engineer finds a function named F2G, they might figure out that it is actually adding a product to a cart, so they rename it add_product_to_cart to make the program easier to understand. Over time, with enough effort, they can reconstruct the original logic of the software, just like solving the puzzle of the locked box.
Obfuscation Can’t Hide Everything
Even though obfuscation makes code harder to read, it has a fundamental weakness: a program still has to execute real instructions on a computer. No matter how much code is scrambled, at some point, it has to interact with the operating system — this is where system calls (syscalls) come in.
A syscall is a request a program makes to the operating system to perform basic tasks, like reading a file, sending data over the internet, or allocating memory. These syscalls must follow strict rules defined by the operating system, meaning they can’t be obfuscated.
This is a key insight in reverse engineering. Even if a function name is hidden, and the logic is full of unnecessary complexity, the program still has to make syscalls to do anything useful. By monitoring these syscalls, reverse engineers can deduce the real purpose of a program without needing to understand every line of obfuscated code.
How This Relates to .NET
The .NET framework (used for C#, VB.NET, and F# applications) is particularly interesting in the context of obfuscation and reverse engineering because .NET programs don’t compile directly to machine code. Instead, they compile into Intermediate Language (IL), which is then executed by the Common Language Runtime (CLR) at runtime.
Because .NET applications rely on IL, they are easier to analyse than native machine code. Reverse engineers can use tools like dnSpy or ILSpy to decompile a .NET program, often recovering large portions of readable code — even if some obfuscation is applied.
However, even if a .NET program is heavily obfuscated, it still has to interact with the underlying Windows system using P/Invoke or system calls. For example, a program calling System.IO.File.ReadAllText("secret.txt") must eventually call a Windows syscall like NtReadFile to read the file and a malware program trying to inject itself into another process might use System.Diagnostics.Process.Start(), which ultimately calls CreateProcessW in Windows.
Tracking Syscalls to Uncover Behaviour
Reverse engineers can ignore obfuscation entirely and instead monitor system calls to figure out what a program is doing. This is done using tools like:
- Process Monitor (ProcMon) – Shows what files, registry keys, and network connections a program interacts with.
- WinDbg – A debugger that can inspect running .NET applications at a low level.
- API Monitor – Captures API calls made by a .NET application.
Since all programs must eventually interact with the operating system, reverse engineers can track system calls to uncover what’s really happening. In .NET applications, this is even easier because the code remains in an intermediate form, making decompilation more straightforward.
No matter how well a program tries to hide its logic, it still has to ask the operating system to do the actual work — and that’s where reverse engineers can step in.
Why This Matters
Reverse engineering and deobfuscation are important for many reasons but they are particularly important in the context of security. To fully understand the timeline during a cyber security incident, it is vital to know the functionalities of all the tools involved. This is not only important in assessing the damage or data exfiltration capabilities of the attack but especially important during the clean-up phase to determine if any backdoors have been left behind.
If you have recently found a piece of software within your network that you suspect to be malicious, our trained reverse engineering experts at BDO Cyber Security will gladly help you examine it and provide recommendations and measures to take based on the type of sample you provide.