Here’s a fun story I like to tell to people about one of the more intimidating challenge categories in the National Cyber League (NCL) Games: Reverse Engineering and Binary Exploitation.

I’ve had plenty of people tell me that so-and-so topic (usually Web App Exploitation or Binary Exploitation) in the NCL Games is too challenging or that they’re not smart enough to do those challenges. Now I will admit, unlike some other categories in the games, to be truly effective at these categories requires training, knowledge and many hours of practice. There are also excellent resources available online to help you build your skills in these categories, but even in the heat of the competition with little time to learn assembly, don’t count yourself out so quickly.

The first time I ever solved a Binary Exploitation challenge that required reverse engineering, I had no idea what I was doing at all! At the time, I had programming knowledge, but very little experience in C and C++, the two most useful languages to know for reverse engineering binaries. Even then, I knew I wasn’t ever going to learn it by just giving up, I had to at least hit my head against it for a few hours before it would be considered socially acceptable to cry.

The challenge itself at the time involved reverse engineering an encryption program to determine how to decrypt a file containing the flag. I had no knowledge or experience with programs like Ghidra that could decompile binary and give me C code to look at. All I knew about at the time was the basics of programming, and that it was technically possible to disassemble the binary to inspect the processor instructions. I used the radare2 GUI wrapper known as Cutter to accomplish this.

Of course, the tools don’t just do everything for you. I opened the binary in cutter to see a screen that basically looked like hundreds of lines of this:

I’m not gonna lie, I have no idea what any of this means. I still don’t, really. But no matter what, don’t let the technology intimidate you. Any problem of any complexity can always be broken down into its essential elements.

I was, fortunately, able to identify the function that would need reverse engineering extremely easily by simply looking at function names.

Once I’m in that function my task doesn’t exactly become any easier, I still have no idea where to start or what any of these instructions mean. I DID know that they corresponded to some C code that had been written and compiled into these processor instructions, so I figured that the easiest way to see what the code was doing was to slowly convert the processor instructions back into some C code that accomplishes the same thing.

Let me just preface this by reiterating that you should never, EVER waste your time like this. This is the desperate work of a desperate man. There exist sophisticated tools to do exactly what I’m about to do, namely Ghidra. Even Cutter has the Ghidra decompiler built straight into it to automate this process. Of course, at the time I didn’t know that.

To figure out this function, I started by copying over the assembly instructions into a text file. I then proceeded to convert each line of assembly into a corresponding line of C code that would accomplish the same thing. Remember, in this line of work especially, Google is your best friend. Since I had no idea what any of the assembly instructions did, I pretty much had to do a couple Google searches for each and every line of assembly. As I converted the lines of assembly into lines of C code, multiple lines would eventually collapsed into single, compound lines, and the code would more closely resemble that of conventional C code.

It look over an hour of grueling, mind-numbing work to do this to the entire function. But at the end, I was able to determine its function from the C code that I had produced and derive a solution. I didn’t go in with any experience, but was simply willing to spend the time learning on the job. Nobody on my NCL team had the experience really, but I was the only one who wasn’t ready to give up as soon as the problem was revealed to be reverse engineering a binary.

Ever since then, I never shy away from challenges that I don’t have any knowledge about. I don’t really finish CTFs in first place or earn the most points. I go in for the hardest challenge, the problems that nobody else has solved, and hopefully learn something new even if the attempt to get the flag doesn’t succeed.

The most important thing to remember is that if you give up before you’ve even put forward your best effort, it wasn’t really the difficulty of the problem that defeated you. Please, give it your all in the NCL Games and don’t let anybody tell you that you can’t succeed in them, especially yourself.

Aaron