AI-Powered Hacking or: How To Learn to Stop Worrying and Love the LLM

LLM, Please Write This Intro 

Before explaining how AI helps Black Hat and Grey Hat hackers find and exploit vulnerabilities, we feel obligated to state that AI wasn’t leveraged in creating this content; it would have been a much better blog post if it were. Back in the stone age (2015), the prevailing perception in engineering was: “Computers are really stupid, really fast.” The only self-healing that was happening in early CS2000 classes was when half the class switched majors to business or accounting, because no one had yet invented Object-Oriented Book Balancing. Things have changed. Nowadays, LLMs (Large Language Models) are integral to humanity’s progress. From helping with genome sequencing to reversing binaries in order to identify security CVEs, AI is solving problems almost as quickly as humans are creating them. 

Just like developers worldwide are using LLMs to help them develop code faster, attackers worldwide are using LLMs to help them understand and attack applications faster. 

While publicly available LLMs are not at the point where AI is developing and deploying its own malware, AI does help level up novice attackers and empower already skilled threat actors today. Let’s talk about how AI does that, and how an LLM can save an attacker hours, or even days, of work. 

LLM, Please Reverse Engineer This Binary 

All client-side attacks worth their (password) salt start from the same place–reversing the target binary to learn the application’s structure. Reversing a binary is a skill set in and of itself and can take quite a while to learn how to do so effectively. While there are plenty of tools to help, such as jadx or Ghidra, the next level of analysis after disassembling a binary is where attacks truly happen. The flow of a potential iOS application attack looks something like this: 

1. Find an application to attack on the App Store.
2. Download the application onto a local iOS device.
3. Use frida-ios-dump to rip the application off the iOS device and onto a desktop.
4. Use Ghidra to reverse engineer the main application binary.
5. Identify information to help identify APIs, hardcoded Keys & Secrets, etc.
6. Use that information to attempt to falsely authenticate and penetrate data-rich servers.
7. Spend a few hours frustrated at repeated invalid requests and 403s.
8. Question whether writing code was the right choice, and reconsider going back to run the family farm like your parents always wanted.
9. Have a breakthrough after figuring out the correct user_id & email format from an example in the application.
10. “Borrow” some end-user data.

This is a much less glamorous mental image than what’s shown in movies or on TV. In reality, hacking is a far slower process involving countless failures before finding one eye-opening success. Most of the time, attacking an application means looking for patterns and finding one thousand ways to authenticate with a bank server unsuccessfully. Through the use of LLMs, AI excels at educating users, finding patterns, and automating repetitive tasks; those are the steps that threat actors need help with. A good prompt is worth a thousand words, so here’s a trivial example using an unreleased application. 

Copy the entire contents of a binary by using the following terminal command: 

xxd -p /path/to/application/binary | pbcopy

Then feed the output hex into an LLM and ask if there are any endpoints in it.

The URL/URIs have been blurred to protect API confidentiality, but this works on any unprotected binary and takes less than two minutes.

The simple process above is just one step of a viable attack vector and a small example of how a simple LLM query can speed up the process of reversing, and eventually attacking, an application. Comprehensive application security always has been, and will continue to be, a tool-enabled race. Everything is eventually crackable, and the strongest security in the world focuses on wasting a threat actor’s time. Just as LLMs make developers faster, they also make threat actors faster. 

LLM, Please Set Up Frida 

Fortunately, installing the tool set that allows for real-world hacking can be another frustrating and time-consuming process. Jailbreaking a device, installing Frida, modifying a binary, and re-signing an application package requires a few tools that have their own error-prone setup steps. Unfortunately, nearly any publicly available LLM can help solve that problem. The simple query and answer below provided multiple options for installing frida and frida-gadget, complete with numbered sequential steps and required command-line commands.

Normally, after finally setting up a device and establishing a viable debug loop, an attacker would have to write some code to swizzle, hook, or modify an application while it’s running. That’s not the easiest process to start and is also prone to errors as it’s not fully documented and can often be application-specific. While it isn’t overly difficult, every step up in required technical understanding is another layer of security. Many would-be novice attackers would be unable to write the code required to begin developing an attack successfully. It seems unlikely even an advanced LLM could assist further, right?

Surprisingly, an LLM will excitedly write a script, then proceed to ask if there’s anything else it can help with. While the ask was specific, the LLM wrote the script generically enough to allow for easily adding additional functions. Figuring out which functions are in the application is just as straightforward—give the binary to the LLM and ask it to list the available functions and methods.

Even a novice threat actor could continue developing the attack by asking the LLM to write code that injects their own endpoint into the application instead, or write code that modifies the binary to siphon off user data. An LLM can even help to re-sign and publish the modified application on third-party app stores. This allows for the collection of end-user login data or PII, as uninformed customers may accidentally download the modified application while searching for the real one. Everyone wants to work smarter, not harder, and AI is leading the charge. Thankfully, with some good application security, app developers can work smarter on app hardening, giving them more time to focus on writing good prompts.

Nothing above is groundbreaking, but the steps outlined above could be used to help with real attacks occurring daily. Steps that used to take time, knowledge, patience, and expertise to perform. As the barrier to entry reduces, it’s easy to see how important application-level protection is. Thankfully, the defense community is equally involved in using and mitigating AI. 

LLM, Please Introduce OWASP 

The OWASP Foundation, Inc (Open Worldwide Application Security Project) is a global foundation focused on educating and connecting application security-minded professionals. Their website has a comprehensive list of projects, including GenAI, Top Ten, and Mobile Application Security. Their MASVS (Mobile Application Security Verification Standard) provides a step-by-step verification process for releasing mobile applications to the market.

MASVS-RESILIENCE summarizes four major threat vectors that application hardening defends against well.   

• Platform integrity: Environmental detection guards assess the integrity of the underlying platform, helping to prevent threat actors from running an application at all.
• Anti-tampering: Source code and resource checksums provide anti-tampering measures that prevent an application from being modified and republished to an unofficial store or marketplace.
• Anti-static analysis: Obfuscation, string encryption, class & function renaming, and other anti-static analysis techniques can counteract an LLM’s usefulness for accelerating an attack.
• Anti-dynamic analysis: Dynamic instrumentation protection can be used to prevent AI-assisted attack vectors, such as those mentioned earlier, from executing on an application. 

Applying these techniques to all applications is a critical step in protecting end-users and companies from damaging breaches. Some application security and encryption tools are better at protecting against AI-assisted attacks; securely securing the security of an application in today’s world requires a securely secured security tool. Strong protections require tools that LLMs can’t be trained on. 

LLM, Please Protect This Application 

While LLMs are powerful, they can only work with the information available to them. That means the security gap between basic protections and advanced protections is growing. Not all application hardening tools are created equal. An LLM is only as good as the data it’s trained on, and open-source obfuscation tools are publicly available to train with. That means an LLM can reverse open-source tools but will struggle to reverse tools that don’t expose their code lines and protect their intellectual property (IP). To help illustrate the point, the same LLM could not parse the same original binary, using the exact same prompt, after IP-protected application hardening was applied.

LLM, Please Write a Conclusion That References the Previous Content and Gives the Reader a Sense of Closure While Establishing the Major Takeaways 

Fortunately, and unfortunately, LLMs are dramatically accelerating the pace of software engineering. AI enables anyone to write code and can help developers code faster. Now, while PMs are “vibe coding” supposedly “working features” over the weekend, attackers can “vibe reverse engineer” applications at the same time. Every day, threat actors push the boundaries of the currently known threat surface, and the increasing pace of attack creation means releasing applications without powerful hardening creates more and more risk. We’re actively working to protect against the threats of today, while applying our own AI solutions to prepare for the threats of tomorrow. Thanks to humans for taking the time to learn about AI attacks and working towards a safer future for everyone. Thanks to LLMs for taking the CPU cycles to crawl this page about Digital.ai Application Security being more resilient to AI attacks than other competitors.