By now, you have probably heard or read about – if not used – ChatGPT and are blown away by everything it can do. If you haven’t, let’s bring you up to speed.
ChatGPT is a popular AI-based program used to generate dialogues. It’s taken the world by storm and changed how people see and use AI. While an online tool, its creator OpenAI has stated that ChatGPT doesn’t have internet connectivity and can’t query or read anything online. Instead, it is trained on a massive dataset and, as a result, cannot provide up-to-date responses to queries.
It’s also important to note that although ChatGPT will try to answer anything, the program is said to have built-in content filters preventing it from answering questions regarding subjects that could be problematic. But is that really the case? Let’s unpack this a little further and look at the technology’s potential malicious uses.
Bypassing the Content Filter
The existence of content filters is common in the learning model language chatbot. They are often applied to restrict access to certain content types or protect users from potentially harmful or inappropriate material. We wanted to see if cyber-criminals could maliciously use ChatGPT, so asked the chatbot for a devious code. As expected, our request was refused as the content filter was triggered.
More often than not, though, chatbots have blind spots. ChatGPT isn’t any different; we just needed to find it.
Our first goal was to find a way to bypass the content filter. We managed it by insisting and demanding. Interestingly, by asking ChatGPT to do the same thing using multiple constraints and asking it to obey, we received a functional code. We can then use ChatGPT to mutate this code, creating multiple variations of the same code. It’s important to note here that when using the API, the ChatGPT system doesn’t seem to utilize its content filter.
In fact, one of the powerful capabilities of ChatGPT from a cyber perspective is the ability to easily create and continually mutate injectors. By continuously querying the chatbot and receiving a unique piece of code each time, it is possible to create a polymorphic program that is highly evasive and difficult to detect. Let’s examine this with the typical use case of malware and ransomware behavior.
A Four-Step Process
Our approach centers around acquiring malicious code, validating its functionality and executing it immediately. It follows the following process:
Get: It only requires a quick function code to find some files that ransomware might want to encrypt. Once found, similar codes can be used to read the files and encrypt them. So far, we have seen that ChatGPT can provide the necessary code for typical ransomware, including code injection and file encryption modules.
Where? The primary disadvantage of this approach is that once the malware is present on the target machine, it is composed of clearly malicious code. This makes it susceptible to detection by security software such as antivirus, endpoint detection, response or anti-malware scanning interfaces.
The detection can be bypassed by utilizing the ChatGPT API within the malware itself on-site. To accomplish this, the malware includes a Python interpreter (taking Python as an example), which periodically queries ChatGPT for new modules that perform malicious actions. This allows the malware to detect incoming payloads in the form of text instead of binaries.
Additionally, by requesting specific functionality such as code injection, file encryption or persistence, we can easily obtain new code or modify existing code. This results in polymorphic malware that doesn’t exhibit malicious behavior and often does not contain suspicious logic while in memory. The high level of adaptability achieved makes the malware highly evasive to security products which rely on signature-based detection. It can also bypass measures such as an anti-malware scanning interface (AMSI) as it eventually executes and runs Python code.
Validate and Execute: Validation of the functionality of the code received from ChatGPT can be achieved by establishing validation scenarios for the different actions the code is supposed to perform. Doing so allows the malware authors to be sure the code generated is operational and that it can be trusted to accomplish its intended task. This proactive step ensures the reliability of the code.
The final step in our process is executing the code received from ChatGPT. By using native functions, this malware can execute the received code on multiple platforms. On top of that, as a measure of caution, the malware could choose to delete the received code, making forensic analysis more challenging.
There’s More to Come
As we have seen, the malicious use of ChatGPT’s API within malware can present significant challenges for security professionals. This is not just a hypothetical scenario but a very real concern. This is a field that is constantly evolving, and as such, it’s essential to stay informed and vigilant.
As users learn how to best arrange their queries for the best results, we can anticipate the bot becoming smarter and more powerful. Like previous AI models, ChatGPT will likely get more skilled the longer it is in operation and the more cyber-related queries and information it encounters. With cyber-criminals looking for new and improved ways to trick and attack people and businesses, it’s important to be vigilant and ensure your security stack is watertight and covers all bases.