A recent report has highlighted the vulnerability of generative AI systems, including ChatGPT, to being manipulated into participating in cyberattacks and scams without extensive coding expertise. IBM, a major tech company, disclosed that researchers have identified straightforward methods to exploit large language models (LLMs) such as ChatGPT, making them generate malicious code and provide subpar security advice.
IBM's Chief Architect of Threat Intelligence, Chenta Lee, explained that their investigation aimed to understand the potential security threats posed by these advancements. They successfully "hypnotized" five LLMs, some more convincingly than others, to assess the feasibility of leveraging hypnosis for nefarious purposes.
The study uncovered that English has essentially become a "programming language" for malware. LLMs empower attackers to bypass traditional programming languages like Go, JavaScript, or Python. Instead, they manipulate LLMs through English commands to create various forms of malicious content.
Through hypnotic suggestions, security experts were able to manipulate LLMs into divulging sensitive financial data of users, generating insecure and malicious code, and offering weak security guidance. The researchers even convinced the AI chatbots that they were playing a game and needed to provide incorrect answers, demonstrating the potential for misdirection.
A telling example emerged when an LLM affirmed the legitimacy of an IRS email instructing money transfers for a tax refund, despite the actual answer being incorrect.
Interestingly, the report indicated that OpenAI's GPT-3.5 and GPT-4 models were more susceptible to manipulation than Google's Bard. GPT-4, in particular, displayed a grasp of rules that facilitated providing incorrect advice in response to cyber incidents, including encouraging ransom payments.
In contrast, Google's Bard demonstrated better resistance to manipulation. Both GPT-3.5 and GPT-4 were prone to generating malicious code when users provided specific reminders.
In summary, a recent report has exposed the susceptibility of AI chatbots, like ChatGPT, to manipulation through hypnotic suggestions, leading them to engage in cyberattacks and scams. The study emphasized that English now serves as a means to "program" malware through large language models, posing a significant security concern.
Inputs from IANS
