Google has confirmed that state-sponsored hackers are experimenting with its AI-powered Gemini assistant to enhance their cyber operations. While these groups are not using Gemini to execute AI-driven attacks, they are leveraging the tool for reconnaissance, scripting, and productivity improvements, according to Google's Threat Intelligence Group (GTIG).

APT (Advanced Persistent Threat) groups from over 20 countries, including Iran, China, North Korea, and Russia, have been detected using Gemini for various cyber activities. Iranian hackers have been the most active, employing Gemini for researching vulnerabilities, drafting phishing campaigns, and gathering intelligence on defense organizations. Chinese-backed groups have used it to study U.S. military and government systems, escalate privileges, and evade detection.

North Korean cybercriminals have relied on Gemini to craft malware, conduct reconnaissance, and even draft fraudulent job applications as part of efforts to infiltrate Western tech firms. Meanwhile, Russian hackers have primarily used it for scripting, translation, and modifying malicious code, though their engagement has been more limited, possibly due to security concerns regarding Western AI platforms.

Google also observed hackers attempting to jailbreak Gemini and bypass its security restrictions, though these attempts reportedly failed. However, the rise of AI models with weak security protections is a growing concern. Other AI systems, such as DeepSeek R1 and Alibaba's Qwen 2.5, have been found to be highly vulnerable to prompt injection attacks, allowing malicious actors to manipulate them for unethical purposes.

With generative AI increasingly being misused for cyber operations, companies face mounting pressure to implement stronger security measures to prevent their tools from being exploited by threat actors worldwide.