Google Reports Over 100,000 Adversarial Prompts Targeted Gemini AI

Google has revealed that it was the target of a large-scale attempt to duplicate its Gemini AI chatbot, according to a recent threat intelligence report published by the company. The tech giant detected "commercially motivated" actors issuing more than 100,000 prompts in an effort to extract proprietary knowledge from Gemini—a major artificial intelligence system designed to generate text-based responses to user queries.

The attempted copying, called "model extraction" by Google, involved attackers systematically soliciting information from Gemini, often in a variety of non-English languages. The extracted responses may have been used to train rival models or develop cheaper, clone-like alternatives based on Gemini’s outputs.

Model extraction attacks have become a growing concern in the generative AI sector. Such attacks involve sending large numbers of inputs, or prompts, to a target AI system—like a chatbot powered by a large language model (LLM)—with the goal of replicating its behavior and knowledge. LLMs, such as Gemini, use advanced neural network architectures known as transformers to process vast amounts of data and generate human-like responses.

In its assessment, Google frames the incident as both a direct threat to its intellectual property and an emerging risk for the industry. The company refers to the activity as "intellectual property theft." However, this position is not without controversy: Google’s own AI models have previously drawn on data scraped from the public web, often without explicit permission from content creators. This practice has sparked debate over the boundaries of permissible data use for training AI systems.

The report also acknowledged scrutiny Google has faced over its rival chatbot development efforts. In 2023, external reports alleged that Google’s Bard chatbot team leveraged outputs from OpenAI’s ChatGPT—obtained via ShareGPT, a public site where users share AI-generated conversations—for training purposes. Although Google denied violating OpenAI’s terms of service, it reportedly ceased the practice following internal warnings.

These incidents reflect the escalating competition—and occasional controversy—among leading AI developers as generative models become increasingly integral to business and technology. Security risks associated with model extraction extend to the broader industry, prompting calls for more robust protection of proprietary AI systems.

As the value and sophistication of generative AI continue to rise, tech companies like Google are likely to encounter further attempts at model duplication and data extraction. The sector is facing mounting pressure to address questions of intellectual property, data provenance, and responsible AI development.

For further details, see the original report at arstechnica.com.