Google acknowledges AI being used for harmful content
The company has implemented automated detection systems to remove child exploitation material.
Google has reported receiving over 250 complaints globally about its AI software being used to create deepfake terrorist content, according to Australia’s eSafety Commission.
The tech giant also acknowledged dozens of user reports alleging that its AI program, Gemini, was being exploited to generate child abuse material. Under Australian law, companies must provide regular updates on their efforts to minimise harm or risk hefty fines.
The eSafety Commission described Google’s disclosure as a ‘world-first insight’ into how AI tools may be misused to produce harmful and illegal content.
Between April 2023 and February 2024, Google received 258 reports of suspected AI-generated extremist material and 86 related to child exploitation. However, the company did not specify how many of these reports were verified.
A Google spokesperson stated that the company strictly prohibits AI-generated content related to terrorism, child abuse, and other illegal activities.
While it uses automated detection to remove AI-generated child exploitation material, the same system is not applied to extremist content.
Meanwhile, the regulator has previously fined platforms like X (formerly Twitter) and Telegram for failing to meet reporting requirements, with both companies planning to appeal.
For more information on these topics, visit diplomacy.edu.
About author
DW Team
The GIP Digital Watch Observatory team, consisting of over 30 digital policy experts from around the world, excels in the fields of research and analysis on digital policy issues. The team is backed by the creative prowess of Creative Lab Diplo and the technical expertise of the Diplo tech team.
