As the world is increasingly flooded with deepfakes, the European Union is taking action to help people distinguish what is real and what is not
It recently started enforcing transparency obligations for tech companies which require them to clearly label any photos, videos and audio which are created using their artificial intelligence (AI) systems and designed to look authentic
This is a step up from the approach of other countries, such as Australia, which only recommend AI developers mark AI-generated content to help people identify it. This matters because research suggests that when confronted with sophisticated AI-generated content, our ability to distinguish fake from real can be little better than a coin-toss
But will the EU’s approach actually work? And could requiring the labelling of AI-generated content have unintended consequences? Could it potentially make us more vulnerable to the very content designed to deceive us?
A set of icons
Article 50 of the EU AI Act took effect from August 2. It introduces various transparency obligations for companies who provide or otherwise deploy AI systems or content
Among these obligations, AI-generated or manipulated images, audio, or videos which resemble existing persons, objects, places, entities or events must now be disclosed as such
AI-generated or manipulated text on matters of public interest, which is not subject to human review or editorial control, must also be disclosed
The EU has developed a set of icons which authentic-looking AI content must be labelled with
For audio deepfakes, a short audible disclaimer should be placed at the commencement of the content
Icons should be clearly visible when users first encounter the content. They should also be placed without obstructing overlays, and embedded into the content so it remains visible when shared or downloaded. Using these EU icons is optional, however disclosure of such content in accordance with the act remains mandatory for companies
Interestingly, for evidently artistic, creative, satirical, or fictional AI-generated content, disclosure is only required in a way that informs audiences without disrupting the presentation or enjoyment of the work
The EU’s new transparency obligations also require AI companies to include invisible, machine-readable watermarks on any content that is generated or even processed with their systems. To comply with the act, leading AI lab Anthropic said it will ensure all Claude chatbot models launched in the EU after August 2 will support machine-readable marking. The company is also working to add watermarks to earlier Claude models
The act applies to companies even outside the EU who create AI-generated content visible to those within the EU. Non-compliant companies may face fines up to €15 million (A$24.5 million), or up to 3% of total worldwide annual turnover
However, the act does not apply to individuals when they are creating content in a personal or non-professional capacity. Therefore, the act has limitations in enforcing the disclosure of some AI-generated content
European Union
Labels may create unintended consequences
Even when labels are applied as intended, we need to understand how people will respond
AI labels are designed to help us identify synthetic content. But they could have an unintended “boomerang effect”. This is where an intervention produces the opposite of what was intended
If we become accustomed to seeing AI-generated content with a label, we may start assuming that content without a label must be real. This is problematic because malicious actors who deliberately create deceptive content are unlikely to disclose AI use
Government efforts to protect citizens through disclosure might in turn be encouraging less critical evaluation of wider digital content. Reliance on labels may therefore make citizens increasingly susceptible to undisclosed and malicious AI use
Our research has shown that disclosure is more complicated than simply adding an “AI-generated” label
The presence and timing of AI disclosure can influence how people respond to the content. This means the effectiveness of disclosure depends not only on the presence of a label, but also on how people interpret and use it
Importantly, labels are only one of many cues people use when deciding whether to trust online content. Our research has also shown that when content begins to go viral or is accompanied by comments that reinforce its message, people can become more susceptible to misinformation
These social cues may compete with, or even override, information indicating whether content is authentic or AI-generated
Literacy, not just labels
People cannot solely rely on labelling to help them identify AI-generated content. They also need to be alert to AI cues such as inconsistencies in facial movements, expressions, speech or voice
However, as AI becomes more sophisticated, these cues are becoming harder and harder to spot. So consideration must also be given to who created or shared the content, whether the can be verified elsewhere
Cultivating AI literacy such as this is vital
AI labels can help. But they must not be a replacement for the critical evaluation of online content. As the boundary between synthetic and authentic content becomes increasingly blurred, governments and tech companies must do more to support AI literacy efforts to help people make informed judgements about what they see online
