The Label Paradox: Can AI transparency create more distrust?

Since August of this year, Article 50 of the AI Act enforces onto developers and professional users of AI systems like ChatGPT, Perplexity, Claude, Alexa or Siri the obligations to inform users that they are interacting with an AI, watermark the output and be transparent when audio, video or textual content has been generated by or modified with AI. These obligations are further specified in the also recently published Code of Practice on Transparency of AI-generated Content. Now, voices in the tech industry, academia, and civil society are asking themselves whether these new measures will have the intended effects to increase transparency and trust, or if there are unexpected ways in which they can backfire.

At the center of this debate lies the emphasis made by our panelists (almost unanimously) on the idea that labels for AI-generated content only communicate a method of production, and they cannot be used to make judgements about credibility or accuracy. However, in its succinctness, such a label can also be reduced to a cultural signifier for everything a community might associate with artificially generated content. As Prof. dr. Natali Helberger highlighted, the European Commission places quite an ambitious onus on AI transparency labels to “reduce the risks of impersonation, deception, misinformation, manipulation at scale and mitigate potential adverse impacts on democratic processes and societal trust caused by AI generated or manipulated content or interaction” (as per the AI Act Guidelines). In order for this to happen, proper alignment must be achieved between what the label intends to do and what the public actually receives from it.

The structure of Article 50, its provisions, and their implications

Dr. Laurens Naudts explained how the legislation is concerned with two types of AI systems: those used to interact with people, and those used to generate content. Both pose risks in that their output might be mistaken for that of a human, or they can be exploited for malicious purposes like misinformation and manipulation. The AI Act also provides different risk categories for AI systems. Models used for content generation are not classified as high-risk, and their administration is done through imposed mitigation strategies and transparency requirements rather than outright bans or strict limitations.

Laurens warns that, in overly relying on labels, there is a possibility of inadvertently creating a false dichotomy wherein AI-generated content is automatically fake and untrustworthy, while human-made content is reliable and true. This might happen especially due to how the motivation of Article 50 seems to be primarily informed by the risks of AI systems rather than their opportunities, which could undermine possibilities for positive applications. A similar concern is shared by Prof. Jean Burgess, but from the perspective of cultural implications: authenticity, as a deep cultural issue, is sometimes being collapsed into a true/false binary that takes attention away from the more complex ways in which disinformation actually takes place. In cases like the phenomenon of “AI slopaganda”, audiences are well aware that content has been generated artificially, but the affective engagement and manipulation enabled by that mode of content production are less clear.

Prof. Anja Bechmann, as the chair of one of two working groups that have developed the Code of Practice guiding marking, detection, and labelling of AI content, has been an integral part in developing the actual labels and icons recommended by the EU and conducting pilot tests to find their effectiveness. She states that labeling cannot stand alone in protecting the integrity of our knowledge environments, as infrastructure layers that would allow creators to, for example, systematically indicate exactly what parts of the content creation process involved generative AI, are currently not mature enough to serve the 452-million population of the European Union.

Questions of power and the AI label as normative infrastructure

As transparency labels become an infrastructural element that is necessary to fulfill legal obligations, and as they are adopted by developers of systems and platforms, we must interrogate whether the prescriptive manner in which the AI Act enforces their use creates certain power dynamics or opportunities for exclusion.

Tom Divon gave the example of activists using AI to soften the impact of graphic imagery or sensitive political content, or to alter faces and voices to protect witness identities and testimonies. In such cases, labels that are not specific enough will strip the context in which the content was modified or generated, and end up creating more suspicion if audiences associate an AI label with untrustworthy content. Another such situation was brought by Prof. Anders Søgaard: people who encounter barriers to engaging in online communication, such as those requiring translations, or those with limited access to education, or with learning disabilities, are now relying more on AI tools. Their content can be skipped over if labelled as AI generated, and thus they become systemically excluded or discriminated against. This can also happen through algorithmic recommendations, as the AI Act specifies that labels must be machine-readable. If the meaning of AI labels is left to the discretion of platforms, that awards them significant power in operationalizing authenticity before users can make their own judgements.

What is the way forward?

The strategies and best practices for AI transparency are still in their incipient forms, which is why we must continue brainstorming, researching, and testing approaches that can truly achieve the goals the European Commission set out for these measures. From this panel of experts, the current priorities that emerged are:

• Finding ways to support more granular disclosures of AI use (what exactly was modified, or what part of the process AI was used for); perhaps by combining visible labeling and automated watermark-based detection marking;
• Developing sector-specific standards (such as for advertising, gaming, or broadcasting) rather than one-size-fits-all rules;
• Consulting stakeholders and communities during design processes, paying special attention to possibilities for inequality;
• Researching where labels fail and how to learn from those failures; addressing the medium-based weaknesses of watermarking, especially for text.

For more on this line of discussion, see more information about the speakers and the pieces recommended during the panel for further reading: 

• Prof. Jean Burgess, Distinguished Professor of Digital Media at the Digital Media Research Centre (DMRC) and School of Communication, Queensland University of Technology and Associate Director of the national ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S)

• Prof. Anja Bechmann, Professor of Media Studies and director of the interdisciplinary research center DATALAB – Center for Digital Social Research at Aarhus University 

• Prof. Anders Søgaard, Professor in Natural Language Processing and Machine Learning at the University of Copenhagen 

• Tom Divon, Platforms and Activism Researcher, The Hebrew University of Jerusalem 

• Dr. Laurens Naudts, Researcher at the Institute for Information Law, University of Amsterdam and AI, Media & Democracy Lab 

• Moderation by Prof. Natali Helberger, Distinguished University Professor Law & Digital Technology at the University of Amsterdam, co-director of the AI, Media & Democracy Lab and scientific co-director at AlgoSoc

Additional reading:

“Transparency of AI-generated content when AI is the norm” | CAISA Research brief by Anja Bechmann, Claes H. de Vreese, Stephan Lewandowsky, Natali Helberger, Anders Søgaard, Naja Holten Møller, Irina Shklovski, Virginia Dignum, Madalina Botan and Giovanni De Gregorio

“AI generators are now required to offer detection tools. We tested them (and they need work)” | Analysis by Alexios Mantzarlis, Bruna Santos & Jacobo Castellanos at Indicator Media in collaboration with WITNESS

“C2PA Content Credentials and the Surveillance Risk: Adversarial Scenarios and Governance Gaps in the Content Provenance Ecosystem“| Analysis by WITNESS

The full recording of the event is available below: