Deepfake

In the EU, it is legally relevant above all because of Article 50 of the AI Act: anyone who uses an AI system to generate or manipulate a deepfake, other than in the course of a personal non-professional activity, must disclose that the content has been artificially generated or manipulated.

How deepfakes are created technically

The first generation of deepfakes was based on generative adversarial networks (GANs). Two neural networks compete against each other: a generator produces candidates, a discriminator tries to distinguish real images from generated ones. Both improve each other until the discriminator can no longer reliably detect the fake. This method shaped the classic face-swap videos of 2017 to 2021.

Today, diffusion models dominate. They learn to reconstruct a sharp image step by step from a noisy one and can steer this process via a text prompt. This lowers the entry barrier drastically: a usable result no longer requires hours of training material; often a few minutes of video footage or a few seconds of voice recording are enough. At the same time, speech synthesis models have advanced so far that they can adopt a person’s voice timbre, speaking pace and intonation from short samples.

The four variants found in practice

• Face swap. A person’s face is inserted into an existing video. The classic, and technically a commodity by now.
• Lip sync and reenactment. The original face remains; only mouth movements and facial expressions are adapted to a new audio track. Particularly hard to detect, because the background, lighting and posture are real.
• Voice cloning. A synthetic voice speaks any text in the timbre of a real person. The most dangerous variant in a corporate environment, because it works over the phone and needs no image quality.
• Fully synthetic persons. Faces that do not belong to any real person, for example for fake profiles and reviews. According to the Commission’s guidelines, these can also fall under the deepfake definition: it is enough that a realistic-looking person could exist in this form.

Deepfakes and the AI Act: what Article 50 requires

The AI Act, i.e. Regulation (EU) 2024/1689, defines a deep fake in Article 3(60) as AI-generated or manipulated image, audio or video content that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful. Article 50 attaches two different obligations to this definition, which are regularly confused.

Provider obligation: machine-readable marking

Providers of AI systems that generate synthetic image, audio, video or text content must mark the outputs in a machine-readable format as artificially generated or manipulated. This means technical methods such as invisible watermarks, cryptographic proof of origin or metadata according to the C2PA standard. This obligation is addressed to the manufacturer of the model or service, not to the company that uses it.

Deployer obligation: disclosure to the audience

Deployers, i.e. everyone who uses an AI system under their own authority, except where this happens in the course of a personal non-professional activity (Article 3(4)), must disclose that a deepfake generated or manipulated with it has been artificially generated or manipulated. This is the obligation that affects the typical SME using an AI video for social media or a synthetic voice for a commercial. The only exception is use that is authorised by law to detect, prevent, investigate or prosecute criminal offences. For evidently artistic, creative, satirical, fictional or analogous works, the obligation does not lapse. There, it is limited to an appropriate notice that does not hamper the display or enjoyment of the work. The Commission’s guidelines interpret this privilege narrowly: purely informative content such as news is not covered, advertising only in certain individual cases. For AI-generated texts published with the purpose of informing the public on matters of public interest, a separate disclosure obligation applies, which lapses if the text has undergone editorial review and a natural or legal person holds editorial responsibility for the publication.

The deadlines that actually apply

A widespread misconception is that the Digital Omnibus on AI postponed Article 50 across the board to December 2026. That is wrong. The only new element is a grace period until 2 December 2026 for machine-readable marking under Article 50(2), and only for systems placed on the market before 2 August 2026. The disclosure obligation for deepfakes has applied since 2 August 2026 without any transition.

What a label has to look like

The Regulation does not prescribe a format but requires clear, distinguishable disclosure at the latest at the first exposure to the content. In June 2026, the European Commission published the Code of Practice on Transparency of AI-generated Content and, on 20 July 2026, the accompanying guidelines. The guidelines explain how the Commission interprets Article 50. The code is voluntary, but the Commission and the AI Board have confirmed it as adequate: anyone who signs and implements it can use it to demonstrate compliance with the marking and disclosure obligations under Article 50(2) and (4). Anyone who does not sign it must demonstrate themselves that their own measures are adequate. In addition, the Commission provides an official icon set for labelling, which is also voluntary to use.

In practice, this means for a company: a visible notice on the content itself, not in the legal notice. For a video, a caption or an overlay; for an image, a caption below it; for a synthetic voice, a spoken or displayed notice. The sentence “This video contains AI-generated content” at the beginning of a commercial fulfils the obligation; a note in the third line of the video description does not. If viewers can foreseeably join later, the guidelines say the notice should, where possible, also appear at later points.

Deepfakes as an attack vector: the case that changed the debate

In February 2024, an employee at the Hong Kong office of the engineering services firm Arup transferred around 25 million US dollars to fraudsters. The reason: a video conference in which the chief financial officer and other colleagues could be seen and heard ordering the transfer. All participants except the employee himself were deepfakes, generated from publicly available video footage. The company later publicly confirmed the incident.

The case is instructive because it shows where protection does not lie. Technical detection did not help, because the fake came through a trusted channel. A process would have helped: a second, independent confirmation via another communication channel before a payment of this magnitude is executed. This is exactly the lesson for SMEs. Anyone who bases payment approvals, master data changes and password resets solely on a voice or a face has a gap that can be exploited with a few minutes of source material.

Four measures that really work

1. Four-eyes principle with a channel switch. Critical approvals need a confirmation via a channel that the attacker does not control, typically a call back to the number stored in the system.
2. Amount thresholds and four-eyes approval in the ERP system. Technically enforced, not phrased as a house rule.
3. Training with real examples. The AI literacy obligation under Article 4 of the AI Act can be filled meaningfully with exactly this content.
4. Reducing the source material. Hours of executive videos and podcasts online are training material. That is no reason to stop communicating, but it is a reason not to tie approval processes to voice and face.

Limits of detection

Detectors that recognise deepfakes by their artefacts are a race that the generating side structurally wins: every published detector is a training signal for the next model. The reverse approach is more robust, namely making the origin of genuine content verifiable. This is exactly what provenance methods such as C2PA and the machine-readable marking under Article 50(2) aim at: instead of exposing the fake, the authentic content is signed. For companies, this is particularly relevant for their own image material, such as product photos and press images.

How it differs from related terms

Not every piece of AI-generated content is a deepfake. However, in addition to persons, the definition also covers objects, places, entities and events. According to the Commission’s guidelines, an AI-generated product image in advertising or on packaging is a deepfake if it can mislead the audience about the appearance, properties or use of the real product. A real product in front of an AI-generated background, on the other hand, is not, as long as the advertising does not mislead about the product itself. The provider’s machine-readable marking obligation applies regardless of whether there is a deepfake; the deployer’s disclosure obligation only applies to deepfakes. According to the guidelines, minor edits such as colour correction, lighting adjustment or removing passers-by in the background generally do not turn content into a deepfake; the context is decisive. In individual cases, the boundary is blurred. The yardstick is expressly not an average viewer but the foreseeable audience in all its breadth, including children, older people and other groups that are more easily deceived.

The legal situation beyond the AI Act

The labelling obligation under Article 50 is only one level. Anyone who depicts a real person in a deepfake in Germany also touches on the general right of personality and, in the case of likenesses, the right to one’s own image. Consent from the person depicted is therefore generally the prerequisite for a publication to be permissible at all, and labelling does not cure its absence. For advertising with the face or voice of well-known persons, claims under property law are added. Depending on the content, criminal offences may be relevant, for example violation of the most personal sphere of life through image recordings, insult and defamation, or forgery of documents in the wider context. If personal data is processed for training, the General Data Protection Regulation also applies.

For a company, this results in a simple order of checks. First, the question of whether a real person is recognisably depicted: if so, consent is needed, and it must be documented. Then the question of labelling under Article 50. Finally, the question of whether the content appears in a context in which the deception itself can cause harm, for example in press releases, product claims or investor communications. Most conflicts in practice arise not from labelling but from missing consent. This applies in particular to employee videos in which a synthetic voice or a subsequently adapted script is used long after the recording was approved.

What companies should do now

Three steps are enough to start with. First, an inventory: where in the company are AI-generated images, videos or voices created, including in agency services and social media teams? Second, a labelling rule that defines what the notice looks like and who adds it, anchored in the approval process for publications. Third, hardening approval processes against voice cloning, because that is where the actual financial damage occurs. The first task belongs to marketing, the third to accounting, and that is precisely why both tend to be left undone.