Lukimo and Article 50 - An Example of doing the AI Act Correctly

What is Lukimo?

Lukimo is a Finnish AI voice generation platform offering ~35 professionally crafted Finnish voices powered by neural networks, where every voice has signed a licensing agreement with Lukimo.

Output is studio-quality TTS, served via web app and REST API, with credits used for watermarked previews and non-commercial downloads. I had an entertaining discussion with the founder and CEO Marko Tähtikumpu where we discussed his excellent service and business model. Essentially it’s like spotify for voice over artists where you get paid as an artist for every time your voice is used either as a mix of voices or as a single voice. So for instance if 3 artists had their voices synthesised and used they would get 33.33% each depending on the amount of plays. If you were the only voice used you you get 100% of the value for each play.

This isn’t an article so much about Lukimo - though I’d highly recommend you check it out if you’re a B2B creative agency or a voice over artist, it’s about the legal aspect and what Marko and his team have covered their article 50 obligations under the EU AI Act. Also what is coming that they still might need to take into account.

Why this matters

Two things matter here legally:

1. It’s a curated catalogue of pre-defined voices, not on-demand cloning of arbitrary people from a customer-supplied sample;
2. The voice talents have given consent. Both narrow Lukimo’s exposure compared to e.g. ElevenLabs-style instant voice cloning.

Where Lukimo sits on the provider/deployer line

Lukimo is the provider of an AI system that generates synthetic audio. The customer typing text into the platform (or hitting the API) is the deployer. Article 50 splits obligations across that line, they’re not all Lukimo’s.
The obligation that does land on Lukimo: Article 50(2)

Because Lukimo’s system generates synthetic audio, it must ensure outputs are marked in a machine-readable format and detectable as artificially generated or manipulated, with the marking being “effective, interoperable, robust and reliable… as far as technically feasible.”

Three points worth mentioning:

• It applies to all output, not just previews. Lukimo currently watermarks the preview audio that’s burned into low-tier downloads. That’s a perceptible watermark for commercial protection. Article 50(2) requires a machine-readable mark on the commercial, clean download too — and the deployer can’t opt out of it. The two purposes are different: Lukimo’s current watermark exists to stop unpaid commercial use; the AI Act’s required mark exists to let third parties detect AI provenance. They probably need both, and the AI Act mark needs to survive normal post-processing (compression, transcoding, cuts, EQ).
  Recommended technical approach: a perceptual audio watermark (Meta’s AudioSeal, Resemble’s PerTh, or similar) embedded into all rendered output, plus a C2PA manifest in the file container.
• Belt-and-braces: the C2PA manifest gives strong, verifiable provenance for files that arrive intact; the watermark survives being stripped or re-encoded.
  The “assistive editing / no substantial alteration” carve-out doesn’t rescue them. That exception covers things like denoising or auto-EQ. TTS generation from text is the canonical case of “substantially altering” — it’s literally creating audio that didn’t exist.
• Penalty if they get this wrong: up to €15M or 3% of global annual turnover under Article 99(4)(g). Practically not a destroying number for Lukimo at their stage, but the reputational damage in the Finnish/EU market would be worse than the fine.

The obligation that lands on Lukimo’s customers: Article 50(4)

Article 50(4) puts the deepfake disclosure burden on the deployer, not Lukimo.

The question is whether Lukimo output is even a “deep fake” under the Act. Article 3(60) defines a deepfake as AI-generated content that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful.

For most Lukimo use cases — branded explainer videos, e-learning, marketing voiceover with a generic-sounding “Voice 17” — this is not a deepfake. The listener isn’t being deceived into thinking they’re hearing a specific identifiable real person. Recital 134 supports treating generic synthetic content differently from impersonation.

It becomes a deepfake when a deployer (IE a b2b Agency using the tool) uses a Lukimo voice to:

• imitate or pass off as a known public figure (even loosely),
• fabricate fake testimonials presented as real customer audio,
• generate fake news audio with apparent authenticity,
• voice a real person’s words without their involvement.

In those cases the deployer owes the disclosure under 50(4), not Lukimo. But Lukimo still has an interest, because:

• The Act’s allocation of fault doesn’t stop reputational damage spilling onto the platform.
• Lukimo’s terms should require deployer compliance and prohibit obvious misuse.
• There’s a soft trend (and the codes of practice in 50(7) will likely formalise this) toward providers giving deployers in-product nudges at the point of generation.

The artistic/satirical carve-out in 50(4) reduces but does not remove the disclosure burden for legit creative work, and it’s a deployer-side argument, not a Lukimo-side one.

What is NOT triggered

• Article 50(1) — AI interaction disclosure. Doesn’t apply: Lukimo’s product produces audio files, not real-time interaction with natural persons. If a deployer wires the API into a voice agent that talks to humans live, the deployer’s deployment triggers 50(1) — Lukimo doesn’t.
• Article 50(3) — emotion recognition / biometric categorisation. Lukimo expresses emotion in output; it doesn’t recognise it. 50(3) is about systems that infer emotional state from people, which Lukimo doesn’t do.
  GPAI obligations (Articles 51–55). Lukimo’s TTS model is a specialised model, not a general-purpose AI model in the Act’s sense. Nowhere near the 10^25 FLOPS systemic-risk threshold either.
• High-risk classification (Annex III, Article 6). Voiceover for marketing/training isn’t an Annex III use case.

A separate point that’s not Article 50 but worth flagging: voice samples used to train the model are biometric data under GDPR Article 9. Lukimo’s licensing agreements with voice talents need explicit consent meeting Art. 9(2)(a). That’s a parallel compliance track, independent of the AI Act.

The good news

The good news for Lukimo: the licensed-voice catalogue model and Finnish data residency posture put them ahead of most competitors on the GDPR side and on the deepfake risk surface.

They have already added machine readable meta data to all their files so in this sense they have done what they are required to have done from the AI act standpoint.

Recital 133 walks through what’s acceptable and explicitly contemplates this: watermarks, metadata identifications, cryptographic methods for proving provenance, logging methods, fingerprinting, “or other techniques.” All of those are inaudible/invisible to a normal listener. The recital actually nudges toward imperceptibility, framing it as a quality of being “robust” and “interoperable” — i.e. surviving compression, transcoding, edits — without degrading the user’s experience of the content.

Where this matters is the line between 50(2) and 50(4):

• 50(2) — provider’s marking obligation: machine-readable. Inaudible is fine. This is Lukimo’s job.
• 50(4) — deployer’s deepfake disclosure obligation: human-readable. The deployer has to tell people the content is AI-generated, in a “clear and distinguishable manner” (50(5)). This is the customer’s job, when their use case crosses into deepfake territory.

So for Lukimo: an inaudible perceptual audio watermark plus a C2PA manifest is exactly the right shape of solution. The existing perceptible “preview” watermark on free downloads is doing a different job (anti-piracy / commercial gating) and isn’t what 50(2) is asking for. They have both, but the 50(2) one is designed to be invisible.

As I said at the beginning this was an interesting case to hear about because it shows what a company in the creative industry has to do with regards the legal implications of the AI act. It’s also a product I can see winning because its great for both the voice over artist and the creative teams.