Regulation as a Design Constraint

A retail chain buys a tool from an HR software vendor. The tool reads job applications and ranks the candidates for each opening. Under the European Union's AI Act, the vendor is building a high-risk AI system, and from December 2027 it will need a risk management system, data governance, automatic logs, a human oversight design, technical documentation, a conformity assessment and an entry in an EU database.

Now suppose the chain decides the tool costs too much. Its recruiters start pasting CVs into a general-purpose chatbot with a prompt that says "rank these candidates". No vendor, no contract, no procurement review. Read plainly, the Act says the chain may have just stepped into the vendor's shoes, and the vendor's whole list of duties may now be its own.

Nobody wrote a line of code. The model underneath could be the very one the vendor used. What changed is what the system is used for, and who decided that.

That is the thing to understand about European AI regulation, and most teams look at it from the wrong end. The AI Act regulates uses and roles far more than models. Your obligations are set by product decisions (who the users are, what the output is used for, what the sales material promises), so the cheapest place to meet them is the design, while those decisions are being made. A compliance binder assembled before launch costs more and protects less.

Four questions decide what you owe

The AI Act has been in force since August 2024 and applies in stages. An amending regulation, the Digital Omnibus on AI, in force since 27 July 2026, moved several of those stages. The reach is wide: a US startup with no European office whose screening tool is used by a Dutch employer is covered. Working out what the law asks of one system mostly comes down to four questions, asked in order.

Scroll sideways to see the whole figure

Is it an AI system at all? The definition turns on one word. An AI system infers, from its input, how to produce outputs such as predictions, content, recommendations or decisions. A loan-eligibility rule set written entirely by people infers nothing; under this law it's ordinary software. Anything built on a language model infers, so for most readers the answer is yes.

Which role do you play? A provider develops a system, or has one developed, and puts it on the market or into service under its own name. A deployer uses a system under its authority, at work. Two consequences catch people out. "Into service" includes your own use, so a company that wraps a vendor's model in an internal assistant is the provider of that assistant and its deployer at once, though not the provider of the model underneath. And roles change after the fact: a deployer that changes what a system is used for, so that it becomes high-risk, turns into its provider. That is the chatbot-and-CVs story.

Which category does the use fall into? One system can sit in more than one.

• Prohibited since February 2025: a short list that includes social scoring, harmful manipulation, untargeted scraping of facial images and inferring people's emotions at work or in education. The Omnibus adds systems that generate non-consensual intimate images of real people or child sexual abuse material, from 2 December 2026.
• High-risk: safety components of regulated products such as toys, lifts or medical devices, and the uses listed in the Act's Annex III, which include biometrics, critical infrastructure, education, employment, credit scoring and public benefits, law enforcement, migration, and justice and elections. This is where the heavy duties live.
• Transparency: systems that talk to people must say they're AI unless that's obvious, generators of text, images, audio or video must mark their output in a machine-readable way, and deep fakes and emotion recognition must be disclosed.
• Minimal: everything else, which the European Commission describes as most AI systems in use. No specific duties beyond AI literacy.

Annex III has an exit, and the exit has a lock. A listed system isn't high-risk if it doesn't materially influence decisions: a narrow procedural task, polishing work a person already finished, preparing material for a human who decides. But a system that profiles people, evaluating their personal characteristics, is always high-risk. Ranking job candidates is profiling. For the HR vendor, the exit is closed.

Which date applies?

Scroll sideways to see the whole figure

The prohibitions, the literacy duty and the rules for general-purpose model providers already apply. Transparency duties have applied since 2 August 2026. The high-risk core doesn't apply yet: the Omnibus moved it to 2 December 2027 for Annex III uses (from August 2026) and to 2 August 2028 for regulated products (from August 2027), citing late standards and late national authorities. Fines reach €35 million or 7 percent of worldwide turnover for prohibited practices and €15 million or 3 percent for most other duties, with lower caps for small companies.

One model, three price tags

Take one commercial model API and wire it into three products.

An internal assistant. A logistics company builds a helper that answers employees' HR and IT questions and drafts replies that customer-support agents edit and send. Nothing here is in Annex III, so the work is transparency. For employees, the Commission's guidance names an internal assistant used by trained, AI-literate staff as an example where AI involvement can be obvious, though it wants that exception read restrictively. The subtle part is marking. Drafts that go to customers are synthetic text leaving the company, so on a literal reading they need machine-readable marking, and the agents' edits don't remove that duty. The company may rely on marking its model vendor embeds, while staying responsible for it.

A CV-screening product. The HR vendor owes the full high-risk list, plus a quality management system, a conformity assessment it can carry out itself for employment uses, CE marking and monitoring after launch. The retailer using it owes deployer duties: follow the instructions, give oversight to people with the competence and authority to exercise it, keep logs for at least six months, inform workers' representatives before using it on its own staff, and tell candidates a high-risk system is involved.

A fine-tuned open model. A five-person startup fine-tunes an open-weight model on contracts and sells a review assistant to law firms. Has it become a model provider, with model-level duties? The Commission's test: only if the fine-tune used more than a third of the original training compute. Training compute grows roughly with the model's size times the number of tokens it trains on, so for the same model a fine-tune's share is about its share of the tokens. One billion tokens on a model pretrained on fifteen trillion is about 0.007 percent, some five thousand times below the line. Contract review isn't in Annex III, so the startup owes transparency and literacy, and nothing at model level.

In each case, the decision that set the obligations was a product decision.

Three ways teams drift across a line

The risky moments are rarely launches. They're small product decisions nobody reads as legal ones.

The shortcut. The CVs in the chatbot. The law makes a deployer the provider of a high-risk system when it repurposes a system that wasn't high-risk, explicitly "including a general-purpose AI system", into one that is. The reading that catches the retailer rests on two conditions: ranking is the organization's practice, decided or tolerated, rather than one employee's private experiment, and the output really ranks or filters candidates rather than summarizing files a person then judges unaided. It's a strong reading, not settled law. Either way, a shortcut that feels like using a tool can be, legally, building one.

The feature. The logistics company adds a dashboard that ranks support agents by the assistant's assessment of their replies. That evaluates performance at work, an Annex III use: high-risk. A version that infers the agents' frustration from their voices would be prohibited. Point the same emotion recognition at customers and it isn't banned, but it's high-risk and must be disclosed. These are product decisions with regulatory prices, best priced before they reach the roadmap.

The sales deck. The Act defines a system's intended purpose partly through the provider's "promotional or sales materials and statements". If the contract startup's deck starts offering the assistant to courts or arbitration bodies, the product has entered Annex III's justice category. The code didn't change. A slide did.

Marketing copy is a compliance document.

Good engineering already does most of this

Most high-risk requirements describe properties a well-run AI system should have for its own reasons. That overlap is an opportunity if you design for it and a trap if you don't, because traceability can't be added for data that was never recorded.

Logs. High-risk systems must record events automatically over their lifetime, and providers and deployers keep the logs they control for at least six months. One record per decision, holding the input, the model and prompt version, the output, the person who reviewed it and what they decided, serves the regulator, the incident review, the evaluation set for the next model and the engineer debugging on a Tuesday (Proof That the Work Happened).

Oversight that actually oversees. Overseers must be able to understand the system's limits, stay alert to automation bias (the habit of trusting a machine's output too readily), override or reverse what it produces, and stop it through "a 'stop' button or a similar procedure". A recruiter clicking approve on two hundred rankings an hour satisfies none of that in practice. A design that does shows the evidence behind each ranking, makes overriding as cheap as accepting, samples decisions for a second look, and treats an override rate near zero as a question to investigate (Human Review Where It Matters).

Marking as a pipeline property. Decide where marks are applied, and test that conversion, editing and export don't strip them.

Documentation as a by-product. The technical file asks for intended purpose, architecture, data, test results, oversight measures and changes over time. A team that keeps a decision record, versioned evaluation results and a change log already has most of it. A team that doesn't will reconstruct its history under deadline.

My view on architecture follows: build the high-risk function as its own component, with its own documentation, logs and oversight, so it's clear what the conformity work covers. No article requires it, and it never licenses relabeling a high-risk function to escape the rules.

Treat the literacy duty as a lever

One obligation already applies to every provider and deployer, whatever the category: AI literacy, meaning the skills people need to use AI knowledgeably, limits included. The Omnibus softened it to taking measures that support it, with no guaranteed level for any individual.

It's tempting to tick the box. I'd treat it as the cheapest lever in the Act. Training that teaches people where a tool fails, how to check it and when to override it is the same training that supports the "obvious" exception for internal assistants, gives overseers of high-risk systems the competence the law demands, and counters automation bias. One program serves all of them (An AI Learning Program That Changes Work).

Know where the lines stop, and don't lean on them

Knowing where obligations end matters as much as knowing where they start. An internal assistant held for months behind a review built for high-risk systems is a block the law doesn't require, and it teaches the organization that AI projects are slow for the wrong reasons. Research, personal use, systems that don't infer and the narrow Annex III exit are legitimate parts of the law.

A boundary you design with is different from a boundary your business depends on. The Commission reviews the high-risk list and the prohibitions every year, and the Omnibus moved in both directions at once, delaying the high-risk rules and adding two prohibitions. Systems already on the market escape the high-risk rules only until their design changes significantly, so a vendor whose margin depends on that grace has agreed not to improve its product. And the Annex III exit holds only while a system stays preparatory, while every roadmap pushes toward ranking and recommending (A Loophole Is Not a Business).

A boundary in the law is a fact to design with. A boundary your business depends on is a bet that it won't move.

Will it kill European AI?

The worry is fair. Mario Draghi's 2024 competitiveness report counted around 100 tech-focused EU laws, economists estimated that the GDPR pushed about a third of apps on Google Play out of the market, and the Omnibus itself concedes "a compliance burden that is heavier than expected". My reading is narrower than either slogan. The heavy duties attach to a defined set of uses that mostly already lived under anti-discrimination, consumer and sector law, Europe's AI investment gap is older than the Act, and the core hasn't applied yet. Until December 2027, the clearly avoidable cost is uncertainty: late standards, late guidance, moving dates.

What to do

Choose the intended purpose deliberately. Selling into an Annex III use means a quality system, documentation and conformity work before revenue. Staying out means saying no to some customers, in the product and in the sales material. Either can be right. Drifting into one without noticing is the expensive option.

Settle the event log, the review screen and the content pipeline while they're being designed. An override button added to a finished workflow rarely changes how people use it.

Bring legal into the design review, not the launch review. The questions that set obligations get answered in product meetings, and so does who answers internally when a system causes harm (Who Is Responsible When an Agent Acts).

Say which shortcuts are off-limits. If staff can rank, score or filter people with a general-purpose chatbot, the organization may be building a high-risk system by accident.

Ask your model vendor what it marks, and test that you keep it.

The models underneath will change several times before December 2027. The obligations follow the use and the role, so they survive every change. A system whose purpose is explicit, whose decisions leave records, whose overseers can actually intervene and whose outputs say what they are has done the hardest part of compliance. It's also easier to debug, to evaluate and to sell.

Check yourself

How can a company become the provider of a high-risk system without writing any code?

By changing what a system is used for. A retailer whose recruiters rank candidates with a general chatbot may step into the vendor's shoes.

Why is marketing copy a compliance document?

The intended purpose is defined partly by sales materials and statements, so a deck that offers the product to courts can move it into a high-risk category with no code change.

Why settle logs and oversight during design?

The requirements describe things a well-run system needs anyway, and traceability cannot be added for data that was never recorded.