If the thing that checks an AI output is itself an AI model, you have two systems that can vary from run to run and be wrong in ways nobody predicted. A control layer built from deterministic software gives the same result for the same input, can be validated with ordinary computerised system methods, and can be explained to an inspector without talking about model internals.
The idea in one sentence
Let AI generate. Let deterministic software verify. The model can draft, summarise, search and suggest. Before anything it produces reaches a person, a record or a process, a separate layer made of plain, testable rules checks it, assigns a risk tier and writes down what happened. That layer contains no AI at all.
Why not use a second model to check the first?
It is a tempting shortcut, and it is common: one model produces the answer and another model grades it. We think it is the wrong foundation for a regulated process, for three reasons.
- It is not repeatable. Ask a language model to grade the same answer twice and you can get two different grades. An audit trail built on that cannot be reproduced when an inspector asks you to show how a decision was reached.
- Its errors are hard to predict. Models trained on similar data can share blind spots. A checker that misses the same kinds of mistake as the generator adds confidence without adding protection.
- It is hard to validate. Quality teams know how to validate software that behaves the same way every time. They have no settled method for validating a judge whose behaviour shifts when the model is updated.
What deterministic means in practice
Deterministic means that the same input, run against the same version of the rules, always gives the same result. Each rule is an explicit check with a clear pass or fail, written down, version controlled and tested. Typical checks look like this:
- The source an answer cites exists in the approved library, and it is the current effective version, not a superseded one.
- A number in the answer matches the number in the cited source.
- A risk score sits inside the scale it belongs to. A score of 14 on a scale of 1 to 4 fails, whatever the model says.
- A quotation is complete. A quotation that stops mid sentence and changes the meaning fails.
- The answer does not stray into topics that are excluded for that use case.
When a check fails, the output is refused or sent to a named person, and the failure is recorded as carefully as a success would have been.
What a deterministic layer cannot do
It cannot tell you whether a summary is well written, or whether a recommendation is wise. It checks what can be checked: that the evidence exists, that it is current, that the output stays inside its boundaries, and that the right person has seen it. Everything else is a human judgement, which is why the tier assigned to each output decides how much human review it gets. The layer is there to make that judgement faster and better evidenced, not to replace it.
How this relates to draft Annex 22 and GAMP 5
The draft EU GMP Annex 22 draws a line for critical GMP applications: static, deterministic models on one side, generative models and large language models on the other. A deterministic control layer is the kind of component that sits on the right side of that line, because it can be specified, tested and validated in the way GAMP 5 describes. The generative model is then used where a qualified person remains responsible for what is done with its output. We describe our design as aligned with these frameworks. We do not claim that any regulator has approved it.
Questions to ask any vendor of an assurance product
- Is any part of the checking itself done by a language model? If so, which part?
- If I run the same output through your checks twice, will I get an identical result and an identical record?
- Can you show me every rule, in plain language, with its version history?
- What happens to an output that fails a check? Where is that recorded?
- If I change the AI model next quarter, which of your evidence stays valid?
Frequently asked questions
Does AiVACO's platform use AI?
AAP works with AI models, but its control layer contains none. The assurance engine, the risk tiering service and the audit trail are deterministic software, so the same input always produces the same result.
Can a deterministic layer catch every AI error?
No. It catches what can be expressed as a check, such as missing sources, superseded documents, figures that do not match and excluded topics. Higher risk outputs are routed to a qualified person for judgement, and acceptance is recorded by electronic signature.
Why not simply validate the AI model itself?
Models change often and behave probabilistically, so validation evidence for one version says little about the next. Validating the deterministic layer once, and testing each model against fixed acceptance criteria, keeps the evidence stable when the model changes.
Sources and further reading
- European Commission: consultation on draft Annex 22 (Artificial Intelligence), revised Annex 11 and Chapter 4
- ISPE GAMP 5 Guide, Second Edition
- ICH quality guidelines, including Q9(R1) Quality Risk Management
This article is general information, not legal or regulatory advice. Regulatory texts change, and some discussed here are drafts: check the current official text before relying on any detail.
