Building an AI Strategy You Can Defend
Regulatory readiness is not a brake on AI adoption. It is what allows an organisation to adopt AI with confidence, defend it to an inspector, and move faster than competitors still waiting for certainty.
Building an AI Strategy You Can DefendWhat this paper covers
Part one makes the case and gives Quality and IT leaders the tools to act on it. The technical annex gives the teams who will build or buy the system the engineering detail to do it defensibly.
Regulatory timeline · four layer stack · Annex 22 boundary matrix · before and after deviation flow · four tier consequence ladder · six phase chevrons · icon keyed checklist · Assurance Layer architecture · event assistant swimlane · finding versus explaining · priority staircase and 90 day Gantt
Seven layer reference architecture · twelve step roadmap · metadata record card · seven gate orchestration sequence · evaluation metric board · threat and control matrix · pharma mapping ladder · five sector comparison · eight failure mode cards · fourteen week Gantt
The consequence model (page 09), the five executive questions (page 13) and the assurance checklist (pages 11 to 12). Together they answer where AI may be used, what to ask about it, and how to evidence it.
Building an AI Strategy You Can DefendThis already happened.
AI regulation in regulated industries has moved from broad principles to specific, inspectable expectations. The organisations that will scale AI safely are the ones that build assurance capability now rather than retrofit it later.
Three things changed between mid 2025 and October 2026. The EU AI Act began applying in stages. The European Commission published draft EU GMP Annex 22 on Artificial Intelligence alongside revised Annex 11 and Chapter 4, setting out which model types are and are not acceptable in critical GMP applications. And the FDA, EMA and other regulators began deploying AI themselves, changing the inspection dynamic from finding information to explaining it.
The common thread across every framework is the same. The closer an AI output gets to a decision that affects safety, quality or data integrity, the stronger the expectation for a defined intended use, evidence of trustworthiness, traceability and retained human accountability. The technology name matters less than the consequence of being wrong.
Regulatory readiness is not a brake on AI adoption. It is what lets an organisation adopt AI with confidence, defend it to an inspector, and move faster than competitors who are still waiting for certainty.
- The regulatory landscape across the EU AI Act, EU GMP, FDA and EMA, and where they converge
- The critical boundary that separates permitted from discouraged AI uses, and why generative AI sits where it does
- Pharmaceutical manufacturing as the reference case for every regulated sector, because it is the hardest environment in which to prove safety and compliance
- A consequence based model for scoping pilots and a six phase framework for building an AI strategy
- A technical assurance checklist that Quality and IT can apply to any AI use case today
- How the AiVACO Assurance Layer applies these principles in practice as a deterministic control layer between AI output and the regulated process
- A technical annex for IT, platform and security teams: a twelve step guide to deploying retrieval augmented generation in a regulated enterprise, with a reference architecture, a pharmaceutical case study and a cross sector mapping
Written for Quality, Validation and IT leaders who are being asked to approve, pilot or scale AI and who need a defensible basis for doing so.
Building an AI Strategy You Can DefendWhy AI governance became a board level issue in 2026
For most regulated organisations, AI adoption started before AI governance. The governance question arrived afterwards, and in 2026 it arrived with regulatory weight behind it.
Drawn to scale, July 2025 to August 2026. Blue: EU regulatory milestones. Red: the draft GMP package. Green: the regulator adopting AI itself.
Employees began using general purpose assistants, suppliers embedded models into enterprise applications, and data science teams deployed classification and prediction models into operational systems. Alongside the regulatory milestones, the FDA launched its own generative AI assistant, Elsa, in June 2025 and extended it with Elsa 4.0 and the HALO data platform in May 2026. The regulator is now equipped with the same class of technology that industry is adopting.
The consequence for Quality and IT leaders is that two previously separate questions have merged. "Can we validate this system?" and "Can we defend this decision to an inspector?" are now the same question, and the answer depends on whether the organisation can show intended use, evidence, traceability and accountability for every AI output that touches a regulated process.
Deploying tools without defined intended use, testing, ownership or oversight, and discovering the gap during an inspection.
Waiting for final regulation while competitors build the capability that every framework is converging on.
The capabilities themselves are already clear: risk management, accountability, data governance, lifecycle control, evidence and human oversight. Building them now is the only strategy that works under both the current drafts and whatever final form they take.
Building an AI Strategy You Can DefendThe regulatory landscape
A regulated organisation deploying AI is now subject to at least four layers of expectation that apply simultaneously.
Layer 1: The EU AI Act
Technology law rather than quality regulation. It classifies AI systems by risk, prohibits a small set of unacceptable practices, imposes obligations on high risk systems, and adds transparency duties for AI that interacts with people or generates content. For most manufacturers the direct effect is on documentation, transparency and supplier due diligence. Prohibitions under the AI Act are statutory. Nothing in sector quality guidance has that force.
Layer 2: EU GMP and the proposed Annex 22
Existing Annex 11 already requires validated, risk managed computerised systems. The proposed package extends this: revised Chapter 4 recognises electronic, imagery, video and audio records; revised Annex 11 strengthens lifecycle, supplier, identity, audit trail, e-signature and archiving controls and links data integrity to ALCOA+; new Annex 22 supplements Annex 11 for systems containing AI in critical applications.
Layer 3: The FDA credibility framework
The January 2025 draft guidance reframes the question from "is this model validated?" to "is there sufficient evidence to trust this model for this context of use?" Evidence depth scales with the model's influence on a decision and the consequence of it being wrong. The same model may warrant very different controls in different applications.
Layer 4: EMA and the joint principles
The EMA reflection paper names the risks of complex, non transparent architectures: bias, reliability and trustworthiness. The ten joint principles signal that future guidance on both sides of the Atlantic will rest on shared foundations. This convergence is the most useful planning assumption available.
Annex 22's headings, from intended use through test data independence to explainability, confidence and operation, are a blueprint for an internal AI assurance programme and the backbone of the Section 7 checklist.
Building an AI Strategy You Can DefendThe critical boundary: intended use and consequence
Acceptability depends on what the AI is asked to do and what happens if it is wrong, not on the name of the technology.
| Model characteristic | Critical GMP application affects patient safety, product quality or data integrity | Non critical application |
|---|---|---|
| StaticDoes not adapt its performance during operation | In scope · acceptable with full assurance | Proportionate governance |
| Deterministic outputIdentical inputs produce identical outputs | In scope · acceptable with full assurance | Proportionate governance |
| DynamicContinuously learns and adapts during operation | Not in scope · should not be used | Proportionate governance |
| Probabilistic outputIdentical inputs may produce different outputs | Not in scope · should not be used | Proportionate governance |
| Generative AI and LLMsProduces novel text, code or media from a prompt | Not in scope · should not be used | Explicitly contemplated · qualified person responsible · human in the loop |
Annex 22 is draft guidance under consultation. "Banned under Annex 22" is too strong. The accurate formulation: the draft indicates the regulatory direction and states these model types "should not be used" in critical GMP applications. Statutory prohibitions exist only under the EU AI Act, and concern unacceptable risk practices, not model architectures.
Annex 22 does not prohibit generative AI in regulated organisations. It places generative AI on the non critical side of the line, under human supervision.
Knowledge discovery across SOPs and records · first drafts for controlled review · regulatory intelligence and change impact analysis · training content. Illustrative, not a whitelist: each still needs intended use, ownership, access controls and an accountable human.
Three questions that define the boundary
- What is this AI being asked to do? The intended use, written precisely enough that an inspector could test the system against it.
- Does the output directly affect patient safety, product quality or data integrity? If yes, the use is critical and the model type restrictions and full assurance expectations apply.
- What happens if the output is wrong? The consequence of error, not the sophistication of the model, sets the depth of evidence and control required.
A generative assistant that summarises a batch record for a qualified reviewer is a different system from one whose output determines whether a batch is released, even if the model is identical. Same technology, different intended use, different answer.
Building an AI Strategy You Can DefendThe pharmaceutical use case: the hardest environment as the reference standard
If an assurance approach holds up under GMP inspection, it will hold up in MedTech, aerospace, defence, financial services or industrial control.
"The AI classifies the deviation and proposes the CAPA"
Deviation raised
Classifies and proposes CAPA
Decision recorded
"The AI prepares, the investigator decides"
Deviation raised
Retrieves history, SOP versions, specification
What was retrieved and shown
Investigator classifies and owns CAPA
Same model, same productivity, a fraction of the validation burden, and a clear answer for the inspector. The controls required: data access governance, version control on sources, an audit record, and investigator training.
An incorrect output that influences batch release, deviation classification or a specification decision can reach a patient. There is no rollback.
ALCOA+ has been inspected against for over a decade. AI raises new questions: what source data did the model see, which version did it retrieve, can the output be reconstructed. Revised Annex 11 answers: AI does not bypass data integrity expectations. It inherits them.
CSV, ICH Q9(R1) risk management and GAMP 5 lifecycle thinking are mature. The challenge is extending them to systems whose behaviour is harder to specify, test and explain.
Building an AI Strategy You Can DefendWhat pharma teaches every regulated sector
Six principles that GMP makes explicit, and their general form for any organisation whose AI output touches a regulated decision.
| Pharma principle | General form for any regulated organisation |
|---|---|
| Scope assurance to critical applications | Classify every AI use by the consequence of a wrong output; scale control to that consequence |
| Exclude dynamic, probabilistic and generative models from critical decisions | Keep non deterministic AI in advisory roles until deterministic controls can verify its output before use |
| Apply ALCOA+ to AI inputs and outputs | Record source, version, provenance and lineage for every AI interaction that touches a controlled record |
| Require qualified personnel and human in the loop | Name an accountable owner; make human acceptance of AI output an explicit, attributable act |
| Keep validated systems in a validated state | Monitor performance, govern change, capture incidents, trigger reassessment |
| Be able to explain the system to an inspector | Design for defensibility: if it cannot be explained and evidenced, it is not ready |
Moving the AI one step back from the decision and inserting a deterministic record and a qualified human is the core of what an assurance layer does, and it generalises far beyond pharma.
Building an AI Strategy You Can DefendA consequence based model for scoping AI pilots
Classify every use case by the consequence of a wrong output, because the consequence determines the depth of evidence, control and human oversight required. A practical interpretation of the regulatory direction, not an official classification.
Low consequence: AI assists a person with information
Examplesdrafting, searching, summarising, knowledge retrieval, document comparison
Approachgovernance, defined intended use, trained human review of every output
Moderate consequence: AI recommends or prioritises
Examplestrend detection, investigation support, risk prioritisation, anomaly flagging
Approachstronger evidence, acceptance criteria, SME review, performance monitoring
High consequence: AI materially influences a critical activity
Scopepatient safety, product quality and data integrity become central to the assessment
Approachformal risk based controls, deterministic verification of output before use
Critical autonomous decision: AI decides with direct GMP consequences
Examplesbatch disposition, deviation classification, specification or release decisions
Approachstatic, deterministic models only, full Annex 22 assurance, explicit human accountability
Most organisations should start their pilots in tiers 1 and 2, where generative AI is explicitly contemplated and the assurance burden is proportionate.
Tier 3 is where a non deterministic model can contribute while a deterministic control verifies, tiers and records the output before it reaches a person or a system.
Tier 4 remains the domain of static, deterministic, fully validated models and explicit human decision.
"Why are we allowed to use this tool here but not there?" The answer is the tier, and the tier is the consequence.
Building an AI Strategy You Can DefendA six phase framework for building an AI strategy or pilot
An AI strategy for a regulated organisation is an operating capability, not a technology purchase. The sequence works for a first pilot and scales to an enterprise programme.
Discover
Classify
Define intended use
Assign ownership
Evidence and controls
Operate and monitor
1 · Discover
Find every place AI is already in use before designing any programme. Look for approved tools; employee led use of general purpose assistants, often on personal accounts; AI embedded inside purchased applications (eQMS, EDMS, ERP, MES, LIMS, LMS) and often enabled by default; supplier and third party AI on your data; and ML models already operating in manufacturing, quality or analytical systems. Without the inventory an inspector's first question cannot be answered.
2 · Classify
For every system: what it does, whether it affects patient safety, product quality or data integrity, whether it supports or makes a decision, and what happens if it is wrong. Assign a tier of the consequence model.
3 · Define intended use
Draft Annex 22 and the FDA credibility framework both make intended use (or context of use) the anchor for every subsequent control. A good statement names the task, the inputs, the output, the decision or process it supports, the boundary of acceptable use and the prohibited uses.
4 · Assign ownership
Every use case needs a named accountable owner, and governance cannot sit with IT alone. Draft Annex 22 expects collaboration between process SMEs, QA, data scientists, IT and consultants across algorithm selection, training, validation, testing and operation.
5 · Establish evidence and controls
Proportionate to the tier. Tiers 1 and 2: data access governance, version control on sources, output review by trained personnel, a record of use. Tiers 3 and 4: acceptance criteria, independent test data, documented test execution, explainability, a confidence measure and formal risk based controls.
6 · Operate and monitor
Monitor model performance, capture incidents, govern changes to models, prompts, data sources and integrations, and trigger reassessment on change or drift. Review periodically, as revised Annex 11 expects for any validated system.
Skip to Phase 5 and the validation package cannot answer an inspector's questions. Stop at Phase 2 and you have a risk register with no controls. The sequence is what makes the capability repeatable.
Building an AI Strategy You Can DefendTechnical assurance checklist for Quality and IT
Organised around draft Annex 22 and the lifecycle expectations of revised Annex 11, so each control maps to an expectation an inspector would recognise. Applies to any AI use case in any regulated sector.
Visibility
- Every AI system in use is inventoried, including employee led, embedded and supplier AI
- Each use case is classified by consequence tier and by whether it affects safety, quality or data integrity
- Model characteristics (static or dynamic, deterministic or probabilistic, generative) are recorded
- The inventory has an owner and a review cadence
Intended use
- The intended purpose, inputs, outputs and supported decision are documented
- The boundary of acceptable use and the prohibited uses are defined and enforced technically where possible
- The consequence of an incorrect output is understood and documented
- Intended use is approved by the process owner and Quality
Governance and people
- A named accountable owner exists for each use case
- Quality, IT and security, process SMEs and data specialists are involved according to the use case
- Responsibilities, qualifications and access rights are documented
- Supplier and third party AI is covered by supplier management controls
Data
- Data sources the model can access are enumerated and access is controlled
- The current approved version of controlled information is the version the model retrieves
- Source provenance can be traced for every output that touches a controlled record
- Confidential, personal and commercially sensitive information is protected in prompts, retrieval and outputs
- ALCOA+ expectations are applied to AI inputs and outputs, not only to the underlying records
Recognise a gap in A to D? AiVACO's AI readiness assessment produces a consequence tiered inventory and a gap analysis against this checklist. See page 30.
Building an AI Strategy You Can DefendTechnical assurance checklist: testing to lifecycle
Groups E to H cover the evidence, explainability, oversight and operation expectations that distinguish a validated AI use from a documented one.
Testing and evidence
- Acceptance criteria are defined before testing, in terms a reviewer can verify
- An appropriate test dataset exists and its independence from training and tuning data is documented
- Test execution is recorded with pass and fail outcomes against the acceptance criteria
- Known failure modes, edge cases and adversarial inputs are included in testing
- Evidence depth is proportionate to the consequence tier
Explainability and confidence
- Responsible users can understand the significance and limitations of an output
- The system's role can be explained to Quality and during inspection
- Where the model provides a confidence measure, its meaning and thresholds are defined
- Outputs that fall below defined confidence are routed to a human rather than used
Human oversight and accountability
- Human review is required at the points the tier demands, and the reviewer is qualified
- The reviewer understands the AI's limitations and can reject its output
- Acceptance of an AI output is an explicit, attributable act, not a default
- Accountability for the decision stays with a named person, never with the system
Lifecycle and monitoring
- Model performance is monitored in operation against defined indicators
- Incidents, errors and near misses are captured and investigated
- Changes to models, prompts, data sources, integrations and suppliers are controlled
- Reassessment is triggered by change or drift, and periodic review is scheduled
- Audit records are complete, attributable, tamper evident and retrievable for the retention period
Rendered as tick box lists so the checklist can be printed and used in a review meeting; the regulatory mapping beneath each group shows an inspector where the expectation originates.
Building an AI Strategy You Can DefendFive questions for the executive conversation
If the whole checklist had to be compressed for a board or executive team.
An organisation that cannot answer all five has an AI governance gap, whatever its technology investment.
Where are we using AI?
What decisions does it influence?
Who is accountable?
What evidence tells us it is trustworthy for that use?
Could we explain and defend its use to an inspector?
Building an AI Strategy You Can DefendThe AiVACO Assurance Layer in practice
How can an organisation benefit from generative and probabilistic models when those models should not make critical decisions? Separate the AI that generates from the control that verifies, and make the control deterministic.
AI outputs from any model, routed through a model agnostic gateway
Generative assistants, LLMs, ML classifiers and third party AI platforms: swappable, never trusted directly
AiVACO Assurance Layer: deterministic by design, contains no AI
Ground
bind output to approved, current source
Validate
encoded rules, pass or fail, identical every time
Risk tier
A1 to D4 by consequence and context
Record
before release, refusals too
Release
or refuse, or escalate
Regulated systems and processes
eQMS, EDMS, ERP, MES, LIMS, LMS
Accountable, qualified person
Accepts residual risk by e-signature, owns the decision
| Stage | What happens | Regulatory expectation it serves |
|---|---|---|
| 1 · Ground | The output is bound to the approved, current version of the source material it claims to rest on. An output that cannot be substantiated against a controlled source does not proceed | Annex 11 data integrity · ALCOA+ Chapter 4 |
| 2 · Validate | Deterministic rule checks encoded from the applicable regulatory and company requirements. Identical inputs give identical results. Rules are authored and version controlled by qualified regulatory specialists | Annex 22 acceptance criteria · test execution Annex 11 validation |
| 3 · Risk tier | The output is assigned a tier from A1 to D4 according to the consequence of it being wrong and the context of use, which determines the route it takes next | Annex 22 intended use · scope FDA credibility |
| 4 · Record | The output, its grounding, its validation result, its tier and any refusal are written to the audit trail before anything is released. Refusals are recorded as fully as releases | Annex 11 audit trails Annex 22 operation |
| 5 · Release | Low tier outputs that pass are released. Higher tier outputs are routed to a qualified person for explicit acceptance by electronic signature. Outputs that fail are refused or escalated | Annex 22 human in the loop Annex 11 e-signatures |
Building an AI Strategy You Can DefendDesign principles that make it defensible, and a worked use case
Engine, tiering service and audit trail are deterministic software: validated under existing computerised system expectations, explained to an inspector without reference to model internals.
No output above the lowest tier is used on the strength of the system alone. Acceptance is an explicit, e-signed decision by a named, qualified person.
An inspector can reconstruct not only what was used but what was rejected and why.
The FMEA behind tiering fixes severity by consequence. Controls reduce occurrence and detectability only, per ICH Q9(R1).
Uses with unacceptable exposure, such as clinical Q&A with pharmacovigilance implications, are excluded by configuration, not policy alone.
Validate the deterministic layer once; govern each new use through the same pipeline: a content pack, a rule set, a tier mapping and an accountable owner. New uses are added by configuration, not by a new validation project.
Validation rules for the pilot: no off label claims, no comparative statements, no clinical advice. Clinical and adverse event enquiries are excluded at platform level. Without assurance this is a generative model producing unreviewed statements about medicines in public, with ABPI Code, pharmacovigilance and promotional compliance exposure.
Identical to the deviation scenario in Section 4, and to any case where an organisation wants the productivity of generative AI without allowing it to make, or appear to make, a regulated decision.
Building an AI Strategy You Can DefendThe inspector is acquiring AI too
The regulator is adopting the same class of technology it is asking industry to govern, and that changes what inspection readiness means.
Finding the information
- Limiting factor: the inspector's capacity to locate and cross reference records
- Sampling of documents and systems
- Gaps in documentation may go unexamined
- Preparation means assembling binders and locating evidence
Explaining the information
- Submission and inspection data processed at scale
- Patterns surfaced across data sources; undocumented AI use is exactly the kind of finding this is well placed to surface
- Quality of data, traceability, audit trails and system ownership become more exposed, not less
- Preparation means being able to explain and defend every output
The FDA launched Elsa, an LLM powered assistant for agency staff, agency wide in June 2025. The FDA states it is used to accelerate clinical protocol reviews, support scientific evaluations, help identify high priority inspection targets, summarise adverse events, compare labels and generate code for nonclinical database work.
In May 2026 the FDA announced Elsa 4.0 and HALO (Harmonized AI and Lifecycle Operations for Data). HALO consolidates more than 40 application and submission data sources across FDA centres so that staff can query data and build workflows without manually uploading documents. Announced capabilities include custom agents, document generation, quantitative analysis and visualisation, secure web search, voice to text, OCR and large repository search.
It is accurate to say that FDA already uses AI to support inspection targeting, since the agency states this directly. It is not accurate, on the public record, to say that Elsa or HALO conducts autonomous inspections or determines findings. Public claims should stay within that line.
For an organisation deploying AI, this cuts two ways. The audit trail from an assurance layer, with every output, refusal and decision recorded and reconstructable, is exactly the kind of evidence an AI equipped inspector can consume efficiently. The organisations best positioned are those whose AI governance produces evidence as a by product of operation rather than as a retrospective exercise.
Building an AI Strategy You Can DefendWhere to start
Sequence investment so that governance capability is always one step ahead of deployment ambition.
AI governance and inventory
Classified inventory with owners and tiers. Nothing else is complete without it.
Knowledge access
Governed retrieval over approved SOPs and records. Highest value, lowest consequence.
Quality team augmentation
Assisted investigation prep, document comparison, trend surfacing for SME review.
Workforce capability
Role based training on intended use, limitations and review responsibilities.
Manufacturing and analytical AI/ML
Static, deterministic models with Annex 22 aligned test evidence.
AI enabled workflows and agents
Only once governance is mature enough to govern autonomy. Every step tiered, recorded, owned.
Not only a working use case. A repeatable method, a trained team, a validated control layer and an evidence pack that can be shown to an auditor or inspector. The second use case takes a fraction of the time.
- Regulatory uncertainty should not mean organisational inactivity
- Annex 22 is fundamentally about whether AI can be trusted for its intended use
- The question is not whether generative AI is useful but whether the intended use and the consequence of failure are compatible with the controls around it
- AI can assist the decision; it does not transfer accountability for the decision
- Start with the process, not the platform
- Do not ask whether you could demonstrate the AI; ask whether you could defend why you trusted it
Technical annexDeploying retrieval augmented generation in a regulated enterprise
Written for the IT, platform, security and data engineering teams who will build or buy it, and for the Quality and validation colleagues who will have to defend it. Industry neutral in its engineering; pharmaceutical GMP as the case study.
Almost every tier 1 and tier 2 use case in this paper, from knowledge retrieval to investigation preparation to first draft documentation, is delivered through this one architecture. A RAG system that satisfies a GMP inspector will satisfy any other regulator's expectations of the same pattern.
A model on its own answers from what it absorbed in training
That knowledge has a cutoff, contains none of the organisation's controlled documents, and is produced by a probabilistic process that will sometimes generate fluent, confident, wrong text. None of that is acceptable where the answer informs a regulated activity.
RAG changes the model's job from remembering to reading
Before the model generates anything, the system retrieves the relevant passages from a governed store of approved documents, places them in front of the model with their identifiers, and constrains it to answer only from those passages and to cite them. The decision about what to do with the answer stays with a person.
RAG does not make the model deterministic, or the system compliant
Retrieval can return the wrong passages, a superseded version, or passages the user should not see. The model can still paraphrase loosely or ignore an instruction. The engineering in this annex closes those gaps with testable controls; the assurance layer verifies the result deterministically before it is used.
Technical annexReference architecture
Separate the three things that must never be confused: the organisation's knowledge, the organisation's controls, and the model. Each layer has its own owner, test evidence and change control, so any one can be replaced without invalidating the others.
Governed sources
Systems of record with lifecycle states and owners
Ingestion pipeline
Connectors, parsing, structure aware chunking, pinned embeddings, event driven sync
Knowledge store
Hybrid index, document level access control, partitions, versioned snapshots
Retrieval service
Query rewriting, hybrid search, reranking, permission filtering, context budget
Orchestration and guardrails: AiVACO Assurance Layer
Prompt as controlled document, screening, cite or refuse, deterministic validate, tier, record, release
Model gateway
Enterprise endpoint or self hosted model: pinned version, zero retention, residency, swappable
Experience and oversight
Embedded in existing tools, citations shown, review queue, e-signed acceptance by tier
| Layer | Typical owner | What breaks if it is wrong |
|---|---|---|
| 1 Governed sources | Document and process owners; Quality | Answers grounded in drafts, superseded versions or unapproved content |
| 2 Ingestion | Data engineering | Index drifts from the controlled state; metadata lost so citations cannot be verified |
| 3 Knowledge store | Platform engineering; information security | Users retrieve content they are not authorised to read; cross client leakage |
| 4 Retrieval | Data and AI engineering | Relevant passages missed, so the model refuses or answers from partial evidence |
| 5 Orchestration | AI engineering with Quality; the Assurance Layer | Ungrounded or unsafe outputs reach users; no evidence trail |
| 6 Model gateway | Platform engineering; supplier management | Silent vendor updates change behaviour; data leaves the boundary |
| 7 Experience | Product and application teams | Reviewers accept outputs by default; feedback and incidents not captured |
The cross cutting services are not optional extras. Identity decides what a user may retrieve. The audit trail is the evidence an inspector reads. Observability is how drift is noticed. Change control keeps a validated system validated after the first release.
Technical annexDeployment steps: from intended use to production
The engineering expansion of phases 3 to 6. Steps 1 and 2 are governance decisions that shape everything downstream; teams that start at step 3 build pipelines over data they have not yet agreed they may use.
Intended use, tier and refusal policy
Who may ask, what is in scope, which decisions answers may inform, what must be refused. One page, before any code.
Corpus governance and metadata schema
Eligible documents, authoritative systems, metadata every chunk carries. Only approved, effective versions enter.
Ingestion pipeline
Event driven sync, structure aware chunking, pinned embeddings, dead letter queue, pipeline as code.
Knowledge store
Hybrid index; permission filter before ranking; hard partitions; versioned snapshots.
Retrieval service
Glossary based rewriting, hybrid search and reranking, context budget, freshness surfaced, diagnostics.
Orchestration and guardrails
Prompt as controlled document; screening; cite or refuse; deterministic validation; tier; record before release.
Model gateway
Version pinned, zero retention, residency, no auto upgrade; fallback is a refusal.
Identity, access and audit trail
SSO, permissions re-evaluated per query, the interaction record, append only hash chained store.
Evaluation harness
Independent golden set, retrieval and generation metrics, thresholds set first, adversarial set, regression.
Security engineering
Threat model the RAG surface: injection via corpus, permission leakage, poisoning, vendor exfiltration.
Observability and operations
Quality indicators, drift alerts, incident route into CAPA, change control, periodic review.
Validation deliverables and the evidence pack
Everything the preceding steps produced, organised under the Annex 22 headings so evidence is re-indexed, not recreated, when guidance changes.
Technical annexThe two controls that decide what the model can see and what it can say
SOP-0114 Dissolution testing
Metadata is authored once in the system of record and propagated. A corpus that depends on engineers hand tagging documents will not survive its first periodic review.
Input screening
Out of scope classes, sensitive data, injection patterns
Retrieval
Permission filtered, within context budget
Cite or refuse
Answer only from passages; structured output
Validate
Deterministic rules, identical every time
Tier
A1 to D4 sets the route
Record
Full record before anything is shown
Release
Or route to e-signed review
The system prompt and templates are versioned, approved and change controlled like an SOP, because they define the model's permitted behaviour. The prompt version is recorded with every logged interaction.
Each cited passage exists in the retrieved set · each cited version is the effective version · claims outside the cited passages are flagged · prohibited content classes are detected · required qualifiers are present.
Retrieved content is wrapped and labelled as data, and the prompt structure makes that distinction explicit. This is the primary defence against prompt injection via the corpus.
Technical annexModel gateway, identity and the audit trail
Step 7 makes the model a swappable commodity behind one interface. Step 8 produces the evidence an inspector or auditor reads, and it is the one most often under built.
- Deployment pattern. Enterprise cloud endpoint with zero retention and residency; private deployment in your tenancy; or self hosted open weight model for full sovereignty
- Version pinning. A vendor model update is a change to a validated system: assess, run the harness, document, release. Never an automatic upgrade path
- Contractual controls. No training on your data, zero retention, named residency, sub processor disclosure, deprecation notice
- Fallback. A refusal, never a different, unvalidated model
For every query, at minimum: user identity · timestamp · original and rewritten query · filters applied · retrieved chunk identifiers with document, version and content hash · prompt version · model identifier and version · sampling parameters · full output · validation results · tier · routing decision · reviewer identity and outcome · refusal reason.
Written to an append only, hash chained store. The property an auditor tests is that a record cannot be changed after the fact without evidence. Retained for the regulated period, and provably reconstructable, including the index snapshot it relied on.
The people who can change the prompt, the rules or the index are not the people who approve those changes, and neither can alter the audit trail.
The gateway makes the model a swappable component; the audit trail makes every use of it reconstructable. Together they are what let the evidence chain survive a model change.
Technical annexEvaluation harness and acceptance criteria
How the organisation demonstrates, with evidence, that the system meets its acceptance criteria for its intended use, and how it detects regression after any change.
Retrieval
Generation
Illustrative values. Red markers are thresholds approved by Quality before testing; a system tuned until it passes a threshold set afterwards has not been tested.
SMEs write representative questions with reference answers and supporting passages: in scope classes, refusal classes, edge and hard cases. Held independent of anything used to tune. A few hundred items per use case.
Injection embedded in corpus documents, questions designed to elicit out of scope answers, permission probing across tenants, questions whose answer changed between versions.
Set pass thresholds per metric before testing, approved by Quality, and record the rationale.
Rerun the harness on every change to model, prompt, chunking, embedding, reranker, rules or index. This is the technical meaning of keeping the system in a validated state.
Sample live outputs for expert review on a schedule and compare expert judgements with automated metrics, so the metrics remain trustworthy proxies.
Technical annexSecurity, operations and the evidence pack
| Threat | How it manifests in RAG | Control |
|---|---|---|
| Prompt injection via the corpus | A document, attachment, ticket or email contains text instructing the model to ignore its rules or exfiltrate data, and is retrieved into context | Corpus admissionContent wrapped as dataScreened at ingestion and outputNo tool capability |
| Prompt injection via the user | A user attempts to override the system prompt or extract it | Input screeningPrompt not exposedAlert on repeat |
| Permission leakage | Restricted content reaches a user through retrieval, relevance scores or paraphrase | Filter before rankPartitionsPermission probing in tests |
| Corpus poisoning | Unapproved or altered content enters the index and grounds answers | Approved states onlyHash verificationPeriodic reconciliation |
| Exfiltration to the model vendor | Prompts, passages or outputs are retained or used for training | Zero retention contractPrivate or self hostedEgress monitoring |
| Over permissioned connectors | Pipeline service accounts can read more than the corpus scope | Least privilegeScoped credentialsAccess reviews |
| Supply chain | Vulnerable or malicious libraries in the orchestration or pipeline stack | Dependency pinningSBOMSigned builds |
| Denial of service and cost abuse | Automated or runaway querying | Rate limits per identityCost ceilingsAnomaly alerting |
Commission penetration testing that includes these threats, not only the web application surface, and repeat it on major change.
- Quality indicators. Refusal rate and trend, low retrieval score share, citation validation failures, reviewer rejection rate. A rising refusal rate often signals corpus drift or a broken connector before anyone reports it
- Drift detection. Retrieval score distributions and validation failure rates against the step 9 baseline
- Incident management. A route into the incident and CAPA process, with triage that distinguishes retrieval, generation, rules and corpus failures, because the fixes differ
- Change control and periodic review. Every change assessed, regression tested, approved and released with evidence; review at a frequency proportionate to the tier
| Step 12 · Evidence item | Content | From |
|---|---|---|
| Intended use and risk | Statement, tier, refusal policy, risk assessment with controls | Step 1 |
| Requirements | Traceable to intended use; acceptance criteria and thresholds | 1, 9 |
| Design and configuration | Architecture, schema, prompt set, rule set, tier mapping, pinned versions | 2 to 7 |
| Supplier assessment | Vendor assessments, contracts, attestations | 7 |
| Security | Threat model, penetration tests, access model | 8, 10 |
| Test evidence | Golden set, executed protocols, results, adversarial report, baselines | 9 |
| Operational procedures | Monitoring, incident, change, review, reconstruction | 8, 11 |
| Audit trail | The live interaction record and its integrity evidence | 8 |
Technical annexPharmaceutical case study: the same steps under GMP
How each step is instantiated in a manufacturer deploying RAG over its quality documentation, and which expectation drives it. Substitute the sector's own system of record, integrity rules and accountable role and the controls are the same.
The same system becomes tier 3 or 4 the moment its output classifies a deviation or informs disposition without a qualified person's explicit decision. The build does not change; the intended use, tier, routing and evidence burden do.
Technical annexTransferring the pattern to other regulated sectors
Every regulated sector has an equivalent of the three things GMP makes explicit. Map those three and the twelve steps transfer directly.
In every case the consequence model applies unchanged: the build is the same for a tier 1 knowledge assistant and a tier 3 decision support tool; the intended use, the tier, the routing and the evidence burden differ.
Technical annexBuild, buy or hybrid
A spectrum rather than a binary, and the regulatory burden does not move with the purchase order.
| Option | What you get | What stays with you regardless | Fits when |
|---|---|---|---|
| Buy a packaged RAG platform | Fast time to pilot; vendor maintained ingestion, search and UI | Intended use, corpus governance, acceptance criteria, evaluation, audit evidence, supplier management | Low consequence use cases; limited engineering capacity |
| Managed components, own orchestration | Managed model endpoint and search service; your own ingestion, guardrails, evaluation and audit | Everything in the compliance layer, by design | Most regulated organisations |
| Build or self host | Full control of every layer including the model; sovereignty | Everything, including inference operations and model lifecycle | Classified, export controlled or sovereignty constrained data |
| Assurance layer plus any of the above | A deterministic, validated control layer independent of retrieval and model choices | Intended use and corpus governance; the layer carries validation, tiering, record and release | One validated control pattern across many use cases and vendors |
Assuming a bought platform's compliance features discharge the organisation's obligations. They do not. An inspector will ask the organisation, not the vendor.
The intended use, the corpus, the acceptance criteria and the evidence are the organisation's whatever the sourcing model. The managed components option keeps the evidence chain under your control while avoiding infrastructure operation, which is why it fits most regulated organisations.
Technical annexFailure modes to design against, and what a first deployment looks like
Stale knowledge
Answer grounded in a superseded document; the index lagged the system of record
Silent scope creep
A system approved for retrieval is used for decision support within weeks
Permission leakage
Restricted content surfaces through retrieval, scores or paraphrase
Embedding or parser change
A library upgrade changes vectors or chunk boundaries; citations stop resolving
Context stuffing
Too many passages reach the model; it synthesises across sources and mis attributes
Reviewer rubber stamping
Outputs accepted by default because review is frictionless and unmeasured
Model update without assessment
A vendor update changes behaviour; metrics drift without anyone noticing
Evidence after the fact
Validation documentation written to fit a system already in production
Eight to fourteen weeks for a single tier 1 or 2 use case over an existing document estate. Corpus cleanup, where the estate is not under lifecycle control, is the most common cause of extension.
| Accountable business owner | Intended use, tier, go live, residual risk |
| Quality or compliance | Acceptance criteria, thresholds, golden set, corpus scope |
| Document and process owners | Corpus membership; evaluation scoring |
| Data and AI engineering | Ingestion, retrieval, orchestration, harness |
| Platform and infrastructure | Hosting, store, gateway, identity, backup |
| Information security | Threat model, residency, pen testing, vendor review |
| Regulatory affairs or legal | Regulatory watch, AI Act, contracts, evidence mapping |
| Training | Role based training; competence records |
Building an AI Strategy You Can DefendCore reading list
The primary sources that underpin this paper. Work from the official documents rather than secondary commentary, and check the status of the draft EU GMP texts, which may have progressed since October 2026.
Official URLs to be added at publication from the publisher sites listed.
Building an AI Strategy You Can DefendIf your organisation is developing an AI strategy, planning a first pilot, or needs to bring existing AI use under governance before an inspection, AiVACO can help in four ways.
AI readiness assessment
A structured discovery and classification exercise producing a consequence tiered inventory and a gap analysis against the checklist in this paper.
Assurance pilot
A 90 day programme that validates the Assurance Layer for one use case and leaves your team with a repeatable method and an evidence pack.
RAG deployment support
Architecture, build or vendor selection support for a governed retrieval system following the twelve steps in the technical annex, with the Assurance Layer at the guardrail tier.
Capability building
Role based training for Quality, IT and process owners on intended use, model types, testing expectations and inspection readiness.
AiVACO is a sovereign AI assurance platform for regulated industries. It provides a deterministic assurance layer that grounds, validates, risk tiers, records and releases AI outputs so that organisations in pharmaceutical, life sciences and other regulated sectors can adopt generative and machine learning AI with evidence they can defend. Its regulatory rule sets are authored by practising pharmaceutical quality specialists, and its architecture is designed around draft EU GMP Annex 22, revised Annex 11, the EU AI Act and the FDA credibility framework. AiVACO's compliance language describes its engagement with the regulatory frameworks; it does not claim regulatory endorsement.
Throughout this paper, phrases such as "aligned to", "engages with" and "designed around" describe how the platform relates to a regulatory framework. They are deliberate. No regulator or industry body endorses AiVACO or any other AI assurance platform, and responsibility for approving an AI use remains with the deploying organisation's Quality, regulatory and compliance functions. What AiVACO provides is the mechanism by which that approval is enforced and evidenced.
Contact AiVACO to arrange an initial scoping discussion on your AI strategy or pilot. AI-Validated & Compliant.