AI, Quality, and Accountability: Six Questions Every Quality Leader Should Ask 

You can delegate the task. You cannot delegate the accountability 

Artificial intelligence is becoming an integral part of clinical development, supporting everything from protocol optimization and site selection to data management and medical writing. According to ACRO’s 2026 State of the Industry Report, 71% of participating member companies reported using AI for strategic study feasibility and site selection and activation, while 64% reported its use in protocol optimization and data management. 

However, not all AI applications carry the same level of risk. Using AI to draft an internal document is fundamentally different from relying on it to interpret clinical data, identify potential safety signals, or support decisions that may affect trial participants. 

In regulated clinical research, the consequences of an AI error can extend far beyond an inaccurate output. They may affect patient safety, compromise data integrity, influence critical decisions, or undermine the reliability of trial results. And as AI systems increasingly depend on external vendors, cloud infrastructure, and third-party models, determining who is responsible for their performance becomes more complex. 

We asked Cromos Pharma’s Head of Quality, Joris De Maeseneire, six questions about how AI is reshaping Quality and what organizations should consider as these technologies become more deeply embedded in regulated clinical research. 

How are recent FDA, EMA, and ICH guidelines shaping AI governance in clinical trials? 

The regulatory framework is becoming clearer. In January 2025, the FDA issued draft guidance on AI in regulatory decision-making, introducing a risk-based credibility framework centered on context of use. In January 2026, the FDA and EMA followed with ten Guiding Principles of Good AI Practice in Drug Development, covering risk-based approaches, data governance, human-centric design, performance assessment, and lifecycle management. 

At the same time, ICH E6(R3) reinforces quality by design, proportionate risk management, data governance, computerized system controls, and sponsor oversight. Its Principles and Annex 1 became effective in the EU in July 2025.  

What I take from these frameworks is a consistent direction: The potential use of AI in the drug product life cycle is broad and rapidly evolving. The level of control should reflect its context of use, the potential consequences of failure, and the risk to participants and trial reliability.

With so many functions involved in AI governance, who is ultimately responsible?

AI governance is inherently cross-functional. Quality, Clinical Operations, Data Management, IT, Legal, Privacy, Information Security, and technology vendors may all have legitimate roles. 

But when many functions participate, there is a real risk that accountability becomes fragmented. 

What matters to me is that for every AI-supported regulated process, organizations can clearly answer some basic questions: Who owns the process? Who accepts the risk? Who approves and monitors the technology? And who has the authority to intervene or suspend its use if concerns emerge? 

The objective is not to centralize every decision within Quality. It is to ensure that ownership, risks, controls, and escalation pathways are clearly defined, and that accountability does not disappear simply because responsibility is distributed across multiple functions.

Does every AI system used in clinical research need the same level of validation?

No. I would not apply the same validation approach to every AI system. An AI tool summarizing meeting notes should not be governed in the same way as one interpreting clinical data or contributing to safety assessment. 

The first questions I would ask are: What is the AI intended to do? What happens if it fails? What data does it access? Could an error affect participants or trial reliability? And what evidence demonstrates adequate performance for that specific use? 

Those answers should determine the level of control. 

This is also consistent with the FDA/EMA risk-based approach. Validation and oversight should be proportionate to the context of use and model risk. 

Ultimately, the goal is to apply controls that are proportionate to the specific risk and context of use.

Is having a “human in the loop” enough to ensure meaningful oversight?

Not necessarily. 

When I hear that an AI system has a “human in the loop,” my next question is what that human can actually see, understand, and challenge. 

Consider a system that reviews 1,000 records and flags 20 for further review. A qualified professional may carefully review all 20. But that does not establish that the other 980 were correctly excluded. 

That distinction is important because simply placing a person at the end of an automated workflow does not automatically make the workflow controlled. 

In my view, meaningful human oversight means that the reviewer has sufficient expertise and information to detect potential failures, can challenge the system’s output, and has the authority to override it when necessary. 

Effective oversight is not measured by the presence of a human reviewer. It is measured by that reviewer’s ability to identify errors, challenge outputs, and intervene when necessary.

If an AI system has already been validated, why does it still need ongoing Quality oversight?

I would not consider validation the end of the Quality process. Unlike many traditional computerized systems, AI-enabled solutions may continuously evolve through changes in data, usage patterns, configurations, or vendor updates. Data may shift, workflows may evolve, configurations may change, and vendors may update underlying models. An AI system that performed adequately when it was introduced may not necessarily perform in exactly the same way six months or a year later. 

The FDA/EMA Good AI Practice principles address lifecycle management, including performance monitoring, periodic re-evaluation, and data drift. 

From a Quality perspective, what I would want to see is evidence that the system continues to perform as intended throughout its use, not simply evidence that it worked at the point of validation. 

That means organizations also need to define what constitutes a meaningful change, how performance will be monitored, and what should trigger reassessment or revalidation. 

What should teams establish before AI enters a regulated clinical trial workflow?

Many of the necessary principles already exist within GCP, computerized system validation, data integrity, privacy, vendor oversight, risk management, and change control. AI changes how these principles need to be applied. 

Before an AI-supported system enters a regulated workflow, several fundamentals should be clear: 

• Context of use: What is the system permitted to do, and how will its output be used? 

• Risk: What happens if the system is wrong, and could that affect participant safety, trial conduct, data integrity, or reliability of results? 

• Data: What information does the system access, where is it processed, and could it be retained or reused? 

• Human oversight: Does the reviewer have the expertise, information, and authority required to meaningfully challenge the system? 

• Traceability: Can AI-supported decisions be reconstructed across inputs, outputs, model versions, human intervention, and final decisions? 

• Change control: What constitutes a significant change, and when does a model, configuration, data source, or vendor update require reassessment or revalidation? 

• Lifecycle monitoring: How will drift, unexpected behavior, issues, and declining performance be detected and addressed? 

• Avoid bias: Ensure that the data on which decisions are based are representative of the entire population (or the population to which the study applies) 

If an organization cannot clearly define these fundamentals, it may be adopting AI, but it is not yet governing it. 

The Bottom Line 

AI has enormous potential to reduce repetitive work, identify patterns humans may miss, accelerate analysis, and make increasingly complex clinical trials more manageable. The objective is not to prevent its use, but to establish clear boundaries around it. 

Quality in the age of AI is not about resisting innovation. It is about ensuring that innovation remains transparent, controlled, and accountable throughout its lifecycle. 

At Cromos Pharma, this is the perspective we bring to emerging technologies in clinical operations: innovation should be enabled by a quality framework strong enough to support it. 

AI may change the tools used to run clinical trials. It does not change what Quality is ultimately there to protect: trial participants, data integrity, and confidence in the decisions built on that data. 

Planning a clinical trial? Contact our team at inquiry@cromospharma.com to discuss how our quality-driven approach can support your development program.