The Role of AI in Regulatory Decision-Making for Drugs & Biologics: the FDA’s Latest Guidance

On January 7, 2025, the Food and Drug Administration (FDA) issued a draft guidance entitled “Considerations for the Use of Artificial Intelligence (AI) To Support Regulatory Decision-Making for Drug and Biological Products.” [FDA-2024-D-4689] This long-awaited guidance provides recommendations to the industry and other interested parties on the use of artificial intelligence (AI) to generate information or data to support regulatory decision making. The draft guidance specifically focuses on the use of AI to support establishing the safety, effectiveness, and quality of drugs using a risk-based credibility assessment framework for a particular context of use (COU). The draft guidance does not address the use of AI in areas such as drug discovery or operational efficiencies in drug development since these generally do not impact participant safety, the reliability of results from clinical or non-clinical studies, or drug quality. This blog post aims to provide an overview of the key points from the FDA’s draft guidance and discuss its implications for the industry.

Defining AI and Its Role in Drug Development

AI is defined in the guidance as “a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments.” It also notes that “AI systems (1) use machine- and human-based inputs to perceive real and virtual environments, (2) abstract such perceptions into models through analysis in an automated manner, and (3) use model inference to formulate options for information or action.” The guidance states that machine learning (ML), one of the subsets of AI, is most used in drug development, but that the guidance is not intended to be limited to those techniques.

AI may contribute to information to support the safety, efficacy or quality of drugs by (1) decreasing the number of nonclinical pharmacokinetic (PK), pharmacodynamic (PD), and toxicologic studies, (2) utilizing predictive modeling for clinical PK and/or exposure-response analyses, (3) combining various databases (natural history, genetic, registries, social media, clinical trials, etc.,) to help better understand disease presentations, disease progression, disease differences, and predictors of progression, (4) utilizing existing real-world data or data from digital health technologies to develop clinical trial endpoints or assessments of outcomes, (5) handling reports on post marketing adverse drug experience information, and (6) helping select appropriate drug manufacturing conditions.

Challenges and Concerns with AI Implementation

There are challenges to using AI in drug development. Because of the variability of datasets used to train AI models, there is concern about the reliability of the results and biases that might be generated depending on the data source. Emphasis is placed on the data being “fit for use” or relevant and reliable. Additionally, the guidance notes that methodological transparency in the development of AI models may be necessary, specifically outlining in the regulatory submission how a particular AI model was developed. The potential for an AI model’s performance to change over time (i.e., data drift) is highlighted and requires life cycle maintenance of AI models.

Steps for Establishing a Credibility Assessment Framework

The risk-based credibility assessment framework is outlined in the guidance and is a stepwise approach which includes defining the question of interest, defining the COU for the AI model, assessing the AI model risk, developing a plan to establish the credibility of the AI model as it pertains to the COU, executing the plan, documenting the results of the credibility assessment plan and any deviations, and determining the adequacy of the AI model for the COU. The guidance includes two examples to illustrate the implementation of the first three steps of the framework, one in clinical development and one in manufacturing, noting the remaining steps are intended to provide a general list of activities to be considered when establishing an AI model’s credibility and cannot be easily applied to the examples.

What is clear from the examples, is that having a “human in the loop” is important to reduce risk. When assessing risk, there are two main components. The first is the AI model’s influence to answer the question of interest, or what is the contribution of the model relative to other evidence . The second is the decision consequence, or what is the significance of any adverse outcome if use of the model results in an incorrect decision. In the clinical development example, the AI model is the sole determinant of whether a participant undergoes more rigorous monitoring, making it possible that if the model is wrong, that some participants will not be adequately monitored and may have a significant adverse outcome. The model influence is high, and this model is considered high risk. But in the case of the manufacturing example used in the draft guidance, where AI is used to direct vial fill volume, the vial fill point is batch checked by a human, mitigating the risk of the vials being filled incorrectly and contributing to dosing errors. The risk of using the AI model in this example is considered medium since although the consequences of a dosing error could be high, the model’s influence is low.

The next steps involve establishing a credibility assessment plan which is contingent on the model risk and the COU. The guidance urges sponsors and developers to engage early with the FDA to discuss credibility assessment activities. High-risk AI models will necessitate that more detailed information be submitted. The FDA will require that the model and model development process be described, including model inputs and outputs, model architecture (what type of model, e.g., neural network), model features and model parameters. The data used to train and tune the model in the development phase, prior to testing the model,  should also be described. Sponsors should include information on whether the data is fit for the COU, how the data is relevant and reliable, and whether federated learning is utilized.

The submission should also describe how the model was trained, the learning methodology used to develop the model, performance metrics used to evaluate the model, whether a pre-trained or multiple pre-trained models were used, and what calibration and quality control procedures are in place. The limitations of the modeling approach, including any potential biases,  should be described. After the credibility assessment plan is established, the plan should be executed, and a report should be generated to establish the credibility of the AI model for the COU. If there are deviations encountered with execution of the plan, these should be described. Depending on what is pre-agreed upon with the FDA, the report may be submitted to the agency or held and made available to the FDA on request.

If after executing the plan the model credibility is not adequate to support the model risk, then there are several actions that the sponsor may take: 1) Additional evidence can be incorporated to answer the question of interest in addition to the evidence generated from the AI model. This will decrease the model’s influence and lower the risk. 2) The sponsor can add additional development data or increase the rigor of the credibility assessment. 3) The sponsor can add additional controls to mitigate risk. 4) The modeling approach can be changed. 5) The sponsor may determine that the AI model’s output is inadequate for the COU and the model’s COU could be revised or refined.

Plans for life cycle maintenance of the AI model should be in place to monitor and ensure the suitability of the model’s performance over time within the COU. Although at the point of regulatory decision making the data used to inform the model is likely locked, there are models that will continue to be utilized during the drug development life cycle. Some models are self-evolving and can adapt and change without human interaction. It is important that these models be evaluated from time to time to ensure that they are “fit for use” for the COU and that risk is assessed. If necessary, changes may need to be made in the model or with the use of the model to mitigate risk and the credibility assessment plan may need to be re-applied and modified. Plans for life cycle maintenance should be included in submissions to the FDA, as appropriate.

Importance of Early Engagement with the FDA

As noted earlier, the FDA stresses the importance of early engagement with the agency when a drug development plan includes utilizing an AI model. This is important to “(1) set expectations regarding the appropriate credibility assessment activities for the proposed model based on model risk and COU and (2) help identify potential challenges and how such challenges may be addressed.” In addition to formal meetings with the agency, the guidance includes a list of other options to engage with the FDA. The path chosen will likely depend on the AI model and the COU.

In the Federal Register Notice for the guidance the FDA notes that the risk-based credibility assessment framework is informed by: “(1) over 800 comments received on the 2023 discussion papers published by CDER [Center for Drug Evaluation and Research] entitled “Using Artificial Intelligence & Machine Learning in the Development of Drug & Biological Products” and “Artificial Intelligence in Drug Manufacturing”; (2) [the] FDA’s experience with reviewing over 300 submissions with AI and machine learning components across all phases of the drug development process; and (3) current regulatory science research.” Noting that this is a rapidly evolving field, the FDA asks for specific feedback on how well the risk-based credibility assessment framework aligns with the experience of the industry and if the options for engagement with the FDA on AI are adequate.

Conclusion

The FDA’s draft guidance detailing considerations for using AI to support regulatory decision-making for drugs and biological products adopts a risk-based credibility assessment framework and focuses on AI applications for ensuring drug safety, effectiveness, and quality. It outlines definitions, potential benefits, challenges, and the necessity of methodological transparency and life cycle maintenance of AI models. The guidance emphasizes early engagement with the FDA, describing a stepwise approach to establishing AI model credibility for specific contexts of use and highlighting the importance of human oversight. The document also seeks public comment on its alignment with industry experience and sufficiency of engagement options with the FDA.

For ethical review and regulatory inquiries, contact WCG’s IRB experts by completing the form below. Our team, comprising of nearly 200 board members, is prepared to provide guidance. Complete the form today for a free consultation.