Town Hall: ARPA-H PRECISE AI Program | October 17, 2024
The One-U RAI, DELPHI Initiative, and DHI invite you to join our funding townhall for ARPA-H’s PRECISE-AI: Performance and Reliability Evaluation for Continuous Modifications and Useability of Artificial Intelligence program. Information about this opportunity is provided below.
Led by One-U RAI, DELPHI, and DHI, the goal of the town hall is to provide an overview of the mechanism, answer questions from the UofU Research community, and provide space for interested parties to network and discuss potential submission(s) from the UofU. The Townhall will be held on Thursday, October 17 from 3:00 – 4:00 PM on Microsoft Teams. Registration is not required. Meeting details are provided below.
Town Hall Details:
Date: Thursday, October 17, 2024
Time: 3:00 – 4:00 PM
Meeting Link: Join the meeting now
Meeting ID: 226 178 879 481
Passcode: czMVsg
Learn more about our UofU Initiatives:
- One-U Responsible AI Initiative (One-U RAI)
- Data Exploration and Learning for Precision Health Intelligence (DELPHI) Initiative
- Digital Health Initiative (DHI)
---
ARPA-H PRECISE-AI
The Big Question What if AI models in health care autocorrected to maintain peak clinical performance?
The Problem Artificial Intelligence (AI) is becoming an increasingly important tool used to help support clinical decision making. Since 2018, the number of available AI-enabled medical devices in the U.S. has increased by tenfold and will likely continue growing at similar rates in the future. However, research suggests that the accuracy of Machine Learning (ML) models may degrade over time due to changes in input data – such as changes in clinical operations, data acquisition, patient population, or even IT infrastructure. The accuracy of AI models in health care is paramount, as an inaccurate output could have dire consequences for a patient’s health outcome and the efficacy of our health system.
The Current State Despite these issues, no current clinical AI models receive regular testing during clinical use to ensure that the accuracy of output is maintained. There are also no requirements to update AI models whose performance have degraded, in part because of a lack of technical solutions. Today, the main method of detecting degradation within AI models is clinical intuition on the part of physician using the technology. However, relying on clinical intuition can be unreliable and highly variable, meaning that AI model degradation may have already caused misdiagnosis before it is noticed.
The Challenge To address these issues, the Performance and Reliability Evaluation for Continuous Modifications and Useability of Artificial Intelligence (PRECISE-AI) program aims to develop capabilities that can automatically detect and mitigate AI model degradation. These tools will monitor the performance of clinical AI models, identify if a degradation has occurred, and provide capabilities that can correct for performance degradations without the need for human oversight, thereby reducing the burden on individual operators. Importantly, this technology will also communicate clear and actionable information about the sources of degradation and allow users to better interpret model uncertainty, and thus help them use their software more effectively.