Artificial intelligence is increasingly being developed and deployed across medical imaging, but evaluating whether an AI system is accurate, reliable, generalizable, and clinically meaningful remains challenging. Benchmarking provides a structured approach to AI evaluation, yet not all benchmarks answer the same question. A benchmark composed of rare, difficult, or “board-style” cases may reveal model limitations and failure modes, while a high-volume benchmark of routine clinical cases may better estimate clinical performance and workflow impact.
This webinar will introduce how AI benchmarks are designed, interpreted, and applied in radiology. Topics will include building benchmark datasets, selecting appropriate reference standards, defining clinically relevant tasks, interpreting performance metrics, identifying bias and dataset limitations, and evaluating generalizability across patient populations, imaging equipment, institutions, and practice settings. Faculty will discuss how benchmark design should align with intended use, clinical evidence, and imaging guidelines. Ongoing RSNA benchmarking efforts will also be described.
The session will also address the rapidly evolving area of radiology foundation models. As models expand beyond single-task applications to multimodal, multi-disease, and general-purpose imaging systems, the need for standardized and clinically meaningful benchmarks becomes increasingly important. Faculty will review the current state of foundation model benchmarking in radiology, including challenges related to task definition, dataset representativeness, multimodal inputs, reporting quality, reproducibility, external validation, and post-deployment monitoring.
Artificial intelligence is increasingly being developed and deployed across medical imaging, but evaluating whether an AI system is accurate, reliable, generalizable, and clinically meaningful remains challenging. Benchmarking provides a structured approach to AI evaluation, yet not all benchmarks answer the same question. A benchmark composed of rare, difficult, or “board-style” cases may reveal model limitations and failure modes, while a high-volume benchmark of routine clinical cases may better estimate clinical performance and workflow impact.
This webinar will introduce how AI benchmarks are designed, interpreted, and applied in radiology. Topics will include building benchmark datasets, selecting appropriate reference standards, defining clinically relevant tasks, interpreting performance metrics, identifying bias and dataset limitations, and evaluating generalizability across patient populations, imaging equipment, institutions, and practice settings. Faculty will discuss how benchmark design should align with intended use, clinical evidence, and imaging guidelines. Ongoing RSNA benchmarking efforts will also be described.
The session will also address the rapidly evolving area of radiology foundation models. As models expand beyond single-task applications to multimodal, multi-disease, and general-purpose imaging systems, the need for standardized and clinically meaningful benchmarks becomes increasingly important. Faculty will review the current state of foundation model benchmarking in radiology, including challenges related to task definition, dataset representativeness, multimodal inputs, reporting quality, reproducibility, external validation, and post-deployment monitoring.
Content Areas (Codes):
The following Content Areas will be printed on the certificate for this course:
- Artificial Intelligence
- Informatics
Learning Objectives:
- Describe the role of benchmarking in evaluating AI performance, reliability, generalizability, and clinical value in radiology.
- Explain key considerations in building benchmark datasets and reference standards for AI in medical imaging, including task definition, data representativeness, annotation quality, ground truth selection, and dataset composition.
- Interpret common AI performance metrics in the context of clinical evidence, imaging guidelines, disease prevalence, workflow integration, and intended use.
- Identify sources of bias, dataset limitations, and generalizability challenges that may affect AI performance across different patient populations, imaging equipment, institutions, and practice environments.
- Discuss emerging challenges in standardizing benchmarks for radiology foundation models, including multimodal evaluation, broad task coverage, reproducibility, external validation, reporting quality, and ongoing performance monitoring.
This Other activity (live and enduring material) is estimated to take 1 hour to complete.
Start Date: 7/29/2026
Online Expiration Date: 7/28/2027
This educational activity was originally presented on 7/29/2026 as an interactive online webinar.
Faculty:
- Maggie Chung, MD
- George L. Shih, MD, MS
Planners:
Price:
Non-Member/Basic Member Rate: $0.00
Standard Member/Full Access Member Rate: $0.00
Refund / Exchange Policy:
RSNA will not issue any refunds or exchanges for online only versions of educational products or activities purchased online. Please review the entire product or activity description prior to purchase.
RSNA Disclaimer:
The opinions or views expressed in this activity are those of the presenters and do not necessarily reflect the opinions, recommendations or endorsement of the RSNA. Participants should critically appraise the information presented and are encouraged to consult appropriate resources for information surrounding any product or device mentioned. Information presented, as well as publications, technologies, products and/or services discussed, are intended to inform the learner about the knowledge, techniques, and experiences of RSNA faculty who are willing to share such information with colleagues. The RSNA disclaims any and all liability for damages to any individual user for all claims which may result from the use of said information, publications, technologies, products and/or services, and events. Courses are best viewed on a desktop computer.