Best Practices for AI Model Selection and Deployment (2025)-W1-CIN18-2025

Video Transcription

Video Summary

The session, moderated by Heisa Na (Stanford), focused on practical guidance for evaluating and integrating AI in radiology. David Larson (Stanford/ACR) emphasized that FDA-cleared imaging AI is not “plug-and-play”: it is both an IT and clinical tool requiring local governance, acceptance testing, defined end-user qualifications, continuous monitoring, and stoppage rules. He outlined ACR’s path from the ARCH-AI recognition program toward formal accreditation, supported by new practice parameters and the ASSESS-AI program, which benchmarks local AI concordance via ACR Connect and automated report–inference comparison.<br /><br />Sreejesh Krishnan (Radiology Partners) discussed vendor partnership as iterative co-development: validate performance on local prevalence, prioritize clinically meaningful metrics like PPV, share specific false-positive/negative examples, and set realistic expectations for radiologists. He described improving a pneumothorax model through feedback cycles until PPV rose and complementary detection emerged.<br /><br />Shlomit Goldberg-Stein (Northwell) described customized workflow orchestration, hard-coded alerting rules to reduce fatigue, SafeSign tools at report signature for AI visibility and data capture, LLM-assisted resident feedback and navigation, and operational dashboards for downtime and delayed-result safety alerts.<br /><br />Leo Bincourt (University Hospitals) explained implementation complexity and argued for intentional strategy: decide when to use standalones versus platforms/marketplaces, connect research/innovation pipelines to production, and ensure radiologists help lead adoption.

Keywords

AI in radiology implementation

FDA-cleared imaging AI governance

ACR ARCH-AI accreditation

ASSESS-AI concordance benchmarking

Acceptance testing and continuous monitoring

Vendor partnership and co-development

Pneumothorax detection model PPV improvement

Workflow orchestration and alert fatigue reduction

Radiology AI platforms vs standalone tools strategy