Okay. It's 3 p.m. Good afternoon. I'm Dr. Patricia Baltazar. I'm an assistant professor of abdominal imaging and imaging informatics at Emory University. And today I'll be moderating the session, The Report of the Future, Interactive Multimedia Reporting. And I just would like to welcome you with a little challenge, just to, as a proof of concept of why we want to engage in multimedia reporting. So this is a description of something that I would like you to try to identify. And as much as I would like to make this radiology relevant, it's not a radiology finding. So let me just try to challenge you to figure out what this is. It's a large, partially obscured, well-circumscribed spherical solid mass with heterogeneous, predominantly dark gray intensity and a radius of approximately 1,700 kilometers. Any guesses? That's a good guess. So what the technique is this spot image acquired using a five-inch reflector celestron with 60 millimeter objective. And as someone guessed, this is a crescent moon. And the credits go to Les Folio for the picture that he took himself. And the concept idea is from Chris Roth. Hello, my name is David Kwan. I am a co-chair of the Hamsim Enterprise Imaging Interactive Multimedia Reporting Technical Working Group. Apologies for not being able to speak with you in person today. Luminaries have presented a vision for the next generation enterprise imaging reporting for some time now. But as you saw from Les' talk, the next generation imaging reporting has arrived. There are solutions that offer this technology and there are sites that are generating IMRs. IMRs contain very rich clinical content in imaging reports and they cover a wide scope of functionality, which brings new challenges to deployment and at times challenges to define it specifically. I am here to speak with you about the challenges faced by this emerging form of reporting, along with some of the current work that's being undertaken by our community to address these challenges. Here in my session objectives, firstly, we're going to cover the challenges faced by implementations of interactive multimedia reporters from a technical perspective. And secondly, talk about some of the work that is underway to address these technical challenges of creating and distributing IMRs. Namely, I'm going to speak about the IHE IMR technical profile that has been developed over the next six months. Let's delve into this, into the challenges of operating IMR. We're going to look at two main areas. IMR authoring or report creation and sharing of IMRs. We're going to speak more on sharing IMRs and I'll let you know why in the following slides. Quickly go over the IMR workflow, looking at three specific areas of significance that contribute to IMRs challenges, as well as how it differs from traditional image reporting. There are numerous content contributors that exist today. Some are automated, like the RISs, EMRs, and some modalities where the content automatically flows into the PACS workstation and onto the report creator. But rich clinical content, not so much automated. And I'm talking to a significant portion of the image and modality content and AI results. I'd also like to draw your attention to the second area, which are, again, differs from traditional image reporting in that from an IMR report, a previous report, you can actually access directly the images from the PACS for UVNA. So that differs. And it also offers the benefit of a more streamlined and quick workflow. And thirdly, and this has been existing for a while, no longer are there just traditional. We always think of who are our users and we think about, you know, clinical clinicians and providers, some high volume users like surgeons and the patient themselves. But we have to keep in mind that other readers significantly depend on imaging reports, such as tumor boards, public health, research, even in monitoring the therapy and management of patients, as well as AI now. So the reporting, this is the current landscape for IMR, the reporting process. Note the communications between actors with dotted lines. These are essentially manual. These lines of communication are either manual or operated entry into target systems. In this case, it's the reporting application and sometimes through proprietary communications. I want to draw your attention to a second area within the reporting process, and that is sharing of these reports. And that the reports, once we're trying to distribute them, they're actually transformed or shredded. And I'll go into that in the following slides a little bit. First, the first major challenge, which is the segregation of reports and images. In other words, the lack of integration between a PACS and the report creator in most PACS today. With the IMR, the required real-time communication between PACS and report creator or synchronization of image content between these systems are something that we're well aware of. In most IMR solutions, there's a very tight synchronization, and it offers tremendous value. A lot of this will be covered by Seth in his talk, his following talk, and I'll leave that to him to get into. Second major challenge is faced by IMR operators today is really a persistence and access to IMR content for the report creator. Presently, any rich content contained in image reports transmitted through the RIS or transported to or through the RIS, HIS, EMRs is really reduced down to text, simple text. We see this with IMRs as well as synoptic reports, where a version of the rich text report is stripped, or in this case, it's actually converted to a dumb version or it's shredded down to a dumb version in HL7 version 2 as text only. And the reason for this conversion is it's due to the transformation when it goes through HIS, RIS, and EMRs to OBX segments for consumption by those systems and downstream. Not sure who attributed the term HL7 shredder, but Seth Berkowitz introduced this to the IMR community. For review, these are the challenges faced by IMR implementers. And a significant impact on one I didn't touch on is security and authorization and the impact as on IMR readers. Has significant impact, especially to readers outside the originating organization. The IHE radiology domain is undertaking development of an IHE-IMR technical profile to begin addressing some of these challenges. The intent of developing the IHE-IMR technical profile will be to evolve the interoperability of IMRs and to facilitate the sharing of the rich content to a wider audience. This shows the technical approach that we're taking in our technical profile. As you can see, it names the actors and the transactions. Some of these transactions exist today and others we will be starting to formulate during the technical profile. It will identify and document new transactions, leverage and expand existing transactions and standards used into two components of this profile. Firstly, the IMR author and report creation. This is defining the real-time communication between image display and report creator. Secondly, sharing of IMRs, communication of the multimedia report from the report creator to the report reader. We will strive to define several payload contents within the profile. The profile will specify using fire diagnostic report and fire for transport. We will leverage the deficient definition of fire cast for IMR in the first component and invoke image display in order to support for IMR. We will need to enhance IHE-ID profile to identify and address any missing transactions. Year one of this profile will focus on the second component, which is communications of the multimedia report to the report creator and from the report creator to the report reader of included data points. Year one will really scope down those data points to the image reference for hyperlink authoring and perhaps, given time, we could also tackle measurements as well. The importance of defining fire diagnostic report today in terms of static reporting, it's really HL7 version 2. IMR operators are pushing industry to look at other report formats. As you can see from all the ones that are contention, this is actually a risk. We are looking at the more formats we will have, we will have a harder time with interoperability and achieving interoperability and unfettered sharing between systems. We are looking to fire to help us do this and define use of fire resources as a container and components for the IMR. Components we are looking at is fire diagnostic report, fire observation and fire imaging study. The IMR profile proposes to use these components or these resources of fire. This aligns with several other emerging IHE profiles as well. The technical profile, the IMR technical profile will leverage and extend existing IHE profiles and existing standards such as work flow, image display, IHE web, image access, as well as CNAR, which is simple image numeric for leveraging their ability to do measurements into a report. The IHE radiology domain has scoped in year one of this profile to focus on the second major component of the IMR profile. We scoped it down to image reference and perhaps measurements. Ending. I wish to thank you. This is my contact email presented here. If you have any questions, please don't hesitate to reach out to me. On behalf of the IMR community, thank you for your interest and attendance. Greetings. My name is Seth Berkowitz. I'm an interventional radiologist and clinical informatician at Beth Israel Dickens Medical Center in Boston, Massachusetts. And today I'm going to talk about interactive multimedia report creation. So I'm going to review some of the barriers to creating and distributing multimedia reports. And I'm going to discuss different approaches to multimedia report creation. I had the honor of learning about the art of radiology from Ferris Hall. And one thing that he wrote in 2009 is that the radiology report, now entering its second century of existence, is the major product of our specialty. And like all products, it must change and evolve over time. Les provided a very nice and compelling vision of the report of the future. And perhaps you were wondering while listening to that, why can't I do this in my own system? Why can't I have hyperlinks in my clinical report that bring my report readers directly to the image of interest? Why am I dictating measurements in my system when those should be automatically imported from the PACS? Why can't these quantitative observations be graphed or grouped into tables? And so in understanding some of the barriers to creating this, I like to think about the reporting life cycle. We use an image display system to look at our images and make measurements and annotations. And we use a report authoring system to usually dictate our reports. And perhaps these are one or two parts of the same system, or perhaps they are different systems. Then this report needs to get transmitted to the electronic medical record. And there our clinicians will view these reports and then pull up an enterprise viewer and try to connect our reports to the images that we're describing. And in IMR, all of these steps are broken. Oftentimes, we have a segregated workflow between image display and report authoring systems. Even when we can create interactive multimedia reports, our electronic medical records can't ingest them. And even when our clinicians are able to see these reports, our image display systems don't support image-based links back to the enterprise viewer. So as David discussed, the IHE Profile is defining a report container and transport mechanism to help hopefully pave the way for interoperable transmission of reports to electronic medical records. The IHE Profile is also expanding the Invoke Image Display protocol for image-level viewer launching. How does this work? However, we still have many challenges ahead of us for interoperable, vendor-neutral, IMR report authoring. Again, we have a segregated workflow where we have image display systems and report authoring tools. And the radiologist is doing the same thing in both systems. And the image display systems are creating measurements and drawing annotations. And the report authoring tool, they're dictating measurements and imaging features. In the image display system, they create key images. And in the report authoring tool, they try to refer to these key images. And it's not because of a lack of standards. The DICOM standard defines presentation states, which are often used in our clinical PAC systems for encoding graphical representations of vector graphics, as well as other presentation states, as well as the DICOM structured reporting protocol, which allows basic annotations in addition to encoding the semantic meaning of those annotations. Key images can be referred to by the DICOM unique identifier. And that UID can be contained in a DICOM key object selection object. DICOM also specifies even more complicated structures, including structure reports that are compatible with the annotation image markup standard, DICOM segmentation objects, or even rich radiotherapy structure sets that can provide organ-level segmentation. However, with all of these increased level of detail require more effort for the clinician to create these annotations. And this is one of the key challenges, is that creating semantic annotations, annotations where we have a graphical object connected with its meaning, is very cumbersome in clinical workflow. Even when we do create these annotations, most PAC systems create these evidence objects at the end of a session in order to capture all the data points created by the radiologist in one object. So our approach has been trying to reassemble an interactive multimedia report from the data that already exists without requiring any change in the radiologist workflow. The images are retrieved using DICOM QR. Annotations are retrieved using either GSPS objects or for PAC systems that don't support that via proprietary interface. The free text report is processed using natural language processing, and the user can launch our IMR web application from a study context URL from the EMR. The result is a web application that represents a fusion of the clinical report and the enterprise viewer. The references that are made in the report are parsed and become hyperlinks to the image of interest. The image of interest is presented in a scrollable stack so that the referring provider can see the image along with the associated adjacent anatomy. And as a reminder, all this is done completely transparent to the dictating radiologist. The dictating radiologist continues to use the same report authoring and the same PAC systems to continue to work in the same way without adding any additional burden to their workflow. However, there are several challenges in this tool. There can be ambiguity in image and series reference. Series identifiers may be long for some modalities. We do require some degree of radiology compliance with image series format. And there are challenges distinguishing between instance and series reference and frame and multi-frame reference. A better solution would be a standards-based context sharing system of instance UIDs between image display and reporting systems. This can be done in several ways. Single vendor systems already exist where context is shared smoothly between the reporting portion of the software and the image display portion. Proprietary interfaces have been built. However, we envision a standard-based communication between image display systems and reporting systems. There are two main architectures that could be used. One is peer-to-peer, which is very similar to the way that our dictation and PAC systems stay in synchronization today. An alternative model is a publication subscribe model in which an orchestrator system manages the messages from a publisher and a subscriber. Firecast is one such publication subscribe protocol for system synchronization. It uses the FHIR standard for authorization as well as a format for the message payload. The first use case is exam-level synchronization between image viewing and dictation systems. But we envision a much more granular context sharing system in which individual image instance UIDs could be transmitted between PACs and a reporting system, as well as individual annotations and measurements. Thinking more broadly, such a real-time context sharing system could enable brand new AI workflows. Imagine a radiologist measures a lesion and that image reference is transmitted to a reporting tool in an AI. That AI might further process the image to produce a semantic label and further quantitative information on the measured lesion and then publish that back to the reporting tool and the PACs. This type of real-time workflow orchestration might bring us to that human-machine collaboration that many of us are envisioning. There's been explosive growth in AI tools that are not integrated into our reporting workflow. Even when they are integrated into our reporting workflow, the provenance of AI results is lost in plain text reports. IMR creation workflow preserves the provenance of AI results and then links the report readers back to the source images that were used when either the radiologist or the augmented system made those findings. I'll remind us of the value cycle that David presented. Reports are created both by image-centric specialists like radiologists and in the future increasingly augmented by AI systems. We desperately need a standards-based context sharing system so that we can create rich interactive multimedia reports that will encode hyperlinks so that our consumers clearly understand the meaning of our words and are able to bring up images in the context of our semantic descriptions. These are not new ideas but we think the time has arrived to define a standards-based cross-vendor workflow for creating interactive multimedia reports. The enhanced value of these annotated images is likely to be so great that referring physicians and patients will demand that they continue to be a part of radiology reports. you

interactive multimedia reporting

radiology

interoperability

AI-enhanced workflows

IHE technical profile

standards-based context sharing