I will be talking on how to publish health equity research. And this graphic is from Red Pen, Blue Pen on the infallibility of evidence. Unfortunately, because of how evidence is generated, racism and all the other isms, it's not a bug, it's actually a feature. It's built into the way that we shape the research, the type of questions that we ask, even the people that we recruit. People who participate in randomized controlled trials are healthier, wealthier, and whiter than the rest of the population. So how do you use, then, evidence that does not have sufficient numbers of persons of color who constitute less than 10% of trial populations, particularly in multi-center clinical trial groups such as ECOG, ACRIN? So it doesn't matter what type of study. We routinely get only about 10% people of color participating. Therefore, increasing minority participation is crucial to generating and applying evidence-based medicine. So as a journal editor, this slide comes from Melissa Simon, who is a health equity researcher at Northwestern and was recently inducted into the IOM. She's also serving on the committee that will be selecting the next JAMA editor. And I share a lot of her perspectives, which is journals are gatekeepers. Now, one of the best aspects of my position as a journal editor is being able to read new science from people who are very excited to be working on what they do. And sometimes the work is great, and sometimes the work is not great. And even when the work is great, it may not necessarily fit the particular direction that the journal is going or fit the particular mission of the journal. So even though we try to put forth the best, in quotes, science, what is the best will vary depending on the perspective of the editor and the reviewers. And most of the publication opportunities, particularly for health equity research, tends to be opinion pieces and briefs or editorials. These pieces are less likely to be cited compared to substantive research articles, which tend to form the majority of submitted and published manuscripts. And certainly, this is the benchmark for things like promotion, and these are the types of articles that tend to attract the most attention. So when there are specific calls for papers, the majority of the health disparities and health equity related pieces tend to be published through opinion pieces, not through standard, quote, unquote, hypothesis-driven research for a variety of reasons. It also limits the time for an individual to create a body of work in health disparities and health equity, because we have a clock. There is a certain expectation of promotions at home institutions, and one of the ways you get promoted is the number of research papers that get published and which publications, quote, unquote, count. So once it actually gets published, what are the key research findings in many of the health disparities and health equity topics? And then, how do you measure impact? All of that takes place against a background of structural inequity, which is often framed by who is appointed to the editorial board to begin with, or who gets selected editor-in-chief, deputy editor, or associate editors, who gets invited to review. So that's a lot of potential barriers. So now let's talk about, how do you actually get published? So these are the next three slides will be tips on getting health equity research published. I want you to think about diversity, equity, and inclusion as one half of the equation. And diversity, equity, and inclusion is inherently inward facing. It's the institution looking inside itself to make sure that it treats its faculty, its allied professionals, and its patients in the appropriate way, decreasing institutionalized racism. It is inward facing. But what we need to do is make that outward facing, because at the end of that arrow is someone's family member. Like, these are my parents. Health equity and social justice is essentially an outward facing activity that is supplemented and supported by diversity, equity, and inclusion efforts, but they are inherently different, and they require inherently different research methodologies. And they ask essentially different questions. What we know about the sources or the epidemiology of diseases, we focus on what we know. So we want to improve health care from the perspective of the care system, because that is what we know. That access to care and care delivery often accounts for only 25% of the variation, 20% to 25% of the variation. So we could be perfect, and we could change the outcome only one fourth of the time. And the rest are governed by individual factors, such as health behaviors, the physical environment itself, food deserts, transportation equity, et cetera, and socioeconomic factors, including the socioeconomic status of the neighborhood, not just of the person living in that neighborhood, because you could be extremely well resourced. But if you live in a food desert or live in an area that doesn't have reliable transformation, your access to care is inherently limited by the neighborhood SES status. Then we get into geographic segregation, the proportion of minority individuals living in particular geographic areas. Or racial isolation, the likelihood that a member of a minority group will encounter another member of a minority group, which has been shown to support cultural identity and resilience. Or distance from city center. Many communities of color are often relegated to the margins of dynamic communities, which means they have potentially lesser quality services or have to travel farther to get to a pharmacy, for example. And now gentrification, formerly marginalized communities being renovated, attracting people with more capital, and re-marginalizing or further marginalizing individuals who had previously lived in that community and had created a community nexus. So on the top, this is an example of how to expand the type of question that you would ask when you're trying to understand health disparities. This graph is a graph of the Bay Area. And the purple is the rates of asthma-related ER visits. And those purple areas, the darker the purple, the more frequent the ER visits. Well, that's great that you're able to describe it. But what's the take-home message? Because it's not as if people can necessarily change where they live. Or we would need to understand, what is it about those areas where you had higher frequencies of ER visits? What's going on before you can actually develop an intervention? Well, someone did map it and correlate it with redlining. So what is redlining? How many of you know what redlining is? Great. Redlining is a practice that was instituted in the 20s, officially retired in the 50s. And this was a way for banks to categorize mortgage risk. And mortgage risk was typically just due to the number of people of color in that community. And D ratings, which fell below the redline, were invariably communities of color, typically black communities. And when you look at a correlation with redlining that stopped in the 50s, you see effects well into the 2000s on health outcomes and health disparities. So social policies have a long temporal reach. And that's one way for you to think about the types of questions that you would ask and how you would expand your understanding of the etiology of some of the disparities that we tend to traditionally ascribe to race. It's no longer sufficient to say there is a race-based difference. Really, in order to get published in a good journal, it would be ideal if you can say something more and unpack the race-based disparity component. So more tips. What you want to do is advance the framework. So this is a study, a TaylorX study. It's a randomized controlled trial where women all received Oncotype DX. And women who had an intermediate score were randomized to chemo, no chemo. And then everyone got endocrine therapy. And they found that if you had an intermediate score, your outcomes were just as good if you didn't do chemo, which is a landmark result. And that was in 2018. In 2021, Albain looked at the same data set and found that controlling for Oncotype DX score for the treatment that you underwent for sociodemographic characteristics, white women lived longer than black women. And they were also less likely to have distal recurrences. Well, so that gets us to there are race-based differences. The next set of papers are trying to understand where the sources of disparities are. The first paper adjusted for neighborhood socioeconomic status. Now you're looking at the built environment of where that woman lived and the type of insurance that they had and showed that, unlike what was expected, black women were more likely to adhere to endocrine therapy. And yet, in the next paper, after adjusting for endocrine therapy use, neighborhood socioeconomic status, and insurance, black women still died younger and earlier than white women. So they already do the right thing by adhering to endocrine therapy. And yet, they still died earlier. Then we looked at, well, what are some of the constituents of why black women were more likely to die earlier? And what we found when we were looking at racial segregation measures, racial isolation in particular, which is something that you can calculate based on one zip code, after adjusting for endocrine therapy, neighborhood socioeconomic status, and insurance, we found that black women who lived in areas with higher racial isolation, meaning their likelihood of running into another black person was low, were more likely to die earlier. So what these series of papers is showing is that each paper drills deeper and deeper into the concept of race-based differences and seeks to explain a framework for how you get the effects that you get that manifest as a race-based outcome gap. And the last word, and this is really stylistic, but I think it also comes from a level of deeper understanding. When you write your methods, you need to say why you collected race and ethnicity, rather than ancestry or zip code, the way the AMA suggests you should. Ideally, you would link this to your theoretical model, or sometimes it's just required by the funding agency. The funding agency wants you to collect race and ethnicity, and that's what you say. You also have to describe how race and ethnicity was identified or categorized. Was it self-report? Did you assign or the investigator assign a race category? Was it from a database or the electronic medical record or a survey instrument, et cetera? And we tend to group people into broad categories because we think there's just not enough people. So Asian-Pacific Islander, Asian-American-Pacific Islander, I would like that expanded to include distinct categorizations of Filipino, Hawaiian, Chinese, Japanese, and other groups that are now under the AAPF umbrella. And one of the more profound examples is that Filipinos are considered Asian. Asian individuals tend to have better outcomes than white individuals for breast cancer. But once you disaggregate it, Filipino women have outcomes that are closer to black women and Hispanic women, Latina women, than to Asian women. And that effect has been masked by being grouped as Asian-American-Pacific Islander. So ultimately, we do the research that we do because we want the right patient to get the right test, the right treatment at the right time and for the right price. And that means having equitable evidence-based care, meaning someone, yourselves, has to publish the evidence. And part of understanding the evidence and teasing apart what parts we can intervene on requires a more nuanced understanding of social determinants of health besides the easy metric of race categorization. Thank you. Thanks for giving me this opportunity to speak. So I'll be speaking about disparity issues from a workforce perspective. So there's a lot of literature that's explored issues around diversity and equity in our patient populations. But there are also additional important issues to explore around diversity and equity of the radiologist workforce providing such care. So I'm going to approach this with a highlight or overview of the work that's been done in this space. Now, as you'll see, it's actually not a whole lot. And these are the themes that I would say it's focused around. So gender, and this is all speaking about the radiologists rather than our patients. So gender, geography, morality, practice patterns such as subspecialty, IR in particular, and modality distributions. And this is really just, I think, a reflection of the available data and what we have to work with. So I'll run through examples. So starting with county level variation. So you can actually get this from publicly available data on the Medicare website, information on all physicians participating in Medicare and get their primary billing address. And in this work, we had categorized each county in terms of the availability of a physician, of a radiologist participating in Medicare with blue, meaning there was at least one generalist. And then the other shade here, they have this greenish more, at least one subspecialist. And there was a lot of counties here, which actually our way didn't have any radiologist billing Medicare, at least for the public records. And maybe there's coverage through teleradiology or they have secondary billing addresses, but we can see large parts of the country with access issues. And we link this up with other data sets that give county characteristics. And there's actually a good amount of data you can find online characterizing counties. And we can look at the ones with and without a local radiologist. And we see differences for all sorts of population characteristics, age, gender, race, and ethnicity, household income, insurance level, employment status. So these are really different groups which have the local access. And we took this further, we normalized our population levels and it really was a population density effect here. So we have generalists and subspecialists per 100,000. And when we color code this, it's really the rurality, which I think is expected, but the industry is actually showing it. But then the more rural areas just don't have a local radiologist. So in this study, we honed in a little more specifically on IR services. And this was interesting. There really was a division when you broke this down as basic and advanced IR services. We did a county level analysis. And for the basic services in many areas, it was the generalists, for general radiologists performing these basic IR procedures. And then when you looked at advanced IR procedures in many areas, there was nobody doing it. There just was, if a patient wanted it, they would have to travel. There just wasn't any local access. So I'm gonna switch gears now and talk a little bit about some upcoming slides on gender variation in the workforce. So in this study, we just basically classified just the percentage of radiologists who were women per Medicare information. And then we color-coded this based on relative to national average. So green was states above the average. Red was below the average. And then we can see some variations. And the more granular you get, the more the variation becomes apparent. So here we did it by county. And even within a state, when you look at adjacent counties, counties right next to each other, they'll flip-flop from being above the average to below the average. And there's gonna be some variations that may even be hard to explain. We linked this with county data. And then there were actually the percentage of women radiologists in the county correlated with various county characteristics, including just the percent of women in the county's population, insurance status, employment status, even just the county's political voting record. So there were some differences that we could bring out. This was a different project where, so if you look at the Medicare data, it only actually classifies radiologists into three groups, diagnostic rads, interventionalists, and nuclear medicine. But we looked at the billed claims to stratify further. And we actually showed gender variation across the sub-specialties. I'm gonna comment a little more on this in a later slide, but IR, and I think we know this, but we see this here on the data, fewer than 10% were women, at least at the time of the study. So Medicare, they also give the year of graduate, medical school graduation for participants. And we could use that as a surrogate of career stage. And in this study, we looked at the impact on career stage on billed claims and the work that radiologists were doing. And we were comparing total RVU and the distribution of that work, basic versus advanced services. And we saw that the late career radiologists, their actual RVU efforts and the distribution of what they were reading, advanced modalities, CT and MR, really was no different from early and mid-career radiologists. And we had a hypothesis, but it wasn't born out. There really was no, until we got, say, about 40, 50 years into somebody's career, it's a really, really late career, whereas there really wasn't any apparent effect. In that study, we did look at the gender distribution of the radiologists in the sample. And as we got to earlier career, we could start seeing a higher percentage of women in the group. So this was an additional study kind of blending the IR and gender data and just focused on gender differences within IR. And we compared for a whole host of factors, procedural complexity, patient complexity, case mix. And there was really no difference between what the male and female interventionists were doing. It was all very similar across a range of things we looked at. The one difference was when we looked at the evaluation management services, so longitudinal patient care, office-based visits, there was a slight increased percentage of women spending time in office-based evaluations. This was from that same study. We looked at the fraction of IRs who were women by state, green being above a national average, red being below, and again, seeing state-level differences. And then this was just one additional study where we looked at what's been described in the literature as occupational horizontal segregation. And here, the concept is that within a field that there's actual disparities in the distribution of tasks and responsibilities, and that this could then lead to, this might be driven by certain biases and prejudices, but then it could have effects on promotion and career advancement. So we tried to really control for this and look at within abdominal, within MSK, within neuro, rural, and academic, and looked at percent reading advanced modalities versus the basic modalities which included ultrasound and X-ray. And there were differences here. When we got more granular with this, within, say, MSK or abdomen, the women radiologists were spending more, based on their billed claims to Medicare, more of their time reading, and it really came down to ultrasound, more time reading ultrasound relative to the male radiologists, again, based on Medicare claims. So again, what were the reasons for that and what could be the impact on that for career implications? So as I kind of draw some themes based on this work, so I think often key in this is to link different databases. If you look at any one database, you kind of get just one angle at it, but you often, to do this successfully, kind of, you know, you get your Medicare data that has info by county, then link it with another database that has county population characteristics, and I think you can just get more insight that way. There are limitations just from the information in the available data, and the existing research has really heavily used claims data, and basically everything I've shown so far has all been using claims data, which has, you know, great, you know, you get huge, it's national, you get huge numbers, and, you know, it's all standardized, but there's limitations to claims data as well, and there's disparities. We'll often get more apparent, you get more granular as you go from state to county and even zip code. Some priorities for future workforce research. So, you know, we've been limited by what Medicare makes available, and, you know, they do have info on zip code, on counties and gender. They don't really have info on the race, ethnicity of the participants, of the people billing Medicare. I think LGBTQ workforce diversity is really important, and also the associations of this diversity and inclusiveness with outcomes, which has been, I think, also lacking, at least from the Medicare data. So I'll just finish with one study that was recently published from another group that, you know, took a different angle to this. They didn't use claims. They used a large teleradiology data bank. This was actually from Europe, and this spanned multiple European countries. It was actually one of the largest teleradiology clients in Europe. So they can get more than just claims, actually look at other downstream outcomes. And here, this was a gender-based study, and they compared male and female radiologists. And in their study, the female radiologists were more likely to get a second opinion on a case and actually have fewer errors based on an adjudication process in their system. So on that note, thank you for your attention, and I appreciate the opportunity to speak. Thank you for the opportunity today to talk about conceptualizing health disparities research in radiology. Just as basic review, all of us have seen and heard of different definitions of health disparities. A health disparity is a particular type of health difference that's linked to socioeconomic or environmental disadvantage. The U.S. Department of Health and Human Services conceptualizes health disparities in more than just racial-ethnic differences. Disparities and differences can be seen across different groups based on a myriad of characteristics, including socioeconomic status, physical disability, sexual orientation, geographic location, and any other characteristic that's linked historically to discrimination or exclusion. When we think about health equity, it refers to the ability of any person to have equal and just opportunities to maintain and improve their health. The Rartwood Johnson Foundation has a definition of health equity where healthcare and quality healthcare is just one aspect and portion of health equity. The other components involve social justice, the ability to have access to good jobs and fair pay, quality education and housing, and safe environments to live and work in. When we talk about having greater health equity, it's decreasing health disparities across different groups. So how do we promote health equity in radiology? Nabil Safdar had a great review article published in the JCR in 2019, where he outlines four different pillars where radiologists can really promote health equity. The first is to educate about health disparities and improve our cultural competency within professional radiology, to foster diversity and inclusion, to do research in disparities and novel interventions for reducing disparities associated with imaging, and to advocate within our societies and through our communities to improve health equity, especially when it comes to imaging. So let's take a deeper dive. In terms of these four different pillars. First, to educate about disparities. How do we educate about disparities if we don't know what the disparities are? So we need to quantify the impact of actual disparities in imaging-related care. When we talk about fostering diversity inclusion, we actually need to demonstrate that including different groups leads to greater health equity, and that involves diversity inclusion in our workforce, as well as our patient populations, and showing through research that this leads to greater health equity. Third, we all wanna advocate for greater health equity, and we need to engage stakeholders, especially vulnerable patient populations, to ensure that all voices are heard. But I would argue that across these different pillars, the most pressing is researching disparities and potential interventions that can reduce disparities. This pillar is necessary in order for us to educate, to foster diversity inclusion, and to advocate for greater health equity. So really, to promote greater health equity in radiology, we need to concentrate on improving our research methodology for examining disparities and identifying novel interventions centered in the imaging community that could just decrease these disparities. When we talk about different factors that lead to health disparities and to health inequity, we have to approach it with a multi-level lens. When we talk about social determinants of health and what goes into one's health, healthcare only contributes to 20% of an individual's overall health. 30% can be attributed to different health behaviors, for instance, whether they smoke, what their alcohol intake is, their diet and exercise habits. 10% is due to physical environments, and 40% due to socioeconomic factors also around their built environment. That includes safety, includes their income level, job status, education, and social support network. To understand why we need to take a multi-level approach to the social determinants of health and research and health disparities, I'd like to give one example. Breast cancer screening, we all know, saves lives. Yet there are persistent disparities that we see in breast cancer incidence and mortality between white and black women. That we see in breast cancer incidence and mortality between white and black women. Over several decades, the incidence of breast cancer has actually converged so that there's very little difference in terms of incidence between white and black women, whereas the mortality gap has actually increased with greater mortality rates among black women versus white women. And we know that we're doing a better job in terms of screening across different populations, and that this persistent disparity is not related to screening access. So let's look at the link between imaging and disparities. It's already generally accepted that screening mammography decreases mortality at the population level, and that women have improved access to screening mammography, regardless of their race or socioeconomic status. Most women in the U.S. could access a mammography unit within 30 minutes of where they live. Yet this persistent disparities in mortality and morbidity continue. Little data is actually out there regarding what happens after screening. So yes, a woman has improved access to mammography, but does a woman with socioeconomically disadvantaged backgrounds have access to the diagnostic ultrasounds, preoperative MRI, image-guided biopsies? And how do these imaging episodes associate with persistent disparities? Much of my research lies in observational studies with large registry data. I lead the Northwest Screening Cancer Outcomes Research Enterprise based at the University of Washington, where we've aggregated breast cancer screening data across three different states, Oregon, Washington, and Alaska. Collectively, with other regional registries throughout the country that are like-minded, we have data on more than 3 million different mammograms in women dating back over decades. Overall, the Breast Cancer Advanced Consortium Registries are representatives by age and race ethnicity of the U.S. screening population. Recently, we've embarked on a new R01 where we're looking at diagnostic disparities after abnormal screening. So while we know that women of racial ethnic minority groups and socioeconomic disadvantage have access to screening, we do not know if they have access to everything that cascades from an abnormal screening episode. So thinking about that multilevel approach, I wanna show you our conceptual framework. This is a busy slide, but what I want to point out is the top of the graphic here where we go over the multilevel factors that may be contributing to diagnostic delays. We have practice level factors, right? The quality of care a woman receives. Do they have advanced imaging availability where they usually get screened? Do they have access to image-guided biopsy, which is the standard of care? What types of communication are they receiving from their screening facility in order to schedule their biopsy in a timely manner? Their residence, we're collecting data and associating their rurality versus urban residence. We have two coded measures of health equity, including using the Area Deprivation Index and Neighborhood Atlas from the University of Wisconsin to associate women's street addresses with factors such as neighborhood income level, education level, safety, and different socio-demographic characteristics that we don't normally capture with our radiology information system or even our electronic medical record. And we have women-level factors, the risk factors that you normally think of like race, ethnicity, and age, previous biopsy, family history, but also their education level, income level, if they're insured or underinsured. And all of these multilevel factors impact different steps in the imaging cascade. After an amyloid mammogram, what kind of diagnostic workup are they getting? How long does it take? What are the diagnostic outcomes for these different steps? This goes along with not only the diagnostic workup, but image-guided biopsy and preoperative planning leading up to definitive surgery and treatment. We're also looking at novel quality indicators of timeliness, time to diagnostic resolution, time to tissue diagnosis, and time to surgery to develop national quality metrics for the diagnostic continuum. What's important to note with this R01 effort is to look at the multilevel approach that we're taking. We're not just focused on that imaging parameters and the healthcare access and quality measures associated with social determinants of health, but we're looking at all the other social determinants of health, like education, income, community context, the built environment, and all the other factors that go into a woman's health and ability to improve their well-being. So I wanna give you an example of one of the multilevel factor analyses that we're doing. And this is preliminary data for the R01 where we're looking at delays in time to tissue diagnosis after abnormal screening across the Breast Cancer Surveillance Consortium. We know from previous studies that delays after mammography, even delays just greater than 30 days after an abnormal mammogram, is associated with later stage breast cancer and lower survival rates. Studies have documented that patient-level factors are associated with delay, including insurance status and minority race ethnicity. But we haven't had many studies factor in facility-level factors and neighborhood-level factors. So our objective was to assess the woman exam and facility-level factors associated with delays in time to tissue diagnosis after abnormal screening. We used data from nine years of BCSC registries, and we looked at women with a BI-RADS 4 or 5 assessment on their diagnostic workup, who also had an abnormal screening. We wanted to look at delays in time to biopsy using a 30-day threshold for delays. We looked at different exposure variables, including women-level variables that are non-modifiable, like age and race ethnicity, but also modifiable factors, like under-insurance. Then we included different exam and facility-level factors, including onsite services, academic affiliation, and different modalities and advanced imaging received. Not gonna go into the specifics of the complicated analysis, but know that we did regression models where we not only included the traditional risk factors of the women-level, but socio-demographic characteristics and SDOH measures like education, income, rurality. These are some of the early results we're seeing in terms of the relative risk of a delay across different race ethnicity groups. And I wanna focus on the last column, which is adjusted for both modifiable and non-modifiable women-level factors, but also exam and facility-level factors. And what we see is persistent disparities in terms of diagnostic delays for black women, Asian women, women of mixed race ethnicity, and Hispanic women compared to white women. And this is adjusting for all the factors that we could think of at the multi-level conceptual model. So what can we conclude from this preliminary analysis? One, about a third of women actually experience a diagnostic delay after abnormal screening. And we're seeing these risks for women of minoritized race ethnicity groups across the board, even after adjusting for many factors. These include screening modality, academic affiliation, onsite biopsy capabilities. So we think that we need to identify new factors, new multi-level factors that may be associated with structural racism that are leading to disparities in time to tissue diagnosis after abnormal screening. And we have to collect these novel measures that we don't usually think of in radiology. So overall, what are the take-home points? First, disparities in health equity research in radiology is really in its infancy. And there needs to be an emphasis on research and evidence generation. Second, radiologists are actually in a great position to observe potential links between services and disparities in care. We have really detailed knowledge of imaging pathway and the diagnostic steps towards diagnosis and treatment of different diseases. Finally, we need to think about developing multi-level conceptual frameworks for specific imaging-related disparities. Particularly, we need to think about the data parameters we're not collecting that we need to collect and think about who we need to engage in terms of stakeholders to get these multi-level data that don't exist in our risk. I'm here to talk to you about using publicly available databases for health equity research. So first, oftentimes, we start off with research questions and we typically go to our own, here we go, institutional databases. So we go to our hospitals and we see what data we have available about a particular topic. And so this is what we often reflexively do, but there are some problems with this approach. So for one, our environment in a hospital is based upon our own unique geographic setting. And this has its own unique patient population. It's got its own geography. It's got its own local climate here. And when we do research with these particular studies, oftentimes, it's problematic because we're trying to make conclusions in some ways that potentially affect other populations across the country and we live in a diverse country. Different parts of the country have different things going on. Just came from Massachusetts, you go down every street corner, there's a Dunkin' Donuts, maybe two even next to each other as well, too. Just moved to Wisconsin, so now in the Midwest, the Dairyland America cheese capitals, I could regale you with several cheese jokes which I learned in the course of my first few months of work in the University of Wisconsin, but I won't spare you that. And then other places as well, too, like Florida, for example, it's got its own things going on as well, too. But we live in a diverse country and it's getting more diverse over time as well, too. And studies that have looked at this, by 2050 or so, about half of the population, more than half of the population is projected to be racial ethnic minorities. So when we think about that, we think about that, and Dr. Carlos mentioned this as well, too, as far as clinical trials, but our clinical trials and our clinical research don't reflect the populations that we serve. So if you just look at, this is data from the FDA looking at clinical trials here. In randomized clinical trials, only about 5% of trial participants were black or African American, and only about 1% or so were Hispanic or Latinx. So this, of course, has significant consequences. So when we create data and we create guidelines and policies and procedures, we're doing it based upon this data that is coming from mostly homogeneous patient populations. So as a result of this, so this is an example of the U.S. Preventive Services Task Force said we should start breast cancer screening in women starting at the age of 50. And what that recommendation is, is based upon randomized control trials that were essentially homogeneous and included basically no racial ethnic minorities. And this is data from SEER that looked at a wide variety of racial ethnic groups, and you can see that black, Hispanic, and Asian patients are much more likely to be diagnosed with breast cancer at much earlier ages, before the age of 50. So now, and we know, of course, from data that black women are about 40% more likely to die from breast cancer compared with white women. And now, we created a lot of these guidelines and these policies and the procedures, and now we're finally reassessing some of these. We're taking a step back and looking at them and seeing where did we go wrong in terms of introducing bias in these things. So what I want to talk to you about is, going forward, how can we look for ways to improve the representation of patients within our studies, so that our studies can reflect the patient populations that we actually serve. So I've got a few strategies here to improve the participation of different patient populations in clinical research. And so, I just want to throw a little brief little plug in here for something, an educational exhibit that's out there right now for NRSNA, one about community-based participatory research here. Arissa Milton is a first-year medical student at the University of Wisconsin, has put this fantastic educational exhibit together about this particular one. So if you are thinking about this as a potential strategy in terms of reaching out to patient populations and really engaging your community in terms of health equity research, I would advise you to take a look at this one. But I'm going to focus on existing data resources right here. So this is a hot topic right now. A lot of funders and organizations are requiring that if you do health equity research and any other type of research, that you post your datasets online. So for example, now, Google itself has a dataset search engine that you can look for. So if you have research questions, you can focus on these data sources for this. But I'm going to focus on one particular publicly available data source. Dr. Rosenkranz mentioned some of the really great county-level work that he's done in terms of workforce for Medicare data sources, too. But I'm going to focus here on the National Center for Health Statistics and some of their data sources here. So what they've done, and I'm just going to show a few examples of some of these things, but they've conducted a wide variety of population-level surveys that ask patients about different aspects of health behaviors and also providers as well, too. So I'm just going to focus on a few examples and then end with a few tips about how to use some of these. So here's one example. This is the National Hospital Ambulatory Medical Care Survey as one example right here, which looks at about 500 or so hospitals, which provides a nationally representative sample to look at oftentimes mostly emergency department and outpatient department behavior. And I just want to cite one example. Dr. Drew Ross, who's over here in the crowd over here in the corner, fantastic health service researcher at the University of Wisconsin, put this paper together and it found that white patients are more likely to receive medical imaging compared with non-white patients. And just to show the power of some of these publicly available databases and some of the inferences that you can draw about minority patients, I just want to draw your attention to this one particular column right here. This is looking at patient visits right here. So you see 300 million patient visits for black patients in this data sample, looking over the course of 10 years. And it was 200 million patient visits for Hispanic patients. And then Asian, other categories, almost 50 million patient visits. So we're talking about a lot of ability to draw meaningful sort of inferences based upon these kinds of data sources here. Another data source, as well as the Behavioral Risk Factor Surveillance System. And this is a telephone survey. This is also publicly available, both landline and cell phone. And it looks at a wide variety of health-related behaviors and preventive services. And this survey is conducted every year. It surveys almost more than 400,000 patients every single year. So it's the largest continuously conducted survey in the year. And just sort of an example of this in the recent literature, I worked on this with Dr. Flores as well recently, this project looking at lung cancer screening eligibility. Many of you have heard recently that the U.S. Preventive Services Task Force came up with new recommendations regarding the eligibility for lung cancer screening. And this is an exciting development. And one of the reasons they cited was that by expanding eligibility, that they can hopefully reduce health disparities. But that was stated and potentially modeled in terms of the data. But we actually tested this to see if this is true using this Behavioral Risk Factor Surveillance System survey. And we found this wasn't true, that African-American and Hispanic patients are still less likely to be eligible for lung cancer screening, even with the updated guidelines. So the ability of this data source to do that. And one last example I wanna show is the National Health Interview Survey as a data source right here. So this is considered to be one of the gold standard for survey databases. And this is in part because of the large response rates. And what this is, is a household cross-sectional interview survey. And it tracks a wide variety of access metrics and national health objectives. And this is about 35,000 households every single year and about 80,000 individuals. And one characteristic about this survey and also with the Behavioral Risk Factor Surveillance System surveys, these surveys are trying to come up with national estimates to draw national inferences. So because of that, they oversample for black, Hispanic and Asian participants to make sure that their results actually represent the country. So unlike other surveys, it's just sort of go about their processes and come up with small numbers of racial ethnic minorities. This implies oftentimes intentional sampling strategies to make sure that the results from these studies actually resemble that of the nation. And so how would you go about using one of these surveys right here? So I just wanna go through a brief sort of little tutorial right here. So first, this starts off with any kind of typical research process right here. So we start off with what's the problem, what are we looking at, do a literature search and then really come up with a defined, specific focus research question. And this is the PICO format here for setting up a good sort of research question right here. But once you have a research question in hand, now we ask ourselves the question, is there a data source? Is this something that we could use one of these publicly available databases to answer one of these questions? So I just have this example, this Google database right here is something, hey, I'm working on this particular topic, let's run through this Google database and see if it can actually address one of these particular questions. So if you do have a database here, the next question is, if you've never worked with something like this before, to talk to your friendly health service researcher, your epidemiologist, your biostatistician, somebody who's worked with this data source before, because oftentimes these things can have little tricks within them, within these data sets that can make things more challenging to work with. And I would just emphasize that instructions are very, very important with these data sources right here. The data sources here essentially have a lot of little tricks in them that are built in that you don't necessarily know if you just plug this into Excel and start cranking out some data, you might be making very, very wrong assumptions about things. So thankfully, actually many of these data sources have detailed documentation about how to use them. So this is the National Health Interview Survey, which has documentation about how to use this. The Behavioral Risk Factor Surveillance System has both Sudan, SAS, and State of Code to help you go through these things. So these are some ways that you can, if you follow all these careful guidelines and ask your local friendly health service researcher for how to use these things, these are some ways you can use these databases for your research. So to sum things up, the research studies are not doing a good job, including racial ethnic minorities and then in these public databases can really improve the representatives and generalizability of our clinical research. So if you're interested in doing this, talk to one of your local health service researchers. So thank you so much for your attention. Good afternoon, and thank you for inviting me to be part of this important symposium. Qualitative research differs from quantitative research in exploring the what, the why, or the how of a phenomenon. Qualitative research maps well onto health equity research, looking to understand, explore, and identify individuals or systems level perceptions, behaviors, attitudes, and beliefs. Qualitative research is a powerful tool to examine effects on the health system of inequities, looking at sociocultural factors on health beliefs, behaviors, and treatment, and a whole host of patients' expectations of care, mistrust, perceptions about an ability to maneuver the health system, diagnostic and treatment factors, and variation in symptom presentation. Qualitative research is expanding and progressing over the past two plus decades, as shown by the increase in proportion of PubMed articles in which qualitative research is used. It's important as a takeaway from this lecture that I impart to you that there is evidence-based quality research and that there are standards that I encourage you to look up to align your research with quality standards. When I work with individuals who are conducting qualitative research, I ask them to think about why they're conducting the question, what they expect to believe, and what they'd surprised if they didn't believe. In qualitative research, the researcher is very close to the data collection and the data interpretation, which is a strength, but one an individual should be aware of in thinking about what clinical, personal, professional expertise they bring to this research question. My team has used qualitative research to look at the different types of inequities in access to care, primary, secondary, and tertiary. In one study, we conducted focus groups to understand women's barriers to abnormal mammography follow-up, and we're able to see community-level barriers, individual-level barriers such as competing demands, and beliefs based on health perceptions from their country of origin. Overall, we were able to understand how to intervene to improve mammography rates in communities. This is another study in which we're exploring attitudes regarding genetics and nicotine, comparing black and white patients' perceptions regarding the role of genetics in addiction to nicotine. We were able to map out responses and themes based on these focus groups to understand patients' perceptions and racial differences in smoking, addiction, disease, and genetics, to help understand barriers that might occur when matching pharmacological treatments based on genetic testing. When conducting qualitative research, qualitative research is different than quantitative in that we are not supporting hypotheses but developing propositions, which are kind of hunches based on the data. When using qualitative methods in equity research, we can use it to learn about a population, to develop these hypotheses, to inform survey or program development, or to elucidate findings. There are many types of qualitative data methods, from focus group and individual interviews, narratives, consensus building, participant observation, document analysis, and cognitive interviews. In creating interview questions, I work with researchers to think about how the research question aligns with the interview question and how the interview question should be written for the audience that it is intended to collect data with. A question I often get asked is the difference between focus groups and individual interviews. This is an important question when collecting data. Focus groups are great for group rapport and getting a great deal of information from groups of individuals. Individual interviews are easier to facilitate and one gets more information per individual, particularly when the topic is sensitive. For one study, working with medical interpreters, we conducted focus groups and individual interviews to identify interpreters' stressors based at work. Through this combining individual group interviews, we are able to identify patient-based interaction with medical care, conflict, and systems-based stressors. Document analysis is a powerful tool that can be used with existing data such as the EHR. I had conducted a study looking at smoking cessation after cancer diagnosis, and based on comments that were received when the paper was published, we put together a framework to understand how people develop stigmatic attitudes toward cancer patients who continue to smoke, particularly lung cancer patients. Another great way to use qualitative research is to add open-ended questions to your surveys. This was one question that we placed on a survey a year and a half ago to understand frontline clinicians' concerns to the pandemic, and we were able to develop seven themes of concerns. Cognitive interviewing is an important qualitative research design to use when developing questions that harness and help understand the thought processes of patients. An example question is, in the last year, have you been bothered by pain in your abdomen? And the edited question after cognitive interviewing included a picture and posed, in the past 12 months, have you felt any pain in your abdomen? An important consideration in doing qualitative research is to think if you're going to mix it with quantitative research. This can be done by conversion, where there are separate quantitative and qualitative endeavors, explanatory, when the quantitative data is done first, exploratory, where the qualitative data collection is done first. In linking data to develop surveys, qualitative research could be used to understand patient or community populations, thinking about what types of survey domains should be used, what are the dimensions of the survey, and what are the wordings that one is using. This is an example of research that we conducted to understand medical residents' perceptions of cross-cultural care. Through qualitative research, we were able to elucidate survey domains, and the survey that was conducted nationally looked at perceptions of preparedness to deliver care, educational experiences, and educational climates. This is an example of a consensus study in which experts were gathered to understand quality indicators of end-of-life care. Adding a mixed-method study onto the ACRIN NLST trial, we were able to administer a survey and qualitative data collection to understand patients' perceptions of lung cancer screening. Using a theory-based model, the health belief model, which helps explain why patients make a behavior change, if they have a trigger to change their behavior, if they perceive their health as a threat, perceive they are at risk for the threat, perceive a benefit for making a health behavior change, and have confidence to make the change. Our qualitative question developed in the health belief model, and our explanatory results helped us understand why undergoing lung screening didn't have the behavior change effect that we thought it would. Overall, patients didn't feel that a screening was a cue to action, wasn't a motivating trigger to change their smoking cessation or the behavior. They didn't feel that the risk that they really had was applied to them. They didn't think about the severity of the disease that much. They had low confidence and worry about this behavior. The NCI developed, based on the NLST trial results, a call for interventions to intervene on patients who were smoking at the time of lung cancer diagnosis. We received a grant to conduct a randomized clinical trial through this mechanism. We explored inequity concerns in particular and developed Screen Assist to develop a centralized smoking cessation program across our healthcare system for patients who underwent lung cancer screening, and we offered this work virtually. We offered this tobacco treatment trial to patients at three points during their lung cancer screening, using a digital recruitment video, when they had their test ordered, when they underwent lung cancer screening, and after they obtained their screening results. Another trial we are conducting nationally, using virtual delivery of smoking cessation treatment, is with NCI-affiliated community cancer centers nationally. We're also using a virtual video and virtual interventions to intervene with patients. In both of these trials, we're using virtual clinician referral and videoconferencing-based counseling to reach out to patients and intervene with them, to increase our access to patients nationally. Dr. Flores has a diversity supplement to identify barriers in particular to Hispanic patients who are smokers enrolling in tobacco treatment, and has developed a culturally tailored video for these patients. We also look at intervening on access based on survivors' use and access of their health insurance. We conducted qualitative and survey studies documenting that child and cancer survivors are at risk for being underinsured and didn't really understand legislation or their own health insurance plans, and have developed a virtual patient navigation intervention, which we are delivering and testing nationally with patients to see if a psychoeducational patient support through a virtual navigator can help patients improve their health literacy and their access and comfort with their health insurance. I want to thank you for your time, for allowing me to take you through this journey of qualitative mixed methods research and virtual care delivery. Thank you for your time and enjoy the conference. All right, so we're going to talk about intervention development to changing clinical practice in radiology. So more than just research, it's really transforming your care delivery. So we, every day in radiology, we see black and white or grays, and people say, well, what's the role of radiology? Because, you know, who can guess their race and who can guess their ethnicity, even if we're seeing patients with COVID or ruling out a fracture? So truly, we have a lot to say because we see many patients that have been discussed earlier today. And our patients really don't need to navigate their journey alone. So health equity research helps us understand the barriers so people don't feel like they're walking in a hallway when they're alone in the big walls of the institution. It really allows us to better gain an understanding of what they're going through so we can accompany them in that journey to better health. So the first step is really mapping out the radiology care journey from referral to arriving to this study, to getting the study done, to follow up on the results and understanding what is the influence of social determinants of health across that journey. We see in this figure from JCR published a couple of years ago, and then we superimpose certain social determinants of health or additional factors outside, such as scheduling conflicts, system barriers, missed care opportunities, but we don't call them no-shows, and then patient portal access so that they want to see their results. And again, if you map out the process and the journey and then the influence in each step, then you can start thinking about ways to improve it. But what better way to tell our patients what we can do for them than allowing them to guide us in that? So patients' needs guide our radiology care delivery. These are quotes from patients from focus groups, for example, talking about mistrust of Medicaid does not want to give us MRIs and CT, talking about lack of opportunities, you don't know how to get a lung cancer screening, talking about equity issues, I don't have a computer, they don't have a computer, I don't have internet access, so more so they're not going to have a patient portal. So when we think about policy change, and this is a graph that Dr. Narayan talked about earlier, it really represents a potential window for advancing equity in radiology with the new guidelines from the USPSTF criteria for lung cancer screening. But again, intentional outreach is needed because the guidelines or blanket solutions are not equitable. So when we think about the study published by Dr. Narayan and colleagues talking using BRFSS data we discussed earlier, when we look at eligibility criteria for patients with lung cancer screening among the red states, which are the states that fill out the survey, no political implications here, we can see that there's a lot of variability in the eligibility, but also among those patients that are eligible, the utilization rates for lung cancer screening remains low, as low as 7% despite the proven benefits. So we also have to think about the sociological model for lung cancer screening and how these barriers become multilevel at the individual, at the provider level, at the community level, and the health system level. And if you think about a patient caring all that way, it's almost like oppressive on the patient. So more so, even if the patient is ready to take the step to get screened, these barriers don't go away. They are weighing on them at the time of the scan, where they get the results, and the timely follow-up. And for example, in this case, barriers to lung cancer screening, but it could be barriers to MRI. So intentional outreach is needed, and this is courtesy of Claudia Munz, who is like an MS-IV applying to radiologists here in the audience. And so for me, when I see them, we're talking about meeting patients where they are, and see them as partners. Not only we're going to tell them what to do, actually they are the ones who should be guiding how we deliver care for them. And the research on health equity needs to be a continuous process, right? So continually pursuing health equity, not a one and done process. We're not going to do one paper or one project and be done in that journey. You're likely going to uncover additional barriers and other areas for improvement. So we can start for detecting with big data analytics or national data, like Dr. Narayan was talking about earlier, or it could be also, but once you identify the gaps, use an understanding with qualitative and quantitative data, as Dr. Park mentioned, with mixed methods approach. Then you can talk about intervention developing or reducing, like Christoph was mentioning earlier today. And then also, not only you want to develop the intervention to improve the gaps, but also you want to use implementation science framework to evaluate and assess the fidelity and then the long term effects of this as well. So we use that framework of detecting, understanding, intervention, and then evaluating. For example, we looked at missed care opportunities or missed appointments for advanced imaging. Some of the factors associated in our practice were, for example, certain patient populations were African-American, Hispanic, people with Medicaid, and household income less than $50,000 had a higher likelihood of missed appointments. But that's not that this population, we don't use this to single out population. We use it to understand that there are populations that may require additional assistance and we have to provide it for them. Then when we think about the understanding approach, then we think about, okay, so what is the way, once you get an order, what is the effect of waiting to get the exam or that gap between getting the order and getting the study done or called wait days. And we will see in the graphs is that in the first graph or graph A, we can see that the longer you have to wait for your exam, the more likely you're going to have a missed appointment. And this is exacerbated among Hispanic and African-American patients. This also exacerbated among patients living in zip codes with a median household income of less than $50,000, and even among those patients with Medicaid. So even if we increase our capacity and say we're going to have MRI and image everyone and close down to zero, there are still going to be gaps. So there are more for us to do. What can we do? We can develop programs for reducing and evaluating, bridging those gaps in radiology. For example, a ride share program that was implemented at one of the outpatient imaging centers in serving a cashman area, predominantly Spanish-speaking patients coming from a low socioeconomic background. And this offer for patients that spontaneously need to express a need to cancel their appointment because of transportation issue. And post-implementation analysis showed that not only improve the timeliness to arrival, which is critical in the COVID-19 time era where we are trying to avoid overcrowding of waiting rooms, but also patients with public insurance, unemployed and those that are older are more likely to utilize it. So you want to make sure that any program that you're developing is addressing gaps and assisting patients that need it. But more than that, we have to think about maximizing each patient encounter. Because if we think about the sociological model and those factors weighing on each patient every single time and patient need to overcome when they see their PCP, when they come to radiology, why not offer them more at each touch point in the healthcare system, such as same-day screening mammography or same-day lung cancer screening when they're seeing their PCP and they're ready to get imaged, then it's our duty to try to offer that encounter right then and there. We can think about also team-based approach to screening in collaboration with other providers beyond primary care with mental health clinicians. So we have patients that are seeing their mental health provider more frequently than their PCP, but health preventative services are relayed to primary care. So why not a team-based approach where we're all responsible for and become more system responsibility to engage patients in health preventative services and achieve that better health outcome. But also we have to think about reducing and integrating radiology care coordination. Don't think of each area of radiology as an isolated area, but think about novel ways of integrating radiology, in this case, screening mammography encounters and lung cancer screening surveys to see if patients are already filling out a breast health intake form, adding a brief survey that could potentially identify who is eligible and among those that are currently smoking, who would benefit from tobacco cessation, why offer them that opportunity and use it as a teachable moment. So when we think about the road ahead to health equity, this is truly person-centered. So when, you know, we think about potential areas and I think of leveraging mHealth to advance health equity, tomorrow Dr. Patty Salazar, who's a former MGH fellow and current staff at Emory, will be talking about the influence of patient portals and access to internet on breast imaging screening and then we're going to be talking about developing a lung cancer screening digital outreach that is community-based as well. But more importantly, we have to remap that radiology healthcare journey and see that when we think about all the processes, we've talked about integrated care, ride share, same-day imaging and patient portal outreach that is needed for our patients and there are going to be more, but don't think about also about the process being linear, think of novel ways of integrating different areas because we can infuse equity in radiology through research and care delivery transformation because health equity pursues social justice in health. Thank you. I'm going to be talking about design thinking and implementation science and how they can be useful in health equity research. So design thinking and implementation science are two very different concepts, but what they share is the concept of making a process or an evidence-based process practice more applicable in the real world. So design thinking focuses more on understanding a process through the end user experience, the end user in a real world setting, and implementation science focuses on bringing evidence-based practice out to the real world. So both are important to health equity research because in order to achieve health equity, we have to focus on the real world, on diverse populations in the real world, and how we can offer these populations in the real world the opportunities to achieve their best health. So what is design thinking? So design thinking is an iterative process. It's a problem-solving method that puts the user's experience at the center. So it uses divergent techniques or gathering ideas from a lot of people and convergent techniques, which is merging and categorizing all those ideas that you gathered from different people, to inform processes. So this can involve empathy mapping, which is when you create a persona or an end user for a certain situation or process, and then you explore that persona's experience. What might that person say about the experience or think about the experience or do or feel? And what I think is really great about design thinking is that this empathy mapping is often done through sort of a quiet or individual process of idea generation. We often do this with sticky notes, where we ask everyone to write down what they're thinking and sort of give them two quiet minutes to do that, and then everybody puts their sticky notes up on the board, so that each person has the opportunity to have their idea heard, instead of having maybe one or two dominant voices in the room have their ideas heard. So this is an example from a design thinking session that we did at our institution when we were working on developing a health equity curriculum. So we gather ideas in that diverging idea gathering phase, and then we, from what everyone's input says, we generate our big ideas. So the big ideas are the solutions to the problem that you're designing for. And so some of the examples of the big ideas that we generated from that design thinking session were, we really need to understand our patient population better. We need to emphasize the importance of empathy. And so these are things when we are creating our curriculum, how we're going to teach our residents, how we're going to teach our faculty. These are the concepts that we need to integrate into that curriculum. Then you can also use these big ideas and put them into a prioritization grid. So what can you actually do? What is sort of low difficulty but high impact? And start with those things and leave the things that are higher difficulty and lower impact for later. So whirlwind tour on design thinking. To shift gears to implementation science, which is a separate tool that we have in our toolkit for health equity research. Implementation science is along the scale of efficacy studies and effectiveness studies. So we talked a lot about big data sets, retrospective work, looking at existing disparities. So we spent a long time, and a lot of the speakers today talked about how we can work with these big data sets to identify our disparities. But once we've identified them and we have some interventions, how can we test these interventions and how can we prove that they are effective? So on the scale of these types, this type of research, efficacy research is, does a certain practice work in a very tightly controlled environment? So in like the clinical trial setting, you know, it's great if it works in that setting, but what happens when we get out to the real world? So the next step is effectiveness research. Does it work in the real world? And one step further is implementation science. How do we get the real world to actually do these interventions the right way? And so my personal interest is looking at this idea of implementation science and how it relates to health equity. So how do we get broad communities in the real world setting that serve diverse populations to implement proven evidence-based practices? And outcomes that we look at in implementation science are a little bit different than usual clinical outcomes that we look at. And so Brian Weiner, who's sort of a renowned implementation scientist, says the sweet spot for implementation science is taking what works and figuring out how much it can be adapted without losing its effectiveness, and then scaling that up to reach large numbers of people who can benefit from it. And these are some of the examples of some different types of outcomes that are looked at. Acceptability, appropriateness, feasibility, sustainability. So this is kind of a basic idea of how implementation science would work. We know there are evidence-based practices such as mammography or breast MRI for high-risk patients or encouraging patients to have breast cancer risk assessment at younger ages to identify who's at higher risk. How do we take these evidence-based practices that we know work, and how do we integrate them into routine, sustained clinical care? And then why is this implementation science idea important for health equity? Well, we know that disparities in care exist, and we've talked a lot about that today. We've seen a lot of the data. So disparities in care and disparities in outcomes exist. So for example, in Tennessee, where I'm from, the purple state there, we have the seventh worst breast cancer mortality rate out of the entire country. So if you stacked up all 50 states on top of each other, Tennessee would be way down there at the bottom, number seven. So what else do we know? If we look at this on a more local, regional level, within Tennessee itself, breast cancer mortality rates vary by county and by community for reasons that were discussed earlier. It might be access to care, access to subspecialty care, trust in the healthcare system, different jobs, good jobs with better access to care. And so here, the county is shaded in the darker pink color, have higher breast cancer mortality rates than the other counties. And so through implementation science techniques, we can go beyond the standard research that usually focuses on larger academic centers in larger cities. And we can work in an implementation science-driven, community-engaged approach to provide an opportunity for all of these counties, and especially the ones with the higher mortality rates, to use the evidence-based practices that we know work, but to have these evidence-based practices out in a broader real-life setting so that we can begin to address known existing disparities. So to recap, design thinking and implementation science share this concept of making a process or an evidence-based practice more applicable to the real world. So by applying a health equity lens to design thinking and implementation science, we have an opportunity to address existing health and healthcare disparities through a focus on the real world, on diverse populations in the real world, and how we can offer these populations in the real world the opportunity to achieve their best health. So thank you so much.

health equity

systematic biases

minority representation

clinical trials

journal editors

social determinants

health disparities

contextual research

structural inequities

mixed-method approaches

design thinking