Good morning, everyone. I'm Rebecca Rocco Penner from UC San Diego. I just want to welcome you to our webinar entitled key elements for successful clinical research. I just as a background, I'm grateful that you're all here and we have some wonderful speakers today. This the purpose of this is so that we are inclusive of all members of the team that help us get grants submitted. And so we're happy that everyone is able to participate, whether it be a grant person or an admin within the department. Everybody is an important aspect and we hope that you are all active and asking questions today. If you can, during the questions during the presentations, please type your questions into the chat box and Dr. Mankoff will be helping moderate that. And we can also have some time at the end for some questions and answers. I'd like to have my two co-moderators introduce themselves before we introduce our first speaker. Dr. Mankoff. Hi, I'm Dave Mankoff from the University of Pennsylvania, and I've had the pleasure of doing this with Rebecca, I think, three years running now. And it's a great series to to to bring the group together that's working in research and radiology. Sounds good. Thank you. Hi, everyone. My name is Janie Lee. I'm from the University of Washington and the Broadhurst Cancer Center. I'm a breast imaging radiologist and a health services researcher. So it's really great to be here with you today. All right, let's get right into it. Our first speaker today is Dr. Janet Erie. She is an associate director of the cancer imaging program at the NCI. She has been a research principal investigator and program leader continuously throughout her career, which began in experimental therapy with radio pharmaceuticals. And she was also a pioneer in molecular imaging. She has expertise in molecular imaging, radionucleotide therapy, translational studies, clinical trial design, imaging and basic science and image analysis, which remain as her research interests. Today, she's going to talk to us with her NCI hat on representing keeping up with the latest RFAs. Dr. Erie, please. Hey, thanks so much. I guess I should add to my bio now managing the federal workforce. Yeah, that's a bit of a different thing entirely, isn't it? So I changed the title a little bit. How to read an RFA to kind of talking about funding announcements. So that's what I'm going to do is go through some basic materials that I had accumulated for various other educational activities. So let's go ahead. And Tori is running my slides right now. So next one, please. OK, so funding mechanisms. These continue to change. And if you don't know what different terms mean, it's probably not you because it's probably a different one. So the most important thing is to know that there is a general class of opportunities called funding opportunity announcements, and there's different types of those. And, you know, the stuff that's online from the government, grants.gov or NIH grants or NCI grants, it's important to get familiar with some of the terminologies. Your grants and contracts folks might know that, but probably not as well as you should. If you have a research idea that you're looking for funding or you have a group that's looking for funding or you're looking for specific topics to be funded. So. Basically, it's the federal agency. Anybody makes known its intentions to award discretionary grants or cooperative agreements and contracts sometimes as well. And sometimes there's a competition for funds and sometimes there is a request for application. Sometimes there's requests for information and sometimes money is put up and said we're looking for applications. So there's all kinds of things that go on. So throughout my presentation, I'm going to talk about how to basically keep up with what's going on and stay current with the different types of funding opportunities. So. Dr. Erie, you're muted. I still can't hear you. I think you have to unmute your screen. Yeah, I did. I was just talking and it just happened. Well, that's kind of sent that censorship. Okay, can you hear me now. Yes. Okay, so an important term is the one at the bottom there in the bottom paragraph. And that's the omnibus application grants kind of like the omnibus federal funding plan that means everything. And those are the parent announcements for the grant mechanisms that you're most familiar with most familiarly with the R01. How about the next slide. Okay, so yes, learn to read the acronyms. When I first started the government, I walked around with a small book and I made my own dictionary which I've since shared with a number of other people who are new to the federal workforce to understand what these things meant. And since that time they have some of them have evolved into other into other names and other concepts and other things and other things in that the NCI, I'm talking about the NIH in general does this as well. It switches up what types of announcements and opportunities they have out there depending upon what kind of funding they have and what kind of funding they'll anticipate. So next please. So, how do we come up with these and I thought I should give you a little bit of a history about where these things come from, they don't just spring up out of the ground so here's here's what we do in program and I'll use the cancer imaging program as an example. So first of all, and I keep telling my staff all this all the time and it is our job to help with you identify gaps in support and areas that need some support, and that sort of thing they shouldn't be ideas that are brained up by the by the program staff at the NCI or the NIH, they should be the result of conversations that you have during workshops, where program staff interacts with the community, and together we identify areas for that need for support or special interest. I underlined important aspects of our ability to communicate and facilitate conversations throughout the research community so we listen to the imaging community about issues in science and patient care, and what might be new developments. And our job is to keep on top of what the research developments are so that's an important task that I asked my staff to do. We have the wonderful opportunity to facilitate conversations by convening groups and workshops here at the NCI at your meetings or, or wherever else we want to get together, and that means that we can play a major role in shaping the agenda and balancing the voices and input, but really we're looking forward to hearing what comes from the imaging community almost at any level in any conversation so we're always happy to participate, listen in on the conversation, make presentations, offer our two cents about different ways of different things and ideas and concepts could be funded. Next slide. We also have outreach communications, we're changing these all the time the cancer imaging program for instance has a website. We send out newsletters occasionally but we're kind of careful we don't want to spam everybody was like our newsletters. So we also have online seminars such as the imaging community call that we have on once a month. We publish papers that are quite often result of a workshop or conversations with the imaging community and those are published just to report the results and the thinking and social media now has switched over to LinkedIn, LinkedIn I'm not sure how that's going to be but websites are important sources of information. We do make promotional materials that we send around to various folks, and we try to connect the community and ourselves with different activities and different opportunities around the NIH and other science organizations. That's an important aspect is the folks that work at the NCI and particularly in the cancer imaging program are very familiar with other opportunities around the NIH for different types of funding opportunities and if they don't know they certainly know somebody who does. So I just described that a funding opportunity is an idea for potential funding that has been synthesized from the, from the community and with program staff, and if an idea has looks like it's taking shape. The program staff writes the scientific premise, the goals, the impact of the work, and that's that document that you find if you look up a funding opportunity on the websites that all that materials written by the scientific staff based on the conversations that that went on. By the time it gets published, it has gone through a number of different review types, and it has been approved at various levels which begins in the branch, then later on in the program, they've led on the division and eventually the scientific program leaders levels at at the NCI so there's a lot of vetting and review that goes on with these, as well as by the folks who actually published the announcements who make sure that the language is standardized and homogenized. In the cancer imaging program quite often we collaborate with different other divisions at the at our own instigation or at the direction of leadership and they may be people like the folks in cancer biology or prevention or other institutes such as head and neck cancer that sort of thing. Anyway, the staff presents the funding opportunity and takes feedback along the way, and then as well for for cleaning up the scientific aspects of it inclusion criteria that sort of thing. And then, of course, all the way through to the drafting of the document itself. Next one. Next slide. Anyway, other things that are that go on in the background that you're probably not aware of, and that is depending upon what type of announcement is going to be envisioned. One of the important things is how these applications that come in and receipt to these announcements, how they reviewed. So the program staff spends a lot of time and energy determining what type of review would be the most that would be the best for the group of applicants and the grants that come in. Now the type of review panel or whether or not it can fit in with existing study sections, whether it needs to be a special emphasis panel or whether or not it needs to be done at the overall review sections at the NIH level or a special section at the NCI. So that's a special emphasis review and that's worth paying attention to when you look at a program announcement. So the goal here is, of course, is to try to get the best review expertise. Why is that important? If you have an idea and you're not sure where it fits and you're looking around for background information, that's one thing you might consider is, in general, who is reviewing the applications in response to these programs. So anyway, I thought that's information you should know about. That's also information you can find out from the program officers. When applications come into a specific program announcement, the program officers listen in on the reviews, whether they're in the special emphasis panels or the standing study sections, so they can get some idea about how the reviews went and what the sense was about an application and they can provide instant kind of ongoing feedback and information for revising funding announcements if that needs to be done. And then, of course, obviously providing background advice to applicants. So there's a lot of information that goes on in exchange that goes on in the background that you all might not be aware of. Okay, next slide. So okay, back to the types of grants here. So there's the omnibus and that's the R01, the R21. These are the ones, the grant types that are most familiar to everybody. Those are just the general unsolicited grant applications. I have an idea, I'm going to submit it. So the things to look for in the omnibus type grants, because there's different versions of these, and that is whether or not clinical trials are optional, required or not. There are different types of these also is whether or not they're investigator initiated early phase or different ones that have specific targets. So for instance, there was a new reissue of the omnibus R01, which is clinical trials required that we sent around to folks to make sure they are aware of that, just so that you understand that they can be picked up and reviewed separately compared to all the other R01s that are coming in. R21s don't normally have clinical trials, and we're talking about clinical trials in particular, so I thought I'd focus my comments on those. And like I say, there's other types. So it's worthwhile kind of doing sorts for clinical trials required or clinical trials optional or clinical trials allowed. There's different types of applications, even within the omnibus types. Next one. Next slide. Okay, what's a PAR? That's a program announcement. Program announcement is a program announcement, and it has an emphasis on particular funding mechanisms. And they usually have a couple of unusual things. Instead of the normal three times a year receipt dates that you would have for regular omnibus R01 or R21, they usually have specific receipt dates that are different. These announcements usually remain active for three years and program gets a free renewal of them, usually depending upon how many applicants are applying to them and what the funding success is and that sort of thing. However, they can be deactivated sooner. Now, in the yellow down below, they're called a NOFO now. Notice of Funding Opportunity. I have a really hard time saying NOFO. It's like, okay, that's ridiculous. What's wrong with PAR? I just don't want to say NOFO in public. I just can't do it. Anyway, these things are now called NOFOs, but that's the same thing as a program announcement, which you've seen before. Okay, next slide. So here's some examples of some NOFOs, when they're called PARs, when they're listed and that are sponsored by the Cancer Imaging Program. And you can see it's quite a wide variety of opportunities. You can see the academic industrial partnership ones are for translation, clinical trials optional. Some of the ones, clinical trials not allowed, molecular imaging of inflammation, that's a basic biology PAR. A couple of the other ones do have clinical trials optional in them. So it's worthwhile looking across the board, making sure that you understand what's out there, seeing how they're changing and what the different requirements are and special conditions for clinical trials. So that's just a smattering of some examples of PARs that we have at CIP that are origin with us. Next slide. And so requests for application. Now, this is something that's very special. This is where there is a narrowly defined area for which set aside money has been put together by the Institute. So that's set aside money. That's what everybody wants is to be able to submit an RFA. So that means that when you submit the funding opportunity, there's money set aside for it. So it's worthwhile knowing what that means, because that's going to be a narrowly defined area, but that means there's money there for it. Other applications that come into program announcements to PARs and to the omnibus are all pitched into the same pool for funding for funding percentile ranking. The RFA is separate because they're set aside money. So it's not, they're not competing for the same pool of funds, and they almost always have a special emphasis panel and only a single receipt date. So that's an important distinction that I wanted to make. Okay, next. The notice of special interest is what at the NCI, the upper management decided to try to convince the staff to use instead of PARs. They thought that there were too many PARs out on the street for folks and it was getting to be confusing and it was hard to fund them into the mix. And so they have now these things called notices of special interests. And what program can do is hang an area of special interest down to any existing mechanism, which includes the omnibus R01. That's a lot of double talk, isn't it? But so, for instance, if there's an area like translation right there, that PAR 2155, translation of quantitative imaging tools and methods. That is, that's a notice of special interest that has been attached to that for the imaging community. Same thing for the one that we have on the synthetic biology application. That's been added, which is some imaging language to make those grants applicable as well. So there's also some restrictions associated with NOSI. I'm not sure how this is working out. It's been in effect now for a couple of years and it may or may not be more confusing. But rather than a specific funding announcement, what it does is we have the opportunity to expand applicability of that funding announcement to area topics of interest of program interest. And so we slide them in sort of almost like addendums or writers to a law or something like that. Okay, next. Okay, yeah. So yeah, there's some examples here. Like, yeah, we put in a NOSI for the dog oncology grants project. Does the NOSI replace the PAR? Maybe or maybe not. Anyway, so look around. Those are also areas that might fit a small niche interest where someone, for instance, wants to explore imaging opportunities under a mechanism and associated with a community that would not necessarily be associated with the imaging community. So it's also an opportunity to kind of expand our reach, our interactions and collaborations and possibly funding success. Okay, next. Okay, so yeah, check out the resources. Know what they are. Look at the websites. The ACR puts out the academy member benefit on funding Friday. That's actually kept up to be pretty up to date and it includes all the institutes at the NIH. I check it every Friday to see what's out there. That's a very rich, very current list. There are other newsletters and funding updates and various other announcements that go out. And I should have put it in red down there. Give us a call. If you know of an investigator or yourself and has an idea you want to get funded, just give us a call. We'll help you navigate that system and see what might be the best fit. It's always worthwhile. I know today, people don't want to pick up the phone, pick up the team's call or something like that, but it's actually enormously helpful. It's great for us as well to get to meet the investigators. Okay, next slide. Okay, now, how do you actually read a funding announcement? And that is scroll down. There's all this format that you have to go through for these things. Scroll down and look at the scientific statement. Topics not limited to but include, and more importantly, what's excluded? Are clinical trials allowed? What's the budget size? How many receipt dates are there and are they unusual outside the normal submission dates? Once again, I alluded to this. What's the review type? Is it a special emphasis panel where the review folks are going to call specific experts for that pool of applications? Are they going to be associated with a standing study section? And is there a letter of interest required? Or is it suggested that it might be helpful? And then other also importantly, at the bottom, at the very bottom, are a number of program officers to contact for questions and usually one for each institute or program. So that's the important slide right here, how to read a funding announcement, which was the original title of my presentation. Make sure you know the answers to those bullet points. There's eight of them. That's how I distill that. Okay, next slide. Okay, so here, here's that car that's still going out towards the sun, right? So I want to encourage you on behalf of your National Institutes of Health and the taxpayers and the folks who have illnesses, please continue to plan and do research. It's hard, but there's a lot of resources available to you. So please do it and keep up the good work. Next slide. Thanks. And thanks from all of us at the NCI and at the NIH. And I'll stop there. Thank you so much. Great. Thank you, Janet, for that wonderful introduction. And that very compact slide towards the end as well. I get to introduce, I think what we are doing is just to confirm that questions are going into the chat. And then we'll go ahead and move on through our presentations. And we'll do a Q&A. So I get to introduce our next speaker, who is Ruth Carlos from the University of Michigan. Unfortunately, Ruth can't be with us live. She had an unexpected schedule change and was not able to join us as she was scheduled to be here. So she has submitted a prerecorded presentation that will start shortly after this introduction. So Dr. Carlos is a professor of radiology at the University of Michigan and also editor-in-chief of the Journal of the American College of Radiology and chair of the ECOG-ACRAN Cancer Care Network. She serves as a principal investigator or co-investigator for multiple multi-institution clinical trials, including the T-MIST trial, which is the tomosynthesis mammography imaging screening trial. Her research focuses on health care value with a recent output from the National Institute of Radiology. She recently presented on the use of race and ethnicity in biomedical research in a workshop sponsored by the Institute of Medicine and the National Academy of Medicine. Her talk is focused on recruiting diverse populations into clinical trials. Thank you, Tori. These are my disclosures. Ultimately, this is what we want for each and every patient, to treat the right patient with the right test, the right treatment at the right time and for the right price. Thank you, Tori. Tha clinical trial reflects the broader population because once it is placed into evidence or published or somehow disseminated as the best science possible, it attains the veneer of infallibility and therefore must be implemented for all, even if the trial does not necessarily represent the broad spectrum of our patients. Many randomized clinical trials that provide evidence base for diagnosis and treatment do not adequately capture the population that will benefit from the test or the drug. Trial participants are younger, healthier, wealthier, and whiter than our diverse populations. For example, in many E. cogiacarin trials, individuals of color constitute between 10 and 20 percent of the trial population, of the total trial population. The generalizability of this evidence, then, has been called into question. Further, participation, for example in clinical trials, is a critical mechanism for improving quality of cancer care and reducing disparities in morbidity and mortality. Therefore, it is important to offer clinical trial participation to all those who qualify. The type of trial informs the strategies to increase diversity. When you talk about a screener diagnosis trial versus a treatment trial versus a quality of life or patient reported outcomes trial or a care delivery implementation science trial, there are a multitude of barriers at multiple levels. First and foremost, at the patient level, they may have difficulty accessing care. They may not be able to afford the co-payments. For example, many of our trials use interventions that are quote-unquote standard of care. That reduces the cost of performing the trial, but it does not reduce the cost for the patients. Patients are often hesitant to participate in trials that would incur additional out-of-pocket costs. Traditionally, trials that have an imaging component have had a lot of difficulty recruiting because imaging is one of the more costly and non-adequately reimbursed clinical trial requirements and services. Some may want to participate and may be able to afford the co-payment, but for job, work or home reasons, they may be limited in their transportation access or they may be limited in the time off that they can take from work. And there are ways to support patients to improve trial participation. Direct patient support can take the form of direct payments through gift cards. I believe MUSC has a $50 gift card that they give for every visit associated with a clinical trial. Others, for example, in Mayo, Arizona, Dr. Bhavika Patel, as you will hear from Elodia Cole, has arranged for transportation services to deliver individuals in a community of color to Mayo Scottsdale in order to participate in the clinical trial. In addition, as investigators, we need to think about ways to decrease burden to patients through trial design. Participating in a clinical trial is time intensive, and if there are ways for us to decrease the number of visits that I need to make for follow-up or obtain samples, meeting patients where they are, it would improve clinical trial participation and completeness of data gathering. A registry study of Asian Americans that is collecting both survey data and samples for ancestry evaluation or whole exome sequencing has sent tubes to patients' houses and requires several pinpricks to collect the blood samples rather than visiting a hospital to get a blood draw. Other studies have been planned to have nurse navigators or home health visitors to draw blood and collect biological samples. If there are patient-reported outcomes associated with the study, trials have used a multitude of patient response mechanisms, such as mail, email, telephone, and even text messaging, short text messages, to collect patient-reported outcomes, often in the moment in which the patients experience them. But having a digital platform does not necessarily guarantee participation. Our trial of collecting patient cost data related to treatment, even though we had an electronic patient-facing web-based platform, we had a 40% non-completion rate for subsequent examinations. The highest completion rates were associated with clinic visits where there were individuals who could help patients fill out all the forms. So sometimes the human touch still is a necessary component, both of recruitment and data completeness. There may also be patient and provider barriers. So for example, patients may speak a non-English language as their primary language at home. They may have lower health-related literacy, even though they are highly literate. There may be systematic mistrust of individual physicians, of institutions, and of the research enterprise in general. And as care providers, we are also sometimes implicated in challenges that patients have in overcoming their trust and or cultural barriers in order to understand the importance to them of participating in a clinical trial, as well as the importance to our ability to demonstrate appropriate evidence. Again, there are multiple levels in which we can collect and support patients. We can provide patient support by providing all materials, including informed consent and all survey information in multiple languages, as well as written in no higher than an eighth grade level educational attainment for improved comprehension for a majority of individuals. We can also have concordant recruitment between the physician and the patient. The QUIT study, which was a remote study of smoking cessation delivered by MGH smoking cessation counselors, patients were recruited nationwide by videos recorded by the patient's own physicians recommending trial participation. The University of Florida has a U01 trialing generative AI platforms for patient and avatar concordance and language translation. ECOG-ACRIN has a trial for patient navigators to improve trial participation. Regarding the provider, there are ways to make individuals aware of implicit bias, but oftentimes providers may not actually even know what trials are available for their patients just because they are so incredibly busy with the actual clinical work. This is where we can also have trial navigators for the providers, alerting them to, one, readily available trials, and two, the patients whom are eligible for the trials that are open in their facility. We end where we began, right patient, right test, right treatment, right time, and for the right price. And what we want is full representation of individuals in all of our clinical trials regardless of the type of trial, and we need to approach patient diversity recruitment and enrollment with intentionality. Often this means providing more resources for the sites in which we recruit. Often it means increasing our budget to provide for translation services, and those are the truly care-focused, patient-centered way to do research that represents all of us. Thank you. Terrific. Thank you. I think what we'll do is we'll proceed to our next speaker, who is Elodia Cole from the American College of Radiology, and again, please continue to put your questions into the chat and we'll discuss those following this next presentation. So Elodia Cole is the Principal Clinical Project Manager with the American College of Radiology Center for Research and Innovation. She specializes in research and diagnostic medical imaging systems for breast cancer detection and diagnosis. She serves as the Deadly Chair Liaison for the Multi-Center International Tomosynthesis Mammographic Imaging Screening Trial, or TMIST, from the ECOG Akron Cancer Research Group, and previously actually was involved with the DMIST trial, Digital Mammography Imaging Screening Trial. So she brings a tremendous perspective for strategies for the successful enrollment to clinical research studies. Elodia? Thank you, Janie. So again, my talk will be Strategies for Successful Enrollment to Clinical Research Studies. There are three key components to enrollment success, interested and motivated sites, recruiting and enrolling sites willing and able to enroll, recruiting patients who are eligible to participate and don't have insurmountable challenges to participation, and of course, monitoring enrollment progress throughout the study. Let's start with recruiting and enrolling sites. Creating a communication strategy around recruitment of sites in particular requires that you reach out and touch base with the decision makers at the various sites, those individuals that would make the decision of whether or not that site could participate in the trial or not. So that would be your potential site PIs, your administrative decision makers, be it in clinic or research. The methods traditionally utilized are those that are set up by the sponsor, but would likely include any of the methods that you see here, set up a webpage, social media, newsletters, presentations at national meetings. Specifically for the TMIS study, we had a couple of web pages, web portals available to our various target audiences. So specifically for the site and the site recruitment efforts, we had a website dedicated to how to participate at the site. So descriptions of the qualifications that are necessary, the payment structure for the trial, the time and resource commitment that we expected, and contracting structure. Again, general information, oftentimes that will allow sites to make some broad stroke ideas of whether or not it's a study that they are willing to participate in, given, again, the time, resources, and funding that's associated. Second, we had a general facing public webpage, and this is more for the general public, including potential participants. The information presented on our sponsor dedicated webpage, ECOG-ACRIN, is very similar to what we would have or what you would see in clinicaltrials.gov. The eligibility criteria, the locations of where the study is taking place, contacts to reach out to if you want additional information about participating at the specific locations. Trial objectives and outline. In addition, there is recruitment materials that are available from this page, along with an introductory video by the team of study chair, Dr. Etta Pisano, the overall PI for the trial. From a social media perspective, we had blurbs that would go out on our professional social media accounts, as individual investigators, and institutional social media accounts, professional organization social media accounts. So, here's an example of one that was advertised via ACR's social media channels. Again, targeting radiology and radiology administration audience, the goal is to simply let them know about the study and basically, again, in short, simple terms, how to participate and what to expect, with a link that would allow individuals who wanted to learn more to go to the homepage or the webpage about participation. Newsletter articles, helpful at study start to introduce the study, obviously, but it can also be used to update on progress and key milestones throughout the study, and that's how we utilize that within the context of TMIST to this day. So, the key things in terms of site recruitment, raise awareness of the study by the site key decision makers, have a call to action for those interested in the study, whether it be emailing the central trial team or completing an online survey. Again, the key is to make sure they have the information that they need in order to make decisions about participation in the study at the site level. And this is just a summary of what all we did as far as site recruitment strategies, which incorporated many of the different aspects that I just mentioned. The second component is interested in eligible patients, so how do you enroll patients into the trial? And so, in terms of recruitment for patients, you know, it all starts with understanding what is possible at the individual sites that are participating. Do we have enough patients to recruit from? Understand the patient volume, the patient population, is our patient population in general interested in participating in research? How many people do we expect and available, how many people do we expect to enroll into the trial? In other words, do I understand the numbers that are available at my site based on the eligibility criteria? Do we have sufficient resources to perform the study task? Do we have the personnel necessary to do the study? Do we have a specific radiology equipment to do the study? Do we have access to lab services to collect blood and buccal samples, for example? Do I have direct access to the patient population or do I need to recruit from a referring physician's office? So, these are all the questions that individuals cite to the trial. Individual sites have to make, in order to make a determination of what all they would need to do operationally to run the trial. So, moving on to the specific clinical resources, particularly in radiology where we may be looking at, say, diagnostic imaging, particularly in the case of TMS, we're looking for sites that had digital breast tomosynthesis machines. And ideally, these machines would be available for performance of screening mammograms. Now, in the U.S., that's a given. At most of these centers, they open the use of these devices for screening mammography near when we open this trial at many centers or shortly thereafter. But at many of our international sites, screening mammography, the digital breast tomosynthesis is not available for screening. It's available only for diagnostic services. So, within the context of the study, the only way that they can have access to screening digital breast tomosynthesis is via participation in a clinical research project. We look at, again, the resources that are available, the number of slots that are available for screening on the specific equipment that we need in order to conduct the study. So, if we have a mammo clinic schedule where screening mammography takes place, where we have 200 women who undergo screening mammography per week, once we look at or once we pre-screen for eligibility, we'll have it narrowed down to a smaller, more manageable number in a given week. Now, if I have limited research coordinator access, that restricts the research availability and that will impact recruitment, so I have a coordinator that's available only on Tuesdays and Wednesdays who would be doing the bulk of the recruitment activities, then that puts my max number of eligible people that I could approach to maybe 40 women. And again, by approach, we'll talk about different strategies and recruitment in just a few minutes. But assuming 10% to 20% of eligible patients do say yes, the site should be able to enroll between 4 and 8 people per week, that's about 16 to 32 per month, which is a reasonable rate for a screening trial. And again, it can differ depending upon the institution and the resources, personnel, staffing, the volume of patients that they happen to have. So, armed with the protocol and institutional restrictions on recruitment volumes per week or per month that may be imposed, you need to determine how best to identify eligible patients. Where in the clinical care workflow will we approach potential participants? And what recruitment methods should we use? Again, given an understanding of our patient population and our clinical workflows. So, for T-MIST, there were three methods that most of our sites, if not all of them, utilize to varying levels of success at different times during the study. Again, the goal was to come up with the, you know, try different methods to see which one works best for your patient population. And again, the resources that you have available at your facility. But in-person recruitment, common, and that was commonly employed at the very beginning of the study. One of the most efficient ways to recruit. So, when a research coordinator was available, they were in the clinic, they were recruiting when they were in the clinic. Phone recruitment was also utilized heavily, specifically at sites that needed more lead time before enrollment. In other words, they had multiple sign-offs or approvals that were necessary in order to enroll a person into the trial. Now, of course, this became also a heavily important recruitment strategy once COVID-19 hit, when clinical trials had restrictions on contact in clinics. This became a valuable resource, a valuable tool for virtually every TMS site for a given period between 2020 and 2022. Initial written material recruitment, where you're sending out maybe letters or you are promoting the study in the patient electronic medical record, patient portal, with an action that the patient takes if they're interested in participating in the trial, checking a box or saying yes, and then a phone follow-up call to tell them more about the study, give them more information, et cetera. And that represents a slightly more passive method of recruitment, especially if you do not have a large amount of personnel time to commit to it. So, the lead time between the initiated recruitment encounter through the consent process will vary from site to site, and it does depend upon who gets assigned to perform recruitment and enrollment activities for a given trial. The steps that would be involved can involve just one person, or it can involve multiple people, depending upon the complexity of the trial and all of the requirements for eligibility. Bringing up the study, confirming eligibility, going over the details of participation, specifically from the patient's perspective, answering questions that patients may have related to participating, and then ultimately performing the consent process and enrollment. Again, it varies from site to site, the amount of time it takes, but that has to factor in to the specific recruitment plan or strategy that you as a site implement. Now, obviously, you're going to need to focus your efforts and be as efficient as possible. So, the ability to pre-screen for eligibility is going to be very important when possible. Again, particularly when there's limited availability of coordinators or team members to help in terms of recruitment. So, many institutions do have research support tools available to help in recruitment to clinical research studies. So, leverage them as much as possible. Learn about the clinical research tools available to assist you within your recruitment efforts inside of your organizations. The clinical research office would be able to provide you with information. Most of this is available on your websites. Maybe public-facing websites are certainly within your portals. Clinical research tools at your institution name, Google search, should give you the basic information for how to get started and how to initiate contact with those that can help you get started. Communication is going to be an important aspect and participant recruiting support is going to be needed. Media kits that sites can use or can provide to their public relations teams can help raise local community awareness. Assess different recruitment strategies with a pilot study or at the very first sites that open to get an understanding of some of the challenges they may face. Make adjustments as needed or add additional elements as needed. Share successful recruitment strategies of top recruiting sites with newer sites is also another method to help in terms of participant recruitment support that the central trial team can at least facilitate. The key thing that we must remember when we are trying to recruit patients to studies, patients must, one, know that the study is going on. Those patients who are eligible should at least be asked to participate. The more people you approach, the more people you have the ability to enroll. So, you know, I think that's the key takeaway we found from TMIST. Those sites that had outreach to more patients recruited more subjects to the trial. Leveraging the existing community relationships that you already have within your hospital or clinic to help support your efforts will be important as well. If you already have public health fairs, you know, you have Breast Cancer Awareness Month and you're doing activities in the community during that period, taking the opportunity to bring up, you know, TMIST was one of the tools that some of our sites were able to utilize. And that can also raise awareness within the community of the trial, even for those who may not be patients at your specific hospital. And then lastly, looking at central monitoring. So, monitoring your enrollment progress. The first step is always going to be establishing the accrual target, which is done in the study design phase and incorporates prevalence of disease in the population, expected effect size, a look at prior sample sizes, and similar design studies. The second thing is to establish a monthly enrollment rate. Now, oftentimes, unless the study has kind of ramped up, you have a critical mass of sites are, you know, close to what you expect to open in your first year. You have some volume of patients that you expect to enroll. And then you establish whatever the enrollment rate is. So, in a study where we have about 20,000 patients to enroll with 50 enrolling sites, we have an enrolling rate of 850 enrollments each month. That's across all of our sites. Now, some sites will do better than a minimum of 16. Some will do less than that. Again, given their staffing at the time, as well as, you know, just their patient population and readiness to recruit. So, keeping track of the overall enrollment progress is going to be important as long, as well as individual site progress. And again, this is from the central trial team's perspective. And obviously, individual sites should be doing the same. But if I'm meeting my 850 target, exceeding that, I'm in good shape, right? I'm going to be on target, on track, or I'll be slightly faster than, you know, I'll recruit faster. But if I'm below it or well below it over an extended period of time, then I need to look at what are some of the challenges that are taking place. Again, I might have an overarching across all of my sites, a quick snapshot of what it looks like on a monthly basis, and then I might have a drill down of individual sites. And I may be particularly focused on some of my higher-growing sites and how they're doing from month to month, week to week, or over a six-month period, depending, again, upon the rate of enrollment that is anticipated for the specific study. When there are challenges in enrollment, we need to find out why as quickly as possible. Monitoring allows us to quickly assess when there are deficiencies and to make adjustments as quickly as possible when necessary. These issues may be unique to a site, or they may be a pattern that we're seeing across a lot of sites. But we need to find out what's going on at each individual site in order to make that determination. So, as Ruth alluded to, we did have a site, our Mayo Clinic site in Phoenix, had an issue with transportation. They would recruit at an enrolling center, but the imaging took place across town. And so, the patients that they would enroll in, that they had access to enroll in the study, didn't always have access to transportation to get to where the digital breast tomosynthesis systems were located. So, they developed a study and received funding to pay for the travel of their enrolled team as participants to go to the imaging center where the DBT, the tomosynthesis imaging, breast imaging would occur for the trial. And that has been successful in keeping these women on the study and undergoing the imaging necessary for the trial. But shortage of staff is another issue, out-of-pocket costs, weather issues. So, winter storms, hurricanes, all of that can knock individual sites out or multiple sites across a region at any given period. Similar to what happened when we went through COVID. Everybody was shut down across the world for a set period of months. So, again, understanding what's going on and intervening when we can is going to be important. We do educational activities, best practices, and recruitment to help improve or motivate our study teams to recruit better, providing new site training with successful sites. Again, those in the trenches working with the new sites coming on board, doing periodic surveys to identify root causes of enrollment issues, particularly early in the study, so that we understand what's going on, that could be challenges, and maybe we'll need some level of intervention or maybe some troubleshooting that may involve our data safety monitoring boards, steering committees, and oversight committees. All right. Thank you so much for your attention. Thank you very much, Lodia. That was great. And we'll now go through a question and answer period. Encourage folks to put your questions and answers on the chat, or questions, at least. We'll give you the answers. And if you'd like to ask your question live, please go ahead and raise your hand. The first question comes from John Moriarty at UCLA, who I think had a duck off, and had a question, they can see in the chat, directed to Janet Deary. Loved the talk on NOFOs, NOSIs, and PARs, and wondered if there are additional resources for early-stage investigators. I also noticed a couple of research vice chairs on this. So I think a number of folks will be interested in that question. Janet, but any directions to send them, besides us passing out your slides? Well, that's always a good one. I think, depending upon the area that the investigator wants to work, that's best handled on an individual basis, because there are some things, not so many, really, specific mechanisms for early investigators, but there are some things early investigators are considered first in the discussions for review when it comes to the budget prioritization and that sort of thing. So it's worthwhile knowing where, at the moment, early investigators are best served. And Janet, at the risk of turning on the phones at your center, I do remember the very first time I was doing this, when we were at the same place. I reached out quite a bit to Ann Mankins, a very helpful program officer. So I think sometimes doing some email or even sending up a call with a program officer can be helpful. And you'll notice that on these announcements, there's usually several program contacts. So is that okay? Like I say, for early stage investigators, these things kind of come and go and they take different forms. So I don't have a one-size-fits-all answer, but it's a good question. Once again, that's the call us category, please. Yes. And Rebecca, do you want to ask your question directly? Yeah, I just wanted to know, is there a different pay line for one of these PARs or NOFOs, however you say it? Is there a different pay line than if we're applying for like a general R01? No. The only difference where there's a pay line is for the set-aside money for the RFAs, requests for applications. So all of these right now, like I mentioned, go into the mix, into the general pool. So they're all competing against each other. R21s and R01 equivalents may have different pay lines. And then of course, large grants such as P01s, U01s, and program projects, SPORs, and that sort of thing are non-percentile. So they're handled a little bit differently. But no, all of these different announcements go into the mix. So what's the benefit of actually applying for one of these NOFOs versus just going for the general R01 pool? Is it because you'll have different experts reviewing it, that type of thing? I think that's one of the advantages. And that's why I mentioned a couple of times, talk with the program officer, read the announcement, look and see what the expertise is from the standing study sections at the Center for Scientific Review, which reviews for all the NIH grants. And sometimes it's worthwhile considering and having a conversation around your institution or with us about who has the best expertise for reviewing what it is that you're going to do. I think that's actually an important part. It's so difficult to obtain funding nowadays. It's an important part of homework and consideration about how you might want to package and craft your work, your application. And I would say, Rebecca, we ran into this at our institution is the NCI is one of the organizations that has only specific PARs for clinical trials. Even the omnibus R01 is somewhat restrictive. It makes a certain amount of sense because they sponsor large clinical trials on the many things they're involved in. So there's a portfolio going on. So I would think, and Janet may want to amplify on this, one of the reasons to keep an eye out for the PARs in the context of clinical research is to actually look for a PAR that might match exactly what you want and would be a good fit for a clinical trial, optional or required. Thanks. Jennifer, speaking of a research vice chair, we have a question. Hey. Hey, everyone. And Dr. Mankov. Yes, I have a question for Janet about the NIH funding research. It's great that NIH has lots of funding resources for the early investigators when they get started, which is great. But I just wonder, NIH also has a funding, especially for the small research program. Like I'm from University of Kentucky, right? Our research program is small, but our radiology program actually is quite big. I just wonder if you have a special funding resource for a program like ours. Yeah, there's small grant opportunities. There's also pilot grant opportunities. There's academic industrial research partnerships. But there are special ones that would be smaller, like for instance, non-medical schools that apply to like undergraduate colleges sometimes have some of these small ones as well. Yes. They do exist, like the R15, that sort of thing. Yeah, like our, again, we have a medical school, we have a CCTS, we have a comprehensive, you know, breast cancer center. But just like the radiology department, we need some help to get it started. Yes. Yeah, it's worthwhile. That's kind of, you know, I'm a big believer in the local grassroots movement. You know, what are people interested in doing? What's the natural resources of your institution? That's your investigators, your patient population, their needs, their diseases, how, where are the community problems, and what you perceive as a problem. That makes for usually a really strong research project is to start trying to have an idea about that. So what's your natural inclination? For the imaging department, who are their natural partners and the clinicians and the patients that they see? Right, Dave? Yeah, and actually, by putting on my cancer center hat as education and training director, cancer centers are required to do this. So there often be good pilot funds. And many of the big P grants or even some of the big U grants will also require a training component or a pilot component, for example, the spore mechanism in the NCI. So, you know, I think, Jennifer, those are great opportunities, if you're at a place that has some of those resources, and you're just trying to get your individual group. Actually, it's also worthwhile looking at what supplements are available to the cancer centers. Those can be nice $100,000, $50,000 pilot projects. So it's worthwhile getting to know the folks at the cancer center. And once again, saying, well, geez, we'd like it if somebody studied X in the breast cancer population with access or something. There's a lot of natural natural affinities that exist within an institution, but it's worthwhile talking to the folks at the cancer center. Also, yeah, good idea. Good questions. Other questions? I have one for Alodia, if that is all right. Thank you, Alodia, for that really wonderful talk, especially about starting up a trial. I'm wondering, for a trial like T-MIS that has multiple rounds of screening, would you have any advice about bringing back patients for follow-up and subsequent rounds? Are there special considerations? So in terms of retention, some of the key things that most of our sites were challenged with was basically just contacting them. Now, they're already coming in for routine screening, typically, but there is the need to reach out to patients. And keeping track of that, a spreadsheet to keep a list of those patients that are coming in, calendar reminders on Outlook can also be utilized to remind you when you need to reach out to specific groups of patients. Can help you stay on top of reaching out to, A, make sure that they are scheduled for their next screening mammogram, or if they've had it at another facility, getting the key information about the outcome from that outside mammogram. So those are some of the things. Having some level of communication via, again, the medical record patient portals, your EPIC systems, and all of those can be used, again, to remind people, as well as a phone call or an email message, whatever the communications mechanisms that have been approved by your local IRB for communication with existing patients enrolled in clinical trials. Thank you. I do want to, while we have a captive group here, advertise a little bit is one of the ways of getting good funding is to have a good grant. And one of the ways of getting a good grant when you're involved in clinical research is actually provided to you by your RSNA. It's good advertising for the RSNA clinical trials methodology workshops, as well as some of their grant writing workshops. A couple of us, a few of us just came back from this year's clinical trials methodology workshop. And it's really just a fantastic opportunity to learn how to structure a clinical trial, which will help you get it done, get done what you want, and usually get it funded because the better written the trial is, the better off you do. So unsolicited free advertising while we had the RSNA on the phone. I see Janie smiling a little bit. Looks like the questions are slowing down. Number one, I think we will make available to this crowd the slide sets with the permissions of the authors. So look for that announcement. Rebecca, Janie, I'll turn the floor back to you. Anything else you wanted to cover or other items? I just want to appreciate everyone for coming. Janie, do you have anything else you want to add? Actually, if it's okay, I have another question for Elodia. I was, I was curious. You talked about how during the pandemic, there became this sort of turn to digital communication tools as part of a way to continue enrolling in trials. Now that we are coming out of it, are you finding that people are wanting to do consents in person at the time of enrollment? Or are you finding people like doing consents in advance? Or do they like the video tools? Like, what is the kind of new equilibrium that you are seeing? So I will say that most of the sites themselves and the personnel working at the sites prefer remote recruitment. So they are, you know, those centers that prefer it are utilizing it still. Now there are just some sites where they have research coordinators that are actually working in the MAMO clinic, it's easier for them to recruit in person. But the majority are actually doing remote because many of their work assignments are not remote. They're not on campus as often as they used to be. And so they can still recruit to these trials, you know, via whatever tools have been cleared for them to do the remote recruitment, consenting, discussing the study, consenting even can be done electronically now. And many of the local IRBs have created the processes that they're comfortable with for doing electronic consent. So, you know, again, it's really up to the sites, but for TeamIST, the majority of them are doing remote consent processes to this day. That's really interesting for just broad clinical trials recruitment. Could I also ask, there's variable, both access to technology, like some people only have electronic access via their phones, for instance, some people are less technically savvy, maybe it's harder for some people to download Zoom to their phones. Are there ways to mitigate these technological barriers for creating disparities in enrollment? Sure. So again, if you're doing remote, you know, you can utilize the phone and you can mail the consent form in advance to the patients while you're having the phone conversation. That actually is the initial method that the CIRB figured most people could do, even without any technological, even with a patient population that does not have access to technology. So regular mail, a phone would be all that would be necessary, and you could still execute a remote consent process. Interesting. Okay, that's good. Thank you. I know we were kind of winding down there a little bit, but I thought those topics were super interesting, and I just wanted to take advantage of your presence here. If I can also ask Tori to share with those on the call, are we releasing, like how, what's the turnaround for releasing the, either the slides or the recording and particularly Dr. Carlos's recording, which may have been a little bit hard to hear, but if people have the opportunity to review it on their own, they may be able to adjust the settings a little bit more. Yeah, so shortly after the call, all registrants will receive an email from me for them to fill out a webinar evaluation. Your feedback will be greatly appreciated, and then also I'll be sending a link to access the slides, and then also the recording will be available in mid-March. All this information will be provided right after this call. Terrific. Thank you. Do we have any other final comments or questions from the group? I feel like it's, we want to take advantage of those who are on the call, because I know you all have terrific perspectives as well. Hearing none, Rebecca, do you want to close us out? Yeah, I just want to thank everyone for participating, our speakers especially, and also for the RS&A staff led by Tori for coordinating this, and my co-moderators here, Dr. Mankoff and Dr. Lee. I hope that you guys found value in this, you know, your participation in asking questions, and I hope that we can continue to do that in the future. Thank you. Thanks, speakers, and Tori. Thank everybody. CF, it's nice to see you all.

clinical research

inclusivity

funding opportunities

diversity

clinical trials

participant recruitment

community engagement

trial design

patient retention

COVID-19 challenges

early-stage investigators