Regulatory Education for Industry (REdI) Annual Conference 2023 Day 2 Session 2

foreign I hope you were able to stretch and get some coffee or tea uh the next we will have two additional presentations this in this next set before lunch break our next presentation is on the modernization of clinical trials through digital Health Technologies decentralized clinical trials and point of care trials presenting is Miss Beth konkowski Health Science policy analyst clinical methodologies in Cedar's Office of Medical

Office of Medical policy after this we will hear about padufa 7 real world evidence from Captain Kimberly Smith who works in real world evidence analytics in Cedar's Office of Medical policy please join me in welcoming our speakers hello my name is Beth kinkoski I am a health science policy analyst in Cedar's Office of Medical policy today I will be discussing the modernization of clinical trials through

digital Health Technologies decentralized clinical trials and point-of-care trials all of us are privileged to be participating in a Technologic revolution technology dominates almost everything we do and has opened all sorts of new opportunities Information Technology knows no Geographic limits data can be instantaneously transmitted from anywhere in the world looking at clinical trials there are many opportunities to embrace these advances it's obvious that many of traditional

processes in clinical trials can be modernized decentralizing activities in clinical trials is not new patient Diaries interactive voice response systems and telephone follow-up have been used in clinical trials for years covid-19 changed how we conduct clinical trials not having patients travel to trial sites allowed many ongoing trials to continue during the public health emergency during this time FDA published a guidance on the conduct of clinical

trials during the public health emergency FDA recently published a draft guidance on decentralized clinical trials for drugs biological products and devices a decentralized clinical trial is a trial where some are all trial related activities occur in locations other than traditional clinical trial sites decentralized clinical trials might involve the use of Telehealth visits digital Health Technologies direct distribution of product to participants use of electronic informed consent

use of Home Health visits to trial participants or the use of local Health Care Providers and Facilities why are Regulators interested in decentralized clinical trials to improve patient accessibility to allow patients with rare diseases or patients with Mobility or cognitive challenges to participate to enhance the diversity of patient populations to enhance patient convenience to facilitate trial efficiencies and to reduce the spread of contagious diseases one

of the first things to consider when conducting a DCT is the design of the DCT itself we see trial activities potentially occurring at locations such as participants personal Physicians offices or their home so when designing the DCT and determining the assessments to be performed consideration should be given as to whether an assessment should produce similar results regardless of whether it is conducted remotely or at a

trial site for example if a participant is performing homeless spirometry the result of this assessment may differ and if it was performed under the direct supervision of an investigator at a trial site further assessments performed by local Health Care Providers as part of routine clinical practice for example evaluation of a participant's symptoms may be more variable and less precise than assessments conducted by trial Personnel this

Factor also should be considered when designing a DCT another consideration for study design as it relates to dcts is the statistical analysis to be performed in non-inferiority Trials if the effect size of an active controlled drug has only been determined in a traditional site-based clinical trial this may differ in a DCT setting in addition as mentioned the variability and precision of the data obtained in DCT

May differ from that of a traditional site-based clinical trial this would not affect the validity of a finding of superiority but it could affect the validity of a finding of non-inferiority in terms of inspections related to clinical trials while many locations used in DCT may be remote there should be a physical location where all clinical trial related records for participants under the investigators care are accessible

and where trial Personnel can be interviewed this location will need to be included on form FDA 1572 for drug trials and in the IDE application for device trials when conducting a decentralized clinical trial you may consider remote trial visits these allow the investigator to oversee trial related activities at other locations there may be challenges associated with remote visits including local regulations on telemedicine inability to conduct

physical examinations use of Photography may be helpful but also has limitations investigators may not have the same relationship with trial participants with the absence of in-person visits if there are complex Drug Administration procedures it could be Troublesome some investigational products may require close medical Supervision in case of adverse effects dcts may also allow for home visits this novel approach extends the physical reach of the trial

and enhances convenience for trial participants this may allow participants to have Mobility challenges to still participate in a trial either dedicated trial Personnel or contracted fee for service staff may go to the patient's home or other location of preference to collect trial related data now there are also mobile trial units that bring a lab to a more rural area to enhance convenience for trial participants just

like in a traditional site-based clinical trial in a DCT the investigator is responsible for the conduct of the trial and the individuals delegated to perform trial related activities a key difference for dcts is the extent to which the investigator uses Telehealth trial Personnel working remotely local Health Care Providers and or digital Health Technologies in the conduct of the trail decentralized features May necessitate additional training coordination

and standard operating procedures to ensure consistent implementation dcts may allow for the direct distribution of investigational product to trial participants in all trials the investigator must control release of the product to trial participants local state laws differ on Direct distribution to patients they may require locally licensed Health Care Professionals such as a pharmacist to distribute the investigational product sponsors must consider the packing handling and shipping

of products another factor that should be described in the protocol is how investigators will track and document their child participants or legally authorized representatives receive investigational products Additionally the protocol should describe the procedures that investigators or participants should use to return or dispose of unused investigational products and how this return or disposal will be documented local Health Care Providers such as doctors or nurses may be

used by sponsors or investigators to perform certain trial related activities this may be done for example on a fee-for-service basis again such local hcps would not be considered trial personnel please note though there are certain stipulations regarding the trial related Services local Health Care Providers should provide these Services should not differ from those that they are qualified to perform in clinical practice and also should not

require a detailed knowledge of the protocol or the investigational product these Services might include for example obtaining vital signs or performing a physical exam on a trial participant if there are activities required by the study protocol that are unique to the research or do require a detailed knowledge of the protocol or the investigational product these activities should be performed by qualified trial Personnel who have been

appropriately trained the idea of allowing local Health Care Providers to perform certain trial related activities that fulfill the criteria just mentioned is that the local health care provider would simply be performing tasks no different than what they may perform in the course of their usual practice in which do not require a specialized training to perform such local Health Care Providers are already licensed and regulated in

terms of providing such Services their expertise could be relied upon to support the clinical trial in terms of Delegation of trial related activities when permitted by the trial protocol investigators can delegate child-related activities to local health care providers in order to allow the conduct of activities that require in-person interactions with trial participants these activities might include a physical exam drawing blood taking blood pressure these activities

may occur at a local Health Care Facility or at a participant home one critical consideration regarding delegation of such trial related activities to local Health Care Providers is the potential for variability in the approach across different practices for certain activities these might include what data are documented when collecting Vital Signs how a physical exam is performed or how evaluation of Adverse Events is conducted in order

to help reduce variability in such activities quality control measures including regular review by investigators of participant data entered by local Health Care Providers should be performed this will help in the assessment of the consistency and completeness of the required procedures the type and scope of quality control measures should be tailored to the criticality of the data the complexity of procedures done by the local health care

provider in addition to delegation of in-person activities to local Health Care Providers investigators May utilize Technologies such as video conferencing to allow for direct oversight of trial activities delegated to trial Personnel these might include activities such as fitting of wearable sensors photographing skin lesions or possibly conducting physical exams investigators conducting drug trials is either traditional or dcds must complete form 1572 the statement of the investigator

this form includes information such as the name and address of the investigator and the research facility the name and address of Clinical Laboratory facilities to be used in the study and the names of any sub-investigators in terms of determining when trial Personnel should be included as sub-investigators on form 1572 if they are contributing directly and significantly to the trial data they should be included as sub-investigators

on form 1572. for further information on completing form 1572 please see the frequently asked questions on the statement of the investigator local Health Care Providers should not be listed on form 1572 as sub-investigators local Health Care Providers should however be included in a task log which we will discuss next to prepare and maintain adequate case histories investigators must maintain a task log of local health care

providers who perform trial related activities the task log should include the name and affiliation of the local health care provider a description of their role and assigned tasks the date these local Health Care Providers are added to the log as well as the locations where these activities are conducted the test log should be dated and signed by the investigator when it is initially created and updated

when a new local healthcare provider is added to the loan the test log should be available to FDA during inspections electronic informed consent is fairly common in all clinical trials these days the regulatory requirements for obtaining informed consent and IRB review do not differ for dcts electronic informed consent allows patients to review and sign documents at their convenience at home investigators May provide videos and other

Graphics to make the process informative and more easily understood FDA published a guidance on electronic informed consent in 2016. if you would like to learn more about the DCT guidance please join us on June 20th for a webinar on the guidance the DCT guidance is a draft guidance please submit comments to the docket by August 1st to test to see what you have learned about investigator

responsibilities in the decentralized clinical trial here's a challenge question all local Health Care Providers participating in a clinical trial would need to be listed on a form FDA 1572 the statement of the investigator is this true or false and the correct answer is false these could be listed on a task log now let's discuss how new technologies have enabled new capabilities in clinical trials we all

recognize the profound impact that new technologies have on our ability to gather clinical data computers that formerly occupied entire buildings are now condensed into our cell phones with extraordinary capacities for data storage and Analysis digital Health Technologies can transmit data from patients wherever they are as well as provide opportunities to quantify clinical features more precisely a digital Health technology is a system that uses Computing platforms

connectivity software and or sensors for healthcare and related uses some common examples include portable sensors such as activity trackers or mobile apps digital Health Technologies allow access to more frequent or continuous data if the ability to detect sporadic events that might not be caught in a clinic such as Caesar's arrhythmias or Falls they can collect patient reported outcomes such as ecological momentary assessments they allow for

collection of data in the real world environments and to record objective functionality here are some examples of digital Health Technologies some of them are sensor based such as continuous glucose monitors ECG monitors or ectography some dhcs are mobile applications run on smartphones such as patient reported outcomes or coordination tests for neurological disorders FDA published a draft guidance on digital Health Technologies for remote data acquisition in

clinical investigations in December 2021 FDA has been reviewing these docket comments and is working towards finalizing the guidance some dhts meet the definition of a medical device others do not this definition is defined in the food drug and cosmetic ducts I have determined my DHT as a device now do I need pre-market clearance or approval to use the device in a clinical investigation for another medical

products devices intended only for use in clinical investigations are typically exempt for many requirements applicable to devices including pre-market clearance or approval as long as the investigation complies with applicable requirements under 21 CFR part 812 also known as the IDE regulations when is an investigational device exemption application to FDA required for a DHT that is a non-significant risk device in ide application to FDA is generally

not required if my DHT is a significant list device when all information required in an IDE application is also contained in an IND FDA generally does not expect sponsors to submit a separate IDE for these clinical investigations if my DHT is a device used in accordance with its authorized intended use an IDE application to FDA is generally not required here's another challenge question All Digital Health

Technologies need to be cleared or approved to be used in a clinical trial true or false false as we just discussed not all devices need to be cleared or approved to be used in a clinical trial when it's time to select a digital Health technology for your clinical investigation it's important to start with some basic fundamental questions of is the digital Health technology comfortable is the

participant going to put it on and wear it all day is it easy to use does the battery last as long as possible what's the process to sync the data to ensure that the sponsor receives all of the data they need for their protocol is it a bring your own approach for a participant is using their own uh digital Health technology after selection of the dhg

it's time to think about verification and validation of the DHC measurement this is a technology question to evaluate the DHT how accurate and precise is the DHT in measuring the targeted feature such as steps or sleep does the algorithm used to interpret the loss signal reliably represent the clinical characteristic or event we're trying to measure are the data recorded by the DHT and trial participants the

same as the data we would report if we were looking directly at the trial participant justification of the endpoint is a clinical question about the feature being measured regardless of the technology used to make that measurements is the endpoint a clinically meaningful measurement of drug effects usually comparison within an existing Benchmark of performance such as the updrs or other patient reported outcome or the six minute

walk distance is a good way to assess this collaboration for many different stakeholders such as patients caregivers Physicians disease experts and Regulators can help to assess if a endpoint is a clinically meaningful measure developing a novel endpoint based on a DHT derived data can be challenging first you have to decide what you're measuring such as steps then you need to define the window of observation and

the response that will be seen in each patient there are many other aspects that should be considered when running a trial using dhts there should be trial participant and Personnel training technical support should be available for their participants and personnel in addition there must be a plan to address updates to DHT hardware and software it is fairly common for security updates to be rolled out regularly

and sponsors must ensure these updates do not impact the Integrity of data collected for the trial 21 CFR part 11 outlines electronic record protection and retention requirements when first designing a trial based on DHT data sponsors should discuss with relevant rear divisions the type of data recorded from each participant to be submitted to the FDA for review the data output of the DHT and its Associated

metadata should be transmitted from the DHT to a durable electronic data Repository The Source data is generally considered the data housed in the durable electronic data repository and not the data stored on individual dhts additional information is available in the newly published draft guidance on part 11. electronic systems electronic records and electronic signatures questions and answers dhts may allow us to modernize the performance of Trials

and improve trial efficiencies they allow for measurement in challenging populations such as neonates or patients with dementia they may allow measurement of rare events which are difficult to capture such as arrhythmias seizures and expose they allow for the measurement of new clinical characteristics such as stamina gain stability and Tremor they also provide a more convenient or precise way to measure existing features such as sleep exercise

or blood pressure here's another challenge question justification of an endpoint is clinically meaningful depends on the type of DHT used in the clinical investigation true or false this is false justification of an endpoint is independent of the dhtus trials in the clinical practice setting the opportunity to use existing clinical infrastructure particularly when supported by interoperative data system has become an area of increasing interests the recovery

trial in the UK for covid-19 reportedly recruited 40 000 covid patients through the national health system in the UK within six weeks they were able to show the mortality benefits of steroids in treating patients hospitalized with covid practice settings allow engagement of large numbers of patients in short periods of time they reflect the effectiveness of treatment and real world environments and the accessibility of clinical trials

to patients who wouldn't normally participate FDA is committed to modernize clinical trials incorporate technological and scientific advances and potentially address the enormous costs and burden of drug development there's interest from Congress industry and others in the community to enhance the efficiencies in drug developments there's interest of new stakeholders Engineers DHT manufacturers to support clinical trials and patients are seeking more convenient ways to participate in research

Congress also passed several pieces of legislation in the last year to modernize clinical trials they passed the prescription drug user fee act to and this includes a specific section on enhancing the use of digital Health Technologies to support drug development and review and at the end of last year they passed the Food and Drug omnimus Reform Act which includes specific sections on decentralized clinical studies and

modernizing clinical trials and both of these have guidance requirements associated with them the producer 7 DHT commitments include developing a framework establishing a DHT steering committee convening public meetings identifying and funding demonstration projects developing guidance expanding review capabilities and enhancing I.T capabilities to review DHT generated data and submissions FDA published the framework for the use of digital Health Technologies and drug and biological product development in

March of this year FDA published a draft guidance on part 11 earlier this year this is electronic systems electronic records and electronic signatures in clinical investigations questions and answers FDA held the first of its public workshops in March of this year on understanding priorities for the use of digital Health Technologies to support clinical trials for drug development and review Cedar has a new web page dhts

for drug development this webpage will highlight all of the work we are doing to meet our padufa commitments one padufa commitment is to drag submissions containing DHT derived data do this we recently updated forms 1571 and 356h next time you complete these forms please be sure to check the box yes if your submission contains digital Health technology data modern clinical trials provide opportunities to improve trial

efficiencies convenience for patients access to divide reverse participants and participants with rare diseases to broaden how we conduct clinical trials to collaborate with many different stakeholders and to facilitate drug development engaging with the FDA if you have plans to conduct a DCT or use ADHD in a clinical investigation please reach out early in the development phase if you have a specific drug you are developing contact

the appropriate therapeutic review Division and request a representative from the DHT steering committee participate in any discussions if you are considering a BHT or DCT not associated with a specific drug development program and we would like to discuss General feasibility please reach out to dhcs for drug development at fda.hhs.gov thank you now Captain Kim Smith will discuss real world evidence at the conclusion of her presentation

we will both be available to answer any questions thank you my name is Kim Smith I'm with the real world evidence analytics team in Cedars Office of Medical policy and I'm happy to be here with you today to discuss the verbal debits Provisions under padufa 7. the objectives of today's talk are to describe the fta's real world evidence program for drugs and biologics to understand fg's

approach to evaluating real world evidence for Effectiveness and to discuss FDA initiatives related to World evidence including new commitments under padufa 7. so to start with a little bit of background on the U.S Effectiveness standard it's grounded in law with the Food and Drug cosmetic Act of 1962 which states that it directs Effectiveness must be established by substantial evidence consisting of adequate and well-controlled investigations that's

generally been interpreted as calling for two adequate and well-controlled trials with the Food and Drug modernization Act of 1997 substantial evidence could also be met by a single trial plus confirmatory evidence which gives some flexibility in the evidence needed to support Effectiveness claims this is further laid out in regulation with 21 CFR 314 126 describing the characteristics of adequate and well-controlled studies to support claims of

Effectiveness for new drugs and in Guidance with the draft guidance published in 2019 on demonstrating substantial evidence of Effectiveness for human drugs and biological products so this is the background on which we're building the real world evidence program for drugs and biological products because our Effectiveness standard has not changed so fast forward to a time when the Health Care system is generating large volumes of clinical

care data that could potentially be harnessed to inform our regulatory decision making now this is nothing new uh FDA has a long history of using real world's data to Monitor and evaluate the safety of approved drugs in the post-marketing setting once we get out of the clinical trial setting we have a lot more patients taking the drugs for a lot longer time and we can sometimes

detect safety signals that you wouldn't ordinarily see in a clinical trial setting or a study setting and so we have always used these types of data to inform our understanding of the safety profile of drugs we have also used real world's data and real world evidence to support product Effectiveness uh but at a really more limited basis it's often that our our decisions have been based

uh on a more traditional clinical trial paradigm that said uh we now have a lot more data than we used to and also some advances in the uh analyzes of real world data and so this is leading to increased interest in the potential use of such data to support regulatory decisions uh in areas where we typically hadn't focused in the past so in 2016 Congress passed

the 21st century cures act which was designed to accelerate medical product development and bring new Innovations faster and more efficiently to patients it included some Provisions related to real world evidence and in response the FDA established a program to evaluate the use of real world evidence to support new indications for a drug or to satisfy post-approval study requirements the agency issued a draft framework in 2018

which describes sources of real world evidence as well as opportunities and challenges in the use of world of evidence for regulatory decision making the agency uh is follow-up to issuance of the framework issue multiple draft guidances between 21 2021 and 2022 and underline all of this as I said a couple slides back our Effectiveness standard remains unchanged so the task before us is really to understand

when can railroad data and railroad evidence actually meet or Effectiveness standard and be used to support a new indication for a drug or to to meet post-approval study requirements so here's just a screenshot of the cover of fdu's framework for real with evidence and there is a link down below if you're interested in learning more the framework applies to the Center for drugs Cedar the center

for biologic sieber as well as the oncology center of excellence oce of note has their own real world evidence program just because their regulatory framework for devices differs somewhat from that of drugs and biologics the framework provides a multi-faceted program to implement real world evidence including internal processes within the agency engagement of external stakeholders the development of guidances as well as demonstration projects the framework is

grounded in two key definitions and I'll use these terms a lot in this presentation so just to be clear from the outset on what I'm referring to real world data are data relating to Patient health status or the delivery of Health Care that are routinely collected from a variety of sources real world evidence is clinical evidence regarding the usage and potential benefits or risks of a

medical product derived from analyzes of real world data to continue with terminology a bit more we have Interventional studies which is what we've traditionally called clinical trials in in Interventional studies patients are assigned to the treatment or the intervention by a study protocol this is in contrast with non-interventional studies often called observational studies in which patients receive the treatment of interest during routine medical care it's

a treatment that was prescribed by their medical provider uh this is not a dichotomy necessarily so studies can have components of both so for example you can have an externally controlled trial where the Interventional treatment arm was determined by a study protocol for instance a single arm trial and the outcomes in those patients are compared with patients who received a different treatment through routine medical care

so the control arm is a non-interventional or observational group there is a spectrum of Reliance on rwd in studies it's not necessarily all or none so on the left in this table you have the traditional randomized trial where you may use real data in the planning of the trial for instance to assess feasibility determine appropriate enrollment criteria maybe select trial sites but all of the other

data from The Trial are collected through trial sources you set for less and in electronic case report forums and so on on the far right uh the opposite end of the spectrum would be a completely non-interventional or observational study so this is where patients are getting routine Medical Care with treatments prescribed by their medical provider and data is collected as part of routine care whatever Labs

the provider is ordering and is being collected in electronic health records or maybe claims data there is uh an area in the middle though where there may be components of a traditional randomized trial but some of the data for the trial are collected through routine care practices uh so maybe your randomizing a patient to treatment but then following that patient through data collected routinely and gleaned

from the medical record or as I mentioned on the last slide an externally controlled trial where there are components of both an Interventional and a non-interventional treatment groups so the framework also lays out how the agency assesses real world evidence for that regulatory decision making and the three buckets uh in terms of key considerations are the real world data are they fit for use uh is

the study design appropriate and then is the study conducted in a cord with regulatory requirements we'll go into each of these in a little more detail in subsequent slides so to start with data so whether the real world data are fit for use there are really two major components of that reliability meaning are the data accurate are they complete are they traceable can we link them

back to an actual measure with an actual patient so that we can have confidence that the data that we have our our real data and then relevance uh are the data that you have relevant to the question of Interest so do you have the data elements you need to answer the question and are the patients representative of patients who will use the drug and you have

sufficient numbers of those patients to draw conclusions from the data we have several draft guidances that have published uh that deal with data considerations so key sources of rural data are electronic health records and medical claims there is a guidance that was published in September of 2021 on assessing ehrs and medical claims data to support regulatory decision making Registries are another common source of railroad data

so we have published a guidance in November 2021 assessing Registries to support regulatory decision making for drugs and biological products and then finally not specifically related to a Davis Source but how those data are handled and submitted to the agency because there are requirements for adherence to certain data standards and so there's another guidance that was published in October 2021 that addresses data standards for drugs

and biological projects product submissions that can chain real world data so it won't go into detail on all of these but uh readily accessible on the web if you're interested in what we say about uh each of each of these topics moving on to another key consideration when evaluating real world uh data and ruled evidence for regulatory decision making and that's the study design so we

need to have confidence that the trial or the study design can provide adequate scientific evidence to answer to help answer the regulatory questions so can the study as designed meet requirements for the intention of the study if it's more effective disclaim would it meet the requirements for the U.S Effectiveness standard and so we do have one guidance that has been published in this space to date

and that is on considerations for the design and conduct of externally controlled trials for drugs and biological products that was published in February of 2023 and as noted on Cedar's 2023 guidance agenda we have two additional guidances that are in development so one is on considerations regarding non-interventional studies for drugs and biological products and another I'm using clinical practice data and randomized controlled trials so stay

tuned for both of those guidances and then finally in terms of the three key considerations uh here are the regulatory considerations so does the study conduct meet FDA regulatory requirements to support a decision a regulatory decision and so here key is protection of human subjects so in situations where you may need to get informed consent uh for for the study um and and to protect data

from Human subjects being transparent about the study design and the study conduct pre-specifying how the study is going to be conducted and analyzed access to data by FDA so patient level data to allow us to explore the data understand the the nature of the data and conduct any additional analyzes we need to to understand the findings of the study and then finally study monitoring so ensuring

that the study is being conducted in accordance with the protocol and to meet the requirements from a regulatory perspective so there is a draft guidance on all of these considerations uh it's called the considerations for the use of verbal data or what evidence to support regulatory decision making and that one was published in December of 2021. a couple excerpts from the regulatory considerations guidance I think

are important to highlight so the first is that regardless of the studies Interventional or non-interventional design the evidence submitted by a sponsor in a marketing application to support safety or Effectiveness must satisfy the applicable legal standards for the application to be approved or licensed and there are a lot of considerations with non-interventional studies that differ somewhat from uh Interventional studies and so it's really important in

this space for sponsors to engage with the agency in the early stages of Designing another Interventional study that's intended to support a marketing application so shifting gears a bit from the 21st century cures Provisions to padufa 7 which will cover fiscal Year's 2023-2027. we have four reward evidence Provisions under pity for seven at the top FDA will establish an advancing rural evidence program will report aggregate

data on an annual basis describing rwb submissions to Cedar and sieber the third is we'll convene a public Workshop or a meeting to discuss rwe that meets regulatory requirements and then finally we will use Lessons Learned in this process to update existing or to generate new guidance documents in the rwe space so to start with the talk uh we were directed to establish a real live

evidence program to identify approaches for rwe that meet regulatory requirements the program is also intended to develop agency processes that promote consistent decision making and finally to increase awareness of world evidence characteristics that support regulatory decisions so the advancing world with evidence program on the left you'll see a screenshot from our website and the link is below in case you'd like to get additional information this

is a program that was announced on October 20th 2022 it allows for up to four meetings with sponsors in the agency to discuss the use of robot evidence in medical product development it is an optional program all of the usual opportunities to meet with the agency through established meeting Pathways remain available but this is a a new opportunity to have some of these more focused discussions

on the use of rwe so the goals of the program are to identify approaches for generating rwe that meet regulatory requirements to develop agency processes related to their review of role of evidence proposals and to promote awareness of real world evidence that can support regulatory decisions as you'll see one of the components of the program is a disclosure agreement between the sponsors and the agency that

allow the agency to discuss uh the work under this program in a public forum to to promote that awareness so to be eligible for the program sponsors must have an IND or a pre-inda for a specific product in indication this is not intended to discuss sort of more General considerations related to a room with evidence and Rural data the real world evidence proposed must be intended

to meet regulatory requirements in support of either labeling for Effectiveness such as new indications populations or dosing informations or be intended to meet post-approval study requirements and there needs to be agreement on the information that that the agency can publicly disclose related to The Proposal in the discussions so the submission deadlines are twice per year March 31st to September 30th and that continues through the March

31st 2027 deadline at the end of padupa seven the initial request is a 12-page summary of The Proposal if you're interested in what that should contain all of it is on the the public website for each cycle FTA will review all proposals received in the preceding six-month cycle and will accept one to two initial requests in the first two fiscal years so FY 23-24 and one

to four uh requests in FY 25 to 27. that decision will be based on you know first and foremost the scientific merits of The Proposal but also given that this is a program intended to encourage learning and exploration about potential uses of real world evidence it will also take into account diversity of data sources study designs methods as well as regulatory indications sponsors will be notified

whether their initial meeting requests of the program is accepted within 45 days of the deadline so here is it's a visual representation of that cycle so on the left you see that the sponsor submits the initial meeting requests on Day Zero which is either March 31st or September 30th the agency has 45 days to evaluate the requests and by day 45 will notify all sponsors of

whether they're accepted uh or they're not accepted and the acceptance is pending reaching agreement on those disclosures and then by d75 we'll have the first advancing rwe program Heating sponsors can take you know information gained from the discussion and and can determine whether follow-up meetings would be a benefit and if so at their discretion they can request up to three additional meetings through the program and

if granted those meetings will occur within 45 days of the follow-up meeting requests so as I mentioned uh once we uh notify a sponsor that they've been tentatively selected to participate we need to reach agreement on disclosures in terms of what design elements the FDA may present publicly as case studies uh and once we have assigned agreement letter we move forward with that initial program meeting

so the second provision under Paducah 7 related to real world evidence by June 30th of 2024 the fdas report aggregate data on an annual basis describing submissions to Cedar and sieber including data sources study designs and types of regulatory requests so more to come on that starting uh in 2024 but at this time just want to call your attention to a final guidance published in September

of 2022nd uh on uh submitting uh documents with real world data World evidence to the agency which requests that when you submit these documents that you include in your cover letter certain information that will allow us to identify that the submission includes rwd rwe so at the guidance in the guidance at the link at the bottom you'll see an example table that goes through different categories

the purpose of using the rule of data realid evidence the study designs uh that you're proposing to employ as well as the real world data sources that you're proposing to is with your submission the third provision under Paducah 7 is that by December 31st 2025 the agency will convene a public Workshop or meeting to discuss case studies and that will be focused on how to generate

uh real world evidence to meet regulatory requirements so more to come in a couple of years on that and then finally I mentioned uh several guidances that we've published and more to come the FDA will use the lessons learned from the advancing World whatever this program and the other work under Paducah 7 to update existing guidances to generate new guidances uh as needed and that will

work will occur by December 31st of 2026. so to summarize uh there's a lot of work being done in the space the fda's world without this program is just advancing as was described in the 2018 framework verbal evidence presents some new challenges with regards to data to study design as well as to regulatory conduct of these studies to try to move this field forward and to

explain our thinking in certain areas FDA has issued multiple guidances intended to inform the use of world evidence for regulatory decision making and we have several new initiatives underway that aim to identify and promote awareness of verbal and evidence-based approaches that meet regulatory requirements with several new provisions so uh moving on to our first challenge question for this talk uh the 21st century cures act changed

the effectiveness standard for new drug approvals to allow for approvals to be based on room with evidence so is this true or false just think about your uh your answer to this question so this one is uh false hopefully you've got that one correct uh the 21st century cures act did not change the effect of this standard it just directed The Agency to develop a world

evidence program and begin to explore how real data and real world evidence could be used to support regulatory decision making challenge question number two which of the following statements is not true a real with evidence can arise from both Interventional and non-interventional studies the use of verbal data to understand trial feasibility is considered real world evidence determining whether data are fit for use includes consideration of

reliability and relevance or it is important to engage FDA early when considering use of a non-interventional study to support a marketing application which of these is not correct all right hopefully you got the uh the use of verbal data to understand try our feasibility is considered real world evidence sometimes we hear it described in this way but really the you know types of Explorations that are

done to understand trial design we don't consider to be real world evidence uh that's sort of traditionally uses of real wood data but real world evidence is really uh understanding the benefits and risks of products through analyzes of real world data so if you're interested in more information on any of the topics I discussed there were links I think throughout the presentation to the various guidances

the framework Publications so that is a good place to start we also have an email address for questions that's shown here on the screen and feel free to reach out if you have specific questions we can help to direct you further so thank you very much for your time and uh we will be available uh soon to answer some questions thank you thank you both for

those very informative presentations we will roll right into our panel session so for the audience if you have not had a chance to enter your questions into the Q a chat pod this is your chance to get your answers uh your questions answered so please do so now we will try to answer as many questions as time allows so it looks like we have some questions

that have already come in for miscon do yours first a few questions for you the first question is have applications been accepted into the advancing real world evidence program what advice could be provided to support a successful application foreign yes and this is Ken Smith I'll take this one thank you for the question so we did uh just pass our first deadline on March 31st uh

for submissions to the program for uh initial meetings and we have completed our evaluation process um and have selected initial programs to participate uh more to come on that once we've had the discussions and have some learnings to share with the group more broadly uh in terms of advice for a successful application um I think start with the uh externally facing program website which goes through

in detail the information we would like to see in the initial meeting proposal uh and make sure you adhere to the format and content guidelines as well as sharing the information requested and is clear and succinctive manner is possible I think that is an excellent starting point it allows us to to carefully evaluate each proposal into German which to select for each cycle thank you thank

you Captain Smith uh since we have you let's give you one more question would the use of real world data such as an external control arm a political trial be used only for critical or rare conditions are there limitations on the types of indications being studied yes thank you for the question so we don't have uh sort of specified limitations on situations where we would be

willing to accept rural evidence to support a regulatory decision so I think at this point it's really on a case-by-case basis through early discussions with the agency you know things that we do consider are situations where there is unmet need where it may not be feasible or practical to conduct a different type of a study um you know I think we all recognize that there can

be some limitations uh in the use of verbal data and real with evidence uh and so generally we're looking for situations where the uncertainty that may result could be minimized and also um it would be reasonable to accept the the level of uncertainty that may result uh from the use of a real world evidence study to support the decision so sometimes that is in situations where

it is a severe rare disease and it's challenging to conduct other types of studies but we are certainly open to proposals in other areas and evaluate them on a case-by-case basis based on the criteria that I shared earlier thank you thank you Captain Smith another question for you why would a sponsor pursue a meeting through the advancing real world evidence program rather than through another meeting

pathway yes thank you so I think the advantage of the advancing real world evidence program is this really is a focused Pathway to discuss proposals involving robot evidence so you know we have a team that's been convened to evaluate these proposals and to discuss these proposals with sponsors that's really has expertise from across the agency in the use of rural data and real world evidence for

regulatory decision making so we know that there are issues that we need to work through some aspects of the use of these types of data in studies differ from our more traditional trial paradigms and so this is really an opportunity to discuss in the context of a specific development program many of those issues with the right people in the room it also allows for some iterative

interactions on a more quick time frame than some of the other meeting Pathways would allow because the program allows for up to four media things on The Proposal with follow-up meetings having being held within 45 days of the request so I think the hope is that the program will allow for number one in all hands on deck approach with all the right people in the room

to address the issues raised by a proposal and also to be able to do so in an iterative fashion through multiple interactions over a relatively short period of time thank you thank you Captain Smith so now let's move on to Ms kankoski so your first question do the decentralized locations need to be listed on the 1572 or is it only the main site thank you for

that question I I think that's the golden question when it comes to a 1572 and decentralized clinical trials um so the guidance discusses that for inspectional purposes there should be a physical location where all clinical trial related uh Records for participants under the investigators care are accessible and where trial Personnel can be interviewed this location should be listed on the form 1572 or for ID uh

IDE applications they must be included in the IDE application and I think some of the other questions that we're going to get into now um talk a little bit more about how you make that distinction in a decentralized trial thank you another question for you are there examples of decentralized clinical trials where an investigational drug a drug that that has not been approved for any indication

is shipped directly to participants home so the specifics of what has or has not been approved in an IND um are generally confidential in nature so I can't discuss that specifically but I can talk a little bit to uh and the guidance it talks a lot about more of the risk-based approach of how to decide um if a BCT is appropriate for whatever drug you're studying

so some of those lower risk uh drugs that a patient can administer on their own and is not likely to have a serious adverse effect would be appropriate for shipping directly to a patient's home again that's discussed a little more detail in the draft DCT guidance that is out right now thank you let's take one more question for you we have one here with local Health

Care Providers need to be listed on the 1572 as sub investigator thank you uh so no um local Health Care Providers um can be listed on what we are considering the task log that is kept um uh by the investigator um and that is just a running list of any local Health Care Providers their name um licensing affiliation where they're located and then it would be

a running list of the activities and the dates and times that they those local Health Care Providers conduct as part of the clinical investigation thank you let Elizabeth with a few more questions Captain Smith when is it appropriate to use real world evidence to support a regulatory decision yes thank you I mean I think it's important to note that rwe can involve various study designs so

I think we're often thinking of non-interventional or purely observational studies but this can also involve single arm trials with comparator arms from railroad data it can also uh involve randomization where your randomizing patients to an intervention of interest but then you're collecting their outcomes uh using real world data so for instance you're you're following electronic health records or medical claims data to assess what happens to

the patients following their randomization to a particular treatment so I think it's important to understand that real with evidence is a fairly broad construct regardless of the type of study design however you know as I mentioned there are substantial evidence standard Remains the Same and so you know we really need to to Think Through situations uh where we can minimize these certainty involved and we can

have a clear sense that the drug is producing the effect that we're approving it to uh produce if you will so I think it's it's again on a case-by-case basis but really understanding uh the data sources the study design as well as the regulatory conduct so that we can have confidence that the results of the study uh warrant the regulatory decision that we're making thank you

thanks Captain Smith we have another question that came in for you is the agent looking for particular types of study proposals for the advancing real world evidence program thank you uh no not necessarily so we are looking for proposals that are linked to a specific drug for a specific indication so not a proposal for instance just to address some some general Concept in the room with

evidence space we really want to be talking about a specific idea that would support a new indication or meet a post-approval study requirement for a drug and beyond that we're going to be looking as I had mentioned at the scientific marriage or the proposal are the proposed data sources likely to be appropriate is the study designed likely to be adequate to answer the question and then

is the sponsor going to be able to conduct the study in a way that would meet our regulatory requirements you know initially we have a you can cast a fairly broad net over time we're going to be looking for ideas that may be more novel because we do want to have a breadth of experience variances through the program in terms of you know different indications different

types of methods data sources Etc so no Beyond a specific drug and a specific indication where we don't have additional criteria at this time that would narrow the types of submissions that we're looking for thank you thank you moving back to miss konkowski um you need to be an agency review of protocol specific definitions of healthcare professional activities to verify that the healthcare professional is the

appropriate designation versus Sub investigator thank you I'll start with the the broader picture of making that distinction of the sub-investigator versus a local Health Care uh provider and there is a guidance that has been out there for some time on uh it's called the information sheet guidance for sponsors clinical investigators and irbs frequently asked questions uh statement of the investigator or form FDA 1572 and this

was updated in May of 2021 and this uh really talks about the distinction between the investigator and the sub investigator and now that we've come out with the decentralized clinical trials guidance we break that down a little farther section three um D2 is the investigator roles in that uh guidance and there it talks about how local Health Care Professionals are contracted to provide trial related services

that are part of routine clinical practice for example performing physical examinations reading radiographs obtaining Vital Signs and we're a detailed knowledge of the protocol uh investigational product and the investigation brochure is not necessary these should not be listed on the 1572 as sub investigators and so this is always a challenging um distinction of where do you draw the line and so this is where your question

is perfect of yes of course we would love to have this discussion with you come talk to the agency early when you're planning your decentralized clinical trial tell us what your proposal is and why um but we really are hoping that we can start to rely more on some of these local Health Care Providers um but that oversight is still the responsibility of the investigator for

the trial um so we're happy to have that discussion with you early on as you are designing your DCT thank you thank you another question for you what are the most significant barriers to DCT adoption uh so as I was just alluding to that balance between the investigator and the sub investigator and the local health care provider um we have certainly heard from many sponsors and

investigators that um it's it's been a challenge to to figure out that role and uh particularly what an investigator is actually comfortable with in delegating and not overseeing day-to-day because on a piece of paper they are still legally responsible um so that is definitely one area we've heard a lot of concerns um also from the planning and Logistics aspect of an a DCT um as some

one of the questions was about shipping investigational products um all of those things there need to be significant planning to successfully carry that out for a large-scale clinical trial there has to be a lot of training for the trial Personnel sometimes contract um study nurses are used those still need to be trained on what the responsibilities are associated with the trial there needs to be the

training with the participants if a participant is using a digital Health technology they have to be trained on how to ensure all that data is appropriately captured um for the trial and so all of these things all have costs associated with them so that can increase some of the upfront costs with running a decentralized clinical trial thank you thank you let's do one more question for

you if a patient's digital Health technology is used what consideration should be given to Source data being available during or future auditing by the f thank you uh so the digital Health Technologies guidance has a section on record potension and retention and also there is now uh the new part 11 electronic systems uh guidance that came out earlier this year and both of those speak to

um Source data and the digital Health technology um we recognize that it is not realistic um for the source data to be stored on the digital Health technology so we are considering once that data is transferred from the digital Health technology to the sponsors data repository um that data repository that the sponsor has responsibility for is considered the source data and so that is what would

be inspected uh if uh there were a FDA inspection thank you okay let's move back to Captain Smith Captain Smith here's a question for you to generate real world data to evaluate the drug for a particular indication should the drug be approved through regular Pathways to approve its safety and effectiveness yes thank you so I assume the question is asking uh you know four a sponsor

to use real wood evidence to support a new indication should the product already have been approved using more traditional study designs such as a randomized trial uh and and have some existing safety and Effectiveness data so should the drug should this be for drugs that are already marketed in the U.S um I think uh you know again there are no absolutes in this space but uh

in general for products who have already been marketed in the US and there may be some off-label use for different indications you know that's a situation uh where it may make sense to consider the use of real world evidence because you will have existing U.S data and so less questions about whether data for instance from other countries is applicable to the U.S population and practice of

medicine and potentially you would have a breadth of experience on which today's determination of efficacy and safety for a new indication so that would be an area where it may make sense to pursue these types of discussions moving forward but again you know this is a fairly new area and there's a growing interest with all of the data that are now out there so we are

open to discussions of other types of proposals depending on the disease area and the study design and the available data sources thank you thanks we have another question for you how do the padufa 7 real world evidence initiatives relate to the broader real world evidence efforts already underway yes thank you for the question so I really view the Paducah 7 uh commitments and initiatives is a

progression and so it's a way for us to begin to engage more in specific areas where we know based on the guidances we publish that there are a lot of questions uh and we we need to learn through interactions and through case studies how to move this area forward uh and so through the advancing Rule and evidence program we'll gain experience with specific sponsors and specific

proposals that information and those Lessons Learned uh we'll be able to discuss publicly with others who are interested in this space through the disclosure agreements and future discussions of case studies publicly and then as we learn more and gain experience in this area we can feed that back into our guidances with either revised or new guidances as well as public meetings in the future so I

think all of this is just moving us towards gaining experience learn learning and then using those learnings to feed back into the guidance that we're providing in the future thank you thanks and one more question for you what happens after sponsors participate in the advancing real world yes thank you so you know as I mentioned uh based on one of the other earlier questions you know

the the up to four meetings through the advancing World evidence program are really intended to uh you know provide sponsors with iterative feedback on proposals and try to work through any issues and challenges that may be presented by the use of robot evidence to support an indication or to meet a post-approval study requirement and and at the end of those instructions I'm sorry interactions uh the

the goal is really to have a generally agreed upon plan uh and then you know the sponsor as with any other development program plan will continue on with the usual Pathways for interactions with the review Division and with the agency so maybe that the next step is to submit a formal protocol to the review division for any additional feedback before implementing the study or if needed

to engage in additional discussions of The Proposal so you know this is is a a program that allows for a large group of folks with expertise to come together with the review divisions and provide input on issues but you know if at any point the sponsor is ready to go back to the usual processes those are always available thank you thank you Captain Smith it looks

like we have a few more minutes so we will go back to Ms konkowski this um is kind of a longer question but when you think is currently on the market with 510k clearance as validated measures for evaluating a drug on trial and later a recall or a device alert occurs for the device used for the investigation is the suggestion to perform a risk assessment and

provide a rationalization for accepting the outcome then list the limitation or potential variation in the validation device as part of the submission thank you this is a fairly specific question but I think um it's a it's a best practice for anything that comes up unanticipated in your clinical trial is come talk to the agency on this um if you're method of collecting data in your trial

is recalled um or there's an alert for it you're you're putting your entire study at Jeopardy so I would certainly recommend approaching the review division as soon as you find out this information um your approach sounds very reasonable of performing a risk assessment and then um addressing how those changes or risks are being addressed in your clinical investigation but really I can encourage you enough for

many of these different aspects of modernizing clinical trials come talk to the agency early and we'll talk through it with you and that way we can both ensure the ability of the data that you're collecting through your study thank you thanks I think we have time for maybe one more question for addition Al if it were considered significant risk is there a guidance or regulation that

points to allowing the information required for an IDE to place be placed in an IND additionally where would the documentation be placed in the IND does this apply to all review divisions thank you that is something that I did address in my presentation earlier uh this morning that uh yes we do foresee that uh most significant risks devices can be reviewed as part of the uh

IND um this also is addressed in the digital Health technology guidance that is out in draft right now um that is something we did receive many docket comments on so we are revising that section to make it a little more robust and I'm making it a little clearer um for the different uh buckets of device regulation and digital Health Technologies um so um I would um

encourage you to include that information within your IND um this is where the cover letter is very important to include in your cover letter that your IND includes a digital Health technology it's considered significant risk the device aspects of that are that would normally be included in the IDE are actually included in IND um and just outline in in your cover letter table of contents where

all that documentation is actually included but you are right the transparency and explaining that up front will really go a long way to facilitate the review process of both centers thank you okay well I think that's all the time we have for questions right now thank you for all those great questions and answers for those excellent responses and the presentations so um let us now uh

we will break um and we will return at is it let me check 12 45 from lunch so