Real World Evidence: The Times They Are a- (Slowly) Changin’

Once considered “junk science,” Real World Evidence (clinical outcomes data not collected in conventional randomized controlled trials) is the new star on the precision medicine horizon. But the tool set for using this treasure trove of health care information is nascent and the tasks as are daunting as the opportunities. The good news is that the FDA is taking this challenge to heart – and in writing.

Per the FDA’s Prescription Drug User Fee Act VI “commitment letter,” the agency will face some real world deadlines to advance the use of real world evidence. But, since we’re dealing with the real world, let’s get real – guidance is unlikely until the end of 2022 at the earliest. (That’s the timeline agreed to via the PDUFA VI negotiations.)

Step One toward these new 21st Century rules of the regulatory road will be a series of public meetings and regulatory workshops. It will be curious to see who shows up at the table.

PDUFA VI User fees will support agency policy development in the area, and FDA has pledged to meet the following benchmarks:

• End of FY 2018 – Convene one or more public workshops with key stakeholders, including patients, biopharmaceutical companies, and academia, to gather input into issues related to real world evidence (RWE) use in regulatory decision-making.
• End of FY 2019 – Initiate (or fund by contract) appropriate activities (e.g., pilot studies or methodology development projects) aimed at addressing key outstanding concerns and considerations in the use of RWE for regulatory decision-making.
• End of FY 2021 – Publish draft guidance on how RWE can contribute to the assessment of safety and effectiveness in regulatory submissions, for example in the approval of new supplemental indications and for the fulfillment of post-marketing commitments and requirements. FDA will work toward the goal of publishing a revised draft or final guidance within 18 months after the close of the public comment period.

Ground Zero for a real-world evidence regulatory pathway will be Sentinel, the existing public/private program aimed that uses a variety of databases to track, collect and analyze adverse event reports about drugs, vaccines and medical devices.

Modeled after successful programs such as the Centers for Disease Control and Prevention’s Vaccine Safety Datalink, Sentinel allows FDA to conduct safety surveillance by actively querying diverse data sources, primarily administrative and insurance claims databases but also data from electronic health record (EHR) systems, to evaluate possible medical product safety issues quickly and securely.

Sentinel, mandated under the Food and Drug Administration Amendments Act, will get a $50 million boost over five years under PDUFA VI. Sentinel currently has information on more than 100 million patients.

Similarly, efforts are underway to establish a National Device Evaluation System. As currently envisioned, the NDES would be established through strategic alliances and shared governance. The system would build upon and leverage information from electronic real-world data sources, such as data gathered through routine clinical practice in device registries, claims data, and EHRs, with linkages activated among specific data sources as appropriate to address specific questions.

The FDA’s Center for Devices and Radiological Health also has been in the vanguard of the agency’s efforts to solicit patient preference data and use it to support approval. In draft guidance released on July 26, FDA outlined its thinking on the use of real-world evidence in making regulatory decisions about medical devices.

The agency described factors it would consider when evaluating the relevance, reliability and quality of real-world evidence, and suggests when it might use such data to make decisions about devices.

To evaluate the reliability of data, FDA will assess how they were collected, their adequacy for answering relevant questions, and whether they were collected in a manner that minimizes bias. The guidance says a prospective protocol is “essential to ensure reliability” of real-world evidence. The guidance says FDA might use such evidence to expand a device’s approved indications, for postmarket surveillance, and as a control for studies of subsequent devices.

“Big Data” and “Valid Evidence” are not the same thing. It is an important distinction that illuminates a crucial difference. When it comes to the patient voice (or any voice), the plural of anecdote isn’t data. But the plural of data is science.

Patient passion is important to share. When combined with data and a more dispassionate understanding of regulatory paradigms, a patient-driven pathway can is, and must evolve into a tool used to impact regulatory decision-making.

As FDA Commissioner Rob Califf and Deputy Commissioner Rachel Sherman recently commented:

“Creating knowledge requires the application of proven analytical methods and techniques to biomedical data in order to produce reliable conclusions … There must be a common approach to how data is presented, reported and analyzed and strict methods for ensuring patient privacy and data security. … Rules of engagement must be transparent and developed through a process that builds consensus across the relevant ecosystem and its stakeholders. … To ensure support across a diverse ecosystem that often includes competing priorities and incentives, the system’s output must be intended for the public good and be readily accessible to all stakeholders.”

We need to develop proposals that modernize the information used in the evaluation of the value of treatments. Just as the key scientific insights guiding the FDA Critical Path program are genetic variations and biomedical informatics that predict and inform individual responses to treatment, we must establish a science-based process that incorporates the knowledge and tools of personalized medicine in reimbursement decisions: true evidence-based, patient-centric medicine.

Today, right now, we need a Critical Path for Healthcare Technology Assessment to begin the process of developing a similar list of ways new discoveries and tools (such as electronic patient records) can be used to improve the predictive and prospective nature of clinical outcomes. In an era of personalized medicine, one-size-fits-all treatments and reimbursement strategies are dangerously outdated. Accepting real world evidence does not mean discarding the randomized “gold standard” – it means augmenting it.

When it comes to the regulatory science of real world evidence, we are still in early days, but the times they are a-changin.’ (And you better start swimmin’ or you’ll sink like a stone.)

Peter J. Pitts, a former FDA associate commissioner, is president of the Center for Medicine in the Public Interest and Chief Regulatory Office at Adherent Health LLC.