Hogan Lovells Publications | Medical Device Alert | 24 October 2016

Clinical Evaluation of Medical Software: FDA Proposes International Guidance

On October 14, 2016, the Food and Drug Administration (FDA or the Agency) released a draft guidance document addressing clinical evaluation for standalone software. The draft guidance, entitled Software as a Medical Device (SaMD): Clinical Evaluation, was originally prepared by the International Medical Device Regulators Forum (IMDRF), representing a global harmonization effort to articulate a process for evaluating the clinical validity of medical software. While on its face the document appears to be more strongly aligned with the European approach to regulation of standalone software regulated as a medical device, the draft guidance does provide some interesting insights into the process FDA may expect when approaching clinical evaluation.

IMDRF

The International Medical Device Regulators Forum (IMDRF) includes representatives from Australia, Brazil, Canada, China, Europe, Japan, Russia, and the U.S. and builds on earlier work by the Global Harmonization Task Force on Medical Devices (GHTF) to develop medical device regulatory approaches that are harmonized across participating jurisdictions. The general goal of harmonization efforts is to decrease the overall burden on regulated companies operating globally by developing policies that are more consistent across jurisdictions. However, the guidelines and other materials developed by IMDRF must be further adopted by each jurisdiction.

One of the areas of focus for the forum has been standalone medical software (referred to as software as a medical device). To that end, the forum has produced several proposed and/or final documents, including Software as a Medical Device (SaMD): Key Definitions, Software as a Medical Device: Possible Framework for Risk Categorization and Corresponding Considerations, and Software as a Medical Device (SaMD): Application of Quality Management System. However, this most recent proposed document, Software as a Medical Device (SaMD): Clinical Evaluation, marks the first time that FDA has released a software-related document from the forum as an official FDA draft guidance. If and when the draft is finalized, it would have the same impact as any other FDA guidance document.

Summary of Draft Guidance

The draft guidance explains that while the purpose of clinical evaluation is the same for standalone software as for other medical devices, software "operates in a complex, highly connected-interactive socio-technical environment in which frequent changes and modifications can be implemented more quickly and efficiently." For this reason, FDA and the IMDRF felt it important to develop a guidance document specifically targeted at discussing the expectations of clinical evaluation for software.

Importantly, the draft guidance does not (and could not) explain the specific data requirements for each new software product. Such requirements are always based on the specific design of the product and the intended use. What the guidance does do is provide a general framework for considering how to approach clinical evaluation given the complexities of medical software.

The draft guidance defines clinical evaluation as follows: "Clinical evaluation is the assessment and analysis of clinical data pertaining to a medical device in order to verify the safety, effectiveness, and performance of the device. Clinical evaluation is an ongoing process conducted during the lifecycle of a medical device." The guidance explains that clinical evaluation should be an iterative and continuous process conducted as part of the quality management system for medical devices.

According to the guidance, clinical evaluation involves a combination of activities to gather and assess the scientific validity, analytical validity, and real-world clinical performance of a SaMD. It defines these three types of evidence as follows:

• Scientific validity – demonstrating the association of the SaMD output to a clinical condition/physiological state;
• Analytical validity – demonstrating the technical performance related to accuracy, reliability, repeatability, and reproducibility; and
• Clinical performance – demonstrating the ability of a SaMD to yield a clinically meaningful output associated to the target use of SaMD output in the health care situation or condition.

The guidance recognizes that the scope of the clinical evaluation is dependent on the intended use of the software, and draws a distinction between software that is intended to treat a condition, diagnostic SaMDs (typically intended to identify early signs, triage, predict risk, screen, detect or diagnose a healthcare situation or condition), and non-diagnostic SaMDs (have generic functionality ... typically provide data to help aid in diagnosis, aid in treatment, inform of options). Scientific validity may already be well-established, in which case it should be documented. If not, the manufacturer should generate scientific validity evidence. Analytical validity should be generated as part of the SaMD verification and validation activities. Clinical performance evaluation should be performed for diagnostic SaMDs with a higher risk profile. The guidance gives details regarding the methods for collecting each type of data, which can include literature reviews, V&V studies, real world use (post-market) data, data on the device or similar devices, etc.

Meanwhile, the level of evidence required would be based on the impact to patients or public health of the information provided by the software. The guidance proposes four categories that are defined by the state of the healthcare situation or condition (non-serious, serious, critical) and the significance of the information provided by the SaMD to the healthcare decision (inform clinical management, drive clinical management, treat, or diagnose).

The guidance also explains that certain SaMD categories may require independent review of the evidence, akin to peer review or design review, to provide users confidence in the clinical validity of the device. The guidance cautions that this concept is separate from the need for premarket review by a regulatory authority.

Finally, the guidance notes that software is unique in its level of connectivity, which may allow the continuous monitoring of the safety, effectiveness, and performance of the SaMD. The IMDRF encourages manufacturers to use this feature to conduct observational studies in addition to monitoring the software performance.

New Guidance in Context of FDA Software Regulation

In recent years, FDA has released several guidance documents aimed at clarifying the Agency's position on whether certain software products would be considered medical devices and when such devices would be subject to FDA enforcement discretion versus active regulation. For example, FDA's 2015 final guidance, Mobile Medical Applications: Guidance for Industry and Food and Drug Administration Staff (MMA Guidance)¹, provides a framework for the regulatory assessment of mobile applications (apps).

Historically, there has been great controversy regarding the extent to which and the mechanisms by which FDA does or does not regulate medical software. In 2012, Congress passed the Food and Drug Administration Safety and Innovation Act (FDASIA), which, among other things, required FDA, in consultation with Federal Communications Commission (FCC) and the Health and Human Services' Office of the National Coordinator for Health Information Technology (ONC), to propose a framework and strategy for the regulation of all health information technology. In response to that legislation, the three agencies convened a working group to advise on the appropriate framework. Then in April of 2014, the three agencies issued a report proposing an overarching framework for the regulation of health IT². The report, FDASIA Health IT Report: Proposed Strategy and Recommendations for a Risk-Based Framework (draft Report), proposes that health IT be classified into one of three categories: (1) administrative health IT functionality, (2) health management health IT functionality, and (3) medical device health IT functionality. Classification into these categories is to be based on risk and function, with the highest risk products falling into the medical device health IT category. Active FDA regulation would be limited to that last category, though health management health IT functionality would also likely see increased regulatory oversight compared to what is present today. Importantly, no further updates on the FDA's status with respect to the FDASIA report have been released since 2014.

Notably, many medical software products are introduced to the market today without FDA oversight. Identifying where the scope of FDA's active regulation lies has been challenging. Of particular importance has been clinical decision support. For some decision support products there is clear FDA precedent for regulation, while for others FDA has not previously taken an active role. The Agency is reportedly working on a guidance document associated with clinical decision support, but it has not yet been released.

This more recent guidance on clinical evaluation comes in the context of this prior FDA history around regulation medical software. One assumes that the clinical evaluation guidance would only apply to software that is actively regulated by FDA. However, this clinical evaluation guidance cross-references earlier IMDRF documents that provide a much broader definition of SaMD (i.e., software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device). This is consistent with the broader focus of European regulation of medical software. Thus, before FDA could adopt the clinical evaluation guidance in final form, the Agency would need to make this consistent with FDA's general approach to regulation of software.

Discussion

As noted above, this is the first time that FDA has released a software-related document from the IMDRF as an official FDA draft guidance. The Agency added a single cover page with standard guidance disclaimer language, but otherwise released the draft without modification. While FDA is seeking comments on the draft, the IMDRF will presumably produce a final version that would then need to be further adopted by FDA. It is unclear whether the Agency would intend to treat that final IMDRF version as a final FDA version or seek additional public comment on that final version.

Of note, the draft guidance as proposed by FDA essentially incorporates by reference the earlier SaMD guidance documents produced by the IMDRF. For example, the draft guidance notes that it is intended to be a companion document to the prior IMDRF guidance document. Those documents have been previously finalized without public comment in the US. Thus, it is unclear to what extent those documents would also carry weight in the U.S. Most important among them is Software as a Medical Device: Possible Framework for Risk Categorization and Corresponding Considerations, which was finalized in 2014 and provides further explanation of the risk classification scheme described in the clinical evaluation guidance. How the draft guidance, if adopted, would square with FDA's classification of medical devices is unclear. The prior risk categorization IMDRF document and this draft clinical evaluation guidance both outline four risk categories, whereas FDA regulations specify three risk categories for devices (I, II, III). 21 C.F.R. §860.3. The Agency has not specified how its risk categories, including its recent use of enforcement discretion for certain low risk products, relate to the guidance risk categories.

Importantly, as the draft guidance notes, the term "clinical evaluation" should not be understood to be limited to conducting a prospective randomized clinical study of the device itself. Collection and analysis of clinical information, which could include publications, information on related devices, and other real-world evidence, such as post-market data, can also be used to prepare a clinical evaluation for software regulated as a medical device. In this way, it could be argued that the terminology and concept of clinical evaluation in the draft guidance tracks more closely with the European concept of a Clinical Evaluation Report (CER). It does not track as closely to language typically used by FDA in the U.S. and so has the potential to be somewhat confusing to companies operating primarily in the U.S. It remains to be seen whether the guidance provides sufficient information to U.S. companies accustomed to dealing with FDA in how to conduct clinical evaluation activities.

It is also somewhat unclear why the Agency would adopt the EU approach to clinical evaluation for software, but not for other types of devices. While certainly there are unique issues posed by software, the risk categories for devices and the types of evidence (scientific validity, etc.) outlined in the guidance are not particular to software. Certainly, medical devices have contained software, both along with hardware and as standalone software, for many years. From an FDA perspective, in general the type of clinical data that is required to support clearance or approval is more related to the proposed indications for use and claims than the nature of the technology.

Finally, the draft guidance's framework outlining the type of evidence needed (scientific validity, analytical validity, and clinical performance), as well as whether independent review is recommended for each device category, is highly complicated and may be confusing to navigate. A more streamlined way for companies to determine what type of clinical evaluation would be helpful.

Companies that have concluded that they have standalone software that may be regulated by FDA have two options. They can place their software on the market until FDA clarifies whether their software product will be actively regulated by the agency. There are some obvious risks in this approach. Alternatively, the company could engage FDA through informal communications with the agency, the filing of a 513(g) petition seeking FDA classification of the software, or through the filing of a pre-submission with FDA setting out the company's regulatory plan, including possibly seeking market clearance or approval. There are companies that have adopted one or more of these approaches. What is clear is that once it is determined that FDA will actively regulate the standalone software product and that clinical data may be required, the company at that point has no choice but to interface with FDA at some level, likely through the pre-submission process. Given this new guidance on clinical evaluation, FDA will likely expect that companies have considered and taken into account the principles in the guidance when developing proposals for review during any pre-submission meeting.

Per the Federal Register notice, interested parties may submit comments on the draft guidance to FDA until December 13, 2016, to docket number FDA–2016–D–2483. Comments may also be submitted directly to the IMDRF (http://www.imdrf.org/consultations/cons-samd-ce.asp); however, the comment period closes earlier (December 2, 2016).

Note: An analysis of the implications of the proposed IMDRF guidance in the EU will be forthcoming on our blog: http://www.hlregulation.com/medical-devices/

---

1 FDA, February 9, 2015, available at http://www.fda.gov/downloads/MedicalDevices/.../UCM263366.pdf

2 Available at http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHReports/UCM391521.pdf

Download PDF Back To Listing