You should submit comments and suggestions regarding this draft document within 90 days of publication in the Federal Register of the notice announcing the availability of the draft guidance. Submit written comments to the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852. Submit electronic comments to http://www.regulations.gov . Identify all comments with the docket number listed in the notice of availability that publishes in the Federal Register.
For questions regarding this document, contact Rachel Turow at 301-796-5094 or by electronic mail at Rachel.Turow@fda.hhs.gov . For questions regarding this document concerning devices regulated by CBER, contact the Office of Communication, Outreach and Development (OCOD) by calling 1-800-835-4709 or 301-827-1800.
Additional copies are available from the Internet. You may also send an e-mail request to firstname.lastname@example.org to receive an electronic copy of the guidance or send a fax request to 301-827-8149 to receive a hard copy. Please use the document number 1772 to identify the guidance you are requesting.
This draft guidance, when finalized, will represent the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.
FDA has developed this draft guidance document to provide greater clarity for FDA reviewers and industry regarding the factors FDA considers when making benefit-risk determinations during the premarket review process for certain medical devices. FDA believes that the uniform application of the factors listed in this guidance document will improve the predictability, consistency, and transparency of the premarket review process.
FDA's guidance documents, including this one, do not establish legally enforceable responsibilities. Instead, guidance documents describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidance documents means that something is suggested or recommended, but not required.
This guidance document explains the factors that FDA considers when making benefit-risk determinations in the premarket review of certain medical devices. The processes discussed in this guidance are applicable to devices subject to premarket approval (PMA) applications and, in limited cases, devices subject to premarket notification (510(k)) requirements. This guidance applies to both diagnostic devices and therapeutic devices. Although guidance is not binding, the concepts and factors described herein generally capture how benefit-risk determinations are made by FDA during the premarket review process.
Under §513(a) of the Federal Food, Drug & Cosmetic Act (“the FD&C Act”), FDA determines whether PMA applications provide a “reasonable assurance of safety and effectiveness” by “weighing any probable benefit to health from the use of the device against any probable risk of injury or illness from such use”. To aid in this process, PMA applicants submit valid scientific evidence, including one or more clinical investigations where appropriate, which FDA reviews to determine whether “the device will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling of the device.” 1 FDA staff review the data submitted as part of the PMA and determine – based on a number of factors – if the data support the claims made by the sponsor for the device, i.e., intended use and/or indications for use, and data analysis demonstrates that the benefits of the device outweigh its risks. This process may also occur on a case-by-case basis in the review of 510(k) devices when there are differences between the target device and the predicate device that can adversely affect the safety and/or effectiveness of the target device. 2
Medical devices can be evaluated using clinical and non-clinical testing methods. Clinical testing methods for medical devices can include, when appropriate, randomized clinical trials in the appropriate target population, partially controlled studies, studies without matched controls, well-documented case histories conducted by qualified experts, reports of significant human experience, and testing on clinically derived human specimens. Non-clinical testing methods can encompass an array of methods including bench testing for product safety/reliability, animal and cell-based studies, and computer simulations. These tests characterize mechanical and electrical properties of the devices that include wear, tensile strength, compression, flow rate, burst pressure, biocompatibility, toxicity, electromagnetic compatibility (EMC), sterility, stability/shelf life data, software validation, and testing of synthetic samples, including cell lines. For in vitro diagnostics, analytical validation, or assurance that the technical measurement aspect of the product is accurate and reliable, is also considered as part of a demonstration of reasonable assurance of safety and effectiveness. The data derived from any clinical and/or non-clinical testing are taken into account during the premarket review process and FDA’s benefit-risk determination.
Although a great deal of emphasis is placed on the importance of clinical data in demonstrating the safety and effectiveness of a medical device, non-clinical data also can be critical to understanding device safety and effectiveness. Medical devices often have attributes that cannot be tested using clinical methods alone and that play a major role in the safety or effectiveness of the device. For example:
Both clinical and non-clinical data can play a role in the FDA’s benefit-risk determinations, and the factors discussed in this guidance are informed by both types of data.
There are many factors that go into weighing the probable benefit of a device versus its probable risk. These factors include, among others, whether the device is a first-of-a-kind treatment or diagnostic, whether the device provides significant improvement in diagnosis and patient management of a serious disease, how known risks of the device can be mitigated, reliability of the study, whether there are multiple studies and the strength of those studies, what amount of risk the target population will tolerate in light of the condition being treated or diagnosed and the probable benefit of the device, and whether there are alternate treatments or diagnostic techniques available.
The factors FDA considers as part of the benefit-risk determination are explained in detail below. We also give examples of how the factors interrelate and how they may end up affecting FDA’s decisions. By providing greater clarity about FDA’s decision-making process, we hope to improve the predictability, consistency, and transparency of the review process for applicable devices.
We have also included for public comment a proposed draft of a worksheet that reviewers would use in making benefit-risk determinations. The worksheet is attached as to this guidance. We believe that the level of documentation proposed by the worksheet would be very helpful to maintaining the consistency of review across different review divisions and better assuring that an appropriate decision is reached. In addition, by documenting reviewers’ thought processes as part of the review file and, in certain cases, the publicly available summary of our decision, sponsors will have a better idea of the basis for FDA’s decisions and gain a greater understanding of what factors were considered in reaching an approval or clearance decision. After receiving feedback from stakeholders and FDA staff, we may integrate use of this or a similar worksheet as part of the premarket review process. However, because the weighting of the factors for a type of device may change over time – such as a device no longer being a first-of-a-kind or the only available treatment as new therapies are approved – the benefit-risk determination for a specific device at one point in time may no longer represent the proper weighting of the factors for a similar type of device in the future.
The factors described below are considered within the intended use of the device, including the target population.
4.1 Measures for Effectiveness of Devices 3
Extent of the probable benefit(s) : The extent of the probable benefit(s) is measured by taking into account the following factors:
Extent of the probable risk(s)/harm(s): The extent of the probable risk(s)/harm(s) is measured by taking into account the following factors:
We also consider the number of different types of harmful events that can potentially result from using the device and the severity of their aggregated effect. When multiple harmful events occur at once, they have a greater aggregated effect. For example, there may be a harmful event that is considered minor when it occurs on its own, but, when it occurs along with other harmful events, the aggregated effect on the patient can be substantial.
4.3 Additional Factors for Weighing Probable Benefits and Risks of Devices
Uncertainty - there is never 100% certainty when determining reasonable assurance of safety and effectiveness of a device, nor need there be to show that the probable benefits outweigh the probable risks. However, the degree of certainty of the benefits and risks of a device is a factor we consider when making benefit-risk determinations. For example, when the probable benefits are large, less certainty regarding the probable risks may be acceptable to support the approval of the device. On the other hand, factors such as poor trial design, conduct, or analysis can make the data produced by the study unreliable. Furthermore, the repeatability of the study results, the results of other similar studies and whether the study is the first of its kind or a standalone investigation can all influence the level of certainty. In addition, the generalizability of the trial results to the intended treatment and user population is important. For example, if the device requires in-depth user training or specialization, the results of the clinical study may not be generalized to a wider physician population. Likewise, if the device is intended to diagnose a disease in a subpopulation, it may not be useful in the general population.
Characterization of the disease – the treated or diagnosed condition, its clinical manifestation, how it affects the patients who have it, how and whether a diagnosed condition is treated, and the condition’s natural history and progression (i.e., does it get progressively better or worse for the patient) are all important factors to consider when characterizing disease.
Patient tolerance for risk - if the risks are identifiable and definable, different patients will have a different risk tolerance and will make a decision as to whether the risks are acceptable. For diagnostic devices, the availability and nature of the treatment for the condition diagnosed may affect patients’ tolerance for risk. Different factors can influence patient risk tolerance, including:
Availability of alternative treatments or diagnostics – when making benefit-risk determinations, FDA considers whether other treatments or diagnostics, including non-device products, have been approved or cleared for the intended condition and patient population, how effective are they, and what known risks they pose; what off-label uses of approved or cleared devices, other products or procedures are available, if there are no devices or other products approved or cleared for the intended condition and patient population, and their benefit-risk profiles; and how well the alternatives address the needs of patients and providers. For example, if a new device has a very small benefit and there is significant uncertainty about that benefit, we may still approve the product if there are no available alternative treatments or diagnostics and the risk profile is acceptable.
Risk mitigation – the use of mitigations, when appropriate, can minimize the likelihood of a harmful event occurring. The most common form of risk mitigation is to include warnings in labeling, or to restrict the indication to a more limited use. Another important consideration in risk mitigation is the type of intervention required to address the harm. Some harms may require surgery; whereas, others may require only minimal intervention. Thus, even if there is a high likelihood of harm and the harm resulting from the risk is severe, it may be easily mitigated with a non-invasive treatment. Finally, some harms can be mitigated through changing device design features.
Novelty of technology – devices representing or incorporating new technologies, especially those that are first-of-a-kind, may provide a less than optimal benefit, but may also offer advantages that did not previously exist. With subsequent iterations of the device its benefit-risk profile may improve, the expected level of safety and effectiveness may increase, and later versions may offer significant advantages over the initial device. In these circumstances, we may approve a device with less benefit or more risk than would be generally tolerated for more established technologies, particularly where providers and patients have limited alternatives available, to facilitate patient access and encourage innovation.
The examples below are hypothetical or simplified and are only offered for illustrative purposes. The decisions described in these examples are not predictive of future FDA decisions, rather they are hypothetical outcomes and are only intended to demonstrate how FDA considers the factors described in this guidance when making benefit-risk determinations. Similar scenarios or devices may result in different approval outcomes depending on the individual characteristics of a particular device.
An implantable device is developed to treat a severe condition. There are alternative treatments available for the condition, but they are only effective for certain subgroups of patients and all of them pose significant risks.
The device is studied in a pivotal clinical trial with a design where all patients are implanted with the device but the device is only turned on in half of them. After completion of the trial, inactive devices can be turned on. The primary endpoint for the trial is the magnitude of the benefit, i.e., the trial is designed to measure how well the device reduces the patient’s symptoms.
The results of the pivotal clinical trial revealed the following:
Effectiveness: The probability that a patient would experience a substantial benefit was only approximately 15%. However, for the small portion of patients who did respond, their symptoms were significantly reduced and their quality of life was significantly improved due to improved mobility. As a general matter patients with this disease who are able to maintain good mobility longer tend to have a longer life expectancy. However, the duration of the benefit cannot be determined because the patients were only followed for one year.
Safety: There is a very low occurrence of harmful events after device implantation. However, all implanted devices that require a surgical procedure carry with them their own set of risks. In addition, permanent implants pose additional risks, namely, they typically remain with the patient for life and may be difficult or impossible to remove. Even in cases where the device is deactivated, it remains implanted and a risk of device fracture, mechanical failure, or an adverse biological response to the device remains. Finally, the surgery to implant the device is not routine and carries with it high risks.
Uncertainty : The results of the pivotal clinical trial are unreliable because both treatment and control groups suffered significant patient lost-to-follow-up (the clinical status of many patients in the study was not known at its conclusion); therefore, it is difficult to tell if patient symptoms improved because of the device action, the placebo effect, or the effect of the surgery itself.
Patient Tolerance for Risk : Patients who are willing to take the risk of having the device implanted even for a small probability of benefit because they have no other treatment options and their symptoms are severe.
Risk Mitigation : The surgery to implant the device is risky, but the risks can be mitigated by requiring the device be implanted by a skilled surgeon.
Approval/Non-Approval Considerations: The device effectiveness is limited because the probability that a patient will experience a benefit is low (15%). In this case, FDA does not have the option to limit the use of the device to only those patients who are most likely to experience a benefit because there is no way to determine the subset of patients who would definitely experience the benefit. Moreover, this type of permanent implantable device poses significant risks and there is moderate uncertainty associated with the trial results. However, for those patients who do experience a benefit, symptom relief and improvement in quality of life is impressive and the risks, although substantial, could be somewhat mitigated through limiting the device use to specialized surgeons. Finally, the device treats a severe and chronic disease for which there are few, if any, alternative treatments. Therefore, FDA is likely to approve the device.
A revolutionary device that replaces a patient’s memory is developed to treat Alzheimer’s, dementia, and other memory disorders. The device is permanently implantable and the patient must undergo a brain resection in order for the device to work properly. The device functions by downloading all of a patient’s memories onto a computer chip. Once the device is implanted, any residual memory the patient retained is no longer accessible to the patient
Effectiveness: A clinical trial of the device showed significant improvement in patients who were in the early stages of dementia and minimal improvement in patients who were in more advanced stages. Patients who received implanted devices when the majority of their memory was intact experienced the greatest benefit.
Safety: The surgery to implant the device is highly risky and must be done at certified centers of excellence by specially trained neurosurgeons. Even with these procedural restrictions, there is an 8% risk of mortality from the surgery alone. In addition, adverse events include partial paralysis, loss of vision, loss of motor skills, and slurred speech. Non-serious adverse events include personality shifts, mood swings, vertigo and insomnia.
Uncertainty : The number of patients eligible and willing to enroll in the trial was small, but the data were robust and the trial was well-designed and conducted. The results of the trial are generalizable across all patients because different patients experienced different levels of benefit depending upon their stage of memory loss.
Patient Tolerance for Risk : Because of the disruptive effect on patients’ quality of life from diseases like Alzheimer’s, dementia, and other conditions that are associated with severe memory loss, patients (and their families) suffering from these diseases often have a very high tolerance for risk. In this case, it is likely that family members responsible for patients who are at more advanced stages of their illness and therefore experiencing more severe symptoms will have a higher risk tolerance than those at the beginning stages of disease.
Availability of Alternative Treatments or Diagnostics : There are currently no alternative treatments available.
Risk Mitigation : The risks associated with this device are great. The risks associated with implantation can be somewhat mitigated by limiting implantation to a highly select group of surgeons, but the risks associated with personality changes cannot be mitigated or predicted.
Novelty of Technology : This technology is a game-changer. There is no other similar technology like it in the world. It is likely that future improvements of the device may allow treatment of many other conditions that affect cognitive function. Also, there are no other treatments that can give patients the level of benefit that this device confers.
The device has a confirmed, substantial benefit for a defined and predictable subset of patients and a moderate benefit for another defined and predictable subset. Even though the clinical trial was small, the quality of the data is good and there is little uncertainty about the results. The risks associated with the device are great and can be mitigated by advising that only highly trained physicians can implant the device. And, because patients experience the greatest benefit when the device is implanted earlier, they must expose themselves to the risks for a longer period of time in order to reap the greatest benefit; therefore, the patients who stand to benefit most also have a higher risk tolerance. Many patients who suffer from memory disorders are willing to try anything to preserve their memories and quality of life. The fact that there are no alternative treatments for this condition and a known benefit with this treatment is also compelling. Furthermore, the risks are known and quantifiable. Therefore, this device, although risky, is approvable. The decision as to whether or not to implant the device is a matter of patient/family member preference and medical opinion. FDA is likely to approve the device knowing that patients (or responsible family members) can give informed consent to the risks the device poses and only a very small group of skilled physicians will be able to implant the device.
A new in vitro diagnostic device (IVD) claims to provide early detection of any type of cancer in the general healthy population by testing patients’ blood. The device is studied in a diverse population of patients with cancer, and correctly identifies patients that have cancer 95% of the time.
Effectiveness: The IVD detects cancer well in patients who are already known to have cancer, but who likely have different characteristics than the general population in which early detection might be beneficial, because they may have more advanced cancer, they may have already received treatment, and they may have widely different demographic features (e.g., cancer patients are on average older than the general healthy population in which early detection would have greatest benefit). The IVD is effective, but not in the population that would most benefit from its use.
Safety: The device gave mostly correct results in patients who were known to have cancer, but was not tested in the intended use population, i.e. the general healthy population. If the device were to deliver a substantial proportion of false results in the intended use population, normal healthy people might be exposed to further invasive procedures or radiation, resulting in significant harm. Or, people might actually have cancer and the test would not detect it, thus preventing them from receiving beneficial treatment, or discouraging their physicians from monitoring them for cancer.
Uncertain Diagnosis : Although the test could detect cancers, it could not report what type of cancer; therefore, much additional diagnostic work-up, some of it invasive, would be required for patients that receive a positive test result.
Patient Tolerance for Risk : Although most seemingly healthy people would probably like to know if they had early stages of certain cancers, a large proportion of false results would not be tolerated because patients’ health at the stage of testing is good. For very aggressive cancers for which the patient was known to be at risk due to exposure or family history, more risk might be tolerable.
Uncertainty about Cancer Natural History : It is believed that some cancers resolve without treatment, through various immune and other natural mechanisms. It is not possible to know which early cancers will progress and which will go away on their own. Aggressive treatment of early cancers may result in more morbidity than not treating.
This IVD device had the general benefit of detecting cancers anywhere in the body, but the study was not designed to address the intended use population of generally healthy people. The differences in test performance in patients who do not have obvious cancer (the intended use population) are unknown. The information yielded by the test is not specific enough to allow a definitive diagnosis without considerable further work-up. Given the risk of serious adverse events resulting from attempts to find the cancer, as well as the likely high risk of false results (overtreatment or failure to treat), and the uncertain benefit of early detection for certain cancers, FDA is not likely to approve the device because it was not demonstrated to be effective in the intended population and it has serious side effects without adequate available mitigations.
1 In addition to §513(a), the criteria for establishing safety and effectiveness of a device are set forth in 21 CFR 860.7. Subsection (b)(1) notes, “In determining the safety and effectiveness of a device … the Commissioner and the classification panels will consider the following, among other relevant factors…The probable benefit to health from the use of the device weighed against any probable injury or illness from such use.” (21 CFR 860.7(b)).
To make this determination, “the agency relies upon only valid scientific evidence.” (21 CFR 860.7(c)(1)). Valid scientific evidence is defined as “evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device, from which it can fairly and responsibly be concluded by qualified experts that there is reasonable assurance of the safety and effectiveness of a device under its conditions of use.” (21 CFR 860.7(c)(2)).
A reasonable assurance of safety occurs when “it can be determined, based upon valid scientific evidence, that the probable benefits … outweigh any probable risks.” The evidence of which is demonstrated by establishing, “the absence of unreasonable risk of illness or injury associated with the use of the device for its intended uses and conditions of use.” (21 CFR 860.7(d)(1).
Similarly, a reasonable assurance of effectiveness occurs when “it can be determined, based upon valid scientific evidence … the use of the device for its intended uses … will provide clinically significant results.” (21 CFR 860.7(e)(1)). The evidence of which is demonstrated principally through “well-controlled investigations” (see 21 CFR 860.7(e)(2)) as defined in 21 CFR 860.7(f)
2 Under §513(a)(2)(C) of the FD&C Act, FDA may weigh “any probable benefit to health from the use of the device against any probable risk of injury or illness from such use” not only in determining there is a reasonable assurance of safety and effectiveness for a device evaluated under a PMA, but also in determining substantial equivalence under §513(i).
For purposes of this guidance the term “studies” is equivalent to the term “investigations.”
3 21 CFR 860.7(e) “There is reasonable assurance that a device is effective when it can be determined, based upon valid scientific evidence, that in a significant portion of the target population, the use of the device for its intended uses and conditions of use, when accompanied by adequate directions for use and warnings against unsafe use, will provide clinically significant results”, as demonstrated by well-controlled investigations.
4 21 CFR 860.7(d) “There is reasonable assurance that a device is safe when it can be determined, based upon valid scientific evidence, that the probable benefits to health from use of the device for its intended uses and conditions of use, when accompanied by adequate directions and warnings against unsafe use, outweigh any probable risks. The valid scientific evidence used to determine the safety of a device shall adequately demonstrate the absence of unreasonable risk of illness or injury associated with the use of the device for its intended uses and conditions of use.”