Why Real-World Results Are So Challenging for Digital Health

February 8, 2017
NEJM Catalyst

By: Joseph C. Kvedar, MD, Vice President of Connected Health, Partners HealthCare; Associate Professor of Dermatology, Harvard Medical School and Alexander L. Fogel, MBA, MD Candidate, Stanford University School of Medicine

Despite years of hype in the field of digital health products — a term that we use here to describe technologies that are designed to have clinical impact on disease — fewer products than expected are being deployed in real-world clinical settings. Many digital health products that demonstrate impressive results in clinical trials often fail to do so in real-world settings.

Why? Much of the success of digital health products is predicated on patient engagement, and clinical trials are among the most engaging environments in health care. Clinical trials involve the use of a variety of tools (e.g., training, close monitoring, payments) to ensure that patients use the technologies appropriately, but few of these tools are used in the real world. In order to cross the chasm from success in clinical trials to success in practice, digital health companies need to focus on patient engagement.

Digital Health Hinges on Engagement

Engagement is so important because many digital health products are designed to achieve behavioral changes for the purpose of preventing or treating chronic diseases. For patients who are at risk for, or are living with, a chronic disease, successful prevention or management requires minute-to-minute, day-to-day changes in decision-making. Patients need to be highly motivated to make behavioral changes, they need to be praised when they follow through, and they need guidance when they slip up. The process needs to be sticky and self-reinforcing in order to maintain patient interest. In a nutshell, patients need to be engaged.

Engagement is important not only for clinical results, but also for the business models of many digital health companies, which frequently incorporate pay-for-performance contracts. Digital health companies need to track data continuously in order to demonstrate their value to the purchaser or user. Products that do not engage patients drag down performance metrics and reduce the viability of the company.

Contrast this with pharmaceuticals: payors pay for drugs before patients take them, and pay-for-performance contracts are the exception; this is part of the reason why nearly 50% of medications for chronic disease are not taken as prescribed.

The Gap Between Clinical Trials and Real-World Results

Successful clinical trials of both digital health products and pharmaceuticals are engaging by definition. In order for a trial to demonstrate significant results — and in order for the results of the trial to be published — the attrition rate for participants needs to be very low. The entire endeavor is designed around ensuring that patients use the product or service appropriately and follow up regularly.

A clinical trial is designed as follows: an artificial scenario is crafted, willing participants volunteer, participants are carefully selected on the basis of optimal criteria, staffers are thoroughly trained, measurements are taken like clockwork, and participants are induced (by means of monetary payments or a barrage of phone calls, emails, and texts) to use the product or service as directed. As a result, the experience of individuals who are involved in a clinical trial typically is much more engaging than that of patients who use the technology in real-world scenarios.

Tools to Drive Engagement

Extending the engagement-boosting practices from a clinical trial to a much larger population of patients in the real world is not feasible because of cost and logistical complexity. As a result, different approaches are needed to facilitate engagement.

  • Maintain engagement-first thinking. Digital health products need to be designed to drive engagement from the ground up. Focusing on an exciting technology or a massive need is not enough. Companies should consider employing principles from behavioral economics, which have been shown to increase engagement and product stickiness by using sophisticated implementations of financial incentives, loyalty points, social structures, and other techniques.

    Social media also have been shown to be a powerful force in patient engagement, with applications ranging from open patient support resources such as PatientsLikeMe to the patient-only social media networks used by Omada Health to drive weight loss. Companies also should think about employing strategies from video games and sports. While “gamification,” as this approach is known, has been mostly met with a tepid response from the physician community, don’t forget that Pokémon GO has performed better than many digital health tools at increasing the physical activity levels of its users.
  • Embrace real-world data to drive real-world evidence. Real-world data (RWD) and real-world evidence (RWE) are topics of increasing interest in the medical community and have even garnered the attention of the FDA. While RWD and RWE typically are used to supplement clinical trials, these information sources are increasingly important for digital health companies because they provide critical evidence of user engagement. Digital health companies can design studies to capture RWD and build RWE by crafting scenarios that mirror how products will be used by real patients

    Companies can look to Omada Health’s diabetes prevention study for an example of how to design a study to collect RWD and RWE. Omada recruited participants on Craigslist, participants relayed weight measurements to the company with use of a digital scale equipped with cellular data-transmitting capabilities, and HbA1c measurements were obtained with use of home kits that were mailed to participants. The study was performed in a similar way to how Omada interacts with real patients and thus provided the company with a good understanding of the real-world performance characteristics of its product.
  • Develop product variations. Engagement is not one-size-fits-all. Different tactics will work well for different populations, and digital products may be more successful if the manufacturer is able to develop variations to serve different segments of the market. For example, Propeller Healthhas developed sensors that can track inhaler use, with the goal of improving outcomes and reducing costs for patients with asthma and chronic obstructive pulmonary disease (COPD). Although patients with both conditions often use inhaled medications, the demographics, etiology, prognosis, and medication regimens for these conditions differ substantially. Product variations specific to the different patient populations using the inhaler will be critical to Propeller’s success in the treatment of these diseases. Companies must think through how to engage each subset of the population that the product is serving.
  • Seek partnerships to reap synergies. Digital health companies are uniquely positioned to develop synergistic partnerships to drive patient engagement. Many digital health companies — particularly those operating outside of FDA or HIPAA jurisdiction — have unique data and insights about specific populations. This information can translate into powerfully engaging products. For example, Mango Health, a medication adherence app, partners with consumer products companies to offer real-world rewards like gift cards as incentives for patients to increase adherence. Similarly, Under Armour, an athletic apparel company, is investing significant resources to develop synergistic digital health products for fitness and weight loss, thereby combining products with technologies that help users to meet their health goals.

Making the Leap from Novelty to Necessity

Clinical trials are a critical process in the evolution of digital health products. Evidence is an important means for convincing health care executives to buy these products. However, simply learning that an intervention was successful in a clinical study is not enough. Real-world validation is also important as digital health companies own the validation process from product development to scale. Once clinical validation is achieved, real-world evidence must be gathered to justify success at scale. In order for digital health products to make the leap from novelty to necessity, manufacturers will need to focus on driving patient engagement in real-world settings. Engagement will require taking lessons from other industries, creating products that are tailored to population subsets, developing synergistic partnerships, and understanding the performance characteristics of products in the real world.

Read NEJM Catalyst Article