User Experience (UX) is key to the success of a digital health product. It doesn’t matter how beneficial a product has the potential to be, if the user isn’t using it as intended due to a poor user experience, they won’t be getting the most out of it.
The System Usability Scale (SUS) is a widely used scale that is typically used to assess how usable software and hardware products are. The scale was not specifically designed to be used for mobile apps, or digital health products, but it has been used by some to do so.
To understand how useful and accurate it is, and to guide digital health suppliers, the ORCHA research team collaborated with Ulster University on a piece of research to discover whether the SUS can be used to reliably assess the usability of digital health products.
PhD researcher, Maciej Hyzy led the investigation, with support from authors including Sophie Hunt, Senior Digital Health Assessor, and Dr Simon Leigh, Director of Research at ORCHA. The team analysed previously published research to find digital health products which had published SUS scores. They then analysed the distribution of these SUS scores for digital health products in different condition areas, to evaluate whether the existing benchmark of a mean SUS score of 68 (widely accepted for software and hardware products) is relevant when evaluating the usability of digital health products. They found that overall, the SUS is a good starting point for measuring usability in digital health products. However, given that the findings are highly subjective, and demonstrated to be variable by condition area, more work needs to be done in this space, perhaps with reference to more objective and evidence-based approaches to assessing UX.
It was clear from the analysis that future work should aim to supplement the subjective user experience assessments provided by the SUS with questions that can be answered more objectively. This will enable us to understand exactly why some digital health products provide a better user experience than others. We can then learn exactly what makes a digital health product with good user experience.
The report has been published in JMIR Publications mHealth and uHealth, and is available to read on Ulster University’s website.
PhD researcher, Maciej, said:
“The System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS scale was not specifically designed to evaluate mobile apps, or indeed digital health products. The objective of this study was to examine whether the widely used SUS distribution for benchmarking (mean of 68 and standard deviation of 12.5) can be used to reliably assess the usability of digital health products.”
If you have any questions about this research, or about how ORCHA assesses user experience, please get in touch with us at hello@orchahealth.com.
Published in: BMJ Open
The traditional model of healthcare delivery is based on providing medical services through systems of hospitals, primary care facilities and outpatient clinics. However, the COVID-19 pandemic has profoundly disrupted the routine delivery of physical healthcare, resulting in the widespread deferral of elective, preventive and outpatient appointments by health authorities. Estimates suggest that as many as 40% of appointments have been cancelled or postponed by patients, as part of efforts to avoid public spaces as much as possible. In the USA, ambulatory care visits fell by 60% in the early phase of the pandemic, while in the UK alone, an estimated 1.5 million elective admissions and 2.6 million outpatient attendances were forgone as a result of COVID-19.
We explored if consumer interest in digital health products (DHPs) changed following the COVID-19 pandemic and the lockdown measures that ensued. We identified all web-based internet searches for DHPs in the UK using ORCHA app libraries, split over two periods, pre-COVID-19 lockdown (January 2019–23 March 2020) and post-COVID-19 lockdown (24 March 2020–31 December 2020).
There were 126 640 searches for DHPs over the study period. Searches for DHPs increased by 343% from 2446 per month prior to COVID-19 lockdown measures being introduced to 8996 per month in the period following the first COVID-19 lockdown in the UK. In total, 23/25 (92%) of condition areas experienced a significant increase in searches for DHPs, with the greatest increases occurring in the first 2 months following lockdown.
Musculoskeletal conditions (2.036%), allergy (1.253%) and healthy living DHPs (1.051%) experienced the greatest increases in searches compared with pre-lockdown. Increased search volumes for DHPs were sustained in the 9 months following the introduction of lockdown measures, with 21/25 (84%) of condition areas experiencing monthly search volumes at least 50% greater than pre-lockdown levels.
Our findings suggest that the COVID-19 pandemic has profoundly disrupted the routine delivery of healthcare, making face-to-face interaction difficult, and contributing to unmet clinical needs. This study has demonstrated significant increases in internet searches for DHPs by members of the UK population since COVID-19, signifying an increased interest in this potential therapeutic medium. Future research should clarify whether this increased interest has resulted in increased acceptance and utilisation of these technologies also.
Read the full research piece here: https://bmjopen.bmj.com/content/11/10/e053891
Published in: JMIR mHealth uHealth
Despite the increasing availability of mobile health services, clinical engagement and use of health-apps remains minimal. This study aimed to identify and prioritise barriers to and drivers of health app use among health care professionals (HCPs) from the United Kingdom.
A discrete choice experiment was conducted with 222 HCPs using a web-based survey between March 2019 and June 2019. Participants were recruited to take part via social media and asked to choose their preferred option of 2 hypothetical health apps to prescribe to a hypothetical patient or to prescribe neither.
Hypothetical health-apps were characterized by differing levels of patient age, cost, published evidence bases, whether they had a National Health Service (NHS) stamp of approval, personal familiarity with the technology, and whether they were recommended by a fellow HCP. These were all reported as important factors in deciding whether HCPs are likely to prescribe a DHT, based on our pilot study published in the Lancet digital health.
We received 230 responses from UK-based HCPs, a total of 96.5% (n=222/230) of respondents understood the survey task and passed the test of rationality. 62.6% (n=139/222) of the health care providers responding to the survey had previously recommended the use of health apps to patients. Health apps were most likely to be prescribed to patients if they had an NHS stamp of approval or if they were recommended by another HCP (both P<.001). Published studies detailing clinical effectiveness were important (P<.001), but it would take five published studies to have the same impact on prescribing behavior as an NHS stamp of approval and two studies to be as convincing as having used the technology personally.
Increasing patient age and costs resulted in significant reductions in digital health prescribing (P<.001), none more so than among allied health professionals. Willingness-to-pay for health apps increased by £124.61 (US $151.14) if an NHS stamp of approval was present and by £29.20 (US $35.42) for each published study.
Overall, 8.1% (n=18/222) of respondents were reluctant to use health apps, always choosing the I would prescribe neither option, particularly among older HCPs, nurses, and those who do not use health apps personally.
The results of this study suggest that an NHS stamp of approval, published studies, and recommendations from fellow HCPs are significant facilitators of digital prescribing, whereas increasing costs and patient age are significant barriers to engagement. These findings suggest that demonstrating assurances of health apps and supporting both the dissemination and peer-to-peer recommendation of evidence-based technologies are critical if the NHS is to achieve its long-term digital transformation ambitions.
About ORCHA
Founded by NHS clinicians, ORCHA is the world’s leading digital health evaluation and distribution organisation. We provide services to national health bodies across three continents, including the NHS in 50% of UK regions, delivering national accreditation frameworks, bespoke Digital Health Libraries, and professional recommendation tools, specific to the needs of our clients. ORCHA’s unique Review Engine assesses digital health solutions against more than 350 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, plus additional criteria depending on needs.