Published Research

How do you best measure the user experience of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

User Experience (UX) is key to the success of a digital health product. It doesn’t matter how beneficial a product has the potential to be, if the user isn’t using it as intended due to a poor user experience, they won’t be getting the most out of it.

The System Usability Scale (SUS) is a widely used scale that is typically used to assess how usable software and hardware products are. The scale was not specifically designed to be used for mobile apps, or digital health products, but it has been used by some to do so.

To understand how useful and accurate it is, and to guide digital health suppliers, the ORCHA research team collaborated with Ulster University on a piece of research to discover whether the SUS can be used to reliably assess the usability of digital health products.

PhD researcher, Maciej Hyzy led the investigation, with support from authors including Sophie Hunt, Senior Digital Health Assessor, and Dr Simon Leigh, Director of Research at ORCHA. The team analysed previously published research to find digital health products which had published SUS scores. They then analysed the distribution of these SUS scores for digital health products in different condition areas, to evaluate whether the existing benchmark of a mean SUS score of 68 (widely accepted for software and hardware products) is relevant when evaluating the usability of digital health products. They found that overall, the SUS is a good starting point for measuring usability in digital health products. However, given that the findings are highly subjective, and demonstrated to be variable by condition area, more work needs to be done in this space, perhaps with reference to more objective and evidence-based approaches to assessing UX.

It was clear from the analysis that future work should aim to supplement the subjective user experience assessments provided by the SUS with questions that can be answered more objectively. This will enable us to understand exactly why some digital health products provide a better user experience than others. We can then learn exactly what makes a digital health product with good user experience.

The report has been published in JMIR Publications mHealth and uHealth, and is available to read on Ulster University’s website.

PhD researcher, Maciej, said:

“The System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS scale was not specifically designed to evaluate mobile apps, or indeed digital health products. The objective of this study was to examine whether the widely used SUS distribution for benchmarking (mean of 68 and standard deviation of 12.5) can be used to reliably assess the usability of digital health products.”

If you have any questions about this research, or about how ORCHA assesses user experience, please get in touch with us at hello@orcha.co.uk.

About ORCHA

Founded by NHS clinicians, ORCHA is the world’s leading digital health evaluation and distribution organisation. We provide services to national health bodies across three continents, including the NHS in 50% of UK regions, delivering national accreditation frameworks, bespoke Digital Health Libraries, and professional recommendation tools, specific to the needs of our clients. ORCHA’s unique Review Engine assesses digital health solutions against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, plus additional criteria depending on needs.

See how ORCHA works

Discover how our services, including Reviews, Digital Health Libraries, and market intelligence reports, can work for your specific needs.

Sign up to our newsletter

For regular updates on digital health, apps, industry news, and more, sign up to our mailing list here.

Your Health and Care App Library

Search ORCHA’s App Library, featuring thousands of independent app reviews across a broad spectrum of health conditions. Every app is evaluated against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, making it easy for you to find the best apps for your needs.

Contact us

For more information about our services, to request a demo, or for advice on any aspect of digital health, please get in touch.