How do you best measure the user experience of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

User Experience (UX) is key to the success of a digital health product. It doesn’t matter how beneficial a product has the potential to be, if the user isn’t using it as intended due to a poor user experience, they won’t be getting the most out of it.

The System Usability Scale (SUS) is a widely used scale that is typically used to assess how usable software and hardware products are. The scale was not specifically designed to be used for mobile apps, or digital health products, but it has been used by some to do so.

To understand how useful and accurate it is, and to guide digital health suppliers, the ORCHA research team collaborated with Ulster University on a piece of research to discover whether the SUS can be used to reliably assess the usability of digital health products.

PhD researcher, Maciej Hyzy led the investigation, with support from authors including Sophie Hunt, Senior Digital Health Assessor, and Dr Simon Leigh, Director of Research at ORCHA. The team analysed previously published research to find digital health products which had published SUS scores. They then analysed the distribution of these SUS scores for digital health products in different condition areas, to evaluate whether the existing benchmark of a mean SUS score of 68 (widely accepted for software and hardware products) is relevant when evaluating the usability of digital health products. They found that overall, the SUS is a good starting point for measuring usability in digital health products. However, given that the findings are highly subjective, and demonstrated to be variable by condition area, more work needs to be done in this space, perhaps with reference to more objective and evidence-based approaches to assessing UX.

It was clear from the analysis that future work should aim to supplement the subjective user experience assessments provided by the SUS with questions that can be answered more objectively. This will enable us to understand exactly why some digital health products provide a better user experience than others. We can then learn exactly what makes a digital health product with good user experience.

The report has been published in JMIR Publications mHealth and uHealth, and is available to read on Ulster University’s website.

PhD researcher, Maciej, said:

“The System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS scale was not specifically designed to evaluate mobile apps, or indeed digital health products. The objective of this study was to examine whether the widely used SUS distribution for benchmarking (mean of 68 and standard deviation of 12.5) can be used to reliably assess the usability of digital health products.”

If you have any questions about this research, or about how ORCHA assesses user experience, please get in touch with us at hello@orchahealth.com.