How do you best measure the usability of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

Published Research

How do you best measure the user experience of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

User Experience (UX) is key to the success of a digital health product. It doesn’t matter how beneficial a product has the potential to be, if the user isn’t using it as intended due to a poor user experience, they won’t be getting the most out of it.

The System Usability Scale (SUS) is a widely used scale that is typically used to assess how usable software and hardware products are. The scale was not specifically designed to be used for mobile apps, or digital health products, but it has been used by some to do so.

To understand how useful and accurate it is, and to guide digital health suppliers, the ORCHA research team collaborated with Ulster University on a piece of research to discover whether the SUS can be used to reliably assess the usability of digital health products.

PhD researcher, Maciej Hyzy led the investigation, with support from authors including Sophie Hunt, Senior Digital Health Assessor, and Dr Simon Leigh, Director of Research at ORCHA. The team analysed previously published research to find digital health products which had published SUS scores. They then analysed the distribution of these SUS scores for digital health products in different condition areas, to evaluate whether the existing benchmark of a mean SUS score of 68 (widely accepted for software and hardware products) is relevant when evaluating the usability of digital health products. They found that overall, the SUS is a good starting point for measuring usability in digital health products. However, given that the findings are highly subjective, and demonstrated to be variable by condition area, more work needs to be done in this space, perhaps with reference to more objective and evidence-based approaches to assessing UX.

It was clear from the analysis that future work should aim to supplement the subjective user experience assessments provided by the SUS with questions that can be answered more objectively. This will enable us to understand exactly why some digital health products provide a better user experience than others. We can then learn exactly what makes a digital health product with good user experience.

The report has been published in JMIR Publications mHealth and uHealth, and is available to read on Ulster University’s website.

PhD researcher, Maciej, said:

“The System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS scale was not specifically designed to evaluate mobile apps, or indeed digital health products. The objective of this study was to examine whether the widely used SUS distribution for benchmarking (mean of 68 and standard deviation of 12.5) can be used to reliably assess the usability of digital health products.”

If you have any questions about this research, or about how ORCHA assesses user experience, please get in touch with us at hello@orcha.co.uk.

About ORCHA

Founded by NHS clinicians, ORCHA is the world’s leading digital health evaluation and distribution organisation. We provide services to national health bodies across three continents, including the NHS in 50% of UK regions, delivering national accreditation frameworks, bespoke Digital Health Libraries, and professional recommendation tools, specific to the needs of our clients. ORCHA’s unique Review Engine assesses digital health solutions against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, plus additional criteria depending on needs.

See how ORCHA works

Discover how our services, including Reviews, Digital Health Libraries, and market intelligence reports, can work for your specific needs.

Sign up to our newsletter

For regular updates on digital health, apps, industry news, and more, sign up to our mailing list here.

Your Health and Care App Library

Search ORCHA’s App Library, featuring thousands of independent app reviews across a broad spectrum of health conditions. Every app is evaluated against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, making it easy for you to find the best apps for your needs.

Contact us

For more information about our services, to request a demo, or for advice on any aspect of digital health, please get in touch.

Web-based internet searches for digital health products in the United Kingdom before and during the COVID-19 pandemic: a time-series analysis using app libraries from ORCHA

Published Research

How do you best measure the user experience of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

Published in: BMJ Open

The traditional model of healthcare delivery is based on providing medical services through systems of hospitals, primary care facilities and outpatient clinics. However, the COVID-19 pandemic has profoundly disrupted the routine delivery of physical healthcare, resulting in the widespread deferral of elective, preventive and outpatient appointments by health authorities. Estimates suggest that as many as 40% of appointments have been cancelled or postponed by patients, as part of efforts to avoid public spaces as much as possible. In the USA, ambulatory care visits fell by 60% in the early phase of the pandemic, while in the UK alone, an estimated 1.5 million elective admissions and 2.6 million outpatient attendances were forgone as a result of COVID-19.

We explored if consumer interest in digital health products (DHPs) changed following the COVID-19 pandemic and the lockdown measures that ensued. We identified all web-based internet searches for DHPs in the UK using ORCHA app libraries, split over two periods, pre-COVID-19 lockdown (January 2019–23 March 2020) and post-COVID-19 lockdown (24 March 2020–31 December 2020).

There were 126 640 searches for DHPs over the study period. Searches for DHPs increased by 343% from 2446 per month prior to COVID-19 lockdown measures being introduced to 8996 per month in the period following the first COVID-19 lockdown in the UK. In total, 23/25 (92%) of condition areas experienced a significant increase in searches for DHPs, with the greatest increases occurring in the first 2 months following lockdown.

Musculoskeletal conditions (2.036%), allergy (1.253%) and healthy living DHPs (1.051%) experienced the greatest increases in searches compared with pre-lockdown. Increased search volumes for DHPs were sustained in the 9 months following the introduction of lockdown measures, with 21/25 (84%) of condition areas experiencing monthly search volumes at least 50% greater than pre-lockdown levels.

Our findings suggest that the COVID-19 pandemic has profoundly disrupted the routine delivery of healthcare, making face-to-face interaction difficult, and contributing to unmet clinical needs. This study has demonstrated significant increases in internet searches for DHPs by members of the UK population since COVID-19, signifying an increased interest in this potential therapeutic medium. Future research should clarify whether this increased interest has resulted in increased acceptance and utilisation of these technologies also.

 

Read the full research piece here: https://bmjopen.bmj.com/content/11/10/e053891

About ORCHA

Founded by NHS clinicians, ORCHA is the world’s leading digital health evaluation and distribution organisation. We provide services to national health bodies across three continents, including the NHS in 50% of UK regions, delivering national accreditation frameworks, bespoke Digital Health Libraries, and professional recommendation tools, specific to the needs of our clients. ORCHA’s unique Review Engine assesses digital health solutions against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, plus additional criteria depending on needs.

Oops! We could not locate your form.

Contact us

For more information about our services, to request a demo, or for advice on any aspect of digital health, please get in touch.

Barriers and Facilitators to the Adoption of Mobile Health Among Health Care Professionals From the United Kingdom: Discrete Choice Experiment

Published Research

How do you best measure the user experience of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

Published in: BMJ Open

The traditional model of healthcare delivery is based on providing medical services through systems of hospitals, primary care facilities and outpatient clinics. However, the COVID-19 pandemic has profoundly disrupted the routine delivery of physical healthcare, resulting in the widespread deferral of elective, preventive and outpatient appointments by health authorities. Estimates suggest that as many as 40% of appointments have been cancelled or postponed by patients, as part of efforts to avoid public spaces as much as possible. In the USA, ambulatory care visits fell by 60% in the early phase of the pandemic, while in the UK alone, an estimated 1.5 million elective admissions and 2.6 million outpatient attendances were forgone as a result of COVID-19.

We explored if consumer interest in digital health products (DHPs) changed following the COVID-19 pandemic and the lockdown measures that ensued. We identified all web-based internet searches for DHPs in the UK using ORCHA app libraries, split over two periods, pre-COVID-19 lockdown (January 2019–23 March 2020) and post-COVID-19 lockdown (24 March 2020–31 December 2020).

There were 126 640 searches for DHPs over the study period. Searches for DHPs increased by 343% from 2446 per month prior to COVID-19 lockdown measures being introduced to 8996 per month in the period following the first COVID-19 lockdown in the UK. In total, 23/25 (92%) of condition areas experienced a significant increase in searches for DHPs, with the greatest increases occurring in the first 2 months following lockdown.

Musculoskeletal conditions (2.036%), allergy (1.253%) and healthy living DHPs (1.051%) experienced the greatest increases in searches compared with pre-lockdown. Increased search volumes for DHPs were sustained in the 9 months following the introduction of lockdown measures, with 21/25 (84%) of condition areas experiencing monthly search volumes at least 50% greater than pre-lockdown levels.

Our findings suggest that the COVID-19 pandemic has profoundly disrupted the routine delivery of healthcare, making face-to-face interaction difficult, and contributing to unmet clinical needs. This study has demonstrated significant increases in internet searches for DHPs by members of the UK population since COVID-19, signifying an increased interest in this potential therapeutic medium. Future research should clarify whether this increased interest has resulted in increased acceptance and utilisation of these technologies also.

 

Read the full research piece here: https://bmjopen.bmj.com/content/11/10/e053891

About ORCHA

Founded by NHS clinicians, ORCHA is the world’s leading digital health evaluation and distribution organisation. We provide services to national health bodies across three continents, including the NHS in 50% of UK regions, delivering national accreditation frameworks, bespoke Digital Health Libraries, and professional recommendation tools, specific to the needs of our clients. ORCHA’s unique Review Engine assesses digital health solutions against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, plus additional criteria depending on needs.

Oops! We could not locate your form.

Contact us

For more information about our services, to request a demo, or for advice on any aspect of digital health, please get in touch.

The role of health-care providers in mHealth adoption

Published Research

How do you best measure the user experience of a digital health product? ORCHA’s research team publishes in JMIR Publications mHealth and uHealth

Published in: BMJ Open

The traditional model of healthcare delivery is based on providing medical services through systems of hospitals, primary care facilities and outpatient clinics. However, the COVID-19 pandemic has profoundly disrupted the routine delivery of physical healthcare, resulting in the widespread deferral of elective, preventive and outpatient appointments by health authorities. Estimates suggest that as many as 40% of appointments have been cancelled or postponed by patients, as part of efforts to avoid public spaces as much as possible. In the USA, ambulatory care visits fell by 60% in the early phase of the pandemic, while in the UK alone, an estimated 1.5 million elective admissions and 2.6 million outpatient attendances were forgone as a result of COVID-19.

We explored if consumer interest in digital health products (DHPs) changed following the COVID-19 pandemic and the lockdown measures that ensued. We identified all web-based internet searches for DHPs in the UK using ORCHA app libraries, split over two periods, pre-COVID-19 lockdown (January 2019–23 March 2020) and post-COVID-19 lockdown (24 March 2020–31 December 2020).

There were 126 640 searches for DHPs over the study period. Searches for DHPs increased by 343% from 2446 per month prior to COVID-19 lockdown measures being introduced to 8996 per month in the period following the first COVID-19 lockdown in the UK. In total, 23/25 (92%) of condition areas experienced a significant increase in searches for DHPs, with the greatest increases occurring in the first 2 months following lockdown.

Musculoskeletal conditions (2.036%), allergy (1.253%) and healthy living DHPs (1.051%) experienced the greatest increases in searches compared with pre-lockdown. Increased search volumes for DHPs were sustained in the 9 months following the introduction of lockdown measures, with 21/25 (84%) of condition areas experiencing monthly search volumes at least 50% greater than pre-lockdown levels.

Our findings suggest that the COVID-19 pandemic has profoundly disrupted the routine delivery of healthcare, making face-to-face interaction difficult, and contributing to unmet clinical needs. This study has demonstrated significant increases in internet searches for DHPs by members of the UK population since COVID-19, signifying an increased interest in this potential therapeutic medium. Future research should clarify whether this increased interest has resulted in increased acceptance and utilisation of these technologies also.

 

Read the full research piece here: https://bmjopen.bmj.com/content/11/10/e053891

About ORCHA

Founded by NHS clinicians, ORCHA is the world’s leading digital health evaluation and distribution organisation. We provide services to national health bodies across three continents, including the NHS in 50% of UK regions, delivering national accreditation frameworks, bespoke Digital Health Libraries, and professional recommendation tools, specific to the needs of our clients. ORCHA’s unique Review Engine assesses digital health solutions against more than 300 measures across Clinical/Professional Assurance, Data u0026amp; Privacy, and Usability u0026amp; Accessibility, plus additional criteria depending on needs.

Oops! We could not locate your form.

Contact us

For more information about our services, to request a demo, or for advice on any aspect of digital health, please get in touch.