Proof of concept for a multi-method approach to evaluate the usability of websites with public health content: The case of Alzheimer association websites

Abstract

Methods for assessing usability are advancing rapidly, and include think-aloud protocols, objective measures such as task timing, and ultra-sensitive measures such as eye tracking and page recording. In this study, we provide proof-of-concept for the new, combined use of these three types of measures within a quasi-experimental paradigm focused on a question relevant to public health. The quasi-experiment specifically involved assessing the usability of four English-language Alzheimer websites from around the world (USA, UK, Ireland, Canada), all of which contained information about cognitive health. Participants completed two tasks; one requiring them to identify strategies to prevent dementia and another requiring them to locate a definition of Alzheimer’s disease. We hypothesized that the websites would have superior usability for the former task and inferior usability for the latter task. We also hypothesized positive correlations among eye tracking measures, simpler performance measures (e.g., task completion time), and subjective usability measures. We found that the new quasi-experimental protocol facilitated comparison of tasks and websites. It also facilitated interpretation by permitting comparison across measures. Overall, this study provides proof of concept for the use of this multi-method approach to evaluate the usability of websites. It also provides information that could potentially be used to advance further pilot or experimental hypothesis testing on this topic.