Evaluation of different landing pages on behavioural engagement with the CARA dashboard: A user research protocol.
Dashboard
General practice
Primary health care
User research
Journal
BMC primary care
ISSN: 2731-4553
Titre abrégé: BMC Prim Care
Pays: England
ID NLM: 9918300889006676
Informations de publication
Date de publication:
20 May 2024
20 May 2024
Historique:
received:
04
01
2024
accepted:
07
05
2024
medline:
21
5
2024
pubmed:
21
5
2024
entrez:
20
5
2024
Statut:
epublish
Résumé
CARA set out to develop a data-visualisation platform to facilitate general practitioners to develop a deeper understanding of their patient population, disease management and prescribing through dashboards. To support the continued use and sustainability of the CARA dashboards, dashboard performance and user engagement have to be optimised. User research places people at the centre of the design process and aims to evaluate the needs, behaviours and attitudes of users to inform the design, development and impact of a product. To explore how different initial key messages impact the level of behavioural engagement with a CARA dashboard. Participating general practices can upload their practice data for analysis and visualisation in CARA dashboards. Practices will be randomised to one of three different initial landing pages: the full dashboard or one of two key messages: a between comparison (their practice prescribing with the average of all other practices) or within comparison (with practice data of the same month the previous year) with subsequent continuation to the full dashboard. Analysis will determine which of the three landing pages encourages user interaction, as measured by the number of 'clicks', 'viewings' and 'sessions'. Dashboard usage data will be collected through Google analytics. This study will provide evidence of behavioural engagement and its metrics during the implementation of the CARA dashboards to optimise and sustain interaction. ISRCTN32783644 (Registration date: 02/01/2024).
Sections du résumé
BACKGROUND
BACKGROUND
CARA set out to develop a data-visualisation platform to facilitate general practitioners to develop a deeper understanding of their patient population, disease management and prescribing through dashboards. To support the continued use and sustainability of the CARA dashboards, dashboard performance and user engagement have to be optimised. User research places people at the centre of the design process and aims to evaluate the needs, behaviours and attitudes of users to inform the design, development and impact of a product.
OBJECTIVE
OBJECTIVE
To explore how different initial key messages impact the level of behavioural engagement with a CARA dashboard.
METHODS
METHODS
Participating general practices can upload their practice data for analysis and visualisation in CARA dashboards. Practices will be randomised to one of three different initial landing pages: the full dashboard or one of two key messages: a between comparison (their practice prescribing with the average of all other practices) or within comparison (with practice data of the same month the previous year) with subsequent continuation to the full dashboard. Analysis will determine which of the three landing pages encourages user interaction, as measured by the number of 'clicks', 'viewings' and 'sessions'. Dashboard usage data will be collected through Google analytics.
DISCUSSION
CONCLUSIONS
This study will provide evidence of behavioural engagement and its metrics during the implementation of the CARA dashboards to optimise and sustain interaction.
TRIAL REGISTRATION
BACKGROUND
ISRCTN32783644 (Registration date: 02/01/2024).
Identifiants
pubmed: 38769539
doi: 10.1186/s12875-024-02420-6
pii: 10.1186/s12875-024-02420-6
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
174Subventions
Organisme : Research Leader Awards (RL) 2020, Health Research Board, Ireland
ID : RL-20200-03
Informations de copyright
© 2024. The Author(s).
Références
Collins C, Homeniuk R. How many general practice consultations occur in Ireland annually? Cross-sectional data from a survey of general practices. BMC Fam Pract. 2021;22(1):40.
doi: 10.1186/s12875-021-01377-0
pubmed: 33610171
pmcid: 7896162
Health Service Executive. Terms of agreement between the Department of Health, the HSE and the IMO regarding GP contractual reform and Service Development. In.; 2019.
Schulze A, Brand F, Geppert J, Bol GF. Digital dashboards visualizing public health data: a systematic review. Front Public Health. 2023;11:999958.
doi: 10.3389/fpubh.2023.999958
pubmed: 37213621
pmcid: 10192578
Gardner L, Ratcliff J, Dong E, Katz A. A need for open public data standards and sharing in light of COVID-19. Lancet Infect Dis. 2021;21(4):e80.
doi: 10.1016/S1473-3099(20)30635-6
pubmed: 32791042
Gotz D, Borland D. Data-Driven Healthcare: challenges and opportunities for interactive visualization. IEEE Comput Graph Appl. 2016;36(3):90–6.
doi: 10.1109/MCG.2016.59
pubmed: 28113160
Dagliati A, Malovini A, Tibollo V, Bellazzi R. Health informatics and EHR to support clinical research in the COVID-19 pandemic: an overview. Brief Bioinform. 2021;22(2):812–22.
doi: 10.1093/bib/bbaa418
pubmed: 33454728
pmcid: 7929411
Garzon-Orjuela N, Parveen S, Amin D, Vornhagen H, Blake C, Vellinga A. The effectiveness of interactive dashboards to Optimise Antibiotic Prescribing in Primary Care: a systematic review. Antibiot (Basel) 2023, 12(1).
Zhuang M, Concannon D, Manley E. A Framework for evaluating dashboards in Healthcare. IEEE Trans Vis Comput Graph. 2022;28(4):1715–31.
doi: 10.1109/TVCG.2022.3147154
pubmed: 35213306
Helminski D, Kurlander JE, Renji AD, Sussman JB, Pfeiffer PN, Conte ML, Gadabu OJ, Kokaly AN, Goldberg R, Ranusch A, et al. Dashboards in Health Care settings: protocol for a scoping review. JMIR Res Protoc. 2022;11(3):e34894.
doi: 10.2196/34894
pubmed: 35234650
pmcid: 8928055
Oyedele Y, Greunen D, Veldsman A. UX Engagement and Interaction. In: 2018 7th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO): 29–31 Aug. 2018 2018; 2018: 792–798.
Sauro J, Lewis JR. Chap. 2 - Quantifying user research. In: Quantifying the User Experience (Second Edition) edn. Edited by Sauro J, Lewis JR. Boston: Morgan Kaufmann; 2016: 9–18.
Song MJ, Ward J, Choi F, Nikoo M, Frank A, Shams F, Tabi K, Vigo D, Krausz M. A process evaluation of a web-based Mental Health Portal (WalkAlong) using Google Analytics. JMIR Ment Health. 2018;5(3):e50.
doi: 10.2196/mental.8594
pubmed: 30126832
pmcid: 6121139
Plaza B. Google Analytics for measuring website performance. Tour Manag. 2011;32(3):477–81.
doi: 10.1016/j.tourman.2010.03.015
Ogrodniczuk JS, Beharry J, Oliffe JL. An evaluation of 5-Year web analytics for HeadsUpGuys: a men’s Depression E-Mental Health Resource. Am J Mens Health. 2021;15(6):15579883211063322.
doi: 10.1177/15579883211063322
pubmed: 34861812
pmcid: 8646842
Crutzen R, Roosjen JL, Poelman J. Using Google Analytics as a process evaluation method for internet-delivered interventions: an example on sexual health. Health Promot Int. 2013;28(1):36–42.
doi: 10.1093/heapro/das008
pubmed: 22377974
Clark DJ, Nicholas D, Jamali HR. Evaluating information seeking and use in the changing virtual world: the emerging role of Google Analytics. Learn Publish. 2014;27(3):185–94.
doi: 10.1087/20140304
Vona P, Wilmoth P, Jaycox LH, McMillen JS, Kataoka SH, Wong M, DeRosier ME, Langley AK, Kaufman J, Tang L, Stein BD. A web-based platform to support an evidence-based Mental Health intervention: lessons from the CBITS web site. Psychiatric Serv. 2014;65(11):1381–4.
doi: 10.1176/appi.ps.201300512
CARA NETWORK. [ https://caranetwork.ie ].
Garzon-Orjuela N, Garcia Pereira A, Vornhagen V, Stasiewicz K, Parveen S, Amin D, Porwol L, Vellinga A. Design of infrastructure for visualising and benchmarking patient data from general practice (CARA). Eur J Public Health. 2023;33(Suppl 2):ckad160.1210. https://doi.org/10.1093/eurpub/ckad160.1210 .
O’Connor N, Breen R, Carton M, Mc Grath I, Deasy N, Collins C, Vellinga A. Improving the quality of antibiotic prescribing through an educational intervention delivered through the out-of-hours general practice service in Ireland. Eur J Gen Pract. 2020;26(1):119–25.
doi: 10.1080/13814788.2020.1784137
pubmed: 32757866
pmcid: 7470051
Government of Ireland. Ireland ’s Second One Health National Action Plan on Antimicrobial Resistance 2021–2025; 2021.
Sein MK, Henfridsson O, Purao S, Rossi M, Lindgren R. Action Design Research. MIS Q. 2011;35(1):37–56.
doi: 10.2307/23043488
Garzon-Orjuela N, Vornhagen H, Stasiewicz K, Garcia Pereira A, Vellinga A. Interactive dashboard development for Irish general practices to visualise and compare patient data. Eur J Public Health. 2023;33(Suppl 2):ckad160.575. https://doi.org/10.1093/eurpub/ckad160.575 .
Meeker D, Linder JA, Fox CR, Friedberg MW, Persell SD, Goldstein NJ, Knight TK, Hay JW, Doctor JN. Effect of behavioral interventions on Inappropriate Antibiotic Prescribing among Primary Care practices: a Randomized Clinical Trial. JAMA. 2016;315(6):562–70.
doi: 10.1001/jama.2016.0275
pubmed: 26864410
pmcid: 6689234
Zeng Y, Shi L, Liu C, Li W, Li J, Yang S, Yang X, Huang Q, Yang L. Effects of social norm feedback on antibiotic prescribing and its characteristics in behaviour change techniques: a mixed-methods systematic review. Lancet Infect Dis. 2023;23(5):e175–84.
doi: 10.1016/S1473-3099(22)00720-4
pubmed: 36521504
Talat U, Schmidtke KA, Khanal S, Chan A, Turner A, Horne R, Chadborn T, Gold N, Sallis A, Vlaev I. A systematic review of Nudge interventions to optimize medication prescribing. Front Pharmacol. 2022;13:798916.
doi: 10.3389/fphar.2022.798916
pubmed: 35145411
pmcid: 8822212
CARA dashboard v1.0. [ http://vmcara-vm2.datascienceinstitute.ie/dashboard/app] ].
Dimensions and metrics. [GA4] Understand user metrics [ https://support.google.com/analytics/answer/12253918?hl=en& ref_topic=11151952&sjid=17122983140791273295-EU]
Sauro J, Lewis JR. Chap. 6 - What sample sizes do we need? Part 1: summative studies. In: Quantifying the User Experience (Second Edition) edn. Edited by Sauro J, Lewis JR. Boston: Morgan Kaufmann; 2016: 103–141.
Lewis JR. Usability Testing. In: Handbook of Human Factors and Ergonomics edn.; 2006: 1275–1316.
Rothfeld R. Advancing Web-based Dashboards: Providing Contextualised Comparisons in an Air Traffic Discovery Dashboard. 2015.
Curtis HJ, Bacon S, Croker R, Walker AJ, Perera R, Hallsworth M, Harper H, Mahtani KR, Heneghan C, Goldacre B. Evaluating the impact of a very low-cost intervention to increase practices’ engagement with data and change prescribing behaviour: a randomized trial in English primary care. Fam Pract. 2021;38(4):373–80.
doi: 10.1093/fampra/cmaa128
pubmed: 33783497
Sauro J, Lewis JR. Chap. 5 - Is there a statistical difference between designs? In: Quantifying the User Experience (Second Edition) edn. Edited by Sauro J, Lewis JR. Boston: Morgan Kaufmann; 2016: 61–102.
Almasi S, Bahaadinbeigy K, Ahmadi H, Sohrabei S, Rabiei R. Usability Evaluation of Dashboards: A Systematic Literature Review of Tools. Biomed Res Int 2023, 2023:9990933.
: MyPractice, Care P. [ https://www.hqontario.ca/Quality Improvement/Practice-Reports/Primary-Care].
Building Rapid Interventions. to reduce antibiotic resisTance (BRIT) [ https://www.britanalytics.uk/ ].
Few S. Information Dashboard Design: The Effective Visual Communication of Data / S. Few. 2006.
Koneska E, Appelbe D, Williamson PR, Dodd S. Usage Metrics of web-based interventions evaluated in Randomized controlled trials: systematic review. J Med Internet Res. 2020;22(4):e15474.
doi: 10.2196/15474
pubmed: 32297870
pmcid: 7193439
Bickmore T, Schulman D, Yin L. Maintaining Engagement in Long-Term interventions with Relational agents. Appl Artif Intell. 2010;24(6):648–66.
doi: 10.1080/08839514.2010.492259
pubmed: 21318052
pmcid: 3035950
Taki S, Lymer S, Russell CG, Campbell K, Laws R, Ong KL, Elliott R, Denney-Wilson E. Assessing user Engagement of an mHealth intervention: development and implementation of the growing healthy app Engagement Index. JMIR Mhealth Uhealth. 2017;5(6):e89.
doi: 10.2196/mhealth.7236
pubmed: 28663164
pmcid: 5509951
Hoang P. Measuring user Engagement from interactive media. The University of Bergen; 2022.
Sarikaya A, Correll M, Bartram L, Tory M, Fisher D. What do we talk about when we talk about dashboards? IEEE Trans Vis Comput Graph. 2019;25(1):682–92.
doi: 10.1109/TVCG.2018.2864903