Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods.
audit logs
electronic health records
systematic review
usability
workflow
Journal
Journal of the American Medical Informatics Association : JAMIA
ISSN: 1527-974X
Titre abrégé: J Am Med Inform Assoc
Pays: England
ID NLM: 9430800
Informations de publication
Date de publication:
01 03 2020
01 03 2020
Historique:
received:
02
08
2019
revised:
07
10
2019
accepted:
18
10
2019
pubmed:
22
11
2019
medline:
20
1
2021
entrez:
22
11
2019
Statut:
ppublish
Résumé
To systematically review published literature and identify consistency and variation in the aims, measures, and methods of studies using electronic health record (EHR) audit logs to observe clinical activities. In July 2019, we searched PubMed for articles using EHR audit logs to study clinical activities. We coded and clustered the aims, measures, and methods of each article into recurring categories. We likewise extracted and summarized the methods used to validate measures derived from audit logs and limitations discussed of using audit logs for research. Eighty-five articles met inclusion criteria. Study aims included examining EHR use, care team dynamics, and clinical workflows. Studies employed 6 key audit log measures: counts of actions captured by audit logs (eg, problem list viewed), counts of higher-level activities imputed by researchers (eg, chart review), activity durations, activity sequences, activity clusters, and EHR user networks. Methods used to preprocess audit logs varied, including how authors filtered extraneous actions, mapped actions to higher-level activities, and interpreted repeated actions or gaps in activity. Nineteen studies validated results (22%), but only 9 (11%) through direct observation, demonstrating varying levels of measure accuracy. While originally designed to aid access control, EHR audit logs have been used to observe diverse clinical activities. However, most studies lack sufficient discussion of measure definition, calculation, and validation to support replication, comparison, and cross-study synthesis. EHR audit logs have potential to scale observational research but the complexity of audit log measures necessitates greater methodological transparency and validated standards.
Identifiants
pubmed: 31750912
pii: 5637327
doi: 10.1093/jamia/ocz196
pmc: PMC7025338
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
480-490Subventions
Organisme : NEI NIH HHS
ID : P30 EY010572
Pays : United States
Organisme : NLM NIH HHS
ID : R00 LM012238
Pays : United States
Organisme : NLM NIH HHS
ID : T15 LM007088
Pays : United States
Informations de copyright
© The Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com.
Références
J Am Med Inform Assoc. 2017 Apr 1;24(e1):e28-e34
pubmed: 27375293
BMC Med Inform Decis Mak. 2013 Apr 17;13:49
pubmed: 23594488
Am J Med. 2014 Sep;127(9):891-5
pubmed: 24907594
J Biomed Inform. 2011 Dec;44(6):1056-67
pubmed: 21867774
JMIR Mhealth Uhealth. 2017 Dec 13;5(12):e178
pubmed: 29237579
BMC Med Inform Decis Mak. 2016 Jan 30;16:12
pubmed: 26831123
J Am Med Inform Assoc. 2011 Mar-Apr;18(2):112-7
pubmed: 21292706
J Am Med Inform Assoc. 2013 Jan 1;20(1):144-51
pubmed: 22733976
J Am Med Inform Assoc. 2019 Jun 1;26(6):506-515
pubmed: 30889243
Am J Manag Care. 2013 Nov;19(10 Spec No):SP345-52
pubmed: 24511889
Stud Health Technol Inform. 2016;225:858-9
pubmed: 27332378
J Am Coll Surg. 2008 Mar;206(3):405-11
pubmed: 18308208
AMIA Annu Symp Proc. 2015 Nov 05;2015:669-76
pubmed: 26958202
Stud Health Technol Inform. 2006;122:273-8
pubmed: 17102263
Int J Med Inform. 2014 Jul;83(7):495-506
pubmed: 24845147
Jt Comm J Qual Patient Saf. 2010 Feb;36(2):72-8
pubmed: 20180439
Int J Med Inform. 2019 Aug;128:46-52
pubmed: 31160011
AMIA Annu Symp Proc. 2018 Apr 16;2017:790-799
pubmed: 29854145
Arch Intern Med. 2011 Mar 28;171(6):568-74
pubmed: 21444847
J Am Med Inform Assoc. 2015 Jan;22(1):206-12
pubmed: 25074989
J Gen Intern Med. 2017 Nov;32(11):1210-1219
pubmed: 28808942
Comput Inform Nurs. 2019 May;37(5):260-265
pubmed: 31094915
J Am Med Inform Assoc. 2018 Jul 1;25(7):790-799
pubmed: 29481625
AMIA Annu Symp Proc. 2007 Oct 11;:135-9
pubmed: 18693813
J Am Med Inform Assoc. 2010 May-Jun;17(3):265-73
pubmed: 20442143
Health Informatics J. 2015 Mar;21(1):73-88
pubmed: 24692078
AMIA Annu Symp Proc. 2018 Apr 16;2017:1820-1827
pubmed: 29854253
J Biomed Inform. 2018 Jan;77:1-10
pubmed: 29174994
Am J Med. 2010 Mar;123(3):286-90
pubmed: 20193841
AMIA Annu Symp Proc. 2003;:145-9
pubmed: 14728151
AMIA Annu Symp Proc. 2018 Dec 05;2018:490-497
pubmed: 30815089
J Am Med Inform Assoc. 2011 Mar-Apr;18(2):125-30
pubmed: 21292704
J Surg Educ. 2018 Nov;75(6):e97-e106
pubmed: 30522828
AMIA Annu Symp Proc. 2008 Nov 06;:809-13
pubmed: 18999307
IEEE Trans Dependable Secure Comput. 2012 May;9(3):332-344
pubmed: 24489520
AMIA Annu Symp Proc. 2018 Apr 16;2017:921-929
pubmed: 29854159
Appl Clin Inform. 2019 Jan;10(1):28-37
pubmed: 30625502
J Am Med Inform Assoc. 2018 Jan 1;25(1):40-46
pubmed: 29036581
J Am Med Inform Assoc. 2006 Jan-Feb;13(1):91-5
pubmed: 16221943
Appl Clin Inform. 2015 Oct 21;6(4):629-37
pubmed: 26767060
Mayo Clin Proc. 2015 Dec;90(12):1600-13
pubmed: 26653297
J Am Med Inform Assoc. 2015 Mar;22(2):299-311
pubmed: 25710558
J Oncol Pract. 2013 Jul;9(4):e103-14
pubmed: 23942926
Stud Health Technol Inform. 2010;160(Pt 1):86-90
pubmed: 20841655
J Digit Imaging. 2018 Oct;31(5):611-614
pubmed: 29696473
Ann Fam Med. 2017 Sep;15(5):419-426
pubmed: 28893811
JMIR Mhealth Uhealth. 2019 Jan 17;7(1):e12041
pubmed: 30664473
Appl Clin Inform. 2014 Apr 09;5(2):368-87
pubmed: 25024755
CODASPY. 2011;2011:63-74
pubmed: 25485309
J Am Med Inform Assoc. 2017 Apr 1;24(e1):e111-e120
pubmed: 27570217
AMIA Annu Symp Proc. 2006;:151-5
pubmed: 17238321
Telemed J E Health. 2014 Mar;20(3):215-22
pubmed: 23909863
Health Aff (Millwood). 2017 Apr 1;36(4):655-662
pubmed: 28373331
J Grad Med Educ. 2014 Mar;6(1):151-4
pubmed: 24701327
Healthc Inform Res. 2015 Oct;21(4):292-8
pubmed: 26618036
J Grad Med Educ. 2016 Feb;8(1):39-44
pubmed: 26913101
AMIA Annu Symp Proc. 2014 Nov 14;2014:749-56
pubmed: 25954381
Int J Med Inform. 2004 Mar 31;73(3):267-70
pubmed: 15066557
J Am Med Inform Assoc. 2011 Sep-Oct;18(5):704-10
pubmed: 21527407
Appl Clin Inform. 2018 Jan;9(1):99-104
pubmed: 30184241
J Med Internet Res. 2009 Oct 27;11(4):e44
pubmed: 19861298
J Am Med Inform Assoc. 2015 Nov;22(6):1179-82
pubmed: 26049532
AMIA Annu Symp Proc. 2018 Apr 16;2017:760-769
pubmed: 29854142
J Am Med Inform Assoc. 2019 Feb 1;26(2):106-114
pubmed: 30517663
AMIA Annu Symp Proc. 2017 Feb 10;2016:1248-1257
pubmed: 28269922
J Med Syst. 2019 Mar 16;43(5):105
pubmed: 30879188
Stud Health Technol Inform. 2004;107(Pt 1):18-22
pubmed: 15360766
AMIA Annu Symp Proc. 2015 Nov 05;2015:416-25
pubmed: 26958173
J Am Med Inform Assoc. 1997 Sep-Oct;4(5):342-55
pubmed: 9292840
JAMA Intern Med. 2016 Feb;176(2):252-4
pubmed: 26642261
J Am Med Inform Assoc. 2011 Jul-Aug;18(4):498-505
pubmed: 21672912
Ophthalmology. 2019 Jun;126(6):783-791
pubmed: 30664893
Proc Annu Symp Comput Appl Med Care. 1993;:107-11
pubmed: 8130443
J Med Internet Res. 2017 Oct 18;19(10):e340
pubmed: 29046269
AMIA Annu Symp Proc. 2015 Nov 05;2015:1909-17
pubmed: 26958290
J Am Med Inform Assoc. 2007 Mar-Apr;14(2):235-8
pubmed: 17213496
Stud Health Technol Inform. 2012;180:656-60
pubmed: 22874273
AMIA Annu Symp Proc. 2011;2011:858-67
pubmed: 22195144
Am J Med. 2016 Sep;129(9):993-999.e1
pubmed: 27103047
JAMA Ophthalmol. 2018 Jan 1;136(1):20-26
pubmed: 29121175
AMIA Annu Symp Proc. 2003;:175-9
pubmed: 14728157
J Interprof Care. 2018 Nov;32(6):666-673
pubmed: 30015537
PLoS One. 2019 Feb 6;14(2):e0205379
pubmed: 30726208
JAMA Ophthalmol. 2017 Nov 1;135(11):1250-1257
pubmed: 29049512
Appl Clin Inform. 2011;2(4):395-405
pubmed: 22574103
J Gen Intern Med. 2019 Nov;34(11):2355-2367
pubmed: 31183688
Ann Fam Med. 2014 Nov-Dec;12(6):573-6
pubmed: 25384822
JMIR Med Inform. 2019 Feb 07;7(1):e12650
pubmed: 30730293
AMIA Annu Symp Proc. 2011;2011:491-500
pubmed: 22195103
Mayo Clin Proc. 2016 Jul;91(7):836-48
pubmed: 27313121
J Am Med Inform Assoc. 2010 May-Jun;17(3):328-36
pubmed: 20442152
J Am Med Inform Assoc. 2014 Nov-Dec;21(6):1001-8
pubmed: 24914013
AMIA Annu Symp Proc. 2005;:141-5
pubmed: 16779018
Stud Health Technol Inform. 2001;84(Pt 2):1440-4
pubmed: 11604964
J Biomed Inform. 2011 Apr;44(2):333-42
pubmed: 21277996
Diagnosis (Berl). 2018 Nov 27;5(4):235-242
pubmed: 30240357
Mach Learn. 2014 Apr 1;95(1):87-101
pubmed: 24683293
EGEMS (Wash DC). 2016 May 02;4(1):1176
pubmed: 27195306
J Biomed Inform. 2014 Jun;49:292-9
pubmed: 24607863
AMIA Annu Symp Proc. 2017 Feb 10;2016:647-656
pubmed: 28269861
J Am Med Inform Assoc. 2018 Nov 1;25(11):1501-1506
pubmed: 30137348
Ophthalmology. 2019 Mar;126(3):347-354
pubmed: 30312629
J Med Syst. 2012 Dec;36(6):3795-803
pubmed: 22527782
Stud Health Technol Inform. 2017;245:1231
pubmed: 29295318
J Am Med Inform Assoc. 2013 Jun;20(e1):e2-8
pubmed: 23355463