A smartphone-based test for the assessment of attention deficits in delirium: A case-control diagnostic test accuracy study in older hospitalised patients

Autoři: Zoë Tieges aff001;  David J. Stott aff003;  Robert Shaw aff003;  Elaine Tang aff001;  Lisa-Marie Rutter aff001;  Eva Nouzova aff001;  Nikki Duncan aff003;  Caoimhe Clarke aff003;  Christopher J. Weir aff004;  Valentina Assi aff004;  Hannah Ensor aff004;  Jennifer H. Barnett aff005;  Jonathan Evans aff007;  Samantha Green aff001;  Kirsty Hendry aff003;  Meigan Thomson aff003;  Jenny McKeever aff001;  Duncan G. Middleton aff008;  Stuart Parks aff008;  Tim Walsh aff009;  Alexander J. Weir aff008;  Elizabeth Wilson aff010;  Tara Quasim aff011;  Alasdair M. J. MacLullich aff001
Působiště autorů: Edinburgh Delirium Research Group, University of Edinburgh, Edinburgh, Scotland, United Kingdom aff001;  Centre for Cognitive Ageing and Cognitive Epidemiology, University of Edinburgh, Scotland, Edinburgh, United Kingdom aff002;  Institute of Cardiovascular and Medical Sciences, University of Glasgow, Glasgow, Scotland, United Kingdom aff003;  Edinburgh Clinical Trials Unit, Usher Institute of Population Health Sciences and Informatics, University of Edinburgh, Edinburgh, United Kingdom aff004;  Cambridge Cognition Ltd, Cambridge, United Kingdom aff005;  Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom aff006;  Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, United Kingdom aff007;  Medical Devices Unit, West Glasgow Ambulatory Care Hospital, Glasgow, Scotland, United Kingdom aff008;  Critical Care Medicine and Anaesthesia, University of Edinburgh, Edinburgh, Scotland, United Kingdom aff009;  Critical Care Medicine and Anaesthesia, Royal Infirmary of Edinburgh, Edinburgh, Scotland, United Kingdom aff010;  Anaesthesia, Critical Care and Pain Medicine, Glasgow Royal Infirmary, Glasgow, Scotland, United Kingdom aff011
Vyšlo v časopise: PLoS ONE 15(1)
Kategorie: Research Article
doi: https://doi.org/10.1371/journal.pone.0227471



Delirium is a common and serious acute neuropsychiatric syndrome which is often missed in routine clinical care. Inattention is the core cognitive feature. Diagnostic test accuracy (including cut-points) of a smartphone Delirium App (DelApp) for assessing attention deficits was assessed in older hospital inpatients.


This was a case-control study of hospitalised patients aged ≥65 years with delirium (with or without pre-existing cognitive impairment), who were compared to patients with dementia without delirium, and patients without cognitive impairment. Reference standard delirium assessment, which included a neuropsychological test battery, was based on Diagnostic and Statistical Manual of Mental Disorders-5 criteria. A separate blinded assessor administered the DelApp arousal assessment (score 0–4) and attention task (0–6) yielding an overall score of 0 to 10 (lower scores indicate poorer performance). Analyses included receiver operating characteristic curves and sensitivity and specificity. Optimal cut-points for delirium detection were determined using Youden’s index.


A total of 187 patients were recruited, mean age 83.8 (range 67–98) years, 152 (81%) women; n = 61 with delirium; n = 61 with dementia without delirium; and n = 65 without cognitive impairment. Patients with delirium performed poorly on the DelApp (median score = 4/10; inter-quartile range 3.0, 5.5) compared to patients with dementia (9.0; 5.5, 10.0) and those without cognitive impairment (10.0; 10.0, 10.0). Area under the curve for detecting delirium was 0.89 (95% Confidence Interval 0.84, 0.94). At an optimal cut-point of ≤8, sensitivity was 91.7% (84.7%, 98.7%) and specificity 74.2% (66.5%, 81.9%) for discriminating delirium from the other groups. Specificity was 68.3% (56.6%, 80.1%) for discriminating delirium from dementia (cut-point ≤6).


Patients with delirium (with or without pre-existing cognitive impairment) perform poorly on the DelApp compared to patients with dementia and those without cognitive impairment. A cut-point of ≤8/10 is suggested as having optimal sensitivity and specificity. The DelApp is a promising tool for assessment of attention deficits associated with delirium in older hospitalised adults, many of whom have prior cognitive impairment, and should be further validated in representative patient cohorts.

Klíčová slova:

Attention – Case-control studies – Cell phones – Cognitive impairment – Dementia – Diagnostic medicine – Geriatrics – Neuropsychological testing


1. Siddiqi N, House AO, Holmes JD. Occurrence and outcome of delirium in medical in-patients: a systematic literature review. Age Ageing. 2006;35(4):350–64. doi: 10.1093/ageing/afl005 16648149.

2. Reynish EL, Hapca SM, De Souza N, Cvoro V, Donnan PT, Guthrie B. Epidemiology and outcomes of people with dementia, delirium, and unspecified cognitive impairment in the general hospital: prospective cohort study of 10,014 admissions. BMC Med. 2017;15(1):140. doi: 10.1186/s12916-017-0899-0 28747225.

3. Davis DH, Muniz Terrera G, Keage H, Rahkonen T, Oinas M, Matthews FE, et al. Delirium is a strong risk factor for dementia in the oldest-old: a population-based cohort study. Brain. 2012;135(Pt 9):2809–16. doi: 10.1093/brain/aws190 22879644.

4. Inouye SK, Westendorp RG, Saczynski JS. Delirium in elderly people. Lancet. 2014;383(9920):911–22. doi: 10.1016/S0140-6736(13)60688-1 23992774.

5. Partridge JS, Martin FC, Harari D, Dhesi JK. The delirium experience: what is the effect on patients, relatives and staff and what can be done to modify this? Int J Geriatr Psychiatry. 2013;28(8):804–12. doi: 10.1002/gps.3900 23112139.

6. Oh ES, Fong TG, Hshieh TT, Inouye SK. Delirium in Older Persons: Advances in Diagnosis and Treatment. JAMA. 2017;318(12):1161–74. doi: 10.1001/jama.2017.12067 28973626.

7. Collins N, Blanchard MR, Tookman A, Sampson EL. Detection of delirium in the acute hospital. Age Ageing. 2010;39(1):131–5. doi: 10.1093/ageing/afp201 19917632.

8. Kakuma R, du Fort GG, Arsenault L, Perrault A, Platt RW, Monette J, et al. Delirium in older emergency department patients discharged home: effect on survival. J Am Geriatr Soc. 2003;51(4):443–50. doi: 10.1046/j.1532-5415.2003.51151.x 12657062.

9. Royal College of Psychiatrists. National Audit of Dementia care in general hospitals 2012–13: Second round audit report and update. London: 2013.

10. National Institute for Health and Clinical Excellence: Delirium: diagnosis, prevention and management. (Clinical Guideline 103) 2010 [on-line]. www.nice.org.uk/CG103 Accessed May 14, 2018.

11. Young J, Meagher D, Maclullich A. Cognitive assessment of older people. BMJ. 2011;343:d5042. doi: 10.1136/bmj.d5042 21900345.

12. Brown LJ, Fordyce C, Zaghdani H, Starr JM, MacLullich AM. Detecting deficits of sustained visual attention in delirium. J Neurol Neurosurg Psychiatry. 2011;82(12):1334–40. Epub 2010/07/01. doi: 10.1136/jnnp.2010.208827 20587493.

13. Meagher DJ, Leonard M, Donnelly S, Conroy M, Saunders J, Trzepacz PT. A comparison of neuropsychiatric and cognitive profiles in delirium, dementia, comorbid delirium-dementia and cognitively intact controls. J Neurol Neurosurg Psychiatry. 2010;81(8):876–81. doi: 10.1136/jnnp.2009.200956 20587481.

14. Oudewortel L, Joling KJ, Hertogh C, Wijnen VJM, van der Brug AAM, van Gool WA. Performance on bedside tests of attention and organized thinking in patients with dementia free from delirium. Int Psychogeriatr. 2019;31(1):73–81. doi: 10.1017/S1041610218000522 30032736.

15. Tieges Z, Evans JJ, Neufeld KJ, MacLullich AMJ. The neuropsychology of delirium: advancing the science of delirium assessment. Int J Geriatr Psychiatry. 2018;33(11):1501–11. doi: 10.1002/gps.4711 28393426.

16. Tieges Z, Brown LJ, Maclullich AM. Objective assessment of attention in delirium: a narrative review. Int J Geriatr Psychiatry. 2014. doi: 10.1002/gps.4131 24760756.

17. Guenther U, Weykam J, Andorfer U, Theuerkauf N, Popp J, Ely EW, et al. Implications of objective vs subjective delirium assessment in surgical intensive care patients. Am J Crit Care. 2012;21(1):e12–20. 22210705.

18. Simon SE, Bergmann MA, Jones RN, Murphy KM, Orav EJ, Marcantonio ER. Reliability of a structured assessment for nonclinicians to detect delirium among new admissions to postacute care. J Am Med Dir Assoc. 2006;7(7):412–5. doi: 10.1016/j.jamda.2006.02.006 16979083

19. Numan T, van den Boogaard M, Kamper AM, Rood PJT, Peelen LM, Slooter AJC, et al. Recognition of Delirium in Postoperative Elderly Patients: A Multicenter Study. J Am Geriatr Soc. 2017;65(9):1932–8. doi: 10.1111/jgs.14933 28497575.

20. Leonard M, McInerney S, McFarland J, Condon C, Awan F, O’Connor M, et al. Comparison of cognitive and neuropsychiatric profiles in hospitalised elderly medical patients with delirium, dementia and comorbid delirium-dementia. BMJ Open. 2016;6(3):e009212. doi: 10.1136/bmjopen-2015-009212 26956160.

21. Meagher J, Leonard M, Donoghue L, O’Regan N, Timmons S, Exton C, et al. Months backward test: A review of its use in clinical studies. World J Psychiatry. 2015;5(3):305–14. 26425444.

22. European Delirium Association, American Geriatric Society. The DSM-5 criteria, level of arousal and delirium diagnosis: inclusiveness is safer. BMC Med. 2014;12. ARTN 141 doi: 10.1186/s12916-014-0141-2 25300023

23. Tieges Z, Stiobhairt A, Scott K, Suchorab K, Weir A, Parks S, et al. Development of a smartphone application for the objective detection of attentional deficits in delirium. Int Psychogeriatr. 2015;27(8):1251–62. doi: 10.1017/S1041610215000186 25742756.

24. Tieges Z, McGrath A, Hall RJ, Maclullich AM. Abnormal level of arousal as a predictor of delirium and inattention: an exploratory study. Am J Geriatr Psychiatry. 2013;21(12):1244–53. doi: 10.1016/j.jagp.2013.05.003 24080383.

25. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (5th ed.). Washington DC 2013.

26. Rutter LM, Nouzova E, Stott DJ, Weir CJ, Assi V, Barnett JH, et al. Diagnostic test accuracy of a novel smartphone application for the assessment of attention deficits in delirium in older hospitalised patients: a prospective cohort study protocol. BMC Geriatr. 2018;18(1):217. doi: 10.1186/s12877-018-0901-5 30223771.

27. Petersen SE, Posner MI. The attention system of the human brain: 20 years after. Annu Rev Neurosci. 2012;35:73–89. doi: 10.1146/annurev-neuro-062111-150525 22524787.

28. Katzman R, Brown T, Fuld P, Peck A, Schechter R, Schimmel H. Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. Am J Psychiatry. 1983;140(6):734–9. doi: 10.1176/ajp.140.6.734 6846631.

29. Marcantonio ER. Delirium in Hospitalized Older Adults. New Engl J Med. 2017;377(15):1456–66. doi: 10.1056/NEJMcp1605501 29020579

30. Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Validation of the Delirium Rating Scale-revised-98: comparison with the delirium rating scale and the cognitive test for delirium. J Neuropsychiatry Clin Neurosci. 2001;13(2):229–42. doi: 10.1176/jnp.13.2.229 11449030.

31. Sessler CN, Gosnell MS, Grap MJ, Brophy GM, O’Neal PV, Keane KA, et al. The Richmond Agitation-Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):1338–44. doi: 10.1164/rccm.2107138 12421743.

32. Han JH, Vasilevskis EE, Schnelle JF, Shintani A, Dittus RS, Wilson A, et al. The Diagnostic Performance of the Richmond Agitation Sedation Scale for Detecting Delirium in Older Emergency Department Patients. Acad Emerg Med. 2015;22(7):878–82. doi: 10.1111/acem.12706 26113020.

33. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695–9. doi: 10.1111/j.1532-5415.2005.53221.x 15817019.

34. Morandi A, McCurley J, Vasilevskis EE, Fick DM, Bellelli G, Lee P, et al. Tools to Detect Delirium Superimposed on Dementia: A Systematic Review. J Am Geriatr Soc. 2012;60(11):2005–13. doi: 10.1111/j.1532-5415.2012.04199.x 23039270

35. Perry RJ, Watson P, Hodges JR. The nature and staging of attention dysfunction in early (minimal and mild) Alzheimer’s disease: relationship to episodic and semantic memory impairment. Neuropsychologia. 2000;38(3):252–71. doi: 10.1016/s0028-3932(99)00079-2 10678692.

36. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98. 1202204.

37. Tanaka H, Nagata Y, Uematsu M, Takebayashi T, Hanada K, Inokawa M, et al. Development of the Cognitive Test for Severe Dementia. Dement Geriatr Cogn Disord. 2015;40(1–2):94–106. doi: 10.1159/000430089 26043780.

38. Voyer P, Champoux N, Desrosiers J, Landreville P, Monette J, Savoie M, et al. Assessment of inattention in the context of delirium screening: one size does not fit all! Int Psychogeriatr. 2016;28(8):1293–301. doi: 10.1017/S1041610216000533 27004924.

39. Morandi A, Davis D, Bellelli G, Arora RC, Caplan GA, Kamholz B, et al. The Diagnosis of Delirium Superimposed on Dementia: An Emerging Challenge. J Am Med Dir Assoc. 2017;18(1):12–8. doi: 10.1016/j.jamda.2016.07.014 27650668.

40. Richardson SJ, Davis DHJ, Bellelli G, Hasemann W, Meagher D, Kreisel SH, et al. Detecting delirium superimposed on dementia: diagnostic accuracy of a simple combined arousal and attention testing procedure. Int Psychogeriatr. 2017;29(10):1585–93. doi: 10.1017/S1041610217000916 28560945.

41. Jorm AF. A short form of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): development and cross-validation. Psychol Med. 1994;24(1):145–53. doi: 10.1017/s003329170002691x 8208879.

42. Schuurmans MJ, Shortridge-Baggett LM, Duursma SA. The Delirium Observation Screening Scale: a screening instrument for delirium. Res Theory Nurs Pract. 2003;17(1):31–50. doi: 10.1891/rtnp. 12751884.

43. Marcantonio ER, Ngo LH, O’Connor M, Jones RN, Crane PK, Metzger ED, et al. 3D-CAM: derivation and validation of a 3-minute diagnostic interview for CAM-defined delirium: a cross-sectional diagnostic test study. Ann Intern Med. 2014;161(8):554–61. doi: 10.7326/M14-0865 25329203.

Článek vyšel v časopise


2020 Číslo 1
Nejčtenější tento týden