Evaluating case management as a complex intervention: Lessons for the future

Autoři: Anne-Sophie Lambert aff001;  Catherine Legrand aff002;  Sophie Cès aff001;  Thérèse Van Durme aff001;  Jean Macq aff001
Působiště autorů: Institute of Health and Society (IRSS), Université Catholique de Louvain, Clos Chapelle aux Champs, Brussels, Belgium aff001;  Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA-IMMAQ), Université Catholique de Louvain, Louvain-la-Neuve, Belgium aff002
Vyšlo v časopise: PLoS ONE 14(10)
Kategorie: Research Article
doi: 10.1371/journal.pone.0224286


The methodological challenges to effectiveness evaluation of complex interventions has been widely discussed. Bottom-up case management for frail older person was implemented in Belgium, and indeed, it was evaluated as a complex intervention. This paper presents the methodological approach we developed to respond to four main methodological challenges regarding the evaluation of case management: (1) the standardization of the interventions, (2) stratification of the frail older population that was used to test various modalities of case management with different risks groups, (3) the building of a control group, and (4) the use of multiple outcomes in evaluating case management. To address these challenges, we developed a mixed-methods approach that (1) used multiple embedded case studies to classify case management types according to their characteristics and implementation conditions; and (2) compared subgroups of beneficiaries with specific needs (defined by Principal Component Analysis prior to cluster analysis) and a control group receiving ‘usual care’, to evaluate the effectiveness of case management. The beneficiaries’ subgroups were matched using propensity scores and compared using generalized pairwise comparison and the hurdle model with the control group. Our results suggest that the impact of case management on patient health and the services used varies according to specific needs and categories of case management. However, these equivocal results question our methodological approach. We suggest to reconsider the evaluation approach by moving away from a viewing case management as an intervention. Rather, it should be considered as a process of interconnected actions taking place within a complex system.

Klíčová slova:

Activities of daily living – Behavior – Cognitive impairment – Complex systems – Critical care and emergency medicine – Depression – Frailty – Long-term care


1. Vandresse M. Perspectives démographiques 2016–2060 Population, ménages et quotients de mortalité prospectifs. Brussels: Bureau du plan, 2017.

2. Fried LP, Ferrucci L, Darer J, Williamson JD, Anderson G. Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 2004;59(3):M255–M63.

3. Van den Bosch K, Willemé P, Geerts J, Breda J, Peeters S, Van de Sande S, et al. Soins résidentiels pour les personnes âgées en Belgique: projections 2011–2025. Brussels: Centre fédéral d’expertise des soins de santé (KCE), 2011.

4. Paulus D, Van den Heede K, Gerkens S, Desomer A, Mertens R. Development of a national position paper for chronic care: example of Belgium. Health policy. 2013;111(2):105–9. doi: 10.1016/j.healthpol.2013.04.010 23664776

5. Cès S, Van Durme T, Schmitz O, Macq J, Maggi P, Delye S, et al. Scientific evaluation of projects of alternative forms of care or support of care for frail elderly, in order to allow them to maintain their autonomy and to live independently in their homes "Protocol 3". Brussels: National Insurance for Health and Disability Institute, 2014.

6. Beswick AD, Rees K, Dieppe P, Ayis S, Gooberman-Hill R, Horwood J, et al. Complex interventions to improve physical function and maintain independent living in elderly people: a systematic review and meta-analysis. The Lancet. 2008;371(9614):725–35. doi: 10.1016/s0140-6736(08)60342-6 18313501

7. Van Durme T, Schmitz O, Cès S, Anthierens S, Maggi P, Delye S, et al. A comprehensive grid to evaluate case management’s expected effectiveness for community-dwelling frail older people: results from a multiple, embedded case study. BMC geriatrics. 2015;15(1):67.

8. America CMSo. Definition of Case Management 2011 2011 [24/05/2018]. Available from: http://www.cmsa.org/PolicyMaker/ResourceKit/AboutCaseManagers/tabid/141/Default.aspx.

9. Köpke S, McCleery J. Systematic reviews of case management: too complex to manage? Cochrane Database of Systematic Reviews. 2015;(1).

10. Lukersmith S, Millington M, Salvador-Carulla L. What is case management? A scoping and mapping review. International journal of integrated care. 2016;16(4).

11. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj. 2008;337:a1655. doi: 10.1136/bmj.a1655 18824488

12. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. Bmj. 2000;321(7262):694–6. doi: 10.1136/bmj.321.7262.694 10987780

13. Vuik SI, Mayer E, Darzi A. Enhancing risk stratification for use in integrated care: a cluster analysis of high-risk patients in a retrospective cohort study. BMJ Open. 2016;6(12):e012903. Epub 2016/12/21. doi: 10.1136/bmjopen-2016-012903 27993905; PubMed Central PMCID: PMCPMC5168666.

14. Team/LTC NED. NHS England: Case Finding & Risk Stratification Handbook. 2015.

15. Lambert A-S, Ces S, Malembaka EB, Van Durme T, Declercq A, Macq J. Evaluation of bottom-up interventions targeting community-dwelling frail older people in Belgium: methodological challenges and lessons for future comparative effectiveness studies. BMC Health Services Research. 2019;19(1):416. doi: 10.1186/s12913-019-4240-9 31234857

16. Ross S, Curry N, Goodwin N. Case management: what it is and how it can best be implemented. London: King's Fund, 2011 1857176308.

17. Creswell JW. Research design: Qualitative, quantitative, and mixed methods approaches: Sage publications; 2013.

18. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? Bmj. 2004;328(7455):1561–3. doi: 10.1136/bmj.328.7455.1561 15217878

19. Mello JDA, Van Durme T, Macq J, Declercq A. Interventions to delay institutionalization of frail older persons: design of a longitudinal study in the home care setting. BMC Public Health. 2012;12(1):615.

20. Rolfson DB, Majumdar SR, Tsuyuki RT, Tahir A, Rockwood K. Validity and reliability of the Edmonton Frail Scale. Age and ageing. 2006;35(5):526–9. doi: 10.1093/ageing/afl041 16757522

21. De Lepeleire J, Iliffe S, Mann E, Degryse JM. Frailty: an emerging concept for general practice. Br J Gen Pract. 2009;59(562):e177–e82. doi: 10.3399/bjgp09X420653 19401013

22. Van Durme T, Schmitz O, Sophie C, Lambert A-S, Billings J, Anthierens S, et al. Why Is Case Management Effective? A Realist Evaluation of Case Management for Frail, Community-Dwelling Older People: Lessons Learned from Belgium. Open Journal of Nursing. 2016;6:863–80.

23. Yin RK. Case study research and applications: Design and methods: Sage publications; 2017.

24. George AL, Bennett A, Lynn-Jones SM, Miller SE. Case studies and theory development in the social sciences: mit Press; 2005.

25. Gerring J. Case study research: Principles and practices: Cambridge university press; 2006.

26. Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. The Milbank Quarterly. 1996:511–44. 8941260

27. Van Durme T, Schmitz O, Maggi P, Delye S, Gosset C, Lopez Hartmann M, et al. Scientific evaluation of projects of alternative forms of care or support of care for frail elderly, in order to allow them to maintain their autonomy and to live independently in their homes: “Protocol 3”—Contextual description of projects’ components and implementation analysis. Brussels: National Institute of Health and Disability Insurance, 2014.

28. Somme D, Trouve H, Dramé M, Gagnon D, Couturier Y, Saint-Jean O. Analysis of case management programs for patients with dementia: a systematic review. Alzheimer's & Dementia. 2012;8(5):426–36.

29. Powell SK, Tahan HM. Case management: A practical guide for education and practice: Lippincott Williams & Wilkins; 2018.

30. O'Rourke N, Tuokko HA. Psychometric properties of an abridged version of the Zarit Burden Interview within a representative Canadian caregiver sample. The Gerontologist. 2003;43(1):121–7. doi: 10.1093/geront/43.1.121 12604753

31. Nosikov A, Gudex C. EUROHIS: Developing common instruments for health surveys 2003.

32. Belgische Burgerlijk Wetboek Art. 487–488. and 489.

33. Beraadslaging nr. 10/028 van 20 April 2010 met betrekking tot de verwerking van persoonsgegevens die de gezondheid betreffen voor de evaluatie van projecten in het kader van het Protocolakkoord nr. 3.

34. Melzer D, McWilliams B, Brayne C, Johnson T, Bond J. Profile of disability in elderly people: estimates from a longitudinal population study. BMJ (Clinical research ed). 1999;318(7191):1108–11. doi: 10.1136/bmj.318.7191.1108 10213718

35. Van Durme T, Lopez Hartmann M, Anthierens S, Remmen R, Flusin D, Macq J. Scientific evaluation of the 25 projects of the second call–implementation analysis Brussels: National Insurance for Health and Disability Institute, 2017.

36. Le S, Josse J, Husson F. FactoMineR: An R Package for Multivariate Analysis. Journal of statistical software. 2008;25(1):1–18. doi: 10.18637/jss.v025.i01

37. Hastie T, Tibshirani R, Friedman J. Unsupervised Learning. In: Springer, editor. The Elements of Statistical Learning Data Mining, Inference, and Prediction. Second Edition ed2009. p. 485–586.

38. Pimouguet C, Bassi V, Somme D, Lavallart B, Helmer C, Dartigues JF. The 2008–2012 French Alzheimer Plan: A unique opportunity for improving integrated care for dementia. Journal of Alzheimer's Disease. 2013;34(1):307–14. doi: 10.3233/JAD-121648 23160009

39. Totten AM, Wagner J, Motu'apuaka M, Hickam DH, Guise J-M. Outpatient Case Management for Adults With Medical Illnesses and Complex Care Needs: Future Research Needs: Identification of Future Research Needs From Comparative Effectiveness Rockville (MD): Agency for Healthcare Research and Quality (US), 2013 13-EHC035-EF.

40. Morris JN, Fries BE, Morris SA. Scaling ADLs within the MDS. The Journals of Gerontology: Series A. 1999;54(11):M546–M53.

41. Hartmaier SL, Sloane PD, Guess HA, Koch GG, Mitchell CM, Phillips CD. Validation of the minimum data set cognitive performance scale: agreement with the mini-mental state examination. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 1995;50(2):M128–M33.

42. Burrows AB, Morris JN, Simon SE, Hirdes JP, Phillips C. Development of a minimum data set-based depression rating scale for use in nursing homes. Age and ageing. 2000;29(2):165–72. doi: 10.1093/ageing/29.2.165 10791452

43. Sekhon JS. Multivariate and propensity score matching software with automated balance optimization: the matching package for R. Journal of statistical software. 2011;42(7).

44. Lero DS, Heckman G, Poss J, Hirdes JP, Betini RS, Cadell S. A longitudinal study looking at and beyond care recipient health as a predictor of long term care home admission. BMC Health Services Research. 2017;17(1):709. doi: 10.1186/s12913-017-2671-8 29121916

45. Williams AP, Challis D, Deber R, Watkins J, Kuluski K, Lum JM, et al. Balancing institutional and community-based care: why some older persons can age successfully at home while others require residential long-term care. Healthcare quarterly. 2009;12(2).

46. Buyse M. Generalized pairwise comparisons of prioritized outcomes in the two‐sample problem. Statistics in medicine. 2010;29(30):3245–57. doi: 10.1002/sim.3923 21170918

47. Speer DC, Greenbaum PE. Five methods for computing significant individual client change and improvement rates: Support for an individual growth curve approach. Journal of consulting and clinical psychology. 1995;63(6):1044–8. doi: 10.1037//0022-006x.63.6.1044 8543708

48. Falissard B. Various procedures used in psychometry. measurements. 2005;20:37–46.

49. Buyse M. Reformulating the hazard ratio to enhance communication with clinical investigators. Clinical Trials. 2008;5(6):641. doi: 10.1177/1740774508098328 19029217

50. Potts JM, Elith J. Comparing species abundance models. Ecological Modelling. 2006;199(2):153–63.

51. Zuur A, Smith G, Savieliev A, Walker N, Ieno E. Zero-truncated and zero-inflated models for count data. In: Springer, editor. Mixed effects models and extensions in ecology with R. New York2009. p. 261–93.

52. Zeileis A, Kleiber C, Jackman S. Regression models for count data in R. Journal of statistical software. 2008;27(8):1–25.

53. Hothorn T, Zeileis A, Farebrother RW, Cummins C, Millo G, Mitchell D. Package ‘lmtest’ 2018 [24/05/2018]. Available from: https://cran.r-project.org/web/packages/lmtest/lmtest.pdf.

54. Canty A, Ripley B. Package 'boot'. 2017.

55. Fox J. Bootstrapping regression models. Appendix to An R and S-PLUS Companion to Applied Regression 2002 [24/05/2018]. Available from: https://socialsciences.mcmaster.ca/jfox/Books/Companion-1E/appendix-bootstrapping.pdf.

56. You EC, Dunt D, Doyle C, Hsueh A. Effects of case management in community aged care on client and carer outcomes: a systematic review of randomized trials and comparative observational studies. BMC Health Services Research. 2012;12(1):395.

57. Sandberg M, Kristensson J, Midlöv P, Jakobsson U. Effects on healthcare utilization of case management for frail older people: A randomized controlled trial (RCT). Archives of gerontology and geriatrics. 2015;60(1):71–81. doi: 10.1016/j.archger.2014.10.009 25459920

58. Hudson B. Sea change or quick fix? Policy on long‐term conditions in England. Health & social care in the community. 2005;13(4):378–85.

59. Brodaty H, Thomson C, Thompson C, Fine M. Why caregivers of people with dementia and memory loss don't use services. Int J Geriatr Psychiatry. 2005;20(6):537–46. Epub 2005/05/28. doi: 10.1002/gps.1322 15920707.

60. Sargent P, Pickard S, Sheaff R, Boaden R. Patient and carer perceptions of case management for long‐term conditions. Health & social care in the community. 2007;15(6):511–9.

61. Hutt R, Rosen R, McCauley J. Case managing long term conditions. London: King's Fund 2004.

62. Gravelle H, Dusheiko M, Sheaff R, Sargent P, Boaden R, Pickard S, et al. Impact of case management (Evercare) on frail elderly patients: controlled before and after analysis of quantitative outcome data. Bmj. 2007;334(7583):31. doi: 10.1136/bmj.39020.413310.55 17107984

63. Kumar GS, Klein R. Effectiveness of case management strategies in reducing emergency department visits in frequent user patient populations: a systematic review. The Journal of emergency medicine. 2013;44(3):717–29. doi: 10.1016/j.jemermed.2012.08.035 23200765

64. Grover CA, Crawford E, Close RJ. The Efficacy of Case Management on Emergency Department Frequent Users: An Eight-Year Observational Study. The Journal of emergency medicine. 2016;51(5):595–604. doi: 10.1016/j.jemermed.2016.06.002 27595372

65. Hickam DH, Weiss JW, Guise J-M, Buckley D, Motu'apuaka M, Graham E, et al. Outpatient case management for adults with medical illness and complex care needs. 2013.

66. Hawe P, Shiell A, Riley T, Gold L. Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. Journal of Epidemiology & Community Health. 2004;58(9):788–93.

67. Lambert A-S, Declercq A, De Almeida Mello J, Cès S, Macq J. Report on quantitative analysis. Brussels: National Insurance for Health and Disability Institute, 2017.

68. Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. Journal of Epidemiology & Community Health. 2002;56(2):119–27.

69. Gerkens S, Merkur S. Health systems in transition: Belgium. Health system review, Copenhagen. 2010.

70. Russell M, Roe B, Beech R, Russell W. Service developments for managing people with long-term conditions using case management approaches, an example from the UK. International journal of integrated care. 2009;9(1).

71. Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. Bmj. 2008;336(7656):1281–3. doi: 10.1136/bmj.39569.510521.AD 18535071

72. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. American journal of community psychology. 2009;43(3–4):267–76. doi: 10.1007/s10464-009-9229-9 19390961

73. Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health policy and planning. 2011;27(5):365–73. doi: 10.1093/heapol/czr054 21821667

Článek vyšel v časopise


2019 Číslo 10