Towards text mining therapeutic change: A systematic review of text-based methods for Therapeutic Change Process Research

Autoři: Wouter Smink aff001;  Anneke M. Sools aff002;  Janneke M. van der Zwaan aff003;  Sytske Wiegersma aff002;  Bernard P. Veldkamp aff002;  Gerben J. Westerhof aff001
Působiště autorů: Department of Psychology, Health & Technology, University of Twente, Enschede, Overijssel, The Netherlands aff001;  Department of Research Methodology, Measurement & Data Analysis, University of Twente, Enschede, Overijssel, The Netherlands aff002;  Netherlands eScience Center (NWO), Amsterdam, Noord-Holland, The Netherlands aff003
Vyšlo v časopise: PLoS ONE 14(12)
Kategorie: Research Article


Therapeutic Change Process Research (TCPR) connects within-therapeutic change processes to outcomes. The labour intensity of qualitative methods limit their use to small scale studies. Automated text-analyses (e.g. text mining) provide means for analysing large scale text patterns. We aimed to provide an overview of the frequently used qualitative text-based TCPR methods and assess the extent to which these methods are reliable and valid, and have potential for automation. We systematically reviewed PsycINFO, Scopus, and Web of Science to identify articles concerning change processes and text or language. We evaluated the reliability and validity based on replicability, the availability of code books, training data and inter-rater reliability, and evaluated the potential for automation based on the example- and rule-based approach. From 318 articles we identified four often used methods: Innovative Moments Coding Scheme, the Narrative Process Coding Scheme, Assimilation of Problematic Experiences Scale, and Conversation Analysis. The reliability and validity of the first three is sufficient to hold promise for automation. While some text features (content, grammar) lend themselves for automation through a rule-based approach, it should be possible to automate higher order constructs (e.g. schemas) when sufficient annotated data for an example-based approach are available.

Klíčová slova:

Apes – Automation – Database searching – Emotions – Language – Psychotherapy – Research validity – Text mining


1. Owen J. Early career perspectives on psychotherapy research and practice: Psychotherapist effects, multicultural orientation, and couple interventions. Psychotherapy. 2013 Dec;50(4):496–502. doi: 10.1037/a0034617 24295457

2. Smink WAC, Fox JP, Tjong Kim Sang E, Sools AM, Westerhof GJ, Veldkamp BP. Understanding Therapeutic Change Process Research through Multilevel Modelling and Text Mining. Frontiers in psychology Frontiers in psychology. 2019 May;10:1186. doi: 10.3389/fpsyg.2019.01186 31191394

3. Imel ZE, Steyvers M, Atkins DC. Computational psychotherapy research: Scaling up the evaluation of patient-provider interactions. Psychotherapy. 2015 Mar;51(1):19–30. doi: 10.1037/a0036841

4. He Q. Text mining and IRT for psychiatric and psychological assessment. Enschede: University of Twente. 2013. 156.

5. Elliott R. Qualitative Methods for Studying Psychotherapy Change Processes. In Thompson AR, Harper D, editors. Qualitative Research Methods in Mental Health and Psychotherapy: A Guide for Students and Practitioners. Chichester: Wiley-Blackwells. 2013. 69–111.

6. Murphy S, Maskit B, Bucci W. Putting feelings into words: Cross-linguistic markers of the referential process. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 2015. 80–88.

7. Gelo OC, Salcuni S, Colli A Text analysis within quantitative and qualitative psychotherapy process research: introduction to special issue Research in Psychotherapy: Psychopathology, Process and Outcome. 2012 Mar;15(2):45–53.

8. Elliott R. Psychotherapy change process research: Realizing the promise Psychotherapy Research. 2010 Jan;20(2):123–135. doi: 10.1080/10503300903470743 20099202

9. Tanana M, Hallgren K, Imel Z, Atkins D, Smyth P, Srikumar V. Recursive neural networks for coding therapist and patient behavior in motivational interviewing. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 2015. 71–79.

10. Adler JM. Living into the story: Agency and coherence in a longitudinal study of narrative identity development and mental health over the course of psychotherapy. Journal of Personality and Social Psychology. 2012;102(2):367–389. doi: 10.1037/a0025289 21910554

11. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cognitive behaviour therapy. 2009 Dec;38(4):196–205. doi: 10.1080/16506070903318960 20183695

12. Atkins DC, Steyvers M, Imel ZE, Smyth P. Scaling up the evaluation of psychotherapy: evaluating motivational interviewing fidelity via statistical text classification. Implementation Science. 2014 Dec;9(1):49. doi: 10.1186/1748-5908-9-49 24758152

13. Howes C, Purver M, McCabe R. LLinguistic indicators of severity and progress in online text-based therapy for depression. In Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 2014 Jun:27. 7–16.

14. Bishop L. Ethical sharing and reuse of qualitative data. Australian Journal of Social Issues. 2009 Mar;44(3):255–272. doi: 10.1002/j.1839-4655.2009.tb00145.x

15. Bennett K, Bennett AJ, Griffiths KM. Security considerations for e-mental health interventions. Journal of Medical Internet Research. 2010 Dec;12(5):e61. doi: 10.2196/jmir.1468 21169173

16. Cariola LA. Semantic Expressions of the Body Boundary Personality in Person-centred Psychotherapy. International Body Psychotherapy Journal. 2015 Jan;14(1):48–64.

17. Mergenthaler E. Emotion–abstraction patterns in verbatim protocols: A new way of describing psychotherapeutic processes. Journal of consulting and clinical psychology. 1996 Dec;64(6):1306. doi: 10.1037//0022-006x.64.6.1306 8991317

18. Degenhardt L, Charlson F, Ferrari A, Santomauro D, Erskine H., Mantilla-Herrara A, Whiteford H, Leung J, Naghavi M, Griswold M, Rehm J, Hall W, Sartorius B, Scott J, Vollset SE, Knudsen AK, Haro JM, Patton G, Kopec J, Carvalho Malta D, Topor-Madry R, McGrath J, Haagsma J, Allebeck P, Phillips M, Salomon J, Hay S, Foreman K, Lim S, Mokdad A, Smith M, Gakidou E, Murray E, Vos T. The global burden of disease attributable to alcohol and drug use in 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. The Lancet Psychiatry. 2018 Dec 1;5(12):987–1012. doi: 10.1016/S2215-0366(18)30337-7

19. de Graaf R, Tuithof M, van Dorsselaer S, ten Have M. Verzuim door psychische en somatische aandoeningen bij werkenden. Resultaten van de ‘Netherlands Mental Health Survey and Incidence Study-2’(NEMESIS-2)[Absenteeism due to psychological or somatic disorders in workers. Results of the ‘Netherlands Mental Health Survey and Incidence Study-2’(NEMESIS-2)]. Utrecht: Trimbos-Instituut. 2011.

20. Garfield SL. Therapies—modern and popular: PsycCRITIQUES 2006. Washington, DC: American Psychological Association. 2006.

21. The effectiveness of psychotherapy. In Lambert MJ, Bergin AE, editors. Handbook of psychotherapy and behavior change. 1994;4:143–189.

22. Campbell LF, Norcross JC, Vasquez MJ, Kaslow NJ. Recognition of psychotherapy effectiveness: The APA resolution. Psychotherapy. 2013 Mar;50(1):98. doi: 10.1037/a0031817 23505985

23. Barkham M, Stiles WB, Shapiro DA. The shape of change in psychotherapy: Longitudinal assessment of personal problems. Journal of Consulting and Clinical Psychology. 1993 Aug;61(4):667–677. doi: 10.1037//0022-006x.61.4.667 8370863

24. Nock MK. Progress review of the psychosocial treatment of child conduct problems. Clinical Psychology: Science and Practice. 2003 Mar;10(1):1–28.

25. Kazdin AE, Nock MK. Delineating mechanisms of change in child and adolescent therapy: Methodological issues and research recommendations. Journal of Child Psychology and Psychiatry. 2003 Nov;44(8):1116–29. doi: 10.1111/1469-7610.00195 14626454

26. Norcross JC, Wampold BE. What works for whom: Tailoring psychotherapy to the person. Journal of Clinical Psychology. 2011 Feb;67(2):127–32. doi: 10.1002/jclp.20764 21108312

27. Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. Jama. 2007 Sep 12;298(10):1209–12. doi: 10.1001/jama.298.10.1209 17848656

28. Tasca GA, Sylvestre J, Balfour L, Chyurlia L, Evans J, Fortin-Langelier B, Francis K, Gandhi J, Huehn L, Hunsley J, Joyce AS, Kinley J, Koszycki D, Leszcz M, Lybanon-Daigle L, Mercer D, Ogrodniczuk JS, Presniak M, Ravitz P, Ritchie K, Talbot J, Wilson B. What clinicians want: Findings from a psychotherapy practice research network survey. Psychotherapy. 2015 Mar;52(1):1–11. doi: 10.1037/a0038252 25528356

29. Gelo OCG, Manzo S. Quantitative approaches to treatment process, change process, and process-outcome research. Chapter 13. In Gelo OCG, Pritz A, Rieken B, editors. Psychotherapy Research. 2015. Vienna: Springer. 247–277.

30. Greenberg LS. A guide to conducting a task analysis of psychotherapeutic change. Psychotherapy Research. 2007 Jan 1;17(1):15–30. doi: 10.1080/10503300600720390

31. Braakmann D. Historical Paths in Psychotherapy Research. Chapter 3. In Gelo OCG, Pritz A, Rieken B, editors. Psychotherapy research: Foundations, process, and outcome. 2015. Vienna: Springer. 39–65.

32. Hill CE, Corbett MM. A perspective on the history of process and outcome research in counseling psychology. Journal of Counseling Psychology. 1993 Jan;40(1):3.

33. Shapiro DA. Finding out how psychotherapies help people change. Psychotherapy Research. 1995 Jan 1;5(1):1–21. doi: 10.1080/10503309512331331106

34. Wallerstein RS. The generations of psychotherapy research: An overview. Psychoanalytic Psychology. 2001;18(2):243. doi: 10.1037/0736-9735.18.2.243

35. Mörtl K, Gelo OCG. Qualitative methods in psychotherapy process research. Chapter 20. In Gelo OCG, Pritz A, Rieken B, editors. Psychotherapy research: Foundations, process, and outcome. 2015. Vienna: Springer. 381–428.

36. Pennebaker JW, Boyd RL, Jordan K, Blackburn K. The Development and Psychometric Properties of LIWC2015. UT Fac Work. 2015.

37. Street RL Jr, Makoul G, Arora NK, Epstein RM. How does communication heal? Pathways linking clinician–patient communication to health outcomes. Patient education and counseling. 2009 Mar 1;74(3):295–301. doi: 10.1016/j.pec.2008.11.015

38. Arntz A, Hawke LD, Bamelis L, Spinhoven P, Molendijk ML. Changes in natural language use as an indicator of psychotherapeutic change in personality disorders. Behaviour research and therapy. 2012 Mar 1;50(3):191–202. doi: 10.1016/j.brat.2011.12.007 22317755

39. Wynn R, Wynn M. Empathy as an interactionally achieved phenomenon in psychotherapy: Characteristics of some conversational resources. Journal of Pragmatics. 2006 Sep 1;38(9):1385–97. doi: 10.1016/j.pragma.2005.09.008

40. Salvatore S, Gennaro A, Auletta AF, Tonti M, Nitti M. Automated method of content analysis: A device for psychotherapy process research. Psychotherapy Research. 2012 May 1;22(3):256–73. doi: 10.1080/10503307.2011.647930 22242553

41. Barak A, Hen L, Boniel-Nissim M, Shapira NA. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. Journal of Technology in Human services. 2008 Jul 3;26(2-4):109–60. doi: 10.1080/15228830802094429

42. Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. Journal of medical Internet research. 2005;7(1):e1. doi: 10.2196/jmir.7.1.e1 15829471

43. Lamers SM, Bohlmeijer ET, Korte J, Westerhof GJ. The efficacy of life-review as online-guided self-help for adults: A randomized trial. Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 2014 Apr 1;70(1):24–34. doi: 10.1093/geronb/gbu030

44. Amichai-Hamburger Y, Klomek AB, Friedman D, Zuckerman O, Shani-Sherman T. The future of online therapy. Computers in Human Behavior. 2014 Dec 1;41:288–94. doi: 10.1016/j.chb.2014.09.016

45. Chung C, Pennebaker JW. The psychological functions of function words. Chapter 13. In Fiedler K, editor. Social communication. New York: Psychology Press. 2007 Sep;1:343–59.

46. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PloS one. 2010 Oct 13;5(10):e13196. doi: 10.1371/journal.pone.0013196 20967242

47. Hoogendoorn M, Berger T, Schulz A, Stolz T, Szolovits P. Predicting social anxiety treatment outcome based on therapeutic email conversations. IEEE journal of biomedical and health informatics. 2016 Aug 17;21(5):1449–59. doi: 10.1109/JBHI.2016.2601123 27542187

48. Abbe A, Grouin C, Zweigenbaum P, Falissard B. Text mining applications in psychiatry: a systematic literature review. International journal of methods in psychiatric research. 2016 Jun;25(2):86–100. doi: 10.1002/mpr.1481 26184780

49. Snow R, O’Connor B, Jurafsky D, Ng AY. Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the conference on empirical methods in natural language processing 2008 Oct 25 (pp. 254-263). Association for Computational Linguistics.

50. Basit T. Manual or electronic? The role of coding in qualitative data analysis. Educational research. 2003 Jun 1;45(2):143–54. doi: 10.1080/0013188032000133548

51. Sools AM, Smink WAC, van der Zwaan JM, Schuffelen PCJ, Tjong Kim Sang E, de Vries BL, Veldkamp BP, Westerhof GJ. Text Mining Research of Psychotherapeutic Processes: A State-of-the-Art Review. Manuscript under review.

52. Jurafsky D, Martin JH. Edition 2. Chapter 1. In Speech and Language Processing. Essex: Pearson Education. 2014. 9–14.

53. Feldman R, Sanger J. The text mining handbook: advanced approaches in analyzing unstructured data. Cambridge university press; 2007.

54. Jurafsky D, Martin JH. Edition 2. Speech and Language Processing. Essex: Pearson Education. 2014. 988.

55. Manning CD, Manning CD, Schütze H. Foundations of statistical natural language processing. MIT press; 1999.

56. Bird S, Klein E, Loper E. Natural language processing with Python: analyzing text with the natural language toolkit. “O’Reilly Media, Inc.”; 2009 Jun 12.

57. Pennebaker JW, Mehl MR, Niederhoffer KG. Psychological aspects of natural language use: Our words, our selves. Annual review of psychology. 2003 Feb;54(1):547–77. doi: 10.1146/annurev.psych.54.101601.145041 12185209

58. Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. Journal of machine Learning research. 2003;3(Jan):993–1022.

59. Atkins DC, Rubin TN, Steyvers M, Doeden MA, Baucom BR, Christensen A. Topic models: A novel method for modeling couple and family text data. Journal of family psychology. 2012 Oct;26(5):816. doi: 10.1037/a0029607 22888778

60. Pfäfflin F, Böhmer M, Cornehl S, Mergenthaler E. What happens in therapy with sexual offenders? A model of process research. Sexual Abuse. 2005 Apr;17(2):141–51. doi: 10.1177/107906320501700205 15974421

61. Anderson T, Bein E, Pinnell B, Strupp H. Linguistic analysis of affective speech in psychotherapy: A case grammar approach. Psychotherapy research. 1999 Jan 1;9(1):88–99.

62. Johnson M. How the statistical revolution changes (computational) linguistics. InProceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous; 2009 Mar 30 (pp. 3–11). Association for Computational Linguistics.

63. Mykowiecka A, Marciniak M, Kupść A. Rule-based information extraction from patients’ clinical data. Journal of biomedical informatics. 2009 Oct 1;42(5):923–36. doi: 10.1016/j.jbi.2009.07.007 19646551

64. Brzozowski JA. Derivatives of regular expressions. Journal of the ACM 1964. doi: 10.1145/321239.321249

65. McNaughton R, Yamada H. Regular expressions and state graphs for automata. IRE transactions on Electronic Computers. 1960 Mar(1):39–47. doi: 10.1109/TEC.1960.5221603

66. Althoff T, Clark K, Leskovec J. Large-scale analysis of counseling conversations: An application of natural language processing to mental health. Transactions of the Association for Computational Linguistics. 2016 Dec;4:463–76. doi: 10.1162/tacl_a_00111 28344978

67. Gallo C, Pantin H, Villamar J, Prado G, Tapia M, Ogihara M, Cruden G, Brown CH. Blending qualitative and computational linguistics methods for fidelity assessment: experience with the Familias Unidas preventive intervention. Administration and Policy in Mental Health and Mental Health Services Research. 2015 Sep 1;42(5):574–85. doi: 10.1007/s10488-014-0538-4 24500022

68. Halfon S, Fişek G, Çavdar A. An empirical study of verb use as indicator of emotional access in therapeutic discourse. Psychoanalytic Psychology. 2017 Jan;34(1):35. doi: 10.1037/pap0000081

69. Martinez A, Martinez W. At the interface of computational linguistics and statistics. Wiley Interdisciplinary Reviews: Computational Statistics. 2015 Jul;7(4):258–74. doi: 10.1002/wics.1353

70. Liberman M. Emotional Prosody Speech and Transcripts. The Linguistic Data Consortium. 2002.

71. Sofaer HR, Hoeting JA, Jarnevich CS. The area under the precision-recall curve as a performance metric for rare binary events. Methods in Ecology and Evolution. 2019; 10(4): 565–577. doi: 10.1111/2041-210X.13140

72. Bates M. Models of natural language understanding. Proceedings of the National Academy of Sciences. 1995; 92(22):9977–9982. doi: 10.1073/pnas.92.22.9977

73. Xiao B, Can D, Gibson J, Imel ZE, Atkins DC, Georgiou PG, Narayanan SS. Behavioral Coding of Therapist Language in Addiction Counseling Using Recurrent Neural Networks. In Interspeech. 2016 Sep. 908–912. doi: 10.21437/Interspeech.2016-1560

74. Banko M, Brill E. Scaling to Very Very Large Corpora for Natural Language Disambiguation. In Proceedings of 39th Annual Meeting of the Association for Computational Linguistics. 2001. 26–33.

75. Smaling A. Inductive, analogical, and communicative generalization. International Journal of Qualitative Methods. 2003;2(1). 52–67. doi: 10.1177/160940690300200105

76. Stiles WB, Elliott R, Llewelyn SP, Firth-Cozens JA, Margison FR, Shapiro DA, et al. Assimilation of problematic experiences by clients in psychotherapy. Psychotherapy. 1990;27(3):411–420. doi: 10.1037/0033-3204.27.3.411

77. Stiles WB, Morrison LA, Harper H, Shapiro DA, Haw SK, Firth-Cozens JA, et al. Longitudinal study of assimilation in exploratory psychotherapy. Psychotherapy. 1991;28(2):195–206. doi: 10.1037/0033-3204.28.2.195

78. Gonçalves MM, Ribeiro AP, Matos M, Santos A, Mendes I. Innovative moments coding system: a methodological procedure for tracking changes in psychotherapy. YIS Yearb idiographic Sci. 2010;2:107–130.

79. Gonçalves MM, Ribeiro AP, Mendes I, Matos M, Santos A. Tracking novelties in psychotherapy process research: The innovative moments coding system. Psychother Res. 2011;21(5):497–509. doi: 10.1080/10503307.2011.560207 21480054

80. Angus LE, Hardtke K, Levitt H. Narrative Processes Coding System training manual. Unpublished manuscript, York University, Toronto, Ontario, Canada. Unpublished manuscript, Angus Narrative Lab, York University, Toronto, Ontario, Canada; 1996.

81. Voutilainen L, Peräkylä A, Ruusuvuori J. Therapeutic change in interaction: Conversation analysis of a transforming sequence. Psychother Res. 2011;21(3):348–65. doi: 10.1080/10503307.2011.573509 21623552

82. Peräkylä A. Conversation analysis in psychotherapy. In: Sidnell J, Stivers T, editors. The Handbook of Conversation Analysis. Oxford, England: Wiley-Blackwell; 2012. p. 551–74.

83. Angus LE, Greenberg LS. Working with narrative in emotion-focused therapy: Changing stories, healing lives. Washington: American Psychological Association; 2011.

84. Angus LE, McLeod J. The handbook of narrative and psychotherapy: practice, theory, and research. New York:Sage Publications; 2004. 404 p.

85. Stivers T. Coding social interaction: A heretical approach in conversation analysis?. Res Lang and Soc Interact. 2015; 48(1), 1–19. doi: 10.1080/08351813.2015.993837

Článek vyšel v časopise


2019 Číslo 12
Nejčtenější tento týden