Assessing the convergent validity between the automated emotion recognition software Noldus FaceReader 7 and Facial Action Coding System Scoring

Autoři: Tanja Skiendziel aff001;  Andreas G. Rösch aff001;  Oliver C. Schultheiss aff001
Působiště autorů: Department of Psychology, Friedrich-Alexander University, Erlangen, Germany aff001
Vyšlo v časopise: PLoS ONE 14(10)
Kategorie: Research Article
doi: 10.1371/journal.pone.0223905


This study validates automated emotion and action unit (AU) coding applying FaceReader 7 to a dataset of standardized facial expressions of six basic emotions (Standardized and Motivated Facial Expressions of Emotion). Percentages of correctly and falsely classified expressions are reported. The validity of coding AUs is provided by correlations between the automated analysis and manual Facial Action Coding System (FACS) scoring for 20 AUs. On average 80% of the emotional facial expressions are correctly classified. The overall validity of coding AUs is moderate with the highest validity indicators for AUs 1, 5, 9, 17 and 27. These results are compared to the performance of FaceReader 6 in previous research, with our results yielding comparable validity coefficients. Practical implications and limitations of the automated method are discussed.

Klíčová slova:

Coding mechanisms – Emotions – Face – Face recognition – Fear – Happiness – Lips – Research validity


1. Izard CE. The Many Meanings/Aspects of Emotion: Definitions, Functions, Activation, and Regulation. Emotion Review. 2010;2(4):363–70. doi: 10.1177/1754073910374661

2. Cohn JF, Ambadar Z, Ekman P. Observer-based measurement of facial expression with the Facial Action Coding System. In: Coan JA, Allen JJB, editors. Handbook of emotion elicitation and assessment. New York, NY: Oxford University Press; 2007. p. 203–21.

3. Harrigan J, Rosenthal R, Scherer K. Introduction. In: Harrigan J, Rosenthal R, Scherer K, editors. The New Handbook of Methods in Nonverbal Behavior Research. New York, NY: Oxford University Press; 2005. p. 1–8.

4. Ekman P, Friesen WV, Hager JC. Facial action coding system. The manual on CD ROM. Salt Lake City, UT: Research Nexus; 2002.

5. Bartlett MS, Hager JC, Ekman P, Sejnowski TJ. Measuring facial expressions by computer image analysis. Psychophysiology. 1999;36(2):253–63. Epub 03/01. undefined. 10194972

6. Lewinski P, den Uyl TM, Butler C. Automated facial coding: Validation of basic emotions and FACS AUs in FaceReader. Journal of Neuroscience, Psychology, and Economics. 2014;7(4):227–36. doi: 10.1037/npe0000028 10.1037/npe0000028.supp (Supplemental). 2014-52742-003.

7. Cohn JF, Ekman P. Measuring facial action. In: Harrigan J, Rosenthal R, Scherer K, editors. The New Handbook of Methods in Nonverbal Behavior Research. New York, NY: Oxford University Press; 2005. p. 9–64.

8. Noldus. FaceReader™ 7: Tool for automatic analysis of facial expressions. Wageningen, The Netherlands: Noldus Information Technology; 2016.

9. Ekman P. An argument for basic emotions. Cognition and Emotion. 1992;6(3–4):169–200. doi: 10.1080/02699939208411068

10. Ekman P. Basic emotions. In: Dalgleish T, Power M, editors. Handbook of cognition and emotion. Chichester: Wiley; 1999. p. 45–60.

11. Ekman P. All emotions are basic. In: Ekman P, Davidson RJ, editors. The nature of emotion: Fundamental questions. New York, NY: Oxford University Press; 1994. p. 15–9.

12. Ekman P, Cordaro D. What is Meant by Calling Emotions Basic. 2011;3(4):364–70. doi: 10.1177/1754073911410740

13. Ekman P. Universal facial expressions of emotion. California Mental Health Research Digest. 1970;8(4):151–8. 1972-06605-001.

14. Ekman P. Facial expression and emotion. Am Psychol. 1993;48(4):384–92. doi: 10.1037//0003-066x.48.4.384 1993-32252-001. 8512154

15. Alvarado N, Jameson K. New findings on the contempt expression. Cognition & Emotion. 1996;10:379–407.

16. Sayette MA, Cohn JF, Wertz JM, Perrott MA, Parrott DJJJoNB. A Psychometric Evaluation of the Facial Action Coding System for Assessing Spontaneous Expression. 2001;25(3):167–85. doi: 10.1023/a:1010671109788

17. Ekman P, Friesen WV. Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press; 1978.

18. Rosenberg E. Is FACS Training right for you? without year [cited 2019 March 19]. Available from:

19. Kapoor A, Qi Y, Picard RW, editors. Fully automatic upper facial action recognition. 2003 IEEE International SOI Conference Proceedings (Cat No03CH37443); 2003 17–17 Oct. 2003.

20. Tian Y, Kanade T, Cohn JF. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2001;23(2):97–115. doi: 10.1109/34.908962 25210210

21. Inc. A. Affdex. Boston, MA: Affectiva Inc.; 2009.

22. Inc. ET. EmoVu. Palo Alto, CA: Eyeris Technologies Inc.; 2013.

23. SA N. nViso. Lausanne, Switzerland: NVISO SA; 2019.

24. Campbell DT, Fiske DW. Covergent and discriminant validation by the multitrait-multimethod matrix. Psych Bull. 1959;56:81–105.

25. Bijlstra G, Dotsch R. FaceReader 4 emotion classification performance on images from the Radboud Faces Database. [Unpublished manuscript]. In press 2011.

26. Danner L, Sidorkina L, Joechl M, Duerrschmid K. Make a face! Implicit and explicit measurement of facial expressions elicited by orange juices using face reading technology. Food Quality and Preference. 2014;32:167–72.

27. Gallant NL, Hadjistavropoulos T. Experiencing Pain in the Presence of Others: A Structured Experimental Investigation of Older Adults. The Journal of Pain. 2017;18(4):456–67. doi: 10.1016/j.jpain.2016.12.009 28062310

28. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994;6:284–90.

29. Rösch AG. The influence of implicit motives on the expression and perception of facial expressions of emotion [Der Einfluss impliziter Motive auf den Ausdruck und die Wahrnehmung emotionaler Gesichtsausdrücke] [Dissertation]. Erlangen, Germany: Friedrich-Alexander University; 2012.

30. Matsumoto D, Ekman P. Japanese and Caucasian facial expressions of emotion (JACFEE) and neutral faces (JACNeuF) [Slides]. San Francisco: Author; 1988.

31. Sporer SL. Recognizing faces of other ethnic groups: An integration of theories. Psychology, Public Policy, and Law. 2001;7(1):36–97. doi: 10.1037/1076-8971.7.1.36 2001-14540-002.

32. Nelson NL, Russell JA. Universality Revisited. 2013;5(1):8–15. doi: 10.1177/1754073912457227

33. Roy-Charland A, Perron M, Beaudry O, Eady K. Confusion of fear and surprise: A test of the perceptual-attentional limitation hypothesis with eye movement monitoring. Cognition and Emotion. 2014;28(7):1214–22. doi: 10.1080/02699931.2013.878687 24460373

34. Cacioppo JT, Petty RE, Losch ME, Kim HS. Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. J Pers Soc Psychol. 1986;50(2):260–8. doi: 10.1037//0022-3514.50.2.260 3701577

35. Larsen JT, Norris CJ, Cacioppo JT. Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology. 2003;40(5):776–85. Epub 2003/12/31. 14696731.

Článek vyšel v časopise


2019 Číslo 10