Cooperation with autonomous machines through culture and emotion

Autoři: Celso M. de Melo aff001;  Kazunori Terada aff002
Působiště autorů: CCDC U.S. Army Research Laboratory, Playa Vista, CA, United States of America aff001;  Gifu University, Gifu, Yanagido, Japan aff002
Vyšlo v časopise: PLoS ONE 14(11)
Kategorie: Research Article
doi: 10.1371/journal.pone.0224758


As machines that act autonomously on behalf of others–e.g., robots–become integral to society, it is critical we understand the impact on human decision-making. Here we show that people readily engage in social categorization distinguishing humans (“us”) from machines (“them”), which leads to reduced cooperation with machines. However, we show that a simple cultural cue–the ethnicity of the machine’s virtual face–mitigated this bias for participants from two distinct cultures (Japan and United States). We further show that situational cues of affiliative intent–namely, expressions of emotion–overrode expectations of coalition alliances from social categories: When machines were from a different culture, participants showed the usual bias when competitive emotion was shown (e.g., joy following exploitation); in contrast, participants cooperated just as much with humans as machines that expressed cooperative emotion (e.g., joy following cooperation). These findings reveal a path for increasing cooperation in society through autonomous machines.

Klíčová slova:

Behavior – Culture – Decision making – Emotions – Ethnicities – Social communication – Social research – Prisoner's dilemma


1. Tajfel H, Turner J (1986) The social identity theory of intergroup behavior. In: Worchel S, Austin W, editors. Psychology of intergroup relations. Nelson-Hall; pp. 7–24.

2. Brewer M (1979) In-group bias in the minimal intergroup situation: A cognitive-motivational analysis. Psychol Bull 86: 307–324.

3. Crisp R, Hewstone M (2007) Multiple social categorization. Adv Exp Soc Psychol 39: 163–254.

4. Baillet D, Wu J, De Dreu C (2014) Ingroup favoritism in cooperation: A meta-analysis. Psychol Bull 140: 1556–1581. doi: 10.1037/a0037737 25222635

5. de Melo C, Marsella S, Gratch J (2019) Human cooperation when acting through autonomous machines. Proc Nat Acad Sci U.S.A. 116: 3482–3487.

6. Bonnefon J-F, Shariff A, Rahwan I (2016) The social dilemma of autonomous vehicles. Science 352: 1573–1576. doi: 10.1126/science.aaf2654 27339987

7. Stone R, Lavine M (2014) The social life of robots. Science 346: 178–179. doi: 10.1126/science.346.6206.178 25301612

8. Richerson P, Newton E, Baldini R, Naar N, Bell A, Newson L et al. (2016) Cultural group selection plays an essential role in explaining human cooperation: A sketch of the evidence. Behav Brain Sci 39, 1–68.

9. Betancourt H, Lopez S (1993) The study of culture, ethnicity, and race in American Psychology. Am Psychol 48: 629–637.

10. Kurzban R, Tooby J, Cosmides L (2001) Can race be erased? Coalitional computation and social categorization. Proc Nat Acad Sci U.S.A. 98: 15387–15392.

11. Pietraszewski D, Cosmides L, Tooby J (2014) The content of our cooperation, not the color of our skin: An alliance detection system regulates categorization by coalition and race, but not sex. PLOS ONE 9, e88534. doi: 10.1371/journal.pone.0088534 24520394

12. Bernhard H, Fischbacher U, Fehr E (2006) Parochial altruism in humans. Nature 442: 912–915. doi: 10.1038/nature04981 16929297

13. Kollock P (1998) Social dilemmas: The anatomy of cooperation. Annu Rev Sociol 24: 183–214.

14. Nass C, Moon Y, Green N (1997) Are computers gender-neutral? Gender stereotypic responses to computers. J App Soc Psychol 27: 864–876.

15. Nass C, Isbister K, Lee E-J (2000) Truth is beauty: Researching embodied conversational agents. In: Cassell J, editor. Embodied conversational agents. MIT Press; pp. 374–402.

16. Khooshabeh P, Dehghani M, Nazarian A, Gratch J (2017) The cultural influence model: When accented natural language spoken by virtual characters matters. AI & Soc 32: 9–16.

17. Reeves B, Nass C (1996) The media equation: How people treat computers, television, and new media like real people and places, Cambridge University Press.

18. de Melo C, Marsella S, Gratch J (2016) People do not feel guilty about exploiting machines. ACM T Comput-Hum Int 23: 1–17.

19. Gallagher H, Anthony J, Roepstorff A, Frith C (2002) Imaging the intentional stance in a competitive game. NeuroImage 16: 814–821. 12169265

20. McCabe K, Houser D, Ryan L, Smith V, Trouard T (2001) A functional imaging study of cooperation in two-person reciprocal exchange. Proc Nat Acad Sci U.S.A. 98: 11832–11835.

21. Kircher T, Blümel I, Marjoram D, Lataster T, Krabbendam L, Weber J et al. (2009) Online mentalising investigated with functional MRI. Neurosci Lett 454: 176–181. doi: 10.1016/j.neulet.2009.03.026 19429079

22. Krach S, Hegel F, Wrede B, Sagerer G, Binkofski F, Kircher T (2008) Can machines think? Interaction and perspective taking with robots investigated via fMRI. PLOS ONE 3: 1–11.

23. Rilling J, Gutman D, Zeh T, Pagnoni G, Berns G, Kilts C (2002) A neural basis for social cooperation. Neuron 35: 395–405. doi: 10.1016/s0896-6273(02)00755-9 12160756

24. Sanfey A, Rilling J, Aronson J, Nystrom L, Cohen J (2003) The neural basis of economic decision-making in the ultimatum game. Science 300: 1755–1758. doi: 10.1126/science.1082976 12805551

25. Waytz A, Gray K, Epley N, Wegner D (2010) Causes and consequences of mind perception. Trends Cogn Sci 14: 383–388. doi: 10.1016/j.tics.2010.05.006 20579932

26. Haslam N (2006) Dehumanization: An integrative review. Pers Soc Psychol Rev 10: 252–264. doi: 10.1207/s15327957pspr1003_4 16859440

27. Terada K, Takeuchi C (2017) Emotional expression in simple line drawings of a robot’s face leads to higher offers in the ultimatum game. Front. Psychol. 8: doi: 10.3389/fpsyg.2017.00724 28588520

28. Rau P, Li Y, Li D (2009) Effects of communication style and culture on ability to accept recommendations from robots. Comp Hum Behav 25: 587–595.

29. Aylett R, Paiva A (2012) Computational modelling of culture and affect. Emotion Rev 4: 253–263.

30. Brett J, Okomura T (1998) Inter- and intracultural negotiations: US and Japanese negotiators. Acad Manag J 41: 495–510.

31. Henrich J, Boyd R, Bowles S, Camerer C, Fehr E et al. (2000) In search of homo economicus: behavioral experiments in 15 small-scale societies. Am Econ Rev 91: 73–78.

32. Efferson C, Lalive R, Fehr E (2008) The coevolution of cultural groups and ingroup favoritism. Science 321: 1844–1849. doi: 10.1126/science.1155805 18818361

33. Damasio A (1994) Descartes' error: Emotion, reason, and the human brain, Putnam Press.

34. Van Kleef G, De Dreu C, Manstead A (2010) An interpersonal approach to emotion in social decision making: The emotions as social information model. Adv Exp Soc Psychol 42: 45–96.

35. Morris M, Keltner D (2000) How emotions work: An analysis of the social functions of emotional expression in negotiations. Res Organ Behav 22: 1–50.

36. Frank R (2004) Introducing moral emotions into models of rational choice. In: Manstead A, Frijda N, Fischer A, editors. Feelings and emotions. Cambridge University Press; pp. 422–440.

37. de Melo C, Carnevale P, Read S, Gratch J (2014) Reading people's minds from emotion expressions in interdependent decision making. J Pers Soc Psychol 106: 73–88. doi: 10.1037/a0034251 24079297

38. Marsella S, Gratch J, Petta P (2010) Computational models of emotion. In: Scherer K, Bänziger T, Roesch E, editors. A blueprint for an affectively competent agent: Cross-fertilization between emotion psychology, affective neuroscience, and affective computing. Oxford University Press; pp. 21–45.

39. Dovidio J, Gaertner S (2000). Aversive racism and selection decisions: 1989 and 1999. Psychol Sci 11: 319–323.

40. Axt J, Ebersole C, Nosek B (2016) An unintentional, robust, and replicable pro-Black bias in social judgment. Soc Cogn 34: 1–39.

41. Goodman B, Flaxman S (2017) European Union regulations on algorithmic decision making and a “Right to Explanation”. AI Magazine Fall 2017: 51–57.

42. Paolacci G, Chandler J, Ipeirotis P (2010) Running experiments on Amazon Mechanical Turk. Judg Decis Making 5: 411–419.

Článek vyšel v časopise


2019 Číslo 11