Eye-gaze information input based on pupillary response to visual stimulus with luminance modulation

Autoři: Yumiko Muto aff001;  Hideka Miyoshi aff001;  Hirohiko Kaneko aff001
Působiště autorů: Dept. of Information and Communications Engineering, Tokyo institute of Technology, Yokohama, Kanagawa, Japan aff001;  Brain Science Institute, Tamagawa university, Tokyo, Japan aff002
Vyšlo v časopise: PLoS ONE 15(1)
Kategorie: Research Article
doi: https://doi.org/10.1371/journal.pone.0226991


This study develops an information-input interface in which a visual stimulus targeted by a user’s eye gaze is identified based on the pupillary light reflex to periodic luminance modulations of the object. Experiment 1 examines how pupil size changes in response to periodic luminance modulation of visual stimuli, and the results are used to develop an algorithm for information input. Experiment 2a examines the effectiveness of interfaces with two objects. The results demonstrate that 98% accurate identification of the gaze targeted object is possible if the luminance modulation frequencies of two objects differ by at least 0.12 Hz. Experiment 2b examines the accuracy of a gaze directed information input method based on a keyboard configuration with twelve responses. The results reveal that keyboard input is possible with an average accuracy of 85% for luminance modulation frequencies from 0.75 to 2.75 Hz. The proposed pupillometry based information-input interface offers several advantages, such as low burden on users, minimal invasiveness, no need for training or experience, high theoretical validity, and no need for calibration. Thus, the pupillometry method presented herein has advantages for practical use without requiring the eye’s position to be calibrated. Additionally, this method has a potential for the design of interfaces that allow patients with severely limited motor function to communicate with others.

Klíčová slova:

Eye movements – Eyes – Light – Luminance – Modulation – Pupil – Reflexes – Vision


1. Bouma H. Size of the Static Pupil as a Function of Wave-length and Luminosity of the Light Incident on the Human Eye. Nature. 1962;193:690. doi: 10.1038/193690a0 13871842

2. Crawford BH. The dependence of pupil size upon external light stimulus under static and variable conditions. Royal Society of London Proceedings Series B Biological Sciences. 1936;121(823):376–395. doi: 10.1098/rspb.1936.0072

3. Woodhouse JM, Campbell FW. The role of the pupil light reflex in aiding adaptation to the dark. Vision Research. 1975;15(6):649–653. https://doi.org/10.1016/0042-6989(75)90279-5 1138479

4. Woodhouse JM. The effect of pupil size on grating detection at various contrast levels. Vision Research. 1975;15(6):645–648. https://doi.org/10.1016/0042-6989(75)90278-3 1138478

5. Loewenfeld IE. Mechanisms of reflex dilatation of the pupil. Documenta Ophthalmologica. 1958;12(1):185–448. doi: 10.1007/bf00913471 13609524

6. Loewenfeld IE. The Pupil: Anatomy, Physiology and Clinical Applications. Iowa State University Press. 1993;.

7. Barbur JL. A Study of Pupil Response Components in Human Vision. In: Basic and Clinical Perspectives in Vision Research. Boston, MA: Springer US; 1995. p. 3–18. Available from: http://link.springer.com/10.1007/978-1-4757-9362-8_1.

8. Reeves P. The Response of the Average Pupil to Various Intensities of Light. Journal of the Optical Society of America. 1920;4(2):35–43. doi: 10.1364/JOSA.4.000035

9. Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology. 1988;70(6):510–523. https://doi.org/10.1016/0013-4694(88)90149-6 2461285

10. Fazel-Rezai R, Allison BZ, Guger C, Sellers EW, Kleih SC, Kübler A. P300 brain computer interface: current challenges and emerging trends. Frontiers in neuroengineering. 2012;5:14. doi: 10.3389/fneng.2012.00014 22822397

11. Stoll J, Chatelle C, Carter O, Koch C, Laureys S, Einhäuser W. Pupil responses allow communication in locked-in syndrome patients. Current Biology. 2013;23(15):R647–R648. doi: 10.1016/j.cub.2013.06.011 23928079

12. Mathôt S, Melmi JB, van der Linden L, Van der Stigchel S. The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry. PLOS ONE. 2016;11(2):e0148805. doi: 10.1371/journal.pone.0148805 26848745

13. Mathôt S, Van der Stigchel S. New Light on the Mind’s Eye: The Pupillary Light Response as Active Vision. Current Directions in Psychological Science. 2015;24(5):374–378. doi: 10.1177/0963721415593725 26494950

14. Binda P, Pereverzeva M, Murray SO. Attention to bright surfaces enhances the pupillary light reflex. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2013;33(5):2199–204. doi: 10.1523/JNEUROSCI.3440-12.2013

15. Mathôt S, van der Linden L, Grainger J, Vitu F. The pupillary light response reveals the focus of covert visual attention. PloS one. 2013;8(10). doi: 10.1371/journal.pone.0078168 24205144

16. Myers GA, Gannon JA, Stark LW. Level dependent signal flow in the light pupil reflex. Biological Cybernetics. 1993;68(3):235–240. doi: 10.1007/bf00224857 8452892

17. Barbur JL, Moro S, Harlow JA, Lam BL, Liu M. Comparison of pupil responses to luminance and colour in severe optic neuritis. Clinical Neurophysiology. 2004;115(11):2650–2658. https://doi.org/10.1016/j.clinph.2004.05.022 15465455

18. Moro S. A study of pupil response components in human vision. Unpublished Doctoral thesis, City University London. 2004;.

19. Silva B, Sfer A, D’Urso Villar MA, Issolio LA, Colombo EM. Pupil dynamics with periodic flashes: effect of age on mesopic adaptation. Journal of the Optical Society of America A. 2016;33(8):1546–1552. doi: 10.1364/JOSAA.33.001546

20. Zangemeister WH, Gronow T, Grzyska U. Pupillary responses to single and sinusoidal light stimuli in diabetic patients. Neurology international. 2009;1(1):e19–e19. doi: 10.4081/ni.2009.e19 21577356

21. Matsuda K, Nagami T, Sugase Y, Takemura A, Kawano K. A Widely Applicable Real-Time Mono/Binocular Eye Tracking System Using a High Frame-Rate Digital Camera. In: Kurosu M, editor. Human-Computer Interaction. User Interface Design, Development and Multimodality. Cham: Springer International Publishing; 2017. p. 593–608.

22. Alexandridis E, Manner M. [Frequency of the pupillary response following flicker stimuli (author’s transl)]. Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie Albrecht von Graefe’s archive for clinical and experimental ophthalmology. 1977;202(3):175–80. doi: 10.1007/bf00407866 301718

23. Press W H Teukolsky S A, Vetterling W T F BP. Numerical recipes in C: the art of scientific computing. Cambridge University Press. 1992;.

24. Kay SM, Marple SL. Spectrum analysis-A modern perspective. Proceedings of the IEEE. 1981;69(11):1380–1419. doi: 10.1109/PROC.1981.12184

25. Calcagnini G, Lino S, Censi F, Cerutti S. Cardiovascular autonomic rhythms in spontaneous pupil fluctuations. In: Computers in Cardiology 1997; 1997. p. 133–136.

26. Yoshida H, Yana K, Okuyama F, Tokoro T. Time-varying properties of respiratory fluctuations in pupil diameter of human eyes. Methods of information in medicine. 1994;33(1):46–8. doi: 10.1055/s-0038-1634990 8177078

27. Yoshida H, Mizuta H, Gouhara T, Suzuki Y, Yana K, Okuyama F. Statistical properties of simultaneously recorded fluctuations in pupil diameter and heart rate. In: Proceedings of 17th International Conference of the Engineering in Medicine and Biology Society. vol. 1; 1995. p. 165–166 vol.1.

28. Mathôt S, van der Linden L, Grainger J, Vitu F. The Pupillary Light Response Reveals the Focus of Covert Visual Attention. PLOS ONE. 2013;8(10):e78168. doi: 10.1371/journal.pone.0078168 24205144

29. Naber M, Alvarez G, Nakayama K. Tracking the allocation of attention using human pupillary oscillations. Frontiers in Psychology. 2013;4(919). doi: 10.3389/fpsyg.2013.00919 24368904

30. Partala T, Surakka V. Pupil size variation as an indication of affective processing. International Journal of Human Computer Studies. 2003;59(1-2):185–198. doi: 10.1016/S1071-5819(03)00017-X

31. Bradley MM, Miccoli L, Escrig MA, Lang PJ. The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology. 2008;45(4):602–607. doi: 10.1111/j.1469-8986.2008.00654.x 18282202

32. Oliva M, Anikin A. Pupil dilation reflects the time course of emotion recognition in human vocalizations. Scientific Reports. 2018;8(1). doi: 10.1038/s41598-018-23265-x

33. Van Orden KF, Limbert W, Makeig S, Jung TP. Eye activity correlates of workload during a visuospatial memory task. Human factors. 2001;43(1):111–21. doi: 10.1518/001872001775992570 11474756

34. Ahlstrom U, Friedman-Berg FJ. Using eye movement activity as a correlate of cognitive workload. International Journal of Industrial Ergonomics. 2006;36(7):623–636. doi: 10.1016/j.ergon.2006.04.002

35. Hasegawa S, Ishikawa S. [Age changes in pupillary light reflex. A demonstration by means of a pupillometer]. Nippon Ganka Gakkai zasshi. 1989;93(10):955–61. 2603847

36. Muto Y, Kaneko H. Proposal of “YES/NO” communication method based on frequency analysis of pupil response. Human Interface Symposium (HI2018), 25D. 2018;.

Článek vyšel v časopise


2020 Číslo 1
Nejčtenější tento týden