Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing
Autoři:
Sang-Ah Yoo aff001; John K. Tsotsos aff002; Mazyar Fallah aff001
Působiště autorů:
Department of Psychology, York University, Toronto, ON, Canada
aff001; Centre for Vision Research, York University, Toronto, ON, Canada
aff002; Active and Attentive Vision Laboratory, Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada
aff003; Visual Perception and Attention Laboratory, School of Kinesiology and Health Science, York University, Toronto, ON, Canada
aff004
Vyšlo v časopise:
PLoS ONE 14(9)
Kategorie:
Research Article
doi:
https://doi.org/10.1371/journal.pone.0223166
Souhrn
It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing.
Klíčová slova:
Animal performance – Ellipses – Flowers – Neurons – Reaction time – Vision – Visual system – Neuronal tuning
Zdroje
1. Fabre-Thorpe M, Delorme A, Marlot C, Thorpe S. A limit to the speed of processing in ultra-rapid visual categorization of novel natural scenes. J Cogn Neurosci. 2001;13(2):171–80. 11244543
2. Keysers C, Xiao DK, Földiák P, Perrett DI. The speed of sight. J Cogn Neurosci. 2001;13(1):90–101. 11224911
3. Potter MC. Short-term conceptual memory for pictures. J Exp Psychol Hum Learn. 1976;2(5):509–22. 1003124
4. Potter MC, Wyble B, Hagmann CE, McCourt ES. Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, Psychophys. 2014;76(2):270–9.
5. Rousselet GA, Macé MJ-M, Fabre-Thorpe M. Is it an animal? Is it a human face? Fast processing in upright and inverted natural scenes. J Vis. 2003 Jul 31;3(6):440–55. doi: 10.1167/3.6.5 12901715
6. Thorpe S, Fize D, Marlot C. Speed of processing in the human visual system. Nature. 1996;381(6582):520–2. doi: 10.1038/381520a0 8632824
7. VanRullen R, Koch C. Visual selective behavior can be triggered by a feed-forward process. J Cogn Neurosci. 2003;15(2):209–17. doi: 10.1162/089892903321208141 12676058
8. VanRullen R, Thorpe SJ. Is it a bird? Is it a plane? Ultra-rapid visual categorisation of natural and artifactual objects. Perception. 2001;30(6):655–68. doi: 10.1068/p3029 11464555
9. Grill-Spector K, Kanwisher N. Visual recognition. Psychol Sci. 2005 Feb 6;16(2):152–60. doi: 10.1111/j.0956-7976.2005.00796.x 15686582
10. Delorme A, Richard G, Fabre-Thorpe M. Ultra-rapid categorisation of natural scenes does not rely on colour cues: A study in monkeys and humans. Vision Res. 2000;40(16):2187–200. doi: 10.1016/s0042-6989(00)00083-3 10878280
11. Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci. 1999 Nov;2(11):1019–25. doi: 10.1038/14819 10526343
12. Serre T, Oliva A, Poggio T. A feedforward architecture accounts for rapid categorization. Proc Natl Acad Sci. 2007;104(15):6424–9. doi: 10.1073/pnas.0700622104 17404214
13. Rousselet GA, Fabre-Thorpe M, Thorpe SJ. Parallel processing in high-level categorization of natural images. Nat Neurosci. 2002;5(7):629–30. doi: 10.1038/nn866 12032544
14. Hung CP, Kreiman G, Poggio T, DiCarlo JJ. Fast readout of object identity from macaque inferior temporal cortex. Science. 2005;310(5749):863–6. doi: 10.1126/science.1117593 16272124
15. Isik L, Meyers EM, Leibo JZ, Poggio T. The dynamics of invariant object recognition in the human visual system. J Neurophysiol. 2014;111(1):91–102. doi: 10.1152/jn.00394.2013 24089402
16. Liu H, Agam Y, Madsen JR, Kreiman G. Timing, timing, timing: Fast decoding of object information from intracranial field potentials in human visual cortex. Neuron. 2009;62(2):281–90. doi: 10.1016/j.neuron.2009.02.025 19409272
17. Kay KN, Winawer J, Mezer A, Wandell BA. Compressive spatial summation in human visual cortex. J Neurophysiol. 2013;110(2):481–94. doi: 10.1152/jn.00105.2013 23615546
18. Salin P-A, Bullier J. Corticocortical connections in the visual system: Structure and function. Physiol Rev. 1995;75(1):107–54. doi: 10.1152/physrev.1995.75.1.107 7831395
19. Tsotsos JK. A “complexity level” analysis of immediate vision. Int J Comput Vis. 1988;1(4):303–320.
20. Tsotsos JK, Culhane SM, Kei Wai WY, Lai Y, Davis N, Nuflo F. Modeling visual attention via selective tuning. Artif Intell. 1995;78(1–2):507–45.
21. Carlson T, Hogendoorn H, Fonteijn H, Verstraten FAJ. Spatial coding and invariance in object-selective cortex. Cortex. 2011;47(1):14–22. doi: 10.1016/j.cortex.2009.08.015 19833329
22. Cichy RM, Chen Y, Haynes JD. Encoding the identity and location of objects in human LOC. Neuroimage. 2011;54(3):2297–307. doi: 10.1016/j.neuroimage.2010.09.044 20869451
23. MacEvoy SP, Epstein RA. Position selectivity in scene- and object-responsive occipitotemporal regions. J Neurophysiol. 2007;98(4):2089–98. doi: 10.1152/jn.00438.2007 17652421
24. DiCarlo JJ, Maunsell JHR. Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position. J Neurophysiol. 2003;89(6):3264–78. doi: 10.1152/jn.00358.2002 12783959
25. Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R. Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron. 1999;24(1):187–203. doi: 10.1016/s0896-6273(00)80832-6 10677037
26. Hemond CC, Kanwisher NG, Op de Beeck HP. A preference for contralateral stimuli in human object- and face-selective cortex. PLoS One. 2007;2(6):3–7.
27. Niemeier M, Goltz HC, Kuchinad A, Tweed DB, Vilis T. A contralateral preference in the lateral occipital area: Sensory and attentional mechanisms. Cereb Cortex. 2005;15(3):325–31. doi: 10.1093/cercor/bhh134 15269109
28. Schwarzlose RF, Swisher JD, Dang S, Kanwisher N. The distribution of category and location information across object-selective regions in human visual cortex. Proc Natl Acad Sci. 2008;105(11):4447–52. doi: 10.1073/pnas.0800431105 18326624
29. Sayres R, Grill-Spector K. Relating retinotopic and object-selective responses in human lateral occipital cortex. J Neurophysiol. 2008;100(1):249–67. doi: 10.1152/jn.01383.2007 18463186
30. Kirchner H, Thorpe SJ. Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Res. 2006;46(11):1762–76. doi: 10.1016/j.visres.2005.10.002 16289663
31. Crouzet SM, Kirchner H, Thorpe SJ. Fast saccades toward faces: Face detection in just 100 ms. J Vis. 2010;10(4):1–17.
32. Tsotsos JK. Analyzing vision at the complexity level. Behav Brain Sci. 1990;13(3):423–45.
33. Tsotsos JK. A computational perspective on visual attention. Cambridge: MIT Press; 2011.
34. Tsotsos JK, Rodríguez-Sánchez AJ, Rothenstein AL, Simine E. The different stages of visual recognition need different attentional binding strategies. Brain Res. 2008;1225:119–32. doi: 10.1016/j.brainres.2008.05.038 18585692
35. Boehler CN, Tsotsos JK, Schoenfeld MA, Heinze HJ, Hopf JM. The center-surround profile of the focus of attention arises from recurrent processing in visual cortex. Cereb Cortex. 2009;19(4):982–91. doi: 10.1093/cercor/bhn139 18755778
36. Hopf JM, Boehler CN, Schoenfeld MA, Heinze HJ, Tsotsos JK. The spatial profile of the focus of attention in visual search: Insights from MEG recordings. Vision Res. 2010;50(14):1312–20. doi: 10.1016/j.visres.2010.01.015 20117126
37. Hopf JM, Boehler CN, Luck SJ, Tsotsos JK, Heinze HJ, Schoenfeld MA. Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. Proc Natl Acad Sci. 2006;103(4):1053–8. doi: 10.1073/pnas.0507746103 16410356
38. Boehler CN, Schoenfeld MA, Heinze H-J, Hopf J-M. Rapid recurrent processing gates awareness in primary visual cortex. Proc Natl Acad Sci. 2008;105(25):8742–7. doi: 10.1073/pnas.0801999105 18550840
39. Wyatte D, Jilk DJ, O’Reilly RC. Early recurrent feedback facilitates visual object recognition under challenging conditions. Front Psychol. 2014;5:674. doi: 10.3389/fpsyg.2014.00674 25071647
40. Mohsenzadeh Y, Qin S, Cichy RM, Pantazis D. Ultra-rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway. Elife. 2018;7:e36329. doi: 10.7554/eLife.36329 29927384
41. Evans KK, Treisman A. Perception of objects in natural scenes: Is it really attention free? J Exp Psychol Hum Percept Perform. 2005;31(6):1476–92. doi: 10.1037/0096-1523.31.6.1476 16366803
42. Moran J, Desimone R. Selective attention gates visual processing in the extrastriate cortex. Science. 1985;229(4715):782–4. doi: 10.1126/science.4023713 4023713
43. VanRullen R, Thorpe SJ. The time course of visual processing: From early perception to decision-making. J Cogn Neurosci. 2001;13(4):454–61. 11388919
44. Chen Y, Byrne P, Crawford JD. Time course of allocentric decay, egocentric decay, and allocentric-to-egocentric conversion in memory-guided reach. Neuropsychologia. 2011;49(1):49–60. doi: 10.1016/j.neuropsychologia.2010.10.031 21056048
45. Rothenstein AL, Rodríguez-Sánchez AJ, Simine E, Tsotsos JK. Visual feature binding within the Selective Tuning attention framework. Int J Pattern Recognit Artif Intell. 2008;22(5):861–81.
46. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, et al. Microsoft COCO: Common objects in context. In: ECCV. Springer; 2014. p. 740–55.
47. Brainard DH. The Psychophysics Toolbox. Spat Vis. 1997;10(4):433–6. 9176952
48. Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spat Vis. 1997;10(4):437–42. 9176953
49. Praß M, Grimsen C, König M, Fahle M. Ultra rapid object categorization: Effects of level, animacy and context. PLoS One. 2013;8(6):2–11.
50. Walker S, Stafford P, Davis G. Ultra-rapid categorization requires visual attention: Scenes with multiple foreground objects. J Vis. 2008;8(4):21. doi: 10.1167/8.4.21 18484860
51. Davenport JL, Potter MC. Scene consistency in object and background perception. Psychol Sci. 2004;15(8):559–64. doi: 10.1111/j.0956-7976.2004.00719.x 15271002
52. Joubert OR, Fize D, Rousselet GA, Fabre-Thorpe M. Early interference of context congruence on object processing in rapid visual categorization of natural scenes. J Vis. 2009;8(13):11–11.
53. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19(6):716–23.
54. Buffalo EA, Fries P, Landman R, Liang H, Desimone R. A backward progression of attentional effects in the ventral stream. Proc Natl Acad Sci. 2010;107(1):361–5. doi: 10.1073/pnas.0907658106 20007766
55. Cutzu F, Tsotsos JK. The selective tuning model of attention: Psychophysical evidence for a suppressive annulus around an attended item. Vision Res. 2003;43(2):205–19. doi: 10.1016/s0042-6989(02)00491-1 12536142
56. Müller NG, Kleinschmidt A. The attentional “spotlight’s” penumbra: Center-surround modulation in striate cortex. Neuroreport. 2004;15(6):977–80. doi: 10.1097/00001756-200404290-00009 15076718
57. Müller NG, Mollenhauer M, Rösler A, Kleinschmidt A. The attentional field has a Mexican hat distribution. Vision Res. 2005;45(9):1129–37. doi: 10.1016/j.visres.2004.11.003 15707921
58. Di Russo F, Martinez A, Hillyard SA. Source analysis of event-related cortical activity during visuo-spatial attention. Cereb Cortex. 2003;13(5):486–99. doi: 10.1093/cercor/13.5.486 12679295
59. Martínez A, Anllo-Vento L, Sereno MI, Frank LR, Buxton RB, Dubowitz DJ, et al. Involvement of striate and extrastriate visual cortical areas in spatial attention. Nat Neurosci. 1999;2(4):364–9. doi: 10.1038/7274 10204544
60. Noesselt T, Hillyard SA, Woldorff MG, Schoenfeld A, Hagner T, Jäncke L, et al. Delayed striate cortical activation during spatial attention. Neuron. 2002;35(3):575–87. doi: 10.1016/s0896-6273(02)00781-x 12165478
61. Martínez A, Di Russo F, Anllo-Vento L, Sereno MI, Buxton RB, Hillyard SA. Putting spatial attention on the map: Timing and localization of stimulus selection processes in striate and extrastriate visual areas. Vision Res. 2001;41(10–11):1437–57. doi: 10.1016/s0042-6989(00)00267-4 11322985
62. Mehta AD, Ulbert I, Schroeder CE. Intermodal selective attention in monkeys. I: distribution and timing of effects across visual areas. Cereb Cortex. 2000;10(4):343–358. doi: 10.1093/cercor/10.4.343 10769247
63. Chakravarthi R, Carlson TA, Chaffin J, Turret J, VanRullen R. The temporal evolution of coarse location coding of objects: Evidence for feedback. J Cogn Neurosci. 2014;26(10):2370–84. doi: 10.1162/jocn_a_00644 24738769
64. Zhang Y, Meyers EM, Bichot NP, Serre T, Poggio TA, Desimone R. Object decoding with attention in inferior temporal cortex. Proc Natl Acad Sci. 2011;108(21):8850–5. doi: 10.1073/pnas.1100999108 21555594
65. Noudoost B, Chang MH, Steinmetz NA, Moore T. Top-down control of visual attention. Curr Opin Neurobiol. 2010;20(2):183–90. doi: 10.1016/j.conb.2010.02.003 20303256
66. Mihalas S, Dong Y, von der Heydt R, Niebur E. Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proc Natl Acad Sci. 2011;108(18):7583–8. doi: 10.1073/pnas.1014655108 21502489
67. Wyatte D, Curran T, O’Reilly R. The limits of feedforward vision: recurrent processing promotes robust object recognition when objects are degraded. J Cogn Neurosci. 2012;24(11):2248–61. doi: 10.1162/jocn_a_00282 22905822
68. Muckli L, De Martino F, Vizioli L, Petro LS, Smith FW, Ugurbil K, et al. Contextual Feedback to Superficial Layers of V1. Curr Biol. 2015;25(20):2690–5. doi: 10.1016/j.cub.2015.08.057 26441356
69. O’Reilly RC, Wyatte D, Herd S, Mingus B, Jilk DJ. Recurrent processing during object recognition. Front Psychol. 2013;4(124).
70. Roland PE, Hanazawa A, Undeman C, Eriksson D, Tompa T, Nakamura H, et al. Cortical feedback depolarization waves: A mechanism of top-down influence on early visual areas. Proc Natl Acad Sci. 2006;103(33):12586–91. doi: 10.1073/pnas.0604925103 16891418
71. Roland PE. Six principles of visual cortical dynamics. Front Syst Neurosci. 2010;4:1–21. doi: 10.3389/neuro.06.001.2010
72. Baldauf D, Deubel H. Attentional landscapes in reaching and grasping. Vision Res. 2010;50(11):999–1013. doi: 10.1016/j.visres.2010.02.008 20219518
Článek vyšel v časopise
PLOS One
2019 Číslo 9
- Proč jsou nemocnice nepřítelem spánku? A jak to změnit?
- Dlouhodobá ketodieta může poškozovat naše orgány
- „Jednohubky“ z klinického výzkumu – 2024/42
- Není statin jako statin aneb praktický přehled rozdílů jednotlivých molekul
- Metamizol jako analgetikum první volby: kdy, pro koho, jak a proč?