“Seeing is believing,” so the saying goes, but new research suggests vision also involves a bit of hearing. Scientists studying brain process involved in sight have found the visual cortex also uses information gleaned from the ears as well as the eyes when viewing the world. They suggest this auditory input enables the visual system to predict incoming information and could confer a survival advantage.
Courtesy: University of Glasgow. In each of the five experiments, ten healthy subjects were scanned with solely auditory stimulation in the absence of visual stimulation. Subjects wore a blindfold and were instructed to keep their eyes closed at all times, and room lights were switched off. In a separate session, retinotopic mapping was performed for all subjects in all experiments to define early visual areas V1, V2, and V3. We show probability maps from the retinotopic mapping data of experiment 1 (n = 10) as derived from functionally informed cortex-based alignment on a flattened Montreal Neurological Institute (MNI) template. White lines indicate mean eccentricity boundaries. Sound-induced blood-oxygen-level-dependent activation patterns from these regions of interest (ROIs) were fed into a multivariate pattern analysis.
“Sounds create visual imagery, mental images, and automatic projections,” says Lars Muckli, PhD, of the Institute of Neuroscience and Psychology at the University of Glasgow, who led the research. “So, for example, if you are in a street and you hear the sound of an approaching motorbike, you expect to see a motorbike coming around the corner. If it turned out to be a horse, you’d be very surprised.”
The study, published in the journal Current Biology, involved conducting five different experiments using functional Magnetic Resonance Imaging (MRI) to examine the activity in the early visual cortex in ten volunteer subjects.
In one experiment they asked the blindfolded volunteers to listen to three different sounds—a bird, traffic noise, and a talking crowd. Using a special algorithm that can identify unique patterns in brain activity, the researchers were able to discriminate between the different sounds being processed in early visual cortex activity. A second experiment revealed even imagined images, in the absence of both sight and sound, evoked activity in the early visual cortex.
“This research enhances our basic understanding of how interconnected different regions of the brain are,” says Muckli. “The early visual cortex hasn’t previously been known to process auditory information, and while there is some anatomical evidence of interconnectedness in monkeys, our study is the first to clearly show a relationship in humans. In future we will test how this auditory information supports visual processing, but the assumption is it provides predictions to help the visual system to focus on surprising events which would confer a survival advantage. This might provide insights into mental health conditions such as schizophrenia or autism and help us understand how sensory perceptions differ in these individuals.”
The project was part of a 5-year study funded by a €1.5m European Research Council consolidator grant entitled “Brain reading of contextual feedback and predictions”, and the Biotechnology and Biological Sciences Research Council.
The researchers will explore more sounds to find out how precise predictive coding in the brain can be.
Source: University of Glasgow
