The Role of the Visual Cortex

It's tempting to think of the visual cortex as one structure doing one job, but it's really a network of dozens of interconnected areas at the back and sides of the brain, each specialising in some aspect of seeing, such as edges, colour, motion, faces or layout (Felleman & Van Essen, 1991). Together they take up a large share of the cortex, which gives you a sense of just how much work vision demands.

V1: the way in

Signals from your eyes, relayed through the thalamus, arrive first at the primary visual cortex, known as V1 or the striate cortex, right at the back of the brain. Its neurons are famously particular. In the 1950s and 1960s, David Hubel and Torsten Wiesel showed that individual V1 cells respond best to edges and bars at a particular angle in a particular place, and much less to plain light (Hubel & Wiesel, 1959), work that went on to win them a Nobel Prize. V1 keeps a rough map of the retina, and its cells are arranged in columns that cycle systematically through the different angles. In effect, V1 breaks every scene down into a huge catalogue of small edge fragments before anything like an "object" exists in the brain at all.

Beyond V1: a division of labour

From V1, signals spread out to a series of further areas, labelled V2 to V5 (also called MT) and beyond. V2 carries on working on edges and contours, and some of its cells respond to illusory contours, which are edges inferred where none is actually drawn (von der Heydt, Peterhans & Baumgartner, 1984). That's the brain activity behind the Kanizsa triangle. Area V4 is heavily involved in colour, including colour constancy, the stable perception of an object's colour under changing light that the checker shadow and the dress take advantage of.

Area V5/MT specialises in motion. Damage to it can cause akinetopsia, a rare condition in which moving things are seen as a series of still snapshots. Josef Zihl and his colleagues described a patient who found pouring tea difficult because the liquid seemed frozen in mid-air (Zihl, von Cramon & Mai, 1983). The motion aftereffect is thought to come from adaptation in motion-sensitive areas like this one.

Further along, in the temporal lobe, there are regions that respond to particular categories of thing. The best known, the fusiform face area, responds much more strongly to faces than to other objects (Kanwisher, McDermott & Chun, 1997). That's part of the reason face illusions such as the Thatcher effect feel so different from geometric ones: they engage dedicated face circuitry, not just general edge and motion detectors.

Contrast everywhere

One theme running through all of these areas is the receptive field, which is the patch of visual space, or the kind of feature, that a neuron responds to. Many receptive fields are organised with a centre and a surround that work against each other, so neurons throughout the visual system respond to differences, not absolute levels. The classic explanation of the Hermann grid rests on this arrangement, although, as I explain on that page, the smudges now seem to come from processing in the cortex, not from the retinal cells that were first blamed for them.

Feedback: not a one-way climb

It's easy to picture the visual cortex as a strict ladder, climbing from simple features in V1 to whole objects further on. The wiring tells a different story. Higher areas send large numbers of connections back down to V1 and the thalamus, so expectation, attention and context can all change how early areas respond to the same input. That feedback is central to how your brain handles ambiguous images. For figures like the Necker cube and the spinning dancer, imaging studies link each switch between interpretations to activity spread across visual, parietal and frontal areas, not to a single "decision" made in one place (Sterzer, Kleinschmidt & Rees, 2009).

Illusions as probes

Because different illusions target different stages, vision scientists use them as probes. An illusion built from contrast and edges tests early processing. One built from depth cues, like the Ponzo illusion, tests how those cues are combined into a sense of layout. One that depends on grouping, as discussed in Gestalt principles of perception, tests how features are organised into objects. Here's a striking example: people whose V1 has a smaller surface area tend to experience stronger Ebbinghaus and Ponzo illusions (Schwarzkopf, Song & Rees, 2011). It's a finding you can set against your own scores, using the tests on those two pages.

Sources and further reading

1. Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat's striate cortex. Journal of Physiology, 148(3), 574–591. Find this reference on Google Scholar
2. Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47. Find this reference on Google Scholar
3. Zihl, J., von Cramon, D., & Mai, N. (1983). Selective disturbance of movement vision after bilateral brain damage. Brain, 106(2), 313–340. Find this reference on Google Scholar
4. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302–4311. Find this reference on Google Scholar
5. Sterzer, P., Kleinschmidt, A., & Rees, G. (2009). The neural bases of multistable perception. Trends in Cognitive Sciences, 13(7), 310–318. Find this reference on Google Scholar
6. von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224(4654), 1260–1262. Find this reference on Google Scholar
7. Schwarzkopf, D. S., Song, C., & Rees, G. (2011). The surface area of human V1 predicts the subjective experience of object size. Nature Neuroscience, 14(1), 28–30. Find this reference on Google Scholar