Skip to content
Robotic Brain Universe

Vision and the occipital lobe

How spikes from the retina become edges, faces and scenes, the equations of receptive fields, and the implants and AI models built on them.

Intermediate · about 10 min · updated 2026-10-02 · awaiting clinical review

See it in 3D:

Illustrative simulation excitatory inhibitory

The route from retina to V1, visual field maps and cortical magnification; simple and complex cells, the ventral and dorsal streams, the fusiform face area and motion vision; critical periods and the speed of recognition; field defects, blindsight and motion blindness; difference-of-Gaussians and Gabor receptive fields, the log-polar map and convolution; deep networks as models of vision, decoding images from fMRI and cortical visual prostheses.

Contents
  1. Seeing is computing
  2. What the visual system is
  3. Why two streams and many maps
  4. How the visual cortex builds a scene
  5. When vision is shaped
  6. When vision fails
  7. The mathematics of seeing
  8. Vision, AI and implants
  9. Milestones
  10. Frontiers
  11. Check yourself

Seeing is computing

Each of your retinas holds on average about 4.6 million cones for colour and detail and 92 million rods for dim light. What they send to the brain is not a picture but a stream of spikes, from which the cortex rebuilds edges, motion, faces and whole scenes, and it can tell whether a photograph flashed for just 20 milliseconds contains an animal within about 150 milliseconds.[1,2]

The visual system has shaped both neuroscience and AI. David Hubel and Torsten Wiesel's recordings of cells that respond to oriented edges won the 1981 Nobel Prize and inspired the layered networks behind today's computer vision; those networks now in turn predict how neurons in the visual cortex respond.[3,4,5,6]

This reading follows vision from retina to cortex, explains the maps and streams of the visual brain, the equations that describe its receptive fields, what happens when parts of it fail, and the first implants that let blind people see letters.[7,8]

What the visual system is

Signals leave the eye along the optic nerve, are relayed in the lateral geniculate nucleus of the thalamus and travel through the optic radiation, a fibre system that curves around the lateral ventricle, partly through the temporal lobe, to the primary visual cortex on the banks of the calcarine sulcus (pericalcarine cortex, V1).[9,10]

Beyond V1, functional MRI has revealed many visual field maps, in which nearby neurons respond to nearby points of the image: V1, V2 and V3 on the medial occipital surface, lateral occipital maps including hMT+, ventral maps such as hV4, dorsal maps (V3A and V3B) and maps in posterior parietal cortex.[7]

Key numbers

Cones in an average human retina
4.6 million[1]
Rods in an average human retina
92 million[1]
Peak cone density at the fovea
about 199,000 per mm²[1]
Time to decide whether a flashed photo contains an animal
under 150 ms[2]
Electrodes in the first long-term cortical visual implant in a blind volunteer
96[8]

Why two streams and many maps

Goodale and Milner proposed that vision serves two different jobs. A ventral stream from V1 to the inferotemporal cortex identifies and recognises objects; a dorsal stream from V1 to the posterior parietal cortex performs the transformations needed to guide actions, such as shaping the hand to pick an object up.[11]

Within these streams, regions specialise. Kanwisher and colleagues found an area in the fusiform gyrus that responds much more to faces than to other objects, the fusiform face area. Motion has its own machinery: after bilateral damage to the lateral temporo-occipital cortex, one patient lost the impression of movement while other aspects of vision were largely spared.[12,13]

How the visual cortex builds a scene

Receptive fields. Each neuron responds to light in a small region of the visual field, its receptive field. In the retina these fields have a centre and an opposing surround. In V1, Hubel and Wiesel found simple cells that respond to edges or bars at a particular orientation and position, and complex cells that respond to the right orientation anywhere within a larger field.[3,14]

Maps that magnify. The centre of gaze gets far more cortex than the periphery. Correlating MRI with visual field defects in patients with occipital lesions, Horton and Hoyt showed that the classic map had underestimated the cortical magnification of central vision.[10]

A hierarchy. Information flows from V1 through further maps, building increasingly complex representations; networks trained only to recognise objects turn out to predict neural responses at the top of the ventral hierarchy, in V4 and inferotemporal cortex.[6,7]

From the eye to the two visual streamsRetinarods and cones; centre–surroundcellsLateral geniculate nucleusthalamic relayOptic radiationloops through the temporal lobeV1orientation, simple and complexcellsVentral streamto inferotemporal cortex: whatDorsal streamto posterior parietal cortex:howFusiform face areafacesVisually guided actionreaching and grasping
From the eye to the two visual streams. The main route from eye to cortex, and the two streams that leave V1: a ventral stream for recognising objects such as faces and a dorsal stream for acting on them.[3,9,11,12]
Text version of the diagram
  1. Retina: rods and cones; centre–surround cells. Leads to Lateral geniculate nucleus.
  2. Lateral geniculate nucleus: thalamic relay. Leads to Optic radiation.
  3. Optic radiation: loops through the temporal lobe. Leads to V1.
  4. V1: orientation, simple and complex cells. Leads to Ventral stream; Dorsal stream.
  5. Ventral stream: to inferotemporal cortex: what. Leads to Fusiform face area.
  6. Dorsal stream: to posterior parietal cortex: how. Leads to Visually guided action.
  7. Fusiform face area: faces.
  8. Visually guided action: reaching and grasping.

When vision is shaped

Visual cortex is wired by experience early in life. When Wiesel and Hubel kept one eye of young kittens closed, cells in the striate cortex largely stopped responding to that eye, the classic demonstration of a critical period in development.[15]

In adults, vision is fast. In a task deciding whether a never-before-seen photo flashed for 20 ms contained an animal, brain signals that distinguished the decisions appeared roughly 150 ms after the image, showing that the visual processing needed can be done in under 150 ms.[2]

When vision fails

Field defects. Because each part of the visual field maps to a particular part of cortex, an occipital lesion causes a matching defect in the same part of the visual field of both eyes (a homonymous defect). Surgery in the temporal lobe can damage the optic radiation where it passes through and cause visual field defects.[9,10]

Blindsight. Weiskrantz and colleagues studied a patient who, after removal of part of the occipital lobe, could still locate and discriminate stimuli presented in the blind part of the visual field while denying seeing them.[16]

Motion blindness. The patient studied by Zihl and colleagues could tell a stationary from a moving target only in the periphery, had no impression of movement in depth, and her eye and finger movements guided by vision were impaired, while movement perceived through hearing and touch was normal.[13]

The mathematics of seeing

Visual neurons are among the best-described in the brain: simple equations capture what their receptive fields compute, and how the visual field is laid out on the cortex.[14,17]

Difference of Gaussians (retinal receptive field)[14]
R(r)=Ac e−r2/2σc2−As e−r2/2σs2,σs>σcR(r) = A_c\, e^{-r^2/2\sigma_c^2} - A_s\, e^{-r^2/2\sigma_s^2}, \qquad \sigma_s > \sigma_c

A retinal ganglion cell's sensitivity as a function of distance from the centre of its receptive field: a narrow excitatory centre minus a broad inhibitory surround. Such a cell responds best to a spot that fills its centre and little to uniform light.

Symbols in Difference of Gaussians (retinal receptive field)
SymbolMeaningUnit
rrdistance from the receptive-field centre°
Ac,AsA_c, A_sstrengths of centre and surround—
σc,σs\sigma_c, \sigma_swidths of centre and surround°
Gabor filter (V1 simple cell)[17]
g(x,y)=exp⁡ ⁣(−x′22σx2−y′22σy2)cos⁡ ⁣(2πfx′+ϕ)g(x, y) = \exp\!\left(-\frac{x'^2}{2\sigma_x^2} - \frac{y'^2}{2\sigma_y^2}\right)\cos\!\left(2\pi f x' + \phi\right)

A wave of preferred spatial frequency ff under a Gaussian window, rotated to the cell's preferred orientation (x′x', y′y' are the rotated coordinates). Jones and Palmer found that two-dimensional Gabor filters fit the spatial response profiles of simple cells in the cat's striate cortex with residual errors indistinguishable from random noise.

Symbols in Gabor filter (V1 simple cell)
SymbolMeaningUnit
x′,y′x', y'position, rotated to the preferred orientation°
σx,σy\sigma_x, \sigma_ysize of the Gaussian window°
ffpreferred spatial frequencycycles/°
ϕ\phiphase: whether the cell prefers a light bar, a dark bar or an edge—
Cortical magnification in human V1[10]
M(E)=17.3E+0.75 mm per degreeM(E) = \frac{17.3}{E + 0.75}\ \text{mm per degree}

How many millimetres of V1 represent one degree of the visual field at eccentricity EE. At the centre of gaze about 23 mm of cortex serve each degree; at 20° from the centre, less than 1 mm does.

Symbols in Cortical magnification in human V1
SymbolMeaningUnit
EEeccentricity: angle from the centre of gaze°
MMlinear cortical magnification factormm/°
Log-polar map from retina to cortex[18]
w=k log⁡(z+a)w = k\,\log(z + a)

Writing a point in the visual field as a complex number zz, Schwartz proposed that its position on the cortex is approximately a complex logarithm. Rings of constant eccentricity become nearly parallel lines and rays from the centre become lines at right angles to them, which is why rotating or scaling an image shifts its cortical pattern.

Symbols in Log-polar map from retina to cortex
SymbolMeaningUnit
zzposition in the visual field as a complex number (eccentricity and angle)—
wwposition on the cortical surface—
k,ak, ascale and foveal constants—
Convolution (as in a convolutional network)[5,19]
(I∗K)(x,y)=∑m∑nI(x−m, y−n) K(m,n)(I * K)(x, y) = \sum_{m}\sum_{n} I(x - m,\, y - n)\, K(m, n)

Sliding the same small filter KK over every position of an image II. It is what a layer of simple-cell-like units with identical receptive fields at every location computes, and it is the core operation of the convolutional networks descended from the neocognitron.

Symbols in Convolution (as in a convolutional network)
SymbolMeaningUnit
IIthe image (or the previous layer's output)—
KKthe filter, learned in a convolutional network—

Vision, AI and implants

From cells to networks. Fukushima's neocognitron copied the hierarchy of simple and complex cells; its descendants, deep convolutional networks, transformed computer vision, and object-recognition networks now predict responses in V4 and inferotemporal cortex better than earlier models.[5,6,19]

Reading what you see. Using models of each fMRI voxel's tuning for position, orientation and spatial frequency, Kay and colleagues identified which of a large set of completely new natural photographs a person was looking at, from brain activity alone.[20]

Restoring sight. A 96-electrode array implanted in the visual cortex of a 57-year-old blind volunteer for six months evoked spots of light (phosphenes); stimulating several electrodes together let the participant identify some letters and recognise the boundaries of objects, with no complications from implantation or removal.[8]

Brain and machine vision[3,5,19,20]
Visual systemComputer vision counterpart
Simple cells: oriented, local filtersConvolutional filters
Complex cells: tolerance to positionPooling
Hierarchy from V1 to inferotemporal cortexStacked layers of a deep network
Receptive-field models of voxelsDecoding the image a person is seeing

Milestones

Seeing how we see

  1. 1962Hubel and Wiesel describe simple and complex cells and the architecture of visual cortex.[3]
  2. 1963Closing one eye of young kittens rewires their visual cortex.[15]
  3. 1965The difference-of-Gaussians model of retinal receptive fields.[14]
  4. 1974Blindsight: residual visual capacity in a hemianopic field.[16]
  5. 1977The retina-to-cortex map is described as a complex logarithm.[18]
  6. 1980Fukushima's neocognitron models the visual hierarchy.[5]
  7. 1981Nobel Prize to Hubel and Wiesel for information processing in the visual system.[4]
  8. 1983Selective loss of motion vision after temporo-occipital damage.[13]
  9. 1987Gabor filters are shown to fit simple-cell receptive fields.[17]
  10. 1991The map of human V1 is revised with MRI.[10]
  11. 1992Separate visual pathways for perception and action are proposed.[11]
  12. 1997The fusiform face area is identified with fMRI.[12]
  13. 2008Natural images are identified from human brain activity.[20]
  14. 2021A cortical implant lets a blind volunteer identify letters.[8]

Frontiers

Cortical visual prostheses are moving from single phosphenes towards patterns: simultaneous stimulation through many electrodes lowered thresholds and produced percepts that could be told apart, the basis for letters and object outlines.[8]

Function and wiring are being mapped together. In mouse visual cortex, the MICrONS project recorded the activity of about 75,000 neurons while the mouse viewed natural and synthetic stimuli, and then reconstructed the same tissue's 200,000 cells and half a billion synapses by electron microscopy.[21]

Check yourself

Check yourself

  1. Trace the main route from the retina to the primary visual cortex.
    Show answer

    Optic nerve, lateral geniculate nucleus of the thalamus, optic radiation, then V1 on the banks of the calcarine sulcus.

  2. What is the difference between simple and complex cells?
    Show answer

    Simple cells respond to an edge or bar at a particular orientation and position; complex cells respond to the right orientation anywhere in a larger field.

  3. What do the ventral and dorsal streams do?
    Show answer

    The ventral stream (to inferotemporal cortex) identifies objects; the dorsal stream (to posterior parietal cortex) guides actions towards them.

  4. Why is the centre of gaze over-represented in V1?
    Show answer

    Cortical magnification: far more millimetres of cortex serve each degree near the fovea, about 17.3/(E + 0.75) mm per degree.

  5. What is blindsight?
    Show answer

    Locating or discriminating stimuli in a blind part of the visual field after occipital damage, without awareness of seeing them.

  6. What happened when one eye of young kittens was kept closed?
    Show answer

    Cells in the visual cortex largely stopped responding to that eye, revealing a critical period.

  7. What did the 2021 cortical implant allow its blind participant to do?
    Show answer

    Perceive phosphenes and, with several electrodes stimulated together, identify some letters and object boundaries.

Glossary[3,7,8,10,13,15,16]

Receptive field
The region of the visual field where light changes a neuron's firing.
Retinotopy
The orderly mapping of the visual field onto a brain area.
Simple cell
A V1 neuron responding to an oriented edge or bar at a particular position.
Complex cell
A V1 neuron responding to an orientation over a larger region, regardless of exact position.
Cortical magnification
Millimetres of cortex devoted to each degree of the visual field.
Optic radiation
The fibre pathway from the lateral geniculate nucleus to primary visual cortex.
Homonymous hemianopia
Loss of the same half of the visual field in both eyes.
Blindsight
Visual capacity in a blind field without conscious seeing.
Akinetopsia
Loss of the perception of visual motion.
Phosphene
A spot of light seen without light entering the eye, for example from cortical stimulation.
Critical period
A window in development when experience strongly shapes neural wiring.

References

  1. Curcio CA, Sloan KR, Kalina RE, Hendrickson AE. Human photoreceptor topography. Journal of Comparative Neurology 1990;292(4):497-523. doi:10.1002/cne.902920402
  2. Thorpe S, Fize D, Marlot C. Speed of processing in the human visual system. Nature 1996;381(6582):520-522. doi:10.1038/381520a0
  3. Hubel DH, Wiesel TN. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of Physiology 1962;160(1):106-154. doi:10.1113/jphysiol.1962.sp006837
  4. Nobel Prize Outreach. The Nobel Prize in Physiology or Medicine 1981. NobelPrize.org 1981. https://www.nobelprize.org/prizes/medicine/1981/summary/
  5. Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 1980;36(4):193-202. doi:10.1007/BF00344251
  6. Yamins DLK, Hong H, Cadieu CF, Solomon EA, Seibert D, DiCarlo JJ. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences of the USA 2014;111(23):8619-8624. doi:10.1073/pnas.1403112111
  7. Wandell BA, Dumoulin SO, Brewer AA. Visual field maps in human cortex. Neuron 2007;56(2):366-383. doi:10.1016/j.neuron.2007.10.012
  8. Fernández E, Alfaro A, Soto-Sánchez C, Gonzalez-Lopez P, Lozano AM, Peña S, et al.. Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex. Journal of Clinical Investigation 2021;131(23):e151331. doi:10.1172/JCI151331
  9. Ebeling U, Reulen HJ. Neurosurgical topography of the optic radiation in the temporal lobe. Acta Neurochirurgica 1988;92(1-4):29-36. doi:10.1007/BF01401969
  10. Horton JC, Hoyt WF. The representation of the visual field in human striate cortex. Archives of Ophthalmology 1991;109(6):816-824. doi:10.1001/archopht.1991.01080060080030
  11. Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends in Neurosciences 1992;15(1):20-25. doi:10.1016/0166-2236(92)90344-8
  12. Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. The Journal of Neuroscience 1997;17(11):4302-4311. doi:10.1523/JNEUROSCI.17-11-04302.1997
  13. Zihl J, von Cramon D, Mai N. Selective disturbance of movement vision after bilateral brain damage. Brain 1983;106(2):313-340. doi:10.1093/brain/106.2.313
  14. Rodieck RW. Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vision Research 1965;5(12):583-601. doi:10.1016/0042-6989(65)90033-7
  15. Wiesel TN, Hubel DH. Single-cell responses in striate cortex of kittens deprived of vision in one eye. Journal of Neurophysiology 1963;26(6):1003-1017. doi:10.1152/jn.1963.26.6.1003
  16. Weiskrantz L, Warrington EK, Sanders MD, Marshall J. Visual capacity in the hemianopic field following a restricted occipital ablation. Brain 1974;97(1):709-728. doi:10.1093/brain/97.1.709
  17. Jones JP, Palmer LA. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology 1987;58(6):1233-1258. doi:10.1152/jn.1987.58.6.1233
  18. Schwartz EL. Spatial mapping in the primate sensory projection: analytic structure and relevance to perception. Biological Cybernetics 1977;25(4):181-194. doi:10.1007/BF01885636
  19. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436-444. doi:10.1038/nature14539
  20. Kay KN, Naselaris T, Prenger RJ, Gallant JL. Identifying natural images from human brain activity. Nature 2008;452(7185):352-355. doi:10.1038/nature06713
  21. The MICrONS Consortium, Bae JA, Baptiste M, et al.. Functional connectomics spanning multiple areas of mouse visual cortex. Nature 2025;640(8058):435-447. doi:10.1038/s41586-025-08790-w

Related readings

  • Neural networks, biological and artificial

    How brains and machines learn from experience, the maths both share, and why AI and neuroscience keep borrowing from each other.

    Intermediate

  • The cerebral cortex and its maps

    What the folded outer layer of the brain is made of, why it folds, and how scientists map it from Brodmann's microscope to petabyte connectomes.

    Introductory

  • Brain–computer interfaces

    How implants like BrainGate and Neuralink read intention from motor cortex, the maths of decoding, and the race to restore speech and movement.

    Intermediate

  • Blood supply and stroke

    The arteries of the brain and their territories, how flow is regulated, what a stroke destroys minute by minute, and the trials that changed treatment.

    Intermediate

  • Touch and the parietal lobe

    How fingertips, the somatosensory map and the parietal cortex turn touch and space into action, and how touch is being restored artificially.

    Intermediate

Template anatomy for education. Not patient-specific. Not for clinical decision-making.