Mind Hacks™: Tips & Tools for Using Your Brain (8 page)

The visual system is a complex network of modules and pathways, all specializing in
different tasks to contribute to our eventual impression of the world.

In a sense, light landing on the retina — the sensory surface at the back of the eye — is
already inside the brain. The whole central nervous system (the brain and spinal column
[
Get Acquainted with the Central Nervous System
]
) is contained within a number of membranes, the outermost of which is
called the
dura mater
. The white of your eye, the surface that
protects the eye itself, is a continuation of this membrane, meaning the eye is inside the
same sac. It’s as if two parts of your brain had decided to bulge out of your head and
become your eyes, but without becoming separate organs.

The retina is a surface of cells at the back of your eye, containing a layer of
photoreceptors
, cells that detect light and convert it to
electrical signals. For most of the eye, signals are aggregated — a hundred photoreceptors
will pass their signal onto a single cell further along in the chain. In the center of the
eye, a place called the fovea, there is no such signal compression. (The population
density of photoreceptors changes considerably across the retina
[
See the Limits of Your Vision
]
.) The resolution at the
fovea is as high as it can be, with cells packed in, and the uncompressed signal
dispatched, along with all the other information from other cells, down the
optic nerve
. The optic nerve is a bundle of projections from the
neurons that sit behind the photoreceptors in the retina, carrying electrical information
toward the brain, the path of information out of the eye. The size of the optic nerve is
such that it creates a hole in our field of vision, as photoreceptors can’t sit over the
spot where it quits the eyeball (that’s what’s referred to as the blind spot
[
Map Your Blind Spot
]
).

Just behind the eyes, in the middle, the optic nerves from each eye meet, split, and
recombine in a new fashion, at the
optic chiasm
. The right halves of
the two retinas are connected to the right of the brain and the left halves to the left
side (from here on, the two hemispheres of the brain are mirror images of each other). It
seems a little odd to divide processing directly down the center of the visual field,
rather than by eye, but this allows a single side of the brain to compare the same scene
as observed by both eyes, which it needs to get access to depth information.

The route plan now is a dash from the optic chiasm right to the back of the brain, to
reach the visual cortex, which is where the real work starts happening. Along the way,
there’s a single pit stop at a small region buried deep within the brain called the
lateral geniculate nucleus
, or LGN (there’s one of these in each
hemisphere, of course).

Actually, the LGN isn’t a simple relay station. It deals almost entirely with optical
information, all 1.5 million cells of it. But it also takes input from areas of the brain
that deal with what you’re paying attention to, as well as from the cortex in general, and
mixes that in too. Before visual features have been extracted from the raw visual
information, sophisticated input from elsewhere is being added — we’re not really sure of
what’s happening here.

There’s another division of the visual signal here, too. The LGN has processing
pathways for two separate signals: coarse, low-resolution data (lacking in color) goes
into the
magnocellular
pathway. High-resolution information goes
along the
parvocellular
pathway. Although there are many subsequent
crossovers, this division remains throughout the visual system.

From the LGN, the signals are sent directly to the visual cortex. At the lower back of
the cerebrum (so about a third of the way up your brain, on the back of your head, and
toward the middle) is an area of the cortex called either the striate or primary visual
cortex. It’s called “striate” simply because it contains a dark stripe when closely
examined.

Why the stripes? The primary visual cortex is literally six layers of cells, with a
thicker and subdivided layer four where the two different pathways from the LGN land.
These projections from LGN create the dark band that gives the striate cortex its name. As
visual information moves through this region, cells in all six layers play a role in
extracting different features. It’s way more complex than the LGN — the striate contains
about 200 million cells.

The first batch of processing takes place in a module called V1.
V1
holds a map of the retina as source material, which looks more
or less like the area of the eye it’s dealing with, only distorted. The part of the map
that represents the fovea — the high-resolution center of the eye — is all out of
proportion because of the number of cells dedicated to it. It’s as large as
the rest of the map put together.

Physically standing on top of this map are what are called hypercolumns. A hypercolumn
is a stack of cells performing processing that sits on top of an individual location and
extracts basic information. So some neurons will become active when they see a particular
color, others when they see a line segment at a particular angle, and other more complex
ones when they see lines at certain angles moving in particular directions. This first map
and its associated hypercolumns constitute the area V1 (
V
for
“vision”); it performs really simple feature extraction.

The subsequent visual processing areas named
V2
and
V3
(again,
V
for “vision,” the number just
denotes order), also in the visual cortex, are similar. Information gets bumped from V1 to
V2 by dumping it into V2’s own map, which acts as the center for its batch of processing.
V3 follows the same pattern: at the end of each stage, the map is recombined and passed
on.

So far visual processing has been mostly linear. There are feedback (the LGN gets
information from elsewhere on the cortex, for example) and crossovers, but mostly the
coarse and fine visual pathways have been processed separately and there’s been a
reasonably steady progression from the eye to the primary visual cortex.

From V3, visual information is sent to dozens of areas all over the cortex. These
modules send information to one another and draw from and feed other areas. It stops being
a production line and turns into a big construction site, with many areas extracting and
associating different features, all simultaneously.

There’s still a broad distinction between the two pathways though. The coarse visual
information, the
magnocellular pathway
, flows up to the top of the
head. It’s called the
dorsal stream
, or, more memorably, the “where”
stream. From here on, there are modules to spot motion and to look for broad
features.

The fine detail of vision from the
parvocellular pathway
comes
out of the primary visual cortex and flows down the
ventral
stream —
the “what” stream. The destination for this stream is the inferior
temporal lobe, the underside of the cerebrum, above and behind the eyes.

As the name suggests, the “what” stream is all about object recognition. On the way to
the temporal lobe, there’s a stop-off for a little further processing at a unit called the
lateral occipital complex
(LOC). What happens here is
key to what’ll happen at the final destination points of the “what” stream.
The LOC looks for similarity in color and orientation and groups parts of the visual map
together into objects, separating them from the background.

Later on, these objects will be recognized as faces or whatever else. It represents a
common method: the visual information is processed to look for features. When found,
information about those features is added to the pool of data, and the whole lot is sent
on.

The wiring diagram for all the subsequent motion detection and object recognition
modules is enormously complex. After basic feature extraction, there’s still number
judgment, following moving objects, and spotting biological motion
[
See a Person in Moving Lights
]
to be done. At a certain
point, the defining characteristic of the cortex as a whole must come into play, and
visual information is processed enough to be associated with memory, language, and reading
emotions. This is where it blends in to the higher-order functions of the whole
brain.

In the hacks that follow, we’ll explore the effects of early and late visual
processing. A common thread through these effects will be the assumptions the visual
system has made about the visual world to expedite its computation — and by looking at the
quirks of vision, we can draw some of these out. Assumptions like the visual world
remaining relatively stable from second to second (so we don’t notice if it doesn’t
[
Blind to Change
]
) and supposing
that dark areas are shadows, which is the quirk that makeup takes advantage of
[
Fool Yourself into Seeing 3D
]
.

In a sense, the fact that we can observe these assumptions suggests that the visual
system assumes as much about the external environment as about its own modules. The visual
system’s expectation that the motion module will report motion correctly (and therefore
our confusion when the module doesn’t identify motion correctly
[
See Movement When All Is Still
]
) is much the same as
the visual system’s expectation that a shadow is reporting 3D shape correctly. While we
could think of the visual system as entirely in the brain, really we should include the
eyes, the head, the body, and the environment as components in this big, messy, densely
connected human visual processing system, all of which report their conclusions into the
mix.

And somehow, in all of this, the visual perception we know and love somehow springs
into existence. There doesn’t seem to be a single place where all this visual processing
is reassembled, no internal television screen that we watch (and even if there were, who
would watch it?). It’s distributed over the whole visual system, and over the environment
too. Not just a picture at the retina, after all.