Kristian Beedholm | THE TORCH

The World Is In the Eye

---

'The Binding Problem' is an unsolved research issue in the study of how the brain processes visual input. It arises from the separate processing of shape, colour and movement information performed by distinct neural pathways in both the eye and brain. In this article we will look at some of these different pathways and into a few of the apparent paradoxes that arise from this separation.

'The Binding Problem' is an unsolved research issue in the study of how the brain processes visual input. It arises from the separate processing of shape, colour and movement information performed by distinct neural pathways in both the eye and brain. In this article we will look at some of these different pathways and into a few of the apparent paradoxes that arise from this separation.

Visual perception is an area of neurophysiology in which there is very good understanding of how sensory and cerebral processes occur, up to a high level of complexity. For this reason, it is primarily data originating from this field that have led people dealing with neuroscience generally to broadly announce that they have given up on dualism and aligned with monism. They have apparently acknowledged that dualism is not meaningful when interpreting neurological phenomena. For this, and for philosophical reasons, it is worth while to study this subject in more detail.

In this article, we shall take a first step toward this, in the form of a brief treatment of a few of the processes that occur along the route from the retina to cortex. At the same time there will be opportunity to look at one of the consequences that a more consistently developed monism has for our simple understanding of neurology.

By far most of what is known about human vision today originates from experiments performed on cats and different types of monkeys, mostly macaques. These are mammals which, like us, have forward pointing eyes and who therefore have a large overlap between the visual fields of each eye. Because of this, there is no immediate reason to assume that the way in which these animals process visual information is radically different from the way in which humans do. In the case of monkeys, the near kinship with humans makes it quite unlikely that we should differ in this regard. This assumption has also been confirmed to a large degree by investigations in which the brains of living humans have been examined in conscious patients undergoing brain surgery.

The Eye

The human eye, like that of other vertebrates, is roughly spherical, with a lens positioned opposite a layer of receptor cells that cover most of the internal surface. This layer is called the retina. Just like in a camera, an inverted image of the light pattern incident on the lens is formed on this light-sensitive layer. This inversion has led to the notion that the brain must somehow be able to flip the image back again, a notion that stems from an erroneous understanding of how sensory information is processed. The 'Inversion Problem' originates in a dualist interpretation of vision, and we shall see later why it is incorrect.

The initiating process in a neurological description of vision is that of light impinging on the retina, which must first and foremost be converted into a form of energy that is compatible with the nervous system. This task is performed by receptor cells, which in the case of vision act in a somewhat unconventional way.

Humans have two types of photoreceptor cells, rods and cones. The rods deal exclusively with vision at low light intensities. These receptors are very sensitive to light, but on the other hand they are colour blind. This does not mean that they respond equally well to light of all wavelengths, rather that there is only one type of receptor. At low light intensities where this cell type operates, there are no free parameters to deliver information about what spectral properties the light has. It is for this reason that we cannot judge colour at night. The other receptor cell type, the cones, exists in three varieties, each of which responds best to particular wavelengths (blue, green, or red). Given this, calculations similar to the following can be made: reasonable activity in Green + the same amount in Red + none in Blue = Yellow.

When the rod or cone receptor cells in the retina absorb a photon of light, they act to increase the electrical potential difference between the outer and inner surfaces of the cell membrane. This is the opposite of what occurs in other neurons, in which the potential difference usually decreases as the nerve is stimulated. From this observation one might say that visual receptor cells act as darkness detectors, a step towards acknowledging that the visual system normally delivers information about changes in contrast to the brain, rather than photointensity.

When a receptor cell in the retina experiences a change in light intensity, it changes the amount of neurotransmitter released to the next link in the chain, the bipolar cells. These manage a key position in the signal pathway, as they deliver input to the ganglion cells, whose outgoing axons form the optic nerve. As far as the eye is concerned, the ganglion cells are the last stage in the processing of incident light. The bipolar cells connecting photoreceptors with ganglion cells, act so that changes in light intensity experienced by the photoreceptors are conveyed to the rest of the nervous system as changes in contrast. Each ganglion cell has inputs from several bipolar cells (perhaps indirectly), and each bipolar cell has contact with several ganglion cells.

Additionally, there are two other main cell types in the retina, which connect to the previously mentioned types in the signal pathway, but these operate 'in the other direction'. Horizontal cells provide connections between the photoreceptors themselves, while amacrine cells connect bipolar cells with each other. The effect of these convoluted inputs to the ganglion cells is that the stimulus that 'best fits' a cell (i.e. elicits the most action potentials), is either a light spot on a darker background (centre-on cell) or the opposite (centre-off cell). The output of the eye is information about local contrasts (with the exception though of when one has been in darkness for a long time).

A class of ganglion cells manages colour perception, for which it is local changes in complementary colours on the retina that give the greatest response from the cell. In this case, we can speak about 'green centre/red surround' or the opposite. This means that the ganglion cell is excited by light of a particular wavelength, which strikes the middle of the area that the cell receives input from, but is inhibited by the same wavelength when it falls in the surrounding area. The colour sensitive receptor cells, the cones, are found concentrated in the retina close to the fovea, the focal point of the lens.

Ganglion cells which have input that originates from receptor cells in this area, 'observe' a very small portion of the visual field, which explains why our resolution of detail is by far best, when light from an object is incident on this part of the retina. The cones also mediate our ordinary sense of daylight photo-intensity, as the outputs of red and green photoreceptors, added together in the middle of a broadband centre-on ganglion cell indicates 'bright'.

Out of the Eye

The output of the retina to the brain is, as mentioned, the axons of the ganglion cells. These are sorted in the optic chiasma so that each brain hemisphere receives axons stimulated by phenomena in the contralateral visual field. For this to occur, those that originate in the portion of the retina closest to the nose must swap sides. As might be expected, we can already here notice the fact that order is maintained of where individual signals in the optic nerve originate. This regulation becomes explicit in the next level of the visual system.

The first processing of the information from the retina occurs in the lateral geniculate nucleus, LGN, which is a part of the structure known as the thalamus. The thalamus receives and forwards nearly all signals, both sensory and motor, to and from the brain and functions in this way as a sort of relay station. In the case of the visual system, signals from the eyes reach into the LGN, whose morphology in a rather extreme manner bears witness to a structure designed to coordinate different types of signal pathways. Ganglion cells with different types of response characteristics and positions in the retina are ordered in this layered structure. It should be mentioned that there are two LGNs, one in each hemisphere (along with everything else discussed here, except for the chiasma).

Those of the retina's ganglion cells that react to rapid changes in contrast, have thick cables to the lowest two layers of the LGN. This signal pathway is called the magnocellular pathway. Each eye contributes with inputs into one of these layers. Ganglion cells in the retina that are colour sensitive and/or have long response times, project via thinner axons to the upper four layers of the LGN, which constitutes a part of the parvocellular pathway. Here as well, there is structure organised by which eye the individual axons originate.

Beyond functioning to separate these pathways, the physiological explanation for the existence of the LGN is not immediately obvious. The responses to stimuli here do not diverge greatly from what occurs in the retina. Additionally, contacts that come directly from the retina comprise only about 15% of the total number of neurons that have input to the LGN. The rest originate from many other regions of the brain, in both the brain stem and cortex.

Hypercolumns and Blobs

The LGN sends axons to a region of the occipital lobe of the cortex, which is denoted Brodmann's area 17, or alternatively V1 (Visual Area 1). The cortex is divided into six layers of cells. Layer 4, which has a number of sub-layers, functions as the input layer. Connections project to other layers from here. It is common to speak of the cortex as being organised in columns. One can imagine that it consists of a close packed forest of columns, each of which has six layers.

In V1 these columns relate to orientation. The retinal stimuli that evoke the strongest responses from individual cells in the cortex are no longer limited to concentric shapes as in the retina, but to a large degree may be lines or contrasts with a specific orientation. Each column has its own preferred orientation of lines, and for each orientation there are additional characteristics, such as preference for stimulus from a particular eye. A small region of the V1 cortex, which contains all combinations of preferred orientation and eye dominance is called a hypercolumn, and covers about 1mm² of the cortex surface.

Within these hypercolumns, there are additional regions, which may be distinguished visually by colouration of the tissue, the so-called blobs. Their function is linked to the parvocellular pathway, more specifically to a subgrouping that deals with colour perception. They do not have a preferred orientation for an object on the retina.

Three Separate Analyses

We can now speak of three systems that operate in parallel, both with respect to morphology and known function. One is the magnocellular system, which in V1 gives rise to cells with a specialisation for the direction of motion of lines or contrasts, and which thereby can be said to be engaged with motion. In addition, there are the two subsections of the parvocellular system, which are quaintly called the parvocellular-interblob system and the parvocellular-blob system, where the latter deals with colour perception and the former with shape.

From V1 these three signal pathways deliver input to V2, which is at least easy to remember. V2 consists of the region surrounding V1, and is known also as Brodmann area 18. Here again, one can see by colour a differentiation of the tissue, which appears striated. These striations arise from the different cell types that constitute the columns. Thick stripes house connections of the magnocellular pathway, while thin stripes manage colour perception, and thereby the parvocellular-blob system. The parvocellular-interblob system is found in between these two.

After V2 the paths of parvo- and magnocellular systems separate, so that there is a clear separation between the regions of the cortex that manage them. The magnocellular-interblob system continues to V3 (yet another surrounding area) and later to V5, which lies on the border between occipital, parietal, and temporal lobes. By very local electrical stimulation of monkeys' cortex in this region, it has been possible to change their perception as to which direction a number of dots were moving. There is very good agreement between perception of motion and activity of neurons in this area. Signals are sent from V5 to the parietal lobe, where it is thought they have importance for visual coordination of the body, whose self-perception amongst other things is located here.

The parvocellular systems continue together, though they still reside in separate cells. From V2 and V4 (another border area) connections end in the inferotemporal cortex, which as the name states, is an area in the lower part of the temporal cortex. Here the parvocellular interblob-system, which is specialised for shape recognition, gives rise to cells that respond to rather complicated stimuli. For example the columns in some areas seem to give greatest response to faces and hands. About 10% of the cells are thought to have the task of searching the visual field for such specific object classes. Other cells have more simple, icon-like objects as their preferred stimulus. Interestingly the visual field of these cells is very large. For such a 'face-neuron' it doesn't matter where the stimulus is positioned within an area of roughly 25° x 25°. Individual neurons have even been observed that cover the entire visual field in this way. The large area of the retina covered by each of these neurons should be viewed in context, in that the columns here are not ordered according to position on the retina, in contrast to V1, V2 or V5 for instance. Localisation is a parameter that is managed by areas other than the inferotemporal cortex.

The colour sensitive cells of the parvocellular system exist separately from shape perception, though high level processing of both systems is performed in the inferotemporal area. That these do not mix can be verified by noting the often very poor way in which cartoons are coloured. If colour perception played a significant role in shape perception, one would have difficulty seeing what pictures portray when the colours are not precisely printed on top of the line drawing. On a more cultured note, one could refer to the painter Van Gogh, who used rather untraditional colours in the reproduction of objects, for instance his own face. One can still readily see what the pictures portray, because the colours, so to speak, have been put on somewhere else in the brain. They do not influence our perception of the object's shape.

'The Binding Problem'

We are then in the situation that different aspects of the visual world appear to reside in different areas of the brain, which are not in direct contact with each other. The parallel way in which the extraction, for example of colour, shape, and motion, occurs could almost have been guessed. It would be strange if the brain contained, for instance, a neuron for every conceivable combination of colour for cubes and the same number for the colour of spheres. There would simply not be enough neurons for this task, even less if all possible directions of motion were to be represented as well. But how are these properties put together, so that for instance we can say, 'the ball and not the box is red'? The answer to this question is, as yet, unknown.

Some Paradoxes

We saw earlier that colour perception plays no role in the perception of shapes. The following is another interesting observation related to the binding problem. If we have green dots moving slowly on a red background of the same light intensity, we cannot see that they are moving. We can only notice after some time that they are no longer in the same position, which is not so much seeing the motion as deducing that it must have occurred. This is of course because the cells that manage motion perception are colour blind. They manage only intensity contrasts. Other colour combinations give the same result.

An evolutionarily older part of our visual system manages how the eye follows an object. This mechanism bypasses both LGN and cortex, with direct signaling from the retina to the superior colliculus, a part of the brain stem. People who have had their primary visual cortex destroyed, should be blind according to what has been discussed here. However, there is evidence that this parallel signal pathway contains poorly studied fibres, which connect the eye with the mediotemporal area of the cortex, which deals with direction of motion. This is thought to be the mechanism by which some 'neurologically blind' people are able to see objects in motion (Anecdote has it that that some can even catch a ball thrown to them). It seems incredible that someone can in this way, have an object in the middle of their visual field, which they can see moving, but which they cannot actually see.

There are other famous examples of people with localised brain damage to the inferotemporal area who cannot recognise, or even notice faces. These sorts of observations lead to a metaphor of the brain as being analogous to a typesetter's tray. Individual neurons are spaces in the tray, and information is processed when the relevant type-block goes into them. This description is especially interesting in the case where two people do not have the same spaces in their tray. They will simply not be able to see the same thing. In keeping with this metaphor, a brain-damaged person who cannot see faces has had the relevant spaces blocked off.

With the separation described here, into categories of shape, colour and motion, one can add to the typesetter's tray, that there occurs (already in the eye for the visual system) a rough allocation of the type-blocks. Actually, the model goes to pieces a bit, as it is almost the blocks themselves that are subdivided.

Dualism and Vision

With what has been discussed up to now, we can better understand that the dualist paradigm, leading to the camera model of the visual system, does not provide a good foundation for understanding the process of perception.

Don't the 'original' electrical signals, from which the brain constructs its visual world, reflect the light impression from objects? Don't these impulses seek to create as precise a reproduction in the brain as possible? No. Vision simply does not work in the same manner as a camera, in which a photographic film reproduces or represents an outer scenario. On the contrary, the visual system comprises many more or less disjunct elements, each performing their part in the construction of a final visual impression. We cannot at all preserve the traditional notion, that what we see looks like that which we are looking at. Actually, we are forced to conclude that in this fundamental survey of the vision system, it is meaningless to talk about an outer object that the visual system depicts at all. There is nothing more than what we see, and there is no reproduction of anything we meaningfully can call 'visual'.

This may seem paradoxical, but on reflection, the classical assumption about the visual system as a camera lens, through which objects are photographed onto the brain, is far more paradoxical, if not absurd.

This assumption demands that in the brain 'someone' views the photographed picture. Who should that be? An 'inner observer', or a 'soul' of sorts? However, if this soul could see the pictures in the brain, then it must itself view them through its own visual system. 'Someone', perhaps a photographing soul inside the soul, is needed to view these pictures inside the soul. This in turn demands a 'point of view' for the next link in the photography chain - and so on ad infinitum.

This is the so-called homunculus problem: Who is it looking at the pictures supposedly found in the brain? Who examines the pictures formed in the brain of whoever looked at the pictures in the first brain?

Dualism, where there is an 'outer world' represented in an 'inner world', observed by some type of 'soul', operates with observer-independent objects, which the visual system depicts. As discussed here, vision does not depict anything in any traditional sense; it constructs a visual world from a monstrously tangled puzzle of nerve impulses.

It may be tempting to view the visual system using a modified 'Kantian' camera model. This would be in contrast to a one-to-one representation, photography of an outer world, with the visual system functioning as a type of accurate photographic developer. One could imagine an outer world so very different from the visual world constructed by the visual system, that it approaches Kant's ideas about an in principle unknowable 'Ding an sich' (Thing to itself), which human cognition (in this case via the visual system) then conceptualises as its own 'Ding für mich' (Thing for me).

The observed, becomes a meaningless and amorphous 'an sich', which the visual system converts into a concrete visual world, 'für mich'. We can never get to 'see' the 'Ding an sich', because the process of seeing itself is a conversion into visual categories which do not look like this, which cannot look like it because 'looking like' involves the very same construct of the visual system.

The main problem with this revised camera model is that such an in principle, not just non-visual but non-knowable 'model' or 'origin' of the actual, knowable visual impression, obviously enough must be viewed as a redundancy. Why assume that what we see is a picture of something completely different, something that we by definition are unable ever to observe? Does there even need to be a picture of 'anything' or can what we see just be what we see, a creation of the visual system?

In any case, such a monism, in contrast to any dualism, seems to be in complete agreement with the fundamental functioning of the visual system.

The Inversion Theorem

It was promised at the start of this article that we would do away with the widespread conception that the brain compensates for the inverse projection of the visual field on the retina by turning the picture back around. The rationale behind the Inversion Theorem was naturally enough the perception of the visual system as a camera, which is inextricably bound to dualism. One imagines an almost digital projection of light intensities, which 'in the brain' are put together into a picture of the visual field, ready to be looked at. When we see the world 'right side up', the pictures that have been formed in the brain must obviously have been inverted. However, this is only true within the dualist paradigm, which operates with an 'outer' scenario, depicted in an 'inner' visual impression. It is not possible to put a finger on where in the brain anything has been rotated. The representation of the visual field on the retina is simply never turned around.

The motion sensitive neuron which reacts when an object moves from left to right in the visual field, reacts then to light or shadow moving from right to left on the retina. However, the neuron 'itself' does not care. It just fires and thereby sends the signal that 'something is moving to the right', which could be called the purpose of the neuron. Alternatively, one could say that when the neuron fires, something is moving to the right. We experience as a matter of course an outer world as pointing the 'right' way, because how else should it point?

Almost all researchers dealing with the visual system have definitively left dualism behind. However, most textbooks on the subject still contain a passage, which reassures that the brain takes into account such physiological problems as the projection of the visual field on the retina being the wrong way up. This begs the question of whether there is such an absolute reference, an original that is photographed the right way up. All research into the visual system must, as we have seen, dismiss this perception.

For the individual neuron in the visual system, there is nothing to change around.

The dualist paradigm is difficult to escape, because it is so rooted in our linguistic reproduction of the visual world.

In the meanwhile we must make do with the fact that dualism is nothing more than an at times useful model, which can sometimes ease communication. It is practical to talk about a world as the 'origin' of visual perception, but fundamentally this is not the case. The human visual system conceptualises a visual world, for which the perception of the observer, as much as observed object, is constructed and not simply registered in the traditional sense.

The world is in the eye, or rather, everything we see, the whole visual world, is the common creation of human visual systems. That we cannot experience this directly does not mean that this is not the case. In the same way, the fact that we cannot experience the blind spot of our eye, because the visual system in a creative manner fills it in, does not mean that there is no blind spot.

By Kristian Beedholm
Published in Faklen (The Torch) No. 8, 1998

---