|Return to Representationalist Site|
(Extracted from Lehar (2000) The Function of Conscious Experience)
Modern critics of Gestalt theory do not challenge the epistemological argument of indirect realism, but rather they focus their criticism on specific details of Gestalt theory, although careful examination reveals that their criticism is only meaningful when viewed from a naïve realist perspective. This is a classic case of a paradigmatic debate where the opposing camps argue at cross purposes, due to different foundational assumptions. I present for illustration a prominent modern vision scientist, Richard Gregory, a man who has written many insightful books and papers on a wide variety of subjects in visual perception. However Gregory also reveals a fundamental naïve realism, of which he is almost certainly unaware, and I suspect that Gregory would be deeply offended at being accused of naïve realism. However Gregory's views on the critical epistemological issues are representative of the prevailing consensus in contemporary psychology, and therefore Gregory's arguments are illustrative of the common errors of contemporary psychology.
In his discussion of Immanuel Kant, Gregory (1981 p. 335) rejects epistemological dualism but makes no mention of the deep logical problems in the monist alternatives that forced Kant to this incredible conclusion. Gregory challenges Kant's argument that perception involves an a priori understanding of the nature of space, i.e. that it is three-dimensional and Euclidean, by pointing out that we can conceive of non-Euclidean spaces as seen for example in Riemannian geometry. However the only way we can conceive of a Riemannian space is as a warped or distorted Euclidean space. If we could not conceive of Euclidean space, we would have great difficulty conceiving a Riemannian space, let alone of the principles of Euclidean geometry. Gregory misses the more fundamental point that perception involves a representation, and a representation has a pre-ordained dimensionality. For example the retina is a representation, and it has two dimensions, so it is impossible for the retina itself to encode a three- dimensional image. The cortex is also a representation, but it too is limited, this time to representing three spatial dimensions (and time). Even if there were four-dimensional objects to be found in our world, we would never be able to perceive them as four-dimensional, all we would ever see is the peculiar morphing of a three-dimensional shape.This is not to deny that the external world also has (at least) three spatial dimensions, but those dimensions would be completely invisible to us were we not also equipped with a three-dimensional perceptual representation. Kant was right therefore, that the dimensionality of the phenomenal world is a property first and foremost of the representational mechanism of the brain, and only in secondary fashion is it also representative of some of the dimensions of external reality.
As to the Gestalt principle of isomorphism Gregory (1981, p. 368) finds "a deep logical problem here concerning identity and analogy." According to Gestalt theory, when we experience a continuous spatial structure in phenomenal experience, that percept is a direct manifestation of some continuous spatial structure in the brain. Even if this were true, Gregory asks how are these perceptual structures themselves conscious? How can a physical process in the brain somehow become conscious of its own spatial structure? Gestalt theory invokes emergent processes such as the soap bubble, or electric fields, as the principle behind the spatial structures observed in perception. But Gregory objects that if the structures in the brain corresponding to perceptual experiences are somehow conscious of themselves, then similar emergent structures such as soap bubbles and electric fields, must also necessarily be somehow conscious. This conclusion is to Gregory so absurd as to preclude its antecedent. But if Gregory denies the possibility of physical processes ever becoming conscious, that precludes any kind of materialist or scientific explanation of consciousness in the brain. For if we take seriously the materialist thesis that mind is a physical process taking place in the physical mechanism of the brain, then that by itself is already an acceptance of the fact that a physical process taking place in a physical mechanism can under certain circumstances be conscious. Furthermore, even if Gregory's objection were valid, it would necessarily apply also to non-spatial or symbolic representations in the brain. But Gregory raises this objection only against spatial or perceptual structures, thus revealing that his objection is not to representations in the brain as such, or their ability to become conscious, but only to spatial representations.
Gregory argues that it is absurd to believe that a part of your brain becomes colored when you perceive a colored surface, or that the percept of a complex three-dimensional object like a house is represented by a three-dimensional model house in the brain. This too is a common criticism of the principle of isomorphism, but it is only intelligible from a naïve realist perspective. For from the indirect realist perspective it is even more absurd to think that the color you are seeing is the external color itself, entering consciousness directly, somehow bypassing the perceptual representations in the brain. The indirect realist view recognizes that the perceived color is not a property of physical light itself, but only of its perceptual replica in the brain. We have no idea what physical color is really like, we can only know how a color is represented in the brain, which by definition is identically equal to the way we see it in phenomenal experience. It is true that the phenomenal experience of red does not mean that the corresponding portion of our brain would be observed to be red under microscopic examination, nor that it would appear red to a micro-electrode implanted in that portion of the brain. But that does not make it any the less red when experienced phenomenally, or any the less an intrinsic property of the physical mechanism of the brain. Similarly, if we deny that the brain is capable of fabricating a three-dimensional spatial replica of a house, then by definition we could never have the spatial experience of such a house. For the spatial structure of the house cannot enter consciousness directly, only by way of its explicit representation in the physical brain. Gregory (1981, p. 370) argues that there are many different ways to represent information; for example the number six can be expressed as the digit "6" or the word "six", or by the presence of six dots as on dice. "Why then" asks Gregory "did the Gestalt psychologists choose isomorphic representation? This is only one of an infinite set of kinds of representation." The answer, Dr. Gregory, is to be found by inspection, for the representation employed in the brain is exactly as it appears in conscious experience.
Richard Gregory's comments on the problem of the inverted retinal image are also representative of the prevailing view in contemporary psychology. Ever since Johannes Kepler identified the retina as the sensory organ of the eye onto which the lens projects an inverted image, the question arose why we do not see the visual world as inverted. A number of philosophers have weighed in on this issue over the centuries, but to this day the question remains unresolved, although very few are even aware of it, for this too is a manifestation of indirect realism in perception that is inconsistent with naive realism. Gregory's view, which again is representative of the prevailing view in psychology, is that the issue is a pseudoproblem which is actually no problem at all. Gregory argues that " it does not in the least matter that [the retinal images] are upside-down with respect to the objects that they image - for there are, so to say, no eyes looking at the retinal images". In other words Gregory raises the oft-refuted "homunculus objection" (of which more below) to argue that there are no spatial representations in the brain, and non- spatial or symbolic representations have no particular orientation. While this argument might seem plausible to a naïve realist who identifies the spatial world of conscious experience with the external world itself, even a naïve realist must recognize that a retinal after-image, seen after exposure to a bright light or camera flash, is a phenomenon internal to the eye rather than out in the world. And the spatial structure of the retinal after-image is clearly evident, and no internal homunculus is required to view it. Furthermore, the retinal after-image is clearly erect relative to the rest of the perceived world, when viewed "internally" by phenomenological observation, although the corresponding patch of over-exposed retina is obviously inverted by the lens in the eye, relative to the external world. The inescapable conclusion is that the retinal image is in register with our perceptual replica of the external world, both of which are inverted relative to the external world itself. So the reason why we do not see the world inverted is not because there are no spatial representations in the brain, but because we cannot see any trace of the external world directly, and therefore we cannot notice that it is inverted relative to the phenomenal world that we do see.
What is most interesting about Gregory's viewpoint is the fact that a man with such extensive knowledge of the physiology, psychology, and phenomenology of vision, does not seem to fully comprehend the position he contests, so his criticisms inevitably miss their intended targets. It is equally interesting that Gregory never strikes at the core issue hidden behind the debate, i.e. the indirect nature of perception. Nor would Gregory explicitly defend naive realism, a position which he would almost certainly reject, although that philosophy unwittingly pervades his arguments on the other peripheral issues. This schizophrenic attitude intermediate between direct and indirect realism underlies much of the confusion over the core issues of perception and consciousness in the contemporary literature.
While the epistemological question is generally not discussed directly in modern psychology, occasionally the issue comes to a head in particular subject domains. For example the phenomenon of hemi-neglect (McFie & Zangwill 1960, Heilman & Watson 1977, Heilman et al. 1985) initially caused a great stir in psychological circles because it appeared to be concrete evidence for an explicit spatial representation in the brain (Denny-Brown & Chambers 1958, de Renzi 1982, Bisiach & Luzzatti 1978, Bisiach et al. 1981). It is curious that half of phenomenal space should have to disappear for psychologists to take account of its existence in the first place! But after the initial excitement, the naïve realists quickly marshalled their defenses with an array of arguments which many believe to dispose of the troublesome issue of hemi- neglect. Some argue that hemi-neglect is not a failure of spatial representation, but rather an imbalance of attention, or `orienting response', i.e. that half of phenomenal space does not actually disappear, but that the neglect patient is merely inclined to ignore its presence. (Heilman & Watson 1977, Heilman et al. 1985, Kinsbourne 1987, 1993) But even if these arguments are valid, they do not account for the presence in visual consciousness of the spatial structure of the phenomenal world whenever it is not being ignored or neglected; they merely offer a convenient escape clause to make neglect syndrome seem no more mysterious than normal spatial perception. Others argue that the phenomenon of hemi-neglect fractionates to a number of distinct patterns of impairment (Vallar 1998 p. 88). For example many neglect patients can describe the global gestalt of a figure, but when copying its local features, leave those on the left side out (Marshall & Halligan 1995). Present accounts of the multiple forms of neglect refer to several spatial maps and their interaction (e.g. Ladavas et al. 1997). This highlights a conflict between the phenomenal and neurophysiological evidence, the former presenting a unified spatial structure in visual experience, while the latter suggests discrete mechanisms in different cortical areas. To the naive realist this suggests that the spatial percept must be somehow illusory, which thereby supposedly relieves neuroscience from any obligation to account for its manifest properties. What is curious about the debate over neglect is the passion that it engenders. The evidence presented by each side never seems to convince the opposition, because the debate is not really about neglect, but about its implications for perceptual representation, and that issue is not so much a matter of experimental evidence but of the interpretation of that evidence, or the foundational assumptions with which one comes to the debate in the first place.
A similar paradigmatic question underlies the contemporary debate on the nature of mental imagery. Ever since the behaviorist movement mental imagery had been strictly off-limits to psychology until the cognitive revolution revived interest in mental phenomena. A number of ingenious psychophysical experiments have since provided quantitative evidence in support of mental imagery (Kosslyn et al. 1995). For example Shepard and Metzler (1971) show that the time required to rotate a mental image is in linear relation to the angle through which it is rotated. Stephen Kosslyn and his coworkers established that the time it takes to scan between two points on the mental image of a memorized map is in direct proportion to the distance between the points, just as with a real map. (Kosslyn et al. 1978) In addition, it takes subjects approximately the same amount of time to scan a real map as it takes them to scan their visual image of a map memorized from a verbal description (Denis & Cocude 1989). Kosslyn (1975) showed that mental images take a finite time to zoom up or down to a different size, as when comparing the mental image of an elephant immediately following an image of a rabbit. As incredible as these properties might seem in the context of contemporary concepts of neurocomputation, these properties are perfectly consistent with the subjective experience of mental imagery as observed phenomenologically. Visual pathologies generate further striking evidence for the existence of mental images. Bisiach and Luzzatti (1978) and Bisiach et al. (1979) showed that patients suffering from hemispatial neglect sometimes exhibit a parallel neglect of one side of their imaged visual field. Llinás and Paré (1991) observe that neglect patients report a similar lack of perception in their dreams, and that the people inhabiting the dreams of prosopagnosic subjects are faceless. Levine et al. (1985) found that patients who had lost the ability to perceive either shape or location, had corresponding difficulties in mental imagery tasks. Mellet et al. (2000) show that imagery based on verbal descriptions can recruit cortical regions known to be engaged in high-order visual processing. As with the debate over neglect syndrome, the imagery debate generates more heat than light, as the opposing camps never seem to be persuaded by the evidence presented by the other side, due to the paradigmatic issue hidden at the core of the debate which remains unaddressed. For if the subjective experience of visual consciousness is considered to be a valid source of evidence for the nature of the representation in the brain, then the existence and spatial nature of mental imagery can be confirmed by inspection, making all that psychophysical verification actually unnecessary.
Ned Block (1981a) provides an excellent summary of the imagery debate, with chapters contributed by both supporters and opponents of mental imagery. The arguments rallied by the opponents of mental imagery clearly reveal their underlying naive realist assumptions. Ned block himself (1981a, p. 2) and Daniel Dennett (1981a p. 53) raise the oft-refuted homunculus objection, of which more below. Block also raises the neurophysiological objection, that "brain scientists have found no pictures in the brain" revealing that he subscribes to the belief that neurophysiological evidence (or the absence of it) trumps phenomenal evidence, i.e. that we cannot be sure that our phenomenal experience is "real" until it is confirmed neurophysiologically. Jerry Fodor (1981a p. 63) simply announces that he considers the idea incredible: "I am, in fact, strongly inclined to doubt the very intelligibility of the suggestion that there is a stage at which cognitive processes are carried out in a medium that is fundamentally nondiscursive" and leaves it at that. Robert Schwartz (1981a p. 122) complains that the principle of isomorphism between mental imagery and brain processes is "vacuous" because "the notion of resemblance is too vague". Zenon Pylyshyn (1981a p. 153) provides the most blatant naive realist argument that what people report as properties of mental images, for example their spatial extension, are properties of the objects they represent, not of the images themselves. Pylyshyn argues as if the quality of spatial extent can somehow bypass the representational mechanism and penetrate directly into the experience of the imaging subject, without leaving an impression in the subject's brain.
The naive basis of Pylyshyn's concept of visual processing is made abundantly clear in his "Theory of Visual Indexes" (Pylyshyn 1998). According to that theory, an early preattentive stage in visual perception involves a resource-limited mechanism for individuating a small number (four to six) of visual tokens in the stimulus. These tokens are then indexed, like books in a library, and the index remains attached to its object as the object changes its retinal location or other properties. So far, this description is not objectionable, and is consistent with both direct and indirect realism. But where is the sensory image in which these tokens are tracked? Is it registered in an image-like sense-data array? Not according to Pylyshyn, who speaks as if that "image" is out in the world itself, for, Pylyshyn argues, the external world can be accessed as if it were an internal memory, as suggested by O'Regan (1992). Pylyshyn writes (p. 217) "it may no longer be necessary for the perceptual representation itself to have metrical or pictorial properties." Of course since the phenomenal world clearly does have both metrical and pictorial properties, Pylyshyn must be confusing the phenomenal world for the external world. The principal argument that Pylyshyn (1981b, 1988) raises against mental imagery is that much of mental imagery is "cognitively penetrable", i.e. it can be influenced by cognitive considerations. Pylyshyn argues that "a process that is sensitive to the logical contents of beliefs must itself contain at least some inferential ... rule-governed process." But this argument hangs on the assumption that inferential rule-governed processes cannot be analogical, they can only be symbolic and non-spatial. I will show later how computational devices can be designed to incorporate inferential rule-governed processes implemented as analog field-like forces, which are in fact "cognitively penetrable", i.e. they can be influenced or modulated by manipulating a few "cognitive variables". The mental image medium does change its properties to correspond to what subjects believe about the world, as Pylyshyn himself points out, with mental imagery tasks such as imagining the color produced by overlapping yellow and blue color filters, or imagining dropping two objects of different weights. The outcome of these mental images will depend on one's knowledge of color mixing theory, and the theory of gravity respectively, so a physicist's mental image will naturally be more physically correct than that of a non-scientist. This, according to Pylyshyn, demonstrates that the answer cannot be produced by imagery mechanisms, but must involve non-spatial symbolic operations. But as is often the case with paradigmatic issues, this self-same evidence can be interpreted as demonstrating that mental imagery is both analogical, and it is "cognitively penetrable". In fact, the very act of willing a mental image into existence is itself a "cognitive penetration", for the mental image medium would be useless unless it can be controlled cognitively to depict whatever visual problem is required to be solved, when that solution is needed. Pylyshyn's definition of what can be validly considered to be mental imagery excludes mental imagery as it is observed.
Pylyshyn (1981b, 1988) complains that adding "free variables" such as voluntary control of the rate of rotation, translation, or scaling of mental imagery, amounts to "ad hoc contrivances" that detract from the theory's explanatory power. Pylyshyn insists that an analog model should be both "principled and constrained", by which he means that the rate of rotation etc. must be fixed. But Pylyshyn has some ad hoc explanations of his own. His own explanation for the linear time-to-rotate data for mental imagery (Shepard & Metzler 1971) is that the subjects know that rotation is constant in the real world (is it?) and therefore they "make it be the case", i.e. they hold back their keypress response until the mental image would have stopped rotating, if they were using a mental image, except that they arn't. This is about as ad hoc an explanation as any I have heard! As to the model remaining "principled and constrained", we must not lose sight of the fact that the explanandum in this case is the cognitive mechanism of the human mind, an extraordinarily complex and sophisticated function, and clearly far more "cognitively penetrable" than the lower level perceptual function. A model of mental imagery that is arbitrarily constrained to fixed rates of rotation and scaling etc. is guaranteed from the outset to fall far short of an adequate model of mental imagery. To the contrary - what is required are more sophisticated and complex models with a great deal of cognitive control to account for this most complex and malleable mental phenomenon.
But perhaps the most surprising aspect of the imagery debate is the fact that even the proponents of mental imagery seem to hold back from a full acceptance of mind-brain identity, and the validity of phenomenological observation of neurophysiological entities. In the words of Ned Block (1981a, p. 2) "no one writing in this book (nor any other serious participants in the debate) thinks that people can literally see and manipulate real internal pictures." This opinion is echoed by perhaps the most ardent proponent of mental imagery, Stephen Kosslyn (1981a, p. 207) "Although no serious researcher today maintains that images are actual pictures in the head, some still find it reasonable to posit quasi-pictorial representations that are supported by a medium that mimics a coordinate space." I find this statement puzzling, since the notion of quasi-pictorial representations supported by a medium that mimics a coordinate space is exactly what I consider to be "actual pictures in the head". It seems that even the proponents of mental imagery have not entirely abandoned all vestiges of their native naive realism. Daniel Dennett (1981a, p. 88) makes the insightful observation that although this issue remains unresolved, and that nobody really knows with any certainty whether there are pictures in the brain, "A curious feature of the debate is the passion it evokes, which is unlike the normal passion of scientific controversy ... everyone, it seems, has a fiercely confident opinion about the nature and existence of mental images. This manifests itself in remarkable ways: in unhesitating predictions of the results of novel psychological experiments, in flat disbelief in the integrity of recalcitrant experiments, in gleeful citation of 'supporting' experimental evidence, coupled with bland imperviousness to contrary evidence." In their summary of the debate to date, Kosslyn et al. (1981a p. 132) offer the observation that... "Not surprisingly neither arguments nor counter-arguments have been definitive, and neither seems to have had enough force to sway most people from whatever position they found most congenial in the first place." This is a sure sign of a paradigmatic debate!
|Return to Representationalist Site|