Resurrection of resemblance theory

Has the Aristotelian “resemblance theory” of representation been abandoned, when we embark on a computational analysis of cognitive processes?  This question came to my mind when reading Geir Kirkebøen’s excellent article on “Descartes’ psychology of vision and cognitive science” (1998). I stumbled upon it after Fodor’s modularity book had sent me to reading texts on computational neuroscience and I came across David Marr’s influential book on vision (1982). It appeared posthumously after his death of leukaemia only at the age of 35. A reprint of the book has been issued quite recently. There are several good reviews of Marr’s conceptualisations.  Kirkebøen traces the historical antecedents of Marr’s view and demonstrates its similarity with Descartes’ theory of vision. This article, once again, demonstrates the immeasurable value of a historical approach to the theoretical issues of psychology. If you can get hold of this article, do read it.

Kirkebøen argues that Marr’s computational understanding is a restatement of Descartes’ solution to the problem, how  res extensa can become res cogitas. Descartes showed that extension (a “picture” on the retina) is transformed into mechanical movements in the optical nerve by a computational process that models two-dimensional objects like his analytical geometry modelled Euclidean forms.  This seems indeed to be the end of the “resemblance theory” of representations, as Fodor pointed out.

However, the idea that our visual perception uses algorithmic procedures to generate three-dimensional perceptibles from two-dimensional sense data does not abolish the problem of the relationship between the source and its retinal imprint. Oron Shagrir’s recent (2010) commentary on Marr’s theory is helpful here. The following quotations are terminologically difficult for us who are not trained in the language of computational neuroscience, but I believe you can recognise the point:

“In the case of edge detection, Marr (1982, 68ff.) refers to the constraint of spatial localization, which means (in this context) that the things in the world that give rise to intensity changes are spatially localized… Another pertinent physical fact is that intensity changes in the [retinal] image result from “surface discontinuities or from reflectance or illumination boundaries.” (Shagrir, 2010, p. 488)

“Marr’s explanation appeals to similarity between the internal mapping relations and external relations between the features that are being represented. The similarity is not at  the level of physical properties. After all, the physiological properties of the brain are quite different from the physical and optical properties that make up our visual field. The similarity is at a more abstract level of mathematical properties.” (p. 489)

“Thus, Marr not only demonstrates that the internal mathematical function correlates with the contingent world that we live in. He also underscores the basis of this correlation, which is a similarity of mathematical structures. This mathematically based similarity, I maintain, is the key in addressing the appropriateness problem…” (p. 489)

Shagrir does not refer to Kirkebøen’s paper and is perhaps not familiar with it. The second quotation, however, reproduces almost literally the Cartesian solution to the difference between the optic imprint and the physiological properties  of the brain. The similarity is established by mathematical modelling. To me this is a restatement of the resemblance theory. It is still about “form without matter” even if the form is not any more understood in pictorial terms. Shagrir also restates Aristotle's idea that vision is about distinguishing light from darkness.

Fodor, Aristotle, and the modular mind

I mentioned some time ago, when discussing Daniel Stern’s attempt at tracing meaningful interpersonal interactions, that there is a convenient way to bypass the knotty problem of how representations - or what ”comes in” - obtain meanings that can be expressed by referential, or symbolic, means. You just ignore the divide and concentrate on either side. Because Stern did not want to do this, he crashed into the problem. His effort to solve it demonstrate that a number representational modes must be assumed and, specifically, one that allows for the transition. Stern called it “proto-narrative envelope”. To me, this construction is not entirely convincing.

Disregarding the number and nature of representational modes that make up our experience of the world, the other problem is the number of mental faculties that must be postulated  to account for the incoming information. Stimulated by Ibn Sina’s Medieval account of the five faculties inthe perceptual soul, I left the issue of meaning for a while and began reading Jerry Fodor’s and Peter Carruthers’ views of mind modules. What an interesting world of thought it is. As a psychotherapist being inspired by Lev Vygotsky, Valentin Voloshinov, Mikhail Bakhtin, and Donald Winnicott, I left behind the cognitive approaches to the mind in the early 1970s. Hence, the more philosophical discourse on  the computational models of the mind is a foreign territory to me and I have to proceed cautiously.

Both Fodor and Carruthers subscribe to the assumption on multiple functions, or modules, that mediate between the flow of sensory input and the mind. Their main difference seems to be in the level and extent of such modules. Fodor claims that the central (computational) processes are non-modular, while Carruthers supports a “massive modularity” theory of the mind. Both share the view that mental modules are innate, hard-wired systems that operate according to their inner rules.

Interestingly, when Fodor published his seminal book “The modularity of mind” (1983), he added “An essay on faculty psychology” as the subheading. Indeed, within the Aristotelian framework of how the mind relates to the world there is logically no alternative.  Although written almost thirty years ago, Fodor’s book is a relevant work that introduces the modern version of the ancient model of understanding the soul’s operations. Here is an illustration how Fodor approaches the issue of incoming forms:

Any mechanism whose states covary with environmental ones can be thought of as registering information about the world; and, given the satisfaction of certain further conditions, the output of such systems can reasonably be thought of as representations of the environmental states with which they covary … But if cognitive processors are computational systems, they have access to such information solely in virtue of the form of the representations in which it is couched. Computational processes are, by definition, syntactic; a device which makes information available to such processes is therefore responsible for it format as well as its quality… if we think of the perceptual mechanisms as analogous to such devices, then we are saying that what perception must do is to so represent the world as to make it accessible to thought. ” (Fodor: The modularity of mind, pp. 39-40, original italics).

It is not easy to unpack this statement. In fact, the whole book is an elaboration of this basic formulation. For the time being, I only pay attention to the first and the last sentences in the passage. Aristotle’s view of the co-varying mechanism covered the sense organs and the internal common sense, whose joint operation produced phantasmata as the output, in a mode that was accessible to thought. This analogue does not do justice to Fodor because, elsewhere, he explicitly rejects the “resemblance theory” of mental representations. 

True, since Descartes representations could not anymore be conceived of as “forms without matter”. The division between “res extensa” and “res cogitans” implied a radical transformation from the material to the ideal mode of existence. The concept of representation has indeed changed a lot. However, the notion of sensory processes that  covary with external states, register information about the world, and present this information to thought, faithfully repeats the ancient model of the mind.

How do representations gain meaning?

Daniel Stern postulated six internal formats that pick up various aspects of the incoming sense data. Probably not aware of it, Stern repeats Aristotle’s problem of how the pieces from different sensory channels are parsed together. By being so distinct and clear, Stern finds exactly the right words to show the problem. I must repeat the passage from my previous post:

...the diverse events and feelings are tied together as necessary elements of a single unified happening that, at one of its higher levels, assumes a meaning. The problem with this and other such solutions is how the meaning, even a very primitive one, slips in or gets assigned or is constructed from the pieces. (Stern: The motherhood constellation, p. 89, my italics)

Indeed, by what mental function does meaning become attached to the representation, or the internal perceptual form of an object or an event? This is a typically Aristotelian problem, as Kaukua and Kukkonen pointed out in their article. An interesting Medieval attempt to solve it was presented by Ibn Sina (the Latin Avicenna), a Persian physician who lived around the turn of the first millennium (980–1037). He was an important developer of Arabian philosophical psychology and a link in the subsequent Western Medieval adoption of Aristotle’s philosophy.

Much in the same manner as Stern, Ibn Sina recognised the need to postulate a number of internal functions, or internal senses, that processed the elemental data, received by the five sense organs. I managed to retrieve a translated excerpt  in which Ibn Sina enlists the five internal senses that perform various synthetic operations on the incoming forms.

One of the animal internal faculties of perception is the faculty of fantasy, i.e., sensus communis, located in the forepart of the front ventricle of the brain. It receives all the forms which are imprinted on the five [external] senses and transmitted to it from them. Next is the faculty of representation located at the rear part of the front ventricle of the brain, which preserves what the sensus communis has received from the five senses even in the absence of the sensed object. … Next is the faculty of the 'sensitive imagination' in relation to the animal soul, and the 'rational imagination' in relation to the human soul. This faculty is located in the middle ventricle of the brain near the vermiform process, and its function is to combine certain things with others in the faculty of representation, and to separate some things from others as it chooses. Then there is the estimative faculty located in the far end of the middle ventricle of the brain, which perceives the non-sensible intentions that exist in the individual sensible objects, like the faculty that judges that the wolf is to be avoided and the child is to be loved. Next there is the retentive and recollective faculty located in the rear ventricle of the brain, which retains what the estimative faculty perceives of the non-sensible intentions existing in individual sensible objects. (Avicenna, translated in Rahman, F.(1952). Avicenna's psychology. London: Oxford University Press,  p. 31)

Ibn Sina is surprisingly modern by regarding these internal faculties as brain modules. Certainly, current brain imaging studies have shown that these modules  of the “sensible soul” do not reside in the ventricles, but the contemporary modular theories of mind follow exactly the same logic of analysis as Ibn Sina does in the excerpt. Going back to Stern’s question about constructing meaning of events I want you to pay attention to the estimative faculty that is capable of perceiving the non-sensible intentions that exist in sensible objects.

Ibn Sina understood well the impoverished character of the Aristotelian sensible forms. He asked, why does the lamb flee when perceiving a wolf. The grey, hairy form does not “inform” a particular danger. Something had to be added, and here we have a conception of meaning that “exists in the individual sensible objects” and is recognised by the estimative faculty (cf. Stern’s “protonarrative envelope”).  Thus, in addition to forms, objects seem to “emit” intentions that are perceived by an internal mental faculty. Actually, the currently popular research on the “theory-of-mind”, or "mind-reading" module represents a truly Avicennian approach to understanding how we make sense of others. I will return to this topic in due course.

According to Kaukua and Kukkonen, Ibn Sina had a sophisticated understanding of intentions.  He did not, actually, regard them as properties of objects or events that they "emit" and are then sensed by the estimative faculty. Instead, intentions vary according to the reciprocal positioning of the agent an object.  Kaukua and Kukkonen illustrate this by an example of the hunter’s way of perceiving a wolf. While the lamb flees, the hunter has a rifle and this alters the intentional balance. Now it is rather the wolf who must escape. But the hunter may be an environmentalist and does not want to harm the wolf. “Acts of meaning” (in Bruner’s terms) are not anything as such. They receive their meaning only in the concrete relationships that are established between the actor and the object.

Representing interaction

In Chapter 5 of The Motherhood Constellation, Daniel Stern traces the path from mother’s overt behaviour to the infant’s subjective experience, or his way of representing  interpersonal events. Stern is aware of the difficulties ahead: “When thinking about infants, we are far less sure what such representations are and how they get there" (i.e., in the infant’s mind). (p. 79). The first problem is the nature of social interaction. Inanimate events can be described in the standard Aristotelian model of incoming forms:

There are several important differences between the representations of inanimate physical happenings and those of subjective interpersonal happenings. For inanimate physical happenings, the mental events in a representation are thought to be isomorphic with the real events: they are simply performed virtually on an internal stage. (Stern, The motherhood constellation, 1995, p. 80)

Joint interpersonal actions have several features that make this straightforward model inadequate. Stern lists six, which I will not reproduce here. Taken together, they make up a particular format of, or a placeholder for representations that Stern calls “schema-of-being-with-another”. It can be seen as an elaboration of his earlier description of “Representations of Interactions that have been Generalized” or RIGs.  Stern warns of equating representations with the relatively simple idea of internalising interactions and adopts a strictly Kantian position:

...these representations are not formed from external events or persons that have  been internalized. They are not put inside from the outside. They are constructed from the inside, from the self-experience of being with another. Nothing is taken in. (ibid., p. 81)

I admire Stern’s bold precision even if by being so explicit he paints himself into the corner. Now he is faced with the issue by what means does the infant construct the internal representations of interactive events. In a truly Kantian manner Stern has to assume a number of innate "principles of judgment” or schemas that do the job for the infant.

What fundamental formats for representing already exist to account for each of the basic elements of an experience as well as to tie them together? Taking each element separately, we have perceptual schemas (e.g., visual images) and we have conceptual schemas (e.g., symbols and words)... Piaget introduced the sensorimotor schema, which added motor acts and their coordination with sensory experiences as yet another fundamental way of representing experience. More recently, another basic form of human representation available to children has been added, namely, an invariant sequence of events that is represented as a single script...  (ibid., p. 82)

These four representational formats are necessary but not sufficient for the construction of the schema-of-being-with-another. Hence, Stern adds two more. The first is a placeholder for the temporal contour of affects, called “feeling shape” and the second has the intriguing name of a “protonarrative envelope”. It is needed in order to make sense of the five other formats in terms of meaning:

...the diverse events and feelings are tied together as necessary elements of a single unified happening that, at one of its higher levels, assumes a meaning.

The problem with this and other such solutions is how the meaning, even a very primitive one, slips in or gets assigned or is constructed from the pieces. There is a way out of this dilemma, and that is to assume that there is yet another fundamental way of representing human events, a sixth schema made up of “acts of meaning”, as Bruner and others have argued. (Ibid., p. 89)

As Jari Kaukua and Taneli Kukkonen pointed out in their article ”Sense perception and self-awareness” , an Aristotelian model of sense perception unavoidably necessitates an assumption of mental faculties that unite the disparate information conveyed by the senses. “What is it that allows us to go beyond mere sensations of red, fragrant, and smooth and to recognise an apple for an apple?” (Kaukua & Kukkonen, 2007, p. 96). Stern’s list of six representational formats summarises modern conceptions in cognitive developmental psychology, but as a solution to the puzzle it has ancient roots. I will get back to these in my next post.