Conceptual research needs a lab

The best way to understand the meaning of concepts is to trace their origin and history. However, even this method does not help us generate alternative viewpoints to the theoretical difficulties and contradictions we have encountered while tracing the development of a concept. The history of representation is a powerful example. It has experienced many transformations since the Aristotelian formulation of forms that are taken in without matter (in-formation). Yet the theoretical contradictions that it creates have remained unsolved in cognitive science to this day.

So how about empirical research? We expect that putting the concept into a rigorous experimental test will lead us forward. In the case of representations, no empirical research (as Cummins has concluded) has been able to account for the gap between sensory input and meaningful representations that Fodor postulated in his Language of Thought hypothesis. 

Ptolemaic astronomy illustrates this problem. No empirical observation that could be made by naked eye undermined the basic theory. Aristarchus of Samos had proposed a heliocentric alternative in the third century BCE, but the geocentric understanding prevailed well over a millennium. 

In a way, we are in a hermeneutic impasse. Our theories dictate what we expect to observe and contradictory observations tend to remain meaningless and neglected.

Yet, laboratories are useful because they help generate varying series of observations. The repeated patterns in this continuous variation may, eventually, show the limits of our current explanatory concepts unless we interpret them to be meaningless noise. Thus, laboratories have a potential, even if the power of our ways of thinking may, initially, mute the alternative signs that are experimentally generated.

My personal experience of exploring the concept of sign makes a good case. Having become excited by Vygotsky’s idea of semiotic mediation as the explanatory principle of mental processes, I wanted to explore its potential for clinical psychology. Psychotherapy was my laboratory. 

In the early 1970’s we formed a study group that tried the idea in individual counseling and clinical assessment. At the time, we were completely ignorant of the conceptual gap that had developed within the Vygotskyan tradition during the Stalin era. 

Vygotsky’s understending of semiotic mediation had been discarded in favor of Leontiev’s activity theory. We adopted his tripartite conceptualization of activity, action, and operation as our point of departure. 

Afer a few years, it became clear that Leontiev's conceptualization did not help understanding the interactive events that we encountered in the consulting rooms. They were far too general. 

Leontiev’s concepts of object and tool had been developed in studying distributed work activities, but we were involved in verbal exchange with the client whose “objects” were accounts of personal experiences. Much later, I perceived that Leontiev's concept of activity included mediation, but he had no clear concept of semiotic mediation. Consequently, instrumental and communicative activities remained separate fields of research, as Radzhikowskii noticed in his 1984 paper. 

What was even more disappointing was Leontiev's complete inability to account for internal actions. He only stated that they share the structure of mediated instrumental actions. 

So, in order to understand  we went to train ourselves in psycho-dynamic and cognitive therapies that appeared to have much more adequate conceptual tools. 

The psychotherapy lab quickly demonstrated that activity theory did not work, but neither it did point out any alternative that might have involved semiotic mediation. Thus, we had to give up our hopes and go back to the existing traditions in the field.

An adequate concept of sign that would serve both communication and psychic activity called for theoretical work.

The Aristotelian shipwreck

The Aristotelian shipwreck, buried in Kantian sand dunes, reappears unexpectedly here and there. Some weeks ago, Harry Procter sent me Mats Bergman's paper “Representationism and Presentationism (2007), which is a valuable account of Peirce’s struggle to unite his semiotic understanding of the consciousness with the doctrine of immediate perception. The article, again, shows how useful is a careful  developmental approach that compares and contrasts Peirce’s early and mature views of perception.

In my reading of the article, Peirce did not manage to reach his aim for two reasons. His early theoretical maxim that everything in the consciousness is semiotic presupposes a unified conception of sign that unites the elementary modes of perception with the “higher” processes in consciousness, such as remembering, imagining, and thinking. Peirce never reached such a conception, although he experimented with different versions of the triadic conception of object, representamen, and interpretant.  His later solution of categorising signs according to the ontological categories of firstness, secondness, and thirdness blurred the picture and created problems of analysing consciousness in terms of the relationships between iconic, indexical, and symbolic signs.

The other problem that I sensed when reading Bergman concerns Peirce’s way of approaching perception from the traditional Aristotelian angle.  The following citation that Bergman quotes  in full illustrates it very well:

. . .the object of a sign, that to which it, virtually at least, professes to be applicable, can itself be only a sign. For example, the object of an ordinary proposition is [a] generalization from a group of perceptual facts. It represents those facts. These perceptual facts are themselves abstract representatives, though we know not precisely what intermediaries, of the percepts themselves; and these are themselves viewed, and are,—if the judgment has any truth,—representations, primarily of impressions of sense, ultimately of a dark underlying something, which cannot be specified without its manifesting itself as a sign of something below. There is, we think, and reasonably think, a limit to this, an ultimate reality like a zero of temperature. But in the nature of things, it can only be approached, it can only be represented. The immediate object which any sign seeks to represent is itself a sign. (MS 599:36–37 [c. 1902]; cf. NEM 4:309–310 [c. 1894?])

The receptive understanding of perception that Aristotle cemented seems here to be modified by a Kantian cautiousness that prevents Peirce to name “the thing in itself” as the starting point. Instead, he refers to “a dark underlying something”. Nevertheless, the following steps seem to come close to the Medieval debates on abstractive cognition, or Roger Bacon’s doctrine of De multiplication specierum. The path from “impressions of sense”, “percepts”, and “perceptual facts” to  “ordinary propositions”  is a faithful reproduction of the Aristotelian theory of cognition.

The inherent dynamics of this account brings about problems that lead to “a multiplication” of the steps that seem to be needed. Bergman’s paper illustrates this nicely by Peirce’s conceptual division between “percept” and “percipuum”. As instances of secondness, percepts only convey the “brute force” of their instigators. Percipuums are a bit like Fodor’s symbolic representations. Still within the perceptual domain, they involve a kind of identifying judgments that make them appearances of something. “But the moment we fix our minds upon it and think the least thing about the percept, it is the perceptual judgment that tells us what we so ‘perceive’.” (CP 7.643 [c. 1903])

The recent commentaries that try to bring some coherence in Peirce’s prolific and contradictory thought seem, unwittingly, to reproduce the tendency of introducing more steps and substages in order to cope with the Aristotelian fallacy. Bergman  presents Carl Hausman’s suggestion to distinguish between percept₁ and percept₂ . The former denotes the dynamic object, or the immediate impact of the brute force, while the second incorporates a cognitive generalization that prepares the percept suitable for becoming the immediate object of thought, or the percipuum.

At this stage, I cannot help concluding that Peirce was, as we still mostly are, too closely tied to the Aristotelian understanding of cognition in order to unite his semiotic maxim of the consciousness with his accounts of perception. Jerry Fodor’s much more recent attempt to fill the rift between propositional attitudes and immediate sense impressions is another reproduction of the ancient problem that remains unsolved despite terminological modifications.

Cognitive models of reasoning

Approaches that endorse the causal nature of representation have difficulties in explaining the selectivity that any living organism shows in its behaviour.  Not every represented feature in the environment is relevant. The organism’s (construed) goal seems to affect the ways by it handles incoming information. Worse still, such information seems to be interpreted in relation to the goal and the organism’s experiences of the conditions for attaining it. This is a long known problem. In cognitive science philosophy it is discussed as the relevance problem.  Richard Samuels has presented a recent summary of the issue and the various attempts at solving it.

Reasoning seems to be the pivotal cognitive process that transforms incoming information into action plans. In order to provide a naturalistic account of cognition, this mental activity should be modelled in terms of some algorithmic procedures. As Fodor suspected in “The mind doesn’t work that way”, and Samuels concludes in 2010, neither the classical computational theory of cognition nor the connectionist approaches have succeeded. He enlists various reasons, but to me the most interesting concerns the ways representations are understood.

Fodor opted for symbolic representations (as the vehicles for the language of thought) very early on, because the resemblance theory of representations  was completely inadequate to account for “propositional attitudes”, like beliefs or wishes about some state of affair. This conception generated the “functional role semantics” with all sorts of paradoxes and Fodor is both an honest man and enough sharp-eyed to admit the difficulties, which many of his followers (Like Susan Schneider and Richard Samuels) have not wanted to do. 

There is a marvellous debate between Steven Pinker and Jerry Fodor in 2005 about how to understand the workings of the mind. Fodor’s “The mind doesn’t work that way” was a critique of the classical computational approach, stimulated by Pinker’s book “How the mind works” (1997). It took Pinker five years to defend his evolutionary account of cognitive processes in the paper published in Mind and Language. You can retrieve it at Pinker’s home page. Fodor’s response in the same issue is, again, most exhilarating but it is sadly locked up in Wiley’s archives from which you cannot get it for free unless you have an institutional access. 

First, Fodor shows how the syntactic and “local” meaning of symbolic representations inevitably cause problems. He also shows how cognitive science tries to account for these by the massive modularity models and by calling evolutionary accounts to help (as Pinker does). Fodor's conclusion is very straightforward: “So how does the mind work? I don’t know. You don’t know. Pinker doesn’t know. And, I rather suspect, such is the current state of the art, that if God were to tell us, we wouldn’t understand him.”

Samuels admits five years later that Fodor’s view of the state of art in cognitive science is accurate. However, he does not want to explain it as the inherent (logical) flaws in the paradigm as Fodor did. He regards it mainly as an empirically solvable problem once the methodological difficulties in studying reasoning have been cleared away. Why is Samuels reluctant to admit the logical problem? He provides the answer, partly in a footnote. I am astonished how often important insights are buried in footnotes. Perhaps they are too dangerous to be inserted in the main line of argument. Here it is: “...if one seeks to provide a substantive general explanation of failures in cognitive science [to explain the problem of relevance], then the most obvious option is to reject the mechanistic assumption. [The footnote:  this is the position that Descartes (1637) famously adopts in the Discourse on method; and for surprisingly similar reasons. Roughly, he thinks that it is impossible to provide a mechanistic account of reason]. 

If our only alternative to computational,  connectionist, dynamic systems etc. models of cognitive processes is to revert back to Cartesian dualism, it is obvious that we must go on trying to fit these models to the mind and hope that improved ways of doing so will eventually get us out of the danger. I do not think we have to renounce a monistic account of the mind. We only have to renounce the representationist accounts of the “tokens in the brain”.

Farewell to Fodor?

I think I have now reached the point at which I can leave behind Jerry Fodor’s long-standing attempt at understanding the causal chain from representations to their symbolic expression. When I sensed the Aristotelian smell in the way Fodor wrote about perceptual modules, I was not yet aware that it was just an offspring of his grand project of overcoming the Aristotelian rift between incoming information and its transformation into language. It is really impressive regarding its scope, its time span, and volume. And yet, in the end, it seems to fail. The source of the failure lurks in the very beginning, in the conception of “Language of thought” , which Fodor launched in a book (1975) with this title. 

If we start by assuming that mental representations are language-like, we are free to look at the complex permutations, combinations and transformations that thinking seems to accomplish with such “symbolic representations”. But how does this coincide with the causal theory of mental representations?  

The most succinct formulation I have found by Fodor on this issue is from an exhilarating paper, jointly written with LePore in 1993, that shows the inconsistencies of inferential role semantics (endorsed by post-structuralists and constructionists of the time).

 “..what makes 'dog' mean dog is some sort of symbol-world connection; perhaps some sort of causal or informational or nomological connection between tokens of the expression and tokens of the animal.”

There are two important things to notice here. First, two sets of “tokens” [presumably in the brain] are postulated. One for symbols and the other for the object to which the symbol refers. The second thing  concerns the nature of the relationship between the two sets. I have got the impression that Fodor emphasised causal nature of the object and the token [object representation?] earlier in his writings. This is not addressed here. Instead, the relationship between the two token sets are characterised by three possible alternatives. So, symbol tokens inherit their meaning content from the [causally] tokened object representations. Such mental symbols can combined, permuted, etc., in the language of thought and, eventually, also be expressed in natural languages.

But how is this “causal or informational or nomological” relationship implemented (in the brain, presumably)? How is the first set of tokens transformed into the second set? Do we not have here exactly the problem of how meaning is attached to representations, that Daniel Stern’s homemade attempt in Motherhood Constellation addressed in a small scale? Fodor’s edifice is obviously much more extensive but nevertheless it runs into the same problems.

It interesting to notice that Fodor’s work is refuted from two sides. Both the representatives of semantic theories, like Jylkkä (2011), and strict cognitive science philosophers, as Cummins, show that his functionalist account entails forbidden amalgamations. On one hand, you cannot do away the inferential nature of symbols, concepts, and their verbal expressions. On the other hand, by assuming “symbolic representations” you destroy the idea of representation as it is used in cognitive science by overextending its meaning. It seems that both camps are justified in their arguments.

In order to accomplish a unitary chain from perception to expression, mediated by a central process, Fodor had to invent a number of bridging concepts. I felt awkward when encountering the terms “symbolic representation”, “language of thought” or LOT, and “tokens”. Reading commentaries and reviews did not help much. Consider the following summary of LOT:

“…cognitive processes are the causal sequencing of tokenings of symbols in the brain… LOT also claims that mental representations have a combinatorial syntax. A representational system has a combinatorial syntax just in case it employs a finite store of atomic representations that may be combined to form compound representations… Relatedly, LOT holds that symbols have a compositional semantics—the meaning of compound representations is a function of the meaning of the atomic symbols, together with the grammar. ” (Schneider & Katz, 2011).

Will I ever get out of such a conceptual mess? Or should I just leave it behind to rest in the peace of countless journal articles, hidden in the archives of publishing houses that charge 30 $ for an access.

It seems that different strands of cognitive science philosophy work with quite different conceptions of representation. The computational approaches to sense perception (illustrated by David Marr’s early research) understand representations as an end product of some basic algorithmic procedures on the sense data, like retinal images.  Symbolic representations, in turn, are products of algorithmic procedures with “atomic symbols”  - if I can make any sense of the above quotation.  

 
But the same problem that can be recognised in Fodor & LePore’s formulation is present. How do we get from the representations that are constructed (computed) out of sense data to the symbolic representations (“atomic symbols”)? We cannot, as Cummins and Roth argue in a recent article.  There is a rift between the causal sequences that operate on “incoming forms"  and the “symbol manipulating operations” performed by the “central processor”.  Perceptual representations are not symbolic.  In my next post, I will summarise this line of arguments. Then I shall explore a bit closer what Schneider and Katz mean by “atomic symbols”.