Contents

"I'm the only President you've got." - Lyndon B. Johnson

"The Earth is the only center of the universe you've got." - a typical reply to Copernicus

1. Introduction

Connectionism is dead. At the very least, Jerry Fodor and Zenon Pylyshyn (1988) claim that connectionism as a cognitive architecture is utterly fruitless and no more time should be wasted trying to save it. It can technically still be used as an implementation model, but even then it is not all that beneficial of a theory.

I disagree. I believe that connectionism is not only alive and thriving but is beginning to suffocate classicism as a new direction for artificial intelligence research. For the purposes of this paper, however, I will merely attempt to show that despite Fodor and Pylyshyn's critique, connectionism is a viable alternative with quite a bit of promise.

First off, let's see just what is at issue here. Artificial intelligence has been an area of intense interest for psychologists, computer scientists, and philosophers. With the emergence of the unified discipline of cognitive science, these fields have been able to interchange ideas far more than ever in the past. The growth of knowledge in this field has been staggering over the past several decades alone.

Not only has significant progress been made towards intelligent machines, but we have also learned much about our own mind in the process. Models of human cognition have come and gone, but we do seem to be narrowing the range. Parallel to this have been the varying models of potential computer cognition. With a substantial amount of detail variation and some occasional, but short-lived, alternatives, a primary model for both human and computer thought has been developed.

The dominant models in each field are remarkably, and I doubt very coincidentally, quite similar and, moreover, are easily compatible with each other. The primary style of computing that nearly all artificial intelligence research uses is the von Neuman style. The most relevant point is that it is a serial, symbolic style; a particular form of the more generalized Turing Machine. Although this is a very broad category, only a general understanding of it is necessary for my purposes.

Whereas von Neuman and Turing Machine models pretty much dominated computer science since its onset, models of human cognition were less consistent. However, for approximately the past 20 years, mild variations of Fodor's Language of Thought model have reigned (1975). I will explain it in detail in the next section, but suffice it to say, it is quite serial and symbolic as well. Both the computer and the cognitive models deal with a series of processes over discrete symbolic representations.

There have always been an upstart or two to challenge each view, usually with little or no success however. Recently, a new breakthrough within computer science has begun a revolution in nearly all the disciplines of cognitive science. It has been called a clear example of a Kuhnian paradigm shift for each of those fields. It is connectionism.

There were ancestors to connectionism in both computer science and philosophy of mind dating back to Kant, but, as Fodor and Pylyshyn so gladly point out, they failed. Why should this new version of an old theory rear its ugly head again and still be taken seriously? It should be because now computer scientists have developed sophisticated models that are able to empirically prove what were merely wild speculations and highly abstract arguments. Previously, there were only guesses and hypotheses of what these models could do, but now we can directly see. Even though computer scientists are now able to directly test some philosophical ideas, the greatest advances have been entirely new discoveries made with the connectionist networks that have surprised and inspired philosophers.

Later in the paper, I will give a better explanation of the broad field of connectionism, but first a quick glimpse for those new to the paradigm. Connectionist models are not serial in nature, but primarily parallel networks of highly interconnected nodes. These nodes, which can be vast in number, are very simple processors, each often just summing its input, perhaps performing a simple calculation, and then giving the appropriate output. A very different picture than the serial models where a single, complex processor does all of the work.

To better understand the differences between the two let's compare how they would each handle a certain operation - data retrieval for instance. Rumelhart and McClelland developed a network that stores and retrieves information, in this case information about the various members of the Jets and Sharks gangs (1986). A serial machine would best store the information in a list-like database as shown in Figure 1. The computer would run a specific retrieval program that would scan certain rows and columns searching for keywords to get the desired information.

The network created by Rumelhart and McClelland works much differently. A scaled down version (for clarity) is diagrammed in Figure 2. Here the nodes are divided into groups. The groups are: name, age, "profession", marital status, gang affiliation, and education. These are connected by way of hidden nodes. These nodes are considered "hidden" because they do not receive direct input or have any sort of meaningful output. They are useful for connecting the different groups and, in some other connectionist network designs (as I will discuss later) they are essential for computations.

To retrieve information about a certain feature, you merely activate that node. For example, you can activate the Ralph node and Ralph's characteristics will also be activated: Junior High, Jet, single, Pusher, and 30's. Since the activation is two-way, there is an added benefit that can be of more use in other networks but is illustrated here. Art is a single Jet pusher with a Junior High education but is in his 40's, so when each of those nodes are activated, activation trickles back to the Art node. It is decreased in level enough that it does not greatly affect the rest of the network, but is significant enough to be influenced by exactly how similar Ralph is to Art. This effect is essential to connectionist systems' ability to generalize as I will address towards the end of my paper. You can retrieve any sort of information from this network. You can find out not only which of the individuals are Jets but activating that node, but also due to the two-way activation and generalizing, you can find out which profession is most prevalent among them just by stimulating that single node. So it is quite apparent that connectionism and classical systems approach computation far differently.

Connectionism was originally inspired by neuroscience, but the analogy should not be taken too far, at least not presently. The ultimate goal of many connections is to reduce these nodes to electronic neurons and therefore bridge the gap between neuroscience and cognitive psychology. Others - in fact a great majority - are content with an inexact structural similarity between the neural level and the cognitive level. It is important to emphasize that connectionism is still in its infancy. The general foundation has only been laid in the past few years and future directions are still being fine-tuned.

The problem now arises, which model of human cognition is correct? It would appear that connectionist networks are incompatible with the Language of Thought model. This is precisely the main thrust of Fodor and Pylyshyn's critique. Two main options are open: a) prove that connectionism, or a related form of it, is indeed compatible with Language of Thought, or b) prove that the connectionist model of cognition is better than Language of Thought. The first option is not very attractive to connections, and the latter is forbidding to say the least. However, these are actually only the two ends of a continuum of possible positions. A third position, intermediate between the two, also exists, according to which, it is possible to form a new cognitive model that is at least as good as Language of Thought, if not better in some ways, while still preserving the some of the best aspects of it. I shall argue for this latter position.

Before we dive in, let's do some road-mapping for this paper. For starters, I will explain in greater detail the Language of Thought model and some of Fodor's main arguments in favor of it. I will then summarize Fodor and Pylyshyn critique of connectionism presenting their arguments as to why connectionism cannot support a Language of Thought. The primary issues are the way in which connectionism handles both mental representations and mental processes.

2.1. General Background for LOT

2.3.1. Argument from Methodology

Fodor believes that this violates a fundamental rule of scientific inference that wants to postulate the least number of "accidents".

Principle P: Suppose there is a kind of event c1 of which the normal effect is a kind of e1; and a kind of event c2 of which the normal effect is a kind of event e2; and a kind of event c3 of which the normal effect is a complex event e1 & e2. Viz.:
c1 => e1
c2 => e2
c3 => e1 & e2
Then, ceteris paribus, it is reasonable to infer that c3 is a complex event whose constituents include c1 and c2. (Fodor p.141, 1987).

Both Fodor (1987) and Kim Sterelny (1991) argue that mental representations are necessary. Sterelny cites problem solving in support of this. People use a hypothesis/test method for problem solving (this includes the full range from abstract psychology experiments in the area to mundane analysis of what is happening in one's environment). This methodology requires some form of representation in which to formulate hypotheses to test. If all of our hypothesis testing had to be done externally rather than mentally, then humans would most likely not have evolved quite as far as we have. Mental trial-and-error has definite survival advantages over external trial-and-error.

Fodor sticks with the sentence comprehension examples and argues that mental processes are defined by the mental representations over which they function (1987). Mental processes, on his account, are a series of mental representations and a transition from one to another is completed by performing operations on the representations. For example, Wh- questions such as "Who is John?" can be easily converted to "John is who?" by performing simple operations on a mental representation that can be seen as a basic parsing tree. Aunty says a somewhat similar thing with her "Unknown Neurological Mechanism" that converts the expression before the listener hears it. Aunty's explanation is just hand-waving, whereas Fodor offers an explanation for how this occurs - a language of thought.

Systematicity is illustrated by the ability to move parts of complex structures to create new ones. Being able to have the mental state John loves Michael also means that you necessarily can have the thought Michael loves John. It is quite apparent that thought is systematic. Parts can be rearranged, removed, and added according to a set of rules to create novel mental states.

AIR must rely on a phrase-book type of system. Since AIR's mental states are indivisible they cannot be systematic. All that Aunty can do is memorize a massive list of sentences, just like a non-native speaker reading from a large phrase-book. However this phrase book would have to be astronomically expansive from the beginning unless you can somehow add new thoughts to it. With thoughts being whole objects, a theory explaining the addition of new ones has its work cut out for it.

With all thoughts coming from a massive phrase book, it is also possible, and even likely, that the situation of being able to think John loves Michael but being utterly unable, no matter how hard you tried, to ever think Michael loves John, could arise. This situation not only does not seem to occur in any moderately complex organisms, but implausible to even conceive of it occurring. This position is not seriously supported by anyone that I have come across.

Language of thought models with their combinatorial semantics can explain the systematicity of thought just as it does for public languages. However, in a (somewhat) noble move, Fodor points a flaw in this reasoning. The argument that thought has combinatorial structure because of systematicity goes along as follows:

(a) There's a certain property [systematicity] that linguistic capacities have in virtue of the fact that natural languages have a combinatorial semantics.
(b) Thought has this property too.
(c) So thought too must have a combinatorial semantics. (Fodor p.148, 1987)

As Fodor puts it, "one man's affirming the consequent is another man's inference to the best explanation" (Fodor p.149, 1987). Since Fodor is only trying to prove that LOT is better than his opponents, he merely has to show that it explains the facts better than the rest. He does not have to prove that it is necessarily true. To further show the power of affirming the consequent to get a conditionally true conclusion, all of science is based completely on this logical fallacy (If my theory is true, then the evidence will come out this way. The evidence came out this way, so my theory is true.)

Compositionality is quite similar to systematicity and Fodor and Pylyshyn even say that "perhaps they should be viewed as aspects of a single phenomenon (1988, p.41)." Pretty much what systematicity is to mental processes, compositionality is to mental representations. It is the clearly defined structure that the mental processes operate over. A mental representation is compositional if and only if it is structured in such a way that the information in it is accessible to mental processes. So the systematicity of mental processes depends on the compositionality of its representations. The compositional structure of the representations depends in turn on combinatorial syntax. The arguments surrounding systematicity apply directly to compositionality, at least for now.

3.1. Fodor and Pylyshyn's Version of Connectionism

Fodor and Pylyshyn define connectionist systems as a large network of nodes that sum all of their input then and output some value according to a certain simple function. Rather than having some single, complex processor to carry out the mental functions, there are a vast number of very simple processors that together carry out the mental functions. This is the fundamental difference between connectionism and classical systems. This apparently small shift in processing style actually leads to some very large differences on key issues.

It will be important to note that Fodor and Pylyshyn discuss a localist version of connectionism. This means that semantics is assigned directly to individual nodes as opposed to being distributed over a large number of nodes. So in a localist network, each node would have a specifically assigned meaning, whereas in a distributed network groups of nodes would be assigned meaning.

Fodor and Pylyshyn see this shift to distributed representations as irrelevant to the issue at hand. In their eyes, distributed representations have no advantage over localist models. They are both still fundamentally connectionist and consequently their arguments should apply similarly. This point is clearly made in their first footnote:

The difference between Connectionist networks in which the state of a single node encodes properties of the world (i.e., the so-called 'localist' networks) and ones in which the pattern of states of an entire population of units does the encoding (the so-called 'distributed' representation networks) is considered to be important by many people working on Connectionist models. Although Connections debate the relative merits of localist (or 'compact') versus distributed representations, the distinction will usually be of little consequence for our purposes, for reasons we give later. (Fodor & Pylyshyn, 1988 p.5)

Fodor and Pylyshyn then point out that the only relevant "primitive relation" in connectionism is the casual relation between nodes (i.e., such and such node affects these other nodes in such and such a way). Classical systems not only recognize causal relations but also "a range of structural relations, of which constituency is paradigmatic" (Fodor and Pylyshyn, 1988, p.12). This fact, that classical systems have constituency and other structural properties and connectionist systems do not, is the focal point of their argument: after all, constituency is where Aunty failed.

The model used by Fodor and Pylyshyn as an allegedly typical case of connectionism is reprinted as Figure 3. It is a simple logical reference machine, a paradigmatic case of language of thought. It is a network in which the complex predicate A & B can be broken down to either or both of its constituents, A and B. Understandably, in reality such a simple network would only be part of a larger machine, but Fodor and Pylyshyn use it as a connectionist network stripped to its essence, with no frills at all. If this "purified" connectionist system can support a language of thought, then there's no problem at all. However, if it cannot, then connectionism is just out of luck.

Fodor and Pylyshyn's logical inference network does not possess any constituent structure, or any internal structure whatsoever. Each node is atomic. Even node 1, which represents the molecular statement A & B, is itself atomic. It is just a single site of activation and lacks structure of any kind, let alone compositional structure.

One might be tempted to say that since node 1 represents A & B, a statement that obviously has constituent structure, this representation can then be split into its individual parts. However, this move misinterprets the nature of this connectionist system. The fault lies in the fact that the characteristics of the labels (i.e. compositionality) are being attributed to the representations (which have no internal structure). The logical statements A, B, A & B are all node labels. These labels are conventions attached to them by the programmers to make the network's activity meaningful.

The representations themselves are merely the nodes themselves. Each representation is a simple yes/no, on/off, 1/0 level of activation. Since this network is a localist one, each node is by definition a representation. With Fodor and Pylyshyn's formulation of it, as with most (but not all) localist nets, there are no other levels of representation. There is only the activation of individual nodes.

It is possible to perform operations that end up exhibiting systematic rules, as our logical inference device does. However, htese are not intrinsic operations. Again, the particular labels are the source of systematicity and not the representations themselves.

The labels of the system play no causal role whatsoever. The fact that node 1, when activated, tends to activate node 2 and node 3, is the only causal relation present. The particular node labels are utterly irrelevant. We could relabel node 1 'Bill the Clown', node 2 'Bill', and node 3 'clown'. It would appear to be systematic still. But what about 'Bill the Clown', 'elephant', and lampshade' being the respective node labels? 'Elephant' and 'lampshade' are not constituents of 'Bill the Clown', yet the causal relations among the nodes remains unchanged. To be anthropomorphic, the computational system couldn't care less what labels we humans put onto it; it will compute exactly the same way. This is another obvious point that Fodor and Pylyshyn elaborate.

Fodor and Pylyshyn do not completely nail the lid shut on connectionism though. They are kind enough to present the only alternatives they see open to die hard connections:

1) Try to show that unstructured mental representations are the correct model for cognitive processes.
2) Rely on structured mental representations but continue to use an associational account of mental processes.
3) Use connectionism only as an implementation theory for a classical architecture.
4) Give up on networks as model for cognitive processes in general and only use for certain less cognitive mental processes (i.e. perception).

4.1. Chalmers' "Simple Refutation"

Fodor and Pylyshyn argue that a connectionist network cannot support a language of thought. Their logical inference example shows a case where this is obviously true. So their conclusion is that connectionism fails as a form of cognitive architecture. They explicitly offer the possibility, however, that connectionism is a viable option as an implementational architecture. More simply stated, Fodor and Pylyshyn believe that connectionism has no place in philosophy and should be confined solely to the computer science domain. This may very well be what they set out to prove, but they did not succeed.

Their arguments, as they stand, prove that no connectionist network can support a language of thought. It is not possible to develop any structure formental representations and processes within a connectionist network. Fodor and Pylyshyn argue that due to its associational basis, it is impossible to form any structured mental process and representations in a connectionist system. I emphasize no connectionist network because ones that are attempted implementations of classical cognitive architectures are still fundamentally connectionist networks. Therefore, it is impossible to implement a classical cognitive architecture (and therefore a language of thought) on a fundamentally connectionist network. However, Fodor and Pylyshyn then claim that they proved that no connectionist systems can instantiate a language of thought except for implementations of classical systems. So what their argument proves goes far beyond what they conclude from it, no tto mention that it also goes beyond what is generally accepted.

The analogy that Chalmers draws is his mad scientist who proves that Earth is the only inhabited planet in the universe. First this scientist runs through an a priori proof that the proper biochemical reactions for life to ever evolve cannot occur. It is necessarily impossible for life to exist. However, life obviously exists on Earth. So the Earth is the only planet with life on it! If you are thinking ad hoc, you are right. This maneuver is the one that Fodor and Pylyshyn subtly use.

This is a big problem for Fodor and Pylyshyn. They could say that they were wrong in their conclusion and that implementations of classical architectures are not an option either. However, connectionist implementations of classical architectures that work just as well as standard implementations are entirely possible. As a computational system, both connectionist networks and von Neuman/Turing Machines are equivalent. Both can compute anything computable. Consequently, Fodor and Pylyshyn are out of luck there.

It is not the fact that the example is so simple and small. The fallacy lies in its purely localist nature. Plenty of useful and well-documented connectionist networks use atomic, localist representations. However, none of them are attempts at modelling cognition. By analogy, I could very well look quite in-depth into this word processing program that I am currently using and claim that it cannot be even remotely cognitive and conclude that no von Neuman computer (or Turing machine, for that matter) can be cognitive.

Sounds quite ridiculous, doesn't it? It parallels Fodor and Pylyshyn's argument exactly. They take an extreme example of connectionism that no true connectionist would support as a model of cognition, and prove that it cannot support a language of thought. Since all other aspects of connectionism are "confusions and irrelevancies" (Fodor and Pylyshyn 1988, p. 6), all connectionist networks are incapable of supporting a language of thought.

Even after the substantial literature on the subject, Fodor still fails to see its importance. He still claims that "connections clearly assume that there are elementary mental representations (typically labeled nodes), and that these have both semantic and causal properties." (1994, p.96) However this is far from being assumed at all. In fact it is the very thing connections argue against.

When one asks what is the deepest philosophical commitment of the connectionist movement, the answer is surely this: the rejection of the atomic symbol as the bearer of meaning. Connections feel that atomic tokens simply do not carry enough information with them to be useful in modeling human cognition. Rather, distributed, subdivisible, malleable representations are the cornerstone of the connectionist endeavor. For this reason, localist networks are regarded by many connections as not really connectionist at all. These networks employ precisely the traditional notion of atomic symbols, with a new twist added by connecting them by associative links. (Chalmers, 1990b, p. 343)

Fodor and Pylyshyn's model though, is only half-way to the true models of cognitive architecture that is supported by connectionists. This example is actually atomic symbols connected by associations. As Chalmers puts it, it is "symbolic AI with soft constraints" (Chalmers, 1990b p.343). It is a blend of the two positions using the representation style of one and a processing style of the other. On top of this, these two characteristics that Fodor and Pylyshyn decide to blend are the weakest of each. In the classic view, atomic symbols are pretty powerless unless they are combined using systematic rules. In the connectionist views, the associations between representations are also quite powerless except for the fact that their representations are distributed ones with a great deal of internal structure. Clearly, Fodor and Pylyshyn's example is not the standard for connectionism.

With distributed representation, the basic level of representation is not a single node, but the pattern of activation over a group of nodes. This is a far cry from the atomic tokens within classical AI, and far more difficult to understand. Atomic tokens are easily understood, most likely due to parallel with variables in computer programs and simple algebra. Understanding distributed representations requires a massive theoretical shift.

For one thing, the nodes themselves are not exactly the representation. They are merely the objects used to instantiate the representations. Consequently, a given set of nodes within a network is able to instantiate a large variety of representations. To give a simplified example, say that we have a connectionist network within which a group of 5 nodes are responsible for representations. The rest of the network can simply be a camera for input, and a speech box that outputs "Hey, that's a ______!". Now particular patterns of activation will correspond to particular representations. For example, 01101 could represent cat, 01100 could represent dog, and 10011 rock. Whenever those 5 nodes match one of those activation patterns, they are said to represent that thing. The 5 nodes could represent any of them or none of them. It's the particular patterns that are the representations.

It is quite a change in representational structure to say the least. All that is really necessary to understand at the beginning is that representations and nodes are not assigned on a one-to-one basis as in the Fodor and Pylyshyn example. Instead, the activation pattern over a group of nodes is the representation.

It is also important to note that these representations are complex without being molecular. According to classical AI, there are only atomic and molecular representations. Atomic representations are indivisible, simple pieces. One piece with no internal structure. Molecular representations are directly composed of atomic or other molecular representations. You can remove a part and it will merely be the same representation missing that piece. For example, a representation may go from "Ralph ate the meat that was spoiled." to "Ralph ate the meat."

Another very large benefit of distributed representations is their ability to generalize. Between the training methods and the very nature of connectionist networks, systematic rules naturally emerge, and the network will generalize beyond its training set. Localist connectionist systems cannot do this at all. It is also interesting to note that while classical models are able to do this, they must be specially modified to do so. With distributed representational networks, this generalizing is automatic.

This assumption is very wrong. In even the earliest connectionist research, it was clear that having representations distributed over many nodes rather than being on a one-to-one relation allowed for far more flexibility and possible characteristics. Apparently, all distributed representations cannot be merely reduced to localist ones. There is a relevant theoretical difference involved, not just an implementational one.

5.1. Brands of Compositionality

However, compositionality itself is a bit broader than this. Constituency is just one method to bring about compositionality. The actual definition of compositionality that I will use is:

The ability to create a representation of a structured object in such a way as to preserve the original structure in a usable form.

I believe that compositionality can take other forms. Constituent structure is the most widely accepted and perhaps even paradigmatic version, but I feel that it is not a necessary one. There is a more general, catch-all category that involves any method of representing a structured object in a way different from Fodor's concatenative compositionality. Tim van Gelder had called this alternative "functional compositionality". A representation of a structured object is created with all relevant information accessible, but not literally present.

The reason that Fodor and others do not believe compositionality can exist without the constituent parts being literally present is that they confuse where the structure must be. Functional compositionality is representation of structure rather than a structured representation. All of the information concerning the original structure is accessible.

All that is needed is functional structure. For a representation to be useful and compositional, it must merely contain the information concerning the original structure. Fodor did not realize this. He thought that the only way to do this was to have the representation directly mirror the original structure. There is no reason for this other than it has worked well so far.

Connectionism's problem is with concatenative compositionality. Using the concatenative method, to represent an object of arbitrary size, you would need an arbitrary amount of space to represent it in. This is not feasible in a connectionist network. The number of nodes is normally specified at the onset, and all representations must fit within a given number of nodes. Programming a connectionist network to change its size as necessary is an extremely challenging task at best, and perhaps even impossible to properly train at worst.

With each of these models, there is an input which is to be represented. This input is then encoded into a distributed representation. It is also possible to output the original pattern of input given only the distributed representation form of it. They all have their own method for doing this, but they each perform the same basic function - transform the input into a distributed representation. The differences between each of these models is irrelevant to our discussions.

6.1. Groundwork

This is easier said than done, though. Since most work has been focused on connectionist compositionality, and that has only been achieved recently, very little has been done in the way of systematicity. Most networks will use a distributed representation until it is necessary to perform any systematic processes on it, then convert it over to the more traditional version.

This leaves one to wonder, perhaps systematicity only works with concatenative compositionality. Even though all of the same information is contained with functional compositionality, the particular structure of the concatenative version is necessary. Since the information in the original structure is merely recoverable within a distributed representation rather than being literally present, there is some plausibility to this notion.

I, surprisingly enough, disagree. Functional compositionality is all that is necessary for systematicity, on one condition. The constituent parts of the distributed representation must be individually recoverable and analyzable. In other words, you do not have to decode the entire thing just to get a single piece of it. You can remove a piece and leave the rest of it in a distributed format. If the representation must be transformed back to its original state entirely to access any bit of it, then storing it as a distributed representation is unnecessary. It has no advantage over classical systems and is most likely inferior.

Pollack was able to make a version of his RAAM architecture that can have simple syntactic transformations performed upon it (Chalmers, 1990a). In this case, the network is able to create distributed representations of sentences, and then transfer them from the active to passive voice (or vice versa). This transformation is done entirely on the distributed representation, it is never restored back to its original state.

The network works best at a 3N-N-3N set-up as diagramed in Figure 4. The 3N is the input layer where the sentence, in its original state, is first fed into the network. The middle layer is the hidden layer where the distributed representations are instantiated. The third layer is the output layer for the transformed sentence. Within this simple architecture, is a very basic version of systematicity.

In addition to this, the system is very reliable, achieving 100% accuracy in its transformations. This accuracy includes sentences (within its vocabulary) that it was never trained on. Even with what it generalizes to, it is perfectly accurate.

7.1. Differing Cognitive Models: The Hard and Soft Approaches

Basically, it seems that Fodor and Pylyshyn were wrong and that connectionism is a possibility for cognitive modeling. Now we are left with their last argument, mostly presented by Fodor himself. He claims that the ultimate argument for the classical view of language of thought is that it is the only viable alternative around. A few others may try to pop up their heads briefly, but then don't last. Therefore, since the classical view is the only one around, it would seem to be the winner for now.

Through showing that connectionism can possess compositionality and systematicity, the two criteria set down by Fodor and Pylyshyn themselves, connectionism has become a viable alternative. It seems that the classical view is no longer the only option around. So which will it be then?

Smolensky (1987) addressed this issue, although from a broader perspective. He refers to the Paradox of Cognition. He claims that there are two distinct approaches to cognitive modeling: (i) the hard approach - the mind seems to be characterized by rules, and (ii) the soft approach - rules alone do not seem to be enough. This is the paradox. On the one hand it seems that the mind follows clear rules, on the other hand, the rules only seem to be able to explain things to a point. The matter of whether or not the mind is only a set of rules seems paradoxical. This is parallel to the classical/connectionist debate since the classical endeavor follows the hard approach and the connectionist the soft.

In actuality the answer most likely lies somewhere in the middle: the majority of the mind is characterized by rules, but not all of it is. Some parts lay below rules. By following both approaches at the same time, it should be much easier to find that spot. Also, the problems of one are compensated by the benefits of the other. They complement each other quite well.

Unfortunately, the vast majority of the research has been done following the hard/classical view, and very little has been done on the soft approach. The majority of the work done on the soft approach has only been in recent years as well. So there does seem to be an imbalance here, an imbalance that Fodor and Pylyshyn wish to preserve.

7.2.1. Those Overlooked Cognitive Abilities

I feel that language is an enormous part of thinking and helps humans to organize our thoughts better, but I do not see how something that has all the same brain power as us, but no language, cannot be intelligent and possess any kind of conscious thought. Supporters of this view have taken linguistic ability (which is allegedly unique to humans) and becuse of its human uniqueness claimed that it the most important aspect of intelligence and even consciousness.

Whereas language is very important, I have not seen any conclusive evidence to support the notion that it is the basis for all intelligence. There are many, many mental processes that are not even remotely linguistic in nature. A partial list by Chalmers of mental processes in which compositionality is unimportant, includes: "perception, categorization, motor control, memory, similarity judgements, associations, and attention" (Chalmers 1990b, p.347).

We seem to be left with two options, classical and connectionist AI. I think that the final answer will be a blend of both (even if it is skewed more towards connectionism). At the very least, connectionism will offer a brand new perspective on the issues. However, connectionism seem to be faring better than that, and may someday be an equal alternative to classical AI, or perhaps even take over the lead that young students will argue against since it would then be "establishment".

It would seem that the classical dominance is at an end. Connectionism is carving out its own place, even if it is with the "lesser" mental functions. Contrary to Fodor and Pylyshyn, it is a force that demands attention, not to mention that the amount of controversy stirred up is revitalizing the field. Many people are dissatisfied with the recent lack of progress in classical AI. Far from being dead, connectionism is alive and thriving extremely well.

Chalmers, D. (1990a). Syntactic transformations on distributed representations. Connection Science, 2: 53-62.

Chalmers, D. (1990b). Why Fodor and Pylyshyn were wrong: The simplest refutation. In Proceedings of the Twelfth Annual Conference of the Cognitive Science Society, pp. 340-347.

Elman, J. (1990). Structured representations and connectionist models. In Proceedings of the Twelfth Annual Conference of the Cognitive Science Society, pp.17-23.

Fodor, J. (1975). Language of Thought. Scranton, PA: Crowell.

Fodor, J. (1987). Psychosemantics: The Problem of Meaning in the Philosophy of Mind. Cambridge, MA: MIT Press.

Fodor, J. (1990). Theory of Content and Other Essays. Cambridge, MA: MIT Press.

Fodor, J. (1994). Concepts: A potboiler. Cognition, 50: 95-113.

Fodor, J. & Pylyshyn, Z. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28: 3-71.

Pollack, J. (1988). Recursive auto-associative memory: Devising compositional distributed representations. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society, pp. 33-39.

Polack, J. (1990). Recursive distributed representations. Artificial Intelligence, 46: 77-105.

Rumelhart, D., McClelland, J., and the PDP Research Group. (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition Vols. 1-2. Cambridge, MA: MIT Press.

Searle, J. (1980). Minds, brains, and programs. Behavorial and Brain Science, 3: 417-458.

Smolensky, P. (1987). The constituent structure of connectionist mental states: A reply to Fodor and Pylyshyn. Southern Journal of Philosophy, 26: 137-163.

Sterelney, K. (1991). The Representational Theory of Mind: An Introduction. Cambridge, MA: Blackwell.

Van Gelder, T. (199?). Compositionality and the explanation of cognitive process.

Van Gelder, T. (1990). Compositionality: A connectionist variation on a theme. Cognitive Science, 14: 355-384.