Negative Expertise by Minsky

Logical vs. Analogical or Symbolic vs Connectionist or Neat vs Scruffy by Minsky

Evolving Artificial Neural Networks by Yao

Chomsky and the Universal Grammar by Cruse

A Cognitive Architecture That Solves A Problem Stated by Minsky by Pomi et al

Formation of Neural Structures by Vaario et al

Evolution of Primate Cognition by Byrne

Minsky – Negative Expertise

The author looks at the nature of knowledge, by disputing the widely held belief that human knowledge is positive. Minsky claims that most knowledge, is in fact negative. He also claims that abstracts such as beauty, humour, pleasure and decision making are not the positives that they are generally believed to be – in fact they are the result of all processes shutting down, except the ones that are directly involved in that process (decision making, for example, can be seen as shutting down the processes that consider alternatives).

Another interesting point is the argument that to create a “creative” machine does not require the introduction of a chaotic or random generator of some kind. Rather, it requires pre-shaping the search space to reduce the number of useless attempts. Minsky believes that creativity and originality come from considering some unconventional alternatives that might not be discovered unless conventional censors are turned off.

The most important aspects of this paper were firstly, the argument supporting negative enforcement in learning. Not necessarily that for learning to occur there must be negative enforcement, but that the current view of positive enforcement is not accurate. Secondly, the notion that a positive system of knowledge forces the owner of that knowledge to generate and then test ideas (for example, heuristics in a computer game that generate game options). A negative  based system would be able to shape the search space for positive solutions more efficiently from the beginning of the search by avoiding areas that are not viable.

Go to Minsky’s “Negative Expertise”:

http://web.media.mit.edu/~minsky/minsky.html

Reference details for Minsky’s “Negative Expertise”:

Minsky – Logical vs. Analogical or Symbolic vs Connectionist or Neat vs Scruffy

This article is Minsky’s opinions on the debate between symbolic and sub-symbolic Artificial Intelligence. He discusses the strengths and weaknesses of both types of systems. He believes that different types of problems need different representations and may require different problem solving techniques.

The strengths of symbolic AI are: they are expressive and have procedural versatility. The weaknesses of symbolic AI are: inflexible and specialised. They require very well-defined problems, and cannot function outside the scope of that problem.

The strengths of sub-symbolic AI are: they allow fuzziness and can be easily adapted. The weaknesses of sub-symbolic AI are: they are unable to perform many of the functions associated with higher level human thinking (such as goal based reasoning and causal analysis).

Minsky concludes by stating that the best way forward is by combining the two approaches. He concludes by outlining his idea of the future AI, where “mind sculpting” will be the norm. The most relevant aspect of this paper was the concise comparisons between symbolic and sub-symbolic AI. An interesting sideline, only briefly mentioned, was the section on connecting a number of networks that perform different functions with the aim of modelling more complex behaviours.

Go to Minsky’s “Logical vs. Analogical or Symbolic vs Connectionist or Neat vs Scruffy”:

http://web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html

Reference details for “Logical vs. Analogical or Symbolic vs Connectionist or Neat vs Scruffy”:

Yao – Evolving Artificial Neural Networks

The aim of this article was to explore different methods of combining evolutionary algorithms (EAs) and artificial neural networks (ANNs). The result of this analysis of both EAs and ANNs is to demonstrate that combinations of the two can result in much better systems than either alone. This article is mainly a literature review, which summarises theories and results of a large body of work in the area.

As a result of the purpose of the article being a literature review and not an addition to the field’s body of knowledge, no experiments or simulations have been carried out (or at least, none were reported in this article). The author discusses a number of areas including using EAs to evolve connection weights, architectures, learning rules and input features of ANNs. These are the most common areas of combinations between EAs and ANNS, although others are briefly mentioned.

The author only briefly mentions Fractal Representations of the architectures of ANNs, however he does give one or two references that should be followed up on. I think the major importance of this article was that it serves as a concise description of the field, and contains an enormous number of references that can direct further study into the field.

Go to Yao’s paper “Evolving Artificial Neural Networks”:

http://www.cs.bham.ac.uk/~xin/journal_papers.html

Reference details for Yao’s paper “Evolving Artificial Neural Networks”:

X. Yao, `Evolving artificial neural networks,' Proceedings of the IEEE, 87(9):1423-1447, September 1999.

Transfer function of a node in an Artificial Neural Network:

Function defining a second-order node in a network:

Learning rule is described by the function:

Don Cruse – Chomsky and the Universal Grammar

The aim of this article is to provide a critique of some of the major concepts conceptual beliefs that lie behind the philosophy of linguistics, in particular  Noam Chomsky’s work on the Universal Grammar (UG). The belief that has the most bearing on Chomsky’s work, is the question of what the main function of the brain is. According to this author, there are two very different view points on this subject. The first is that the brain is a thought generator which has been developed over billions of years through evolution (as in Darwinian theory). The second view point is that the primary function of the brain is as an “organ of perception”.

Noam Chomsky ascribed to the first view (materialism), which may have presented some difficulties in describing the mechanism of the UG. Chomsky believed that UG existed in everyone and that all human languages originated from it. However, the materialist point of view would indicate that the grammar was part of the physical workings of the brain and that it is a result of the evolution of the brain of many generations. To explain the apparent conflict in his beliefs, Chomsky used “mechanistic imagery”, describing UG as an intricate, partially wired up system. The author is not convinced that this argument is completely valid. His argument is based on the belief that using mechanistic imagery to prove materialistic assumptions (such as Chomsky’s for the UG), is an error in causal logic.

Go to Cruse’s “Chomsky and the Universal Grammar”:

http://www.southerncrossreview.org/9/chomsky.htm

E-review, downloaded 27/04/02.

Pomi and Mizraji – A Cognitive Architecture That Solves A Problem Stated by Minsky

The aim of this article was to design and build a cognitive architecture that was able to solve a problem titled by the authors as “Minsky’s problem”. This problem is basically the challenge of performing a good diagnosis based on multiple criteria (or partial clues) that arrive in successive time steps. The implementation of the architecture takes the form of a neural network made of distributed associative memories.

The cognitive architecture that the authors used consisted of three parts – an attribute-object associator (AOA), a working memory (WM) and an intersection filter (IF). Time is assumed to be a discrete variable. The AOA is a memory module that is used to associate attributes with objects. The objects are then sent to the IF, which determines the intersection between two objects. The IF also takes input from the WM (which is the output from the IF at the previous time step). The network is implemented as matrices where the coefficients represent synaptic weights.

The authors claim that this kind of task is the basis of medical diagnosis, and that the important aspect of this experiment is the memories that allow diagnosis from asynchronously presented pieces of information. To me, the next question that arises is that of concurrent clues, or the case were time is not assumed to be discrete.

Reference details for Pomi & Mizraji’s “A Cognitive Architecture That Solves A Problem Stated by Minsky”:

Pomi, A. and Mizraji, E., 2001, A Cognitive Architecture That Solves A Problem Stated by Minsky in IEEE Transactions on Systems, Man and Cybernetics, Part B, 31:5 , Oct. 2001 pp 729 -734.

Representation of the Attribute Object Associator module (correlation memory) as a matrix:

Processing of an input vector a_k by the AOA:

Vaario, Onitsuka and Shimohara – Formation of Neural Structures

The aim of this article is to demonstrate the capabilities of a proposed new method that models the growth of neural structures. The method is based on diffusion field modelling, which enables the control of both the neural growth and the genetic factors.

The basis of the diffusion process (which is normally used for bacteria modelling) is that the diffusion of the environment is very fast when compared with the growth process of the organism. It is assumed that the environment will become a steady state. This is the basis for the growth process of the neural network as a function of the environment. The modelling method consists of three parts: the neural dynamics (connections are grown, signals are propagated along the connections and the learnability of the structure is determined), the creature to be controlled by the network is created (with sensors and a motor) and the evolution parameters (the genetic coding and fitness function) are determined.

The behaviour that the network was expected to learn was the ability to direct the creature within a simulated environment. The neural structure created using the method briefly outlined was able to navigate in the environment (although there were examples were the behaviour was not perfect). The effect of using diffusion field control in this experiment was that the amount of genetic information required to code the network was reduced. Instead the environment was enabled to take control of aspects of the network growth.

Go to Vaario, Onitsuka and Shimohara’s “Formation of Neural Structures”:

http://citeseer.nj.nec.com/vaario97formation.html

Reference details for Vaario, Onitsuka and Shimohara’s “Formation of Neural Structures”:

Vaario, J., Onitsuka, A. and Shimohara, K., 1997, Formation of neural structures, in Proceedings of the Fourth European Conference on Artificial Life, ECAL97, pp 214-223 The MIT Press, 15.

The equation to calculate the signal i between A and B:

The signal is either positive or negative based on