"A system of physical symbols has the necessary and sufficient means of general intelligent action" (Allen Newell and Herbert Simon).
"Man's symbolic behavior arises because it has the characteristics of a system of physical symbols" (Allen Newell and Herbert Simon).
Simon and Newell's IPS model
Herbert Simon and Allen Newell hypothesized that "symbol manipulation" is the essence of intelligence common to humans and machines.
According to Simon and Newell, the human mind and computers are so similar that it is possible to develop a new theory that unifies psychology and computer science. Both are symbol-processing systems.
Simon and Newell hoped to explain the complexity of human thought from a number of simple psychological mechanisms, leaving it to the computer to generate the complexity from those simple mechanisms.
In their work "Human Problem Solving" [1972], Simon and Newell presented the general structure of a unified model: the IPS model (Information Processing System). The IPS model is based on the conviction that human reasoning and the subsequent decisions are structurable and, therefore, standardizable and programmable following a certain process that can be modeled on a computer.
The IPS model consists of:
Symbols. It is a set of elementary entities.
Expressions. Symbols form structures or expressions by means of a set of relations.
Processes. An information process is a serial process that has expressions as input and output. Processes can also be represented as expressions.
Characteristics:
Symbols do not occupy memory, but expressions and processes do.
An expression designates, references, or points to an object or another expression.
There is a short-term memory for input/output expressions. There is a long-term memory where expressions are stored. And there is also external memory. The memory is associative because of the internal relationships between expressions.
There is a set of elementary information processes, which are 9 generic actions that serve to model any information process. They are structured according to the following scheme:
The IPS model −called "symbol system"− was intended to raise the level of abstraction of a Turing machine. It is an example of a universal machine. To prove that this model is universal, it is enough to show that it can emulate a Turing machine, which is very easy to demonstrate. This system was presented as the first model of the human mind, the so-called "symbolic" or "computational" model.
Examples of symbol systems
Logic.
Symbols: propositional variables and logical operators.
Expressions: well-formed logical expressions with symbols.
Processes: inference rules.
Algebra.
Symbols: numbers, numeric variables and arithmetic operators.
Expressions: well-formed arithmetic or algebraic expressions with symbols.
Processes: evaluation of expressions to produce results.
Chess.
Symbols: the chess pieces.
Expressions: states of the board (positions of the pieces on the board).
Processes: legal movements of the pieces that make pass from one state of the board to another.
Applications of the IPS Model
Based on the IPS model, Newell and Simon developed the first two artificial intelligence applications: the Logic Theorist and the General Problem Solver (GPS).
Simon and Newell presented the Logic Theorist at the famous "Darmouth Conference" (at Darmouth College, in the summer of 1956 −a conference organized by John McCarthy, Marvin Minsky, Claude Shannon and Nathan Rochester− considered the founding event of artificial intelligence.
Although the term "artificial intelligence" was coined by McCarthy, the field of artificial intelligence is considered to have been founded by Newell and Simon (with Logic Theorist and GPS), as they are considered models of human reasoning. John McCarthy, Marvin Minsky and Oliver Selfridge are also considered founders [McCorduck, 2004].
The Logic Theorist
The Logic Theorist was a program developed by Herbert Simon and Allen Newell, with the help of programmer Cliff Shaw in 1955.
According to its authors, this program provided a link between psychology and computer science. It was able to prove several theorems of symbolic logic, specifically it proved 38 of the first 52 theorems in Chapter 2 of Principia Mathematica, by Russell and Whitehead. And he performed a more elegant than human proof of Theorem 2.85. This proof was attempted to be published in "The Journal of Symbolic Logic", but was rejected because the proof was not remarkable, ignoring the fact that the author was a computer program.
To develop Theoretical Logic they created a new programming language, the IPL language (Information Processing Language) [see Addenda], since no language existing at that time met the required needs of power, generality and flexibility to be able to implement the project. This language was also used later in the GPS.
The Logic Theorist represented a milestone in the development of artificial intelligence and in the understanding of intelligence in general. It introduced new concepts that later became central to the field of artificial intelligence:
Reasoning as search.
The Logical Theorist explored a search tree. The root was the initial state. Each branch was a deduction based on the rules of logic. One of the states of the tree was the goal (the proposition to be proved). The path taken along the branches was the demonstration.
Heuristic rules.
As the search tree grew exponentially (what is called "combinatorial explosion" occurred), it was necessary to prune some branches to avoid paths that were unlikely (or unlikely) to lead to the solution. To do this they used intuitive, practical, simplifying, informal and general rules. They called these rules "heuristics," a term introduced by George Pólya in his classic book "How to solve it".
Pólya's book focuses on mathematical problem solving, but many of his techniques are generic. The term "heuristics" derives from the Greek and basically means "help or guidance in finding or discovering something." Heuretics is the science that studies heuristics and the methods that use them. Heuristic rules are synthetic, generic, right-brain rules. Normal rules are analytical, detailed, from the left side of the brain.
The list process.
IPL implemented a list processing technique with which expressions could be associated. The elements of the list were linked by pointers, that is, the sequentiality was logical, not physical. Thanks to the links between pointers, the memory was associative.
"The Logic Theorist was the positive demonstration that a machine could execute tasks hitherto considered intelligent, creative, and uniquely human" [McCorduck, 2004].
A detailed description of the Logic Theorist can be found in [Steffernd, 1963].
The General Problem Solver, GPS)
GPS is a computer program created in 1957 by Herbert Simon, Allen Newell and Cliff Shaw. It was a generalization of The Logic Theorist to try to solve all kinds of particular problems using a general procedure. GPS was also implemented in IPL.
In GPS the strategy "problem space theory" (problem space theory) was applied where the objects reside. The knowledge of the problem was expressed (in a formal symbolic language) by objects and by the operations that could be performed between objects to give rise to other objects. Operations could be restricted to apply only to certain classes of objects. An operation could produce more than one object as a result.
GPS represented another major milestone in artificial intelligence. It was the first computer program in which knowledge of the problem (the "what") was separated from the strategy for solving it (the "how"), always using the same general reasoning mechanism, regardless of the problem to be solved.
GPS could be applied to a wide variety of problems: proof of theorems, games, recursive problems (such as the towers of Hanoi), algebraic identities, cryptoarithmetic, geometric problems, etc.
In the 8-puzzle or 15-puzzle game, the objects are the different configurations of the pieces. The operations are the possible moves that make you move from one object to another.
In the game of chess, objects are the different arrangements of the pieces on the board (board states). Operations are the legal moves that give rise to new board states.
In mathematics, in a formal axiomatic system, the objects are the axioms. The operations are the rules of inference. To prove a theorem is to transform some initial objects (the axioms) into a previously specified object (the theorem). Once a theorem has been proved, it is added to the list of objects.
Build a computer program. The objects are the possible contents of the memory. Operations are the programming language (or native computer language) instructions that alter the contents of memory. A program is a sequence of operations that transforms one state of memory into another. The problem in programming is to find a sequence of instructions that leads from an initial state of the memory to the final state.
GPS used two heuristics:
means-ends analysis.
Means are the objects and the operations between the objects to achieve the goal. An analysis is performed at each point in time of the available alternatives and the objects and operations that come closest to the objective are selected.
Planning.
Planning makes it possible to construct a solution in general terms, before considering the details. To do this, certain details of the objects and operations are omitted to simplify the problem, ignoring non-significant differences between objects and between operations. Once the target state is reached, the details are considered.
In general, the process for reaching a goal is recursive, and is as follows:
Create a list with only one element: the initial object.
Go through all the objects in the list and all the possible operations to obtain new objects, selecting those objects that are closest to the final object, adding them at the end of the list.
Remove the expanded objects from the list.
If one of the new objects is the target, finish the process successfully.
If there are no more possible new states, end the process with failure. Otherwise, return to point 2.
The structure of possible objects can be traversed in breadth or depth. In breadth, new objects are obtained by expanding horizontally all objects of the previous level. In depth, new objects are obtained by vertically expanding the objects of the previous level.
The GPS system is described in a book that is considered a scientific landmark of the 20th century: "Human Problem Solving" [Newell & Simon, 1972].
The Physical Symbol System Hypothesis
The successes of the symbolic paradigm (with the IPS and GPS systems) encouraged Simon and Newell to go a step further and propose the Physical Symbol System Hypothesis: "A physical symbol system has the necessary and sufficient means of general intelligent action."
In other words, the necessary and sufficient condition for a physical system to exhibit general intelligence is that it be a Physical Symbol System (PSS). "Necessary" means that any physical system that exhibits intelligence is a PSS. "Sufficient" means that any PSS can be organized so that it exhibits general intelligence. "General intelligence" means the same as human intelligence.
The PSS hypothesis cannot be demonstrated at the logical level. It is subject to its empirical validation. The hypothesis is considered validated if intelligent symbolic programs can be constructed and if psychological studies show that humans think by manipulating symbols.
Symbolic systems are labeled "physical" to indicate that they exist in real-world physical devices, computers, and brains. In brains they exist naturally. In computers they exist artificially. Computers have made it possible to transfer symbol systems from the Platonic world of ideas to the empirical world of real processes.
A PSS consists of the same thing as a (theoretical) symbol system, but with the particularity that the system is physical, that is, it is a physical implementation of a symbol system. And it is subject to the laws of physics, i.e., they are realizable or implementable by engineering or electronic computation. Symbolic systems are abstract. PSS are concrete.
Newell's physical symbol system
Newell, in his publication "Physical Symbol Systems" [1980, 1987], presented a variation of the IPS model, with 10 operations:
Input/output: Input, Read, Write.
Control: Do, Exit-if, Continue-if, Behave, Copy, Assign, Quote (Do not evaluate).
The first four control operations are for loops. The "Quote" operator treats processes as data. The "Assign" operator implements the concept of assignment or representation and, according to Newell, plays a very important role, the most fundamental role in a symbol system, since it establishes a basic relationship between a symbol and the entity it designates or represents.
Contents and characteristics of a PSS
A PSS consists of:
Symbols. It is a set of entities that are fundamental physical patterns. The symbol is the elementary unit of information processing.
Expressions. They are structures that relate symbols in a physical way.
Processes. They create, modify, reproduce and delete expressions.
Features:
At any instant, the state of an PSS is a collection of expressions and a collection of processes operating on those expressions. An PSS is a system that evolves through the dynamics of expressions.
An expression can refer to itself as a computational process that the system can evoke and execute. An expression can also designate an object in the world or another expression.
There is a long-term memory that is associative and has virtually infinite capacity to store information. There is a short-term memory that has a limited capacity, which decays over time (it is not permanent) and has less access time than long-term memory. There is also external memory.
The problem space is where the problem solving takes place. It contains the solution and possible solutions. The effectiveness of finding a solution depends on the structure of the problem space.
Implications of the PSS hypothesis
Artificial intelligence.
The hypothesis implies that machines can have minds (or intelligence) like people. This position was termed "strong artificial intelligence" by philosopher John Searle. He called "weak artificial intelligence" the simulation of human intelligence.
For Searle, machines cannot have minds because they are only symbol processors, they are only syntactic, without semantics. To show this he gave his famous example of the Chinese room (or box).
Computing.
An PSS has the computational power of a universal computer. It represents a universal model of computation. A few elementary computational resources (as in the Turing machine) are enough to solve all kinds of problems. At the artificial level (computers), symbols are data, expressions are combinations of data, and processes are programs that manipulate symbols and expressions.
Psychology.
Human symbolic reasoning has the characteristics of a PSS. Humans process symbolic information to solve problems. An PSS can exhibit as much intelligence as a human.
The PSS hypothesis was intended to solve two important problems: 1) the problem of the mind-body relationship, a problem that was posed by Descartes; 2) the problem of the existence of mind in this physical universe. The answer to both problems is that the mind exists as a system of physically implemented symbols.
The PSS hypothesis is equally applicable in cognitive psychology and to artificial intelligence, since it constitutes an abstract description of both the human mind (as an information processor) and a computer. This implies a direct connection between computer science and experimental psychology. Therefore, cognitive processes can be studied with the help of the computer.
Human thought and cognitive processes consist of computational processes that operate on expressions, such as the creation, modification, replication and deletion of expressions, where some expressions act on others.
At the natural (human) level expressions are thoughts, symbols are encoded in our brains, and processes are mental operations of thinking.
Model of the mind.
The representational theory of mind (which is a consequence of the PSS hypothesis) asserts that cognition is best understood as computations that use mental representations, just as arithmetic computations use numerals.
The computational theory of mind, advocated by several authors, including Jerry Fodor. Symbol processing is the basis of human thought and cognition. Mental representations form the language of thought (Mentalés).
Linguistics.
According to Chomsky, the grammars of natural languages require precisely the computational power of a PSS.
Philosophy.
The idea that human thought is a kind of symbol manipulation has its philosophical roots in several authors, most notably:
Hobbes. Reasoning is nothing but calculation.
Leibniz. He tried to create a logical calculus of human reasoning.
Kant. All experience is controlled by internal mental rules.
The PSS hypothesis was proposed by Simon and Newell in a famous paper entitled "Computer Science as Empirical Enquiry: Symbols and Search", published in 1976, which is the record of their 1975 Turing Award lecture.
"The concept of the physical symbol system is the most fundamental contribution of artificial intelligence and computer science to cognitive science" [Newell, in Norman 1981].
"We have invented a computer program capable of thinking non-numerically, and thus solving the venerable mind-body problem by explaining how a system composed of matter can have the properties of mind" [Herbert Simon, quoted in Crevier, 1993].
Criticisms of the Physical Symbol System Hypothesis
The PSS hypothesis has been criticized for five main reasons:
Intelligence is more than just the manipulation of symbols. Intelligence requires feeling, acting, and experiencing.
Searle's objection: computers have no awareness of what they do. They are just symbol processors, without semantics.
In addition to the discrete processing done with symbols, analog processing is also required, mainly linked to perception and pattern recognition.
The brain is not a computer and computation is not a model of intelligence. The brain learns and computers have no initiative; they just do what they are programmed to do.
Many intelligent behaviors transcend the rational to an intuitive plane.
In short, both modes of consciousness are needed: analytic and synthetic, digital and analog, rational and intuitive, semantics and syntax, and so on.
MENTAL vs. Physical Symbol System Hypothesis
We can establish the following analogies and differences:
Abstraction.
MENTAL is at a higher level of abstraction than IPS/PSS. Its primitives are degrees of freedom, so it does not implement any heuristics or special procedures. But its programming is very simple, as it has generic and flexible resources.
IPS/PSS is an abstraction superior to the Turing machine. MENTAL is the supreme conceptual abstraction.
Symbols.
Symbols have no foundation, no semantics. MENTAL primitives are semantic, with which all kinds of expressions can be constructed. MENTAL uses symbols for primitives. And lexical semantics is the same as structural semantics.
Instructions.
IPS/PSS instructions have more of an implementational character. They are not primary concepts, such as those used by MENTAL.
Expressions.
IPS/PSS expressions correspond to MENTAL expressions. Expressions designate themselves (if self-assessing) or designate others (if not self-assessing).
Information processes.
Information processes consist of manipulation and evaluation of expressions.
Model of the mind.
IPS/PSS is considered a model of the mind, which today we call "computationalism". But the mind is not a concrete thing and therefore has no concrete model. On the contrary, MENTAL is not a concrete model of the mind; it is a metamodel of the mind based on degrees of freedom that are the archetypes of consciousness.
Cognitive science.
The subject of the mind is apparently so complex that it needs a multidisciplinary approach. For this reason, Cognitive Science was created, formed by the so-called "cognitive hexagon": neuroscience, anthropology, artificial intelligence, psychology, linguistics and philosophy. MENTAL integrates in a simple way the last four by means of a universal language that is the foundation of all formal sciences.
Paradigms.
IPS/PSS is a computational paradigm. MENTAL is a universal paradigm based on the generic concept of "expression", which can be descriptive or operational. As a universal paradigm it allows expressing all kinds of particular paradigms.
Primitives.
Both systems use a set of primitives. IPS/PSS uses operational primitives to model information processes, according to the computational paradigm. MENTAL is more generic, as it is based on universal semantic primitives.
IPS/PSS primitives are simple. MENTAL primitives are pairs of opposites and configured as the language of consciousness.
General problem solver.
MENTAL is not a "general problem solver", but problems are simplified, clarified or solved by using primitives of supreme level of abstraction. In this sense, MENTAL can be considered a GPS.
Artificial Intelligence.
MENTAL is a programming language and it is also an artificial intelligence language.
Universality.
IPS/PSS is a system that was intended to unite psychology and computer science. MENTAL is more ambitious: it claims to be the foundation of all formal sciences and universal science.
MENTAL vs. IPS model
No.
IPS Instruction
MENTAL
1
Read
(1)
2
Write
3
Store
(2)
4
Move
(x = y) (y = θ)
5
Copy
(x = y)
6
Compare
(x op y) → action (3)
7
Delete
(y = θ)
8
Aggregate
Grouping (series or parallel)
9
Disaggregate
group↓
(1) In MENTAL there are no input/output operations because they are considered implementer-type, transfer operations between internal memory (abstract space) and the external world.
(2) The store operation is implicit. All expression resides in abstract space.
(3) "op" is a comparison operator.
MENTAL vs. Newell's PSS
No.
PSS Instruction
MENTAL
1
Enter
(1)
2
Read
3
Write
4
Do
(2)
5
Exit if
6
Continue if
7
Act
Execute (!)
8
Copy
(x = y)
9
Assign
(x = y) or (x =: y) (3)
10
Quote
Do not evaluate (x°)
(1) In MENTAL there are no input/output operations for the reasons mentioned above.
(2) These three instructions refer to loops. In MENTAL loops are implemented by means of primitives. There are conditional operations. In MENTAL the condition is an independent primitive.
(3) The operation "Assign" corresponds in MENTAL to the primitive "Substitution", in its two variants: as immediate and deferred substitution (or representation).
A fundamental primitive that has neither IPS nor the physical symbol system is "Generalization", a "meta" primitive that allows defining different programming paradigms.
Addenda
More about Simon and Newell
Simon was one of the founding fathers of several scientific fields, including artificial intelligence, information processing, cognitive psychology, decision making, problem solving, organization theory, management theory, econometrics, design science, social and political science, complex systems, operations research, and computer simulation. He received the Nobel Prize in Economics in 1978.
Simon was an interdisciplinary scientist and universalist. His main motivation was to understand the mechanisms of human rational thinking in problem solving and their application to machines. He was always looking for the common root of everything and at the same time for maximum simplicity. According to Simon, complexity emerges from simplicity, so one must look for the underlying hidden simple patterns.
His 1947 book "Administrative Behaviour: a Study of Decision Making Processes in Administration Behaviour" is considered the most influential book of the 20th century in the field of management. In it he connects different disciplines (political science, economics, management, logic, psychology, etc.) and states two things: 1) that decision making is the foundation of all management; 2) that management theory must be derived from the logic and psychology of human choice (or decision).
Simon associated intelligence with decision making. The essence of intelligence is the ability to choose the best option among available alternatives and according to existing constraints. Simon spoke of "bounded rationality" at the personal level for three reasons: 1) by the available information; 2) by the cognitive limitations of the subject; 3) by the time available to make the decision.
In "The Sciences of the Artificial," Simon [2006] states his hypothesis that computation and the human mind are symbol-processing systems.
For his part, Newell based his entire scientific career on the question of the nature of mind and intelligence. During his lifetime he received a large number of honors.
Simon and Newell were jointly awarded the Turing Award in 1975 for their contributions to artificial intelligence and cognitive psychology. The two used the award lecture to present their Physical Symbol System hypothesis and the basic principles of intelligence as information processing.
IPL (Information Processing Language)
IPL was a language with which Logic Theorist and GPS were implemented. IPL implemented general type features such as: lists, property lists, schemas, recursion, symbol processing, higher-order functions, generators, and cooperative multitasking. IPL was the first programming language to implement lists and the first to support recursion.
IPL was soon replaced by Lisp, a high-level language with even more powerful features and a simpler syntax.
SOAR
The GPS paradigm evolved, with Simon and Newell, into SOAR (State Operator And Result), a unified theory of human cognition and an architecture for problem solving and learning.
SOAR is based on a set of cognitive hypotheses similar to the structure of human problem solving:
Long-term memory is structured as a production system (set of rules). Rules can be applied serially or in parallel.
Short-term memory (working memory) is a memory where perceptions and facts inferred by long-term rules are stored.
Problem solving is performed in a state space. The states, the rules to be applied and the effects produced by the rules are explored. All results (intermediate and final) are recorded for future reference.
There are meta-rules, rules that establish the rules to be selected and applied among a set of possible applicable rules, operator preference rules, and rules of preference of some states over others.
Bibliography
Crevier, Daniel. AI: The Tumultuous History of the Search for Artificial Intelligence. Basic Books, 1994.
Ernst, G.; Newell, Allen. GPS: A Case Study in Generality and Problem Solving. Academic Press, 1969.
Le Moigne, J.L. La modélisation des Systèmes complexes. Editions AFCET Système, Dunod 1990.
McCorduck, Pamela. Machines who think. A.K. Peters, 2004.
Newell, Allen. Physical Symbol Systems. Cognitive Science, 4, 135-183, 1980.
Newell, Allen. Sistemas de símbolos físicos. En Donald A. Norman (ed.), Perspectivas de la Ciencia Cognitiva, Paidós Ibérica, pp. 51-107, 1987.
Newell, Allen. Unified Theories of Cognition. Harvard University Press, 1994.
Newell, A., Shaw, J.C.; Simon, H.A. Elements of a theory of human problem solving. Psychological Review, 65, 151-166, 1958.
Newell, Allen; Simon, Herbert A. Human Problem Solving. Prentice-Hall, 1972.
Newell, Allen; Simon, Herbert A. The Logic Theory Machine: A Complex Information Processing System. The Rand Corporation P-868, June 15, 1956. Disponible online.
Newell, Allen; Simon, Herbert A. Computer Science as Empirical Enquiry: Symbols and Search. Communications of the ACM, 10 (3): 113-126, March 1976. Disponible online.
Newell, Allen; Simon, Herbert A. GPS, a program that simulates human thought. En Edward A. Feigenbaum & Julian Feldman Computer and Thought. MacGraw-Hill, 1963.
Newell, A., Shaw, J.C.; Simon, H.A. Elements of a theory of human problem solving. Psychological Review, 65, 151-166, 1958.
Newell, Allen. A guide to the general problem-solver program GPS-2-2. RAND Corporation, Santa Monica, California. Technical Report No. RM-3337-PR, 1963.
Newell, A.; Shaw, J.C.; Simon, H.A. Report on a general problem-solving program. Proceedings of the International Conference on Information Processing. pp. 256-264, 1959.
Nilsson, Nils J. The Physical Symbol System Hypothesis: Status and Prospects. Internet.
Norvig, Peter. Paradigms of Artificial Intelligence Programming. Morgan Kaufmann Publishing, 1991.
Russell, Stuart; Norvig, Peter. Artificial Intelligence: A Modern Approach (AIMA). Prentice-Hall, 2009. Disponible online. (Es el texto sobre inteligencia artificial más difundido del mundo.)
Simon, Herbert A. Las ciencias de lo artificial. Comares, 2006. Disponible online.
Steffernd, Einar. The Logic Theory Machine: A Model Heuristic Program. Rand Corporation, memorandum RM-3731-CC, June 1963.
