Darwin's Dangerous Idea (83 page)

Maelzel was both a crane-maker and crane-stealer of great talent.

434 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Sword in the Stone
435

suspicions. An obviously mechanical swami figure sat at a suspiciously en-FIGURE 15.1. Von Kempelen's

closed cabinet, the doors and drawers of which were sequentially opened (but chess automaton.

never all at once), permitting the audience to "see" that there was nothing but machinery inside. The swami figure then commenced to play a game of that man is capable to impart intellect to matter: for
mind
is no less chess, picking up and moving the chess pieces on a board in response to the requisite in the operations of the game of chess, than it is in the prosecu-moves of a human opponent—and usually winning! But was there, literally, a tion of a chain of abstract reasoning. We recommend those, whose credu-homunculus inside, a little man doing all the mind-work? If AI is possible, lity has in this instance been taken captive by plausible appearance; and all, the cabinet
could
be filled with some collection or other of cranes and other whether credulous or not, who admire an ingenious train of inductive reasoning, to read this article attentively: each and all must arise from its bits of machinery. If AI is impossible, then there must be a skyhook in the perusal convinced that a
mere machine
cannot bring into requisition the cabinet, a Mind pretending to be a Machine.

intellect which this intricate game demands.... [Poe 1836b, p. 89.]

Poe was absolutely certain that Maelzel's machine concealed a human being, and his ingenious sleuthing confirmed his suspicions, which he de-We now know that, however convincing this argument
used
to be, its back scribed in detail with an appropriate air of triumph in an article in the has been broken by Darwin, and the particular conclusion Poe drew about
Southern Literary Messenger
(1836a). At least as interesting as his reasoning chess has been definitively refuted by the generation of artificers following about how the hoax was perpetrated is his reasoning, in a letter accom-in Art Samuel's footsteps. What, though, of Descartes's test—now known as panying the publication of his article, about why it
had
to be a hoax, a line of the Turing Test? That has generated controversy ever since Turing proposed argument that perfectly echoes John Locke's "proof" (back in chapter 1): his nicely operationalized version of it, and has even led to a series of real, if restricted, competitions, which confirm what everybody who had thought We have never, at any time, given assent to the prevailing opinion, that carefully about the Turing Test already knew (Dennett 1985): it is human agency is not employed by Mr. Maelzel. That such agency is em-embarrassingly easy to fool the naive judges, and astronomically ployed cannot be questioned, unless it may be satisfactorily demonstrated 436 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Library of Toshiba
437

difficult to fool the expert judges—a problem, once more, of not having a proper "sword-in-the-stone" feat to settle the issue. Holding a conversation or argument, but, in Penrose's eyes, they didn't agree on what "the" lethal flaw winning a chess match is not a suitable feat, the former because it is too
was.
This was itself a measure of how widely he had missed the mark, since open-ended for a contestant to secure unambiguous victory in spite of its the critics had found many different ways of zeroing in on one big misunderstanding, about the very nature of AI and its use of algorithms.

severe difficulty, and the latter because it is demonstrably within the power of a machine after all. Might the implications of Godel's Theorem provide a better contest? Suppose we put a mathematician in box A and a computer—

any computer you like—in box B, and ask each of them questions about the 2. THE LIBRARY OF TOSHIBA

truth and falsehood of sentences of arithmetic. Would
this
be a test that would surely unmask the machine? The trouble is that human mathematicians
The people who are going to like the book best, however, will probably
all make mistakes, and Godel's Theorem offers no verdict at all about the
be those who don't understand it. As an evolutionary biologist, I have
likelihood, let alone impossibility, of less-than-perfect truth detection by an
learned over the years that most people do not want to see themselves
algorithm. It does not appear, then, that there is any fair arithmetic test we
as lumbering robots programmed to ensure the survival of their genes. I
can put to the boxes that will clearly distinguish the man from the machine.

don't think they will want to see themselves as digital computers either.

This difficulty had been widely seen as systematically blocking any argu-To be told by someone with impeccable scientific credentials that they ment from Godel's Theorem to the impossibility of AI. Certainly everybody
are nothing of the kind can only be pleasing.

in AI has always known about Godel's Theorem, and they have all continued,

—JOHN MAYNARD SMITH 1990 (

unworried, with their labors. In fact, Hofstadter's classic
Godel Escher Bach
review of Penrose )

(1979) can be read as the demonstration that Godel is an unwilling
champion
of AI, providing essential insights about the paths to follow to strong AI, not Consider the set of all Turing machines—in other words, the set of all showing the futility of the field. But Roger Penrose, Rouse Ball Professor of possible algorithms. Or, rather, to ease the task of imagination, consider Mathematics at Oxford, and one of the world's leading mathematical instead a Vast but finite subset of them, relativized to a particular language, physicists, thinks otherwise. His challenge has to be taken seriously, even if, and consisting of "volumes" of a particular length: the set of all possible as I and others in AI are convinced, he is making a fairly simple mistake. When strings of 0 and 1 (bit strings), up to the length of one megabyte (eight Penrose's book appeared, I pointed out the problem in a review: his argument million 0's and l's). Consider the reader of these strings to be my old laptop is highly convoluted, and bristling with details of physics and mathematics, computer, a Toshiba T-1200, with its twenty-megabyte hard disk (we'll prohibit using any additional memory, just for finiteness' sake). It should and it is unlikely that such an enterprise would succumb to a single, come as no surprise that the Vast majority of these bit strings do nothing at crashing oversight on the part of its creator—that the argument could be all worth mentioning if an attempt is made to "run" them as programs on the

'refuted' by any simple observation. So I am reluctant to credit my obser-Toshiba. Programs, after all, are not random strings of bits, but highly vation that Penrose seems to make a fairly elementary error right at the designed sequences of bits, the products of thousands of hours of R and D.

beginning, and at any rate fails to notice or rebut what seems to be an The fanciest program that ever could be is still something that can be obvious objection. [Dennett 1989b.]

expressed as one or another string of 0's and l's, and although my old Toshiba is too small to run some of the truly huge programs that have been devised, it is quite capable of running a handsome and representative subset of them: My surprise and disbelief were soon echoed, first by the usual assortment word-processors, spread sheets, chess-players, Artificial Life simulations, of commentators to a target article (based on his book) by Penrose in
Be-logic-proof-checkers, and, yes, even a few automatic arithmetic-truth-havioral
and
Brain Sciences,
and then by Penrose in turn. In "The Nonal-provers. Call any such runnable program, actual or envisaged, an
interesting
gorithmic Mind" (1990), Penrose's reply to his critics, he expressed mild program (it is roughly analogous to a readable book, actual or imaginary,
in
astonishment at the strong language some of them used: "quite fallacious,"

the Library of
Babel, or a viable genotype in the Library of Mendel). We

"wrong," "lethal flaw" and "inexplicable mistake," "invalid," "deeply flawed."

don't have to worry about the boundary separating the interesting from the The AI community was, not surprisingly, united in its dismissal of Penrose's uninteresting; when in doubt, throw it out. No matter how we rule, there are Vastly many interesting programs in the Library of

438 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Library of Toshiba
439

Toshiba, but they are Vanishingly hard to "find"—that's why software com-certain
proposition follows from the soundness of a certain system. He then panies make quite a few millionaires along with their software.

goes to some length to argue that there could be no algorithm, or at any rate Now,
every
megabyte-length bit string is an algorithm in one sense—the no practical algorithm, "for" mathematical insight. But, in going to all this sense that matters to us: it is a recipe, stupid or wise, that can be followed by trouble, he overlooks the possibility that some algorithm—many different a mechanism, my Toshiba. If we try bit strings at random, most of the time algorithms, in fact—might yield mathematical insight even though that was the Toshiba will just sit there emitting a faint hum (it won't even flash an not just what it was "for." We can see the mistake clearly in a parallel amber light); there are Vastly more ways of being a dead program than a live argument.

one, to echo Dawkins. Only a Vanishing subset of these algorithms are Chess is a finite game (since there are rules for terminating go-nowhere interesting in any way at all, and only a Vanishing subset of
them
have games as draws). That means that there is, in principle, an algorithm for anything at all to do with truths of arithmetic, and only a Vanishing subset of determining either checkmate or a draw—I have no idea which. In fact, I can
these
attempt to generate formal proofs of arithmetical truths, and only a specify the algorithm for you quite simply: (1) Draw the entire decision tree Vanishing subset of
those
are consistent. Godel shows us that not a single of all possible chess games (a Vast but finite number). (2) Go to the end node one of the algorithms in
that
subset ( and there are still Vastly many of them, of each game; it will be either a win for white or black, or a draw. (3) "Color"

even for my little Toshiba) can generate proofs of
all
the truths of arithmetic.

the node black, white, or gray, depending on the outcome. (4) Work But Godel's Theorem tells us nothing at all about any other algorithm in backwards, one
whole
step ( one white move plus one black move ) at a time; the Library of Toshiba. It does not tell us whether there are any algorithms if on the previous move
all
the paths from
any one
of white's moves lead that can play decent chess. There are in fact Vastly many, and a few actual through all black's responses to a white-colored node, color that node white ones reside on my actual Toshiba, and I've never beaten any of them! It does and move back again, and so forth. (5 ) Do the same for any guaranteed not tell us whether there are any algorithms that are pretty darn good at winning paths for black. (6) Color all other nodes gray. At the end of this playing the Turing Test or imitation game. In fact, there is one actual one on procedure (way past the universe's bedtime), you will have colored in every my Toshiba, a stripped-down version of Joseph Weizenbaum's famous ELIZA node of the tree of all possible chess games, leading back to white's opening program, and I have seen it fool uninitiated people into concluding, like move. Now it is time to play. If any one of the twenty legal moves is colored Edgar Allan Poe, that there
must
be a human being issuing the answers. At white, take it! There is a guaranteed checkmate ahead that can be reached just first I was baffled by how any sane human being could think there was a tiny by always staying on the white nodes. Shun any black move, of course, since guy in my laptop Toshiba, sitting, unattached to anything, on a card table, but that opens up a guaranteed win for your opponent. If there are no white I had forgotten how resourceful a persuaded mind can be—there must be, moves at the outset, choose a gray move, and hope that sometime later in the these wily skeptics concluded, a
cellular phone
in my Toshiba!