How do we learn? Learning about learning isn’t easy, but since the rewards are potentially enormous, it’s worth looking hard to find answers. One way to begin is to ask what non-human examples, from biology and technology, teach us about learning.
How Nature Learns
The world’s most impressive example of learning is all around us, in the phenomenon of life. Living organisms have “learned” how to gather energy and information amidst a complex environment to grow, to repair themselves, and to produce offspring that repeat those feats.
Biological evolution’s learning strategy, through which the biosphere’s organisms became so sophisticated, is, of course, natural selection. As elucidated by Darwin, natural selection has two basic components: descent with modification, and what he called the struggle for existence. The idea is that organisms produce offspring similar but not identical to themselves, and that the “fittest” offspring — those best adapted to their conditions of life — are more likely to reproduce abundantly. Over time, successive generations become better adapted.
The definition of fitness is circular, but not empty. The fittest organisms are, by definition, those which succeed in producing many descendants. Increasing fitness, in that sense, does not in itself imply increasing sophistication or complexity. (For example, simple bacteria are still extremely abundant!) But evolutionary history shows that increasing sophistication and complexity is one major path to fitness.
Much has been learned since Darwin’s day about how descent with modification actually works. Parents provide a set of coded instructions, in the form of DNA molecules, for assembling children. The code is embodied in DNA’s sequence of nucleotides, which can be visualized as a long succession of As, Cs, Gs, and Ts.
Natural selection’s learning scheme has been a fruitful model for machine learning. “Genetic algorithms” are a prime example. In order to find programs that achieve some goal, one creates a population of functioning programs, each with different properties, specified by its “genome.” Then one runs the programs and compares their results. The least successful are discarded, while the most successful leave several descendants, whose genomes may be slightly different. One can repeat that cycle many times. Impressive programs, which would have been very difficult to design directly, have been created this way, through artificial “evolution.”
It’s important to emphasize that the design of genetic algorithms involves many choices — the design of the master program (which uses the genome); the initial set of “genes”; the rules for culling, reproduction, and mutation; and, crucially, the definition of what counts as “success.”
Other Success Stories
Prior to Darwin’s natural selection, a different theory of evolution had been advanced by Jean-Baptiste Lamarck. According to Lamarck, the “modifications” in “descent with modification” result from the life experiences of the parents. Thus, for example, Lamarck taught that giraffes gradually acquired their long necks as successive generations of proto-giraffes stretched to graze on the leaves of tall trees.
As a theory of biological evolution, Lamarck’s idea was superseded by Darwin’s. But its spirit survives, as an important strategy for machine learning.
In reinforcement learning, as in genetic algorithms, one has a skeletal master program and experiments with alternative ways of filling it in to create functional programs — or, to use a friendlier word, “plans.”
Plans are prescriptions for making a sequence of choices among available options. When things work out successfully — or when they don’t — we can’t necessarily tell which choice was responsible. In reinforcement learning we make incremental changes in the probabilities for selecting among the available options, based on their appearance in successful outcomes.
Put more simply, the basic idea of reinforcement learning is to compare many plans, and to fine-tune them by building up sub-plans that participate in success. Reinforcement learning, therefore, is a kind of Lamarckian evolution.
Typical applications of reinforcement learning include helping robots to move efficiently, and helping game-playing computers to play well. Reinforcement learning was central to AlphaGo, the AI program that, the human Go champion, by 4 to 1 in a five-game match. Good Go-playing requires pattern recognition and long-term thinking, “intuitive” abilities that computer scientists have found difficult to program using conventional, top-down approaches. AlphaGo reached new levels of excellence by playing many games against itself and improving its strategies by experience, using reinforcement learning.
In recent years, many other games — including even Jeopardy and poker — have been “conquered” by machine learning, in the sense that the world’s best players are computer programs.
The programs that conquered checkers and chess follow a simpler version of the approach we’ve already seen: before making a move, they imagine lots of possibilities, evaluate the results, and select the most successful one. (Of course, the best programs add many wrinkles and refinements to this guiding strategy. They are marvels of human ingenuity!) But in the fierce “struggle for existence” among individual moves there is only one survivor, and that survivor leaves no descendants — the whole process starts from scratch at the next move. It’s quite possible that even stronger chess programs could be developed by incorporating more evolutionary ideas.
Imagination and Play
What do these spectacular success stories in non-human learning teach us about human learning? Perhaps ironically, one big takeaway is the value of a quintessentially human characteristic: imagination.
At first that conclusion might sound strange. Do genes or computers really have imagination? But, on deeper reflection, I think you’ll find it compelling. All the approaches we’ve discussed feature, at their core, spinning out and examining many alternative possibilities. And what is imagination, but the ability to consider what is not, but might be?
Many creative people, including many physicists, have testified to the value of imagination in their work. In one of his most famous quotes,: “Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world.”
The great quantum physicist Paul Dirac, when asked about how he made his epochal discoveries,: “I like to play about with equations, just looking for beautiful mathematical relations which maybe don’t have any physical meaning at all. Sometimes they do.”
And Richard Feynman: “The game I play is a very interesting one. It’s imagination, in a tight straitjacket.” The straitjacket, he goes on to explain, is that if your goal is to discover new things about physical reality, you have to respect known facts, accumulated over centuries of research.
Perhaps the most impressive testimonial to scientific imagination comes from James Clerk Maxwell, the leading theoretician of electromagnetism, in, its leading experimenter. The self-taught Faraday had to rely on his visual imagination, because his mathematical training was sketchy. This led him to new ways of thinking, as Maxwell described:
Faraday, in his mind’s eye, saw lines of force traversing all space where the mathematicians saw centres of force attracting at a distance: Faraday saw a medium where they saw nothing but distance; Faraday sought the seat of the phenomena in real actions going on in a medium.
Later, Maxwell later converted Faraday’s imaginative visions into wonderful new kinds of equations, which we still use today.
Having recognized the value of imagination as a tool for problem-solving, how can we cultivate it?
In their free play, children exercise extraordinary imagination spontaneously. So, one promising idea is to encourage a playful approach to problem-solving. Humans are predisposed to enjoy this — as witness the racks of popular magazines at airports or supermarkets, which usually feature collections of word puzzles (e.g., crosswords and acrostics), math puzzles (e.g., Sudoku and KenKen), and logic puzzles, some of which are quite challenging.
I’d like to mention some wonderful examples of imagination-expanding resources, suitable for young people, that take things to a higher level. Each of these has meant a lot to me, personally. Edwin Abbott’sis a mind-expanding novel that explores life in different numbers of dimensions. Robert Forward’s imagines life arising on a neutron star, based on nuclear physics. It too is mind-expanding fun. George Gamow’s Mr. Tompkins is a kind of scientific Gulliver, who in the illustrated novels finds himself moving very fast, where the effects of special relativity come into play, or shrunken down to atomic size, where quantum theory rules, among other adventures. And the late Ray Smullyan wrote several witty collections of logic puzzles, including , which go quite deep (touching on, for example, Gödel’s incompleteness theorems) while advancing humorous narratives. I especially love his retro chess puzzles, and . These present challenging problems in visual imagination and logical reasoning, posed in the context of the classic, slightly exotic characters and situations suggested by their titles.
Schools at all levels would do well to bring these and other imagination-expanding resources — including such computer games as Sim City, Civilization, and Minecraft — front and center.
But imagination is only half the story. In each of the problem-solving strategies mentioned above, the process of imagining possibilities must be married to a method for evaluating their success.
In games, the final goal — like winning or solving the puzzle — usually is specified clearly. But in interesting games it’s not possible to see a clear path to that goal straight away. To make progress you must form more limited, tractable, intermediate goals, and aim for those. In other words, you’ve got to decide what to wish for.
Wishful thinking, in this sense, is an essential part of problem-solving. This becomes even more true when we move outside the context of games, where often there are no set rules that supply the definition of ultimate success.
It’s very plausible, then, that an important step toward achievement, whether for machines or human beings, is to cultivate systematic wishful thinking. New Year’s resolutions, business plans, and visionary “to do” lists embody that strategy.
The issue of goals arises most keenly at the highest levels of learning and problem-solving.
Trying to make a scientific breakthrough, write a novel or compose a symphony, or create some other great work of art can be hard, frustrating work. Success is not guaranteed, and the tangible, economic rewards are usually modest. What keeps people going?
Some motivations are obvious. The dopamine rush that accompanies successful problem-solving can be its own reward. And successful problem-solving can earn the esteem of others — another thing people find rewarding in itself. But this isn’t the whole story.
For many of the greatest physicists, religion was a big motivator. Galileo, Newton, Faraday, and Maxwell were all deeply believing, if not entirely orthodox, Christians. Here’s how John Maynard Keynes, the economist, described Isaac Newton, in his famous lecture “”:
He looked on the whole universe and all that is in it as a riddle, as a secret which could be read by applying pure thought to certain evidence, certain mystic clues which God had lain about the world to allow a sort of philosopher’s treasure hunt to the esoteric brotherhood. He believed that these clues were to be found partly in the evidence of the heavens and in the constitution of elements (and that is what gives the false suggestion of his being an experimental natural philosopher), but also partly in certain papers and traditions handed down by the brethren in an unbroken chain back to the original cryptic revelation in Babylonia. He regarded the universe as a cryptogram set by the Almighty.
And, delighting in his discoveries:
The vast interplanetary and interstellar regions will no longer be regarded as waste places in the universe, which the Creator has not seen fit to fill with the symbols of the manifold order of His kingdom. We shall find them to be already full of this wonderful medium; so full, that no human power can remove it from the smallest portion of space, or produce the slightest flaw in its infinite continuity.
These scientists were determined,, to “know the mind of God.” That inspired them to work prodigiously hard.
Though Albert Einstein was not religious in any conventional sense, he, too, was driven by a passion to know. In his, he recounts his fascination as a child when his father showed him a compass: “This experience made a deep and lasting impression upon me. Something deeply hidden had to be behind things.” And elsewhere he of his struggle to reach the general theory of relativity: “The years of searching in the dark for a truth that one feels but cannot express, the intense desire and the alternations of confidence and misgiving until one breaks through to clarity and understanding.”
Although I am no longer a believer, I am grateful for the training I received in Roman Catholicism, which taught me to see the world as having grandeur and hidden meaning. That vision helped inspire my wishful searching for grandeur and meaning in the physical world, which became habitual.
Finally, let me add that in deciding what to wish for, a feeling for beauty is an invaluable asset. Exposure to beautiful objects and sounds — art and music — as well as beautiful ideas can develop that sense.
- In addition to imagination and wishful thinking, are there other habits of thought that be helpful in learning and specifically in science?
- Are there other books or games, beyond those mentioned here, that can encourage imaginative play?
- Are there any limitations to using computer programs as models for thinking about learning?