How We Learn (15 page)

• Juba

• Maseru

• Cotonou

• N’Djamena

(Friend: “Maseru”)

And so on. You’ve just taken a test on which you’ve guessed, if you’re anything like me, mostly wrong. Has taking that test improved your knowledge of those twelve capitals? Of course it has. Your friend
gave you the answers
after each question. Nothing surprising there.

We’re not quite done, though. That was Phase 1 of our experiment, pretesting. Phase 2 will be what we think of as traditional studying. For that, you will need to choose another twelve unfamiliar
nations, with the correct answer listed alongside, and then sit down and try to memorize them. Nigeria—Abuja. Eritrea—Asmara. Gambia—Banjul. Take the same amount of time—two minutes—as you took on the multiple-choice test. That’s it. You’re done for the day.

You have now effectively studied the capital cities of twenty-four African nations. You studied the first half by taking a multiple-choice pretest. You studied the other half the old-fashioned way, by straight memorization. We’re going to compare your knowledge of the first twelve to your knowledge of the second twelve.

Tomorrow, take a multiple-choice test on all twenty-four of those nations, also with five possible choices under each nation. When you’re done, compare the results. If you’re like most people, you scored 10 to 20 percent higher on the countries in that first group, the ones where you guessed before hearing the correct answer. In the jargon of the field, your “unsuccessful retrieval attempts potentiated learning, increasing successful retrieval attempts on subsequent tests.”

In plain English: The act of guessing engaged your mind in a different and more demanding way than straight memorization did, deepening the imprint of the correct answers. In even plainer English, the pretest drove home the information in a way that studying-as-usual did not.

Why? No one knows for sure. One possible explanation is that pretesting is another manifestation of desirable difficulty. You work a little harder by guessing first than by studying directly. A second possibility is that the wrong guesses eliminate the fluency illusion, the false impression that you knew the capital of Eritrea because you just saw or studied it. A third is that, in simply memorizing, you saw only the correct answer and weren’t thrown off by the other four alternatives—the way you would be on a test. “Let’s say you’re studying capitals and you see that Australia’s is Canberra,” Robert Bjork told me. “Okay, that seems easy enough. But when the exam question appears, you see all sorts of other possibilities—Sydney, Melbourne,
Adelaide—and suddenly you’re not so sure. If you’re studying just the correct answer, you don’t appreciate all the other possible answers that could come to mind or appear on the test.”

Taking a practice test provides us something else as well—a glimpse of the teacher’s hand. “Even when you get wrong answers, it seems to improve subsequent study,” Robert Bjork added, “because the test adjusts our thinking in some way to the kind of material we need to know.”

That’s a good thing, and not just for us. It’s in the teacher’s interest, too. You can teach facts and concepts all you want, but what’s most important in the end is how students think about that material—how they organize it, mentally, and use it to make judgments about what’s important and what’s less so. To Elizabeth Bjork, that seemed the best explanation for why a pretest would promote more effective subsequent studying—it primes students to notice important concepts later on. To find out, she decided to run a pretesting trial in one of her own classes.

Bjork decided to start small, in her Psychology 100B class at UCLA, on research methods. She wouldn’t give a comprehensive pre
final
on the first day of class. “It was a pilot study, really, and I decided to give the pretests for three individual lectures,” she said. “The students would take each pretest a day or two before each of those lectures; we wanted to see whether they
remembered the material better later.”

She and Nicholas Soderstrom, a postdoctoral fellow, designed the three short pretests to have forty questions each, all multiple-choice. They also put together a cumulative exam to be given
after
the three lectures. The crucial question they wanted to answer was: Do students comprehend and retain pretested material better and longer than they do material that’s not on a pretest but
is
in the lectures? To answer that, Bjork and Soderstrom did something clever on the final exam. They filled it with two kinds of questions: those that were related to the pretest questions and those that were not. “If pretesting
helps, then students should do better on related questions during a later exam than on material we covered in the lectures but was not pretested,” Bjork said. This is analogous to the African nation test we devised above. The first twelve capitals were “pretested”; the second twelve were not—they were studied in the usual way. By comparing our scores on the first twelve to the second twelve, on a comprehensive test of all twenty-four, we could judge whether pretesting made any difference.

Bjork and Soderstrom would compare students’ scores on pretest-related questions to their scores on non-pretested ones on the cumulative final. The related questions were phrased differently but often had some of the same possible answers. For example, here’s a pair of related questions, one from the pretest and the next from the cumulative exam:

Which of the following is true of scientific explanations?

a. They are less likely to be verified by empirical observation than other types of explanations.

b. They are accepted because they come from a trusted source or authority figure.

c. They are accepted only provisionally.

d. In the face of evidence that is inconsistent with a scientific explanation, the evidence will be questioned.

e. All of the above are true about scientific explanations.

Which of the following is true of explanations based on belief?

a. They are more likely to be verified by empirical observation than other types of explanations.

b. They are accepted because they come from a trusted source or authority figure.

c. They are assumed to be true absolutely.

d. In the face of evidence that is inconsistent with an explanation based on belief, the belief will be questioned.

e. b and c above

The students tanked each pretest. Then they attended the relevant lecture a day or two later—in effect, getting the correct answers to the questions they’d just tried to answer. Pretesting is most helpful when people get prompt feedback (just as we did on our African capitals test).

Did those bombed tests make any difference in what the students remembered later? The cumulative exam, covering all three pretested lectures, would tell. Bjork and Soderstrom gave that exam two weeks after the last of the three lectures was presented, and it used the same format as the others: forty multiple-choice questions, each with five possible answers. Again, some of those exam questions were related to pretest ones and others were not. The result? Success. Bjork’s Psych 100B class scored about 10 percent higher on the
related
questions than on the unrelated ones. Not a slam dunk, 10 percent—but not bad for a first attempt. “The best way you could say it for now,” she told me, “is that on the basis of preliminary data, giving students a pretest on topics to be covered in a lecture improves their ability to answer related questions about those topics on a later final exam.” Even when students bomb a test, she said, they get an opportunity to see the vocabulary used in the coming lectures and get a sense of what kinds of questions and distinctions between concepts are important.

Pretesting is not an entirely new concept. We have all taken practice tests at one time or another as a way of building familiarity—and to questionable effect. Kids have been taking practice SATs for years, just as adults have taken practices MCATs and GMATs and LSATs. Yet the SAT and tests like it are general-knowledge exams,
and the practice runs are primarily about reducing anxiety and giving us a feel for format and timing. The research that the Bjorks, Roediger, Kornell, Karpicke and others have done is different. Their testing effect—pre-or post-study—applies to learning the kind of concepts, terms, and vocabulary that form a
specialized
knowledge base, say of introductory chemistry, biblical analysis, or music theory.

In school, testing is still testing. That’s not going to change, not fundamentally. What is changing is our appreciation of what a test is. First, thanks to Gates, the Columbia researcher who studied recitation, it appeared to be at least equivalent to additional study: Answering does not only measure what you remember, it increases overall retention. Then, testing proved itself to be
superior
to additional study, in a broad variety of academic topics, and the same is likely true of things like music and dance, practicing from memory. Now we’re beginning to understand that some kinds of tests improve later learning—even if we do poorly on them.

Is it possible that one day teachers and professors will give “prefinals” on the first day of class? Hard to say. A prefinal for an intro class in Arabic or Chinese might be a wash, just because the notations and symbols and alphabet are entirely alien. My guess is that prefinals are likely to be much more useful in humanities courses and the social sciences, because in those courses our minds have some scaffolding of language to work with, before making a guess. “At this point, we don’t know what the ideal applications of pretesting are,” Robert Bjork told me. “It’s still a very new area.”

Besides, in this book we’re in the business of discovering what we can do for ourselves, in our own time. Here’s what I would say, based on my conversations with the Bjorks, Roediger, and others pushing the limits of retrieval practice: Testing—recitation, self-examination, pretesting, call it what you like—is an enormously powerful technique capable of much more than simply measuring knowledge. It vanquishes the fluency trap that causes so many of us to think that
we’re poor test takers. It amplifies the value of our study time. And it gives us—in the case of pretesting—a detailed, specific preview of how we should begin to think about approaching a topic.

Testing has brought fear and self-loathing into so many hearts that changing its definition doesn’t come easily. There’s too much bad blood. Yet one way to do so is to think of the examination as merely one application of testing—one of many. Those applications remind me of what the great Argentine writer Jorge Luis Borges once said about his craft: “Writing long books is a laborious and impoverishing act of foolishness: expanding in five hundred pages an idea that could be perfectly explained in a few minutes. A better procedure is to pretend that those books already exist and to
offer a summary, a commentary.”

Pretend that the book already exists. Pretend you already know. Pretend you already can play something by Sabicas, that you already inhaled the St. Crispin’s Day speech, that you have philosophy logic nailed to the door. Pretend you already are an expert and give a summary, a commentary—pretend and
perform
. That is the soul of self-examination: pretending you’re an expert, just to see what you’ve got. This goes well beyond taking a quick peek at the “summary questions” at the end of the history chapter before reading, though that’s a step in the right direction. Self-examination can be done at home. When working on guitar, I learn a few bars of a piece, slowly, painstakingly—then try to play it from memory several times in a row. When reading through a difficult scientific paper, I put it down after a couple times through and try to explain to someone what it says. If there’s no one there to listen (or pretend to listen), I say it out loud to myself, trying as hard as I can to quote from the paper its main points. Many teachers have said that you don’t really know a topic until you have to
teach
it, until you have to make it clear to someone else. Exactly right. One very effective way to think of self-examination is to say, “Okay, I’ve studied this stuff; now it’s time to tell my brother, or spouse, or teenage daughter what it all means.” If
necessary, I write it down from memory. As coherently, succinctly, and clearly as I can.

Remember: These apparently simple attempts to communicate what you’ve learned, to yourself or others, are not merely a form of self-testing, in the conventional sense, but
studying
—the high-octane kind, 20 to 30 percent more powerful than if you continued sitting on your butt, staring at that outline. Better yet, those exercises will dispel the fluency illusion. They’ll expose what you don’t know, where you’re confused, what you’ve forgotten—and fast.

That’s ignorance of the best kind.

Part Three