Part 5 (1/2)
They are, in this book's terms, hedgehogs. Their kind dominates the op-ed pages of newspapers, pundit panels, lecture circuits, and best-seller lists.
Now, if this is true, and if it's also true that the predictions of hedgehogs are even less accurate than those of the average expert-who does as well as a flipped coin, remember-then a disturbing conclusion should follow. The experts who dominate the media won't be the most accurate. In fact, they will be the least accurate. And that is precisely what Philip Tetlock discovered. Using Google hits as a simple way to measure the fame of each of his 284 experts, Tetlock found that the more famous the expert, the worse he did.
That result may seem more than a little bizarre. Predictions are a big part of what media experts do, after all. Surely experts who consistently deliver lousy results will be weeded out, while the expert who does better than average will be rewarded with a spot on the talking-head shows and all that goes with it. The cream should rise, and yet, it doesn't. In the world of expert predictions, the cream sinks. What rises is the stuff that should be, but isn't, skimmed off and thrown away.
How is this possible? Very simply, it's what people want. Or to put it in economic terms, it's supply and demand: We demand it; they supply.
INTRODUCING THE RENOWNED PROFESSOR DR. MYRON L. FOX.
For as long as students have evaluated professors, professors have complained about student evaluations. They aren't based on substance, professors say. A charming, funny, and confident teacher will be rated highly whether the students actually learn or not, while a serious and challenging professor who isn't so charming, funny, and confident will suffer. This complaint isn't unique to professors, of course. It can be heard anywhere people are rated by others, but professors have a unique way of fighting back.
In the early 1970s, three psychologists from the University of Southern California devised a simple but brilliant experiment. At the center of it would be a distinguished professor named Dr. Myron L. Fox.
Dr. Fox didn't exist. The psychologists invented him, complete with a suitably impressive resume. To play the role of Dr. Fox, they hired an actor who fit the popular image of a distinguished professor. The researchers then crafted an hour-long lecture on ”mathematical game theory as applied to physician education” that was nonsense, and they coached the actor ”to present his topic and conduct his question-and-answer period with an excessive use of double talk, neologisms, non sequiturs, and contradictory statements. All this was to be interspersed with parenthetical humor and meaningless references to unrelated topics.” If substance were the basis of how people judge experts, Dr. Fox would be humiliated.
But Dr. Fox didn't just look the part of a distinguished professor. He talked it. He spoke with clarity, confidence, and authority. That was all that mattered. When Dr. Fox delivered his lecture at a teacher-training conference-before an audience consisting of psychiatrists, psychologists, and social-worker educators-the evaluations were positive across the board. Dr. Fox even got a perfect 100 percent ”yes” on the question ”Did he stimulate your thinking?” The researchers then showed a videotape of the lecture to a similar group and got more enthusiastic responses. A third showing of the videotape-this time to a group of educators and administrators enrolled in a graduate-level university cla.s.s-garnered still more praise. ”Considering the educational sophistication of the subjects,” the psychologists concluded, ”it is striking that none of them detected the lecture for what it was.”
Clearly, people were impressed by Dr. Fox. Just as clearly, it wasn't his substance that impressed them. So what was it exactly? Was it the aura of ”expertness” emanating from his t.i.tle and his authoritative speaking style? Or was it simply the enthusiasm and confidence he projected? It's impossible to tell from the experiment itself, but the answer is likely both.
As social animals, we are exquisitely sensitive to status. An expert, in the appropriate circ.u.mstances, has considerable status. We respect that, even defer to it, whether consciously or not, sometimes with bizarre results. In a Texas study, researchers had a man at a street corner cross against a traffic light and watched to see how many other people waiting at the corner would follow the man's lead and cross the street. The critical variable? Sometimes the man wore ordinary street clothes. Other times, he wore a sharp business suit and tie. As the saying goes, the clothes make the man: Three and a half times more people followed the man across the street when he wore the business suit. An Australian experiment produced even odder results, thanks to the well-doc.u.mented connection between status and perceived size. In a series of university cla.s.srooms, a man was introduced to the students as a visitor from Cambridge University. In some cla.s.ses, the man was said to be a student at Cambridge. In others, he was a lecturer, or a senior lecturer, or a professor. Afterward, students were asked to estimate the visitor's height: With each step up the ladder of status, the man grew by half an inch.
The power of authority was most famously demonstrated in Stanley Milgram's cla.s.sic experiment involving electric shocks administered to a supposed test subject by a volunteer. The shocks weren't real; the ”test subject” was actually an actor. The real subject was the volunteer who flipped switches under the supervision of a white-coated scientist. As the shocks got to supposedly dangerous and even deadly levels, volunteers got anxious and upset. They sweated and moaned. They begged the scientist to stop. But very few refused to do as directed when the scientist told them to throw the switch. A less famous experiment that is perhaps even more disturbing started with a phone call to twenty-two nurses' stations in various hospitals and wards. The man on the phone identified himself as a physician and told the nurses to administer a large dose of a certain drug to a patient. The nurses had plenty of reason to refuse. They didn't know this supposed doctor; it was against hospital policy for doctors to direct treatments over the phone; the drug prescribed by the ”doctor” hadn't been cleared for use; and the label of the drug clearly stated that the maximum daily dose was half what the ”doctor” had ordered them to inject in the patient. Despite all this, 95 percent of the nurses got the drug and were on their way to the patient's room when the researchers put a stop to the experiment. In effect, the nurses stopped thinking independently the moment they heard the t.i.tle ”Dr.”
”Con artists,” notes psychologist Robert Cialdini, ”drape themselves with the t.i.tles, clothes, and trappings of authority. They love nothing more than to emerge elegantly dressed from a fine automobile and to introduce themselves to their prospective 'mark' as Doctor or Judge or Professor or Commissioner Someone. They understand that when they are so equipped, their chances for compliance are greatly increased.” Experts, TV producers, newspaper editors, and book publishers act in better faith than con men, one hopes. But they, too, intuitively understand that the trappings of authority make people much more likely to find the expert persuasive. It's why pundits on business shows wear expensive business suits. It's why ”Ph.D.” so often appears alongside the author's name on the covers of books, and why an economist from Harvard or Oxford or some other prestigious university is far more likely to have her university affiliation mentioned in an introduction than an economist from a lesser inst.i.tution. It's why the CNBC host in the video clip I described at the start of this chapter introduced Arthur Laffer as ”chief investment officer of Laffer Investments and former economic adviser to President Reagan.” There may be no rational reason to introduce Laffer by citing a job he held decades earlier, but that's not the point. Much like the phrase Harvard economist and a con man's gold watch, it establishes authority at a level deeper than rationality. The audience may not consciously think, ”He was an adviser to the president of the United States! He must be right!” But nudging us toward that conclusion is certainly the effect.
THE CONFIDENCE GAME.
In much the same way, a strong, enthusiastic, confident speaking style has a power that transcends mere rationality-a power the psychologist Stephen Ceci demonstrated in an ingenious little experiment. For more than twenty years, Ceci had taught the same cla.s.s in developmental psychology at Cornell University. Sometimes he taught it twice in one year, once in the fall semester and again in the spring semester. Using the same course structure, the same lecture outlines, the same material, it was all perfectly routine. But one year, in the break between fall and spring semesters, Ceci and some other professors attended a workshop taught by a professional media consultant. Each professor was videotaped giving a lecture. The consultant critiqued the performance and suggested changes that would make the delivery more expressive and enthusiastic. ”Underscore points with hand gestures,” the consultant might say, or ”Vary the pitch of your voice.” The substance of the lectures wasn't discussed. This was strictly about style.
Ceci sensed an opportunity. He would follow the media consultant's advice about speaking and gesturing in the next semester but otherwise teach his cla.s.s exactly as he always had. Then he would compare the student evaluations from the fall and spring semesters. If his ratings were higher in the spring, he would know if the stylistic changes alone had made a difference. And they did. Ceci's ratings improved across the board. Students judged him to be more organized, more accessible, and more tolerant. They even considered him to be more knowledgeable, with his average score on a five-point scale rising from 3.5 to 4.
Ceci's experiment clarified things by taking expert status out of the equation, but it still leaves some ambiguity. What exactly was it in his new speaking style that people responded to? Was he more likable? Or was it simply the greater enthusiasm and confidence he projected? That's not clear. But other research suggests confidence is a critical factor. In one study that examined how one person persuaded another when they disagreed, researchers concluded that ”persuasion is a function not of intelligence, prediscussion conviction, position with respect to the issue, manifest ability, or volubility, but of the expression of confidence during the discussion itself.” Very simply: Confidence convinces.
Another group of researchers asked people to tackle various problems-math questions, a.n.a.logy puzzles, forecasts-and state how confident they were that the answer they came up with was correct. Then they were put in groups and asked to decide collectively what the answer was and how confident they were that their answer was correct. The researchers found the group responses tended to match those of the most confident person in the group, whether that person was actually right or not. In a third study, people were asked to watch the videotaped evidence of an eyewitness to a crime. The researchers varied ten different variables, including the circ.u.mstances of the crime, how a police lineup was conducted, and the witness's confidence in her own judgment-in one version, the witness says she is 80 percent sure she correctly identified the suspect; in another, she says she is 100 percent sure. The only factor that made a big difference to every measured outcome was confidence. Researchers have also shown that financial advisers who express considerable confidence in their stock forecasts are more trusted than those who are less confident, even when their objective records are the same.
This research, and much more like it, suggests there is a ”confidence heuristic”: If someone's confidence is high, we believe they are probably right; if they are less certain, we feel they are less reliable. Obviously, this means we deem those who are dead certain the best forecasters, while those who make ”probabilistic” calls-”It is probable this will happen but not certain”-must be less accurate, and anyone who dares to say the odds of something are fifty-fifty will be met with scorn. People ”took such judgments as indications that the forecasters were either generally incompetent, ignorant of the facts in a given case, or lazy, unwilling to expend the effort required to gather information that would justify greater confidence,” one researcher found.
This ”confidence heuristic,” like the ”availability” and other heuristics, isn't necessarily a conscious decision path. We may not actually say to ourselves, ”She's so sure of herself she must be right!” It's something that happens at the edge of consciousness, or even without any awareness at all. It is automatic and instantaneous. We simply hear the person speak and suddenly we have the sense that, yes, she is probably right. If someone suggested her confidence played a key role in our conclusion, we may deny it. After all, we didn't think about her confidence; we may not have even noticed her confidence. At least not consciously.
Using confidence as a proxy for accuracy isn't all that unreasonable. In general, people's accuracy really does rise as their confidence increases. The person who mutters, ”Gee whiz, I think so, but I'm not sure,” probably is less likely to be right than the one who shouts, ”I'm right! I'm right! I'd stake my life on it!” Which is why a ”confidence heuristic” would work. But as is true with all heuristics, the confidence heuristic is far from perfect.
One problem is overconfidence. As we saw, most people are far too sure of themselves, and this gives us trouble if we use confidence to gauge accuracy. Robert s.h.i.+ller is an interesting ill.u.s.tration. Aside from his very impressive t.i.tle-Yale economist-s.h.i.+ller is the very ant.i.thesis of the loud, quick-talking, dead-sure-of-himself pundit. He speaks quietly and is often hesitant, even a little inarticulate. He qualifies his statements and mentions reasons why he might be wrong. He is seldom simple, clear, and certain. But he has a track record that includes correctly calling both the high-tech bubble of the late 1990s and the real estate bubble that followed, and that record got him airtime on business shows normally dominated by c.o.c.ksure pundits. A July 2009 interview on CNBC was typical. The American real estate market was ”still in an abysmal situation,” s.h.i.+ller said, but there was a great deal of diversity within the national market and ”people have gotten very speculative in their att.i.tudes toward housing.” This made it possible that in certain regional markets ”there could be another bubble.” But ”this is not my more probable scenario,” he added. It was more likely that prices ”will languish for many years.” By the standards of TV punditry, it was a nuanced and thoughtful overview of a complex situation plagued with uncertainties. And people hated it. Posted on a Web site, the interview drew 170 comments. Most ignored s.h.i.+ller altogether and instead offered dead-certain predictions of the sort that are usually heard on TV. Some were contemptuous. ”This guy is really hedging his bet. He doesn't want to be underexposed or overexposed. I sure wouldn't take advice from him.” Bring on the blowhards. As British politician Norman Lamont once said, admiringly, of one of his favorite newspaper columnists, ”He is often wrong but he's never in doubt.”
Another problem with the confidence heuristic is that people may look and sound more confident than they really are. Con men do this deliberately. We all do, to some degree. Of course most of us don't do it as brazenly as con men-one hopes-but we all sense intuitively that confidence is convincing. And so, when we are face-to-face with people we want to convince, we downplay our doubts, or bury them entirely, and put on a brave face. And that's before compet.i.tion enters the equation. A financial adviser doesn't just want to convince clients that he can forecast the stock market. He wants to convince clients that he can do it better than other financial advisers. So he beats his chest a little louder than the other guys. But the other financial advisers want to land the same clients, so they answer this chest-beating with even more vigorous displays of bravado. Psychologists Joseph R. Radzevick and Don A. Moore tested this dynamic with an experiment in which people were a.s.signed the role of either ”guesser” or ”adviser.” The job of guessers was to estimate the weight of people in photos. The more accurate they were, the more money they made. The advisers were to provide estimates, including indications of confidence in their accuracy. Guessers were free to choose any adviser's estimate, so advisers made money based on the number of guessers who took their advice. Not surprisingly, advisers were overconfident in the first rounds of the experiment. But they weren't punished for being inaccurate. In fact, guessers preferred the more confident advisers, and advisers responded by getting steadily more confident as the experiment progressed-even though their accuracy never improved. Compet.i.tion ”magnifies” overconfidence, the researchers concluded.
Most people are overconfident to begin with. When they try to convince others, they become even more sure of themselves. Reward them for convincing others, and have them compete for those rewards, and it's just a matter of time before they are insisting they are 100 percent certain what the future holds.
TELL ME A STORY.
People love stories, both the listening and the telling. It's a central part of human existence, found in every culture, in every place, in every time. That universality suggests its origins are biological, and therefore evolutionary.
There are many potential advantages storytelling gave our ancestors. It allowed experience to be distilled into knowledge and knowledge to be transmitted. It strengthened social bonds. It provided an opportunity to rehea.r.s.e possible outcomes. But at an even more fundamental level, storytelling is a work-sharing agreement: If I use my brain's ”Interpreter” neural network to produce an explanation for one set of facts and you use yours to explain another, we can share our explanations by swapping stories. If all fifty members of the tribe do the same, we'll all get fifty explanations in exchange for doing the heavy lifting on one. That's an efficient way to make sense of the world.
For explanation-sharing to work, however, a story cannot conclude with ”I don't know” or ”The answer isn't clear.” The Interpreter insists on knowing. An explanatory story must deliver. When the movie No Country for Old Men ends as the killer walks off with nothing resolved, it disturbs us because the narrative isn't complete. What happens next? How does it end? This nonending worked for a movie (and novel) that was intended to be unsettling, but a story normally has to wrap everything up and come to a clear conclusion. Only that will satisfy the Interpreter's hunger for order and reason.
Other elements of a good story are as universal as storytelling itself. It has to be about people, not statistics or other abstractions. It should elicit emotion. Surprise is valuable, thanks to our evolved tendency to zero in on novelty. It also helps if the story involves a threat of some kind, thanks to the ”negativity bias” mentioned in the last chapter.
”Confirmation bias” also plays a critical role for the very simple reason that none of us is a blank slate. Every human brain is a vast warehouse of beliefs and a.s.sumptions about the world and how it works. Psychologists call these ”schemas.” We love stories that fit our schemas; they're the cognitive equivalent of beautiful music. But a story that doesn't fit-a story that contradicts basic beliefs-is dissonant. (And n.o.body but a few oddb.a.l.l.s enjoys dissonant music.) This is why there is no such creature as a universally acclaimed pundit. The expert who makes a prediction based on an explanatory story that fits neatly with the basic beliefs of an American free-market enthusiast, for example, is likely to get the attention and applause of American free-market enthusiasts. But that expert is just as likely to get a cold shoulder from European social democrats. Same story, same evidence, same logic, but completely different reactions. This sort of disparity appears routinely. Will man-made climate change savage civilization if we don't act now? Many scientists, activists, and politicians make that case. Some people find their evidence and arguments very compelling. Others snort. Whether a person falls in one camp or the other isn't up to a coin toss. Their prior beliefs-their schemas-make all the difference. If I were to describe an American who thinks gun control doesn't work, Ronald Reagan was a great leader, and international terrorism is a major threat, which side of the climate change debate is he likely to come down on? What about an American who supports strict gun control, thinks Reagan was dishonest and dangerous, and the threat of terrorism is overblown? We all know the answer-the first person is much more likely to snort-even though with regard to evidence and logic, gun control, Ronald Reagan, and terrorism have absolutely nothing to do with climate change. But they do reveal a person's schemas.
The importance of explanatory stories in convincing others cannot be exaggerated. ”Imagine that you are the vice-president of a fairly large corporation,” researchers asked a group of forty-four advanced university students with some training in basic decision making. As vice-president, you have been given the job of selecting a law firm to put on retainer. Among other things, the firm must be good at predicting the outcome of litigation. So you ask them to review a hundred pending cases in detail, predict whether the plaintiff or the defendant will win, and say how confident they are on a scale from 50 percent to 100 percent. Now, the researchers directed, ”outline the strategy you would use in evaluating the accuracy of each firm that took part in the exercise.” The big winner? One might think it would be a statistical a.n.a.lysis of the firm's accuracy, but people weren't so interested in that. What they wanted to hear was a good explanation: How did firms make predictions? If the method sounds good, the predictions must be as well.
Statistics be d.a.m.ned. Tell me a story.
As it happens, I've been told that many times, almost word for word, by people who should know. As a money manager based in Houston, Texas, Mike Robertson is one. Handling two billion dollars of other people's cash, Robertson is the fifth-largest independent wealth adviser in the United States. His clients are all multimillionaires; two are billionaires. He knows what it takes to convince very rich people to take his advice. And what it takes is not statistical evidence of sound judgment.
”Do I trust you? Do I think you care about me? Do I think you know what the h.e.l.l you're doing?” That's what matters, Robertson told me. Getting people to say yes requires the ability to connect on a personal level, to make the potential client feel you're caring and trustworthy. Robertson is a big, friendly, likable guy. He's got the human stuff covered. As for competence, he establishes that with a good story-a confident, clear, concise explanation of how he makes decisions that leaves the potential client nodding his head and saying, ”Yes, that makes sense.” Robertson's story is drawn from the demographic a.n.a.lysis of business guru Harry Dent and it's not complicated: ”Instead of coming in with reams of research papers and all this kind of stuff, you sit down and talk about potato chips and motorcycles.” Who eats potato chips like crazy? Fourteen-year-olds. ”I've had two fourteen-year-olds. I can attest to that.” So if demographic projections show growing numbers of fourteen-year-olds in the coming years, you buy potato chip stocks. Same with motorcycles. They may be a young man's dream but Harley-Davidsons are expensive, so Harleys are mainly bought by people in their late forties. ”Everybody says, 'Yeah, absolutely. The last guy I saw riding that thing had gray hair.'” So the future number of people in their late forties tells you whether you should get into Harley-Davidson.
Robertson doesn't deny that his story is very simplistic, or that there's far more to his thinking. But he doesn't go further because this story is what people want. Anything more is needless complication. ”You don't have to sit there and bring in all these charts and stuff,” he says. ”If you don't trust me, it won't make a difference.”
Fundamentally, says Robertson, convincing others that he has a handle on the future is not a rational process. ”Statistics are rational. People are not.”
NOW PUT IT ALL TOGETHER AND GO ON THE TONIGHT SHOW.
”This is a little different for us,” says Johnny Carson, the legendary host of The Tonight Show. After an hour kibitzing with the comedian Buddy Hackett about such weighty matters as Hackett's hair-”You can comb it till your nose drops off and it stays like that,” Hackett observes-Carson will now moderate a debate about the fate of humanity. On one side is the journalist Ben Wattenberg. On the other is a familiar face. ”Dr. Paul Ehrlich has been with us a few times before,” Carson says. ”He's a population biologist at Stanford University and his book The Population Bomb has sold nearly a million copies.”
As one might guess from the brown suits, sideburns, and ads for cigarettes-”New Kent Menthol has got it all together!”-this unusual moment in television history took place in August 1970. As always, Carson sat at his desk. Ehrlich took the chair normally occupied by grinning movie stars, while Wattenberg sat on the couch reserved for sidekicks and second-tier guests.
Carson asked Ehrlich to get things started by summarizing the basic argument of his book. ”The main premise is there are 3.6 billion people in the world today. We're adding about seventy million a year and that's too many,” Ehrlich said. ”It's too many because we are getting desperately short of food. Matter of fact, recent indications are that the socalled Green Revolution is going to be less of a success than we thought it was going to be.” Ehrlich's deep voice is calm and steady and his words flow smoothly. His left elbow is propped casually on the edge of the chair. He's young, but with his suit and tie, his relaxed confidence, and ”DR. PAUL EHRLICH” flashed on-screen, he is every inch an authority. He knows what he's talking about. And he knows what's coming. ”The very delicate life-support systems of the planet, the things that supply us with all our food, ultimately with all our oxygen, with all our waste disposal, are now severely threatened. I would say that trained ecologists are divided into two schools. There's the optimistic school, of which I'm a member, that thinks that if we should stop what we're doing now very rapidly, that there's some chance that we'll prevent a breakdown of these systems. There are others who feel that the changes in the weather, that the permanent poisons that we've already added to the planet, have already set in train the sequence of events that will lead to disaster. They feel it's already too late. I think the only practical thing to do is pretend that it's not too late. So we're in deep trouble and I'm worried about it.”
Wattenberg gives a decent reply, but it's obvious from the beginning he's no match. His delivery is hesitant, and his message is diffuse, unfocused. He accepts some of what Ehrlich is saying but suggests it's ”overstated” and should be more balanced, but his alternative vision is as fuzzy as Ehrlich's is sharp and vivid.
Wattenberg tries gamely to parry Ehrlich's attacks but Ehrlich is far too quick-witted. When Wattenberg claims, ”The new cars have sixty percent less pollution,” Ehrlich shoots back, ”That's nonsense.”
Wattenberg looks a little stunned. ”Well, that's my data,” he says.
”That's not your data. That's the automobile manufacturers' data for cars that have never run anywhere,” Ehrlich responds.
Wattenberg leans back in his chair. ”I can't debate the scientific data with you.”
Ehrlich smiles gently. ”True,” he says. The audience laughs. It's like watching Muhammad Ali float like a b.u.t.terfly and sting like a bee. You can't help but feel sorry for the poor chump in the ring.
Ehrlich delivers all the elements that make a powerful presentation-evident expertise, confidence, clarity, enthusiasm, and charm. He tells a story that is simple and clear and fits the audience's beliefs and current concerns. (”I consider the Vietnam War and racism to be part of the same mess,” Ehrlich says. ”So it's really one big crisis.”) And Ehrlich is able to move from one element to the other as smoothly as Johnny Carson working his way through the opening monologue.