Experts, apparently, are worthless. From Nick Kristof:
The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.
The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.
“It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience,” Mr. Tetlock wrote.
Indeed, the only consistent predictor was fame — and it was an inverse relationship. The more famous experts did worse than unknown ones. That had to do with a fault in the media. Talent bookers for television shows and reporters tended to call up experts who provided strong, coherent points of view, who saw things in blacks and whites.
The most important thing to remember is that experts are human – they have biases just like the rest of us. After the jump, I have a story to tell you.
When reporting this story, I talked to Jim Fell, a researcher at the Pacific Institute for Research and Education. Fell appeared to be a great source – a local scientist who had done extensive research on the subject I was talking about.
I interviewed him and it generally went well. I knew the result of his study coming in (that the lowering of the drinking age saved lives) and wasn’t suprised by what he had to say.
But a bit of googling turned up that Fell also served on the Board of Directors for Mothers Against Drunk Driving. Now, this doesn’t mean I regret talking to him or that his research is wrong or anything like that, it just indicates that he clearly has an opinion about what he is researching.
But who wouldn’t? Theoretically, researchers are incredibely well-informed about what they are researching, so shouldn’t they be able to develop opinions about it? How can you know so much about a subject but not somehow critically evaluate it all and form an opinion?