This past Sunday, Carl Zimmer published a piece in The New York Times opinion pages, where he makes the important point that even though, in principle, science is a self-correcting enterprise, in practice things are far from perfect: science fixes its mistakes more slowly, more fitfully and with more difficulty than we believe.
He goes on to give a series of examples in the recent literature, where a published result elicits criticisms of all sorts but rarely an attempt to replicate the data in the laboratory. Ideally, of course, experimental refutation is only complete when other groups try to reproduce a certain result in the lab and fail.
Enormous care must be taken to do this: the experimental conditions — as described in the original paper — must be exactly replicated, including the various chemicals or specific material samples used, which must have the same degree of purity. The best is to purchase the stuff from the same distributors as the original experiment.
It's clear that to reproduce an experimental result is troublesome. And the payoff is somewhat on the small side: scientists want to be the first to discover something new, not to be the ones to show that another scientist's discovery is flawed.
It's not very glamorous to be the party pooper. But yes, it is extremely important. Without validation by the community, science simply wouldn't work.
Of course, in practice, no experiment can be exactly duplicated. We must settle for very close at best. A cynic (or a postmodernist) could argue that this is science's Achilles' heel: after all, if you can't duplicate exactly an experimental set-up, how do you know if the results truly stand? How can science have any claim on the "truth?"
This reminds me of the famous saying attributed to the Greek Heraclitus: you cannot step in the same river twice.
Fortunately, in science at least, you don't really need to. (Actually, when do you? Everything in Nature is in constant flux, which is what Heraclitus had in mind.)
Any scientific result comes with error bars that embody the precision of the experimental procedure. No measurement is ever exact. For example, when measuring your weight on a scale graded with a half-pound grid, the accuracy of the measurement is at most a quarter pound, that is, half of the smallest reading. (So, the dieting pessimist can say I gained a quarter of a pound, while the optimist can say, I lost a quarter of a pound!)
As a consequence, scientific results are always quoted within error bars. If an experimental result is correct, different groups will reproduce it within the allowed range of error. (For the interested reader,
here is a short intro to error treatment in the sciences, where the two main kinds of errors, systematic and experimental, are briefly explained.)
To decrease the experimental errors, we need to increase the precision of the measurements. This is usually achieved with better equipment and more readings of data. Still, there is no final "true" result, only the best that current measurements can determine. And that works just fine for confirming experimental results and to validate a hypothesis.
We may not, as Protagoras claimed around 450 BCE, be the measure of all things. But we are the things that can measure! And although our measurements are never exact, the fact that we have a universal procedure to distinguish right from wrong in natural phenomena is a remarkable achievement.
Science may be far from perfect, but the alternative, unchecked subjectivism, is far worse.