Report Finds Forensic Evidence Lacking A new report from the National Academy of Sciences says much of what's commonly called "forensic sciences" doesn't meet scientific standards. Experts say the country's forensics methods and systems — from fingerprint identification to bite-mark analysis — need an overhaul.
NPR logo

Report Finds Forensic Evidence Lacking

  • Download
  • <iframe src="https://www.npr.org/player/embed/101242472/101242456" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
Report Finds Forensic Evidence Lacking

Report Finds Forensic Evidence Lacking

  • Download
  • <iframe src="https://www.npr.org/player/embed/101242472/101242456" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript

IRA FLATOW, host:

Up next - it is the not-so-scientific world of forensic science. You know, when you tune in to "CSI" and you see the detective/scientist in the lab - they have those blue lights; they're analyzing bite marks and carpet fibers or grains of pollen or all kinds of stuff that they take off those crime scenes. Don't let the lab coats and the pipettes fool you, because according to a new government report, much of what of passes for forensic science isn't science at all. At least - or at least many of the commonly used forensic techniques have, according to the report, never been exposed to stringent scientific scrutiny.

What did they single out? Things like bite mark analysis, tool mark analysis, and even fingerprints, to name a few. And these all have never been rigorously tested, and scientists who analyzed the evidence say in the report that some of these people who do the analyses may not be qualified to give expert testimony. There is even one case in which an Indiana high school girl at the age of 17 was certified as a deputy coroner. The report says our country's forensic science system, quote, "has serious problems and we need to overhaul the current structure."

Among the recommendations of the committee behind the report: establish a National Institute of Forensic Science. Also, accredit and certify forensic labs and technicians, and get rid of the system in many parts of the country where elected officials - coroners - conduct forensic work. This report is massive. It makes many more recommendations. We'll try to get into as many as we can.

And joining me now to talk more about it are my guests. Constantine Gatsonis is co-chairman of the National Academy of Sciences Committee on Identifying the Needs of the Forensic Science Community. He is a professor of biostatistics and director of the Center for Statistical Sciences at Brown University in Providence. Thanks for talking with us today, Dr. Gatsonis.

Dr. CONSTANTINE GATSONIS (Co-Chairman, National Academy of Sciences Committee on Identifying the Needs of the Forensic Science Community; Professor, Biostatistics; Director, Center for Statistical Sciences, Brown University): Thank you for having me.

FLATOW: You're welcome. Barry A.J. Fisher is the crime laboratory director for the Los Angeles Sheriff's Department. Welcome back to Science Friday, Dr. - Mr. Fisher.

Mr. BARRY A.J. FISHER (Director, Crime Laboratory, Los Angeles County Sheriff's Department): Thank you for having me, too.

FLATOW: Were you surprised at the results, Barry?

Mr. FISHER: No.

FLATOW: No. Why not?

Mr. FISHER: No. Most people don't realize it was the forensic science community that went to lobby Congress to have this study conducted by the National Academy of Sciences. Many of these issues that were discussed have been deliberated for a number of years without much resolution, and we felt that by having a prestigious group, such as the National Academies, weigh in, we would get enough inertia behind many of these of these things to move the whole issue forward.

FLATOW: So, you've been talking about this for years?

Mr. FISHER: Yes.

FLATOW: And how does that affect the work in your lab?

Mr. FISHER: Well, there's going to have to be some - a number of changes. First of all, our lab, as well as probably 75, 80 percent of the public crime labs, are accredited. It's a voluntary process. There's only a couple of states in the United States that require laboratories to be accredited. There's likely to be a lot more oversight than we're seeing right now. I suspect that in some of the areas - the things that we usually categorize as pattern match kinds of evidence that have yet to go through the type of scrutiny that I think we all agree we would like to see - people will have to get used to how that kind of evidence is being accepted in the courtroom and the kinds of questions that examiners are going to be facing by defense attorneys.

FLATOW: Dr. Gatsonis, I'm sure this will be a topic of discussion in courtrooms.

Dr. GATSONIS: Please?

FLATOW: I say, I'm sure this will be a topic of discussion in courtrooms - this report. And I - we only have less than - we have about 30 seconds to go. So, I want to ask you a question that you're going to not have time to answer. So, I'm going to ask both of you stay on the line. Talking with Constantine Gatsonis, who's co-chair of the National Academy of Sciences Committee Identifying the Needs of Forensic Science Community. He is a professor at Brown University. And Barry A.J. Fisher is the crime laboratory director for the Los Angeles County Sheriff's Department. We're going to talk about the need to change the way forensic science is conducted in labs around the country. So, stay with us. We'll be right back after this short break.

(Soundbite of Talk of the Nation theme)

FLATOW: I'm Ira Flatow. This is Science Friday from NPR News.

(Soundbite of music)

FLATOW: You're listening to Science Friday on NPR News. I'm Ira Flatow, talking this hour about National Academy of Sciences' report - Committee on Identifying the Needs of the Forensic Science Community. With us is Constantine Gatsonis, who is co-chair of that report - of that committee, also, Barry A.J. Fisher, who is crime laboratory director for the Los Angeles County Sheriff's Department. Our number - 1-800-989-8255.

Dr. Gatsonis, let's get into some of the specific details of some of the problems with forensic science. Let me give you a hypothetical question - something I just saw on one of these crime scene investigation stories just last week. Let's say there's a crime committed and there's a bite mark on the victim and there's an expert who testifies about who may have made that bite. What is the problem with that testimony and that scenario?

Dr. GATSONIS: Specifically about bite marks, the committee heard testimony and looked at the published literature that we could find. And our understanding of this is that there's a lot that we really don't know about bite mark analysis and the scientific basis for bite mark analysis has not been developed. And the evaluation of what actually bite mark analysis experts do has not been developed. So, there is a little chapter in one of the sections of the report that discusses bite marks in detail, but I think it's fair to say that what the committee did is it took a broad-brush view of the forensics disciplines, both on the science side and on the practice side. So, the committee examined the forensics sciences system and made its recommendations and its findings that really relate to the full system.

FLATOW: So, bite marks were included with other tests and tools, basically saying that there's no real scientific - heavily - heavy duty scientific testing behind these to verify that there's a science going on here - and bite marks and tool use and other kinds of things that go on in the laboratory?

Dr. GATSONIS: I think that a way of presenting the report, I think, that will make it more sort of realistic is the following: The report show - looked at the forensics disciplines across the spectrum of disciplines and found that, with the exception of nuclear DNA analysis, there is a large range of scientific development in the rest of the disciplines. This is particularly true in the disciplines that developed inherently from within forensics and were not essentially technology transfer from other parts. For instance, DNA analysis came from biology, analysis of chemical substances comes from chemistry and so on. Those methods have been developed and have been evaluated.

But there's a fair number of methods that come inherently from the forensic sciences world, and for many of these methods, what is necessary is a scientific basis for the approaches and a rigorous research infrastructure that will evaluate what is here and that will also innovate - that will bring in innovation about all of the disciplines.

FLATOW: How do we make sure that the people who testify in court and who are the experts in the crime labs actually are good at what they say they're good at?

Dr. GATSONIS: Well, one of the recommendations that we are making about the practice - and now you're going on the side of what the committee looked at. That recommendation's called - calls for mandatory certification programs for experts, but it also calls for rigorous education programs - higher education programs - that go from bachelors to masters to Ph.D. You need a variety of educational offerings for those who are training, so you need to train people. And at the same time, you also need to continue the education of people who are out there in the field. Once they go out in the field, then you need certification programs that will be renewed on a regular basis. And that's one of the ways in which many other disciplines evaluate people who are doing analysis in the field.

FLATOW: Barry Fisher, would you have any problem with that idea?

Mr. FISHER: Well, the - I think the issue is going to come down to most state and local crime laboratories are not under the jurisdiction of the federal government, so the federal government is going to have to figure out a way to help these organizations step up and do the things that are needed. There's probably about, in forensic science today, something like 10,000 or so practitioners. The question about getting them all certified in the various areas, getting the laboratories that they work in accredited - although many of them are already accredited - is going to be a significant task to accomplish. So, you know, there are resource considerations, there are infrastructure considerations that need to be addressed. And I think that the proposal of the National Institute of Forensic Science or some entity to kind of offer some guidance and focus in these areas will help a great deal to move this whole thing along.

FLATOW: So, what you're also talking about among all those words was money?

Mr. FISHER: Yes, absolutely.

FLATOW: (Laughing)There's not - there's no money to do all the training and certification, and certainly, small towns now, in this environment, are not going to be able to afford that.

Dr. GATSONIS: Well, it's - there is money. There is a money issue. There is also the issue of having a system that requires these things to happen. In other words, you need to have the certification mandatory. You need to have bodies that administer this and whatever regulations exist, they need to have teeth. You also need to provide for educational opportunities and to develop educational programs in the universities - a lot more than what there is today.

FLATOW: Well, we have certification in one case in particular. I mentioned at the beginning that there was teenager who was 17 who was certified to become a coroner.

Dr. GATSONIS: (Laughing) Yeah, well, that's - I mean that's…

FLATOW: She was certified by the, you know…

Dr. GATSONIS: Yes, but I mean, in the coroner system, there's a lot of variability there. Typically, to become a coroner, you have to be - either to win an election or to have some very minimal criteria. This was in assistant coroner, a deputy coroner. All in all, that's not certification really, especially in the case of coroners. What are we talking about here in the case of forensic experts is certification developed by professional bodies and administered in ways that are mandatory.

FLATOW: Let's go to the phones - 1-800-989-8255. Vanessa from Denton, Texas. Hi, Vanessa.

VANESSA (Caller): Hello.

FLATOW: Hi, there.

VANESSA: Hi. I just wanted to talk to you guys about bones, specifically teeth bones.

FLATOW: Go ahead.

VANESSA: OK. They can chew stuff really well.

FLATOW: OK. Do you have a point you want to make?

VANESSA: Yeah. When they bite into human skin, they leave marks.

FLATOW: Are you a scientist?

VANESSA: Yes.

FLATOW: And so what is the point you're trying to make?

VANESSA: They can leave marks that can be studied.

FLATOW: OK. Dr. Fisher - Barry Fisher, you could disagree with that.

Mr. FISHER: Well, bite mark analysis is just one of many areas that - involved in tool mark examination, where different subjects are leaving marks behind on tissue, whether it's skin or wood or metal or what have you. And what examiners are trying to do is to find an association between the tool and the mark that's left. It's a lot of observation. It's a lot of experiential-based training.

But Dr. Constantine is correct. There hasn't been a great deal of academic study in these areas. It's a lot of - there have been studies, and some of it has been published in forensic literature, but I think it would add some insight to have non-traditional forensic investigators take a look at some of the things that we're doing and tell us how good we are or not. And we can react from there.

I think at the end of the day, when you look at things like fingerprints and footwear evidence and firearms evidence and what have you, you're going to find that it works, that you come up - you can - you come up with some information that have some probative value to a court. Now, what may very well happen is that we have to ratchet back a little bit on the certainty in which testimony is given about what the meanings of these things are. But I'm still quite certain that they'll be a showing, when the research is done, that these things are appropriate kinds of procedures that we're using. What's lacking here is to have the mainstream scientists weigh in to many of these areas that we're currently using.

FLATOW: And even though your specialist may give expert testimony about the bite mark or something else, we have no way of finding out whether that person is right or wrong about it. We have to take his or her word for it. And without certification, without a track record of what their experience was or their education, we just have to - the jury, meaning we, whoever's listening - has to say, OK, I believe it.

Mr. FISHER: Well, first, there's a process involved. The court - the judges in the courtrooms are the so-called gatekeepers. They determine whether or not the jury can even hear the evidence that's being presented in a trial. And judges are certainly not going to permit things like astrology, for example. You know, there are many people that read the daily newspapers about astrology, but that kind of information is not going to be admitted into a courtroom.

The question comes down for the judge and the jury to consider is how much value does the expert provide in trying to assist in a case? And oftentimes, that becomes a judgment call on the part of the jury. And what usually happens is that the other side of one of these litigations - say, the defense - will bring in their own experts to try to counter what the government experts are saying. And, you know, they will decide - try to figure out what's going on. What the report is saying is that they would like to tighten up on some of these processes that we've been using in the past.

FLATOW: In fact - go ahead, Dr. Gatsonis.

Dr. GATSONIS: If I may add to that, the report makes a couple of recommendations that are related to how the forensics are presented in the court. We're making recommendation that the language and the structure of the report should be standardized and uniform, and the content of the report should also be standardized and uniform, so that we understand what was done, how it was done. And the reporting should be comparable. So, that would help in the courtroom.

But one thing to understand clearly about the report is that this report did not look at past cases. It's not making any points about future cases in the court, and it really focused on the forensic science per se as a scientific enterprise, as an - and as a practice. The way the forensic evidence is presented in court is part of what we are recommending to standardize. But at the same time, that evidence is only one piece, as Mr. Fisher just said. There's only one piece of the information that is available to the jury, and the jury is operating in the context of an adversarial, you know, system of justice. So, it's a different process from that point on.

FLATOW: We're talking about forensic science this hour on Science Friday from NPR News. I'm Ira Flatow. We're talking with Constantine Gatsonis and Barry Fisher - 1-800-989-8255. Dr. Gatsonis, in fact, you say that the way our current system works, the police and the prosecution are responsible for the forensics, and your committee thinks that should change.

Dr. GATSONIS: Yes, that's a key recommendation that we're making. And in other words, what we are calling for is to - for the forensic disciplines to have relative autonomy. That means administrative and fiscal autonomy, vis a vis the law enforcement agencies. The reason for that is, on the one hand, the forensic sciences are, you know, are a scientific enterprise and should be driven by science.

And on the other hand, we also want to make sure that whatever potential for bias there is in the results of an analysis and in the context of an analysis, the bias - coming from the fact that the law enforcement agency would be in a sense the overseer and the supervisor and the employer of the scientist - that kind of bias is removed. I think there have been studies that show that various types of biases, conscious or unconscious, could creep into this. And that's another major consideration, on the basis of which we are recommending that forensic science and forensics laboratories have relative autonomy.

FLATOW: Mm hmm. Barry Fisher, what about this idea of an independent laboratory?

Mr. FISHER: Well, there are some labs around the country that are independent of police agencies. The lab that I'm employed in is not. I think there are alternative ways to accomplish that particular goal that the study is suggesting. And I think that if firewalls are set up to avoid that kind of undue influence, if it exists to any great degree, that may set us - satisfy some of the critics. The reality is that many laboratories who are not in law enforcement agencies have difficulty with their local budgets. I think police and sheriffs' departments have a better chance of getting the kind of resources that are needed to support crime labs, which are not an inexpensive proposition for governmental entities to support.

So, it comes out to be a trade off. You know, obviously we want to assure that there's no bias, that there's no undue pressure among laboratories. But by the same token, we want to make sure that they're adequately funded. When you look at this report, in addition to this separation of laboratories from police agencies, you also find a lot of statements in there about the lack of resources that crime labs have to do their job. And we - many of us sit on large backlogs, and we can't get cases out in a timely basis. So, this is, you know - all of these things are related one to another, and that's why I think this national institute or some other entity to take a holistic approach in looking at these - all of these issues will help out a great deal.

Dr. GATSONIS: I think the resource issues an important one, and we have discussed it in detail in the report. But the independence of the resource is also an issue. In other words, resources have to be provided explicitly to the forensic laboratories and independently, without the control of those resources from the law enforcement agency. Otherwise, there's no firewall.

FLATOW: OK, we have to take a break. We'll come back and talk more about the - repairing the forensics science services around the country. Our number - 1-800-989-8255. Talking with Barry Fisher and Constantine Gatsonis. Stay with us. We'll be right back after the short break.

(Soundbite of Talk of the Nation theme)

FLATOW: I'm Ira Flaw. This is Science Friday from NPR News.

(Soundbite of music)

FLATOW: You're listening to Science Friday on NPR News. We're talking this hour about the science behind forensic science with my guests, Constantine Gatsonis, who is co-chair of the National Academy of Sciences committee that examined the state of forensic sciences in the country. Also with us is Barry A.J. Fisher. He's the crime laboratory director at the Los Angeles County Sheriff's Department. Our number - 1-800-989-8255.

Barry, even if it can be shown that the techniques are ironclad science and that they are ironclad, do you think that will have any significant impact on the way juries decide? I mean, don't they decide on a case by case basis?

Mr. FISHER: Well, sure.

FLATOW: What's going on in the courtroom?

Mr. FISHER: It's a strange environment because this is an adversarial system. You have the two sides - the prosecution and defense - duking it out in the courtroom. Scientists from both sides are trying to be - provide objective information to the finders of facts, whether it be the jury or the judge. And there are no certainties there. All - when the dust settles on all these issues, the defense attorneys, whose job is to try to convince juries that there is some degree of reasonable doubt, are going to look for other areas of fertile issues that they can raise to accomplish their objective.

FLATOW: Right. Dr. Gatsonis, what's going to happen next, because Congress has asked for this report to be prepared, so what do you think they're going to doing with the findings? Do they have time to even consider these things these days?

Dr. GATSONIS: Yes, they do. We briefed both the Senate and the Congress on - before we released the report. We also briefed the Department of Justice; we briefed the White House. So, there is a lot of interest in Washington about what the report is saying, and my understanding is that in the coming months, there may be hearings of committees and so on. So, there will be processes, and I also understand that the forensic science professional organizations are very interested in this. So, I expect things will move on.

FLATOW: All right. Let's see if we can get one question in before we have to go. Brian in Birmingham, Alabama. Hi, Brian.

BRIAN (Caller): Hey, nice to hear from you. I love the show. Had a question for the panel, please. Did I understand during your introduction correctly that there's also a question as to the probative value or the dependability of fingerprint evidence, as well as other evidence?

FLATOW: Dr. Gatsonis?

Dr. GATSONIS: What we are saying about fingerprints and many of the forensic disciplines is that we need to do the studies that show how accurate these are, and we need to do studies that show how they can be improved. So, this applies to fingerprints, and it applies to other forensic disciplines as well. The actual probative value is something that is decided in court. It's not a...

BRIAN: Well…

Dr. GATSONIS: Scientific question per se.

FLATOW: Explain the difference of what court evidence is versus what the science tells us, Dr. Gatsonis.

Dr. GATSONIS: Yes. I mean…

FLATOW: Why is it not ironclad science?

Dr. GATSONIS: Yeah, let's get back to the question you were asking earlier. Suppose the science was very good, how will this play out in court? What science's concern is the following: Suppose you have a procedure that detects things, and it has a certain accuracy, and the accuracy is 90 percent. What science is looking is the denominator and the numerator of that 90 percent and tries to asses whether it's 90 percent, is it 95 percent, is it 99 percent, et cetera.

The legal question is not that. The legal question is the individual who is in court. Are they guilty or innocent? Or the specific issue about where a particular piece of evidence comes from - that is either yes or no. It's neither 95 or it's neither 99. The reason that you want to have a procedure that is very accurate because it gives you extra credence that whatever statement you're making about this one case may be right. But it's not something that is 100 percent.

FLATOW: And you would be able to assign - by scientifically study - assign the accuracy of that technique in general?

Dr. GATSONIS: Well, this has been quantified and studied, let's say in the case of DNA analysis, where you can reasonably assess what's the probability that there's a random match and so on. So, there have been - there has been progress made in some of these disciplines, but for many of the other disciplines, including fingerprints, these kinds of probabilities - for instance of random matches and what the false positives are and what the false negatives are, et cetera, et cetera - have not been adequately studied.

FLATOW: All right. We have to leave with that point. I want to thank you both for taking time to be with us. Constantine Gatsonis, co-chair of the National Academy of Sciences committee that examined the state of forensics in the country. He's also professor of biostatistics and director of the Center for Statistical Sciences at Brown University. Thank you, Doctor Gatsonis.

Dr. GATSONIS: Thank you very much for having me.

FLATOW: Barry A.J. Fisher is the crime laboratory director for the Los Angeles Sheriff Department. Thank you, Barry, for taking time to be with us.

Mr. FISHER: Thank you.

FLATOW: Good to have you again. Have a great weekend.

Mr. FISHER: Bye-bye.

Copyright © 2009 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.