United States: Assessing Expert Methodology: Daubert: in the Third Circuit and the District of New Jersey

This article was previously published in the February 2004 issue of New Jersey Lawyer Magazine, a publication of the New Jersey State Bar Association, and is reprinted here with permission.

Federal Rule of Evidence 702 and the Daubert Decision
In Daubert, the Supreme Court established a detailed standard for a trial court’s admission of expert opinion, based in large measure upon the trial judge’s evaluation of the scientific foundation of the expert’s opinion. Before Daubert, the central authority was the 70-year-old, two-page opinion of the United States Court of Appeals for the District of Columbia Circuit in Frye v. United States.² Frye required that expert testimony be premised upon scientific principle "sufficiently established to have gained general acceptance in the particular field in which it belongs." ³ When Frye provided the governing test, the standard for the admission of expert testimony focused upon the question of scientific consensus rather than the quality of the scientific method.

Daubert is an interpretation of Federal Rule of Evidence 702, the language of which signaled a departure from the Frye general acceptance test. When Daubert was decided, the rule provided:

If scientific, technical or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise.

In Daubert, a Bendectin products liability case, the Court considered the admissibility of the testimony of the plaintiff’s expert interpreting epidemiological studies by others. The expert’s testimony had been rejected by the trial court and the United States Court of Appeals for the Ninth Circuit under the Frye standard. The Supreme Court held that the adoption of Rule 703 had effectively overruled the Frye test, "{given the Rules’ permissive backdrop and their inclusion of a specific rule on expert testimony that does not mention ‘general acceptance.'" ⁴ It substituted a case-specific inquiry by the trial judge, applicable not only to "unconventional evidence" but to other scientific testimony.

Faced with a proffer of expert scientific testimony, then, the trial judge must determine at the outset, pursuant to Rule 104(a), whether the expert is proposing to testify as to (1) scientific knowledge that (2) will assist the trier of fact to understand or determine a fact in issue. This entails a preliminary assessment of whether the reasoning or methodology properly can be applied to the facts in issue. We are confident that federal judges possess the capacity to undertake this review. ⁵

The Court also determined that trial judges have more control over experts than over lay witnesses in applying Federal Rule of Evidence 403 to exclude testimony "if its probative value is substantially outweighed by the danger of unfair prejudice, confusion of the issues, or misleading the jury . . ." ⁶

With that general introduction, the Supreme Court established the following factors for admissibility in the context of the Bendectin issue before it — factors that became the framework of the Daubert test:

1. whether the theory or technique that the expert contends cons titutes scientific knowledge has been tested;
2. whether the theory or technique has been subject to peer review and publication (but publication is not dispositive);
3. the known or potential error rate and the existence or maintenance of standards controlling the technique’s operation; and
4. "'general acceptance’ can yet have a bearing on the inquiry," in the sense that widespread acceptance can be an indicator of reliability, and "a known technique which has been able to attract only minimal support within the community ... may properly be viewed with skepticism." ⁷

The federal trial courts were thus assigned a substantial task, well beyond the parameters of the general acceptance test of Frye: validation of the scientific technique that the expert employs, in its broader application and its case-specific use. In General Electric Co. v. Joiner, decided a few years after Daubert, the Supreme Court underscored the importance of the trial court’s gatekeeper role, holding that the district court’s determination is subject to an "abuse of discretion" standard on appeal, and approving the exclusion of expert evidence when the data and the opinion are insufficiently connected. ⁸

Yet an important issue remained to be determined: Was Daubert limited to scientific evidence such as the epidemiology at issue in that case, or did it extend to experts in other fields, including those whose methodology was less distinct? The Court answered that question in the affirmative in Kumho Tire Co. v. Carmichael, in which the contested expert was a tire engineer. ⁹ There, the Supreme Court held that the Daubert standard extends to all fields in which experts have technical or otherwise specialized knowledge, and that the test may vary with the scientific, technical or other area of expertise at issue. Daubert’s reach is broad indeed.

The drafters of the federal rules amended Federal Rule of Evidence 702 in 2000 to conform to the Supreme Court’s definition of the district court’s inquiry:

If scientific, technical or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education. may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.¹⁰

The rule thus confirmed the trial court’s responsibility to understand and critically evaluate the expert’s scientific or technical methodology, even when that evaluation could determine the outcome of a case.

The Supreme Court’s articulation of general principles was not a comprehensive blueprint for the district court’s gatekeeper function; indeed, Daubert was and remains the subject of voluminous commentary and vigorous debate. Few absolute rules apply to a process that is by its terms expert-specific, particularly when that process is governed by an abuse of discretion standard on appeal. However, opinions by the Third Circuit and of several district and magistrate judges in the District of New Jersey provide useful guidance to the practitioner seeking to admit or exclude expert evidence.

In almost every case, a Daubert challenge will trigger an in limine hearing under Federal Rule of Evidence 104(a). In Daubert, the Supreme Court instructed trial courts to determine expert opinion admissibility as a preliminary inquiry under Rule 104(a), and to require proof by a preponderance of the evidence.¹¹ The hearing is ordinarily requested by counsel, but may be directed by the Court.

The Third Circuit has confirmed that a Rule 104 hearing is usually, if not always, essential to Daubert gatekeeping. In In re TMI Litigation,¹² the Court noted that while an in limine hearing is not absolutely mandatory, "when the ruling on admissibility turns on factual issues, at least in the summary judgment context, failure to hold a hearing may be an abuse of discretion." Quoting its prior holding in Padillas v. Stork-Gamco, Inc.,¹³ the Court held that a trial court might be compelled to hold an in limine hearing, even when the expert’s proponent fails to request it, by virtue of its "independent responsibility for the proper management of complex litigation and because the plaintiff needs an opportunity to be heard on issues of scientific reliability and validity." However, the Court noted that "Padillas certainly does not establish that a District Court must provide a plaintiff with an open-ended and never-ending opportunity to meet a Daubert challenge until plaintiff ‘gets it right.'" Note that the gatekeeper role is primarily focused upon evidence in a jury trial; "where the Court itself acts as the ultimate trier of fact at a bench trial, the Court’s role as a gatekeeper pursuant to Daubert is arguably less essential." ¹⁴

In the wake of Daubert, the use of expert affidavits or testimony to validate or criticize the disputed expert testimony has become increasingly popular among lawyers; such expert evidence affords to the trial court credible and valuable assistance on qualifications, methodology and scientific consensus. Shortly after Daubert was decided, the Third Circuit issued a lengthy and comprehensive opinion on various Daubert issues in In re Paoli R.R. Yard P.C.B. Litigation. ¹⁵ It held that "because under Daubert a judge at an in liminehearing must make findings of fact on the reliability of complicated scientific methodologies and this fact-finding can decide the case, it is important that each side have an opportunity to depose the other side’s experts in order to develop strong critiques and defenses of their experts’ methodologies."

The right to explore the opinions of experts called to testify about the opinions of other experts has its limits, however. In Magistrini v. One Hour Martinizing Dry Cleaning, the plaintiff challenged, on Daubert grounds, the affidavit prepared by defense experts critiquing the methodology of the plaintiffs’ experts. The Court held that the "[p]laintiff’s challenge asks this Court to act as a gatekeeper to itself, to undertake a hearing within a hearing and to exclude [the affidavit] from the Court’s own consideration;" it declined to do so.¹⁶ The affidavit was admitted.

Counsel should anticipate the scheduling of an in limine hearing, most likely in the context of a summary judgment hearing, shortly before trial or during trial, at times on relatively short notice, and should be prepared to present their evidence for or against the admission of expert testimony in that hearing. Testimony by qualified experts other than the witness whose testimony is disputed can prove helpful to a district court.

Like the courts of many other jurisdictions, the Third Circuit has summarized the Daubert test as a "trilogy of restrictions" consisting of "qualification, reliability and fit." ¹⁷ Unless it is conceded by the challenging party, each of the factors must be addressed in the Daubert hearing.

The First Factor: Qualification
The qualification factor "refers to the requirement that the witness possess specialized expertise ... a broad range of knowledge, skills and training qualify an expert." ¹⁸ The issue is not whether the expert has the best conceivable qualifications to opine on a given s ubject. ¹⁹ Instead, the court considers "whether the proposed expert witness has sufficient knowledge, skill, training, education or experience to testify with authority on the particular subject matter or issue on which he or she proposes to opine." ²⁰ The requirement is construed liberally, and the "imposition of overly rigorous requirements of expertise is eschewed." ²¹

In appropriate cases, expertise developed through professional experience can compensate for an absence of formal training. In Waldorf v. Shuta, the court rejected a challenge to the testimony of an expert on vocational rehabilitation despite his lack of formal training in that field. After reciting the expert’s long government experience in programs for the disabled, the Third Circuit found that he had "substantially more knowledge than an average lay person regarding employment opportunities for disabled individuals." However, the proposed expert must have more than a general familiarity, acquired through work experience, with the subject of his or her testimony. ²²

In Aloe Coal Co. v. Clark Equipment Co., the Third Circuit precluded a tractor sales representative from testifying as an expert regarding the cause of a tractor fire, on the grounds that he was not an engineer, a designer of construction machinery or a mechanic, and had no experience in determining the cause of fires or operating machinery. ²³ Work experience may be sufficient to qualify an expert, but it must be directly relevant to the subject of his or her testimony.

A witness qualified to give one component of his or her proposed opinion may be unqualified to give another. In Poust v. Huntleigh Healthcare, the court found the plaintiff’s engineering and safety expert to be qualified to render an opinion on the alleged defect in a medical device. ²⁴ However, the court rejected the expert’s attempt to opine that the defect he found caused the plaintiff’s injuries. The court limited his trial testimony accordingly.

The parameters of the witness’s expertise should be fully explored from the time he or she is retained, and the expert’s opinion should remain within those parameters. The more specific the expert’s proposed testimony, the more precisely relevant his or her qualifications must be; as the Third Circuit recently held in Calhoun v. Yamaha Motor Corporation, U.S.A., "[w]hile the background, education, and training may provide an expert with general knowledge to testify about general matters, more specific knowledge is required to support more specific opinions." ²⁵

The Second Factor: Reliability
The reliability factor is the centerpiece of Daubert analysis, and it is the most challenging component of the test. It requires the trial judge to understand and critically evaluate the methodology by which a scientist, physician, engineer, accountant or other expert collects information, analyzes data and reaches conclusions. As the post -Daubert law has matured, the federal courts have become increasingly familiar with a broad variety of scientific and technical fields, and in some instances with the prior work of particular experts.

The reliability requirement differentiates testimony premised upon "the methods and procedures of science" as opposed to "subjective belief or unsupported speculation." ²⁶ The non-exclusive criteria adopted in Daubert have been refined in this and other circuits to entail a total of eight factors: 1) whether the theory or technique can be tested; 2) whether the theory or technique has been subjected to peer review; 3) whether there is a high rate of known or potential error; 4) whether there are standards controlling the technique’s operation; 5) whether the theory enjoys general acceptance; 6) whether there is a sufficient relationship between the technique and methods which have been established to be reliable; 7) whether the expert witness’s qualifications are sufficient; and 8) whether the method has been put to non-judicial uses. ²⁷

In Magistrini, the district court compiled a useful list of other factors, found by other federal courts to be relevant considerations in specific cases. They include:

(i) whether the expert’s proposed testimony grows naturally and directly out of research the expert has conducted independent of the litigation;" (ii) whether the expert has unjustifiably extrapolated from an accepted premise to an unfounded conclusion; (iii) whether the expert has adequately accounted from alternative explanations; (iv) whether the expert is being as careful as he would be in his professional work outside of the litigation; and (v) whether the field of expertise asserted by the expert is known to reach reliable results for the type of opinion proffered by the expert. ²⁸

As does the original test articulated by the Supreme Court in Daubert, these factors focus the Court’s attention upon the distinction between methodology proven in scientific practice and methodology developed for purposes of the case.

To satisfy the reliability prong of the Daubert test, the expert’s methodology must be carefully documented and explained, preferably by reference to non-litigation applications. Opinions evaluating expert testimony on grounds of unreliability often focus on the sufficiency of explanation and documentation. ²⁹

Sometimes, the methodology is so weak the court determines a mistaken admission to be inconsequential. In Better-box Communications Ltd., v. BB Technologies, the Third Circuit held that the expert’s opinion, based upon "his personal knowledge or experience" rather than a methodology that satisfies the Daubert factors, was so simple that it was unlikely to have played a role in the jury’s decision. ³⁰

Finally, the reliability inquiry focuses "solely on principles and methodology, not on the conclusions that they generate."³¹ That does not mean, however, that the content of those conclusions is irrelevant. The Supreme Court noted in Joiner that "nothing in Daubert or the Federal Rules of Evidence requires a District Court to admit opinion evidence that is connected to existing data only by the ipsi dixit of the expert."³²

A New Jersey District Court reconciled the two principles as follows:

a court must engage in limited review of an expert’s conclusions ‘in order to determine whether they could reliably flow from the facts known to the expert and the methodology used.’ A court may conclude, after viewing an expert’s conclusions in light of the evidence on which he relies and the methodology employed, that there is simply too great an analytical gap between the data and the opinion proffered. ³³

The reliability determination is an individualized inquiry, yet these general principles govern every case. The methodology should be subject to objective criteria and testing, proven in the scientific or technical sphere rather than created for litigation, and documented with precision and care.

The Third Factor: Fit
The fit component of the Daubert test concerns "the proffered connection between the scientific research or test results to be used and particular disputed factual issues in the case."³⁴ It requires that the testimony "must in fact assist the jury, by providing it with relevant information, necessary for a reasoned decision of the case."³⁵

This factor focuses upon the nexus between the methodology and the case-specific opinion. It bars admission of otherwise reliable opinion evidence if the expert attempts too ambitious an extrapolation of theory to facts. For example, in In re TMI Litigation, the Third Circuit rejected the admission of a meteorologist’s "plume movie," designed to illustrate the movement of a radioactive plume, on the ground that it was irrelevant to the Three Mile Island nuclear accident at issue in the case.³⁶ The expert conceded that he lacked adequate details about the Three Mile Island conditions to track the radioactive plume with precision, but attempted to support the plume movie as "the beginning of an investigation, not the end." The Third Circuit affirmed its exclusion on grounds of insufficient "fit."

Similarly, in Calhoun, the Third Circuit determined that while the disputed witness "possessed expertise in relevant fields, he failed to apply this expertise to the matter at hand," relying for his opinion on jet ski acceleration upon "his familiarity with outboard motors [and] his recollection of a friend’s motorcycle."³⁷ In short, this final factor of the Daubert test can derail the testimony of an admittedly qualified witness using sound methodology.

Ten years after Daubert, in light of numerous opinions by the lower courts refining and explaining its test, much of the alarm and confusion that initially greeted the Supreme Court’s opinion has dispersed. Lawyers are now in a better position to anticipate Daubert issues from the inception of a case and choose their experts accordingly. The process of selecting an expert now requires a thorough understanding of the methodology to be used, and a sophisticated critique of that methodology with a future Daubert hearing in mind.

1. 509 U.S. 579 (1993).
2. 293 F. 1013 (D.C. Cir. 1923).
3. 293 F. at 1014.
4. 509 U.S. at 589.
5. 509 U.S. at 592-593 and n. 11.
6. 509 U.S. at 595, (quoting Weinstein, Rule 702 of the Federal Rules of Evidence is Sound; It Should Not Be Amended, 138 F.R.D. 631 (1991)).
7. Daubert, 509 U.S. at 593-94.
8. General Electric Co. v. Joiner, 522 U.S. 136, 146-147 (1997).
9. Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999).
10. Fed. R. Evid. 702.
11. 509 U.S. 592 n. 10.
12. 199 F.3d 158, 159 (3rd Cit. 2000).
13. 186 F.3d 412 (3rd Cir. 1999).
14. Magistrini v. One Hour Martinizing Dry Cleaning, 180 F. Supp. 2d 584, 596 n. 10 (D.N.J. 2002), aff’d, No. 02-2331, 2003 WL 21467223 (3rd Cir. June 25, 2003); Gibbs v. Gibbs, 210 F.3d 491 (5th Cir. 2000); Volk v. United States, 57 F. Supp. 2d 888 (N.D. Cal. 1999).
15. 35 F.3d 717, 739 (3rd Cir. 1994), cert. denied, 513 U.S. 1190 (1995).
16. 180 F. Supp. 2d at 597.
17. Calhoun v. Yamaha Motor Corp., 350 F.3d 316 (3rd Cir. 2003) at 321; Schneider v. Fried, 320 F.3d 396, 404 (3rd Cir. 2003).
18. Schneider, 320 F.3d at 404.
19. Holbrook v. Lykes Brothers Steamship Co., 80 F.3d 777, 782 (3rd Cir. 1996).
20. Yarchak v. Trek Bicycle Corp., 208 F. Supp. 2d 470, 494-95 (D.NJ. 2002).
21. Waldorf V. Shuta, 142 F.3d 601, 625 (3rd Cir. 1998).
22. Paoli, 35 F.3d at 741; Lithuanian Commerce Corp. v. Sara Lee Hosiery, 177 F.R.D. 245, 255 (D.N.J. 1997).
23. 816 F.2d 110, 114 (3rd Cir.), cert. denied, 484 U.S. 853 (1987).
24. 998 F. Supp. 478, 492-93 (D.N.J. 1998).
25. 350 F3d 316 (3rd Cir. 2003) at 322.
26. Paoli, 35 F.3d at 742.
27. Daubert, 509 U.S. at 592-95; Paoli, 35 F.3d at 742 n. 7; Magistrini, 180 F. Supp. 2d 584 (D.N.J. 2002).
28. 180 F. Supp. 2d at 594-595 (citations omitted).
29. See, e.g., Magistrini, 180 F. Supp. 2d at 604-608 (finding expert opinion unreliable); Voilas v. General Motors Corp., 73 F. Supp. 2d 452, 460-63 (D.N.J. 1999) (finding expert opinion reliable).
30. 300 F.3d 325, 329-330 (3rd Cir. 2002).
31. Daubert , 509 U.S. at 595.
32. General Electric Co. v. Joiner, 522 U.S. at 145-146.
33. Yarchak v. Trek Bicycle Corp., 208 F. Supp. 2d 470, 495 (D.N.J. 2002).
34. Paoli, 35 F.3d at 743.
35. Yarchak, 208 F. Supp. 2d at 496; Magistrini, 180 F. Supp. 2d at 595.
36. 193 F.3d 613, 669-671 (3rd Cir. 1999).
37. 350 F.3d 316 (3rd Cir. 2003) at 324.