Mental Health: A Report by the Surgeon General

In science, no single study by itself, however well designed, is generally considered sufficient to establish causation. The findings need to be replicated by other investigators to gain widespread acceptance by the scientific community.

The strength of the evidence amassed for any scientific fact or conclusion is referred to as “the level of evidence.” The level of evidence, for example, to justify the entry of a new drug into the marketplace has to be substantial enough to meet with approval by the U.S. Food and Drug Administration (FDA). According to U.S. drug law, a new drug’s safety and efficacy must be established through controlled clinical trials conducted by the drug’s manufacturer or sponsor (FDA, 1998). The FDA’s decision to approve a drug represents the culmination of a lengthy, research-intensive process of drug development, which often consumes years of animal testing followed by human clinical trials (DiMasi & Lasagna, 1995). The FDA requires three phases of clinical trials3 before a new drug can be approved for marketing (FDA, 1998).

With psychotherapy, the level of evidence similarly must be high. Although there are no formal Federal laws governing which psychotherapies can be introduced into practice, professional groups and experts in the field strive to assess the level of evidence in a given area through task forces, review articles, and other methods for evaluating the body of published studies on a topic. This Surgeon General’s report is replete with references to such evaluations. One of the most prominent series of evaluations was set in motion by a group within the American Psychological Association (APA), one of the main professional organizations of psychologists. Beginning in the mid-1990s, the APA’s Division of Clinical Psychology convened task forces with the objective of establishing which psychotherapies were of proven efficacy. To guide their evaluation, the first task force created a set of criteria that also was used or adapted by subsequent task forces. The first task force actually developed two sets of criteria: the first, and more rigorous, set of criteria was for Well-Established Treatments, while the other set was for Probably Efficacious Treatments (Chambless et al., 1996). For a psychotherapy to be well established, at least two experiments with group designs or similar types of studies must have been published to demonstrate efficacy. Chapters 3 through 5 of this report describe the findings of the task forces in relation to psychotherapies for children, adults, and older adults. Some types of psychotherapies that do not meet the criteria might be effective but may not have been studied sufficiently.

Another way of evaluating a collection of studies is through a formal statistical technique called a meta-analysis. A meta-analysis is a way of combining results from multiple studies. Its goal is to determine the size and consistency of the “effect” of a particular treatment or other intervention observed across the studies. The statistical technique makes the results of different studies comparable so that an overall “effect size” for the treatment can be identified. A meta-analysis determines if there is consistent evidence of a statistically significant effect of a specified treatment and estimates the size of the effect, according to widely accepted standards for a small, medium, or large effect.

³ The first phase is to establish safety (Phase I), while the latter two phases establish efficacy through small and then large-scale randomized controlled clinical trials (Phases II and III) (FDA, 1998).

Next

Back to the Mental Health: The Surgeon General's Report Table of Contents

Back to Mental Health Articles