Levels of Evidence for ADHD Interventions
CHADD members and others affected by ADHD are continually looking for ways to minimize the negative impact of ADHD symptoms on their own lives or the lives of their children. They are often bombarded by advertisements for assessment services, “treatments,” or other “interventions” that make claims that may or may not be supported by evidence-based research or clinical practice.
While CHADD does not endorse or promote particular assessments or treatment interventions, CHADD always looks to science in evaluating ADHD interventions. CHADD is committed to helping those making decisions for themselves or others evaluate important questions: Does this assessment tool do what it claims to do? Will this intervention make my/my child’s life easier, and help manage my/my child’s ADHD? Does it work?
CHADD encourages its members and others reviewing CHADD’s publications to make these decisions in consultation with trained medical/mental health professionals who know the full medical/mental health history of the individual with ADHD and who have particular expertise and experience in diagnosing and treating ADHD.
There continue to be a number of emerging services, products, or interventions that are broadly called “alternative or complementary.” To assist CHADD members and the general public in making decisions, CHADD’s Professional Advisory Board (PAB) has developed a “Levels of Evidence” scale. CHADD’s Professional Advisory board will use this scale to provide a frame of reference for assessing whether, and to what extent, a specific service, program, product or intervention is deemed to be evidence-based for the assessment or treatment of ADHD symptoms.
The PAB will evaluate specific services, programs, products or interventions, as opposed to general approaches that include a number of specific assessments or interventions. A review of the broader literature will be included for consideration as background, and will be essential in order for a program to be considered as Level 2 (“Promising”) or 1 (“Evidence-based”).
To be clear, an intervention may be effective for an individual without having specific scientific evidence, and new interventions have not have had the opportunity to be researched thoroughly. At the same time, an intervention with solid research may not be effective for a specific individual. But we look to research to give us an independent evaluation of the effectiveness. Use this information and consult with your treating doctor to decide what will work best for you or for your family member.
It is important to note that in the following, the effectiveness of the intervention is considered with respect to whether or not it reduces symptoms of ADHD and/or functional impairment associated with ADHD.
CHADD’s 4-point Levels of Evidence Scale
Level 4: Non-evidence-based assessments and interventions1 are approaches without peer-reviewed, published evidence, or with inadequate evidence in peer-reviewed, published research. These clinical methods may use basic general principles that are evidence-based and may even have a positive effect on the symptoms of ADHD. However, the way they utilize such principles has yet to be systematically evaluated and tested. If the description of the assessment/intervention is not adequately specified, it will be assigned to Level 4. Additionally, if well-designed clinical trials have been conducted and the preponderance of the data indicate negative findings for the efficacy of the treatment, Level 4 should be assigned. In order to further explain the lack of evidence, Level 4 rankings will be subdivided as follows:
4.a: No peer-reviewed, published evidence
4.b: Some negative evidence not balanced by positive evidence
4.c: Substantial negative evidence
Level 3: Evidence-informed assessments and interventions are informed by peer-reviewed research but are experimental. This level includes strategies that are explicitly linked to pre-existing empirical or research-based evidence or to portions of an already validated evidence-based assessment or intervention, suggesting that there may be some evidence base. The reasons that these programs do not achieve a higher level of evidence include the following:
- Lack of specification of targeted clinical problems (and appropriate outcomes).
- Lack of research on the specific program or product itself. (If the program is described as falling within a broad category, such as “brain-based assessment” or “cognitive interventions,” in which there exist programs with some empirical support, there nonetheless must be peer-reviewed research on the particular program in question in order for it to be assigned to Level 3.)
Confidence that the intervention will produce desired clinical outcomes is low, though they may have a positive effect in particular situations. While an assessment or treatment with this label may be effective, we cannot know that it is. Additionally, if well-designed clinical trials have been conducted and the preponderance of the data indicate negative findings for the efficacy of the treatment, Level 4 should be assigned.
Level 2: Promising assessments and interventions. Promising assessments and interventions are clearly specified methods that have some evidence supporting them, but not a sufficient amount and/or of sufficient quality to allow the conclusion that they are effective, though they may be effective in particular situations. The manuals are highly specified, such that procedures can be clearly understood and can be replicated by others.
Examples of reasons for placing an assessment or intervention approach at this level include the following:
- The research pertaining to ADHD symptoms and impairment (impact on day to day functioning in the real world) may demonstrate only preliminary results.
- The peer-reviewed findings pertaining to ADHD have been inconsistent and/or inconclusive.
- The intervention or assessment may have either a very limited number of studies supporting it, studies may have occurred only at a single site, or there may be ample studies but of less than optimal methodological rigor.
Level 1: Evidence-Based assessments and interventions. These programs have systematic, high-quality evidence pertaining to the clinical problems they are designed to impact. For interventions, such evidence should include at least two outcome studies by different investigators to show that the program can produce outcomes greater than gained from the normal development and improvement process typical in that population. Evidence may demonstrate that the program produces outcomes better than those obtained by clinically viable alternatives, and that the program is effective with various clients/problems/settings. There is a broad consensus of quality research. Procedures are highly specified, typically through treatment or assessment manuals, enabling replication by others. The characteristics of the samples are clearly described. As with Level 2, in order to assign a program to Level 1, the PAB will look beyond information provided on the website, including relevant comprehensive literature. Additionally, for Level 1 programs a specifier will be added to address effect size, as follows:
1 a: Small effect size (small amount of benefit)
1 b: Medium effect size (medium amount of benefit)
1 c: Large effect size (large amount of benefit)
1For the purpose of this document and scale, “intervention” is used to refer to anything intended to ameliorate the symptoms of ADHD.