Increase Font Size Decrease Font Size View as PDF Print

2015 Dietary Guidelines Advisory Committee (DGAC) Nutrition Evidence Library (NEL) Methodology

Extract Data and Assess the Risk of Bias

Key information from each study included in a systematic review (SR) was extracted and a risk of bias assessment was performed by a NEL abstractor. NEL abstractors are National Service Volunteers from across the United States with advanced degrees in nutrition or a related field, who were trained to review individual research articles included in NEL systematic reviews (a list of the Volunteers is included in Acknowledgments). From the evidence grids, summary tables are created for each SR that highlight the most relevant data from the reviewed papers.

The risk of bias (i.e., internal validity) for each study was assessed using the NEL Bias Assessment Tool (BAT). This tool helped in determining whether any systematic error existed to either over- or underestimate the study results. This tool was developed in collaboration with a panel of international SR experts.

NEL staff reviewed the work of abstractors, resolved inconsistencies, and generated a draft of a descriptive summary of the body of evidence. The DGAC reviewed this work and used it to inform their synthesis of the evidence.

Nutrition Evidence Library Bias Assessment Tool (BAT)

The NEL BAT is used to assess the risk of bias of each individual study included in a SR. The types of bias that are addressed in the NEL BAT include:
 
Selection Bias Systematic differences between baseline characteristics of the groups that are compared; error in choosing the individuals or groups taking part in a study
Performance Bias Systematic differences between groups in the intervention/exposure received, or in experience with factors other than the interventions/exposures of interest
Detection Bias Systematic differences between groups in how outcomes are determined; outcomes are more likely to be observed or reported in certain subjects
Attrition Bias Systematic differences between groups in withdrawals from a study, particularly if those who drop out of the study are systematically different from those who remain in the study
Adapted from: Cochrane Bias Methods Group: http://bmg.cochrane.org/assessing-risk-bias-included-studies
 
The NEL BAT is tailored by study design, with different sets of questions applying to randomized controlled trials (RCTs) (14 questions), non-randomized controlled trials (14 questions), and observational studies (12 questions). Abstractors complete the NEL BAT after data extraction for each article. There are four response options:
  • Yes: Information provided in the article is adequate to answer “yes” (score=0)
  • No: Information provided in the article clearly indicates an answer of “no” (score=2)
  • Cannot Determine: No information or insufficient information is provided in the article, so an answer of “yes” or “no” is not possible (score=1)
  • N/A: The question is not applicable to the article (score=0).
 The NEL Bias Assessment Tool (NEL BAT)
Risk of Bias Questions Study Designs Type of Bias
Were the inclusion/exclusion criteria similar across study groups? Controlled trials
Observational studies
Selection Bias
Was the strategy for recruiting or allocating participants similar across study groups? Controlled trials
Observational studies
Selection Bias
Was the allocation sequence randomly generated? RCTs Selection Bias
Was the group allocation concealed (so that assignments could not be predicted)? RCTs Selection Bias
Performance Bias
Was distribution of health status, demographics, and other critical confounding factors similar across study groups at baseline? If not, does the analysis control for baseline differences between groups? RCTs
Controlled trials
Observational studies
Selection Bias
Did the investigators account for important variations in the execution of the study from the proposed protocol or research plan? RCTs
Controlled trials
Observational studies
Performance Bias
Was adherence to the study protocols similar across study groups? RCTs
Controlled trials
Observational studies
Performance Bias
Did the investigators account for the impact of unintended/unplanned concurrent interventions or exposures that were differentially experienced by study groups and might bias results? RCTs
Controlled trials
Observational studies
Performance Bias
Were participants blinded to their intervention or exposure status? RCTs
Controlled trials
Performance Bias
Were investigators blinded to the intervention or exposure status of participants? RCTs
Controlled trials
Performance Bias
Were outcome assessors blinded to the intervention or exposure status of participants? RCTs
Controlled trials
Observational studies
Detection Bias
Were valid and reliable measures used consistently across all study groups to assess inclusion/exclusion criteria, interventions/exposures, outcomes, participant health benefits and harms, and confounding? RCTs
Controlled trials
Observational studies
 
Detection Bias
Was the length of follow-up similar across study groups? RCTs
Controlled trials
Observational studies
Attrition Bias
In cases of high or differential loss to follow-up, was the impact assessed (e.g., through sensitivity analysis or other adjustment method)? RCTs
Controlled trials
Observational studies
Attrition Bias
Were other sources of bias taken into account in the design and/or analysis of the study (e.g., through matching, stratification, interaction terms, multivariate analysis, or other statistical adjustment such as instrumental variables)? RCTs
Controlled trials
Observational studies
Attrition, Detection, Performance, and Selection Bias
Were the statistical methods used to assess the primary outcomes adequate? RCTs
Controlled trials
Observational studies
Detection Bias
 
The completed NEL BAT is used to rate the overall risk of bias for the article by tallying the responses to each question. Each “Yes” response receives 0 points, each “Cannot Determine” response receives 1 point, each “No” response receives 2 points, and each “N/A” response receives 0 points. Since 14 questions are answered for RCTs and non-randomized controlled trials, they will be assigned a risk of bias rating out of a maximum of 28 points; while observational studies will be out of 24 points. The lower the number of points received, the lower the risk of bias.

Related 2015 DGAC Methodology topics:

Last Updated: 02/02/2015