Increase Font Size Decrease Font Size View as PDF Print

Frequently Asked Questions (FAQs)

Introduction Systematic Review Process Dietary Guidelines General


Introduction

 What is the Nutrition Evidence Library (NEL)?

The NEL is a group of nutritionists and librarians who specialize in systematic review methodology within the USDA’s Center for Nutrition Policy and Promotion (CNPP) that uses state-of-the-art methodology to search, evaluate, and synthesize food and nutrition-related research. This rigorous, protocol-driven methodology is designed to minimize bias, maximize transparency, and ensure relevant, timely, and high-quality systematic reviews to inform federal nutrition-related policies, programs, and recommendations and future research. In addition, the NEL is a key resource for making food and nutrition research accessible to all Americans.
 
Plain language: The NEL specializes in doing systematic reviews, or pulling together the best available scientific research in an unbiased way to answer important food and nutrition-related questions. These reviews provide the scientific foundation for federal policies, programs, and recommendations to ensure they align with the most current research.

Back to Top  

What are the benefits of the Nutrition Evidence Library (NEL)?

The NEL:

  • Evaluates and synthesizes scientific literature using gold standard systematic review methodology;
  • Utilizes a process that is transparent, reproducible and and minimizes bias;
  • Works collaboratively with the nation’s leading scientists to develop objective criteria to select and analyze the highest quality scientific studies to answer policy-relevant food and nutrition questions;
  • Provides timely and credible scientific information to support nutrition-related policies and programs; and
  • Ensures compliance with the Consolidated Appropriations Act of 2001 (Data Quality Act), which mandates that federal agencies ensure the quality, objectivity, utility, and integrity of the information used to form federal guidance.

Back to Top

What can be found on the NEL website? 

NEL.gov provides public access to documentation from each step of the systematic review process for all reviews completed by the NEL. This includes:

  • Research questions
  • Literature selection criteria (inclusion/exclusion criteria)
  • Search strategies
  • Lists of included and excluded studies (with rationale)
  • Assessment of the risk of bias of included studies
  • Evidence summaries and tables
  • Key findings
  • Graded conclusion statements
  • Research recommendations.
 NEL systematic reviews address topics related to the relationships between diet and nutrient intakes and:
  • Health outcomes
  • Social, behavioral, and/or environmental factors relevant to federal nutrition policy and programs
  • Effective communication of dietary guidance

Back to Top

Systematic Review Process

What is a NEL systematic review?

A systematic review is a critical assessment of all research studies that address a particular issue. Systematic reviews originated to address clinical issues in the medical field and have expanded into other disciplines. A NEL systematic review is a state-of-the-art method for evaluating scientific evidence on food and nutrition to answer a precise question or series of questions in areas that may impact federal policy and programs. In keeping with best practices, NEL systematic reviews are conducted by a multidisciplinary research team to:

  • Develop systematic review questions
  • Search, screen, and select studies to review
  • Extract data and assess the risk of bias of the research
  • Describe and synthesize the evidence
  • Develop conclusion statements and grade the evidence
  • Identify research recommendations.

Each step of the process is documented in detail to ensure objectivity, transparency, and reproducibility of the process.

Back to Top

How are NEL systematic review topics selected?

Many factors are considered when determining topics for NEL systematic reviews. First, the scope of the project is defined to provide guidance to the number and breadth of topics in areas of high public health concern. Then, multidisciplinary teams of scientific and/or federal program experts provide recommendations based on the state of science relating to nutrition and health. When working with a Federal Advisory Committee, such as the Dietary Guidelines Advisory Committee, the Committee identifies the topics and systematic questions of interest. Topics are then refined and prioritized according to established criteria by the multidisciplinary team, and high priority topics are selected for review.

Back to Top

Who are the experts with whom the NEL collaborates?

The NEL works with different expert groups who are convened to review the state of the science on nutrition and health. These groups include domain experts, nutrition scientists, clinicians, epidemiologists, methodologists, communicators, and/or end users of a review. One type of expert group is a Technical Expert Collaborative (TEC) consisting of six to eight leading scientists who are convened to participate in a systematic review project on a specific topic. Members of the TEC provide perspective and insight regarding issues related to the topic and help synthesize synthesize evidence to answer important diet-related review questions. Another type of expert group is a Federal Advisory Committee, such as the 2015 Dietary Guidelines Advisory Committee, which was tasked with conducting systematic reviews of the latest nutrition and health research and developing a report to advise the USDA and HHS regarding updates to the 2010 Dietary Guidelines for Americans.
 
How are NEL systematic review questions developed?

The research questions are developed by expert groups and the NEL staff ensures the questions are accompanied by an analytical framework in a standardized format that includes the following key elements (Examples relevant to the question “What is the relationship between dietary patterns and bone health?”):

  • Population of interest [e.g., adults and children (aged two and older), healthy and at risk for chronic disease]
  • Intervention or exposure (e.g., adherence to dietary patterns)
  • Comparator (e.g., different levels of adherence to a dietary pattern; adherence to a different dietary pattern)
  • Outcome(s) of interest [e.g., intermediate outcomes (bone mass, bone mineral density, bone mineral content) and endpoint health outcomes (osteoporosis, rickets, risk of fracture)]

The staff also makes certain that directionality (i.e., the relationship) is not implied by the way the question is asked. For example, a question would read: “what is the impact of X on Y?”, not “how does X increase Y?” Question development is an iterative process that includes vetting by the expert groups and in the future, solicitation of public comments to ensure objectivity.

Back to Top

How are the included and excluded studies determined? 

The NEL works with a team of scientific experts, such as the Dietary Guidelines Advisory Committee or a Technical Expert Collaborative, to develop literature search and sort plans for each research question or family of questions. Each predetermined plan identifies:

  • Search terms to find the relevant body of literature;
  • Relevant databases to be searched; and
  • Inclusion/exclusion criteria for determining which of the search results will be included in the body of evidence.

Multiple databases are searched in an iterative fashion to identify relevant citations. Once the search is complete, the identified studies are independently reviewed by two or more CNPP professional staff at the title, abstract, and full-text level to determine which studies meet the inclusion and exclusion criteria in the predetermined search and sort plans.  When a study is excluded, the reason for exclusion is documented. The search and sort plans for each question are available on the NEL website to enhance transparency and to ensure that the reviews could be replicated.

Back to Top

Are industry sponsored research studies included in the NEL review?

All peer-reviewed studies that met the predetermined literature search and sort plan inclusion criteria were considered in the NEL systematic review. The evidence worksheet developed for each paper identifies the study’s funding sources and evaluates the potential for bias.

Why do NEL reviews not include any animal studies as part of the research examined?

The expert groups define the scope of each review and the most relevant evidence to answer each systematic review question. To date, NEL reviews have answered systematic review questions related to human nutrition; thus, only research identified as applicable to humans has been included.
 

How is the search date range determined? 

The literature search date range varies across questions to ensure that the most timely and relevant studies are included in the evidence base. For example, the search date range for questions that build upon findings from previous NEL projects start when the last literature search ended. For some 2015 Dietary Guidelines Advisory Committee literature searches, the search range began in 2009, which is when most of the 2010 Committee’s literature searches ended. The search date ranges for new 2015 Committee’s questions were determined by the Committee members.

Back to Top

Which type of study design is preferred?

A literature search and sort plan was developed for each question to define the eligibility criteria for studies selected for inclusion in each systematic review. All searches were limited to human studies, developed countries, English language, and peer-reviewed publications. Unpublished data, including abstracts and conference proceedings, were not included. A step-wise process is used to determine which study designs are most relevant. The conventional hierarchy of study designs is used as a guide, but the advantages and disadvantages of each study design as it contributes to the evidence is carefully weighed. Experimental and observational study designs may be included. The literature search and sort plan for each question defines additional eligibility criteria for studies to be included in each systematic review. All searches are limited to those studies published in the English language and peer-reviewed publications. Unpublished data, including abstracts and conference proceedings, are not included.

Back to Top

What if I do not see a study included that I think should have been included?

If you identify a study that you believe met the inclusion criteria  (documented in the question search plan and results) for a NEL systematic review, but was not included, check to see if the citation and a reason for exclusion is documented in the excluded articles list in the corresponding literature search and sort plan.  If it was not on the excluded list, please contact us at nutritionevidencelib@cnpp.usda.gov.

 

What process is used to abstract the articles and evaluate the studies included in a NEL systematic review?

NEL evidence abstractors (i.e., highly-qualified professionals with an advanced degree in nutrition or a related field) critically appraise each article and extract relevant data from each included study. These data include the study design and purpose; descriptions of the study population, protocol, and exposures/interventions; statistical analyses; key findings; conclusions; limitations; and funding sources. One component of study evaluation is the NEL Bias Assessment Tool (NEL BAT), an instrument used to assess the risk of bias for each study. This tool helps determine whether any systematic error existed to either over- or underestimate the study results. NEL staff review the extracted data and completed NEL BAT for accuracy. The experts, such as the Dietary Guidelines Advisory Committee Technical Expert Collaborative members, make the final determination regarding key data to extract and quality ratings for each individual study.
 
Note: Prior to the 2015 Dietary Guidelines Advisory Committee, the NEL Research Design and Implementation Checklist was used to assess the methodological quality of each included study.

Back to Top

What criteria are used to determine the quality rating for each study?

The NEL Bias Assessment Tool (NEL BAT), an instrument designed with assistance from a team of leading international systematic review experts, is used to assess the risk of bias (i.e., internal validity) of each individual study included in a NEL systematic review. The types of bias addressed in the NEL BAT include selection bias, performance bias, detection bias, and attrition bias. The tool is tailored by study design, with different sets of questions applying to different study types.

Note: Prior to the 2015 Dietary Guidelines Advisory Committee, the NEL Research Design and Implementation Checklist was used to assess the methodological quality of each included study.

Back to Top

How are conclusions drawn from the evidence and what is the role of the NEL staff?

The research questions are developed by expert groups and the NEL staff ensures the questions are accompanied by an analytical framework in a standardized format that includes the following key elements (Examples relevant to the question “What is the relationship between dietary patterns and bone health?”):

  • Population of interest [e.g., adults and children (aged two and older), healthy and at risk for chronic disease]
  • Intervention or exposure (e.g., adherence to dietary patterns)
  • Comparator (e.g., different levels of adherence to a dietary pattern; adherence to a different dietary pattern)
  • Outcome(s) of interest [e.g., intermediate outcomes (bone mass, bone mineral density, bone mineral content) and endpoint health outcomes (osteoporosis, rickets, risk of fracture)]

The staff also makes certain that directionality (i.e., the relationship) is not implied by the way the question is asked. For example, a question would read: “what is the impact of X on Y?”, not “how does X increase Y?” Question development is an iterative process that includes vetting by the expert groups and in the future, solicitation of public comments to ensure objectivity.

Back to Top

Who determines the quality rating for each study and the body of evidence?

The expert group determines the quality rating for each study and the body of the evidence after the following process:

  • The NEL evidence abstractors objectively record study data in a similar way from each included article
  • During data extraction, abstractors complete the NEL Bias Assessment Tool (NEL BAT), an instrument used to assess the risk of bias for each individual study included in the review
  • NEL staff review the extracted data and quality ratings for accuracy and the expert group (such as the Dietary Guidelines Advisory Committee or Technical Expert Collaborative members, review the materials and make the final determination regarding an individual study’s quality rating
  • The expert group deliberates until reaching consensus about the individual study findings and the quality of the body of evidence as a whole.
 
Note: Prior to the 2015 Dietary Guidelines Advisory Committee, the NEL Research Design and Implementation Checklist was used to assess the methodological quality of each included study.

Back to Top

 

Dietary Guidelines

What involvement does the NEL have in development of systematic reviews used to inform the Dietary Guidelines for Americans?

The NEL assists the Dietary Guidelines Advisory Committee’s implementation of the systematic review process to ensure consistency with NEL methodology, transparency of all key decisions, and reproducibility of the review. NEL staff uses standardized documents to guide development of the review framework, conduct the literature search and screening, and coordinate data extraction. From these documents, NEL staff prepares evidence summary materials to support the Committee’s deliberations. The Committee makes all substantive decisions required during the process.

Back to Top

How does the NEL enhance the transparency of the review of the science by the Dietary Guidelines Advisory Committee?

The NEL uses state-of-the-art methodology as part of a structured process that provides visibility to all key decisions made at each step in the process. Each step of the process is documented to ensure transparency and reproducibility. Specific information about each review is available at www.NEL.gov, including the research questions, the related literature search protocol, literature selection decisions, an assessment of the methodological quality of each included study, evidence summary materials, evidence tables, a description of key findings, graded conclusion statements, and identification of research limitations and gaps. The review of the science by the Committee is transparent due to their presentation of systematic review questions, the corresponding analytical frameworks, relevant evidence, key findings, and their conclusions at the public meetings. In addition, the references for studies included in each review were posted online prior to the Committee’s public meetings (www.DietaryGuidelines.gov). Members of the public can provide comments and feedback via the public comment application, which was open throughout the Dietary Guidelines Advisory Committee process. The Committee’s final report, which includes advice to HHS and USDA, is informed by the NEL reviews and includes a summary of the materials available at www.NEL.gov.

Back to Top

How does the NEL work to minimize bias in the review of the science by the Dietary Guidelines Advisory Committee?

Quality control is a key aspect of the predefined six-step NEL process designed to ensure objectivity, transparency, and reproducibility. Bias is minimized at each step in the process as summarized:

  • Inclusion and exclusion criteria to guide the literature search to identify the strongest research are developed by the Committee before the literature search begins.
  • Two reviewers independently screen articles to ensure accurate and unbiased application of inclusion criteria
  • NEL staff create data extraction worksheets, tailored for each systematic review, to ensure that evidence abstractors (i.e., highly-qualified volunteers with an advanced degree in nutrition or a related field) objectively record study data in a similar way from each included article
  • Abstractors complete the NEL Bias Assessment Tool (BAT), an instrument designed with assistance from a team of international systematic review experts, to assess the risk of bias for each individual study included in the review
  • NEL staff peer-review data tables and evidence summaries prior to Committee review.
 
Note: Prior to the 2015 Dietary Guidelines Advisory Committee, the NEL Research Design and Implementation Checklist was used to assess the methodological quality of each included study.

Back to Top

How does the Dietary Guidelines Advisory Committee Report differ from the materials found on the NEL website?

The NEL website provides a detailed evidence portfolio for each of the Dietary Guidelines Advisory Committee’s systematic reviews. Each evidence portfolio contains the question; literature search and sort plan with lists of included and excluded articles; a summary of the body of evidence with an overview table; a graded conclusion statement that answers the research question; recommendations for future research; and a technical abstract. The Advisory Committee’s Report provides overviews of the graded evidence, conclusions, and implications related to all aspects of the Committee’s review process. In addition to summaries of the systematic review findings, the report includes dietary intake data analyses, information about dietary pattern modeling, and information from other resources that informed the Committee’s advice to HHS and USDA.
 
Are the conclusions on the NEL website the same as those in the Dietary Guidelines Advisory Committee’s Report?

Yes. The full Committee deliberated and agreed upon conclusions based on the systematic review of the evidence and open discussion at public meetings. These conclusions are found in the Advisory Report and the NEL website provides transparency to the Committee’s systematic review process.

Back to Top

What is the total number of research articles that were reviewed by the 2015 Dietary Guidelines Advisory Committee and considered for NEL systematic reviews?

The 2015 Advisory Committee reviewed approximately 4,000 manuscripts, and nearly 300 studies were used in the systematic reviews since they met the predetermined criteria for the systematic review questions.

Was the process of reviewing the evidence different for 2015 than it was for the 2010 Dietary Guidelines Advisory Committee?

The systematic review process was updated after the 2010 Dietary Guidelines Advisory Committee’s work to evolve with the latest advancements in systematic review science and approaches. The current process is the gold standard and was informed by leading international systematic review experts, as well as by the process used by the Agency for Healthcare Research and Quality. The main differences from 2010 to 2015 include:
  • Use of an externally validated instrument to assess the risk of bias in included studies (NEL Bias Assessment Tool replaced the Research Design and Implementation Checklists)
  • Independent screening by two staff members for screening and selecting studies for inclusion
  • Research recommendations are highlighted for each question
  • Use of existing systematic reviews and/or reports to answer questions when high-quality reviews are available (quality is rated based on a standardized system, known as AMSTAR)  

Back to Top

Did the public have an opportunity to give input on the Dietary Guidelines Advisory Committee’s process?

Yes. A public comments application was accessible for submitting written comments and support material. A total of 972 written comments on the Committee’s process were received between May 29, 2013 and December 30, 2014. A total of X comments on the Committee’s Report were received from X to Y, 2015. Public comments can be viewed at www.DietaryGuidelines.gov.
 
Oral testimony was heard at the 2nd Dietary Guidelines Advisory Committee meeting. In addition, oral testimony will occur at a public meeting regarding the Committee’s Report in March 2015. A total of 53 organizations or individuals provided oral comments on the Dietary Guidelines Advisory Committee process on January 14, 2014 (2nd public meeting), and X organizations or individuals provided oral comments on the Committee’s Report on March X, 2015.

Back to Top



General

How can I get a copy of the peer-reviewed articles cited?

The NEL cannot provide them as they are copyrighted by the publishing journal. Please use your usual library resources to obtain the cited articles.

Who do I give credit to if I use information cited by the NEL?

If you wish to reference information cited within an NEL review, please cite the original publication. We strongly encourage you to obtain and review the original document to ensure you understand the full context of the information you wish to cite prior to referencing the information.

May I use the information I find on the NEL and apply it to my own program analysis?

Yes. The Nutrition Evidence Library (NEL) is a publically available source of nutrition-related systematic reviews designed to aid policy makers, scientists, educators, students and stakeholders in understanding the state of the science at this point in time, and identify where gaps in the evidence exist.

Can the public submit research to be included in the NEL?

To be considered in a NEL systematic review, research must be published in a peer-reviewed journal. Additionally, studies must meet predetermined inclusion criteria developed by the expert groups. Please submit any questions or comments to: nutritionevidencelib@cnpp.usda.gov.

Back to Top