Increase Font Size Decrease Font Size View as PDF Print

Frequently Asked Questions (FAQs)

Introduction NEL Systematic Review Process Dietary Guidelines General


Introduction

 What is the Nutrition Evidence Library (NEL)?

The NEL is a systematic review library that uses a state-of-the-art methodology to review, evaluate and synthesize food and nutrition-related research. This rigorous, protocol-driven, transparent methodology is designed to minimize bias and ensure relevant, timely, and high-quality systematic reviews to inform Federal nutrition-related policies, recommendations and programs.

Back to Top  

What are the benefits of the Nutrition Evidence Library (NEL)?

The NEL conducts systematic reviews, which are considered the gold standard for objectively synthesizing and interpreting literature research findings to inform policy recommendations and program development.  NEL systematic reviews objectively assess the quality of research on which to base judgments of its strength to support research findings and conclusions.  NEL reviews provide Government policymakers and program leaders a timely, credible, and transparent scientific foundation from which to make decisions that are based on the strongest available scientific evidence.  NEL reviews also ensure Government compliance with the Consolidated Appropriations Act of 2001 (Data Quality Act), which mandates that Federal agencies ensure the quality, objectivity, utility, and integrity of the information used to form Federal guidance.

Back to Top

What can be found on the NEL website? 

The Nutrition Evidence Library (NEL) serves as a publicly available store house for all systematic review documentation. The NEL contains systematic review research questions and related literature search protocols, literature selection decisions, evidence worksheets and an assessment of the methodological quality of each included study, evidence summary materials, and graded conclusion statements.

Back to Top

What is the purpose of the USDA NEL?

The USDA Nutrition Evidence Library was created to review, evaluate and synthesize food and nutrition-related research using a transparent methodology to ensure that relevant, timely and high-quality evidence-based reviews are available to inform nutrition-related public health policies, recommendations and programs. A secondary product of the NEL is the identification of gaps in the data or methodological limitations related to a particular area of study and recommendations for future research.

Back to Top

NEL Systematic Review Process

What is a NEL systematic review?

A NEL systematic review is a state-of-the-art method for evaluating scientific evidence to answer a precise question or series of questions that may form the foundation for Federal policy and programs. Nutrition Evidence Library systematic reviews are conducted by a multidisciplinary research team based on a predefined approach and criteria to:

  • Develop systematic review questions
  • Search, screen, and select studies to review
  • Extract data and assess the risk of bias of the research
  • Describe and synthesize the evidence
  • Develop conclusion statements and grade the evidence
  • Identify research recommendations

Electronic tools are used to describe and document each step to ensure, objectivity, transparency, and reproducibility of the process.

Back to Top

How are NEL systematic review topics selected?

For the 2010 Dietary Guidelines Advisory Committee (DGAC), the charge was to review the state of the science on nutrition and health, and advise USDA and HHS if the 2005 Dietary Guidelines for Americans (DGA) warranted revision. The DGAC used the 2005 DGA topics as a baseline and identified additional related and emerging topics for consideration and prioritization.

Back to Top

How are the included and excluded studies determined? 

A team of scientific experts such as the DGAC or Technical Expert Collaborative (TEC), assisted by the NEL staff and research librarian, developed and executed literature search and sort plans for each research question or family of questions. Each predetermined plan identified:

  • Search terms to find the related body of literature
  • Inclusion/exclusion criteria for sorting the search results

Multiple databases were searched in iterative fashion to identify relevant citations.  Committee members and NEL staff also scoured the references of included studies and related review papers to identify additional papers that were germane to the question.

Once the list of relevant citations was assembled, the pre-determined search and sort plan inclusion and exclusion criteria were used to review and sort the literature.  Lists were sorted at the title, abstract, and full text level to determine which studies met the inclusion and exclusion criteria.  When a study was excluded, the reason for exclusion was documented on the excluded articles list in the search and sort plan for that question, which is posted on the NEL website.

Back to Top

Were industry sponsored research studies included in the NEL review?

All peer-reviewed studies that met the predetermined literature search and sort plan inclusion criteria were considered in the NEL systematic review. The evidence worksheet developed for each paper identifies the study’s funding sources and evaluates the potential for bias.

How was the search date range determined? 

The literature search date range varied by individual question or family of questions. For example, the search date range for questions that built upon 2005 DGAC findings generally started in June 2004, which is when the 2005 Committee’s literature search ended.  The search date ranges for new 2010 DGAC questions were determined by the Subcommittee members. For some questions, such as the effect of dietary sodium reduction on blood pressure in children, no date limit was set.  For others, the Subcommittee based the range on their knowledge of the literature.  Frequently, iterative exploratory searches were conducted to help refine the search results.

Back to Top

Which type of study design is preferred?

A literature search and sort plan was developed for each question to define the eligibility criteria for studies selected for inclusion in each systematic review. All searches were limited to human studies, developed countries, English language, and peer-reviewed publications. Unpublished data, including abstracts and conference proceedings, were not included. 

A step-wise process was used to determine which research designs were considered to examine a question. Study designs included intervention trials, observational studies, ecological studies, systematic reviews, and meta-analyses. If systematic reviews were used, primary studies included in these reviews were excluded. If multiple systematic reviews considered an overlapping body of primary studies, this was noted in the evidence summary. The conventional hierarchy of study designs (below) was used as a guide, but the advantages and disadvantages of each study design as it contributed to the evidence has been carefully weighted.

  • Hierarchy of Study Designs
    • Experimental studies
      • Meta-analysis of randomized controlled trials
      • Randomized controlled trial
      • Quasi-experimental
    • Observational studies
      • Cohort study
      • Case-control study
      • Cross-sectional study
      • Before and after study
      • Case series
Back to Top

Why were cross-sectional studies excluded from some literature reviews?

Many searches initially included all study designs. However, for a number of questions, cross-sectional studies were eventually excluded from a review when sufficient evidence from studies with a stronger design was available.

Why were systematic reviews given priority over narrative reviews?

Conducting a systematic review involves an objective, rigorous scientific process designed to reduce the risk of error or bias. A detailed protocol is written to guide:

  • Question development
  • Identification of the scientific literature used to answer the question
  • Critically evaluation of the methodological rigor of each study
  • Synthesis of the evidence
  • Conclusion development and grading of the strength and quality of evidence supporting the conclusion

Use of these rigorous and well defined methods distinguishes systematic reviews from narrative reviews. The quality of systematic reviews may vary, so the NEL incorporates the Research Design and Implementation Checklists to rate the quality of each systematic review used. Since narrative reviews are more susceptible to error or bias, narrative reviews are typically excluded from systematic reviews, but may be hand searched to identify primary studies for inclusion in the systematic review.

Back to Top

Are systematic reviews from other sources considered for inclusion?

To determine whether to incorporate an existing systematic review or not, the following criteria are considered:

  • Relevancy to the question
  • Search dates
  • Strength and consistency of methodology

Comprehensive systematic reviews, with well-documented methodology and rigorous criteria for judging methodological quality of included studies and grading the body of evidence, were occasionally selected to serve as a baseline for a review in cases where the seminal research on a question was considered to be “settled science.” For the 2010 Dietary Guidelines, a number of American Dietetic Association Evidence Analysis Library reviews were updated in the NEL. Available systematic reviews (e.g. 2009 AHRQ report Vitamin D and Calcium: A Systematic Review of Health Outcomes) or reports based on systematic reviews (e.g. Physical Activity Guidelines Advisory Committee Report, 2008) that were deemed to be current and comprehensive representations of available literature were not duplicated. Results from the 2007 World Cancer Research Fund/American Institute for Cancer Research; Food, Nutrition, Physical Activity, and the Prevention of Cancer: A Global Perspective report were used to substantiate recommendations related to food, nutrient, and diet intake and cancer-related outcomes.

Back to Top

What do I do if I don’t see a study included that I think should have been included?

If you identify a study that you believe met the inclusion criteria for a NEL systematic review, but was not included. First, check for the citation and reason for exclusion in the excluded articles list in the relevant NEL literature search and sort plan.  If it was not on the excluded list, please contact us nutritionevidencelib@cnpp.usda.gov.

What process was used to evaluate the included studies cited in a NEL systematic review?  How were the studies graded?  Who graded the studies? 

An evidence worksheet template was used to extract relevant evidence from each included study.  The worksheet includes the study design and purpose; descriptions of the study population, protocol and interventions, and statistical measures; as well as, summary tables, key findings, and author’s conclusions. Each evidence worksheet also contains a Research Design and Implementation Checklist of preset criteria to assess the methodological rigor and quality of the study’s design and implementation. This assessment determines the study’s quality rating. NEL evidence abstractors prepared the draft evidence worksheets and NEL staff reviewed them for accuracy.  The DGAC or TEC members made the final determination regarding an individual study’s quality rating.

Back to Top

What criteria are used to determine the quality rating for each study?

The NEL Research Design and Implementation Checklist was used to assess the methodological rigor and quality of each included primary and review study.   The checklist for primary studies consists of four relevancy questions and ten validity questions based on the Agency for Healthcare Research Quality (AHRQ) domains for research studies. Sub-questions under each validity question identify important aspects of sound study design and execution relevant to each domain. Some sub-questions also identify how the domain applies in specific research designs. The ten validity questions are:

  1. Was the research question clearly stated?
  2. Was the selection of subjects free from bias?
  3. Were the study groups comparable?
  4. Was the method of handling withdrawals described?
  5. Was blinding used to prevent bias?
  6. Were the intervention/exposures/procedures described in detail?
  7. Were the outcomes clearly defined. Measurements valid and reliable?
  8. Was the statistical analysis appropriate for study design and outcomes?
  9. Were the conclusions supported by results? Biases and limitations considered?
  10. Was bias due to study funding or sponsorship unlikely?

Review studies are assessed using a similar style checklist with four relevancy questions and ten validity questions. This Research Design and Implementation Checklist is a component of the evidence worksheet for all review articles found on the portal.

Back to Top

Who determines the quality rating for each study?

The NEL evidence abstractors prepare the draft evidence worksheets, which include the quality rating determination. NEL staff reviewed the worksheets for accuracy. The DGAC or TEC members approved the worksheet’s content and made the final determination regarding an individual study’s quality rating.

Who abstracts the articles and creates the evidence worksheets? 

NEL evidence abstractors critically appraised each article and prepared the draft evidence worksheets. NEL staff reviewed the worksheets for accuracy. The DGAC or TEC members reviewed and approved the worksheet’s content and made the final determination regarding an individual study’s quality rating.

If there was disagreement about assessment of the literature and respective studies, what was the process to resolve these differences?

For the 2010 DGAC, subcommittees and full Committee held meetings to discuss and come to consensus on the assessment and interpretation of individual studies, as well as the body of evidence considered for each systematic review.

How were Conclusions drawn from the evidence and what was the role of the NEL staff?

Developing and grading each Conclusion was a deliberative and time-consuming process that benefited from group interaction. The strength of the evidence supporting the conclusion statement was graded using the DGAC’s predetermined criteria, which assessed the quality (relevance and validity) and size of the studies, the quantity of studies, the consistency and agreement across studies, the magnitude of the effect or public health impact, and generalizability to the population of interest.

All conclusion statements were developed and graded by the DGAC or TEC; the NEL staff’s role was to provide guidance on methodology, implement the DGAC or TEC's research protocols, and assist in summarizing the evidence for the DGAC or TEC's review and synthesis.

Back to Top

Why do USDA NEL evidence-based reviews not include any animal studies as part of the research examined?

The DGAC or TEC defined the scope of the review at the beginning of the process. Since the objective of the DGAC advisory report was to update the Dietary Guidelines for Americans, only science developed far enough to show clear effect in humans was included.

How does expert opinion interface with the NEL process?

NEL systematic reviews included peer-reviewed primary studies, systematic reviews and meta-analyses, not editorial publications.  Also, the DGAC or TEC employed its collective professional judgment to synthesize the systematic review findings and develop the science-based conclusion statements and implications.

Back to Top

Dietary Guidelines

Are the conclusions found on the NEL website the same as those found in the 2010 DGAC Report?

Yes. The full Committee discussed and agreed upon conclusions based on the systematic review of the evidence and open discussion at public meetings. These conclusions are found in the Advisory Report and the NEL website provides transparency to the Dietary Guideline Advisory Committee systematic review process.

What is the total number of research articles that were reviewed by the 2010 DGAC?

The subcommittee reviewed approximately 1950 manuscripts, and approximately 900 were used in the systematic reviews. Over 100 of these studies were systematic reviews or meta-analyses which represent a larger number of studies.

How does the 2010 DGAC Report differ from the materials found on the NEL website?

The NEL website provides the detailed evidence portfolio for each of the 2010 Dietary Guidelines Advisory Committee’s systematic reviews. Each evidence portfolio in the NEL contains the literature search and sort plan with lists of included and excluded articles; evidence worksheets for each included article; an overview table and evidence summary, which synthesize the body of evidence; and a graded conclusion statement that answers the research question. The 2010 DGAC Report provides overviews, interpretations, and implications related to all aspects of the Committee’s Dietary Guidelines review process. It includes summaries of systematic review findings, information about dietary pattern modeling, dietary intake data analyses, and information about other resources that informed its review.

Back to Top

General

How can I get a copy of the actual articles from peer-reviewed journals that are cited?

The NEL cannot provide them as they are copyrighted by the publishing journal. Please use your usual library resources to obtain the cited articles.

Who do I give credit to if I use information cited by the USDA NEL?

If you wish to cite information that is cited within a NEL systematic review, please cite the original publication. Before citing information that is cited in another document, it is advisable to obtain the original document and review it to understand the full context of the information that you wish to cite.

Can I use the information I find on the NEL and apply it to my own program analysis?

Yes. The Nutrition Evidence Library is a publically available source of nutrition-related systematic reviews designed to aid policy makers, scientists, educators, students and stakeholders in understanding the state of the science at this point in time, and identify where gaps in the evidence exist.

Can the public submit research to be included in the Nutrition Evidence Library (NEL)?

To be considered by the 2015 DGAC in a NEL systematic review, research should be published in a peer-reviewed journal by December 31, 2013. Additionally, to be included in a NEL systematic review, studies must meet predetermined inclusion criteria developed by the DGAC. The public can inform the DGAC about relevant research by submitting a Public Comment at www.DietaryGuidelines.gov.

Back to Top