Introduction

Prev Chronic Dis

Preventing Chronic Disease

1545-1151

Centers for Disease Control and Prevention

20950540

2995597

PCDv76_09_0195

Systematic Review

Peer Reviewed

Quality of Systematic Reviews of Observational Nontherapeutic Studies

Shamliyan

Tatyana

MD, MSMinnesota Evidence-based Practice Center, University of Minnesota School of Public Health

D351 Mayo (MMC 197), 420 Delaware St SE, Minneapolis, MN 55455

612-624-1185shaml005@umn.edu

Kane

Robert L.

MDUniversity of Minnesota School of Public Health, Minneapolis, MinnesotaJansen

Stacy

MPHUniversity of Minnesota School of Public Health, Minneapolis, Minnesota

112010

15102010

A133

Introduction

High-quality epidemiologic research is essential in reducing chronic diseases. We analyzed the quality of systematic reviews of observational nontherapeutic studies.

Methods

We searched several databases for systematic reviews of observational nontherapeutic studies that examined the prevalence of or risk factors for chronic diseases and were published in core clinical journals from 1966 through June 2008. We analyzed the quality of such reviews by using prespecified criteria and internal quality evaluation of the included studies.

Results

Of the 145 systematic reviews we found, fewer than half met each quality criterion; 49% reported study flow, 27% assessed gray literature, 2% abstracted sponsorship of individual studies, and none abstracted the disclosure of conflict of interest by the authors of individual studies. Planned, formal internal quality evaluation of included studies was reported in 37% of systematic reviews. The journal of publication, topic of review, sponsorship, and conflict of interest were not associated with better quality. Odds of formal internal quality evaluation (odds ratio [OR], 1.10 per year; 95% confidence interval [CI], 1.02-1.19) and either planned, formal internal quality evaluation or abstraction of quality criteria of included studies (OR, 1.17 per year; 95% CI, 1.08-1.26) increased over time, without positive trends in other quality criteria from 1990 through June 2008. Systematic reviews with internal quality evaluation did not meet other quality criteria more often than those that ignored the quality of included studies.

Conclusion

Collaborative efforts from investigators and journal editors are needed to improve the quality of systematic reviews.

Introduction

Valid epidemiologic research is essential in preventing chronic diseases (1-3). Assessing the quality of observational studies is an important part of evidence synthesis (4). Systematic reviews have become key tools in evidence synthesis from a growing number of epidemiologic studies (5). Producing high-quality systematic reviews is essential to developing generalizable and actionable conclusions (6,7). Quality criteria for systematic reviews have been proposed by working groups that developed the Meta-analysis of Observational Studies in Epidemiology (MOOSE), Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), and a measurement tool for assessment of multiple systematic reviews (AMSTAR) (8-12). The working groups and the Cochrane handbook (13) addressed those criteria for systematic reviews that more likely result in biased results, including bias in selection of the studies or the information within studies by the reviewers (14-18) or bias in the publication of positive significant results (6,15,19,20).

Previous research and guidelines (13,21-23) focus on systematic reviews of interventional therapeutic studies. Validity of observational nontherapeutic studies of prevalence of chronic diseases or risk factors for diseases is essential for effective preventive public health actions (24,25). Our aim was to evaluate the quality of systematic reviews of observational nontherapeutic studies that examined the incidence and prevalence of chronic conditions and risk factors for diseases. The criteria we used to determine the reporting and methodologic quality in systematic reviews were from published standards (8-12). We hypothesized that the quality of systematic reviews differs by the time when the study was published, the country in which the study was conducted, the journal of publication, the sponsorship of the study, and whether a conflict of interest was disclosed. We hypothesized also that systematic reviews with internal quality evaluation of the included studies would have better quality, demonstrating commitment to quality of evidence.

MethodsData sources

We searched MEDLINE via PubMed and via Ovid MEDLINE, the Cochrane Library (26) and working groups, WorldCat (27), and Scirus (28) to find systematic reviews of observational nontherapeutic studies published in English from 1966 through June 2008 in core clinical journals (exact search string is listed in Appendix Table 1). We used the definitions of core clinical journals from the Abridged Index Medicus (119 indexed titles). We defined observational nontherapeutic studies as observations of patient outcomes that did not examine procedures concerned with the remedial treatment or prevention of diseases (29).

Study selection

Three investigators independently decided on the eligibility of the studies according to recommendations from the Cochrane Handbook for Systematic Reviews of Interventions (13). We reviewed abstracts to exclude comments, expert opinions, letters, case reports, systematic reviews of interventional studies, and systematic reviews of studies of diagnostic accuracy of tests.

Data extraction

Evaluations of the studies and data extraction were performed independently by 2 researchers. Predefined categorical responses to the checklist items were abstracted into our spreadsheet. Errors in data extraction were assessed by a comparison of the data charts with the original articles (13,30). Any discrepancies were discussed and resolved. The quality criteria that we abstracted were based on guidelines for determining the reporting and methodologic quality of systematic reviews (8-12).

To evaluate selection bias, we abstracted whether the authors of systematic reviews described the search strategy (yes, no, or partially); yes indicated that the authors reported time periods of searches, searched databases, and exact search string. We abstracted whether the authors of systematic reviews described study flow (yes, no, or partially); yes indicated that the authors reported the list of retrieved citations, the list of excluded studies, and justification for exclusion.

We abstracted as dichotomous variables whether the authors of systematic reviews did any of the following:

Stated the aim of the review and the primary and secondary hypotheses of the review.

Included or justified exclusion of articles published in languages other than English.

Searched for gray literature, including abstracts and unpublished studies, to evaluate publication bias (21).

Described any contact with authors of the included studies.

Analyzed sponsorship of and conflict of interest in the included studies.

We abstracted how the authors of systematic reviews described obtained statistical methods with justification and models for pooling with fixed or random effects models in sufficient detail to be replicated (no pooling, random, or fixed). We abstracted whether the authors of pooling analyses reported statistical tests for heterogeneity and whether heterogeneity was statistically significant (not reported, not significant, or significant).

We used 3 categories to classify whether the authors of systematic reviews had evaluated the quality of included studies by using developed or previously published checklists or scales (31): 1) the authors stated planned, formal internal quality evaluations; 2) the authors abstracted selected criteria of external or internal validity without using a planned, formal, and comprehensive internal quality evaluation; and 3) the authors did not conduct internal quality evaluations. We further categorized the studies that evaluated quality criteria to compare studies with no mention of internal quality evaluation of the included studies. We also compared studies with and without planned formal internal quality evaluation. We abstracted with dichotomous responses blinding and reliability testing (reported or not reported) of internal quality evaluations.

We abstracted several explanatory variables that could be related to the quality of systematic reviews:

The year of publication, defined as a continuous variable. We created categories of 4- or 5-year periods: 1990 to 1994, 1995 to 1999, 2000 to 2004, and 2005 through June 2008.

The journals of publication.

The country where the systematic reviews were performed.

The sponsorship of the reviews. Those that had either governmental or foundational support or were fellowships were defined as having nonprofit support.

The disclosure of conflict of interest by authors of reviews (either not disclosed, disclosed as no conflict of interest, or disclosed conflict of interest).

The number of disclosed relationships with industry, defined as a continuous variable.

The sponsor's participation in data collection, analysis, and interpretation of the results of the review.

The review outcomes as risk factors for prevalence or incidence of chronic conditions or diseases.

Data synthesis

We summarized the results in evidence tables. We used prespecified categories of dependent and independent variables and did not force the data into binary categories for definitive tests of significance. We used univariate logistic regression to examine the association between internal quality evaluation and the year of the publication by using the Wald test. Odds ratios (ORs) were calculated with binary logit models and Fisher's scoring method technique. We computed the fractions of systematic reviews meeting various quality criteria in each of the 4 time periods considered. The proportions of systematic reviews that met different levels of each quality criterion were evaluated by using χ² tests and Fisher's exact tests in cases of small numbers. All calculations were performed at 95% confidence intervals (CIs) by using 2-sided P values with SAS version 9.1.3 (SAS Institute Inc, Cary, North Carolina).

Results

We found 145 eligible systematic reviews of observational nontherapeutic studies (study flow in the Appendix Figure) (32-176). The number of published systematic reviews increased from 17 during 1990-1994 to 56 during 2005-2008. Most of the studies were conducted in the United States (55 publications) or in the United Kingdom (28 publications) (Appendix Table 2). Half of the systematic reviews (73 publications) were funded by nonprofit organizations; 56 (39%) reviews did not publish their funding sources, 4 reviews received industry support, and 10 were sponsored jointly by industry and nonprofit organizations. Almost three-fourths (106) of the authors of systematic reviews did not disclose conflict of interest; 35 publications stated that the authors do not have any conflict of interest; and 4 studies were conducted by authors who reported conflict of interest. The studies were published in 49 journals. Most systematic reviews (122 studies) assessed risk factors for chronic diseases, 19 summarized estimates of prevalence or incidence, 2 studies reported prevalence and associations with risk factors, and 2 studies examined levels of risk factors. Most studies reported incidence and risk factors for cardiovascular diseases (46 studies) or cancer (26 studies).

Quality of systematic reviews

Less than half of the studies reported study flow (49%), assessed gray literature (27%), or addressed language bias (29%) (Table 1). Only 2% of reviews abstracted sponsorship of individual studies and none abstracted the disclosure of conflict of interest by the authors of individual studies that were eligible for the reviews. Pooling was performed in 137 studies; of these, 62% used a random effects model; 57% reported detecting significant heterogeneity across the studies; and 19% did not provide any information about statistical heterogeneity in pooled estimates. The proportion of systematic reviews that met quality criteria including study flow, assessment of gray literature, or the abstraction of funding sources of included studies did not show significant trends from 1990 through 2008. The proportion of systematic reviews that assessed language bias increased from 8% during 1995-1999 to 41% during 2005-2008. In later years, more studies reported using random effects models (79% during 2005-2008 vs 39% during 1995-1999) and tests for statistical heterogeneity (89% during 2005-2008 vs 65% during 1995-1999).

Internal quality evaluation

Planned and detailed quality assessment of included studies was reported in 37% of systematic reviews, and 18% abstracted more than 1 criterion of external or internal quality; significant positive trends were reported during the evaluated time (Table 1). Quality assessment was masked in 3 studies. Development of the appraisals, including references to previously published tools, was reported in 32 studies, but only 6 tested interobserver agreement for quality assessment.

Quality of systematic review by explanatory factors

The quality of systematic reviews did not differ much by study location or by the journal of publication. Systematic reviews of prevalence or incidence or risk factors of the diseases did not differ in their quality measures. Sponsorship was not associated with quality of the reviews. The role of conflict of interest was impossible to establish because the authors of 56 reviews did not disclose funding and authors of 106 reviews did not disclose conflict of interest.

Explanatory factors of internal quality evaluation of included studies

The journal of publication, topic of the review, and continent where the review was conducted were not associated with the likelihood of internal quality evaluation. Systematic reviews of risk factors tended to conduct internal quality evaluation of the included studies more often than reviews of incidence or prevalence or of levels of risk factors. Systematic reviews sponsored by nonprofit organizations conducted internal quality evaluations of individual studies more often than reviews that received corporate funding. Systematic reviews that disclosed conflict of interest conducted internal quality evaluation of individual studies less frequently (10 of 39 studies; 26%) than reviews with no disclosure (44 of 106 studies; 42%). Odds of formal internal quality evaluation (OR, 1.10 per year; 95% CI, 1.02-1.19) and either planned, formal internal quality evaluation or abstraction of quality criteria (OR, 1.17 per year; 95% CI, 1.08-1.26) increased over time. Disclosure of conflict of interest by the authors of systematic reviews was not associated with greater odds of internal quality evaluation.

Quality of systematic reviews by internal quality evaluation

Complete documentation of the literature search including time period, databases searched, and exact literature search strings was less common among reviews with planned, formal internal quality evaluation (48 studies, 35%) than among reviews without it (90 studies, 65%) (Table 2). However, reviews that either abstracted selected quality criteria or planned, formal internal quality evaluation reported partial (6 studies) or complete (74 studies) information about the literature search more often than studies that did not evaluate quality of included studies (64 studies). Reviews that did not justify exclusion of non-English studies ignored quality of individual studies more often (72 studies) than reviews with planned, formal internal quality evaluation (31 studies). The same pattern was present for publication bias: the reviews that did not mention gray literature also ignored the quality of individual studies. The reviews reporting attempts to contact the authors of included studies either performed planned, formal internal quality evaluation or abstracted selected quality criteria more often than reviews without such attempts (OR, 2.3; 95% CI, 1.1-4.7). Reviews with complete reporting of study flow performed planned, formal internal quality evaluation or abstracted quality criteria more often (51 studies) than reviews without study flows (20 studies). More than half of systematic reviews without planned, formal internal quality evaluation (44 studies) also did not report study flow.

The association between quality of systematic reviews and sponsor participation in the data collection, analyses, and interpretation was difficult to analyze because this information was either omitted or reported in various ways. Less than 10% of systematic reviews contained a clear statement that the sponsors did not play any role in gathering the studies or analyzing or interpreting the results and did not influence the content of the manuscript. Other reviews omitted mention of the role of the sponsor in approval of the manuscript or provided a general statement that sponsors did not influence the conclusions or the content of the paper. Two reviews included statements of unconditional or unrestricted sponsorship of the meta-analyses.

Discussion

Our analyses showed that less than half of the systematic reviews of nontherapeutic observational studies that were published in core clinical journals met each quality criterion. Quality of systematic reviews did not improve over time. Planned, formal internal quality evaluations of the included studies was reported in less than half of systematic reviews, but the prevalence of internal quality evaluations has increased during the last decade. Our findings are in concordance with previously published methodologic analyses of systematic reviews that also found inconsistent quality and incomplete internal quality evaluation of individual studies (6). Methodologic analyses of systematic reviews that focused on particular diseases or conditions demonstrated that half of the publications had major flaws in design and reporting. For instance, systematic reviews of therapies for renal diseases failed to assess the methodologic quality of included studies (177). Methodologic analyses of systematic reviews of interventions showed that 69% of those randomly selected in MEDLINE meta-analyses did not analyze quality of trials (22). Most (68%) systematic reviews of diagnostic tests for cancer did not provide formal assessments of study quality (178). We also found that the quality of reviews did not differ among types of studies (incidence or risk factors for diseases), types of diseases, or journal of publication.

Journal commitment to high-quality research, however, was associated with improved reporting quality of the publications. For example, adoption by journals of the Consolidated Standards of Reporting Trials (CONSORT) improved the quality of the publications of interventional studies (179,180). An endorsement of the developed standards for observational studies including MOOSE and STROBE checklists may also improve quality of the publications. We did not analyze how many core clinical journals adopted these standards and how quality of the publications changed depending on this adaptation. Peer review of submitted manuscripts should include quality assessment using validated tools (12).

We could not identify the factors that can explain differences in quality of systematic reviews. The role of sponsorship and conflict of interest could not be estimated because of poor reporting of this information. The quality and reliability of quality evaluation of the included studies is unclear because development of the appraisals was described in a small proportion of systematic reviews (32 of 80 studies), and only 6 of 80 studies tested interobserver agreement for quality assessment. We did not evaluate all reviews of observational studies that were published in epidemiologic journals. However, it is unlikely that the quality of reviews published in other journals would be better than those in core clinical journals. Future research should investigate the factors that can explain differences in the quality of systematic reviews.

Peer reviewed publications of high-quality systematic reviews can provide the best available research evidence for evidence-based public health (24). Evidence-based decisions can improve public health practice in preventing incidence and progression of chronic diseases (25). In our analysis, less than half of the systematic reviews of observational nontherapeutic studies met quality criteria established in the MOOSE, STROBE, and AMSTAR statements. Internal quality evaluation of included studies should be an essential part of evidence synthesis, but only half of the reviews reported such evaluation. Collaborative efforts from investigators and journal editors are needed to improve quality of systematic reviews.

This article is based on research conducted by the Minnesota Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, Maryland (contract no. 290-02-0009).

We thank our reviewers David Atkins, MD, John Hoey, MD, and Christine Laine, MD, for reviewing and commenting on the draft; our collaborating experts, Mohammed Ansari, MBBS, Ethan Balk, MD, Nancy Berkman, PhD, Chantelle Garritty, Mark Grant, MD, Gail Janes, PhD, Margaret Maglione, MPP, David Moher, PhD, Mona Nasser, DDS, Gowri Raman, MD, Karen Robinson, MD, Jodi Segal, MD, and Thomas Trikalinos, PhD, for their scientific input throughout this project; and Carmen Kelly, PharmD, our task order officer, and Stephanie Chang, MD, medical officer, at AHRQ for their guidance throughout the project. We also thank librarian Judith Stanke for her contributions to the literature search; research assistants Emily Zabor, candidate for the master of science degree (MS) in biostatistics, and Akweley Ablorh, candidate for MS in biostatistics, for the data abstraction, quality control, and synthesis of evidence; Zhihua Bian, candidate for MS in biostatistics, for her statistical help; Zhiyuan Xu, candidate for MS in applied economics, for his work creating the ACCESS database; Dean McWilliams for his assistance in database development; Qi Wang, research fellow, for her statistical expertise in reliability testing; Susan Duval, PhD, for her help estimating sample size; Marilyn Eells for editing and formatting the report; and Nancy Russell, MLS, and Rebecca Schultz for their assistance gathering data from the experts and formatting the tables, and Christa Prodzinski for quality control of the data.

The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the US Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors' affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above. URLs for nonfederal organizations are provided solely as a service to our users. URLs do not constitute an endorsement of any organization by CDC or the federal government, and none should be inferred. CDC is not responsible for the content of Web pages found at these URLs.

Suggested citation for this article: Shamliyan T, Kane RL, Jansen S. Quality of systematic reviews of observational nontherapeutic studies. Prev Chronic Dis 2010;7(6) http://www.cdc.gov/pcd/issues/2010/nov/09_0195.htm. Accessed [date].

Bero

Jadad

127119973742

How consumers and policymakers can use systematic reviews for decision making

Ann Intern Med9214251

Chan

Morton

Shekelle

11 Pt 1

2004806812

Systematic reviews for evidence-based management: how to find them and what to do with them

Am J Manag Care15623270

Biss

Zaza

Pappaioanou

Fielding

Wright-De

Agüero

LTruman

181 Suppl20003543

Developing an evidence-based Guide to Community Preventive Services — methods. The Task Force on Community Preventive Services

Am J Prev Med10806978

West

King

Carey

Lohr

McKoy

Sutton

Rockville (MD)

Agency for Healthcare Research and Quality

2002Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment no. 47 (Prepared by the Research Triangle Institute — University of North Carolina Evidence-based Practice Center under Contract no. 290-97-0011)AHRQ Publication no. 02-E016

US Preventive Services Task Force, Agency for Healthcare Research and Quality

2008

Rockville (MD)

Agency for Healthcare Research and Quality

The guide to clinical preventive services, 2008: recommendations of the US Preventive Services Task Force

Moher

Tetzlaff

Tricco

Sampson

Altman

432007e78

Epidemiology and reporting characteristics of systematic reviews

PLoS Med17388659

Tricco

Tetzlaff

Sampson

Fergusson

Cogo

Horsley

6152008422434

Few systematic reviews exist documenting the extent of bias: a systematic review

J Clin Epidemiol18394534

Stroup

Berlin

Morton

Olkin

Williamson

Rennie

28315200020082012

Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group

JAMA10789670

Tooth

Ware

Bain

Purdie

Dobson

16132005280288

Quality of reporting of observational longitudinal research

Am J Epidemiol15671260

Vandenbroucke

von Elm

Altman

Gøtzsche

Mulrow

Pocock

1862007805835

Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration

Epidemiology18049195

von Elm

Altman

Egger

Pocock

Gotzsche

Vandenbroucke

3709596200714531457

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies

Lancet18064739

Shea

Grimshaw

Wells

Boers

Andersson

Hamel

2007710

Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

BMC Med Res Methodol17302989

Higgins

JPT

Green

2008

Chichester, West Sussex (GB)

John Wiley and Sons Ltd

Cochrane handbook for systematic reviews of interventions

Whitehead

2002

Chichester, West Sussex (GB)

John Wiley and Sons Ltd

Meta-analysis of controlled clinical trials

Egger

Smith

316712419986166

Bias in location and selection of studies

BMJ9451274

Dickersin

26310199013851389

The existence of publication bias and risk factors for its occurrence

JAMA2406472

Dickersin

91 Suppl19971521

How important is publication bias? A synthesis of available data

AIDS Educ Prev9083596

Dickersin

Min

1993428

NIH clinical trials and publication bias

Online J Curr Clin TrialsDoc No 50:[4,967 words; 53 paragraphs]

Ravnskov

305684419921519

Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome

BMJ1638188

Moher

Soeken

Sampson

Ben-Porat

Berman

200223

Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine

BMC Pediatr11914146

Methods guide for comparative effectiveness reviews. Agency for Healthcare Research and QualityAccessed June 2010

http://www.effectivehealthcare.ahrq.gov/repFiles/20090427IdenttifyingTopics.pdf

Jadad

Cook

Jones

Klassen

Tugwell

Moher

28031998278280

Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals

JAMA9676681

Liberati

Altman

Tetzlaff

Mulrow

Gøtzsche

Ioannidis

2009339b2700

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration

BMJ19622552

Brownson

Fielding

Maylahn

200930175201

Evidence-based public health: a fundamental concept for public health practice

Annu Rev Public Health19296775

Brownson

Gurney

Land

5519998697

Evidence-based decision making in public health

J Public Health Manag Pract10558389

Higgins

Green

The Cochrane handbook for systematic reviews of interventions 4.2.6, updated September 2006Accessed July 2010

http://www2.cochrane.org/resources/handbook/handbook.pdf

WorldCat. OCLC Online Computer Library CenterAccessed July 2010

http://www.oclc.org/worldcat/

Scirus for scientific information onlyElsevier

Accessed July 2010

http://www.scirus.com

Principles

epidemiology

2006

Atlanta (GA)

US Department of Health and Human Services, Centers for Disease Control and Prevention, Office of Workforce and Career Development

Principles of epidemiology in public health practice: an introduction to applied epidemiology and biostatistics3rd edition

Higgins

JPT

Green

Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.0.2 [updated September 2009]London (GB)

Cochrane Collaboration

2009

Accessed July 2010

http://www.cochrane-handbook.org/

Lundh

Gøtzsche

2008822

Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies

BMC Med Res Methodol18426565

Bracken

3 Pt 2

1990552557

Oral contraception and congenital malformations in offspring: a review and meta-analysis of the prospective studies

Obstet Gynecol2143279

Romieu

Berlin

Colditz

6611199022532263

Oral contraceptives and breast cancer. Review and meta-analysis

Cancer2147122

Haughey

Gates

Arfken

Harvey

101

2 Pt 1

1992105112

Meta-analysis of second malignant tumors in head and neck cancer: the case for an endoscopic screening protocol

Ann Otol Rhinol Laryngol1531402

Lemon

Heidel

Rodriguez-Sierra

6921992457465

Increased catechol estrogen metabolism as a risk factor for nonfamilial breast cancer

Cancer1309434

McKenna

93119926977

Differences in vitamin D status between countries in young adults and the elderly

Am J Med1385673

Morris

Audet

Angelillo

Chalmers

Mosteller

8271992955963

Chlorination, chlorination by-products, and cancer: a meta-analysis

Am J Public Health1535181

Myers

Basinski

1529199217671772

Coffee and coronary heart disease

Arch Intern Med1387780

Becker

Smith

Rhodes

22119938691

Incidence of cardiac arrest: a neglected factor in evaluating survival rates

Ann Emerg Med8424622

Brownson

Novotny

Perry

15341993469475

Cigarette smoking and adult leukemia. A meta-analysis

Arch Intern Med8435026

Ernst

Resch

118121993956963

Fibrinogen as a cardiovascular risk factor: a meta-analysis and review of the literature

Ann Intern Med8489110

Katerndahl

18191993539544

Panic and prolapse. Meta-analysis

J Nerv Ment Dis8245921

Harris

Barraclough

7361994281296

Suicide as an outcome for medical disorders

Medicine (Baltimore)7984079

Kawachi

Colditz

Stone

7231994269275

Does coffee drinking increase the risk of coronary heart disease? Results from a meta-analysis

Br Heart J7946780

Law

Thompson

Wald

30869251994373379

Assessing possible hazards of reducing serum cholesterol

BMJ8124144

Law

Wald

Thompson

30869251994367372

By how much and how quickly does reduction in serum cholesterol concentration lower risk of ischaemic heart disease?

BMJ8043072

Steffen

Kane

Shapiro

Billo

Schoellhorn

van Damme

272111994885889

Epidemiology and prevention of hepatitis A in travelers

JAMA8078167

Zhang

Begg

2341994682690

Is Trichomonas vaginalis a cause of cervical neoplasia? Results from a combined analysis of 24 studies

Int J Epidemiol8002180

Everhart

Wright

27320199516051609

Diabetes mellitus as a risk factor for pancreatic cancer

JAMA7745774

Feinberg

Blackshear

Laupacis

Kronmal

Hart

15551995469473

Prevalence, age distribution, and gender of patients with atrial fibrillation. Analysis and implications

Arch Intern Med7864703

Ritchie

Kildea

34689801995931934

Is senile dementia "age-related" or "ageing-related"? — evidence from meta-analysis of dementia prevalence in the oldest old

Lancet7564727

Raman-Wilms

Tseng

Wighardt

Einarson

Koren

8511995141149

Fetal genital effects of first-trimester sex hormone exposure: a meta-analysis

Obstet Gynecol7800312

Hatsukami

Fischman

27619199615801588

Crack cocaine and cocaine hydrochloride. Are the differences myth or reality?

JAMA8918856

Hill

Schoener

1539199611431146

Age-dependent decline of attention deficit hyperactivity disorder

Am J Psychiatry8780416

Hackshaw

Law

Wald

31571141997980988

The accumulated evidence on lung cancer and environmental tobacco smoke

BMJ9365295

Kluijtmans

Kastelein

Lindemans

Boers

Heil

Bruschke

968199725732577

Thermolabile methylenetetrahydrofolate reductase in coronary artery disease

Circulation9355896

Law

Hackshaw

31571121997841846

A meta-analysis of cigarette smoking, bone mineral density and risk of hip fracture: recognition of a major effect

BMJ9353503

Law

Morris

Wald

31571141997973980

Environmental tobacco smoke exposure and ischaemic heart disease: an evaluation of the evidence

BMJ9365294

Danesh

Collins

Appleby

Peto

27918199814771482

Association of fibrinogen, C-reactive protein, albumin, or leukocyte count with coronary heart disease: meta-analyses of prospective studies

JAMA9600484

French

Brocklehurst

10581998827835

The effect of pregnancy on survival in women infected with HIV: a systematic review of the literature and meta-analysis

Br J Obstet Gynaecol9746374

Forgie

Wells

Laupacis

Fergusson

15861998610616

Preoperative autologous donation decreases allogeneic transfusion but increases exposure to all red blood cell transfusion: results of a meta-analysis. International Study of Perioperative Transfusion (ISPOT) Investigators

Arch Intern Med9521225

Huang

Sridhar

Chen

Hunt

1146199811691179

Meta-analysis of the relationship between Helicobacter pylori seropositivity and gastric cancer

Gastroenterology9609753

Johnston

Colford

JrGress

5121998411418

Oral contraceptives and the risk of subarachnoid hemorrhage: a meta-analysis

Neurology9710012

Lazarou

Pomeranz

Corey

27915199812001205

Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies

JAMA9555760

Ray

15819199821012106

Meta-analysis of hyperhomocysteinemia as a risk factor for venous thromboembolic disease

Arch Intern Med9801176

Spencer-Green

15861998595600

Outcomes in primary Raynaud phenomenon: a meta-analysis of the frequency, rates, and predictors of transition to secondary diseases

Arch Intern Med9521223

Stratton

Pharoah

Smith

Easton

Ponder

10551998493499

A systematic review and meta-analysis of family history and risk of ovarian cancer

Br J Obstet Gynaecol9637117

Zock

Katan

6811998142153

Linoleic acid intake and cancer risk: a review and meta-analysis

Am J Clin Nutr9665108

Zondervan

Yudkin

Vessey

Dawes

Barlow

Kennedy

105119989399

The prevalence of chronic pelvic pain in women in the United Kingdom: a systematic review

Br J Obstet Gynaecol9442169

Angelillo

Villari

77111999906915

Residential exposure to electromagnetic fields and childhood leukaemia: a meta-analysis

Bull World Health Organ10612886

Vupputuri

Allen

Prerost

Hughes

Whelton

340121999920926

Passive smoking and the risk of coronary heart disease — a meta-analysis of epidemiologic studies

N Engl J Med10089185

Shaffer

Hall

Vander

Bilt

899199913691376

Estimating the prevalence of disordered gambling behavior in the United States and Canada: a research synthesis

Am J Public Health10474555

Wittrup

Tybjaerg-Hansen

Nordestgaard

9922199929012907

Lipoprotein lipase mutations, plasma lipids and lipoproteins, and risk of ischemic heart disease. A meta-analysis

Circulation10359734

Yoder

Sabbagha

Gross

Zelop

935 Pt 21999869872

The second-trimester fetus with isolated choroid plexus cysts: a meta-analysis of risk of trisomies 18 and 21

Obstet Gynecol10912435

Christen

Ajani

Glynn

Hennekens

16042000422434

Blood levels of homocysteine and increased risks of cardiovascular disease: causal or casual?

Arch Intern Med10695683

Cleophas

Hornstra

van Hoogstraten

van der Meulen

869200010051009

1005-9, A8

Homocysteine, a risk factor for coronary artery disease or not? A meta-analysis

Am J Cardiol11053715

DiMatteo

Lepper

Croghan

16014200021012107

Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence

Arch Intern Med10904452

35592022000451455

Effect of breastfeeding on infant and child mortality due to infectious diseases in less developed countries: a pooled analysis. WHO Collaborative Study Team on the Role of Breastfeeding on the Prevention of Infant Mortality

Lancet10841125

Wilson

Gibson

Willan

Cook

16072000939944

Effect of smoking cessation on mortality after myocardial infarction: meta-analysis of cohort studies

Arch Intern Med10761958

Zeegers

Tan

Dorant

van

Den

Brandt

8932000630639

The impact of characteristics of cigarette smoking on urinary tract cancer risk: a meta-analysis of epidemiologic studies

Cancer10931463

Danesh

Whincup

Walker

Lennon

Thomson

Appleby

10319200123232327

Fibrin D-dimer and coronary heart disease: prospective study and meta-analysis

Circulation11352877

Eaden

Abrams

Mayberry

4842001526535

The risk of colorectal cancer in ulcerative colitis: a meta-analysis

Gut11247898

Faraone

Doyle

Mick

Biederman

1587200110521057

Meta-analysis of the association between the 7-repeat allele of the dopamine D(4) receptor gene and attention deficit hyperactivity disorder

Am J Psychiatry11431226

Horta

Kramer

Platt

9122001304307

Maternal smoking and the risk of early weaning: a meta-analysis

Am J Public Health11211645

Rebora

13772001943947

Baldness and coronary artery disease: the dermatologic point of view of a controversial issue

Arch Dermatol11453815

Cannon

Jones

Murray

1597200210801092

Obstetric complications and schizophrenia: historical and meta-analytic review

Am J Psychiatry12091183

Hellermann

Jacobsen

Gersh

Rodeheffer

Reeder

Roger

11342002324330

Heart failure after myocardial infarction: a review

Am J Med12361819

Huang

Sridhar

Hunt

359930020021422

Role of Helicobacter pylori infection and non-steroidal anti-inflammatory drugs in peptic-ulcer disease: a meta-analysis

Lancet11809181

Huncharek

Kupelnick

927200211731177

Use of topical sunscreens and the risk of malignant melanoma: a meta-analysis of 9067 patients from 11 case-control studies

Am J Public Health12084704

Juul

Tybjaerg-Hansen

Steffensen

Kofoed

Jensen

Nordestgaard

10012002310

Nordestgaard BG. Factor V Leiden: The Copenhagen City Heart Study and 2 meta-analyses

Blood12070000

Kelly

Rosand

Kistler

Shih

Silveira

Plomaritoglou

5942002529536

Homocysteine, MTHFR 677C-->T polymorphism, and risk of ischemic stroke: results of a meta-analysis

Neurology12196644

Klerk

Verhoef

Clarke

Blom

Kok

Schouten

28816200220232031

MTHFR 677C-->T polymorphism and risk of coronary heart disease: a meta-analysis

JAMA12387655

Kozer

Nikfar

Costei

Boskovic

Nulman

Koren

1876200216231630

Aspirin consumption during the first trimester of pregnancy and congenital anomalies: a meta-analysis

Am J Obstet Gynecol12501074

Law

Watt

Wald

16221200224052410

The underlying risk of death after myocardial infarction in the absence of treatment

Arch Intern Med12437397

Wald

Law

Morris

325737420021202

Homocysteine and cardiovascular disease: evidence on causality from a meta-analysis

BMJ12446535

Wald

Link

185120024552

Risk of human immunodeficiency virus infection in herpes simplex virus type 2-seropositive persons: a meta-analysis

J Infect Dis11756980

Benjamin

JrPoole

Steinbach

Rowen

Walsh

112

3 Pt 1

2003634640

Neonatal candidemia and end-organ damage: a critical appraisal of the literature using meta-analytic techniques

Pediatrics12949295

Clarfield

16318200322192229

The decreasing prevalence of reversible dementias: an updated meta-analysis

Arch Intern Med14557220

Cole

Dendukuri

1606200311471156

Risk factors for depression among elderly community subjects: a systematic review and meta-analysis

Am J Psychiatry12777274

100

Gisbert

Garcia-Buey

Pajares

Moreno-Otero

1256200317231732

Prevalence of hepatitis C virus infection in B-cell non-Hodgkin's lymphoma: systematic review and meta-analysis

Gastroenterology14724825

101

Glatt

Faraone

Tsuang

16032003469476

Association between a functional catechol O-methyltransferase gene polymorphism and schizophrenia: meta-analysis of case-control and family-based studies

Am J Psychiatry12611827

102

Halbert

Isonaka

George

Iqbal

1235200316841692

Interpreting COPD prevalence estimates: what is the true burden of disease?

Chest12740290

103

Huang

Zheng

Sumanac

Irvine

Hunt

1256200316361644

Meta-analysis of the relationship between cagA seropositivity and gastric cancer

Gastroenterology14724815

104

Rey

Kahn

David

Shrier

36193612003901908

Thrombophilic disorders and fetal loss: a meta-analysis

Lancet12648968

105

Riboli

Norat

783 Suppl2003559S569S

Epidemiologic evidence of the protective effect of fruit and vegetables on cancer risk

Am J Clin Nutr12936950

106

Scholten-Peeters

GGM

Verhagen

Bekkering

van

der

WindtBarnsley

Oostendorp

10412003303322

Prognostic factors of whiplash-associated disorders: a systematic review of prospective cohort studies

Pain12855341

107

Thurnham

McCabe

Northrop-Clewes

Nestel

3629401200320522058

Effects of subclinical infection on plasma retinol concentrations and assessment of prevalence of vitamin A deficiency: meta-analysis

Lancet14697804

108

Zeegers

Jellema

Ostrer

978200318941903

Empiric risk of prostate carcinoma for relatives of patients with prostate carcinoma: a meta-analysis

Cancer12673715

109

Burzotta

Paciaroni

De Stefano

Crea

Maseri

Leone

90120048286

G20210A prothrombin gene polymorphism and coronary ischaemic syndromes: a phenotype-specific meta-analysis of 12,034 subjects

Heart14676252

110

Casas

Bautista

Humphries

Hingorani

10911200413591365

Endothelial nitric oxide synthase genotype and ischemic heart disease: meta-analysis of 26 studies involving 23,028 subjects

Circulation15007011

111

Casas

Hingorani

Bautista

Sharma

6111200416521661

Meta-analysis of genetic studies in ischemic stroke: thirty-two genes involving approximately 18,000 cases and 58,000 controls

Arch Neurol15534175

112

Song

Daviglus

Liu

Van Horn

Dyer

10922200427052711

Accumulated evidence on fish consumption and coronary heart disease mortality: a meta-analysis of cohort studies

Circulation15184295

113

Huang

Chen

Poole

6212200421982202

APOE-[epsilon]2 allele associated with higher prevalence of sporadic Parkinson disease

Neurology15210882

114

Klement

Cohen

Boxman

Joseph

Reif

805200413421352

Breastfeeding and risk of inflammatory bowel disease: a systematic review with meta-analysis

Am J Clin Nutr15531685

115

Kovalevsky

Gracia

Berlin

Sammel

Barnhart

16452004558563

Evaluation of the association between hereditary thrombophilias and recurrent pregnancy loss: a meta-analysis

Arch Intern Med15006834

116

Levitan

Song

Ford

Liu

16419200421472155

Is nondiabetic hyperglycemia a risk factor for cardiovascular disease? A meta-analysis of prospective studies

Arch Intern Med15505129

117

Lovett

Coull

Rothwell

6242004569573

Early risk of recurrence by subtype of ischemic stroke in population-based incidence studies

Neurology14981172

118

Mitsikostas

Sfikakis

Goadsby

127

Pt 5

200412001209

A meta-analysis for headache in systemic lupus erythematosus: the evidence and the myth

Brain15047589

119

Montanez

Ruskin

Hebert

Lamas

Hennekens

16492004943948

Prolonged QTc interval and risks of total and cardiovascular mortality and sudden death in the general population: a review and qualitative overview of the prospective cohort studies

Arch Intern Med15136301

120

Woodbury

Houghton

501020042224

22-4, 26, 28 passim

Prevalence of pressure ulcers in Canadian healthcare settings

Ostomy Wound Manage15509880

121

Bolland

Grey

Gamble

Reid

903200515251530

Association between primary hyperparathyroidism and increased body weight: a meta-analysis

J Clin Endocrinol Metab15613408

122

Contopoulos-Ioannidis

Manoli

Ioannidis

11552005963972

Meta-analysis of the association of beta2-adrenergic receptor polymorphisms with asthma phenotypes

J Allergy Clin Immunol15867853

123

Dauchet

Amouyel

Dallongeville

658200511931197

Fruit and vegetable consumption and risk of stroke: a meta-analysis of cohort studies

Neurology16247045

124

Etminan

Takkouche

Isorna

Samii

3307482200563

Risk of ischaemic stroke in people with migraine: systematic review and meta-analysis of observational studies

BMJ[Errata published in: BMJ 2005;330(7491):596. BMJ 2005;330(7487):345.]

15596418

125

Fazel

Wheeler

Danesh

3659467200513091314

Prevalence of serious mental disorder in 7000 refugees resettled in western countries: a systematic review

Lancet15823380

126

García-Closas

Malats

Silverman

Dosemeci

Kogevinas

Hein

36694862005649659

NAT2 slow acetylation, GSTM1 null genotype, and risk of bladder cancer: results from the Spanish Bladder Cancer Study and meta-analyses

Lancet16112301

127

Lee

Witte

Momot

Schmidt

Kaufman

Harley

5212200539663974

The mannose-binding lectin gene polymorphisms and systemic lupus erythematosus: two case-control studies and a meta-analysis

Arthritis Rheum16320344

128

Lin

August

10512005182192

Genetic thrombophilias and preeclampsia: a meta-analysis

Obstet Gynecol15625161

129

McDonald

Murphy

Beyene

Ohlsson

19312005141152

Perinatal outcomes of in vitro fertilization twins: a systematic review and meta-analyses

Am J Obstet Gynecol16021072

130

Palmer

Pankratz

Bostwick

6232005247253

The lifetime risk of suicide in schizophrenia: a reexamination

Arch Gen Psychiatry15753237

131

Sin

Man

1276200519521959

The relationship between reduced lung function and cardiovascular mortality: a population-based study and a systematic review of the literature

Chest15947307

132

Boudville

Prasad

Knoll

Muirhead

Thiessen-Philbrook

Yang

14532006185196

Meta-analysis: risk for hypertension in living kidney donors

Ann Intern Med16880460

133

Clark

Tobias

Ness

11722006e291e297

Association between bone density and fractures in children: a systematic review and meta-analysis

Pediatrics16452336

134

de Boer

Verbeek

van Dijk

10712006111

Adult survivors of childhood cancer and unemployment: a metaanalysis

Cancer16718655

135

Di Castelnuovo

Costanzo

Bagnardi

Donati

Iacoviello

de Gaetano

16622200624372445

Alcohol dosing and total mortality in men and women: an updated meta-analysis of 34 prospective studies

Arch Intern Med17159008

136

Flores-Mateo

Navas-Acien

Pastor-Barriuso

Guallar

8442006762773

Selenium and coronary heart disease: a meta-analysis

Am J Clin Nutr17023702

137

Galassi

Reynolds

119102006812819

Metabolic syndrome and risk of cardiovascular disease: a meta-analysis

Am J Med17000207

138

Huxley

Barzi

Woodward

332753320067378

Excess risk of fatal coronary heart disease associated with diabetes in men and women: meta-analysis of 37 prospective cohort studies

BMJ16371403

139

Kahlenborn

Modugno

Potter

Severs

8110200612901302

Oral contraceptive use as a risk factor for premenopausal breast cancer: a meta-analysis

Mayo Clin Proc17036554

140

Larsson

Giovannucci

Wolk

1314200612711283

Folate intake, MTHFR polymorphisms, and risk of esophageal, gastric, and pancreatic cancer: a meta-analysis

Gastroenterology17030196

141

Mahid

Minor

Soto

Hornung

Galandiuk

8111200614621471

Smoking and inflammatory bowel disease: a meta-analysis

Mayo Clin Proc17120402

142

Owen

Martin

Whincup

Smith

Cook

845200610431054

Does breastfeeding influence risk of type 2 diabetes in later life? A quantitative analysis of published evidence

Am J Clin Nutr17093156

143

Ownby

Crocco

Acevedo

John

Loewenstein

6352006530538

Depression and risk for Alzheimer disease: systematic review, meta-analysis, and metaregression analysis

Arch Gen Psychiatry16651510

144

Pavia

Pileggi

Nobile

Angelillo

835200611261134

Association between fruit and vegetable consumption and oral cancer: a meta-analysis of observational studies

Am J Clin Nutr16685056

145

Riddle

Sanders

Putnam

Tribble

7452006891900

Incidence, etiology, and impact of diarrhea among long-term travelers (US military and similar populations): a systematic review

Am J Trop Med Hyg16687698

146

Rutledge

Reis

Linke

Greenberg

Mills

488200615271537

Depression in heart failure: a meta-analytic review of prevalence, intervention effects, and associations with clinical outcomes

J Am Coll Cardiol17045884

147

Smith

Lichtman

Bracken

Shlipak

Phillips

DiCapua

4710200619871996

Renal impairment and outcomes in heart failure: systematic review and meta-analysis

J Am Coll Cardiol16697315

148

Weis

Shah

Lajous

Shields

124120065460

The association between host susceptibility factors and uveal melanoma: a meta-analysis

Arch Ophthalmol16401785

149

Williams

Higgins

Brayne

9112006815

Systematic review of prevalence studies of autism spectrum disorders

Arch Dis Child15863467

150

Bahekar

Singh

Saha

Molnar

Arora

15452007830837

The prevalence and incidence of coronary heart disease is significantly increased in periodontitis: a meta-analysis

Am Heart J17967586

151

Baurecht

Irvine

Novak

Illig

Bühler

Ring

1206200714061412

Toward a major risk factor for atopic eczema: meta-analysis of filaggrin polymorphism data

J Allergy Clin Immunol17980411

152

Bellamy

Casas

Hingorani

Williams

33576272007974

Pre-eclampsia and risk of cardiovascular disease and cancer in later life: systematic review and meta-analysis

BMJ17975258

153

Conde-Agudelo

Rosas-Bermudez

Kafury-Goeta

19642007297308

Effects of birth spacing on maternal health: a systematic review

Am J Obstet Gynecol17403398

154

Dehghan

Kardys

de Maat

Uitterlinden

Sijbrands

Bootsma

5632007872878

Genetic variation, C-reactive protein levels, and incidence of diabetes

Diabetes17327459

155

Eichler

Puhan

Steurer

Bachmann

15352007722731

722-31, 731.e1-8

Prediction of first coronary events with the Framingham score: a systematic review

Am Heart J17452145

156

Gami

Witt

Howard

Erwin

Gami

Somers

4942007403414

Metabolic syndrome and risk of incident cardiovascular events and death: a systematic review and meta-analysis of longitudinal studies

J Am Coll Cardiol17258085

157

Grulich

van Leeuwen

Falster

Vajdic

370958120075967

Incidence of cancers in people with HIV/AIDS compared with immunosuppressed transplant recipients: a meta-analysis

Lancet17617273

158

Havemann

Henderson

El-Serag

5612200716541664

The association between gastro-oesophageal reflux disease and asthma: a systematic review

Gut17682001

159

Hirtz

Thurman

Gwinn-Hardy

Mohamed

Chaudhuri

Zalutsky

6852007326337

How common are the "common" neurologic disorders?

Neurology17261678

160

Huxley

Owen

Whincup

Cook

Rich-Edwards

Smith

855200712441250

Is birth weight a risk factor for ischemic heart disease in later life?

Am J Clin Nutr17490959

161

Krishna

Kim

10732007522529

Ethnic differences in risk factors for subarachnoid hemorrhage

J Neurosurg17886550

162

Langan

Flohr

Williams

14312200715701577

The role of furry pets in eczema: a systematic review

Arch Dermatol18087010

163

Larsson

Wolk

8632007556565

Obesity and colon and rectal cancer risk: a meta-analysis of prospective studies

Am J Clin Nutr17823417

164

Larsson

Wolk

1325200717401745

Coffee consumption and risk of liver cancer: a meta-analysis

Gastroenterology17484871

165

Liu

Korantzopoulos

4915200716421648

Association between C-reactive protein and recurrence of atrial fibrillation after successful electrical cardioversion: a meta-analysis

J Am Coll Cardiol17433956

166

Loza

Chang

12032007578585

Association between Q551R IL4R genetic variants and atopic asthma risk demonstrated by meta-analysis

J Allergy Clin Immunol17586032

167

Pittas

Lau

Dawson-Hughes

926200720172029

The role of vitamin D and calcium in type 2 diabetes. A systematic review and meta-analysis

J Clin Endocrinol Metab17389701

168

Polanczyk

de Lima

Horta

Biederman

Rohde

16462007942948

The worldwide prevalence of ADHD: a systematic review and metaregression analysis

Am J Psychiatry17541055

169

Rona

Keil

Summers

Gislason

Zuidmeer

Sodergren

12032007638646

The prevalence of food allergy: a meta-analysis

J Allergy Clin Immunol17628647

170

Sarwar

Danesh

Eiriksdottir

Sigurdsson

Wareham

Bingham

11542007450458

Triglycerides and the risk of coronary heart disease: 10,158 incident cases among 262,525 participants in 29 Western prospective studies

Circulation17190864

171

Snoep

Hovens

Eikenboom

van der Bom

Jukema

Huisman

15422007221231

Clopidogrel nonresponsiveness in patients undergoing percutaneous coronary intervention with stenting: a systematic review and meta-analysis

Am Heart J17643570

172

Zintzaras

Kaditis

16122007172178

Sleep-disordered breathing and blood pressure in children: a meta-analysis

Arch Pediatr Adolesc Med17283303

173

Ageno

Becattini

Brighton

Selby

Kamphuisen

1171200893102

Cardiovascular risk factors and venous thromboembolism: a meta-analysis

Circulation18086925

174

Barclay

Petocz

McMillan-Price

Flood

Prvan

Mitchell

8732008627637

Glycemic index, glycemic load, and chronic disease risk — a meta-analysis of observational studies

Am J Clin Nutr18326601

175

Conde-Agudelo

Villar

Lindheimer

19812008722

Maternal infection and risk of preeclampsia: systematic review and metaanalysis

Am J Obstet Gynecol18166297

176

Schunkert

Gotz

Braund

McGinnis

Tregouet

Mangino

11713200816751684

Repeated replication and a prospective meta-analysis of the association between chromosome 9p21.3 and coronary artery disease

Circulation18362232

177

Mrkobrada

Thiessen-Philbrook

Haynes

Iansavichus

Rehman

Garg

34200811021114

Need for quality improvement in renal systematic reviews

Clin J Am Soc Nephrol18400967

178

Mallett

Deeks

Halligan

Hopewell

Cornelius

Altman

33375652006413

Systematic reviews of diagnostic tests in cancer: review of methods and reporting

BMJ16849365

179

Needleman

Worthington

Moher

Schulz

Altman

2112008712

Improving the completeness and transparency of reports of randomized trials in oral health: the CONSORT statement

Am J Dent18435368

180

Plint

Moher

Morrison

Schulz

Altman

Hill

18552006263267

Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review

Med J Aust16948622

Appendix

Table 1

Search Strategy and Exact Search Strings Used to Identify Systematic Reviews of Observational Studies, Scales and Checklists for Internal Quality Evaluation, and Studies About Bias in Observational Research, 1966 Through June 2008

Search Method	No. of Articles Identified
Search strategy for Ovid MEDLINE
1. exp Research Design/st [Standards]	4,303
2. exp Chronic Disease/ep [Epidemiology]	1,619
3. exp Urinary Incontinence/ep [Epidemiology]	1,155
4. exp Fecal Incontinence/ep [Epidemiology]	328
5. exp "Sleep Initiation and Maintenance Disorders"/ep [Epidemiology]	565
6. exp Depression/ep [Epidemiology]	4,700
7. exp Depressive Disorder/ep [Epidemiology]	6,816
8. exp Myocardial Infarction/	43,531
9. 6 or 7	11,214
10. 8 and 9	105
11. 2 or 3 or 4 or 5 or 10	3,636
12. 1 and 11	9
13. exp Data Collection/mt, st [Methods, Standards]	36,173
14. exp "Bias (Epidemiology)"/	25,369
15. exp Questionnaires/st [Standards]	3,879
16. exp Evidence-Based Medicine/	27,487
17. 13 or 14 or 15 or 16	86,857
18. 11 and 17	127
19. 12 or 18	133
20. limit 19 to english language	124
21. exp "Predictive Value of Tests"/	62,290
22. exp "Reproducibility of Results"/	126,475
23. 21 or 22	182,941
24. 11 and 23	126
25. limit 24 to english language	121
26. 20 or 25	224
27. exp randomized controlled trial/	151,027
28. 11 and 27	74
29. exp research design/	134,468
30. 28 and 29	15
31. 1 and 16	547
32. ep.fs.	434,923
33. exp epidemiology/	6,500
34. 32 or 33	437,784
35. 31 and 34	29
36. exp incidence/	81,260
37. exp prevalence/	83,713
38. 36 or 37	157,239
39. 31 and 38	14
40. 26 or 30 or 35 or 39	268
41. limit 40 to english language	267
42. limit 41 to journal article	251
43. from 42 keep 1-251	251
MEDLINE search via PubMed
("Biomedical Research/methods"[MeSH] OR "Biomedical Research/organization and administration"[MeSH] OR "Biomedical Research/standards"[MeSH] OR "Biomedical Research/statistics and numerical data"[MeSH] OR "Biomedical Research/trends"[MeSH]) Limits: Humans, Journal Article, English	3,703
"Epidemiologic Studies"[MeSH] AND "Research Design/standards"[MeSH] AND ("Evaluation Studies as Topic/classification"[MeSH] OR "Evaluation Studies as Topic/methods"[MeSH] OR "Evaluation Studies as Topic/standards"[MeSH]) Limits: Humans, Journal Article, English	59
"Publishing/standards"[MeSH] AND "Epidemiologic Methods"[MeSH] AND "Research Design/standards"[MeSH] Limits: Humans, Journal Article, English	65
"STROBE Initiative"[Corporate Author]	10
"Bias (Epidemiology)"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Epidemiologic Methods"[MeSH] AND "Research Design/standards"[MeSH] Limits: Humans, Journal Article, English	97
"Evidence-Based Medicine"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Epidemiologic Methods"[MeSH] AND "Research Design/standards"[MeSH] Limits: Humans, Journal Article, English	25
"Research Design/standards"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Epidemiologic Measurements"[MeSH] AND "Bias (Epidemiology)"[MeSH] Limits: Humans, Journal Article, English AND "Incidence"[MeSH] Limits: Humans, Journal Article, English	8
"Research Design/standards"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Epidemiologic Measurements"[MeSH] AND "Bias (Epidemiology)"[MeSH] Limits: Humans, Journal Article, English AND "Prevalence"[MeSH] Limits: Humans, Journal Article, English	7
("Prevalence"[MeSH]) AND systematic[sb] "Working group" Limits: English	15
[CN] Limits: Humans, Meta-Analysis, English, Core clinical journals	2
("Prevalence"[MeSH]) AND systematic[sb] Limits: Humans, Meta-Analysis, English, Core clinical journals	83
Moher D[author]	198
"Epidemiologic Studies"[MeSH] Limits: Humans, Meta-Analysis, English AND "Incidence"[MeSH] Limits: Humans, Meta-Analysis, English Limits: Humans, Meta-Analysis, English, Core clinical journals	57
"Epidemiologic Studies"[MeSH] AND "Incidence"[MeSH] Limits: Humans, Meta-Analysis, English	236
"Epidemiologic Studies"[MeSH] AND "Incidence"[MeSH] AND Evidence Limits: Humans, Meta-Analysis, English	52
"Incidence"[MeSH] Limits: Humans, Meta-Analysis, English	635
"Risk"[MeSH] AND "Epidemiologic Studies"[MeSH] Limits: Humans, Meta-Analysis, English, Core clinical journals	273
"Prevalence"[MeSH] Limits: Humans, Meta-Analysis, English, Core clinical journals	84
Altman DG[author]	7
Higgins J[author]	3
"Review Literature as Topic"[MeSH] AND "Research Design/standards"[MeSH] AND "Epidemiologic Studies"[MeSH] Limits: Humans, English, Core clinical journals	0
"Review Literature as Topic"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Quality control"[MeSH]	1
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH] AND "Research Design/standards"[MeSH]	0
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH]	0
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	0
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND ("Data Collection/methods"[MeSH] OR "Data Collection/standards"[MeSH])	5
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Bias (Epidemiology)"[MeSH]	1
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND ("Questionnaires/methods"[MeSH] OR "Questionnaires/standards"[MeSH])	0
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Evidence-Based Medicine"[MeSH]	2
"Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Reproducibility of Results"[MeSH]	3
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH] AND "Research Design/standards"[MeSH]	0
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH]	0
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	0
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND ("Data Collection/methods"[MeSH] OR "Data Collection/standards"[MeSH])	16
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Bias (Epidemiology)"[MeSH]	6
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND ("Questionnaires/methods"[MeSH] OR "Questionnaires/standards"[MeSH])	1
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Evidence-Based Medicine"[MeSH]	0
"Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Reproducibility of Results"[MeSH]	12
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH] AND "Research Design/standards"[MeSH]	0
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH]	0
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	1
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND ("Data Collection/methods"[MeSH] OR "Data Collection/standards"[MeSH])	18
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Bias (Epidemiology)"[MeSH]	7
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND ("Questionnaires/methods"[MeSH] OR "Questionnaires/standards"[MeSH])	1
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Evidence-Based Medicine"[MeSH]	4
"Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Reproducibility of Results"[MeSH]	10
"Health Care Quality, Access, and Evaluation"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH] AND "Research Design/standards"[MeSH]	0
"Health Care Quality, Access, and Evaluation"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Peer Review, Research"[MeSH]	0
"Health Care Quality, Access, and Evaluation"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	4
"Health Care Quality, Access, and Evaluation"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Evidence-Based Medicine"[MeSH]	8
"Health Care Quality, Access, and Evaluation"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Bias (Epidemiology)"[MeSH]	33
"Models, Statistical"[MeSH] AND "Risk Factors"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	0
"Models, Statistical"[MeSH] AND "Incidence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	0
"Models, Statistical"[MeSH] AND "Prevalence"[MeSH] AND "Chronic Disease/epidemiology"[MeSH] AND "Research Design/standards"[MeSH]	0
"Epidemiologic Studies"[MeSH] AND "Models, Statistical"[MeSH] AND "Research Design/standards"[MeSH]	47
"Prevalence"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Models, Statistical"[MeSH] AND "Bias (Epidemiology)"[MeSH]	61
"Incidence"[MeSH] AND "Epidemiologic Studies"[MeSH] AND "Models, Statistical"[MeSH] AND "Bias (Epidemiology)"[MeSH]	66
"Research Design/standards"[MeSH] AND ("Biomedical Research/methods"[MeSH] OR "Biomedical Research/organization and administration"[MeSH] OR "Biomedical Research/standards"[MeSH] OR "Biomedical Research/statistics and numerical data"[MeSH] OR "Biomedical Research/trends"[MeSH]) Limits: Humans, Journal Article, English	62

Abbreviations: MeSH, Medical Subject Heading term; sb, subset; CN, corporate author.

Figure 1

Study flow to identify systematic reviews of observational studies, scales, and checklists for planned formal internal quality evaluation, and studies about bias in observational research, 1990 through June 2008.

Flow chart

This flow chart shows 3 boxes. The top box reads

Total references — 1,053

Medline via PubMed — 904

Related publications — 14

WorldCat — 2

Scirus — 28

Manual search — 70

Guidelines, textbooks, consensus statements — 23

Published by the authors — 11

Cochrane Library — 1

It is connected by a vertical line to 2 additional boxes. The left box reads

Included studies — 397

Systematic reviews for the present article — 145

Tools (scales and checklists) — 96

Working groups — 13

Methodologic studies about bias in observational research — 143

The right box reads

Excluded studies — 656

Ethical guidelines — 3

Reviews — 23

Centers for Disease Control and Prevention study — 4

Related to interventions — 209

Related to diagnostic studies — 54

Not relevant to research questions — 363

Table 2

Quality of Systematic Review and Meta-Analyses of Nontherapeutic Observational Studies Published in Core Clinical Journals, 1990 through June 2008

Publication Characteristics	Outcome	Estimate	Assessment of Quality of Included Studies
Bracken, 1990 (32)Country: United StatesJournal: Obstet GynecolSponsorship: Not reportedConflict of interest (COI): Not reportedSponsor participation in data analyses: Not reported	Congenital malformations in offspring	Risk	No
Romieu et al, 1990 (33)Country: United StatesJournal: CancerSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Breast cancer	Risk	Quality criteria abstracted
Haughey et al, 1992 (34)Country: United StatesJournal: Ann Otol Rhinol LaryngolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Second malignant tumors in head and neck cancer	Risk	No
Lemon et al, 1992 (35)Country: United StatesJournal: CancerSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Nonfamilial breast cancer	Continuous variable	No
McKenna, 1992 (36)Country: United KingdomJournal: Am J MedSponsorship: Nonprofit organization, nursing homeCOI: Not reportedSponsor participation in data analyses: Not reported	Differences in vitamin D status	Prevalence	Quality criteria abstracted
Morris et al, 1992 (37)Country: United StatesJournal: Am J Public HealthSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Cancer	Risk	Yes
Myers and Basinski, 1992 (38)Country: CanadaJournal: Arch Intern MedSponsorship: Nonprofit organization, awardCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	No
Becker et al, 1993 (39)Country: United StatesJournal: Ann Emerg MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Survival of cardiac arrest	Risk	No
Brownson et al, 1993 (40)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Adult leukemia	Risk	Yes
Ernst and Resch, 1993 (41)Country: AustriaJournal: Ann Intern Med Sponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Cardiovascular risk factor	Risk	No
Katerndahl, 1993 (42)Country: United StatesJournal: J Nerv Ment DisSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Panic disorder and mitral valve prolapse	Risk	Yes
Harris and Barraclough, 1994 (43)Country: United KingdomJournal: MedicineSponsorship: IndustryCOI: Not reportedSponsor participation in data analyses: Not reported	Suicide	Risk	No
Kawachi et al, 1994 (44)Country: United StatesJournal: Br Heart JSponsorship: Industry, scholarshipCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	No
Law et al, 1994 (45)Country: United KingdomJournal: BMJSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Hazards of reducing serum cholesterol	Risk	No
Law et al, 1994 (46)Country: United KingdomJournal: BMJSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Ischemic heart disease	Risk	No
Steffen et al, 1994 (47)Country: SwitzerlandJournal: JAMASponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Hepatitis A	Risk	No
Zhang and Begg, 1994 (48)Country: United StatesJournal: Int J EpidemiolSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Cervical neoplasia	Risk	No
Everhart and Wright, 1995 (49)Country: United StatesJournal: JAMASponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Pancreatic cancer	Risk	No
Feinberg et al, 1995 (50)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Atrial fibrillation	Prevalence	No
Ritchie and Kildea, 1995 (51)Country: FranceJournal: LancetSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Senile dementia	Prevalence	No
Raman-Wilms et al, 1995 (52)Country: CanadaJournal: Obstet GynecolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Fetal genital effects	Risk	No
Hatsukami and Fischman, 1996 (53)Country: United StatesJournal: JAMASponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Use of crack cocaine and cocaine hydrochloride	Prevalence	No
Hill and Schoener, 1996 (54)Country: United StatesJournal: Am J PsychiatrySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Attention deficit hyperactivity disorder	Prevalence	No
Hackshaw et al, 1997 (55)Country: United KingdomJournal: BMJSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The views expressed are those of the authors and not necessarily those of the Department of Health."	Lung cancer	Risk	No
Kluijtmans et al, 1997 (56)Country: NetherlandsJournal: CirculationSponsorship: Nonprofit organization, industryCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary artery disease	Risk	No
Law and Hackshaw, 1997 (57)Country: United KingdomJournal: BMJSponsorship: NoneCOI: Reported as not a conflict of interestSponsor participation in data analyses: None	Hip fracture	Risk	No
Law et al, 1997 (58)Country: United KingdomJournal: BMJSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The Department of Health (England) supported this work, although the views are our own."	Ischemic heart disease	Risk	No
Danesh et al, 1998 (59)Country: United KingdomJournal: JAMASponsorship: Scholarship, nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	Yes
French and Brocklehurst, 1998 (60)Country: United KingdomJournal: Br J Obstet GynaecolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Survival in women infected with human immunodeficiency virus	Risk	Yes
Forgie et al, 1998 (61)Country: CanadaJournal: Arch Intern MedSponsorship: Industry, government, fellowships, nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Allogeneic blood transfusion	Risk	No
Huang et al, 1998 (62)Country: CanadaJournal: GastroenterologySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Gastric cancer	Risk	Yes
Johnston et al, 1998 (63)Country: United StatesJournal: NeurologySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Subarachnoid hemorrhage	Risk	No
Lazarou et al, 1998 (64)Country: CanadaJournal: JAMASponsorship: Scholarship, nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Adverse drug reactions in hospitalized patients	Prevalence	Quality criteria abstracted
Ray, 1998 (65)Country: CanadaJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Venous thromboembolic disease	Risk	Quality criteria abstracted
Spencer-Green, 1998 (66)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Secondary diseases from primary Reynaud phenomenon	Risk	Yes
Stratton et al, 1998 (67)Country: United KingdomJournal: Br J Obstet GynaecolSponsorship: Research fellowship, nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Ovarian cancer	Risk	No
Zock and Katan, 1998 (68)Country: NetherlandsJournal: Am J Clin NutrSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Breast, colorectal, and prostate cancer	Risk	Quality criteria abstracted
Zondervan et al, 1998 (69)Country: United KingdomJournal: Br J Obstet GynaecolSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Chronic pelvic pain in women	Prevalence	No
Angelillo and Villari, 1999 (70)Country: ItalyJournal: Bull World Health OrganSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Childhood leukemia	Risk	Yes
He et al, 1999 (71)Country: United StatesJournal: N Engl J MedSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	No
Shaffer et al, 1999 (72)Country: United StatesJournal: Am J Public HealthSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Disordered gambling behavior	Prevalence	No
Wittrup et al, 1999 (73)Country: DenmarkJournal: CirculationSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Ischemic heart disease	Risk	Yes
Yoder et al, 1999 (74)Country: United StatesJournal: Obstet GynecolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Fetus with isolated choroid plexus cysts	Risk	No
Christen et al, 2000 (75)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Cardiovascular disease	Risk	Yes
Cleophas et al, 2000 (76)Country: NetherlandsJournal: Am J CardiolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary artery disease	Risk	Yes
DiMatteo et al, 2000 (77)Country: United StatesJournal: Arch Intern MedSponsorship: Industry, scholarshipCOI: Not reportedSponsor participation in data analyses: Not reported	Noncompliance with medical treatment	Risk	Quality criteria abstracted
WHO Collaborative Study Team on the Role of Breastfeeding on the Prevention of Infant Mortality, 2000 (78)Country: BrazilJournal: LancetSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Infant and child mortality	Risk	No
Wilson et al, 2000 (79)Country: CanadaJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Mortality after myocardial infarction	Risk	Yes
Zeegers et al, 2000 (80)Country: NetherlandsJournal: CancerSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Urinary tract cancer	Risk	Yes
Danesh et al, 2001 (81)Country: United KingdomJournal: CirculationSponsorship: Government, scholarshipCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	No
Eaden et al, 2001 (82)Country: United KingdomJournal: GutSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Colorectal cancer	Risk	Yes
Faraone et al, 2001 (83)Country: United StatesJournal: Am J PsychiatrySponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Attention deficit hyperactivity disorder	Risk	Yes
Horta et al, 2001 (84)Country: BrazilJournal: Am J Public HealthSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Early weaning	Risk	Yes
Rebora, 2001 (85)Country: ItalyJournal: Arch DermatolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary artery disease	Risk	Yes
Cannon et al, 2002 (86)Country: United KingdomJournal: Am J PsychiatrySponsorship: Research fellowship, nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Schizophrenia	Risk	No
Hellermann et al, 2002 (87)Country: United StatesJournal: Am J MedSponsorship: Government, nonprofit organization, fellowshipCOI: Not reportedSponsor participation in data analyses: Not reported	Heart failure	Risk	No
Huang et al, 2002 (88)Country: CanadaJournal: LancetSponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Peptic-ulcer disease	Risk	Yes
Huncharek et al, 2002 (89)Country: United StatesJournal: Am J Public HealthSponsorship: Nonprofit organization, industryCOI: Not reportedSponsor participation in data analyses: Not reported	Malignant melanoma	Risk	Yes
Juul et al, 2002 (90)Country: DenmarkJournal: BloodSponsorship: Government, nonprofit organizationCOI: Not reportedSponsor participation in data analyses: "They had no role in gathering, analyzing, or interpreting the data and had no right to approve or disapprove the submitted paper."	Factor V Leiden	Risk	Yes
Kelly et al, 2002 (91)Country: United StatesJournal: NeurologySponsorship: Nonprofit organization, industry, fellowshipCOI: Not reportedSponsor participation in data analyses: Not reported	Risk of ischemic stroke	Risk	No
Klerk et al, 2002 (92)Country: NetherlandsJournal: JAMASponsorship: Government, "public/private partnership"COI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	Yes
Kozer et al, 2002 (93)Country: CanadaJournal: Am J Obstet GynecolSponsorship: IndustryCOI: Not reportedSponsor participation in data analyses: Not reported	Congenital anomalies	Risk	No
Law et al, 2002 (94)Country: United KingdomJournal: Arch Intern MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Death after myocardial infarction	Risk	No
Wald et al, 2002 (95)Country: United KingdomJournal: BMJSponsorship: NoneCOI: Reported as not a conflict of interestSponsor participation in data analyses: None	Cardiovascular disease	Risk	No
Wald and Link, 2002 (96)Country: United StatesJournal: J Infect DisSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Human immunodeficiency virus infection	Risk	No
Benjamin et al, 2003 (97)Country: United StatesJournal: PediatricsSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	End-organ damage	Prevalence	No
Clarfield, 2003 (98)Country: IsraelJournal: Arch Intern MedSponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Reversible dementias	Prevalence	No
Cole and Dendukuri, 2003 (99)Country: CanadaJournal: Am J PsychiatrySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Depression among elderly community subjects	Risk	Yes
Gisbert et al, 2003 (100)Country: SpainJournal: GastroenterologySponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Hepatitis C virus infection	Risk	Yes
Glatt et al, 2003 (101)Country: United StatesJournal: Am J PsychiatrySponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Schizophrenia	Risk	No
Halbert et al, 2003 (102)Country: United StatesJournal: ChestSponsorship: IndustryCOI: Not reportedSponsor participation in data analyses: Not reported	Prevalence estimates for chronic obstructive pulmonary disease	Prevalence	No
Huang et al, 2003 (103)Country: CanadaJournal: GastroenterologySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Gastric cancer	Risk	Yes
Rey et al, 2003 (104)Country: CanadaJournal: LancetSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Fetal loss	Risk	Yes
Riboli and Norat, 2003 (105)Country: FranceJournal: Am J Clin NutrSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Cancer risk	Risk	No
Scholten-Peeters et al, 2003 (106)Country: NetherlandsJournal: PainSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Whiplash-associated disorders	Risk	Yes
Thurnham et al, 2003 (107)Country: United KingdomJournal: LancetSponsorship: Government, fellowshipCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The funding source had no role in study design, data collection, data analysis, data interpretation, or in the writing of this report."	Vitamin A deficiency	Continuous variable	No
Zeegers et al, 2003 (108)Country: NetherlandsJournal: CancerSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Prostate carcinoma	Risk	No
Burzotta et al, 2004 (109)Country: ItalyJournal: HeartSponsorship: FellowshipCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary ischemic syndromes	Risk	No
Casas et al, 2004 (110)Country: United KingdomJournal: CirculationSponsorship: Government, 1 author holds a chair of nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Ischemic heart disease	Risk	No
Casas et al, 2004 (111)Country: United KingdomJournal: Arch NeurolSponsorship: FellowshipCOI: Not reportedSponsor participation in data analyses: Not reported	Ischemic stroke	Risk	No
He et al, 2004 (112)Country: United StatesJournal: CirculationSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease mortality	Risk	No
Huang et al, 2004 (113)Country: United StatesJournal: NeurologySponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Sporadic Parkinson disease	Risk	No
Klement et al, 2004 (114)Country: IsraelJournal: Am J Clin NutrSponsorship: Medical centerCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Inflammatory bowel disease	Risk	Yes
Kovalevsky et al, 2004 (115)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Recurrent pregnancy loss	Risk	No
Levitan et al, 2004 (116)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Cardiovascular disease	Risk	No
Lovett et al, 2004 (117)Country: United KingdomJournal: NeurologySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Subtype of ischemic stroke	Risk	Yes
Mitsikostas et al, 2004 (118)Country: GreeceJournal: BrainSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Headache	Risk	No
Montanez et al, 2004 (119)Country: United StatesJournal: Arch Intern MedSponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Total and cardiovascular mortality and sudden death	Risk	No
Woodbury and Houghton, 2004 (120)Country: CanadaJournal: Ostomy Wound ManageSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Pressure ulcers	Prevalence	Yes
Bolland et al, 2005 (121)Country: New ZealandJournal: J Clin Endocrinol MetabSponsorship: ScholarshipCOI: Not reportedSponsor participation in data analyses: Not reported	Increased body weight	Risk	Quality criteria abstracted
Contopoulos-Ioannidis et al, 2005 (122)Country: GreeceJournal: J Allergy Clin ImmunolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Asthma phenotypes	Risk	Quality criteria abstracted
Dauchet et al, 2005 (123)Country: FranceJournal: NeurologySponsorship: Nonprofit organization, educational instituteCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Stroke	Risk	No
Etminan et al, 2005 (124)Country: CanadaJournal: BMJSponsorship: Government, fellowshipCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Ischemic stroke	Risk	Yes
Fazel et al, 2005 (125)Country: United KingdomJournal: LancetSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The sponsors of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication."	Serious mental disorder	Prevalence	Quality criteria abstracted
García-Closas et al, 2005 (126)Country: United StatesJournal: LancetSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The study sponsors had no role in the design of the study; in the collection, analysis, or interpretation of the data; or in the writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit the paper for publication."	Bladder cancer	Risk	No
Lee et al, 2005 (127)Country: United StatesJournal: Arthritis RheumSponsorship: Government, industryCOI: Not reportedSponsor participation in data analyses: Unrestricted	Systemic lupus erythematosus	Risk	No
Lin and August, 2005 (128)Country: United StatesJournal: Obstet GynecolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Preeclampsia	Risk	No
McDonald et al, 2005 (129)Country: CanadaJournal: Am J Obstet GynecolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Perinatal outcomes	Risk	Yes
Palmer, 2005 (130)Country: United StatesJournal: Arch Gen PsychiatrySponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Lifetime risk of suicide in schizophrenia	Prevalence	Quality criteria abstracted
Sin et al, 2005 (131)Country: CanadaJournal: ChestSponsorship: Nonprofit organization, educational instituteCOI: Not reportedSponsor participation in data analyses: Not reported	Cardiovascular mortality	Risk	Yes
Boudville et al, 2006 (132)Country: CanadaJournal: Ann Intern MedSponsorship: Government, fellowshipCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The study sponsors had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication."	Hypertension	Risk	No
Clark et al, 2006 (133)Country: United KingdomJournal: PediatricsSponsorship: FellowshipCOI: Not reportedSponsor participation in data analyses: Not reported	Fractures	Risk	Yes
de Boer et al, 2006 (134)Country: NetherlandsJournal: CancerSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Unemployment	Risk	Yes
Di Castelnuovo et al, 2006 (135)Country: ItalyJournal: Arch Intern MedSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: "The sponsor of the study had no involvement in study design; data collection, analysis, or interpretation; writing of the report; or in the decision to submit the paper for publication."	Total mortality in men and women	Risk	Yes
Flores-Mateo et al, 2006 (136)Country: United StatesJournal: Am J Clin NutrSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	Yes
Galassi et al, 2006 (137)Country: United StatesJournal: Am J MedSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Cardiovascular disease	Risk	Quality criteria abstracted
Huxley et al, 2006 (138)Country: AustraliaJournal: BMJSponsorship: Government, fellowship, industryCOI: Reported as not a conflict of interestSponsor participation in data analyses: Unconditional	Fatal coronary heart disease	Risk	Quality criteria abstracted
Kahlenborn et al, 2006 (139)Country: United StatesJournal: Mayo Clin ProcSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Premenopausal breast cancer	Risk	Quality criteria abstracted
Larsson et al, 2006 (140)Country: SwedenJournal: GastroenterologySponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Esophageal, gastric, and pancreatic cancer	Risk	Quality criteria abstracted
Mahid et al, 2006 (141)Country: United StatesJournal: Mayo Clin ProcSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: Not reported	Inflammatory bowel disease	Risk	Yes
Owen et al, 2006 (142)Country: United KingdomJournal: Am J Clin NutrSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Type 2 diabetes	Risk	Quality criteria abstracted
Ownby et al, 2006 (143)Country: United StatesJournal: Arch Gen PsychiatrySponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Alzheimer disease	Risk	Yes
Pavia et al, 2006 (144)Country: ItalyJournal: Am J Clin NutrSponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Oral cancer	Risk	Yes
Riddle et al, 2006 (145)Country: United StatesJournal: Am J Trop Med HygSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Diarrhea	Prevalence	Yes
Rutledge et al, 2006 (146)Country: United StatesJournal: J Am Coll CardiolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Depression	Prevalence/ risk	Quality criteria abstracted
Smith et al, 2006 (147)Country: United StatesJournal: J Am Coll CardiolSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Renal impairment	Risk	Yes
Weis et al, 2006 (148)Country: United StatesJournal: Arch OphthalmolSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Uveal melanoma	Risk	Quality criteria abstracted
Williams et al, 2006 (149)Country: United KingdomJournal: Arch Dis ChildSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Autism spectrum disorders	Prevalence/ risk	Quality criteria abstracted
Bahekar et al, 2007 (150)Country: United StatesJournal: Am Heart JSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Coronary heart disease	Risk	Yes
Baurecht et al, 2007 (151)Country: GermanyJournal: J Allergy Clin ImmunolSponsorship: Government, universityCOI: Reported as a conflict of interestSponsor participation in data analyses: Not reported	Atopic eczema	Risk	No
Bellamy et al, 2007 (152)Country: United KingdomJournal: BMJSponsorship: Government, fellowshipCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Cardiovascular disease	Risk	Quality criteria abstracted
Conde-Agudelo et al, 2007 (153)Country: ColombiaJournal: Am J Obstet GynecolSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: "The content of the paper has not been influenced by the sponsor."	Maternal health	Risk	Yes
Dehghan et al, 2007 (154)Country: NetherlandsJournal: DiabetesSponsorship: University, governmentCOI: Not reportedSponsor participation in data analyses: Not reported	Diabetes	Risk	No
Eichler et al, 2007 (155)Country: SwitzerlandJournal: Am Heart JSponsorship: Nonprofit organizationCOI: Not reportedSponsor participation in data analyses: "The funding source had no influence on study design; in the collection, analysis, and interpretation of the data; in the writing of the manuscript; and in the decision to submit the manuscript for publication."	First coronary events	Risk	Yes
Gami et al, 2007 (156)Country: United StatesJournal: J Am Coll CardiolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Cardiovascular events and death	Risk	Yes
Grulich et al, 2007 (157)Country: AustraliaJournal: LancetSponsorship: Government, fellowship, scholarshipCOI: Reported as a conflict of interestSponsor participation in data analyses: "There was no funding source for this study. All authors had access to all the data. The corresponding author had final responsibility for the decision to submit for publication."	Cancers	Risk	Yes
Havemann et al, 2007 (158)Country: United StatesJournal: GutSponsorship: IndustryCOI: Reported as a conflict of interestSponsor participation in data analyses: Not reported	Asthma	Risk	Quality criteria abstracted
Hirtz et al, 2007 (159)Country: United StatesJournal: NeurologySponsorship: Not reportedCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Common neurologic disorders	Prevalence	Yes
Huxley et al, 2007 (160)Country: AustraliaJournal: Am J Clin NutrSponsorship: Government, nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: "None of the funding sources had any role in the study design, data analysis, data interpretation, writing of the paper, or the decision to submit the paper for publication."	Ischemic heart disease	Risk	Yes
Krishna and Kim, 2007 (161)Country: United StatesJournal: J NeurosurgSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: Not reported	Risk factors for subarachnoid hemorrhage	Risk	Quality criteria abstracted
Langan et al, 2007 (162)Country: United KingdomJournal: Arch DermatolSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The sponsor had no role in the design and conduct of the study; in the collection, analysis, and interpretation of data; or in the preparation, review, or approval of the manuscript."	Eczema	Risk	Quality criteria abstracted
Larsson and Wolk, 2007 (163)Country: SwedenJournal: Am J Clin NutrSponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Colon and rectal cancer risk	Risk	Quality criteria abstracted
Larsson and Wolk, 2007 (164)Country: SwedenJournal: GastroenterologySponsorship: Nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: "The sponsor had no role in the study design or in the collection, analysis, and interpretation of the data."	Liver cancer	Risk	Quality criteria abstracted
Liu et al, 2007 (165)Country: ChinaJournal: J Am Coll CardiolSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Recurrence of atrial fibrillation after successful electrical cardioversion	Risk	Yes
Loza and Chang, 2007 (166)Country: United StatesJournal: J Allergy Clin ImmunolSponsorship: Government, nonprofit organizationCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Atopic asthma risk	Risk	Yes
Pittas et al, 2007 (167)Country: United StatesJournal: J Clin Endocrinol MetabSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Type 2 diabetes	Risk	No
Polanczyk et al, 2007 (168)Country: BrazilJournal: Am J PsychiatrySponsorship: Industry, foreign grantsCOI: Not reportedSponsor participation in data analyses: "There was no involvement of any funding source in the study design, data collection, analysis, interpretation of data, and writing of this article or in the decision to submit the article for publication."	Attention deficit hyperactivity disorder	Prevalence	No
Rona et al, 2007 (169)Country: United KingdomJournal: J Allergy Clin ImmunolSponsorship: GovernmentCOI: Reported as a conflict of interestSponsor participation in data analyses: Not reported	Food allergy	Prevalence	Quality criteria abstracted
Sarwar et al, 2007 (170)Country: United KingdomJournal: CirculationSponsorship: Government, scholarship, industryCOI: Reported as not a conflict of interestSponsor participation in data analyses: Unrestricted	Coronary heart disease	Risk	No
Snoep et al, 2007 (171)Country: NetherlandsJournal: Am Heart JSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Clopidogrel nonresponsiveness	Prevalence	Yes
Zintzaras and Kaditis, 2007 (172)Country: GreeceJournal: Arch Pediatr Adolesc MedSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Blood pressure	Risk	Yes
Ageno et al, 2008 (173)Country: ItalyJournal: CirculationSponsorship: Not reportedCOI: Not reportedSponsor participation in data analyses: Not reported	Venous thromboembolism	Risk	Yes
Barclay et al, 2008 (174)Country: AustraliaJournal: Am J Clin NutrSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Chronic disease risk	Risk	Quality criteria abstracted
Conde-Agudelo et al, 2008 (175)Country: United StatesJournal: Am J Obstet GynecolSponsorship: GovernmentCOI: Not reportedSponsor participation in data analyses: "The views expressed in this document are solely the responsibility of the authors and do not necessarily represent the views of the World Health Organization."	Risk of preeclampsia	Risk	Yes
Schunkert et al, 2008 (176)Country: GermanyJournal: CirculationSponsorship: GovernmentCOI: Reported as not a conflict of interestSponsor participation in data analyses: Not reported	Coronary artery disease	Risk	No

Table 1

Quality Criteria of Systematic Reviews of Observational Nontherapeutic Studies Published in Core Clinical Journals, by Year of Publication, 1990 Through June 2008

Evaluated Criteria	1990-1994, n (N = 17)	1995-1999, n (N = 26)	2000-2004, n (N = 46)	2005-2008, n (N = 56)	Total, n (N = 145)	P Valuea
Literature search
No information	0	0	1	0	1	.7
Documented partially	1	1	3	1	6
Complete documenting of databases used, exact search strings used, and time periods of searches	16	25	42	55	138
Contact with authors of the included studies
No information	13	17	31	31	92	.4
The authors of the review attempted to contact the authors of included studies	4	9	15	25	53	.4
Study flow
Study flow not reported	10	15	29	18	72	.04
Study flow partially reported	0	0	0	2	2
Study flow reported with the list of retrieved citations, the list of excluded studies, and justification for exclusion for each study	7	11	17	36	71
Articles published in languages other than English
Language bias was not addressed	15	24	31	33	103	.01
Language bias was addressed: the authors included or justified exclusion of the non-English publications	2	2	15	23	42	.01
Gray literature
Gray literature was not assessed	15	17	36	38	106	.25
Reporting of the method of handling abstracts and unpublished studies	2	9	10	18	39	.25
Conflict of interest from included studies
Conflict of interest in included studies was not abstracted	17	26	46	56	145	NA
Sponsorship of the included studies
Sponsorship of included studies was not analyzed	16	25	46	55	142	.45
Sponsorship of included studies was analyzed	1	1	0	1	3	.45
Pooled model obtained in the review
Pooling was not obtained	2	0	4	2	8	<.001
Fixed effects model was obtained for meta-analyses	10	16	11	10	47
Random effects model was obtained for meta-analyses	5	10	31	44	90
Heterogeneity across included studies
Heterogeneity across studies was not reported	6	9	7	6	28	.04
Heterogeneity across studies was not significant	5	6	13	11	35
Heterogeneity across studies was significant	6	11	26	39	82
Formal internal quality evaluation of included studies
Planned, formal internal quality evaluation with developed or previously published checklists or scales	3	6	20	25	54	<.001
Some selected criteria of external or internal quality of included studies were abstracted without planned, formal internal quality evaluation	2	3	1	20	26
No internal quality evaluation	12	17	25	11	65
Reliability of internal quality evaluation reported	2	4	8	18	32	.99
Internal quality evaluation was masked	1	1	0	1	3	.11

Abbreviation: NA, not applicable.

P values for overall χ² test.

Table 2

Quality of Systematic Reviews, by Internal Quality Evaluation of Included Studies, 1990 Through June 2008

Quality Criterion	Definition of Formal Internal Quality Evaluation
Quality Criterion	Planned, Formal Internal Quality Evaluation or Abstraction of Some Quality Criteria, n	Neither Planned, Formal Internal Quality Evaluation nor Abstraction of Some Quality Criteria, n	Planned, Formal Internal Quality Evaluation, n	No Planned, Formal Internal Quality Evaluation, n
Literature search	P = .04a		P = .004b
No information	0	1	0	1
Documented partially	6	0	6	0
Complete documenting of databases used, exact search strings used, and time periods of searches	74	64	48	90
Contact with authors of the included studies	P = .02a		P = .25b
No information	44	48	31	61
The authors of the review attempted to contact the authors of included studies	36	17	23	30
Study flow	P < .001a		P = .003b
Study flow not reported	28	44	17	55
Study flow partially reported	1	1	1	1
Study flow reported with the list of retrieved citations, the list of excluded studies, and justification for exclusion for each study	51	20	36	35
Articles published in languages other than English	P = .001a		P = .01b
No information	48	55	31	72
Inclusion of non-English studies or justification for exclusion	32	10	23	19
Gray literature	P = .09a		P = .04b
No information	54	52	34	72
Reporting of the method of handling abstracts and unpublished studies	26	13	20	19
Conflict of interest from included studies
No information	80	65	54	91
Sponsorship of the included studies	P = .44a		P = .18b
No information	79	63	54	88
Sponsorship of included studies was abstracted	1	2	0	3
Pooled model obtained in the review	P < .001a		P = .06b
Not applicable (no pooling)	6	2	5	3
Fixed effects model	15	32	12	35
Random effects model	59	31	37	53
Heterogeneity across included studies	P = .27a		P = .67b
Not reported	13	15	9	19
Heterogeneity was not significant	17	18	15	20
Heterogeneity was significant at least for one association	50	32	30	52

P value for overall χ² test between planned, formal internal quality evaluation or abstraction of some quality criteria versus neither planned, formal internal quality evaluation nor abstraction of some quality criteria.

P value for overall χ² test between planned, formal internal quality evaluation versus no planned, formal internal quality evaluation.