A Key Characteristic of a Systematic Review Is That It Contains a Meta-analysis True False
Systematic Review and Meta-assay: When I Study Is Just non Plenty
We live in the data age, and the do of medicine is becoming increasingly specialized. In the biomedical literature, the number of published studies has dramatically increased: There are now more than xv million citations in MEDLINE, with ten,000 to 20,000 new citations added each week (ane). Multiple relevant studies ordinarily guide almost clinical decisions. These studies oftentimes vary in their design; methodologic quality; population studied; and the intervention, test, or condition considered. Because fifty-fifty highly cited trials may exist challenged or refuted over fourth dimension (2), clinical conclusion-making requires ongoing reconciliation of studies that provide unlike answers to the same question. Both clinicians and researchers can as well benefit from a summary of where uncertainty remains. Because it is often impractical for readers to track downwardly and review all of the primary studies (3), review articles are an important source of summarized show on a detail topic (four).
Narrative Review, Systematic Review, and Meta-analysis
Review manufactures take traditionally taken the form of a narrative review, whereby a content skillful writes nearly a detail field, status, or treatment (5–seven). Narrative reviews have many benefits, including a broad overview of relevant information tempered past years of applied knowledge from an experienced author. Indeed, this article itself is in a narrative format, from authors who have published a number of meta-analyses in previous years.
In some circumstances, a reader wants to become very knowledgeable virtually specific details of a topic and wants some assurance that the data presented is both comprehensive and unbiased. A narrative review typically uses an implicit process to compile evidence to support the statements beingness fabricated. The reader oft cannot tell which recommendations were based on the author'due south clinical experience, the breadth to which available literature was identified and compiled, and the reasons that some studies were given more than emphasis than others. It is sometimes uncertain whether the author of a narrative review selectively cited reports that reinforced his or her preconceived ideas or promoted specific views of a topic. Too, a quantitative summary of the literature is often absent-minded in a narrative review.
A systematic review uses a process to identify comprehensively all studies for a specific focused question (fatigued from research and other sources), appraise the methods of the studies, summarize the results, present key findings, identify reasons for different results across studies, and cite limitations of current knowledge (viii,9). In a systematic review, all decisions used to compile data are meant to exist explicit, assuasive the reader to gauge for him- or herself the quality of the review process and the potential for bias. In this mode, systematic reviews tend to be more transparent than their narrative cousins, although they too can be biased if the selection or emphasis of sure primary studies is influenced by the preconceived notions of the authors or funding sources (10).
Depending on the nature of the data, the results of a systematic review can exist summarized in text or graphic form. In graphic form, it is common for different trials to be depicted in a plot where the point gauge and 95% confidence interval for each study are presented on an private line (11). When results are mathematically combined (a procedure sometimes referred to equally pooling), this is referred to every bit meta-analysis. Graphically, the pooled result is often presented as a diamond at the bottom of the plot.
When performing a meta-analysis, a review team normally combines aggregate-level information reported in each chief study (point and variance approximate of the summary measure). On occasion, a review team will obtain all of the individual patient data from each of the primary studies (12,xiii). Although challenging to bear (14), individual patient meta-analyses may have sure advantages over aggregate-level analyses. As highlighted in a review of angiotensin-converting enzyme (ACE) inhibitors for nondiabetic kidney disease, this includes the utilise of common definitions, coding and cutoff points between studies, addressing questions not examined in the original publication, and a meliorate sense of the impact of individual patient (versus study level) characteristics (12,15).
Equally first highlighted a decade ago (sixteen), the number of systematic reviews in nephrology and other fields has increased dramatically with time, paralleling the rapid growth of biomedical literature during the by half century. Initiatives such as the Cochrane Collaboration have further increased the profile and rigor of the systematic review process (details of the structured procedure of Cochrane systematic reviews are bachelor through their Web site) (17,18). From 1990 to 2005, at that place were more 400 systematic reviews and meta-analyses published in the discipline of nephrology (Figure 1). Of these reviews, forty% pertained to chronic kidney illness or glomerulonephritis and 20, sixteen, 15, and 7% pertained to kidney transplantation, dialysis, acute kidney injury, and pediatric nephrology, respectively. Equally a publication type, however, systematic reviews take not been without controversy: Some authors consider a meta-analysis the best possible use of all available data, whereas others question whether they add together anything meaningful to scientific knowledge (19). The strengths and weaknesses of this publication type are described next.
- Download figure
- Open in new tab
- Download powerpoint
Figure i.
There have been more than 400 systematic reviews and meta-analyses published in the subject of nephrology since 1990, with the annual number increasing with time. Frequencies were estimated from a MEDLINE and EMBASE search performed past an experienced renal librarian in December 2006. Citations were reviewed by a nephrologist for relevance. Indistinguishable publications from the aforementioned grouping of authors were counted only once.
Strengths of Systematic Review and Meta-analysis
Physicians brand better clinical decisions when they understand the circumstances and preferences of their patients and combine their personal experience with clinical evidence underlying the available options (20). The public likewise expects that their physicians will integrate research findings into practice in a timely mode (21). Thus, sound clinical or wellness policy decisions are facilitated by reviewing the available evidence (and its limitations), understanding reasons why some studies differ in their results (a finding sometimes referred to as heterogeneity among the main studies), coming upwards with an assessment of the expected effect of an intervention or exposure (for questions of therapy or etiology), and then integrating the new data with other relevant treatment, patient, and health care system factors.
In this respect, reading a properly conducted systematic review is an efficient way to become familiar with the best available research evidence for a focused clinical question. The review team may also take obtained information from the primary authors which was not available in the original reports. The presented summary allows the reader to accept into account a whole range of relevant findings from inquiry on a item topic. The procedure can likewise establish whether the scientific findings are consistent and generalizable across populations, settings, and treatment variations and whether findings vary significantly by item subgroups. Once again, the potential force of a systematic review lies in the transparency of each phase of the synthesis process, allowing the reader to focus on the claim of each decision made in compiling the information, rather than a simple contrast of one report to another as sometimes occurs in other types of reviews.
For example, studies demonstrating a significant effect of treatment are more probable to be published than studies with negative findings, are more likely to be published in English, and more likely to be cited past others (22–27). A well-conducted systematic review attempts to reduce the possibility of bias in the method of identifying and selecting studies for review, past using a comprehensive search strategy and specifying inclusion criteria that ideally accept not been influenced by a priori knowledge of the principal studies.
Mathematically combining data from a series of well-conducted primary studies may provide a more than precise estimate of the underlying "true effect" than any individual study (28). In other words, by combining the samples of the individual studies, the size of the "overall sample" is increased, enhancing the statistical ability of the analysis and reducing the size of the confidence interval for the point estimate of the effect. It is also more than efficient to communicate a pooled summary than to describe the results for each of the individual studies. Sometimes, if the treatment effect in small trials shows a nonsignificant trend toward efficacy, then pooling the results may establish the benefits of therapy (16). For example, 10 trials examined whether ACE inhibitors were more than effective than other antihypertensive agents for the prevention of nondiabetic kidney failure (29). Many of the 95% confidence intervals for the estimate provided by each study overlapped with a finding of no effect; even so, the overall pooled estimate established a benefit of ACE inhibitors.
For these reasons, a meta-analysis of similar, well-conducted, randomized, controlled trials has been considered one of the highest levels of evidence (30–32). It is important to stress that the primary trials all accept to be conducted with high methodologic rigor for the meta-assay to be definitive. Alternatively, when the existing studies accept important scientific and methodologic limitations, including smaller sized samples (which is more than oft the instance), the systematic review may identify where gaps be in the available literature. In this case, an exploratory meta-analysis can provide a plausible gauge of effect that can be tested in subsequent studies (33,34).
Limitations of Systematic Review and Meta-analysis
This type of publication type has many potential limitations that should be appreciated by all readers. Kickoff, the summary provided in a systematic review and meta-analysis of the literature is only equally reliable as the methods used to estimate the effect in each of the primary studies. In other words, conducting a meta-analysis does not overcome problems that were inherent in the design and execution of the main studies. Information technology likewise does not correct biases as a result of selective publication, whereby studies that report dramatic furnishings are more likely to exist identified, summarized, and subsequently pooled in meta-analysis than studies that report smaller effect sizes (an issue referred to as publication bias). Because more than three quarters of meta-analyses did not report any empirical cess of publication bias (35), the truthful frequency of this form of bias is unknown.
Controversies likewise arise around the estimation of summarized results, particularly when the results of discordant studies are pooled in meta-analysis (36). The review procedure inevitably identifies studies that are diverse in their design, methodologic quality, specific interventions used, and types of patients studied. In that location is often some subjectivity when deciding how similar studies must exist before pooling is appropriate. Combining studies of poor quality with those that were more rigorously conducted may non be useful and tin can lead to worse estimates of the underlying truth or a faux sense of precision effectually the truth (36). A false sense of precision may also arise when diverse subgroups of patients divers by characteristics such as their age or gender differ in their observed response. In such cases, reporting an aggregate pooled effect might be misleading if at that place are important reasons to explain variable handling effects beyond dissimilar types of patients (36–twoscore).
Finally, simply labeling a manuscript equally a "systematic review" or "meta-assay" does non guarantee that the review was conducted or reported with due rigor (41). To reduce the chance of arriving at misleading conclusions, guidelines on the bear and reporting of systematic reviews were recently published (42,43); notwithstanding, important methodologic flaws of systematic reviews published in peer-reviewed journals have been well described (44–54). For instance, of the 86 renal systematic reviews published in 2005, the majority (58%) had important methodologic flaws (Mrkobrada Chiliad, Thiessen-Philbrook H, Haynes RB, Iansavichus AV, Rehman F, and Garg AX, submitted). The most common flaws among these renal reviews were failure to assess the methodologic quality of included principal studies and failure to avoid bias in written report inclusion (Mrkobrada M, Thiessen-Philbrook H, Haynes RB, Iansavichus AV, Rehman F, and Garg AX, submitted). In some cases, industry-supported reviews of drugs accept had fewer reservations about methodologic limitations of the included trials than rigorously conducted Cochrane reviews on the same topic (10); however, the hypothesis that less rigorous reviews more often written report positive conclusions than good-quality reviews of the same topic has not been borne out in empirical cess (48,53,55). Nonetheless, like all proficient consumers, users of systematic reviews should carefully consider the quality of the production and adhere to the dictum "caveat emptor": Permit the buyer beware. The limitations described in this section may explain differences in the results of meta-analyses as compared with subsequent big, randomized, controlled trials, which accept occurred in approximately one 3rd of cases (56).
How to Appraise Critically a Systematic Review and Meta-analysis
Users of systematic reviews need to assure themselves that the underlying methods used to assemble relevant data were sound. Earlier because the results or how the information could be appropriately applied in patient care (9), there are a few questions that the reader tin ask him- or herself when assessing the methodologic quality of a systematic review (Table 1).
Table 1.
Questions to ask when assessing the quality of a systematic reviewa
Was the Review Conducted Co-ordinate to a Prespecified Protocol?
It is reassuring if a review was guided past a written protocol (prepared in advance) that describes the research question(southward), hypotheses, review method, and plan for how the data volition be extracted and compiled. Such an arroyo minimizes the likelihood that the results or the expectations of the reviewing team influenced study inclusion or synthesis. Although most systematic reviews are conducted in a retrospective manner, reviews and meta-analyses can in theory be defined at the time several similar trials are being planned or under way. This allows a set of specific hypotheses, information collection procedures, and analytic strategies to be specified in accelerate before whatever of the results from the chief studies are known. Such a prospective try may provide more reliable answers to medically relevant questions than the traditional retrospective approach (41).
Was the Question Focused?
Clinical questions ofttimes deal with issues of handling, etiology, prognosis, and diagnosis. A well-formulated question usually specifies the patient's trouble or diagnosis, the intervention or exposure of interest, whatever comparison group (if relevant), and the chief and secondary outcomes of interest (57).
Were the "Right" Types of Studies Eligible for the Review?
Dissimilar study designs can exist used to answer dissimilar clinical questions. Randomized, controlled trials; observational studies; and cross-sectional diagnostic studies may each be appropriate depending on the master question posed in the review. When examining the eligible criteria for study inclusion, the reader should feel confident that a potential bias in the selection of studies was avoided. Specifically, the reader should ask her- or himself whether the eligibility criteria for written report inclusion were appropriate for the question asked. Whether the correct types of studies were selected for the review also depends on the depth and breadth of the underlying literature search.
For example, some review teams will consider simply studies that were published in English. There is evidence that journals from certain countries publish a higher proportion of positive trials than others (58). Excluding non-English language studies seemed to change the results of some reviews (59,60) only non others (61,62).
Some review teams use wide criteria for their inclusion of primary studies (east.g., furnishings of agents that cake the renin-angiotensin organisation on renal outcomes [63]), whereas other teams utilise more than narrow inclusion criteria (due east.g., restricting the analysis only to patients who have diabetes without evidence of nephropathy [64]). There is often no single correct arroyo; however, the conclusions of any meta-analysis that is highly sensitive to altering the entry criteria of included studies should be interpreted with some caution (25). For example, two different review teams considered whether constructed dialysis membranes resulted in meliorate clinical outcomes compared with cellulose-based membranes in patients with acute renal failure. In one meta-analysis (65) merely not the other (66), synthetic membranes reduced the adventure for death. The discordant results were due to the inclusion of a study that did not see eligibility for the 2nd review (67).
Was the Method of Identifying All Relevant Information Comprehensive?
Identifying relevant studies for a given clinical question among the many potential sources of information is commonly a laborious process (68). Biomedical journals are the most mutual source of information, and bibliographic databases are frequently used to search for relevant articles. MEDLINE currently indexes approximately 4800 medical journals and contains xiii million citations (69). Similarly, EMBASE indexes approximately 5000 medical journals and contains more than than 11 million records. At that place are some key differences between EMBASE and MEDLINE, and the review team should have searched both databases (70–72). For example, EMBASE provides the best coverage of European research besides as pharmaceutical research including renal adverse events (73). Positive studies may be more often published in journals that are indexed in MEDLINE, compared with nonindexed journals (25).
Depending on the question posed, other databases may likewise take been searched. For example, if a team is summarizing the effects of practise training in patients who receive maintenance hemodialysis, then searching the Cumulative Index to Nursing and Allied Health Literature (CINAHL) database would be appropriate (74). Alternatively, the ECONOLIT database may be useful for identifying information on the out-of-pocket expenses incurred by living kidney donors (75). As a supplementary method of identifying information, searching databases such as the Science Commendation Index (which identifies all articles that cite a relevant article), besides every bit newer Internet search engines such as Google Scholar and Elsevier'southward Scirus, tin be useful for identifying manufactures that are not indexed well in traditional bibliographic databases (76). Searching bibliographies of retrieved manufactures can also identify relevant articles that were missed.
Whatever bibliographic database was used, the review squad should have used a search strategy that maximized the identification of relevant articles (77,78). Considering in that location is some subjectivity in screening databases, citations should exist reviewed independently and in duplicate by two members of the reviewing squad, with the total-text article retrieved for any citation deemed relevant past any of the reviewers. At that place is also some subjectivity in assessing the eligibility of each full-text commodity, and the hazard for incorrectly discarding relevant reports is reduced when two reviewers independently perform each cess in a reliable fashion (79).
Of import sources of information other than journal manufactures should not be disregarded. Briefing proceedings, abstracts, books, and manufacturers all tin can be sources of potentially valuable information. Inquiries to experts, including those listed in trial registries, may have also proved useful (28).
A comprehensive search of bachelor literature reduces the possibility of publication bias, which occurs when studies with statistically significant results are more probable to exist published and cited (fourscore,81). It is interesting that some recent reviews of acetylcysteine for the prevention of contrast nephropathy analyzed every bit few as 5 studies, despite existence submitted for publication almost i yr after publication of a review of 12 studies (82). Although there are many potential reasons for this, one cannot exclude the possibility that some search strategies missed eligible trials. In addition to a comprehensive search method, which makes it unlikely that relevant studies were missed, information technology is often reassuring if the review squad used graphic and statistical methods to confirm that there was little adventure that publication bias influenced the results (83).
Was the Data Abstraction from Each Study Advisable?
In compiling relevant information, the review team should have used a rigorous and reproducible method of abstracting all relevant information from the chief studies. Often two reviewers abstract fundamental information from each primary written report, including report and patient characteristics, setting, and details almost the intervention, exposure, or diagnostic test as is appropriate. Linguistic communication translators may be needed. Teams who behave their review with due rigor will indicate that they contacted the principal authors from each of the primary studies to confirm the accuracy of abstracted information as well every bit to provide additional relevant data that was not provided in the primary report. Some authors will go through the boosted effort of blinding or masking the results from other study characteristics so that information brainchild is as objective as possible (84,85).
One chemical element that should have been abstracted is the methodologic quality of each principal study (recognizing this is not e'er equally straightforward as information technology may first seem) (86–91). The question to be posed by the reader is whether the reviewing team considered if each of the main studies was designed, conducted, and analyzed in a way to minimize or avert biases in the results (92). For randomized, controlled trials, lack of concealment of resource allotment, inadequate generation of the allocation sequence, and lack of double blinding can exaggerate estimates of the treatment effect (54,ninety,93). The value of abstracting such information is that it may assistance to explain important differences in the results amid the master studies (90).
For example, long-term chance estimates can go unreliable when participants are lost to study follow-upwardly; those who participate in follow-up oftentimes systematically differ from nonparticipants. For this reason, prognosis studies are vulnerable to bias, unless the loss to follow-upward is less than 20% (94). In a systematic review of 49 studies on the renal prognosis of diarrhea associated hemolytic uremic syndrome, on boilerplate, 21% of patients were lost to follow-upwards (range 0 to 59% across studies) (95). It was hypothesized that patients who were lost to follow-up would contribute to worse estimates of long-term prognosis because they are typically healthier than those who continue to be followed by their nephrologists. Indeed, studies with a higher proportion of patients lost to follow-up demonstrated a higher proportion of patients with long-term renal sequelae, explaining 28% of the between-study variability.
How Was the Information Synthesized and Summarized?
In cases in which the primary studies differ in the pattern, populations studied, interventions and comparisons used, or outcomes measured, information technology may have been advisable for the review team simply to written report the results descriptively using text and tables. When the primary studies are similar in these characteristics and the studies provide a similar estimate of a true effect, then meta-analysis may have been used to derive a more than precise estimate of this effect (96). In meta-analysis, data from the individual studies are not simply combined every bit though they were from a single study; rather, greater weights are given to the results from studies that provide more than information, because they are likely to exist closer to true effect being estimated. Mathematically combining the results from the private studies tin exist accomplished under the assumption of "fixed" effects or "random" furnishings model. Although a thorough description and merits of each arroyo is described elsewhere (97), information technology is fair to say that a random-furnishings model is more conservative than the stock-still-effects approach, and a finding that is statistically significant with the latter simply not the onetime should be viewed with skepticism.
Whenever individual studies are pooled in meta-analysis, it is of import for the reader to determine whether it was reasonable to exercise so. 1 mode to appraise the similarity of various studies is to audit the graphic display of the results, looking for similarities in the direction of the estimated outcome. Even without considering any combined meta-analytic effect, a reader becomes much more than confident when a similar consequence is being observed across many studies (i.e., the results have replicated across many studies). Some review teams may report a statistical test to determine how different the studies are from one some other (as described previously, this is often termed heterogeneity of the study results [98]). This can help to evidence or disprove that differences in the results that were observed between the primary studies is no different from what would be expected by take a chance. The nearly common statistical exam to quantify heterogeneity is something called the Q statistic, which is like in concept to a χii test. Although a nonsignificant result (by convention P > 0.1) is ofttimes taken to indicate that in that location are no substantial differences between the studies, it is important to consider that this test is underpowered, peculiarly when the number of studies beingness pooled is modest. A new statistic that is frequently existence reported in meta-analysis these days is something called the I2 statistic. This statistic describes the percentage variability between the studies that is present beyond what would be expected past chance. When interpreting an I2 statistic, values of 0 to 30, 31 to 50, and >50% represent mild, moderate, and marked differences between the studies, respectively (99).
Whenever a review team identifies pregnant differences between the primary studies, they should endeavor to explain possible reasons for these differences. This can be done in an informal style by analyzing certain types of studies separately or by selectively combining studies to determine which are specially unlike from the remaining studies. Alternatively, a statistical arroyo can be taken to explore differences across studies, using a technique like to linear or logistic regression (which at the written report level is something chosen meta-regression) (100). Either way, a careful exploration of why study results differ can yield important information about potential determinants of the effect existence observed.
Conclusions
Similar all types of research, systematic reviews and meta-analyses have both potential strengths and weaknesses. With the growth of renal clinical studies, an increasing number of these types of summary publications will certainly go available to nephrologists, researchers, administrators, and policy makers who seek to keep abreast of recent developments. To maximize their advantages, it is essential that future reviews be conducted and reported properly, with judicious estimation past the discriminating reader.
Acknowledgments
A.10.G. was supported by a Clinician Scientist Honor from the Canadian Institutes of Health Inquiry (CIHR). D.H. was supported by a CIHR Fellowship Accolade, the Chisholm Memorial Fellowship, and the Clinician-Scientist Training Program of the Academy of Toronto. K.T. was supported by a Population Wellness Investigator Laurels from the Alberta Heritage Foundation for Medical Inquiry and a New Investigator Award from the CIHR.
We give thanks Drs. Chi Hsu and Harvey Feldman for assistance and advice. We thank Arthur Iansavichus, MLIS, who helped compile systematic reviews published in the discipline of nephrology.
Footnotes
-
Published online ahead of print. Publication date available at world wide web.cjasn.org.
- Copyright © 2008 past the American Guild of Nephrology
References
- ↵
- ↵
- ↵
- ↵
- ↵
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
- ↵
-
- ↵
- ↵
- ↵
- ↵
-
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
-
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
-
- ↵
-
-
-
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
Source: https://cjasn.asnjournals.org/content/3/1/253
0 Response to "A Key Characteristic of a Systematic Review Is That It Contains a Meta-analysis True False"
Post a Comment