Finding your way out of the forest without a trail of bread crumbs: development and evaluation of two novel displays of forest plots

Heterogeneity in meta-analysis: a comprehensive overview

In recent years, meta-analysis has evolved to a critically important field of Statistics, and has significant applications in Medicine and Health Sciences. In this work we briefly present existing methodologies to conduct meta-analysis along with any discussion and recent developments accompanying them. Undoubtedly, studies brought together in a systematic review will differ in one way or another. This yields a considerable amount of variability, any kind of which may be termed heterogeneity. To this end, reports of meta-analyses commonly present a statistical test of heterogeneity when attempting to establish whether the included studies are indeed similar in terms of the reported output or not. We intend to provide an overview of the topic, discuss the potential sources of heterogeneity commonly met in the literature and provide useful guidelines on how to address this issue and to detect heterogeneity. Moreover, we review the recent developments in the Bayesian approach along with the various graphical tools and statistical software that are currently available to the analyst. In addition, we discuss sensitivity analysis issues and other approaches of understanding the causes of heterogeneity. Finally, we explore heterogeneity in meta-analysis for time to event data in a nutshell, pointing out its unique characteristics.

Say farewell to bland regression reporting: Three forest plot variations for visualizing linear models

Regression ranks among the most popular statistical analysis methods across many research areas, including psychology. Typically, regression coefficients are displayed in tables. While this mode of presentation is information-dense, extensive tables can be cumbersome to read and difficult to interpret. Here, we introduce three novel visualizations for reporting regression results. Our methods allow researchers to arrange large numbers of regression models in a single plot. Using regression results from real-world as well as simulated data, we demonstrate the transformations which are necessary to produce the required data structure and how to subsequently plot the results. The proposed methods provide visually appealing ways to report regression results efficiently and intuitively. Potential applications range from visual screening in the model selection stage to formal reporting in research papers. The procedure is fully reproducible using the provided code and can be executed via free-of-charge, open-source software routines in R.

The fraternal birth-order effect as a statistical artefact: convergent evidence from probability calculus, simulated data, and multiverse meta-analysis

The fraternal-birth order effect (FBOE) is a research claim which states that each older brother increases the odds of homosexual orientation in men via an immunoreactivity process known as the maternal immune hypothesis. Importantly, older sisters supposedly either do not affect these odds, or affect them to a lesser extent. Consequently, the fraternal birth-order effect predicts that the association between the number of older brothers and homosexual orientation in men is greater in magnitude than any association between the number of older sisters and homosexual orientation. This difference in magnitude represents the main theoretical estimand of the FBOE. In addition, no comparable effects should be observable among homosexual vs heterosexual women. Here, we triangulate the empirical foundations of the FBOE from three distinct, informative perspectives, complementing each other: first, drawing on basic probability calculus, we deduce mathematically that the body of statistical evidence used to make inferences about the main theoretical estimand of the FBOE rests on incorrect statistical reasoning. In particular, we show that throughout the literature researchers ascribe to the false assumptions that effects of family size should be adjusted for and that this could be achieved through the use of ratio variables. Second, using a data-simulation approach, we demonstrate that by using currently recommended statistical practices, researchers are bound to frequently draw incorrect conclusions. And third, we re-examine the empirical evidence of the fraternal birth-order effect in men and women by using a novel specification-curve and multiverse approach to meta-analysis (64 male and 17 female samples, N = 2,778,998). When analyzed correctly, the specific association between the number of older brothers and homosexual orientation is small, heterogenous in magnitude, and apparently not specific to men. In addition, existing research evidence seems to be exaggerated by small-study effects.

Elucidating the Associations Between Achievement Goals and Academic Dishonesty: a Meta-analysis

Academic dishonesty is a pervasive problem undermining the effectiveness of educational institutions. From a motivational perspective, researchers have proposed achievement goals as antecedents of academic dishonesty. Empirical findings corroborate the notion that mastery goals (focus on learning and competence development) are negatively linked to academic dishonesty. However, even though theoretical considerations suggest positive links between performance goals (focus on competence demonstration) and academic dishonesty, empirical findings are mixed. To provide a better understanding of how goals matter for academic dishonesty, we conducted three-level meta-analyses encompassing 163 effect sizes from 33 studies and a total of 19,787 participants. We found a disproportional use of correlational designs (using self-report measures of academic dishonesty) and personal goal measures (opposed to surrounding goal structures). Evidence of publication bias was not found. Our results confirmed the expected negative associations between mastery goals and academic dishonesty and revealed heterogenous findings for performance goals, with indications of positive associations within behavioral and intentional dishonesty measures, but not within self-reports. To further clarify the associations between achievement goals and academic dishonesty, we call for more methodological rigor in the measurement of goals and dishonesty as well as multi-methods approaches when investigating their interplay.

Phenotypic characteristics and variability in CHARGE syndrome: a PRISMA compliant systematic review and meta-analysis

Background

CHARGE syndrome (OMIM #214800) is a phenotypically complex genetic condition characterised by multi-system, multi-sensory impairments. Behavioural, psychological, cognitive and sleep difficulties are not well delineated and are likely associated with biopsychosocial factors.

Methods

This meta-analysis investigated the prevalence of clinical features, physical characteristics and conditions, behavioural, psychological, cognitive and sleep characteristics in CHARGE syndrome, and statistically evaluated directional associations between these characteristics. Pooled prevalence estimates were calculated using reliable, prespecified quality weighting criteria, and meta-regression was conducted to identify associations between characteristics.

Results

Of the 42 eligible studies, data could be extracted for 1675 participants. Prevalence estimates were highest for developmental delay (84%), intellectual disability (64%), aggressive behaviour (48%), self-injurious behaviour (44%) and sleep difficulties (45%). Meta-regression indicated significant associations between intellectual disability and choanal atresia, intellectual disability and inner ear anomalies, sleep difficulties and growth deficiency, and sleep difficulties and gross motor difficulties.

Conclusions

Our comprehensive review of clinical features, behavioural, psychological, cognitive and physical characteristics, conditions and comorbidities in CHARGE syndrome provides an empirically based foundation to further research and practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s11689-022-09459-5.

Vitamin D and SARS-CoV2 infection, severity and mortality: A systematic review and meta-analysis

To assess the evidence on SARS-CoV2 infection and Covid-19 in relation to deficiency and supplementation of vitamin D, we conducted a systematic review up to April 2021. We summarised data from 38 eligible studies, which presented risk estimates for at least one endpoint, including two RCT and 27 cohort-studies: 205565 patients with information on 25OHD status and 2022 taking vitamin D supplementation with a total of 1197 admitted to the ICU or who needed invasive mechanical ventilation or intubation and hospital stay, and more than 910 Covid-19 deaths. Primary outcomes were severity and mortality and the main aim was to evaluate the association with vitamin D supplementation. Random effects models showed that supplementation was associated with a significant lower risk of both Covid-19 severe disease (SRR 0.38, 95% CI 0.20-0.72, 6 studies) and mortality (SRR 0.35, 95% CI 0.17-0.70, 8 studies). There were no statistically significant dose differences between studies: summary estimates with regular doses remain statistically significant, suggesting that higher doses are not necessary. For patients on vitamin D supplementation, a greater reduction in mortality risk emerged in older individuals and at higher latitudes. Regarding the quality of studies, assessed using the New Castle-Ottawa quality scale, the analysis revealed in most cases no statistically significant differences between low, medium or high quality studies. We found significant associations of vitamin D supplementation with Covid-19, encompassing risks of disease worsening and mortality, especially in seasons characterized by 25OHD deficiency and with not severe patients. Dedicated randomized clinical studies are encouraged to confirm these results.

Depression and Psychosis Risk Shared Vulnerability for Motor Signs Across Development, Symptom Dimensions, and Familial Risk

BACKGROUND

Motor abnormalities are strong transdiagnostic indicators of psychopathology risk that reflect emerging neural network abnormalities. Indeed, motor signs, such as motor slowing and agitation, are widely recognized as core features of both psychosis and depression. However, it is unclear whether these reflect shared or distinct etiology.

METHODS

A sample of 11 878 adolescents completed self-reported clinical measures of rated psychotic-like experiences (PLEs) and depression. Familial risk for psychopathology and the presence of motor signs were drawn from parental reports, including developmental motor delays (eg, sitting, walking), and adolescent motor signs (eg, dyscoordination, psychomotor retardation, and psychomotor agitation). Finally, motor network connectivity in theoretically relevant networks (cortico-striatal, cortico-thalamic, and cortico-cerebellar) were related to symptoms and familial risk for psychopathology.

RESULTS

Developmental motor delays related to increased PLEs, increased depression symptoms, and greater familial risk. Familial risk for both PLEs and depression showed higher rates of developmental motor delays than all other groups. Adolescent motor signs, however, showed unique patterns of relationships to symptoms and familial risk such that dyscoordination reflected risk for PLEs, both psychomotor agitation and retardation reflected depression risk, and psychomotor agitation reflected transdiagnostic risk. Cortico-striatal connectivity was related to depression and PLEs, but cortico-cerebellar connectivity was linked to PLEs only.

CONCLUSIONS

Motor signs may be a transdiagnostic marker of vulnerability for psychopathology. Early developmental motor delays could belie pluripotent, familial risk features. Unique items, eg, dyscoordination specifically related to PLEs, possibly reflecting processes inherent in distinct emerging forms of psychopathology.

Investigating weighted fishing hooks for seabird bycatch mitigation

Fisheries bycatch threatens the viability of some seabird populations and reduces fishing efficiency. Albatross bycatch in a US North Pacific tuna longline fishery has increased over the past decade and now exceeds 1000 annual captures. Seabirds interacting with this fishery reach hooks at depths up to 1 m. A branchline weight's mass and distance from the hook affect seabird catch rates. We conducted experimental fishing to compare the commercial viability of a weighted hook relative to conventional gear with weights attached 0.75 m from the hook. We used a Bayesian random effects meta-analytic regression modelling approach to estimate pooled expected species-specific log relative risk of capture on conventional versus experimental gear. There was a significant 53% (95% HDI: - 75 to - 25%) decrease in retained species' catch rates on experimental hooks, indicating an unacceptable economic cost, and no significant effect for discarded species. Using a Bayesian general linear mixed regression modelling approach, experimental hooks sank to 85 cm ca. 1.4 times (95% HDI: 1.37-1.48) faster than control hooks. Given their potential to reduce seabird catch rates, eliminate safety risks from bite-offs and facilitate robust compliance monitoring, it is a priority to find a weighted hook design with acceptable catch rates.

Meta-analyzing intelligence and religiosity associations: Evidence from the multiverse

Over the past century, a remarkable body of research about the relationship of intelligence and religiosity has accumulated. So far, the majority of studies that investigated this relationship showed a negative correlation, indicating lower cognitive abilities of individuals reporting stronger religious beliefs. Although the effect direction has been observed to be largely consistent across studies, the reported effect strength varied substantially across studies. Several potentially moderating variables such as different intelligence and religiosity assessment methods, educational status of samples, and participant sex have been proposed as likely candidates for explaining systematic differences in effect strengths. However, the effects of these moderators are to date unclear. Consequently, we focused in investigating effects of these moderating variables on the intelligence and religiosity link in an update of prior meta-analytical investigations in n = 89 (k = 105; N = 201,457) studies. Random-effects analyses showed a small but robust negative association between intelligence and religiosity r = -.14 (p < .001; 95% CI [-.17, -.12]). Effects were stronger for (i) psychometric intelligence tests than for proxy measures such as grade point averages and (ii) general population and college samples than pre-college samples. Moreover, we provide evidence from combinatorial, multiverse, and specification curve analyses that further corroborates the robustness of the investigated association. Out of 192 reasonable specifications all 135 (70.4%) significant summary effects were negative. In all, our results show small but robust negative associations between religiosity and intelligence that are differentiated in strength but generalize in terms of direction over moderating variables.

Motor abnormalities, depression risk, and clinical course in adolescence

Background

Motor abnormalities, such as psychomotor agitation and retardation, are widely recognized as core features of depression. However, it is not currently known if motor abnormalities connote risk for depression.

Methods

Using data from the Adolescent Brain Cognitive Development (ABCD) Study, a nationally representative sample of youth (n=10,835, 9-11 years old), the present paper examines whether motor abnormalities are associated with (a) depression symptoms in early adolescence, (b) familial risk for depression (familial risk loading), and (c) future depression symptoms. Motor abnormalities measures included traditional (DSM) motor signs such as psychomotor agitation and retardation as well as other motor domains such as developmental motor delays and dyscoordination.

Results

Traditional motor abnormalities were less prevalent (agitation=3.2%, retardation=0.3%) than non-traditional domains (delays=13.79%, coordination=35.5%) among adolescents. Motor dysfunction was associated with depression symptoms (Cohen's ds=0.02 to 0.12). Familial risk for depression was related to motor abnormalities (Cohen's ds=0.08 to 0.27), with the exception of motor retardation. Family vulnerability varied in sensitivity to depression risk (e.g., retardation: .53%; dyscoordination: 32.05%). Baseline endorsement of motor abnormalities predicted future depression symptoms at one-year follow-up.

Conclusions

These findings suggest that motor signs reflect a novel, promising future direction for examining vulnerability to depression risk in early adolescence.

In vitro susceptibility of Trypanosoma cruzi discrete typing units (DTUs) to benznidazole: A systematic review and meta-analysis

BACKGROUND

Chagas disease, a neglected tropical disease endemic to Latin America caused by the parasite Trypanosoma cruzi, currently affects 6-7 million people and is responsible for 12,500 deaths each year. No vaccine exists at present and the only two drugs currently approved for the treatment (benznidazole and nifurtimox), possess serious limitations, including long treatment regimes, undesirable side effects, and frequent clinical failures. A link between parasite genetic variability and drug sensibility/efficacy has been suggested, but remains unclear. Therefore, we investigated associations between T. cruzi genetic variability and in vitro benznidazole susceptibility via a systematic article review and meta-analysis.

METHODOLOGY/PRINCIPAL FINDINGS

In vitro normalized benznidazole susceptibility indices (LC50 and IC50) for epimastigote, trypomastigote and amastigote stages of different T. cruzi strains were recorded from articles in the scientific literature. A total of 60 articles, which include 189 assays, met the selection criteria for the meta-analysis. Mean values for each discrete typing unit (DTU) were estimated using the meta and metaphor packages through R software, and presented in a rainforest plot. Subsequently, a meta-regression analysis was performed to determine differences between estimated mean values by DTU/parasite stage/drug incubation times. For each parasite stage, some DTU mean values were significantly different, e.g. at 24h of drug incubation, a lower sensitivity to benznidazole of TcI vs. TcII trypomastigotes was noteworthy. Nevertheless, funnel plots detected high heterogeneity of the data within each DTU and even for a single strain.

CONCLUSIONS/SIGNIFICANCE

Several limitations of the study prevent assigning DTUs to different in vitro benznidazole sensitivity groups; however, ignoring the parasite's genetic variability during drug development and evaluation would not be advisable. Our findings highlight the need for establishment of uniform experimental conditions as well as a screening of different DTUs during the optimization of new drug candidates for Chagas disease treatment.

Harnessing technological solutions for childhood obesity prevention and treatment: a systematic review and meta-analysis of current applications

BACKGROUND

Technology holds promise for delivery of accessible, individualized, and destigmatized obesity prevention and treatment to youth.

OBJECTIVES

This review examined the efficacy of recent technology-based interventions on weight outcomes.

METHODS

Seven databases were searched in April 2020 following PRISMA guidelines. Inclusion criteria were: participants aged 1-18 y, use of technology in a prevention/treatment intervention for overweight/obesity; weight outcome; randomized controlled trial (RCT); and published after January 2014. Random effects models with inverse variance weighting estimated pooled mean effect sizes separately for treatment and prevention interventions. Meta-regressions examined the effect of technology type (telemedicine or technology-based), technology purpose (stand-alone or adjunct), comparator (active or no-contact control), delivery (to parent, child, or both), study type (pilot or not), child age, and intervention duration.

FINDINGS

In total, 3406 records were screened for inclusion; 55 studies representing 54 unique RCTs met inclusion criteria. Most (89%) included articles were of high or moderate quality. Thirty studies relied mostly or solely on technology for intervention delivery. Meta-analyses of the 20 prevention RCTs did not show a significant effect of prevention interventions on weight outcomes (d = 0.05, p = 0.52). The pooled mean effect size of n = 32 treatment RCTs showed a small, significant effect on weight outcomes (d = ‒0.13, p = 0.001), although 27 of 33 treatment studies (79%) did not find significant differences between treatment and comparators. There were significantly greater treatment effects on outcomes for pilot interventions, interventions delivered to the child compared to parent-delivered interventions, and as child age increased and intervention duration decreased. No other subgroup analyses were significant.

CONCLUSIONS

Recent technology-based interventions for the treatment of pediatric obesity show small effects on weight; however, evidence is inconclusive on the efficacy of technology based prevention interventions. Research is needed to determine the comparative effectiveness of technology-based interventions to gold-standard interventions and elucidate the potential for mHealth/eHealth to increase scalability and reduce costs while maximizing impact.

Visual representations of meta-analyses of multiple outcomes: Extensions to forest plots, funnel plots, and caterpillar plots

Meta-analytic datasets can be large, especially when in primary studies multiple effect sizes are reported. The visualization of meta-analytic data is therefore useful to summarize data and understand information reported in primary studies. The gold standard figures in meta-analysis are forest and funnel plots. However, none of these plots can yet account for the existence of multiple effect sizes within primary studies. This manuscript describes extensions to the funnel plot, forest plot and caterpillar plot to adapt them to three-level meta-analyses. For forest plots, we propose to plot the study-specific effects and their precision, and to add additional confidence intervals that reflect the sampling variance of individual effect sizes. For caterpillar plots and funnel plots, we recommend to plot individual effect sizes and averaged study-effect sizes in two separate graphs. For the funnel plot, plotting separate graphs might improve the detection of both publication bias and/or selective outcome reporting bias.

Commercial viability of locating pelagic longline branchline weights at the hook to reduce seabird bycatch

Bycatch in pelagic longline fisheries threatens the viability of some seabird populations and reduces fishing efficiency. The mass of a branchline fishing weight and its distance from the hook significantly explain seabird catch risk during setting and hauling. We surveyed Hawaii tuna longline fishers to obtain their perspectives on the practicality, safety and economic viability of alternative designs which place weights adjacent to hooks instead of conventional designs with weighted swivels attached 0.6 m from the hook. We conducted experimental fishing to explore the commercial viability of a combination weighted swivel attached to the eye of the hook. We used a Bayesian random effects meta-analytic regression modeling approach to estimate pooled expected species-specific log relative risk of capture on conventional, as opposed to experimental, branchlines. There were significant decreases in catch rates on experimental branchlines of 54% (95% highest posterior density interval [HDI]: 34.3 to 73.8) for retained species and 28% (95% HDI: -7.5 to 93.3) for discarded species. For about 10% of experimental weighted hooks, the point of the hook tangled on the swivel, likely contributing to lower catch rates and making it more difficult for crew to coil branchlines. We identify possible factors explaining why the experimental gear reduced catch rates. The study demonstrates the benefits of stakeholder scoping to determine bycatch mitigation options with the highest promise for practicality, economic viability and safety, as well as benefits of conducting a pilot study to assess commercial viability to determine if a full experiment to assess bycatch mitigation efficacy is warranted.

The orchard plot: Cultivating a forest plot for use in ecology, evolution, and beyond

"Classic" forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution, meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a "forest-like plot," showing point estimates (with 95% confidence intervals [CIs]) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the "orchard plot." Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also include 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis

BACKGROUND

Data-visualization methods are essential to explore and communicate meta-analytic data and results. With a large number of novel graphs proposed quite recently, a comprehensive, up-to-date overview of available graphing options for meta-analysis is unavailable.

METHODS

We applied a multi-tiered search strategy to find the meta-analytic graphs proposed and introduced so far. We checked more than 150 retrievable textbooks on research synthesis methodology cover to cover, six different software programs regularly used for meta-analysis, and the entire content of two leading journals on research synthesis. In addition, we conducted Google Scholar and Google image searches and cited-reference searches of prior reviews of the topic. Retrieved graphs were categorized into a taxonomy encompassing 11 main classes, evaluated according to 24 graph-functionality features, and individually presented and described with explanatory vignettes.

RESULTS

We ascertained more than 200 different graphs and graph variants used to visualize meta-analytic data. One half of these have accrued within the past 10 years alone. The most prevalent classes were graphs for network meta-analysis (45 displays), graphs showing combined effect(s) only (26), funnel plot-like displays (24), displays showing more than one outcome per study (19), robustness, outlier and influence diagnostics (15), study selection and p-value based displays (15), and forest plot-like displays (14). The majority of graphs (130, 62.5%) possessed a unique combination of graph features.

CONCLUSIONS

The rich and diverse set of available meta-analytic graphs offers a variety of options to display many different aspects of meta-analyses. This comprehensive overview of available graphs allows researchers to make better-informed decisions on which graphs suit their needs and therefore facilitates using the meta-analytic tool kit of graphs to its full potential. It also constitutes a roadmap for a goal-driven development of further graphical displays for research synthesis.

Are viral-infections associated with Ménière's Disease? A systematic review and meta-analysis of molecular-markers of viral-infection in case-controlled observational studies of MD

Despite considerable research, it remains controversial as to whether viral-infections are associated with Meniere’s Disease (MD), a clinically heterogeneous set of chronic inner-ear disorders strongly associated with endolymphatic hydrops. Here, we investigated whether viral-infections are associated with MD through a systematic review and meta-analysis of observational clinical studies using molecular-diagnostics. Eligible for inclusion were case-controlled studies which ascertained molecular-determinants of past or present viral-infection through either viral nucleic acids or host serological marker in MD cases and non-MD controls. Across online databases and grey literature, we identified 210 potentially relevant articles in the English language, from which a total of 14 articles fully satisfied our eligibility criteria such that meta-groups of 611 MD-cases and 373 controls resulted. The aggregate quality of the modest-sized (14 studies) body of evidence was limited and varied considerably with regards to participant selection, matching, and ascertainment(s) and determinant(s) of viral-infection. Most data identified concerned the human cytomegalovirus (CMV), and meta-analysis of eligible studies revealed that evidence of CMV-infection was associated approximately three-fold with MD compared to controls, however the timing of the infections was indeterminate as the pooled analyses combined antiviral serological markers with viral nucleic acid markers. No association was found for any of HSV-1, -2, VZV, or EBV. Associative analyses of any viral species not aforementioned were precluded by limited data, and thus potential associations between other viral species and MD, especially other than Herpesviridae, are yet to be characterised. Overall, we have found a small association between CMV-infection and MD, however it is to be determined for what sub-groups of MD this finding may be relevant, and ideally the reported association remains would be reproduced by a greater volume of higher quality evidence.

Multiple uses of forest plots in presenting analysis results in health research: A Tutorial

BACKGROUND AND OBJECTIVES

Forest plots are an important graphical method in meta-analyses used to show results from individual studies and pooled analyses. Forest plots are easy and straightforward to understand because they provide tabular and graphical information about estimates of comparisons or associations, corresponding precision, and statistical significance. This visual representation also makes it easier to see variations between individual study results. Forest plots are widely used in not only systematic reviews and meta-analyses but also observational studies and clinical trials. In this study, we aimed to show readers the various uses of forest plots in displaying analysis results.

STUDY DESIGN AND SETTING

We used a tutorial type to show multiple uses of forest plots in meta-analyses, clinical trials, and observational studies according to the PICO (Population or Subgroup, Intervention or Exposure, Control, and Outcome) framework.

RESULTS

We introduced forest plots' structure, application, current practice, and research advances in health research. We provided some examples from the literature to show the various uses of forest plot-type graphics in health research including meta-analyses, clinical trials, and observational studies.

CONCLUSION

It is expected that our discussion of the current multiple uses of forest plots in meta-analyses, clinical trials, and observational studies provides a glimpse about their potential in displaying results in a way that makes comparisons between items easier.

Which Data to Meta-Analyze, and How?

Abstract. Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.

Rainforest plots for the presentation of patient-subgroup analysis in clinical trials

While the conventional forest plot is useful to present results within subgroups of patients in clinical studies, it has been criticized for several reasons. First, small subgroups are visually overemphasized by long confidence interval lines, which is misleading. Second, the point estimates of large subgroups are difficult to discern because of the large box representing the precision of the estimate within subgroups. Third, confidence intervals depicted by lines might incorrectly convey the impression that all points within the interval are equally likely. Rainforest plots have been proposed to overcome these potentially misleading aspects of conventional forest plots. The metaviz package enables to generate rainforest plots for meta-analysis within the statistical computing environment R. We suggest the application of rainforest plots for the depiction of subgroup analysis in clinical trials. In this tutorial, detailed step-by-step guidance on the generation of rainforest plot for this purpose is provided.

Cross-hairs: a scatterplot for meta-analysis in R

We describe a meta-analytic scatterplot that indicates precision of points for two variables paired within studies; this is equivalent in form to a 'cross-hairs' plot used to portray specificity and sensitivity in diagnostic testing. At the user's discretion, the plot also displays boxplots for each of the X and Y variable distributions, means for each of the variables, and the correlation between the two. The cross-hairs may be suppressed for dense point clouds. The program is written in R, so it can be modified by the user and can serve as a companion to existing meta-analysis programs. Some of the program's novel uses are described and illustrated with (1) independent effect sizes, (2) dependent effect sizes, and (3) shrunken estimates. Copyright © 2016 John Wiley & Sons, Ltd.

An evaluation of harvest plots to display results of meta-analyses in overviews of reviews: a cross-sectional study

BACKGROUND

Harvest plots are used to graphically display evidence from complex and diverse studies or results. Overviews of reviews bring together evidence from two or more systematic reviews. Our objective was to determine the feasibility of using harvest plots to depict complex results of overviews of reviews.

METHODS

We conducted a survey of 279 members of Cochrane Child Health to determine their preferences for graphical display of data, and their understanding of data presented in the form of harvest plots. Preferences were rated on a scale of 0-100 (100 most preferred) and tabulated using descriptive statistics. Knowledge and accuracy were assessed by tabulating the number of correctly answered questions for harvest plots and traditional data summary tables; t-tests were used to compare responses between formats.

RESULTS

53 individuals from 7 countries completed the survey (19%): 60% were females; the majority had an MD (38%), PhD (47%), or equivalent. Respondents had published a median of 3 systematic reviews (inter-quartile range 1 to 8). There were few differences between harvest plots and tables in terms of being: well-suited to summarize and display results from meta-analysis (52 vs. 56); easy to understand (53 vs. 51); and, intuitive (49 vs. 44). Harvest plots were considered more aesthetically pleasing (56 vs. 44, p = 0.03). 40% felt the harvest plots could be used in conjunction with tables to display results from meta-analyses; additionally, 45% felt the harvest plots could be used with some improvement. There was no statistically significant difference in percentage of knowledge questions answered correctly for harvest plots compared with tables. When considering both types of data display, 21% of knowledge questions were answered incorrectly.

CONCLUSIONS

Neither harvest plots nor standard summary tables were ranked highly in terms of being easy to understand or intuitive, reflecting that neither format is ideal to summarize the results of meta-analyses in overviews of reviews. Responses to knowledge questions showed some misinterpretation of results of meta-analyses. Reviewers should ensure that messages are clearly articulated and summarized in the text to avoid misinterpretation.