Statistical inference and effect measures in abstracts of randomized controlled trials, 1975-2021. A systematic review

OBJECTIVE

To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs).

STUDY DESGIN AND SETTINGS

We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to detect reporting of statistical inference (p-values, confidence intervals, significance terminology), statistical reporting style of results, and effect measures for binary outcomes, including time-to-event measures. We validated the text mining algorithms by random samples of abstracts.

RESULTS

A total of 320 676 abstracts contained statistical inference. The percentage of abstracts including statistical inference increased from 65% (1975) to 87% (2006) and then decreased slightly. From 1975 to 1990, the sole reporting of language regarding statistical significance was predominant. Since 1990, reporting of p-values without confidence intervals has been the most common reporting style. Reporting of confidence intervals increased from 0.5% (1975) to 29% (2021). The two most common effect measures for binary outcomes were hazard ratios and odds ratios. Number needed to treat and number needed to harm are reported in less than 5% of abstracts with binary endpoints.

CONCLUSIONS

Reporting of statistical inference in abstracts of RCTs has increased over time. Increasingly, p-values and confidence intervals are reported rather than just mentioning the presence of "statistical significance". The reporting of odds ratios comes with the liability that the untrained reader will interpret them as risk ratios, which is often not justified, especially in RCTs.

Gender composition and wage gaps in the Canadian health policy research workforce in comparative perspective

BACKGROUND

Gendered challenges have been shown to persist among health practitioners in countries at all levels of development. Less is known about non-clinical professionals, that is, those who do not deliver services directly but are essential to health systems performance, such as health policy researchers. This national observational study examined gender occupational segregation and wage gaps in the Canadian health policy research workforce using a cross-domain comparative labour market analysis approach.

METHODS

Sourcing data from the 2016 population census, we applied linear regression and Oaxaca-Blinder decomposition techniques to assess wage differentials by sex, traditional human capital measures (e.g., age, education, place of work), and social identity variables intersecting with gender (household head, childcare, migrant status) among health policy researchers aged 25-54. We compared the gender composition and wage gap with seven non-health policy and programme domains, as mapped under the national occupational classification by similarity in the types of work performed.

RESULTS

The health policy research workforce (N = 19 955) was characterized by gender segregation: 74% women, compared with 58% women among non-health policy research occupations (N = 102 555). Women health policy researchers earned on average 4.8% (95% CI 1.5‒8.0%) less than men after adjusting for other professional and personal variables. This gap was wider than among education policy researchers with similar gender composition (75% women; adjusted wage gap of 2.6%). Wages among health policy researchers were 21.1% (95% CI 19.4‒22.8%) lower than their counterparts in the male-dominated economics policy domain, all else being equal. Overall, women's earnings averaged 3.2% lower than men's due to factors that remained unexplained by policy domain or other measured predictors.

CONCLUSIONS

This investigation found that the gender inequalities already widely seen among clinical practitioners are replicated among health policy researchers, potentially hindering the competitiveness of the health sector for attracting and retaining talent. Our findings suggest intersectoral actions are necessary to tackle wage gaps and devaluation of female-dominated health professions. Accountability for gender equity in health must extend to the professionals tasked with conducting equity-informative health policy research.

Influence of non-clinical factors on restorative rectal cancer surgery: An analysis of four specialized population-based digestive cancer registries in France

BACKGROUND

This study aims to measure the association between deprivation, health care accessibility and health care system with the likelihood of receiving non-restorative rectal cancer surgery (NRRCS).

METHODS

All adult patients who had rectal resection for invasive adenocarcinoma diagnosed between 2007 and 2016 in four French specialised cancer registries were included. A multilevel logistic regression with random effect was used to assess the link between patient and health care structure characteristics on the probability of NRRCS.

RESULTS

2997 patients underwent rectal cancer resection in 68 health care structures: 708 (23.63%) had NRRCS. The likelihood of receiving NRCCS was associated with patients' characteristics (97%): age, sub peritoneal rectal tumors, neoadjuvant therapy, residual tumour and stage III . There was no impact of European Deprivation Index or remoteness on NRRCS. Inter-health care structure variability was modest (3%), of which 50% was explained by the high group volume of colorectal procedures and the type of health care structure which were associated with less NRRCS (p<0.01).

CONCLUSION

There is an influence of operating volume and type of structure on the probability of NRRCS, but it has truly little importance in explaining differences in performances. The probability of NRRCS is mainly affected by clinical determinant.

Incorporating radiomics into clinical trials: expert consensus endorsed by the European Society of Radiology on considerations for data-driven compared to biologically driven quantitative biomarkers

Existing quantitative imaging biomarkers (QIBs) are associated with known biological tissue characteristics and follow a well-understood path of technical, biological and clinical validation before incorporation into clinical trials. In radiomics, novel data-driven processes extract numerous visually imperceptible statistical features from the imaging data with no a priori assumptions on their correlation with biological processes. The selection of relevant features (radiomic signature) and incorporation into clinical trials therefore requires additional considerations to ensure meaningful imaging endpoints. Also, the number of radiomic features tested means that power calculations would result in sample sizes impossible to achieve within clinical trials. This article examines how the process of standardising and validating data-driven imaging biomarkers differs from those based on biological associations. Radiomic signatures are best developed initially on datasets that represent diversity of acquisition protocols as well as diversity of disease and of normal findings, rather than within clinical trials with standardised and optimised protocols as this would risk the selection of radiomic features being linked to the imaging process rather than the pathology. Normalisation through discretisation and feature harmonisation are essential pre-processing steps. Biological correlation may be performed after the technical and clinical validity of a radiomic signature is established, but is not mandatory. Feature selection may be part of discovery within a radiomics-specific trial or represent exploratory endpoints within an established trial; a previously validated radiomic signature may even be used as a primary/secondary endpoint, particularly if associations are demonstrated with specific biological processes and pathways being targeted within clinical trials. KEY POINTS: • Data-driven processes like radiomics risk false discoveries due to high-dimensionality of the dataset compared to sample size, making adequate diversity of the data, cross-validation and external validation essential to mitigate the risks of spurious associations and overfitting. • Use of radiomic signatures within clinical trials requires multistep standardisation of image acquisition, image analysis and data mining processes. • Biological correlation may be established after clinical validation but is not mandatory.

Coronary heart disease mortality is decreasing in Argentina, and Colombia, but keeps increasing in Mexico: a time trend study

Background

Mortality rates due to coronary heart disease (CHD) have decreased in most countries, but increased in low and middle-income countries. Few studies have analyzed the trends of coronary heart disease mortality in Latin America, specifically the trends in young-adults and the effect of correcting these comparisons for nonspecific causes of death (garbage codes). The objective of this study was to describe and compare standardized, age-specific, and garbage-code corrected mortality trends for coronary heart disease from 1985 to 2015 in Argentina, Colombia, and Mexico.

Methods

Deaths from coronary heart disease were grouped by country, year of registration, sex, and 10-year age bands to calculate age-adjusted and age and sex-specific rates for adults aged ≥25. We corrected for garbage-codes using the methodology proposed by the Global Burden of Disease. Finally, we fitted Joinpoint regression models.

Results

In 1985, age-standardized mortality rates per 100,000 population were 136.6 in Argentina, 160.6 in Colombia, and 87.51 in Mexico; by 2015 rates decreased 51% in Argentina and 6.5% in Colombia, yet increased by 61% in Mexico, where an upward trend in mortality was observed in young adults. Garbage-code corrections produced increases in mortality rates, particularly in Argentina with approximately 80 additional deaths per 100,000, 14 in Colombia and 13 in Mexico.

Conclusions

Latin American countries are at different stages of the cardiovascular disease epidemic. Garbage code correction produce large changes in the mortality rates in Argentina, yet smaller in Mexico and Colombia, suggesting garbage code corrections may be needed for specific countries. While coronary heart disease (CHD) mortality is falling in Argentina, modest falls in Colombia and substantial increases in Mexico highlight the need for the region to propose and implement population-wide prevention policies.

An Introduction to Statistics - Data Types, Distributions and Summarizing Data

The purpose of research is to gather data, which can then be used to inform decision-making. Data can be of various types and an understanding of this is crucial for its proper analysis and interpretation. In this article, we look at various types and distributions of data, and methods to summarize this data.

How to cite this article

Ranganathan P, Gogtay NJ, An Introduction to Statistics - Data Types, Distributions and Summarizing Data. Indian J Crit Care Med 2019;23(Suppl 2):S169-S170.

Experience and Availability of Pelvic Binders at Swedish Trauma Units; A Nationwide Survey

Objective:

To assess availability, experience, and knowledge about the Pelvic Circumferential Compression Device (PCCD) in Sweden.

Methods:

A telephone interview with the current on-call trauma doctors at all trauma units in Sweden was conducted. After a short presentation and oral consent, the doctors were asked to answer four short questions. We asked the doctors to answer whether they knew if they had PCCDs available in their emergency room, how many times had they applied a PCCD, which is the correct level of application for a PCCD, and if a PCCD can stop arterial bleeding.

Results:

The on-call trauma doctors at the nine University hospitals, twenty-two General hospitals and twenty-one District General hospitals, with response rate of 100%, were interviewed. Availability of PCCD was 85 % and there was no difference between hospital types (p=0.546). In all hospitals 29/52 (56%) of those interviewed had used a PCCD at least once. There were significantly more doctors that had used a PCCD at least once in the University hospitals (8/9), compared to General hospitals (13/22) and District General hospitals (8/21) (p=0.034). A total of 43/52 (83 %) doctors defined the greater trochanters as the correct level of application for a PCCD. No difference was found when comparing hospitals (p=0.208). Only 22/52 (42 %) of doctors answered that a PCCD could not stop an arterial bleeding. No difference was found between hospitals (p=0.665)

Conclusion:

Less than half of the doctors knew that a PCCD cannot stop arterial bleeding, while the majority knew the correct level of application of a PCCD.

The importance of data structure in statistical analysis of dendritic spine morphology

BACKGROUND

Dendritic spine morphology is heterogeneous and highly dynamic. To study the changing or aberrant morphology in test setups, often spines from several neurons from a few experimental units e.g. mice or primary neuronal cultures are measured. This strategy results in a multilevel data structure, which, when not properly addressed, has a high risk of producing false positive and false negative findings.

METHODS

We used mixed-effects models to deal with data with a multilevel data structure and compared this method to analyses at each level. We apply these statistical tests to a dataset of dendritic spine morphology parameters to illustrate advantages of multilevel mixed-effects model, and disadvantages of other models.

RESULTS

We present an application of mixed-effects models for analyzing dendritic spine morphology datasets while correcting for the data structure.

COMPARISON WITH EXISTING METHODS

We further show that analyses at spine level and aggregated levels do not adequately account for the data structure, and that they may lead to erroneous results.

CONCLUSION

We highlight the importance of data structure in dendritic spine morphology analyses and highly recommend the use of mixed-effects models or other appropriate statistical methods to deal with multilevel datasets. Mixed-effects models are easy to use and superior to commonly used methods by including the data structure and the addition of other explanatory variables, for example sex, and age, etc., as well as interactions between variables or between variables and level identifiers.

Cohort study of trials submitted to ethics committee identified discrepant reporting of outcomes in publications

OBJECTIVES

To identify factors associated with discrepant outcome reporting in randomized drug trials.

STUDY DESIGN AND SETTING

Cohort study of protocols submitted to a Swiss ethics committee 1988-1998: 227 protocols and amendments were compared with 333 matching articles published during 1990-2008. Discrepant reporting was defined as addition, omission, or reclassification of outcomes.

RESULTS

Overall, 870 of 2,966 unique outcomes were reported discrepantly (29.3%). Among protocol-defined primary outcomes, 6.9% were not reported (19 of 274), whereas 10.4% of reported outcomes (30 of 288) were not defined in the protocol. Corresponding percentages for secondary outcomes were 19.0% (284 of 1,495) and 14.1% (334 of 2,375). Discrepant reporting was more likely if P values were <0.05 compared with P ≥ 0.05 [adjusted odds ratio (aOR): 1.38; 95% confidence interval (CI): 1.07, 1.78], more likely for efficacy compared with harm outcomes (aOR: 2.99; 95% CI: 2.08, 4.30) and more likely for composite than for single outcomes (aOR: 1.48; 95% CI: 1.00, 2.20). Cardiology (aOR: 2.34; 95% CI: 1.44, 3.79) and infectious diseases (aOR: 1.77; 95% CI: 1.01, 3.13) had more discrepancies compared with all specialties combined.

CONCLUSION

Discrepant reporting was associated with statistical significance of results, type of outcome, and specialty area. Trial protocols should be made freely available, and the publications should describe and justify any changes made to protocol-defined outcomes.