Developing a quality scoring system for epidemiological surveys of genetic disorders

A scoping review shows that no single existing risk of bias assessment tool considers all sources of bias for cross-sectional studies

OBJECTIVES

Different tools to assess the potential risk of bias (RoB) for cross-sectional studies have been developed, but it is unclear whether all pertinent bias concepts are addressed. We aimed to identify RoB concepts applicable to cross-sectional research validity and to explore coverage for each in existing appraisal tools.

STUDY DESIGN AND SETTING

This scoping review followed the Joanna Briggs Institute methodology. We included records of any study design describing or reporting methods, concepts or tools used to consider RoB in health research reported to be descriptive/prevalence survey or analytic/association (cross-sectional) study designs. Synthesis included quantitative and qualitative analysis.

RESULTS

Of the 4556 records screened, 90 were selected for inclusion; 67 (74%) described the development of, or validation process for, appraisal tools, 15 (17%) described methodological content or theory relevant to RoB for cross-sectional studies and 8 (9%) records of methodological systematic reviews. Review of methodological reports identified important RoB concepts for both descriptive/prevalence and analytic/association studies. Tools identified (n = 64 unique tools) were either intended to appraise quality or assess RoB in multiple study designs including cross-sectional studies (n = 21; 33%) or cross-sectional designs alone (n = 43; 67%). Several existing tools were modified (n = 17; 27%) for application to cross-sectional studies. The RoB items most frequently addressed in the RoB tools were validity and reliability of the exposure (53%) or outcome (65%) measurement and representativeness of the study population (59%). Most tools did not consider nonresponse or missingness appropriately or at all.

CONCLUSION

Assessing cross-sectional studies involve unique RoB considerations. We identified RoB tools designed for broad applicability across various study designs as well as those specifically tailored for cross-sectional studies. However, none of the identified tools comprehensively address all potential biases pertinent to cross-sectional studies. Our findings indicate a need for continued improvement of RoB tools and suggest that the development of context-specific or more precise tools for this study design may be necessary.

Birth prevalence of phenylalanine hydroxylase deficiency: a systematic literature review and meta-analysis

BACKGROUND

Phenylalanine hydroxylase (PAH) deficiency is an autosomal recessive disorder that results in elevated concentrations of phenylalanine (Phe) in the blood. If left untreated, the accumulation of Phe can result in profound neurocognitive disability. The objective of this systematic literature review and meta-analysis was to estimate the global birth prevalence of PAH deficiency from newborn screening studies and to estimate regional differences, overall and for various clinically relevant Phe cutoff values used in confirmatory testing.

METHODS

The protocol for this literature review was registered with PROSPERO (International prospective register of systematic reviews). Pubmed and Embase database searches were used to identify studies that reported the birth prevalence of PAH deficiency. Only studies including numeric birth prevalence reports of confirmed PAH deficiency were included.

RESULTS

From the 85 publications included in the review, 238 birth prevalence estimates were extracted. After excluding prevalence estimates that did not meet quality assessment criteria or because of temporal and regional overlap, estimates from 45 publications were included in the meta-analysis. The global birth prevalence of PAH deficiency, estimated by weighting regional birth prevalences relative to their share of the population of all regions included in the study, was 0.64 (95% confidence interval [CI] 0.53-0.75) per 10,000 births and ranged from 0.03 (95% CI 0.02-0.05) per 10,000 births in Southeast Asia to 1.18 (95% CI 0.64-1.87) per 10,000 births in the Middle East/North Africa. Regionally weighted global birth prevalences per 10,000 births by confirmatory test Phe cutoff values were 0.96 (95% CI 0.50-1.42) for the Phe cutoff value of 360 ± 100 µmol/L; 0.50 (95% CI 0.37-0.64) for the Phe cutoff value of 600 ± 100 µmol/L; and 0.30 (95% CI 0.20-0.40) for the Phe cutoff value of 1200 ± 200 µmol/L.

CONCLUSIONS

Substantial regional variation in the birth prevalence of PAH deficiency was observed in this systematic literature review and meta-analysis of published evidence from newborn screening. The precision of the prevalence estimates is limited by relatively small sample sizes, despite widespread and longstanding newborn screening in much of the world.

A unified framework for bias assessment in clinical research

Methodological flaws, limitations, and inadequate practices in research are well known and pose threats to the internal validity of any research study. However, there are ways of safeguarding research conduct to reduce the chance of research producing distorted results. Numerous tools now exist to assess the incorporation of such safeguards into primary research studies (also known as quality and/or risk-of-bias assessment). These tools typically include a variety of items that are then checked against those implemented in the study. Despite a lot of research in this area, no comprehensive generic classification of safeguards across study designs exist, although attempts have been made to clarify aspects of this. We review the developments in this area as well as use preliminary data from 100 methodological studies to illustrate our proposed approach. We conclude by proposing a new framework for identifying research studies at risk of being biased and the information in this article will promote a unification of the diverse approaches to facilitating bias assessment in clinical research.

Development, content validation, and reliability of the Assessment of Real-World Observational Studies (ArRoWS) critical appraisal tool

PURPOSE

The objective was to develop and test a pragmatic critical appraisal tool, the Assessment of Real-World Observational Studies (ArRoWS), to quickly and easily assess the quality of real-world evidence studies using electronic health records.

METHODS

The initial ArRoWS tool was developed by identifying items frequently found in existing validated assessment instruments and adapting these items to specifically assess real-world evidence studies. The tool was revised based on recommendations from an expert panel of 14 senior academic individuals specializing in epidemiology and content validity was measured. During March 2018-January 2019, 47 large, observational studies related to cardiometabolic medicine were identified through a search algorithm and assessed by three pairs of raters using the ArRoWS tool.

RESULTS

The final version of the ArRoWS had 16 items including nine core items and seven study design-specific items with item-specific content validity indexes ranging from 0.64 to 1.00. The scale-level content validity index of the ArRoWS appraisal tool was 0.91. When the ArRoWS tool was pilot tested, the observed agreement between assessor pairs on whether the study provided high-quality real-world evidence was 85.7%, 68.8%, and 58.8%. The prevalence adjusted bias-adjusted kappa for the assessor pairs was 0.71, 0.38, and 0.18.

CONCLUSION

The ArRoWS is a simple tool to standardize the assessment of real-world evidence studies.

Quality assessment of prevalence studies: a systematic review

OBJECTIVES

The objective of the study is to identify items and domains applicable for the quality assessment of prevalence studies.

STUDY DESIGN AND SETTING

We searched databases and the gray literature to identify tools or guides about the quality assessment of prevalence studies. After study selection, we abstracted questions applicable for prevalence studies and classified into at least one of the following domains: 'population and setting', 'condition measurement', 'statistics', and 'other'. PROSPERO registration:CRD42018088437.

RESULTS

We included 30 tools: eight (26.7%) specifically designed to appraise prevalence studies and 22 (73.3%) adaptable for this purpose. We identified 12 unique items in the domain "population and setting", 16 in the domain "condition measurement", and 14 in the domain "statistics". Of those, 25 (59.5%) were identified in the eight specific tools. Regarding the domain "other", we identified 77 unique items, mainly related to manuscript writing and reporting (n = 48, 62.3%); of those, 24 (31.2%) were identified in the eight specific tools and 53 (68.8%) in the additional 22 nonspecific tools.

CONCLUSION

We provide a comprehensive set of items classified by domains that can guide the appraisal of prevalence studies, conduction of primary prevalence studies, and update or development of tools to evaluate prevalence studies.

PRECEPT: an evidence assessment framework for infectious disease epidemiology, prevention and control

Decisions in public health should be based on the best available evidence, reviewed and appraised using a rigorous and transparent methodology. The Project on a Framework for Rating Evidence in Public Health (PRECEPT) defined a methodology for evaluating and grading evidence in infectious disease epidemiology, prevention and control that takes different domains and question types into consideration. The methodology rates evidence in four domains: disease burden, risk factors, diagnostics and intervention. The framework guiding it has four steps going from overarching questions to an evidence statement. In step 1, approaches for identifying relevant key areas and developing specific questions to guide systematic evidence searches are described. In step 2, methodological guidance for conducting systematic reviews is provided; 15 study quality appraisal tools are proposed and an algorithm is given for matching a given study design with a tool. In step 3, a standardised evidence-grading scheme using the Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE) methodology is provided, whereby findings are documented in evidence profiles. Step 4 consists of preparing a narrative evidence summary. Users of this framework should be able to evaluate and grade scientific evidence from the four domains in a transparent and reproducible way.

Perspective: Essential Study Quality Descriptors for Data from Nutritional Epidemiologic Research

Pooled analysis of secondary data increases the power of research and enables scientific discovery in nutritional epidemiology. Information on study characteristics that determine data quality is needed to enable correct reuse and interpretation of data. This study aims to define essential quality characteristics for data from observational studies in nutrition. First, a literature review was performed to get an insight on existing instruments that assess the quality of cohort, case-control, and cross-sectional studies and dietary measurement. Second, 2 face-to-face workshops were organized to determine the study characteristics that affect data quality. Third, consensus on the data descriptors and controlled vocabulary was obtained. From 4884 papers retrieved, 26 relevant instruments, containing 164 characteristics for study design and 93 characteristics for measurements, were selected. The workshop and consensus process resulted in 10 descriptors allocated to "study design" and 22 to "measurement" domains. Data descriptors were organized as an ordinal scale of items to facilitate the identification, storage, and querying of nutrition data. Further integration of an Ontology for Nutrition Studies will facilitate interoperability of data repositories.

Human metapneumovirus in patients hospitalized with acute respiratory infections: A meta-analysis

This meta-analysis aimed to estimate the prevalence of human metapneumovirus (hMPV) infections in patients hospitalized for acute respiratory infection (ARI) and to study factors associated with this prevalence. Medline and ScienceDirect databases were searched for prospective observational studies that screened hospitalized patients with ARI for hMPV by RT-PCR, with data available at December 27, 2014. The risk of bias was assessed regarding participation rate, definition of ARI, description of diagnostic technique, method of inclusion identical for all subjects, standardized and identical sampling method for all subjects, analysis performed according to the relevant subgroups, and presentation of data sources. Random-effect meta-analysis with arcsine transformation and meta-regressions was used. In the 75 articles included, the prevalence of hMPV among hospitalized ARI was 6.24% (95% CI 5.25-7.30). An effect of the duration of the inclusion period was observed (p=0.0114), with a higher prevalence of hMPV in studies conducted during periods of 7-11 months (10.56%, 95% CI 5.97-16.27) or complete years (7.55%, 95% CI 5.90-9.38) than in periods of 6 months or less (5.36%, 95% CI 4.29-6.54). A significant increase in the incidence with increasing distance from the equator was observed (p=0.0384). hMPV should be taken into account as a possible etiology in hospitalized ARI.

Epidemiology of irritable bowel syndrome in children and adolescents in Asia

OBJECTIVES

Several cross-sectional surveys have been conducted to study the prevalence of irritable bowel syndrome (IBS) in children. The aim of the present study was to conduct a systematic review and a meta-analysis of published literature to estimate the prevalence of IBS among Asian children.

METHOD

A computer-assisted search of MEDLINE, EMBASE, PsycINFO, and regional databases of Asia was carried out. Selected articles were reviewed in depth and data were extracted. Pooled prevalence, sex differences, and 95% confidence intervals were calculated. Heterogeneity of the studies was assessed using the I(2) test.

RESULTS

A total of 16 cross-sectional studies that reported prevalence of IBS (in children and adolescents) and qualified to be included were taken into the final analysis containing 38,076 patients. Selected studies are from China, Korea, Japan, Iran, Sri Lanka, and Saudi Arabia. Studies showed a marked heterogeneity with I(2) of 98.59 (P < 0.0001). Prevalence of IBS ranges from 2.8% to 25.7%, with a pooled prevalence of 12.41% (95% confidence interval 9.87-14.95). Prevalence risk ratio of girl:boy is 1.39. Prevalence of subtypes is diverse and varies between studies.

CONCLUSIONS

The published data indicate that IBS is a significant problem among Asian children and adolescents. Female sex predisposes children and adolescents to develop IBS.

Evidence-based decision-making in infectious diseases epidemiology, prevention and control: matching research questions to study designs and quality appraisal tools

BACKGROUND

The Project on a Framework for Rating Evidence in Public Health (PRECEPT) was initiated and is being funded by the European Centre for Disease Prevention and Control (ECDC) to define a methodology for evaluating and grading evidence and strength of recommendations in the field of public health, with emphasis on infectious disease epidemiology, prevention and control. One of the first steps was to review existing quality appraisal tools (QATs) for individual research studies of various designs relevant to this area, using a question-based approach.

METHODS

Through team discussions and expert consultations, we identified 20 relevant types of public health questions, which were grouped into six domains, i.e. characteristics of the pathogen, burden of disease, diagnosis, risk factors, intervention, and implementation of intervention. Previously published systematic reviews were used and supplemented by expert consultation to identify suitable QATs. Finally, a matrix was constructed for matching questions to study designs suitable to address them and respective QATs. Key features of each of the included QATs were then analyzed, in particular in respect to its intended use, types of questions and answers, presence/absence of a quality score, and if a validation was performed.

RESULTS

In total we identified 21 QATs and 26 study designs, and matched them. Four QATs were suitable for experimental quantitative study designs, eleven for observational quantitative studies, two for qualitative studies, three for economic studies, one for diagnostic test accuracy studies, and one for animal studies. Included QATs consisted of six to 28 items. Six of the QATs had a summary quality score. Fourteen QATs had undergone at least one validation procedure.

CONCLUSIONS

The results of this methodological study can be used as an inventory of potentially relevant questions, appropriate study designs and QATs for researchers and authorities engaged with evidence-based decision-making in infectious disease epidemiology, prevention and control.

Familial Risks for Diseases of Myoneural Junction and Muscle in Siblings Based on Hospitalizations and Deaths in Sweden

Diseases of the myoneural junction and muscle are disabling and some are life-threatening. Recent successes in the identification of the underlying genetic mechanisms have had profound implication for their diagnostics, treatment and classification. We define familial risks for siblings who were hospitalized for or deceased from diseases of the myoneural junction and muscle. A nationwide database on diseases of the myoneural junction and muscle was constructed by linking the Multigeneration Register on 0- to 69-year-old siblings to the Hospital Discharge Register and the Causes of Death Register from years 1987 to 2001. Standardized risk ratios (SIRs) were calculated for affected sibling pairs by comparing to those whose siblings had no diseases of myoneural junction and muscle. Among a total of 2307 patients, myasthenia gravis, muscular dystrophy and myotonic disorders were commonest diagnoses. The sibling risks for these disease were 22, 190 and 198, respectively, when a sibling was diagnosed with any disease of the myoneural junction and muscle. The concordant SIRs, both siblings presenting the same disease, were 42 for myasthenia gravis, 737 for muscular dystrophy, 2000 for congenital myopathy, 1211 for myotonic disorder, 909 for periodic paralysis and 209 for unspecified myopathy. Only a few discordant sibling pairs were noted. The very high overall SIRs for the diseases of the myoneural junction and muscle imply that the sporadic forms of these diseases are relatively rare and these diseases are overwhelmingly heritable.

Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists

OBJECTIVE

To develop two checklists for the quality of observational studies of incidence or risk factors of diseases.

STUDY DESIGN AND SETTING

Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles.

RESULTS

The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers.

CONCLUSION

We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

A systematic review of the incidence of schizophrenia: the distribution of rates and the influence of sex, urbanicity, migrant status and methodology

BACKGROUND

Understanding variations in the incidence of schizophrenia is a crucial step in unravelling the aetiology of this group of disorders. The aims of this review are to systematically identify studies related to the incidence of schizophrenia, to describe the key features of these studies, and to explore the distribution of rates derived from these studies.

METHODS

Studies with original data related to the incidence of schizophrenia (published 1965-2001) were identified via searching electronic databases, reviewing citations and writing to authors. These studies were divided into core studies, migrant studies, cohort studies and studies based on Other Special Groups. Between- and within-study filters were applied in order to identify discrete rates. Cumulative plots of these rates were made and these distributions were compared when the underlying rates were sorted according to sex, urbanicity, migrant status and various methodological features.

RESULTS

We identified 100 core studies, 24 migrant studies, 23 cohort studies and 14 studies based on Other Special Groups. These studies, which were drawn from 33 countries, generated a total of 1,458 rates. Based on discrete core data for persons (55 studies and 170 rates), the distribution of rates was asymmetric and had a median value (10%-90% quantile) of 15.2 (7.7-43.0) per 100,000. The distribution of rates was significantly higher in males compared to females; the male/female rate ratio median (10%-90% quantile) was 1.40 (0.9-2.4). Those studies conducted in urban versus mixed urban-rural catchment areas generated significantly higher rate distributions. The distribution of rates in migrants was significantly higher compared to native-born; the migrant/native-born rate ratio median (10%-90% quantile) was 4.6 (1.0-12.8). Apart from the finding that older studies reported higher rates, other study features were not associated with significantly different rate distributions (e.g. overall quality, methods related to case finding, diagnostic confirmation and criteria, the use of age-standardization and age range).

CONCLUSIONS

There is a wealth of data available on the incidence of schizophrenia. The width and skew of the rate distribution, and the significant impact of sex, urbanicity and migrant status on these distributions, indicate substantial variations in the incidence of schizophrenia.