The Validity, Reliability, and Feasibility of Measurement Tools Used to Assess Sleep of Pre-school Aged Children: A Systematic Rapid Review

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews

PURPOSE

Systematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards.

METHODS

One hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings.

RESULTS

A quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use.

CONCLUSION

Despite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study

AIM

The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.

METHODS

Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.

RESULTS

The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.

CONCLUSIONS

The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.

Comparison of actigraphy-measured and parent-reported sleep in association with weight status among preschool children

This study compared weekday and weekend actigraphy-measured and parent-reported sleep in relation to weight status among preschool-aged children. Participants were 3-6 years old preschoolers from the cross-sectional DAGIS-study with sleep data for ≥2 weekday and ≥2 weekend nights. Parents-reported sleep onset and wake-up times were gathered alongside 24 h hip-worn actigraphy. An unsupervised Hidden-Markov Model algorithm provided actigraphy-measured night time sleep without the guidance of reported sleep times. Waist-to-height ratio and age-and-sex-specific body mass index characterised weight status. Comparison of methods were assessed with consistency in quintile divisions and Spearman correlations. Associations between sleep and weight status were assessed with adjusted regression models. Participants included 638 children (49% girls) with a mean ± SD age of 4.76 ± 0.89. On weekdays, 98%-99% of actigraphy-measured and parent-reported sleep estimates were classified in the same or adjacent quintile and were strongly correlated (rs  = 0.79-0.85, p < 0.001). On weekends, 84%-98% of actigraphy-measured and parent-reported sleep estimates were respectively classified and correlations were moderate to strong (rs  = 0.62-0.86, p < 0.001). Compared with actigraphy-measured sleep, parent-reported sleep had consistently earlier onset, later wake-up, and greater duration. Earlier actigraphy-measured weekday sleep onset and midpoint were associated with a higher body mass index (respective β-estimates: -0.63, p < 0.01 and -0.75, p < 0.01) and waist-to-height ratio (-0.004, p = 0.03 and -0.01, p = 0.02). Though the sleep estimation methods were consistent and correlated, actigraphy measures should be favoured as they are more objective and sensitive to identifying associations between sleep timing and weight status compared with parent reports.

Co-design and content validity of the movement measurement in the early years (MoveMEY) tool for assessing movement behaviour of pre-school aged children

BACKGROUND

Movement behaviours (physical activity, sedentary behaviour, and sleep) are important for pre-school children's health and development. Currently, no tools with appropriate content validity exist that concurrently capture these movement behaviours in young children. The aim of this study was to co-design and assess the content validity of a novel tool to concurrently measure movement behaviours in pre-school aged children (aged 3-4 years).

METHODS

We followed four distinct steps to develop and assess the content validity of Movement Measurement in the Early Years (MoveMEY): (1) We conducted an extensive literature search, to identify pre-existing proxy measurement tools (questionnaires and diaries) to inform the design of a novel tool, which aimed to effectively capture movement behaviour guidelines of pre-school aged children. (2) We facilitated focus group discussions with parents and carers of pre-school aged children (n = 11) and (3) a qualitative survey with free text responses was completed by topic relevant researchers (n = 6), to co-design the measurement tool. (4) We assessed the content validity of the developed tool, MoveMEY, through interviews with parents of pre-school aged children (n = 12) following piloting of the tool.

RESULTS

We developed an initial version of MoveMEY based on the format of an existing questionnaire and by mapping the content of questions to the guidelines. Co-design of MoveMEY resulted in changes to the format (e.g. short questionnaire to a seven-day diary) and content (e.g. inclusion of 'general information' questions on illness, disabilities and sleep disturbances; question on screen time before bed). Content validity assessment demonstrated that the items of MoveMEY were relevant and comprehensive for the assessment of children's movement behaviours. MoveMEY was felt to be comprehensible, however, parental suggestions were implemented to finalise and improve MoveMEY (e.g. adding examples to questions aiming to detect moderate to vigorous physical activity).

CONCLUSION

MoveMEY is the first co-designed measurement tool that has relevance for assessing the movement behaviour guidelines of pre-school aged children. Parent/carer and topic relevant researcher involvement throughout the development process resulted in a seven-day daily reported activity diary that is comprehensive of children's movement behaviours and comprehensible to parents and carers.

Parental Views on the Acceptability and Feasibility of Measurement Tools Used to Assess Movement Behaviour of Pre-School Children: A Qualitative Study

Movement behaviours (physical activity, sedentary behaviour, and sleep) are important for the health and development of pre-school children (aged 3-4 years). There is limited qualitative research examining the acceptability and feasibility of tools used to assess movement behaviours in pre-schoolers. This study explored parental views on various measurement tools in three deprived areas in England, UK (West Yorkshire, County Durham and Northumberland). The study consisted of a demonstration of the different tools (accelerometers, a diary and a questionnaire), directly followed by focus group discussions. Three focus group discussions with a total of eleven parents and carers were transcribed verbatim and analysed using thematic analysis. Findings revealed four main themes: (1) importance of contextual information when using any measurement tool (e.g., child illness, capturing different routines); (2) practical issues associated with devices (e.g., aversion to devices being attached directly to the skin of their child; concern of larger devices during sleep time); (3) encouraging children to wear a device (e.g., making devices attractive to children-'superpowers'); and (4) presentation of diaries and questionnaires (e.g., age-appropriate movement activities, preference for real-time recording over recall). Practical recommendations for the use of the tools to measure movement behaviours of pre-school children are provided.