Best Practices and Methodological Strategies for Addressing Generalizability in Neuropsychological Assessment

Generalizability considerations are widely discussed and a core foundation for understanding when and why treatment effects will replicate across sample demographics. However, guidelines on assessing and reporting generalizability-related factors differ across fields and are inconsistently applied. This paper synthesizes obstacles and best practices to apply recent work on measurement and sample diversity. We present a brief history of how knowledge in psychology has been constructed, with implications for who has been historically prioritized in research. We then review how generalizability remains a contemporary threat to neuropsychological assessment and outline best practices for researchers and clinical neuropsychologists. In doing so, we provide concrete tools to evaluate whether a given assessment is generalizable across populations and assist researchers in effectively testing and reporting treatment differences across sample demographics.

Reporting in Experimental Philosophy: Current Standards and Recommendations for Future Practice

Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods (n = 134) that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact (numbers of citations). We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible.

Meta-Analysis of Adherence Promotion Interventions in Pediatric Asthma

BACKGROUND

Youth with asthma commonly have suboptimal adherence to inhaled corticosteroids (ICS). It is critical to systematically evaluate the effectiveness of ICS adherence promotion interventions and discern which techniques are most effective.

OBJECTIVE

This study aims to (1) quantify the extent to which interventions improve ICS adherence in pediatric asthma, (2) explore differences in effect size estimates based on intervention and study characteristics, and (3) characterize the risk of bias across interventions.

METHODS

We conducted literature searches across five databases. Included studies quantitatively measured ICS adherence as an intervention outcome among youth (<18 years old) diagnosed with asthma and were published after 1997. We analyzed aggregate effect sizes and moderator variables using random-effects models and characterized risk of bias using the Cochrane Collaboration tool.

RESULTS

Thirty-three unique studies met inclusion criteria. At post-intervention, the aggregate effect size for pediatric ICS adherence promotion interventions was small but significant (n = 33, g = 0.39, 95% confidence interval [CI] = 0.24-0.54); however, the aggregate effect size at follow-up was not statistically significant (n = 6, g = 0.38, 95% CI = -0.08 to 0.83). Method of adherence measurement and intervention format were significant moderators. Most interventions had a high risk of performance bias and an unclear risk of bias in one or more domains.

CONCLUSIONS

ICS adherence promotion interventions are effective among youth with asthma. Additional longitudinal research is needed to quantify a more precise measure of intervention effectiveness over time, and moderators of intervention effectiveness should be reassessed as the literature base expands.

Reporting characteristics of cancer pain: a systematic review and quantitative analysis of research publications in palliative care journals

Objective:

A common disorder requiring symptom palliation in palliative and end-of-life care is cancer. Cancer pain is recognized as a global health burden. This paper sought to systematically examine the extent to which there is an adequate scientific research base on cancer pain and its reporting characteristics in the palliative care journal literature.

Materials and Methods:

Search conducted in MEDLINE and CINAHL sought to locate all studies published in 19 palliative/ hospice/ supportive/ end-of-life care journals from 2009 to 2010. The journals included were: American Journal of Hospice and Palliative Care, BMC Palliative Care, Current Opinion in Supportive and Palliative Care, End of Life Care Journal, European Journal of Palliative Care, Hospice Management Advisor, Indian Journal of Palliative Care, International Journal of Palliative Nursing, Internet Journal of Pain Symptom Control and Palliative Care, Journal of Pain and Palliative Care Pharmacotherapy, Journal of Palliative Care, Journal of Palliative Medicine, Journal of Social Work in End-of-life and Palliative Care, Journal of Supportive Oncology, Palliative Medicine, Palliative and Supportive Care, and Supportive Care in Cancer. Journal contents were searched to identify studies that included cancer pain in abstract.

Results:

During the years 2009 and 2010, of the selected 1,569 articles published in the journals reviewed, only 5.86% (92 articles) were on cancer pain.

Conclusion:

While researchers in the field of palliative care have studied cancer pain, the total percentage for studies is still a low 5.86%. To move the field of palliative care forward so that appropriate guidelines for cancer pain management can be developed, it is critical that more research be reported upon which to base cancer pain therapy in an evidence-based palliative care model.

Assessing the quality of randomized controlled trials examining psychological interventions for pediatric procedural pain: recommendations for quality improvement

OBJECTIVE

Systematic reviews of randomized controlled trials (RCTs) support the efficacy of psychological interventions for procedural pain management. However, methodological limitations (e.g., inadequate randomization) have affected the quality of this research, thereby weakening RCT findings.

METHODS

Detailed quality coding was conducted on 28 RCTs included in a systematic review of psychological interventions for pediatric procedural pain.

RESULTS

The majority of RCTs were of poor to low quality (criteria reported in <50% of RCTs). Commonly reported criteria addressed study background, conditions, statistical analyses, and interpretation of results. Commonly nonreported criteria included treatment administration, evaluation of treatment efficacy (effect sizes, summary statistics, intention-to-treat analyses), caregiver demographics, follow-up, and participant flow. Quality was greater in more recent trials, and did not vary by journal type (psychology vs. medical).

CONCLUSION

Despite poor quality ratings, quality reporting in psychological RCTs for pediatric procedural pain has improved over time. Recommendations for quality enhancement are provided.