1
|
Verdonschot A, Beauchamp MR, Brusseau TA, Chinapaw MJM, Christiansen LB, Daly-Smith A, Eather N, Fairclough SJ, Faulkner G, Foweather L, García-Hermoso A, Ha AS, Harris N, Jaakkola T, Jago R, Kennedy SG, Lander NJ, Lonsdale C, Manios Y, Mazzoli E, Murtagh E, Nathan N, Naylor PJ, Noetel M, O'Keeffe B, Resaland GK, Ridgers ND, Ridley K, Riley N, Rosenkranz RR, Rosenkranz SK, Sääkslahti A, Sczygiol SM, Skovgaard T, van Sluijs EMF, Smith JJ, Smith M, Stratton G, Vidal-Conti J, Webster CA, Young ES, Lubans DR. Development and evaluation of the Capability, Opportunity, and Motivation to deliver Physical Activity in School Scale (COM-PASS). Int J Behav Nutr Phys Act 2024; 21:93. [PMID: 39187858 PMCID: PMC11346190 DOI: 10.1186/s12966-024-01640-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Accepted: 08/01/2024] [Indexed: 08/28/2024] Open
Abstract
BACKGROUND Teachers are recognized as 'key agents' for the delivery of physical activity programs and policies in schools. The aim of our study was to develop and evaluate a tool to assess teachers' capability, opportunity, and motivation to deliver school-based physical activity interventions. METHODS The development and evaluation of the Capability, Opportunity, and Motivation to deliver Physical Activity in School Scale (COM-PASS) involved three phases. In Phase 1, we invited academic experts to participate in a Delphi study to rate, provide recommendations, and achieve consensus on questionnaire items that were based on the Capability, Opportunity, and Motivation Behavior (COM-B) model. Each item was ranked on the degree to which it matched the content of the COM-B model, using a 5-point scale ranging from '1 = Poor match' to '5 = Excellent match'. In Phase 2, we interviewed primary and secondary school teachers using a 'think-aloud' approach to assess their understanding of the items. In Phase 3, teachers (n = 196) completed the COM-PASS to assess structural validity using confirmatory factor analysis (CFA). RESULTS Thirty-eight academic experts from 14 countries completed three rounds of the Delphi study. In the first round, items had an average rating score of 4.04, in the second round 4.51, and in the third (final) round 4.78. The final tool included 14 items, which related to the six constructs of the COM-B model: physical capability, psychological capability, physical opportunity, social opportunity, reflective motivation, and automatic motivation. In Phase 2, ten teachers shared their interpretation of COM-PASS via a 20-min interview, which resulted in minor changes. In Phase 3, CFA of the 3-factor model (i.e., capability, opportunity, and motivation) revealed an adequate fit to the data (χ2 = 122.6, p < .001, CFI = .945, TLI = .924, RMSEA = .066). The internal consistencies of the three subscale scores were acceptable (i.e., capability: α = .75, opportunity: α = .75, motivation: α = .81). CONCLUSION COM-PASS is a valid and reliable tool for assessing teachers' capability, opportunity, and motivation to deliver physical activity interventions in schools. Further studies examining additional psychometric properties of the COM-PASS are warranted.
Collapse
Affiliation(s)
- A Verdonschot
- Centre for Active Living and Learning, School of Education, University of Newcastle, Newcastle, Australia
- Active Living Research Program, Hunter Medical Research Institute, New Lambton Heights, New South Wales, Australia
| | - M R Beauchamp
- School of Kinesiology, University of British Columbia, Vancouver, British Columbia, Canada
| | - T A Brusseau
- Department of Health and Kinesiology, University of Utah, Salt Lake City, USA
| | - M J M Chinapaw
- Department of Public and Occupational Health, Amsterdam UMC, Location Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - L B Christiansen
- Active Living, Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense M, Denmark
| | - A Daly-Smith
- Faculty of Health Studies, University of Bradford, Bradford, UK
- Centre for Applied Education Research, Wolfson Centre for Applied Health Research, Bradford, UK
| | - N Eather
- Centre for Active Living and Learning, School of Education, University of Newcastle, Newcastle, Australia
- Active Living Research Program, Hunter Medical Research Institute, New Lambton Heights, New South Wales, Australia
| | - S J Fairclough
- Sport, Physical Activity, Health, & Wellbeing Research Group, and Department of Sport & Physical Activity, Edge Hill University, Ormskirk, UK
| | - G Faulkner
- School of Kinesiology, University of British Columbia, Vancouver, British Columbia, Canada
| | - L Foweather
- Physical Activity Exchange, Research Institute of Sport and Exercise Sciences, Liverpool John Moores University, Liverpool, UK
| | - A García-Hermoso
- Navarrabiomed, Hospital Universitario de Navarra, Universidad Pública de Navarra, IdiSNA, Pamplona, Navarra, Spain
| | - A S Ha
- Department of Sports Science and Physical Education, Faculty of Education, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - N Harris
- Human Potential Centre, Auckland University of Technology, Auckland, New Zealand
| | - T Jaakkola
- Faculty of Sport and Health Sciences, University of Jyväskylä, Jyväskylä, Finland
| | - R Jago
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - S G Kennedy
- School of Health Sciences, Western Sydney University, Kingswood, New South Wales, Australia
| | - N J Lander
- Institute for Physical Activity and Nutrition (IPAN), School of Exercise and Nutrition Sciences, Faculty of Health, Deakin University, Geelong, Victoria, Australia
| | - C Lonsdale
- Institute for Positive Psychology and Education, Australian Catholic University, North Sydney, New South Wales, Australia
| | - Y Manios
- Department of Nutrition and Dietetics, School of Health Science and Education, Harokopio University, Athens, Greece
- Institute of Agri-Food and Life Sciences, University Research & Innovation Center, H.M.U.R.I.C., Hellenic Mediterranean University, Crete, Greece
| | - E Mazzoli
- Institute for Physical Activity and Nutrition (IPAN), School of Exercise and Nutrition Sciences, Faculty of Health, Deakin University, Geelong, Victoria, Australia
- School of Health and Social Development, Faculty of Health, Deakin University, Geelong, Victoria, Australia
| | - E Murtagh
- Department of Physical Education and Sport Sciences, University of Limerick, Limerick, Ireland
| | - N Nathan
- Hunter New England Population Health, Hunter New England Local Health District, Wallsend, New South Wales, Australia
- College of Health, Medicine and Wellbeing, School of Medicine and Public Health, The University of Newcastle, Newcastle, New South Wales, Australia
- National Centre of Implementation Science (NCOIS), The University of Newcastle, Newcastle, New South Wales, Australia
- Hunter Medical Research Institute, New Lambton Heights, New South Wales, Australia
| | - P J Naylor
- School of Exercise Science, Physical and Health Education, University of Victoria, Victoria, British Columbia, Canada
| | - M Noetel
- School of Psychology, The University of Queensland, Brisbane, Queensland, Australia
| | - B O'Keeffe
- Department of Physical Education and Sport Sciences, University of Limerick, Limerick, Ireland
| | - G K Resaland
- Centre for Physically Active Learning, Faculty of Education, Arts and Sports, Western Norway University of Applied Sciences, Sogndal, Norway
| | - N D Ridgers
- Alliance for Exercise, Nutrition and Activity (ARENA), Allied Health and Human Performance, University of South Australia, Adelaide, South Australia, Australia
| | - K Ridley
- College of Education, Psychology and Social Work, Flinders University, Adelaide, South Australia, Australia
| | - N Riley
- Centre for Active Living and Learning, School of Education, University of Newcastle, Newcastle, Australia
- Active Living Research Program, Hunter Medical Research Institute, New Lambton Heights, New South Wales, Australia
| | - R R Rosenkranz
- Department of Kinesiology and Nutrition Sciences, University of Nevada, Las Vegas, Las Vegas, Nevada, USA
| | - S K Rosenkranz
- Department of Kinesiology and Nutrition Sciences, University of Nevada, Las Vegas, Las Vegas, Nevada, USA
| | - A Sääkslahti
- Faculty of Sport and Health Sciences, University of Jyväskylä, Jyväskylä, Finland
| | - S M Sczygiol
- Department of Neuromotor Behaviour and Exercise, University of Münster, Münster, Germany
| | - T Skovgaard
- Active Living, Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense M, Denmark
| | | | - J J Smith
- Centre for Active Living and Learning, School of Education, University of Newcastle, Newcastle, Australia
- Active Living Research Program, Hunter Medical Research Institute, New Lambton Heights, New South Wales, Australia
| | - M Smith
- School of Nursing, University of Auckland, Auckland, New Zealand
| | - G Stratton
- Applied Sport Technology Exercise and Medicine Research Centre, Faculty Science and Engineering, Swansea University, Wales, UK
- Sport and Exercise Sciences, University of Western Australia, Perth, Western Australia, Australia
| | - J Vidal-Conti
- Physical Activity and Sport Sciences Research Group (GICAFE), University of the Balearic Islands, Palma, Spain
| | - C A Webster
- Department of Kinesiology, Texas A and M University - Corpus Christi, Corpus Christi, Texas, USA
| | - E S Young
- Faculty of Health Studies, University of Bradford, Bradford, UK
- Centre for Applied Education Research, Wolfson Centre for Applied Health Research, Bradford, UK
| | - D R Lubans
- Centre for Active Living and Learning, School of Education, University of Newcastle, Newcastle, Australia.
- Active Living Research Program, Hunter Medical Research Institute, New Lambton Heights, New South Wales, Australia.
- Faculty of Sport and Health Sciences, University of Jyväskylä, Jyväskylä, Finland.
| |
Collapse
|
2
|
Sajatovic M, Beem Jelley P, Schrag K, Varanese P, Needham K, Black J, Adeniyi C, Barigye R, Yala J, Ghearing GR, Glasgow RE, Briggs F. A case study in science-to-service application of epilepsy self-management. Epilepsy Behav 2024; 159:109945. [PMID: 39121751 DOI: 10.1016/j.yebeh.2024.109945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/10/2024] [Revised: 06/12/2024] [Accepted: 07/07/2024] [Indexed: 08/12/2024]
Abstract
AIMS Epilepsy self-management (ESM), the overall approach of reducing seizures and optimizing whole-health, is a targeted approach to improve population health for people with epilepsy (PWE). "Self-management for people with epilepsy and a history of negative health events" (SMART) is an 8-session group-format, remotely delivered ESM. This report describes the evolution of SMART development, testing and scale-up, taking advantage of ESM team expertise, community relationships and infrastructure established by social service agencies that deliver support to PWE. METHODS This is a case-study dissemination and implementation (D&I) science-to-service model using the RE-AIM framework approach (Reach, Effectiveness, Adoption, Implementation, and Maintenance) focused on 5 dimensions of individual- and setting-level outcomes important to program adoption, impact and sustainability. Performance evaluations include participation representativeness, ESM attendance and acceptability as well as change in relevant health outcomes. RESULTS SMART D & I is implemented via a collaboration of 3 unique regional, epilepsy-focused nonprofit social service organizations and a university team that developed SMART. The ongoing collaboration is expanding SMART delivery to PWE across 13 U.S. states. Thus far, we have trained 17 Nurse and Peer Educators (NEs and PEs). PEs (N = 10) have a mean age 51.1 (SD 10.4) years and a mean age of epilepsy diagnosis of 29.4 (SD 19.3). Of 128 participants offered SMART, and who provided age data (N = 86) mean age was 37.7 years (SD 14.4). Of participants who provided data on gender and race (N = 89), 65 were women (73.9 %), 18 African-American (20.2 %). Mean age of epilepsy diagnosis was 19.4 years (SD 16.6) and 59 (52.2 %) of PWE reported having seizures in the last 30 days pre-SMART sessions. Among those with attendance data (N = 103), mean number of SMART groups attended was 5.7 (SD 2.3). Mean values for past 30-day seizure frequency, 9-item Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder Inventory (GAD-7) and 10-item Quality of Life in Epilepsy Scale (QOLIE-10) for PWE that provided both pre and post SMART data were 7.6 (SD 15.8) vs 2.8 (SD 3.4) p = 0.3, 7.63 (SD 6.6) vs 6.3 (SD 5.7) p = 0.95, 6.6 (SD 5.7) vs 6.67(SD 5.3) p = 0.47 and 2.8 (SD 0.8) vs 2.7 (SD 1.0) p = 0.07 respectively. CONCLUSIONS Implementing ESM using a RE-AIM/Iterative RE-AIM framework links intervention developers and community partners. While PWE have substantial barriers to health, including frequent seizures, they are able engage in the SMART program. Although a major limitation to patient-level evaluation is challenges in collecting post-SMART follow-up data, preliminary findings suggest a trend for improved quality of life.
Collapse
Affiliation(s)
- Martha Sajatovic
- Department of Psychiatry and of Neurology, Case Western Reserve University School of Medicine, Cleveland, OH, USA.
| | - Peggy Beem Jelley
- Epilepsy Association of Western and Central Pennsylvania, Pittsburgh, PA, USA
| | | | | | | | - Jessica Black
- Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Clara Adeniyi
- Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Richard Barigye
- Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Joy Yala
- Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | | | | | - Farren Briggs
- Department of Public Health Sciences, Division of Epidemiology, Miller School of Medicine, University of Miami, Miami, FLA, USA
| |
Collapse
|
3
|
Ravalihasy A. L'évaluateur et l'évaluation d'impact d'une intervention comme levier potentiel de l' empowerment au niveau communautaire : un exercice réflexif à partir de l'intervention MAKASI. Glob Health Promot 2024:17579759241258193. [PMID: 39087362 DOI: 10.1177/17579759241258193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/02/2024]
Affiliation(s)
- Andrainolo Ravalihasy
- Ceped (IRD-Université Paris Cité-Inserm ERL 1244), Paris, France
- DeSCID, IAME UMR1137 (Université Paris Cité-Inserm), Paris, France
- Institut Convergences Migrations, Aubervilliers, France
| |
Collapse
|
4
|
Shoesmith A, Nathan N, Lum M, Yoong S, Nolan E, Wolfenden L, Shelton RC, Cooper B, Lane C, Grady A, Imad N, Riley-Gibson E, McCarthy N, Pearson N, Hall A. Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C): development and psychometric evaluation of a measure of sustainability determinants in the early childhood education and care setting. Implement Sci 2024; 19:41. [PMID: 38902763 PMCID: PMC11188265 DOI: 10.1186/s13012-024-01372-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 06/13/2024] [Indexed: 06/22/2024] Open
Abstract
BACKGROUND There is a need for valid and reliable measures of determinants of sustainability of public health interventions in early childhood education and care (ECEC) settings. This study aimed to develop and evaluate the psychometric and pragmatic properties of such a measure - the Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C). METHODS We undertook a two-phase process guided by the COnsensus-based Standards for the selection of health status Measurement INstruments checklist (COSMIN) and Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Phase 1 involved measure development; i.e., determining items and scales through an iterative process and assessment of face and content validity. Phase 2 involved the evaluation of psychometric and pragmatic properties. The 29-item measure completed by service executives (directors and nominated supervisors) was embedded in a larger survey from a national sample of Australian ECEC services assessing their implementation of nutrition and physical activity programs. Structural validity, concurrent validity, known groups validity, internal consistency, floor and ceiling effects, norms, and pragmatic qualities of the measure were assessed according to the PAPERS criteria. RESULTS The final measure contained 26 items, with respondents reporting how strongly they agreed or disagreed on a five-point Likert scale. Phase 1 assessments confirmed the relevance, and face and content validity of the scale. In Phase 2, we obtained 482 completed surveys, of which 84% (n = 405) completed the entire measure across 405 ECEC settings (one executive per service). Three of the four fit indices for the confirmatory factor analysis met the pre-specified criteria (SRMR = 0.056, CFI = 0.993, RMSEA = 0.067) indicating 'good' structural validity. The IMPRESS-C illustrated: 'good' internal consistency, with Cronbach's alpha values from 0.53 to 0.92; 'emerging' concurrent validity; 'poor' known groups validity; 'good' norms; and 'good' overall pragmatic qualities (cost, readability, length, and assessor burden). CONCLUSIONS The IMPRESS-C possesses strong psychometric and pragmatic qualities for assessing service executive-level perceptions of determinants influencing sustainment of public health interventions within ECEC settings. To achieve a full range of perspectives in this setting, future work should be directed to also develop and test measures of sustainability determinants at the implementer level (e.g., among individual educators and staff).
Collapse
Affiliation(s)
- Adam Shoesmith
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia.
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia.
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia.
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia.
| | - Nicole Nathan
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Melanie Lum
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- Faculty of Health, School of Health and Social Development, Global Centre for Preventive Health and Nutrition, Institute for Health Transformation, Deakin University, Geelong, VIC, 3220, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Serene Yoong
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- Faculty of Health, School of Health and Social Development, Global Centre for Preventive Health and Nutrition, Institute for Health Transformation, Deakin University, Geelong, VIC, 3220, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Erin Nolan
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Luke Wolfenden
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Rachel C Shelton
- Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University, New York, NY, 10032, USA
| | - Brittany Cooper
- Department of Human Development, Washington State University, Pullman, WA, 99164, USA
| | - Cassandra Lane
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Alice Grady
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Noor Imad
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Faculty of Health, School of Health and Social Development, Global Centre for Preventive Health and Nutrition, Institute for Health Transformation, Deakin University, Geelong, VIC, 3220, Australia
- School of Health Sciences, Department of Nursing and Allied Health, Swinburne University of Technology, Hawthorn, VIC, 3122, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Edward Riley-Gibson
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Nicole McCarthy
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Nicole Pearson
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Alix Hall
- School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, 2308, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, 2287, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, 2305, Australia
- National Centre of Implementation Science, University of Newcastle, Callaghan, NSW, 2308, Australia
| |
Collapse
|
5
|
Meffert SM, Mathai MA, Ongeri L, Neylan TC, Mwai D, Onyango D, Akena D, Rota G, Otieno A, Obura RR, Wangia J, Opiyo E, Muchembre P, Oluoch D, Wambura R, Mbwayo A, Kahn JG, Cohen CR, Bukusi DE, Aarons GA, Burger RL, Jin C, McCulloch CE, Njuguna Kahonge S. Defining a screening tool for post-traumatic stress disorder in East Africa: a penalized regression approach. Front Public Health 2024; 12:1383171. [PMID: 38947359 PMCID: PMC11211862 DOI: 10.3389/fpubh.2024.1383171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Accepted: 05/13/2024] [Indexed: 07/02/2024] Open
Abstract
Background Scalable PTSD screening strategies must be brief, accurate and capable of administration by a non-specialized workforce. Methods We used PTSD as determined by the structured clinical interview as our gold standard and considered predictors sets of (a) Posttraumatic Stress Checklist-5 (PCL-5), (b) Primary Care PTSD Screen for the DSM-5 (PC-PTSD) and, (c) PCL-5 and PC-PTSD questions to identify the optimal items for PTSD screening for public sector settings in Kenya. A logistic regression model using LASSO was fit by minimizing the average squared error in the validation data. Area under the receiver operating characteristic curve (AUROC) measured discrimination performance. Results Penalized regression analysis suggested a screening tool that sums the Likert scale values of two PCL-5 questions-intrusive thoughts of the stressful experience (#1) and insomnia (#21). This had an AUROC of 0.85 (using hold-out test data) for predicting PTSD as evaluated by the MINI, which outperformed the PC-PTSD. The AUROC was similar in subgroups defined by age, sex, and number of categories of trauma experienced (all AUROCs>0.83) except those with no trauma history- AUROC was 0.78. Conclusion In some East African settings, a 2-item PTSD screening tool may outperform longer screeners and is easily scaled by a non-specialist workforce.
Collapse
Affiliation(s)
- Susan M. Meffert
- Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, San Francisco, CA, United States
| | | | | | - Thomas C. Neylan
- Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, San Francisco, CA, United States
| | - Daniel Mwai
- Department of Health Economics, University of Nairobi, Nairobi, Kenya
| | | | - Dickens Akena
- Department of Psychiatry, Makerere University, Kampala, Uganda
| | - Grace Rota
- Department of Psychiatry, Makerere University, Kampala, Uganda
| | - Ammon Otieno
- Department of Psychiatry, Makerere University, Kampala, Uganda
| | | | | | | | | | - Dennis Oluoch
- Global Programs for Research and Training, Nairobi, Kenya
| | | | - Anne Mbwayo
- Department of Psychiatry, University of Nairobi, Nairobi, Kenya
| | - James G. Kahn
- Institute for Health Policy Studies, University of California, San Francisco, San Francisco, CA, United States
- Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, CA, United States
| | - Craig R. Cohen
- Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, San Francisco, CA, United States
| | - David E. Bukusi
- Department of Psychiatry, University of Nairobi, Nairobi, Kenya
| | - Gregory A. Aarons
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, United States
| | - Rachel L. Burger
- Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, San Francisco, CA, United States
| | - Chengshi Jin
- Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, CA, United States
| | - Charles E. McCulloch
- Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, CA, United States
| | | |
Collapse
|
6
|
Lawson GM, Azad G. School-Based Mental Health Interventions: Recommendations for Selecting and Reporting Implementation Strategies . THE JOURNAL OF SCHOOL HEALTH 2024; 94:581-585. [PMID: 38627895 DOI: 10.1111/josh.13458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 03/22/2024] [Accepted: 03/28/2024] [Indexed: 05/14/2024]
Affiliation(s)
- Gwendolyn M Lawson
- Department of Child and Adolescent Psychiatry and Behavioral Sciences, Children's Hospital of Philadelphia, Philadelphia, PA; Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Gazi Azad
- Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University Irving Medical Center, New York, NY, USA
| |
Collapse
|
7
|
Boltz M, Van Haitsma K, Baier RR, Sefcik JS, Hodgson NA, Jao YL, Kolanowski A. Ready or Not: A Conceptual Model of Organizational Readiness for Embedded Pragmatic Dementia Research. Res Gerontol Nurs 2024; 17:149-160. [PMID: 38598780 PMCID: PMC11163963 DOI: 10.3928/19404921-20240403-02] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/12/2024]
Abstract
The National Institute on Aging Alzheimer's Disease/Alzheimer's Disease and Related Dementias Research Implementation Milestones emphasize the need for implementation research that maximizes up-take and scale-up of evidence-based dementia care practices across settings, diverse populations, and disease trajectories. Organizational readiness for implementation is a salient consideration when planning and conducting embedded pragmatic trials, in which interventions are implemented by provider staff. The current article examines the conceptual and theoretical underpinnings of organizational readiness for implementation and the operationalization of this construct. We offer a preliminary conceptual model for explicating and measuring organizational readiness and describe the unique characteristics and demands of implementing evidence-based interventions targeting persons with dementia and/or their care partners. [Research in Gerontological Nursing, 17(3), 149-160.].
Collapse
|
8
|
Jumper MBE, Friedman BR, Becker-Haimes EM, Dong F, Kohler CG, Hurford I, Calkins ME. Implementation of an Adapted Fidelity Scale for Pennsylvania Coordinated Specialty Care Programs for First Episode Psychosis. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2024; 25:421-435. [PMID: 37943446 DOI: 10.1007/s11121-023-01607-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/18/2023] [Indexed: 11/10/2023]
Abstract
Coordinated Specialty Care (CSC) is an evidence-based model for early intervention of first episode psychosis (FEP). Monitoring fidelity to CSC models is essential for proper evaluation of program outcomes and quality improvement. To address variability across CSC programs and fidelity assessment-associated burden, an adapted fidelity scale was developed and implemented statewide in Pennsylvania. This report describes the design and preliminary performance of the adapted scale. The Pennsylvania FEP Fidelity Scale (PA-FEP-FS) assesses adherence to the CSC model by focusing on essential model components of other established fidelity scales, in tandem with program evaluation data. Initial data from fourteen PA-FEP sites from 2018 to 2021 were examined as preliminary validation. Assessment-associated burdens and costs were also estimated. PA-FEP-FS captured essential components of CSC models and proved feasible for implementation across varying program structures, while minimizing burden and cost. Programs utilized annual feedback as CSC benchmarks, evidenced by increased scores over time. PA-FEP-FS provides a model for adapting CSC fidelity scales to meet state- or local-level requirements while reducing assessment burdens and costs that may be associated with existing scales.
Collapse
Affiliation(s)
- Megan B E Jumper
- Pennsylvania Early Intervention Center (PEIC)/HeadsUp, Neurodevelopment and Psychosis Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 10 Gates, 3400 Spruce Street, Philadelphia, PA, 19104, USA
| | - Bess Rose Friedman
- Division of Community Behavioral Health, Department of Psychiatry and Behavioral Sciences, University of New Mexico School of Medicine, Albuquerque, NM, USA
| | - Emily M Becker-Haimes
- Pennsylvania Early Intervention Center (PEIC)/HeadsUp, Neurodevelopment and Psychosis Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 10 Gates, 3400 Spruce Street, Philadelphia, PA, 19104, USA
| | - Fanghong Dong
- Pennsylvania Early Intervention Center (PEIC)/HeadsUp, Neurodevelopment and Psychosis Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 10 Gates, 3400 Spruce Street, Philadelphia, PA, 19104, USA
| | - Christian G Kohler
- Pennsylvania Early Intervention Center (PEIC)/HeadsUp, Neurodevelopment and Psychosis Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 10 Gates, 3400 Spruce Street, Philadelphia, PA, 19104, USA
| | - Irene Hurford
- Pennsylvania Early Intervention Center (PEIC)/HeadsUp, Neurodevelopment and Psychosis Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 10 Gates, 3400 Spruce Street, Philadelphia, PA, 19104, USA
| | - Monica E Calkins
- Pennsylvania Early Intervention Center (PEIC)/HeadsUp, Neurodevelopment and Psychosis Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 10 Gates, 3400 Spruce Street, Philadelphia, PA, 19104, USA.
| |
Collapse
|
9
|
Patel-Syed Z, Becker S, Olson M, Rinella H, Scott K. What do you think it means? Using cognitive interviewing to improve measurement in implementation science: description and case example. Implement Sci Commun 2024; 5:14. [PMID: 38355677 PMCID: PMC10865651 DOI: 10.1186/s43058-024-00549-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 01/16/2024] [Indexed: 02/16/2024] Open
Abstract
Pragmatic measures are essential to evaluate the implementation of evidence-based interventions. Cognitive interviewing, a qualitative method that collects partner feedback throughout measure development, is particularly useful for developing pragmatic implementation measures. Measure developers can use cognitive interviewing to increase a measure's fit within a particular implementation context. However, cognitive interviewing is underused in implementation research, where most measures remain "homegrown" and used for single studies. We provide a rationale for using cognitive interviewing in implementation science studies and illustrate its use through a case example employing cognitive interviewing to inform development of a measurement-based care protocol for implementation in opioid treatment programs. Applications of cognitive interviewing, including developing a common language with partners and collecting multi-level feedback on assessment procedures, to improve measurement in implementation science are discussed.
Collapse
Affiliation(s)
- Zabin Patel-Syed
- Northwestern University Feinberg School of Medicine, Institute for Public Health and Medicine, Center for Dissemination and Implementation Science, Chicago, USA.
| | - Sara Becker
- Northwestern University Feinberg School of Medicine, Institute for Public Health and Medicine, Center for Dissemination and Implementation Science, Chicago, USA
| | - Miranda Olson
- Northwestern University Feinberg School of Medicine, Institute for Public Health and Medicine, Center for Dissemination and Implementation Science, Chicago, USA
| | - Hailey Rinella
- Northwestern University Feinberg School of Medicine, Institute for Public Health and Medicine, Center for Dissemination and Implementation Science, Chicago, USA
| | - Kelli Scott
- Northwestern University Feinberg School of Medicine, Institute for Public Health and Medicine, Center for Dissemination and Implementation Science, Chicago, USA
| |
Collapse
|
10
|
Yang LH, Bass JK, Le PD, Singh R, Gurung D, Velasco PR, Grivel MM, Susser E, Cleland CM, Muñoz RA, Kohrt BA, Bhana A. A Case Study of the Development of a Valid and Pragmatic Implementation Science Measure: The Barriers and Facilitators in Implementation of Task-Sharing Mental Health Interventions (BeFITS-MH) Measure. RESEARCH SQUARE 2024:rs.3.rs-3877031. [PMID: 38343864 PMCID: PMC10854285 DOI: 10.21203/rs.3.rs-3877031/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/18/2024]
Abstract
Background Few implementation science (IS) measures have been evaluated for validity, reliability and utility - the latter referring to whether a measure captures meaningful aspects of implementation contexts. In this case study, we describe the process of developing an IS measure that aims to assess Barriers and Facilitators in Implementation of Task-Sharing in Mental Health services (BeFITS-MH), and the procedures we implemented to enhance its utility. Methods We summarize conceptual and empirical work that informed the development of the BeFITS-MH measure, including a description of the Delphi process, detailed translation and local adaptation procedures, and concurrent pilot testing. As validity and reliability are key aspects of measure development, we also report on our process of assessing the measure's construct validity and utility for the implementation outcomes of acceptability, appropriateness, and feasibility. Results Continuous stakeholder involvement and concurrent pilot testing resulted in several adaptations of the BeFITS-MH measure's structure, scaling, and format to enhance contextual relevance and utility. Adaptations of broad terms such as "program," "provider type," and "type of service" were necessary due to the heterogeneous nature of interventions, type of task-sharing providers employed, and clients served across the three global sites. Item selection benefited from the iterative process, enabling identification of relevance of key aspects of identified barriers and facilitators, and what aspects were common across sites. Program implementers' conceptions of utility regarding the measure's acceptability, appropriateness, and feasibility were seen to cluster across several common categories. Conclusions This case study provides a rigorous, multi-step process for developing a pragmatic IS measure. The process and lessons learned will aid in the teaching, practice and research of IS measurement development. The importance of including experiences and knowledge from different types of stakeholders in different global settings was reinforced and resulted in a more globally useful measure while allowing for locally-relevant adaptation. To increase the relevance of the measure it is important to target actionable domains that predict markers of utility (e.g., successful uptake) per program implementers' preferences. With this case study, we provide a detailed roadmap for others seeking to develop and validate IS measures that maximize local utility and impact.
Collapse
Affiliation(s)
- Lawrence H Yang
- New York University School of Global Public Health, Department of Social and Behavioral Sciences
| | - Judy K Bass
- Johns Hopkins Bloomberg School of Public Health, Department of Mental Health
| | - PhuongThao Dinh Le
- New York University School of Global Public Health, Department of Social and Behavioral Sciences
| | - Ritika Singh
- George Washington University, Division of Global Mental Health, Department of Psychiatry and Behavioral Sciences
| | - Dristy Gurung
- Transcultural Psychosocial Organization (TPO) Nepal; King's College London, Denmark Hill Campus
| | - Paola R Velasco
- Universidad O'Higgins; Universidad Católica de Chile; Universidad de Chile
| | - Margaux M Grivel
- 1 New York University School of Global Public Health, Department of Social and Behavioral Sciences
| | - Ezra Susser
- Columbia University Mailman School of Public Health; New York State Psychiatric Institute
| | - Charles M Cleland
- New York University Grossman School of Medicine, Department of Population Health
| | | | - Brandon A Kohrt
- George Washington University, Division of Global Mental Health, Department of Psychiatry and Behavioral Sciences
| | - Arvin Bhana
- University of KwaZulu-Natal, Centre for Rural Health; South African Medical Research Council, Health Systems Research Unit
| |
Collapse
|
11
|
Terrana A, Viglione C, Rhee K, Rabin B, Godino J, Aarons GA, Chapman J, Melendrez B, Holguin M, Osorio L, Gidwani P, Juarez Nunez C, Firestein G, Hekler E. The core functions and forms paradigm throughout EPIS: designing and implementing an evidence-based practice with function fidelity. FRONTIERS IN HEALTH SERVICES 2024; 3:1281690. [PMID: 38292916 PMCID: PMC10826509 DOI: 10.3389/frhs.2023.1281690] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 12/21/2023] [Indexed: 02/01/2024]
Abstract
There are numerous frameworks for implementing evidence-based practices (EBPs) in novel settings to achieve "fidelity." However, identifying appropriate referents for fidelity poses a challenge. The Core Functions and Forms paradigm offers a model that can inform adaptation decisions throughout all phases of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. We applied the Core Functions-Forms paradigm throughout the Exploration and Preparation phases of EPIS in the design of two EBPs targeting family protective factors among Latinos in San Diego, as well as describe plans for its use in Implementation and Sustainment. We employed a distinct approach for each intervention element to contrast adaptation decisions that prioritize adherence to either form or function fidelity. We describe our application of the functions-forms paradigm within the EPIS framework, focusing on the Preparation phase. We also provide functions-forms matrices that map out the relationship between individual intervention components (forms) and the essential processes (functions) by which components are theorized to exert their impact. This case study of how the core functions-forms framework can be mapped onto EPIS can support a conceptual shift from prioritizing form fidelity to also focusing on function fidelity. This might allow interventionists to target appropriate fidelity referents when adapting an EBP, rather than defaulting to maintaining fidelity to forms as described in the protocol. We see great promise for using this framework for guiding actions throughout all EPIS phases and informing future applications of this paradigm to foster more robust fidelity to function.
Collapse
Affiliation(s)
- Alec Terrana
- School of Medicine, University of California San Diego, San Diego, CA, United States
| | - Clare Viglione
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
| | - Kyung Rhee
- School of Medicine, University of California San Diego, San Diego, CA, United States
| | - Borsika Rabin
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, CA, United States
| | - Job Godino
- Family Health Centers of San Diego, San Diego, CA, United States
| | - Gregory A. Aarons
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
| | - Jessica Chapman
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
| | - Blanca Melendrez
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
| | | | - Liliana Osorio
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
| | | | | | - Gary Firestein
- Altman Clinical and Translational Research Institute, School of Medicine, University of California, San Diego, CA, United States
| | - Eric Hekler
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, CA, United States
| |
Collapse
|
12
|
Berkel C, Murry VM, Thomas NA, Bekele B, Debreaux ML, Gonzalez C, Hanebutt RA. The Strong African American Families Program: Disrupting the Negative Consequences of Racial Discrimination Through Culturally Tailored, Family-Based Prevention. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2024; 25:44-55. [PMID: 36107276 PMCID: PMC11178634 DOI: 10.1007/s11121-022-01432-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/25/2022] [Indexed: 10/14/2022]
Abstract
Racism continues to be a major source of stress for African Americans and can impair psychological functioning. Adolescents experiencing discrimination may engage in self-soothing, but risky behaviors, which leave them at risk for negative life trajectories. Black pride has been identified as a key factor in explaining the heterogeneity in responses to discrimination. Racial socialization, strategies parents use to promote Black pride and protect youth from discrimination, is an important focus of family-based prevention programs serving African American families. This study tests the efficacy of a culturally tailored preventive intervention for rural African American families to disrupt the negative consequences of discrimination on adolescent psychological functioning. Four waves of data from the Strong African American Families (SAAF) efficacy trial (Murry & Brody in Journal of Marital & Family Therapy 30(3):271-283, 2004) with 667 African American families in rural Georgia were used for this study. Structural equation modeling was used to test study hypotheses. Adolescent experiences with discrimination at age 15 predicted concurrent psychological functioning and multiple risk behaviors at age 16, including sexual risk behavior, substance use problems, academic failure, and juvenile justice involvement. Mediation analyses demonstrated that psychological functioning was a significant mediator of these relations. The SAAF program was associated with increases in racial socialization, which in turn fostered gains in adolescent Black pride. Black pride was indirectly associated with reduced risk behavior through adolescent psychological functioning, but Black pride did not moderate the effect of discrimination on psychological functioning. This study confirms that family-based prevention can support African American adolescent mental health in the context of discrimination. However, more emphasis on reducing exposure to discrimination is needed.
Collapse
Affiliation(s)
- Cady Berkel
- College of Health Solutions, Arizona State University, 425 N. 5th St, Phoenix, AZ, 85004, USA.
- REACH Institute, Arizona State University, Tempe, USA.
| | - Velma McBride Murry
- Departments of Health Policy and Human & Organizational Development, Vanderbilt University, Nashville, USA
| | - Nalani A Thomas
- College of Health Solutions, Arizona State University, Phoenix, USA
| | - Beza Bekele
- Department of Psychology, Arizona State University, Phoenix, USA
| | - Marlena L Debreaux
- Department of Human & Organizational Development, Vanderbilt University, Nashville, USA
| | - Catherine Gonzalez
- Department of Human & Organizational Development, Vanderbilt University, Nashville, USA
| | - Rachel A Hanebutt
- Department of Human & Organizational Development, Vanderbilt University, Nashville, USA
| |
Collapse
|
13
|
Dear ER, McLeod BD, Peterson NM, Sutherland KS, Broda MD, Dopp AR, Lyon AR. A case study of pragmatic measure development of the Treatment Integrity for Elementary Settings self-report measure for teachers. IMPLEMENTATION RESEARCH AND PRACTICE 2024; 5:26334895231220262. [PMID: 38322805 PMCID: PMC10775717 DOI: 10.1177/26334895231220262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2024] Open
Abstract
Introduction Due to usability, feasibility, and acceptability concerns, observational treatment fidelity measures are often challenging to deploy in schools. Teacher self-report fidelity measures with specific design features might address some of these barriers. This case study outlines a community-engaged, iterative process to adapt the observational Treatment Integrity for Elementary Settings (TIES-O) to a teacher self-report version designed to assess the use of practices to support children's social-emotional competencies in elementary classrooms. Method Cognitive walkthrough interviews were conducted with teachers to improve the usability of the teacher self-report measure, called the Treatment Integrity for Elementary Schools-Teacher Report (TIES-T). Qualitative content analysis was used to extract themes from the interviews and inform changes to the measure. Results Increasing clarity and interactive elements in the measure training were the dominant themes, but suggestions for the measure format and jargon were also suggested. Conclusion The suggested changes resulted in a brief measure, training, and feedback system designed to support the teacher's use of practices to support children's social-emotional competencies in elementary classrooms. Future research with the TIES-T will examine the score reliability and validity of the measure.
Collapse
Affiliation(s)
- Emma R. Dear
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, USA
| | - Bryce D. McLeod
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, USA
| | | | | | - Michael D. Broda
- School of Education, Virginia Commonwealth University, Richmond, VA, USA
| | - Alex R. Dopp
- Department of Behavioral and Policy Sciences, Rand Corporation, Santa Monica, CA, USA
| | - Aaron R. Lyon
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| |
Collapse
|
14
|
Gillespie BM, Harbeck E, Sutherland-Fraser S, Nicholson P, Boric T. Psychometric validation of the Perceived Perioperative Competence Scale-Revised Short Form. J Adv Nurs 2023; 79:4732-4746. [PMID: 37313973 DOI: 10.1111/jan.15735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Revised: 04/18/2023] [Accepted: 05/30/2023] [Indexed: 06/15/2023]
Abstract
AIM To develop a parsimonious, shortened version of the Perceived Perioperative Competence Scale-Revised for perioperative nurses to complete as part of their specialty training while retaining good psychometric properties. DESIGN A longitudinal online survey was adopted. METHODS A national sample of perioperative nurses from Australia completed an online survey at two different time points 6 months apart between February and October 2021. Confirmatory factor analysis was conducted for item reduction and construct validity, while criterion, convergent validity and internal consistency were examined. RESULTS Usable data for psychometric assessment were obtained from 485 operating room nurses at time 1 and 164 nurses at time 2. The original 40-item revised scale was reduced to an 18-item measure, maintaining its six domains. Cronbach's alpha for the 18-item scale was .92 at time 1 and .90 at time 2. Scale validation demonstrated moderate to weak positive relationships in perceived competence scores relative to general self-efficacy, years of operating room experience, postgraduate education and recency of training. CONCLUSIONS Results suggest the 18-item Perceived Perioperative Competence Scale-Revised Short Form has initial robust psychometric properties and may be implemented in clinical settings as part of perioperative transition-to-practice, orientation programs and yearly professional development reviews. IMPLICATIONS FOR THE PROFESSION This short-form scale can help prepare perioperative nurses to demonstrate clinical competence in a climate of increasing professional demands using a valid measure of competence required in clinical practice. IMPACT Short and validated scales of perioperative competence are needed in clinical practice. Assessment of practising operating room nurses' perceived competence is essential in quality care provision, workforce planning and human resource management. This study provides an 18-item measure of the previously validated 40-item Perceived Perioperative Competence Scale-Revised. This scale can help provide an option for future testing of perioperative nurses' competence in clinical or research settings. PATIENT OR PUBLIC CONTRIBUTION Perioperative nurses were involved in the design of the study, particularly in the assessment of validation of the tools used in the study.
Collapse
Affiliation(s)
- Brigid M Gillespie
- National Health and Medical Research Council Centre of Research Excellence in Wiser Wound Care, Menzies Health Institute Queensland, Griffith University, Southport, Queensland, Australia
- Gold Coast University Hospital, Gold Coast Health Nursing and Midwifery Education and Research Unit, Gold Coast, Queensland, Australia
| | - Emma Harbeck
- National Health and Medical Research Council Centre of Research Excellence in Wiser Wound Care, Menzies Health Institute Queensland, Griffith University, Southport, Queensland, Australia
| | | | - Patricia Nicholson
- School of Nursing and Midwifery, Deakin University, Centre for Quality and Patient Safety Research in the Institute for Health Transformation, Geelong, Victoria, Australia
| | - Tina Boric
- Perioperative Nurse Educator, The Prince Charles Hospital, Metro North Health, Chermside, Queensland, Australia
- Conjoint Nurse Educator Australian Catholic University, Brisbane, Australia
| |
Collapse
|
15
|
Gabbard JL, Carpenter JG, Ernecoff NC, Mournighan K, Cornea I, McKone M, Hanson LC. Evaluating the pragmatic characteristics of advance care planning outcome measures in dementia clinical trials: A scoping review. J Am Geriatr Soc 2023; 71:3595-3608. [PMID: 37439456 PMCID: PMC10787044 DOI: 10.1111/jgs.18495] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 06/09/2023] [Accepted: 06/13/2023] [Indexed: 07/14/2023]
Abstract
BACKGROUND Advance care planning (ACP) is a process that involves discussing a person's goals, values, and preferences; it is particularly important for persons living with dementia (PLWD) given that dementia is incurable and progressive. To ensure results that will impact real-world practices, ACP outcome measures must be psychometrically strong, meaningful to key partners, and pragmatic to collect. Therefore, we conducted a scoping review of outcome measures utilized in ACP randomized controlled clinical trials (RCTs) enrolling PLWD or their care partners and evaluated their pragmatic characteristics. METHODS We searched MEDLINE/PubMed, EMBASE, CINAHL, PsycINFO, and Web of Science for peer-reviewed ACP RCTs enrolling PLWD or their care partners from 2011 to 2021. We abstracted characteristics of primary and secondary outcome measures, including pragmatic characteristics using an adapted Psychometric and Pragmatic Evidence Rating Scale and ACP outcome domains using the standardized ACP Outcome Framework (i.e., process, action, healthcare, or quality of care). RESULTS We included 21 ACP RCTs. Trials included 103 outcome measures (39 primary and 64 secondary), of which 11% measured process, 14% measured action, 49% measured healthcare, and 26% measured quality of care. Twenty-four (23%) outcome measures were highly pragmatic, the majority of which (67%) reflected healthcare outcome measures. Sixty-one (59%) outcomes were assessed as highly relevant to PLWD or their care partners. Only 20% (n = 21) of outcome measures were embedded into clinical practice. Most (62%) RCTs were conducted in nursing homes, and 33% were focused PLWD with advanced stage disease. CONCLUSIONS In RCTs testing ACP interventions to support PLWD, only 23% of outcome measures were highly pragmatic, and most of these measured healthcare utilizations. Outcome assessments were rarely integrated into the EHR during routine clinical care. New outcome measures that address the lived experience of PLWD and their care partners plus have high pragmatic characteristics are needed for embedded pragmatic clinical trials.
Collapse
Affiliation(s)
- Jennifer L Gabbard
- School of Medicine, Section of Gerontology and Geriatrics, Wake Forest University, Winston-Salem, North Carolina, USA
| | - Joan G Carpenter
- School of Nursing, Department of Organizational and Adult Health, University of Maryland, College Park, Maryland, USA
| | | | - Kimberly Mournighan
- Division of Geriatric Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, USA
| | - Isabella Cornea
- School of Medicine, Section of Gerontology and Geriatrics, Wake Forest University, Winston-Salem, North Carolina, USA
| | - Mark McKone
- School of Medicine, Section of Gerontology and Geriatrics, Wake Forest University, Winston-Salem, North Carolina, USA
| | - Laura C Hanson
- Division of Geriatric Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
16
|
Rivera J, Malone S, Puerto-Torres M, Prewitt K, Counts L, Wiphatphumiprates P, Sakaan F, Al Zebin Z, Arias AV, Bhattacharyya P, Gunasekera S, Johnson S, Kambugu J, Kaye EC, Mandrell B, Mack J, McArthur J, Mendez A, Morrissey L, Sharara-Chami R, Snaman J, Sniderman E, Luke DA, Graetz DE, Agulnik A. CritCom: assessment of quality of interdisciplinary communication around deterioration in pediatric oncologic patients. Front Oncol 2023; 13:1207578. [PMID: 37886167 PMCID: PMC10598383 DOI: 10.3389/fonc.2023.1207578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Accepted: 09/22/2023] [Indexed: 10/28/2023] Open
Abstract
Background High-quality clinical care requires excellent interdisciplinary communication, especially during emergencies, and no tools exist to evaluate communication in critical care. We describe the development of a pragmatic tool focusing on interdisciplinary communication during patient deterioration (CritCom). Methods The preliminary CritCom tool was developed after a literature review and consultation with a multidisciplinary panel of global experts in communication, pediatric oncology, and critical care to review the domains and establish content validity iteratively. Face and linguistic validity were established through cognitive interviews, translation, and linguistic synthesis. We conducted a pilot study among an international group of clinicians to establish reliability and usability. Results After reviewing 105 potential survey items, we identified 52 items across seven domains. These were refined through cognitive interviews with 36 clinicians from 15 countries. CritCom was piloted with 433 clinicians (58% nurses, 36% physicians, and 6% other) from 42 hospitals in 22 countries. Psychometric testing guided the refinement of the items for the final tool. CritCom comprised six domains with five items each (30 total). The final tool has excellent reliability (Cronbach's alpha 0.81-0.86), usability (93% agree or strongly agree that the tool is easy to use), and similar performance between English and Spanish tools. Confirmatory factor analysis was used to establish the final 6-domain structure. Conclusions CritCom is a reliable and pragmatic bilingual tool to assess the quality of interdisciplinary communication around patient deterioration for children in diverse resource levels globally. Critcom results can be used to design and evaluate interventions to improve team communication.
Collapse
Affiliation(s)
- Jocelyn Rivera
- Pediatric Emergency Department, Hospital Infantil Teletón de Oncología (HITO), Querétaro, Mexico
| | - Sara Malone
- Washington University in St. Louis, Brown School, St. Louis, MO, United States
| | - Maria Puerto-Torres
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Kim Prewitt
- Washington University in St. Louis, Brown School, St. Louis, MO, United States
| | - Lara Counts
- Washington University in St. Louis, Brown School, St. Louis, MO, United States
| | - Parima Wiphatphumiprates
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Firas Sakaan
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Zebin Al Zebin
- Pediatric Hematology and Oncology, King Hussein Cancer Center, Amman, Jordan
| | - Anita V. Arias
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | | | - Sanjeeva Gunasekera
- Department of Pediatric Oncology National Cancer Institute, Maharagama, Sri Lanka
| | - Sherry Johnson
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Joyce Kambugu
- Pediatric Oncology, Uganda Cancer Institute, Kampala, Uganda
| | - Erica C. Kaye
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Belinda Mandrell
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Jennifer Mack
- Department of Hematology and Oncology, Dana-Farber Cancer Institute and Boston Children’s Hospital, Boston, MA, United States
| | - Jennifer McArthur
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Alejandra Mendez
- Pediatric Intensive Care Unit, Unidad Nacional de Oncología pediátrica (UNOP), Guatemala City, Guatemala
| | - Lisa Morrissey
- Department of Hematology and Oncology, Dana-Farber Cancer Institute and Boston Children’s Hospital, Boston, MA, United States
| | - Rana Sharara-Chami
- Pediatric Critical Care Medicine, American University of Beirut, Beirut, Lebanon
| | - Jennifer Snaman
- Department of Hematology and Oncology, Dana-Farber Cancer Institute and Boston Children’s Hospital, Boston, MA, United States
| | - Elizabeth Sniderman
- Northern Alberta Children’s Cancer Program, Stollery Children’s Hospital, Edmonton, AB, Canada
| | - Douglas A. Luke
- Washington University in St. Louis, Brown School, St. Louis, MO, United States
| | - Dylan E. Graetz
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Asya Agulnik
- Division of Critical Care Medicine, Department of Global Pediatric Medicine, St. Jude Children’s Research Hospital, Memphis, TN, United States
| |
Collapse
|
17
|
Carlson MA, Fradgley EA, Yates D, Morris S, Tait J, Paul CL. Response to Gorter et al. regarding "Acceptability and feasibility of neurocognitive assessments with adults with primary brain cancer and brain metastases: A systematic review". Neurooncol Pract 2023; 10:493-494. [PMID: 37720391 PMCID: PMC10502773 DOI: 10.1093/nop/npad042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/19/2023] Open
Affiliation(s)
- Melissa A Carlson
- School of Medicine and Public Health, College of Health, Medicine, and Wellbeing, University of Newcastle, Callaghan, NSW, Australia
| | - Elizabeth A Fradgley
- School of Medicine and Public Health, College of Health, Medicine, and Wellbeing, University of Newcastle, Callaghan, NSW, Australia
| | - Della Yates
- School of Medicine and Public Health, College of Health, Medicine, and Wellbeing, University of Newcastle, Callaghan, NSW, Australia
| | - Sarah Morris
- School of Medicine and Public Health, College of Health, Medicine, and Wellbeing, University of Newcastle, Callaghan, NSW, Australia
| | - Jordan Tait
- NSW and ACT Research and Evaluation Unit, GP Synergy, Mayfield West, NSW, Australia
| | - Christine L Paul
- School of Medicine and Public Health, College of Health, Medicine, and Wellbeing, University of Newcastle, Callaghan, NSW, Australia
| |
Collapse
|
18
|
Chaitow L, McCabe P, Munro N, Purcell A. Language and early literacy professional development: A complex intervention for early childhood educators and speech-language pathologists. INTERNATIONAL JOURNAL OF SPEECH-LANGUAGE PATHOLOGY 2023; 25:656-666. [PMID: 36062764 DOI: 10.1080/17549507.2022.2115136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Purpose: Improving language and literacy skills in preschoolers can lead to better life outcomes. One way speech-language pathologists (SLPs) can improve these skills in preschoolers is by supporting educators through professional development (PD). However, PD in early childhood education and care (ECEC) settings is a complex intervention. To improve preschoolers' language and literacy skills using PD, SLPs must first work with educators to change or increase educators' language and literacy-promoting behaviours. This paper aimed to describe educator behaviours and preschooler skills following a real-world language and literacy PD intervention facilitated by two community SLPs.Method: Two pragmatic studies were conducted across four ECEC centres: (1) an observation study of 13 educators' self-reported language and literacy promoting behaviours, and (2) a non-randomised controlled trial investigating the language and literacy skills of 82 preschoolers as reported by their educators and parents/carers.Result: After the intervention, educators rated themselves as performing language and literacy-promoting behaviours more frequently. Educators also rated the early reading skills of preschoolers more highly after the PD intervention, but not preschoolers' oral language or early writing skills. Parents/carers did not report any significant improvements in preschoolers' skills.Conclusion: PD as an SLP intervention, whilst promising, showed mixed outcomes. Educator outcomes improved; however, preschooler outcomes were varied.
Collapse
Affiliation(s)
- Lauren Chaitow
- Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia, and
| | - Patricia McCabe
- Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia, and
| | - Natalie Munro
- Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia, and
| | - Alison Purcell
- School of Health Sciences, Western Sydney University, Campbelltown, Australia
| |
Collapse
|
19
|
Vico R, Martín J, González M. Functional Assessment of Handwriting Among Children: A Systematic Review of the Psychometric Properties. Am J Occup Ther 2023; 77:7705205050. [PMID: 37877571 DOI: 10.5014/ajot.2023.050174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2023] Open
Abstract
IMPORTANCE Handwriting legibility and speed assessments have a critical role in identifying and evaluating handwriting problems, which are common among children. OBJECTIVE The objective was to evaluate the psychometric properties and clinical utility of handwriting assessments for children ages 3 to 16 yr. DATA SOURCES A systematic review was conducted in CINAHL, PubMed (MEDLINE), Scopus, and education databases, with no time limits. The search strategy included a combination of the following keywords: handwriting, write, children, assessment, and validity. The exclusion criteria were assessment tools that were electronic, that focused on cognitive components of handwriting, or that only evaluated alphabets other than Latin. STUDY SELECTION AND DATA COLLECTION The systematic review was carried out on the basis of the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations. The protocol was registered in the Prospective Register of Systematic Reviews (PROSPERO). FINDINGS The 14 included instruments had a total sample of 4,987 children. Internal consistency ranged from moderate (.73; Writing Readiness Inventory Tool in Context) to high (.98; Letter Writing). The interexaminer reliability values of the 11 direct assessment instruments ranged from .77 (Systematic Screening for Handwriting Difficulties) to 1.00 (Handwriting Speed Test). CONCLUSIONS AND RELEVANCE In this systematic review, existing tools were evaluated by clinical utility and the quality of psychometric properties. Direct assessments showed good psychometric properties. Indirect and self-assessment tools demonstrated poor psychometric properties. Further research on screening tools and tools in other languages is needed. What This Article Adds: Specific learning disorders (e.g., dysgraphia) negatively affect academic learning and, when prolonged in time, self-concept. However, handwriting legibility and speed assessments could be used to identify and evaluate these learning disorders if an early referral to occupational therapy is carried out.
Collapse
Affiliation(s)
- Rocío Vico
- Rocío Vico, OTD, OTR/L, is PhD Candidate, Department of Physiotherapy, University of Malaga, Malaga, Andalucía, Spain, and Occupational Therapist, Department of Occupational Therapy, Hospital Juan Ramón Jiménez, Andalusian Public Health Service, Huelva, Andalucía, Spain
| | - Jaime Martín
- Jaime Martín, PhD, is Assistant Professor, Department of Human Anatomy, Legal and Forensic Medicine Area, University of Malaga, Malaga, Andalucía, Spain, and Researcher, Malaga Institute of Biomedical Research, Malaga, Andalucía, Spain;
| | - Manuel González
- Manuel González, PhD, is Associate Professor, Department of Physiotherapy, University of Malaga, Malaga, Andalucía, Spain, and Researcher, Malaga Institute of Biomedical Research, Malaga, Andalucía, Spain
| |
Collapse
|
20
|
O'Neill CJ, Morris-Baguley H, Alam AS, Carlson MA, Blefari N, Rowe CW, Fradgley EA, Paul C. Thyroid cancer patient reported outcome measures in clinical practice: analysing acceptability and optimizing recruitment. ANZ J Surg 2023; 93:2214-2221. [PMID: 37391881 DOI: 10.1111/ans.18578] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 06/13/2023] [Accepted: 06/14/2023] [Indexed: 07/02/2023]
Abstract
BACKGROUND Patient reported outcomes measures (PROMs) can provide valuable metrics in clinical trials and cancer registries. To ensure relevance, patient participation must be optimized and PROMs be highly acceptable to patients. There are few data reporting methods to maximize recruitment and a lack of consensus regarding appropriate PROMs for thyroid cancer survivors. METHODS All patients with a new diagnosis of thyroid (excluding micropapillary and anaplastic) cancer within a single Australian health district between January 2020 and December 2021 were invited to complete PROMs electronically, and self-report ease of use and comprehensiveness of each tool. Participants completed Short Form-12 (SF-12), European Organization of Research and Treatment of Cancer (EORTC-QLQ-C30), City of Hope Quality of Life-Thyroid Version (COH-TV) and Thyroid Cancer Quality of Life Survey (ThyCaQoL). Semi-structured qualitative telephone interviews explored patient priorities. An enhanced, multimodal recruitment strategy was instituted after 12 months due to low response rates. RESULTS Survey completion improved under enhanced recruitment (37/62, 60% versus 19/64, 30%, P = 0.0007) with no differences in demographic or clinical characteristics. Few (4%-7%) respondents rated surveys as difficult to complete. No single PROM comprehensively captured health-related quality of life, with disease-specific tools performing marginally better (54% ThyCaQoL and 52% CoH-TV) compared to generic tools (38% SF-12 and 42% EOROTC-QLQ-C30). Qualitative data suggested that concurrent diagnoses, and survey invitation prior to surgery, made surveys more difficult to complete. CONCLUSION A comprehensive and representative assessment of PROMs in thyroid cancer survivors requires the use of multiple survey tools and specialized staff to maximize recruitment.
Collapse
Affiliation(s)
- Christine J O'Neill
- Surgical Services, John Hunter Hospital, Newcastle, New South Wales, Australia
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| | - Harriet Morris-Baguley
- Surgical Services, John Hunter Hospital, Newcastle, New South Wales, Australia
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
| | - Ahmad S Alam
- Surgical Services, John Hunter Hospital, Newcastle, New South Wales, Australia
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
| | - Melissa A Carlson
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| | - Nicholas Blefari
- Surgical Services, John Hunter Hospital, Newcastle, New South Wales, Australia
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
| | - Christopher W Rowe
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
- Department of Endocrinology, John Hunter Hospital, Newcastle, New South Wales, Australia
| | - Elizabeth A Fradgley
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| | - Christine Paul
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| |
Collapse
|
21
|
Daza JF, Chesney TR, Alibhai SMH, Kennedy ED, Lebovic G, Lightfoot D, Mbadjeu Hondjeu AR, Morales JF, Pivetta B, Jolley R, Racz E, Wilmshurst L, Wijeysundera DN. Clinical tools to assess functional capacity before elective non-cardiac surgery: a scoping review protocol. JBI Evid Synth 2023; 21:1632-1647. [PMID: 37006169 DOI: 10.11124/jbies-22-00397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/04/2023]
Abstract
OBJECTIVE The objective of this scoping review is to map the evidence on clinical tools to assess functional capacity prior to elective non-cardiac surgery. INTRODUCTION Functional capacity is a strong prognostic indicator before surgery, which can be used to identify patients at elevated risk of postoperative complications, yet, there is no consensus on which clinical tools should be used to assess functional capacity in patients prior to non-cardiac surgery. INCLUSION CRITERIA This review will consider any randomized or non-randomized studies that evaluate the performance of a functional capacity assessment tool in adults (≥18 years) prior to non-cardiac surgery. For studies to be included, the tool must be used clinically for risk stratification. We will exclude studies on lung and liver transplant surgery, as well as ambulatory procedures performed under local anesthesia. METHODS The review will be conducted in line with the JBI methodology for scoping reviews. A peer-reviewed search strategy will be used to query relevant databases (ie, MEDLINE, Embase, EBM Reviews). Additional sources of evidence will include databases of non-peer-reviewed literature and the reference lists of included studies. Two independent reviewers will identify eligible studies in 2 stages: stage 1, based on titles and abstracts; and stage 2, based on full texts. Information on study details, measurement properties, pragmatic qualities, and/or clinical utility metrics will be charted in duplicate onto standardized data collection forms. The results will be presented using descriptive summaries, frequency tables, and visual plots that highlight the extent of evidence and remaining gaps in the validation process of each tool. REVIEW REGISTRATION Open Science Framework https://osf.io/6nfht.
Collapse
Affiliation(s)
- Julian F Daza
- Department of Surgery, Division of General Surgery, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
| | - Tyler R Chesney
- Department of Surgery, Division of General Surgery, University of Toronto, Toronto, ON, Canada
- Department of Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada
| | - Shabbir M H Alibhai
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
- Department of Medicine, University Health Network, Toronto, ON, Canada
| | - Erin D Kennedy
- Department of Surgery, Division of General Surgery, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- Department of Surgery, Mount Sinai Hospital, Toronto, ON, Canada
| | - Gerald Lebovic
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
| | - David Lightfoot
- Health Sciences Library, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Arnaud R Mbadjeu Hondjeu
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Juan F Morales
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Bianca Pivetta
- Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Rachel Jolley
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
| | - Elizabeth Racz
- Patient and Family Partner Program, Unity Health Toronto, Toronto, ON, Canada
| | - Luke Wilmshurst
- Patient and Family Partner Program, Unity Health Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| |
Collapse
|
22
|
Lawson GM, Jawad AF, Comly R, Khanna M, Glick HA, Beidas RS, Goldstein J, Brizzolara-Dove S, Wilson T, Rabenau-McDonnell Q, Eiraldi R. A comparison of two group cognitive behavioral therapy protocols for anxiety in urban schools: appropriateness, child outcomes, and cost-effectiveness. Front Psychiatry 2023; 14:1105630. [PMID: 37426105 PMCID: PMC10328418 DOI: 10.3389/fpsyt.2023.1105630] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Accepted: 05/16/2023] [Indexed: 07/11/2023] Open
Abstract
Background Cognitive behavioral therapy (CBT) for pediatric anxiety is efficacious for reducing anxiety symptoms and improving functioning, but many children are unable to access CBT for anxiety in community settings. Schools are an important setting in which children access mental health care, including therapy for anxiety. In this setting, therapy is usually delivered by Masters-level therapists. Objectives Friends for Life (FRIENDS), a 12-session, manualized, group CBT program for anxiety has demonstrated effectiveness when implemented in schools. However, prior research has also found challenges regarding feasibility and cultural fit when delivering FRIENDS in the urban school context. To address these challenges, we adapted FRIENDS for implementation in the school setting so that it might be more feasible and culturally appropriate for low-income, urban schools in the United States, while maintaining the core components of treatment. The current study uses a mixed-method approach to compare the effectiveness, cost-effectiveness, and perceived appropriateness of FRIENDS and CATS when delivered by Masters-level therapists with train-the-trainer support. Materials and methods First, we compared change scores for student outcomes (i.e., child-report MASC-2 total score, parent-report MASC-2 total score, teacher-report Engagement and Disaffection subscale scores) from pre- to post- treatment between students receiving FRIENDS and students receiving CATS to assess whether the two conditions resulted in equivalent outcomes. Second, we compared the cost and cost-effectiveness between the groups. Finally, we used an applied thematic analysis to compare appropriateness of the interventions as perceived by therapists and supervisors. Results The mean change score for the child-reported MASC-2 was 1.9 (SE = 1.72) points in the FRIENDS condition and 2.9 (SE = 1.73) points in the CATS condition; results indicated that the conditions were similar in their treatment effects, and symptom reductions were small in both groups. The modified protocol, CATS, was shown to cost significantly less to implement compared to FRIENDS and showed greater cost-effectiveness. Finally, compared to therapists and supervisors in the CATS condition, therapists and supervisors in the FRIENDS condition more strongly described aspects of the intervention that were not appropriate for their context and in need of more extensive adaptations. Conclusion Relatively brief, group CBT for anxiety, with adaptations to improve cultural fit, is a promising approach to treat youth anxiety symptom when delivered by school-based therapists with train-the-trainer implementation support.
Collapse
Affiliation(s)
- Gwendolyn M. Lawson
- Children’s Hospital of Philadelphia, Philadelphia, PA, United States
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Abbas F. Jawad
- Children’s Hospital of Philadelphia, Philadelphia, PA, United States
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Rachel Comly
- Children’s Hospital of Philadelphia, Philadelphia, PA, United States
| | - Muniya Khanna
- OCD and Anxiety Institute, Bryn Mawr, PA, United States
| | - Henry A. Glick
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Rinad S. Beidas
- Department of Medical Social Sciences, Feinberg School of Medicine Northwestern University, Chicago, IL, United States
| | - Jessica Goldstein
- Children’s Hospital of Philadelphia, Philadelphia, PA, United States
| | | | - Tara Wilson
- Children’s Hospital of Philadelphia, Philadelphia, PA, United States
| | | | - Ricardo Eiraldi
- Children’s Hospital of Philadelphia, Philadelphia, PA, United States
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
23
|
Lee W, Kim H, Shim J, Kim D, Hyeon J, Joo E, Joo BE, Oh J. The simplification of the insomnia severity index and epworth sleepiness scale using machine learning models. Sci Rep 2023; 13:6214. [PMID: 37069247 PMCID: PMC10106896 DOI: 10.1038/s41598-023-33474-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Accepted: 04/13/2023] [Indexed: 04/19/2023] Open
Abstract
Insomnia and excessive daytime sleepiness (EDS) are the most common complaints in sleep clinics, and the cost of healthcare services associated with them have also increased significantly. Though the brief questionnaires such as the Insomnia Severity Index (ISI) and Epworth Sleepiness Scale (ESS) can be useful to assess insomnia and EDS, there are some limitations to apply for large numbers of patients. As the researches using the Internet of Things technology become more common, the need for the simplification of sleep questionnaires has been also growing. We aimed to simplify ISI and ESS using machine learning algorithms and deep neural networks with attention models. The medical records of 1,241 patients who examined polysomnography for insomnia or EDS were analyzed. All patients are classified into five groups according to the severity of insomnia and EDS. To develop the model, six machine learning algorithms were firstly applied. After going through normalization, the process with the CNN+ Attention model was applied. We classified a group with an accuracy of 93% even with only the results of 6 items (ISI1a, ISI1b, ISI3, ISI5, ESS4, ESS7). We simplified the sleep questionnaires with maintaining high accuracy by using machine learning models.
Collapse
Affiliation(s)
- Woodo Lee
- Department of Physics, Korea University, Seoul, 02841, South Korea
| | - Hyejin Kim
- College of Pharmacy, Sookmyung Women's University, Seoul, 04310, South Korea
| | - Jaekwoun Shim
- Institute of Educational Research, Korea University, Seoul, 02841, South Korea
| | - Dongsin Kim
- Sleep Research Center, NYX Corporation, Hanam, 12902, South Korea
| | - Janghun Hyeon
- Semiconductor Research Institute, Korea University, Seoul, 02841, South Korea
| | - Eunyeon Joo
- Department of Neurology, Samsung Medical Center, Seoul, 06351, South Korea
| | - Byung-Euk Joo
- Department of Neurology, Soonchunhyang University Seoul Hospital, Soonchunhyang University, Seoul, 31151, South Korea
| | - Junhyoung Oh
- Institute for Business Research and Education, Korea University, Seoul, 02841, South Korea.
| |
Collapse
|
24
|
Berger MB, Chisholm M, Miller HN, Askew S, Kay MC, Bennett GG. "We bleed for our community:" A qualitative exploration of the implementation of a pragmatic weight gain prevention trial from the perspectives of community health center professionals. BMC Public Health 2023; 23:695. [PMID: 37060053 PMCID: PMC10103522 DOI: 10.1186/s12889-023-15574-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 03/31/2023] [Indexed: 04/16/2023] Open
Abstract
BACKGROUND Clinical trial implementation continues to shift toward pragmatic design, with the goal of increasing future adoption in clinical practice. Yet, few pragmatic trials within clinical settings have qualitatively assessed stakeholder input, especially from those most impacted by research implementation and outcomes, i.e., providers and staff. Within this context, we conducted a qualitative study of the implementation of a pragmatic digital health obesity trial with employees at a Federally qualified health center (FQHC) network in central North Carolina. METHODS Participant recruitment was conducted through purposive sampling of FQHC employees from a variety of backgrounds. Two researchers conducted semi-structured qualitative interviews and collected demographic data. Interviews were digitally recorded, professionally transcribed and double-coded by two independent researchers using NVivo 12. Coding discrepancies were reviewed by a third researcher until intercoder consensus was reached. Responses were compared within and across participants to elucidate emergent themes. RESULTS Eighteen qualitative interviews were conducted, of whom 39% provided direct medical care to patients and 44% worked at the FQHC for at least seven years. Results illuminated the challenges and successes of a pragmatically designed obesity treatment intervention within the community that serves medically vulnerable patients. Although limited time and staffing shortages may have challenged recruitment processes, respondents described early buy-in from leadership; an alignment of organizational and research goals; and consideration of patient needs as facilitators to implementation. Respondents also described the need for personnel power to sustain novel research interventions and considerations of health center resource constraints. CONCLUSIONS Results from this study contribute to the limited literature on pragmatic trials utilizing qualitative methods, particularly in community-based obesity treatment. To continue to merge the gaps between research implementation and clinical care, qualitative assessments that solicit stakeholder input are needed within pragmatic trial design. For maximum impact, researchers may wish to solicit input from a variety of professionals at trial onset and ensure that shared common goals and open collaboration between all partners is maintained throughout the trial. TRIAL REGISTRATION This trial was registered with ClinicalTrials.gov (NCT03003403) on December 28, 2016.
Collapse
Affiliation(s)
- Miriam B Berger
- Duke Digital Health Science Center, Duke University, 417 Chapel Drive Room 048, Campus Box 90086, Durham, NC, 27708-0086, USA.
| | - Miriam Chisholm
- Duke Digital Health Science Center, Duke University, 417 Chapel Drive Room 048, Campus Box 90086, Durham, NC, 27708-0086, USA
| | - Hailey N Miller
- Duke Digital Health Science Center, Duke University, 417 Chapel Drive Room 048, Campus Box 90086, Durham, NC, 27708-0086, USA
- School of Nursing, Duke University, 307 Trent Drive, Durham, NC, 27710, USA
| | - Sandy Askew
- Duke Digital Health Science Center, Duke University, 417 Chapel Drive Room 048, Campus Box 90086, Durham, NC, 27708-0086, USA
| | - Melissa C Kay
- Duke Digital Health Science Center, Duke University, 417 Chapel Drive Room 048, Campus Box 90086, Durham, NC, 27708-0086, USA
- Department of Pediatrics, Duke University, 3116 N. Duke Street, Room 1029, 27704, Durham, NC, USA
| | - Gary G Bennett
- Duke Digital Health Science Center, Duke University, 417 Chapel Drive Room 048, Campus Box 90086, Durham, NC, 27708-0086, USA
- Department of Psychology and Neuroscience, Duke University, 222 Reuben-Cooke, Durham, NC, 27708, USA
| |
Collapse
|
25
|
Chouinard J, Turcotte M, Harrison S, Loignon J, Morisset AS, Saulais L, Labonté MÈ. Vers une stratégie d’évaluation de la qualité de l’offre alimentaire dans les établissements de santé du Québec : sélection d’outils existants et retours d’utilisateurs potentiels. CAHIERS DE NUTRITION ET DE DIÉTÉTIQUE 2023. [DOI: 10.1016/j.cnd.2023.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/14/2023]
|
26
|
Kingston Ú, Adamakis M, Lester D, Costa J. A Scoping Review on Quality Physical Education Programmes and Their Outcomes on Primary-Level Pupils. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:3575. [PMID: 36834274 PMCID: PMC9965463 DOI: 10.3390/ijerph20043575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 02/14/2023] [Accepted: 02/15/2023] [Indexed: 06/18/2023]
Abstract
A scoping review was carried out on the literature relating to the evaluation of the implementation of quality physical education (QPE) programmes and related outcomes on final-stage primary-level pupils' attitudes towards physical education (ATPE), physical activity behaviour (PAB), mental wellbeing (MWB) and academic achievement (AA). The scoping review included studies published between 2000 and 2020 in the PubMed, Elsevier, SCOPUS and CINAHL databases and was completed in accordance with the PRISMA extension for scoping reviews' guidelines. Based on the inclusion criteria, 15 out of 2869 studies were included in the review. A thematic analysis was used to inductively and deductively analyse the studies for common themes of features of QPE programmes in primary schools, arising from nine different countries, considering the four outcome dimensions (ATPE, PAB, MWB and AA). The common themes identified as features of QPE across all four dimensions were as follows: (1) government leadership; (2) PE curriculum; (3) school principal and leaders; (4) organisational management from leadership in school; (5) teachers; (6) parental involvement; and (7) community partnerships. Based on these findings, recommendations were made for an evaluation framework on QPE in primary education.
Collapse
Affiliation(s)
- Úna Kingston
- Sports Studies and Physical Education Programme, School of Education, University College Cork, T12 CY82 Cork, Ireland
| | - Manolis Adamakis
- School of Physical Education and Sport Science, National and Kapodistrian University of Athens, 10556 Athens, Greece
| | - Diarmuid Lester
- Sports Studies and Physical Education Programme, School of Education, University College Cork, T12 CY82 Cork, Ireland
| | - João Costa
- Sports Studies and Physical Education Programme, School of Education, University College Cork, T12 CY82 Cork, Ireland
| |
Collapse
|
27
|
Randall CL. Dissemination and implementation research for oral and craniofacial health: Background, a review of literature and future directions. Community Dent Oral Epidemiol 2023; 51:119-132. [PMID: 36744988 PMCID: PMC10364974 DOI: 10.1111/cdoe.12841] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Revised: 12/15/2022] [Accepted: 12/22/2022] [Indexed: 02/07/2023]
Abstract
Oral conditions are highly prevalent globally and have profound consequence on individuals and communities. Clinical (e.g. dental treatments, behavioural counselling) and non-clinical (e.g. community-based programming, water fluoridation, oral health policy) evidence-based interventions have been identified, recommended and applied at the clinic, community and policy levels. Still, the burden of oral conditions persists, with inequitable distribution across populations. A major driver of this lack of progress is poor translation of research findings, which results in an evidence-to-practice gap. Dissemination and implementation science (DIS) has emerged to address this gap. A relatively new field, application of DIS represents an important avenue for achieving good dental, oral and craniofacial health for all. The goal of this introductory article is to provide a brief background on DIS relevant to researchers in dentistry and oral health. The problem of knowledge translation, basic concepts and terminology in DIS, and approaches to doing dissemination and implementation research-including implementation strategies, key outcomes, and implementation theories, models and frameworks-are discussed. Additionally, the article reviews literature applying DIS to dentistry and oral health. Results of published studies and their implications for the field are presented. Drawing on the literature review and contemporary thinking in DIS, current gaps, opportunities and future directions are discussed. Resources for understanding and applying DIS are provided throughout. This article serves as a primer on DIS for dental and oral health researchers of all types working across a range of contexts; it also serves as a call to action for increased application of DIS to address the burden of oral conditions globally.
Collapse
Affiliation(s)
- Cameron L Randall
- Department of Oral Health Sciences, University of Washington School of Dentistry, Seattle, Washington, USA
| |
Collapse
|
28
|
Robinson CH, Damschroder LJ. A pragmatic context assessment tool (pCAT): using a Think Aloud method to develop an assessment of contextual barriers to change. Implement Sci Commun 2023; 4:3. [PMID: 36631914 PMCID: PMC9835384 DOI: 10.1186/s43058-022-00380-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Accepted: 11/27/2022] [Indexed: 01/13/2023] Open
Abstract
BACKGROUND The Consolidated Framework for Implementation Research (CFIR) is a determinant framework that can be used to guide context assessment prior to implementing change. Though a few quantitative measurement instruments have been developed based on the CFIR, most assessments using the CFIR have relied on qualitative methods. One challenge to measurement is to translate conceptual constructs which are often described using highly abstract, technical language into lay language that is clear, concise, and meaningful. The purpose of this paper is to document methods to develop a freely available pragmatic context assessment tool (pCAT). The pCAT is based on the CFIR and designed for frontline quality improvement teams as an abbreviated assessment of local facilitators and barriers in a clinical setting. METHODS Twenty-seven interviews using the Think Aloud method (asking participants to verbalize thoughts as they respond to assessment questions) were conducted with frontline employees to improve a pilot version of the pCAT. Interviews were recorded and transcribed verbatim; the CFIR guided coding and analyses. RESULTS Participants identified several areas where language in the pCAT needed to be modified, clarified, or allow more nuance to increase usefulness for frontline employees. Participants found it easier to respond to questions when they had a recent, specific project in mind. Potential barriers and facilitators tend to be unique to each specific improvement. Participants also identified missing concepts or that were conflated, leading to refinements that made the pCAT more understandable, accurate, and useful. CONCLUSIONS The pCAT is designed to be practical, using everyday language familiar to frontline employees. The pCAT is short (14 items), freely available, does not require research expertise or experience. It is designed to draw on the knowledge of individuals most familiar with their own clinical context. The pCAT has been available online for approximately two years and has generated a relatively high level of interest indicating potential usefulness of the tool.
Collapse
Affiliation(s)
- Claire H Robinson
- VA Center for Clinical Management Research, VA Ann Arbor Healthcare System, 2215 Fuller Road (152), Ann Arbor, MI, 48105, USA.
| | - Laura J Damschroder
- VA Center for Clinical Management Research, VA Ann Arbor Healthcare System, 2215 Fuller Road (152), Ann Arbor, MI, 48105, USA
| |
Collapse
|
29
|
Mrklas KJ, Boyd JM, Shergill S, Merali S, Khan M, Nowell L, Goertzen A, Pfadenhauer LM, Paul K, Sibley KM, Swain L, Vis-Dunbar M, Hill MD, Raffin-Bouchal S, Tonelli M, Graham ID. Tools for assessing health research partnership outcomes and impacts: a systematic review. Health Res Policy Syst 2023; 21:3. [PMID: 36604697 PMCID: PMC9817421 DOI: 10.1186/s12961-022-00937-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Accepted: 11/08/2022] [Indexed: 01/07/2023] Open
Abstract
OBJECTIVE To identify and assess the globally available valid, reliable and acceptable tools for assessing health research partnership outcomes and impacts. METHODS We searched Ovid MEDLINE, Embase, CINAHL Plus and PsycINFO from origin to 2 June 2021, without limits, using an a priori strategy and registered protocol. We screened citations independently and in duplicate, resolving discrepancies by consensus and retaining studies involving health research partnerships, the development, use and/or assessment of tools to evaluate partnership outcomes and impacts, and reporting empirical psychometric evidence. Study, tool, psychometric and pragmatic characteristics were abstracted using a hybrid approach, then synthesized using descriptive statistics and thematic analysis. Study quality was assessed using the quality of survey studies in psychology (Q-SSP) checklist. RESULTS From 56 123 total citations, we screened 36 027 citations, assessed 2784 full-text papers, abstracted data from 48 studies and one companion report, and identified 58 tools. Most tools comprised surveys, questionnaires and scales. Studies used cross-sectional or mixed-method/embedded survey designs and employed quantitative and mixed methods. Both studies and tools were conceptually well grounded, focusing mainly on outcomes, then process, and less frequently on impact measurement. Multiple forms of empirical validity and reliability evidence was present for most tools; however, psychometric characteristics were inconsistently assessed and reported. We identified a subset of studies (22) and accompanying tools distinguished by their empirical psychometric, pragmatic and study quality characteristics. While our review demonstrated psychometric and pragmatic improvements over previous reviews, challenges related to health research partnership assessment and the nascency of partnership science persist. CONCLUSION This systematic review identified multiple tools demonstrating empirical psychometric evidence, pragmatic strength and moderate study quality. Increased attention to psychometric and pragmatic requirements in tool development, testing and reporting is key to advancing health research partnership assessment and partnership science. PROSPERO CRD42021137932.
Collapse
Affiliation(s)
- K. J. Mrklas
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10, 3280 Hospital Drive NW, Calgary, AB T2N 4Z6 Canada
- Strategic Clinical Networks™, Provincial Clinical Excellence, Alberta Health Services, Calgary, AB Canada
| | - J. M. Boyd
- Knowledge Translation Program, St Michael’s Hospital, Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, ON Canada
| | - S. Shergill
- Cumming School of Medicine, University of Calgary, Calgary, AB Canada
| | - S. Merali
- Faculty of Kinesiology, University of Calgary, Calgary, AB Canada
| | - M. Khan
- Department of Community Health Sciences, University of Manitoba, Winnipeg, MB Canada
| | - L. Nowell
- Faculty of Nursing, University of Calgary, Calgary, AB Canada
| | - A. Goertzen
- Faculty of Science, University of Alberta, Edmonton, AB Canada
| | - L. M. Pfadenhauer
- Institute for Medical Information Processing, Biometry, and Epidemiology–IBE, Ludwig-Maximilian Universität Munich, Munich, Germany
- Pettenkofer School of Public Health, Munich, Germany
| | - K. Paul
- University of Calgary Summer Studentships Program, Calgary, AB Canada
| | - K. M. Sibley
- Department of Community Health Sciences, University of Manitoba, Winnipeg, MB Canada
- George & Fay Yee Centre for Healthcare Innovation, University of Manitoba, Winnipeg, MB Canada
| | - L. Swain
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10, 3280 Hospital Drive NW, Calgary, AB T2N 4Z6 Canada
| | - M. Vis-Dunbar
- University of British Columbia - Okanagan, Kelowna, BC Canada
| | - M. D. Hill
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10, 3280 Hospital Drive NW, Calgary, AB T2N 4Z6 Canada
- Departments of Clinical Neurosciences, Medicine and Radiology, Cumming School of Medicine, University of Calgary, Calgary, AB Canada
- Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Calgary, AB Canada
| | | | - M. Tonelli
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB Canada
- Office of the Vice-President (Research), University of Calgary, Calgary, AB Canada
| | - I. D. Graham
- Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, ON Canada
- School of Epidemiology and Public Health & School of Nursing, University of Ottawa, Ottawa, ON Canada
| |
Collapse
|
30
|
Mrklas KJ, Merali S, Khan M, Shergill S, Boyd JM, Nowell L, Pfadenhauer LM, Paul K, Goertzen A, Swain L, Sibley KM, Vis-Dunbar M, Hill MD, Raffin-Bouchal S, Tonelli M, Graham ID. How are health research partnerships assessed? A systematic review of outcomes, impacts, terminology and the use of theories, models and frameworks. Health Res Policy Syst 2022; 20:133. [PMID: 36517852 PMCID: PMC9753311 DOI: 10.1186/s12961-022-00938-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Accepted: 11/09/2022] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Accurate, consistent assessment of outcomes and impacts is challenging in the health research partnerships domain. Increased focus on tool quality, including conceptual, psychometric and pragmatic characteristics, could improve the quantification, measurement and reporting partnership outcomes and impacts. This cascading review was undertaken as part of a coordinated, multicentre effort to identify, synthesize and assess a vast body of health research partnership literature. OBJECTIVE To systematically assess the outcomes and impacts of health research partnerships, relevant terminology and the type/use of theories, models and frameworks (TMF) arising from studies using partnership assessment tools with known conceptual, psychometric and pragmatic characteristics. METHODS Four electronic databases were searched (MEDLINE, Embase, CINAHL Plus and PsycINFO) from inception to 2 June 2021. We retained studies containing partnership evaluation tools with (1) conceptual foundations (reference to TMF), (2) empirical, quantitative psychometric evidence (evidence of validity and reliability, at minimum) and (3) one or more pragmatic characteristics. Outcomes, impacts, terminology, definitions and TMF type/use were abstracted verbatim from eligible studies using a hybrid (independent abstraction-validation) approach and synthesized using summary statistics (quantitative), inductive thematic analysis and deductive categories (qualitative). Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). RESULTS Application of inclusion criteria yielded 37 eligible studies. Study quality scores were high (mean 80%, standard deviation 0.11%) but revealed needed improvements (i.e. methodological, reporting, user involvement in research design). Only 14 (38%) studies reported 48 partnership outcomes and 55 impacts; most were positive effects (43, 90% and 47, 89%, respectively). Most outcomes were positive personal, functional, structural and contextual effects; most impacts were personal, functional and contextual in nature. Most terms described outcomes (39, 89%), and 30 of 44 outcomes/impacts terms were unique, but few were explicitly defined (9, 20%). Terms were complex and mixed on one or more dimensions (e.g. type, temporality, stage, perspective). Most studies made explicit use of study-related TMF (34, 92%). There were 138 unique TMF sources, and these informed tool construct type/choice and hypothesis testing in almost all cases (36, 97%). CONCLUSION This study synthesized partnership outcomes and impacts, deconstructed term complexities and evolved our understanding of TMF use in tool development, testing and refinement studies. Renewed attention to basic concepts is necessary to advance partnership measurement and research innovation in the field. Systematic review protocol registration: PROSPERO protocol registration: CRD42021137932 https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=137932 .
Collapse
Affiliation(s)
- Kelly J. Mrklas
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10-3280 Hospital Drive NW, Calgary, AB T2N 4Z6 Canada
- Strategic Clinical Networks™, Provincial Clinical Excellence, Alberta Health Services, Calgary, AB Canada
| | - Sera Merali
- Faculty of Kinesiology, University of Calgary, Calgary, AB Canada
| | - Masood Khan
- Department of Community Health Sciences, University of Manitoba, Winnipeg, MB Canada
| | - Sumair Shergill
- Cumming School of Medicine, University of Calgary, Calgary, AB Canada
| | - Jamie M. Boyd
- Knowledge Translation Program, St Michael’s Hospital, Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, ON Canada
| | - Lorelli Nowell
- Faculty of Nursing, University of Calgary, Calgary, AB Canada
| | - Lisa M. Pfadenhauer
- Institute for Medical Information Processing, Biometry, and Epidemiology-IBE, Ludwig-Maximilian Universität Munich, Munich, Germany
- Pettenkofer School of Public Health, Munich, Germany
| | - Kevin Paul
- University of Calgary Summer Studentships Program, University of Calgary, Calgary, AB Canada
| | - Amelia Goertzen
- Faculty of Science, University of Alberta, Edmonton, AB Canada
| | - Liam Swain
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10-3280 Hospital Drive NW, Calgary, AB T2N 4Z6 Canada
| | - Kathryn M. Sibley
- Department of Community Health Sciences, University of Manitoba, Winnipeg, MB Canada
- George & Fay Yee Centre for Healthcare Innovation, University of Manitoba, Winnipeg, MB Canada
| | | | - Michael D. Hill
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10-3280 Hospital Drive NW, Calgary, AB T2N 4Z6 Canada
- Departments of Clinical Neurosciences, Medicine and Radiology, Cumming School of Medicine, University of Calgary, Calgary, AB Canada
- Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Calgary, AB Canada
| | | | - Marcello Tonelli
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB Canada
- Office of the Vice-President (Research), University of Calgary, Calgary, AB Canada
| | - Ian D. Graham
- Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, ON Canada
- School of Epidemiology and Public Health & School of Nursing, University of Ottawa, Ottawa, ON Canada
| |
Collapse
|
31
|
Hull L, Boulton R, Jones F, Boaz A, Sevdalis N. Defining, conceptualizing and evaluating pragmatic qualities of quantitative instruments measuring implementation determinants and outcomes: a scoping and critical review of the literature and recommendations for future research. Transl Behav Med 2022; 12:1049-1064. [PMID: 36318228 PMCID: PMC9677469 DOI: 10.1093/tbm/ibac064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
The pragmatic (i.e., practical) quality of quantitative implementation measures has received increased attention in the implementation science literature in recent years. Implementation measures that are judged to be pragmatic by implementation stakeholders are thought to be more likely to be applied in research and practice. Despite the need for pragmatic implementation measures, ambiguity and uncertainty regarding what constitutes a pragmatic measure remains. This study sought to identify and critically appraise the published literature to understand (i) how pragmatism is defined as a measurement construct/quality of implementation determinants and outcome instruments; (ii) how pragmatic qualities of instruments are evaluated; (iii) identify key gaps and limitations of the current evidence-base and (iv) identify recommendations for future research. We conducted a scoping review of the literature also employing methods of critical review. PubMed and PsycINFO databases, using the OVID interface, were searched for relevant articles published between January 2010 and September 2020. Articles that contained a definition and/or described characteristics of "pragmatism" as a measurement construct of quantitative implementation outcomes (as defined by Proctor's Implementation Outcomes taxonomy) and/or implementation determinants were eligible for inclusion. Nine articles met inclusion criteria. A degree of overlap in definitions and terms used to describe the pragmatic qualities of quantitative implementation determinant and outcome instruments were found. The most frequently cited descriptors of pragmatism were "not burdensome", "brief", "reliable", "valid" and "sensitive to change". 3 of the 9 included articles involved international implementation stakeholders in defining and conceptualizing pragmatism and employed specific methods to do so, including a systematic literature review, stakeholder interviews, concept mapping, and a Delphi process. All other articles defined pragmatism, with or without citing relevant literature. One article objectively assessed the pragmatic qualities, above and beyond the psychometric qualities, of implementation measures, using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The evidence base within the implementation instrumentation literature on what pragmatism is and how it might be assessed is limited. Some of the research identified in the review provides a strong foundation to build upon, by testing its applicability in other settings (including healthcare areas and countries) and among a more diverse group of stakeholders. We discuss directions for further development of the concept of pragmatism relating to the measurement of implementation determinants and outcomes.
Collapse
Affiliation(s)
| | - Richard Boulton
- Centre for Health and Social Care, St George’s, University of London and Kingston University, UK
| | - Fiona Jones
- Centre for Health and Social Care, St George’s, University of London and Kingston University, UK
| | - Annette Boaz
- Faculty of Public Health & Policy, London School of Hygiene & Tropical Medicine, London, UK
| | - Nick Sevdalis
- Centre for Implementation Science, Health Service and Population Research Department, King’s College London, London, UK
| |
Collapse
|
32
|
Becker-Haimes EM, Marcus SC, Klein MR, Schoenwald SK, Fugo PB, McLeod BD, Dorsey S, Williams NJ, Mandell DS, Beidas RS. A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy. Behav Ther 2022; 53:1191-1204. [PMID: 36229116 PMCID: PMC9680992 DOI: 10.1016/j.beth.2022.06.001] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Revised: 05/16/2022] [Accepted: 06/01/2022] [Indexed: 12/01/2022]
Abstract
Clinician fidelity to cognitive behavioral therapy (CBT) is an important mechanism by which desired clinical outcomes are achieved and is an indicator of care quality. Despite its importance, there are few fidelity measurement methods that are efficient and have demonstrated reliability and validity. Using a randomized trial design, we compared three methods of assessing CBT adherence-a core component of fidelity-to direct observation, the gold standard. Clinicians recruited from 27 community mental health agencies (n = 126; M age = 37.69 years, SD = 12.84; 75.7% female) were randomized 1:1:1 to one of three fidelity conditions: self-report (n = 41), chart-stimulated recall (semistructured interviews with the chart available; n = 42), or behavioral rehearsal (simulated role-plays; n = 43). All participating clinicians completed fidelity assessments for up to three sessions with three different clients that were recruited from clinicians' caseloads (n = 288; M age = 13.39 years SD = 3.89; 41.7% female); sessions were also audio-recorded and coded for comparison to determine the most accurate method. All fidelity measures had parallel scales that yielded an adherence maximum score (i.e., the highest-rated intervention in a session), a mean of techniques observed, and a count total of observed techniques. Results of three-level mixed effects regression models indicated that behavioral rehearsal produced comparable scores to observation for all adherence scores (all ps > .01), indicating no difference between behavioral rehearsal and observation. Self-report and chart-stimulated recall overestimated adherence compared to observation (ps < .01). Overall, findings suggested that behavioral rehearsal indexed CBT adherence comparably to direct observation, the gold-standard, in pediatric populations. Behavioral rehearsal may at times be able to replace the need for resource-intensive direct observation in implementation research and practice.
Collapse
Affiliation(s)
- Emily M Becker-Haimes
- University of Pennsylvania; Hall Mercer Community Mental Health, University of Pennsylvania Health System.
| | | | | | | | | | | | | | | | | | | |
Collapse
|
33
|
Siraneh Y, Woldie M, Birhanu Z. Assessment of Implementation Outcome Measures for Positive Deviance Approach as a New Strategy to Promote Exclusive Breastfeeding: A Psychometric Follow-Up Study. Int J Womens Health 2022; 14:1579-1601. [DOI: 10.2147/ijwh.s385662] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 11/04/2022] [Indexed: 11/16/2022] Open
|
34
|
Macauda MM, Arent MA, Sakhuja M, Yelton B, Noblet S, Fedrick D, Zona D, New C, Isenhower WD, Wandersman A, Friedman DB. Elements for successful implementation of a clinic-based health literacy intervention. Front Public Health 2022; 10:977765. [PMID: 36388330 PMCID: PMC9650509 DOI: 10.3389/fpubh.2022.977765] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Accepted: 10/07/2022] [Indexed: 01/25/2023] Open
Abstract
Improving health literacy is a national public health priority. Given the context of the COVID-19 pandemic, it is even more critical for health and medical information to be clear and understandable for patients and their families. Clinic-based programs to improve health literacy need to be pragmatic, feasible, and helpful for the implementing clinic and patients. This paper describes the development, implementation, and evaluation of a pragmatic, clinic-based health literacy intervention in a safety-net clinic that serves uninsured and indigent patients. Study methods are guided by a previous pilot study and components recommended for pragmatic interventions. An electronic readiness assessment was distributed to out-patient clinics affiliated with a statewide hospital association. The AskMe3 tool was used for the intervention as it is evidence informed and relatively easy to implement. Implementation included ongoing dialogue between the clinic and the academic research team. Within the implementing clinic, data collected from patients via verbally administered questionnaires was analyzed using descriptive statistics and chi-squares. Interview data collected from the clinic director was analyzed qualitatively for themes. The implementing clinic had some of the lowest average scores of the 34 clinics who participated in the initial readiness assessment. Despite this, they were able to successfully implement the health literacy intervention during a global pandemic. Eighty-eight participants completed patient questionnaires at this clinic. Most patients (96%) agreed the AskMe3 questions helped them talk with the doctor or nurse at their current appointment. Most (99%) also perceived the AskMe3 tool to be very helpful when used in a clinical setting. The clinic director offered that the staff initially thought the intervention would be difficult to implement. However, implementation by clinic volunteers with encouragement and prioritization of health literacy by the clinic director contributed to success. When considering interventions for clinical settings, a pragmatic approach can help with selection and implementation of a program that fits with the realities on the ground. Further, frequent technical assistance can help resolve implementation barriers. Interventions utilizing tools such as AskMe3, because of their simplicity, allow creative solutions to capacity issues for clinics who see a need for health literacy improvements.
Collapse
Affiliation(s)
- Mark M. Macauda
- Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States,Center for Applied Research and Evaluation, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States,*Correspondence: Mark M. Macauda
| | - Michelle A. Arent
- Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States
| | - Mayank Sakhuja
- Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States
| | - Brooks Yelton
- Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States
| | - Samuel Noblet
- Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States,Prevention Research Center, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States
| | | | - Diana Zona
- South Carolina Hospital Association, Columbia, SC, United States
| | - Cyndi New
- South Carolina Hospital Association, Columbia, SC, United States
| | | | | | - Daniela B. Friedman
- Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC, United States
| |
Collapse
|
35
|
Lewis CC, Klasnja P, Lyon AR, Powell BJ, Lengnick-Hall R, Buchanan G, Meza RD, Chan MC, Boynton MH, Weiner BJ. The mechanics of implementation strategies and measures: advancing the study of implementation mechanisms. Implement Sci Commun 2022; 3:114. [PMID: 36273224 PMCID: PMC9588220 DOI: 10.1186/s43058-022-00358-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 09/28/2022] [Indexed: 11/08/2022] Open
Abstract
BACKGROUND There is a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery. Until this gap is addressed, scientific knowledge and practical guidance about which implementation strategies to use in which contexts will remain elusive. This research project aims to identify plausible strategy-mechanism linkages, develop causal models for mechanism evaluation, produce measures needed to evaluate such linkages, and make these models, methods, and measures available in a user-friendly website. The specific aims are as follows: (1) build a database of strategy-mechanism linkages and associated causal pathway diagrams, (2) develop psychometrically strong, pragmatic measures of mechanisms, and (3) develop and disseminate a website of implementation mechanisms knowledge for use by diverse stakeholders. METHODS For the first aim, a combination of qualitative inquiry, expert panel methods, and causal pathway diagramming will be used to identify and confirm plausible strategy-mechanism linkages and articulate moderators, preconditions, and proximal and distal outcomes associated with those linkages. For the second aim, rapid-cycle measure development and testing methods will be employed to create reliable, valid, pragmatic measures of six mechanisms of common strategies for which no high-quality measures exist. For the third aim, we will develop a user-friendly website and searchable database that incorporates user-centered design, disseminating the final product using social marketing principles. DISCUSSION Once strategy-mechanism linkages are identified using this multi-method approach, implementation scientists can use the searchable database to develop tailored implementation strategies and generate more robust evidence about which strategies work best in which contexts. Moreover, practitioners will be better able to select implementation strategies to address their specific implementation problems. New horizons in implementation strategy development, optimization, evaluation, and deployment are expected to be more attainable as a result of this research, which will lead to enhanced implementation of evidence-based interventions for cancer control, and ultimately improvements in patient outcomes.
Collapse
Affiliation(s)
- Cara C Lewis
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.
| | - Predrag Klasnja
- School of Information, University of Michigan, Ann Arbor, MI, USA
| | - Aaron R Lyon
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Byron J Powell
- Center for Mental Health Services Research, Brown School, Washington University in St. Louis, St Louis, MO, USA
- Center for Dissemination & Implementation, Institute for Public Health, Washington University in St. Louis, St. Louis, MO, USA
- Division of Infectious Diseases, John T. Milliken Department of Medicine, School of Medicine, Washington University in St. Louis, St. Louis, MO, USA
| | - Rebecca Lengnick-Hall
- Center for Mental Health Services Research, Brown School, Washington University in St. Louis, St Louis, MO, USA
| | - Gretchen Buchanan
- Center for Mental Health Services Research, Brown School, Washington University in St. Louis, St Louis, MO, USA
| | - Rosemary D Meza
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Michelle C Chan
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Marcella H Boynton
- Division of General Medicine and Clinical Epidemiology, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, NC, USA
- NC TraCS Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
- Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Bryan J Weiner
- Department of Global Health, University of Washington, Seattle, WA, USA
| |
Collapse
|
36
|
Hoy S, Helgadóttir B, Norman Å. Quantitative Measurements for Factors Influencing Implementation in School Settings: Protocol for A Systematic Review and A Psychometric and Pragmatic Analysis. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:12726. [PMID: 36232024 PMCID: PMC9564866 DOI: 10.3390/ijerph191912726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 10/01/2022] [Accepted: 10/02/2022] [Indexed: 06/16/2023]
Abstract
INTRODUCTION In order to address the effectiveness and sustainability of school-based interventions, there is a need to consider the factors affecting implementation success. The rapidly growing field of implementation-focused research is struggling to determine how to assess and measure implementation-relevant constructs. Earlier research has identified the need for strong psychometric and pragmatic measures. The aims of this review are therefore to (i) systematically review the literature to identify measurements of the factors influencing implementations which have been developed or adapted in school settings, (ii) describe each measurement's psychometric and pragmatic properties, (iii) describe the alignment between each measurement and the corresponding domain and/or construct of the Consolidated Framework for Implementation Research (CFIR). METHODS Six databases (Medline, ERIC, PsycInfo, Cinahl, Embase, and Web of Science) will be searched for peer-reviewed articles reporting on school settings, published from the year 2000. The identified measurements will be mapped against the CFIR, and analyzed for their psychometric and pragmatic properties. DISCUSSION By identifying measurements that are psychometrically and pragmatically impactful in the field, this review will contribute to the identification of feasible, effective, and sustainable implementation strategies for future research in school settings.
Collapse
Affiliation(s)
- Sara Hoy
- Department of Movement, Culture, and Society, The Swedish School of Sport and Health Sciences (GIH), 114 86 Stockholm, Sweden
| | - Björg Helgadóttir
- Department of Physical Activity and Health, The Swedish School of Sport and Health Sciences (GIH), 114 33 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institute, Tomtebodavägen 18A, 171 77 Stockholm, Sweden
| | - Åsa Norman
- Department of Clinical Neuroscience, Karolinska Institute, Tomtebodavägen 18A, 171 77 Stockholm, Sweden
- Department of Psychology, Stockholm University, 106 91 Stockholm, Sweden
| |
Collapse
|
37
|
Lachance K, Štětinová K, Rieske R, Peer S. Repetitive Behavior Scale for Early Childhood (RBS-EC): Psychometrics and Developmental Effects with a Community Sample. Child Psychiatry Hum Dev 2022; 53:863-875. [PMID: 33871734 DOI: 10.1007/s10578-021-01166-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 03/30/2021] [Indexed: 11/30/2022]
Abstract
Restricted, repetitive behaviors and interests (RRBIs) occur commonly in young children in the course of typical development, but too frequent or persistent RRBIs have been linked to a variety of psychosocial and neurodevelopmental disorders. Unfortunately, a lack of psychometrically sound, feasible instruments stymies RRBI study and intervention in early childhood. Therefore, this study validated an extant RRBI measure (i.e., Repetitive Behavior Scale for Early Childhood; RBS-EC) for predominately neurotypical community children ages 2-7 years via an MTurk survey of 300 caregivers. Confirmatory factor analyses best supported a 4-factor model for the RBS-EC (i.e., repetitive motor, rituals and routines, restricted interests, self-directed behaviors). RBS-EC scores varied significantly with age and gender; namely, RRBIs were typically (1) less frequent but more problematic with older versus younger children and (2) more frequent and problematic with boys versus girls. Findings extend the RBS-EC's utility in RRBI assessment and treatment during early childhood.
Collapse
Affiliation(s)
- Kathryn Lachance
- Department of Psychology, Idaho State University, 921 S 8th Ave STOP 8201, Pocatello, ID, 83209, USA
| | - Karolina Štětinová
- Department of Psychology, Idaho State University, 921 S 8th Ave STOP 8201, Pocatello, ID, 83209, USA
| | - Robert Rieske
- Department of Psychology, Idaho State University, 921 S 8th Ave STOP 8201, Pocatello, ID, 83209, USA.
| | - Samuel Peer
- Department of Psychology, Idaho State University, 921 S 8th Ave STOP 8201, Pocatello, ID, 83209, USA
| |
Collapse
|
38
|
Hjort AV, Kuipers MAG, Stage M, Pisinger C, Klinker CD. Intervention Activities Associated with the Implementation of a Comprehensive School Tobacco Policy at Danish Vocational Schools: A Repeated Cross-Sectional Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:12489. [PMID: 36231788 PMCID: PMC9565121 DOI: 10.3390/ijerph191912489] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Revised: 09/24/2022] [Accepted: 09/26/2022] [Indexed: 06/16/2023]
Abstract
School tobacco policies are often poorly implemented, which may explain their limited effectiveness. Further, constructs to measure implementation outcomes of school tobacco policies are missing. The Smoke-Free Vocational Schools intervention was designed to stimulate the implementation of a comprehensive school tobacco policy into routine practice. This study (1) developed implementation fidelity outcomes measures for the school tobacco policy and (2) examined associations between intervention activities and implementation fidelity at two time points. We applied a repeated cross-sectional survey study design across seven schools: the first time point was >5 months after the policy was established and the second time point > 14 months after policy establishment. The dependent/outcome variables were four binary fidelity domains as well as a total score across domains. A total of six intervention activities were measured among either students (e.g., new school-break facilities) or staff/managers (e.g., a joint workshop before policy implementation). Associations were analyzed separately for students and staff/managers using generalized linear mixed models, adjusted for confounders. A total of n = 2674 students and n = 871 staff/managers participated. The total implementation fidelity scores increased over time among both students and staff/managers. Three intervention activities were consistently associated with the total implementation fidelity score, including: new school-break facilities (BT1 = 0.08, 95% CI = 0.03; 0.12; BT2 = 0.07, 95% CI = 0.04-0.50), the joint workshop before policy implementation (BT1 = 0.13, 95% CI = 0.02; 0.25; BT2 = 0.13, 95% CI = 0.004; 0.24), and internalization of fixed procedures for enforcement (BT1 = 0.19, 95% CI = 0.13-0.26; BT2 = 0.16, 95% CI = 0.13-0.26). These findings can be applied by schools and other actors in practice. The developed implementation fidelity outcomes measures can be applied in future research on school tobacco policies.
Collapse
Affiliation(s)
- Anneke Vang Hjort
- Health Promotion Research, Steno Diabetes Center Copenhagen—Copenhagen University Hospital, 2730 Herlev, Denmark
- National Institute of Public Health, University of Southern Denmark, 5230 Odense, Denmark
| | - Mirte A. G. Kuipers
- Department of Public and Occupational Health, Amsterdam UMC Location University of Amsterdam, 1105 AZ Amsterdam, The Netherlands
- Health Behaviors & Chronic Diseases, Amsterdam Public Health Research Institute, 1081 BT Amsterdam, The Netherlands
| | - Maria Stage
- The Danish Cancer Society, 2100 Copenhagen, Denmark
| | - Charlotta Pisinger
- Center for Clinical Research and Prevention, Capital Region, 2000 Frederiksberg, Denmark
- The Danish Heart Foundation, 1120 Copenhagen, Denmark
| | - Charlotte Demant Klinker
- Health Promotion Research, Steno Diabetes Center Copenhagen—Copenhagen University Hospital, 2730 Herlev, Denmark
| |
Collapse
|
39
|
Elks M, Young J, Kearney L, Bernard A. The impact of an autonomous nurse-led high-flow nasal cannula oxygen protocol on clinical outcomes of infants with bronchiolitis. J Clin Nurs 2022. [PMID: 36164265 DOI: 10.1111/jocn.16525] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Revised: 08/01/2022] [Accepted: 08/14/2022] [Indexed: 11/26/2022]
Abstract
OBJECTIVE To investigate the relationship of the implementation of a nurse-led high-flow nasal cannula oxygen protocol on the clinical outcomes of infants with bronchiolitis in a regional paediatric unit. BACKGROUND Bronchiolitis is a common lower respiratory illness and is the leading cause for hospitalisation of infants globally. Standard care involves the provision of supportive measures. Historically, supplemental oxygen was provided by low-flow nasal cannula. High-flow nasal cannula oxygen has been increasingly adopted despite limited evidence of its efficacy. METHODS This study employed non-equivalent, post-implementation only design to explore clinical outcomes of infants with bronchiolitis admitted for high-flow nasal cannula oxygen therapy. The study compared infants in the 24 months before and after the initiation of a high-flow nasal cannula protocol. The primary clinical outcome was length of stay, secondary outcomes included time on high flow, weaning time, escalation of care and time outside of physiological parameters. Implementation strategy evaluation was measured by compliance with applying the protocol, reported as episodes of variance, and duration of variance. The StaRI checklist was selected as the most appropriate reporting guideline. RESULTS A total of 80 patients were admitted with bronchiolitis and received high-flow nasal cannula oxygen therapy during a 48-month period; 37 patients were prior, and 43 after, the introduction of a nurse-led high-flow nasal cannula protocol. Length of stay was significantly reduced in the post-implementation group compared to the historical control group (83.8 vs. 61.3 h). Time on high flow and weaning time was decreased in the post-implementation group compared to the control group (33.5 vs. 26.7 h and 26 vs.12.25 h, respectively); however, these did not reach statistical significance. There was varied application of the HFNC protocol. CONCLUSIONS The implementation of a nurse-led high-flow nasal cannula protocol was associated with a reduced length of stay. RELEVANCE TO CLINICAL PRACTICE This study demonstrated that infants with bronchiolitis that were treated with a nurse-led high-flow nasal cannula (HFNC) therapy protocol had positive effects on clinical outcomes including a shorter length of stay than compared with those with physician-directed care in a regional paediatric unit. A weight-based (2 L/kg) HFNC therapy was safely administered to infants with bronchiolitis in a regional hospital paediatric ward with no paediatric intensive care unit (PICU).
Collapse
Affiliation(s)
- Michelle Elks
- Sunshine Coast Hospital and Health Service, University of the Sunshine Coast, Sunshine Coast, Queensland, Australia
| | - Jeanine Young
- School of Nursing, Midwifery and Paramedicine, University of the Sunshine Coast, Sunshine Coast, Queensland, Australia
| | - Lauren Kearney
- School of Nursing, Midwifery & Paramedicine University of the Sunshine Coast, Sunshine Coast, Queensland, Australia
| | - Anne Bernard
- Institute for Molecular Bioscience, The University of Queensland, Brisbane, Australia
| |
Collapse
|
40
|
Rabin BA, Cakici J, Golden CA, Estabrooks PA, Glasgow RE, Gaglio B. A citation analysis and scoping systematic review of the operationalization of the Practical, Robust Implementation and Sustainability Model (PRISM). Implement Sci 2022; 17:62. [PMID: 36153628 PMCID: PMC9509575 DOI: 10.1186/s13012-022-01234-3] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 08/30/2022] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND The Practical, Robust Implementation and Sustainability Model (PRISM) was developed in 2008 as a contextually expanded version of the broadly used Reach, Adoption, Effectiveness, Implementation, and Maintenance (RE-AIM) framework. PRISM provides researchers a pragmatic and intuitive model to improve translation of research interventions into clinical and community practice. Since 2008, the use of PRISM increased across diverse topics, populations, and settings. This citation analysis and scoping systematic review aimed to assess the use of the PRISM framework and to make recommendations for future research. METHODS A literature search was conducted using three databases (PubMed, Web of Science, Scopus) for the period of 2008 and September 2020. After exclusion, reverse citation searches and invitations to experts in the field were used to identify and obtain recommendations for additional articles not identified in the original search. Studies that integrated PRISM into their study design were selected for full abstraction. Unique research studies were abstracted for information on study characteristics (e.g., setting/population, design), PRISM contextual domains, and RE-AIM outcomes. RESULTS A total of 180 articles were identified to include PRISM to some degree. Thirty-two articles representing 23 unique studies integrated PRISM within their study design. Study characteristics varied widely and included studies conducted in diverse contexts, but predominately in high-income countries and in clinical out-patient settings. With regards to use, 19 used PRISM for evaluation, 10 for planning/development, 10 for implementation, four for sustainment, and one for dissemination. There was substantial variation across studies in how and to what degree PRISM contextual domains and RE-AIM outcomes were operationalized and connected. Only two studies directly connected individual PRISM context domains with RE-AIM outcomes, and another four included RE-AIM outcomes without direct connection to PRISM domains. CONCLUSIONS This is the first systematic review of the use of PRISM in various contexts. While there were low levels of 'integrated' use of PRISM and few reports on linkage to RE-AIM outcomes, most studies included important context domains of implementation and sustainability infrastructure and external environment. Recommendations are provided for more consistent and comprehensive use of and reporting on PRISM to inform both research and practice on contextual factors in implementation.
Collapse
Affiliation(s)
- Borsika A. Rabin
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, 9500 Gillman Drive, La Jolla, CA 92037 USA
- UC San Diego Altman Clinical and Translational Research Center Dissemination and Implementation Science Center, UC San Diego, 9500 Gillman Drive, La Jolla, CA 92037 USA
- Dissemination and Implementation Science Program, Adult & Child Center for Health Outcomes Research & Delivery Science; and Department of Family Medicine, University of Colorado, 1635 Aurora Ct, Aurora, CO 80045 USA
| | - Julie Cakici
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, 9500 Gillman Drive, La Jolla, CA 92037 USA
- UC San Diego Altman Clinical and Translational Research Center Dissemination and Implementation Science Center, UC San Diego, 9500 Gillman Drive, La Jolla, CA 92037 USA
- School of Public Health, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182 USA
| | - Caitlin A. Golden
- University of Nebraska Medical Center, 42nd and Emile St, Omaha, NE 68198 USA
| | - Paul A. Estabrooks
- Health & Kinesiology, College of Health, University of Utah, 248 HPER North, 260 South 1850 East, Salt Lake City, UT 84112 USA
| | - Russell E. Glasgow
- Dissemination and Implementation Science Program, Adult & Child Center for Health Outcomes Research & Delivery Science; and Department of Family Medicine, University of Colorado, 1635 Aurora Ct, Aurora, CO 80045 USA
| | - Bridget Gaglio
- Patient-Centered Research, Evidera, 7101 Wisconsin Ave, Suite 1400, Bethesda, MD 20814 USA
| |
Collapse
|
41
|
Kuo GM, Trinkley KE, Rabin B. Research and Scholarly Methods: Implementation Science Studies. JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY 2022; 5:995-1004. [PMID: 36212610 PMCID: PMC9534307 DOI: 10.1002/jac5.1673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 06/14/2022] [Indexed: 11/08/2022]
Abstract
Traditional research focuses on efficacy or effectiveness of interventions but lacks evaluation of strategies needed for equitable uptake, scalable implementation, and sustainable evidence-based practice transformation. The purpose of this introductory review is to describe key implementation science (IS) concepts as they apply to medication management and pharmacy practice, and to provide guidance on literature review with an IS lens. There are five key ingredients of IS, including: (1) evidence-based intervention; (2) implementation strategies; (3) IS theory, model, or framework; (4) IS outcomes and measures; and (5) stakeholder engagement, which is key to a successful implementation. These key ingredients apply across the three stages of IS research: (1) pre-implementation; (2) implementation; and (3) sustainment. A case example using a combination of IS models, PRISM (Practical, Robust Implementation and Sustainability model) and RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance), is included to describe how an IS study is designed and conducted. This case is a cluster randomized trial comparing two clinical decision support tools to improve guideline-concordant prescribing for patients with heart failure and reduced ejection fraction. The review also includes information on the Standards for Reporting Implementation Studies (StaRI), which is used for literature review and reporting of IS studies,as well as IS-related learning resources.
Collapse
Affiliation(s)
- Grace M Kuo
- Texas Tech University Health Sciences Center and Professor Emerita at University of California San Diego; Address: 1300 S. Coulter Street, Suite 104, Amarillo, TX 79106
| | - Katy E Trinkley
- University of Colorado Skaggs Schools of Medicine and Pharmacy and Pharmaceutical Sciences at the Anschutz Medical Campus; Aurora, Colorado
| | - Borsika Rabin
- Herbert Wertheim School of Public Health and Human Longevity Science and Co-Director of the UC San Diego ACTRI Dissemination and Implementation Science Center at University of California San Diego; La Jolla, California
| |
Collapse
|
42
|
Beidas RS, Dorsey S, Lewis CC, Lyon AR, Powell BJ, Purtle J, Saldana L, Shelton RC, Stirman SW, Lane-Fall MB. Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem. Implement Sci 2022; 17:55. [PMID: 35964095 PMCID: PMC9375077 DOI: 10.1186/s13012-022-01226-3] [Citation(s) in RCA: 54] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 07/20/2022] [Indexed: 12/26/2022] Open
Abstract
BACKGROUND Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers. MAIN BODY Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches. CONCLUSION We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.
Collapse
|
43
|
de Bles NJ, Gast DAA, van der Slot AJC, Didden R, van Hemert AM, Rius-Ottenheim N, Giltay EJ. Lessons learned from two clinical trials on nutritional supplements to reduce aggressive behaviour. J Eval Clin Pract 2022; 28:607-614. [PMID: 35040231 PMCID: PMC9543803 DOI: 10.1111/jep.13653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/29/2021] [Accepted: 12/14/2021] [Indexed: 12/01/2022]
Abstract
BACKGROUND Setting up and conducting a randomised controlled trial (RCT) has many challenges-particularly trials that include vulnerable individuals with behavioural problems or who reside in facilities that focus on care as opposed to research. These populations are underrepresented in RCTs. APPROACH In our paper, we describe the challenges and practical lessons learned from two RCTs in two care settings involving long-stay psychiatric inpatients and people with intellectual disabilities. We describe five main difficulties and how these were overcome: (1) multisite setting, (2) inclusion of vulnerable participants, (3) nutritional supplements and placebos, (4) assessment of behavioural outcomes, and (5) collecting bio samples. CONCLUSIONS By sharing these practical experiences, we hope to inform other researchers how to optimally design their trials, while avoiding and minimising the difficulties that we encountered, and to facilitate the implementation of a trial. Both trials were registered in the Clinical Trials Register (RCT A: NCT02498106; RCT B: NCT03212092).
Collapse
Affiliation(s)
- Nienke J de Bles
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands
| | - David A A Gast
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands.,Gemiva-SVG Group, Gouda, The Netherlands
| | - Abe J C van der Slot
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands
| | - Robert Didden
- Behavioural Science Institute, Radboud University, Nijmegen, The Netherlands.,Trajectum, Zwolle, The Netherlands
| | - Albert M van Hemert
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands
| | | | - Erik J Giltay
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands
| |
Collapse
|
44
|
Guerin R, Glasgow R, Tyler A, Rabin B, Huebschmann A. Methods to improve the translation of evidence-based interventions: A primer on dissemination and implementation science for occupational safety and health researchers and practitioners. SAFETY SCIENCE 2022; 152:105763. [PMID: 37854304 PMCID: PMC10583726 DOI: 10.1016/j.ssci.2022.105763] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/20/2023]
Abstract
Objective A limited focus on dissemination and implementation (D&I) science has hindered the uptake of evidence-based interventions (EBIs) that reduce workplace morbidity and mortality. D&I science methods can be used in the occupational safety and health (OSH) field to advance the adoption, implementation, and sustainment of EBIs for complex workplaces. These approaches should be responsive to contextual factors, including the needs of partners and beneficiaries (such as employers, employees, and intermediaries). Methods By synthesizing seminal literature and texts and leveraging our collective knowledge as D&I science and/or OSH researchers, we developed a D&I science primer for OSH. First, we provide an overview of common D&I terminology and concepts. Second, we describe several key and evolving issues in D&I science: balancing adaptation with intervention fidelity and specifying implementation outcomes and strategies. Next, we review D&I theories, models, and frameworks and offer examples for applying these to OSH research. We also discuss widely used D&I research designs, methods, and measures. Finally, we discuss future directions for D&I science application to OSH and provide resources for further exploration. Results We compiled a D&I science primer for OSH appropriate for practitioners and evaluators, especially those newer to the field. Conclusion This article fills a gap in the OSH research by providing an overview of D&I science to enhance understanding of key concepts, issues, models, designs, methods and measures for the translation into practice of effective OSH interventions to advance the safety, health and well-being of workers.
Collapse
Affiliation(s)
- R.J. Guerin
- Division of Science Integration, National Institute for
Occupational Safety and Health, Centers for Disease Control and Prevention, 1090
Tusculum Ave., MS C-10, Cincinnati, OH 45226, USA
| | - R.E. Glasgow
- Dissemination and Implementation Science Program,
University of Colorado Adult and Child Center for Outcomes Research and Delivery
Science, Anschutz Medical Campus, Aurora, CO 80045, USA
- University of Colorado Department of Family Medicine,
Anschutz Medical Campus, Aurora, CO 80045, USA
| | - A. Tyler
- Dissemination and Implementation Science Program,
University of Colorado Adult and Child Center for Outcomes Research and Delivery
Science, Anschutz Medical Campus, Aurora, CO 80045, USA
- University of Colorado Department of Pediatrics, Section
of Hospital Medicine, Anschutz Medical Campus, Aurora, CO 80045, USA
| | - B.A. Rabin
- Herbert Wertheim School of Public Health and Human
Longevity Science, University of California San Diego, La Jolla, CA 92037, USA
- UC San Diego Altman Clinical and Translational Research
Institute Dissemination and Implementation Science Center, University of California
San Diego, La Jolla, CA 92037, USA
| | - A.G. Huebschmann
- Dissemination and Implementation Science Program,
University of Colorado Adult and Child Center for Outcomes Research and Delivery
Science, Anschutz Medical Campus, Aurora, CO 80045, USA
- University of Colorado Division of General Internal
Medicine, Anschutz Medical Campus, Aurora, CO 80045, USA
- University of Colorado Ludeman Family Center for
Women’s Health Research, Anschutz Medical Campus, Aurora, CO 80045, USA
| |
Collapse
|
45
|
Lyon AR, Corbin CM, Brown EC, Ehrhart MG, Locke J, Davis C, Picozzi E, Aarons GA, Cook CR. Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS). Implement Sci 2022; 17:48. [PMID: 35854385 PMCID: PMC9295535 DOI: 10.1186/s13012-022-01222-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2021] [Accepted: 06/29/2022] [Indexed: 11/25/2022] Open
Abstract
Background Strategic implementation leadership is a critical determinant of successful implementation, hypothesized to create a more supportive implementation climate conducive to the adoption and use of evidence-based practices. Implementation leadership behaviors may vary significantly across contexts, necessitating studies that examine the validity of established measurement tools in novel health service delivery sectors. The education sector is the most common site for delivering mental health services to children and adolescents in the USA, but research focused on implementation leadership in schools is in the early phases, and there is a need for adaptation and expansion of instruments in order to tailor to the school context. The current study adapted and validated the School Implementation Leadership Scale (SILS) (based on the Implementation Leadership Scale) in a sample of elementary school personnel from six school districts who were implementing one of two well-established prevention programs for supporting children’s mental health. Methods Participants were 441 public school teachers from 52 elementary schools in the Midwest and West Coast of the USA. Participants completed a survey that contained: (1) an adapted and expanded version of the SILS with additional items generated for four existing subscales as well as three new subscales (communication, vision/mission, and availability), and (2) additional tools to evaluate convergent and divergent validity (i.e., measures of general/molar leadership and teaching attitudes). Data underwent (1) examination of item characteristic curves to reduce items and ensure a pragmatic instrument, (2) confirmatory factor analyses to establish structural validity, and (3) evaluation of convergent and divergent validity. Results Item reduction analyses resulted in seven subscales of three items each. Results indicated acceptable fit for a seven-factor structural model (CFI = .995, TLI = .99, RMSEA = .07, SRMR = 0.02). Second-order factor loadings were high (λ = .89 to .96), suggesting that the SILS subscales comprise a higher-order implementation leadership factor. All subscales demonstrated good inter-item reliability (α = .91–.96). Convergent and divergent validity results were generally as hypothesized, with moderate to high correlations between SILS subscales and general leadership, moderate correlations with teaching attitudes, and low correlations with school demographics. Conclusions Overall, results provided strong structural, convergent, and divergent validity evidence for the 21-item, 7-factor SILS instrument. Implications for the measurement of implementation leadership in schools are discussed, as well as strategies to support leaders to enhance their strategic behaviors related to the implementation of mental health prevention programs (e.g., adaptation of existing leadership-focused implementation strategies). Supplementary Information The online version contains supplementary material available at 10.1186/s13012-022-01222-7.
Collapse
Affiliation(s)
- Aaron R Lyon
- Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA.
| | - Catherine M Corbin
- Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA
| | - Eric C Brown
- Department of Public Health Sciences, Miller School of Medicine, University of Miami, 1120 NW 14th Street, Office, 104, Miami, FL, 33136, USA
| | - Mark G Ehrhart
- Department of Psychology, University of Central Florida, P.O. Box 161390, Orlando, FL, 32816-1390, USA
| | - Jill Locke
- Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA
| | - Chayna Davis
- Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA
| | - Elissa Picozzi
- Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, USA.,UC San Diego ACTRI Dissemination and Implementation Science Center, San Diego, USA.,Child and Adolescent Services Research Center, San Diego, USA
| | - Clayton R Cook
- Department of Organizational Leadership and Policy Development, University of Minnesota, 206 Burton Hall-178 Pillsbury Drive SE, Minneapolis, MN, 55455, USA
| |
Collapse
|
46
|
Corbin CM, Hugh ML, Ehrhart MG, Locke J, Davis C, Brown EC, Cook CR, Lyon AR. Teacher Perceptions of Implementation Climate Related to Feasibility of Implementing Schoolwide Positive Behavior Supports and Interventions. SCHOOL MENTAL HEALTH 2022. [DOI: 10.1007/s12310-022-09528-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
AbstractFor Schoolwide Positive Behavior Interventions and Supports (SWPBIS) to be effective, educators must use the practices as intended. Whether a teacher uses a practice as intended can depend, in part, on how feasible the practice is perceived to be. Identifying malleable factors associated with teachers’ perceptions of SWPBIS feasibility can help schools identify targeted supports to improve feasibility. Implementation climate, or the shared perception among implementers that their school supports implementation efforts, is known to promote high quality implementation. However, little is known about how individuals’ perceptions of their school’s implementation climate may influence their perceptions of feasibility. The lack of empirical evidence points to a need to explore whether educators’ shared and individual perceptions of implementation climate influence feasibility of implementing SWPBIS. Therefore, this study examines the link between teachers’ individual and shared perceptions of implementation climate related to the feasibility of implementing SWPBIS in a sample of 348 K-5 general education teachers across 39 elementary schools in the pacific northwestern United States. Results indicate that teachers who experience their schools’ implementation climate as positive are more likely to report SWPBIS as feasible, controlling for teachers’ shared perceptions of implementation climate. Implications for schools aiming to improve their implementation of SWPBIS, including the development of individualized implementation supports (e.g., tailoring implementation strategies to support each and every teacher), are discussed.
Collapse
|
47
|
Fishleder S, Harris JR, Petrescu-Prahova M, Kohn M, Helfrich CD. Development and Feasibility Testing of the Clinical-Community Linkage Self-Assessment Survey for Community Organizations. Front Public Health 2022; 10:797468. [PMID: 35669755 PMCID: PMC9163549 DOI: 10.3389/fpubh.2022.797468] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Accepted: 04/05/2022] [Indexed: 11/22/2022] Open
Abstract
Introduction Clinical-community linkages (CCLs) can improve health, but few instruments exist to evaluate these partnerships. To address this gap, we develop and test the Clinical-Community Linkage Self-Assessment Survey (CCL Self-Assessment). Materials and Methods We built on an existing framework and conducted a literature review to guide the design of our survey, and obtained feedback from academic, clinical, and community-based experts. To pretest the instrument, we conducted 10 think-aloud interviews with community-based health-promotion organizations. We performed feasibility testing with 38 staff from 20 community organizations, followed by criterion-validity testing. Results The 15-item final instrument includes five domains: Nature of the Relationship, Communication, Referral Process, Feedback Loop, and Timeliness. Expert feedback included keeping the CCL Self-Assessment brief and actionable. Think-aloud interviews produced a range of revisions related to item wording, instructions, brevity, and formatting. Feasibility testing showed high response rate and ease of administration. Sites scoring high on the CCL Self-Assessment also scored high on the criterion measure. Discussion We demonstrate feasibility, as well as face, content, construct, and criterion validity. Initial results suggest the CCL Self-Assessment survey may be used by community organizations to identify strengths and weaknesses of their linkages. Next steps include additional statistical validation and testing to determine how the CCL Self-Assessment survey works in the field as well as providing specific tools to improve linkages.
Collapse
Affiliation(s)
- Sarah Fishleder
- Health Promotion Research Center, Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA, United States
- *Correspondence: Sarah Fishleder
| | - Jeffrey R. Harris
- Health Promotion Research Center, Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA, United States
| | - Miruna Petrescu-Prahova
- Health Promotion Research Center, Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA, United States
| | - Marlana Kohn
- Health Promotion Research Center, Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA, United States
| | - Christian D. Helfrich
- US Department of Veterans Affairs Health Services Research & Development, Washington, DC, United States
| |
Collapse
|
48
|
Aldridge LR, Kemp CG, Bass JK, Danforth K, Kane JC, Hamdani SU, Marsch LA, Uribe-Restrepo JM, Nguyen AJ, Bolton PA, Murray LK, Haroz EE. Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries. Implement Sci Commun 2022; 3:54. [PMID: 35590428 PMCID: PMC9118868 DOI: 10.1186/s43058-022-00301-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2021] [Accepted: 04/26/2022] [Indexed: 01/18/2023] Open
Abstract
BACKGROUND Existing implementation measures developed in high-income countries may have limited appropriateness for use within low- and middle-income countries (LMIC). In response, researchers at Johns Hopkins University began developing the Mental Health Implementation Science Tools (mhIST) in 2013 to assess priority implementation determinants and outcomes across four key stakeholder groups-consumers, providers, organization leaders, and policy makers-with dedicated versions of scales for each group. These were field tested and refined in several contexts, and criterion validity was established in Ukraine. The Consumer and Provider mhIST have since grown in popularity in mental health research, outpacing psychometric evaluation. Our objective was to establish the cross-context psychometric properties of these versions and inform future revisions. METHODS We compiled secondary data from seven studies across six LMIC-Colombia, Myanmar, Pakistan, Thailand, Ukraine, and Zambia-to evaluate the psychometric performance of the Consumer and Provider mhIST. We used exploratory factor analysis to identify dimensionality, factor structure, and item loadings for each scale within each stakeholder version. We also used alignment analysis (i.e., multi-group confirmatory factor analysis) to estimate measurement invariance and differential item functioning of the Consumer scales across the six countries. RESULTS All but one scale within the Provider and Consumer versions had Cronbach's alpha greater than 0.8. Exploratory factor analysis indicated most scales were multidimensional, with factors generally aligning with a priori subscales for the Provider version; the Consumer version has no predefined subscales. Alignment analysis of the Consumer mhIST indicated a range of measurement invariance for scales across settings (R2 0.46 to 0.77). Several items were identified for potential revision due to participant nonresponse or low or cross- factor loadings. We found only one item, which asked consumers whether their intervention provider was available when needed, to have differential item functioning in both intercept and loading. CONCLUSION We provide evidence that the Consumer and Provider versions of the mhIST are internally valid and reliable across diverse contexts and stakeholder groups for mental health research in LMIC. We recommend the instrument be revised based on these analyses and future research examine instrument utility by linking measurement to other outcomes of interest.
Collapse
Affiliation(s)
- Luke R Aldridge
- Johns Hopkins University Bloomberg School of Public Health, Baltimore, USA.
| | - Christopher G Kemp
- Johns Hopkins University Bloomberg School of Public Health, Baltimore, USA
| | - Judith K Bass
- Johns Hopkins University Bloomberg School of Public Health, Baltimore, USA
| | - Kristen Danforth
- University of Washington Department of Global Health, Seattle, USA
| | - Jeremy C Kane
- Columbia University Mailman School of Public Health, New York, USA
| | - Syed U Hamdani
- University of Liverpool Institute of Population Health, Liverpool, UK
| | - Lisa A Marsch
- Dartmouth Center for Technology & Behavioral Health, Lebanon, USA
| | - José M Uribe-Restrepo
- Pontificia Universidad Javeriana Department of Psychiatry and Mental Health, Bogota, Colombia
| | - Amanda J Nguyen
- University of Virginia School of Education and Human Development, Charlottesville, USA
| | - Paul A Bolton
- Johns Hopkins University Bloomberg School of Public Health, Baltimore, USA
| | - Laura K Murray
- Johns Hopkins University Bloomberg School of Public Health, Baltimore, USA
| | - Emily E Haroz
- Johns Hopkins University Bloomberg School of Public Health, Baltimore, USA
| |
Collapse
|
49
|
Heid AR, Abbott KM, Rovine MJ, Eshraghi K, Madrigal C, Crumbie V, Van Haitsma K. The Stability of Nursing Home Residents' Ratings of Importance of Recreation Preferences Over One Year. J Appl Gerontol 2022; 41:1942-1951. [PMID: 35506670 DOI: 10.1177/07334648221089239] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Long-term stability of nursing home (NH) residents' everyday preference remains unknown. We examined 1-year stability in reports of importance of 34-recreational activity preferences (8-MDS 3.0 Section F items; 26-Preferences for Everyday Living Inventory-NH items) by NH residents (N = 161). We examined mean differences on demographic and clinical characteristics of residents for preferences showing change. Importance ratings of preferences were highly stable over 1-year, with 91% of items retaining the same valence of importance for the majority of the sample (<20% change). Three preferences showed greater change. More functionally able residents were more likely to change their importance on "being with groups of people," and older residents were more likely to change their preferences for being "involved in religious practices" and "around animals such as pets". Overall, annual assessments of recreational activity preferences capture an accurate representation of preferences with reassessment only needed in a few circumstances.
Collapse
Affiliation(s)
| | - Katherine M Abbott
- Department of Sociology and Gerontology, Scripps Gerontology Center, Miami University, Oxford, OH, USA
| | - Michael J Rovine
- Graduate School of Education, University of Pennsylvania, Philadelphia, PA, USA
| | - Karen Eshraghi
- The Pennsylvania State University, College of Nursing, University Park, PA, USA
| | - Caroline Madrigal
- Advanced Fellow in Health Services Research, Center for Innovation in Long-Term Services & Supports, Providence VA, Providence, RI, USA
| | - Victoria Crumbie
- The Pennsylvania State University, College of Nursing, University Park, PA, USA
| | | |
Collapse
|
50
|
Wakschlag LS, Finlay-Jones AL, MacNeill LA, Kaat AJ, Brown CH, Davis MM, Franklin P, Berkel C, Krogh-Jespersen S, Smith JD. Don't Get Lost in Translation: Integrating Developmental and Implementation Sciences to Accelerate Real-World Impact on Children's Development, Health, and Wellbeing. Front Public Health 2022; 10:827412. [PMID: 35493380 PMCID: PMC9046665 DOI: 10.3389/fpubh.2022.827412] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Accepted: 03/24/2022] [Indexed: 01/14/2023] Open
Abstract
Translation of developmental science discoveries is impeded by numerous barriers at different stages of the research-to-practice pipeline. Actualization of the vast potential of the developmental sciences to improve children's health and development in the real world is imperative but has not yet been fully realized. In this commentary, we argue that an integrated developmental-implementation sciences framework will result in a translational mindset essential for accelerating real world impact. We delineate key principles and methods of implementation science of salience to the developmental science audience, lay out a potential synthesis between implementation and developmental sciences, provide an illustration of the Mental Health, Earlier Partnership (MHE-P), and set actionable steps for realization. Blending these approaches along with wide-spread adoption of the translational mindset has transformative potential for population-level impact of developmental science discovery.
Collapse
Affiliation(s)
- Lauren S. Wakschlag
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Institute for Innovations in Developmental Sciences, Northwestern University, Chicago, IL, United States
| | - Amy L. Finlay-Jones
- Early Neurodevelopment and Mental Health Team, Telethon Kids Institute, Nedlands, WA, Australia
- School of Population Health, Curtin University, Bentley, WA, Australia
| | - Leigha A. MacNeill
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Institute for Innovations in Developmental Sciences, Northwestern University, Chicago, IL, United States
| | - Aaron J. Kaat
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Institute for Innovations in Developmental Sciences, Northwestern University, Chicago, IL, United States
| | - C. Hendricks Brown
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
| | - Matthew M. Davis
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Institute for Innovations in Developmental Sciences, Northwestern University, Chicago, IL, United States
- Department of Pediatrics, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, United States
| | - Patricia Franklin
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
| | - Cady Berkel
- College of Health Solutions, Arizona State University, Tempe, AZ, United States
| | - Sheila Krogh-Jespersen
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
- Institute for Innovations in Developmental Sciences, Northwestern University, Chicago, IL, United States
| | - Justin D. Smith
- Institute for Innovations in Developmental Sciences, Northwestern University, Chicago, IL, United States
- Department of Population Health Sciences, Spencer Fox Eccles School of Medicine at the University of Utah, Salt Lake City, UT, United States
| |
Collapse
|