1
|
Qin Z, Zhu Y, Shi DD, Chen R, Li S, Wu J. The gap between statistical and clinical significance: time to pay attention to clinical relevance in patient-reported outcome measures of insomnia. BMC Med Res Methodol 2024; 24:177. [PMID: 39118002 PMCID: PMC11308508 DOI: 10.1186/s12874-024-02297-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Accepted: 07/25/2024] [Indexed: 08/10/2024] Open
Abstract
BACKGROUND Appropriately defining and using the minimal important change (MIC) and the minimal clinically important difference (MCID) are crucial for determining whether the results are clinically significant. The aim of this study is to survey the status of randomized controlled trials (RCTs) for insomnia interventions to assess the inclusion and interpretation of MIC/MCID values. METHODS We conducted a cross-sectional study to survey the status of RCTs for insomnia interventions to assess the inclusion and appropriate interpretation of MIC/MCID values. A literature search was conducted by searching the main sleep medicine journals indexed in PubMed, the Excerpta Medica Database (EMBASE), and the Cochrane Central Register of Controlled Trials (CENTRAL) to identify a broad range of search terms. We included RCTs with no restriction on the intervention. The included studies used the Insomnia Severity Index (ISI) or the Pittsburgh Sleep Quality Index (PSQI) questionnaire as the outcome measures. RESULTS 81 eligible studies were identified, and more than one-third of the included studies used MIC/MCID (n = 31, 38.3%). Among them, 21 studies with ISI as the outcome used MIC defined as a relative decrease ranging from 3 to 8 points. The most frequently used MIC value was a 6-point decrease (n = 7), followed by 8-point (n = 6) and 7-point decrease (n = 4), a 4 to 5-points decrease (n = 3), and a 30% reduction from baseline; 6 studies used MCID values, ranging from 2.8 to 4 points. The most frequently used MCID value was a 4-point decrease in the ISI (n = 4). 4 studies with PSQI as the outcome used a 3-point change as the MIC (n = 2) and a 2.5 to 2.7-point difference as MCID (n = 2). 4 non-inferiority design studies considered interval estimation when drawing clinically significant conclusions in their MCID usage. CONCLUSIONS The lack of consistent MIC/MCID interpretation and usage in outcome measures for insomnia highlights the urgent need for further efforts to address this issue and improve reporting practices.
Collapse
Affiliation(s)
- Zongshi Qin
- Clinical Research Institute, Institute of Advanced Clinical Medicine, Peking University, Beijing, China.
| | - Yidan Zhu
- Clinical Research Institute, Institute of Advanced Clinical Medicine, Peking University, Beijing, China
| | - Dong-Dong Shi
- Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Rumeng Chen
- School of Life Sciences, Beijing University of Chinese Medicine, Beijing, China
| | - Sen Li
- School of Life Sciences, Beijing University of Chinese Medicine, Beijing, China.
| | - Jiani Wu
- Guang'anmen Hospital, China Academy of Chinese Medical Sciences, Beijing, China.
| |
Collapse
|
2
|
Malandrini F, Meregaglia M, Di Maio M, Pinto C, De Lorenzo F, Ciani O. The development of an archive of patient-reported outcome measures (PROMs) in oncology: The Italian PRO4All project. Eur J Cancer 2024; 207:114161. [PMID: 38870746 DOI: 10.1016/j.ejca.2024.114161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2024] [Revised: 05/27/2024] [Accepted: 05/30/2024] [Indexed: 06/15/2024]
Abstract
BACKGROUND Choosing the most adequate measure of patient-reported outcomes (PROs) for a specific medical condition is not straightforward. This study aimed to develop a comprehensive archive of patient-reported outcome measures (PROMs), observer-reported outcome measures (ObsROMs) and caregiver-reported outcome measures (CROMs) in oncology and identify their main characteristics and target outcome domains. MATERIALS AND METHODS As part of the Italian PRO4All Project, we retrieved questionnaires through an extensive search of online databases. We developed a data extraction form to collect information on cancer type, questionnaire variant(s), recall period, and scoring system. We performed a content analysis of the questionnaires to assign each item a specific outcome domain according to a predefined 38-item taxonomy. RESULTS A total of 386 PROMs (n = 356), ObsROMs (n = 13) and CROMs (n = 17) were identified and described; of these, 358 were also analyzed in their content. 47.3 % of the instruments were cancer type-specific, 45.1 % were generic for cancer and 7.9 % were developed for the general population but also recommended in oncology. The great majority (92.2 %) were patient-reported. In 50.3 % the recall period was "last week". The mean number of items per questionnaire was 22.0 (range: 1-130). 7794 items were assigned an outcome domain, the most frequent being emotional functioning/wellbeing (22 %), physical functioning (15.7 %), general outcomes (10.1 %) and delivery of care (8.9 %). CONCLUSIONS There are a variety of patient and caregiver-reported measures in oncology. This archive can guide researchers and practitioners in selecting the most suitable measures and fostering a patient-centered approach in clinical trials, clinical practice, and regulatory activities.
Collapse
Affiliation(s)
- Francesco Malandrini
- Center for Research on Health and Social Care Management, SDA Bocconi School of Management, Milan, Italy
| | - Michela Meregaglia
- Center for Research on Health and Social Care Management, SDA Bocconi School of Management, Milan, Italy
| | - Massimo Di Maio
- Department of Oncology, University of Turin, A.O.U. Città della Salute e della Scienza di Torino, Turin, Italy; Italian Association of Medical Oncology (AIOM), Italy
| | - Carmine Pinto
- Medical Oncology Unit, Comprehensive Cancer Center, AUSL-IRCCS di Reggio Emilia, Reggio Emilia, Italy; Federation of Italian Cooperative Oncology Groups (FICOG), Florence, Italy
| | | | - Oriana Ciani
- Center for Research on Health and Social Care Management, SDA Bocconi School of Management, Milan, Italy.
| |
Collapse
|
3
|
Anderson M, van Kessel R, Wood E, Stokes A, Fistein J, Porter I, Mossialos E, Valderas JM. Understanding factors impacting patient-reported outcome measures integration in routine clinical practice: an umbrella review. Qual Life Res 2024:10.1007/s11136-024-03728-7. [PMID: 39023733 DOI: 10.1007/s11136-024-03728-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/24/2024] [Indexed: 07/20/2024]
Abstract
PURPOSE Patient-report outcome measures (PROMs) have gained widespread support as a mechanism to improve healthcare quality. We aimed to map out key enablers and barriers influencing PROMs implementation strategies in routine clinical practice. METHODS An umbrella review was conducted to identify reviews exploring enablers and barriers related to the integration of PROMs in routine clinical practice from January 2000 to June 2023. Information on key enablers and barriers was extracted and summarised thematically according to the Theoretical Domains Framework. RESULTS 34 reviews met our criteria for inclusion. Identified reviews highlighted barriers such as limited PROMs awareness among clinicians and patients, perceived low value by clinicians and patients, PROMs that were too complex or difficult for patients to complete, poor usability of PROMs systems, delayed feedback of PROMs data, clinician concerns related to use of PROMs as a performance management tool, patient concerns regarding privacy and security, and resource constraints. Enablers encompassed phased implementation, professional training, stakeholder engagement prior to implementation, clear strategies and goals, 'change champions' to support PROMs implementation, systems to respond to issues raised by PROMs, and integration into patient pathways. No consensus favoured paper or electronic PROMs, yet offering both options to mitigate digital literacy bias and integrating PROMs into electronic health records emerged as important facilitators. CONCLUSIONS The sustainable implementation of PROMs is a complex process that requires multicomponent organisational strategies covering training and guidance, necessary time and resources, roles and responsibilities, and consultation with patients and clinicians.
Collapse
Affiliation(s)
- Michael Anderson
- Health Organisation, Policy, Economics (HOPE), Centre for Primary Care & Health Services Research, The University of Manchester, Manchester, UK.
- LSE Health, Department of Health Policy, London School of Economics and Political Science, London, UK.
| | - Robin van Kessel
- LSE Health, Department of Health Policy, London School of Economics and Political Science, London, UK
- Department of International Health, Care and Public Health Research Institute (CAPHRI), Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Eleanor Wood
- LSE Health, Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Adam Stokes
- Centre for Global Health, St Georges, University of London, London, UK
| | - Jon Fistein
- Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
| | - Ian Porter
- Health Services and Policy Research Group, University of Exeter, Exeter, UK
| | - Elias Mossialos
- LSE Health, Department of Health Policy, London School of Economics and Political Science, London, UK
- Institute of Global Health Innovation, Imperial College London, London, UK
| | - Jose M Valderas
- Health Services and Policy Research Group, University of Exeter, Exeter, UK
- Centre for Research On Health Systems Performance, National University of Singapore, Singapore, Singapore
| |
Collapse
|
4
|
Gabel CP, Cuesta-Vargas A, Dibai-Filho AV, Mokhtarinia HR, Melloh M, Bejer A. Developing a shortened spine functional index (SFI-10) for patients with sub-acute/chronic spinal disorders: a cross-sectional study. BMC Musculoskelet Disord 2024; 25:236. [PMID: 38532353 PMCID: PMC10964542 DOI: 10.1186/s12891-024-07352-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 03/12/2024] [Indexed: 03/28/2024] Open
Abstract
BACKGROUND Brief whole-spine patient-reported outcome measures (PROMs) provide regional solutions and future directions for quantifying functional status, evidence, and effective interventions. The whole-spine regional Spine Functional Index (SFI-25) is used internationally in clinical and scientific contexts to assess general sub-acute/chronic spine populations. However, to improve structural validity and practicality a shortened version is recommended. This study developed a shortened-SFI from the determined optimal number of item questions that: correlated with criteria PROMs being highly with whole-spine, moderately with regional-spine, condition-specific and patient-specific, and moderately-low with general-health and pain; retained one-dimensional structural validity and high internal consistency; and improved practicality to reduce administrative burden. METHODS A cross-sectional study (n = 505, age = 18-87 yrs., average = 40.3 ± 10.1 yrs) of sub-acute/chronic spine physiotherapy outpatients from an international sample of convenience. Three shortened versions of the original SFI-25 were developed using 1) qualitative 'content-retention' methodology, 2) quantitative 'factorial' methodology, and 3) quantitative 'Rasch' methodology, with a fourth 'random' version produced as a comparative control. The clinimetric properties were established for structural validity with exploratory (EFA) and confirmatory (CFA) factorial analysis, and Rasch analysis. Criterion validity used the: whole-spine SFI-25 and Functional Rating Index (FRI); regional-spine Neck Disability Index (NDI), Oswestry Disability Index (ODI), and Roland Morris Questionnaire (RMQ), condition-specific Whiplash Disability Questionnaire (WDQ); and patient-specific functional scale (PSFS); and determined floor/ceiling effect. A post-hoc pooled international sub-acute/chronic spine sample (n = 1433, age = 18-91 yrs., average = 42.0 ± 15.7 yrs) clarified the findings and employed the general-health EuroQuol-Index (EQ-5D), and 11-point Pain Numerical Rating Scale (P-NRS) criteria. RESULTS A 10-item SFI retained structural validity with optimal practicality requiring no computational aid. The SFI-10 concept-retention-version demonstrated preferred criterion validity with whole-spine criteria (SFI-25 = 0.967, FRI = 0.810) and exceeded cut-off minimums with regional-spine, condition-specific, and patient-specific measures. An unequivocal one-dimensional structure was determined. Internal consistency was satisfactory (α = 0.80) with no floor/ceiling effect. Post-hoc analysis of the international sample confirmed these findings. CONCLUSION The SFI-10 qualitative concept-retention version was preferred to quantitative factorial and Rasch versions, demonstrated structural and criterion validity, and preferred correlation with criteria measures. Further longitudinal research is required for reliability, error, and responsiveness, plus an examination of the practical characteristics of readability and administrative burden.
Collapse
Affiliation(s)
| | - Antonio Cuesta-Vargas
- Department of Psychiatry and Physiotherapy, Faculty of Medicine, Malaga University, Malaga, Spain
| | | | - Hamid Reza Mokhtarinia
- Department of Ergonomics, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran.
| | - Markus Melloh
- Queensland University of Technology, School of Public Health and Social Work, Brisbane, Australia
| | - Agnieszka Bejer
- Institute of Health Sciences, Medical College, Rzeszow University, Rzeszow, Poland
- The Holy Family Specialist Hospital, Rudna Mała 600, 36-060, Głogów Małopolski, Poland
| |
Collapse
|
5
|
Cerimele JM, Franta G, Blanchard BE, Leasure W, Fortney JC. Bipolar Disorder Symptom Monitoring Measures: A Mixed-Methods Study of Patient Preferences. J Acad Consult Liaison Psychiatry 2024; 65:148-156. [PMID: 37967752 PMCID: PMC11032232 DOI: 10.1016/j.jaclp.2023.11.266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 10/17/2023] [Accepted: 11/09/2023] [Indexed: 11/17/2023]
Abstract
OBJECTIVE To determine the perceived acceptability and helpfulness of bipolar disorder symptom measures and rank the measures in order of preference among individuals with bipolar disorder. METHODS We recruited 20 participants 18 years of age or older with any type of bipolar disorder from a primary care clinical site and a national advocacy organization. We used a simultaneous complementary mixed-method design involving completion of symptom measures, a semistructured interview, and numerical ranking of measures. Participants completed three symptom measures or combination of measures: 1) Affective Self-Rating Scale; 2) combination Patient Mania Questionnaire-9 (PMQ-9) and Patient Health Questionnaire-9 (PHQ-9); and 3) combination Altman Self-Rating Mania Rating Scale and PHQ-9. A semistructured interview was conducted, and participants ranked their preferences for measures. Interviews focused on participants' rationale for measuring preferences. Interviews were analyzed by two psychiatrist-investigators using content analysis, and themes were determined. Average rank of each measure was determined. RESULTS The average rank for each measure was 1.48 for the combination PMQ-9 and PHQ-9, 1.68 for the Affective Self-Rating Scale, and 2.85 for the combination Altman Self-Rating Mania Rating Scale and PHQ-9, indicating that the combination PMQ-9 and PHQ-9 (top-ranked measure by 55% of participants) was the most preferred among the three measures. Major themes that emerged from the data were: 1) measure format; 2) patient experience; 3) clinical practice; and 4) therapeutic effects. CONCLUSIONS Individuals with bipolar disorder preferred the combination PMQ-9 and PHQ-9 for use in monitoring treatment due to perceived strengths such as format, ease of completion and interpretation, accurate description of experiences, and feasibility of use in practice.
Collapse
Affiliation(s)
- Joseph M Cerimele
- Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle, WA.
| | - Gabriel Franta
- Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle, WA
| | - Brittany E Blanchard
- Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle, WA
| | - William Leasure
- Department of Psychiatry & Psychology, Mayo Clinic, Rochester, MN
| | - John C Fortney
- Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle, WA; Department of Veterans Affairs, HSR&D Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, Seattle, WA
| |
Collapse
|
6
|
Jansson-Fröjmark M, Badinlou F, Lundgren T, Schonfeld IS, Bianchi R. Validation of the Occupational Depression Inventory in Sweden. BMC Public Health 2023; 23:1505. [PMID: 37553626 PMCID: PMC10411009 DOI: 10.1186/s12889-023-16417-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 07/28/2023] [Indexed: 08/10/2023] Open
Abstract
BACKGROUND The Occupational Depression Inventory (ODI) was recently devised to assess depressive symptoms that individuals specifically attribute to their work. One purpose of the ODI is to overcome limitations in current assessments of job-related distress. This study aimed to validate the Swedish version of the ODI. METHODS The study involved 365 individuals employed in Sweden. In addition to the ODI, the study included the Satisfaction with Life Scale, the Effort-Reward Imbalance Questionnaire, the Demand-Control-Support Questionnaire, the GAD-2, and the PHQ-9. We inquired into the factorial validity, dimensionality, scalability, test-score reliability, criterion validity, convergent validity, discriminant validity, and measurement invariance of the ODI. RESULTS Exploratory structural equation modeling bifactor analysis indicated that the ODI's Swedish version meets the requirements for essential unidimensionality (e.g., explained common variance = 0.872). Measurement invariance held across sexes, age groups, and occupational categories. The instrument exhibited strong scalability (e.g., H = 0.662). The observed total scores thus accurately ranked respondents on the latent continuum underlying the scale. The ODI's total-score reliability was high (e.g., McDonald's ω = 0.929). Speaking to the instrument's criterion validity, we found occupational depression to correlate, in the expected direction, with various work (e.g., job support) and nonwork (e.g., general anxiety) variables. Occupational depression showed large correlations with effort-reward imbalance (r = 0.613) and demand-control imbalance (r = 0.566) at work. Multiple regression analyses supported these associations further. As expected, we observed both a degree of convergent validity and a degree of discriminant validity when examining the ODI against the PHQ-9, an attribution-free measure of depression. DISCUSSION This study indicates that the ODI performs well within the Swedish context, consistent with the findings obtained in other linguistic and geographic contexts. The ODI promises to help researchers, practitioners, and public health decision-makers address job-related distress more effectively.
Collapse
Affiliation(s)
- Markus Jansson-Fröjmark
- Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institute, and Stockholm Health Care Services, Region of Stockholm, Stockholm, Sweden.
| | - Farzaneh Badinlou
- Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institute, and Stockholm Health Care Services, Region of Stockholm, Stockholm, Sweden
| | - Tobias Lundgren
- Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institute, and Stockholm Health Care Services, Region of Stockholm, Stockholm, Sweden
| | - Irvin Sam Schonfeld
- Department of Psychology, The City College and the Graduate Center of the City University of New York, New York City, NY, USA
| | - Renzo Bianchi
- Department of Psychology, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
| |
Collapse
|
7
|
Terwee CB, Elders PJM, Blom MT, Beulens JW, Rolandsson O, Rogge AA, Rose M, Harman N, Williamson PR, Pouwer F, Mokkink LB, Rutters F. Patient-reported outcomes for people with diabetes: what and how to measure? A narrative review. Diabetologia 2023; 66:1357-1377. [PMID: 37222772 PMCID: PMC10317894 DOI: 10.1007/s00125-023-05926-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 03/03/2023] [Indexed: 05/25/2023]
Abstract
Patient-reported outcomes (PROs) are valuable for shared decision making and research. Patient-reported outcome measures (PROMs) are questionnaires used to measure PROs, such as health-related quality of life (HRQL). Although core outcome sets for trials and clinical practice have been developed separately, they, as well as other initiatives, recommend different PROs and PROMs. In research and clinical practice, different PROMs are used (some generic, some disease-specific), which measure many different things. This is a threat to the validity of research and clinical findings in the field of diabetes. In this narrative review, we aim to provide recommendations for the selection of relevant PROs and psychometrically sound PROMs for people with diabetes for use in clinical practice and research. Based on a general conceptual framework of PROs, we suggest that relevant PROs to measure in people with diabetes are: disease-specific symptoms (e.g. worries about hypoglycaemia and diabetes distress), general symptoms (e.g. fatigue and depression), functional status, general health perceptions and overall quality of life. Generic PROMs such as the 36-Item Short Form Health Survey (SF-36), WHO Disability Assessment Schedule (WHODAS 2.0), or Patient-Reported Outcomes Measurement Information System (PROMIS) measures could be considered to measure commonly relevant PROs, supplemented with disease-specific PROMs where needed. However, none of the existing diabetes-specific PROM scales has been sufficiently validated, although the Diabetes Symptom Self-Care Inventory (DSSCI) for measuring diabetes-specific symptoms and the Diabetes Distress Scale (DDS) and Problem Areas in Diabetes (PAID) for measuring distress showed sufficient content validity. Standardisation and use of relevant PROs and psychometrically sound PROMs can help inform people with diabetes about the expected course of disease and treatment, for shared decision making, to monitor outcomes and to improve healthcare. We recommend further validation studies of diabetes-specific PROMs that have sufficient content validity for measuring disease-specific symptoms and consider generic item banks developed based on item response theory for measuring commonly relevant PROs.
Collapse
Affiliation(s)
- Caroline B Terwee
- Amsterdam UMC, Department of Epidemiology and Data Science, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands.
- Amsterdam Public Health Research Institute, Methodology, Amsterdam, the Netherlands.
| | - Petra J M Elders
- Amsterdam Public Health Research Institute, Methodology, Amsterdam, the Netherlands
- Amsterdam UMC, Department of General Practice, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Marieke T Blom
- Amsterdam Public Health Research Institute, Methodology, Amsterdam, the Netherlands
| | - Joline W Beulens
- Amsterdam UMC, Department of Epidemiology and Data Science, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Olaf Rolandsson
- Department of Public Health and Clinical Medicine, Family Medicine, Umeå University, Umeå, Sweden
| | - Alize A Rogge
- Center for Patient-Centered Outcomes Research, Department of Psychosomatic Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Matthias Rose
- Center for Patient-Centered Outcomes Research, Department of Psychosomatic Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Nicola Harman
- Department of Health Data Science, University of Liverpool, Liverpool, UK
| | - Paula R Williamson
- Department of Health Data Science, University of Liverpool, Liverpool, UK
| | - Frans Pouwer
- Steno Diabetes Center Odense, Odense, Denmark
- Department of Psychology, University of Southern Denmark, Odense, Denmark
- Amsterdam UMC, Department of Medical Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Lidwine B Mokkink
- Amsterdam UMC, Department of Epidemiology and Data Science, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health Research Institute, Methodology, Amsterdam, the Netherlands
| | - Femke Rutters
- Amsterdam UMC, Department of Epidemiology and Data Science, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
8
|
Mokanyk AR, Taylor CL, De la Garza Ramos R, Tadepalli S, Girasulo SA, Rossi MCS, O'Donnell BA, Bauman JA, Sekhar R, Abbed KM, Matmati N, Yanamadala V. System-wide integration of patient reported outcome measure collection through an electronic medical record system: A state-wide retrospective study. J Clin Neurosci 2023; 114:137-143. [PMID: 37392561 DOI: 10.1016/j.jocn.2023.06.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 06/09/2023] [Accepted: 06/11/2023] [Indexed: 07/03/2023]
Abstract
BACKGROUND AND PURPOSE In spine neurosurgery practice, patient-reported outcome measures (PROMs) are tools used to convey information about a patient's health experience and are an integral component of a clinician's decision-making process as they help guide treatment strategies to improve outcomes and minimize pain. Currently, there is limited research showing effective integration strategies of PROMs into electronic medical records. This study aims to provide a framework for other healthcare systems by outlining the process from start to finish in seven Hartford Healthcare Neurosurgery outpatient spine clinics throughout the state of Connecticut. METHODS On March 1, 2021, a pilot implementation program began in one clinic and on July 1, 2021, all outpatient clinics were implementing the revised clinical workflow that included the electronic collection of PROMs within the electronic health record (EHR). A retrospective chart analysis studied all adult (18+) new patient visits in seven outpatient clinics by comparing the rates of PROMs collection in Half 1 (March 1, 2021-August 31, 2022) and in Half 2 (September 1, 2022-February 28, 2022) across all sites. Additionally, patient characteristics were studied to identify any variables that may lead to higher rates of collection. RESULTS During the study period, 3528 new patient visits were analyzed. There was a significant change in rates of PROMs collection across all departments between H1 and H2 (p < 0.05). Additional significant predictors for PROMs collection were the sex and ethnicity of the patient as well as the provider type for the visit (p < 0.05). CONCLUSIONS This study proved that implementing the electronic collection of PROMs into an already existing clinical workflow reduces previously identified collection barriers and enables PROMs collection rates that meet or exceed current benchmarks. Our results provide a successful step-by-step framework for other spine neurosurgery clinics to implement a similar approach.
Collapse
Affiliation(s)
- Abigail R Mokanyk
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA
| | - Christopher L Taylor
- Frank H. Netter MD School of Medicine, 370 Bassett Rd, North Haven, CT 06473, USA
| | - Rafael De la Garza Ramos
- Department of Neurosurgery, Montefiore Medical Center, Albert Einstein College of Medicine in New York City, USA
| | - Swetha Tadepalli
- University of Connecticut, College of Liberal Art and Sciences, Storrs, CT 06269, USA
| | - Samantha A Girasulo
- University of Connecticut, College of Liberal Art and Sciences, Storrs, CT 06269, USA
| | - Maria C S Rossi
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA
| | - Brooke A O'Donnell
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA
| | - Joel A Bauman
- Department of Neurosurgery, Hartford HealthCare Medical Group, Hartford, CT 06106, USA
| | - Rajat Sekhar
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA; Frank H. Netter MD School of Medicine, 370 Bassett Rd, North Haven, CT 06473, USA
| | - Khalid M Abbed
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA; Frank H. Netter MD School of Medicine, 370 Bassett Rd, North Haven, CT 06473, USA
| | - Nabil Matmati
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA; Frank H. Netter MD School of Medicine, 370 Bassett Rd, North Haven, CT 06473, USA.
| | - Vijay Yanamadala
- Department of Neurosurgery, Hartford HealthCare Medical Group, St. Vincent's Medical Center, Bridgeport, CT 06606, USA; Frank H. Netter MD School of Medicine, 370 Bassett Rd, North Haven, CT 06473, USA
| |
Collapse
|
9
|
Kroenke K, Lam V, Ruddy KJ, Pachman DR, Herrin J, Rahman PA, Griffin JM, Cheville AL. Prevalence, Severity, and Co-Occurrence of SPPADE Symptoms in 31,866 Patients With Cancer. J Pain Symptom Manage 2023; 65:367-377. [PMID: 36738867 PMCID: PMC10106386 DOI: 10.1016/j.jpainsymman.2023.01.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Revised: 01/21/2023] [Accepted: 01/24/2023] [Indexed: 02/05/2023]
Abstract
OBJECTIVES To examine the prevalence, severity, and co-occurrence of SPPADE symptoms as well as their association with cancer type and patient characteristics. BACKGROUND The SPPADE symptoms (sleep disturbance, pain, physical function impairment, anxiety, depression, and low energy /fatigue) are prevalent, co-occurring, and undertreated in oncology and other clinical populations. METHODS Baseline SPPADE symptom data were analyzed from the E2C2 study, a stepped wedge pragmatic, population-level, cluster randomized clinical trial designed to evaluate a guideline-informed symptom management model targeting the six SPPADE symptoms. Symptom prevalence and severity were measured with a 0-10 numeric rating (NRS) scale for each of the six symptoms. Prevalence of severe (NRS ≥ 7) and potential clinically relevant (NRS ≥ 5) symptoms as well as co-occurrence of clinical symptoms were determined. Distribution-based methods were used to estimate the minimally important difference (MID). Associations of cancer type and patient characteristics with a SPPADE composite score were analyzed. RESULTS A total of 31,886 patients were assessed for SPPADE symptoms prior to, during, or soon after an outpatient medical oncology encounter. The proportion of patients with a potential clinically relevant symptom ranged from 17.5% for depression to 33.4% for fatigue. Co-occurrence of symptoms was high, with the proportion of patients with three or more additional clinically relevant symptoms ranging from 45.2% for fatigue to 68.6% for depression. The summed SPPADE composite score demonstrated good internal reliability (Cronbach's alpha of 0.86), with preliminary MID estimates of 4.1-4.3. Symptom burden differed across several types of cancer but was generally similar across most sociodemographic characteristics. CONCLUSION The high prevalence and co-occurrence of SPPADE symptoms in patients with all types of cancer warrants clinical approaches that optimize detection and management.
Collapse
Affiliation(s)
- Kurt Kroenke
- Indiana University School of Medicine (K.K.), Indianapolis, Indiana, USA; Regenstrief Institute, Inc. (K.K.), Indianapolis, Indiana, USA.
| | - Veronica Lam
- Department of Physical Medicine and Rehabilitation (V.L., A.L.C.), Mayo Clinic, Rochester, Minnesota, USA
| | - Kathryn J Ruddy
- Division of Medical Oncology (K.J.R.), Mayo Clinic, Rochester, Minnesota, USA
| | - Deirdre R Pachman
- Division of Community Internal Medicine, Geriatrics, and Palliative Care (D.R.P.), Mayo Clinic, Rochester, Minnesota, USA
| | - Jeph Herrin
- Yale University School of Medicine (J.H.), New Haven, Connecticut, USA
| | - Parvez A Rahman
- Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery (P.A.R., J.M.G., A.L.C.), Mayo Clinic, Rochester, Minnesota, USA
| | - Joan M Griffin
- Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery (P.A.R., J.M.G., A.L.C.), Mayo Clinic, Rochester, Minnesota, USA; Division of Health Care Delivery Research (J.M.G.), Mayo Clinic, Rochester, Minnesota, USA
| | - Andrea L Cheville
- Department of Physical Medicine and Rehabilitation (V.L., A.L.C.), Mayo Clinic, Rochester, Minnesota, USA; Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery (P.A.R., J.M.G., A.L.C.), Mayo Clinic, Rochester, Minnesota, USA
| |
Collapse
|
10
|
Koh ZH, Skues J, Murray G. Digital self-report instruments for repeated measurement of mental health in the general adult population: a protocol for a systematic review. BMJ Open 2023; 13:e065162. [PMID: 36693693 PMCID: PMC9884895 DOI: 10.1136/bmjopen-2022-065162] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/26/2023] Open
Abstract
INTRODUCTION Digital technologies present tremendous opportunities for enabling long-term measurement of mental health in the general population. Emerging studies have established preliminary efficacy of collecting self-report data digitally. However, a key challenge when developing a new self-report instrument is navigating the abundance of existing instruments to select relevant constructs for measurements. This review is a precursor to developing a novel future integrated digital instrument for repeated measurements. We interrogate the literature as the first step towards optimal measurement of the multifaceted mental health concept, in the context of digital repeated measurement. This review aims to identify (1) digital self-report instruments administered repeatedly to measure the mental health of the general adult population; (2) their structure and format; (3) their psychometric properties; (4) their usage in empirical studies; and (5) the constructs these instruments were designed to measure (as characterised in the original publication), and the constructs the instruments have been used to measure in the identified empirical studies. METHODS AND ANALYSIS Five major electronic databases will be searched. Studies administering mental health instruments (in English) repeatedly to community dwellers in the general adult population are eligible. A reviewer will preliminarily screen for eligible studies. Then, two reviewers will independently screen the full text of the eligible articles and extract data. Both reviewers will resolve any disagreement through discussion or with a third reviewer. After the data extraction, a reviewer will manually search for the structure, format, psychometric properties and the original constructs these instruments were developed to measure. This review will synthesise the results in a narrative approach. The reporting in this review will be guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist. ETHICS AND DISSEMINATION Ethical approval is not required as no data will be collected. Findings of the systematic review will be disseminated through peer-reviewed publications and conference presentations. PROSPERO REGISTRATION NUMBER CRD42022306547.
Collapse
Affiliation(s)
- Zhao Hui Koh
- Centre for Mental Health, Swinburne University of Technology, Hawthorn, Victoria, Australia
| | - Jason Skues
- Department of Psychological Sciences, Swinburne University of Technology, Hawthorn, Victoria, Australia
| | - Greg Murray
- Centre for Mental Health, Swinburne University of Technology, Hawthorn, Victoria, Australia
| |
Collapse
|
11
|
Are Ecological Momentary Assessments of Pain Valid and Reliable? A Systematic Review and Meta-analysis. Clin J Pain 2023; 39:29-40. [PMID: 36524770 DOI: 10.1097/ajp.0000000000001084] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 10/17/2022] [Indexed: 12/23/2022]
Abstract
OBJECTIVES This systematic review and meta-analysis aimed to determine the level of evidence for the psychometric properties of Ecological Momentary Assessment (EMA) in populations with persistent pain. MATERIALS AND METHODS Five databases were searched from 1980 to December 2021. Two reviewers independently screened the titles, abstracts, and full text, extracted data, and assessed adherence to reporting standards and methodological rigor before evaluating the quality of evidence. A meta-analysis, including the pooling of correlations for the relevant EMA pain outcomes, was completed. RESULTS Overall, 3270 studies were identified, with 14 studies meeting inclusion criteria. Meta-analyses confirmed good to excellent relationships demonstrated between EMA and recalled pain intensity and interference across different timeframes. Many of the included studies did not fully adhere to recommended reporting standards, and the quality of included studies was either doubtful or inadequate due to methodological flaws. The level of evidence for measurement properties of pain outcomes was low for the criterion validity of pain intensity and interference and very low for reliability and construct validity of pain intensity and interference. DISCUSSION Ecological momentary assessment of pain experience appears both valid and reliable. Although the levels of evidence were low or very low, these findings provide preliminary support for the use of EMA in clinical practice and research settings. Potential strengths of EMA include providing measures with greater ecological validity while also reducing recall bias, both pertinent in pain outcome measurement. More research, including higher-quality studies, is needed to demonstrate further support for EMA, including the need for establishing other types of validity.
Collapse
|
12
|
Patient Reported Outcome Measures: An Exploration of their Utility in Functional Assessment and Rehabilitation. Arch Phys Med Rehabil 2022; 103:S1-S2. [DOI: 10.1016/j.apmr.2022.02.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 02/09/2022] [Indexed: 11/19/2022]
|
13
|
Lee MK, Beebe TJ, Yost KJ, Eton DT, Novotny PJ, Dueck AC, Frost M, Sloan JA. Score equivalence of paper-, tablet-, and interactive voice response system-based versions of PROMIS, PRO-CTCAE, and numerical rating scales among cancer patients. J Patient Rep Outcomes 2021; 5:95. [PMID: 34533663 PMCID: PMC8448797 DOI: 10.1186/s41687-021-00368-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2021] [Accepted: 09/04/2021] [Indexed: 11/21/2022] Open
Abstract
Background The study tests the effects of data collection modes on patient responses associated with the multi-item measures such as Patient-Reported Outcomes Measurement System (PROMIS®), and single-item measures such as Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE), and Numerical Rating Scale (NRS) measures. Methods Adult cancer patients were recruited from five cancer centers and administered measures of anxiety, depression, fatigue, sleep disturbance, pain intensity, pain interference, ability to participate in social roles and activities, global mental and physical health, and physical function. Patients were randomized to complete the measures on paper (595), interactive voice response (IVR, 596) system, or tablet computer (589). We evaluated differential item functioning (DIF) by method of data collection using the R software package, lordif. For constructs that showed no DIF, we concluded equivalence across modes if the equivalence margin, defined as ± 0.20 × pooled SD, completely surrounds 95% confidence intervals (CI's) for difference in mean score. If the 95% CI fell totally outside the equivalence margin, we concluded systematic score difference by modes. If the 95% CI partly overlaps the equivalence margin, we concluded neither equivalence nor difference. Results For all constructs, no DIF of any kind was found for the three modes. The scores on paper and tablet were more comparable than between IVR and other modes but none of the 95% CI’s were completely outside the equivalence margins, in which we established neither equivalence nor difference. Percentages of missing values were comparable for paper and tablet modes. Percentages of missing values were higher for IVR (2.3% to 6.5% depending on measures) compared to paper and tablet modes (0.7% to 3.3% depending on measures and modes), which was attributed to random technical difficulties experienced in some centers. Conclusion Across all mode comparisons, there were some measures with CI’s not completely contained within the margin of small effect. Two visual modes agreed more than visual-auditory pairs. IVR may induce differences in scores unrelated to constructs being measured in comparison with paper and tablet. The users of the surveys should consider using IVR only when paper and computer administration is not feasible.
Collapse
Affiliation(s)
- Minji K Lee
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA.
| | - Timothy J Beebe
- Division of Health Policy and Management, University of Minnesota School of Public Health, 625 Michigan Ave, 27th Floor, Chicago, IL, 60611, USA
| | - Kathleen J Yost
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA
| | - David T Eton
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA
| | - Paul J Novotny
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA
| | - Amylou C Dueck
- Department of Quantitative Health Sciences, Mayo Clinic, 13400 E. Shea Blvd., Scottsdale, AZ, 85259, USA
| | - Marlene Frost
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA
| | - Jeff A Sloan
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA
| |
Collapse
|
14
|
Hawkins M, Elsworth GR, Nolte S, Osborne RH. Validity arguments for patient-reported outcomes: justifying the intended interpretation and use of data. J Patient Rep Outcomes 2021; 5:64. [PMID: 34328558 PMCID: PMC8324704 DOI: 10.1186/s41687-021-00332-y] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2020] [Accepted: 06/23/2021] [Indexed: 12/28/2022] Open
Abstract
BACKGROUND Contrary to common usage in the health sciences, the term "valid" refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane's argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. METHODS The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane's five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study. RESULTS The general HLQ interpretive argument was built to demonstrate Kane's five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument. CONCLUSIONS Kane's five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.
Collapse
Affiliation(s)
- Melanie Hawkins
- Swinburne University of Technology, Centre for Global Health and Equity, School of Health Sciences, PO Box 218, Hawthorn, Melbourne, Victoria 3122 Australia
| | - Gerald R. Elsworth
- Swinburne University of Technology, Centre for Global Health and Equity, School of Health Sciences, PO Box 218, Hawthorn, Melbourne, Victoria 3122 Australia
| | - Sandra Nolte
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Medical Department, Division of Psychosomatic Medicine, Berlin, Germany
| | - Richard H. Osborne
- Swinburne University of Technology, Centre for Global Health and Equity, School of Health Sciences, PO Box 218, Hawthorn, Melbourne, Victoria 3122 Australia
| |
Collapse
|