Reliability and validity of the Patient Experiences Questionnaire for Interdisciplinary Treatment for Substance Dependence - Continuous Electronic Measurement (PEQ-ITSD - CEM)

BACKGROUND

Inpatient experiences with interdisciplinary treatment for substance dependence and mental health care are measured using continuous electronic measurements in Norway. Major changes in data collection from cross-sectional surveys to continuous measurements necessitated the revalidation of the instrument. The main purpose of the present study was to determine the psychometric properties of the Patient Experiences Questionnaire for Interdisciplinary Treatment for Substance Dependence - Continuous Electronic Measurement (PEQ-ITSD - CEM). We also aimed to develop a short version of this tool, since completing the original version can be burdensome for some patients.

METHODS

The study included adult inpatients (aged ≥ 16 years) who received substance-dependence treatment at 102 different sections in Norway during 2020-2022 (n = 2,850). Factor structure and item performance were assessed. A short version was developed based on the psychometric testing results that included item response theory analysis.

RESULTS

The PEQ-ITSD - CEM comprised three empirically based scales with good internal consistency, reliability and validity, which covers treatment and personnel (14 items), milieu (6 items) and outcome (5 items). The results supported a seven-item short version, with three items selected for the treatment and personnel scale, two items for the milieu scale and two items for the outcome scale.

CONCLUSIONS

The PEQ-ITSD - CEM can be recommended for future assessments of patient experiences with interdisciplinary treatment for substance dependence in Norway and in other countries with similar healthcare systems. This short-form version can be applied when respondent burden is a crucial issue.

Reliability and validity of the Psychiatric Inpatient Patient Experience Questionnaire - Continuous Electronic Measurement (PIPEQ-CEM)

Background

The increasing emphasis on patient-centred care has accelerated the demand for high-quality assessment instruments, but the development and application of measures of the quality of care provided for mental health have lagged behind other areas of medicine. The main objective of this study was to determine the psychometric properties of the Psychiatric Inpatient Patient Experience Questionnaire – Continuous Electronic Measurement (PIPEQ-CEM), which consists of large-scale measurements from a Norwegian population. The change from cross-sectional surveys to continuous measurements necessitated further validation of the instrument. The secondary objective was to develop a short version of the PIPEQ-CEM.

Methods

The data included responses from the first year of continuous measurement, and included adult inpatients (age ≥ 18 years) who received specialized mental healthcare from 191 different sections in Norway (n = 3,249). Missing data, ceiling effects, factor structure and internal consistency levels were assessed. The short scale was developed by exploring missing items, ceiling effects, results from exploratory factor analysis (EFA) and item performance from item response theory (IRT) analyses.

Results

Psychometric testing supported previous results and illustrated that the PIPEQ-CEM comprises three empirically based scales with good internal consistency, reliability and validity, and covers structure and facilities, patient-centred interactions, and outcomes. A seven-item short form was developed, which provides an efficient approach for brief yet comprehensive measurements that can be applied in the future.

Conclusion

The PIPEQ-CEM can be recommended for use in future national surveys that assess patient experience with inpatient psychiatric care in Norway and in other countries with similar healthcare systems. The short form can be applied where respondent burden and cognitive load are crucial issues.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08307-5.

Are parents' geographical origin associated with their evaluation of child and adolescent mental health services? Results from a national survey in Norway

The objective was to assess the association between parents' geographical origin and their evaluation of outpatient child and adolescent mental health services (CAMHS). Data were collected in a national parent's experience survey of all outpatient CAMHS in Norway in 2017. Following exclusions, 16,143 parents were part of the study, of which 5932 responded (36.1%). Diagnosis and global psychosocial functional level were collected from the National Patient Register. Multilevel regression was used to assess the association between parents' geographical origin and parent evaluation of the outpatient CAMHS on ten indicators. Sentiment and content analysis was conducted on open-ended comments from parents. The estimated regression coefficients showed that parents born in Eastern Europe scored the services significantly poorer than parents born in Norway on outcome of treatment (- 7.73, p < 0.01), general satisfaction (- 5.53, p < 0.05), ease of getting in contact with health personnel outside of scheduled appointments (- 17.04, p < 0.001), and knowledge of the services that the child has received at the service (- 10.63, p < 0.001). Parents born in Asia/Africa/South America scored the services similar as Norwegian parents on eight of ten indicators, better on one (waiting time) and poorer on one (ease of getting in contact). Sentiment analysis showed that 54% of the comments from parents born in Eastern Europe were negative, compared to 42% for the Norwegian group and 36% for Asia/Africa/South America. The parents' evaluation of the outpatient CAMHS were partly associated with their geographical origin, with parents born in Eastern Europe reporting poorer experiences than parents born in Norway.

Patient-reported experiences with general practitioners: a randomised study of mail and web-based approaches following a national survey

OBJECTIVE

The standard data-collection procedure in the Norwegian national patient experience survey programme is post-discharge mail surveys, which include a pen-and-paper questionnaire with the option to answer electronically. A purely electronic protocol has not previously been explored in Norway. The aim of this study was to compare response rates, background characteristics, data quality and main study results for a survey of patient experiences with general practitioners (GPs) administered by the standard mail data-collection procedure and a web-based approach.

DESIGN

Cross-sectional survey.

SETTING

GP offices in Norway.

PARTICIPANTS

The sample consisted of 6999 patients aged 16 years and older registered with a GP in November 2018.

INTERVENTION

Based on a three-stage sampling design, 6999 patients of GPs aged 16 or older were randomised to one of two survey administration protocols: Group A, who were mailed an invitation with both a pen-and-paper including an electronic response option (n=4999) and Group B, who received an email invitation with electronic response option (n=2000).

MAIN OUTCOME MEASURES

Response rates, background characteristics, data quality and main study results.

RESULTS

The response rate was markedly higher for the mail survey (42.6%) than for the web-based survey (18.3%). A few of the background variables differed significantly between the two groups, but the data quality and patient-reported experiences were similar.

CONCLUSIONS

Web-based surveys are faster and less expensive than standard mail surveys, but their low response rates and coverage problems threaten their usefulness and legitimacy. Initiatives to increase response rates for web-based data collection and strategies for tailoring data collection to different groups should be key elements in future research.

The Adolescent Patient Experiences of Diabetes Care Questionnaire (APEQ-DC): Reliability and Validity in a Study Based on Data from the Norwegian Childhood Diabetes Registry

Purpose

Patient-reported experiences are a key source of information on quality in health care. Most patient experience surveys only include adults’ assessments including parent or proxy surveys in child health care settings. The aim of this study was to determine the psychometric properties of the Adolescent Patient Experiences of Diabetes Care Questionnaire, a new instrument developed to measure adolescent experiences of paediatric diabetes care at hospital outpatient departments in Norway.

Patients and Methods

The questionnaire was developed based on a literature review, qualitative interviews with adolescents, expert-group consultations, pretesting of the questionnaire and a pilot study. The pilot study involved adolescents aged 12–17 years with type 1 diabetes, sampled from the four largest paediatric outpatient departments in Norway. We assessed the levels of missing data, ceiling effects, factor structure, internal consistency, item discriminant validity and construct validity.

Results

The pilot study included responses from 335 (54%) patients. Low proportions of missing or “not applicable” responses were found for 17 of the 19 items, and 14 of these 19 items were below the ceiling-effect criterion. Five indicators were identified: consultation, information on food and physical activity/exercise, nurse contact, doctor contact and outcome. All except one indicator met the criterion of 0.7 for Cronbach’s alpha. Each of the single items had a stronger correlation with its hypothesized indicator than with any of the other indicators. The construct validity of the instrument was supported by 38 out of 45 significant associations.

Conclusion

The content validity of the instrument was secured by a rigorous development process. Psychometric testing produced good evidence for data quality, internal consistency and construct validity. Further research is needed to assess the usefulness of the Adolescent Patient Experiences of Diabetes Care Questionnaire as a basis for quality indicators.

Associations between adolescent experiences, parent experiences and HbA1c: results following two surveys based on the Norwegian Childhood Diabetes Registry (NCDR)

OBJECTIVE

The aim of the current study was to determine the association between the experiences of adolescents and their parents with paediatric diabetes care at hospital outpatient departments and the association between these experiences and the Hemoglobin A1c (HbA1c) levels of adolescents.

DESIGN

Cross-sectional survey.

SETTING

Paediatric diabetes care at hospital outpatient departments in Norway.

PARTICIPANTS

Parents of all outpatients registered in the Norwegian Childhood Diabetes Registry and patients in the same registry aged 12-17 years.

INTERVENTION

1399 parents participated in a national pilot survey and 335 patients aged 12-17 years from the four largest paediatric outpatient departments in Norway responded in another pilot study. 181 paired parental and patient questionnaires were analysed.

MAIN OUTCOME MEASURES

The correlations between single items, indicator scores and overall scores were explored, as was that between indicator scores and HbA1c levels.

RESULTS

There was a moderate but significant correlation between the responses of the patients and parents. For 40 of the 42 associations the correlations were significant, ranging from 0.16 to 0.42. A weak but significant negative correlation was found between the indicator scores of parents and the HbA1c levels of the adolescents. The strongest correlations were between HbA1c level and nurse contact and organisation, both with a correlation coefficient of 0.21 (p<0.01). There was no significant correlation between HbA1c level and patient indicator scores.

CONCLUSIONS

These results highlight the need to obtain information from both parents and adolescents, and indicate that the views of adolescents are not always mirrored by their parents. Three of the seven parent experience indicators were significantly related to the HbA1c levels of adolescents, but replication in future research with larger sample sizes is warranted.

The value of Facebook in nation-wide hospital quality assessment: a national mixed-methods study in Norway

Objectives

The objective was to assess the possibility of using a combination of official and unofficial Facebook ratings and comments as a basis for nation-wide hospital quality assessments in Norway.

Methods

All hospitals from a national cross-sectional patient experience survey in 2015 were matched with corresponding Facebook ratings. Facebook ratings were correlated with both case-mix adjusted and unadjusted patient-reported experience scores, with separate analysis for hospitals with official site ratings and hospitals with unofficial site ratings. Facebook ratings were also correlated with patient-reported incident scores, hospital size, 30-day mortality and 30-day readmission. Facebook comments from 20 randomly selected hospitals were analysed, contrasting the content and sentiments of official versus unofficial Facebook pages.

Results

Facebook ratings were significantly correlated with most patient-reported indicators, with the highest correlations relating to unadjusted scores for organisation (0.60, p<0.000) and nursing services (0.57, p<0.000). Facebook ratings were significantly correlated with hospital size (−0.40, p=0.003) and 30-day mortality (0.31, p=0.040). Sentiment analysis showed that 84.7% of the comments from unofficial Facebook sites included neutral comments that did not give any specific description of experiences of the quality of care at the hospital. Content analysis identified common themes on official and unofficial Facebook pages.

Conclusions

Facebook ratings were associated with patient-reported indicators, hospital size, and 30-day mortality. Qualitative comments from official Facebook are more relevant for hospital evaluation than unofficial sites. More research is needed on using Facebook ratings as a standalone indicator of patient experiences in national quality measurement, and such ratings should be reported together with research-based patient experience indicators and with explicit criteria for the inclusion of unofficial sites.

Patient experiences with interdisciplinary treatment for substance dependence: an assessment of quality indicators based on two national surveys in Norway

Purpose: The quality of health care is often measured using quality indicators, which can be utilized to compare the performance of health-care providers. Conducting comparisons in a meaningful and fair way requires the quality indicators to be adjusted for patient characteristics and other individual-level factors. The aims of the study were to develop and test a case-mix adjustment model for quality indicators based on patient-experience surveys among inpatients receiving interdisciplinary treatment for substance dependence, and to establish whether the quality indicators discriminate between health care providers. Patients and methods: Data were collected through two national surveys involving inpatients receiving residential treatment in Norway in 2013 and 2014. The same questionnaire was used in both surveys, and comprised three patient-experience scales. The scales are reported as national quality indicators, and associations between the scales and patient characteristics were tested through multilevel modeling to establish a case-mix model. The intraclass correlation coefficient was computed to assess the amount of variation at the hospital-trust level. Results: The intraclass correlation coefficient for the patient-reported experience scales varied from 2.3% for "treatment and personnel" to 8.1% for "milieu". Multivariate multilevel regression analyses showed that alcohol reported as the most frequently used substance, gender and age were significantly associated with two of the three scales. The length of stay at the institution, pressure to be admitted for treatment, and self-perceived health were significantly related to all three scales. Explained variance at the individual level was approximately 7% for all three scales. Conclusion: This study identified several important case-mix variables for the patient-based quality indicators and systematic variations at the hospital-trust level. Future research should assess the association between patient-based quality indicators and other quality indicators, and the predictive validity of patient-experience indicators based on on-site measurements.

Inpatients' assessment of outcome at psychiatric institutions: an analysis of predictors following a national cross-sectional survey in Norway

OBJECTIVES

The objective was to assess the importance of different types of predictors for patient-reported outcome, both background factors at the patient level and healthcare predictors related to structure and processes of healthcare.

DESIGN

Cross-sectional patient experience survey.

SETTING

All 280 secondary care institutions in Norway providing inpatient care for adult psychiatric patients.

PARTICIPANTS

1683 inpatients responded to the questionnaire on-site (73.4%).

PRIMARY OUTCOME MEASURES

The outcome scale of the Psychiatric Inpatient Patient Experience Questionnaire-On-Site was the primary dependent variable. The scale consists of five items relating to overall patient satisfaction, benefit of treatment and patient enablement. Regressions were used to assess predictors, for all patients and for five different patient groups reported by the patients including anxiety/depression, drug-related problems and eating disorders.

RESULTS

Multilevel linear regression for all patients showed that background factors related to overall current state, self-perceived mental health before admission and admission type were the most important predictors for patient-assessed outcome. Poor current state was associated with poor assessment of outcome (estimate: 8.64, p<0.001), poor health before admission was associated with better outcome (estimate: -6.89, p<0.001) and patients with urgent admission had poorer scores on the outcome scale (estimate: 4.40, p<0.001). A range of structure and healthcare variables were related to patient-assessed outcome, the most important being clinicians/personnel understanding your situation, treatment adjusted to your situation and adequate information about mental health condition.

CONCLUSIONS

Self-perceived mental health before admission, current overall state and type of admission were the most important background factors for patient-assessed outcome. The most important structure and process variables were related to patient-centred interaction. The background factors should be considered in case-mix adjustments of quality indicators, while the process variables could be used as focus areas in work aiming to improve patients' assessment of outcome.

Parent experiences of diabetes care questionnaire (PEQ-DC): reliability and validity following a national survey in Norway

BACKGROUND

Patient experiences are acknowledged as an important aspect of health care quality but no validated instruments have been identified for the measurement of either parent or patient experiences with outpatient paediatric diabetes care. The aim of the current study was to assess the psychometric properties of a new instrument developed to measure parent experiences of paediatric diabetes care at hospital outpatient departments in Norway.

METHODS

The development of the questionnaire was based on a literature review of existing questionnaires, qualitative interviews with both parents and children/adolescents, expert-group consultations, pretesting of the questionnaire and a pilot study. The national pilot study included parents of 2606 children/adolescents aged 0-17 years with Type 1 Diabetes registered in The Norwegian Childhood Diabetes Registry, a nationwide, population-based registry. Levels of missing data, ceiling effects, factor structure, internal consistency, item discriminant validity and construct validity were assessed.

RESULTS

A total of 2606 patients were included in the survey, but 80 were excluded due to incorrect addresses. 1399 (55%) parents responded to the questionnaire. Low levels of missing or "not applicable" responses were found for 31 of the 35 items (< 10%), and 27 of 35 items were below the ceiling-effect criterion. Psychometric testing and theoretical considerations identified six scales: Consultation (six items), organisation (five items), equipment (three items), nurse contact (four items), doctor contact (four items) and outcome (five items). All six scales met the 0.7 criterion for Cronbach's alpha (range: 0.71-0.90). As expected, each item had a higher correlation with its hypothesised scale than with any of the other five scales. The construct validity of the Parent Experiences of Diabetes Care Questionnaire (PEQ-DC) was supported by 17 out of 18 associations with variables expected to be related to parent experiences.

CONCLUSION

The psychometric testing of the PEQ-DC showed good evidence for data quality, internal consistency and construct validity. The instrument includes important aspects of diabetes care at paediatric outpatient departments from the perspective of the parent. The content validity of the PEQ-DC was secured by a rigorous development process, and the instrument was tested following a national survey in Norway, securing generalisability across Norway.

A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care

BACKGROUND

The objective of this study was to compare three data collection methods for the measurement of parent experiences with hospital outpatient care for child and adolescent diabetes, based on a randomised national trial in Norway involving both pen-and-paper and electronic response options.

METHODS

The sample frame was patients registered in the Norwegian Childhood Diabetes Registry. Parents of patients were randomised into the following groups (n = 2606): group A, who were posted questionnaires with only a pen-and-paper response option (n = 859); group B, who were posted questionnaires with both an electronic and a pen-and-paper response option (n = 886); and group C, who were posted questionnaires with only an electronic response option (n = 861). The three groups were compared on response rate, background variables about respondents, main study results and survey costs. Statistical analysis included logistic regression to test group differences in response probabilities and multilevel linear regression to test differences in parent experiences.

RESULTS

The response rate was 61.8% for group A, 62.4% for group B and 41.6% for group C. The probability of answering was significantly higher for group A (OR = 2.3, p < 0.001) and B (OR = 2.3, p < 0.001) compared to group C. Respondent age, gender, education, living with the child and the degree of participation in consultations did not differ significantly between the three groups. Group differences in parent-reported experiences were small, varying from 1.0 (equipment and doctor contact) to 2.4 (outcome), on a scale from 0 to 100. Only one of 18 group differences was significant: the mixed group had significantly higher score than the electronic group on the organization scale (p < 0.05). The total cost of the electronic model was less than half the cost of the other models, and cost per response was 5.1 euros for the electronic model compared to 8.2 euros for group A and 7.6 euros for group B.

CONCLUSIONS

The models with pen-and paper questionnaire included had more than 20% higher response rate than the model with an electronic-only response option. Background variables and parent-reported experiences were similar between the three groups, and the electronic model was the more cost-effective model.

The Norwegian patient experiences with GP questionnaire (PEQ-GP): reliability and construct validity following a national survey

OBJECTIVES

The aim of this study was to test the reliability and validity of a new questionnaire for measuring patient experiences with general practitioners (PEQ-GP) following a national survey.

SETTING

Postal survey among patients on any of 500 GPs patient lists in Norway. GPs were stratified by practice size and geographical criteria.

PARTICIPANTS

4964 patients who had at least one consultation with their regular GP in the foregoing 12 months were included in the study. The patients were randomly selected after the selection of GPs. 2377 patients (49%) responded to the survey.

PRIMARY AND SECONDARY OUTCOME MEASURES

The items were assessed for missing data and ceiling effects. Factor structure was assessed using exploratory factor analyses. Reliability was tested with item-total correlation, Cronbach's alpha and test-retest correlations. Item discriminant validity was tested by correlating items with all scales. Construct validity was assessed through associations of scale scores with health status, the patients' general satisfaction with the services, whether the patient had been incorrectly treated by the GP and whether the patient would recommend the GP to others.

RESULTS

Item missing varied from 1.0% to 3.1%, while ceiling effects varied from 16.1% to 45.9%. The factor analyses identified three factors. Reliability statistics for scales based on these three factors, and two theoretically derived scales, showed item-total correlations ranging from 0.63 to 0.85 and Cronbach's alpha values from 0.77 to 0.93. Test-retest correlation for the five scales varied from 0.72 to 0.88. All scales had the expected association with other variables.

CONCLUSIONS

The PEQ-GP has good evidence for data quality, internal consistency and construct validity. The PEQ-GP is recommended for use in local, regional and national surveys in Norway, but further studies are needed to assess the instrument's ability to detect differences over time and between different GPs.

Patient experiences questionnaire for interdisciplinary treatment for substance dependence (PEQ-ITSD): reliability and validity following a national survey in Norway

BACKGROUND

Patient experiences are an important aspect of health care quality, but there is a lack of validated instruments for their measurement in the substance dependence literature. A new questionnaire to measure inpatients' experiences of interdisciplinary treatment for substance dependence has been developed in Norway. The aim of this study was to psychometrically test the new questionnaire, using data from a national survey in 2013.

METHODS

The questionnaire was developed based on a literature review, qualitative interviews with patients, expert group discussions and pretesting. Data were collected in a national survey covering all residential facilities with inpatients in treatment for substance dependence in 2013. Data quality and psychometric properties were assessed, including ceiling effects, item missing, exploratory factor analysis, and tests of internal consistency reliability, test-retest reliability and construct validity.

RESULTS

The sample included 978 inpatients present at 98 residential institutions. After correcting for excluded patients (n = 175), the response rate was 91.4%. 28 out of 33 items had less than 20.5% of missing data or replies in the "not applicable" category. All but one item met the ceiling effect criterion of less than 50.0% of the responses in the most favorable category. Exploratory factor analysis resulted in three scales: "treatment and personnel", "milieu" and "outcome". All scales showed satisfactory internal consistency reliability (Cronbach's alpha ranged from 0.75-0.91) and test-retest reliability (ICC ranged from 0.82-0.85). 17 of 18 significant associations between single variables and the scales supported construct validity of the PEQ-ITSD.

CONCLUSION

The content validity of the PEQ-ITSD was secured by a literature review, consultations with an expert group and qualitative interviews with patients. The PEQ-ITSD was used in a national survey in Norway in 2013 and psychometric testing showed that the instrument had satisfactory internal consistency reliability and construct validity.

The Universal Patient Centeredness Questionnaire: reliability and validity of a one-page questionnaire following surveys in three patient populations

Background

This study developed and tested the reliability and validity of the Universal Patient Centeredness Questionnaire (UPC-Q).

Methods

The UPC-Q developed in this study has three parts: 1) the aspects that patients consider the most important when receiving a relevant health care service, rating the health care services on these aspects and their prioritization, 2) the overall experiences of patients using the relevant health care service, and 3) suggestions for improvements. The UPC-Q was tested in four different patient-experience surveys in 2015, including psychiatric inpatients (n=109), general practitioner (GP) patients (n=1,059), and inpatients from two hospital samples (n=973, n=599). The UPC-Q was tested for item completeness and ceiling effects, while the UPC-Q scale consisting of the first part of the UPC-Q was tested for internal consistency reliability and construct validity.

Results

The percentage of patients rating at least one aspect was 70.6% for psychiatric inpatients, 77.6% for hospital inpatients, and 90.6% for GP patients, while 88.9% of the psychiatric inpatients, 93.1% of the hospital inpatients, and 95.3% of the GP patients were able to prioritize the aspects. The internal consistency reliability of the UPC-Q scale was acceptable in all samples (Cronbach’s alpha >0.7), and construct validity was supported by 20 of 21 significant associations between the UPC-Q and related variables. The UPC-Q total score was skewed toward positive evaluations, but the ceiling effect was smaller for an unbalanced response scale than for a balanced scale.

Conclusion

The UPC-Q includes ratings of what is most important for individual patients, while at the same time providing data for improving the quality of health care and making it possible to monitor trends within and across patient populations. This study included psychiatric inpatients, hospital inpatients, and GP patients, and found that the UPC-Q performed well in terms of acceptance, internal consistency reliability, and construct validity.

The Universal Patient Centeredness Questionnaire: scaling approaches to reduce positive skew

PURPOSE

Surveys of patients' experiences typically show results that are indicative of positive experiences. Unbalanced response scales have reduced positive skew for responses to items within the Universal Patient Centeredness Questionnaire (UPC-Q). The objective of this study was to compare the unbalanced response scale with another unbalanced approach to scaling to assess whether the positive skew might be further reduced.

PATIENTS AND METHODS

The UPC-Q was included in a patient experience survey conducted at the ward level at six hospitals in Norway in 2015. The postal survey included two reminders to nonrespondents. For patients in the first month of inclusion, UPC-Q items had standard scaling: poor, fairly good, good, very good, and excellent. For patients in the second month, the scaling was more positive: poor, good, very good, exceptionally good, and excellent. The effect of scaling on UPC-Q scores was tested with independent samples t-tests and multilevel linear regression analysis, the latter controlling for the hierarchical structure of data and known predictors of patient-reported experiences.

RESULTS

The response rate was 54.6% (n=4,970). Significantly lower scores were found for all items of the more positively worded scale: UPC-Q total score difference was 7.9 (P<0.001), on a scale from 0 to 100 where 100 is the best possible score. Differences between the four items of the UPC-Q ranged from 7.1 (P<0.001) to 10.4 (P<0.001). Multivariate multilevel regression analysis confirmed the difference between the response groups, after controlling for other background variables; UPC-Q total score difference estimate was 8.3 (P<0.001).

CONCLUSION

The more positively worded scaling significantly lowered the mean scores, potentially increasing the sensitivity of the UPC-Q to identify differences over time and between providers. However, none of the groups exhibited large positive skew and ceiling effects, implying that such effects might not be a big measurement problem for either scaling format. We recommend using the standard scaling in surveys producing external indicators for inter-provider comparisons. The more positively worded scaling has greater relevance for local measurement work where the results of patient experience surveys have shown large positive skew, and intra-provider comparison is the primary goal.

PIPEQ-OS--an instrument for on-site measurements of the experiences of inpatients at psychiatric institutions

BACKGROUND

The Psychiatric Inpatient Patient Experience Questionnaire (PIPEQ) was developed for post-discharge measurements of experiences, but the low response rates associated with post-discharge surveys restrict their usefulness. A new questionnaire was developed based on the PIPEQ for on-site measurements of patient experiences: the PIPEQ-OS. The aim of this study was to psychometrically test the PIPEQ-OS using data from a nationally representative survey conducted in Norway in 2014.

METHODS

Data were collected using a nationally representative patient-experience survey; 25% of the institutions in each of the 4 health regions in Norway were randomly selected, yielding a total of 26 institutions. The PIPEQ-OS questionnaire was completed by patients on-site on an agreed day in week 37 of 2014. Item missing and ceiling effects were assessed, and factor analysis was used to assess the structure of the items included in the PIPEQ-OS. The scales were tested for internal consistency reliability, test-retest reliability and construct validity.

RESULTS

The initial sample comprised 857 patients. Of these, 60 were excluded for ethical reasons and 57 were excluded because they were absent on the day of the survey. Of the remaining 740 patients, 552 (74.6% of the included population) returned the questionnaire. Low levels of missing or "not applicable" responses were found for 18 of the 21 items (<20%), and 20 of 21 items were below the ceiling-effect criterion. Psychometric testing identified three scales: structure and facilities (six items), patient-centred interaction (six items) and outcomes (five items). All scales met the criterion of 0.7 for Cronbach's alpha (range: 0.79-0.91) and test-retest reliability (range: 0.83-0.84). The construct validity of the scales was supported by 14 of 15 significant associations with variables known to be related to psychiatric inpatient experiences.

CONCLUSIONS

The PIPEQ-OS comprises three scales with satisfactory internal consistency reliability and construct validity. This instrument can be used for on-site assessments of psychiatric inpatient patient experiences, but further research is needed to evaluate its usefulness as basis for external quality indicators.

Patient-reported experiences with hospitals: comparison of proxy and patient scores using propensity-score matching

OBJECTIVE

The objective of this study was to compare patient-experience scores between patients with a proxy response and without a proxy response, using propensity-score matching to maximize the comparability between these two groups.

DESIGN

Cross-sectional survey.

SETTING

Hospital inpatient services in Norway.

PARTICIPANTS

Patients were randomly selected from each of the 61 hospitals in Norway during spring 2011. Postal questionnaires were mailed to 23 420 patients after their discharge from hospital.

INTERVENTION

No intervention.

MAIN OUTCOME MEASURES

All of the patient respondents (n = 8744) and the matched cases (n = 734) were compared with patients with a proxy response (n = 734) on 12 patient-experience indicators using t-tests.

RESULTS

Compared with patient respondents, patients with a proxy response had poorer health, were older and were more often discharged from the hospital to another health-care institution (P < 0.001). Patients with a proxy response yielded significantly lower patient-experience scores than those without a proxy response for 9 of the 12 indicators. Compared with the matched patient group, patients with a proxy response had significantly lower scores for 3 of the 12 indicators and a significantly higher score for one indicator. Differences in scores between patients with a proxy response and the matched patient sample were small, with the largest difference being 4 on a scale of 0-100.

CONCLUSIONS

Patients with a proxy response report somewhat poorer experiences than those without a proxy response; however, proxies represent a different patient group than the patient group as a whole, and the results were very similar after controlling for these differences.

The Patient-Reported Incident in Hospital Instrument (PRIH-I): assessments of data quality, test-retest reliability and hospital-level reliability

Background

The objective of this study was to test the data quality, test–retest reliability and hospital-level reliability of the Patient-Reported Incident in Hospital Instrument (PRIH-I).

Methods

13 incident questions were included in a national patient-experience survey in Norway during the spring of 2011. All questions and a composite incident index were assessed by calculating missing-item rates, test–retest reliability and hospital-level reliability. A multivariate linear regression on a global item regarding incorrect treatment was used to assess the main sources of variation in patient-perceived incorrect treatment at hospitals.

Results

Five of the 13 patient-incident questions had a missing-item rate of >20%. Only one item met the criterion of 0.7 for test–retest reliability (wrong or delayed diagnosis), seven items had a score of >0.5, while the remainder had a reliability score of <0.5. However, the reliability was >0.7 for six of 10 items tested at the hospital level, and >0.6 for the remaining four items. A patient-incident index based on 12 of the incident items had no missing data, the test–retest reliability was 0.6 and the hospital-level reliability was 0.85.

Conclusions

The PRIH-I comprises 13 questions about patient-perceived incidents in hospitals, and can be easily and cost-effectively included in national patient-experience surveys with an acceptable increase in respondent burden. Although the missing-item rate and test–retest reliability were poor for several items, the hospital-level reliability was satisfactory for most of the items. The incident items contribute to a patient-reported incident index, with excellent data quality and hospital-level reliability.

Classification of patients based on their evaluation of hospital outcomes: cluster analysis following a national survey in Norway

BACKGROUND

A general trend towards positive patient-reported evaluations of hospitals could be taken as a sign that most patients form a homogeneous, reasonably pleased group, and consequently that there is little need for quality improvement. The objective of this study was to explore this assumption by identifying and statistically validating clusters of patients based on their evaluation of outcomes related to overall satisfaction, malpractice and benefit of treatment.

METHODS

Data were collected using a national patient-experience survey of 61 hospitals in the 4 health regions in Norway during spring 2011. Postal questionnaires were mailed to 23,420 patients after their discharge from hospital. Cluster analysis was performed to identify response clusters of patients, based on their responses to single items about overall patient satisfaction, benefit of treatment and perception of malpractice.

RESULTS

Cluster analysis identified six response groups, including one cluster with systematically poorer evaluation across outcomes (18.5% of patients) and one small outlier group (5.3%) with very poor scores across all outcomes. One-Way ANOVA with post-hoc tests showed that most differences between the six response groups on the three outcome items were significant. The response groups were significantly associated with nine patient-experience indicators (p < 0.001), and all groups were significantly different from each of the other groups on a majority of the patient-experience indicators. Clusters were significantly associated with age, education, self-perceived health, gender, and the degree to write open comments in the questionnaire.

CONCLUSIONS

The study identified five response clusters with distinct patient-reported outcome scores, in addition to a heterogeneous outlier group with very poor scores across all outcomes. The outlier group and the cluster with systematically poorer evaluation across outcomes comprised almost one-quarter of all patients, clearly demonstrating the need to tailor quality initiatives and improve patient-perceived quality in hospitals. More research on patient clustering in patient evaluation is needed, as well as standardization of methodology to increase comparability across studies.