Measuring the patient experience in community mental health services for older people: A study of the Net Promoter Score using the Friends and Family Test in England

Psychometric validation of a patient-reported experience measure for older adults attending the emergency department: the PREM-ED 65 study

INTRODUCTION

Optimising emergency department (ED) patient experience is vital to ensure care quality. However, there are few validated instruments to measure the experiences of specific patient groups, including older adults. We previously developed a draft 82-item Patient Reported Experience Measure (PREM-ED 65) for adults ≥65 attending the ED. This study aimed to derive a final item list and provide initial validation of the PREM-ED 65 survey.

METHODS

A cross-sectional study involving patients in 18 EDs in England. Adults aged 65 years or over, deemed eligible for ED discharge, were recruited between May and August 2021 and asked to complete the 82-item PREM at the end of the ED visit and 7-10 days post discharge. Test-retest reliability was assessed 7-10 days following initial attendance. Analysis included descriptive statistics, including per-item proportions of responses, hierarchical item reduction, exploratory factor analysis (EFA), reliability testing and assessment of criterion validity.

RESULTS

Five hundred and ten initial surveys and 52 retest surveys were completed. The median respondent age was 76. A similar gender mix (men 47.5% vs women 50.7%) and reason for attendance (40.3% injury vs 49.0% illness) was observed. Most participants self-reported their ethnicity as white (88.6%).Hierarchical item reduction identified 53/82 (64.6%) items for exclusion, due to inadequate engagement (n=33), ceiling effects (n=5), excessive inter-item correlation (n=12) or significant differential validity (n=3). Twenty-nine items were retained.EFA revealed 25 out of the 29 items demonstrating high factor loadings (>0.4) across four scales with an Eigenvalue >1. These scales were interpreted as measuring 'relational care', 'the ED environment', 'staying informed' and 'pain assessment'. Cronbach alpha for the scales ranged from 0.786 to 0.944, indicating good internal consistency. Test-retest reliability was adequate (intraclass correlation coefficient 0.67). Criterion validity was fair (r=0.397) when measured against the Friends and Families Test question.

CONCLUSIONS

Psychometric testing demonstrates that the 25-item PREM-ED 65 is suitable for administration to adults ≥65 years old up to 10 days following ED discharge.

Effectiveness of a chatbot in improving the mental wellbeing of health workers in Malawi during the COVID-19 pandemic: A randomized, controlled trial

We conducted a randomized, controlled trial (RCT) to investigate our hypothesis that the interactive chatbot, Vitalk, is more effective in improving mental wellbeing and resilience outcomes of health workers in Malawi than the passive use of Internet resources. For our 2-arm, 8-week, parallel RCT (ISRCTN Registry: trial ID ISRCTN16378480), we recruited participants from 8 professional cadres from public and private healthcare facilities. The treatment arm used Vitalk; the control arm received links to Internet resources. The research team was blinded to the assignment. Of 1,584 participants randomly assigned to the treatment and control arms, 215 participants in the treatment and 296 in the control group completed baseline and endline anxiety assessments. Six assessments provided outcome measures for: anxiety (GAD-7); depression (PHQ-9); burnout (OLBI); loneliness (ULCA); resilience (RS-14); and resilience-building activities. We analyzed effectiveness using mixed-effects linear models, effect size estimates, and reliable change in risk levels. Results support our hypothesis. Difference-in-differences estimators showed that Vitalk reduced: depression (-0.68 [95% CI -1.15 to -0.21]); anxiety (-0.44 [95% CI -0.88 to 0.01]); and burnout (-0.58 [95% CI -1.32 to 0.15]). Changes in resilience (1.47 [95% CI 0.05 to 2.88]) and resilience-building activities (1.22 [95% CI 0.56 to 1.87]) were significantly greater in the treatment group. Our RCT produced a medium effect size for the treatment and a small effect size for the control group. This is the first RCT of a mental health app for healthcare workers during the COVID-19 pandemic in Southern Africa combining multiple mental wellbeing outcomes and measuring resilience and resilience-building activities. A substantial number of participants could have benefited from mental health support (1 in 8 reported anxiety and depression; 3 in 4 suffered burnout; and 1 in 4 had low resilience). Such help is not readily available in Malawi. Vitalk has the potential to fill this gap.

Ultrasound-Guided Peripheral Venipuncture Decreases the Procedure's Pain and Positively Impacts Patient's Experience: The PRECISE Randomized Clinical Trial

This study aimed to compare patients' experience of pain during ultrasound (US)-guided peripheral venipuncture versus conventional peripheral venipuncture. This randomized clinical trial was conducted at a public university hospital in 2021. Adult patients with indication for intravenous therapy compatible with peripheral intravenous catheters (PIVCs) were included: intervention group (IG), US peripheral venipuncture executed by specialist nurses; control group (CG), conventional peripheral venipuncture executed by clinical practice nurses. The primary outcome was patient experience of pain during the procedure and patient experience related to the PIVC placement method. Sixty-four patients were included, 32 for each group. The pain experienced was none-to-mild in the IG for 25 patients (78.1%) and moderate-to-severe in the CG for 21 patients (65.7%; P < .001). The overall pain rating was 2 (1-3) in the IG and 4 (3-6) in the CG (P < .001). The recommendation of the procedure in IG (net promoter score [NPS] + 90.6%) versus CG (NPS + 18.8%) was considered excellent and good, respectively (P < .001). Patients had less pain and significantly recommended the US-guided procedure. Patient experience with US-guided PIVC, performed by a specialist nurse, was superior to that of conventional peripheral venipuncture.

Engaging staff to improve COVID-19 vaccination response at long-term care facilities (ENSPIRE): A cluster randomized trial of co-designed, tailored vaccine promotion materials

BACKGROUND

COVID-19 vaccination rates among long-term care center (LTCC) workers are among the lowest of all frontline health care workers. Current efforts to increase COVID-19 vaccine uptake generally focus on strategies that have proven effective for increasing influenza vaccine uptake among health care workers including educational and communication strategies. Experimental evidence is lacking on the comparative advantage of educational strategies to improve vaccine acceptance and uptake, especially in the context of COVID-19. Despite the lack of evidence, education and communication strategies are recommended to improve COVID-19 vaccination rates and decrease vaccine hesitancy (VH), especially strategies using tailored messaging for disproportionately affected populations.

METHODS

We describe a cluster-randomized comparative effectiveness trial with 40 LTCCs and approximately 4000 LTCC workers in 2 geographically, culturally, and ethnically distinct states. We compare the effectiveness of two strategies for increasing COVID-19 booster vaccination rates and willingness to promote COVID-19 booster vaccination: co-design processes for tailoring educational messages vs. an enhanced usual care comparator. Our study focuses on the language and/or cultural groups that are most disproportionately affected by VH and low COVID-19 vaccine uptake in these LTCCs.

CONCLUSION

Finding effective methods to increase COVID-19 vaccine uptake and decrease VH among LTCC staff is critical. Beyond COVID-19, better approaches are needed to improve vaccine uptake and decrease VH for a variety of existing vaccines as well as vaccines created to address novel viruses as they emerge.

Net Promoter Score: a prospective, single-centre observational study assessing if a single question determined treatment success after primary or revision hip arthroplasty

BACKGROUND

Our study aimed to identify the relationship between treatment outcome assessed by patient-reported outcomes (PROMs) and satisfaction measured by calculation of the Net Promoter Score (NPS), which identifies promoters, following total hip arthroplasty (THA). The aim was to evaluate this association separately in primary and revision THA and to determine thresholds based on PROMs that identify detractors of the surgical procedure or the centre.

METHODS

A total of 1,243 patients who underwent primary or revision THA at our hospital were asked to complete questionnaires of the Oxford Hip Score (OHS), Euroquol-5D (EQ-5D) and information on pain intensity preoperatively, three and 12 months after surgery. Postoperatively, the patients were additionally asked about their satisfaction with the procedure and the hospital by using three different NPS questions. The association between PROMs and NPS was evaluated based on group comparisons of primary or revision THA and receiver operating characteristics analysis (ROC) to determine threshold values.

RESULTS

At 12 months the NPS of all three questions were invariably linked to treatment outcome in patients after primary THA and patients with a single revision. In these two treatment groups, promoters always showed significantly better PROM scores than detractors. The NPS score was always higher in the primary group in comparison to the single revision group, e.g. 66.4% would undergo the procedure again in the first group, while only 33.0% would opt for this in the latter group. The high thresholds for the PROMs at 12 months, that were calculated by ROC analysis to identify promoter/detractors, indicate that patients` satisfaction required very good joint function and pain relief. However, the NPS was not a suitable tool to identify patients who need further care in an early phase after surgery.

CONCLUSIONS

With NPS already a single question or a single parameter provides the desired information regarding patient satisfaction and also treatment success.

TRIAL REGISTRATION

The study was approved by the Ethics Committee at the Medical Faculty of the University Rostock: "Ethikkommission an der Medizinischen Fakultät der Universität Rostock", Address: St.-Georg Str. 108 18055 Rostock, Germany, reference number: A2015-0055.

Respecting the Patient's Right to Information in Hospital Wards in Poland - Socio-Economic Determinants of Patients' Opinions

Background

Implementation of the patient's statutory right to information is one of determinants the quality and safety of medical services. The patients' opinions survey is an element of patient-centric care, which is now one of the most important things in modern healthcare. The main aim of the work is to examine patients' opinions on the observance of patients' right to information in the hospital and quality of healthcare in the hospital. A secondary objective was to examine the impact of socioeconomic variables on patients' opinions.

Materials and Methods

The study was based on the Computer-Assisted Web Interview (CAWI) questionnaire, carried out in March 2022. The sample included 801 persons. Respondents were patients hospitalized within a maximum of 12 months. The questionnaire contained closed questions about the quality of medical care, observance of the patient's right to information.

Results

Probability of referral was measured using the Net Promoter Score (NPS). Respondents answered the question about the recommendation on a scale from 0 to 10. People who rated the hospital ward at 10 and 9 are promoters, people who rated the hospital ward at 8 and 7 are neutral people. In turn, people who rated the hospital ward at 6, 5, 4, 3, 2, 1 or 0 are critics, ie people who will not recommend the hospital to their relatives. Most patients in hospital wards were the detractors (37%). The percentage of promoters was only 1% lower (36%) than that of the detractors. Patients was rated medical and nursing staff at a similar level with regard to the observance of the patient's right to information.

Conclusion

Most statistically significant differences in the assessment of medical care in the hospital ward were associated with such sociodemographic variables as age and income. People aged 55-64 and 65+ rated most of the analyzed aspects better.

Patients report being satisfied with private musculoskeletal physical therapy when therapists educate effectively and facilitate shared decision making

BACKGROUND

Overall satisfaction with physical therapy care can improve patient adherence and active involvement in their management. However, which individual factors most influence satisfaction with private practice physical therapy care is not well established.

OBJECTIVE

To identify which aspects of the private practice musculoskeletal physical therapy experience best delineated "completely satisfied" and "dissatisfied patients".

METHODS

The MedRisk Instrument for Measuring Patient Satisfaction with Physical Therapy Care (MRPS) was used in a cross-sectional design within 18 Australian private musculoskeletal physical therapy practices. The area under the curve (AUC) of receiver operator characteristic curves (ROC) was used to quantify the ability of the individual patient experience questions to classify the global impressions of satisfaction and likelihood to recommend to others.

RESULTS

1712 patients completed the survey (out of 7320 survey recipients - response rate 23%). High scores were identified for overall satisfaction (4.8/5 ± 0.61) and likelihood to recommend (4.78/5 ± 0.67). Individual items relating to education (AUC = 0.839 and 0.838) and shared decision making (AUC = 0.832 and 0.811) were the most accurate indicators of satisfaction and likelihood to recommend to others, respectively.

CONCLUSION

Individual questionnaire items relating to education and shared decision making were the most accurate indicators of satisfaction and likelihood to recommend in patients attending private practice musculoskeletal physical therapy in Australia. Clinicians and educators should focus on developing these skills to encourage an effective therapeutic alliance and promote greater levels of patient satisfaction.

Net Promoter Score (NPS) as a tool to assess parental satisfaction in pediatric intensive care units

OBJECTIVE

To verify the performance of the Net Promoter Score (NPS) as a tool to assess parental satisfaction in pediatric intensive care units (PICUs).

METHODS

The authors conducted an observational cross-sectional multicenter study in the PICUs of 5 hospitals in Brazil. Eligible participants were all parents or legal guardians of PICU-admitted children, aged 18 years or over. The NPS was administered together with the EMpowerment of PArents in THe Intensive Care (EMPATHIC-30), used as the gold standard, and a sociodemographic questionnaire. For analysis, the results were dichotomized into values greater than or equal to the median of the tests. The associations between the 2 tools were evaluated and the distribution of their results was compared.

RESULTS

The parents or legal guardians of 78 PICU-admitted children were interviewed. Of the respondents, 85% were women and 62% were in a private hospital. The median NPS was 10 (IQR, 10-10), and the median EMPATHIC-30 score was 5.7 (IQR, 5.4-5.9). Compared with the gold standard, the NPS had a sensitivity of 100% at all cutoff points, except at cutoff 10, where the sensitivity was slightly lower (97.5%). As for specificity, NPS performance was poorer, with values ranging from 0% (NPS ≥ 5) to 47.4% (NPS = 10).

CONCLUSIONS

NPS proved to be a sensitive tool to assess parental satisfaction, but with poor ability to identify dissatisfied users in the sample.

Understanding Patient Experience with Outpatient Cancer Rehabilitation Care

BACKGROUND

Understanding patient experience is key to optimize access and quality of outpatient cancer rehabilitation (physical or occupational therapy, PT/OT) services.

METHODS

We performed a retrospective mixed-method analysis of rehabilitation medical record data to better understand patient experience and aspects of care that influenced experience. From the medical record, we extracted case characteristics, patient experience data (Net Promoter Survey^®, NPS) and patient-reported outcome (PRO) data. We categorized cases as 'promoters' (i.e., highly likely to recommend rehabilitation) or 'detractors', then calculated NPS score (-100 [worst] to 100 [best]). We identified key themes from NPS free-text comments using inductive content analysis, then used Pearson [r] or Spearman [ρ] correlation to explore relationships between NPS, characteristics, and PRO improvement.

RESULTS

Patients (n = 383) were 60.51 ± 12.02 years old, predominantly women with breast cancer (69.2%), and attended 14.23 ± 12.37 visits. Most were 'promoters' (92%); NPS score was 91.4. Patients described two experiences (themes) that influenced their likelihood to recommend rehabilitation: (1) feeling comfortable with the process and (2) observable improvement in health/functioning, and described attributes of clinic staff, environment and clinical care that influenced themes. Likelihood to recommend rehabilitation was associated with achieving the minimal clinical important difference on a PRO (ρ = 0.21, p < 0.001) and cancer type (ρ = 0.10, p < 0.001).

CONCLUSION

Patients who received specialized cancer PT/OT were highly likely to recommend rehabilitation. Feeling comfortable with the rehabilitation process and making observable improvements in health and/or functioning influenced likelihood to recommend. Rehabilitation providers should leverage the findings of this study optimize access to and quality of cancer rehab services.

Patients' Opinions on the Quality of Services in Hospital Wards in Poland

INTRODUCTION

Patient opinion surveys have become a widely used method for assessing key aspects of the functioning of medical facilities and, thus, of the functioning of the entire health care system. They are a prerequisite for developing patient-centered care and an essential component of quality improvement programs. In many countries, including Poland, patient opinion surveys are written into the accreditation standards of medical institutions. Patient's readiness to recommend a hospital is a recognized indicator of the quality of patient-centered care. In a report on strategies for improving the quality of health care in Europe published in 2019 by WHO and the OECD (Organisation for Economic Cooperation and Development), patient's readiness to recommend a hospital was cited as one of the basic indicators of 'patient centeredness' along with patient satisfaction. Therefore, as well consideration of the quality of medical care, a patient recommendation index was also used in the study presented in this paper. The index was based on the answers to questions about the patient's readiness to recommend a hospital ward to family and friends.

AIM

The aim of the study was to investigate patients' opinions on the quality of services in particular hospital wards. A patient opinion survey can be used to improve the quality of services and monitor the effects of health-related activities, identify areas that need improvement, motivate medical staff and prevent their burnout, build a trusting relationship with patients, and compare the quality of health care in various facilities.

MATERIAL AND METHODS

The study was carried out in March 2022. The patient opinion survey was conducted using the CAWI (Computer-Assisted Web Interview). The sample selection was purposive. The respondents were patients with a history of hospitalization. The sample selection used an algorithm for the random selection of patients who met the criteria for the sample. The inclusion criterion was hospitalization in the 12 months prior to the study. A standardized questionnaire was used that was aimed at the assessment of the quality of medical care and the patient's rights to information. Additionally, the survey contained questions about the demographic characteristics of the respondents.

RESULTS

A total of 38% of patients with a history of hospitalization expressed criticisms. The majority of statistically significant differences were observed when differentiating respondents according to age. Elderly persons significantly more often declared having been treated with respect and interest. They also rated more highly the meals served in the hospital, effective pain treatment, and respect for the patient's dignity and intimacy during diagnosis and treatment. Younger persons assessed all these aspects of hospitalization less favorably.

CONCLUSIONS

Variables including age and the level of income had a statistically significant influence on the opinion of the respondents. Elderly persons assessed most aspects of the quality of care in a hospital ward more favorably. There were a similar number of "promoters" (36%) and "detractors" (38%) of the quality of hospital services. Detractors mainly pointed to long waiting times for hospital admission, the poor quality of medical and nursing care, and unappealing meals. The promoters emphasized the high quality of medical and nursing care and the favorable conditions of the accommodation. Regular patient satisfaction surveys are helpful in identifying areas in which the functioning of a medical entity requires changes.

The ultimate question? Evaluating the use of Net Promoter Score in healthcare: A systematic review

Background

Patient experience is a complex phenomenon that presents challenges for appropriate and effective measurement. With the lack of a standardized measurement approach, efforts have been made to simplify the evaluation and reporting of patient experience by using single‐item measures, such as the Net Promoter Score (NPS). Although NPS is widely used in many countries, there has been little research to validate its effectiveness and value in the healthcare setting. The aim of this study was to systematically evaluate the evidence that is available about the application of NPS in healthcare settings.

Methods

Studies were identified using words and synonyms that relate to NPS, which was applied to five electronic databases: Medline, CINAHL, Proquest, Business Journal Premium, and Scopus. Titles and abstracts between January 2005 and September 2020 were screened for relevance, with the inclusion of quantitative and qualitative studies in the healthcare setting that evaluated the use of NPS to measure patient experience.

Results

Twelve studies met the inclusion criteria. Four studies identified benefits associated with using NPS, such as ease of use, high completion rates and being well‐understood by a range of patients. Three studies questioned the usefulness of the NPS recommendation question in healthcare settings, particularly when respondents are unable to select their service provider. The free‐text comments section, which provides additional detail and contextual cues, was viewed positively by patients and staff in 4 of 12 studies. According to these studies, NPS can be influenced by a wide range of variables, such as age, condition/disease, intervention and cultural variation; therefore, caution should be taken when using NPS for comparisons. Four studies concluded that NPS adds minimal value to healthcare improvement.

Conclusion

The literature suggests that many of the proposed benefits of using NPS are not supported by research. NPS may not be sufficient as a stand‐alone metric and may be better used in conjunction with a larger survey. NPS may be more suited for use in certain healthcare settings, for example, where patients have a choice of provider. Staff attitudes towards the use of NPS for patient surveying are mixed. More research is needed to validate the use of NPS as a primary metric of patient experience.

Patient or Public Contribution

Consumer representatives were provided with the research findings and their feedback was sought about the study. Consumers commented that they found the results to be useful and felt that this study highlighted important considerations when NPS data is used to evaluate patient experience.

Same-Day Patient Consultation and Cochlear Implantation: Patient Experiences and Barriers to Implementation

OBJECTIVE

This study aimed to assess the efficacy of same-day cochlear implant (SDCI) evaluation and surgery in improving patient experience.

PATIENTS

Adult CI candidates participated in this study.

INTERVENTIONS

Education materials and communication with providers were offered via telephone, e-mail, and/or telemedicine. Patients then arrived for in-person consultation, imaging studies, and outpatient CI surgery in one visit and received a 3-month postoperative survey.

MAIN OUTCOME MEASURES

Preoperative hearing, referral-to-surgery time, travel burden, and patient satisfaction were included as outcome measures.

RESULTS

Of 35 patients who qualified, 14 were successfully contacted regarding the same-day program: 9 underwent CI, 1 enrolled but did not ultimately meet candidacy criteria, and 4 declined because of coronavirus and/or active medical conditions and did not pursue a CI. For the nine patients who underwent SDCI, mean age was 78 years, and mean preoperative consonant-nucleus-consonant score was 16% in the implanted ear. Mean referral-to-surgery time was 103 days and, after accounting for cancellations because of coronavirus, was 52 days. Mean travel distance to institution was 234 miles. Of the seven patients who completed the follow-up survey, none felt rushed for surgery, and mean program experience was rated 8.6 out of 10. Net promoter score was positive (+72), supporting high experience favorability among patients. Barriers to program expansion included patient recruitment and education, surgery scheduling, and the coronavirus pandemic.

CONCLUSIONS

No patients declined the SDCI program to pursue traditional CI evaluation, and all patients were satisfied with their experience. The SDCI program is a feasible and successful model that overcomes barriers to implantation, including travel burden, and improves access to care.

Use and subjective experience of the impact of motor-assisted movement exercisers in people with amyotrophic lateral sclerosis: a multicenter observational study

Motor-assisted movement exercisers (MME) are devices that assist with physical therapy in domestic settings for people living with ALS. This observational cross-sectional study assesses the subjective experience of the therapy and analyzes users' likelihood of recommending treatment with MME. The study was implemented in ten ALS centers between February 2019 and October 2020, and was coordinated by the research platform Ambulanzpartner. Participants assessed symptom severity, documented frequency of MME use and rated the subjective benefits of therapy on a numerical scale (NRS, 0 to 10 points, with 10 being the highest). The Net Promotor Score (NPS) determined the likelihood of a participant recommending MME. Data for 144 participants were analyzed. Weekly MME use ranged from 1 to 4 times for 41% of participants, 5 to 7 times for 42%, and over 7 times for 17%. Particularly positive results were recorded in the following domains: amplification of a sense of achievement (67%), diminution of the feeling of having rigid limbs (63%), diminution of the feeling of being immobile (61%), improvement of general wellbeing (55%) and reduction of muscle stiffness (52%). Participants with more pronounced self-reported muscle weakness were more likely to note a beneficial effect on the preservation and improvement of muscle strength during MME treatment (p < 0.05). Overall, the NPS for MME was high (+ 61). High-frequency MME-assisted treatment (defined as a minimum of five sessions a week) was administered in the majority of participants (59%) in addition to physical therapy. Most patients reported having achieved their individual therapeutic objectives, as evidenced by a high level of satisfaction with MME therapy. The results bolster the justification for extended MME treatment as part of a holistic approach to ALS care.

Do Patient Experiences Have Mediating Roles on Patient Loyalty?

The study aimed to evaluate the mediating roles of patient experiences on patient loyalty. The data were collected through an electronic questionnaire regarding feedback from 5732 patients received outpatient clinics. Patient loyalty was evaluated using the Net Promoter Score (NPS₁₁) that patients were asked whether they would like to recommend the hospital to their relatives or friends. Patient experiences with physicians, nurses, and waiting times were also asked in the questionnaire. After preliminary analysis, mediation analyses were performed to evaluate direct and indirect causal effects among variables for NPS_11. While patient experiences are used as possible mediators, Branch Groups in the first and Admission Time in the second model are independent variables. In the analyses, Surgical Medical Science (p = 0.019) and Day Shift (p = 0.000) have a direct mediating effect on NPS₁₁. Nursing care experiences were found to be a mediator variable for NPS₁₁ in both models (p = 0.000 for both). Patient loyalty was associated with Surgical Medical Science and Day Shift primarily whereas Nursing care experience had a mediating role.

Evaluating patient-reported outcome measures in Peru: a cross-sectional study of satisfaction and net promoter score using the 2016 EnSuSalud survey

BACKGROUND

Patient-reported measures attempt to quantify the value health services provide to users. Satisfaction is a common summative measure, but often has limited utility in identifying poor quality care. We compared satisfaction and the net promoter score (NPS), which was developed to help businesses quantify consumer sentiment, in a nationally representative survey in Peru. We aimed to compare NPS and satisfaction as individual ratings of care, assess the relationship of patient-reported experience ratings to these outcome measures and consider the utility of these measures as indicators of facility performance based on reliability within facilities and capacity to discriminate between facilities.

METHODS

We analysed the 2016 National Survey on User Satisfaction of Health Services, a cross-sectional outpatient exit survey. We assessed ratings by patient characteristics and compared the distributions of satisfaction and NPS categories. We tested the association of patient-reported experience measures with each outcome using multilevel ordinal logistic regression. We used intraclass correlation (ICC) from these models to predict minimum sample for reliable assessment and compared patient-reported experience measures in facilities with average satisfaction but below or above average NPS.

RESULTS

13 434 individuals rated services at 184 facilities. Satisfaction (74% satisfied) and NPS (17% reported at least 9 out of 10) were largely concordant within individuals but weakly correlated (0.37). Ratings varied by individual factors such as age and visit purpose. Most domains of patient-reported experience were associated with both outcomes. Adjusted ICC was higher for NPS (0.26 vs 0.11), requiring a minimum of 7 (vs 20) respondents for adequate reliability. Within the 70% of facilities classified as average based on satisfaction, NPS-based classification revealed systematic differences in patient-reported experience measures.

CONCLUSION

While satisfaction and NPS were broadly similar at an individual level, this evidence suggests NPS may be useful for benchmarking facility performance as part of national efforts in Peru and throughout Latin America to identify deficits in health service quality.

Factors affecting doctor’s recommendation for mobile health services

Objective

As a new medical service mode, the value of mobile health (mHealth） services has received increasing attention and recognition. However, compared with the owners of mobile devices, the user scale of mHealth services is still small. It is well known that doctors’ recommendations have an important impact on what kind of medical service patients choose. To explore the key factors affecting doctors’ recommendation of mHealth services to patients, and to provide countermeasures for mHealth service providers and hospital managers, so as to promote doctors to recommend mHealth services to more patients.

Methods

Through literature review, expert consultation and pre-test, a questionnaire including 22 questions was designed, and 114 valid questionnaires were collected by online research. Net Promoter Score (NPS) was used to evaluate doctors’ recommendation willingness, and multivariate logistics analysis was used to evaluate the key factors affecting doctors’ recommendation willingness.

Results

The NPS of doctors was 6.06%, among which the recommenders, neutrals and critics accounted for 29.56%, 46.96% and 23.48%, respectively. The attitude towards mHealth services and whether they pay attention to and/or are willing to try new technologies are the key factors affecting the doctors’ recommendation, and the usefulness for patients most often emphasized by mHealth service providers to doctors does not affect doctors’ recommendation willingness. In addition, whether mHealth services can help doctors establish personal brands may be a potential factor to enhance doctors’ recommendation willingness.

Conclusion

In order to improve the recommendation willingness of doctors, mHealth service providers and hospital managers should focus on doctors who have a positive attitude towards mHealth services and are highly innovative (which often means younger and lower professional levels). At the same time, they should think about how to use mHealth services to help doctors establish personal brands in the future.

Patient Satisfaction of Ambulatory Care Pharmacy Services

The Ministry of Health (MOH) recently developed the Health Sector Transformation Program (HTSP) to focus on patient satisfaction, safety, and care. This study explored patient satisfaction with outpatient care services.

A semi-structured questionnaire was designed to collect the data of patient satisfaction with pharmacy care services through face-to face interviews of participants and over the internet using an electronic format of the same questionnaire (web-based survey).

A total of 508 participants responded of whom, 51.8% were male and 83% were below 46 years old. The average scores of domains 1, 2, 3, 4, 5, 6, 7, and 8 were 4.51, 4.11, 4.30, 3.70, 3.57, 3.82, 3.63, and 4.15, respectively. The general evaluation of pharmacy services demonstrated that respondents were very satisfied (245; 48.2%), satisfied (143; 28%), and willing to recommend the pharmacy to their families (380; 76.4%). There was no statistically significant difference in the overall satisfaction with pharmacy services among the 2 hospitals (P > .05).

Most respondents were satisfied with outpatient care pharmacy services. More efforts are needed to improve the service of following up on patients’ adherence by the pharmacists.

Orthotic Device Use in Canine Patients: Owner Perception of Quality of Life for Owners and Patients

Orthotic devices are an established treatment for neuromusculoskeletal disease in the human population. Orthoses are an emerging veterinary therapy due to limited practitioner experience, availability of devices, and published data from veterinary patient outcomes. Expanding client education and veterinary expertise in the application of orthoses may allow greater access and successful utilization of these devices to treat appendicular disease. While orthoses have the potential to improve quality of life for veterinary patients, consideration needs to be made for owner related factors with device use. Owner satisfaction and experience may greatly impact compliance with treatment recommendations; therefore, it is crucial that owner expectations are met. The purpose of the present study was to evaluate owner-reported outcomes of orthosis for canine patients and their owner's subjective responses about the shared pet/owner experience utilizing a promoter score. It was hypothesized that owner's impressions of their pet's experience with the orthotic device would influence owner perceptions of quality of life for both the owner and the pet, and these factors would impact the likelihood of the owner to recommend a veterinary orthosis to a friend. An anonymous online survey was sent to 136 clients of a single veterinary orthoses manufacturer. Fifty-six surveys were completed and included for analysis. The owner's reported quality of life was in agreement (P = 0.02) with reported pet quality of life. There was also a higher likelihood (P = 0.02) for the owner to recommend a veterinary orthotic device to a friend when owner perceptions of pet quality of life were positive as compared to negative or neutral. Willingness to recommend an experience to a friend is a reflection of satisfaction with the experience. The dependence of owner and pet quality of life should therefore guide therapeutic decisions for patient management and client communication to ensure that the orthosis experience is positive for both patient and owner.

Comparison of KOOS Scores of Middle-Aged Patients Undergoing Total Knee Arthroplasty to the General Dutch Population Using KOOS Percentile Curves: The LOAS Study

BACKGROUND

We aimed to investigate the application of the Knee Injury and Osteoarthritis Outcome Score (KOOS) percentile curves, using preoperative and postoperative data of patients with knee osteoarthritis undergoing total knee arthroplasty (TKA).

METHODS

We used Longitudinal Leiden Orthopedics Outcomes of Osteo-Arthritis study data of patients between 45 and 65 years and undergoing primary TKA. KOOS scores (0-100) were obtained preoperatively and 6, 12, and 24 months after TKA. Preoperative knee radiographs were assessed according to Kellgren-Lawrence (KL) in a subset (37%) of patients. Comorbidities were self-reported using a standardized questionnaire. The median (interquartile range) population-level KOOS scores were plotted on previously developed population-based KOOS percentile curves. In addition, we assessed the application of the curves on patient level and investigated differences in scores between patients with preoperative KL scores ≤2 and ≥3 and presence (vs absence) of comorbidities.

RESULTS

The study population consisted of 853 patients (62% women, mean age 59 years, body mass index 30 kg/m²) with knee osteoarthritis undergoing primary TKA. Preoperatively, median KOOS scores of all subscales were at or below the 2.5^th percentile. Scores increased to approximately the 25^th percentile 12 months postoperatively. Greater improvements were observed in pain and less improvements in sport and recreational function and quality of life. Patients with higher preoperative KL scores and without comorbidities showed greater improvements.

CONCLUSION

The KOOS percentile curves provided visual insights in knee complaints of patients relative to the general population. Furthermore, the KOOS percentile curves give insight in how preoperative patient characteristics are correlated with postoperative results.

Evaluating a Multicomponent Strategy to Address Burnout, Job Engagement, and Job Satisfaction Among Telepsychiatrists

INTRODUCTION

Preventing and managing burnout among psychiatrists in the United States is a national priority. Telepsychiatry is a promising web-based modality for the delivery of psychiatric services that has the potential for reducing burnout rates. However, it may also be associated with certain occupational challenges such as social and professional isolation, which may impact telepsychiatrists' performance and well-being. Therefore, it is important to develop ways to enhance retention rates of telepsychiatrists and reduce their risk of burnout.

METHODS

In this study, we examine approaches implemented by a telepsychiatry organization between 2018 and 2019, as part of a strategy to enhance the job satisfaction, engagement, and well-being of telepsychiatrists. This article describes the development, implementation, and outcomes associated with this strategy.

RESULTS

Individual and organizational-level outcomes showed overall higher job satisfaction, higher retention rates, and higher output rates over the period in which the strategy was implemented compared with previously, which translate to increased access to mental health care services for patients and imply a reduced risk of burnout among telepsychiatrists.

CONCLUSIONS

Given the risk for social, professional, and academic isolation among telepsychiatrists, organizational engagement efforts are advisable, feasible, and may be associated with significant improvement in satisfaction and retention among telepsychiatrists. Our findings have important implications for enhancing the performance and job satisfaction of telepsychiatrists and potentially reducing burnout rates in this population of providers.

PRACTICAL APPLICATIONS

Effective strategy efforts included: (1) enhancing and standardizing recruitment efforts to ensure proper matching of the psychiatrist with organization, site, and patient population; (2) ensuring congruence between the individual's and the organization's values; (3) providing support for nonclinical administrative tasks; (4) maintaining professional and social communication, support, and supervision via multiple channels.

A Multi-Faceted Strategy for Evidence Translation Reduces Healthcare Waiting Time: A Mixed Methods Study Using the RE-AIM Framework

Background: Waiting lists are often thought to be inevitable in healthcare, but strategies that address patient flow by reducing complexity, combining triage with initial management, and/or actively managing the relationship between supply and demand can work. One such model, Specific Timely Appointments for Triage (STAT), brings these elements together and has been found in multiple trials to reduce waiting times by 30–40%. The next challenge is to translate this knowledge into practice.

Method: A multi-faceted knowledge translation strategy, including workshops, resources, dissemination of research findings and a community of practice (CoP) was implemented. A mixed methods evaluation of the strategy was conducted based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework, drawing on an internal database and a survey of workshop and CoP participants.

Results: Demonstrating reach, at July 2020 an internal database held details of 342 clinicians and managers from 64 health services who had participated in the workshop program (n = 308) and/or elected to join an online CoP (n = 227). 40 of 69 (58%) respondents to a survey of this population reported they had adopted the model, with some providing data demonstrating that the STAT model had been efficacious in reducing waiting time. Perceived barriers to implementation included an overwhelming existing waiting list, an imbalance between supply and demand and lack of resources.

Conclusion: There is high quality evidence from trials that STAT reduces waiting time. Using the RE-AIM framework, this evaluation of a translation strategy demonstrates uptake of evidence to reduce waiting time in health services.

Impact of the COVID-19 Pandemic on the Hospital: Inpatient's Perceived Quality in Spain

Although the Coronavirus disease 2019 (COVID-19) pandemic has generated a large amount of studies, the patient-perceived quality of care (PQ) in this context is still not well known, so more studies intending to focus on this issue are strongly needed. This study assesses changes on PQ in patients hospitalized in Spain during the first month of the COVID-19 pandemic and investigates differences between those admitted for this cause and the rest a descriptive study using the “Net Promoters Score” and the hospital regular monitoring plan. Due to this point of view, ethical approval is not applicable. Four PQ dimensions (nurse, physician, and nurse assistant actions [NA], and discharge information [DI]) were measured in all COVID patients (57) and in a sample of non-COVID patients (60) discharged at home during the first month of the pandemic, and also compared with another sample (384) from an immediately previous period. The COVID patients scored worse (8.2) than non-COVID ones (9.0; P < .0001), especially in NA and DI, and were more likely to be detractors (odds ratio [OR]: 3.05, P < .0001) and less to be promoters (OR: 0.64, P < .05). Global and DI net promoters score values before the pandemic were higher than afterward. In conclusion, the COVID-19 pandemic negatively and significantly influenced the health care quality as perceived by inpatients, both in COVID and in non-COVID ones, but more intensely in the former. As a health care organization, this knowledge meant an opportunity from improvement and to be better qualified to face the pandemic.

[Influence of the covid-19 pandemic period of alert on perceived quality of hospital emergencies]

Objetivo

Valorar los cambios en la calidad percibida por los pacientes que precisaron atención urgente hospitalaria antes y durante el periodo de alarma por la pandemia covid-19.

Métodos

Estudio observacional descriptivo retrospectivo usando Net Promoters Score (NPS), que clasifica a los pacientes como promotores o detractores. Se midieron 3 dimensiones de calidad percibida (accesibilidad, profesionales e información) en 160 pacientes dados de alta a domicilio antes del periodo de alarma y en otros 160 en los primeros 2 meses durante ella. Se comprobó, mediante aceptación de lotes (LQAS), un estándar ≥90% promotores y ≤10% detractores, en diferentes vías de acceso y horarios. Los factores relacionados con la probabilidad de que un paciente fuera promotor o detractor se analizaron mediante regresión logística.

Resultados

La puntuación media fue menor en accesibilidad que en las otras dimensiones (8,6 vs. 9,1 y 9,0; p < 0,0001). La accesibilidad obtuvo mejores resultados durante el estado de alarma (NPS 70 vs. 32; p < 0,001). Respecto a la vía de acceso, mejoraron infantil y oftalmología. Maternal no experimentó cambios. En LQAS, se aceptaron más lotes durante la alarma (85 vs. 72%).

La probabilidad de que un paciente fuera promotor fue mayor durante la alarma (OR 1,85; p < 0,0001) y en la dimensión accesibilidad (OR 3,08; p < 0,0001). La probabilidad de que fuera detractor se redujo (OR 0,54; p< 0,05), también para la dimensión accesibilidad (OR 0,39; p < 0,05).

Conclusiones

La calidad percibida durante el periodo de alarma ha mejorado, en la que su propia declaración es el factor más influyente. La mejora es patente en pacientes pediátricos y oftalmológicos, pero imperceptible en gestantes o traumatología, quizá porque la pandemia ejerció como correctora de la adecuación.

The Use of the Net Promoter Score (NPS) in an Outpatient Allergy and Pulmonary Clinic: An Innovative Look into Using Tablet-Based Tool vs Traditional Survey Method

Introduction

Patient satisfaction has become an essential metric in addition to the type of care they receive. Phone calls, emails, and text to patients after their healthcare visit are the typical way of obtaining the data reflecting patient satisfaction. The purpose of this retrospective quality improvement study is to compare the traditional post-outpatient clinic survey method with an onsite concise two-question survey using a tablet method immediately after the patient visit using Net Promoter Score (NPS) questions.

Methods

Data were collected retrospectively from February to August 2018 from an outpatient subspecialty clinic in southern California using an existing database from two different sources: the traditional method (TM) and the tablet-based tool (TBT), using NPS. The TM data were obtained from a third-party company using two questions via phone, email, and text collected 2–4 weeks after the patient’s visit. The TBT has only two questions that were given to patients upon their visit check-out. These two questions assessed both provider and clinic’s performance using the NPS method.

Results

In total, there were 1708 patients seen from February to August 2018. In the TM, the total outgoing messages during this period were 580 (34.0%) with 156 responses (27%). In the TBT, 648 out of 1708 (37.9%) surveys were collected with a 100% response rate. The NPS score showed that 99.2% of the providers were promoters. The NPS score for the clinic was 96% which reflects a promoter score.

Conclusion

Our results indicate that when using the TBT immediately after their visit to the clinic, a higher response rate was noted. In addition, both methods had similar outcomes in terms of patient satisfaction NPS scores. Future prospective studies with a larger sample size are warranted to evaluate the effectiveness of the TBT tool in assessing patient satisfaction.

A Combined Method of r-NPS and t-NPS Evaluations for Identification of Negative Triggers of Detractors’ Experience

User experience has garnered increasing interest from researchers in the field of sustainable human–computer interaction (HCI) design. One of the key pillars to building sustainable user experience for digital applications is to be able to identify powerful triggers of detracting behavior and especially those of radical detraction. In the past, traditional methods of user experience analysis have often led to the assumption that pragmatic aspects constitute the priority issues for user experience improvement given the prevalence of these aspects in user feedback surveys. However, our econometric analysis based on net promoter score (NPS) survey results defeats this assumption and reveals that the most powerful degraders of the detractors’ experience are in fact the emotive aspects of experience such as transparency of transactions and customer service interaction. Based on our analysis, we arrive at several insights regarding the building of a sustainable HCI strategy. First, hedonic aspects of user experience are the most significant determinants of the degree of user detraction among the detractors. Second, membership cannot be taken for granted as a token of customer loyalty. Building on the theoretical framework of Hassenzahl, Haines-Gadd, and others, we generate new evidence for the importance of servicing hedonic aspects of user experience for digital applications businesses to form a sustainable customer relation and product strategy.

Measuring and improving the quality of tuberculosis care: A framework and implications from the Lancet Global Health Commission

In this article, we describe the framework of the Lancet Global Health Commission on High Quality Health Systems, propose new and undermeasured indicators of TB care quality, and discuss implications of the Commission's key conclusions for measuring and improving the quality of TB care services. The Commission contends that measurement of quality should focus on the processes of care and their impacts. In addition to monitoring treatment coverage and the availability of tools, governments should consider indicators of clinical competence (for e.g. ability of providers to correctly diagnose TB and adhere to treatment guidelines), of timely, continuous and integrated care and of respectful and patient-centered care. Indicators of impact include TB mortality and treatment success rates, but also quality of life and daily functioning among TB patients, public trust in TB services, and bypassing of the formal health system for TB care. Cascades of care, from initial care seeking to recurrence-free survival, should be built in every high-burden country to monitor quality longitudinally. In turn, improvement efforts should target the foundations of health systems and consider the Commission's four universal actions: governing for quality, redesigning service delivery, transforming the health workforce and igniting demand for quality TB services. Important work remains to validate new indicators of TB care quality, develop data collection systems for new measures, and to test new strategies for improving the delivery of competent and respectful TB care.