Instruments to assess patient satisfaction after teleconsultation and triage: a systematic review

Psychometric evaluation of the Telenursing Interaction and Satisfaction Scale

AIMS AND OBJECTIVES

Interaction between caller and telenurse in telenursing is important for caller satisfaction and subsequent compliance. Despite this, satisfaction measures with focus on interaction in telenursing are scarce and rarely anchored in nursing theory. The aim was to evaluate the psychometric properties of the Telenursing Interaction and Satisfaction Scale (TISS) with focus on data quality, factor structure, convergent validity, and reliability.

METHODOLOGICAL DESIGN AND JUSTIFICATION

This psychometric study was based on cross-sectional data.

RESEARCH METHODS, INSTRUMENTS, AND/OR INTERVENTIONS

Callers to the National Medical Advisory Service in Sweden (n = 616) completed the 60-item Telenursing Interaction and Satisfaction Questionnaire based on Cox's Interaction Model of Client Health Behavior. Twenty-five of these items were selected to form the TISS in four subscales according to the model. Data quality was evaluated in terms of missing data patterns and score distributions. The factor structure was evaluated with confirmatory factor analysis for ordinal data, convergent validity with Spearman correlations, internal consistency with ordinal alpha, scale reliability with composite reliability coefficients, and test-retest reliability with intraclass correlations.

RESULTS

The amount of missing data was acceptable and equally distributed. Data deviated significantly from a normal distribution. All response options were endorsed. The factor analysis confirmed the hypothesised four-factor structure; factor loadings ranged from 0.56 to 0.97 and factor correlations were high (0.88-0.96). Internal consistency (ordinal alpha = 0.82-0.97), scale reliability (0.88-0.99), and test-retest reliability (ICC = 0.77-0.86) were satisfactory for all scales.

STUDY LIMITATIONS

The study design did not allow drop-out analysis.

CONCLUSIONS

The TISS showed satisfactory psychometric properties in the study sample. It provides a measure that enables quantitative measurement of caller satisfaction with interaction in telenursing.

Translation and validation study of the Chinese version of the service user technology acceptability questionnaire

Objective

This study aimed to assess the reliability and validity of the translated Chinese version of the Service User Technology Acceptability Questionnaire (C-SUTAQ).

Methods

Patients with cancer (n ​= ​554) from a tertiary hospital in China completed the C-SUTAQ. Item analysis, content and construct validity test, internal consistency test, and test-retest reliability analysis were conducted on the instrument to test its applicability.

Results

The critical ratio of each item of the C-SUTAQ ranged from 11.869 to 29.656; the correlation of each item and subscale ranged from 0.736 to 0.929. The Cronbach's α value for each subscale ranged from 0.659 to 0.941, and the test-retest reliability ranged from 0.859 to 0.966. The content validity index of the scale level and the item level content validity index of the instrument were both 1. Exploratory factor analysis indicated it was reasonable that the C-SUTAQ consists of six subscales after rotation. Confirmatory factor analysis demonstrated good construct validity (χ²/df ​= ​2.459, comparative fit index ​= ​0.922, incremental fit index ​= ​0.907, standardized root mean square residual ​= ​0.060, root-mean-square error of approximation ​= ​0.073, goodness of fit index ​= ​0.875, normed fit index ​= ​0.876.

Conclusions

The C-SUTAQ had good reliability and validity and may be useful to assess Chinese patients' acceptability of telecare. However, the small sample size limited generalization and there is a need to expand the sample to include persons with other diseases. Further studies are required using the translated questionnaire.

Patient satisfaction with nursing care: A bibliometric and visualization analysis (1950-2021)

AIM

The current study aimed to provide an overview of research publications on patient satisfaction with nursing care.

BACKGROUND

Research activity on patient satisfaction with nursing care is an indicator of the commitment of policymakers to improve the quality of health services.

DESIGN

This was a descriptive cross-sectional bibliometric study.

METHODS

The Scopus database was used to retrieve relevant documents for the period from 1950 to 31 December 2021. The keywords 'patient satisfaction' and 'nursing care' were used in the search strategy. Two volunteers validated the search strategy.

RESULTS

The search strategy found 387 documents authored by 1470 researchers and disseminated through 241 journals. Research on patient satisfaction with nursing care (a) has experienced a steep growth over the last decade; (b) was mainly published by scholars from a limited number of countries; (c) has limited cross-country collaboration and limited author collaboration; (d) has addressed research topics concerning the quality of care, instrument validation, emergency department and hospital care and (e) has shown the highest readability for topics related to nurses' burnout and patient satisfaction.

CONCLUSION

Researchers and educators, especially in low- and middle-income countries, need to focus on this field to support activities to upgrade health services and patient outcomes.

A Proposed Patient-Inclusive Methodology for Developing and Validating Telehealth Surveys that Include Social Determinants of Health

This paper discusses a method to develop and validate telehealth surveys that include social determinants of health domains. We performed a scoping review of literature on measuring social determinants of health and extracted 50 social determinants of health domains. We evaluated 14 validated telehealth surveys for questions associated with social determinants of health. We categorized the questions from the validated telehealth surveys using our extracted social determinants of health. We found that current validated telehealth-specific surveys only cover 16 (32%) of social determinants of health domains, with the most commonly evaluated domains being "Medical Needs" and "Social Connections/Isolation". Telehealth services are a valuable modality to provide care to patients. Surveying patients is integral to performing quality improvement and improving patient outcomes. Social determinants of health are important factors in determining patient outcomes. We propose an approach to validating the missing domains and evaluating survey validity.

Assessment of Patients' Confidence Regarding a New Triage Concept in a Medical Retina Clinic during the First COVID-19 Outbreak

BACKGROUND

During the first COVID-19 pandemic outbreak, a new triage concept had to be implemented for patients with retinal diseases having a scheduled appointment at the medical retina clinic. In this study, we aimed to assess patients' confidence in this triage concept and patients' satisfaction regarding the received treatment during the outbreak.

METHODS

This retrospective study included all patients with a diagnosed retinal disease, triaged into three priority groups based on their condition's urgency during lockdown. After restrictions were eased, a subset of previously triaged patients was interviewed to assess their confidence in the triage and their satisfaction regarding the received treatment during the pandemic.

RESULTS

In total, 743 patients were triaged during the lockdown. Over 80% received an urgent appointment (priority 1). Among all priority 1 patients, over 84% attended their appointment and 77% received an intravitreal injection (IVI), while 7% cancelled their appointment due to COVID-19. In post-lockdown interviews of 254 patients, 90% trusted the emergency regimen and received treatment.

CONCLUSIONS

Our triage seemed to be useful in optimizing access to treatment for patients with retinal diseases. An excellent rating of patients' confidence in the triage and satisfaction regarding the received treatment during the first COVID-19 outbreak could be achieved.

Does an emergency access button increase the patients' satisfaction and feeling of safety with the out-of-hours health services? A randomised controlled trial in Denmark

OBJECTIVE

To investigate if the option to bypass the telephone queue can increase satisfaction and feeling of safety in callers.

DESIGN

Randomised controlled parallel superiority trial. Data from questionnaire survey.

SETTING

Two out-of-hours (OOH) services in Denmark.

PARTICIPANTS

217 510 citizens who called the OOH services between 4 September 2017 and 30 November 2017.

RANDOMISATION

Two-faze study period: First half with randomisation of participants based on their date of birth; even date randomised to intervention, uneven date randomised to control group. Second half with all participants included in intervention group.

INTERVENTION

Providing randomised callers (intervention group n=146 355) with the option to bypass the telephone waiting line through an emergency access button (EAB), while the rest got the normal service (control group n=71 155). All EAB users were invited to a questionnaire survey as well as random participants who did not use the EAB (of whom approximately 50% did not have the EAB option).

MAIN OUTCOME MEASURES

Satisfaction and feeling of safety in callers.

RESULTS

2208 of 6704 (32.9%) invited callers answered the questionnaire (intervention group n=1415 (users n=621, non-users n=794); control group n=793). The OR for answering in the two categories with highest satisfaction when provided with the EAB option was 1.34 (95% CI 1.07 to 1.68) for satisfaction with the waiting time, 1.21 (95% CI 0.91 to 1.60) for overall satisfaction and 1.46 (95% CI 1.12 to 1.89) for feeling of safety. Approximately 72% (441/621) of EAB users reported that the EAB option increased their feeling of safety with the OOH services 'to a high degree' compared with 25% (197/794) of callers who had the EAB option without using it.

CONCLUSIONS

The EAB can provide fast access to OOH telephone advice in case of severe illness. It favours citizens perceived in most need of urgent healthcare and significantly increases both feeling of safety and patient satisfaction.

TRIAL REGISTRATION DETAILS

NCT02572115 (5 October 2015).

Development and content validation of the Telenursing Interaction and Satisfaction Questionnaire (TISQ)

Background

Caller satisfaction with telephone advice nursing (TAN) is generally high, and the interaction is essential. However, a valid questionnaire exploring caller satisfaction in TAN with focus on perceived interaction is lacking.

Objective

To develop and assess content validity and test‐retest reliability of a theoretically anchored questionnaire, the Telenursing Interaction and Satisfaction Questionnaire (TISQ), that explores caller satisfaction in TAN by focusing on perceived interaction between the caller and the telenurse.

Methods

The study was performed in three stages. First, variables relevant for patient satisfaction in health care were identified through a literature search. Variables were then structured according to the Interaction Model of Client Health Behavior (IMCHB), which provided theoretical guidance. Items relevant for a TAN context were developed through consensus discussions. Then, evaluation and refinement were performed through cognitive interviews with callers and expert ratings of the Content Validity Index (CVI). Finally, test‐retest reliability of items was evaluated in a sample of 109 individuals using intraclass correlation coefficients (ICC).

Results

The TISQ consists of 60 items. Twenty items cover perceived interaction in terms of health information, affective support, decisional control and professional/technical competence. Five items cover satisfaction with interaction and five items overall satisfaction. Remaining items reflect singularity of the caller and descriptive items of the call. The TISQ was found to exhibit good content validity, and test‐retest reliability was moderate to good (ICC = 0.39‐0.84).

Conclusions

The items in the TISQ form a comprehensive and theoretically anchored questionnaire with satisfactory content validity and test‐retest reliability.

A Conceptual Framework and Pilot Study for Examining Telemedicine Satisfaction Research

Stakeholder satisfaction is often considered a key to the success of telemedicine systems. However, it can be difficult to understand and compare satisfaction evaluations because of variations in reporting and study designs. This research will contribute to the knowledge by developing a conceptual framework around key concepts that relate to understanding studies on telemedicine satisfaction. The framework is built based on a developmental review of the telemedicine and telehealth literature obtained from searches of PubMed and Google Scholar. Using a conceptual matrix, researchers have synthesized the results into a framework that includes: satisfaction dimensions, stakeholders, type of care, type of system, context and methodologies. This research expands these concepts by discussing attributes of each and tests the framework by conducting a pilot study that identifies the concepts in primary study sources. The results of the framework and the pilot study are reported.

The Service User Technology Acceptability Questionnaire: Psychometric Evaluation of the Norwegian Version

Background

When developing a mobile health app, users’ perception of the technology should preferably be evaluated. However, few standardized and validated questionnaires measuring acceptability are available.

Objective

The aim of this study was to assess the validity of the Norwegian version of the Service User Technology Acceptability Questionnaire (SUTAQ).

Methods

Persons with type 2 diabetes randomized to the intervention groups of the RENEWING HEALTH study used a diabetes diary app. At the one-year follow-up, participants in the intervention groups (n=75) completed the self-reported instrument SUTAQ to measure the acceptability of the equipment. We conducted confirmatory factor analysis for evaluating the fit of the original five-factor structure of the SUTAQ.

Results

We confirmed only 2 of the original 5 factors of the SUTAQ, perceived benefit and care personnel concerns.

Conclusions

The original five-factor structure of the SUTAQ was not confirmed in the Norwegian study, indicating that more research is needed to tailor the questionnaire to better reflect the Norwegian setting. However, a small sample size prevented us from drawing firm conclusions about the translated questionnaire.

Telehealth: A Balanced Look at Incorporating This Technology Into Practice

This article explores the use of telehealth and the ability to provide health care to patients through interactive technology and telecommunication tools. The use of telehealth continues to grow, reaching multiple medical specialties and providing care to a greater population of patients. The purpose of this article is to increase the knowledge of telehealth, identify positive outcomes associated with telehealth as well as potential barriers, and evaluate the benefits against the downfalls to determine whether the positive outcomes outweigh the barriers.

Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience

Background

There has been an increased focus towards improving quality of care within the NHS in the last 15 years; as part of this, there has been an emphasis on the importance of patient feedback within policy, through National Service Frameworks and the Quality and Outcomes Framework. The development and administration of large-scale national patient surveys to gather representative data on patient experience, such as the national GP Patient Survey in primary care, has been one such initiative. However, it remains unclear how the survey is used by patients and what impact the data may have on practice.

Objectives

Our research aimed to gain insight into how different patients use surveys to record experiences of general practice; how primary care staff respond to feedback; and how to engage primary care staff in responding to feedback.

Methods

We used methods including quantitative survey analyses, focus groups, interviews, an exploratory trial and an experimental vignette study.

Results

(1)Understanding patient experience data. Patients readily criticised their care when reviewing consultations on video, although they were reluctant to be critical when completing questionnaires. When trained raters judged communication during a consultation to be poor, a substantial proportion of patients rated the doctor as ‘good’ or ‘very good’. Absolute scores on questionnaire surveys should be treated with caution; they may present an overoptimistic view of general practitioner (GP) care. However, relative rankings to identify GPs who are better or poorer at communicating may be acceptable, as long as statistically reliable figures are obtained. Most patients have a particular GP whom they prefer to see; however, up to 40% of people who have such a preference are unable regularly to see the doctor of their choice. Users of out-of-hours care reported worse experiences when the service was run by a commercial provider than when it was run by a not-for profit or NHS provider. (2)Understanding patient experience in minority ethnic groups. Asian respondents to the GP Patient Survey tend to be registered with practices with generally low scores, explaining about half of the difference in the poorer reported experiences of South Asian patients than white British patients. We found no evidence that South Asian patients used response scales differently. When viewing the same consultation in an experimental vignette study, South Asian respondents gave higher scores than white British respondents. This suggests that the low scores given by South Asian respondents in patient experience surveys reflect care that is genuinely worse than that experienced by their white British counterparts. We also found that service users of mixed or Asian ethnicity reported lower scores than white respondents when rating out-of-hours services. (3)Using patient experience data. We found that measuring GP–patient communication at practice level masks variation between how good individual doctors are within a practice. In general practices and in out-of-hours centres, staff were sceptical about the value of patient surveys and their ability to support service reconfiguration and quality improvement. In both settings, surveys were deemed necessary but not sufficient. Staff expressed a preference for free-text comments, as these provided more tangible, actionable data. An exploratory trial of real-time feedback (RTF) found that only 2.5% of consulting patients left feedback using touch screens in the waiting room, although more did so when reminded by staff. The representativeness of responding patients remains to be evaluated. Staff were broadly positive about using RTF, and practices valued the ability to include their own questions. Staff benefited from having a facilitated session and protected time to discuss patient feedback.

Conclusions

Our findings demonstrate the importance of patient experience feedback as a means of informing NHS care, and confirm that surveys are a valuable resource for monitoring national trends in quality of care. However, surveys may be insufficient in themselves to fully capture patient feedback, and in practice GPs rarely used the results of surveys for quality improvement. The impact of patient surveys appears to be limited and effort should be invested in making the results of surveys more meaningful to practice staff. There were several limitations of this programme of research. Practice recruitment for our in-hours studies took place in two broad geographical areas, which may not be fully representative of practices nationally. Our focus was on patient experience in primary care; secondary care settings may face different challenges in implementing quality improvement initiatives driven by patient feedback. Recommendations for future research include consideration of alternative feedback methods to better support patients to identify poor care; investigation into the factors driving poorer experiences of communication in South Asian patient groups; further investigation of how best to deliver patient feedback to clinicians to engage them and to foster quality improvement; and further research to support the development and implementation of interventions aiming to improve care when deficiencies in patient experience of care are identified.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Test-retest stability of patient experience items derived from the national GP patient survey

Purpose

The validity and reliability of various items on the GP Patient Survey (GPPS) survey have been reported, however stability of patient responses over time has not been tested. The purpose of this study was to determine the test–retest reliability of the core items from the GPPS.

Methods

Patients who had recently consulted participating GPs in five general practices across the South West England were sent a postal questionnaire comprising of 54 items concerning their experience of their consultation and the care they received from the GP practice. Patients returning the questionnaire within 3 weeks of mail-out were sent a second identical (retest) questionnaire. Stability of responses was assessed by raw agreement rates and Cohen’s kappa (for categorical response items) and intraclass correlation coefficients and means (for ordinal response items).

Results

348 of 597 Patients returned a retest questionnaire (58.3 % response rate). In comparison to the test phase, patients responding to the retest phase were older and more likely to have white British ethnicity. Raw agreement rates for the 33 categorical items ranged from 66 to 100 % (mean 88 %) while the kappa coefficients ranged from 0.00 to 1.00 (mean 0.53). Intraclass correlation coefficients for the 21 ordinal items averaged 0.67 (range 0.44–0.77).

Conclusions

Formal testing of items from the national GP patient survey examining patient experience in primary care highlighted their acceptable temporal stability several weeks following a GP consultation.

Assessment of Patients' Perception of Telemedicine Services Using the Service User Technology Acceptability Questionnaire

Introduction:

The purpose of this paper is to assess if similar telemedicine services integrated in the management of different chronic diseases are acceptable and well perceived by patients or if there are any negative perceptions.

Theory and methods:

Participants suffering from different chronic diseases were enrolled in Veneto Region and gathered into clusters. Each cluster received a similar telemedicine service equipped with different disease-specific measuring devices. Participants were patients with diabetes (n = 163), chronic obstructive pulmonary disease (n = 180), congestive heart failure (n = 140) and Cardiac Implantable Electronic Devices (n = 1635). The Service User Technology Acceptability Questionnaire (SUTAQ) was initially translated, culturally adapted and pretested and subsequently used to assess patients’ perception of telemedicine. Data were collected after 3 months and after 12 months from the beginning of the intervention. Data for patients with implantable devices was collected only at 12 months.

Results:

Results at 12 months for all clusters are similar and assessed a positive perception of telemedicine. The SUTAQ results for clusters 2 (diabetes), 5 (COPD) and 7 (CHF) after 3 months of intervention were confirmed after 12 months.

Conclusions:

Telemedicine was perceived as a viable addition to usual care. A positive perception for telemedicine services isn’t a transitory effect, but extends over the course of time.

Patient satisfaction with a teleradiology service in general practice

BACKGROUND

Accessibility to secondary health services is not always easy for patients who live at a great distance of hospital. In these circumstances, transferring diagnostic tools and treatment options to primary care could prove beneficial for patients. To do so, the quality of medical care and the costs and benefits of the approach need to be assessed. However, the patient perspective is equally important, offering important insights.

METHODS

In a cross-sectional study we investigate the satisfaction of patients toward a new teleradiology facility offered a general practice on Ameland, an island in the Netherlands. A questionnaire was created based on the Dutch version of the Patient Satisfaction Questionnaire III and completed by all patients after receiving an x-ray in primary care between June 1, 2007 and June 1, 2009. Those who received more than one x-ray in that period were included only once. The technical and interpersonal skills of doctors were rated out the sum score of the questionnaire namely 25 and 30, respectively. Analysis of variance (ANOVA) was used to analyze the differences between the means of the satisfaction subscales and the patient characteristics.

RESULTS

The response proportion was after reminder 65% (381/587 patients). Satisfaction with the technical skills of the doctor providing the teleradiology service was 22.4 ± 3.7, while satisfaction with the interpersonal skills of the doctor during the diagnostic phase was 26.8 ± 3.8. Island residents, the elderly, and those with no history of trauma were more satisfied with the technical and interpersonal aspects of the consultation than non-residents, younger patients, and those with a history of trauma.

CONCLUSION

Patients in the island community of Ameland experienced high levels of satisfaction with the teleradiology service offered in primary care.

Establishing a centralised telehealth service increases telehealth activity at a tertiary hospital

BACKGROUND

The Princess Alexandra Hospital Telehealth Centre (PAH-TC) is a project jointly funded by the Australian national government and Queensland Health. It seeks to provide a whole-of-hospital telehealth service using videoconferencing and store-and-forward capabilities for a range of specialities. The aim of this study was to investigate whether the introduction of a new telehealth coordination service provided by a tertiary hospital centre increased telehealth activities of a tertiary hospital. Evaluation included service delivery records and stakeholder satisfaction.

METHODS

Telehealth service delivery model before and after the establishment of the centre is described as well as the project implementation. The study retrieved data related to the number and scope of previous, and current, telehealth service episodes, to ascertain any change in activity levels following the introduction of the new telehealth coordination service. In addition, using a cross-sectional research design, the satisfaction of patients, clinicians and administrators was surveyed. The survey focused on technical utility and perceived clinical validity.

RESULTS

Introduction of a new centralised telehealth coordination service was associated with an increase in the scope of telehealth from five medical disciplines, in the year before the establishment, to 34 disciplines two years after the establishment. The telehealth consultations also increases from 412 (the year before), to 735 (one year after) and 1642 (two years after) the establishment of the centre. Respondents to the surveys included patients (27), clinicians who provided the consultations (10) and clinical or administrative staff who hosted the telehealth consultations in the remote site (8). There were high levels of agreement in relation to the telehealth option saving time and money, and an important health service delivery model. There was evidence from the remote site that modifying roles to incorporate this new service was challenging.

CONCLUSION

The introduction of a centralised coordination for telehealth service of a tertiary hospital was associated with the increase in the scope and level of telehealth activity of the hospital. The project and model of health care delivery described in this paper can be adopted by tertiary hospitals to grow their telehealth activities, and potentially reduce costs associated with the delivery of services at a distance.

Establishing the validity of English GP Patient Survey items evaluating out-of-hours care

Background

A 2014 national audit used the English General Practice Patient Survey (GPPS) to compare service users’ experience of out-of-hours general practitioner (GP) services, yet there is no published evidence on the validity of these GPPS items.

Objectives

Establish the construct and concurrent validity of GPPS items evaluating service users’ experience of GP out-of-hours care.

Methods

Cross-sectional postal survey of service users (n=1396) of six English out-of-hours providers. Participants reported on four GPPS items evaluating out-of-hours care (three items modified following cognitive interviews with service users), and 14 evaluative items from the Out-of-hours Patient Questionnaire (OPQ). Construct validity was assessed through correlations between any reliable (Cochran's α>0.7) scales, as suggested by a principal component analysis of the modified GPPS items, with the ‘entry access’ (four items) and ‘consultation satisfaction’ (10 items) OPQ subscales. Concurrent validity was determined by investigating whether each modified GPPS item was associated with thematically related items from the OPQ using linear regressions.

Results

The modified GPPS item-set formed a single scale (α=0.77), which summarised the two-component structure of the OPQ moderately well; explaining 39.7% of variation in the ‘entry access’ scores (r=0.63) and 44.0% of variation in the ‘consultation satisfaction’ scores (r=0.66), demonstrating acceptable construct validity. Concurrent validity was verified as each modified GPPS item was highly associated with a distinct set of related items from the OPQ.

Conclusions

Minor modifications are required for the English GPPS items evaluating out-of-hours care to improve comprehension by service users. A modified question set was demonstrated to comprise a valid measure of service users’ overall satisfaction with out-of-hours care received. This demonstrates the potential for the use of as few as four items in benchmarking providers and assisting services in identifying, implementing and assessing quality improvement initiatives.