Is it time to abandon paper? The use of emails and the Internet for health services research--a cost-effectiveness and qualitative study

Quality of total hip arthroplasty health care based on four years of patient-reported outcomes in the Netherlands

BACKGROUND

Joint arthroplasty registries have incorporated patient-reported outcomes (PROs) to evaluate outcomes from a patients' perspective to improve total hip arthroplasty (THA). To draw valid conclusions on PROs, a minimum response rate (RR) of 60% is advised. This study investigated (1) if the quality of THA health care based on PROs improved over the years in the Netherlands, (2) if RRs improved over the years, and (3) difference in PROs over the years in hospitals with RR ≥ 60% compared to RR < 60%.

METHODS

Longitudinal study with publicly available datasets from 2016 to 2019. Primary outcome was increase/decrease in PRO change scores including 95%CI ranges over the years between preoperatively and 3 months postoperatively (pre-3 m), and 12 months postoperatively (pre-12 m). Improved quality of health care was arbitrary defined as when ≥ 3 of 4 included scores or ranges were statistically significant improved. Secondary outcome was increase/decrease in RRs over the years. Subgroups RR ≥ 60% and RR < 60% were compared.

RESULTS

Hospitals (%) collecting THA PROs increased from 78 to 92%. EQ VAS change score increased over the years, and 95%CI ranges of EQ VAS, EQ-5D descriptive system and NRS pain during activity decreased over the years at pre-3 m (p < 0.05). All THA pre-12 m PRO change scores and 95%CI ranges remained equal (p > 0.05). Pre-3 m RR remained equal (around 43%, p = 0.107) and pre-12 m RR decreased 9% (49% to 40%, p = 0.008). Pre-3 m subgroup RR ≥ 60% was too small to analyse (5%). No difference was found between pre-12 m subgroups (RR ≥ 60% = 16%), p > 0.05).

CONCLUSIONS

Quality of THA health care based on PROs seems equal in the Netherlands between 2016 and 2019. Although more hospitals participated in PRO collection, low RRs with large IQRs are observed and only 16% of the hospitals achieved the advised RR ≥ 60%. Multiple recommendations are provided to improve PRO collection and use.

Response to survey directed to patient portal members differs by age, race, and healthcare utilization

Health care systems are increasingly utilizing electronic medical record—associated patient portals to facilitate communication with patients and between providers and their patients. These patient portals are growing in recognition as potentially valuable research tools. While there is much information about the response rates and demographics of internet-based surveys as well as the demographics of patients who are portal members, not much is known about the response rate of internet-based surveys directed to a group of patient portal members or the demographics of which portal members respond to internet-based surveys issued within that specific population. The objective of these analyses was to determine the demographics of patient portal users who respond to an internet-based survey request. We hypothesized that respondents would more likely be: (1) older (65+), (2) European American, (3) married, (4) female, (5) college educated, (6) have higher medical care utilization, (7) have more comorbidities, and (8) have a private practice primary care physician (as opposed to a salaried group practice primary care physician). We found that our respondents tended to be older, of European geographic ancestry, and more frequent users of healthcare. While patient portal members are an easily identifiable and contactable group that are potentially valuable participants for research, it is important to understand that respondents to surveys solicited from this sampling frame may not be entirely representative. It will be important to develop strategies to more fully engage populations that represent the target population in order to increase overall and subgroup response rates.

Response rate and costs for automated patient-reported outcomes collection alone compared to combined automated and manual collection

Background

The response rate on patient-reported outcome measurements (PROMs) necessary to adequately evaluate a treatment and improve patient care is unknown. Hospitals generally aim for the highest possible response rate without insight into the increase in costs involved. This study aimed to investigate which PROMs response rate is achievable in relation to the costs in an orthopaedic practice.

Methods

In an observational study, patients planned for orthopaedic surgery were asked to participate per surgical procedure (5769 surgical procedures at 5300 patients). Patient-reported outcomes (PROs) collection with a digital online automated PROMs collection system (minimal effort) was compared to a combined automated system and manual collection (maximal effort). Response rate was calculated preoperative and at two postoperative time points separately, and on all three time points together. Costs were calculated for the study period, per year and per surgical procedure. Calculations were executed for all surgical procedures and for three subgroups: knee arthroplasty, hip arthroplasty and anterior cruciate ligament reconstruction (ACLR).

Results

Using maximal effort the response rate increased for all surgical procedures compared to minimal effort; the preoperative response rate from 86% to 100% and the postoperative response rates from 55% to 83% (3 or 6 months) and 53% to 83% (12 months). Concerning the response at all three time points for all surgical procedures, minimal effort resulted in 44% response rate and increased to 76% with maximal effort. Lowest postoperative response rates were found in the ACLR group for both maximal and minimal effort. A costs difference of €5.55–€5.98 per surgical procedure between maximal and minimal effort was found.

Conclusions

A two times higher PROMs response rate for patients responding at all three time points (44% versus 76%) is achievable with maximal effort compared to the use of an automated PROMs collection system only. Manual collection adds a cost of €5.5–€6 per surgical procedure to automated PROMs collection alone. It is debatable if these additional costs are justifiable from a value-based health care perspective as the response rate for adequate evaluation of a treatment is still unknown.

Data Collection via In-Depth Email Interviewing: Lessons From the Field

This methods article is a reflection on the use of in-depth email interviewing in a qualitative descriptive study. The use of emailing to conduct interviews is thought to be an effective way to collect qualitative data. Building on current methodological literature in qualitative research regarding in-depth email interviewing, we move the conversation toward elicitation of quality data and management of multiple concurrent email interviews. Excerpts are shared from a field journal that was kept throughout one study, with commentary on developing insights. Valuable lessons learned include the importance of (a) logistics and timing related to the management of multiple concurrent email interviews, (b) language and eliciting the data, (c) constructing the email, and (d) processing text-based data and preparing transcripts. Qualitative researchers seeking deeply reflective answers and geographically diverse samples may wish to consider using in-depth email interviews.

Measuring Patient-Centeredness of Care for Seriously Ill Individuals: Challenges and Opportunities for Accountability Initiatives

Patient- and family-centeredness of care is particularly important for individuals with serious illness. In this article, we describe methodological challenges of using measures of patient- and family-centeredness in accountability initiatives such as public reporting and pay for performance. We begin with background on measuring patient- and family-centered care using standardized surveys, describe evidence of the use of these measures for quality improvement, and highlight methodological challenges in the development and implementation of these measures for use in accountability. To ensure that patient- and family-centeredness is the cornerstone of public and private accountability initiatives designed to promote high-quality care to seriously ill patients, we recommend development of (1) a nationally endorsed survey instrument that assesses patient and family experiences of serious illness care across the full range of patient trajectories and care settings in which this care is provided; (2) administrative data infrastructure that allows for identification and outreach to the most knowledgeable respondents for the survey, regardless of the patient's setting of care; and (3) a broad toolkit of quality improvement approaches to ensure that as the emphasis on accountability grows, providers across settings have access to tools that can help them improve patient- and family-centeredness of care for the seriously ill.

Technological aids for improving longitudinal research on substance use disorders

Background

There is a broad consensus that addictive behaviors tend to be chronic and relapsing. But for field studies of substance users, successfully tracking, locating, and following up with a representative sample of subjects is a challenge.

Methods

The purpose of this paper is to provide a general overview of how current technological aids can support and improve the quality of longitudinal research on substance use disorders. The review is grouped into four domains: (1) tracking and locating, (2) prompting/engaging, (3) incentivizing, and (4) collecting data.

Results & conclusions

Although the technologies described in this review will be modified or replaced over time, our findings suggest that incorporating some or all of these currently available approaches may improve research efficiency, follow-up rates, and data quality.

The Development of the Recovery Assessments by Phone Points (RAPP): A Mobile Phone App for Postoperative Recovery Monitoring and Assessment

BACKGROUND

In Sweden, day surgery is performed in almost 2 million patients per year. Patient satisfaction is closely related to potential adverse events during the recovery process. A way to empower patients and give them the opportunity to affect care delivery is to let them evaluate their recovery process. The most common evaluation method is a follow-up telephone call by a nurse one or two days after surgery. In recent years, mHealth apps have been used to evaluate the nurse-patient relationship for self-management in chronic diseases or to evaluate pain after surgery. To the best of our knowledge, no previous research has explored the recovery process after day surgery via mobile phone in a Swedish cohort.

OBJECTIVE

The objective of the study is to describe the process of developing a mobile phone app using a Swedish Web-based Quality of Recovery (SwQoR) questionnaire to evaluate postoperative recovery after day surgery.

METHODS

The development process included five steps: (1) setting up an interdisciplinary task force, (2) evaluating the potential needs of app users, (3) developing the Swedish Web version of a QoR questionnaire, (4) constructing a mobile phone app, and (5) evaluating the interface and design by staff working in a day-surgery department and patients undergoing day surgery. A task force including specialists in information and communication technology, eHealth, and nursing care worked closely together to develop a Web-based app. Modifications to the QoR questionnaire were inspired by instruments used in the field of recovery for both children and adults. The Web-based app, Recovery Assessment by Phone Points (RAPP) consists of two parts: (1) a mobile app installed on the patient's private mobile phone, and (2) an administrator interface for the researchers.

RESULTS

The final version of the SwQoR questionnaire, which includes 31 items, was successfully installed in RAPP. The interface and the design were evaluated by asking for user opinions about the design and usefulness of the app with 10 day surgery patients. Some minor adjustments were made concerning text size and screen color.

CONCLUSIONS

Taking advantage of joint expertise, a useable Web-based app adaptable to different technical platforms was constructed. In addition, the SwQoR was successfully transferred into digital format for use on mobile phones.

Interformat reliability of digital psychiatric self-report questionnaires: a systematic review

Background

Research on Internet-based interventions typically use digital versions of pen and paper self-report symptom scales. However, adaptation into the digital format could affect the psychometric properties of established self-report scales. Several studies have investigated differences between digital and pen and paper versions of instruments, but no systematic review of the results has yet been done.

Objective

This review aims to assess the interformat reliability of self-report symptom scales used in digital or online psychotherapy research.

Methods

Three databases (MEDLINE, Embase, and PsycINFO) were systematically reviewed for studies investigating the reliability between digital and pen and paper versions of psychiatric symptom scales.

Results

From a total of 1504 publications, 33 were included in the review, and interformat reliability of 40 different symptom scales was assessed. Significant differences in mean total scores between formats were found in 10 of 62 analyses. These differences were found in just a few studies, which indicates that the results were due to study effects and sample effects rather than unreliable instruments. The interformat reliability ranged from r=.35 to r=.99; however, the majority of instruments showed a strong correlation between format scores. The quality of the included studies varied, and several studies had insufficient power to detect small differences between formats.

Conclusions

When digital versions of self-report symptom scales are compared to pen and paper versions, most scales show high interformat reliability. This supports the reliability of results obtained in psychotherapy research on the Internet and the comparability of the results to traditional psychotherapy research. There are, however, some instruments that consistently show low interformat reliability, suggesting that these conclusions cannot be generalized to all questionnaires. Most studies had at least some methodological issues with insufficient statistical power being the most common issue. Future studies should preferably provide information about the transformation of the instrument into digital format and the procedure for data collection in more detail.

Efficiencies of Internet-based digital and paper-based scientific surveys and the estimated costs and time for different-sized cohorts

AIMS

To evaluate the relative efficiencies of five Internet-based digital and three paper-based scientific surveys and to estimate the costs for different-sized cohorts.

METHODS

Invitations to participate in a survey were distributed via e-mail to employees of two university hospitals (E1 and E2) and to members of a medical association (E3), as a link placed in a special text on the municipal homepage regularly read by the administrative employees of two cities (H1 and H2), and paper-based to workers at an automobile enterprise (P1) and college (P2) and senior (P3) students. The main parameters analyzed included the numbers of invited and actual participants, and the time and cost to complete the survey. Statistical analysis was descriptive, except for the Kruskal-Wallis-H-test, which was used to compare the three recruitment methods. Cost efficiencies were compared and extrapolated to different-sized cohorts.

RESULTS

The ratios of completely answered questionnaires to distributed questionnaires were between 81.5% (E1) and 97.4% (P2). Between 6.4% (P1) and 57.0% (P2) of the invited participants completely answered the questionnaires. The costs per completely answered questionnaire were $0.57-$1.41 (E1-3), $1.70 and $0.80 for H1 and H2, respectively, and $3.36-$4.21 (P1-3). Based on our results, electronic surveys with 10, 20, 30, or 42 questions would be estimated to be most cost (and time) efficient if more than 101.6-225.9 (128.2-391.7), 139.8-229.2 (93.8-193.6), 165.8-230.6 (68.7-115.7), or 188.2-231.5 (44.4-72.7) participants were required, respectively.

CONCLUSIONS

The study efficiency depended on the technical modalities of the survey methods and engagement of the participants. Depending on our study design, our results suggest that in similar projects that will certainly have more than two to three hundred required participants, the most efficient way of conducting a questionnaire-based survey is likely via the Internet with a digital questionnaire, specifically via a centralized e-mail.