Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency

Differences in Center for Epidemiologic Studies Depression Scale, Generalized Anxiety Disorder-7 and Kessler Screening Scale for Psychological Distress Scores between Smartphone Version versus Paper Version Administration: Evidence of Equivalence

The use of electronic patient-reported outcomes has increased recently, and smartphones offer distinct advantages over other devices. However, previous systematic reviews have not investigated the reliability of the Center for Epidemiologic Studies Depression Scale (CES-D), Generalized Anxiety Disorder-7 (GAD-7), and Kessler Screening Scale for Psychological Distress (K6) when used with smartphones, and this has not been fully explored. This study aimed to evaluate the equivalence of the paper and smartphone versions of the CES-D, GAD-7, and K6, which were compared following a randomized crossover design method in 100 adults in Gunma, Japan. Participants responded to the paper and smartphone versions at 1-week intervals. The equivalence of paper and smartphone versions was evaluated using the intraclass correlation coefficient (ICC_agreement). The mean participant age was 19.86 years (SD = 1.08, 23% male). The ICC_agreements for the paper and smartphone versions of the CES-D, GAD-7, and K6 were 0.76 (95% confidence interval [CI] 0.66-0.83), 0.68 (95% CI 0.59-0.77), and 0.83 (95% CI 0.75-0.88), respectively. Thus, the CES-D and K6 scales are appropriate for use in a smartphone version, which could be applied to clinical and research settings in which the paper or smartphone versions could be used as needed.

Curricular Integration of Virtual Reality in Nursing Education

AIM

The purpose of this article was to evaluate the ability of an interactive virtual reality (VR) platform guided by standards of best practice to provide an effective immersive learning environment. We specifically evaluated usability of the platform and learners' perceptions of the experience.

BACKGROUND

A variety of strategies are needed to train a highly competent nursing workforce.

METHODS

We conducted a quantitative cross-sectional study to evaluate the VR experience using the System Usability Scale (SUS)^® and the Simulation Effectiveness Tool-Modified (SET-M).

RESULTS

Post-simulation evaluations were completed by 127 prelicensure and 28 advanced practice students. On the SUS scale, students found the overall VR system easy to navigate, and on the SET-M, they rated the VR experience positively.

CONCLUSION

Immersive technology such as VR with a defined curriculum and facilitated debriefing can be valuable for student learning and may ultimately effect patient care. [J Nurs Educ. 2023;62(X):XXX-XXX.].

Public awareness and attitudes toward biobank and sample donation: A regional Chinese survey

Background

The biobank is an extraordinary aid to research and scientific progress. Public involvement in biobanks, necessary for their development, is limited due to inadequate knowledge of biobanking and concerns about sample donation. This study explores the effectiveness of different publicity methods in improving participants' willingness to donate, and assesses public motivations and concerns. It aims to identify an efficient method of improving participants' awareness of biobanking and promoting sample donation.

Methods

A structured 20-item questionnaire was formulated to evaluate participants' knowledge of and attitudes toward biobanks and sample donation. In total, 1,500 questionnaires were disseminated to three groups of 500 participants who received, respectively, picture-based promotional material, text-based promotional material, or who attended a biobank-related lecture. Of these, 945 completed questionnaires were received. All the participants completed the questionnaires twice, before and after the corresponding publicity education.

Results

After each of the three methods of publicity based on text, pictures and a lecture, respondents' willingness to donate samples was significantly increased (P < 0.001), the lecture being more effective than the other two methods (P = 0.001). Participants with a medical background were more willing to donate biospecimens after publicity than those without medical backgrounds (P < 0.005) but had common motivations for donation including altruism and aiding medical research. The main concern hindering respondents' willingness to donate was the security of personal information.

Conclusion

Different types of biobank-related publicity based on text material, pictorial material and a lecture all improved respondents' willingness to donate and reduced concerns regarding sample donation. Medical background was a critical factor affecting attitudes toward sample donation after publicity. The results of this study suggest strategies that may popularize biobanks and enhance sample donation, further promoting the development of biobanks.

"Nobody Will Tell You. You've Got to Ask!": An Examination of Patient-Provider Communication Needs and Preferences among Black and White Women with Early-Stage Breast Cancer

Patient-provider communication is a critical component of healthcare and is associated with treatment quality and outcomes for women with breast cancer. This qualitative study examines similarities and differences in patient perspectives of communication needs between Black and White breast cancer survivors. We conducted four focus groups (N = 28) involving women with early-stage breast cancer on adjuvant endocrine therapy (AET), stratified by race and length of time on AET (< 6 months and >6 months). Each group was moderated by a race-concordant moderator and analyzed by emergent themes. Participants expressed common patient-provider communication needs, namely increased sensitivity from oncologists during the initial cancer diagnosis, personalized information to facilitate treatment decisions, emotional support during the transition from active treatment to maintenance, and rapid provider responses to mobile app-based queries. Communication differences by race also emerged. Black women were less likely than White women to describe having their informational needs met. White women praised longstanding relationships with providers, while Black women shared personal stories of disempowered interactions and noted the importance of patient advocates. White women more often reported privacy concerns about technology use. Unlike White women, Black women reported willingness to discuss sensitive topics, both online and offline, but believed those discussions made their providers feel uncomfortable. Early-stage breast cancer patients on AET, regardless of race, have similar needs for patient-centered communication with their oncologists. However, Black women were more likely to report experiencing poorer communication with providers than White women, which may be improved by technology and advocates.

Repeated Digitized Assessment of Risk and Symptom Profiles During Inpatient Treatment of Affective Disorder: Observational Study

BACKGROUND

Predictive models have revealed promising results for the individual prognosis of treatment response and relapse risk as well as for differential diagnosis in affective disorders. Yet, in order to translate personalized predictive modeling from research contexts to psychiatric clinical routine, standardized collection of information of sufficient detail and temporal resolution in day-to-day clinical care is needed. Digital collection of self-report measures by patients is a time- and cost-efficient approach to gain such data throughout treatment.

OBJECTIVE

The objective of this study was to investigate whether patients with severe affective disorders were willing and able to participate in such efforts, whether the feasibility of such systems might vary depending on individual patient characteristics, and if digitally acquired assessments were of sufficient diagnostic validity.

METHODS

We implemented a system for longitudinal digital collection of risk and symptom profiles based on repeated self-reports via tablet computers throughout inpatient treatment of affective disorders at the Department of Psychiatry at the University of Münster. Tablet-handling competency and the speed of data entry were assessed. Depression severity was additionally assessed by a clinical interviewer at baseline and before discharge.

RESULTS

Of 364 affective disorder patients who were approached, 242 (66.5%) participated in the study; 88.8% of participants (215/242) were diagnosed with major depressive disorder, and 27 (11.2%) had bipolar disorder. During the duration of inpatient treatment, 79% of expected assessments were completed, with an average of 4 completed assessments per participant; 4 participants (4/242, 1.6%) dropped out of the study prematurely. During data entry, 89.3% of participants (216/242) did not require additional support. Needing support with tablet handling and slower data entry pace were predicted by older age, whereas depression severity at baseline did not influence these measures. Patient self-reporting of depression severity showed high agreement with standardized external assessments by a clinical interviewer.

CONCLUSIONS

Our results indicate that digital collection of self-report measures is a feasible, accessible, and valid method for longitudinal data collection in psychiatric routine, which will eventually facilitate the identification of individual risk and resilience factors for affective disorders and pave the way toward personalized psychiatric care.

Barriers and Enablers to Using a Patient-Facing Electronic Questionnaire: A Qualitative Theoretical Domains Framework Analysis

BACKGROUND

Electronic patient questionnaires are becoming ubiquitous in health care. To address care gaps that contribute to poor asthma management, we developed the Electronic Asthma Management System, which includes a previsit electronic patient questionnaire linked to a computerized clinical decision support system.

OBJECTIVE

This study aims to identify the determinants (barriers and enablers) of patient uptake and completion of a previsit mobile health questionnaire.

METHODS

We conducted semistructured interviews with adult patients with asthma in Toronto, Canada. After demonstrating the questionnaire, participants completed the questionnaire using their smartphones and were then interviewed regarding perceived barriers and enablers to using and completing the questionnaire. Interview questions were based on the Theoretical Domains Framework to identify the determinants of health-related behavior. We generated themes that addressed the enablers and barriers to the uptake and completion of the questionnaire.

RESULTS

In total, 12 participants were interviewed for saturation. Key enablers were as follows: the questionnaire was easy to complete without additional knowledge or skills and was perceived as a priority and responsibility for patients, use could lead to more efficient and personalized care, completion on one's own time would be convenient, and uptake and completion could be optimized through patient reminders. Concerns about data security, the usefulness of questionnaire data, the stress of completing it accurately and on time, competing priorities, and preferences to complete the questionnaire on other devices were the main barriers.

CONCLUSIONS

The barriers and enablers identified by patients should be addressed by developing implementation strategies to enhance e-questionnaire use and completion by patients. As the use of e-questionnaires grows, our findings will contribute to implementation efforts across settings and diseases.

Primary Care Pre-Visit Electronic Patient Questionnaire for Asthma: Uptake Analysis and Predictor Modeling

Background

mHealth tablet-based interventions are increasingly being studied and deployed in various health care settings, yet little knowledge exists regarding patient uptake and acceptance or how patient demographics influence these important implementation metrics.

Objective

To determine which factors influence the uptake and successful completion of an mHealth tablet questionnaire by analyzing its implementation in a primary care setting.

Methods

We prospectively studied a patient-facing electronic touch-tablet asthma questionnaire deployed as part of the Electronic Asthma Management System. We describe tablet uptake and completion rates and corresponding predictor models for these behaviors.

Results

The tablet was offered to and accepted by patients in 891/1715 (52.0%) visits. Patients refused the tablet in 33.0% (439/1330) visits in which it was successfully offered. Patients aged older than 65 years of age (odds ratio [OR] 2.30, 95% CI 1.33-3.95) and with concurrent chronic obstructive pulmonary disease (OR 2.22, 95% CI 1.05-4.67) were more likely to refuse the tablet, and those on an asthma medication (OR 0.55, 95% CI 0.30-0.99) were less likely to refuse it. Once accepted, the questionnaire was completed in 784/891 (88.0%) instances, with those on an asthma medication (OR 0.53, 95% CI 0.32-0.88) being less likely to leave it incomplete.

Conclusions

Older age predicted initial tablet refusal but not tablet questionnaire completion, suggesting that perceptions of mHealth among older adults may negatively impact uptake, independent of usability. The influence of being on an asthma medication suggests that disease severity may also mediate mHealth acceptance. Although use of mHealth questionnaires is growing rapidly across health care settings and diseases, few studies describe their real-world acceptance and its predictors. Our results should be complemented by qualitative methods to identify barriers and enablers to uptake and may inform technological and implementation strategies to drive successful usage.

A Real-Time Automated Patient Screening System for Clinical Trials Eligibility in an Emergency Department: Design and Evaluation

Background

One critical hurdle for clinical trial recruitment is the lack of an efficient method for identifying subjects who meet the eligibility criteria. Given the large volume of data documented in electronic health records (EHRs), it is labor-intensive for the staff to screen relevant information, particularly within the time frame needed. To facilitate subject identification, we developed a natural language processing (NLP) and machine learning–based system, Automated Clinical Trial Eligibility Screener (ACTES), which analyzes structured data and unstructured narratives automatically to determine patients’ suitability for clinical trial enrollment. In this study, we integrated the ACTES into clinical practice to support real-time patient screening.

Objective

This study aimed to evaluate ACTES’s impact on the institutional workflow, prospectively and comprehensively. We hypothesized that compared with the manual screening process, using EHR-based automated screening would improve efficiency of patient identification, streamline patient recruitment workflow, and increase enrollment in clinical trials.

Methods

The ACTES was fully integrated into the clinical research coordinators’ (CRC) workflow in the pediatric emergency department (ED) at Cincinnati Children’s Hospital Medical Center. The system continuously analyzed EHR information for current ED patients and recommended potential candidates for clinical trials. Relevant patient eligibility information was presented in real time on a dashboard available to CRCs to facilitate their recruitment. To assess the system’s effectiveness, we performed a multidimensional, prospective evaluation for a 12-month period, including a time-and-motion study, quantitative assessments of enrollment, and postevaluation usability surveys collected from the CRCs.

Results

Compared with manual screening, the use of ACTES reduced the patient screening time by 34% (P<.001). The saved time was redirected to other activities such as study-related administrative tasks (P=.03) and work-related conversations (P=.006) that streamlined teamwork among the CRCs. The quantitative assessments showed that automated screening improved the numbers of subjects screened, approached, and enrolled by 14.7%, 11.1%, and 11.1%, respectively, suggesting the potential of ACTES in streamlining recruitment workflow. Finally, the ACTES achieved a system usability scale of 80.0 in the postevaluation surveys, suggesting that it was a good computerized solution.

Conclusions

By leveraging NLP and machine learning technologies, the ACTES demonstrated good capacity for improving efficiency of patient identification. The quantitative assessments demonstrated the potential of ACTES in streamlining recruitment workflow and improving patient enrollment. The postevaluation surveys suggested that the system was a good computerized solution with satisfactory usability.

Evaluation of Electronic and Paper-Pen Data Capturing Tools for Data Quality in a Public Health Survey in a Health and Demographic Surveillance Site, Ethiopia: Randomized Controlled Crossover Health Care Information Technology Evaluation

BACKGROUND

Periodic demographic health surveillance and surveys are the main sources of health information in developing countries. Conducting a survey requires extensive use of paper-pen and manual work and lengthy processes to generate the required information. Despite the rise of popularity in using electronic data collection systems to alleviate the problems, sufficient evidence is not available to support the use of electronic data capture (EDC) tools in interviewer-administered data collection processes.

OBJECTIVE

This study aimed to compare data quality parameters in the data collected using mobile electronic and standard paper-based data capture tools in one of the health and demographic surveillance sites in northwest Ethiopia.

METHODS

A randomized controlled crossover health care information technology evaluation was conducted from May 10, 2016, to June 3, 2016, in a demographic and surveillance site. A total of 12 interviewers, as 2 individuals (one of them with a tablet computer and the other with a paper-based questionnaire) in 6 groups were assigned in the 6 towns of the surveillance premises. Data collectors switched the data collection method based on computer-generated random order. Data were cleaned using a MySQL program and transferred to SPSS (IBM SPSS Statistics for Windows, Version 24.0) and R statistical software (R version 3.4.3, the R Foundation for Statistical Computing Platform) for analysis. Descriptive and mixed ordinal logistic analyses were employed. The qualitative interview audio record from the system users was transcribed, coded, categorized, and linked to the International Organization for Standardization 9241-part 10 dialogue principles for system usability. The usability of this open data kit-based system was assessed using quantitative System Usability Scale (SUS) and matching of qualitative data with the isometric dialogue principles.

RESULTS

From the submitted 1246 complete records of questionnaires in each tool, 41.89% (522/1246) of the paper and pen data capture (PPDC) and 30.89% (385/1246) of the EDC tool questionnaires had one or more types of data quality errors. The overall error rates were 1.67% and 0.60% for PPDC and EDC, respectively. The chances of more errors on the PPDC tool were multiplied by 1.015 for each additional question in the interview compared with EDC. The SUS score of the data collectors was 85.6. In the qualitative data response mapping, EDC had more positive suitability of task responses with few error tolerance characteristics.

CONCLUSIONS

EDC possessed significantly better data quality and efficiency compared with PPDC, explained with fewer errors, instant data submission, and easy handling. The EDC proved to be a usable data collection tool in the rural study setting. Implementation organization needs to consider consistent power source, decent internet connection, standby technical support, and security assurance for the mobile device users for planning full-fledged implementation and integration of the system in the surveillance site.

Development process and patient usability preferences for a touch screen tablet-based questionnaire

We sought to design a touch tablet asthma questionnaire while identifying patient preferences for usability features of such questionnaires. We created an evidence-based prototype and employed rapid-cycle design (semi-structured focus group testing, analysis, corresponding modifications, re-testing) with asthma patients aged ⩾16 years. We analyzed transcripts using deductive and inductive content analysis. Quantitative measures included Likert-type-scale responses, the System Usability Scale, and questionnaire completion times. There were 20 participants across five focus groups (15/20 female, age 49.1 ± 15.6 years). Usability-related themes included (1) "Touch Technology" (hygiene, touch technology familiarity, ease of use) and (2) "Questionnaire Design" (visual characteristics, navigation). Completion time was 11.7 ± 5.9 min. Summative Likert-type scale responses suggested high system usability, as did a System Usability Scale score of 84.2 ± 14.7. In summary, Attention to specific technology- and design-related preferences can result in a highly usable patient-facing touch tablet questionnaire. Our findings can inform touch questionnaire design across other diseases.

Patient preferences for a touch screen tablet-based asthma questionnaire

OBJECTIVE

Assessment of asthma control and provision of asthma action plans are seldom performed in practice, partly due to limited time for physicians to ascertain required information. A patient-facing electronic asthma questionnaire could facilitate information collection. We sought to design a touch-tablet asthma questionnaire for use in the clinic waiting room and to describe patient preferences for the content of such a questionnaire.

METHODS

We created a questionnaire prototype based on best evidence and employed rapid-cycle design (semi-structured focus group testing; analysis; corresponding modifications and re-testing) with asthma patients aged ≥16 years. We analyzed transcripts using deductive and inductive content analysis. Quantitative measures included Likert-scale questions and questionnaire completion times.

RESULTS

There were 20 participants across five focus groups (15/20 (75%) female, age 49.1 ± 15.6 years). Content-related themes included: 1) comprehensibility (language) (how questionnaire language affected ease and accuracy of comprehension) and b) information collection (issues arising during information collection in the following identified subthemes: personal asthma symptoms and triggers; asthma control; asthma medications and contact information). Average questionnaire completion time was 11.7 ± 5.9 min. Summative Likert scale responses suggested high levels of question comprehension and confidence with responses.

CONCLUSIONS

Our analysis provides novel insight about how best to formulate and present asthma-related content in an electronic questionnaire. Such questionnaires might facilitate quality improvement by improving efficiency of data collection, enabling better assessment of asthma control and medication adherence, and personalization of asthma action plans. Future studies should measure real-world uptake of such a questionnaire and impact on care.

Are Electronic and Paper Questionnaires Equivalent to Assess Patients with Overactive Bladder?

PURPOSE

Overactive bladder syndrome is defined as urinary urgency, usually accompanied by frequency and nocturia, with or without urgency urinary incontinence in the absence of urinary tract infection or another obvious pathological condition. Electronic questionnaires have been used in a few specialties with the hope of improving treatment outcomes and patient satisfaction. However, they have not been widely used in the urological field. When treating overactive bladder, the main outcome is to improve patient quality of life. The primary objective of this study was to evaluate whether electronic questionnaires would be equally accepted as or preferred to paper questionnaires. The secondary objective was to look at the preference in relation to patient age, education and iPad® tablet familiarity.

MATERIALS AND METHODS

We prospectively evaluated the iList® electronic questionnaire application using a friendly iPad tablet in patients with overactive bladder who presented to the urology clinic at our institution. Each of the 80 patients who were recruited randomly completed the validated OABSS (Overactive Bladder Symptom Score) and the PPBC (Patient Perception of Bladder Condition) questionnaires in paper and electronic format on the tablet. Variables potentially associated with the outcomes of interest included demographic data, questionnaire method preference, patient response rate and iPad familiarity. We used the 2-sided Z-test to determine whether the proportion of patients who considered the tablet to be the same, better or much better than paper was significantly greater than 50%. The 2-sided chi-square test was applied to assess whether the intervention effect significantly differed among the demographic subgroups.

RESULTS

A total of 80 patients 21 to 87 years old were enrolled in the study from November 2015 to August 2016. Of the patients 53% were female and 49% were 65 years or younger. The incidence of those who considered the tablet to be the same or better than paper was 82.5% (95% CI 74.2-90.8, p <0.001). The incidence of patients who considered the tablet to be the same or better than paper ranged from 76% to 97% regardless of age, gender and education subgroup as well as in those with any familiarity with the tablet (each p <0.001). Of the 20 patients who were not familiar with the tablet 45% preferred the electronic questionnaire (p = 0.654).

CONCLUSIONS

We found that the proportion of patients who considered electronic questionnaires to be equivalent to or better than paper versions was higher than those who preferred paper questionnaires regardless of age, gender or education level.

Changes in User Perceptions of a Telemedicine System Over Time: From Initial Implementation to Everyday Use

BACKGROUND

Benefits associated with telemedicine are contingent upon positive user perceptions. Despite this, research on user perceptions of telemedicine remains limited.

INTRODUCTION

Usability approaches offer a robust way to assess user perceptions, but have rarely been applied in telemedicine. In this study, a usability approach was employed to examine how user perceptions toward a telemedicine system changed over the course of everyday use.

MATERIALS AND METHODS

A telemedicine system was introduced to a hospital in the home service. Ten mobile nurses completed the System Usability Scale (SUS) after initial use, then again after 18 months of everyday use. Results were compared. Analysis included Bangor et al.'s (2009) adjective rating scale.

RESULTS

The initial SUS mean was 83 (standard deviation [SD] = 7.98), indicating "excellent" usability. After 18 months, the SUS mean was 64.38, indicating "OK" usability (SD = 14.25, p < 0.05, 95% confidence interval [CI]). Over time, users had lower desire to use the system frequently (p < 0.05, 95% CI), found it more complex (p < 0.05, 95% CI), and perceived greater inconsistency in its design (p < 0.05, 95% CI).

DISCUSSION

Considered with existing evidence, our usability findings indicate that a temporary period of positive user perceptions occurs when new telemedicine systems are used for the first few months. This fades with everyday use, with design inconsistency and perceived complexity becoming more noticeable. Although other factors such as user satisfaction and efficiency may also contribute, further studies are needed for confirmation.

CONCLUSIONS

User perceptions of telemedicine vary with time. To help maximize the benefits and longevity of telemedicine systems, responding to intermittent user appraisal is desirable.

Large-scale clinical implementation of PROMIS computer adaptive testing with direct incorporation into the electronic medical record

The objective of this research was to assess the implementation of collecting patient-reported outcomes data in the outpatient clinics of a large academic hospital and identify potential barriers and solutions to such an implementation. Three PROMIS computer adaptive test instruments, (1) physical function, (2) pain interference, and (3) depression, were administered at 23,813 patient encounters using a novel software platform on tablet computers. The average time to complete was 3.50 ± 3.12 min, with a median time of 2.60 min. Registration times for new patients did not change significantly, 6.87 ± 3.34 to 7.19 ± 2.69 min. Registration times increased for follow-up (p = .007) from 2.94 ± 1.57 (p < .01) min to 3.32 ± 1.78 min. This is an effective implementation strategy to collect patient-reported outcomes and directly import the results into the electronic medical record in real time for use during the clinical visit.

Management of chronic pruritus: from the dermatological office to the specialized itch center: a review

Patient care for those affected by chronic pruritus is remarkably complex due to its high prevalence and multifactorial nature. It requires a comprehensive assessment of the patient's medical history, extensive diagnostic procedures, and long treatment duration, including management of possible accompanying disorders such as sleep disturbances and mental distress. It is important to prioritize patient's needs when developing a therapeutic treatment plan. Standardized questionnaires and scales should be used to better analyze the patient history, quality of life, symptom intensity, and course of treatment. These can be distributed via digital platforms, allowing for more effective communication between the treating agents and gathering of large volumes of data in central databases. In today's health care system, it is essential for physicians with itch-related specializations and specialized itch centers to cooperate. It is thus crucial to focus efforts on the further development of specialized treatment centers and training courses for medical practitioners. There are, however, various regulatory and economic barriers to overcome in the modern health care system before patients with chronic pruritus can be offered the best possible care. Accordingly, health care authorities should be made aware of the difficulties associated with the management of chronic pruritus and of the high individual and societal burden it represents.

Towards Usable E-Health. A Systematic Review of Usability Questionnaires

BACKGROUND

The use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen.

OBJECTIVES

To identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality.

METHODS

We conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table.

RESULTS

Using a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7).

CONCLUSIONS

Existing questionnaires provide a foundation for research on e-health usability. However, future research is needed to broaden the coverage of the usability attributes and psychometric properties of the available questionnaires.

[Chronic pruritus : Care in daily practice]

Chronic pruritus is a highly prevalent, multifactorial symptom requiring extensive diagnostics, treatment and consideration of accompanying symptoms (reduced quality of life, sleep disorders, psychic factors). Patient care is thus complex and requires consideration of individual treatment goals. Patients indicate their wish for a symptom-free life an explanation of the causes and a trustful physician-patient relationship. The targeted use of questionnaires is thus advisable in order to structurally survey the history, pruritus intensity, quality of life and treatment progression. Nevertheless, there are many administrative and economical hurdles in the health care system to overcome in order to provide patients with chronic pruritus the best possible care, also per the recommended guidelines. The development of specialized centers and training courses for medical practitioners is thus urgently needed.

Two years of unintended consequences: introducing an electronic health record system in a hospice in Scotland

AIMS AND OBJECTIVES

To explore the impact of implementing an electronic health record system on staff at a Scottish hospice.

BACKGROUND

Electronic health records are broadly considered preferable to paper-based systems. However, changing from one system to the other is difficult. This study analysed the impact of this change in a Scottish hospice.

DESIGN

Naturalistic prospective repeated-measures mixed-methods approach.

METHODS

Data on the usability of the system, staff engagement and staff experience were obtained at four time points spanning 30 months from inception. Quantitative data were obtained from surveys, and qualitative from concurrent analysis of free-text comments and focus group. Participants were all 150 employees of a single hospice in Scotland.

RESULTS

Both system usability and staff engagement scores decreased for the first two years before recovering at 30 months. Staff experience data pointed to two main challenges: (1) Technical issues, with subthemes of accessibility and usability. (2) Cultural issues, with subthemes of time, teamwork, care provision and perception of change.

CONCLUSIONS

It took 30 months for system usability and staff engagement scores to rise, after falling significantly for the first two years. The unintended outcomes of implementation included challenges to the way the patient story was both recorded and communicated. Nevertheless, this process of change was found to be consistent with the 'J-curve' theory of organisational change, and as such, it is both predictable and manageable for other organisations.

RELEVANCE TO CLINICAL PRACTICE

It is known that implementing an electronic health record system is complex. This paper puts parameters on this complexity by defining both the nature of the complexity ('J' curve) and the time taken for the organisation to begin recovery from the challenges (two years). Understanding these parameters will help health organisations across the world plan more strategically.

Expectancy, usage and acceptance by general practitioners and patients: exploratory results from a study in the German outpatient sector

Objective

The study’s objective was to assess factors contributing to the use of smart devices by general practitioners (GPs) and patients in the health domain, while specifically addressing the situation in Germany, and to determine whether, and if so, how both groups differ in their perceptions of these technologies.

Methods

GPs and patients of resident practices in the Hannover region, Germany, were surveyed between April and June 2014. A total of 412 GPs in this region were invited by email to participate via an electronic survey, with 50 GPs actually doing so (response rate 12.1%). For surveying the patients, eight regional resident practices were visited by study personnel (once each). Every second patient arriving there (inclusion criteria: of age, fluent in German) was asked to take part (paper-based questionnaire). One hundred and seventy patients participated; 15 patients who did not give consent were excluded.

Results

The majority of the participating patients (68.2%, 116/170) and GPs (76%, 38/50) owned mobile devices. Of the patients, 49.9% (57/116) already made health-related use of mobile devices; 95% (36/38) of the participating GPs used them in a professional context. For patients, age (P < 0.001) and education (P < 0.001) were significant factors, but not gender (P > 0.99). For doctors, neither age (P = 0.73), professional experience (P > 0.99) nor gender (P = 0.19) influenced usage rates. For patients, the primary use case was obtaining health (service)-related information. For GPs, interprofessional communication and retrieving information were in the foreground. There was little app-related interaction between both groups.

Conclusions

GPs and patients use smart mobile devices to serve their specific interests. However, the full potentials of mobile technologies for health purposes are not yet being taken advantage of. Doctors as well as other care providers and the patients should work together on exploring and realising the potential benefits of the technology.

Usability evaluation of the SMART application for youth with mTBI

OBJECTIVE

There is a dearth of evidence-based treatments available to address the significant morbidity associated with mild traumatic brain injury (mTBI). To address this gap, we designed a novel user-friendly, web-based application. We describe the preliminary evaluation of feasibility and usability of the application to promote recovery following mTBI in youth, the Self-Monitoring Activity-Restriction and Relaxation Treatment (SMART). SMART incorporates real-time recommendations for individualized symptom management and activity restriction along with training in cognitive-behavioral coping strategies.

METHODS

We conducted a usability evaluation to assess and modify the SMART system prior to further study and deployment. Children ages 11-18 years presenting to the emergency department were recruited after symptoms resolved. Usability was assessed using a 60-min think-aloud protocol of teens and parents describing their interaction with the application. Upon completion of the tasks, each participant also completed the system usability scale (SUS).

RESULTS

We performed tests with 4 parent/child dyads. The average age of the children was 13 years (standard deviation=1.8). The parents were an average of 41.5 years old (standard deviation=6.2). Research revealed that the participants were enthusiastic about the interactive portions of the tool particularly the video based sessions. Parents were concerned about the speed at which their child might move through the program and the children thought that the system required large amounts of reading. Based on user feedback, researchers modified SMART to include an audio file in every module and improved the system's aesthetic properties. The mean SUS score was 85, with high SUS scores (>68) indicating satisfactory usability.

CONCLUSION

High initial usability and favorable user feedback provide a foundation for further iterative development and testing of the SMART application as a tool for managing recovery from concussion.

The Mediterranean healthy eating, ageing, and lifestyle (MEAL) study: rationale and study design

There is accumulating evidence suggesting that Mediterranean lifestyles, including nutrition and sleeping patterns as well as social integration, may play a role in reducing age-related diseases. However, the literature is mostly deficient of evidence provided by Italian Mediterranean islands that more closely adhered to the originally described lifestyles. In this paper, we described the rationale and the study design of the Mediterranean healthy Eating, Ageing, and Lifestyle (MEAL) study, a prospective population-based cohort established in Sicily, southern Italy. The main exposures investigated are classical determinants of health, including demographic, nutritional habits, smoking and physical activity status, as well as eating-related behaviors, sleeping habits, sun exposure, social resources, and perceived stress. Anthropometric measurements will be collected. The main outcomes included depression, quality of life, and, after the follow-up period, also cardiovascular disease and cancer. The MEAL study may provide important data to increase our knowledge regarding the prevalence, incidence, and risk factors of age-related disorders in the Mediterranean region.

[Experience in measuring the quality of treatment in interventional pain therapy : The Activity Index on a touchscreen PC]

BACKGROUND

There is a growing interest in patient-related outcome measurement. In this field questionnaires on touch screens are becoming more common. This study was designed to identify problems in usability and feasibility of a web-based questionnaire.

STUDY PARTICIPANTS AND METHODS

Patients who underwent a lumbar infiltration were recruited in 5 centers and 50 patients participated of which half were older than 62 years. One third of the patients had basically no former experience with computers or touch screens. The outcome was assessed before treatment and during follow-up on a simple web-based patient questionnaire, the Activity Index. Results were presented graphically and discussed during consultation. Patients, nurses and doctors were asked for standardized feedback.

RESULTS

Of the patients 84% completed the questionnaire in up to 6 min. An adapted form of the system usability scale (SUS) achieved an acceptance score of 71.8%. Problems in handling occurred mostly in older patients (>65 years). The system was scored with 72.9% and 78.5% in efficiency and handling, respectively, by the nurses. The attending physicians rated the usefulness and comprehensibility of the graphical representation of the results on average as 83.3% and both were scored neutral to positive; however, an average of 11.6% rated some aspects of the report to be suboptimal.

CONCLUSION

This web-based questionnaire is the first of its kind to be evaluated in everyday practice of interventional pain therapy for lumbar back pain. The vast majority of the patients were able to efficiently complete the questionnaire. The questionnaire was highly acceptable to patients, nurses and doctors. We found some usability problems but mainly in the older age group.

Assessing and comparing the usability of Chinese EHRs used in two Peking University hospitals to EHRs used in the US: A method of RUA

OBJECTIVES

The objective of this study was to investigate the usability level of Chinese hospital Electronic Health Records (EHRs) by assessing the completion times of EHRs for seven "meaningful use (MU)" relevant tasks conducted at two Chinese tertiary hospitals and comparing the results to those of relevant research conducted in US EHRs.

METHODS

Using Rapid Usability Assessment (RUA) developed by the National Center for Cognitive Informatics and Decision Making (NCCD), the usability of EHRs from two Peking University hospitals was assessed using a three-step Keystroke Level Model (KLM) in a laboratory environment.

RESULTS

(1) The total EHR task completion time for 7 MU relevant test tasks showed no significant differences between the two Chinese EHRs and their US counterparts, in which the time for thinking was relatively large and comprised 35.6% of the total time. The time for the electronic physician order was the largest. (2) For specific tasks, the mean completion times of the 2 hospital EHR systems spent on recording, modifying and searching (RMS) the medication orders were similar to those for the RMS radioactive tests; the mean time spent on the RMS laboratory test orders were much less. (3) There were 85 usability problems identified in the 2 hospital EHR systems.

DISCUSSION

In Chinese EHRs, a substantial amount of time is required to complete tasks relevant to MU targets and many preventable usability problems can be discovered. The task completion time of the 2 Chinese EHR systems was a little shorter than in the 5 reported US EHR systems, while the differences in smoking status and CPOE tasks were obvious; one main reason for these differences was the use of structured data entry.

CONCLUSIONS

The efficiency of Chinese and US EHRs was not significantly different. The key to improving the efficiency of both systems lies in expediting the Computerized physician order entry (CPOE) task. Many usability problems can be identified using heuristic assessments and improved by corresponding actions.

A systematic review of mixed methods research on human factors and ergonomics in health care

This systematic literature review provides information on the use of mixed methods research in human factors and ergonomics (HFE) research in health care. Using the PRISMA methodology, we searched four databases (PubMed, PsycInfo, Web of Science, and Engineering Village) for studies that met the following inclusion criteria: (1) field study in health care, (2) mixing of qualitative and quantitative data, (3) HFE issues, and (4) empirical evidence. Using an iterative and collaborative process supported by a structured data collection form, the six authors identified a total of 58 studies that primarily address HFE issues in health information technology (e.g., usability) and in the work of healthcare workers. About two-thirds of the mixed methods studies used the convergent parallel study design where quantitative and qualitative data were collected simultaneously. A variety of methods were used for collecting data, including interview, survey and observation. The most frequent combination involved interview for qualitative data and survey for quantitative data. The use of mixed methods in healthcare HFE research has increased over time. However, increasing attention should be paid to the formal literature on mixed methods research to enhance the depth and breadth of this research.

A prospective study of electronic quality of life assessment using tablet devices during and after treatment of head and neck cancers

OBJECTIVES

Electronic data collection is increasingly used for quality of life (QOL) assessments in the field of oncology. It is important to assess the feasibility of these new data capture technologies.

MATERIALS AND METHODS

Patients at our institution who were 18 years or older with a pathological diagnosis of head and neck cancer were prospectively enrolled. Each patient completed two questionnaires [EORTC-QLQ-C30 and EORTC-QLQ-H&N35] administered on a touch-screen tablet device (iPad) at initial consult, during treatment, at the completion of treatment and at each subsequent follow up visit for one year after treatment.

RESULTS

A total of 50 patients were included in this study. Although all patients completed the surveys at the initial consult, 86% of initially enrolled patients completed surveys at the end of radiation treatment, and 48% of initially enrolled patients completed surveys by the fourth follow-up visit. Average time to complete the survey for all patients over all time points was 9.8 min (standard deviation 6.1). Age as a continuous variable was significantly associated with time for survey completion (p<0.001), with older age associated with longer survey completion times.

CONCLUSION

QOL assessment using tablet devices in head and neck cancer patients is feasible, but may be more challenging in elderly patients. Patients ⩾70 years old may benefit from more assistance with electronic forms and should be allotted more time for completing tablet-based QOL surveys.

Mammography Decision Aid Reduces Decisional Conflict for Women in Their Forties Considering Screening

BACKGROUND

Clinical guidelines recommend a personalized approach to mammography screening for women in their forties; however, methods to do so are lacking. An evidence-based mammography screening decision aid was developed as an electronic mobile application and evaluated in a before-after study.

METHODS

The decision aid (Mammopad) included modules on breast cancer, mammography, risk assessment, and priority setting about screening. Women aged 40-49 years who were patients of rural primary care clinics, had no major risk factors for breast cancer, and no mammography during the previous year were invited to use the decision aid. Twenty women participated in pretesting of the decision aid and 75 additional women completed the before-after study. The primary outcome was decisional conflict measured before and after using Mammopad. Secondary outcomes included decision self-efficacy and intention to begin or continue mammography screening. Differences comparing measures before versus after use were determined using Wilcoxon signed rank tests.

RESULTS

After using Mammopad, women reported reduced decisional conflict based on mean Decisional Conflict Scale scores overall (46.33 versus 8.33; Z = -7.225; p < 0.001) and on all subscales (p < 0.001). Women also reported increased mean Decision Self-Efficacy Scale scores (79.67 versus 95.73; Z = 6.816, p < 0.001). Although 19% of women changed their screening intentions, this was not statistically significant.

CONCLUSIONS

Women reported less conflict about their decisions for mammography screening, and felt more confident to make decisions after using Mammopad. This approach may help guide women through the decision making process to determine personalized screening choices that are appropriate for them.

Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods

BACKGROUND

Self-administered survey questionnaires are an important data collection tool in clinical practice, public health research and epidemiology. They are ideal for achieving a wide geographic coverage of the target population, dealing with sensitive topics and are less resource-intensive than other data collection methods. These survey questionnaires can be delivered electronically, which can maximise the scalability and speed of data collection while reducing cost. In recent years, the use of apps running on consumer smart devices (i.e., smartphones and tablets) for this purpose has received considerable attention. However, variation in the mode of delivering a survey questionnaire could affect the quality of the responses collected.

OBJECTIVES

To assess the impact that smartphone and tablet apps as a delivery mode have on the quality of survey questionnaire responses compared to any other alternative delivery mode: paper, laptop computer, tablet computer (manufactured before 2007), short message service (SMS) and plastic objects.

SEARCH METHODS

We searched MEDLINE, EMBASE, PsycINFO, IEEEXplore, Web of Science, CABI: CAB Abstracts, Current Contents Connect, ACM Digital, ERIC, Sociological Abstracts, Health Management Information Consortium, the Campbell Library and CENTRAL. We also searched registers of current and ongoing clinical trials such as ClinicalTrials.gov and the World Health Organization (WHO) International Clinical Trials Registry Platform. We also searched the grey literature in OpenGrey, Mobile Active and ProQuest Dissertation & Theses. Lastly, we searched Google Scholar and the reference lists of included studies and relevant systematic reviews. We performed all searches up to 12 and 13 April 2015.

SELECTION CRITERIA

We included parallel randomised controlled trials (RCTs), crossover trials and paired repeated measures studies that compared the electronic delivery of self-administered survey questionnaires via a smartphone or tablet app with any other delivery mode. We included data obtained from participants completing health-related self-administered survey questionnaire, both validated and non-validated. We also included data offered by both healthy volunteers and by those with any clinical diagnosis. We included studies that reported any of the following outcomes: data equivalence; data accuracy; data completeness; response rates; differences in the time taken to complete a survey questionnaire; differences in respondent's adherence to the original sampling protocol; and acceptability to respondents of the delivery mode. We included studies that were published in 2007 or after, as devices that became available during this time are compatible with the mobile operating system (OS) framework that focuses on apps.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data from the included studies using a standardised form created for this systematic review in REDCap. They then compared their forms to reach consensus. Through an initial systematic mapping on the included studies, we identified two settings in which survey completion took place: controlled and uncontrolled. These settings differed in terms of (i) the location where surveys were completed, (ii) the frequency and intensity of sampling protocols, and (iii) the level of control over potential confounders (e.g., type of technology, level of help offered to respondents). We conducted a narrative synthesis of the evidence because a meta-analysis was not appropriate due to high levels of clinical and methodological diversity. We reported our findings for each outcome according to the setting in which the studies were conducted.

MAIN RESULTS

We included 14 studies (15 records) with a total of 2275 participants; although we included only 2272 participants in the final analyses as there were missing data for three participants from one included study.Regarding data equivalence, in both controlled and uncontrolled settings, the included studies found no significant differences in the mean overall scores between apps and other delivery modes, and that all correlation coefficients exceeded the recommended thresholds for data equivalence. Concerning the time taken to complete a survey questionnaire in a controlled setting, one study found that an app was faster than paper, whereas the other study did not find a significant difference between the two delivery modes. In an uncontrolled setting, one study found that an app was faster than SMS. Data completeness and adherence to sampling protocols were only reported in uncontrolled settings. Regarding the former, an app was found to result in more complete records than paper, and in significantly more data entries than an SMS-based survey questionnaire. Regarding adherence to the sampling protocol, apps may be better than paper but no different from SMS. We identified multiple definitions of acceptability to respondents, with inconclusive results: preference; ease of use; willingness to use a delivery mode; satisfaction; effectiveness of the system informativeness; perceived time taken to complete the survey questionnaire; perceived benefit of a delivery mode; perceived usefulness of a delivery mode; perceived ability to complete a survey questionnaire; maximum length of time that participants would be willing to use a delivery mode; and reactivity to the delivery mode and its successful integration into respondents' daily routine. Finally, regardless of the study setting, none of the included studies reported data accuracy or response rates.

AUTHORS' CONCLUSIONS

Our results, based on a narrative synthesis of the evidence, suggest that apps might not affect data equivalence as long as the intended clinical application of the survey questionnaire, its intended frequency of administration and the setting in which it was validated remain unchanged. There were no data on data accuracy or response rates, and findings on the time taken to complete a self-administered survey questionnaire were contradictory. Furthermore, although apps might improve data completeness, there is not enough evidence to assess their impact on adherence to sampling protocols. None of the included studies assessed how elements of user interaction design, survey questionnaire design and intervention design might influence mode effects. Those conducting research in public health and epidemiology should not assume that mode effects relevant to other delivery modes apply to apps running on consumer smart devices. Those conducting methodological research might wish to explore the issues highlighted by this systematic review.

Tablet-based screening improves continence management in multiple sclerosis

Objective

To investigate whether electronic continence questionnaires aid early identification and optimizes management of sphincter dysfunction in a multiple sclerosis clinic.

Methods

A custom designed, tablet-based cross-platform software tool was designed to capture validated multiple sclerosis (MS) patient-reported outcomes. An unselected cohort of MS patients from a tertiary referral clinic completed electronic tablet-based versions of the Bladder Control Scale (BLCS) and the Bowel Control Scale in the waiting room. Data were captured wirelessly “on-the-fly” and stored in a deidentified, secure database; and individual questionnaire results were immediately available to the treating neurologist in the electronic medical record. Scores of ≥2 on either questionnaire generated an automated electronic referral to the clinic MS continence nurse (MS CN).

Results

One hundred and fifty-seven MS patients completed a total of 184 electronic continence test sets and on two occasions only the BLCS was completed. An automatic electronic referral for formal continence review was generated 128 times in 108 patients. Fifty-seven formal continence assessments were undertaken by the MS CN following automated referral. All reviews resulted in at least one clinical intervention being made.

Interpretation

Tablet-based data capture and automated continence referral using this software tool is an efficient, sensitive, and feasible method of screening MS patients for bladder and bowel dysfunction. Concordance with the results of formal continence assessment in this pilot study validates the use of this technology as a screening tool.

Empirical studies on usability of mHealth apps: a systematic literature review

The release of smartphones and tablets, which offer more advanced communication and computing capabilities, has led to the strong emergence of mHealth on the market. mHealth systems are being used to improve patients' lives and their health, in addition to facilitating communication between doctors and patients. Researchers are now proposing mHealth applications for many health conditions such as dementia, autism, dysarthria, Parkinson's disease, and so on. Usability becomes a key factor in the adoption of these applications, which are often used by people who have problems when using mobile devices and who have a limited experience of technology. The aim of this paper is to investigate the empirical usability evaluation processes described in a total of 22 selected studies related to mHealth applications by means of a Systematic Literature Review. Our results show that the empirical evaluation methods employed as regards usability could be improved by the adoption of automated mechanisms. The evaluation processes should also be revised to combine more than one method. This paper will help researchers and developers to create more usable applications. Our study demonstrates the importance of adapting health applications to users' need.

Tablet, web-based, or paper questionnaires for measuring anxiety in patients suspected of breast cancer: patients' preferences and quality of collected data

BACKGROUND

Electronic applications are increasingly being used in hospitals for numerous purposes.

OBJECTIVE

Our aim was to assess differences in the characteristics of patients who choose paper versus electronic questionnaires and to evaluate the data quality of both approaches.

METHODS

Between October 2012 and June 2013, 136 patients participated in a study on diagnosis-induced stress and anxiety. Patients were asked to fill out questionnaires at six different moments during the diagnostic phase. They were given the opportunity to fill out the questionnaires on paper or electronically (a combination of tablet and Web-based questionnaires). Demographic characteristics and completeness of returned data were compared between groups.

RESULTS

Nearly two-thirds of patients (88/136, 64.7%) chose to fill out the questionnaires on paper, and just over a third (48/136, 35.3%) preferred the electronic option. Patients choosing electronic questionnaires were significantly younger (mean 47.3 years vs mean 53.5 in the paper group, P=.01) and higher educated (P=.004). There was significantly more missing information (ie, at least one question not answered) in the paper group during the diagnostic day compared to the electronic group (using a tablet) (28/88 vs 1/48, P<.001). However, in the week after the diagnostic day, missing information was significantly higher in the electronic group (Web-based questionnaires) compared to the paper group (41/48 vs 38/88, P<.001).

CONCLUSIONS

Younger patients and patients with a higher level of education have a preference towards filling out questionnaires electronically. In the hospital, a tablet is an excellent medium for patients to fill out questionnaires with very little missing information. However, for filling out questionnaires at home, paper questionnaires resulted in a better response than Web-based questionnaires.

Efficiencies of Internet-based digital and paper-based scientific surveys and the estimated costs and time for different-sized cohorts

AIMS

To evaluate the relative efficiencies of five Internet-based digital and three paper-based scientific surveys and to estimate the costs for different-sized cohorts.

METHODS

Invitations to participate in a survey were distributed via e-mail to employees of two university hospitals (E1 and E2) and to members of a medical association (E3), as a link placed in a special text on the municipal homepage regularly read by the administrative employees of two cities (H1 and H2), and paper-based to workers at an automobile enterprise (P1) and college (P2) and senior (P3) students. The main parameters analyzed included the numbers of invited and actual participants, and the time and cost to complete the survey. Statistical analysis was descriptive, except for the Kruskal-Wallis-H-test, which was used to compare the three recruitment methods. Cost efficiencies were compared and extrapolated to different-sized cohorts.

RESULTS

The ratios of completely answered questionnaires to distributed questionnaires were between 81.5% (E1) and 97.4% (P2). Between 6.4% (P1) and 57.0% (P2) of the invited participants completely answered the questionnaires. The costs per completely answered questionnaire were $0.57-$1.41 (E1-3), $1.70 and $0.80 for H1 and H2, respectively, and $3.36-$4.21 (P1-3). Based on our results, electronic surveys with 10, 20, 30, or 42 questions would be estimated to be most cost (and time) efficient if more than 101.6-225.9 (128.2-391.7), 139.8-229.2 (93.8-193.6), 165.8-230.6 (68.7-115.7), or 188.2-231.5 (44.4-72.7) participants were required, respectively.

CONCLUSIONS

The study efficiency depended on the technical modalities of the survey methods and engagement of the participants. Depending on our study design, our results suggest that in similar projects that will certainly have more than two to three hundred required participants, the most efficient way of conducting a questionnaire-based survey is likely via the Internet with a digital questionnaire, specifically via a centralized e-mail.

Evaluating tablet computers as a survey tool in rural communities

PURPOSE

Although tablet computers offer advantages in data collection over traditional paper-and-pencil methods, little research has examined whether the 2 formats yield similar responses, especially with underserved populations. We compared the 2 survey formats and tested whether participants' responses to common health questionnaires or perceptions of usability differed by survey format. We also tested whether we could replicate established paper-and-pencil findings via tablet computer.

METHODS

We recruited a sample of low-income community members living in the rural southern United States. Participants were 170 residents (black = 49%; white = 36%; other races and missing data = 15%) drawn from 2 counties meeting Florida's state statutory definition of rural with 100 persons or fewer per square mile. We randomly assigned participants to complete scales (Center for Epidemiologic Studies Depression Inventory and Regulatory Focus Questionnaire) along with survey format usability ratings via paper-and-pencil or tablet computer. All participants rated a series of previously validated posters using a tablet computer. Finally, participants completed comparisons of the survey formats and reported survey format preferences.

FINDINGS

Participants preferred using the tablet computer and showed no significant differences between formats in mean responses, scale reliabilities, or in participants' usability ratings.

CONCLUSIONS

Overall, participants reported similar scales responses and usability ratings between formats. However, participants reported both preferring and enjoying responding via tablet computer more. Collectively, these findings are among the first data to show that tablet computers represent a suitable substitute among an underrepresented rural sample for paper-and-pencil methodology in survey research.

Does single-source create an added value? Evaluating the impact of introducing x4T into the clinical routine on workflow modifications, data quality and cost-benefit

OBJECTIVES

The first objective of this study is to evaluate the impact of integrating a single-source system into the routine patient care documentation workflow with respect to process modifications, data quality and execution times in patient care as well as research documentation. The second one is to evaluate whether it is cost-efficient using a single-source system in terms of achieved savings in documentation expenditures.

METHODS

We analyzed the documentation workflow of routine patient care and research documentation in the medical field of pruritus to identify redundant and error-prone process steps. Based on this, we established a novel documentation workflow including the x4T (exchange for Trials) system to connect hospital information systems with electronic data capture systems for the exchange of study data. To evaluate the workflow modifications, we performed a before/after analysis as well as a time-motion study. Data quality was assessed by measuring completeness, correctness and concordance of previously and newly collected data. A cost-benefit analysis was conducted to estimate the savings using x4T per collected data element and the additional costs for introducing x4T.

RESULTS

The documentation workflow of patient care as well as clinical research was modified due to the introduction of the x4T system. After x4T implementation and workflow modifications, half of the redundant and error-prone process steps were eliminated. The generic x4T system allows direct transfer of routinely collected health care data into the x4T research database and avoids manual transcription steps. Since x4T has been introduced in March 2012, the number of included patients has increased by about 1000 per year. The average entire documentation time per patient visit has been significantly decreased by 70.1% (from 1116±185 to 334±83 s). After the introduction of the x4T system and associated workflow changes, the completeness of mandatory data elements raised from 82.2% to 100%. In case of the pruritus research study, the additional costs for introducing the x4T system are €434.01 and the savings are 0.48ct per collected data element. So, with the assumption of a 5-year runtime and 82 collected data elements per patient, the amount of documented patients has to be higher than 1102 to create a benefit.

CONCLUSION

Introduction of the x4T system into the clinical and research documentation workflow can optimize the data collection workflow in both areas. Redundant and cumbersome process steps can be eliminated in the research documentation, with the result of reduced documentation times as well as increased data quality. The usage of the x4T system is especially worthwhile in a study with a large amount of collected data or a high number of included patients.

Obtaining patient feedback in an outpatient lithotripsy service is facilitated by use of a touch-screen tablet (iPad™) survey

There is now a requirement for every doctor in the UK to obtain patient feedback for revalidation. This can be an onerous and time-consuming task. The objective of this study was to evaluate a novel electronic patient feedback method in an outpatient lithotripsy service setting. Between September 2013 and January 2014, 100 patients attending an outpatient lithotripsy service in Oxford were asked to complete a selection of pre-approved NHS questions about the service they had received. Questions were presented on a tablet device (iPad™) and answered using the touch screen. Departmental staff were unaware of the questions in the survey. Patients were asked to complete the survey by an independent research nurse. Questions were created online in a free-to-use web-based survey application and presented on the tablet device in a user-friendly format via an application. Data were uploaded via wifi™ to the online system. Data were viewed, automatically analysed and displayed graphically. The age range of the patients surveyed was 20-80 years of age. All 100 patients completed the survey without difficulty. All patients answered every question. Data could be automatically viewed, analysed and presented graphically. This method of collecting patient feedback proved to be rapid and efficient. The feedback highlighted a high patient satisfaction with the lithotripsy service. A touch screen tablet device is an efficient and effective method of collecting truly objective patient feedback. This method of patient feedback could be employed in other clinical environments to collect data for revalidation purposes.

Electronic health records: new opportunities for clinical research

Clinical research is on the threshold of a new era in which electronic health records (EHRs) are gaining an important novel supporting role. Whilst EHRs used for routine clinical care have some limitations at present, as discussed in this review, new improved systems and emerging research infrastructures are being developed to ensure that EHRs can be used for secondary purposes such as clinical research, including the design and execution of clinical trials for new medicines. EHR systems should be able to exchange information through the use of recently published international standards for their interoperability and clinically validated information structures (such as archetypes and international health terminologies), to ensure consistent and more complete recording and sharing of data for various patient groups. Such systems will counteract the obstacles of differing clinical languages and styles of documentation as well as the recognized incompleteness of routine records. Here, we discuss some of the legal and ethical concerns of clinical research data reuse and technical security measures that can enable such research while protecting privacy. In the emerging research landscape, cooperation infrastructures are being built where research projects can utilize the availability of patient data from federated EHR systems from many different sites, as well as in international multilingual settings. Amongst several initiatives described, the EHR4CR project offers a promising method for clinical research. One of the first achievements of this project was the development of a protocol feasibility prototype which is used for finding patients eligible for clinical trials from multiple sources.

CADe system integrated within the electronic health record

The latest technological advances and information support systems for clinics and hospitals produce a wide range of possibilities in the storage and retrieval of an ever-growing amount of clinical information as well as in detection and diagnosis. In this work, an Electronic Health Record (EHR) combined with a Computer Aided Detection (CADe) system for breast cancer diagnosis has been implemented. Our objective is to provide to radiologists a comprehensive working environment that facilitates the integration, the image visualization, and the use of aided tools within the EHR. For this reason, a development methodology based on hardware and software system features in addition to system requirements must be present during the whole development process. This will lead to a complete environment for displaying, editing, and reporting results not only for the patient information but also for their medical images in standardised formats such as DICOM and DICOM-SR. As a result, we obtain a CADe system which helps in detecting breast cancer using mammograms and is completely integrated into an EHR.

[Assessment of pruritus - current standards and implications for clinical practice : consensus paper of the Action Group Pruritus Parameter of the International Working Group on Pruritus Research (AGP)]

With a prevalence of 20%, chronic pruritus is a symptom of many diseases with major impact on healthcare costs. The lack of specific therapeutic measures makes the development of new drugs and their testing in clinical trials urgent. It is not possible to measure pruritus in an objective way. For these reasons, it is necessary to have a series of standardized measures to characterize pruritus in a reliable way. Intensity scales such as the visual analog scale (VAS) are most frequently used to document the course of the symptoms. However, for assessing pruritus intensity, VAS is not an optimal instrument, although it cannot be dispensed with. The VAS should be combined with other scales in clinical studies in order to internally test the consistency of data. Other instruments for assessing intensity and course of pruritus are in the process of development. Presently scratch activity and scratch-associated lesions can be documented in a descriptive fashion. There are some studies that have employed devices to document scratch activity; however, methodological studies are not yet available. The patient-benefit index is an indispensable tool in clinical trials. A questionnaire for gathering data on the history and some pruritus-specific parameters has been developed and published. Questionnaires on patient quality of life, anxiety and depression are helpful in obtaining data on other cost-relevant parameters. A questionnaire on the quality of life, for instance, can provide important help in the assessment of the burden of the disease. The results of these questionnaires can be correlated with data on pruritus intensity scales. The relevant questionnaires have been partially digitalized so that they are available immediately as part of patient care. Additional methodological developments and studies are required in order to define a robust set of instruments for measuring pruritus in daily practice and in clinical studies.

Mapping local laboratory interface terms to LOINC at a German university hospital using RELMA V.5: a semi-automated approach

OBJECTIVE

Logical Observation Identifiers Names and Codes (LOINC) mapping of laboratory data is often a question of the effort of mapping compared with the benefits of the structure achieved. The new LOINC mapping assistant RELMA (version 2011) has the potential to reduce the effort required for semi-automated mapping. We examined quality, time effort, and sustainability of such mapping.

METHODS

To verify the mapping quality, two samples of 100 laboratory terms were extracted from the laboratory system of a German university hospital and processed in a semi-automated fashion with RELMA V.5 and LOINC V.2.34 German translation DIMDI to obtain LOINC codes. These codes were reviewed by two experts from each of two laboratories. Then all 2148 terms used in these two laboratories were processed in the same way.

RESULTS

In the initial samples, 93 terms from one laboratory system and 92 terms from the other were correctly mapped. Of the total 2148 terms, 1660 could be mapped. An average of 500 terms per day or 60 terms per hour could be mapped. Of the laboratory terms used in 2010, 99% could be mapped.

DISCUSSION

Semi-automated LOINC mapping of non-English laboratory terms has become promising in terms of effort and mapping quality using the new version RELMA V.5. The effort is probably lower than for previous manual mapping. The mapping quality equals that of manual mapping and is far better than that reported with previous automated mapping activities.

CONCLUSION

RELMA V.5 and LOINC V.2.34 offer the opportunity to start thinking again about LOINC mapping even in non-English languages, since mapping effort is acceptable and mapping results equal those of previous manual mapping reports.