Evaluating the Accuracy of Google Translate for Diabetes Education Material

Performance of ChatGPT and Google Translate for Pediatric Discharge Instruction Translation

BACKGROUND AND OBJECTIVES

Patients who speak languages other than English face barriers to equitable healthcare delivery. Machine translation systems, including emerging large language models, have the potential to expand access to translation services, but their merits and limitations in clinical practice remain poorly defined. We aimed to assess the performance of Google Translate and ChatGPT for multilingual translation of pediatric discharge instructions.

METHODS

Twenty standardized discharge instructions for pediatric conditions were translated into Spanish, Brazilian Portuguese, and Haitian Creole by professional translation services, Google Translate and ChatGPT-4.0, and evaluated for adequacy (preserved information), fluency (grammatical correctness), meaning (preserved connotation), and severity (clinical harm), along with assessment of overall preference. Domain-level ratings and preferred translation source were summarized with descriptive statistics and compared with professional translations.

RESULTS

Google Translate and ChatGPT demonstrated similar domain-level ratings to professional translations for Spanish and Portuguese. For Haitian Creole, compared with both Google Translate and ChatGPT, professional translations demonstrated significantly greater adequacy, fluency meaning, and severity scores. ChatGPT (33.3%, P < .001) and Google Translate (23.3%, P = .024) contained more potentially clinically significant errors (severity score ≤3) for Haitian Creole than professional translations (8.3%). Professional Haitian Creole (48.3%) and Portuguese (43.3%), but not Spanish (15%), translations were most frequently preferred among translation sources.

CONCLUSIONS

Machine translation platforms have comparable performance to professional translations for Spanish and Portuguese but shortcomings in quality, accuracy, and preference persist for Haitian Creole. Diverse multilingual training data are needed, along with regulations ensuring safe and equitable applications of machine translation in clinical practice.

Usability of technological tools to overcome language barriers in health care: a scoping review protocol

INTRODUCTION

In many healthcare contexts globally, where the languages of care providers and patients do not match, miscommunication or non-communication can lead to inaccurate diagnoses and subpar treatment outcomes. In order to bridge these language barriers, a range of informal practices are used, such as family members or staff acting as interpreters, 'receptive multilingualism' or machine translation. The development and use of technological tools are increasing, but factors such as translation quality for complex health-related texts vary widely between languages. The objective of this scoping review is to (1) identify and describe the technological tools used in direct patient-provider communication to overcome a language barrier in a healthcare setting, (2) identify how the usability of these tools was evaluated and (3) assess the usability of the technological tools.

METHODS AND ANALYSIS

The scoping review will follow the Joanna Briggs Institute methodology. A search strategy using variations of the keywords 'technological tools', 'language barrier' and 'healthcare' will be applied in the following databases and research platforms: PubMed, PsycArticle, Scopus, EBSCOhost, ProQuest and Web of Science. All literature where individuals use a technological tool to overcome a language barrier in a healthcare context will be included and exported into the screening assistant software Rayyan. The search will be limited to articles written in German or English. Two independent reviewers will screen the articles, and all relevant extracted data will be presented in a descriptive summary.

ETHICS AND DISSEMINATION

This scoping review does not require ethical approval, as the study's methodology consists of collecting data from publicly available sources. The findings will be disseminated through publication in an open-access, peer-reviewed journal and presentations at scientific conferences. The scoping review results will also guide future research in a multinational project investigating multilingualism in providing (mental) healthcare to migrants.

Customization options in consumer health information materials on type-2 diabetes mellitus-an analysis of modifiable features in different types of media

Introduction

The understanding of health-related information is essential for making informed decisions. However, providing health information in an understandable format for everyone is challenging due to differences in consumers' health status, disease knowledge, skills, and preferences. Tailoring health information to individual needs can improve comprehension and increase health literacy.

Objective

The aim of our research was to analyze the extent to which consumers can customize consumer health information materials (CHIMs) for type-2 diabetes mellitus through various media types.

Methods

We conducted a comprehensive search for various CHIMs across various media types, such as websites, apps, videos, and printed or printable forms. A representative sample of CHIMs was obtained for analysis through blocked randomization across the various media types. We conducted a quantitative content analysis to determine the frequency of user-centered customization options. Cross-comparisons were made to identify trends and variations in modifiable features among the media.

Results

In our representative sample of 114 CHIMs, we identified a total of 24 modifiable features, which we grouped into five main categories: (i) language, (ii) text, (iii) audiovisual, (iv) presentation, and (v) medical content. Videos offered the most customization opportunities (95%), while 47% of websites and 26% of apps did not allow users to tailor health information. None of the printed or printable materials provided the option to customize the information. Overall, 65% of analyzed CHIMs did not allow users to tailor health information according to their needs.

Conclusion

Our results show that CHIMs for type-2 diabetes mellitus could be significantly improved by providing more customization options for users. Further research is needed to investigate the effectiveness and usability of these options to enhance the development and appropriate provision of modifiable features in health information.

Providing care to marginalised communities: a qualitative study of community pharmacy teams

BACKGROUND

Health inequalities in the UK are widening, particularly since the COVID-19 pandemic. Community pharmacies are the most visited healthcare provider in England and are ideally placed to provide and facilitate access to care for those most disadvantaged.

AIM

To explore the experiences and needs of community pharmacy teams in providing care for marginalised groups and how this has changed since the COVID-19 pandemic.

DESIGN AND SETTING

A qualitative study in community pharmacy and across primary care.

METHOD

Semi-structured interviews were undertaken with members of community pharmacy teams, primary care network (PCN) pharmacists, GPs, and nurses in the North of England.

RESULTS

In total, 31 individuals participated in an interview (26 pharmacy staff, three GPs, and two nurses). Most participants acknowledged that their pharmacy had become busier since COVID-19 because of increased footfall compounded by patient difficulties in navigating remote digital systems. Few participants had received any formal training on working with marginalised communities; however, organisational barriers (such as lack of access to translation facilities) combined with interorganisational barriers (such as lack of integrated care) made it more difficult to provide care for some marginalised groups. Despite this, the continuity of care provided by many pharmacies was viewed as an important factor in enabling marginalised groups to access and receive care.

CONCLUSION

There are opportunities to better utilise the skills of community pharmacy teams. Resources, such as access to translation services, and interventions to enable better communication between community pharmacy teams and other primary care services, such as general practice, are essential.

The Use of Machine Translation for Outreach and Health Communication in Epidemiology and Public Health: Scoping Review

BACKGROUND

Culturally and linguistically diverse groups are often underrepresented in population-based research and surveillance efforts, leading to biased study results and limited generalizability. These groups, often termed "hard-to-reach," commonly encounter language barriers in the public health (PH) outreach material and information campaigns, reducing their involvement with the information. As a result, these groups are challenged by 2 effects: the medical and health knowledge is less tailored to their needs, and at the same time, it is less accessible for to them. Modern machine translation (MT) tools might offer a cost-effective solution to PH material language accessibility problems.

OBJECTIVE

This scoping review aims to systematically investigate current use cases of MT specific to the fields of PH and epidemiology, with a particular interest in its use for population-based recruitment methods.

METHODS

PubMed, PubMed Central, Scopus, ACM Digital Library, and IEEE Xplore were searched to identify articles reporting on the use of MT in PH and epidemiological research for this PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews)-compliant scoping review. Information on communication scenarios, study designs and the principal findings of each article were mapped according to a settings approach, the World Health Organization monitoring and evaluation framework and the service readiness level framework, respectively.

RESULTS

Of the 7186 articles identified, 46 (0.64%) were included in this review, with the earliest study dating from 2009. Most of the studies (17/46, 37%) discussed the application of MT to existing PH materials, limited to one-way communication between PH officials and addressed audiences. No specific article investigated the use of MT for recruiting linguistically diverse participants to population-based studies. Regarding study designs, nearly three-quarters (34/46, 74%) of the articles provided technical assessments of MT from 1 language (mainly English) to a few others (eg, Spanish, Chinese, or French). Only a few (12/46, 26%) explored end-user attitudes (mainly of PH employees), whereas none examined the legal or ethical implications of using MT. The experiments primarily involved PH experts with language proficiencies. Overall, more than half (38/70, 54% statements) of the summarizing results presented mixed and inconclusive views on the technical readiness of MT for PH information.

CONCLUSIONS

Using MT in epidemiology and PH can enhance outreach to linguistically diverse populations. The translation quality of current commercial MT solutions (eg, Google Translate and DeepL Translator) is sufficient if postediting is a mandatory step in the translation workflow. Postediting of legally or ethically sensitive material requires staff with adequate content knowledge in addition to sufficient language skills. Unsupervised MT is generally not recommended. Research on whether machine-translated texts are received differently by addressees is lacking, as well as research on MT in communication scenarios that warrant a response from the addressees.

Validation testing of a language translation device for suitability in assisting Australian radiation therapists to communicate with Mandarin-speaking patients

Introduction

Clear, timely communication between practitioners and patients is key in ensuring equitable access to health services and optimal care. Australia's linguistically diverse population adds complexity to healthcare provision. This paper describes a validation study to assess clinical suitability of a language translation device, intended for use with Mandarin speaking patients undergoing radiotherapy (RT).

Materials and methods

After a comprehensive device selection process, common phrases used in RT practice were curated within one clinical center and translated by interpreters. Phrases were categorized by conversation type and readability (according to Flesch-Kincaid and FORCAST scores). Validation of device performance was undertaken by purposely selected radiation therapists (RTTs) who tested and evaluated the device using a survey with 5-point Likert scale responses. Statistical analysis was undertaken on Excel using Pearson's chi-square, z-test, interrater reliability/agreement and linear regression analyses.

Results

Six RTTs and two interpreters volunteered to participate in this study. 188 common phrases were spoken verbatim into the device and scored on a 5-point Likert scale, yielding an overall output accuracy of 66%. A z-test confirmed significance against prior comparative research and Linear regression analysis observed improved output between consecutive participants. 62.7% of interpreter scores were identical; a further 29.1% constituted a single point scoring variation. Poorer outcomes were observed with colloquial English and lower readability.

Conclusions

This study found the device produced suitable translation accuracy and identified language styles that should be avoided with use. Further research could consider clinical application, expanded languages and/or health disciplines, and development of a national RTT phrase list.

Does Access to Point-of-Care Medical Information Improve Trauma and General Surgeons' Clinical Knowledge in a Middle-Income Country? A Mixed-Methods Study with Random Assignment

BACKGROUND

Investing in continued medical education strengthens surgical systems. This study assessed the effectiveness of an evidence-based practice (EBP) tutorial and access to UpToDate (UTD) to improve EBP and understand how and why providers practice using evidence.

STUDY DESIGN

This is a mixed-methods, implementation study at 9 public hospitals in Peru consisting of a didactic session for surgeons on EBP and Google Translate and support of applications for UTD access. Change in clinical knowledge scores (CKS), access and use of UTD, and impact of language pre-and postintervention were measured. Qualitative interviews uncovered reasons for these changes.

RESULTS

Intervention participants had lower CKS at follow-up compared with baseline (odds ratio [OR] of higher score 0.41 [0.18,0.98]; p = 0.044), and this effect was modified (p = 0.003) to the extent that the reverse was true for control participants (OR 2.30 [1.13,4.71]; p = 0.022). Participants with 1 to 20 years of experience had significantly improved CKS compared with students/residents (1 to 10 years: OR 4.5 [1.1,18]; 11 to 20 years: OR 4.9 [1.4,17]); there was no evidence of a different CKS between providers with >20 years of experience compared with students/residents (OR 1.3 [0.5,3.7]). Administrative disconnect, usability, motivation, education, time, resources, and age influenced point-of-care medical information systems impact on knowledge and EBP. Participants reporting low English proficiency translated medical literature mostly used Google Translate. Those with low/no English reading proficiency had higher odds of reporting a negative impact on research than those with working (p = 0.007) or professional (p < 0.001) proficiency.

CONCLUSIONS

Providing education on EBP, free UTD access, and translation solutions did not correlate with increased CKS due to complex barriers to using point-of-care medical information systems.

Machine vs. Radiologist-Based Translations of RadLex: Implications for Multi-language Report Interoperability

The purpose of this study was to evaluate the feasibility of translation of RadLex lexicon from English to German performed by Google Translate, using the RadLex ontology as ground truth. The same comparison was also performed for German to English translations. We determined the concordance rate of the Google Translate-rendered translations (for both English to German and German to English translations) to the official German RadLex (translations provided by the German Radiological Society) and English RadLex terms via character-by-character concordance analysis (string matching). Specific term characteristics of term character count and word count were compared between concordant and discordant translations using t-tests. Google Translate-rendered translations originally considered incongruent (2482 English terms and 2500 German terms) were then reviewed by German and English-speaking radiologists to further evaluate clinical utility. Overall success rates of both methods were calculated by adding the percentage of terms marked correct by string comparison to the percentage marked correct during manual review extrapolated to the terms that had been initially marked incorrect during string analysis. 64,632 English and 47,425 German RadLex terms were analyzed. 3507 (5.4%) of the Google Translate-rendered English to German translations were concordant with the official German RadLex terms when evaluated via character-by-character concordance. 3288 (6.9%) of the Google Translate-rendered German to English translations matched the corresponding English RadLex terms. Human review of a random sample of non-concordant machine translations revealed that 95.5% of such English to German translations were understandable, whereas 43.9% of such German to English translations were understandable. Combining both string matching and human review resulted in an overall Google Translate success rate of 95.7% for English to German translations and 47.8% for German to English translations. For certain radiologic text translation tasks, Google Translate may be a useful tool for translating multi-language radiology reports into a common language for natural language processing and subsequent labeling of datasets for machine learning. Indeed, string matching analysis alone is an incomplete method for evaluating machine translation. However, when human review of automated translation is also incorporated, measured performance improves. Additional evaluation using longer text samples and full imaging reports is needed. An apparent discordance between English to German versus German to English translation suggests that the direction of translation affects accuracy.

A Pragmatic Assessment of Google Translate for Emergency Department Instructions

BACKGROUND

Because many hospitals have no mechanism for written translation, ED providers resort to the use of automated translation software, such as Google Translate (GT) for patient instructions. A recent study of discharge instructions in Spanish and Chinese suggested that accuracy rates of Google Translate (GT) were high.

STUDY OBJECTIVE

To perform a pragmatic assessment of GT for the written translation of commonly used ED discharge instructions in seven commonly spoken languages.

METHODS

A prospective assessment of the accuracy of GT for 20 commonly used ED discharge instruction phrases, as evaluated by a convenience sample of native speakers of seven commonly spoken languages (Spanish, Chinese, Vietnamese, Tagalog, Korean, Armenian, and Farsi). Translations were evaluated using a previously validated matrix for scoring machine translation, containing 5-point Likert scales for fluency, adequacy, meaning, and severity, in addition to a dichotomous assessment of retention of the overall meaning.

RESULTS

Twenty volunteers evaluated 400 google translated discharge statements. Volunteers were 50% female and spoke Spanish (5), Armenian (2), Chinese (3), Tagalog (4), Korean (2), and Farsi (2). The overall meaning was retained for 82.5% (330/400) of the translations. Spanish had the highest accuracy rate (94%), followed by Tagalog (90%), Korean (82.5%), Chinese (81.7%), Farsi (67.5%), and Armenian (55%). Mean Likert scores (on a 5-point scale) were high for fluency (4.2), adequacy (4.4), meaning (4.3), and severity (4.3) but also varied.

CONCLUSION

GT for discharge instructions in the ED is inconsistent between languages and should not be relied on for patient instructions.

Testing the use of translation apps to overcome everyday healthcare communication in Australian aged-care hospital wards-An exploratory study

Aims and objectives

To trial three mobile translation apps in the healthcare setting to address language barriers in everyday care between healthcare staff and older people with limited English proficiency (LEP).

Design

A mixed‐methods exploratory study.

Methods

A two‐month trial of three translation apps was conducted across four aged‐care hospital wards. Observed interactions during use of translation apps were recorded, and staff surveys regarding the use of translation apps were collected at the end of the trial. Data were analysed using descriptive statistics and thematic content analysis of open‐ended responses in the surveys and observations. Findings from the thematic content analysis are reported using the Standards for Reporting of Qualitative Research (SRQR) checklist.

Results

Translation apps were mostly used for identifying pain and assisting with activities of daily living. Qualitative findings revealed that translation apps aided staff in providing care and improved rapport; however, practical shortcomings were identified.

Enabling Nurse-Patient Communication With a Mobile App: Controlled Pretest-Posttest Study With Nurses and Non-English-Speaking Patients

BACKGROUND

There is growing concern regarding the implications of miscommunication in health care settings, the results of which can have serious detrimental impacts on patient safety and health outcomes. Effective communication between nurses and patients is integral in the delivery of timely, competent, and safe care. In a hospital environment where care is delivered 24 hours a day, interpreters are not always available. In 2014, we developed a communication app to support patients' interactions with allied health clinicians when interpreters are not present. In 2017, we expanded this app to meet the needs of the nursing workforce. The app contains a fixed set of phrases translated into common languages, and communication is supported by text, images, audio content, and video content.

OBJECTIVE

This study aims to evaluate the efficacy of the communication app to support nursing staff during the provision of standard care to patients from non-English-speaking backgrounds when an interpreter is not available.

METHODS

This study used a one-group pretest-posttest sequential explanatory mixed methods research design, with quantitative data analyzed using inferential statistics and qualitative data analyzed via thematic content analysis. A total of 134 observation sessions (82 pretest and 52 posttest) of everyday nurse-patient interactions and 396 app use sessions were recorded. In addition, a total of 134 surveys (82 pretest and 52 posttest) with nursing staff, 7 interviews with patients, and 3 focus groups with a total of 9 nursing staff participants were held between January and November 2017.

RESULTS

In the absence of the app, baseline interactions with patients from English-speaking backgrounds were rated as more successful (t₈₀=5.69; P<.001) than interactions with patients from non-English-speaking backgrounds. When staff used the app during the live trial, interactions with patients from non-English-speaking backgrounds were rated as more successful than interactions without the app (F_2,119=8.17; P<.001; η²=0.37). In addition, the level of staff frustration was rated lower when the app was used to communicate (t₈₀=2.71; P=.008; r=0.29). Most participants indicated that the app assisted them in communicating.

CONCLUSIONS

Through the use of the app, a number of patients from non-English-speaking backgrounds experienced better provision of standard care, similar to their English-speaking peers. Thus, the app can be seen as contributing to the delivery of equitable health care.

Accuracy of Google Translate in translating the directions and counseling points for top-selling drugs from English to Arabic, Chinese, and Spanish

PURPOSE

A study was conducted to evaluate the accuracy of Google Translate (Google LLC, Mountain View, CA) when used to translate directions for use and counseling points for the top 100 drugs used in the United States into Arabic, Chinese (simplified), and Spanish.

METHODS

Directions for use and common counseling points for the top 100 drugs were identified by 2 clinicians. This information was translated from English to Arabic, Chinese (simplified), and Spanish using Google Translate. Two nonclinician, bilingual native speakers of each language back-translated the Google Translate translation into English and determined if the sentence made sense in their native language. Two clinicians reviewed the back-translations to determine the clinical significance of each inaccurate translation.

RESULTS

For the top 100 drugs, 38 unique directions for use and 170 unique counseling points were identified for translation. For the 38 directions for use, 29 (76.3%) of the Arabic translations were accurate, 34 (89.5%) of the Chinese (simplified) translations were accurate, and 27 (71%) of the Spanish translations were accurate. For the 170 counseling points, 92 (54.1%) of the Arabic translations were accurate, 130 (76.5%) of the Chinese (simplified) translations were accurate, and 65 (38.2%) of the Spanish translations were accurate. Of the 247 inaccurate translations, 72 (29.1%) were classified as highly clinically significant or potentially life-threatening.

CONCLUSION

Certified translators should be used to translate directions for use and common counseling points for prescription medications into Arabic, Chinese (simplified), and Spanish. Clinicians should be aware of the risk of inaccurate translation when Google Translate is used.

Care Staff Perspectives on Using Mobile Technology to Support Communication in Long-Term Care: Mixed Methods Study

BACKGROUND

Long-term care (LTC) homes provide 24-hour care for people living with complex care needs. LTC staff assist older adults living with chronic conditions such as Alzheimer disease, related dementias, and stroke, which can cause communication disorders. In addition to the complex cognitive challenges that can impact communication, further difficulties can arise from cultural-language differences between care staff and residents. Breakdowns in caregiver-resident communication can negatively impact the delivery of person-centered care. Recent advances in mobile technology, specifically mobile devices (tablets and smartphones) and their software apps, offer innovative solutions for supporting everyday communication between care staff and residents. To date, little is known about the care staff's perspectives on the different ways that mobile technology could be used to support communication with residents.

OBJECTIVE

This study aims to identify care staff's perspectives on the different ways of using devices and apps to support everyday communication with adults living in LTC homes and the priority care areas for using mobile technology to support communication with residents.

METHODS

This descriptive study employed concept mapping methods to explore care staff's perspectives about ways of using mobile technology with residents and to identify the usefulness, practicality, and probable uses of mobile technology to support communication in priority care areas. Concept mapping is an integrated mixed methods approach (qualitative and quantitative) that uses a structured process to identify priority areas for planning and evaluation. In total, 13 care staff from a single LTC home participated in this study. Concept mapping includes 2 main data collection phases: (1) statement generations through brainstorming and (2) statement structuring through sorting and rating. Brainstorming took place in person in a group session, whereas sorting and rating occurred individually after the brainstorming session. Concept mapping data were analyzed using multidimensional scaling and cluster analysis to generate numerous interpretable data maps and displays.

RESULTS

Participants generated 67 unique statements during the brainstorming session. Following the sorting and rating of the statements, a concept map analysis was performed. In total, 5 clusters were identified: (1) connect, (2) care management, (3) facilitate, (4) caregiving, and (5) overcoming barriers. Although all 5 clusters were rated as useful, with a mean score of 4.1 to 4.5 (Likert: 1-5), the care staff rated cluster 2 (care management) as highest on usefulness, practicality, and probable use of mobile technology to support communication in LTC.

CONCLUSIONS

This study provided insight into the viewpoints of care staff regarding the different ways mobile technology could be used to support caregiver-resident communication in LTC. Our findings suggest that care management, facilitating communication, and overcoming barriers are 3 priority target areas for implementing mobile health interventions to promote person-centered care and resident-centered care.

The perceptions of translation apps for everyday health care in healthcare workers and older people: A multi-method study

AIMS AND OBJECTIVES

To understand the attitudes and perceptions of older people with limited English proficiency (LEP) and healthcare workers to using mobile translation technology for overcoming language barriers in the healthcare setting.

BACKGROUND

Australia's cohort of people aged 65 and over has a sizeable population with LEP. In healthcare settings, difficulties with communication may potentially result in inadequate care. Mobile language translation applications have been identified as a potential way to improve communication between patients and healthcare staff when used as an adjunct to professional interpreters in low-risk scenarios; however, the perceptions of the use of mobile translation applications for such communication is unknown.

METHODS

A multi-method design was used. Focus group discussions were conducted with older people from culturally and linguistically diverse (CALD) backgrounds and nursing and allied health professionals to understand their perceptions of translation technology. Qualitative data were analysed using inductive content analysis. Qualitative findings were reported using the Standards for Reporting of Qualitative Research (SRQR) checklist. Participants also appraised three existing translation apps via survey and results were analysed using descriptive statistics.

RESULTS

Overall, older people from CALD backgrounds (n = 12) and healthcare staff (n = 17) agreed that translation technology could play a role in reducing communication barriers. There was enthusiasm amongst older people to learn and use the technology, while healthcare staff saw the potential to address communication barriers in their own work. Barriers identified by older people and healthcare staff included: accuracy of translation and phrases, possible technological learning curves, risk of mistranslation in high-risk conversation and inability to check accuracy of translation. Fixed-phrase translation apps were seen as more favourable than real-time voice-to-voice mobile translation applications.

CONCLUSIONS

Older people from CALD backgrounds and healthcare staff were open to the use of mobile translation applications for everyday healthcare communications.

RELEVANCE TO CLINICAL PRACTICE

Translation applications may have a role in reducing language barriers in everyday healthcare communication but context, accuracy and ease of use need to be considered.

[Usability and Efficiency of a Digital Communication Assistance Tool to Obtain Medical History from Non-German-Speaking Patients]

AIM

In the DICTUM-Friedland study, a medical history-taking app was implemented in an initial reception facility for refugees and asylum seekers. This app can be used to obtain from patients information about the most frequent general medical complaints and risk factors in 13 different languages or dialects. A report (synopsis) is generated, based on each patient's given data, in the German language. Usability and efficiency of obtaining medical history were the criteria evaluated in order to test and optimize the app for use in a broader general medical setting.

METHODS

From May to December 2018, the app was tested in the waiting room of the general medical ward in the refugee camp in Friedland in southern Lower Saxony, Germany. Patients were asked to complete a short digital questionnaire about the usability of the app immediately after finishing the query. Sociodemographic data were partly collected with the same questionnaire and partly extracted from patient records. The efficiency was evaluated on the basis of the correspondence between the entered complaints (ICPC-2) and the diagnoses of the doctors (ICD-10).

RESULTS

In total, the app was used 353 times, and based on these, 283 medical histories were taken. The median time for entering the complaints was 10:27 minutes. The use of the audio output (60% of patients used this option) did not influence the usability. The majority of the users (76%) rated the app as easy to use and 65% of the users stated that they were able to enter their main complaints. Both assessments were independent of the age and sex of the users, but the educational level had a slightly more positive influence. The efficiency test showed that 51% (n=144) of the synopses matched fully and 28% (n=79) of the synopses matched partially with the medical diagnoses.

CONCLUSION

The systematic survey of the most frequent general medical complaints enables a more precise medical history taking in patients with whom a normal interview is not possible due to language barriers. Thus it serves as a support for the medical history taking. The app is easy to use and not prone to the errors of online translations.

A Digital Communication Assistance Tool (DCAT) toObtain Medical History from Foreign-LanguagePatients: Development and Pilot Testing in a PrimaryHealth Care Center for Refugees

Background: Language barriers play a critical role in the treatment of migrant and refugee patients. In Germany, primary care interpreters are often not available especially in rural areas or if patients demand spontaneous or urgent consultations. Methods: In order to enable patients and their physicians to communicate effectively about the current illness history, we developed a digital communication assistance tool (DCAT) for 19 different languages and dialects. This paper reports the multidisciplinary process of the conceptual design and the iterative development of this cross-cultural user-centered application in an action-oriented approach. Results: We piloted our app with 36 refugee patients prior to a clinical study and used the results for further development. The acceptance and usability of the app by patients was high. Conclusion: Using digital tools for overcoming language barriers can be a feasible approach when providing health care to foreign-language patients.

European Rural Development Policy Approaching Health Issues: An Exploration of Programming Schemes

Malnutrition, obesity, type 2 diabetes, micronutrient deficiencies, and the increase in non-communicable diseases are among the future European key challenges in health and welfare. Agriculture and rural development policies can positively contribute to a healthier and nutritious supply of food. The objective of the research is to analyze to what extent European 2007-2013 and 2014-2020 rural development programmes address the nexus between agriculture, food, health, and nutrition to respond to the evolving dietary needs. The research carries out a quali-quantitative content analysis on all 210 European rural development programmes. Results show that the interconnection between agriculture, food, health, and nutrition is present, with differences in the European agricultural and rural policy programming periods. The main interlinking issues of the nexus are food safety, food quality, diseases, nutritional aspect, animal health and welfare, plant health, and environmental health. Healthier and nutritious food-related issues are emerging, addressing dietary needs, and sustaining consumer food trends. Healthy and nutritious food is pursued by combating foodborne communicable diseases and non-communicable diseases. The future Common Agricultural Policy, including its rural dimensions, should support the consumption of healthy foods produced in ways that are environmentally and economically sustainable.

Evaluating the Usefulness of Translation Technologies for Emergency Response Communication: A Scenario-Based Study

Background

In the United States, language barriers pose challenges to communication in emergency response and impact emergency care delivery and quality for individuals who are limited English proficient (LEP). There is a growing interest among Emergency Medical Services (EMS) personnel in using automated translation tools to improve communications with LEP individuals in the field. However, little is known about whether automated translation software can be used successfully in EMS settings to improve communication with LEP individuals.

Objective

The objective of this work is to use scenario-based methods with EMS providers and nonnative English-speaking users who identified themselves as LEP (henceforth referred to as LEP participants) to evaluate the potential of two automated translation technologies in improving emergency communication.

Methods

We developed mock emergency scenarios and enacted them in simulation sessions with EMS personnel and Spanish-speaking and Chinese-speaking (Mandarin) LEP participants using two automated language translation tools: an EMS domain-specific fixed-sentence translation tool (QuickSpeak) and a statistical machine translation tool (Google Translate). At the end of the sessions, we gathered feedback from both groups through a postsession questionnaire. EMS participants also completed the System Usability Scale (SUS).

Results

We conducted a total of 5 group sessions (3 Chinese and 2 Spanish) with 12 Chinese-speaking LEP participants, 14 Spanish-speaking LEP participants, and 17 EMS personnel. Overall, communications between EMS and LEP participants remained limited, even with the use of the two translation tools. QuickSpeak had higher mean SUS scores than Google Translate (65.3 vs 48.4; P=.04). Although both tools were deemed less than satisfactory, LEP participants showed preference toward the domain-specific system with fixed questions (QuickSpeak) over the free-text translation tool (Google Translate) in terms of understanding the EMS personnel’s questions (Chinese 11/12, 92% vs 3/12, 25%; Spanish 12/14, 86% vs 4/14, 29%). While both EMS and LEP participants appreciated the flexibility of the free-text tool, multiple translation errors and difficulty responding to questions limited its usefulness.

Conclusions

Technologies are emerging that have the potential to assist with language translation in emergency response; however, improvements in accuracy and usability are needed before these technologies can be used safely in the field.

CALD Assist-Nursing: Improving communication in the absence of interpreters

AIMS AND OBJECTIVES

To develop a communication app to support nursing staff during the provision of standard care of patients from non-English-speaking backgrounds (NESBs), when an interpreter is not available. This paper reports on the user needs analysis phase that informed the development, content and functionality of the app.

BACKGROUND

In 2014 we developed CALD Assist, a communication app to support patient interactions with allied health clinicians when interpreters are not present. It includes 95 commonly used phrases professionally interpreted into 10 languages and grouped by discipline. This work expands upon our previous app to meet the needs of the nursing workforce.

DESIGN

Qualitative through focus groups, observations and interviews, with a quantitative component from observational data and staff surveys.

METHODS

Four focus groups with hospital staff, ten interviews with patients from NESBs and 85 observation sessions of everyday patient-staff interactions followed by staff surveys (n = 85) were held between January and June 2017.

RESULTS

Baseline data prior to app development revealed that staff confidence of the patients' level of understanding and the success of the interaction were significantly greater for English-speaking (ES) patients, than for non-English-speaking patients. A total of 143 phrases were identified and subdivided into 16 categories for inclusion in the new app.

CONCLUSION

Staff participants highlighted that patients from NESBs are a challenging patient group to interact with. Patient and staff participants identified a range of areas where the nursing app could benefit, including pain management, mobility, hygiene and nutrition.

RELEVANCE TO CLINICAL PRACTICE

The proposed app can be used to reduce variances in practice and provide a timely and positive patient experience for patients from NESBs who are unable to communicate in English during hospital admissions.

Assessing Health Literacy Among Chinese Speakers in the U.S. with Limited English Proficiency

Background:

Limited English proficiency compounds the problem of low health literacy, making certain population groups in the United States especially vulnerable to health disparities.

Objective:

This study clarified the mechanisms underlying low health literacy among people with limited English proficiency using a modified theory-based health literacy assessment survey.

Methods:

We modified and tested the All Aspects of Health Literacy Scale (AAHLS) with a sample of Chinese speakers who have limited English proficiency in the U.S. The AAHLS is a theory-based health literacy survey assessing functional health literacy, interactive health literacy, information appraisal, and empowerment. We adapted the survey, created dual language scenarios (English and Chinese), translated the questions into Chinese, and conducted cognitive interviews to revise the questions. We examined the health literacy score distributions and performed Confirmatory Factor Analysis (CFA) to evaluate the appropriateness of our modified AAHLS to elicit valid data.

Key Results:

A total of 405 participants completed our AAHLS survey. Compared to the English language scenario, aside from the item assessing if participants would question health care providers, participants had significantly higher health literacy levels when they were immersed in communication using Chinese (p < .001). We also found that more than three-quarters of the participants were not likely to question their doctor's and nurse's advice regardless of language scenarios and most of them had limited empowerment capabilities at the level of community and social engagement. The CFA results showed that the modified Chinese model exhibited good fit (RMSEA [root mean square error of approximation] = 0.06, CFI [the comparative fit index] = 0.98, TLI [Tucker-Lewis index] = 0.97, WRMR [weighted root mean square residual] = 1).

Conclusions:

The results showed that our modified AAHLS yielded reliable and valid data among U.S. Chinese speakers. Researchers should consider native languages and cultural differences before conducting health literacy assessments. Public health professionals should incorporate health interventions and policy approaches to improve Chinese immigrants' English proficiency and empowerment capabilities. [HLRP: Health Literacy Research and Practice. 2018;2(2):e94–e106.]

Plain Language Summary:

To clarify the mechanisms underlying low health literacy among populations with limited English proficiency (LEP) in the United States, this study aims to tailor a theory-based health literacy survey with dual-language scenarios among Chinese speakers with LEP. The modified survey yielded reliable and valid data. Participants had higher health literacy levels when they were immersed in communication using Chinese rather than English.

Developing Digital Facilitation of Assessments in the Absence of an Interpreter: Participatory Design and Feasibility Evaluation With Allied Health Groups

BACKGROUND

To ensure appropriate and timely care, interpreters are often required to aid communication between clinicians and patients from non-English speaking backgrounds. In a hospital environment, where care is delivered 24 hours a day, interpreters are not always available. Subsequently, culturally and linguistically diverse (CALD) patients are sometimes unable to access timely assessment because of clinicians' inability to communicate directly with them.

OBJECTIVE

The aim of this study was to design and evaluate CALD Assist, a tablet app to assist communication between patients and allied health clinicians in the absence of an interpreter. CALD Assist uses key phrases translated into common languages and uses pictorial, written, and voice-over prompts to facilitate communication during basic patient assessment.

METHODS

CALD Assist's design, functionality, and content were determined through focus groups with clinicians and informed by interpreting and cultural services. An evaluation was conducted in a live trial phase on eight wards across 2 campuses of a hospital in Victoria, Australia.

RESULTS

A commercial grade CALD Assist mobile app for five disciplines within allied health was developed and evaluated. The app includes a total of 95 phrases in ten different languages to assist clinicians during their initial assessment. Evaluation results show that clinicians' confidence in their assessment increased with use of the CALD Assist app: clinicians' reports of "complete confidence" increased from 10% (3/30) to 42% (5/12), and assessment reports of "no confidence" decreased from 57% (17/30) to 17% (2/12). Average time required to complete an assessment with patients from non-English speaking backgrounds reduced from 42.0 to 15.6 min.

CONCLUSIONS

Through the use of CALD Assist, clinician confidence in communicating with patients from non-English speaking backgrounds in the absence of an interpreter increased, providing patients from non-English speaking backgrounds with timely initial assessments and subsequent care in line with their English speaking peers. Additionally, the inclusion of images and video demonstrations in CALD Assist increased the ability to communicate with patients and overcome literacy-related barriers. Although a number of hurdles were faced, user uptake and satisfaction were positive, and the app is now available in the Apple App Store.

Health Literacy and Use and Trust in Health Information

There is a need to investigate which health information sources are used and trusted by people with limited health literacy to help identify strategies for addressing knowledge gaps that can contribute to preventable illness. We examined whether health literacy was associated with people's use of and trust in a range of potential health information sources. Six hundred participants from a GfK Internet survey panel completed an online survey. We assessed health literacy using the Newest Vital Sign, the sources participants used to get health information, and the extent to which participants trusted health information from these sources. We performed multivariable regressions, controlling for demographic characteristics. Lower health literacy was associated with lower odds of using medical websites for health information and with higher odds of using television, social media, and blogs or celebrity webpages. People with lower health literacy were less likely to trust health information from specialist doctors and dentists, but more likely to trust television, social media, blogs/celebrity webpages, friends, and pharmaceutical companies. People with limited health literacy had higher rates of using and trusting sources such as social media and blogs, which might contain lower quality health information compared to information from healthcare professionals. Thus, it might be necessary to enhance the public's ability to evaluate the quality of health information sources. The results of this study could be used to improve the reach of high-quality health information among people with limited health literacy and thereby increase the effectiveness of health communication programs and campaigns.

Machine or Human? Evaluating the Quality of a Language Translation Mobile App for Diabetes Education Material

Background

Diabetes is a major health crisis for Hispanics and Asian Americans. Moreover, Spanish and Chinese speakers are more likely to have limited English proficiency in the United States. One potential tool for facilitating language communication between diabetes patients and health care providers is technology, specifically mobile phones.

Objective

Previous studies have assessed machine translation quality using only writing inputs. To bridge such a research gap, we conducted a pilot study to evaluate the quality of a mobile language translation app (iTranslate) with a voice recognition feature for translating diabetes patient education material.

Methods

The pamphlet, “You are the heart of your family…take care of it,” is a health education sheet for diabetes patients that outlines three recommended questions for patients to ask their clinicians. Two professional translators translated the original English sentences into Spanish and Chinese. We recruited six certified medical translators (three Spanish and three Chinese) to conduct blinded evaluations of the following versions: (1) sentences interpreted by iTranslate, and (2) sentences interpreted by the professional human translators. Evaluators rated the sentences (ranging from 1-5) on four scales: Fluency, Adequacy, Meaning, and Severity. We performed descriptive analyses to examine the differences between these two versions.

Results

Cronbach alpha values exhibited high degrees of agreement on the rating outcomes of both evaluator groups: .920 for the Spanish raters and .971 for the Chinese raters. The readability scores generated using MS Word’s Flesch-Kincaid Grade Level for these sentences were 0.0, 1.0, and 7.1. We found iTranslate generally provided translation accuracy comparable to human translators on simple sentences. However, iTranslate made more errors when translating difficult sentences.

Conclusions

Although the evidence from our study supports iTranslate’s potential for supplementing professional human translators, further evidence is needed. For this reason, mobile language translation apps should be used with caution.