1
|
Gartrell K, Warren JI, Fontelo P. PubMed4Hh: A Point-of-Care Mobile App for Evidence-Based Clinical Decision Support for Nurse Residents in Maryland. Comput Inform Nurs 2023; 41:983-992. [PMID: 38062547 PMCID: PMC10746293 DOI: 10.1097/cin.0000000000001063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Evidence-based clinical decision-making is considered a core competency for professional nurses. However, barriers such as lack of time in clinical practice; information overload; restricted access to resources; lack of evidence appraisal skills, awareness, and knowledge; and unreliable resources prevent them from adopting evidence-based practice. This study examined the usefulness of the PubMed4Hh (PubMed for Handhelds) mobile application for clinical decision-making at the point of care among Maryland newly licensed RNs. Using iOS and Android smartphones, 178 newly licensed RNs participated in a 6-month trial from November 2019 to April 2020. Nurses manually entered free-text queries or used voice inputs on the "Patient, Intervention, Comparison, Outcome" or "askMEDLINE" search engines. The results retrieved were presented as journal article abstracts or short summaries called "the bottom line," designed for quick reading at the point of care. Both Patient, Intervention, Comparison, Outcome and askMEDLINE were rated highly for their usefulness, and participants said they would continue using PubMed4Hh and recommend it to others. Newly licensed RNs had a significantly higher perception of the usefulness of PubMed4Hh when the results of "the bottom line" or abstracts confirmed, led, or modified their nursing skills, knowledge, or the patient's care plan.
Collapse
Affiliation(s)
- Kyungsook Gartrell
- Author Affiliations: Department of Organizational Systems and Adult Health, University of Maryland School of Nursing, Baltimore (Dr Gartrell); Maryland Organization of Nurse Leaders, Inc/Maryland Nurse Residency Collaborative, Ellicott City (Dr Warren); and Applied Clinical Informatics Branch, National Library of Medicine, Bethesda (Dr Fontelo), MD
| | | | | |
Collapse
|
2
|
Janakiram C, Okunev I, Tranby EP, Fontelo P, Iafolla TJ, Dye BA. Opioids for acute and chronic pain when receiving psychiatric medications. PLoS One 2023; 18:e0286179. [PMID: 37751410 PMCID: PMC10522028 DOI: 10.1371/journal.pone.0286179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 05/07/2023] [Indexed: 09/28/2023] Open
Abstract
BACKGROUND People with mental health disorders (MHD) like depression and anxiety are more likely to experience substance use disorders (SUDs) than those without MHD. This study assesses opioid prescription patterns for acute or chronic pain management in patients receiving medication for depression and/or anxiety. METHODS AND FINDINGS Cross-sectional data trend analysis of 24.5 million adult medical claims was conducted using medical and pharmacy data (2012-2019) for adults aged 21-64 from the IBM Watson MarketScan Medicaid Multi-State Database. Information on sex, age, race, provider type, acute or chronic pain, and prescriptions for opioids and antidepressant and/or antianxiety medication from outpatient encounters were analyzed. For those receiving opioid prescriptions within 14 days of a pain diagnosis, ICD-10-CM codes were used to categorize diagnoses as chronic pain (back pain, neck pain, joint pain, and headache); or acute pain (dental-, ENT-, and orthopedic-related pain). Nearly 8 million adults had at least one prescription for antidepressant or antianxiety medications (MHD), with 2.5 million of those (32%) also diagnosed with an acute or chronic pain condition (pain + MHD). Among the pain + MHD group, 34% (0.85 million) received an opioid prescription within 14 days of diagnosis. Individuals with chronic pain diagnoses received a higher proportion of opioid prescriptions than those with acute pain. Among individuals with pain + MHD, the majority were aged 50-64 (35%), female (72%), and non-Hispanic white (65.1%). Nearly half (48.2%) of the opioid prescriptions given to adults with an MHD were provided by physicians. Compared to other physician types, Health Care Providers (HCPs) in emergency departments were 50% more likely to prescribe an opioid for dental pain to those with an MHD, whereas dentists were only half as likely to prescribe an opioid for dental pain management. Although overall opioid prescriptions for pain management declined from 2012 to 2019, adults with an MHD received opioids for pain management at nearly twice the level as adults without an MHD. CONCLUSIONS Although HCPs have reduced opioids for acute or chronic pain to patients at high-risk for SUD, for example, those with MHD, the use of opioids for pain management has remained at consistently higher levels for this SUD high-risk group, suggesting the need to revisit pain management guidelines for those receiving antidepressant or antianxiety drugs.
Collapse
Affiliation(s)
- Chandrashekar Janakiram
- Amrita School of Dentistry, Amrita Vishwa Vidhyapeetham, Cochin, India
- National Library of Medicine, National Institute of Health, Bethesda, Maryland, United States of America
| | - Ilya Okunev
- Health Data Analytics Institute, Dedham, Massachusetts, United States of America
| | - Eric P. Tranby
- Analytics and Evaluation, Care Quest Institute for Oral Health, Boston, Massachusetts, United States of America
| | - Paul Fontelo
- National Library of Medicine, National Institute of Health, Bethesda, Maryland, United States of America
| | - Timothy J. Iafolla
- National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, Maryland, United States of America
| | - Bruce A. Dye
- National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, Maryland, United States of America
- University of Colorado School of Dental Medicine, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| |
Collapse
|
3
|
Zolnour A, Eldredge CE, Faiola A, Yaghoobzadeh Y, Khani M, Foy D, Topaz M, Kharrazi H, Fung KW, Fontelo P, Davoudi A, Tabaie A, Breitinger SA, Oesterle TS, Rouhizadeh M, Zonnor Z, Moen H, Patrick TB, Zolnoori M. A risk identification model for detection of patients at risk of antidepressant discontinuation. Front Artif Intell 2023; 6:1229609. [PMID: 37693012 PMCID: PMC10484003 DOI: 10.3389/frai.2023.1229609] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Accepted: 08/04/2023] [Indexed: 09/12/2023] Open
Abstract
Purpose Between 30 and 68% of patients prematurely discontinue their antidepressant treatment, posing significant risks to patient safety and healthcare outcomes. Online healthcare forums have the potential to offer a rich and unique source of data, revealing dimensions of antidepressant discontinuation that may not be captured by conventional data sources. Methods We analyzed 891 patient narratives from the online healthcare forum, "askapatient.com," utilizing content analysis to create PsyRisk-a corpus highlighting the risk factors associated with antidepressant discontinuation. Leveraging PsyRisk, alongside PsyTAR [a publicly available corpus of adverse drug reactions (ADRs) related to antidepressants], we developed a machine learning-driven algorithm for proactive identification of patients at risk of abrupt antidepressant discontinuation. Results From the analyzed 891 patients, 232 reported antidepressant discontinuation. Among these patients, 92% experienced ADRs, and 72% found these reactions distressful, negatively affecting their daily activities. Approximately 26% of patients perceived the antidepressants as ineffective. Most reported ADRs were physiological (61%, 411/673), followed by cognitive (30%, 197/673), and psychological (28%, 188/673) ADRs. In our study, we employed a nested cross-validation strategy with an outer 5-fold cross-validation for model selection, and an inner 5-fold cross-validation for hyperparameter tuning. The performance of our risk identification algorithm, as assessed through this robust validation technique, yielded an AUC-ROC of 90.77 and an F1-score of 83.33. The most significant contributors to abrupt discontinuation were high perceived distress from ADRs and perceived ineffectiveness of the antidepressants. Conclusion The risk factors identified and the risk identification algorithm developed in this study have substantial potential for clinical application. They could assist healthcare professionals in identifying and managing patients with depression who are at risk of prematurely discontinuing their antidepressant treatment.
Collapse
Affiliation(s)
- Ali Zolnour
- School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
| | | | - Anthony Faiola
- College of Health Sciences, University of Kentucky, Lexington, KY, United States
| | | | - Masoud Khani
- Biomedical and Health Informatics, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | - Doreen Foy
- School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, United States
| | - Maxim Topaz
- School of Nursing and Data Science Institute, Columbia University, New York, NY, United States
- Center for Home Care Policy and Research, VNS Health, New York, NY, United States
| | - Hadi Kharrazi
- Department of Health Policy and Management, Johns Hopkins University, Baltimore, MD, United States
| | - Kin Wah Fung
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Paul Fontelo
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Anahita Davoudi
- Center for Home Care Policy and Research, VNS Health, New York, NY, United States
| | - Azade Tabaie
- Center of Biostatistics, Informatics, and Data Science, MedStar Health Research Institute, Washington, DC, United States
| | - Scott A. Breitinger
- Department of Psychiatry and Psychology, Mayo Clinic, Rochester, MN, United States
| | - Tyler S. Oesterle
- Department of Psychiatry and Psychology, Mayo Clinic, Rochester, MN, United States
| | - Masoud Rouhizadeh
- Collage of Pharmacy, University of Florida, Gainesville, FL, United States
| | - Zahra Zonnor
- Department of Biomechanics, Bu-Ali Sina University, Hamedan, Iran
| | - Hans Moen
- Department of Computer Science, Aalto University, Otaniemi, Finland
| | - Timothy B. Patrick
- Biomedical and Health Informatics, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | - Maryam Zolnoori
- School of Nursing and Data Science Institute, Columbia University, New York, NY, United States
- Department of Psychiatry and Psychology, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
4
|
Abstract
INTRODUCTION The American Dental Association (ADA) defines evidence-based dentistry (EBD) as "an approach to oral healthcare that requires the judicious integration of systematic assessments of clinically relevant scientific evidence, relating to the oral and medical condition and history, with the dentist's clinical expertise and the patient's treatment needs and preferences." Clinical practice guidelines (CPGs) are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Therefore, ADA CPGs are the most rigorous examples of EBD to inform clinical practice. CPGs should be of the highest level of quality to ensure the appropriateness and timeliness of clinical recommendations. OBJECTIVES The aim of this study was to measure the methodological rigor and transparency of the ADA CPGs. METHODS Each ADA CPG was appraised by 4 independent assessors using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. Quantitative quality scores were obtained for 6 domains and overall quality. In addition, assessors provided a qualitative analysis by providing comments for each item and an appraisal of the full recommendation. RESULTS A quality score of 75% was used as the threshold for high-quality guidelines. Using this metric, 6 of the current 10 current ADA CPGs were considered to be of high quality, 1 was slightly below the quality threshold, and 3 were considered marginal. Even among those evaluated to be high quality in overall assessment, certain domains did not reach the quality threshold of 75%. CONCLUSION Overall, the ADA CPGs collectively provide high-quality guidance for the clinician. While the AGREE appraisal guidelines have been used in CPG development since 2016, there is still room for improvement in certain domains (i.e., stakeholder involvement, rigor of development, applicability, and editorial independence). KNOWLEDGE TRANSFER STATEMENT The results of this study summarize the methodological rigor and transparency of the 10 current ADA clinical practice guidelines. Since adoption of AGREE standards (2016), CPGs have been uniformly of high quality. The quality of older CPGs was somewhat lower but overall deemed acceptable. Thus, ADA CPGs may be used with confidence to inform practitioners of treatment options supported by rigorous evidence-based dentistry standards. However, there is still room for improvement in methodological quality.
Collapse
Affiliation(s)
- S D London
- Office of Science Policy and Analysis, National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD, USA
- Applied Clinical Informatics Branch, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
- Department of Oral Biology and Pathology, Stony Brook University School of Dental Medicine, Stony Brook, NY, USA
| | - S Chamut
- Department of Oral Health Policy and Epidemiology, Harvard School of Dental Medicine, Boston, MA, USA
| | - P Fontelo
- Applied Clinical Informatics Branch, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - T Iafolla
- Office of Science Policy and Analysis, National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD, USA
| | - B A Dye
- Office of Science Policy and Analysis, National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD, USA
- Department of Community Dentistry and Population Health, University of Colorado School of Dental Medicine, Aurora, CO, USA
| |
Collapse
|
5
|
Bonilla H, Peluso MJ, Rodgers K, Aberg JA, Patterson TF, Tamburro R, Baizer L, Goldman JD, Rouphael N, Deitchman A, Fine J, Fontelo P, Kim AY, Shaw G, Stratford J, Ceger P, Costantine MM, Fisher L, O’Brien L, Maughan C, Quigley JG, Gabbay V, Mohandas S, Williams D, McComsey GA. Therapeutic trials for long COVID-19: A call to action from the interventions taskforce of the RECOVER initiative. Front Immunol 2023; 14:1129459. [PMID: 36969241 PMCID: PMC10034329 DOI: 10.3389/fimmu.2023.1129459] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 02/06/2023] [Indexed: 03/11/2023] Open
Abstract
Although most individuals recover from acute SARS-CoV-2 infection, a significant number continue to suffer from Post-Acute Sequelae of SARS-CoV-2 (PASC), including the unexplained symptoms that are frequently referred to as long COVID, which could last for weeks, months, or even years after the acute phase of illness. The National Institutes of Health is currently funding large multi-center research programs as part of its Researching COVID to Enhance Recover (RECOVER) initiative to understand why some individuals do not recover fully from COVID-19. Several ongoing pathobiology studies have provided clues to potential mechanisms contributing to this condition. These include persistence of SARS-CoV-2 antigen and/or genetic material, immune dysregulation, reactivation of other latent viral infections, microvascular dysfunction, and gut dysbiosis, among others. Although our understanding of the causes of long COVID remains incomplete, these early pathophysiologic studies suggest biological pathways that could be targeted in therapeutic trials that aim to ameliorate symptoms. Repurposed medicines and novel therapeutics deserve formal testing in clinical trial settings prior to adoption. While we endorse clinical trials, especially those that prioritize inclusion of the diverse populations most affected by COVID-19 and long COVID, we discourage off-label experimentation in uncontrolled and/or unsupervised settings. Here, we review ongoing, planned, and potential future therapeutic interventions for long COVID based on the current understanding of the pathobiological processes underlying this condition. We focus on clinical, pharmacological, and feasibility data, with the goal of informing future interventional research studies.
Collapse
Affiliation(s)
- Hector Bonilla
- Department of Medicine and Infectious Diseases, Stanford University, Palo Alto, CA, United States
| | - Michael J. Peluso
- Department of Medicine and Infectious Diseases, University of California, San Francisco, San Francisco, CA, United States
| | - Kathleen Rodgers
- Center for Innovations in Brain Science, University of Arizona, Tucson, AZ, United States
| | - Judith A. Aberg
- Department of Medicine, Infectious Diseases, Icahn School of Medicine at Mount Sinai, Chief, Division of Infectious Disease, New York, NY, United States
| | - Thomas F. Patterson
- Department of Medicine, Infectious Diseases, The University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | - Robert Tamburro
- Division of Intramural Research, National Institute of Health, Bethesda, MD, United States
| | - Lawrence Baizer
- National Heart Lung and Blood Institute, Division of Lung Diseases, National Institutes of Health, Bethesda, MD, United States
| | - Jason D. Goldman
- Department of Medicine, Organ Transplant and Liver Center, Swedish Medical Center, Seattle, WA, United States
- Division of Allergy and Infectious Diseases, University of Washington, Seattle, WA, United States
| | - Nadine Rouphael
- Department of Medicine, Division of Infectious Diseases, Emory University School of Medicine, Atlanta, GA, United States
| | - Amelia Deitchman
- Department of Clinical Pharmacy, University of California, San Francisco, San Francisco, CA, United States
| | - Jeffrey Fine
- Department of Rehabilitation Medicine at New York University (NYU) Grossman School of Medicine, Physical Medicine and Rehabilitation Service, New York University (NYU), New York University Medical Center, New York, NY, United States
| | - Paul Fontelo
- Applied Clinical Informatics Branch, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Arthur Y. Kim
- Department of Medicine at Harvard Medical School, Division of Infectious Disease, Boston, MA, United States
| | - Gwendolyn Shaw
- Research Triangle Institute (RTI), International, Durham, NC, United States
| | - Jeran Stratford
- Research Triangle Institute (RTI), International, Durham, NC, United States
| | - Patricia Ceger
- Research Triangle Institute (RTI), International, Durham, NC, United States
| | - Maged M. Costantine
- Department of Obstetrics and Gynecology, The Ohio State University, Columbus, OH, United States
| | - Liza Fisher
- Long COVID Families, Houston, TX, United States
| | - Lisa O’Brien
- Utah Covid-19 Long Haulers, Salt Lake City, UT, United States
| | | | - John G. Quigley
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, United States
| | - Vilma Gabbay
- Department of Medicine, Albert Einstein College of Medicine, New York, NY, United States
| | - Sindhu Mohandas
- Department of Pediatrics, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States
| | - David Williams
- Department of Medicine, University of Michigan, Ann Arbor, MI, United States
| | - Grace A. McComsey
- Department of Pediatrics and Medicine, Case Western Reserve University, Cleveland, OH, United States
| |
Collapse
|
6
|
London SD, Fontelo P, Boroumand S, Dye BA. COVID-19 provides an opportunity for integration of dentistry into the health informatics system. J Am Dent Assoc 2022; 153:3-8. [PMID: 34996533 PMCID: PMC8599016 DOI: 10.1016/j.adaj.2021.11.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 10/11/2021] [Accepted: 11/09/2021] [Indexed: 12/11/2022]
|
7
|
Atanda A(J, Livinski A, Weatherspoon D, Fontelo P, Boroumand S. The Impact of Tooth Retention on Health and Quality of Life in Older Adults. Innov Aging 2021. [PMCID: PMC8968400 DOI: 10.1093/geroni/igab046.2334] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
America is aging rapidly, and older adults (age ≥65 y) are retaining more of their natural teeth, a trend expected to continue. Although much is known about the impact of complete tooth loss on overall health and well-being, less is known about the effect of partial tooth loss. We conducted a systematic review to advance our understanding of the impact of retaining ≥20 teeth on health and quality of life (QoL) in older adults using two tooth retention concepts – shortened dental arch (SDA) and functional dentition (FD). We searched seven scientific databases from 1981–2019 for publications on tooth retention and outcomes and impact on health and QoL. Ninety-six studies were included in this review. Most were assessed with low risk of bias (n=74) and of good quality (n=73) using the revised Cochrane Risk of Bias tool and Newcastle-Ottawa Scale. Tooth retention was defined as FD in 82 studies, SDA in 10 studies, and four studies used both. Most were cross-sectional and only seven were from the US. We found an increasing trend among published studies in using FD and SDA to describe natural dentition retention (50 articles in 2015-19 vs one in 1995-99). In general, having <20 teeth was associated with increased likelihood for functional dependence, onset of disability, declines in higher-level functioning, and lower QoL. New information is needed to facilitate clinical decision-making, care-giving, and to help health providers better meet the future oral health needs of an aging US population.
Collapse
|
8
|
Ben-Omran MO, Livinski AA, Kopycka-Kedzierawski DT, Boroumand S, Williams D, Weatherspoon DJ, Iafolla TJ, Fontelo P, Dye BA. The use of teledentistry in facilitating oral health for older adults: A scoping review. J Am Dent Assoc 2021; 152:998-1011.e17. [PMID: 34521539 DOI: 10.1016/j.adaj.2021.06.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2020] [Revised: 05/13/2021] [Accepted: 06/02/2021] [Indexed: 12/31/2022]
Abstract
BACKGROUND Teledentistry is used in many countries to provide oral health care services. However, using teledentistry to provide oral health care services for older adults is not well documented. This knowledge gap needs to be addressed, especially when accessing a dental clinic is not possible and teledentistry might be the only way for many older adults to receive oral health care services. TYPES OF STUDIES REVIEWED Nine databases were searched and 3,396 studies were screened using established eligibility criteria. Included studies were original research or review articles in which the intervention of interest was delivered to an older adult population (≥ 60 years) via teledentistry. The authors followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Review criteria. RESULTS Nineteen studies were identified that met the criteria for inclusion. Only 1 study was from the United States. Seven studies had results focusing on older adult participants only, with most of those conducted in elder care facilities. The remainder consisted of studies with mixed-age populations reporting distinct results or information for older adults. The included studies used teledentistry, in both synchronous and asynchronous modes, to provide services such as diagnosis, oral hygiene promotion, assessment and referral of oral emergencies, and postintervention follow-up. CONCLUSIONS AND PRACTICAL IMPLICATIONS Teledentistry comprises a variety of promising apps. The authors identified and described uses, promising possibilities, and limitations of teledentistry to improve the oral health of older adults.
Collapse
|
9
|
McCall T, Ali MO, Yu F, Fontelo P, Khairat S. Development of a Mobile App to Support Self-management of Anxiety and Depression in African American Women: Usability Study. JMIR Form Res 2021; 5:e24393. [PMID: 34133313 PMCID: PMC8408754 DOI: 10.2196/24393] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Revised: 02/28/2021] [Accepted: 06/15/2021] [Indexed: 01/19/2023] Open
Abstract
Background Anxiety and depressive disorders are the most common mental health conditions among African American women. Despite the need for mental health care, African American women significantly underuse mental health services. Previous mobile health studies revealed significant improvements in anxiety or depressive symptoms after intervention. The use of mobile apps offers the potential to eliminate or mitigate barriers for African American women who are seeking access to mental health services and resources. Objective This study aims to evaluate the usability of the prototype of an app that is designed for supporting the self-management of anxiety and depression in African American women. Methods Individual usability testing sessions were conducted with 15 participants in Chapel Hill, North Carolina. Cognitive walkthrough and think-aloud protocols were used to evaluate the user interface. Eye-tracking glasses were used to record participants’ visual focus and gaze path as they performed the tasks. The Questionnaire for User Interface Satisfaction was administered after each session to assess the participants’ acceptance of the app. Results Participants rated the usability of the prototype positively and provided recommendations for improvement. The average of the mean scores for usability assessments (ie, overall reactions to the software, screen, terminology and app information, learning, and app capabilities) ranged from 7.2 to 8.8 on a scale of 0-9 (low to high rating) for user tasks. Most participants were able to complete each task with limited or no assistance. Design recommendations included improving the user interface by adding graphics and color, adding a tutorial for first-time users, curating a list of Black women therapists within the app, adding details about tracking anxiety and depression in the checkup graphs, informing users that they can use the talk-to-text feature for journal entries to reduce burden, relabeling the mental health information icon, monitoring for crisis support, and improving clickthrough sequencing. Conclusions This study provides a better understanding of user experience with an app tailored to support the management of anxiety and depression for African American women, which is an underserved group. As African American women have high rates of smartphone ownership, there is a great opportunity to use mobile technology to provide access to needed mental health services and resources. Future work will include incorporating feedback from usability testing and focus group sessions to refine and develop the app further. The updated app will undergo iterative usability testing before launching the pilot study to evaluate the feasibility and acceptability of the prototype.
Collapse
Affiliation(s)
- Terika McCall
- Center for Medical Informatics, Yale School of Medicine, New Haven, CT, United States.,Division of Health Informatics, Department of Biostatistics, Yale School of Public Health, New Haven, CT, United States.,Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Muhammad Osama Ali
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Fei Yu
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.,School of Information & Library Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.,Health Sciences Library, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Paul Fontelo
- National Library of Medicine, Bethesda, MD, United States
| | - Saif Khairat
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.,School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.,Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| |
Collapse
|
10
|
Abstract
CONTEXT A higher incidence of thromboembolic disorders in COVID-19 has been reported by many clinicians worldwide. OBJECTIVE, DESIGN AND DATA SOURCES Selected studies found in PubMed that reported thromboembolic events were included for meta-analysis using weighted fixed and random effects. Data from 19 articles on cohort studies in patients diagnosed with COVID-19 and thromboembolic events, including thrombosis and embolism were included in this review. RESULTS The likelihood for developing thromboembolic disorders in hospitalized COVID-19 patients was 0.28 (95% CI 0.21-0.36). CONCLUSION This study further validates the increased risk of VTE in COVID-19 patients when compared to healthy, non-hospitalized people, and hospitalized patients. These findings will be useful to researchers and medical practitioners caring for COVID-19 patients.
Collapse
Affiliation(s)
- Paul Fontelo
- Applied Clinical Informatics Branch, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, MD, 20894, USA.
| | - Mrigendra M Bastola
- Applied Clinical Informatics Branch, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, MD, 20894, USA
| | - Zhaonian Zheng
- Applied Clinical Informatics Branch, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, MD, 20894, USA
| | - Seo Hyon Baik
- Applied Clinical Informatics Branch, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, MD, 20894, USA
| |
Collapse
|
11
|
Fontelo P, Bastola MM, Zheng Z, Baik SH. A Review of Thromboembolic Events in Hospitalized COVID-19 Patients. Res Sq 2021:rs.3.rs-393440. [PMID: 33851147 PMCID: PMC8043461 DOI: 10.21203/rs.3.rs-393440/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
Context: A higher incidence of thromboembolic disorders in COVID-19 has been reported by many clinicians worldwide. Objective, Design and Data Sources: Selected studies found in PubMed that reported thromboembolic events were included for meta-analysis using weighted fixed and random effects. Data from 19 articles on cohort studies in patients diagnosed with COVID-19 and thromboembolic events, including thrombosis and embolism were included in this review. Results: The likelihood for developing thromboembolic disorders in hospitalized COVID-19 patients was 0.28 (95% CI 0.21â€"0.36). Conclusion: This study further validates the increased risk of VTE in COVID-19 patients when compared to healthy, non-hospitalized people, and hospitalized patients. These findings will be useful to researchers and medical practitioners caring for COVID-19 patients.
Collapse
|
12
|
Bastola M, Locatis C, Fontelo P. Diagnostic Reliability of In-Person Versus Remote Dermatology: A Meta-Analysis. Telemed J E Health 2021; 27:247-250. [PMID: 32639856 PMCID: PMC7958987 DOI: 10.1089/tmj.2020.0043] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2020] [Accepted: 05/13/2020] [Indexed: 11/13/2022] Open
Abstract
Background:Studies comparing teledermatology with in-person dermatologists report wide variations in diagnostic agreement. Teledermatology studies should have two independent in-person consultations establishing a baseline for comparing diagnoses made face-to-face and those made remotely.Objective:To perform a meta-analysis of comparison studies having two in-person dermatologists and at least one remote dermatologist examining the same patients to determine the overall preponderance of agreement.Method:Studies having two in-person diagnosticians were identified from previous teledermatology research reviews and independent searches of PubMed and other databases. Data from six studies identified were meta-analyzed.Results:Some studies showed high levels of diagnostic concordance, while others did not. Meta-analysis revealed that concordance rates reported in the teledermatology and clinical (in-person) consultations were significantly different (odds ratio = 0.55 [Mantel-Haenszel, fixed effect model, 95% confidence interval = 0.42-0.72], χ2 = 11.87, p < 0.05, I2 = 58%). Overall results showed that in-person primary diagnoses are significantly more concordant than remote. The results also suggest that diagnoses made in-person and teledermatology were marginally but significantly different than remote.Conclusion:Although the results of this study suggest teledermatology diagnoses are less reliable than those in-person, there are still valid reasons for using teledermatology to improve access, reduce costs, and triage patients to determine those warranting further in-person consultation and/or laboratory tests. More caution should be exercised in teledermatology when diagnoses involve risky skin conditions. There is evidence that this happens in practice.
Collapse
Affiliation(s)
- Mrigendra Bastola
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Craig Locatis
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| |
Collapse
|
13
|
Bastola MM, Locatis C, Fontelo P. Diagnostic Laboratory Tests for COVID-19 in US: Methodology and Performance. Res Sq 2020:rs.3.rs-43374. [PMID: 32743564 PMCID: PMC7386512 DOI: 10.21203/rs.3.rs-43374/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
Background COVID-19 is a global pandemic caused by a new coronavirus strain. Innovative tests have been developed to diagnose and characterize the spread of COVID-19. Only a few studies have reported the diagnostic value of currently available tests. The diagnostic performance of the tests is a major concern after the recent resurgence in COVID-19. Methods Published papers and FDA data on the currently available tests were used for analysis. Likelihood ratios, and predictive values of tests were computed. Only FDA approved tests were included. RT-PCR performance among different specimen types were also explored. Main results All the published reports on the COVID-19 tests reported RT-PCR as the validation tool for their results. Not all available COVID-19 tests reported their sensitivity and specificity. Among the publications which reported, the positive likelihood ratio ranged between 0.15 to 0.88 and tests had high negative likelihood ratio (0.99). Conclusion Although most recent publications showed high positive and negative likelihood ratios and high predictive values, the publications on test accuracy and validity have limited scope primarily due to their small sample size and insufficiencies in methodology and published data. Although most lab tests reported high sensitivity and specificity, false omission and false discovery rates were found notable in several COVID-19 lab tests. These results suggest need for caution on test results' interpretation. Practitioners also need to integrate evidence that is evolving rapidly.
Collapse
|
14
|
Bastola MM, Locatis C, Maisiak R, Fontelo P. The Effectiveness of Mobile Phone-Based Text Messaging to Intervene with Problem Drinking in Youth and Younger Adult Population: A Meta-Analysis. Telemed J E Health 2020; 26:270-277. [PMID: 30985258 PMCID: PMC7071024 DOI: 10.1089/tmj.2018.0307] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2019] [Accepted: 02/22/2019] [Indexed: 12/12/2022] Open
Abstract
Background: Mobile phone-based text messages have been used to address alcohol use disorder in younger populations by promoting abstinence, decreased alcohol intake, and moderation. Methods: A meta-analysis was conducted to summarize the effectiveness of mobile phone text messaging to address problem drinking by youth and younger adults. Results: Authors systematically searched PubMed, Embase, CINAHL, Web of Science, APA PsycNET, and the Cochrane Central Registry of Controlled Trials for literature published in the past 8 years (2010-2018). Randomized control trials and pre-post studies of younger people that used the problem drinking criteria of the National Institute on Alcohol Abuse and Alcoholism (NIAAA) were included in the meta-analysis. Conclusions: The meta-analysis suggests that text message-based interventions might not be effective in decreasing alcohol intake in the younger populations.
Collapse
Affiliation(s)
- Mrigendra M. Bastola
- National Library of Medicine, National Institutes of Health, Rockville, Maryland
| | - Craig Locatis
- National Library of Medicine, National Institutes of Health, Rockville, Maryland
| | - Richard Maisiak
- School of Medicine, University of Alabama, Birmingham, Alabama
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Rockville, Maryland
| |
Collapse
|
15
|
Janakiram C, Venkitachalam R, Fontelo P, Iafolla TJ, Dye BA. Effectiveness of herbal oral care products in reducing dental plaque & gingivitis - a systematic review and meta-analysis. BMC Complement Med Ther 2020; 20:43. [PMID: 32046707 PMCID: PMC7076867 DOI: 10.1186/s12906-020-2812-1] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2019] [Accepted: 12/31/2019] [Indexed: 01/24/2023] Open
Abstract
Background Despite the large number of trials conducted using herbal oral care products for the reduction of dental plaque or gingivitis, results are conflicting and inconclusive. Objective To assess the effectiveness of herbal oral care products compared to conventional products in reducing dental plaque and gingivitis adults. Methods We searched the following databases for Randomised controlled trials (RCTs): MEDLINE Ovid, EMBASE Ovid etc. which yielded 493 trails. Of which 24 RCTs comparing herbal toothpaste or mouth rinse with over the counter toothpaste or mouth rinse in adults aged 18 to 65 years were included. Two authors extracted information and assessed the methodological quality of the included studies using Risk of Bias. Meta-analyses using the random-effects model were conducted for four outcomes for tooth paste and mouth rinse respectively. Mean difference (MD) or standardized mean difference (SMD) were used to estimate the effect, with 95% confidence intervals. Results A total of 1597 adults participated in 24 RCT studies. These were classified as herbal toothpaste (HTP) (15 trials, 899 participants) and herbal mouth rinse (HMR) (9 trials, 698 participants) compared with non-herbal toothpaste (NHTP) or non-herbal mouth rinse (NHMR). We found that HTP was superior over NHTP (SMD 1.95, 95% CI (0.97–2.93)) in plaque reduction. The long-term use of NHMR was superior in reduction of dental plaque over HMR (SMD -2.61, 95% (CI 4.42–0.80)). From subgroup analysis it showed that HTP was not superior over fluoride toothpaste (SMD 0.99, 95% CI (0.14–2.13)) in reducing dental plaque. However, HTP was favoured over non-fluoride toothpaste (SMD 4.64, 95% CI (2.23–7.05)). Conclusion For short-term reduction in dental plaque, current evidence suggests that HTP is as effective as compared to NHTP; however, evidence is from low quality studies.
Collapse
Affiliation(s)
- Chandrashekar Janakiram
- National Institutes of Health, National Library of Medicine and National Institute of Dental and Craniofacial Research, 31 Center Drive, Suite 4B62, Bethesda, MD, 20892-2190, USA
| | - Ramanarayanan Venkitachalam
- Department of Public Health Dentistry, Amrita Vishwa Vidyapeetham, Amrita School of Dentistry, Kochi, 682041, India
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, 8500 Rockville Pike, Bethesda, MD, 20894, USA
| | - Timothy J Iafolla
- National Institute of Dental and Craniofacial Research, 31 Center Drive, Bethesda, MD, 20892-2190, USA
| | - Bruce A Dye
- National Institute of Dental and Craniofacial Research, 31 Center Drive, Bethesda, MD, 20892-2190, USA.
| |
Collapse
|
16
|
Bastola MM, Locatis C, Maisiak R, Fontelo P. Selenium, copper, zinc and hypertension: an analysis of the National Health and Nutrition Examination Survey (2011-2016). BMC Cardiovasc Disord 2020; 20:45. [PMID: 32005161 PMCID: PMC6995060 DOI: 10.1186/s12872-020-01355-x] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2019] [Accepted: 01/21/2020] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Hypertension is a major cardiovascular illness worldwide with many underlying causes. The role of trace elements selenium, copper, and zinc in hypertension is uncertain. The objective of this study was to evaluate the role of these trace elements in hypertension. METHOD Data from 6683 National Health and Nutrition Examination Survey (NHANES) participants from 2011 to 2016 were analyzed using Statistical Analytical System (SAS, version 9.4) software for the role of trace elements in hypertension in age range 8 to 80 years, irrespective of the antihypertensive medication taken. Recent American Heart Association guidelines and pediatric practice guidelines for hypertension were used. RESULTS Findings showed a significant positive association between serum selenium levels and hypertension but not serum zinc and copper. At optimal levels for transport and distribution, serum selenium levels of 120 μg/L or higher (reference level 70-150 μg/L) were significantly associated with hypertension (OR = 1.46, 95% CI = 1.29-1.66) after adjusting for confounding factors. At serum selenium level greater than 150 μg/L, the association with hypertension strengthened (OR = 1.69, 95% CI = 1.32-2.17). CONCLUSION A positive association was found between serum selenium and hypertension, irrespective of age or anti-hypertensive medications intake. These findings also suggest that the reference levels of serum levels in healthy individuals may need to be re-determined, if supported by additional studies. If validated, patients with hypertension may also need to be cautioned about selenium intake.
Collapse
Affiliation(s)
- Mrigendra M Bastola
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA.
| | - Craig Locatis
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | | | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| |
Collapse
|
17
|
Janakiram C, Chalmers NI, Fontelo P, Huser V, Mitnik GL, Iafolla TJ, Brow AR, Dye BA. Sex and race or ethnicity disparities in opioid prescriptions for dental diagnoses among patients receiving Medicaid. J Am Dent Assoc 2019; 150:e135-e144. [PMID: 31561765 PMCID: PMC6768087 DOI: 10.1016/j.adaj.2019.06.016] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2019] [Revised: 05/30/2019] [Accepted: 06/11/2019] [Indexed: 11/22/2022]
Abstract
BACKGROUND The objective of this study was to identify specific factors (sex, race or ethnicity, and health care provider type) associated with patient receipt of an opioid prescription after a dental diagnosis. METHODS The authors used Medicaid claims dated from January 1, 2013, through September 30, 2015, for 13 U.S. states in this study. The authors identified oral health-related conditions by using International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis codes 520.0 through 529.9. RESULTS During the 2013 through 2015 study period, among the more than 1,008,400 Medicaid patients with a dental diagnosis, 19.8% filled an opioid prescription within 14 days of diagnosis. Female patients were 50% more likely to receive an opioid prescription for pain management of a dental condition than were men (odds ratio [OR], 1.50; 95% confidence interval [CI], 1.49 to 1.52). Non-Hispanic whites and African Americans were approximately twice as likely to receive opioids than were Hispanics (OR, 2.12; 95% CI, 2.05 to 2.19; OR, 1.90; 95% CI, 1.84 to 1.96, respectively). Patients receiving oral health care in an emergency department were more than 7 times more likely to receive an opioid prescription than were patients treated in a dental office (OR, 7.28; 95% CI, 7.13 to 7.43). Patients with a dental condition diagnosed were more than 4 times as likely to receive an opioid from a nurse practitioner as from a dentist (OR, 4.31; 95% CI, 4.19 to 4.44). Opioid use was substantially higher among African American female patients (OR, 2.02; 95% CI, 1.93 to 2.10) and non-Hispanic white female patients (OR, 2.16; 95% CI, 2.07 to 2.24) than among Hispanic female patients. CONCLUSIONS Opioid prescribing patterns differ depending on patient race or ethnicity, sex, and health care provider source in patients with a dental diagnosis in the United States. PRACTICAL IMPLICATIONS Dentists are providing substantially fewer opioid prescriptions compared with their medical colleagues for pain treatment after a dental diagnosis in the Medicaid population. When considering pain management for dental and related conditions, dentists should continue with conservative prescribing practices as recommended.
Collapse
|
18
|
Janakiram C, Fontelo P, Huser V, Chalmers NI, Lopez Mitnik G, Brow AR, Iafolla TJ, Dye BA. Opioid Prescriptions for Acute and Chronic Pain Management Among Medicaid Beneficiaries. Am J Prev Med 2019; 57:365-373. [PMID: 31377093 PMCID: PMC6713282 DOI: 10.1016/j.amepre.2019.04.022] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2018] [Revised: 04/03/2019] [Accepted: 04/04/2019] [Indexed: 12/19/2022]
Abstract
INTRODUCTION Millions of Americans are affected by acute or chronic pain every year. This study investigates opioid prescription patterns for acute and chronic pain management among U.S. Medicaid patients. METHODS The study used medical and pharmacy claims data obtained from the multistate Truven MarketScan Medicaid Database from 2013 to 2015 for Medicaid patients receiving health care. Medicaid beneficiaries who utilized an outpatient healthcare facility for back pain, neck pain (cervicalgia), joint pain (osteoarthritis and rheumatoid arthritis), orthopedics (simple/closed fractures and muscle strains/sprains), headache (cluster headaches and migraines), dental conditions, or otorhinolaryngologic (otalgia) diagnoses, based on the International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis codes, and received an opioid prescription within 14 days of diagnosis were included in this study. RESULTS There were 5,051,288 patients with 1 of the 7 diagnostic groupings; 18.8% had an opioid prescription filled within 14 days of diagnosis. Orthopedic pain (34.8%) was the primary reason for an opioid prescription, followed by dental conditions (17.3%), back pain (14.0%), and headache (12.9%). Patients receiving an opioid for conditions associated with acute pain management, such as otorhinolaryngologic (OR=1.93, 95% CI=1.85, 2.0), dental (OR=1.50, 95% CI=1.48, 1.53), or orthopedic conditions (OR=1.31, 95% CI=1.29, 1.32), were more likely to receive the prescription from an emergency department provider versus a general practitioner. However, compared with general practitioners, other providers were more likely to prescribe opioids for conditions associated with chronic pain management. CONCLUSIONS More than half of Medicaid beneficiaries receiving an opioid for pain management do so for orthopedic- and dental-related reasons, with emergency department providers more likely to prescribe opioids. Modifications to the guidelines addressing temporary acute pain management practices with opioids would be likely to benefit emergency department providers the most.
Collapse
Affiliation(s)
- Chandrashekar Janakiram
- National Library of Medicine, Bethesda, Maryland; National Institute of Dental and Craniofacial Research, Bethesda, Maryland
| | - Paul Fontelo
- National Library of Medicine, Bethesda, Maryland
| | | | - Natalia I Chalmers
- DentaQuest Partnership for Oral Health Advancement, Boston, Massachusetts
| | | | - Avery R Brow
- DentaQuest Partnership for Oral Health Advancement, Boston, Massachusetts
| | - Timothy J Iafolla
- National Institute of Dental and Craniofacial Research, Bethesda, Maryland
| | - Bruce A Dye
- National Institute of Dental and Craniofacial Research, Bethesda, Maryland.
| |
Collapse
|
19
|
Zolnoori M, Fung KW, Patrick TB, Fontelo P, Kharrazi H, Faiola A, Shah ND, Shirley Wu YS, Eldredge CE, Luo J, Conway M, Zhu J, Park SK, Xu K, Moayyed H. The PsyTAR dataset: From patients generated narratives to a corpus of adverse drug events and effectiveness of psychiatric medications. Data Brief 2019; 24:103838. [PMID: 31065579 PMCID: PMC6495095 DOI: 10.1016/j.dib.2019.103838] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2019] [Revised: 02/22/2019] [Accepted: 03/07/2019] [Indexed: 11/23/2022] Open
Abstract
The "Psychiatric Treatment Adverse Reactions" (PsyTAR) dataset contains patients' expression of effectiveness and adverse drug events associated with psychiatric medications. The PsyTAR was generated in four phases. In the first phase, a sample of 891 drugs reviews posted by patients on an online healthcare forum, "askapatient.com", was collected for four psychiatric drugs: Zoloft, Lexapro, Cymbalta, and Effexor XR. For each drug review, patient demographic information, duration of treatment, and satisfaction with the drugs were reported. In the second phase, sentence classification, drug reviews were split to 6009 sentences, and each sentence was labeled for the presence of Adverse Drug Reaction (ADR), Withdrawal Symptoms (WDs), Sign/Symptoms/Illness (SSIs), Drug Indications (DIs), Drug Effectiveness (EF), Drug Infectiveness (INF), and Others (not applicable). In the third phases, entities including ADRs (4813 mentions), WDs (590 mentions), SSIs (1219 mentions), and DIs (792 mentions) were identified and extracted from the sentences. In the four phases, all the identified entities were mapped to the corresponding UMLS Metathesaurus concepts (916) and SNOMED CT concepts (755). In this phase, qualifiers representing severity and persistency of ADRs, WDs, SSIs, and DIs (e.g., mild, short term) were identified. All sentences and identified entities were linked to the original post using IDs (e.g., Zoloft.1, Effexor.29, Cymbalta.31). The PsyTAR dataset can be accessed via Online Supplement #1 under the CC BY 4.0 Data license. The updated versions of the dataset would also be accessible in https://sites.google.com/view/pharmacovigilanceinpsychiatry/home.
Collapse
Affiliation(s)
- Maryam Zolnoori
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
- Department of Health Informatics & Administration, University of Wisconsin Milwaukee, Milwaukee, WI, United States
- Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Kin Wah Fung
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Timothy B. Patrick
- Department of Health Informatics & Administration, University of Wisconsin Milwaukee, Milwaukee, WI, United States
| | - Paul Fontelo
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Hadi Kharrazi
- Department of Health Policy and Management, Johns Hopkins University, Baltimore, MD, United States
| | - Anthony Faiola
- Department of Biomedical and Health Information Sciences, University of Illinois at Chicago, Chicago, IL, United States
| | - Nilay D. Shah
- Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | | | | | - Jake Luo
- Department of Health Informatics & Administration, University of Wisconsin Milwaukee, Milwaukee, WI, United States
| | - Mike Conway
- Department of Biomedical Informatics, Utah University, Salt Lake City, UT, United States
| | - Jiaxi Zhu
- Emmes Corporation, Rockville, MD, United States
| | - Soo Kyung Park
- Department of Epidemiology, Johns Hopkins University, Baltimore, MD, United States
| | - Kelly Xu
- School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, United States
| | - Hamideh Moayyed
- College of Letters and Science, University of Wisconsin Milwaukee, WI, United States
| |
Collapse
|
20
|
Zolnoori M, Fung KW, Fontelo P, Kharrazi H, Faiola A, Wu YSS, Stoffel V, Patrick T. Identifying the Underlying Factors Associated With Patients' Attitudes Toward Antidepressants: Qualitative and Quantitative Analysis of Patient Drug Reviews. JMIR Ment Health 2018; 5:e10726. [PMID: 30287417 PMCID: PMC6876546 DOI: 10.2196/10726] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/07/2018] [Revised: 08/20/2018] [Accepted: 08/25/2018] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND Nonadherence to antidepressants is a major obstacle to deriving antidepressants' therapeutic benefits, resulting in significant burdens on the individuals and the health care system. Several studies have shown that nonadherence is weakly associated with personal and clinical variables but strongly associated with patients' beliefs and attitudes toward medications. Patients' drug review posts in online health care communities might provide a significant insight into patients' attitude toward antidepressants and could be used to address the challenges of self-report methods such as patients' recruitment. OBJECTIVE The aim of this study was to use patient-generated data to identify factors affecting the patient's attitude toward 4 antidepressants drugs (sertraline [Zoloft], escitalopram [Lexapro], duloxetine [Cymbalta], and venlafaxine [Effexor XR]), which in turn, is a strong determinant of treatment nonadherence. We hypothesized that clinical variables (drug effectiveness; adverse drug reactions, ADRs; perceived distress from ADRs, ADR-PD; and duration of treatment) and personal variables (age, gender, and patients' knowledge about medications) are associated with patients' attitude toward antidepressants, and experience of ADRs and drug ineffectiveness are strongly associated with negative attitude. METHODS We used both qualitative and quantitative methods to analyze the dataset. Patients' drug reviews were randomly selected from a health care forum called askapatient. The Framework method was used to build the analytical framework containing the themes for developing structured data from the qualitative drug reviews. Then, 4 annotators coded the drug reviews at the sentence level using the analytical framework. After managing missing values, we used chi-square and ordinal logistic regression to test and model the association between variables and attitude. RESULTS A total of 892 reviews posted between February 2001 and September 2016 were analyzed. Most of the patients were females (680/892, 76.2%) and aged less than 40 years (540/892, 60.5%). Patient attitude was significantly (P<.001) associated with experience of ADRs, ADR-PD, drug effectiveness, perceived lack of knowledge, experience of withdrawal, and duration of usage, whereas oth age (F4,874=0.72, P=.58) and gender (χ24=2.7, P=.21) were not found to be associated with patient attitudes. Moreover, modeling the relationship between variables and attitudes showed that drug effectiveness and perceived distress from adverse drug reactions were the 2 most significant factors affecting patients' attitude toward antidepressants. CONCLUSIONS Patients' self-report experiences of medications in online health care communities can provide a direct insight into the underlying factors associated with patients' perceptions and attitudes toward antidepressants. However, it cannot be used as a replacement for self-report methods because of the lack of information for some of the variables, colloquial language, and the unstructured format of the reports.
Collapse
Affiliation(s)
- Maryam Zolnoori
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States.,Department of Health Informatics and Administration, College of Health Sciences, University of Wisconsin-Milwaukee, Milwaukee, WI, United States.,Section of Medical Informatics, Department of Health Science Research, Mayo Clinic, Rochester, MN, United States
| | - Kin Wah Fung
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Paul Fontelo
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Hadi Kharrazi
- Center for Population Health IT, Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States
| | - Anthony Faiola
- Biomedical and Health Information Sciences, College of Applied Health Sciences, University of Illinois at Chicago, Chicago, IL, United States
| | - Yi Shuan Shirley Wu
- UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC, United States
| | - Virginia Stoffel
- Department of Occupational Science & Technology, College of Health Sciences, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | - Timothy Patrick
- Industrial and Manufacturing Engineering, College of Engineering & Applied Science, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| |
Collapse
|
21
|
Abstract
BACKGROUND Evidence-based medicine relies on current best evidence from the medical literature, the patient's history, and the clinician's own experience to provide the best care for patients. Systematic reviews and meta-analysis are considered the highest levels of evidence for informing clinical decisions. Recently, reports have shown an increase in the number but a decrease in quality of meta-analysis publications. We reviewed publication trends and determined the countries with the most journal articles and types of publications in PubMed from 1995 to 2015. METHODS We examined journal entries in PubMed from 1995 to 2015 from top publishing countries for total number of publications and citations in core clinical journals and in specific publication types (systematic reviews, meta-analysis, randomized controlled trials). RESULTS Yearly, only 30 countries generated 94.6% of all publications and 98.1% of core clinical journals worldwide. All publication types increased but with a significant increase in meta-analysis publications from China. Collaborative and co-authored papers among the 30 countries also showed an increasing trend. CONCLUSION The USA leads in all publication citations and specific publication types, except for meta-analysis where China publishes more. Collaborative publishing among international collaborators is also increasing.
Collapse
Affiliation(s)
- Paul Fontelo
- National Library of Medicine, 8600 Rockville Pike, Bethesda, MD, 20894, USA.
| | - Fang Liu
- National Library of Medicine, 8600 Rockville Pike, Bethesda, MD, 20894, USA
| |
Collapse
|
22
|
Sylim P, Liu F, Marcelo A, Fontelo P. Blockchain Technology for Detecting Falsified and Substandard Drugs in Distribution: Pharmaceutical Supply Chain Intervention. JMIR Res Protoc 2018; 7:e10163. [PMID: 30213780 PMCID: PMC6231844 DOI: 10.2196/10163] [Citation(s) in RCA: 70] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2018] [Revised: 07/05/2018] [Accepted: 07/09/2018] [Indexed: 11/22/2022] Open
Abstract
Background Drug counterfeiting is a global problem with significant risks to consumers and the general public. In the Philippines, 30% of inspected drug stores in 2003 were found with substandard/spurious/falsely-labeled/falsified/counterfeit drugs. The economic burden on the population drug expenditures and on governments is high. The Philippine Food and Drug Administration (FDA) encourages the public to check the certificates of product registration and report any instances of counterfeiting. The National Police of Philippines responds to such reports through a special task force. However, no literature on its impact on the distribution of such drugs were found. Blockchain technology is a cryptographic ledger that is allegedly immutable through repeated sequential hashing and fault-tolerant through a consensus algorithm. This project will develop and test a pharmacosurveillance blockchain system that will support information sharing along the official drug distribution network. Objective This study aims to develop a pharmacosurveillance blockchain system and test its functions in a simulated network. Methods We are developing a Distributed Application (DApp) that will run on smart contracts, employing Swarm as the Distributed File System (DFS). Two instances will be developed: one for Ethereum and another for Hyperledger Fabric. The proof-of-work (PoW) consensus algorithm of Ethereum will be modified into a delegated proof-of-stake (DPoS) or practical Byzantine fault tolerance (PBFT) consensus algorithm as it is scalable and fits the drug supply chain environment. The system will adopt the GS1 pedigree standard and will satisfy the data points in the data standardization guidelines from the US FDA. Simulations will use the following 5 nodes: for FDA, manufacturer, wholesaler, retailer, and the consumer portal. Results Development is underway. The design of the system will place FDA in a supervisory data verification role, with each pedigree type–specific data source serving a primary data verification role. The supply chain process will be initiated by the manufacturer, with recursive verification for every transaction. It will allow consumers to scan a code printed on the receipt of their purchases to review the drug distribution history. Conclusions Development and testing will be conducted in a simulated network, and thus, results may differ from actual practice. The project being proposed is disruptive; once tested, the team intends to engage the Philippine FDA to discuss implementation plans and formulate policies to facilitate adoption and sustainability. Registered Report Identifier RR1-10.2196/10163
Collapse
Affiliation(s)
- Patrick Sylim
- National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Fang Liu
- National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| | - Alvin Marcelo
- Standards and Interoperability Lab for Asia, University of the Philippines, Manila, Philippines
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Bethesda, MD, United States
| |
Collapse
|
23
|
Gartrell K, Brennan CW, Wallen GR, Liu F, Smith KG, Fontelo P. Clinicians' perceptions of usefulness of the PubMed4Hh mobile device application for clinical decision making at the point of care: a pilot study. BMC Med Inform Decis Mak 2018; 18:27. [PMID: 29739392 PMCID: PMC5941474 DOI: 10.1186/s12911-018-0607-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2017] [Accepted: 04/27/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Although evidence-based practice in healthcare has been facilitated by Internet access through wireless mobile devices, research on the effectiveness of clinical decision support for clinicians at the point of care is lacking. This study examined how evidence as abstracts and the bottom-line summaries, accessed with PubMed4Hh mobile devices, affected clinicians' decision making at the point of care. METHODS Three iterative steps were taken to evaluate the usefulness of PubMed4Hh tools at the NIH Clinical Center. First, feasibility testing was conducted using data collected from a librarian. Next, usability testing was carried out by a postdoctoral research fellow shadowing clinicians during rounds for one month in the inpatient setting. Then, a pilot study was conducted from February, 2016 to January, 2017, with clinicians using a mobile version of PubMed4Hh. Invitations were sent via e-mail lists to clinicians (physicians, physician assistants and nurse practitioners) along with periodic reminders. Participants rated the usefulness of retrieved bottom-line summaries and abstracts and indicated their usefulness on a 7-point Likert scale. They also indicated location of use (office, rounds, etc.). RESULTS Of the 166 responses collected in the feasibility phase, more than half of questions (57%, n = 94) were answerable by both the librarian using various resources and by the postdoctoral research fellow using PubMed4Hh. Sixty-six questions were collected during usability testing. More than half of questions (60.6%) were related to information about medication or treatment, while 21% were questions regarding diagnosis, and 12% were specific to disease entities. During the pilot study, participants reviewed 34 abstracts and 40 bottom-line summaries. The abstracts' usefulness mean scores were higher (95% CI [6.12, 6.64) than the scores of the bottom-line summaries (95% CI [5.25, 6.10]). The most frequent reason given was that it confirmed current or tentative diagnostic or treatment plan. The bottom-line summaries were used more in the office (79.3%), and abstracts were used more at point of care (51.9%). CONCLUSIONS Clinicians reported that retrieving relevant health information from biomedical literature using the PubMed4Hh was useful at the point of care and in the office.
Collapse
Affiliation(s)
- Kyungsook Gartrell
- Department of Nursing, Towson University, Linthicum Hall Room 201J, 8000 York Road, Towson, MD 21252 USA
| | - Caitlin W. Brennan
- National Institutes of Health Clinical Center Nursing Department, 10 Center Drive, Bldg. 10/6-3523, Bethesda, MD 20892-1151 USA
| | - Gwenyth R. Wallen
- National Institutes of Health Clinical Center Nursing Department, 10 Center Drive, 6-1484, Bethesda, MD 20892 USA
| | - Fang Liu
- National Library of Medicine, Lister Hill National Center for Biomedical Communications, B1N30N, 38A, 8600 Rockville Pike, Bethesda, MD 20894 USA
| | - Karen G. Smith
- National Institutes of Health/Library, 10 Center Drive, Bethesda, MD 20892 USA
| | - Paul Fontelo
- National Library of Medicine, Lister Hill National Center for Biomedical Communications, B1N30L, 38A, 8600 Rockville Pike, Bethesda, MD 20894 USA
| |
Collapse
|
24
|
Janakiram C, Chalmers NI, Fontelo P, Huser V, Lopez Mitnik G, Iafolla TJ, Brow AR, Dye BA. Sex and race or ethnicity disparities in opioid prescriptions for dental diagnoses among patients receiving Medicaid. J Am Dent Assoc 2018; 149:246-255. [PMID: 29599018 PMCID: PMC6152927 DOI: 10.1016/j.adaj.2018.02.010] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2018] [Revised: 02/09/2018] [Accepted: 02/12/2018] [Indexed: 02/03/2023]
Abstract
BACKGROUND The objective of this study was to identify specific factors (sex, race or ethnicity, and health care provider type) associated with patient receipt of an opioid prescription after a dental diagnosis. METHODS The authors used Medicaid claims dated from January 1, 2013, through September 30, 2015, for 13 US states in this study. The authors identified oral health-related conditions by using International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis codes 520.0 through 529.9. RESULTS During the 2013-2015 study period, among the more than 890,000 Medicaid patients with a dental diagnosis, 23% received an opioid within 14 days of diagnosis. Female patients were 50% more likely to receive an opioid for pain management of a dental condition than were men (odds ratio [OR], 1.53; 95% confidence interval [CI], 1.52 to 1.55). Non-Hispanic whites and African Americans were approximately twice as likely to receive opioids than were Hispanics (OR, 2.11; 95% CI, 2.05 to 2.17 and OR, 1.88; 95% CI, 1.83 to 1.93, respectively). Patients receiving oral health care in an emergency department were nearly 5 times more likely to receive an opioid prescription than were patients treated in a dental office (OR, 4.66; 95% CI, 4.59 to 4.74). Patients with a dental condition diagnosed were nearly 3 times as likely to receive an opioid from a nurse practitioner as from a dentist (OR, 2.64; 95% CI, 2.57 to 2.70). Opioid use was substantially higher among African American female patients (OR, 3.29; 95% CI, 3.18 to 3.40) and non-Hispanic white female patients (OR, 3.24; 95% CI, 3.14 to 3.35) than among Hispanic female patients. CONCLUSIONS Opioid prescribing patterns differ depending on patient race or ethnicity, sex, and health care provider source in patients with a dental diagnosis in the United States. PRACTICAL IMPLICATIONS Dentists are providing substantially less opioid prescriptions compared to their medical colleagues for pain treatment following a dental diagnosis in the Medicaid population. When considering pain management for dental and related conditions, dentists should continue with conservative prescribing practices as recommended.
Collapse
|
25
|
Gimbel RW, Pirrallo RG, Lowe SC, Wright DW, Zhang L, Woo MJ, Fontelo P, Liu F, Connor Z. Effect of clinical decision rules, patient cost and malpractice information on clinician brain CT image ordering: a randomized controlled trial. BMC Med Inform Decis Mak 2018. [PMID: 29530029 PMCID: PMC5848437 DOI: 10.1186/s12911-018-0602-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Background The frequency of head computed tomography (CT) imaging for mild head trauma patients has raised safety and cost concerns. Validated clinical decision rules exist in the published literature and on-line sources to guide medical image ordering but are often not used by emergency department (ED) clinicians. Using simulation, we explored whether the presentation of a clinical decision rule (i.e. Canadian CT Head Rule - CCHR), findings from malpractice cases related to clinicians not ordering CT imaging in mild head trauma cases, and estimated patient out-of-pocket cost might influence clinician brain CT ordering. Understanding what type and how information may influence clinical decision making in the ordering advanced medical imaging is important in shaping the optimal design and implementation of related clinical decision support systems. Methods Multi-center, double-blinded simulation-based randomized controlled trial. Following standardized clinical vignette presentation, clinicians made an initial imaging decision for the patient. This was followed by additional information on decision support rules, malpractice outcome review, and patient cost; each with opportunity to modify their initial order. The malpractice and cost information differed by assigned group to test the any temporal relationship. The simulation closed with a second vignette and an imaging decision. Results One hundred sixteen of the 167 participants (66.9%) initially ordered a brain CT scan. After CCHR presentation, the number of clinicians ordering a CT dropped to 76 (45.8%), representing a 21.1% reduction in CT ordering (P = 0.002). This reduction in CT ordering was maintained, in comparison to initial imaging orders, when presented with malpractice review information (p = 0.002) and patient cost information (p = 0.002). About 57% of clinicians changed their order during study, while 43% never modified their imaging order. Conclusion This study suggests that ED clinician brain CT imaging decisions may be influenced by clinical decision support rules, patient out-of-pocket cost information and findings from malpractice case review. Trial registration NCT03449862, February 27, 2018, Retrospectively registered. Electronic supplementary material The online version of this article (10.1186/s12911-018-0602-1) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Ronald W Gimbel
- Department of Public Health Sciences, Clemson University, 501 Edwards Hall, Clemson, SC, 29634-0745, USA.
| | - Ronald G Pirrallo
- Department of Emergency Medicine, Greenville Health System, Greenville, SC, USA
| | - Steven C Lowe
- Department of Radiology, Greenville Health System, Greenville, SC, USA
| | - David W Wright
- Department of Emergency Medicine, Emory University, Atlanta, GA, USA
| | - Lu Zhang
- Department of Public Health Sciences, Clemson University, 501 Edwards Hall, Clemson, SC, 29634-0745, USA
| | - Min-Jae Woo
- Department of Public Health Sciences, Clemson University, 501 Edwards Hall, Clemson, SC, 29634-0745, USA
| | - Paul Fontelo
- Lister Hill National Center for Biomedical Communication, National Library of Medicine, Bethesda, MD, USA
| | - Fang Liu
- Lister Hill National Center for Biomedical Communication, National Library of Medicine, Bethesda, MD, USA
| | - Zachary Connor
- Department of Public Health Sciences, Clemson University, 501 Edwards Hall, Clemson, SC, 29634-0745, USA.,Department of Radiology, Greenville Health System, Greenville, SC, USA
| |
Collapse
|
26
|
Ghosh A, Brown GT, Fontelo P. Telepathology at the Armed Forces Institute of Pathology: A Retrospective Review of Consultations From 1996 to 1997. Arch Pathol Lab Med 2017; 142:248-252. [DOI: 10.5858/arpa.2017-0055-oa] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Context.—
Telepathology is the practice of pathology at a distance, transmitting images using telecommunication methods for second opinion and/or diagnostic assistance, or for educational purposes. It may be the only means of consultation for some pathologists.
Objective.—
To retrospectively review and evaluate a subset of telepathology consultations from June 1996 to March 1997, and to determine the concordance between the telepathology diagnosis of the contributor and pathologists at the Armed Forces Institute of Pathology (AFIP), Washington, District of Columbia, as well as the concordance between the telepathology diagnosis and the glass slide diagnosis, when available.
Design.—
Photocopies of de-identified telepathology reports from the AFIP during a 15-month period between June 1996 and March 1997 were reviewed. Contributor versus telepathology diagnosis was graded as 1 (complete agreement), 2 (partial agreement), 3 (disagreement; usually a diagnosis of benign versus malignant), and deferred. Data were analyzed using descriptive statistical methods.
Results.—
Of the 262 cases, 194 (74%) were in complete agreement with the contributor's diagnosis, 34 of 262 (13%) were in minor disagreement, and 21 of 262 (8%) were in major disagreement. Diagnoses were deferred in 5% (13 of 262) of cases.
Conclusions.—
Using commercial off-the-shelf technology and despite telecommunication challenges during that time, the AFIP demonstrated that telepathology could be conducted reliably.
Collapse
|
27
|
Gardner CL, Liu F, Fontelo P, Flanagan MC, Hoang A, Burke HB. Assessing the usability by clinicians of VISION: A hierarchical display of patient-collected physiological information to clinicians. BMC Med Inform Decis Mak 2017; 17:41. [PMID: 28410579 PMCID: PMC5391572 DOI: 10.1186/s12911-017-0435-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2016] [Accepted: 03/29/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The inability of patients to accurately and completely recount their clinical status between clinic visits reduces the clinician's ability to properly manage their patients. One way to improve this situation is to collect objective patient information while the patients are at home and display the collected multi-day clinical information in parallel on a single screen, highlighting threshold violations for each channel, and allowing the viewer to drill down to any analog signal on the same screen, while maintaining the overall physiological context of the patient. All this would be accomplished in a way that was easy for the clinician to view and use. METHODS Patients used five mobile devices to collect six heart failure-related clinical variables: body weight, systolic and diastolic blood pressure, pulse rate, blood oxygen saturation, physical activity, and subjective input. Fourteen clinicians practicing in a heart failure clinic rated the display using the System Usability Scale that, for acceptability, had an expected mean of 68 (SD, 12.5). In addition, we calculated the Intraclass Correlation Coefficient of the clinician responses using a two-way, mixed effects model, ICC (3,1). RESULTS We developed a single-screen temporal hierarchical display (VISION) that summarizes the patient's home monitoring activities between clinic visits. The overall System Usability Scale score was 92 (95% CI, 87-97), p < 0.0001; the ICC was 0.89 (CI, 0.79-0.97), p < 0.0001. CONCLUSION Clinicians consistently found VISION to be highly usable. To our knowledge, this is the first single-screen, parallel variable, temporal hierarchical display of both continuous and discrete information acquired by patients at home between clinic visits that presents clinically significant information at the point of care in a manner that is usable by clinicians.
Collapse
Affiliation(s)
- Cubby L Gardner
- 59th Medical Wing, Science and Technology, JBSA-Lackland, TX, USA.
| | - Fang Liu
- National Library of Medicine, Bethesda, MD, 20814, USA
| | - Paul Fontelo
- National Library of Medicine, Bethesda, MD, 20814, USA
| | - Michael C Flanagan
- Cardiology Clinic, Walter Reed National Military Medical Center, Bethesda, MD, USA
| | - Albert Hoang
- F. Edward Hébert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, MD, USA
| | - Harry B Burke
- F. Edward Hébert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, MD, USA
| |
Collapse
|
28
|
Abstract
Background: In the year 2014, Android smartphones accounted for one-third of mobile connections globally but are predicted to increase to two-thirds by 2020. In developing countries, where teleconsultations can benefit health-care providers most, the ratio is even higher. This study compared the use of two Android phones, an 8 megapixel (MP) and a 16 MP phone, for capturing microscopic images. Method: The Android phones were used to capture images and videos of a gastrointestinal biopsy teaching set of referred cases from the Armed Forces Institute of Pathology (AFIP). The acquired images and videos were reviewed online by two pathologists for image quality, adequacy for diagnosis, usefulness of video overviews, and confidence in diagnosis, on a 5-point Likert scale. Results: The results show higher means in a 5-point Likert scale for the 8 MP versus the 16 MP phone that were statistically significant in adequacy of images (4.0 vs. 3.75) for rendering diagnosis and for agreement with the reference diagnosis (2.33 vs. 2.07). Although the quality of images was found higher in the 16 MP phone (3.8 vs. 3.65), these were not statistically significant. Adding video images of the entire specimen was found to be useful for evaluating the slides (combined mean, 4.0). Conclusion: For telepathology and other image dependent practices in developing countries, Android phones could be a useful tool for capturing images.
Collapse
Affiliation(s)
- Donald Ekong
- Department of Electrical and Computer Engineering, School of Engineering, Mercer University, Macon, GA 31207, USA
| | - Fang Liu
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, Maryland, USA
| | - G Thomas Brown
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, Maryland, USA
| | - Arunima Ghosh
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, Maryland, USA
| | - Paul Fontelo
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, Maryland, USA
| |
Collapse
|
29
|
Abstract
For the treatment of HIV, compliance in regard to appointment attendance and medication usage is critical. Various methods have been attempted to increased HIV care compliance, and a method that has inspired many published studies is text message reminders. We conducted a meta-analysis of the literature from inception through May 2016 using the following databases: Pubmed, Embase, CINAHL, Web of Science, and Cochrane. Examples of terms used in the search included exploded versions of "HIV, "AIDS", "cell phone", "SMS", "text message", "reminder". After abstract and manuscript review, articles were discussed with co-author and included based on consensus. We excluded qualitative analyses, observational studies without an intervention, and studies without a control or pre-intervention group. We used random-effects models to calculate odds ratios (OR) and standardized mean differences (SMDs) for the text message intervention. Thirty-four unique studies were found and included in the meta-analysis. For the seven articles relating to non-attendance, text message reminders significantly reduced the rates of non-attendance (OR, 0.66; 95% CI, 0.48-0.92; P = .01; I2 = 52%). For the 20 articles on drug adherence, text message reminders significantly increased adherence (SMD, 0.87; 95% CI, 0.06-1.68; P = .04; I2 = 99%). For the 11 articles with physiologic measures (CD4 count or viral load), text message reminders led to significant improvement (SMD, 1.53; 95% CI, 0.52-2.55; P = .003; I2 = 99%). This meta-analysis reveals that text message reminders are a promising intervention that can be used to increase HIV care compliance when logistically feasible. Further study should focus on which populations benefit the most from this intervention, and successful implementers could create an established technological infrastructure for other clinics to adopt when seeking to boost compliance.
Collapse
Affiliation(s)
- Jonathan E Mayer
- a Department of Medicine , Johns Hopkins Bayview Medical Center, Johns Hopkins University School of Medicine , Baltimore , MD , USA
| | - Paul Fontelo
- b National Library of Medicine , Lister Hill National Center for Biomedical Communications, National Institutes of Health , Bethesda , MD , USA
| |
Collapse
|
30
|
Affiliation(s)
- Paul Fontelo
- National Library of Medicine, Bethesda, Maryland, USA
| | - Fang Liu
- National Library of Medicine, Bethesda, Maryland, USA
| | - Raymonde C Uy
- National Library of Medicine, Bethesda, Maryland, USA
| |
Collapse
|
31
|
Marceglia S, Fontelo P, Rossi E, Ackerman MJ. A Standards-Based Architecture Proposal for Integrating Patient mHealth Apps to Electronic Health Record Systems. Appl Clin Inform 2015; 6:488-505. [PMID: 26448794 DOI: 10.4338/aci-2014-12-ra-0115] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2014] [Accepted: 05/07/2015] [Indexed: 01/09/2023] Open
Abstract
BACKGROUND Mobile health Applications (mHealth Apps) are opening the way to patients' responsible and active involvement with their own healthcare management. However, apart from Apps allowing patient's access to their electronic health records (EHRs), mHealth Apps are currently developed as dedicated "island systems". OBJECTIVE Although much work has been done on patient's access to EHRs, transfer of information from mHealth Apps to EHR systems is still low. This study proposes a standards-based architecture that can be adopted by mHealth Apps to exchange information with EHRs to support better quality of care. METHODS Following the definition of requirements for the EHR/mHealth App information exchange recently proposed, and after reviewing current standards, we designed the architecture for EHR/mHealth App integration. Then, as a case study, we modeled a system based on the proposed architecture aimed to support home monitoring for congestive heart failure patients. We simulated such process using, on the EHR side, OpenMRS, an open source longitudinal EHR and, on the mHealth App side, the iOS platform. RESULTS The integration architecture was based on the bi-directional exchange of standard documents (clinical document architecture rel2 - CDA2). In the process, the clinician "prescribes" the home monitoring procedures by creating a CDA2 prescription in the EHR that is sent, encrypted and de-identified, to the mHealth App to create the monitoring calendar. At the scheduled time, the App alerts the patient to start the monitoring. After the measurements are done, the App generates a structured CDA2-compliant monitoring report and sends it to the EHR, thus avoiding local storage. CONCLUSIONS The proposed architecture, even if validated only in a simulation environment, represents a step forward in the integration of personal mHealth Apps into the larger health-IT ecosystem, allowing the bi-directional data exchange between patients and healthcare professionals, supporting the patient's engagement in self-management and self-care.
Collapse
Affiliation(s)
- S Marceglia
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine , Bethesda, MD 20994, USA ; Clinical Center for Neurostimulation , Neurotechnology, and Movement DisordersFondazione IRCCS Ca'Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - P Fontelo
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine , Bethesda, MD 20994, USA
| | - E Rossi
- eHealthLAB, Dipartimento di Elettronica, Informazione e Bioingegneria , Politecnico di Milano, Milan, Italy
| | - M J Ackerman
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine , Bethesda, MD 20994, USA
| |
Collapse
|
32
|
Abstract
BACKGROUND Mobile networks and smartphones are growing in developing countries. Expert telemedicine consultation will become more convenient and feasible. We wanted to report on our experience in using a smartphone and a 3-D printed adapter for capturing microscopic images. METHODS Images and videos from a gastrointestinal biopsy teaching set of referred cases from the AFIP were captured with an iPhone 5 smartphone fitted with a 3-D printed adapter. Nine pathologists worldwide evaluated the images for quality, adequacy for telepathology consultation, and confidence rendering a diagnosis based on the images viewed on the web. RESULTS Average Likert scales (ordinal data) for image quality (1=poor, 5=diagnostic) and adequacy for diagnosis (1=No, 5=Yes) had modes of 3 and 4, respectively. Adding a video overview of the specimen improved diagnostic confidence. The mode of confidence in diagnosis based on the images reviewed was four. In 31 instances, reviewers' diagnoses completely agreed with AFIP diagnosis, with partial agreement in 9 and major disagreement in 5. There was strong correlation between image quality and confidence (r = 0.78), image quality and adequacy of image (r = 0.73) and whether images were found adequate when reviewers were confident (r = 0.72). Intraclass Correlation for measuring reliability among the four reviewers who finished a majority of cases was high (quality=0.83, adequacy= 0.76 and confidence=0.92). CONCLUSIONS Smartphones allow pathologists and other image dependent disciplines in low resource areas to transmit consultations to experts anywhere in the world. Improvements in camera resolution and training may mitigate some limitations found in this study.
Collapse
Affiliation(s)
- Paul Fontelo
- Office of High Performance Computing and Communications, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Fang Liu
- Office of High Performance Computing and Communications, Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Yukako Yagi
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
33
|
Marceglia S, Fontelo P, Ackerman MJ. Transforming consumer health informatics: connecting CHI applications to the health-IT ecosystem. J Am Med Inform Assoc 2015; 22:e210-2. [PMID: 25665702 DOI: 10.1093/jamia/ocu030] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2014] [Accepted: 11/21/2014] [Indexed: 11/13/2022] Open
Affiliation(s)
- Sara Marceglia
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20994, USA
| | - Paul Fontelo
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20994, USA
| | - Michael J Ackerman
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20994, USA
| |
Collapse
|
34
|
Uy RC, Sarmiento RF, Gavino A, Fontelo P. Confidence and Information Access in Clinical Decision-Making: An Examination of the Cognitive Processes that affect the Information-seeking Behavior of Physicians. AMIA Annu Symp Proc 2014; 2014:1134-1140. [PMID: 25954424 PMCID: PMC4419936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Clinical decision-making involves the interplay between cognitive processes and physicians' perceptions of confidence in the context of their information-seeking behavior. The objectives of the study are: to examine how these concepts interact, to determine whether physician confidence, defined in relation to information need, affects clinical decision-making, and if information access improves decision accuracy. We analyzed previously collected data about resident physicians' perceptions of information need from a study comparing abstracts and full-text articles in clinical decision accuracy. We found that there is a significant relation between confidence and accuracy (φ=0.164, p<0.01). We also found various differences in the alignment of confidence and accuracy, demonstrating the concepts of underconfidence and overconfidence across years of clinical experience. Access to online literature also has a significant effect on accuracy (p<0.001). These results highlight possible CDSS strategies to reduce medical errors.
Collapse
Affiliation(s)
| | | | - Alex Gavino
- National Library of Medicine, Bethesda, MD 20894
| | - Paul Fontelo
- National Library of Medicine, Bethesda, MD 20894
| |
Collapse
|
35
|
Burke HB, Sessums LL, Hoang A, Becher DA, Fontelo P, Liu F, Stephens M, Pangaro LN, O'Malley PG, Baxi NS, Bunt CW, Capaldi VF, Chen JM, Cooper BA, Djuric DA, Hodge JA, Kane S, Magee C, Makary ZR, Mallory RM, Miller T, Saperstein A, Servey J, Gimbel RW. Electronic health records improve clinical note quality. J Am Med Inform Assoc 2014; 22:199-205. [PMID: 25342178 PMCID: PMC4433367 DOI: 10.1136/amiajnl-2014-002726] [Citation(s) in RCA: 44] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
BACKGROUND AND OBJECTIVE The clinical note documents the clinician's information collection, problem assessment, clinical management, and its used for administrative purposes. Electronic health records (EHRs) are being implemented in clinical practices throughout the USA yet it is not known whether they improve the quality of clinical notes. The goal in this study was to determine if EHRs improve the quality of outpatient clinical notes. MATERIALS AND METHODS A five and a half year longitudinal retrospective multicenter quantitative study comparing the quality of handwritten and electronic outpatient clinical visit notes for 100 patients with type 2 diabetes at three time points: 6 months prior to the introduction of the EHR (before-EHR), 6 months after the introduction of the EHR (after-EHR), and 5 years after the introduction of the EHR (5-year-EHR). QNOTE, a validated quantitative instrument, was used to assess the quality of outpatient clinical notes. Its scores can range from a low of 0 to a high of 100. Sixteen primary care physicians with active practices used QNOTE to determine the quality of the 300 patient notes. RESULTS The before-EHR, after-EHR, and 5-year-EHR grand mean scores (SD) were 52.0 (18.4), 61.2 (16.3), and 80.4 (8.9), respectively, and the change in scores for before-EHR to after-EHR and before-EHR to 5-year-EHR were 18% (p<0.0001) and 55% (p<0.0001), respectively. All the element and grand mean quality scores significantly improved over the 5-year time interval. CONCLUSIONS The EHR significantly improved the overall quality of the outpatient clinical note and the quality of all its elements, including the core and non-core elements. To our knowledge, this is the first study to demonstrate that the EHR significantly improves the quality of clinical notes.
Collapse
Affiliation(s)
- Harry B Burke
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Laura L Sessums
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Albert Hoang
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Dorothy A Becher
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Fang Liu
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Mark Stephens
- Department of Family, Medicine, Uniformed Services, University of the Health Sciences, Bethesda, Maryland, USA
| | - Louis N Pangaro
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Patrick G O'Malley
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Nancy S Baxi
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Christopher W Bunt
- Department of Family, Medicine, Uniformed Services, University of the Health Sciences, Bethesda, Maryland, USA
| | - Vincent F Capaldi
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Julie M Chen
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Barbara A Cooper
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | | | | | - Shawn Kane
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Charles Magee
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Zizette R Makary
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Renee M Mallory
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Thomas Miller
- Department of Family, Medicine, Uniformed Services, University of the Health Sciences, Bethesda, Maryland, USA
| | - Adam Saperstein
- Department of Family, Medicine, Uniformed Services, University of the Health Sciences, Bethesda, Maryland, USA
| | - Jessica Servey
- Department of Family, Medicine, Uniformed Services, University of the Health Sciences, Bethesda, Maryland, USA
| | - Ronald W Gimbel
- Department of Public Health Sciences, Clemson University, Clemson, SC, USA
| |
Collapse
|
36
|
Burke HB, Hoang A, Becher D, Fontelo P, Liu F, Stephens M, Pangaro LN, Sessums LL, O'Malley P, Baxi NS, Bunt CW, Capaldi VF, Chen JM, Cooper BA, Djuric DA, Hodge JA, Kane S, Magee C, Makary ZR, Mallory RM, Miller T, Saperstein A, Servey J, Gimbel RW. QNOTE: an instrument for measuring the quality of EHR clinical notes. J Am Med Inform Assoc 2014; 21:910-6. [PMID: 24384231 PMCID: PMC4147610 DOI: 10.1136/amiajnl-2013-002321] [Citation(s) in RCA: 41] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2013] [Revised: 12/04/2013] [Accepted: 12/06/2013] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND AND OBJECTIVE The outpatient clinical note documents the clinician's information collection, problem assessment, and patient management, yet there is currently no validated instrument to measure the quality of the electronic clinical note. This study evaluated the validity of the QNOTE instrument, which assesses 12 elements in the clinical note, for measuring the quality of clinical notes. It also compared its performance with a global instrument that assesses the clinical note as a whole. MATERIALS AND METHODS Retrospective multicenter blinded study of the clinical notes of 100 outpatients with type 2 diabetes mellitus who had been seen in clinic on at least three occasions. The 300 notes were rated by eight general internal medicine and eight family medicine practicing physicians. The QNOTE instrument scored the quality of the note as the sum of a set of 12 note element scores, and its inter-rater agreement was measured by the intraclass correlation coefficient. The Global instrument scored the note in its entirety, and its inter-rater agreement was measured by the Fleiss κ. RESULTS The overall QNOTE inter-rater agreement was 0.82 (CI 0.80 to 0.84), and its note quality score was 65 (CI 64 to 66). The Global inter-rater agreement was 0.24 (CI 0.19 to 0.29), and its note quality score was 52 (CI 49 to 55). The QNOTE quality scores were consistent, and the overall QNOTE score was significantly higher than the overall Global score (p=0.04). CONCLUSIONS We found the QNOTE to be a valid instrument for evaluating the quality of electronic clinical notes, and its performance was superior to that of the Global instrument.
Collapse
Affiliation(s)
- Harry B Burke
- Biomedical Informatics Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
- Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Albert Hoang
- Biomedical Informatics Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Dorothy Becher
- Biomedical Informatics Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Fang Liu
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Mark Stephens
- Family Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Louis N Pangaro
- Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Laura L Sessums
- Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Patrick O'Malley
- Biomedical Informatics Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
- Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Nancy S Baxi
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Christopher W Bunt
- Family Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Vincent F Capaldi
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Julie M Chen
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Barbara A Cooper
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - David A Djuric
- Fort Belvoir Community Hospital, Fort Belvoir, Virginia, USA
| | - Joshua A Hodge
- Fort Belvoir Community Hospital, Fort Belvoir, Virginia, USA
| | - Shawn Kane
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Charles Magee
- Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Zizette R Makary
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Renee M Mallory
- Internal Medicine Service, Walter Reed National Military Medical Center, Bethesda, Maryland, USA
| | - Thomas Miller
- Family Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Adam Saperstein
- Family Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Jessica Servey
- Family Medicine Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Ronald W Gimbel
- Biomedical Informatics Department, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| |
Collapse
|
37
|
Sheets L, Liu F, Sarmiento RF, Gavino A, Fontelo P. Optimizing the txt2MEDLINE search portal for low-resource clinical decision support. AMIA Annu Symp Proc 2013; 2013:1277-1281. [PMID: 24551407 PMCID: PMC3900144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
txt2MEDLINE provides access to high-quality medical evidence via text-messaging in settings with inadequate Internet access. We optimized the txt2MEDLINE search technique by parsing queries for MeSH (Medical Subject Heading) terms and searching MEDLINE for articles containing these terms in their titles or abstracts. We compared our results to the existing txt2MEDLINE tool by compiling benchmark queries from low-income and low-middle-income countries, and asking doctors and nurses with practice experience in low-resource areas to evaluate them. The median scores on a 5-point Likert scale were 2.9 for the existing txt2MEDLINE vs. 3.8 for the modified version (p=0.015). This reached our predefined criterion for clinical significance, a difference of 0.5 standard deviations. Improving this technology could improve clinical information resources in the world's most medically underserved communities.
Collapse
Affiliation(s)
- Lincoln Sheets
- University of Missouri Informatics Institute, Columbia MO ; National Library of Medicine, Bethesda MD
| | - Fang Liu
- National Library of Medicine, Bethesda MD
| | | | | | | |
Collapse
|
38
|
Fontelo P, Liu F. Mobile app versus Web app: a comparison using 2008-2012 "PubMed for Handhelds" server data. AMIA Annu Symp Proc 2013; 2013:445-450. [PMID: 24551349 PMCID: PMC3900190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Recent surveys show that mobile apps are more popular than Web apps. Apple's iTunes Store, now has about 800,000 apps and reported to have about 40 billion downloads. Android apps, although fewer, is available to the most number of smartphones today. About 40,000 apps are medical or health related. We developed a PubMed4Hh mobile app for iPhone/iPad users to search MEDLINE/PubMed with same features as our Web-based search tools, in use since 2002. Five-year (2008-2012) server data for PubMed4Hh and Web app were analyzed. Searches using the mobile app significantly increased compared to the same five-year time period. Month-by-month comparison showed a 3 to 5-fold increase in queries. The six-month total accesses comparison increased 280% from the previous four-year average. A review of 500 randomly selected queries revealed that the majority of queries were clinical questions ((97.8%) and 61% of these queries are searches related to therapy.
Collapse
Affiliation(s)
- Paul Fontelo
- National Library of Medicine, Bethesda, MD 20894
| | - Fang Liu
- National Library of Medicine, Bethesda, MD 20894
| |
Collapse
|
39
|
Sheets L, Gavino A, Callaghan F, Fontelo P. Do language fluency and other socioeconomic factors influence the use of PubMed and MedlinePlus? Appl Clin Inform 2013; 4:170-84. [PMID: 23874356 DOI: 10.4338/aci-2013-01-ra-0006] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2013] [Accepted: 03/30/2013] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND Increased usage of MedlinePlus by Spanish-speakers was observed after introduction of MedlinePlus in Spanish. This probably reflects increased usage of MEDLINE and PubMed by those with greater fluency in the language in which it is presented; but this has never been demonstrated in English speakers. Evidence that lack of English fluency deters international healthcare personnel from using PubMed could support the use of multi-language search tools like Babel-MeSH. OBJECTIVES This study aims to measure the effects of language fluency and other socioeconomic factors on PubMed MEDLINE and MedlinePlus access by international users. METHODS We retrospectively reviewed server pageviews of PubMed and MedlinePlus from various periods of time, and analyzed them against country statistics on language fluency, GDP, literacy rate, Internet usage, medical schools, and physicians per capita, to determine whether they were associated. RESULTS We found fluency in English to be positively associated with pageviews of PubMed and MedlinePlus in countries with high literacy rates. Spanish was generally found to be positively associated with pageviews of MedlinePlus en Español. The other parameters also showed varying degrees of association with pageviews. CONCLUSIONS After adjusting for the other factors investigated in this study, language fluency was a consistently significant predictor of the use of PubMed, MedlinePlus English and MedlinePlus en Español. This study may support the need for multi-language search tools and may increase access of health information resources from non-English speaking countries.
Collapse
Affiliation(s)
- L Sheets
- University of Missouri Informatics Institute, Columbia, Missouri 65211, USA.
| | | | | | | |
Collapse
|
40
|
Gavino AI, Ho BLC, Wee PAA, Marcelo AB, Fontelo P. Information-seeking trends of medical professionals and students from middle-income countries: a focus on the Philippines. Health Info Libr J 2013; 30:303-17. [PMID: 24251892 DOI: 10.1111/hir.12032] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2011] [Accepted: 04/29/2013] [Indexed: 11/30/2022]
Abstract
BACKGROUND Increased emphasis has been given to the practice of evidence-based medicine (EBM) worldwide. Access to quality health information is essential to the practice of EBM in developing countries. OBJECTIVES To understand the information needs and sources of information of physicians from low- and middle-income countries (LMICs). METHODS Medical doctors and students participated in an 18-question online or paper study. RESULTS Of the 156 respondents from six LMICs, 146 (94%) came from the Philippines. Eighty-eight per cent encountered at least one clinical question daily, while 58% were very likely to search for answers. A basic mobile phone was the most used device at home (94%) and at work (82%). More than half had Internet connectivity at home (62%) and just under half at work (46%). In decreasing order, short messaging services (SMS), email, instant messaging and multimedia messaging services (MMS) were the most commonly used messaging tools at home and at work. The primary source for medication questions was a formulary, but for diagnostic dilemmas, colleagues were consulted first. PubMed use was high for therapy and management questions. CONCLUSION The use of health information from the Internet through mobile devices may be increasing. Access to health information was higher at home than at work. These results may be useful when planning resources for healthcare givers in resource-poor settings.
Collapse
|
41
|
Fontelo P, Gavino A, Sarmiento RF. Comparing data accuracy between structured abstracts and full-text journal articles: implications in their use for informing clinical decisions. ACTA ACUST UNITED AC 2013; 18:207-11. [PMID: 23786759 DOI: 10.1136/eb-2013-101272] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
BACKGROUND The abstract is the most frequently read section of a research article. The use of 'Consensus Abstracts', a clinician-oriented web application formatted for mobile devices to search MEDLINE/PubMed, for informing clinical decisions was proposed recently; however, inaccuracies between abstracts and the full-text article have been shown. Efforts have been made to improve quality. METHODS We compared data in 60 recent-structured abstracts and full-text articles from six highly read medical journals. RESULTS Data inaccuracies were identified and then classified as either clinically significant or not significant. Data inaccuracies were observed in 53.33% of articles ranging from 3.33% to 45% based on the IMRAD format sections. The Results section showed the highest discrepancies (45%) although these were deemed to be mostly not significant clinically except in one. The two most common discrepancies were mismatched numbers or percentages (11.67%) and numerical data or calculations found in structured abstracts but not mentioned in the full text (40%). There was no significant relationship between journals and the presence of discrepancies (Fisher's exact p value =0.3405). Although we found a high percentage of inaccuracy between structured abstracts and full-text articles, these were not significant clinically. CONCLUSIONS The inaccuracies do not seem to affect the conclusion and interpretation overall. Structured abstracts appear to be informative and may be useful to practitioners as a resource for guiding clinical decisions.
Collapse
Affiliation(s)
- Paul Fontelo
- Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, , Bethesda, Maryland, USA
| | | | | |
Collapse
|
42
|
Marcelo A, Gavino A, Isip-Tan IT, Apostol-Nicodemus L, Mesa-Gaerlan FJ, Firaza PN, Faustorilla JF, Callaghan FM, Fontelo P. A comparison of the accuracy of clinical decisions based on full-text articles and on journal abstracts alone: a study among residents in a tertiary care hospital. Evid Based Med 2013; 18:48-53. [PMID: 22782923 PMCID: PMC3607116 DOI: 10.1136/eb-2012-100537] [Citation(s) in RCA: 40] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
BACKGROUND Many clinicians depend solely on journal abstracts to guide clinical decisions. OBJECTIVES This study aims to determine if there are differences in the accuracy of responses to simulated cases between resident physicians provided with an abstract only and those with full-text articles. It also attempts to describe their information-seeking behaviour. METHODS Seventy-seven resident physicians from four specialty departments of a tertiary care hospital completed a paper-based questionnaire with clinical simulation cases, then randomly assigned to two intervention groups-access to abstracts-only and access to both abstracts and full-text. While having access to medical literature, they completed an online version of the same questionnaire. FINDINGS The average improvement across departments was not significantly different between the abstracts-only group and the full-text group (p=0.44), but when accounting for an interaction between intervention and department, the effect was significant (p=0.049) with improvement greater with full-text in the surgery department. Overall, the accuracy of responses was greater after the provision of either abstracts-only or full-text (p<0.0001). Although some residents indicated that 'accumulated knowledge' was sufficient to respond to the patient management questions, in most instances (83% of cases) they still sought medical literature. CONCLUSIONS Our findings support studies that doctors will use evidence when convenient and current evidence improved clinical decisions. The accuracy of decisions improved after the provision of evidence. Clinical decisions guided by full-text articles were more accurate than those guided by abstracts alone, but the results seem to be driven by a significant difference in one department.
Collapse
Affiliation(s)
- Alvin Marcelo
- National Telehealth Center, University of the Philippines, Manila, Philippines
- Department of Surgery, Philippine General Hospital, Manila, Philippines
| | - Alex Gavino
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | | | | | | | - Paul Nimrod Firaza
- National Telehealth Center, University of the Philippines, Manila, Philippines
| | | | - Fiona M Callaghan
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| | - Paul Fontelo
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA
| |
Collapse
|
43
|
Goldbach H, Chang AY, Kyer A, Ketshogileng D, Taylor L, Chandra A, Dacso M, Kung SJ, Rijken T, Fontelo P, Littman-Quinn R, Seymour AK, Kovarik CL. Evaluation of generic medical information accessed via mobile phones at the point of care in resource-limited settings. J Am Med Inform Assoc 2013; 21:37-42. [PMID: 23535665 DOI: 10.1136/amiajnl-2012-001276] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVE Many mobile phone resources have been developed to increase access to health education in the developing world, yet few studies have compared these resources or quantified their performance in a resource-limited setting. This study aims to compare the performance of resident physicians in answering clinical scenarios using PubMed abstracts accessed via the PubMed for Handhelds (PubMed4Hh) website versus medical/drug reference applications (Medical Apps) accessed via software on the mobile phone. METHODS A two-arm comparative study with crossover design was conducted. Subjects, who were resident physicians at the University of Botswana, completed eight scenarios, each with multi-part questions. The primary outcome was a grade for each question. The primary independent variable was the intervention arm and other independent variables included residency and question. RESULTS Within each question type there were significant differences in 'percentage correct' between Medical Apps and PubMed4Hh for three of the six types of questions: drug-related, diagnosis/definitions, and treatment/management. Within each of these question types, Medical Apps had a higher percentage of fully correct responses than PubMed4Hh (63% vs 13%, 33% vs 12%, and 41% vs 13%, respectively). PubMed4Hh performed better for epidemiologic questions. CONCLUSIONS While mobile access to primary literature remains important and serves an information niche, mobile applications with condensed content may be more appropriate for point-of-care information needs. Further research is required to examine the specific information needs of clinicians in resource-limited settings and to evaluate the appropriateness of current resources in bridging location- and context-specific information gaps.
Collapse
Affiliation(s)
- Hayley Goldbach
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
44
|
Johnson HL, Fontelo P, Olsen CH, Jones KD, Gimbel RW. Family nurse practitioner student perception of journal abstract usefulness in clinical decision making: a randomized controlled trial. J Am Assoc Nurse Pract 2013; 25:597-603. [PMID: 24170534 DOI: 10.1111/1745-7599.12013] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
PURPOSE To assess family nurse practitioner (FNP) student perception of research abstract usefulness in clinical decision making. DATA SOURCES A randomized controlled trial conducted in a simulated environment with graduate FNP students of the Graduate School of Nursing, Uniformed Services University of the Health Sciences. Given a clinical case study and modified MEDLINE search tool accessible via an iPad device, participants were asked to develop a treatment plan and complete a data collection form. The primary measure was perceived usefulness of the research abstracts in clinical decision making regarding a simulated obese patient seeking to prevent type 2 diabetes. Secondary measures related to participant demographics and accessibility and usefulness of full-text manuscripts. CONCLUSIONS The majority of NP students identified readily available research abstracts as useful in shaping their clinical decision making. The presence or absence of full-text manuscripts associated with the abstracts did not appear to influence the perceived abstract usefulness. The majority of students with full-text manuscript access in the timed simulated clinical encounter read at least one paper, but cited insufficient time to read full-text as a constraint. IMPLICATIONS FOR PRACTICE Research abstracts at point of care may be valuable to FNPs if easily accessible and integrated into clinical workflow.
Collapse
Affiliation(s)
- Heather L Johnson
- Graduate School of Nursing, Uniformed Services University of the Health Sciences, Bethesda, Maryland
| | | | | | | | | |
Collapse
|
45
|
Sheets L, Callaghan F, Gavino A, Liu F, Fontelo P. Usability of selected databases for low-resource clinical decision support. Appl Clin Inform 2012; 3:326-33. [PMID: 23646080 DOI: 10.4338/aci-2012-06-ra-0024] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2012] [Accepted: 08/27/2012] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND Smartphones are increasingly important for clinical decision support, but smartphone and Internet use are limited by cost or coverage in many settings. txt2MEDLINE provides access to published medical evidence by text messaging. Previous studies have evaluated this approach, but we found no comparisons with other tools in this format. OBJECTIVES To compare txt2MEDLINE with other databases for answering clinical queries by text messaging in low-resource settings. METHODS Using varied formats, we searched txt2MEDLINE and five other search portals (askMEDLINE, Cochrane, DynaMed, PubMed PICO, and UpToDate) to develop optimal strategies for each. We then searched each database again with five benchmark queries, using the customized search-optimization formats. We truncated the results to less than 480 characters each to simulate delivering them to a maximum of three text messages. Clinicians with practice experience in low-resource areas scored the results on a 5-point Likert scale. RESULTS Median scores and standard deviations from 17 reviewers were: txt2M2MEDLINE, 3.2±0.82 (control); askMEDLINE, 3.2±0.90 (p = 0.918); Cochrane, 3.8±0.58 (p = 0.073); DynaMed, 3.6±0.65 (p = 0.105); PubMed PICO, 3.6±0.82 (p = 0.005); and UpToDate, 4.0±0.52 (p = 0.002). Our sample size was sufficiently powered to find differences of 1.0 point. CONCLUSIONS Comparing several possible sources for texting-based clinical-decision-support information, our results did not demonstrate one-point differences in usefulness on a scale of 1 to 5. PubMed PICO and UpToDate were significantly better than txt2MEDLINE, but with relatively small improvements in Likert score (0.4 and 0.8, respectively). In a texting-only setting, txt2MEDLINE is comparable to simulated alternatives based on established reference sources.
Collapse
Affiliation(s)
- L Sheets
- Missouri University Informatics Institute, Columbia, MO 65211, USA.
| | | | | | | | | |
Collapse
|
46
|
Fontelo P, Faustorilla J, Gavino A, Marcelo A. Digital pathology - implementation challenges in low-resource countries. Anal Cell Pathol (Amst) 2012; 35:31-6. [PMID: 22233702 PMCID: PMC4605723 DOI: 10.3233/acp-2011-0024] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
Background: Medical education in pathology and histology in low-resource countries face many obstacles because of equipment cost and telecommunication deficiencies. Digital Pathology may provide solutions. We report student experience to virtual slides on a local network and a remote image server. Methods: Using an iPad tablet device, fifty 3rd and 4th year medical students viewed digital pathology slides from a Web server at the National Library of Medicine and a mirror server on the local network. Results: The quality of images from both servers was found to be satisfactory, but the local server was deemed faster and preferred by the participants in this study (p < 0.005). Conclusions: Virtual slides on a local network server may provide solutions to equipment and technical obstacles and could enhance student learning in developing countries.
Collapse
Affiliation(s)
- Paul Fontelo
- National Library of Medicine, Bethesda, MD 20894, USA.
| | | | | | | |
Collapse
|
47
|
Armstrong K, Liu F, Seymour A, Mazhani L, Littman-Quinn R, Fontelo P, Kovarik C. Evaluation of txt2MEDLINE and development of short messaging service-optimized, clinical practice guidelines in Botswana. Telemed J E Health 2011; 18:14-7. [PMID: 22150636 DOI: 10.1089/tmj.2011.0014] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE Currently clinicians in sub-Saharan Africa have limited access to the Internet, whereas mobile phone access and use is extensive. The University of Pennsylvania in collaboration with the National Library of Medicine launched txt2MEDLINE, a short messaging service (SMS) query of PubMed/MEDLINE, and SMS-optimized clinical guidelines in Botswana. The objective of this project was to establish and evaluate the utility of these tools for clinicians in Botswana. MATERIALS AND METHODS A local server was established at the University of Botswana that allowed clinicians to send queries and receive results via local (in-country) SMS text messaging on any type of cellular phone. The queries sent via txt2MEDLINE were returned as abbreviated "the bottom line" summaries of abstracts. The 2007 Botswana Treatment Guide was converted into a format that can be queried by SMS. Various types of healthcare workers were recruited to use and evaluate these services. RESULTS Seventy-six healthcare workers attended training sessions for these services. In the preusage survey, most said they would use the services daily or weekly. During a 4-week trial period, use of these services dropped off dramatically. Participant feedback was collected and indicated that improvements in ease of use would increase the usage. CONCLUSIONS This pilot project enables clinicians to query and receive PubMed abstract summaries and country-specific clinical guidelines using mobile phones. Feedback offers insight on how to improve this technology so that it can be adopted for long-term use. With further adjustments, these resources may provide an effective working model for other countries where limited Internet access impedes upon patient care.
Collapse
Affiliation(s)
- Kathleen Armstrong
- Department of Medicine, University of Toronto Medical School, Toronto, Ontario, Canada
| | | | | | | | | | | | | |
Collapse
|
48
|
Abstract
Translational Science Search (TSS; http://tscience.nlm.nih.gov) is a web application for finding MEDLINE/PubMed journal articles that are regarded by their authors as novel, promising, or may have potential clinical application. A set of "translational" filters and related terms was created by reviewing journal articles published in clinical and translational science (TS) journals. Through E-Utilities, a user's query and TS filters are submitted to PubMed, and then, the retrieved PubMed citations are matched with a database of MeSH terms (for disease conditions) and RxNorm (for interventions) to locate the search term, translational filters found, and associated interventions in the title and abstract. An algorithm ranks the interventions and conditions, and then highlights them in the results page for quick reading and evaluation. Using previously searched terms and standard formulas, the precision and recall of TSS were 0.99 and 0.47, compared to 0.58 and 1.0 for PubMed Entrez, respectively.
Collapse
Affiliation(s)
- Paul Fontelo
- National Library of Medicine, Bethesda, Maryland, USA.
| | | |
Collapse
|
49
|
Affiliation(s)
- Paul Fontelo
- Office of High Performance Computing and Communications, Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, Maryland, USA.
| | | |
Collapse
|
50
|
Affiliation(s)
- Paul Fontelo
- National Library of Medicine, Bethesda, MD, USA.
| | | | | | | |
Collapse
|