The role of data science and machine learning in Health Professions Education: practical applications, theoretical contributions, and epistemic beliefs

Generative AI and Otolaryngology-Head & Neck Surgery

The increasing development of artificial intelligence (AI) generative models in otolaryngology-head and neck surgery will progressively change our practice. Practitioners and patients have access to AI resources, improving information, knowledge, and practice of patient care. This article summarizes the currently investigated applications of AI generative models, particularly Chatbot Generative Pre-trained Transformer, in otolaryngology-head and neck surgery.

The Cooperation Between Nurses and a New Digital Colleague "AI-Driven Lifestyle Monitoring" in Long-Term Care for Older Adults: Viewpoint

Technology has a major impact on the way nurses work. Data-driven technologies, such as artificial intelligence (AI), have particularly strong potential to support nurses in their work. However, their use also introduces ambiguities. An example of such a technology is AI-driven lifestyle monitoring in long-term care for older adults, based on data collected from ambient sensors in an older adult's home. Designing and implementing this technology in such an intimate setting requires collaboration with nurses experienced in long-term and older adult care. This viewpoint paper emphasizes the need to incorporate nurses and the nursing perspective into every stage of designing, using, and implementing AI-driven lifestyle monitoring in long-term care settings. It is argued that the technology will not replace nurses, but rather act as a new digital colleague, complementing the humane qualities of nurses and seamlessly integrating into nursing workflows. Several advantages of such a collaboration between nurses and technology are highlighted, as are potential risks such as decreased patient empowerment, depersonalization, lack of transparency, and loss of human contact. Finally, practical suggestions are offered to move forward with integrating the digital colleague.

Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship

Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.

A hybrid graph network model for ASD diagnosis based on resting-state EEG signals

Autism spectrum disorder (ASD) is a common neurodevelopmental disorder and early diagnosis is crucial for effective treatment. Stable and effective biomarkers are essential for understanding the underlying causes of the disorder and improving diagnostic accuracy. Electroencephalography (EEG) signals have proven to be reliable biomarkers for diagnosing ASD. Extracting stable connectivity patterns from EEG signals helps ensure robustness in ASD diagnostic systems. In this study, we propose a hybrid graph convolutional network framework called Rest-HGCN, which utilizes resting-state EEG signals to capture differential patterns of brain connectivity between normal children and ASD patients using graph learning strategies. The Rest-HGCN combines brain network analysis techniques and data-driven strategies to extract discriminative graph features from resting-state EEG signals. By automatically extracting differential graph patterns from these signals, the Rest-HGCN achieves reliable ASD diagnosis. To evaluate the performance of Rest-HGCN, we conducted ASD diagnosis experiments using k-fold cross-validation on the public ABC-CT resting EEG dataset. The proposed Rest-HGCN model achieved accuracies of 87.12 % and 85.32 % in single-subject and cross-experiment analyses, respectively. The results suggest that Rest-HGCN can effectively capture discriminant graph patterns from resting EEG signals and achieve robust ASD diagnosis. This may provide an effective and convenient tool for clinical ASD diagnosis.

Using learning analytics in clinical competency committees: Increasing the impact of competency-based medical education

Graduate medical education (GME) and Clinical Competency Committees (CCC) have been evolving to monitor trainee progression using competency-based medical education principles and outcomes, though evidence suggests CCCs fall short of this goal. Challenges include that evaluation data are often incomplete, insufficient, poorly aligned with performance, conflicting or of unknown quality, and CCCs struggle to organize, analyze, visualize, and integrate data elements across sources, collection methods, contexts, and time-periods, which makes advancement decisions difficult. Learning analytics have significant potential to improve competence committee decision making, yet their use is not yet commonplace. Learning analytics (LA) is the interpretation of multiple data sources gathered on trainees to assess academic progress, predict future performance, and identify potential issues to be addressed with feedback and individualized learning plans. What distinguishes LA from other educational approaches is systematic data collection and advanced digital interpretation and visualization to inform educational systems. These data are necessary to: 1) fully understand educational contexts and guide improvements; 2) advance proficiency among stakeholders to make ethical and accurate summative decisions; and 3) clearly communicate methods, findings, and actionable recommendations for a range of educational stakeholders. The ACGME released the third edition CCC Guidebook for Programs in 2020 and the 2021 Milestones 2.0 supplement of the Journal of Graduate Medical Education (JGME Supplement) presented important papers that describe evaluation and implementation features of effective CCCs. Principles of LA underpin national GME outcomes data and training across specialties; however, little guidance currently exists on how GME programs can use LA to improve the CCC process. Here we outline recommendations for implementing learning analytics for supporting decision making on trainee progress in two areas: 1) Data Quality and Decision Making, and 2) Educator Development.

The Application of Adaptive Minimum Match k-Nearest Neighbors to Identify At-Risk Students in Health Professions Education

INTRODUCTION

When learners fail to reach milestones, educators often wonder if any warning signs could have allowed them to intervene sooner. Machine learning can predict which students are at risk for failing a high-stakes certification examination. If predictions can be made well before the examination, educators can meaningfully intervene before students take the examination to reduce their chances of failing.

METHODS

The authors used already-collected, first-year student assessment data from 5 cohorts in a single Master of Physician Assistant Studies program to implement an "adaptive minimum match" version of the k-nearest neighbors algorithm using changing numbers of neighbors to predict each student's future examination scores on the Physician Assistant National Certifying Exam (PANCE). Validation occurred in 2 ways by using leave-one-out cross-validation (LOOCV) and by evaluating predictions in a new cohort.

RESULTS

"Adaptive minimum match" version of the k-nearest neighbors algorithm achieved an accuracy of 93% in LOOCV. "Adaptive minimum match" version of the k-nearest neighbors algorithm generates a predicted PANCE score for each student one year before they take the examination. Students are classified into extra support, optional extra support, or no extra support categories. Then, one year remains to provide appropriate support to each category of student.

DISCUSSION

Predictive analytics can identify at-risk students who might need additional support or remediation before high-stakes certification examinations. Educators can use the included methods and code to generate predicted test outcomes for students. The authors recommend that educators use predictive modeling responsibly and transparently, as one of many tools used to support students. More research is needed to test alternative machine learning methods across a variety of educational programs.

Clinical use of artificial intelligence requires AI-capable organizations

Artificial intelligence-based algorithms are being widely implemented in health care, even as evidence is emerging of bias in their design, problems with implementation, and potential harm to patients. To achieve the promise of using of AI-based tools to improve health, healthcare organizations will need to be AI-capable, with internal and external systems functioning in tandem to ensure the safe, ethical, and effective use of AI-based tools. Ideas are starting to emerge about the organizational routines, competencies, resources, and infrastructures that will be required for safe and effective deployment of AI in health care, but there has been little empirical research. Infrastructures that provide legal and regulatory guidance for managers, clinician competencies for the safe and effective use of AI-based tools, and learner-centric resources such as clear AI documentation and local health ecosystem impact reviews can help drive continuous improvement.

Artificial scholarship: LLMs in health professions education research

This editorial examines the implications of artificial intelligence (AI), specifically large language models (LLMs) such as ChatGPT, on the authorship and authority of academic papers, and the potential ethical concerns and challenges in health professions education (HPE).

The fundamentals of Artificial Intelligence in medical education research: AMEE Guide No. 156

The use of Artificial Intelligence (AI) in medical education has the potential to facilitate complicated tasks and improve efficiency. For example, AI could help automate assessment of written responses, or provide feedback on medical image interpretations with excellent reliability. While applications of AI in learning, instruction, and assessment are growing, further exploration is still required. There exist few conceptual or methodological guides for medical educators wishing to evaluate or engage in AI research. In this guide, we aim to: 1) describe practical considerations involved in reading and conducting studies in medical education using AI, 2) define basic terminology and 3) identify which medical education problems and data are ideally-suited for using AI.

Application of machine and deep learning algorithms in optical microscopic detection of Plasmodium: A malaria diagnostic tool for the future

Machine and deep learning techniques are prevalent in the medical discipline due to their high level of accuracy in disease diagnosis. One such disease is malaria caused by Plasmodium falciparum and transmitted by the female anopheles mosquito. According to the World Health Organisation (WHO), millions of people are infected annually, leading to inevitable deaths in the infected population. Statistical records show that early detection of malaria parasites could prevent deaths and machine learning (ML) has proved helpful in the early detection of malarial parasites. Human error is identified to be a major cause of inaccurate diagnostics in the traditional microscopy malaria diagnosis method. Therefore, the method would be more reliable if human expert dependency is restricted or entirely removed, and thus, the motivation of this paper. This study presents a systematic review to understand the prevalent machine learning algorithms applied to a low-cost, portable optical microscope in the automation of blood film interpretation for malaria parasite detection. Peer-reviewed papers were downloaded from selected reputable databases eg. Elsevier, IEEExplore, Pubmed, Scopus, Web of Science, etc. The extant literature suggests that convolutional neural network (CNN) and its variants (deep learning) account for 41.9% of the microscopy malaria diagnosis using machine learning with a prediction accuracy of 99.23%. Thus, the findings suggest that early detection of the malaria parasite has improved through the application of CNN and other ML algorithms on microscopic malaria parasite detection.

Informatics education for translational research teams: An unrealized opportunity to strengthen the national research infrastructure

Objective

To identify the informatics educational needs of clinical and translational research professionals whose primary focus is not informatics.

Introduction

Informatics and data science skills are essential for the full spectrum of translational research, and an increased understanding of informatics issues on the part of translational researchers can alleviate the demand for informaticians and enable more productive collaborations when informaticians are involved. Identifying the level of interest in different topics among various types of of translational researchers will help set priorities for development and dissemination of informatics education.

Methods

We surveyed clinical and translational science researchers in Clinical and Translational Science Award (CTSA) programs about their educational needs and preferences.

Results

Researchers from 23 out of the 62 CTSA hubs responded to the survey. 67% of respondents across roles and topics expressed interest in learning about informatics topics. There was high interest in all 30 topics included in the survey, with some variation in interest depending on the role of the respondents.

Discussion

Our data support the need to advance training in clinical and biomedical informatics. As the complexity and use of information technology and data science in research studies grows, informaticians will continue to be a limited resource for research collaboration, education, and training. An increased understanding of informatics issues across translational research teams can alleviate this burden and allow for more productive collaborations. To inform a roadmap for informatics education for research professionals, we suggest strategies to use the results of this needs assessment to develop future informatics education.

Beliefs and Practice Evaluation Based on Artificial Intelligence Models under the IP Environment

The digitization of thought theory is not yet sufficient to meet the needs of the students. It is very necessary to strengthen the construction of ideological and political (IP) courses, strengthen the education of mainstream ideology, and occupy the initiative of discourse. There are effective ways and means to study the deep integration of information technologies into the new age of philosophy and philosophy education of students, which can greatly improve the quality of teaching and the effectiveness of humanities courses. The intuitive development of intellectual and political education through artificial intelligence is both a real prerequisite for modern development and technological innovation and for new ideas confronting the specific problems of thought-politics education, and it is a necessary prerequisite to ensure the quality and efficiency to improve the teaching of thought politics. AI in ways that embed technology provides a powerful impetus for contradictory movements in thought-politics discourse that lead the effective coordination of its internal elements to high-quality developments. In a practical way, we should take full advantage of the technical advantages of artificial intelligence, through intelligent mehrfachanalyse corresponding algorithms, artificial intelligenzbilder, and artificial roboterbilder photos and profiles that are used to write and accurately provide more precise, leading, stacking, and accurate estimates to maximize and to improve the accuracy of thought and policy education. Philosophy and political theory are the keys to guiding people to accomplish basic human tasks. With the passage of time and with continuous innovation, ideology and philosophical principles are both prerequisites for the self-development of the emotions of the age and necessary for improving the outcomes and effectiveness of scientific education. Advances in the field of artificial intelligence have fundamentally changed human life and have also impacted traditional school and university education systems. This brings new opportunities and challenges to students. This includes the use of smart technology to improve the learning process. Ideological teachers and policymakers must keep up with the all-round trend of the times, make full use of the benefits brought by intelligent new technologies and platforms, effectively improve the effect of ideological education, and further increase the attractiveness, attraction, persuasion, and contagion of ideological education.

Exploring how feedback reflects entrustment decisions using artificial intelligence

CONTEXT

Clinical supervisors make judgements about how much to trust learners with critical activities in patient care. Such decisions mediate trainees' opportunities for learning and competency development and thus are a critical component of education. As educators apply entrustment frameworks to assessment, it is important to determine how narrative feedback reflecting entrustment may also address learners' educational needs.

METHODS

In this study, we used artificial intelligence (AI) and natural language processing (NLP) to identify characteristics of feedback tied to supervisors' entrustment decisions during direct observation encounters of clerkship medical students (3328 unique observations). Supervisors conducted observations of students and collaborated with them to complete an entrustment-based assessment in which they documented narrative feedback and assigned an entrustment rating. We trained a deep neural network (DNN) to predict entrustment levels from the narrative data and developed an explainable AI protocol to uncover the latent thematic features the DNN used to make its prediction.

RESULTS

We found that entrustment levels were associated with level of detail (specific steps for performing clinical tasks), feedback type (constructive versus reinforcing) and task type (procedural versus cognitive). In justifying both high and low levels of entrustment, supervisors detailed concrete steps that trainees performed (or did not yet perform) competently.

CONCLUSIONS

Framing our results in the factors previously identified as influencing entrustment, we find a focus on performance details related to trainees' clinical competency as opposed to nonspecific feedback on trainee qualities. The entrustment framework reflected in feedback appeared to guide specific goal-setting, combined with details necessary to reach those goals. Our NLP methodology can also serve as a starting point for future work on entrustment and feedback as similar assessment datasets accumulate.

Using theory-informed data science methods to trace the quality of dental student reflections over time

This study describes a theory-informed application of data science methods to analyze the quality of reflections made in a health professions education program over time. One thousand five hundred reflections written by a cohort of 369 dental students over 4 years of academic study were evaluated for an overall measure of reflection depth (No, Shallow, Deep) and the presence of six theoretically-indicated elements of reflection quality (Description, Analysis, Feeling, Perspective, Evaluation, Outcome). Machine learning models were then built to automatically detect these qualities based on linguistic features in the reflections. Results showed a dramatic increase from No to Shallow reflections from the start to end of year one (20%  →  66%), but only a limited gradual rise in Deep reflections across all four years (2%  →  26%). The presence of all six reflection elements increased over time, but inclusion of Feelings and Analysis remained relatively low even at the end of year four (found in 44% and 60% of reflections respectively). Models were able to reliably detect the presence of Description (κ_TEST = 0.70) and Evaluation (κ_TEST = 0.65) in reflections; models to detect the presence of Analysis (κ_TEST = 0.50), Feelings (κ_TEST = 0.54), and Perspectives (κ_TEST = 0.53) showed moderate performance; the model to detect Outcomes suffered from overfitting (κ_TRAIN = 0.90, κ_TEST = 0.53). A classifier for overall depth built on the reflection elements showed moderate performance across all time periods (κ_TEST > 0.60) but relied almost exclusively on the presence of Description. Implications for the conceptualization of reflection quality and providing personalized learning support to help students develop reflective skills are discussed.

Reflections on epistemological aspects of artificial intelligence during the COVID-19 pandemic

Artificial intelligence plays an important role and has been used by several countries as a health strategy in an attempt to understand, control and find a cure for the disease caused by Coronavirus. These intelligent systems can assist in accelerating the process of developing antivirals for Coronavirus and in predicting new variants of this virus. For this reason, much research on COVID-19 has been developed with the aim of contributing to new discoveries about the Coronavirus. However, there are some epistemological aspects about the use of AI in this pandemic period of Covid-19 that deserve to be discussed and need reflections. In this scenario, this article presents a reflection on the two epistemological aspects faced by the COVID-19 pandemic: (1) The epistemological aspect resulting from the use of patient data to fill the knowledge base of intelligent systems; (2) the epistemological problem arising from the dependence of health professionals on the results/diagnoses issued by intelligent systems. In addition, we present some epistemological challenges to be implemented in a pandemic period.