Patient No-Show Prediction: A Systematic Literature Review

Factors Associated With Patient Failure To Attend Dental Appointments: A Retrospective Analysis

Nonattendance at scheduled dental appointments has a significant impact on healthcare systems worldwide. This study examines the factors associated with missed appointments at the Dental Center in the Department of Oral and Dental Health in Riyadh, Saudi Arabia, through a retrospective secondary data analysis. Existing medical records from January 1, 2024, to May 1, 2024 were analyzed to identify patterns or factors contributing to nonattendance. Data were collected using a standardized sheet and analyzed with statistical methods, including correlation analysis, ANOVA, and chi-square tests, to determine significant associations and factors affecting nonattendance. The results indicated that the majority of nonattendees were single (56.2%), with a higher proportion of females (60.7%) compared to males (39.3%). Only 3.8% of those who missed their appointments were over 55 years old. Tuesdays had the highest incidence of nonattendance (331 cases). No significant association was found between age groups and the time (F = 0.224, p = 0.925) or date (F = 0.840, p = 0.500) of appointments. Patients were less likely to attend morning appointments compared to evening ones. The high rate of missed appointments reduces the effectiveness and efficiency of the Dental Center's resources. The identified patterns and factors can guide managers and policymakers in developing strategies to reduce missed appointments and improve overall appointment adherence.

Evaluation of no-show rate in outpatient clinics with open access scheduling system: A systematic review

Background

Patients' missed appointments can cause interference in the functions of the clinics and the visit of other patients. One of the most effective strategies to solve the problem of no-show rate is the use of an open access scheduling system (OA). This systematic review was conducted with the aim of investigating the impact of OA on the rate of no-show of patients in outpatient clinics.

Methods

Relevant articles in English were investigated based on the keywords in title and abstract using PubMed, Scopus, and Web of Science databases and Google Scholar search engine (July 23, 2023). The articles using OA and reporting the no-show rate were included. Exclusion criteria were as follows: (1) review articles, opinion, and letters, (2) inpatient scheduling system articles, and (3) modeling or simulating OA articles. Data were extracted from the selected articles about such issues as study design, outcome measures, interventions, results, and quality score.

Findings

From a total of 23,403 studies, 16 articles were selected. The specialized fields included family medicine (62.5%, 10), pediatrics (25%, four), ophthalmology, podiatric, geriatrics, internal medicine, and primary care (6.25%, one). Of 16 articles, 10 papers (62.5%) showed a significant decrease in the no-show rate. In four articles (25%), the no-show rate was not significantly reduced. In two papers (12.5%), there were no significant changes.

Conclusions

According to this study results, it seems that in most outpatient clinics, the use of OA by considering some conditions such as conducting needs assessment and system design based on the patients' and providers' actual needs, and cooperating of all system stakeholders through consistent training caused a significant decrease in the no-show rate.

Dynamic Price Application to Prevent Financial Losses to Hospitals Based on Machine Learning Algorithms

Hospitals that are considered non-profit take into consideration not to make any losses other than seeking profit. A model that ensures that hospital price policies are variable due to hospital revenues depending on patients with appointments is presented in this study. A dynamic pricing approach is presented to prevent patients who have an appointment but do not show up to the hospital from causing financial loss to the hospital. The research leverages three distinct machine learning (ML) algorithms, namely Random Forest (RF), Gradient Boosting (GB), and AdaBoost (AB), to analyze the appointment status of 1073 patients across nine different departments in a hospital. A mathematical formula has been developed to apply the penalty fee to evaluate the reappointment situations of the same patients in the first 100 days and the gaps in the appointment system, considering the estimated patient appointment statuses. Average penalty cost rates were calculated based on the ML algorithms used to determine the penalty costs patients will face if they do not show up, such as 22.87% for RF, 19.47% for GB, and 14.28% for AB. As a result, this study provides essential criteria that can help hospital management better understand the potential financial impact of patients missing appointments and can be considered when choosing between these algorithms.

Understanding the causes of missingness in primary care: a realist review

BACKGROUND

Although missed appointments in healthcare have been an area of concern for policy, practice and research, the primary focus has been on reducing single 'situational' missed appointments to the benefit of services. Little attention has been paid to the causes and consequences of more 'enduring' multiple missed appointments in primary care and the role this has in producing health inequalities.

METHODS

We conducted a realist review of the literature on multiple missed appointments to identify the causes of 'missingness.' We searched multiple databases, carried out iterative citation-tracking on key papers on the topic of missed appointments and identified papers through searches of grey literature. We synthesised evidence from 197 papers, drawing on the theoretical frameworks of candidacy and fundamental causation.

RESULTS

Missingness is caused by an overlapping set of complex factors, including patients not identifying a need for an appointment or feeling it is 'for them'; appointments as sites of poor communication, power imbalance and relational threat; patients being exposed to competing demands, priorities and urgencies; issues of travel and mobility; and an absence of choice or flexibility in when, where and with whom appointments take place.

CONCLUSIONS

Interventions to address missingness at policy and practice levels should be theoretically informed, tailored to patients experiencing missingness and their identified needs and barriers; be cognisant of causal domains at multiple levels and address as many as practical; and be designed to increase safety for those seeking care.

The impact of the COVID-19 pandemic on rates and predictors of missed hospital appointments in multiple outpatient clinics of The Royal Hospital, Sultanate of Oman: a retrospective study

BACKGROUND

The global outbreak of the COVID-19 pandemic resulted in significant changes in the delivery of health care services such as attendance of scheduled outpatient hospital appointments. This study aimed to evaluate the impact of COVID-19 on the rate and predictors of missed hospital appointment in the Sultanate of Oman.

METHODS

A retrospective single-centre analysis was conducted to determine the effect of COVID-19 on missed hospital appointments at various clinics at The Royal Hospital (tertiary referral hospital) in Muscat, Sultanate of Oman. The study population included scheduled face-to-face and virtual appointments between January 2019 and March 2021. Logistic regression models were used with interaction terms (post COVID-19) to assess changes in the predictors of missed appointments.

RESULTS

A total of 34, 3149 scheduled appointments was analysed (320,049 face-to-face and 23,100 virtual). The rate of missed face-to-face hospital appointments increased from 16.9% pre to 23.8% post start of COVID-19, particularly in early pandemic (40.5%). Missed hospital appointments were more frequent (32.2%) in virtual clinics (post COVID-19). Increases in missed face-to-face appointments varied by clinic (Paediatrics from 19.3% pre to 28.2% post; Surgery from 12.5% to 25.5%; Obstetrics & Gynaecology from 8.4% to 8.5%). A surge in the frequency of missed appointments was seen during national lockdowns for face-to-face and virtual appointments. Most predictors of missed appointments did not demonstrate any appreciable changes in effect (i.e., interaction term not statistically significant). Distance of patient residence to the hospital revealed no discernible changes in the relative effect pre and post COVID-19 for both face-to-face and virtual clinic appointments.

CONCLUSION

The rate of missed visits in most clinics was directly impacted by COVID-19. The case mix of patients who missed their appointments did not change. Virtual appointments, introduced after start of the pandemic, also had substantial rates of missed appointments and cannot be viewed as the single approach that can overcome the problem of missing hospital appointments.

Leveraging machine learning to create user-friendly models to mitigate appointment failure at dental school clinics

PURPOSE/OBJECTIVES

This study had a twofold outcome. The first aim was to develop an efficient, machine learning (ML) model using data from a dental school clinic (DSC) electronic health record (EHR). This model identified patients with a high likelihood of failing an appointment and provided a user-friendly system with a rating score that would alert clinicians and administrators of patients at high risk of no-show appointments. The second aim was to identify key factors with ML modeling that contributed to patient no-show appointments.

METHODS

Using de-identified data from a DSC EHR, eight ML algorithms were evaluated: simple decision tree, bagging regressor classifier, random forest classifier, gradient boosted regression, AdaBoost regression, XGBoost regression, neural network, and logistic regression classifier. The performance of each model was assessed using a confusion matrix with different threshold level of probability; precision, recall and predicted accuracy on each threshold; receiver-operating characteristic curve (ROC) and area under curve (AUC); as well as F1 score.

RESULTS

The ML models agreed on the threshold of probability score at 0.20-0.25 with Bagging classifier as the model that performed best with a F1 score of 0.41 and AUC of 0.76. Results showed a strong correlation between appointment failure and appointment confirmation, patient's age, number of visits before the appointment, total number of prior failed appointments, appointment lead time, as well as the patient's total number of medical alerts.

CONCLUSIONS

Altogether, the implementation of this user-friendly ML model can improve DSC workflow, benefiting dental students learning outcomes and optimizing personalized patient care.

Predicting ophthalmic clinic non-attendance using machine learning: Development and validation of models using nationwide data

BACKGROUND

Ophthalmic clinic non-attendance in New Zealand is associated with poorer health outcomes, marked inequities and costs NZD$30 million per annum. Initiatives to improve attendance typically involve expensive and ineffective brute-force strategies. The aim was to develop machine learning models to accurately predict ophthalmic clinic non-attendance.

METHODS

This multicentre, retrospective observational study developed and validated predictive models of clinic non-attendance. Attendance data for 3.1 million appointments from all New Zealand government-funded ophthalmology clinics from 2009 to 2018 were aggregated for analysis. Repeated ten-fold cross validation was used to train and optimise XGBoost and logistic regression models on several demographic and clinic-related variables. Models developed using the entire training set were compared with those restricted to regional subsets of the data.

RESULTS

In the testing data set from 2019, there were 407 574 appointments (median [range] age, 66 [0-105] years; 210 365 [51.6%] female) with a non-attendance rate of 5.7% (n = 23 309 missed appointments), XGBoost models trained on each region's data achieved the highest mean AUROC of 0.764 (SD 0.058) and mean AUPRC of 0.157 (SD 0.072). XGBoost performed better than logistic regression (mean AUROC = 0.756, p = 0.002). Training individual XGBoost models for each region led to better performance than training a single model on the complete nationwide dataset (mean AUROC = 0.754, p = 0.04).

CONCLUSION

Machine learning algorithms can predict ophthalmic clinic non-attendance with relatively basic demographic and clinic data. These findings suggest further research examining implementation of such algorithms in scheduling systems or public health interventions may be useful.

Behavioural economic interventions to reduce health care appointment non-attendance: a systematic review and meta-analysis

BACKGROUND

Appointment non-attendance - often referred to as "missed appointments", "patient no-show", or "did not attend (DNA)" - causes volatility in health systems around the world. Of the different approaches that can be adopted to reduce patient non-attendance, behavioural economics-oriented mechanisms (i.e., psychological, cognitive, emotional, and social factors that may impact individual decisions) are reasoned to be better suited in such contexts - where the need is to persuade, nudge, and/ or incentivize patients to honour their scheduled appointment. The aim of this systematic literature review is to identify and summarize the published evidence on the use and effectiveness of behavioural economic interventions to reduce no-shows for health care appointments.

METHODS

We systematically searched four databases (PubMed/Medline, Embase, Scopus, and Web of Science) for published and grey literature on behavioural economic strategies to reduce no-shows for health care appointments. Eligible studies met four criteria for inclusion; they were (1) available in English, Spanish, or French, (2) assessed behavioural economics interventions, (3) objectively measured a behavioural outcome (as opposed to attitudes or preferences), and (4) used a randomized and controlled or quasi-experimental study design.

RESULTS

Our initial search of the five databases identified 1,225 articles. After screening studies for inclusion criteria and assessing risk of bias, 61 studies were included in our final analysis. Data was extracted using a predefined 19-item extraction matrix. All studies assessed ambulatory or outpatient care services, although a variety of hospital departments or appointment types. The most common behaviour change intervention assessed was the use of reminders (n = 56). Results were mixed regarding the most effective methods of delivering reminders. There is significant evidence supporting the effectiveness of reminders (either by SMS, telephone, or mail) across various settings. However, there is a lack of evidence regarding alternative interventions and efforts to address other heuristics, leaving a majority of behavioural economic approaches unused and unassessed.

CONCLUSION

The studies in our review reflect a lack of diversity in intervention approaches but point to the effectiveness of reminder systems in reducing no-show rates across a variety of medical departments. We recommend future studies to test alternative behavioural economic interventions that have not been used, tested, and/or published before.

Machine Learning Predictions on Outpatient No-Show Appointments in a Malaysia Major Tertiary Hospital

Introduction

A no-show appointment occurs when a patient does not attend a previously booked appointment. This situation can cause other problems, such as discontinuity of patient treatments as well as a waste of both human and financial resources. One of the latest approaches to address this issue is predicting no-shows using machine learning techniques. This study aims to propose a predictive analytical approach for developing a patient no-show appointment model in Hospital Kuala Lumpur (HKL) using machine learning algorithms.

Methods

This study uses outpatient data from the HKL's Patient Management System (SPP) throughout 2019. The final data set has 246,943 appointment records with 13 attributes used for both descriptive and predictive analyses. The predictive analysis was carried out using seven machine learning algorithms, namely, logistic regression (LR), decision tree (DT), k-near neighbours (k-NN), Naïve Bayes (NB), random forest (RF), gradient boosting (GB) and multilayer perceptron (MLP).

Results

The descriptive analysis showed that the no-show rate was 28%, and attributes such as the month of the appointment and the gender of the patient seem to influence the possibility of a patient not showing up. Evaluation of the predictive model found that the GB model had the highest accuracy of 78%, F1 score of 0.76 and area under the curve (AUC) value of 0.65.

Conclusion

The predictive model could be used to formulate intervention steps to reduce no-shows, improving patient care quality.

Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative

BACKGROUND

Appointment no shows are prevalent in safety-net healthcare systems. The efficacy and equitability of using predictive algorithms to selectively add resource-intensive live telephone outreach to standard automated reminders in such a setting is not known.

OBJECTIVE

To determine if adding risk-driven telephone outreach to standard automated reminders can improve in-person primary care internal medicine clinic no show rates without worsening racial and ethnic show-rate disparities.

DESIGN

Randomized controlled quality improvement initiative.

PARTICIPANTS

Adult patients with an in-person appointment at a primary care internal medicine clinic in a safety-net healthcare system from 1/1/2022 to 8/24/2022.

INTERVENTIONS

A random forest model that leveraged electronic health record data to predict appointment no show risk was internally trained and validated to ensure fair performance. Schedulers leveraged the model to place reminder calls to patients in the augmented care arm who had a predicted no show rate of 15% or higher.

MAINE MEASURES

The primary outcome was no show rate stratified by race and ethnicity.

KEY RESULTS

There were 5840 appointments with a predicted no show rate of 15% or higher. A total of 2858 had been randomized to the augmented care group and 2982 randomized to standard care. The augmented care group had a significantly lower no show rate than the standard care group (33% vs 36%, p < 0.01). There was a significant reduction in no show rates for Black patients (36% vs 42% respectively, p < 0.001) not reflected in white, non-Hispanic patients.

CONCLUSIONS

In this randomized controlled quality improvement initiative, adding model-driven telephone outreach to standard automated reminders was associated with a significant reduction of in-person no show rates in a diverse primary care clinic. The initiative reduced no show disparities by predominantly improving access for Black patients.

Development of an evidence-based model for predicting patient, provider, and appointment factors that influence no-shows in a rural healthcare system

BACKGROUND

No-show appointments pose a significant challenge for healthcare providers, particularly in rural areas. In this study, we developed an evidence-based predictive model for patient no-shows at the Marshfield Clinic Health System (MCHS) rural provider network in Wisconsin, with the aim of improving overbooking approaches in outpatient settings and reducing the negative impact of no-shows in our underserved rural patient populations.

METHODS

Retrospective data (2021) were obtained from the MCHS scheduling system, which included 1,260,083 total appointments from 263,464 patients, as well as their demographic, appointment, and insurance information. We used descriptive statistics to associate variables with show or no-show status, logistic regression, and random forests utilized, and eXtreme Gradient Boosting (XGBoost) was chosen to develop the final model, determine cut-offs, and evaluate performance. We also used the model to predict future no-shows for appointments from 2022 and onwards.

RESULTS

The no-show rate was 6.0% in both the train and test datasets. The train and test datasets both yielded 5.98. Appointments scheduled further in advance (> 60 days of lead time) had a higher (7.7%) no-show rate. Appointments for patients aged 21-30 had the highest no-show rate (11.8%), and those for patients over 60 years of age had the lowest (2.9%). The model predictions yielded an Area Under Curve (AUC) of 0.84 for the train set and 0.83 for the test set. With the cut-off set to 0.4, the sensitivity was 0.71 and the positive predictive value was 0.18. Model results were used to recommend 1 overbook for every 6 at-risk appointments per provider per day.

CONCLUSIONS

Our findings demonstrate the feasibility of developing a predictive model based on administrative data from a predominantly rural healthcare system. Our new model distinguished between show and no-show appointments with high performance, and 1 overbook was advised for every 6 at-risk appointments. This data-driven approach to mitigating the impact of no-shows increases treatment availability in rural areas by overbooking appointment slots on days with an elevated risk of no-shows.

Association between no-show rates and interpreter use in a pediatric otolaryngology clinic

OBJECTIVES

To understand how primary language and interpreter use affect no-show rates in pediatric otolaryngology.

METHODS

This is a retrospective cohort study using medical records of new patients in a pediatric otolaryngology clinic from 2014 to 2019. Data was collected on patient demographics including age, primary language, insurance type, maternal education level, maternal primary language, interpreter use at the first visit, total number of appointments scheduled, number of missed appointments, and number of completed appointments. Inferential statistics using parametric (ANOVA) and non-parametric (Mann-Whitney U tests, Kruskal-Wallis tests, and Spearman correlation coefficient) methods were used.

RESULTS

Primary language was associated with significant differences in no-show rates (p = 0.0474), with Spanish and English speakers having the lowest no-show rate (33%). Overall, interpreter use at the first visit was not significantly associated with subsequent appointment attendance (p = 0.3674). Patients with a documented Spanish interpreter at the first visit had the lowest average no-show rate (31% ± 19%) compared to Haitian Creole (42% ± 18%) and all other languages (32% ± 19%) (p = 0.0265). Hispanic ethnicity, maternal education level, and maternal primary language were not associated with attendance.

CONCLUSION

Interpreter use at the first visit was not significantly correlated with no-show rates, but among patients that did require an interpreter at the first visit, those receiving services in Spanish had the best clinic attendance.

To charge or not to charge: reducing patient no-show

BACKGROUND

In order to reduce patient no-show, the Israeli government is promoting legislation that will allow Health Plans to require a co-payment from patients when reserving an appointment. It is hoped that this will create an incentive for patients to cancel in advance rather than simply not show up. The goal of this policy is to improve patient access to medical care and ensure that healthcare resources are utilized effectively. We explore this phenomenon to support evidence-based decision making on this issue, and to determine whether the proposed legislation is aligned with the findings of previous studies.

MAIN BODY

No-show rates vary across countries and healthcare services, with several strategies in place to mitigate the phenomenon. There are three key stakeholders involved: (1) patients, (2) medical staff, and (3) insurers/managed care organizations, each of which is affected differently by no-shows and faces a different set of incentives. The decision whether to impose financial penalties for no-shows should take a number of considerations into account, such as the fine amount, service type, the establishment of an effective fine collection system, the patient's socioeconomic status, and the potential for exacerbating disparities in healthcare access. The limited research on the impact of fines on no-show rates has produced mixed results. Further investigation is necessary to understand the influence of fine amounts on no-show rates across various healthcare services. Additionally, it is important to evaluate the implications of this proposed legislation on patient behavior, access to healthcare, and potential disparities in access.

CONCLUSION

It is anticipated that the proposed legislation will have minimal impact on attendance rates. To achieve meaningful change, efforts should focus on enhancing medical service availability and improving the ease with which appointments can be cancelled or alternatively substantial fines should be imposed. Further research is imperative for determining the most effective way to address the issue of patient no-show and to enhance healthcare system efficiency.

Predicting and explaining absenteeism risk in hospital patients before and during COVID-19

In order to address one of the most challenging problems in hospital management - patients' absenteeism without prior notice - this study analyses the risk factors associated with this event. To this end, through real data from a hospital located in the North of Portugal, a prediction model previously validated in the literature is used to infer absenteeism risk factors, and an explainable model is proposed, based on a modified CART algorithm. The latter intends to generate a human-interpretable explanation for patient absenteeism, and its implementation is described in detail. Furthermore, given the significant impact, the COVID-19 pandemic had on hospital management, a comparison between patients' profiles upon absenteeism before and during the COVID-19 pandemic situation is performed. Results obtained differ between hospital specialities and time periods meaning that patient profiles on absenteeism change during pandemic periods and within specialities.

Sociodemographic and Clinical Factors Associated with Nonattendance at the Hepatology Clinic

BACKGROUND

Absenteeism from clinic appointments reduces efficiency, wastes resources, and contributes to longer wait times. There are limited data regarding factors associated with nonattendance in hepatology clinics. Identifying factors related to appointment nonattendance may help in the design of interventions for reducing absenteeism.

METHODS

We aim to identify sociodemographic, clinical, and appointment-related factors associated with absenteeism following referral to a liver clinic in a tertiary academic center located in the US Midwest. We designed a case-control study using data from electronic medical records of patients scheduled for appointments between January 2016 and December 2021. Cases were defined as patients who canceled appointments on the same day or resulting in no-shows, and controls were those who completed the referral visit. Information about patients' sociodemographic characteristics, appointment details, and etiology of liver disease were recorded. Hierarchical logistic regression was used to analyze factors related to nonattendance.

RESULTS

Of 3404 scheduled appointments, 460 (13.5%) missed visits were recorded. In the multivariable logistic regression models, hepatitis C and alcohol-associated liver disease were associated with greater odds of nonattendance [odds ratio (OR) 4.0 (95% CI 3.2-4.9), OR 2.7 (1.7-4.2), respectively] compared to those with other liver disease. Sociodemographic characteristics associated with risk of nonattendance included being Black [OR 2.6, (1.8-3.7)], Medicaid insurance or no insurance [OR 2.3 (1.7-2.9), OR 2.5 (1.6-3.7), respectively], non-English speaking [OR 1.8 (1.1-3.1)], being unmarried [OR 1.8 (1.4-2.2)], and longer wait time (> 30 days) until appointments [OR 1.8 (1.5-2.2)].

CONCLUSION

Several sociodemographic and administrative characteristics, as well as hepatitis C and alcohol-associated liver disease were associated with appointment nonattendance. Targeted future interventions may help to decrease nonattendance.

Development and validation of nonattendance predictive models for scheduled adult outpatient appointments in different medical specialties

INTRODUCTION

Nonattendance is a critical problem that affects health care worldwide. Our aim was to build and validate predictive models of nonattendance in all outpatients appointments, general practitioners, and clinical and surgical specialties.

METHODS

A cohort study of adult patients, who had scheduled outpatient appointments for General Practitioners, Clinical and Surgical specialties, was conducted between January 2015 and December 2016, at the Italian Hospital of Buenos Aires. We evaluated potential predictors grouped in baseline patient characteristics, characteristics of the appointment scheduling process, patient history, characteristics of the appointment, and comorbidities. Patients were divided between those who attended their appointments, and those who did not. We generated predictive models for nonattendance for all appointments and the three subgroups.

RESULTS

Of 2,526,549 appointments included, 703,449 were missed (27.8%). The predictive model for all appointments contains 30 variables, with an area under the ROC (AUROC) curve of 0.71, calibration-in-the-large (CITL) of 0.046, and calibration slope of 1.03 in the validation cohort. For General Practitioners the model has 28 variables (AUROC of 0.72, CITL of 0.053, and calibration slope of 1.01). For clinical subspecialties, the model has 23 variables (AUROC of 0.71, CITL of 0.039, and calibration slope of 1), and for surgical specialties, the model has 22 variables (AUROC of 0.70, CITL of 0.023, and calibration slope of 1.01).

CONCLUSION

We build robust predictive models of nonattendance with adequate precision and calibration for each of the subgroups.

A Predictive Decision Analytics Approach for Primary Care Operations Management: A Case Study of Double-Booking Strategy Design and Evaluation

Primary care plays a vital role for individuals and families in accessing care, keeping well, and improving quality of life. However, the complexities and uncertainties in the primary care delivery system (e.g., patient no-shows/walk-ins, staffing shortage, COVID-19 pandemic) have brought significant challenges in its operations management, which can potentially lead to poor patient outcomes and negative primary care operations (e.g., loss of productivity, inefficiency). This paper presents a decision analytics approach developed based on predictive analytics and hybrid simulation to better facilitate management of the underlying complexities and uncertainties in primary care operations. A case study was conducted in a local family medicine clinic to demonstrate the use of this approach for patient no-show management. In this case study, a patient no-show prediction model was used in conjunction with an integrated agent-based and discrete-event simulation model to design and evaluate double-booking strategies. Using the predicted patient no-show information, a prediction-based double-booking strategy was created and compared against two other strategies, namely random and designated time. Scenario-based experiments were then conducted to examine the impacts of different double-booking strategies on clinic's operational outcomes, focusing on the trade-offs between the clinic productivity (measured by daily patient throughput) and efficiency (measured by visit cycle and patient wait time for doctor). The results showed that the best productivity-efficiency balance was derived under the prediction-based double-booking strategy. The proposed hybrid decision analytics approach has the potential to better support decision-making in primary care operations management and improve the system's performance. Further, it can be generalized in the context of various healthcare settings for broader applications.

Predicting no-show appointments in a pediatric hospital in Chile using machine learning

The Chilean public health system serves 74% of the country's population, and 19% of medical appointments are missed on average because of no-shows. The national goal is 15%, which coincides with the average no-show rate reported in the private healthcare system. Our case study, Doctor Luis Calvo Mackenna Hospital, is a public high-complexity pediatric hospital and teaching center in Santiago, Chile. Historically, it has had high no-show rates, up to 29% in certain medical specialties. Using machine learning algorithms to predict no-shows of pediatric patients in terms of demographic, social, and historical variables. To propose and evaluate metrics to assess these models, accounting for the cost-effective impact of possible intervention strategies to reduce no-shows. We analyze the relationship between a no-show and demographic, social, and historical variables, between 2015 and 2018, through the following traditional machine learning algorithms: Random Forest, Logistic Regression, Support Vector Machines, AdaBoost and algorithms to alleviate the problem of class imbalance, such as RUS Boost, Balanced Random Forest, Balanced Bagging and Easy Ensemble. These class imbalances arise from the relatively low number of no-shows to the total number of appointments. Instead of the default thresholds used by each method, we computed alternative ones via the minimization of a weighted average of type I and II errors based on cost-effectiveness criteria. 20.4% of the 395,963 appointments considered presented no-shows, with ophthalmology showing the highest rate among specialties at 29.1%. Patients in the most deprived socioeconomic group according to their insurance type and commune of residence and those in their second infancy had the highest no-show rate. The history of non-attendance is strongly related to future no-shows. An 8-week experimental design measured a decrease in no-shows of 10.3 percentage points when using our reminder strategy compared to a control group. Among the variables analyzed, those related to patients' historical behavior, the reservation delay from the creation of the appointment, and variables that can be associated with the most disadvantaged socioeconomic group, are the most relevant to predict a no-show. Moreover, the introduction of new cost-effective metrics significantly impacts the validity of our prediction models. Using a prototype to call patients with the highest risk of no-shows resulted in a noticeable decrease in the overall no-show rate.

Exploring potential barriers in equitable access to pediatric diagnostic imaging using machine learning

In this work, we examine magnetic resonance imaging (MRI) and ultrasound (US) appointments at the Diagnostic Imaging (DI) department of a pediatric hospital to discover possible relationships between selected patient features and no-show or long waiting room time endpoints. The chosen features include age, sex, income, distance from the hospital, percentage of non-English speakers in a postal code, percentage of single caregivers in a postal code, appointment time slot (morning, afternoon, evening), and day of the week (Monday to Sunday). We trained univariate Logistic Regression (LR) models using the training sets and identified predictive (significant) features that remained significant in the test sets. We also implemented multivariate Random Forest (RF) models to predict the endpoints. We achieved Area Under the Receiver Operating Characteristic Curve (AUC) of 0.82 and 0.73 for predicting no-show and long waiting room time endpoints, respectively. The univariate LR analysis on DI appointments uncovered the effect of the time of appointment during the day/week, and patients' demographics such as income and the number of caregivers on the no-shows and long waiting room time endpoints. For predicting no-show, we found age, time slot, and percentage of single caregiver to be the most critical contributors. Age, distance, and percentage of non-English speakers were the most important features for our long waiting room time prediction models. We found no sex discrimination among the scheduled pediatric DI appointments. Nonetheless, inequities based on patient features such as low income and language barrier did exist.

Improving Hospital Outpatient Clinics Appointment Schedules by Prediction Models

Patient no-shows and suboptimal patient appointment length scheduling reduce clinical efficiency and impair the clinic's quality of service. The main objective of this study is to improve appointment scheduling in hospital outpatient clinics. We developed generic supervised machine learning models to predict patient no-shows and patient's length of appointment (LOA). We performed a retrospective study using more than 100,000 records of patient appointments in a hospital outpatient clinic. Several machine learning algorithms were used for the development of our prediction models. We trained our models on a dataset that contained patients', physicians', and appointments' characteristics. Our feature set combines both unstudied features and features adopted from previous studies. In addition, we identified the influential features for predicting LOA and no-show. Our LOA model's performance was 6.92 in terms of MAE, and our no-show model's performance was 92.1% in terms of F-score. We compared our models' performance to the performance of previous research models by applying their methods to our dataset; our models demonstrated better performance. We show that the major effector of such differences is the use of our novel features. To evaluate the effect of our prediction results on the quality of schedules produced by appointment systems (AS), we developed an interface layer between our prediction models and the AS, where prediction results comprise the AS input. Using our prediction models, there was an 80% improvement in the daily cumulative patient waiting time and a 33% reduction in the daily cumulative physician idle time.

Predictive model-based interventions to reduce outpatient no-shows: a rapid systematic review

OBJECTIVE

Outpatient no-shows have important implications for costs and the quality of care. Predictive models of no-shows could be used to target intervention delivery to reduce no-shows. We reviewed the effectiveness of predictive model-based interventions on outpatient no-shows, intervention costs, acceptability, and equity.

MATERIALS AND METHODS

Rapid systematic review of randomized controlled trials (RCTs) and non-RCTs. We searched Medline, Cochrane CENTRAL, Embase, IEEE Xplore, and Clinical Trial Registries on March 30, 2022 (updated on July 8, 2022). Two reviewers extracted outcome data and assessed the risk of bias using ROB 2, ROBINS-I, and confidence in the evidence using GRADE. We calculated risk ratios (RRs) for the relationship between the intervention and no-show rates (primary outcome), compared with usual appointment scheduling. Meta-analysis was not possible due to heterogeneity.

RESULTS

We included 7 RCTs and 1 non-RCT, in dermatology (n = 2), outpatient primary care (n = 2), endoscopy, oncology, mental health, pneumology, and an magnetic resonance imaging clinic. There was high certainty evidence that predictive model-based text message reminders reduced no-shows (1 RCT, median RR 0.91, interquartile range [IQR] 0.90, 0.92). There was moderate certainty evidence that predictive model-based phone call reminders (3 RCTs, median RR 0.61, IQR 0.49, 0.68) and patient navigators reduced no-shows (1 RCT, RR 0.55, 95% confidence interval 0.46, 0.67). The effect of predictive model-based overbooking was uncertain. Limited information was reported on cost-effectiveness, acceptability, and equity.

DISCUSSION AND CONCLUSIONS

Predictive modeling plus text message reminders, phone call reminders, and patient navigator calls are probably effective at reducing no-shows. Further research is needed on the comparative effectiveness of predictive model-based interventions addressed to patients at high risk of no-shows versus nontargeted interventions addressed to all patients.

Predicting no-shows for dental appointments

Patient no-shows is a significant problem in healthcare, reaching up to 80% of booked appointments and costing billions of dollars. Predicting no-shows for individual patients empowers clinics to implement better mitigation strategies. Patients' no-show behavior varies across health clinics and the types of appointments, calling for fine-grained studies to uncover these variations in no-show patterns. This article focuses on dental appointments because they are notably longer than regular medical appointments due to the complexity of dental procedures. We leverage machine learning techniques to develop predictive models for dental no-shows, with the best model achieving an Area Under the Curve (AUC) of 0.718 and an F1 score of 66.5%. Additionally, we propose and evaluate a novel method to represent no-show history as a binary sequence of events, enabling the predictive models to learn the associated future no-show behavior with these patterns. We discuss the utility of no-show predictions to improve the scheduling of dental appointments, such as reallocating appointments and reducing their duration.

Treatment Discontinuation Prediction in Patients With Diabetes Using a Ranking Model: Machine Learning Model Development

BACKGROUND

Treatment discontinuation (TD) is one of the major prognostic issues in diabetes care, and several models have been proposed to predict a missed appointment that may lead to TD in patients with diabetes by using binary classification models for the early detection of TD and for providing intervention support for patients. However, as binary classification models output the probability of a missed appointment occurring within a predetermined period, they are limited in their ability to estimate the magnitude of TD risk in patients with inconsistent intervals between appointments, making it difficult to prioritize patients for whom intervention support should be provided.

OBJECTIVE

This study aimed to develop a machine-learned prediction model that can output a TD risk score defined by the length of time until TD and prioritize patients for intervention according to their TD risk.

METHODS

This model included patients with diagnostic codes indicative of diabetes at the University of Tokyo Hospital between September 3, 2012, and May 17, 2014. The model was internally validated with patients from the same hospital from May 18, 2014, to January 29, 2016. The data used in this study included 7551 patients who visited the hospital after January 1, 2004, and had diagnostic codes indicative of diabetes. In particular, data that were recorded in the electronic medical records between September 3, 2012, and January 29, 2016, were used. The main outcome was the TD of a patient, which was defined as missing a scheduled clinical appointment and having no hospital visits within 3 times the average number of days between the visits of the patient and within 60 days. The TD risk score was calculated by using the parameters derived from the machine-learned ranking model. The prediction capacity was evaluated by using test data with the C-index for the performance of ranking patients, area under the receiver operating characteristic curve, and area under the precision-recall curve for discrimination, in addition to a calibration plot.

RESULTS

The means (95% confidence limits) of the C-index, area under the receiver operating characteristic curve, and area under the precision-recall curve for the TD risk score were 0.749 (0.655, 0.823), 0.758 (0.649, 0.857), and 0.713 (0.554, 0.841), respectively. The observed and predicted probabilities were correlated with the calibration plots.

CONCLUSIONS

A TD risk score was developed for patients with diabetes by combining a machine-learned method with electronic medical records. The score calculation can be integrated into medical records to identify patients at high risk of TD, which would be useful in supporting diabetes care and preventing TD.

Prospective Evaluation of a Machine-Learning Prediction Model for Missed Radiology Appointments

The term “no-show” refers to scheduled appointments that a patient misses, or for which she arrives too late to utilize medical resources. Accurately predicting no-shows creates opportunities to intervene, ensuring that patients receive needed medical resources. A machine-learning (ML) model can accurately identify individuals at high no-show risk, to facilitate strategic and targeted interventions. We used 4,546,104 non-same-day scheduled appointments in our medical system from 1/1/2017 through 1/1/2020 for training data, including 631,386 no-shows. We applied eight ML techniques, which yielded cross-validation AUCs of 0.77–0.93. We then prospectively tested the best performing model, Gradient Boosted Regression Trees, over a 6-week period at a single outpatient location. We observed 123 no-shows. The model accurately identified likely no-show patients retrospectively (AUC 0.93) and prospectively (AUC 0.73, p < 0.0005). Individuals in the highest-risk category were three times more likely to no-show than the average of all other patients. No-show prediction modeling based on machine learning has the potential to identify patients for targeted interventions to improve their access to medical resources, reduce waste in the medical system and improve overall operational efficiency. Caution is advised, due to the potential for bias to decrease the quality of service for patients based on race, zip code, and gender.

A Review of Technological Forecasting from the Perspective of Complex Systems

Technology forecasting (TF) is an important way to address technological innovation in fast-changing market environments and enhance the competitiveness of organizations in dynamic and complex environments. However, few studies have investigated the complex process problem of how to select the most appropriate forecasts for organizational characteristics. This paper attempts to fill this research gap by reviewing the TF literature based on a complex systems perspective. We first identify four contexts (technology opportunity identification, technology assessment, technology trend and evolutionary analysis, and others) involved in the systems of TF to indicate the research boundary of the system. Secondly, the four types of agents (field of analysis, object of analysis, data source, and approach) are explored to reveal the basic elements of the systems. Finally, the visualization of the interaction between multiple agents in full context and specific contexts is realized in the form of a network. The interaction relationship network illustrates how the subjects coordinate and cooperate to realize the TF context. Accordingly, we illustrate suggest five trends for future research: (1) refinement of the context; (2) optimization and expansion of the analysis field; (3) extension of the analysis object; (4) convergence and diversification of the data source; and (5) combination and optimization of the approach.

Reducing non-attendance in outpatient appointments: predictive model development, validation, and clinical assessment

Background

Non-attendance to scheduled hospital outpatient appointments may compromise healthcare resource planning, which ultimately reduces the quality of healthcare provision by delaying assessments and increasing waiting lists. We developed a model for predicting non-attendance and assessed the effectiveness of an intervention for reducing non-attendance based on the model.

Methods

The study was conducted in three stages: (1) model development, (2) prospective validation of the model with new data, and (3) a clinical assessment with a pilot study that included the model as a stratification tool to select the patients in the intervention. Candidate models were built using retrospective data from appointments scheduled between January 1, 2015, and November 30, 2018, in the dermatology and pneumology outpatient services of the Hospital Municipal de Badalona (Spain). The predictive capacity of the selected model was then validated prospectively with appointments scheduled between January 7 and February 8, 2019. The effectiveness of selective phone call reminders to patients at high risk of non-attendance according to the model was assessed on all consecutive patients with at least one appointment scheduled between February 25 and April 19, 2019. We finally conducted a pilot study in which all patients identified by the model as high risk of non-attendance were randomly assigned to either a control (no intervention) or intervention group, the last receiving phone call reminders one week before the appointment.

Results

Decision trees were selected for model development. Models were trained and selected using 33,329 appointments in the dermatology service and 21,050 in the pneumology service. Specificity, sensitivity, and accuracy for the prediction of non-attendance were 79.90%, 67.09%, and 73.49% for dermatology, and 71.38%, 57.84%, and 64.61% for pneumology outpatient services. The prospective validation showed a specificity of 78.34% (95%CI 71.07, 84.51) and balanced accuracy of 70.45% for dermatology; and 69.83% (95%CI 60.61, 78.00) for pneumology, respectively. The effectiveness of the intervention was assessed on 1,311 individuals identified as high risk of non-attendance according to the selected model. Overall, the intervention resulted in a significant reduction in the non-attendance rate to both the dermatology and pneumology services, with a decrease of 50.61% (p<0.001) and 39.33% (p=0.048), respectively.

Conclusions

The risk of non-attendance can be adequately estimated using patient information stored in medical records. The patient stratification according to the non-attendance risk allows prioritizing interventions, such as phone call reminders, to effectively reduce non-attendance rates.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07865-y.

Neighborhood Effects on Missed Appointments in a Large Urban Academic Multispecialty Practice

BACKGROUND

Missed appointments diminish the continuity and quality of care.

OBJECTIVE

To determine whether missing scheduled appointments is associated with characteristics of the populations in places where patients reside.

DESIGN

Retrospective cross-sectional study using data extracted from electronic health records linked to population descriptors for each patient's census tract of residence.

PATIENTS

A total of 58,981 patients ≥18 years of age with 275,682 scheduled appointments during 2014-2015 at a multispecialty outpatient practice.

MAIN MEASURES

We used multinomial generalized linear mixed models to examine associations between the outcomes of scheduled appointments (arrived, canceled, or missed) and selected characteristics of the populations in patients' census tracts of residence (racial/ethnic segregation based on population composition, levels of poverty, violent crime, and perceived safety and social capital), controlling for patients' age, gender, type of insurance, and type of clinic service.

KEY RESULTS

Overall, 17.5% of appointments were missed. For appointments among patients residing in census tracts in the highest versus lowest quartile for each population metric, adjusted odds ratios (aORs) for missed appointments were 1.27 (CI 1.19, 1.35) for the rate of violent crime, 1.27 (CI 1.20, 1.34) for the proportion Hispanic, 1.19 (CI 1.12, 1.27) for the proportion living in poverty, 1.13 (CI 1.05, 1.20) for the proportion of the census tract population that was Black, and 1.06 (CI 1.01, 1.11 for perceived neighborhood safety.

CONCLUSIONS

Characteristics of the places where patients reside are associated with missing scheduled appointments, including high levels of racial/ethnic segregation, poverty, and violent crime and low levels of perceived neighborhood safety. As such, targeting efforts to improve access for patients living in such neighborhoods will be particularly important to address underlying social determinants of access to health care.

Predictive and Causal Analysis of No-Shows for Medical Exams During COVID-19: A Case Study of Breast Imaging in a Nationwide Israeli Health Organization

"No-shows", defined as missed appointments or late cancellations, is a central problem in healthcare systems. It has appeared to intensify during the COVID-19 pandemic and the nonpharmaceutical interventions, such as closures, taken to slow its spread. No-shows interfere with patients' continuous care, lead to inefficient utilization of medical resources, and increase healthcare costs. We present a comprehensive analysis of no-shows for breast imaging appointments made during 2020 in a large medical network in Israel. We applied advanced machine learning methods to provide insights into novel and known predictors. Additionally, we employed causal inference methodology to infer the effect of closures on no-shows, after accounting for confounding biases, and demonstrate the superiority of adversarial balancing over inverse probability weighting in correcting these biases. Our results imply that a patient's perceived risk of cancer and the COVID-19 time-based factors are major predictors. Further, we reveal that closures impact patients over 60, but not patients undergoing advanced diagnostic examinations.

Application of Machine Learning Techniques to Predict a Patient’s No-Show in the Healthcare Sector

The health sector faces a series of problems generated by patients who miss their scheduled appointments. The main challenge to this problem is to understand the patient’s profile and predict potential absences. The goal of this work is to explore the main causes that contribute to a patient’s no-show and develop a prediction model able to identify whether the patient will attend their scheduled appointment or not. The study was based on data from clinics that serve the Unified Health System (SUS) at the University of Vale do Itajaí in southern Brazil. The model obtained was tested on a real collected dataset with about 5000 samples. The best model result was performed by the Random Forest classifier. It had the best Recall Rate (0.91) and achieved an ROC curve rate of 0.969. This research was approved and authorized by the Ethics Committee of the University of Vale do Itajaí, under opinion 4270,234, contemplating the General Data Protection Law.

A novel transition clinic structure for adolescent and young adult patients with childhood onset rheumatic disease improves transition outcomes

BACKGROUND

The transition of health care from Pediatric to Adult providers for adolescents and young adults with chronic disease is associated with poor outcomes. Despite the importance of this transition, over 80% of these patients do not receive the services necessary to transition to Adult health care. In 2018, we initiated a transition clinic structure, integrating an Internal Medicine - Pediatrics trained Adult Rheumatologist in a Pediatric Rheumatology clinic to guide this transition. Our goal was to improve transition outcomes. We report the methods of this clinic and its preliminary outcomes.

METHODS

For patients referred to the transition clinic, the Adult Rheumatologist assumed medical management and implemented a six-part modular transition curriculum. This curriculum included a Transition Policy, Transition Readiness Assessment, medication review and education, diagnosis review and education, and counseling on differences between Pediatric and Adult-oriented care. Eligible patients and their families were enrolled in a prospective observational outcomes research registry. Initial data from this transition clinic is reported including adherence with certain aspects of the transition curriculum and clinic utilization.

RESULTS

The transition clinic Adult Rheumatologist saw 177 patients in 2 years, and 57 patients were eligible for, approached, and successfully enrolled in the registry. From this registry, all patients reviewed the Transition Policy with the Adult Rheumatologist and 45 (78.9%) completed at least one Transition Readiness Assessment. Of the 22 patients for whom transition was indicated, all were successfully transitioned to an Adult Rheumatologist. 17 (77.3%) continued care post-transition with the transition clinic Adult Rheumatologist, and 5 (22.7%) continued care post-transition with a different Adult Rheumatologist. The median time between the last transition clinic visit and first Adult clinic visit was 5.1 months.

CONCLUSIONS

Our experience demonstrated the success of our clinic model regarding participation in the transition curriculum and improved clinic utilization data. Our results are an improvement over transition rates reported elsewhere that did not implement our model. We believe that this structure could be applied to other primary care and subspecialty clinics.

TRIAL REGISTRATION

This research was approved by the University of Utah Institutional Review Board (IRB) in January 2019 (IRB_00115964). Patients were retrospectively registered if involved prior to this date.

A Probabilistic Patient Scheduling Model with Time Variable Slots

One of the current challenges faced by health centers is to reduce the number of patients who do not attend their appointments. The existence of these patients causes the underutilization of the center's services, which reduces their income and extends patient's access time. In order to reduce these negative effects, several appointment scheduling systems have been developed. With the recent availability of electronic health records, patient scheduling systems that incorporate the patient's no-show prediction are being developed. However, the benefits of including a personalized individual variable time slot for each patient in those probabilistic systems have not been yet analyzed. In this article, we propose a scheduling system based on patients' no-show probabilities with variable time slots and a dynamic priority allocation scheme. The system is based on the solution of a mixed-integer programming model that aims at maximizing the expected profits of the clinic, accounting for first and follow-up visits. We validate our findings by performing an extensive simulation study based on real data and specific scheduling requirements provided by a Spanish hospital. The results suggest potential benefits with the implementation of the proposed allocation system with variable slot times. In particular, the proposed model increases the annual cumulated profit in more than 50% while decreasing the waiting list and waiting times by 30% and 50%, respectively, with respect to the actual appointment scheduling system.