Enhancing site selection strategies in clinical trial recruitment using real-world data modeling

Slow patient enrollment or failing to enroll the required number of patients is a disruptor of clinical trial timelines. To meet the planned trial recruitment, site selection strategies are used during clinical trial planning to identify research sites that are most likely to recruit a sufficiently high number of subjects within trial timelines. We developed a machine learning approach that outperforms baseline methods to rank research sites based on their expected recruitment in future studies. Indication level historical recruitment and real-world data are used in the machine learning approach to predict patient enrollment at site level. We define covariates based on published recruitment hypotheses and examine the effect of these covariates in predicting patient enrollment. We compare model performance of a linear and a non-linear machine learning model with common industry baselines that are constructed from historical recruitment data. Performance of the methodology is evaluated and reported for two disease indications, inflammatory bowel disease and multiple myeloma, both of which are actively being pursued in clinical development. We validate recruitment hypotheses by reviewing the covariates relationship with patient recruitment. For both indications, the non-linear model significantly outperforms the baselines and the linear model on the test set. In this paper, we present a machine learning approach to site selection that incorporates site-level recruitment and real-world patient data. The model ranks research sites by predicting the number of recruited patients and our results suggest that the model can improve site ranking compared to common industry baselines.

A Practical Approach to Predicting Surgical Site Infection Risk Among Patients Before Leaving the Operating Room

A surgical site infection (SSI) prediction model that identifies at-risk patients before leaving the operating room can support efforts to improve patient safety. In this study, eight pre-operative and five perioperative patient- and procedure-specific characteristics were tested with two scoring algorithms: 1) count of positive factors (manual), and 2) logistic regression model (automated). Models were developed and validated using data from 3,440 general and oncologic surgical patients. In the automated algorithm, two pre-operative (procedure urgency, odds ratio [OR]: 1.7; and antibiotic administration >2 hours before incision, OR: 1.6) and three intraoperative risk factors (open surgery [OR: 3.7], high-risk procedure [OR: 3.5], and operative time OR: [2.6]) were associated with SSI risk. The manual score achieved an area under the curve (AUC) of 0.831 and the automated algorithm achieved AUC of 0.868. Open surgery had the greatest impact on prediction, followed by procedure risk, operative time, and procedure urgency. At 80% sensitivity, the manual and automated scores achieved a positive predictive value of 16.3% and 22.0%, respectively. Both the manual and automated SSI risk prediction algorithms accurately identified at-risk populations. Use of either model before the patient leaves the operating room can provide the clinical team with evidence-based guidance to consider proactive intervention to prevent SSIs.

Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark

PURPOSE

Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill.

METHODS

To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment.

RESULTS

F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team).

CONCLUSION

Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

Surgical skill assessment using machine learning algorithms

Objective

Surgical skill is correlated with clinical outcomes. Therefore, the assessment of surgical skill is of major importance to improve clinical outcomes and increase patient safety. However, surgical skill assessment often lacks objectivity and reproducibility. Furthermore, it is time-consuming and expensive. Therefore, we developed an automated surgical skill assessment using machine learning algorithms.

Methods

Surgical skills were assessed in videos of laparoscopic cholecystectomy using a three-step machine learning algorithm. First, a three-dimensional convolutional neural network was trained to localize and classify the instruments within the videos. Second, movement patterns of the instruments were recorded over time and extracted. Third, the movement patterns were correlated with human surgical skill ratings using a linear regression model to predict surgical skill ratings automatically. Machine ratings were compared against human ratings of four board certified surgeons using a score ranging from 1 (poor skills) to 5 (excellent skills).

Results

Human raters and machine learning algorithms assessed surgical skills in 242 videos. Inter-rater reliability for human raters was excellent (79%, 95%CI 72-85%). Instrument detection showed an average precision of 78% and average recall of 82%. Machine learning algorithms showed an 87% accuracy in predicting good or poor surgical skills, when compared to human raters.

Conclusion

Machine learning algorithms can be trained to distinguish good and poor surgical skills with high accuracy.

This work was published in Sci Rep 11, 5197 (2021). https://doi.org/10.1038/s41598-021-84295-6

Automation of surgical skill assessment using a three-stage machine learning algorithm

Surgical skills are associated with clinical outcomes. To improve surgical skills and thereby reduce adverse outcomes, continuous surgical training and feedback is required. Currently, assessment of surgical skills is a manual and time-consuming process which is prone to subjective interpretation. This study aims to automate surgical skill assessment in laparoscopic cholecystectomy videos using machine learning algorithms. To address this, a three-stage machine learning method is proposed: first, a Convolutional Neural Network was trained to identify and localize surgical instruments. Second, motion features were extracted from the detected instrument localizations throughout time. Third, a linear regression model was trained based on the extracted motion features to predict surgical skills. This three-stage modeling approach achieved an accuracy of 87 ± 0.2% in distinguishing good versus poor surgical skill. While the technique cannot reliably quantify the degree of surgical skill yet it represents an important advance towards automation of surgical skill assessment.

Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge

Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. While numerous methods for detecting, segmenting and tracking of medical instruments based on endoscopic video images have been proposed in the literature, key limitations remain to be addressed: Firstly, robustness, that is, the reliable performance of state-of-the-art methods when run on challenging images (e.g. in the presence of blood, smoke or motion artifacts). Secondly, generalization; algorithms trained for a specific intervention in a specific hospital should generalize to other interventions or institutions. In an effort to promote solutions for these limitations, we organized the Robust Medical Instrument Segmentation (ROBUST-MIS) challenge as an international benchmarking competition with a specific focus on the robustness and generalization capabilities of algorithms. For the first time in the field of endoscopic image processing, our challenge included a task on binary segmentation and also addressed multi-instance detection and segmentation. The challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures from three different types of surgery. The validation of the competing methods for the three tasks (binary segmentation, multi-instance detection and multi-instance segmentation) was performed in three different stages with an increasing domain gap between the training and the test data. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap. While the average detection and segmentation quality of the best-performing algorithms is high, future research should concentrate on detection and segmentation of small, crossing, moving and transparent instrument(s) (parts).

Olfactory habituation in Drosophila-odor encoding and its plasticity in the antennal lobe

A ubiquitous feature of an animal's response to an odorant is that it declines when the odorant is frequently or continuously encountered. This decline in olfactory response, termed olfactory habituation, can have temporally or mechanistically different forms. The neural circuitry of the fruit fly Drosophila melanogaster's olfactory system is well defined in terms of component cells, which are readily accessible to functional studies and genetic manipulation. This makes it a particularly useful preparation for the investigation of olfactory habituation. In addition, the insect olfactory system shares many architectural and functional similarities with mammalian olfactory systems, suggesting that olfactory mechanisms in insects may be broadly relevant. In this chapter, we discuss the likely mechanisms of olfactory habituation in context of the participating cell types, their connectivity, and their roles in sensory processing. We overview the structure and function of key cell types, the mechanisms that stimulate them, and how they transduce and process odor signals. We then consider how each stage of olfactory processing could potentially contribute to behavioral habituation. After this, we overview a variety of recent mechanistic studies that point to an important role for potentiation of inhibitory synapses in the primary olfactory processing center, the antennal lobe, in driving the reduced response to familiar odorants. Following the discussion of mechanisms for short- and long-term olfactory habituation, we end by considering how these mechanisms may be regulated by neuromodulators, which likely play key roles in the induction, gating, or suppression of habituated behavior, and speculate on the relevance of these processes for other forms of learning and memory.