How to assess the risks associated with the usage of a medical device based on predictive modeling: the case of an anemia control model certified as medical device

BACKGROUND

The successful application of Machine Learning (ML) to many clinical problems can lead to its implementation as a medical device (MD), which is important to assess the associated risks.

METHODS

An anemia control model (ACM), certified as MD, may face adverse events as a result of wrong predictions that are translated into suggestions of doses of erythropoietic stimulating agents to dialysis patients. Risks are assessed as the combination of severity and probability of a given hazard. While severities are typically assessed by clinicians, probabilities are tightly related to the performance of the predictive model.

RESULTS

A postmarketing data set formed by all adult patients registered in French, Portuguese, and Spanish clinics, belonging to an international network, was considered; 3876 patients and 11,508 suggestions were eventually included. The achieved results show that there are no statistical differences between the probabilities of adverse events that are estimated in the ACM test set (using only Spanish clinics) and those actually observed in the postmarketing cohort.

CONCLUSIONS

The risks of an ACM-MD can be accurately and robustly estimated, thus enhancing patients' safety. The proposed methodology is applicable to other clinical decisions based on predictive models since our proposal does not depend on the particular predictive model.

P1378PROCESS NORMALIZATION AND CLINICAL OUTCOMES FOLLOWING IMPLEMENTATION OF THE ANEMIA CONTROL MODEL FOR PERSONALIZED ANEMIA MANAGEMENT

Background and Aims

Anemia management is still challenging in a large share of dialysis patients. ESA resistance show wide variations across patients as well as fluctuations along time within the same patient. The Anemia Control Model (ACM) is a software application which can be integrated in electronic health record systems. ACM is based on an artificial neural network system trained on 1 million health records which simulates patient’s personalized dose-response relationship to ESA and Iron. Based on patients’ response forecast, ACM suggests the optimal drug dose recommendations to reach or maintain the hemoglobin level within target and to minimize the drug consumption. The ACM has earned medical device certification and it is currently used in 13 countries within the EMEA FMC network. Effectiveness and safety of ACM has been demonstrated in previous real-world evidence studies. We report ACM use and its clinical outcomes among hemodialysis patients registered in the Spanish FME Network

Method

We recorded all medical prescriptions and their clinical outcome for which an ACM recommendation was computed (ACM prescription). prescription occurred in 9 clinics located in Catalunya (Spain) from September 2017 to September 2018. We report dynamics of ACM suggestion acceptance rate, ESA consumption and Hb target achievement in the study period.

Results

There were 1189 active patients per month during the study period on average. ACM suggestion acceptance rate was 73,3%. ESA monthly consumption ranged from 234:7±281:9 – 310.2 ± 310:8 UI/Kg; 86% of patients receiving ESA doses according to ACM recommendations achieved or remained within Hb target range; on the contrary, when physicians rejected ACM recommendations and prescribed a different dosage, only 77% of patients reached or maintained the Hb level on target (p<0.05).

Conclusion

Our study demonstrated that ACM suggestions were frequently accepted by physicians. Acceptance of ACM suggestion was associated with higher Hb target achievement.

Performance of a Predictive Model for Long-Term Hemoglobin Response to Darbepoetin and Iron Administration in a Large Cohort of Hemodialysis Patients

Anemia management, based on erythropoiesis stimulating agents (ESA) and iron supplementation, has become an increasingly challenging problem in hemodialysis patients. Maintaining hemodialysis patients within narrow hemoglobin targets, preventing cycling outside target, and reducing ESA dosing to prevent adverse outcomes requires considerable attention from caregivers. Anticipation of the long-term response (i.e. at 3 months) to the ESA/iron therapy would be of fundamental importance for planning a successful treatment strategy. To this end, we developed a predictive model designed to support decision-making regarding anemia management in hemodialysis (HD) patients treated in center. An Artificial Neural Network (ANN) algorithm for predicting hemoglobin concentrations three months into the future was developed and evaluated in a retrospective study on a sample population of 1558 HD patients treated with intravenous (IV) darbepoetin alfa, and IV iron (sucrose or gluconate). Model inputs were the last 90 days of patients’ medical history and the subsequent 90 days of darbepoetin/iron prescription. Our model was able to predict individual variation of hemoglobin concentration 3 months in the future with a Mean Absolute Error (MAE) of 0.75 g/dL. Error analysis showed a narrow Gaussian distribution centered in 0 g/dL; a root cause analysis identified intercurrent and/or unpredictable events associated with hospitalization, blood transfusion, and laboratory error or misreported hemoglobin values as the main reasons for large discrepancy between predicted versus observed hemoglobin values. Our ANN predictive model offers a simple and reliable tool applicable in daily clinical practice for predicting the long-term response to ESA/iron therapy of HD patients.

Computational intelligence for the Balanced Scorecard: studying performance trends of hemodialysis clinics

OBJECTIVES

The Balanced Scorecard (BSC) is a general, widely employed instrument for enterprise performance monitoring based on the periodic assessment of strategic Key Performance Indicators that are scored against preset targets. The BSC is currently employed as an effective management support tool within Fresenius Medical Care (FME) and is routinely analyzed via standard statistical methods. More recently, the application of computational intelligence techniques (namely, self-organizing maps) to BSC data has been proposed as a way to enhance the quantity and quality of information that can be extracted from it. In this work, additional methods are presented to analyze the evolution of clinic performance over time.

METHODS

Performance evolution is studied at the single-clinic level by computing two complementary indexes that measure the proportion of time spent within performance clusters and improving/worsening trends. Self-organizing maps are used in conjunction with these indexes to identify the specific drivers of the observed performance. The performance evolution for groups of clinics is modeled under a probabilistic framework by resorting to Markov chain properties. These allow a study of the probability of transitioning between performance clusters as time progresses for the identification of the performance level that is expected to become dominant over time.

RESULTS

We show the potential of the proposed methods through illustrative results derived from the analysis of BSC data of 109 FME clinics in three countries. We were able to identify the performance drivers for specific groups of clinics and to distinguish between countries whose performances are likely to improve from those where a decline in performance might be expected. According to the stationary distribution of the Markov chain, the expected trend is best in Turkey (where the highest performance cluster has the highest probability, P=0.46), followed by Portugal (where the second best performance cluster dominates, with P=0.50), and finally Italy (where the second best performance cluster has P=0.34).

CONCLUSION

These results highlight the ability of the proposed methods to extract insights about performance trends that cannot be easily extrapolated using standard analyses and that are valuable in directing management strategies within a continuous quality improvement policy.

An interdomain interaction of the androgen receptor is required for its aggregation and toxicity in spinal and bulbar muscular atrophy

Polyglutamine expansion within the androgen receptor (AR) causes spinal and bulbar muscular atrophy (SBMA) and is associated with misfolded and aggregated species of the mutant AR. We showed previously that nuclear localization of the mutant AR was necessary but not sufficient for SBMA. Here we show that an interdomain interaction of the AR that is central to its function within the nucleus is required for AR aggregation and toxicity. Ligands that prevent the interaction between the amino-terminal FXXLF motif and carboxyl-terminal AF-2 domain (N/C interaction) prevented toxicity and AR aggregation in an SBMA cell model and rescued primary SBMA motor neurons from 5α-dihydrotestosterone-induced toxicity. Moreover, genetic mutation of the FXXLF motif prevented AR aggregation and 5α-dihydrotestosterone toxicity. Finally, selective androgen receptor modulators, which prevent the N/C interaction, ameliorated AR aggregation and toxicity while maintaining AR function, highlighting a novel therapeutic strategy to prevent the SBMA phenotype while retaining AR transcriptional function.

Proteasomal and autophagic degradative activities in spinal and bulbar muscular atrophy

Spinal and bulbar muscular atrophy (SBMA or Kennedy's disease) is a fatal neurodegenerative disease characterized by the selective loss of motor neurons in the bulbar region of the brain and in the anterior horns of the spinal cord. The disease has been associated to an expansion of a CAG triplet repeat present in the first coding exon of the androgen receptor (AR) gene. SBMA was the first identified member of a large class of neurodegenerative diseases now known as CAG-related diseases, which includes Huntington's disease (HD), several types of spinocerebellar ataxia (SCAs), and dentatorubral and pallidoluysian atrophy (DRPLA). The expanded CAG tract is translated to an aberrantly long polyglutamine tract (ARpolyQ) in the N-terminal region of the AR protein. The elongated polyQ tract seems to confer a neurotoxic gain-of-function to the mutant AR, possibly via the generation of aberrant conformations (misfolding). Protein misfolding is thought to be a trigger of neurotoxicity, since it perturbs a wide variety of motor neuronal functions. The first event is the accumulation of the ARpolyQ into ubiquitinated aggregates in a ligand (testosterone) dependent manner. The mutant ARpolyQ also impairs proteasome functions. The autophagic pathway may be activated to compensate these aberrant events by clearing the mutant ARpolyQ from motor neuronal cells. This review illustrates the mechanisms at the basis of ARpolyQ degradation via the proteasomal and autophagic systems.

The role of the polyglutamine tract in androgen receptor

The androgen receptor (AR) is a ligand-activated transcription factor which is responsible for the androgen responsiveness of target cells. Several types of mutations have been found in the AR and linked to endocrine dysfunctions. Surprisingly, the polymorphism involving the CAG triplet repeat expansion of the AR gene, coding for a polyglutamine (PolyGln) tract in the N-terminal transactivation domain of the AR protein, has been involved either in endocrine or neurological disorders. For example, among endocrine-related-diseases, the PolyGln size has been proposed to be associated to prostate cancer susceptibility, hirsutism, male infertility, cryptorchidism (in conjunction with polyglycine stretches polymorphism), etc.; the molecular mechanisms of these alterations are thought to involve a modulation of AR transcriptional competence, which inversely correlates with the PolyGln length. Among neurological alterations, a decreased AR function seems to be also involved in depression. Moreover, when the polymorphic PolyGln becomes longer than 35-40 contiguous glutamines (ARPolyGln), the ARPolyGln acquires neurotoxicity, because of an unknown gain-of-function. This mutation has been linked to a rare inherited X-linked motor neuronal disorder, the Spinal and Bulbar Muscular Atrophy, or Kennedy's disease. The disorder is characterized by death of motor neurons expressing high levels of AR. The degenerating motor neurons are mainly located in the anterior horns of the spinal cord and in the bulbar region; some neurons of the dorsal root ganglia may also be involved. Interestingly, the same type of PolyGln elongation has been found in other totally unrelated proteins responsible for different neurodegenerative diseases. A common feature of all these disorders is the formation of intracellular aggregates containing the mutated proteins; at present, but their role in the disease is largely debated. This review will discuss how the PolyGln neurotoxicity of SBMA AR may be either mediated or decreased by aggregates, and will present data on the dual role played by testosterone on motor neuronal functions and dysfunctions.

Mutation of SOD1 in ALS: a gain of a loss of function

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease caused by motoneuron loss. Some familial cases (fALS) are linked to mutations of superoxide dismutase type-1 (SOD1), an antioxidant enzyme whose activity is preserved in most mutant forms. Owing to the similarities in sporadic and fALS forms, mutant SOD1 animal and cellular models are a useful tool to study the disease. In transgenic mice expressing either wild-type (wt) human SOD1 or mutant G93A-SOD1, we found that wtSOD1 was present in cytoplasm and in nuclei of motoneurons, whereas mutant SOD1 was mainly cytoplasmic. Similar results were obtained in immortalized motoneurons (NSC34 cells) expressing either wtSOD1 or G93A-SOD1. Analyzing the proteasome activity, responsible for misfolded protein clearance, in the two subcellular compartments, we found proteasome impairment only in the cytoplasm. The effect of G93A-SOD1 exclusion from nuclei was then analyzed after oxidative stress. Cells expressing G93A-SOD1 showed a higher DNA damage compared with those expressing wtSOD1, possibly because of a loss of nuclear protection. The toxicity of mutant SOD1 might, therefore, arise from an initial misfolding (gain of function) reducing nuclear protection from the active enzyme (loss of function in the nuclei), a process that may be involved in ALS pathogenesis.