The role of episodic memory sampling in evaluation

Many models of choice assume that people retrieve memories of past experiences and use them to guide evaluation and choice. In this paper, we examine whether samples of recalled past experiences do indeed underpin our evaluations of options. We showed participants sequences of numerical values and asked them to recall as many of those values as possible and also to state how much they would be willing to pay for another draw from the sequence. Using Bayesian mixed effects modeling, we predicted participants' evaluation of the sequences at the group level from either the average of the values they recalled or the average of the values they saw. Contrary to the predictions of recall-based models, people's evaluations appear to be sensitive to information beyond what was actually recalled. Moreover, we did not find consistent evidence that memory for specific items is sufficient to predict evaluation of sequences. We discuss the implications for sampling models of memory and decision-making and alternative explanations.

Outcome harvesting as a methodology for the retrospective evaluation of small-scale community development interventions

PURPOSE

Undertaking evaluations of public health interventions can be problematic, particularly where there are multiple stakeholders and high degrees of complexity. This is especially true with regards to the evaluation of community interventions that can include multiple actors with differing priorities, budgets and resources. Using the example of a place-based community wellbeing project, this paper discusses the practical application of 'Outcome Harvesting' as a methodology for retrospective, responsive and collaborative evaluation in public health research.

RESULTS

Our example shows that Outcome Harvesting can be effectively applied to the evaluation of community development initiatives and is likely to have potential in evaluating public health interventions more generally. Our case study also found it was well suited to retrospective evaluation, although the earlier the work can begin the more beneficial it is in understanding the project and in gathering sufficient data.

CONCLUSION

Outcome Harvesting shows promise for evaluating community development initiatives, allowing evaluators to work closely with key stakeholders to arrive at a mutually agreed and verifiable set of outcomes. However, challenges remain with regards to engaging a broad range of participants and with ensuring adequate time and resources are available to evaluate initiatives effectively.

Capacity Building for Refugee Mental Health in Resettlement: Implementation and Evaluation of Cross-Cultural Trauma-Informed Care Training

Refugee mental health needs are heightened during resettlement but are often neglected due to challenges in service provision, including lack of opportunities for building capacity and partnership among providers. We developed and implemented culturally-responsive refugee mental health training, called Cross-Cultural Trauma-Informed Care (CC-TIC) training. We evaluated CC-TIC, using a free listing and semi-structured retrospective pre- and post-training evaluation with five localities in two states in the U.S. The results showed significant improvement in providers’ knowledge of trauma impacts, cultural expressions of trauma/stress-related symptoms, and culturally-responsive trauma-informed care. Trauma-informed care specific to refugee resettlement was regarded as the most helpful topic and community partnership building as the most requested area for future training. This study emphasizes that culturally-responsive trauma-informed approaches can help bridge gaps between mental health care and resettlement services and promote exchanges of knowledge and expertise to build collaborative care and community partnership.

A multicenter retrospective evaluation of Chronic Myeloid Leukemia (CML) therapy in Austria assessing the impact of early treatment response on patient outcomes in a real-life setting : R-EFECT study

Background

Several clinical trials in chronic phase (CP) chronic myeloid leukemia (CML) showed that early response to tyrosine kinase inhibitor (TKI) treatment results in an improved long-term survival and progression-free survival. This study assessed whether patients achieving early treatment response (ETR; partial cytogenetic response or BCR-ABL1 mRNA ≤10% at 3 months) in daily practice also have a long-term survival benefit.

Methods

The Retrospective Evaluation of Early response in CML for long-term Treatment outcome (R-EFECT), a multicenter, retrospective chart review, documented patients with newly diagnosed CML-CP starting first-line TKI therapy in routine clinical practice. The primary aim was to assess the 5‑year overall survival rate.

Results

Of the 211 patients from 12 centers across Austria (January 2004–May 2010), 176 (median age, 56 years) were included in the analysis. All patients received first-line therapy with imatinib. Overall, 136 patients (77.3%) achieved ETR (ETR+ group), whereas 40 (22.7%) did not reach ETR (ETR− group). The ETR+ group had higher 5‑year overall survival (92.5% vs. 77.5%, P = 0.018) and progression-free survival (95.6% vs. 87.5%, P = 0.06) rates compared with the ETR− group. As expected, more patients in the ETR− group were switched to another TKI. At the last contact, 120 patients were still on imatinib and 44 had switched to another TKI (25 to nilotinib, 15 to dasatinib, and 4 to bosutinib).

Conclusion

The data are in line with randomized trials demonstrating that ETR is associated with improved survival and thus confirmed these results in patients treated in daily clinical routine.

Industry-scale application and evaluation of deep learning for drug target prediction

Artificial intelligence (AI) is undergoing a revolution thanks to the breakthroughs of machine learning algorithms in computer vision, speech recognition, natural language processing and generative modelling. Recent works on publicly available pharmaceutical data showed that AI methods are highly promising for Drug Target prediction. However, the quality of public data might be different than that of industry data due to different labs reporting measurements, different measurement techniques, fewer samples and less diverse and specialized assays. As part of a European funded project (ExCAPE), that brought together expertise from pharmaceutical industry, machine learning, and high-performance computing, we investigated how well machine learning models obtained from public data can be transferred to internal pharmaceutical industry data. Our results show that machine learning models trained on public data can indeed maintain their predictive power to a large degree when applied to industry data. Moreover, we observed that deep learning derived machine learning models outperformed comparable models, which were trained by other machine learning algorithms, when applied to internal pharmaceutical company datasets. To our knowledge, this is the first large-scale study evaluating the potential of machine learning and especially deep learning directly at the level of industry-scale settings and moreover investigating the transferability of publicly learned target prediction models towards industrial bioactivity prediction pipelines.

Necessity for retrospective evaluation of past-positive chemicals in in vitro chromosomal aberration tests using recommended cytotoxicity indices

We have demonstrated that retrospective evaluation of existing data of in vitro chromosomal aberration test using the new cytotoxicity indices RICC (relative increase in cell count) or RPD (relative population doubling) reduces the false-positive rate. We have constructed an algorithm to predict the likelihood that past-positive results would differ when retested accordingly. Here, we emphasize the importance of reviewing existing in vitro chromosomal aberration test results. The present Letter not only supports the rediscovery of potentially useful chemicals excluded from further development as a result of misclassification due to in vitro false-positive results, but also contributes to the development of a precise Quantitative Structure-Activity Relationship (QSAR) model by providing an appropriate training data-set. Furthermore, re-evaluation is expected to provide novel insights into underlying mechanisms and/or key structures involved in the development of chromosomal aberrations.

Altered time-perception performance in individuals with high schizotypy levels

The possibility of altered time-perception in high schizotypy scorers, as postulated through previous differences shown in performance between high and low scorers in schizotypy on schedules of reinforcement with temporal elements, was examined using a series of retrospective timing tasks. Three stimuli ratio manipulations were made across two experiments, and, using an adjusted version of the bisection-point method for data analysis, results showed that high scorers on the Unusual Experiences subscale of the O-LIFE(B) estimated the mid point of the stimulus range to be at a significantly longer interval than low scorers. This was true when the ratios between "short" and "long" standard stimuli were 4:1 (Experiment 1), 3:1 and 2:1 (Experiment 2). These findings are consistent with the notion of altered time-perception for high schizotypals.