Needs and feasibility of living systematic reviews (LSRs): Experience from LSRs on COVID-19 vaccine effectiveness

During 2021 and 2023, a team of researchers at the Robert Koch Institute (RKI) and partnering institutions conducted two living systematic reviews (LSRs) on the effectiveness of COVID-19 vaccines in different age groups to inform recommendations of the Standing Committee on Vaccination in Germany (Ständige Impfkommission, STIKO). Based on our experience from the realization of these LSRs, we developed certain criteria to assess the needs and feasibility of conducting LSRs. Combining these with previously established criteria, we developed the following set to inform future planning of LSRs for STIKO: Needs criterion (N)1: Relevance of the research question, N2: Certainty of evidence (CoE) at baseline; N3: Expected need for Population-Intervention-Comparator-Outcome (PICO) adaptations; N4: Expected new evidence over time; N5: Expected impact of new evidence on CoE; Feasibility criterion (F)1: Availability of sufficient human resources; F2: Feasibility of timely dissemination of the results to inform decision-making. For each criterion we suggest rating options which may support the decision to conduct an LSR or other forms of evidence synthesis when following the provided flowchart. The suggested criteria were developed on the basis of the experiences from exemplary reviews in a specific research field (i.e., COVID-19 vaccination), and did not follow a formal development or validation process. However, these criteria might also be useful to assess whether questions from other research fields can and should be answered using the LSR approach, or assist in determining whether the use of an LSR is sensible and feasible for specific questions in health policy and practice.

Audiovisual simultaneity windows reflect temporal sensory uncertainty

The ability to judge the temporal alignment of visual and auditory information is a prerequisite for multisensory integration and segregation. However, each temporal measurement is subject to error. Thus, when judging whether a visual and auditory stimulus were presented simultaneously, observers must rely on a subjective decision boundary to distinguish between measurement error and truly misaligned audiovisual signals. Here, we tested whether these decision boundaries are relaxed with increasing temporal sensory uncertainty, i.e., whether participants make the same type of adjustment an ideal observer would make. Participants judged the simultaneity of audiovisual stimulus pairs with varying temporal offset, while being immersed in different virtual environments. To obtain estimates of participants' temporal sensory uncertainty and simultaneity criteria in each environment, an independent-channels model was fitted to their simultaneity judgments. In two experiments, participants' simultaneity decision boundaries were predicted by their temporal uncertainty, which varied unsystematically with the environment. Hence, observers used a flexibly updated estimate of their own audiovisual temporal uncertainty to establish subjective criteria of simultaneity. This finding implies that, under typical circumstances, audiovisual simultaneity windows reflect an observer's cross-modal temporal uncertainty.

Evidence for cortical adjustments to perceptual decision criteria during word recognition in noise

Extensive increases in cingulo-opercular frontal activity are typically observed during speech recognition in noise tasks. This elevated activity has been linked to a word recognition benefit on the next trial, termed "adaptive control," but how this effect might be implemented has been unclear. The established link between perceptual decision making and cingulo-opercular function may provide an explanation for how those regions benefit subsequent word recognition. In this case, processes that support recognition such as raising or lowering the decision criteria for more accurate or faster recognition may be adjusted to optimize performance on the next trial. The current neuroimaging study tested the hypothesis that pre-stimulus cingulo-opercular activity reflects criterion adjustments that determine how much information to collect for word recognition on subsequent trials. Participants included middle-age and older adults (N = 30; age = 58.3 ± 8.8 years; m ± sd) with normal hearing or mild sensorineural hearing loss. During a sparse fMRI experiment, words were presented in multitalker babble at +3 dB or +10 dB signal-to-noise ratio (SNR), which participants were instructed to repeat aloud. Word recognition was significantly poorer with increasing participant age and lower SNR compared to higher SNR conditions. A perceptual decision-making model was used to characterize processing differences based on task response latency distributions. The model showed that significantly less sensory evidence was collected (i.e., lower criteria) for lower compared to higher SNR trials. Replicating earlier observations, pre-stimulus cingulo-opercular activity was significantly predictive of correct recognition on a subsequent trial. Individual differences showed that participants with higher criteria also benefitted the most from pre-stimulus activity. Moreover, trial-level criteria changes were significantly linked to higher versus lower pre-stimulus activity. These results suggest cingulo-opercular cortex contributes to criteria adjustments to optimize speech recognition task performance.

An Innovative Approach to Modelling the Optimal Treatment Sequence for Patients with Relapsing-Remitting Multiple Sclerosis: Implementation, Validation, and Impact of the Decision-Making Approach

INTRODUCTION

An innovative computational model was developed to address challenges regarding the evaluation of treatment sequences in patients with relapsing-remitting multiple sclerosis (RRMS) through the concept of a 'virtual' physician who observes and assesses patients over time. We describe the implementation and validation of the model, then apply this framework as a case study to determine the impact of different decision-making approaches on the optimal sequence of disease-modifying therapies (DMTs) and associated outcomes.

METHODS

A patient-level discrete event simulation (DES) was used to model heterogeneity in disease trajectories and outcomes. The evaluation of DMT options was implemented through a Markov model representing the patient's disease; outcomes included lifetime costs and quality of life. The DES and Markov models underwent internal and external validation. Analyses of the optimal treatment sequence for each patient were based on several decision-making criteria. These treatment sequences were compared to current treatment guidelines.

RESULTS

Internal validation indicated that model outcomes for natural history were consistent with the input parameters used to inform the model. Costs and quality of life outcomes were successfully validated against published reference models. Whereas each decision-making criterion generated a different optimal treatment sequence, cladribine tablets were the only DMT common to all treatment sequences. By choosing treatments on the basis of minimising disease progression or number of relapses, it was possible to improve on current treatment guidelines; however, these treatment sequences were more costly. Maximising cost-effectiveness resulted in the lowest costs but was also associated with the worst outcomes.

CONCLUSIONS

The model was robust in generating outcomes consistent with published models and studies. It was also able to identify optimal treatment sequences based on different decision criteria. This innovative modelling framework has the potential to simulate individual patient trajectories in the current treatment landscape and may be useful for treatment switching and treatment positioning decisions in RRMS.

The imputation of structure in random patterns as a cognitive factor predicting adult picky eating behavior

We raise the hypothesis that adult picky eating behavior may be related to subjects' tendency to impute structure in random stimuli. We test said hypothesis in an empirical study which operationalizes subjects' objective abilities and subjective decision criteria in discriminating between correlated and independent signals in terms of the classic Signal Detection Theory. A robust regularized Bayesian multiple regression analysis indicates that liberal subjective decision criteria which indicate a tendency to over-impute structure when trying to discriminate between random and correlated patterns are associated with elevated scores of picky eating while controlling for several other variables. The predictive power of our discriminating procedure appears comparable to that of related eating dispositions. Our results strongly suggest that cognitive factors might indeed have substantial and meaningful relationships with certain eating behaviors.

When complexity matters: a step-by-step guide to incorporating a complexity perspective in guideline development for public health and health system interventions

BACKGROUND

Guidelines on public health and health system interventions often involve considerations beyond effectiveness and safety to account for the impact that these interventions have on the wider systems in which they are implemented. This paper describes how a complexity perspective may be adopted in guideline development to facilitate a more nuanced consideration of a range of factors pertinent to decisions regarding public health and health system interventions. These factors include acceptability and feasibility, and societal, economic, and equity and equality implications of interventions.

MAIN MESSAGE

A 5-step process describes how to incorporate a complexity perspective in guideline development with examples to illustrate each step. The steps include: (i) guideline scoping, (ii) formulating questions, (iii) retrieving and synthesising evidence, (iv) assessing the evidence, and (v) developing recommendations. Guideline scoping using stakeholder consultations, complexity features, evidence mapping, logic modelling, and explicit decision criteria is emphasised as a key step that informs all subsequent steps.

CONCLUSIONS

Through explicit consideration of a range of factors and enhanced understanding of the specific circumstances in which interventions work, a complexity perspective can yield guidelines with better informed recommendations and facilitate local adaptation and implementation. Further work will need to look into the methods of collecting and assessing different types of evidence beyond effectiveness and develop procedural guidance for prioritising across a range of decision criteria.

Choice of first line systemic treatment in pancreatic cancer among national experts

BACKGROUND

Treatment options for patients with metastatic pancreatic cancer depend on various factors, including performance status, tumor burden and patient preferences. Metastatic pancreatic cancer is incurable and many systemic treatment options have been investigated over the past decades. This analysis of patterns of practice was performed to identify decision criteria and their impact on the choice of first-line management of metastatic pancreatic cancer.

MATERIALS AND METHODS

Members of the Swiss Group for Clinical Cancer Research (SAKK) Gastrointestinal Cancer Group were contacted and agreed to participate in this analysis. Decision trees for the first line treatment of metastatic pancreatic cancer from 9 centers in Switzerland were collected and analyzed based on the objective consensus methodology to identify consensus and discrepancies in clinical decision-making.

RESULTS

The final treatment algorithms included 3 decision criteria (comorbidities, performance status and age) and 5 treatment options: FOLFIRINOX, FOLFOX, gemcitabine + nab-paclitaxel, gemcitabine mono and best supportive care.

CONCLUSION

We identified multiple decision criteria relevant to all participating centers. We found consensus for the treatment of young (age below 65) patients with good performance status with FOLFIRINOX. For patients with increasing age and reducing performance status there was a decreasing trend to use gemcitabine + nab-paclitaxel. Gemcitabine monotherapy was typically offered to patients in the presence of comorbidities. For patients with ECOG 3-4, most of the experts recommended BSC.

Emotions as Social Information in Shared Decision-Making in Oncology

Emotions play an important role in decision-making and they can impact individual as well as shared decisions. With increasing complexity of the decision, the potential for emotions to influence the outcome increases. Emotions are thus an influential factor in oncological decision-making which is a complex and high-stakes situation. As the shared decision-making process is at the center of patient-centric decisions, we model emotions as social information that inform the shared decision-making process. We present and explain a range of emotional concepts, together with a specific clinical example, that can impact the shared decision-making process. Our process model shows that emotions are experienced in various combinations before, during, and after a shared decision is made and how patients' and physicians' emotions interact and spill over during a shared decision situation. Overall, our process model and specific example show how emotions can impact shared decision-making in oncology in a multitude of ways. With this paper, we want to raise awareness of the role of emotions in the shared decision-making process, as emotions are often not explicitly recognized as decision criteria. Increased awareness of emotions may help their optimal utilization and reduce their influence as a bias in shared decision-making.

Radiotherapy for pelvic nodal recurrences after radical prostatectomy: patient selection in clinical practice

Aim

There is no general consensus on the optimal treatment for prostate cancer (PC) patients with intrapelvic nodal oligorecurrences after radical prostatectomy. Besides androgen deprivation therapy (ADT) as standard of care, both elective nodal radiotherapy (ENRT) and stereotactic body radiotherapy (SBRT) as well as salvage lymph node dissection (sLND) are common treatment options. The aim of our study was to assess decision making and practice patterns for salvage radiotherapy (RT) in this setting.

Methods

Treatment recommendations from 14 Swiss radiation oncology centers were collected and converted into decision trees. An iterative process using the objective consensus methodology was applied to assess differences and consensus.

Results

PSMA PET/CT was recommended by 93% of the centers as restaging modality. For unfit patients defined by age, comorbidities or low performance status, androgen deprivation therapy (ADT) alone was recommended by more than 70%. For fit patients with unfavorable tumor characteristics such as short prostate-specific antigen (PSA) doubling time or initial high-risk disease, the majority of the centers (57–71%) recommended ENRT + ADT for 1–4 lesions. For fit patients with favorable tumor characteristics, there were low levels of consensus and a wide variety of recommendations. For 1–4 nodal lesions, focal SBRT was offered by 64% of the centers, most commonly as a 5-fraction course.

Conclusions

As an alternative to ADT, ENRT or SBRT for pelvic nodal oligorecurrences of PC are commonly offered to selected patients, with large treatment variations between centers. The exact number of lymph nodes had a major impact on treatment selection.

Agency alters perceptual decisions about action-outcomes

Humans experience themselves as agents, capable of controlling their actions and the outcomes they generate (i.e., the sense of agency). Inferences of agency are not infallible. Research shows that we often attribute outcomes to our agency even though they are caused by another agent. Moreover, agents report the sensory events they generate to be less intense compared to the events that are generated externally. These effects have been assessed using highly suprathreshold stimuli and subjective measurements. Consequently, it remains unclear whether experiencing oneself as an agent lead to a decision criterion change and/or a sensitivity change. Here, we investigate this issue. Participants were told that their key presses generated an upward dot motion but that on 30 % of the trials the computer would take over and display a downward motion. The upward/downward dot motion was presented at participant's discrimination threshold. Participants were asked to indicate whether they (upward motion) or the computer (downward motion) generated the motion. This group of participants was compared with a 'no-agency' group who performed the same task except that subjects did not execute any actions to generate the dot motion. We observed that the agency group reported seeing more frequently the motion they expected to generate (i.e., upward motion) than the no-agency group. This suggests that agency distorts our experience of (allegedly) caused events by altering perceptual decision processes, so that, in ambiguous contexts, externally generated events are experienced as the outcomes of one's actions.

Algorithms in the First-Line Treatment of Metastatic Clear Cell Renal Cell Carcinoma--Analysis Using Diagnostic Nodes

BACKGROUND

With the advent of targeted therapies, many treatment options in the first-line setting of metastatic clear cell renal cell carcinoma (mccRCC) have emerged. Guidelines and randomized trial reports usually do not elucidate the decision criteria for the different treatment options. In order to extract the decision criteria for the optimal therapy for patients, we performed an analysis of treatment algorithms from experts in the field.

MATERIALS AND METHODS

Treatment algorithms for the treatment of mccRCC from experts of 11 institutions were obtained, and decision trees were deduced. Treatment options were identified and a list of unified decision criteria determined. The final decision trees were analyzed with a methodology based on diagnostic nodes, which allows for an automated cross-comparison of decision trees. The most common treatment recommendations were determined, and areas of discordance were identified.

RESULTS

The analysis revealed heterogeneity in most clinical scenarios. The recommendations selected for first-line treatment of mccRCC included sunitinib, pazopanib, temsirolimus, interferon-α combined with bevacizumab, high-dose interleukin-2, sorafenib, axitinib, everolimus, and best supportive care. The criteria relevant for treatment decisions were performance status, Memorial Sloan Kettering Cancer Center risk group, only or mainly lung metastases, cardiac insufficiency, hepatic insufficiency, age, and "zugzwang" (composite of multiple, related criteria).

CONCLUSION

In the present study, we used diagnostic nodes to compare treatment algorithms in the first-line treatment of mccRCC. The results illustrate the heterogeneity of the decision criteria and treatment strategies for mccRCC and how available data are interpreted and implemented differently among experts.

IMPLICATIONS FOR PRACTICE

The data provided in the present report should not be considered to serve as treatment recommendations for the management of treatment-naïve patients with multiple metastases from metastatic clear cell renal cell carcinoma outside a clinical trial; however, the data highlight the different treatment options and the criteria used to select them. The diversity in decision making and how results from phase III trials can be interpreted and implemented differently in daily practice are demonstrated.

Assessment of three approaches for regulatory decision making on pesticides with endocrine disrupting properties

Recent EU legislation has introduced endocrine disrupting properties as a hazard-based "cut-off" criterion for the approval of active substances as pesticides and biocides. Currently, no specific science-based approach for the assessment of substances with endocrine disrupting properties has been agreed upon, although this new legislation provides interim criteria based on classification and labelling. Different proposals for decision making on potential endocrine disrupting properties in human health risk assessment have been developed by the German Federal Institute for Risk Assessment (BfR) and other regulatory bodies. All these frameworks, although differing with regard to hazard characterisation, include a toxicological assessment of adversity of the effects, the evaluation of underlying modes/mechanisms of action in animals and considerations concerning the relevance of effects to humans. Three options for regulatory decision making were tested upon 39 pesticides for their applicability and to analyze their potential impact on the regulatory status of active substances that are currently approved for use in Europe: Option 1, based purely on hazard identification (adversity, mode of action, and the plausibility that both are related); Option 2, based on hazard identification and additional elements of hazard characterisation (severity and potency); Option 3, based on the interim criteria laid down in the recent EU pesticides legislation. Additionally, the data analysed in this study were used to address the questions, which parts of the endocrine system were affected, which studies were the most sensitive and whether no observed adverse effect levels were observed for substance with ED properties. The results of this exercise represent preliminary categorisations and must not be used as a basis for definitive regulatory decisions. They demonstrate that a combination of criteria for hazard identification with additional criteria of hazard characterisation allows prioritising and differentiating between substances with regard to their regulatory concern. It is proposed to integrate these elements into a decision matrix to be used within a weight of evidence approach for the toxicological categorisation of relevant endocrine disruptors and to consider all parts of the endocrine system for regulatory decision making on endocrine disruption.

Management of contaminated marine marketable resources after oil and HNS spills in Europe

Different risk evaluation approaches have been used to face oil and hazardous and noxious substances (HNS) spills all over the world. To minimize health risks and mitigate economic losses due to a long term ban on the sale of sea products after a spill, it is essential to preemptively set risk evaluation criteria and standard methodologies based on previous experience and appropriate scientifically sound criteria. Standard methodologies are analyzed and proposed in order to improve the definition of criteria for reintegrating previously contaminated marine marketable resources into the commercialization chain in Europe. The criteria used in former spills for the closing of and lifting of bans on fisheries and harvesting are analyzed. European legislation was identified regarding food sampling, food chemical analysis and maximum levels of contaminants allowed in seafood, which ought to be incorporated in the standard methodologies for the evaluation of the decision criteria defined for oil and HNS spills in Europe. A decision flowchart is proposed that opens the current decision criteria to new material that may be incorporated in the decision process. Decision criteria are discussed and compared among countries and incidents. An a priori definition of risk criteria and an elaboration of action plans are proposed to speed up actions that will lead to prompt final decisions. These decisions, based on the best available scientific data and conducing to lift or ban economic activity, will tend to be better understood and respected by citizens.

New decision criteria for selecting delta check methods based on the ratio of the delta difference to the width of the reference range can be generally applicable for each clinical chemistry test item

BACKGROUND

Many laboratories use 4 delta check methods: delta difference, delta percent change, rate difference, and rate percent change. However, guidelines regarding decision criteria for selecting delta check methods have not yet been provided. We present new decision criteria for selecting delta check methods for each clinical chemistry test item.

METHODS

We collected 811,920 and 669,750 paired (present and previous) test results for 27 clinical chemistry test items from inpatients and outpatients, respectively. We devised new decision criteria for the selection of delta check methods based on the ratio of the delta difference to the width of the reference range (DD/RR). Delta check methods based on these criteria were compared with those based on the CV% of the absolute delta difference (ADD) as well as those reported in 2 previous studies.

RESULTS

The delta check methods suggested by new decision criteria based on the DD/RR ratio corresponded well with those based on the CV% of the ADD except for only 2 items each in inpatients and outpatients. Delta check methods based on the DD/RR ratio also corresponded with those suggested in the 2 previous studies, except for 1 and 7 items in inpatients and outpatients, respectively.

CONCLUSIONS

The DD/RR method appears to yield more feasible and intuitive selection criteria and can easily explain changes in the results by reflecting both the biological variation of the test item and the clinical characteristics of patients in each laboratory. We suggest this as a measure to determine delta check methods.