Cognitive Processes and the Assessment of Subjective Probability Distributions

Multivariate associations between dopamine receptor availability and risky investment decision-making across adulthood

Enhancing dopamine increases financial risk taking across adulthood but it is unclear whether baseline individual differences in dopamine function are related to risky financial decisions. Here, thirty-five healthy adults completed an incentive-compatible risky investment decision task and a PET scan at rest using [11C]FLB457 to assess dopamine D2-like receptor availability. Participants made choices between a safe asset (bond) and a risky asset (stock) with either an expected value less than the bond ("bad stock") or expected value greater than the bond ("good stock"). Five measures of behavior (choice inflexibility, risk seeking, suboptimal investment) and beliefs (absolute error, optimism) were computed and D2-like binding potential was extracted from four brain regions of interest (midbrain, amygdala, anterior cingulate, insula). We used canonical correlation analysis to evaluate multivariate associations between decision-making and dopamine function controlling for age. Decomposition of the first dimension (r = 0.76) revealed that the strongest associations were between measures of choice inflexibility, incorrect choice, optimism, amygdala binding potential, and age. Follow-up univariate analyses revealed that amygdala binding potential and age were both independently associated with choice inflexibility. The findings suggest that individual differences in dopamine function may be associated with financial risk taking in healthy adults.

Aggregating predictions from experts: a review of statistical methods, experiments, and applications

Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse or rapidly changing, statistical models may not be able to make accurate predictions. Expert judgmental forecasts-models that combine expert-generated predictions into a single forecast-can make predictions when training data is limited by relying on human intuition. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This review surveyed recent literature on aggregating expert-elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace.

Return-to-Player Percentage in Gaming Machines: Impact of Informative Materials on Player Understanding

The addictive potential of electronic gaming machines (EGMs) is currently explained within a cognitive-behavioural framework. This framework explains that various erroneous cognitions regarding players' likelihood of winning contribute to persistent EGM gambling behaviour. Related to these cognitions is the pervasive misunderstanding among players regarding the operation of EGMs. However, little research has focussed specifically on player understanding of the theoretical proportion returned to players over the lifetime of a machine; return to player percentage. This study aimed to investigate the extent to which players understand the concept return to player percentage presented in different educative formats. A sample of 112 university students were randomly allocated to one of four conditions pertaining to a different mode of information delivery; infographic, vignette, brochure, or mandated legislation (control). Participants completed post-intervention measures to determine changes in knowledge. As predicted, participants exhibited a lack of understanding of the concept of return to player at baseline. However, contrary to predictions, exposure to any of the experimental conditions did not result in a greater understanding of return to player than controls. The study findings emphasise the difficulty individuals have in understanding complex concepts related to return to player percentages when presented in current formats and content. Treatment and responsible gambling policies need to adopt strategies to effectively improve knowledge of this aspect of the structural characteristics of gaming machines.

Method to Obtain a Vector of Hyperparameters: Application in Bernoulli Trials

The main difficulties when using the Bayesian approach are obtaining information from the specialist and obtaining hyperparameters values of the assumed probability distribution as representative of knowledge  external to the  data. In addition to the  fact  that  a large  part  of the  literature on this subject is characterized by considering prior conjugated distributions for the parameter of interest. An method is proposed  to find the hyperparameters of a nonconjugated prior  distribution. The following  scenarios were considered for Bernoulli trials: four prior distributions (Beta, Kumaraswamy, Truncated Gamma   and   Truncated  Weibull) and four scenarios  for  the  generating process. Two necessary,  but not sufficient  conditions were  identified to ensure   the  existence of  a  vector of  values for  the  hyperparameter. The Truncated Weibull prior distribution performed the worst.  The methodology was  used  to estimate the  prevalence of two  transmitted sexually infections in an Colombian  indigenous community.

An Examination of Surprise and Emotions in the Processing of Anecdotal Evidence

Low-probability and atypical cases are commonly used as anecdotal evidence to support arguments that are divergent from general medical knowledge, thus challenging current efforts of health promotion and education. The present study proposes an emotion-based model that centers on surprise to explain the effects of critical anecdotal evidence information on risk perception, need for uncertainty reduction, and information searching behavior. Using experimental design and two-group path structural equation models, the study results supported a key role of surprise followed by two development routes that led to a series of attitudinal and behavioral changes after exposure to anecdotal evidence. First, surprise caused a group of correlated negative emotions (sadness, fear, and anger). Second, negative emotions were part of the intermediate stage that subsequently resulted in risk perception and need for uncertainty reduction prior to information seeking behaviors. The research provides a model to explain and predict the emotional, cognitive, and behavioral outcomes associated with anecdotal evidence.

Development of a practical approach to expert elicitation for randomised controlled trials with missing health outcomes: Application to the IMPROVE trial

Background/aims:

The analyses of randomised controlled trials with missing data typically assume that, after conditioning on the observed data, the probability of missing data does not depend on the patient’s outcome, and so the data are ‘missing at random’ . This assumption is usually implausible, for example, because patients in relatively poor health may be more likely to drop out. Methodological guidelines recommend that trials require sensitivity analysis, which is best informed by elicited expert opinion, to assess whether conclusions are robust to alternative assumptions about the missing data. A major barrier to implementing these methods in practice is the lack of relevant practical tools for eliciting expert opinion. We develop a new practical tool for eliciting expert opinion and demonstrate its use for randomised controlled trials with missing data.

Methods:

We develop and illustrate our approach for eliciting expert opinion with the IMPROVE trial (ISRCTN 48334791), an ongoing multi-centre randomised controlled trial which compares an emergency endovascular strategy versus open repair for patients with ruptured abdominal aortic aneurysm. In the IMPROVE trial at 3 months post-randomisation, 21% of surviving patients did not complete health-related quality of life questionnaires (assessed by EQ-5D-3L). We address this problem by developing a web-based tool that provides a practical approach for eliciting expert opinion about quality of life differences between patients with missing versus complete data. We show how this expert opinion can define informative priors within a fully Bayesian framework to perform sensitivity analyses that allow the missing data to depend upon unobserved patient characteristics.

Results:

A total of 26 experts, of 46 asked to participate, completed the elicitation exercise. The elicited quality of life scores were lower on average for the patients with missing versus complete data, but there was considerable uncertainty in these elicited values. The missing at random analysis found that patients randomised to the emergency endovascular strategy versus open repair had higher average (95% credible interval) quality of life scores of 0.062 (−0.005 to 0.130). Our sensitivity analysis that used the elicited expert information as pooled priors found that the gain in average quality of life for the emergency endovascular strategy versus open repair was 0.076 (−0.054 to 0.198).

Conclusion:

We provide and exemplify a practical tool for eliciting the expert opinion required by recommended approaches to the sensitivity analyses of randomised controlled trials. We show how this approach allows the trial analysis to fully recognise the uncertainty that arises from making alternative, plausible assumptions about the reasons for missing data. This tool can be widely used in the design, analysis and interpretation of future trials, and to facilitate this, materials are available for download.

Bayesian survival analysis in clinical trials: What methods are used in practice?

Background

Bayesian statistics are an appealing alternative to the traditional frequentist approach to designing, analysing, and reporting of clinical trials, especially in rare diseases. Time-to-event endpoints are widely used in many medical fields. There are additional complexities to designing Bayesian survival trials which arise from the need to specify a model for the survival distribution. The objective of this article was to critically review the use and reporting of Bayesian methods in survival trials.

Methods

A systematic review of clinical trials using Bayesian survival analyses was performed through PubMed and Web of Science databases. This was complemented by a full text search of the online repositories of pre-selected journals. Cost-effectiveness, dose-finding studies, meta-analyses, and methodological papers using clinical trials were excluded.

Results

In total, 28 articles met the inclusion criteria, 25 were original reports of clinical trials and 3 were re-analyses of a clinical trial. Most trials were in oncology (n = 25), were randomised controlled (n = 21) phase III trials (n = 13), and half considered a rare disease (n = 13). Bayesian approaches were used for monitoring in 14 trials and for the final analysis only in 14 trials. In the latter case, Bayesian survival analyses were used for the primary analysis in four cases, for the secondary analysis in seven cases, and for the trial re-analysis in three cases. Overall, 12 articles reported fitting Bayesian regression models (semi-parametric, n = 3; parametric, n = 9). Prior distributions were often incompletely reported: 20 articles did not define the prior distribution used for the parameter of interest. Over half of the trials used only non-informative priors for monitoring and the final analysis (n = 12) when it was specified. Indeed, no articles fitting Bayesian regression models placed informative priors on the parameter of interest. The prior for the treatment effect was based on historical data in only four trials. Decision rules were pre-defined in eight cases when trials used Bayesian monitoring, and in only one case when trials adopted a Bayesian approach to the final analysis.

Conclusion

Few trials implemented a Bayesian survival analysis and few incorporated external data into priors. There is scope to improve the quality of reporting of Bayesian methods in survival trials. Extension of the Consolidated Standards of Reporting Trials statement for reporting Bayesian clinical trials is recommended.

Changing the Paradigm for the Treatment and Development of New Therapies for FSGS

Focal segmental glomerulosclerosis (FSGS) is a renal pathology finding that represents a constellation of rare kidney diseases, which manifest as proteinuria, edema nephrotic syndrome, hypertension, and increased risk for kidney failure. Therapeutic options for FSGS are reviewed displaying the expected efficacy from 25 to 69% depending on specific therapy, patient characteristics, cost, and common side effects. This variability in treatment response is likely caused, in part, by the heterogeneity in the etiology and active molecular mechanisms of FSGS. Clinical trials in FSGS have been scant in number and slow to recruit, which may stem, in part, from reliance on classic clinical trial design paradigms. Traditional clinical trial designs based on the "learn and confirm" paradigm may not be appropriate for rare diseases, such as FSGS. Future drug development and testing will require novel approaches to trial designs that have the capacity to enrich study populations and adapt the trial in a planned way to gain efficiencies in trial completion timelines. A clinical trial simulation is provided that compares a classical and more modern design to determine the maximum tolerated dose in FSGS.

Eliciting prior distributions for extra parameters in some generalized linear models

To elicit an informative prior distribution for a normal linear model or a gamma generalized linear model (GLM), expert opinion must be quantified about both the regression coefficients and the extra parameters of these models. The latter task has attracted comparatively little attention. In this article, we introduce two elicitation methods that aim to complete the prior structure of the normal and gamma GLMs. First, we develop a method of assessing a conjugate prior distribution for the error variance in normal linear models. The method quantifies an expert's opinions through assessments of a median and conditional medians. Second, we propose a novel method for eliciting a lognormal prior distribution for the scale parameter of gamma GLMs. Given the mean value of a gamma distributed response variable, the method is based on conditional quartile assessments. It can also be used to quantify an expert's opinion about the prior distribution for the shape parameter of any gamma random variable, if the mean of the distribution has been elicited or is assumed to be known. In the context of GLMs, the mean value is determined by the regression coefficients. Interactive graphics is the medium through which assessments for the two proposed methods are elicited. Examples illustrating use of the methods are given. Computer programs that implement both methods are available.

Bayesian methods for the design and interpretation of clinical trials in very rare diseases

This paper considers the design and interpretation of clinical trials comparing treatments for conditions so rare that worldwide recruitment efforts are likely to yield total sample sizes of 50 or fewer, even when patients are recruited over several years. For such studies, the sample size needed to meet a conventional frequentist power requirement is clearly infeasible. Rather, the expectation of any such trial has to be limited to the generation of an improved understanding of treatment options. We propose a Bayesian approach for the conduct of rare-disease trials comparing an experimental treatment with a control where patient responses are classified as a success or failure. A systematic elicitation from clinicians of their beliefs concerning treatment efficacy is used to establish Bayesian priors for unknown model parameters. The process of determining the prior is described, including the possibility of formally considering results from related trials. As sample sizes are small, it is possible to compute all possible posterior distributions of the two success rates. A number of allocation ratios between the two treatment groups can be considered with a view to maximising the prior probability that the trial concludes recommending the new treatment when in fact it is non-inferior to control. Consideration of the extent to which opinion can be changed, even by data from the best feasible design, can help to determine whether such a trial is worthwhile.

Prior elicitation and Bayesian analysis of the Steroids for Corneal Ulcers Trial

PURPOSE

To elicit expert opinion on the use of adjunctive corticosteroid therapy in bacterial corneal ulcers. To perform a Bayesian analysis of the Steroids for Corneal Ulcers Trial (SCUT), using expert opinion as a prior probability.

METHODS

The SCUT was a placebo-controlled trial assessing visual outcomes in patients receiving topical corticosteroids or placebo as adjunctive therapy for bacterial keratitis. Questionnaires were conducted at scientific meetings in India and North America to gauge expert consensus on the perceived benefit of corticosteroids as adjunct treatment. Bayesian analysis, using the questionnaire data as a prior probability and the primary outcome of SCUT as a likelihood, was performed. For comparison, an additional Bayesian analysis was performed using the results of the SCUT pilot study as a prior distribution.

RESULTS

Indian respondents believed there to be a 1.21 Snellen line improvement, and North American respondents believed there to be a 1.24 line improvement with corticosteroid therapy. The SCUT primary outcome found a non-significant 0.09 Snellen line benefit with corticosteroid treatment. The results of the Bayesian analysis estimated a slightly greater benefit than did the SCUT primary analysis (0.19 lines verses 0.09 lines).

CONCLUSION

Indian and North American experts had similar expectations on the effectiveness of corticosteroids in bacterial corneal ulcers; that corticosteroids would markedly improve visual outcomes. Bayesian analysis produced results very similar to those produced by the SCUT primary analysis. The similarity in result is likely due to the large sample size of SCUT and helps validate the results of SCUT.

Quantifying experts' uncertainty about the future cost of exotic diseases

Since the foot-and-mouth disease outbreak of 2001 in the United Kingdom, there has been debate about the sharing, between government and industry, both the costs of livestock disease outbreaks and responsibility for the decisions that give rise to them. As part of a consultation into the formation of a new body to manage livestock diseases, government veterinarians and economists produced estimates of the average annual costs for a number of exotic infectious diseases. In this article, we demonstrate how the government experts were helped to quantify their uncertainties about the cost estimates using formal expert elicitation techniques. This has enabled the decisionmakers to have a greater appreciation of government experts' uncertainty in this policy area.

Animation-based education as a gambling prevention tool: correcting erroneous cognitions and reducing the frequency of exceeding limits among slots players

In light of the financial harm that often accompanies problem gambling, and the difficulty in resolving it, there is a pressing need for prevention resources. In the present study, we examined the preventive effects of an animation-based video that educated participants on how slot machines function, the prudence of setting financial limits, and strategies to avoid problems. Non-problem gamblers (N = 242) at a slots venue were randomly assigned to watch either an animation or a control video. Compared to participants who watched the control video, those who watched the animation endorsed strategies to gamble within financial limits, reported greater behavioral intentions to use the strategies, and exceeded their pre-set limits less frequently during their subsequent gambling session. Some effects waned over a 30-day period suggesting booster sessions may be required for long term sustainability. The effectiveness of animation-based education as a prevention tool and the need for adjunctive measures is discussed.

Weighted Area Under the Receiver Operating Characteristic Curve and Its Application to Gene Selection

Partial area under the ROC curve (PAUC) has been proposed for gene selection in Pepe et al. (2003) and thereafter applied in real data analysis. It was noticed from empirical studies that this measure has several key weaknesses, such as an inability to reflect nonuniform weighting of different decision thresholds, resulting in large numbers of ties. We propose the weighted area under the ROC curve (WAUC) in this paper to address the problems associated with PAUC. Our proposed measure enjoys a greater flexibility to describe the discrimination accuracy of genes. Nonparametric and parametric estimation methods are introduced, including PAUC as a special case, along with theoretical properties of the estimators. We also provide a simple variance formula, yielding a novel variance estimator for nonparametric estimation of PAUC, which has proven challenging in previous work. The proposed methods permit sensitivity analyses, whereby the impact of differing weight functions on gene rankings may be assessed and results may be synthesized across weights. Simulations and re-analysis of two well-known microarray datasets illustrate the practical utility of WAUC.

Can human irrationality be experimentally demonstrated?

The object of this paper is to show why recent research in the psychology of deductive and probabilistic reasoning does not have "bleak implications for human rationality," as has sometimes been supposed. The presence of fallacies in reasoning is evaluated by referring to normative criteria which ultimately derive their own credentials from a systematisation of the intuitions that agree with them. These normative criteria cannot be taken, as some have suggested, to constitute a part of natural science, nor can they be established by metamathematical proof. Since a theory of competence has to predict the very same intuitions, it must ascribe rationality to ordinary people.

Accordingly, psychological research on this topic falls into four categories. In the first, experimenters investigate conditions under which their subjects suffer from genuine cognitive illusions. The search for explanations of such performance errors may then generate hypotheses about the ways in which the relevant information-processing mechanisms operate. In the second category, experimenters investigate circumstances in which their subjects exhibit mathematical or scientific ignorance: these are tests of the subjects' intelligence or education. In the third and fourth categories, experimenters impute a fallacy where none exists, either because they are applying the relevant normative criteria in an inappropriate way or because the normative criteria being applied are not the appropriate ones.

Planning and decision theory

Research on planning in AI can be separated into the two major areas: plan generation and plan representation. Most AI planners to date have been based on the STRIPS planning representation. This representation has a number of limitations. Much recent work in plan representation has addressed these limitations. It was shown that Decision Theory can be used to remove a number of the limitations. Furthermore, the decision theoretic framework provides a precise definition of rational behaviour. There remain open questions within decision theory regarding belief revision and causality. It should be noted that these problems are not artifacts of the representation. Rather they arise because the rich representation allows their formulation. Some work integrating AI and decision theoretic approaches to planning has been done but this remains a largely untouched research area.

We see two main avenues for fruitful research. First, the straightforward decision theoretic formulation of planning is computationally impractical. Techniques need to be developed to do efficient decision theoretic planning. Work in AI plan generation has exploited information contained the structure of qualitative representations to guide efficient plan construction. These techniques should be applied to decision theoretic representations as well. Second, AI has developed many representations that allow useful structuring of knowledge about the world. Decision Theory has concentrated on representing beliefs and desires. Integration of AI and decision theoretic representations would yield powerful representation languages. Some of the benefits of such work can already be seen in the research combining temporal and decision theoretic representations.