Epistemic uncertainty in Bayesian predictive probabilities

Bayesian predictive probabilities have become a ubiquitous tool for design and monitoring of clinical trials. The typical procedure is to average predictive probabilities over the prior or posterior distributions. In this paper, we highlight the limitations of relying solely on averaging, and propose the reporting of intervals or quantiles for the predictive probabilities. These intervals formalize the intuition that uncertainty decreases with more information. We present four different applications (Phase 1 dose escalation, early stopping for futility, sample size re-estimation, and assurance/probability of success) to demonstrate the practicality and generality of the proposed approach.

A hybrid approach to comparing parallel-group and stepped-wedge cluster-randomized trials with a continuous primary outcome when there is uncertainty in the intra-cluster correlation

BACKGROUND/AIMS

To evaluate how uncertainty in the intra-cluster correlation impacts whether a parallel-group or stepped-wedge cluster-randomized trial design is more efficient in terms of the required sample size, in the case of cross-sectional stepped-wedge cluster-randomized trials and continuous outcome data.

METHODS

We motivate our work by reviewing how the intra-cluster correlation and standard deviation were justified in 54 health technology assessment reports on cluster-randomized trials. To enable uncertainty at the design stage to be incorporated into the design specification, we then describe how sample size calculation can be performed for cluster- randomized trials in the 'hybrid' framework, which places priors on design parameters and controls the expected power in place of the conventional frequentist power. Comparison of the parallel-group and stepped-wedge cluster-randomized trial designs is conducted by placing Beta and truncated Normal priors on the intra-cluster correlation, and a Gamma prior on the standard deviation.

RESULTS

Many Health Technology Assessment reports did not adhere to the Consolidated Standards of Reporting Trials guideline of indicating the uncertainty around the assumed intra-cluster correlation, while others did not justify the assumed intra-cluster correlation or standard deviation. Even for a prior intra-cluster correlation distribution with a small mode, moderate prior densities on high intra-cluster correlation values can lead to a stepped-wedge cluster-randomized trial being more efficient because of the degree to which a stepped-wedge cluster-randomized trial is more efficient for high intra-cluster correlations. With careful specification of the priors, the designs in the hybrid framework can become more robust to, for example, an unexpectedly large value of the outcome variance.

CONCLUSION

When there is difficulty obtaining a reliable value for the intra-cluster correlation to assume at the design stage, the proposed methodology offers an appealing approach to sample size calculation. Often, uncertainty in the intra-cluster correlation will mean a stepped-wedge cluster-randomized trial is more efficient than a parallel-group cluster-randomized trial design.

Volume matters: Breast induration is associated with irradiated breast volume in the Danish Breast Cancer Group phase III randomized Partial Breast Irradiation trial

PURPOSE

The relation between breast induration grade 2-3 at 3 years after radiation therapy and irradiated breast volume was investigated for patients in the Danish Breast Cancer Group (DBCG) Partial Breast Irradiation (PBI) trial. METHODS Treatment plan data was obtained from the Danish radiotherapy plan database. Dosimetric parameters for breast and organs at risk were determined. Breast induration data was obtained from the DBCG database. The volume of the whole breast (CTVp_breast) treated to various dose levels was determined for treatment plans in both arms. Logistic regression was used to assess the frequency of induration on breast volume irradiated to ≥40 Gy. RESULTS PBI and WBI was given to 433 and 432 patients, respectively. Median and interquartile ranges (IQR) for CTVp_breast were 710 mL (467-963 mL; PBI) and 666 mL (443-1012 mL; WBI) (p = 0.98). Median and IQR for CTVp_breast treated to ≥40 Gy was 24.9% (18.6-32.6%; PBI) and 59.8% (53.6-68.5%; WBI). Grade 2-3 induration was observed in 5% (PBI) and 10% (WBI) of the patients. A dose-response relationship was established between irradiated breast volume and frequency of breast induration. From the model, 5% and 10% risks of breast induration were observed for ≥40 Gy delivered to CTVp_breast volumes of 177 mL (95%CI, 94-260 mL) and 426 mL (95%CI, 286-567 mL), respectively. CONCLUSION The frequency of breast induration increased significantly with increasing irradiated breast volume, strongly favouring small volumes and PBI. Thus, treated breast volume - not the breast size itself - is the risk factor for induration. This is the first report directly linking the 40 Gy irradiated breast volume to breast induration.

A Review of Bayesian Perspectives on Sample Size Derivation for Confirmatory Trials

Sample size derivation is a crucial element of planning any confirmatory trial. The required sample size is typically derived based on constraints on the maximal acceptable Type I error rate and minimal desired power. Power depends on the unknown true effect and tends to be calculated either for the smallest relevant effect or a likely point alternative. The former might be problematic if the minimal relevant effect is close to the null, thus requiring an excessively large sample size, while the latter is dubious since it does not account for the a priori uncertainty about the likely alternative effect. A Bayesian perspective on sample size derivation for a frequentist trial can reconcile arguments about the relative a priori plausibility of alternative effects with ideas based on the relevance of effect sizes. Many suggestions as to how such "hybrid" approaches could be implemented in practice have been put forward. However, key quantities are often defined in subtly different ways in the literature. Starting from the traditional entirely frequentist approach to sample size derivation, we derive consistent definitions for the most commonly used hybrid quantities and highlight connections, before discussing and demonstrating their use in sample size derivation for clinical trials.

Opportunities to enhance ward audit: a multi-site qualitative study

BACKGROUND

Hospitals in many countries are encouraged to develop audits to assess and improve the quality of care. Ward audit is a specific form of audit and feedback that is commonly used but little studied. The aim of this study is to describe the content and application of hospital ward audit in order to identify potential enhancements to such audits.

METHODS

Multiple qualitative methods were used to study a diversity sample of four English National Health Service organisations over a 16-month period. We undertook semi-structured interviews (n = 32), documentary analysis (n = 44) and 25 h of observations of healthcare workers involved in the design and implementation of ward audit. Data were analysed using framework analysis. Findings were presented iteratively to stakeholders who used them to develop a description of the content and delivery of ward audit.

RESULTS

Ward audit consisted of seven stages: impetus; method; preparation of staff; assessing practice; analysis; feedback; and decide on action to improve. Two key stages were the monthly assessment of practice using case note data extraction, and the resulting feedback to clinical staff, ward managers, matrons and directors of nursing. At three organisations, the case note data were extracted by staff and there was evidence that this resulted in misrepresentation of the clinical performance audited. The misrepresentation appeared to be associated with the anticipation of punitive feedback from directors of nursing and matrons, as well as time pressures and a lack clarity about the method of audit data collection. Punitive feedback was reported to occur if no data were collected, if data demonstrated poor performance or if performance did not improve.

CONCLUSIONS

Organisations invest considerable clinical resources in ward audit, but such audits may have unintended, potentially negative, consequences due to the impacts from punitive feedback. We discuss potential enhancements to ward audit (e.g. providing feedback recipients with suggested actions for improvement) and discuss implications for theory. There is a need to reduce the use of punitive feedback.

Optimal designs for phase II/III drug development programs including methods for discounting of phase II results

Background

Go/no-go decisions after phase II and sample size chosen for phase III are usually based on phase II results (e.g., the treatment effect estimate of phase II). Due to the decision rule (only promising phase II results lead to phase III), treatment effect estimates from phase II that initiate a phase III trial commonly overestimate the true treatment effect. Underpowered phase III trials are the consequence. Optimistic findings may then not be reproduced, leading to the failure of potentially expensive drug development programs. For some disease areas these failure rates are described to be quite high: 62.5%.

Methods

We integrate the ideas of multiplicative and additive adjustment of treatment effect estimates after go decisions in a utility-based framework for optimizing drug development programs. The design of a phase II/III program, i.e., the “right amount of adjustment”, the allocation of the resources to phase II and III in terms of sample size, and the rule applied to decide whether to stop or to proceed with phase III influences its success considerably. Given specific drug development program characteristics (e.g., fixed and variable per patient costs for phase II and III, probable gain in case of market launch), optimal designs with respect to the maximal expected utility can be identified by the proposed Bayesian-frequentist approach. The method will be illustrated by application to practical examples characteristic for oncological studies.

Results

In general, our results show that the program set-ups with adjusted treatment effect estimate used for phase III planning are superior to the “naïve” program set-ups with respect to the maximal expected utility. Therefore, we recommend considering an adjusted phase II treatment effect estimate for the phase III sample size calculation. However, there is no one-fits-all design.

Conclusion

Individual drug development planning for a specific program is necessary to find the optimal design. The optimal choice of the design parameters for a specific drug development program at hand can be found by our user friendly R Shiny application and package (both assessable open-source via [1]).

National official assurance systems for international trade in animals and animal products, with reference to the standards of the World Organisation for Animal Health

In the context of trade, national official assurance systems are the mechanism through which countries provide official assurance to other countries that their products are safe to trade. Regardless of the form in which it is conveyed, an official assurance, for the most part, is a statement from one competent authority to another about the conformity of a consignment with agreed requirements. Effectively, one government is providing a level of guarantee to the other government about matters such as the disease or pest status that exists nationally or regionally and/ or about the risk management activities that have been undertaken as relevant to the traded consignment. Accordingly, the degree of confidence that the importing competent authority has in the ethics, competence and capability of the exporting country's competent authority is central to how much trust the importing country places in the official assurances from the exporting country. The World Organisation for Animal Health Terrestrial Animal Health Code and Aquatic Animal Health Code (Section 5 of both) set out veterinary certificate requirements relating to animal health and zoonoses for both importing and exporting countries engaging in the trade of animals and animal products. These requirements are supplemented by the guidance developed by the Codex Committee on Food Inspection and Certification Systems, which covers the inspection and certification system requirements related to food safety and other non-health-related technical matters (e.g. composition, grade or organic status), as relevant to the international trade in food. This review discusses the need for countries to further align the form and content of their official assurance requirements with the relevant international standards and recommendations. It also notes, however, that there is currently a paucity of recommended standardised attestations. It highlights the increasing movement towards electronic certification and the potential this brings for further amalgamation of different certificate types and the coordination of border clearance processes. The basic components and principles that apply to national official assurance systems are identified and explained. Lastly, future trends and challenges for national official assurance systems, such as the impact of electronic commerce and regional distribution hubs, and the increasing recognition of containment zones and/or risk mitigations, such as treatments, are discussed.

National Veterinary Services and the private sector: the role of private veterinarians, aquatic animal health professionals and veterinary para-professionals in animal health and food safety assurance

Establishing trust in national systems for assurance of animal health and food safety is a key step in any importing country's consideration of whether a potential trading partner can meet its appropriate level of protection. Private veterinarians, veterinary para-professionals (VPPs) and aquatic animal health professionals (AAHPs) play a crucial role in national Veterinary Services, formally or informally, and across the whole spectrum of national animal and public health activities. Private veterinarians, AAHPs or VPPs are engaged as part of the national Veterinary Services and in various forms of public-private partnerships (PPPs) in many countries worldwide. In many cases, authorised private veterinarians, AAHPs and VPPs enable the national Veterinary or Aquatic Animal Health Services as a whole to do more work over a wider geographical area and thus have a greater impact than publicly employed professionals working alone. The deployment of private veterinarians, AAHPs and VPPs directly or in PPP arrangements strengthens national services and enhances their ability to deliver reliable animal health and food safety assurance. To ensure that private veterinarians, AAHPs and VPPs deliver to their full potential, effective and efficient systems for training, accreditation, monitoring and audit are essential. This article draws on data from published OIE Performance of Veterinary Services evaluations (from 2007 to the present) and unpublished responses to the OIE 2017 questionnaire on PPPs, to draw insights into the use and accreditation of private veterinarians, AAHPs and VPPs globally.

Biovigilance for the Quality and Safety of Medical Products of Human Origin

Progress in science and technology in the health services has led to the development of methods of regenerating and replacing solid organs, tissues and cells, using human body components to create medical products of human origin intended for clinical use. In the activities in which products of human origin are used, however, from the point of donation and harvesting to the subsequent care of the recipient, medical products of human origin are exposed to the risk of specific complications related to the transmission of infectious diseases, and further side-effects. Biovigilance system application is a basic requirement for ensuring the quality and safety of tissues and cells intended for human use. The quality system focuses on error prevention, maintaining a consistent pattern of agreed assays for tissues and cells intended for clinical use. The implementation of quality and safety standards, the development of medical protocols and cooperation protocols between member states, the implementation of Single European Code (SEC), and the development of electronic traceability systems, all aim at vigilance and the surveillance of medical products of human origin from donation to transplantation.

An anaesthesia information management system as a tool for a quality assurance program: 10years of experience

INTRODUCTION

Anaesthesia Information Management Systems (AIMS) generate large amounts of data, which might be useful for quality assurance programs. This study was designed to highlight the multiple contributions of our AIMS system in extracting quality indicators over a period of 10years.

METHODS

The study was conducted from 2002 to 2011. Two methods were used to extract anaesthesia indicators: the manual extraction of individual files for monitoring neuromuscular relaxation and structured query language (SQL) extraction for other indicators which were postoperative nausea and vomiting (PONV), pain, sedation scores, pain-related medications, scores and postoperative hypothermia. For each indicator, a program of information/meetings and adaptation/suggestions for operating room and PACU personnel was initiated to improve quality assurance, while data were extracted each year.

RESULTS

The study included 77,573 patients. The mean overall completeness of data for the initial years ranged from 55 to 85% and was indicator-dependent, which then improved to 95% completeness for the last 5years. The incidence of neuromuscular monitoring was initially 67% and then increased to 95% (P<0.05). The rate of pharmacological reversal remained around 53% throughout the study. Regarding SQL data, an improvement of severe postoperative pain and PONV scores was observed throughout the study, while mild postoperative hypothermia remained a challenge, despite efforts for improvement.

DISCUSSION

The AIMS system permitted the follow-up of certain indicators through manual sampling and many more via SQL extraction in a sustained and non-time-consuming way across years. However, it requires competent and especially dedicated resources to handle the database.