A Nationwide Network of Health AI Assurance Laboratories

Importance

Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed.

Observations

While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings.

Conclusion and Relevance

The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

Assessing a new pre-screening score for the simplified evaluation of the clinical quality and relevance of e-health applications. (Preprint)

Background

In 2020, more than 250 eHealth solutions were added to app stores each day, or 90,000 in the year; however, the vast majority of these solutions have not undergone clinical validation, their quality is unknown, and the user does not know if they are effective and safe. We sought to develop a simple prescreening scoring method that would assess the quality and clinical relevance of each app. We designed this tool with 3 health care stakeholder groups in mind: eHealth solution designers seeking to evaluate a potential competitor or their own tool, investors considering a fundraising candidate, and a hospital clinician or IT department wishing to evaluate a current or potential eHealth solution.

Objective

We built and tested a novel prescreening scoring tool (the Medical Digital Solution scoring tool). The tool, which consists of 26 questions that enable the quick assessment and comparison of the clinical relevance and quality of eHealth apps, was tested on 68 eHealth solutions.

Methods

The Medical Digital Solution scoring tool is based on the 2021 evaluation criteria of the French National Health Authority, the 2022 European Society of Medical Oncology recommendations, and other provided scores. We built the scoring tool with patient association and eHealth experts and submitted it to eHealth app creators, who evaluated their apps via the web-based form in January 2022. After completing the evaluation criteria, their apps obtained an overall score and 4 categories of subscores. These criteria evaluated the type of solution and domain, the solution’s targeted population size, the level of clinical assessment, and information about the provider.

Results

In total, 68 eHealth solutions were evaluated with the scoring tool. Oncology apps (22%, 20/90) and general health solutions (23%, 21/90) were the most represented. Of the 68 apps, 32 (47%) were involved in remote monitoring by health professionals. Regarding clinical outcomes, 5% (9/169) of the apps assessed overall survival. Randomized studies had been conducted for 21% (23/110) of the apps to assess their benefit. Of the 68 providers, 38 (56%) declared the objective of obtaining reimbursement, and 7 (18%) out of the 38 solutions seeking reimbursement were assessed as having a high probability of reimbursement. The median global score was 11.2 (range 4.7-17.4) out of 20 and the distribution of the scores followed a normal distribution pattern (Shapiro-Wilk test: P=.33).

Conclusions

This multidomain prescreening scoring tool is simple, fast, and can be deployed on a large scale to initiate an assessment of the clinical relevance and quality of a clinical eHealth app. This simple tool can help a decision-maker determine which aspects of the app require further analysis and improvement.

External quality assessment of COVID-19 real time reverse transcription PCR laboratories in India

Sudden emergence and rapid spread of COVID-19 created an inevitable need for expansion of the COVID-19 laboratory testing network across the world. The strategy to test-track-treat was advocated for quick detection and containment of the disease. Being the second most populous country in the world, India was challenged to make COVID-19 testing available and accessible in all parts of the country. The molecular laboratory testing network was augmented expeditiously, and number of laboratories was increased from one in January 2020 to 2951 till mid-September, 2021. This rapid expansion warranted the need to have inbuilt systems of quality control/ quality assurance. In addition to the ongoing inter-laboratory quality control (ILQC), India implemented an External Quality Assurance Program (EQAP) with assistance from World Health Organization (WHO) and Royal College of Pathologists, Australasia. Out of the 953 open system rRTPCR laboratories in both public and private sector who participated in the first round of EQAP, 891(93.4%) laboratories obtained a passing score of > = 80%. The satisfactory performance of Indian COVID-19 testing laboratories has boosted the confidence of the public and policy makers in the quality of testing. ILQC and EQAP need to continue to ensure adherence of the testing laboratories to the desired quality standards.

The sixth edition of the WHO manual on semen examination: ensuring quality and standardization in basic examination of human ejaculates

A basic semen investigation has established principles that are necessary for ascertaining reliable and internationally comparable results. Although these principles have been present in the WHO manual since its inception, the baseline issue across most published studies and practice in reproductive medicine (in which the male is considered) is repetitive failure to adhere to these principles, thereby leading to relevant comparable data and accuracy. To address this failure, the sixth edition of the WHO manual includes revised basic methods, and a complementary formal standard of the International Standards Organization (ISO23162:2021) for basic semen examination has been published. Perhaps the most significant change in the sixth edition is the reintroduction of the four-category distinction of sperm motility, which causes additional work for laboratories in changing reporting parameters but is clinically important. Another essential change is the widened focus from mainly a prognostic tool for medically assisted reproduction to additionally raising awareness of semen examination as a measure of male reproductive functions and general male health.

How to assure the quality of clinical records? A 7-year experience in a large academic hospital

INTRODUCTION

Clinical record (CR) is the primary tool used by healthcare workers (HCWs) to record clinical information and its completeness can help achieve safer practices. CR is the most appropriate source in order to measure and evaluate the quality of care. In order to achieve a safety climate is fundamental to involve a responsive healthcare workforce thorough peer-review and feedbacks. This study aims to develop a peer-review tool for clinical records quality assurance, presenting the seven-year experience in the evolution of it; secondary aims are to describe the CR completeness and HCWs' diligence toward recording information in it.

METHODS

To assess the completeness of CRs a peer-review tool was developed in a large Academic Hospital of Northern Italy. This tool included measurable items that examined different themes, moments and levels of the clinical process. Data were collected every three months between 2010 and 2016 by appointed and trained HCWs from 42 Units; the hospital Quality Unit was responsible for of processing and validating them. Variations in the proportion of CR completeness were assessed using Cochran-Armitage test for trends.

RESULTS

A total of 9,408 CRs were evaluated. Overall CR completeness improved significantly from 79.6% in 2010 to 86.5% in 2016 (p<0.001). Doctors' attitude showed a trend similar to the overall completeness, while nurses improved more consistently (p<0.001). Most items exploring themes, moments and levels registered a significant improvement in the early years, then flattened in last years. Results of the validation process were always above the cut-off of 75%.

CONCLUSIONS

This peer-review tool enabled the Quality Unit and hospital leadership to obtain a reliable picture of CRs completeness, while involving the HCWs in the quality evaluation. The completeness of CR showed an overall positive and significant trend during these seven years.

Quantity and quality of complementary and alternative medicine recommendations in clinical practice guidelines for type 2 diabetes mellitus: A systematic review

AIMS

Approximately 70% of Americans with diabetes have used complementary and alternative medicine (CAM) in the past year. Healthcare providers often receive minimal training on these therapies and subsequently rely on clinical practice guidelines (CPGs) to supplement their knowledge about the safe and effective use of CAM for the treatment/management of type 2 diabetes mellitus (T2DM). The purpose of this systematic review is to determine the quantity and assess the quality of CAM recommendations in CPGs for the treatment and/or management of T2DM.

DATA SYNTHESIS

MEDLINE, EMBASE, and CINAHL were systematically searched from 2009 to 2020, in addition to the Guidelines International Network and the National Center for Complementary and Integrative Health websites. CPGs containing treatment and/or management recommendations for T2DM were eligible; those with CAM recommendations were quality-assessed with the AGREE II instrument twice, once for the overall CPG and once for the CAM sections. Twenty-seven CPGs were deemed eligible, of which 7 made CAM recommendations. Mean scaled domain percentages were (overall, CAM): scope and purpose (89.7%, 79.8%), clarity of presentation (85.7%, 48.4%), stakeholder involvement (67.9%, 28.2%), applicability (54.8%, 20.2%), rigour of development (49.7%, 35.7%), and editorial independence (44.1%, 44.1%).

CONCLUSIONS

Quality varied within and across CPGs; domain scores across CAM sections generally scored lower than the overall CPG. Given that CAM therapies for T2DM are only represented in one-quarter of eligible CPGs and are of lower quality, a knowledge gap exists for healthcare providers who seek evidence-based information on this topic in order to effectively counsel inquiring patients.

Identification and Critical Appraisal of Food Service Satisfaction Questionnaires for Use in Nursing Homes: A Systematic Review

BACKGROUND

Food service provision in nursing homes is a complex, adaptive system through which multiple stakeholders interface. Organizational stakeholders include staff involved in preparing and delivering meals. Consumer stakeholders are the end users including residents and family. Questionnaires can be an economical and efficient method of measuring food service satisfaction in nursing homes and a powerful quality improvement tool.

OBJECTIVE

(1) To identify questionnaires that measure food service satisfaction of various stakeholders in a nursing homes and (2) to critically appraise the psychometric properties of identified questionnaires.

METHODS

Five electronic databases were searched (Cumulative Index to Nursing and Allied Health Literature, Medline, ProQuest, Scopus, and Cochrane) in April 2020. Data from the eligible studies were extracted, and the psychometric properties were critically appraised using the Consensus-Based Standards for the Selection of Health Measurement Instruments.

RESULTS

This review identified 129 studies that used a questionnaire to measure food service satisfaction in nursing homes. Of those, 107 studies representing 75 unique general nursing home satisfaction questionnaires were excluded for failing to adequately explore aspects related to food service. From the remaining 22 studies, 7 food service satisfaction questionnaires were identified; 5 intended for consumers (residents) and 2 intended for organizational stakeholders (staff). Using the Consensus-Based Standards for the Selection of Health Measurement Instruments quality criteria, most questionnaires had flaws in content validity and construct validity, primarily due to small sample sizes. No questionnaires explored food service satisfaction from the family perspective.

CONCLUSIONS

Nursing homes collect satisfaction information for accreditation, marketing, benchmarking, and quality improvement. Although questionnaires are easy to administer, the quality of the data they collect is impacted by the validity and reliability of the questionnaires used. Using unreliable satisfaction data may mean that nursing homes are not accurately able to understand the impact of changes in the system on stakeholder satisfaction.

Integrating quality control and external quality assurance

Effective management of clinical laboratories relies upon an understanding of Quality Control and External Quality Assurance principles. These processes, when applied effectively, reduce patient risk and drive quality improvement. In this Review, we will describe the purpose of QC and EQA and their role in identifying analytical and process error. The two concepts are linked, and we will illustrate that linkage. Some EQA providers offer far more than analytical surveillance. They facilitate training and education and extend quality improvement and identify areas where there is potential for patient harm into the pre-and post-analytical phases of the total testing process.

Quality assurance of radiotherapy in the ongoing EORTC 1420 "Best of" trial for early stage oropharyngeal, supraglottic and hypopharyngeal carcinoma: results of the benchmark case procedure

INTRODUCTION

The current phase III EORTC 1420 Best-of trial (NCT02984410) compares the swallowing function after transoral surgery versus intensity modulated radiotherapy (RT) in patients with early-stage carcinoma of the oropharynx, supraglottis and hypopharynx. We report the analysis of the Benchmark Case (BC) procedures before patient recruitment with special attention to dysphagia/aspiration related structures (DARS).

MATERIALS AND METHODS

Submitted RT volumes and plans from participating centers were analyzed and compared against the gold-standard expert delineations and dose distributions. Descriptive analysis of protocol deviations was conducted. Mean Sorensen-Dice similarity index (mDSI) and Hausdorff distance (mHD) were applied to evaluate the inter-observer variability (IOV).

RESULTS

65% (23/35) of the institutions needed more than one submission to achieve Quality assurance (RTQA) clearance. OAR volume delineations were the cause for rejection in 53% (40/76) of cases. IOV could be improved in 5 out of 12 OARs by more than 10 mm after resubmission (mHD). Despite this, final IOV for critical OARs in delineation remained significant among DARS by choosing an aleatory threshold of 0.7 (mDSI) and 15 mm (mHD).

CONCLUSIONS

This is to our knowledge the largest BC analysis among Head and neck RTQA programs performed in the framework of a prospective trial. Benchmarking identified non-common OARs and target delineations errors as the main source of deviations and IOV could be reduced in a significant number of cases after this process. Due to the substantial resources involved with benchmarking, future benchmark analyses should assess fully the impact on patients' clinical outcome.

Quality assurance in radiation oncology

The Children's Oncology Group (COG) has a strong quality assurance (QA) program managed by the Imaging and Radiation Oncology Core (IROC). This program consists of credentialing centers and providing real-time management of each case for protocol compliant target definition and radiation delivery. In the International Society of Pediatric Oncology (SIOP), the lack of an available, reliable online data platform has been a challenge and the European Society for Paediatric Oncology (SIOPE) quality and excellence in radiotherapy and imaging for children and adolescents with cancer across Europe in clinical trials (QUARTET) program currently provides QA review for prospective clinical trials. The COG and SIOP are fully committed to a QA program that ensures uniform execution of protocol treatments and provides validity of the clinical data used for analysis.

Progress will come in the balancing of needs: Professional desires, consumer demand, and why we have to specialize

Conoley, Powers, and Gutkin (2020) have provided an assessment of the progress and lack thereof of school psychology as a profession toward school reformation, especially as related to their various earlier calls for changes in training and practice. Conoley and Gutkin especially (and I have been colleagues of both) have called for decades for an emphasis on working with teachers, other adults in the school system, and the school system itself to alter the delivery of education. In some earlier work, they have characterized this as adult school psychology. In many ways this is a reconceptualization of earlier mental health consultation models of school psychology practice that were not widely adopted for reasons Conoley et al. (2020) explain. School psychology has a long history of wanting to be seen and to practice differently from in a primary role of test and place, and indeed I would characterize the primary organizations that represent the profession (the National Association of Psychologists and the American Psychological Association's Division of School Psychology) as being in many ways antagonistic toward psychological testing. Yet, it is what most school psychologists are in fact hired to do as their primary role, and training in psychological testing and assessment is one of the key skills that sets us apart and distinguishes us from others in the schools. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Quality of oxytocin and misoprostol in health facilities of Rwanda

Sustainable Development Goal 3.1 calls for a reduction of the maternal mortality ratio to less than 70 per 100,000 live births by 2030. The most important cause of maternal mortality is post-partum haemorrhage (PPH). Oxytocin injections and misoprostol tablets are medicines of first choice for the management of PPH in low- and middle-income countries (LMICs). Unfortunately, both substances are chemically unstable, and previous studies have revealed serious quality problems of these medicines in LMICs. The present study is the first report on their quality in Rwanda. From 40 randomly selected health facilities (hospitals, health centers, retail pharmacies and private clinics) in different parts of Rwanda, as well as from six wholesalers and government stores, oxytocin injections and misoprostol tablets were collected. Oxytocin storage temperatures in the health facilities were monitored for six months using temperature data loggers, and found to correctly follow the storage requirements stated by the manufacturers (2–8°C, or room temperature) with few minor deviations. Oxytocin injections (57 samples, representing seven batches of four brands) were tested for their oxytocin content and pH value according to the United States Pharmacopeia. Twenty-four samples from three European manufacturers passed all tests. However, all nine samples of one batch of a Chinese manufacturer showed an excessive content of oxytocin (range 117.2–121.5% of the declared amount). Another batch of the same manufacturer showed extreme variations of the concentration of the preservative benzyl alcohol. Misoprostol tablets (25 samples, representing ten batches of six brands) were tested for content and dissolution according to the International Pharmacopoeia. Fifteen samples passed, but all 10 samples of two brands from India failed with extreme deviations, containing only 42.5–48.7% of the stated amount of misoprostol. In conclusion, oxytocin quality in Rwanda was better than reported from other African countries. However, two extremely substandard brands of misoprostol tablets were found. The Rwandan authorities reacted quickly and efficiently, and recalled these substandard medicines from the market. For oxytocin and misoprostol, with their well-known problems of quality and stability, procurement should possibly be restricted to medicines which are WHO-prequalified or which have been manufactured in countries with stringent regulatory authorities.

Optimizing hepcidin measurement with a proficiency test framework and standardization improvement

Objectives

Hepcidin measurement advances insights in pathophysiology, diagnosis, and treatment of iron disorders, but requires analytically sound and standardized measurement procedures (MPs). Recent development of a two-level secondary reference material (sRM) for hepcidin assays allows worldwide standardization. However, no proficiency testing (PT) schemes to ensure external quality assurance (EQA) exist and the absence of a high calibrator in the sRM set precludes optimal standardization.

Methods

We developed a pilot PT together with the Dutch EQA organization Stichting Kwaliteitsbewaking Medische Laboratoriumdiagnostiek (SKML) that included 16 international hepcidin MPs. The design included 12 human serum samples that allowed us to evaluate accuracy, linearity, precision and standardization potential. We manufactured, value-assigned, and validated a high-level calibrator in a similar manner to the existing low- and middle-level sRM.

Results

The pilot PT confirmed logistical feasibility of an annual scheme. Most MPs demonstrated linearity (R2>0.99) and precision (duplicate CV>12.2%), although the need for EQA was shown by large variability in accuracy. The high-level calibrator proved effective, reducing the inter-assay CV from 42.0% (unstandardized) to 14.0%, compared to 17.6% with the two-leveled set. The calibrator passed international homogeneity criteria and was assigned a value of 9.07±0.24 nmol/L.

Conclusions

We established a framework for future PT to enable laboratory accreditation, which is essential to ensure quality of hepcidin measurement and its use in patient care. Additionally, we showed optimized standardization is possible by extending the current sRM with a third high calibrator, although international implementation of the sRM is a prerequisite for its success.

Papanicolaou Staining and Processing Troubleshooting Exercise Using Digital Pathology: Peer Comparison in an External Quality Assurance Setting

BACKGROUND

The Royal College of Pathologists of Australasia Quality Assurance Programs (RCPAQAP) offers a comprehensive range of External Quality Assurance programs for all disciplines of pathology and is committed to delivering programs to advance outcomes in patient care and promote excellence in laboratory diagnostic and technical proficiency. Proficiency testing in cytopreparatory techniques became available for formal enrolment in 2018. The 2019 technical program focused on the demonstration of a laboratory to troubleshoot problems encountered during staining and processing. The exercise provided a peer comparison of a laboratory's ability to (1) identify the problem viewed from a digital image and (2) describe the troubleshooting steps to rectify the problem using assessment guidelines.

METHODS

Ten troubleshooting exercises including whole-slide z-stacked virtual images, a reference slide, and an online Result Entry questionnaire were provided. Common staining and processing problems were demonstrated in these exercises. Multiple-choice responses were made available to participants for each case with 1 response requiring submission. Participants were then allowed free text to describe how they would rectify the problem.

RESULTS

Of the 72 participants who participated, 62 participants (86%) achieved a satisfactory assessment, 6 participants (8%) received a borderline assessment, and 4 participants (6%) received an unsatisfactory assessment. The average mark obtained was 32.7 out of a total of 45 marks, which equates to a satisfactory assessment. The highest mark obtained for this survey was 42.5.

CONCLUSIONS

Quality cytopathology preparatory techniques are integral to mitigating the risk of diagnostic error. The digital pathology platform allowed homogeneity of samples for all participants and appeared to be a satisfactory mode for demonstrating the technical problems to participants. Laboratories participating in this survey showed good performance in identifying processing and staining problems on virtual images and working through the troubleshooting steps. The exercise also identified a method for troubleshooting formalin vapour-affected slides using citrate buffer which may be useful for laboratories following in-house validation.

[The use of RCGO triggers in the obstetric - gynecological procedures: the impact on the reduction of adeverse events. The experience of the Lombardia Region]

The last few weeks of pregnancy are critical to a baby's health because important organs, including the brain and lungs, are not completely developed until the end of pregnancy. The adverse events during labor and childbirth can have very serious physical, psychological and financial consequences for the child, the family, health professionals and the whole community. These events can be reduced through interventions aimed at improving the safety and quality of care, based on evidence-based knowledge, guidelines and practices that must be widely and effectively applied. This work reports the experience of the Lombardy Region on improvement actions in the obstetric and gynecological procedures for the reduction of adverse events and sentinel events through the monitoring and management of the RCGS trigger tool.

Personal-recovery-oriented community mental healthcare: qualitative evaluation of a developmental project

OBJECTIVES

Mental healthcare is commonly aimed at reducing symptoms in individual service users. When only symptomatic recovery is addressed, not all service users experience sufficient recovery, and when care is aimed only at individuals (instead of the neighbourhood), not all people in need of mental healthcare are reached. This study evaluated a project that aimed to improve mental healthcare in a neighbourhood, by improving healthcare providers' outreach to the residents living in the neighbourhood, by improving collaboration among healthcare providers and focussing on the residents' personal recovery. This project was carried out by several public health services. It aimed to change the goal of mental healthcare provided in the neighbourhood from symptom reduction to personal recovery.

DESIGN

The study included qualitative focus groups and inductive content analysis.

SETTING

Primary and secondary mental healthcare that healthcare workers from different healthcare services provided.

PARTICIPANTS

The evaluation was conducted through three focus group interviews with services users, their friends and relatives, neighbourhood residents, neighbourhood representatives and the healthcare services that were involved (n = 24).

RESULTS

Evaluation indicated that the most valued part of the project was the utilisation of peer workers at the initiation of mental healthcare. Improved communication among healthcare providers that the project fostered was also highly regarded. The aim of the project to align it with existing initiatives in the neighbourhood was also considered important, although it was difficult to achieve.

CONCLUSIONS

The project did not find a panacea for recovery-oriented community mental healthcare. A variety of its components did, however, contribute to the mental health of the community residents.

Technological quality requirements for stereotactic radiotherapy : Expert review group consensus from the DGMP Working Group for Physics and Technology in Stereotactic Radiotherapy

This review details and discusses the technological quality requirements to ensure the desired quality for stereotactic radiotherapy using photon external beam radiotherapy as defined by the DEGRO Working Group Radiosurgery and Stereotactic Radiotherapy and the DGMP Working Group for Physics and Technology in Stereotactic Radiotherapy. The covered aspects of this review are 1) imaging for target volume definition, 2) patient positioning and target volume localization, 3) motion management, 4) collimation of the irradiation and beam directions, 5) dose calculation, 6) treatment unit accuracy, and 7) dedicated quality assurance measures. For each part, an expert review for current state-of-the-art techniques and their particular technological quality requirement to reach the necessary accuracy for stereotactic radiotherapy divided into intracranial stereotactic radiosurgery in one single fraction (SRS), intracranial fractionated stereotactic radiotherapy (FSRT), and extracranial stereotactic body radiotherapy (SBRT) is presented. All recommendations and suggestions for all mentioned aspects of stereotactic radiotherapy are formulated and related uncertainties and potential sources of error discussed. Additionally, further research and development needs in terms of insufficient data and unsolved problems for stereotactic radiotherapy are identified, which will serve as a basis for the future assignments of the DGMP Working Group for Physics and Technology in Stereotactic Radiotherapy. The review was group peer-reviewed, and consensus was obtained through multiple working group meetings.

Definition and quality requirements for stereotactic radiotherapy: consensus statement from the DEGRO/DGMP Working Group Stereotactic Radiotherapy and Radiosurgery

Stereotactic radiotherapy with its forms of intracranial stereotactic radiosurgery (SRS), intracranial fractionated stereotactic radiotherapy (FSRT) and stereotactic body radiotherapy (SBRT) is today a guideline-recommended treatment for malignant or benign tumors as well as neurological or vascular functional disorders. The working groups for radiosurgery and stereotactic radiotherapy of the German Society for Radiation Oncology (DEGRO) and for physics and technology in stereotactic radiotherapy of the German Society for Medical Physics (DGMP) have established a consensus statement about the definition and minimal quality requirements for stereotactic radiotherapy to achieve best clinical outcome and treatment quality in the implementation into routine clinical practice.

Development and Validation of a Tool to Assess the Quality of Clinical Practice Guideline Recommendations

IMPORTANCE

Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined.

OBJECTIVE

To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations.

DESIGN, SETTING, AND PARTICIPANTS

A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users.

EXPOSURES

Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey.

MAIN OUTCOMES AND MEASURES

Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach α, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters.

RESULTS

A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach α = 0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r = 0.81) and the clinical credibility score (r = 0.76).

CONCLUSIONS AND RELEVANCE

This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.

Tuberculosis Supranational Reference Laboratories: A Global Approach

The World Health Organization Supranational TB Reference Laboratory Network (SRLN) has served as the backbone for TB drug-resistance surveillance and diagnosis since 1994 and remains a key WHO programme for antimicrobial resistance (AMR) surveillance at the global level. SRLN is a great technical resource for proficiency testing to ensure accuracy of drug-susceptibility testing, scale-up, capacity development in countries and provides unique support to the reliable detection of drug resistance. Technical assistance from individual SRLs has been supported by a variety of mechanisms but funding for the SRLN has become increasingly challenging.

Management of type 2 diabetes using non-insulin glucose-lowering therapies: a critical appraisal of clinical practice guidelines with the AGREE II instrument

AIM

Type 2 diabetes is a major global epidemic affecting over 400 million people worldwide. The objective of this systematic review was to provide an overview of recommendations from clinical practice guidelines (guidelines) addressing non-insulin based pharmacological management of among non-pregnant adults in an outpatient setting, and critically appraise their methodological development.

METHODS

We systematically searched MEDLINE and Embase databases, for relevant guidelines using the Ovid interface. We scanned the bibliographies of all eligible guidelines for additional relevant citations. Teams of two reviewers, independently and in duplicate, screened titles and abstracts and potentially eligible full text reports to determine eligibility and appraised the reporting quality of guidelines using the Advancing Guideline Development, Reporting and Evaluation in Health Care instrument II (AGREE II) instrument.

RESULTS

Our search yielded 11264 unique citations, of which 124 were retrieved for full-text review; 17 guidelines proved eligible. The highest scoring AGREE domain was 'clarity of presentation' (66%; range 7-92%), followed by 'scope and purpose' (58%; range 25-92%), 'editorial independence' (55%; range 0-91%), 'stakeholder involvement' (45%; range 11-90%) and 'rigour of development' (43%; range 4-92%). The poorest domain was 'applicability' (37%; range 6-84%). The guidelines authored by the World Health Organization group achieved the highest AGREE overall score.

CONCLUSIONS

Most of the guidelines provided recommendations with a local jurisdictional focus and showed significant variation in the quality. Nevertheless, only a small number of those scored well overall.

Is patient safety in the NHS in England a postcode lottery?

John Tingle, Lecturer in Law, Birmingham Law School, University of Birmingham, discusses some recent patient safety publications from the World Health Organization and the Care Quality Commission.

Validation of the Dutch version of the primary care resources and support for self-management tool: A tool to assess the quality of self-management support

Introduction

Enhancing the self-management activities of patients improves the quality of care and is an integrated element of current healthcare provision. However, self-management support (SMS) is not yet common in healthcare. The Primary Care Resources and Support for Self-Management (PCRS) is a tool for healthcare professionals to assess the quality of SMS. In this study, we assessed the validity and reliability of the Dutch version of the PCRS.

Method

The validation of the PCRS was performed in Dutch healthcare centres. Correlations between the PCRS scores and the Assessment of Chronic Illness Care (ACIC) and Clinician Support for Patient Activation Measure (CS-PAM) scores were calculated to assess the convergent and discriminant validity. A confirmatory factor analysis (CFA) was performed to test the factor structure. Lastly, the internal consistency and face validity were assessed.

Results

The convergent and discriminant validity were good, with respective correlations of 0.730 (p < 0.001) and 0.030 (p > 0.050) between the PCRS and the ACIC SMS subscale and the PCRS and the CS-PAM. Although 49% of the variance of the PCRS was explained by one factor, the CFA could not confirm a fit between a one-factor model and the data. The reliability was excellent (Cronbach’s α = 0.921).

Conclusion

The PCRS showed good validity and excellent internal consistency. However, the evidence for its validity was inconclusive. We therefore suggest rephrasing specific items.

A Review of Best Practices for Monitoring and Improving Inpatient Pediatric Patient Experiences

CONTEXT

Achieving high-quality patient-centered care requires assessing patient and family experiences to identify opportunities for improvement. With the Child Hospital Consumer Assessment of Healthcare Providers and Systems Survey, hospitals can assess performance and make national comparisons of inpatient pediatric experiences. However, using patient and family experience data to improve care remains a challenge.

OBJECTIVE

We reviewed the literature on best practices for monitoring performance and undertaking activities aimed at improving pediatric patient and family experiences of inpatient care.

DATA SOURCES

We searched PubMed, Cumulative Index to Nursing and Allied Health Literature, and PsychINFO.

STUDY SELECTION

We included (1) English-language peer-reviewed articles published from January 2000 to April 2019; (2) articles based in the United States, United Kingdom, or Canada; (3) articles focused on pediatric inpatient care; (4) articles describing pediatric patient and family experiences; and (5) articles including content on activities aimed at improving patient and family experiences. Our review included 25 articles.

DATA EXTRACTION

Two researchers reviewed the full article and abstracted specific information: country, study aims, setting, design, methods, results, Quality Improvement (QI) initiatives performed, internal reporting description, best practices, lessons learned, barriers, facilitators and study implications for clinical practice, patient-experience data collection, and QI activities. We noted themes across samples and care settings.

RESULTS

We identified 10 themes of best practice. The 4 most common were (1) use evidence-based approaches, (2) maintain an internal system that communicates information and performance on patient and family experiences to staff and hospital leadership, (3) use experience survey data to initiate and/or evaluate QI interventions, and (4) identify optimal times (eg, discharge) and modes (eg, print) for obtaining patient and family feedback. These correspond to adult inpatient best practices.

CONCLUSIONS

Both pediatric and adult inpatient best practices rely on common principles of culture change (such as evidence-based clinical practice), collaborative learning, multidisciplinary teamwork, and building and/or supporting a QI infrastructure that requires time, money, collaboration, data tracking, and monitoring. QI best practices in both pediatric and adult inpatient settings commonly rely on identifying drivers of overall ratings of care, rewarding staff for successful implementation, and creating easy-to-use and easy-to-access planning and QI tools for staff.

Radiotherapy quality assurance for mesorectum treatment planning within the multi-center phase II STAR-TReC trial: Dutch results

BACKGROUND

The STAR-TReC trial is an international multi-center, randomized, phase II study assessing the feasibility of short-course radiotherapy or long-course chemoradiotherapy as an alternative to total mesorectal excision surgery. A new target volume is used for both (chemo)radiotherapy arms which includes only the mesorectum. The treatment planning QA revealed substantial variation in dose to organs at risk (OAR) between centers. Therefore, the aim of this study was to determine the treatment plan variability in terms of dose to OAR and assess the effect of a national study group meeting on the quality and variability of treatment plans for mesorectum-only planning for rectal cancer.

METHODS

Eight centers produced 25 × 2 Gy treatment plans for five cases. The OAR were the bowel cavity, bladder and femoral heads. A study group meeting for the participating centers was organized to discuss the planning results. At the meeting, the values of the treatment plan DVH parameters were distributed among centers so that results could be compared. Subsequently, the centers were invited to perform replanning if they considered this to be necessary.

RESULTS

All treatment plans, both initial planning and replanning, fulfilled the target constraints. Dose to OAR varied considerably for the initial planning, especially for dose levels below 20 Gy, indicating that there was room for trade-offs between the defined OAR. Five centers performed replanning for all cases. One center did not perform replanning at all and two centers performed replanning on two and three cases, respectively. On average, replanning reduced the bowel cavity V20Gy by 12.6%, bowel cavity V10Gy by 22.0%, bladder V35Gy by 14.7% and bladder V10Gy by 10.8%. In 26/30 replanned cases the V10Gy of both the bowel cavity and bladder was lower, indicating an overall lower dose to these OAR instead of a different trade-off. In addition, the bowel cavity V10Gy and V20Gy showed more similarity between centers.

CONCLUSIONS

Dose to OAR varied considerably between centers, especially for dose levels below 20 Gy. The study group meeting and the distribution of the initial planning results among centers resulted in lower dose to the defined OAR and reduced variability between centers after replanning.

TRIAL REGISTRATION

The STAR-TReC trial, ClinicalTrials.gov Identifier: NCT02945566. Registered 26 October 2016, https://clinicaltrials.gov/ct2/show/NCT02945566).

Quality assessment of interpretative commenting and competency comparison of comment providers in China

Background This study aimed to evaluate the ability of comment providers who were responsible for interpreting results in clinical laboratories in China and to improve the quality of interpretative comments. Methods Basic information and interpretative comments for five cases of 1912 routine chemistry External Quality Assessment (EQA) participant laboratories were collected by web-based EQA system in May 2018. EQA organizers assigned scores to each key phrase of comments based on predetermined marking scale and calculated total scores for each participant's answer. Final scores and ranking were calculated according to scores of cases. Finally, we comprehensively analyzed the type of hospital and the professional title of participants. Results In total, 772 clinical laboratories, 1472 participants, from different Chinese provinces submitted interpretative comments. Median scores, interquartile ranges and score ranges of the five cases were 13 (11-15, 1-20), 13 (10-16, 0-20), 15 (12-17, 0-21), 7 (5-9, -2 to 14) and 12 (10-13, -2 to 18). The final scores and ranking of participants that came from tertiary hospitals were higher than those from secondary and other hospitals; however, there were no significant differences (0.774). When grouped by professional title, we found that although no significant variability existed among senior, intermediate, junior and others (0.699), it existed between laboratory physicians and technicians, as the median final scores of the former were higher than the latter. Conclusions Practice and quality of interpretative comments are indeed different among different laboratories and participants in China. Laboratories should train and assess the interpretative ability of personnel. EQA organizers should also improve the scoring method and establish peer assessors team through this survey.

POSTAL TLD AUDIT OF HETEROGENEITY CORRECTIONS IN RADIOTHERAPY IN THE CZECH REPUBLIC

In the Czech Republic, a more advanced version of postal audit in radiotherapy (RT) is available. It covers dose measurements with thermoluminescent dosemeters (TLD) in more complex conditions of irradiation, when dose distribution is affected by heterogeneities in the irradiated volume. Relative deviation between doses measured with TLDs and doses stated by RT centre should not exceed 3%. During 2015-2017, all Czech RT centres equipped with modern linear accelerators were subjected to this more advanced TLD audit. A total of  70% of participants complied with the limit of 3% in the first round of this audit.

COMPARISON OF RADIATION DOSE AND IMAGE QUALITY IN CTA OF THE PERIPHERAL ARTERIES

The aim of this study is to investigate the possibility of replacing the standard CTA protocol for peripheral arteries with a low dose CTA protocol without affecting the diagnostic image quality. Therefore a single centre retrospective study was conducted involving 200 exams of patients undergoing lower limb angiography. All exams were performed on a 64-row detector CT and the vascular density, muscle density, noise and radiation dose of each image were assessed. The subjective image quality was evaluated additionally by an experienced radiologist. Significant differences were observed in radiation dose and image quality between the standard CTA protocol and the lower dose CTA protocol. No differences were found between objective and subjective image quality. Using 80kVp instead of 120kVp as the tube voltage for lower limb CTA reduces the radiation dose without affecting the diagnostic image quality.

Realist evaluation of UK medical education quality assurance

OBJECTIVES

The aim of the study was to explore what components of the General Medical Council's (GMC) Quality Assurance Framework work, for whom, in what circumstances and how?

SETTING

UK undergraduate and postgraduate medical education and training.

PARTICIPANTS

We conducted interviews with a stratified sample of 36 individuals. This included those who had direct experiences, as well as those with external insights, representing local, national and international organisations within and outside medicine.

INTERVENTION

The GMC quality assure education to protect patient and public safety utilising complex intervention components including meeting standards, institutional visits and monitoring performance. However, the context in which these are implemented matters. We undertook an innovative realist evaluation to test an initial programme theory. Data were analysed using framework analysis.

RESULTS

Across components of the intervention, we identified key mechanisms, including transparent reporting to promote quality improvement; dialogical feedback; partnership working facilitating interactions between regulators and providers, and role clarity in conducting proportionate interventions appropriate to risk. The GMC's framework was commended for being comprehensive and enabling a broad understanding of an organisation's performance. Unintended consequences included confusion over roles and boundaries in different contexts which often undermined effectiveness.

CONCLUSIONS

This realist evaluation substantiates the literature and reveals deeper understandings about quality assuring medical education. While standardised approaches are implemented, interventions need to be contextually proportionate. Routine communication is beneficial to verify data, share concerns and check risk; however, ongoing partnership working can foster assurance. The study provides a modified programme theory to explicate how education providers and regulators can work more effectively together to uphold education quality, and ultimately protect public safety. The findings have influenced the GMC's approach to quality assurance which impacts on all medical students and doctors in training.

PATIENT DOSE SURVEY BASED ON SIZE-SPECIFIC DOSE ESTIMATE AND ACCEPTABLE QUALITY DOSE IN CHEST AND ABDOMEN/PELVIS CT EXAMINATIONS

The practical aspects of two recently developed patient dose optimization methods in computed tomography (CT) examinations, size-specific dose estimate (SSDE) and acceptable quality dose (AQD), were verified for the chest and abdomen/pelvis examinations. A dose survey was performed in a CT institute by considering patients lateral diameter, weight and body mass index (BMI). The AQD tables for weight and BMI groups and SSDE threshold curves were obtained. The mean of volume CT dose index and dose length product for standard-size patients were compared with the national diagnostic reference levels (NDRLs) of Iran. The results show that patient doses are below the NDRLs. It is more reliable to report the AQDs based on SSDE and for BMI groups which can well take into account patient size in the dose optimization process. The SSDE threshold curves can be determined with more precision by including dose data of all possible sizes in the curves.

External Quality Assessment Program for Newborn Screening of Acylcarnitine in China, 2018

BACKGROUND

The aim of this study was to analyze the 2018 external quality assessment (EQA) results of newborn screening by MS/MS of acylcarnitine by Chinese National Center for Clinical Laboratories and to determine the performance of clinical laboratories.

METHODS

Five dried blood spots were distributed to participants every round. Satisfactory performance was defined as scores more than 80 of acceptable results within the evaluation criterion. The robust coefficient of variability (RCV) of each sample was calculated by measurement systems. The chi-square (2) test was used to compare the correct recognition rates.

RESULTS

EQA results were collected from 150 laboratories for 15 different acylcarnitines between C0 - C18. The overall acceptable rates of the qualitative results were between 81.21% and 96.67%, and the proportion of acceptable quantitative results were between 78.38% and 93.24%. There were significant differences in the rates of acceptable quantitative results among different items and between the four methods.

CONCLUSIONS

Most of the participant laboratories had satisfactory performance for the quantitative results in this EQA scheme. But for qualitative assessment, a laboratory should re-evaluate and validate their reference intervals on a regular basis to improve the consistency of clinical assessment.

Development and validation of a comprehensive patient-specific quality assurance program for a novel stereotactic radiation delivery system for breast lesions

PURPOSE

The GammaPod is a dedicated prone breast stereotactic radiosurgery (SRS) machine composed of 25 cobalt-60 sources which rotate around the breast to create highly conformal dose distributions for boosts, partial-breast irradiation, or neo-adjuvant SRS. We describe the development and validation of a patient-specific quality assurance (PSQA) system for the GammaPod.

METHODS

We present two PSQA methods: measurement based and calculation based PSQA. The measurements are performed with a combination of absolute and relative dose measurements. Absolute dosimetry is performed in a single point using a 0.053-cc pinpoint ionization chamber in the center of a polymethylmethacrylate (PMMA) breast phantom and a water-filled breast cup. Relative dose distributions are verified with EBT3 film in the PMMA phantom. The calculation-based method verifies point doses with a novel semi-empirical independent-calculation software.

RESULTS

The average (± standard deviation) breast and target sizes were 1263 ± 335.3 cc and 66.9 ± 29.9 cc, respectively. All ion chamber measurements performed in water and the PMMA phantom agreed with the treatment planning system (TPS) within 2.7%, with average (max) difference of -1.3% (-1.9%) and -1.3% (-2.7%), respectively. Relative dose distributions measured by film showed an average gamma pass rate of 97.0 ± 3.2 when using a 3%/1 mm criteria. The lowest gamma analysis pass rate was 90.0%. The independent calculation software had average agreements (max) with the patient and QA plan calculation of 0.2% (2.2%) and -0.1% (2.0%), respectively.

CONCLUSION

We successfully implemented the first GammaPod PSQA program. These results show that the GammaPod can be used to calculate and deliver the predicted dose precisely and accurately. For routine PSQA performed prior to treatments, the independent calculation is recommended as it verifies the accuracy of the planned dose without increasing the risk of losing vacuum due to prolonged waiting times.

Outcome measures for oncology alternative payment models: practical considerations and recommendations

OBJECTIVES

This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures, which are often underutilized. We also present a set of "core outcome measures" that may be considered in future oncology alternative payment models (APMs).

STUDY DESIGN

Our research consists of a focused literature review, content analysis, and quality measure synthesis and categorization.

METHODS

We conducted a focused literature review to generate key evidence on quality measures in oncology. We studied 7 oncology quality assessment frameworks, encompassing 142 quality metrics, and synthesized recommendations using the Center for Medicare and Medicaid Innovation APM toolkit, focusing on outcome measures.

RESULTS

We present 34 outcome-based oncology quality measures for consideration, which are classified into 5 domains: clinical care (eg, hospital and emergency department visits, treatment effectiveness, mortality), safety (eg, infections, hospital adverse events), care coordination (for hospital and hospice care), patient and caregiver experience, and population health and prevention. Both general and indication-specific outcome measures should be considered in oncology APMs, as appropriate. Utilizing outcome-based measures will require addressing multiple challenges, ranging from risk adjustment to data quality assurance.

CONCLUSIONS

Oncology care will benefit from a more rigorous approach to quality assessment. The success of oncology APMs will require a robust set of quality measures that are relevant to patients, providers, and payers.

The Application of the Toyota Production System LEAN 5S Methodology in the Operating Room Setting

The purpose of the Toyota Production System (TPS) Lean 5S methodology project is to improve the efficiency and effectiveness in a process by eliminating identified process waste of (1) defects (errors), (2) overproduction, (3) waiting, (4) confusion, (5) motion/travel, (6) excess inventory, (7) overprocessing, and (8) human potential. The specific aim of this quality improvement project was to evaluate the impact of the TPS 5S tool process, a problem-solving, space-organizing tool, on distractions and interruptions in the neurosurgery operating room (OR) workflow with a goal to decrease neurosurgery craniotomy infection rates in a neurosurgery OR suite within a 3-month period.

Strengths and weaknesses of national confidential case reviews of maternal and newborn morbidity and mortality

Confidential case reviews have been established as a means to improve the quality of maternity care for several decades. Impacts of these programmes, while difficult to demonstrate, have been observed on maternal and neonatal mortality rates, maternity policy and clinical practice. At a national level, maternal and newborn case review programmes identify messages to improve care through multidisciplinary review of medical records of all, or a sample, of mothers and infants who have died or have a specific morbidity. The major strength of such national programmes is that they allow lessons to be identified to improve care at all levels of the health system from national government policy to individual clinical practice. However, strengthening translation of recommendations into action, whether through a more active link with implementation processes or further development of dissemination strategies grounded in implementation science, is an important continuing focus.

An applicable method for PET/CT image quality assessment and comparison among three PET/CT systems with similar physical performance in cancer patients

PURPOSE

This research proposes a method with specific procedure guideline for clinical PET/CT image quality assessment according to physicians' behavior of image interpretation and explore the relationship between image quality and image systems with similar physical performance.

METHODS

Clinical PET/CT were divided according to body location: brain, chest, abdomen and pelvic cavity. We explored the lesions and suspicious regions where radiologists concerned most through eye-tracker and behavior observation study to generate an assessment checklist. Fifty-five patients who were statistically consistent in age, weight and height were studied. Thirty-seven were scanned with an experimental scanner A and control systems B or C because their clinical pathways required PET/CT examinations at short intervals, the other 18 were scanned with scanners A and C. The grade of every system's PET, CT and PET/CT image performance on the four parts was calculated by subtraction of mean value and variance between experimental and control systems.

RESULTS

The scoring checklist was set for PET, CT and PET/CT images in four parts respectively, and a standard procedure guideline was formulated for assessment. Using assessment criteria, the statistical results objectively reflected certain systems' superiority on certain modalities and certain parts of the body.

CONCLUSION

Our criteria for clinical PET/CT image quality assessment and comparison were efficient.

Characterization and longitudinal assessment of daily quality assurance for an MR-guided radiotherapy (MRgRT) linac

PURPOSE

To describe and characterize daily machine quality assurance (QA) for an MR-guided radiotherapy (MRgRT) linac system, in addition to reporting a longitudinal assessment of the dosimetric and mechanical stability over a 7-month period of clinical operation.

METHODS

Quality assurance procedures were developed to evaluate MR imaging/radiation isocenter, imaging and patient handling system, and linear accelerator stability. A longitudinal assessment was characterized for safety interlocks, laser and imaging isocenter coincidence, imaging and radiation (RT) isocentricity, radiation dose rate and output, couch motion, and MLC positioning. A cylindrical water phantom and an MR-compatible A1SL detector were utilized. MR and RT isocentricity and MLC positional accuracy was quantified through dose measured with a 0.40 cm2  x 0.83 cm2 field at each cardinal angle. The relationship between detector response to MR/RT isocentricity and MLC positioning was established through introducing known errors in phantom position.

RESULTS

Correlation was found between detector response and introduced positional error (N = 27) with coefficients of determination of 0.9996 (IEC-X), 0.9967 (IEC-Y), 0.9968 (IEC-Z) in each respective shift direction. The relationship between dose (DoseMR/RT+MLC ) and the vector magnitude of MLC and MR/RT positional error (Errormag ) was calculated to be a nonlinear response and resembled a quadratic function: DoseMR/RT+MLC [%] = -0.0253 Errormag [mm]2  - 0.0195 Errormag [mm]. For the temporal assessment (N = 7 months), safety interlocks were functional. Laser coincidence to MR was within ±2.0 mm (99.6%) and ±1.0 mm (86.8%) over the 7-month assessment. IGRT position-reposition shifts were within ±2.0 mm (99.4%) and ±1.0 mm (92.4%). Output was within ±3% (99.4%). Mean MLC and MR/RT isocenter accuracy was 1.6 mm, averaged across cardinal angles for the 7-month period.

CONCLUSIONS

The linac and IGRT accuracy of an MR-guided radiotherapy system has been validated and monitored over seven months for daily QA. Longitudinal assessment demonstrated a drift in dose rate, but temporal assessment of output, MLC position, and isocentricity has been stable.

Quantifying the accuracy of deformable image registration for cone-beam computed tomography with a physical phantom

PURPOSE

Kilo-voltage cone-beam computed tomography (CBCT) is widely used for patient alignment, contour propagation, and adaptive treatment planning in radiation therapy. In this study, we evaluated the accuracy of deformable image registration (DIR) for CBCT under various imaging protocols with different noise and patient dose levels.

METHODS

A physical phantom previously developed to facilitate end-to-end testing of the DIR accuracy was used with Varian Velocity v4.0 software to evaluate the performance of image registration from CT to CT, CBCT to CT, and CBCT to CBCT. The phantom is acrylic and includes several inserts that simulate different tissue shapes and properties. Deformations and anatomic changes were simulated by changing the rotations of both the phantom and the inserts. CT images (from a head and neck protocol) and CBCT images (from pelvis, head and "Image Gently" protocols) were obtained with different image noise and dose levels. Large inserts were filled with Mobil DTE oil to simulate soft tissue, and small inserts were filled with bone materials. All inserts were contoured before the DIR process to provide a ground truth contour size and shape for comparison. After the DIR process, all deformed contours were compared with the originals using Dice similarity coefficient (DSC) and mean distance to agreement (MDA). Both large and small volume of interests (VOIs) for DIR volume selection were tested by simulating a DIR process that included whole patient image volume and clinical target volumes (CTV) only (for CTVs propagation).

RESULTS

For cross-modality DIR registration (CT to CBCT), the DSC were >0.8 and the MDA were <3 mm for CBCT pelvis, and CBCT head protocols. For CBCT to CBCT and CT to CT, the DIR accuracy was improved relative to the cross-modality tests. For smaller VOIs, the DSC were >0.8 and MDA <2 mm for all modalities.

CONCLUSIONS

The accuracy of DIR depends on the quality of the CBCT image at different dose and noise levels.

Performance of Laboratories after 10 Years of Participating in External Quality Monitoring in Cervical Cytology

OBJECTIVE

To evaluate the performance of laboratories accredited by the National Health System that perform cytopathology examination of the uterine cervix after 10 years of participation in external quality monitoring (eQM).

STUDY DESIGN

Seven laboratories were assessed in this study. To assess the concordance of the representation of epithelia and results, 6,536 examinations (3,433 in 2007 and 3,103 in 2017) were reviewed. Statistical analysis was performed using the κ coefficient as well as the χ2 and the Fisher exact test (p < 0.05).

RESULTS

All laboratories showed adequate infrastructure and internal quality monitoring. Regarding the representation of the epithelia, the concordance remained excellent (κ between 0.84 and 0.94). In 2007, 26 false-negative results (FN), 157 false-positive results (FP), and 79 delays in clinical conduct (DCC) were identified (κ = 0.80). In 2017, 24 FN, 42 FP, and 38 DCC were identified (κ = 0.90). Five laboratories showed improvement in concordance from 2007 to 2017. We observed significant improvement in cytomorphological criteria of the results: atypical squamous cells of undetermined significance (p < 0.001), atypical squamous cells cannot exclude high-grade squamous intraepithelial lesion (p < 0.016), and low-grade squamous intraepithelial lesion (p < 0.001). We also observed a considerable improvement in the results of cellular abnormalities in glandular epithelium (p < 0.0504).

CONCLUSIONS

Ten years after the implementation of eQM, improvements in the concordance of results were observed as well as reductions in FN, FP, and DCC in all laboratories monitored.

Quality appraisal of clinical practice guidelines for diabetes mellitus published in China between 2007 and 2017 using the AGREE II instrument

OBJECTIVE

The aim of this study was to systematically evaluate the quality of the clinical practice guidelines (CPGs) for diabetes mellitus published in China over the period of January 2007 to April 2017.

METHODS

We searched the China National Knowledge Infrastructure, Chinese Biomedical Literature database, VIP database and WanFang databases and guideline websites for CPGs for diabetes mellitus published between January 2007 and April 2017 in China. Two reviewers independently screened the literature according to the inclusion and exclusion criteria and extracted data. We used the the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool (Canadian Institutes of Health Research, Ottawa, Canada) to evaluate the quality of the included guidelines, calculated the scores of each domain and evaluated the consistency among the assessors via use of the intragroup correlation coefficient. And then we compared the results with Chinese CPGs and international CPGs. We conducted a subgroup analysis based on different classification criteria and compared scores of each domain subgroup analyses.

RESULTS

A total of 98 guidelines were identified. The correlation coefficient within the group was 0.93, suggesting that the consistency between the evaluators was good. The scores of the six domains of AGREE II were described in median (IQR) as follows: scope and purpose 53.7 (50.0-59.7), stakeholder involvement 31.5 (27.3-37.0), rigour of development 19.1 (15.3-22.2), clarity of presentation 59.3 (50.0-64.8), applicability 18.1 (13.9-25.7) and editorial independence 0.0 (0.0-0.0). The mean score in each domain of quality of Chinese diabetes CPGs was lower than that of CPGs published worldwide but higher than the mean score of Chinese guidelines of all topics. A funding source, the updated version, organisation and publishers of the guidelines and target fields are all the factors influencing the quality of CPGs to a certain degree.

CONCLUSIONS

A large number of Chinese diabetes CPGs have been produced. Their quality remain unsatisfactorily low compared with CPGs worldwide, there is still room for improvement. Chinese guideline developers should pay more attention to the transparency of methodology, and use the AGREE II instrument to develop and report guidelines.

Streamlining Regulatory Activities Within Radiation Therapy Departments Using QATrack

Radiation therapy departments are faced with the challenge of tracking numerous quality control tests as well as monitoring service events affecting radiation therapy treatment units. Service events, in particular, pose a challenge since the clinic must be able to provide evidence to the regulatory body that both the service work and any required follow-up tests were recorded and authorized by the appropriate staff. This article presents an integrated approach to tracking quality control tests and service event logs using QATrack+. The newly developed version of this quality assurance software integrates quality control tracking with the service event log, allowing a direct link between a service event and any initiating routine tests or follow-up tests that are performed. This improves the ability of a licensee to ensure compliance with regulations and permits a simple platform from which to access all machine equipment tests and service events. Furthermore, this improves the ability of a department to assess the service record of equipment and to identify trends in failure modes.

A systematic evaluation of the error detection abilities of a new diode transmission detector

Transmission detectors meant to measure every beam delivered on a linear accelerator are now becoming available for monitoring the quality of the dose distribution delivered to the patient daily. The purpose of this work is to present results from a systematic evaluation of the error detection capabilities of one such detector, the Delta⁴ Discover. Existing patient treatment plans were modified through in‐house‐developed software to mimic various delivery errors that have been observed in the past. Errors included shifts in multileaf collimator leaf positions, changing the beam energy from what was planned, and a simulation of what would happen if the secondary collimator jaws did not track with the leaves as they moved. The study was done for simple 3D plans, static gantry intensity modulated radiation therapy plans as well as dynamic arc and volumetric modulated arc therapy (VMAT) plans. Baseline plans were delivered with both the Discover device and the Delta⁴ Phantom+ to establish baseline gamma pass rates. Modified plans were then delivered using the Discover only and the predicted change in gamma pass rate, as well as the detected leaf positions were evaluated. Leaf deviations as small as 0.5 mm for a static three‐dimensional field were detected, with this detection limit growing to 1 mm with more complex delivery modalities such as VMAT. The gamma pass rates dropped noticeably once the intentional leaf error introduced was greater than the distance‐to‐agreement criterion. The unit also demonstrated the desired drop in gamma pass rates of at least 20% when jaw tracking was intentionally disabled and when an incorrect energy was used for the delivery. With its ability to find errors intentionally introduced into delivered plans, the Discover shows promise of being a valuable, independent error detection tool that should serve to detect delivery errors that can occur during radiotherapy treatment.