The Patient's Perspective on the Functioning of the Primary Healthcare Centres in Bangalore, India: An Illustrated Guide

India's healthcare system is, for a large part, organized around a vast network of Primary Healthcare Centres (PHCs) that form the pillar on which the public healthcare sector functions. The World Health Organization (WHO) has emphasized the important role that PHCs play in strengthening community health and the provision of healthcare. Although a few studies have assessed specific elements of services offered by PHCs, only a few have studied the patients' perspectives on the functioning and performance of PHCs in the Indian context. A qualitative research methodology was employed to explore the opinions of 188 patients attending one of three PHCs in Bengaluru (India), using in-depth interviews and thematic analysis. Results showed that patients assessed PHC based on the nine themes that broadly can be classified into components of the center, and that of the personnel. The patients valued the behavioural aspects of the personnel during service delivery and should be configured into the PHC performance.

Tool to assess the groundstroke technique of preadolescent tennis players

Objective

In this study, we develop a tool that can be used by tennis coaches to evaluate the groundstroke (forehand and backhand) technique of preadolescent tennis players.

Methods

The participants of the study were 60 children (30 males and 30 females) aged 10-12 years, with at least two years of training in tennis. The Groundstroke Correction Checklist (GCC) was translated into Turkish by using a blind procedure. The Turkish translation was then evaluated by 15 coaches of the Turkish Tennis Federation who had at least a level-3 coaching license and more than five years of coaching experience. The technical components related to technique in the checklist were labeled as unimportant, important, and very important. Following this, the GCC was converted into a Groundstroke Technique Assessment Test Tool (GTATT) by a selection committee consisting of three experts, and its reliability and validity were assessed by using it in the field. Spearman's correlation was used to analyze the correlation (test-r test) between the technical evaluation scores assigned to the players by the tennis coaches based on the GTATT in the first and second weeks. Intra- and inter-rater reliability was used to analyze the overall scores of technical evaluations in the first and second weeks to assess the reliability of the scale used. We determined each player's number of years of playing experience (TPY), technical evaluation (TE), international tennis-level test score (ITN), I-cord classification order (ICCO), and the number of games won (GW) in a tournament organized among themselves and evaluated the correlations among these parameters by using Spearman's correlation analysis.

Conclusion

A statistically high and significant correlation was observed between the technical evaluations of the players' forehand and backhand groundstrokes by the tennis coaches by using the GTATT in the first and second weeks (r > .90, p < .01). The analysis of the intra- and inter-rater reliability of the GTATT yielded excellent agreement between the technical observations of the three coaches of the players' forehand and backhand strokes in the first and second weeks.

Key Performance Indicators: A Framework for Allied Healthcare Educational Institutions

Background

Performance evaluation in the allied healthcare education sector is complex, making it essential for policymakers and managers to approach it comprehensively and thoughtfully to understand their performance. Hence, the development and monitoring of Key Performance Indicators (KPIs) in this domain must be considered one of the key priorities for the policymakers in AHIs.

Aim

This study aims to develop a framework for the AHIs to extract and profile the indicators, measure, and report the results appropriately.

Methods

The authors adopted a general review of the literature approach to study the primary goals of the institutional KPI framework, emphasizing the need for benchmarking while implementing KPIs and how to track performance using a KPI dashboard.

Results

The study provides the scope, relevant KPI categories, and a list of KPIs for evaluating the effectiveness of allied healthcare programs. The study findings also emphasized the need for benchmarking the KPIs and establishing a KPI dashboard while measuring and monitoring performance.

Conclusion

KPIs are considered an invaluable tool that contributes immensely to the performance monitoring process of AHIs, irrespective of the specialties. This helps to identify and guide AHIs for developing KPIs and the associated minimum data set to measure organizational performance and monitor the quality of teaching and learning. In addition, the KPI framework reported in this study is a tool to assist performance monitoring that can subsequently contribute to the overall quality of AHIs.

Further Validity Evidence for Patient-Specific Virtual Reality Temporal Bone Surgical Simulation

OBJECTIVE

Patient-specific virtual reality (VR) simulation of cochlear implant (CI) surgery potentially enables preoperative rehearsal and planning. We aim to gather supporting validity evidence for patient-specific simulation through the analysis of virtual performance and comparison with postoperative imaging.

METHODS

Prospective, multi-institutional study. Pre- and postoperative cone-beam CT scans of CI surgical patients were obtained and processed for patient-specific VR simulation. The virtual performances of five trainees and four attendings were recorded and (1) compared with volumes removed during actual surgery as determined in postoperative imaging, and (2) assessed using the Copenhagen Cochlear Implant Surgery Assessment Tool (CISAT) by two blinded raters. The volumes compared were cortical mastoidectomy, facial recess, and round window (RW) cochleostomy as well as violation of the facial nerve and chorda.

RESULTS

Trainees drilled more volume in the cortical mastoidectomy and facial recess, whereas attendings drilled more volume for the RW cochleostomy and made more violations. Except for the cochleostomy, attendings removed volumes closer to that determined in postoperative imaging. Trainees achieved a higher CISAT performance score compared with attendings (22.0 vs. 18.4 points) most likely due to lack of certain visual cues.

CONCLUSION

We found that there were differences in performance of trainees and attendings in patient-specific VR simulation of CI surgery as assessed by raters and in comparison with actual drilled volumes. The presented approach of volume comparison is novel and might be used for further validation of patient-specific VR simulation before clinical implementation for preoperative rehearsal in temporal bone surgery.

LEVEL OF EVIDENCE

n/a Laryngoscope, 134:1403-1409, 2024.

Speed Effects on the Accuracy of Heart Rate as Oxygen-Uptake Indicator in Short-Distance Shuttle Running

Purpose: Despite the accuracy of heart rate (HR) as an indicator of the aerobic engagement has been evaluated in several intermittent on-court activities, its validity as an oxygen uptake (V ˙ O 2 ) indicator during shuttle running over short paths remains uncertain. Moreover, it is unclear whether speed may affect such validity. This study evaluated the HR ability in estimating the V ˙ O 2 during 5-m shuttle running at different speeds. Methods: V ˙ O 2 and HR of 12 physically active young men were recorded during an incremental forward running (FW) protocol and a 5-m shuttle test at 50%, 60%, and 75% of maximal aerobic speed (MAS). Slope and intercept of the relationship between HR and V ˙ O 2 (HR/V ˙ O 2 ) were individually determined, in both protocols. The HR measured during the shuttle test was used in the FW HR/V ˙ O 2 to estimate V ˙ O 2 at each shuttle speed. A paired Student's t-test compared slopes and intercepts of the two HR/V ˙ O 2 . A two-way RM-ANOVA and an equality test examined, respectively, the differences and the equality between measured and estimated V ˙ O 2 . Lastly, a Bland-Altman plot described the accuracy and precision of the estimated V ˙ O 2 at each shuttle intensity. Results: Slopes and intercepts of the HR/V ˙ O 2 appeared not different between FW and shuttle running. At 50%MAS, HR underestimated the V ˙ O 2 (~7%), whereas returned accurate values at the two higher velocities, although with high variability (±18%). Conclusions: When using HR as V ˙ O 2 indicator during shuttle running over short paths, a separated analysis of the HR validity as V ˙ O 2 indicator is recommended especially when administering different exercise intensities.

Test-Retest Reliability of Immersive Virtual Reality Measures of Perceptual-Motor Performance

The duration, accuracy, and consistency of responses to various types of stimuli are widely accepted as indirect indicators of the efficiency of brain information processing, but current clinical tests appear to lack sufficient sensitivity to detect subtle impairments. Immersive virtual reality (VR) offers a new means to acquire measures of perceptual-motor responses to moving visual stimuli that require rapid conflict resolution, but their test-retest reliability has not yet been demonstrated. Repeated measures. We analyzed data from 19 healthy young adults who performed a 40-trial VR test on three consecutive days. We focused on response time (RT) and perceptual latency (PL) for eye, neck, arm, and whole-body step displacements involved in executing a reaching/lunging movement in a right or left direction toward a peripherally located virtual target. Measures of RT and PL included a 40-trial mean, an intra-individual variability (IIV) value, and a rate correct score (RCS) that incorporated both response duration and accuracy. Most mean and IIV values for PL and RT demonstrated a positive distributional skew that was substantially reduced by natural logarithm transformation. While a learning effect was evident between sessions 1 and 2 for 7 of 8 mean PL and RT measures, 3-session intraclass correlation coefficient (ICC) values were moderate to excellent for 15 of 16 transformed PL and RT measures (range: .618 to .922). The composite RCS metric did not require transformation for either PL or RT, whose respective 3-session ICC values were .877 and .851. This moderate to excellent test-retest reliability for various VR measures of perceptual-motor function, combined with evidence of their validity from both past and future research, suggest that these measures can advance clinical detection of impaired brain processing and longitudinal assessments of potentially modifiable performance deficiencies.

Performance Evaluation of Urban Emergency Governance: Conceptual Connotations and Empirical Research Based on the Crisis Life Cycle Theory

Although scholars have discussed in depth about the evaluation content and the construction of evaluation index system of emergency management and crisis management, a unified and standardized interpretation of system construction concepts and empirical research on the performance evaluation of urban emergency management is still lacking. In view of this, this paper is based on the theory of the  crisis life cycle, with the four phases of urban emergency management and the content of the task as a clue, constructed a performance evaluation index system containing 12 primary indicators and 44 secondary indicators, and centered on the emergency management work situation of S city. The study reveals that emergency management underperforms in S city, with a score of 5.948, qualifying as the "defined level". The "prevention and preparedness stage" receives poor evaluation, indicating a significant bias for crisis response instead of prevention. The report suggests the improvement of emergency material stockpile planning, the formation of grassroots emergency response teams, the emergency management publicity, and special emergency drill plans, etc., with a view to providing reference for city managers to comprehensively review and evaluate the emergency management system, and to promote the construction of a more systematic, complete and scientific urban emergency management system.

Exploring objective measures for assessing team performance in healthcare: an interview study

Introduction

Effective teamwork plays a critical role in achieving high-performance outcomes in healthcare. Consequently, conducting a comprehensive assessment of team performance is essential for providing meaningful feedback during team trainings and enabling comparisons in scientific studies. However, traditional methods like self-reports or behavior observations have limitations such as susceptibility to bias or being resource consuming. To overcome these limitations and gain a more comprehensive understanding of team processes and performance, the assessment of objective measures, such as physiological parameters, can be valuable. These objective measures can complement traditional methods and provide a more holistic view of team performance. The aim of this study was to explore the potential of the use of objective measures for evaluating team performance for research and training purposes. For this, experts in the field of research and medical simulation training were interviewed to gather their opinions, ideas, and concerns regarding this novel approach.

Methods

A total of 34 medical and research experts participated in this exploratory qualitative study, engaging in semi-structured interviews. During the interview, experts were asked for (a) their opinion on measuring team performance with objective measures, (b) their ideas concerning potential objective measures suitable for measuring team performance of healthcare teams, and (c) their concerns regarding the use of objective measures for evaluating team performance. During data analysis responses were categorized per question.

Results

The findings from the 34 interviews revealed a predominantly positive reception of the idea of utilizing objective measures for evaluating team performance. However, the experts reported limited experience in actively incorporating objective measures into their training and research. Nevertheless, they identified various potential objective measures, including acoustical, visual, physiological, and endocrinological measures and a time layer. Concerns were raised regarding feasibility, complexity, cost, and privacy issues associated with the use of objective measures.

Discussion

The study highlights the opportunities and challenges associated with employing objective measures to assess healthcare team performance. It particularly emphasizes the concerns expressed by medical simulation experts and team researchers, providing valuable insights for developers, trainers, researchers, and healthcare professionals involved in the design, planning or utilization of objective measures in team training or research.

Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model

Rating scale analysis techniques provide researchers with practical tools for examining the degree to which ordinal rating scales (e.g., Likert-type scales or performance assessment rating scales) function in psychometrically useful ways. When rating scales function as expected, researchers can interpret ratings in the intended direction (i.e., lower ratings mean "less" of a construct than higher ratings), distinguish between categories in the scale (i.e., each category reflects a unique level of the construct), and compare ratings across elements of the measurement instrument, such as individual items. Although researchers have used these techniques in a variety of contexts, studies are limited that systematically explore their sensitivity to problematic rating scale characteristics (i.e., "rating scale malfunctioning"). I used a real data analysis and a simulation study to systematically explore the sensitivity of rating scale analysis techniques based on two popular polytomous item response theory (IRT) models: the partial credit model (PCM) and the generalized partial credit model (GPCM). Overall, results indicated that both models provide valuable information about rating scale threshold ordering and precision that can help researchers understand how their rating scales are functioning and identify areas for further investigation or revision. However, there were some differences between models in their sensitivity to rating scale malfunctioning in certain conditions. Implications for research and practice are discussed.

The science and art of testing in ice hockey: a systematic review of twenty years of research

Introduction

Ice hockey is a complex sport requiring multiple athletic and technical attributes. Considering the variety of tests developed, on-ice testing protocols have been created to measure the physiological and mechanical attributes associated with performance. To our knowledge, a lack of technical resources exists to help stakeholders opt for on-ice protocols from among those developed. It becomes crucial for researchers and practitioners to select relevant and context-specific procedures. This systematic review of the literature outlines an inventory of the on-ice tests that have been used in the domain of ice hockey research over the last twenty years, and summarize protocols mostly used in major athletic components.

Methods

A search was performed on three databases (PubMed, SPORTDiscus and Scopus) by following the PRISMA guidelines. Specific keywords were selected to find publications using on-ice testing protocols in the methodology. Four aspects of athletic attributes were used to categorize the protocols: aerobic capacity, acceleration-speed, agility-change of direction and ability to repeat skating sprints. Analyses were conducted regarding four categories of observations: population under study, on-ice reported test(s), outcomes measures and main findings.

Results

A total of 107 articles were included, resulting in 55 on-ice tests related to the on-ice assessments of four major athletic components: aerobic capacity (n = 7), acceleration-speed (n = 6), agility and change of direction (n = 23) and repeated skating sprint ability (n = 19). Testing in male and older cohorts (≥16 years old) predominates, with a primary focus on the competitive amateur level. The selected tests were mainly designed for assessing on-ice physiological responses and fitness (n = 38), talent identification-team selection (n = 19), efficiency of interventions (n = 17) and validation purposes (n = 16).

Conclusion

A prevalence of on-ice skating tests to assess the ability to repeat intense efforts, agility, acceleration and speed components exists, which are relevant and linked to match requirement. The wealth of on-ice tests used in the literature reflects the need to adapt the on-ice evaluation process to the population, constraints, and goals. This review is a valid toolbox and can benefit for researchers and practitioners interested in testing hockey players from different levels, with a variety of aims and needs, by helping them to select the relevant procedures to their environment and practice context.

Does Sparseness Matter? Examining the Use of Generalizability Theory and Many-Facet Rasch Measurement in Sparse Rating Designs

Sparse rating designs, where each examinee's performance is scored by a small proportion of raters, are prevalent in practical performance assessments. However, relatively little research has focused on the degree to which different analytic techniques alert researchers to rater effects in such designs. We used a simulation study to compare the information provided by two popular approaches: Generalizability theory (G theory) and Many-Facet Rasch (MFR) measurement. In previous comparisons, researchers used complete data that were not simulated-thus limiting their ability to manipulate characteristics such as rater effects, and to understand the impact of incomplete data on the results. Both approaches provided information about rating quality in sparse designs, but the MFR approach highlighted rater effects related to centrality and bias more readily than G theory.

Narrowing the gap: expected versus deployment performance

OBJECTIVES

Successful model development requires both an accurate a priori understanding of future performance and high performance on deployment. Optimistic estimations of model performance that are unrealized in real-world clinical settings can contribute to nonuse of predictive models. This study used 2 tasks, predicting ICU mortality and Bi-Level Positive Airway Pressure failure, to quantify: (1) how well internal test performances derived from different methods of partitioning data into development and test sets estimate future deployment performance of Recurrent Neural Network models and (2) the effects of including older data in the training set on models' performance.

MATERIALS AND METHODS

The cohort consisted of patients admitted between 2010 and 2020 to the Pediatric Intensive Care Unit of a large quaternary children's hospital. 2010-2018 data were partitioned into different development and test sets to measure internal test performance. Deployable models were trained on 2010-2018 data and assessed on 2019-2020 data, which was conceptualized to represent a real-world deployment scenario. Optimism, defined as the overestimation of the deployed performance by internal test performance, was measured. Performances of deployable models were also compared with each other to quantify the effect of including older data during training.

RESULTS, DISCUSSION, AND CONCLUSION

Longitudinal partitioning methods, where models are tested on newer data than the development set, yielded the least optimism. Including older years in the training dataset did not degrade deployable model performance. Using all available data for model development fully leveraged longitudinal partitioning by measuring year-to-year performance.

MISM: A Medical Image Segmentation Metric for Evaluation of Weak Labeled Data

Performance measures are an important tool for assessing and comparing different medical image segmentation algorithms. Unfortunately, the current measures have their weaknesses when it comes to assessing certain edge cases. These limitations arise when images with a very small region of interest or without a region of interest at all are assessed. As a solution to these limitations, we propose a new medical image segmentation metric: MISm. This metric is a composition of the Dice similarity coefficient and the weighted specificity. MISm was investigated for definition gaps, an appropriate scoring gradient, and different weighting coefficients used to propose a constant value. Furthermore, an evaluation was performed by comparing the popular metrics in the medical image segmentation and MISm using images of magnet resonance tomography from several fictitious prediction scenarios. Our analysis shows that MISm can be applied in a general way and thus also covers the mentioned edge cases, which are not covered by other metrics, in a reasonable way. In order to allow easy access to MISm and therefore widespread application in the community, as well as reproducibility of experimental results, we included MISm in the publicly available evaluation framework MISeval.

Assessment of the Study Habits of Residents in Physical Medicine and Rehabilitation Programs in Saudi Arabia: A Cross-Sectional Study

Background

Residents in training must employ a variety of study strategies, as they not only participate in academic studies but also interact with patients. This study aimed to evaluate the study practices and factors affecting those practices among Saudi Arabian physical medicine and rehabilitation residents during their residency program.

Methods

In this cross-sectional study, a previously used questionnaire was distributed to Saudi Arabian physiatry residents from July 1 to August 15, 2022, via a social media platform and completed using a Google Forms survey. A Microsoft Excel spreadsheet was used to collect, clean, and import the data before IBM SPSS Statistics for Windows, version 22.0 was utilized for statistical analysis.

Results

The data of 94.91% of respondents were included in the analysis. Individuals who were female, unmarried or divorced, and without children predominated. Only 17.9% (n = 10) of the residents believed that their training program effectively prepared them to pass the board examination, which was the most strongly motivating factor for studying for 85.7% of respondents. Over two-thirds of the residents mentioned that they regularly exercise. Residents who studied more than 11 hours per week had a significantly lower score in the category of factors that negatively affect examination performance (M = 12.33 ± 2.82, F = 2.794, P < 0.05). Females, final-year residents, and Riyadh residents studied more than their counterparts.

Conclusion

Our study is the first to investigate how Saudi physiatrists study, with the finding that current physiatry residents employ a combination of traditional and contemporary learning strategies. This information can help stakeholders to understand current training challenges, improve the quality of training for physiatry residents, and create an ideal learning environment.

Introducing value-based healthcare perspectives into hospital performance assessment: A scoping review

OBJECTIVE

Value-based healthcare (VBHC) puts patient outcomes at the center of the healthcare process while optimizing the use of hospital resources across multiple stakeholders. This scoping review was conducted to summarize how VBHC had been represented in theory and in practice, how it had been applied to assess hospital performance, and how well it had been ultimately implemented.

METHODS

For this review, we followed the PRISMA-ScR protocol and searched five major online databases for articles published between January 2006 and July 2022. We included original articles that used the concept of VBHC to conduct performance assessments of healthcare organizations. We extracted and analyzed key concepts and information on the dimensions of VBHC, specific strategies and methods for using VBHC in performance assessment, and the effectiveness of the assessment.

RESULTS

We identified 48 eligible studies from 7866 articles. Nineteen nonempirical studies focused on the development of a VBHC performance assessment indicator system, and 29 empirical studies reported on the ways and points of introducing VBHC into performance assessment and its effectiveness. Ultimately, we summarized the key dimensions, processes, and effects of performance assessment after introducing VBHC.

CONCLUSION

Current healthcare performance assessment has begun to focus on implementing VBHC as an integrated strategy, and future work should further clarify the reliability of metrics and their association with evaluation outcomes and consider the effective integration of clinical outcomes and patient-reported outcomes.

Investigating the Impact of Two Major Programming Environments on the Accuracy of Deep Learning-Based Glioma Detection from MRI Images

Brain tumors have been the subject of research for many years. Brain tumors are typically classified into two main groups: benign and malignant tumors. The most common tumor type among malignant brain tumors is known as glioma. In the diagnosis of glioma, different imaging technologies could be used. Among these techniques, MRI is the most preferred imaging technology due to its high-resolution image data. However, the detection of gliomas from a huge set of MRI data could be challenging for the practitioners. In order to solve this concern, many Deep Learning (DL) models based on Convolutional Neural Networks (CNNs) have been proposed to be used in detecting glioma. However, understanding which CNN architecture would work efficiently under various conditions including development environment or programming aspects as well as performance analysis has not been studied so far. In this research work, therefore, the purpose is to investigate the impact of two major programming environments (namely, MATLAB and Python) on the accuracy of CNN-based glioma detection from Magnetic Resonance Imaging (MRI) images. To this end, experiments on the Brain Tumor Segmentation (BraTS) dataset (2016 and 2017) consisting of multiparametric magnetic MRI images are performed by implementing two popular CNN architectures, the three-dimensional (3D) U-Net and the V-Net in the programming environments. From the results, it is concluded that the use of Python with Google Colaboratory (Colab) might be highly useful in the implementation of CNN-based models for glioma detection. Moreover, the 3D U-Net model is found to perform better, attaining a high accuracy on the dataset. The authors believe that the results achieved from this study would provide useful information to the research community in their appropriate implementation of DL approaches for brain tumor detection.

A Multimodal Intervention for Reducing Unnecessary Repeat Glycated Hemoglobin Testing

OBJECTIVES

Reducing unnecessary tests that do not enhance quality can promote health-care value. Glycated hemoglobin (A1C) is often ordered at a frequency exceeding the recommendation of once every 3 months. We conducted a quality improvement (QI) initiative aimed to reduce unnecessary repeat testing by 75% at a tertiary care academic hospital.

METHODS

A retrospective baseline analysis was conducted on laboratory data from 2019 that enumerated unnecessary A1C tests, defined as repeat tests ordered within 60 days. A multifaceted change intervention with iterative plan-do-study-act cycles was introduced in March 2019 to educate providers and to automatically cancel A1C tests requested within 60 days. Monthly totals of A1C testing processed were plotted on statistical process control charts.

RESULTS

In 2019, 11% of all A1C tests ordered were unnecessary. Between March 2020 and January 2021, 11% of the tests (N=14,247 tests) were unnecessary, of which 84% were cancelled with our intervention. Providers in cardiology and nephrology accounted for over half (55%) of the unnecessary tests ordered.

CONCLUSIONS

A 2-pronged approach informed by root-cause analysis, and comprised of gatekeeping and provider education, can effectively promote resource stewardship for reducing unnecessary A1C testing.

Application of "Behind the Barriers" Model at Neighbourhood Scale to Improve Water Management under Multi-Risks Scenarios: A Case Study in Lyon, France

In modern urban areas, water management highly depends on the socio-ecological urban water cycle (UWC) that heavily relies on water infrastructures. However, increasing water-related hazards, natural and/or human-based, makes it difficult to balance water resources in the socio-ecological UWC. In the last decade, urban infrastructure resilience has rapidly become a popular topic in disaster risk management and inspired many studies and operational approaches. Among these theories and methods, the "Behind the Barriers" model (BB model), developed by Barroca and Serre in 2013, is considered a theory that allows effective and comprehensive analysis of urban infrastructure resilience through cognitive, functional, correlative, and organisational dimensions. Moreover, this analysis can be a reference to develop actions that improve infrastructure resilience under critical scenarios. Therefore, this study aims to study resilience design actions based on the BB model to achieve socio-ecological water balance and assess the performance of these actions. The study focuses on water management on a neighbourhood scale, which is considered the essential urban unit to study and improve the resilience of critical infrastructures, such as water services. The Part-Dieu neighbourhood in Lyon, France is selected as a case study, and it highlights the need to develop indicators to assess the performance of implemented actions in a structural and global resilience framework, to understand urban systems as complex and dynamic systems to provide decision support, and to strengthen crisis prevention and management perspectives in a dynamic approach.

A method to benchmark the balance resilience of robots

Robots that work in unstructured scenarios are often subjected to collisions with the environment or external agents. Accordingly, recently, researchers focused on designing robust and resilient systems. This work presents a framework that quantitatively assesses the balancing resilience of self-stabilizing robots subjected to external perturbations. Our proposed framework consists of a set of novel Performance Indicators (PIs), experimental protocols for the reliable and repeatable measurement of the PIs, and a novel testbed to execute the protocols. The design of the testbed, the control structure, the post-processing software, and all the documentation related to the performance indicators and protocols are provided as open-source material so that other institutions can replicate the system. As an example of the application of our method, we report a set of experimental tests on a two-wheeled humanoid robot, with an experimental campaign of more than 1100 tests. The investigation demonstrates high repeatability and efficacy in executing reliable and precise perturbations.

An Objective Structured Laboratory Animal Science Examination (OSLASE) to ensure researchers' professional competence in laboratory animal science

The evaluation of the competence of personnel working with laboratory animals is currently a challenge. Directive 2010/63/EU establishes that staff must have demonstrated competence before they perform unsupervised work with living animals. Nevertheless, there is a lack of research into education and training in laboratory animal science, and the establishment of assessment strategies to confirm researchers' competence remains largely unaddressed.In this study, we analysed the implementation of a practical assessment strategy over three consecutive years (2018-2021) using the Objective Structured Laboratory Animal Science Exam (OSLASE) developed previously by us to assess professional competence. The interrater reliability (IRR) was determined based on the assessors' rating of candidates' performance at different OSLASE stations using weighted kappa (Kw) and percentage of agreement. Focus group interviews were conducted to access trainees' acceptability regarding the OSLASE.There was a moderate-to-good Kw for the majority of the scales' items (0.79 ± 0.20 ≤ Kw ≥ 0.45 ± 0.13). The percentages of agreement were also acceptable (≥75%) for all scale items but one. Trainees reported that the OSLASE had a positive impact on their engagement during practical training, and that it clarified the standards established for their performance and the skills that required improvement. These preliminary results illustrate how assessment strategies, such as the OSLASE, can be implemented in a manner that is useful for both assessors and trainees.Examen structuré objectif de science animale de laboratoire (OSASSE) pour assurer la compétence professionnelle des chercheurs en SAL.

Cost-Effectiveness Analysis in Performance Assessments: A Case Study of the Objective Structured Clinical Examination

Medical education assessments are becoming more complex, resulting in the inappropriateness of traditional methods primarily consisting of direct observations, oral examinations, and multiple-choice tests. Advancements in research methods have led to the formation of new modalities, namely performance assessments, which are, on the other hand, always costly in development and implementation. Proposing using the Program Effectiveness and Cost Generalization flow within an assessment context (PRECOG-A), this brief report explores the real financial cost drivers associated with an assessment case in the context of medical education, presents the steps in bridging the effectiveness with its psychometric properties via cost-effectiveness analysis, and evaluates the two-side outcomes for further evaluation decision-making. Referentially providing a framework to investigators and researchers, the illustration of PRECOG-A in this study outlines instructional guidelines for conducting cost-effectiveness analysis in a performance assessment.

Developing Situated Measures of Science Instruction Through an Innovative Electronic Portfolio App for Mobile Devices: Reliability, Validity, and Feasibility

Adoption of new instructional standards in science demands high-quality information about classroom practice. Teacher portfolios can be used to assess instructional practice and support teacher self-reflection anchored in authentic evidence from classrooms. This study investigated a new type of electronic portfolio tool that allows efficient capture of classroom artifacts in multimedia formats using mobile devices. We assess the psychometric properties of measures of quality instruction in middle school science classrooms derived from the contents of portfolios collected using this novel tool-with instruction operationalized through dimensions aligned to the Next Generation Science Standards. Results reflect low rater error and adequate reliability for several dimensions, a dominant underlying factor, and significant relations to some relevant concurrent indicators. Although no relation was found to student standardized test scores or course grades, portfolio ratings did relate to student self-efficacy perceptions and enjoyment of science. We examine factors influencing measurement error, and consider the broader implications of the results for assessing the validity of portfolio score interpretations, and the feasibility and potential value of this type of tool for summative and formative uses, in the context of large-scale instructional improvement efforts.

A Multi-Criteria Approach to Assess the Performance of the Brazilian Unified Health System

Brazil's Unified Health System (SUS) provides universal free access to health services and is considered a model for the rest of the world. One of the tools used by the Brazilian government to assess this system is the Index of Unified Health System Performance (IDSUS). However, this method has a number of limitations, such as disregarding the opinion of healthcare decision makers. Thus, the aim of the present study was to propose a model based on the R-TOPSIS in order to assess the performance of the SUS. Methodologically, the main steps for proposing multi-criteria models were followed, and to validate the model, a real case study with a set of six cities (alternatives) of the state of São Paulo was used. The results provide a clearer picture of the differences in terms of potential and obtained access, as well as the effectiveness of health services in the cities analyzed. Likewise, the proposal of integrating multiple criteria as well as considering healthcare decision makers proved to be decisive for the results obtained, even in comparison with the other approaches. It was concluded that the proposed method provides a robust and adequate analysis of health systems performance.

Dunning-Kruger Effect Between Self-Peer Ratings of Surgical Performance During a MASCAL Event and Pre-Event Assessed Trauma Procedural Capabilities

Objectives

The research question asked to what extent do self-rated performance scores of individual surgeons correspond to assessed procedural performance abilities and to peer ratings of procedural performance during a mass casualty (MASCAL) event?

Background

Self-assessment using performance rating scales is ubiquitous in surgical education as a proxy for direct measurement of competence. The validity and reliability of self-ratings as competency measures are susceptible to cognitive biases such as Dunning-Kruger effects, which describe how individuals over/underestimate their own performance compared to assessments from independent sources. The ability of surgeons to accurately self-assess their procedural performance remains undetermined.

Methods

A purposive sample of military surgeons (N = 13) who collectively cared for trauma patients during a MASCAL event participated in the study. Pre-event performance assessment scores for 32 trauma procedures were compared with post-event self and peer performance ratings using F tests (P < 0.05) and effect sizes (Cohen's d).

Results

There were no significant differences between peer ratings and performance assessment scores. There were significant differences between self-ratings and both peer ratings (P < 0.001) and performance assessment scores (P < 0.001). Effect sizes were very large for self to peer rating comparison (Cohen's d = 2.34) and self to performance assessment comparison (Cohen's d = 2.77).

Conclusions

The outcomes demonstrate that self-ratings were significantly lower than the independently determined assessment scores for each surgeon, revealing a Dunning-Kruger effect for highly skilled individuals underestimating their abilities. These outcomes underscore the limitations of self-assessment for measuring competence.

Performance metric curve analysis framework to assess impact of the decision variable threshold, disease prevalence, and dataset variability in two-class classification

Purpose: The aim of this study is to (1) demonstrate a graphical method and interpretation framework to extend performance evaluation beyond receiver operating characteristic curve analysis and (2) assess the impact of disease prevalence and variability in training and testing sets, particularly when a specific operating point is used. Approach: The proposed performance metric curves (PMCs) simultaneously assess sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and the 95% confidence intervals thereof, as a function of the threshold for the decision variable. We investigated the utility of PMCs using six example operating points associated with commonly used methods to select operating points (including the Youden index and maximum mutual information). As an example, we applied PMCs to the task of distinguishing between malignant and benign breast lesions using human-engineered radiomic features extracted from dynamic contrast-enhanced magnetic resonance images. The dataset had 1885 lesions, with the images acquired in 2015 and 2016 serving as the training set (1450 lesions) and those acquired in 2017 as the test set (435 lesions). Our study used this dataset in two ways: (1) the clinical dataset itself and (2) simulated datasets with features based on the clinical set but with five different disease prevalences. The median and 95% CI of the number of type I (false positive) and type II (false negative) errors were determined for each operating point of interest. Results: PMCs from both the clinical and simulated datasets demonstrated that PMCs could support interpretation of the impact of decision threshold choice on type I and type II errors of classification, particularly relevant to prevalence. Conclusion: PMCs allow simultaneous evaluation of the four performance metrics of sensitivity, specificity, PPV, and NPV as a function of the decision threshold. This may create a better understanding of two-class classifier performance in machine learning.

Representativeness of Workplace-Based Operative Performance Assessments for Resident Operative Experience

OBJECTIVE

Workplace-based assessment is increasingly prevalent in surgical education, especially for assessing operative skill. With current implementations, not all observed clinical performances are assessed, in part because trainees often have discretion about when they seek assessment. As a result, these samples of observed operative performances may not be representative of the full breadth of experience of surgical trainees. Therefore, analyses of these samples may be biased. We aimed to benchmark patterns of procedures logged in the SIMPL operative performance assessment system against records of trainee experience in Accreditation Council for Graduate Medical Education (ACGME) case logs.

DESIGN

We analyzed SIMPL longitudinal intraoperative performance assessments from categorical trainees in US general surgery residency programs. We compared overall patterns of how procedures are logged in SIMPL and in ACGME case logs using a Pearson correlation, and we examined differences in how individual procedures are logged in each system using Fisher's exact test.

RESULTS

Total procedure frequency from the SIMPL dataset was strongly correlated with total procedure frequency from ACGME case logs (r = 0.86, 95% CI 0.80-0.90). A subset of these procedures (10 of 116 procedures) was logged more frequently in the SIMPL dataset. These 10 procedures accounted for 56% of SIMPL observations and 30% of ACGME logged cases. Case complexity was comparable for assessments initiated by residents and faculty.

CONCLUSIONS

Samples of intraoperative performance ratings gathered using the SIMPL application largely resemble ACGME case logs. There is no evidence to indicate that residents preferentially select fewer complex cases for assessment.

Assessment of countermovement jump with and without arm swing using a single inertial measurement unit

The countermovement vertical jump height, flight time, and jump duration are used to assess athletic performance. Force-plate and motion-capture cameras are used to estimate these parameters, yet, their application is limited to dedicated lab environments. Despite the potential of inertial measurement units (IMU) for estimating the jump height, their accuracy has not been validated. This study investigates the accuracy of our proposed method to estimate the jump height using a sacrum-mounted IMU, during countermovement jumping. Eleven individuals performed four jumps each. To obtain the jump height, we transformed the IMU readouts into anatomical planes, and double-integrated the vertical acceleration after correction for zero velocity and vertical displacement. The accuracy of jump height obtained by IMU was compared to force-plate and motion-capture cameras during jumps without arm swing (mean error (standard deviation) of 0.3(2.2) cm and 1.0(3.0) cm, and correlation coefficient of 0.83 and 0.82, respectively) and during jumps with arm swing (-1.1(2.1) cm and 0.5(1.9) cm, and 0.92 and 0.89). The correlation coefficients were high, and the errors were comparable to the difference between the jump height obtained by force-plate and cameras. Therefore, a sacrum-mounted IMU can be recommended for in-field assessment of countermovement jump with and without arm swing.

Comparative analysis of machine learning-based approaches for identifying therapeutic peptides targeting SARS-CoV-2

Coronavirus disease 2019 (COVID-19) has impacted public health as well as societal and economic well-being. In the last two decades, various prediction algorithms and tools have been developed for predicting antiviral peptides (AVPs). The current COVID-19 pandemic has underscored the need to develop more efficient and accurate machine learning (ML)-based prediction algorithms for the rapid identification of therapeutic peptides against severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). Several peptide-based ML approaches, including anti-coronavirus peptides (ACVPs), IL-6 inducing epitopes and other epitopes targeting SARS-CoV-2, have been implemented in COVID-19 therapeutics. Owing to the growing interest in the COVID-19 field, it is crucial to systematically compare the existing ML algorithms based on their performances. Accordingly, we comprehensively evaluated the state-of-the-art IL-6 and AVP predictors against coronaviruses in terms of core algorithms, feature encoding schemes, performance evaluation metrics and software usability. A comprehensive performance assessment was then conducted to evaluate the robustness and scalability of the existing predictors using well-constructed independent validation datasets. Additionally, we discussed the advantages and disadvantages of the existing methods, providing useful insights into the development of novel computational tools for characterizing and identifying epitopes or ACVPs. The insights gained from this review are anticipated to provide critical guidance to the scientific community in the rapid design and development of accurate and efficient next-generation in silico tools against SARS-CoV-2.

Plutonium Disposition: Using and Explaining Complex Risk-Related Methods

Applying risk assessment and management tools to plutonium disposition is a long-standing challenge for the U.S. government. The science is complicated, which has helped push risk assessment and management tools in new creative directions. Yet, communicating effectively about increasingly complicated risk-science issues like plutonium disposition requires careful planning and speakers who can address why specific tools are selected, the past record of applying these tools, why assumptions sometimes are applied instead of reliable data, and how uncertainty is characterized. Speakers addressing risk issues must also overcome obstacles in communication arising from expert-audience differences in knowledge and legal restrictions on disclosing information. This perspective seeks to highlight and illustrate five key risk questions, about probabilistic risk assessment (PRA) and performance assessment (PA) in the context of managing plutonium defense nuclear waste: objectives, experience, gaps, transparency, and difficulty of applying and communicating using each tool. While the general public needs to be involved, some issues require a level of expertise that is typically beyond local communities and therefore an expert panel should support community access.

Does Experience Make Hucul Horses More Resistant to Stress? A Pilot Study

The aim of the study was to determine whether experience related to horse age and number of starts in championships influences stress level, measured by salivary cortisol concentration. The study involved 18 clinically healthy Hucul mares who participated in the Polish Championships for Hucul Horses. Evaluation of performance value was carried out in accordance with the guidelines specified in the breeding and genetic resources conservation program for this breed. The championship lasted two days, consisting of conformation evaluation, endurance, and Hucul path. Saliva was collected at baseline (T1), after arena assessment (T2), after endurance (T3), and on the second day after the Hucul path (T4). Cortisol levels increased from an average of 2.73 ± 1.18 ng/mL (T1) to 10.46 ± 8.03 ng/mL after T3. Significantly lower levels of free cortisol were detected in the saliva of the younger mares, up to 9 years old, and mares who participated in only one qualifying path after each element of the championship. The highest levels of cortisol (T3) were found in mares competing repeatedly on the qualifying path. No correlation was found between cortisol levels and the championship results. Participation of mares in the championship was associated with stress, which was reflected in the increase in cortisol levels in saliva.

Whole-Body Reactive Agility Metrics to Identify Football Players With a Core and Lower Extremity Injury Risk

Clinical prediction models are useful in addressing several orthopedic conditions with various cohorts. American football provides a good population for attempting to predict injuries due to their relatively high injury rate. Physical performance can be assessed a variety of ways using an assortment of different tests to assess a diverse set of metrics, which may include reaction time, speed, acceleration, and deceleration. Asymmetry, the difference between right and left performance has been identified as a possible risk factor for injury. The purpose of this study was to determine the whole-body reactive agility metrics that would identify Division I football players who were at elevated risk for core, and lower extremity injuries (CLEI). This cohort study utilized 177 Division I football players with a total of 57 CLEI suffered who were baseline tested prior to the season. Single-task and dual-task whole-body reactive agility movements in lateral and diagonal direction reacting to virtual reality targets were analyzed separately. Receiver operator characteristic (ROC) analyses narrowed the 34 original predictor variables to five variables. Logistic regression analysis determined the three strongest predictors of CLEI for this cohort to be: lateral agility acceleration asymmetry, lateral flanker deceleration asymmetry, and diagonal agility reaction time average. Univariable analysis found odds ratios to range from 1.98 to 2.75 for these predictors of CLEI. ROC analysis had an area under the curve of 0.702 for any combination of two or more risk factors produced an odds ratio of 5.5 for risk of CLEI. These results suggest an asymmetry of 8-15% on two of the identified metrics or a slowed reaction time of ≥0.787 s places someone at increased risk of injury. Sixty-three percent (36/57) of the players who sustained an injury had ≥2 positive predictors In spite of the recognized limitation, these finding support the belief that whole-body reactive agility performance can identify Division I football players who are at elevated risk for CLEI.

A Critical Review on the Design, Manufacturing and Assessment of the Bone Scaffold for Large Bone Defects

In recent years, bone tissue engineering has emerged as a promising solution for large bone defects. Additionally, the emergence and development of the smart metamaterial, the advanced optimization algorithm, the advanced manufacturing technique, etc. have largely changed the way how the bone scaffold is designed, manufactured and assessed. Therefore, the aim of the present study was to give an up-to-date review on the design, manufacturing and assessment of the bone scaffold for large bone defects. The following parts are thoroughly reviewed: 1) the design of the microstructure of the bone scaffold, 2) the application of the metamaterial in the design of bone scaffold, 3) the optimization of the microstructure of the bone scaffold, 4) the advanced manufacturing of the bone scaffold, 5) the techniques for assessing the performance of bone scaffolds.

Detecting Rater Biases in Sparse Rater-Mediated Assessment Networks

Practical constraints in rater-mediated assessments limit the availability of complete data. Instead, most scoring procedures include one or two ratings for each performance, with overlapping performances across raters or linking sets of multiple-choice items to facilitate model estimation. These incomplete scoring designs present challenges for detecting rater biases, or differential rater functioning (DRF). The purpose of this study is to illustrate and explore the sensitivity of DRF indices in realistic sparse rating designs that have been documented in the literature that include different types and levels of connectivity among raters and students. The results indicated that it is possible to detect DRF in sparse rating designs, but the sensitivity of DRF indices varies across designs. We consider the implications of our findings for practice related to monitoring raters in performance assessments.

STALLION: a stacking-based ensemble learning framework for prokaryotic lysine acetylation site prediction

Protein post-translational modification (PTM) is an important regulatory mechanism that plays a key role in both normal and disease states. Acetylation on lysine residues is one of the most potent PTMs owing to its critical role in cellular metabolism and regulatory processes. Identifying protein lysine acetylation (Kace) sites is a challenging task in bioinformatics. To date, several machine learning-based methods for the in silico identification of Kace sites have been developed. Of those, a few are prokaryotic species-specific. Despite their attractive advantages and performances, these methods have certain limitations. Therefore, this study proposes a novel predictor STALLION (STacking-based Predictor for ProkAryotic Lysine AcetyLatION), containing six prokaryotic species-specific models to identify Kace sites accurately. To extract crucial patterns around Kace sites, we employed 11 different encodings representing three different characteristics. Subsequently, a systematic and rigorous feature selection approach was employed to identify the optimal feature set independently for five tree-based ensemble algorithms and built their respective baseline model for each species. Finally, the predicted values from baseline models were utilized and trained with an appropriate classifier using the stacking strategy to develop STALLION. Comparative benchmarking experiments showed that STALLION significantly outperformed existing predictor on independent tests. To expedite direct accessibility to the STALLION models, a user-friendly online predictor was implemented, which is available at: http://thegleelab.org/STALLION.

The Impact of COVID-19 on the Preparation for the Tokyo Olympics: A Comprehensive Performance Assessment of Top Swimmers

BACKGROUND

The Olympic preparation of athletes has been highly influenced by COVID and post-COVID syndrome. As the complex screening of athletes is essential for safe and successful sports, we aimed to repeat the 2019-year sports cardiology screening of the Olympic Swim Team before the Olympics and to compare the results of COVID and non-COVID athletes.

METHODS

Patient history, electrocardiogram, laboratory tests, body composition analysis, echocardiography, cardiopulmonary exercise test (CPET) were performed. We used time-ranking points to compare swimming performance.

RESULTS

From April 2019, we examined 46 elite swimmers (24 ± 4 years). Fourteen swimmers had COVID infection; all cases were mild. During CPET there was no difference in the performance of COVID (male: VO2 max 55 ± 4 vs. 56.5 ± 5 mL/kg/min, p = 0.53; female: VO2 max 54.6 ± 4 vs. 56 ± 5.5 mL/kg/min, p = 0.86) vs. non-COVID athletes (male VO2 max 56.7 ± 5 vs. 55.5 ± 4.5 mL/kg/min, p = 0.50; female 49.6 ± 3 vs. 50.7 ± 2.6 mL/kg/min, p = 0.47) between 2019 and 2021. When comparing the time results of the National Championships, 54.8% of the athletes showed an improvement (p = 0.75).

CONCLUSIONS

COVID infection with short-term detraining did not affect the performance of well-trained swimmers. According to our results, the COVID pandemic did not impair the effectiveness of the preparation for the Tokyo Olympics.

Providers' perspectives on the performance of primary healthcare centres in India: The missing link

BACKGROUND

Primary healthcare centres (PHCs) form the foundation of the Indian public health system, and thus their effective functioning is paramount in ensuring the population's health. The World Health Organisation (WHO) has set six aspects of performance assessment for general health systems, which are hardly applicable to the PHC setup in a low- and middle-income country. The Primary Health Care Performance Initiative (PHCPI) has prescribed a framework with five domains consisting of 36 indicators for primary healthcare performance assessment from a policy point of view. For the assessment to be realistic, it should include inputs from stakeholders involved in care delivery, so this study examines the perspectives of healthcare providers at PHCs in India.

METHODOLOGY

The authors used qualitative research methodology in the form of responsive evaluations of healthcare provider's interviews to understand the indicators of PHC performance.

RESULTS AND CONCLUSION

The study results showed that healthcare providers considered efficient teamwork, opportunities for enhancing provider skills and knowledge, job satisfaction, effective PHC administration, and good community relationship as PHC performance assessment. These domains of performance could be considered the 'missing link' in PHC assessment, since they are deemed important by providers and did not coincide with the WHO aspects and the PHCPI performance assessment framework.

Associations between Testing and Game Performance in Ice Hockey: A Scoping Review

BACKGROUND

Despite the exhaustive body of literature on the demands of ice hockey, less is known about the relationships between functional performance testing protocols (on ice and off ice) and performance in a game situation. The objective of this review is to provide an overview of these associations.

METHODS

This review aims to identify on- and off-ice testing currently used in the scientific literature and their possible transfer to game performance as well as identifying research gaps in this field.

RESULTS

The 17 selected studies showed that off-ice and on-ice fitness test results can be modestly transferred to the player's selection as well as global and advanced performance indicators.

CONCLUSION

This review of the literature reinforces the importance of strength and conditioning coaches administering previously validated fitness tests. Regarding the academic research, it is also proposed to use performance markers that are directly related to the players' on-ice performance to represent more accurately the relationship between the players' fitness level and their work output. Three research gaps were also identified in relation to targeted populations, choice of performance markers and data measurement methods.

ANPELA: analysis and performance assessment of the label-free quantification workflow for metaproteomic studies

Label-free quantification (LFQ) with a specific and sequentially integrated workflow of acquisition technique, quantification tool and processing method has emerged as the popular technique employed in metaproteomic research to provide a comprehensive landscape of the adaptive response of microbes to external stimuli and their interactions with other organisms or host cells. The performance of a specific LFQ workflow is highly dependent on the studied data. Hence, it is essential to discover the most appropriate one for a specific data set. However, it is challenging to perform such discovery due to the large number of possible workflows and the multifaceted nature of the evaluation criteria. Herein, a web server ANPELA (https://idrblab.org/anpela/) was developed and validated as the first tool enabling performance assessment of whole LFQ workflow (collective assessment by five well-established criteria with distinct underlying theories), and it enabled the identification of the optimal LFQ workflow(s) by a comprehensive performance ranking. ANPELA not only automatically detects the diverse formats of data generated by all quantification tools but also provides the most complete set of processing methods among the available web servers and stand-alone tools. Systematic validation using metaproteomic benchmarks revealed ANPELA's capabilities in 1 discovering well-performing workflow(s), (2) enabling assessment from multiple perspectives and (3) validating LFQ accuracy using spiked proteins. ANPELA has a unique ability to evaluate the performance of whole LFQ workflow and enables the discovery of the optimal LFQs by the comprehensive performance ranking of all 560 workflows. Therefore, it has great potential for applications in metaproteomic and other studies requiring LFQ techniques, as many features are shared among proteomic studies.

Physical and Anthropometric Characteristics Do Not Differ According to Birth Year Quartile in High-Level Junior Australian Football Players

The aim of this study was to explore differences in the physical fitness and anthropometric profiles between birth year quartiles of players attending the Australian Football League (AFL) National Draft Combine. Date of birth, anthropometric, 20 m sprint, vertical and running vertical jump, AFL planned agility, and 20 m Multi-Stage Fitness Test (MSFT) data were obtained for players selected to attend the Combine between 1999 and 2019 (n = 1549; Mage = 18.1; SDage = 0.3). The underlying density distributions of the data were visually explored using violin plots overlaid with box and whisker plots. A multivariate analysis of variance (MANOVA) was then used to model the main effect of birth quartile (four levels) on the physical and anthropometric scores. Results showed that physical and anthropometric test scores did not significantly differ according to birth quartile (V = 0.008, F = 0.880, p = 0.631). We conclude that the physical and anthropometric profiles of high-level junior Australian Football players were similar according to birth year quartile across the modeled period. Therefore, how players utilize their physical and anthropometric attributes during game-play via contextualized, representative assessments, such as small-sided games, should be considered when examining potential causes of a RAE.

Developing a Comprehensive Testing Battery for Mixed Martial Arts

Mixed martial arts (MMA) is a combat sport that employs techniques from different combat disciplines. There are a multitude of technical and physiological characteristics that contribute to competitive success. Developing a single scientific assessment that can predict competitive outcomes poses great difficulty due to the complexity of MMA. While previous research has investigated some important physiological characteristics, there is no accepted best-practice for a comprehensive testing battery. As such, this study aimed to design and utilize a battery of physiological assessments to cover aerobic and anaerobic function, strength measures including explosive and maximal strength, body composition and repeat effort ability in Australian MMA athletes. Six participants with competitive experience were recruited. Testing involved a familiarization, three experimental sessions and including assessments such as the isometric midthigh pull, Wingate test, graded exercise test, countermovement jump and body composition scan. Results showed the testing battery in this study was realistic and able to be completed by the participants without issue and that regional Australian MMA athletes were similar physiologically to elite standard compared with previous research taken from a range of sources. However, future research with the testing battery is required with larger and more diverse samples to better understand the full profiles of MMA athletes. The results of the study can help inform athletes, researchers and support staff alike when deciding upon which testing protocols to use for MMA athletes. Future research should aim to develop normative data using the battery proposed in the current study.

Large-scale comparative review and assessment of computational methods for anti-cancer peptide identification

Anti-cancer peptides (ACPs) are known as potential therapeutics for cancer. Due to their unique ability to target cancer cells without affecting healthy cells directly, they have been extensively studied. Many peptide-based drugs are currently evaluated in the preclinical and clinical trials. Accurate identification of ACPs has received considerable attention in recent years; as such, a number of machine learning-based methods for in silico identification of ACPs have been developed. These methods promote the research on the mechanism of ACPs therapeutics against cancer to some extent. There is a vast difference in these methods in terms of their training/testing datasets, machine learning algorithms, feature encoding schemes, feature selection methods and evaluation strategies used. Therefore, it is desirable to summarize the advantages and disadvantages of the existing methods, provide useful insights and suggestions for the development and improvement of novel computational tools to characterize and identify ACPs. With this in mind, we firstly comprehensively investigate 16 state-of-the-art predictors for ACPs in terms of their core algorithms, feature encoding schemes, performance evaluation metrics and webserver/software usability. Then, comprehensive performance assessment is conducted to evaluate the robustness and scalability of the existing predictors using a well-prepared benchmark dataset. We provide potential strategies for the model performance improvement. Moreover, we propose a novel ensemble learning framework, termed ACPredStackL, for the accurate identification of ACPs. ACPredStackL is developed based on the stacking ensemble strategy combined with SVM, Naïve Bayesian, lightGBM and KNN. Empirical benchmarking experiments against the state-of-the-art methods demonstrate that ACPredStackL achieves a comparative performance for predicting ACPs. The webserver and source code of ACPredStackL is freely available at http://bigdata.biocie.cn/ACPredStackL/ and https://github.com/liangxiaoq/ACPredStackL, respectively.

Performance assessment using sensor technology

Over the past 30 years, there have been numerous, noteworthy successes in the development, validation, and implementation of clinical skills assessments. Despite this progress, the medical profession has barely scratched the surface towards developing assessments that capture the true complexity of hands-on skills in procedural medicine. This paper highlights the development implementation and new discoveries in performance metrics when using sensor technology to assess cognitive and technical aspects of hands-on skills.

Psychometric Suitability of Adaptations to the Special Judo Fitness Test for Athletes With Visual Impairment

This study aimed to test the reliability, sensitivity, construct and logical validity of an adapted Special Judo Fitness Test (SJFT) for judo athletes with visual impairment . Twenty judo athletes with visual impairments performed both the adapted SJFT with tactile and sonorous stimuli (experimental conditions) and the typically administered SJFT (standard condition). We used analyses of variance (ANOVAs) with repeated-measures to compare the groups' SJFT performances, and one-way ANOVAs to compare different visual ability classes of athletes (B1, B2 and B3). We used t-tests to compare SJFT variables between elite and sub-elite groups. We set statistical significance for all tests at p < 0.05. The standard SJFT showed excellent test-retest reliability for number of throws and overall index (ICC = 0.91-0.95), and both sonorous and tactile sensitivity adaptations of the SJFT showed medium sensitivity for detecting performance changes. The number of throws and SJFT index were higher with the sonorous adaptation of the test, compared to the tactile and standard versions (p < 0.001). Athletes who were blind (B1) presented similar performances to athletes who were partially sighted (B2 and B3) only on the SJFT with the sonorous stimulus. Moreover, only the sonorous SJFT adaptation discriminated between the performances of elite and sub-elite athletes (p < 0.001). In conclusion, both SJFT adaptations showed excellent reliability and medium sensitivity on test-retest, but, only the SJFT with the sonorous stimulus seemed valid for assessing judo athletes with varying degrees of visual impairment, and only the sonorous stimulus SJFT discriminated elite from sub-elite athletes.

Quality Assessment and Validation of High-Throughput Sequencing for Grapevine Virus Diagnostics

Development of High-Throughput Sequencing (HTS), also known as next generation sequencing, revolutionized diagnostic research of plant viruses. HTS outperforms bioassays and molecular diagnostic assays that are used to screen domestic and quarantine grapevine materials in data throughput, cost, scalability, and detection of novel and highly variant virus species. However, before HTS-based assays can be routinely used for plant virus diagnostics, performance specifications need to be developed and assessed. In this study, we selected 18 virus-infected grapevines as a test panel for measuring performance characteristics of an HTS-based diagnostic assay. Total nucleic acid (TNA) was extracted from petioles and dormant canes of individual samples and constructed libraries were run on Illumina NextSeq 500 instrument using a 75-bp single-end read platform. Sensitivity was 98% measured over 264 distinct virus and viroid infections with a false discovery rate (FDR) of approximately 1 in 5 positives. The results also showed that combining a spring petiole test with a fall cane test increased sensitivity to 100% for this TNA HTS assay. To evaluate extraction methodology, these results were compared to parallel dsRNA extractions. In addition, in a more detailed dilution study, the TNA HTS assay described here consistently performed well down to a dilution of 5%. In that range, sensitivity was 98% with a corresponding FDR of approximately 1 in 5. Repeatability and reproducibility were assessed at 99% and 93%, respectively. The protocol, criteria, and performance levels described here may help to standardize HTS for quality assurance and accreditation purposes in plant quarantine or certification programs.

A nationwide participatory programme to measure person-centred hospital care in Italy: Results and implications for continuous improvement

Background

Patient‐centredness has been targeted by the Italian government as a key theme for the future development of health services.

Objective

Measuring patient‐centred health services in partnership with citizens, health professionals and decision makers.

Design

National participatory survey in a large test set of hospitals at national level.

Setting and participants

A total of 387 hospital visits conducted in 16 Italian regions by over 1,500 citizens and health professionals during 2017‐2018.

Main variables and outcome measures

An ad hoc checklist was used to assess person‐centredness in hospital care through 243 items, grouped in 4 main areas, 12 sub‐areas and 29 person‐centred criteria (scored 0‐10). GEE linear multivariate regression was used to explore the relation between hospital characteristics and person‐centredness.

Results

Person‐centred scores were moderately high, with substantial variation overall (median score: 7.0, range: 3.2‐9.5) and by area (Care Processes: 6.8, 2.0‐9.8; Access: 7.4, 2.7‐9.7; Transparency: 6.7, 3.4‐9.5 and Relationship: 7.3, 0.8‐10.0). Multivariate regression found higher scores for increasing volumes of activity (quartile increase: +0.21; 95% CI: 0.13, 0.29) and lower scores in the south and islands (−1.03; −1.62,‐0.45).

Discussion

The checklist has been applied successfully by over 1,500 collaborators who assessed hospitals in 16 distinct Regions and Autonomous Provinces of Italy. Despite an overall positive mark, all scores were highly variable by location and hospital characteristics.

Conclusion and patient or public contribution

A national participatory programme to improve patient‐centredness in Italian hospitals highlighted critical areas with the direct input of citizens.

A Hospital Performance Assessment Model Using the IPOCC Approach

BACKGROUND

Developing a practical model to assess hospital performance improves the quality of services and leads to patient satisfaction. This study aims to develop and present such a model using the IPOCC (Input, Process, Output, Control and Context) approach.

METHODS

This study used a mixed-method research. The statistical population of the qualitative part included 27 experts who were purposefully selected and the sampling process was continued by the snowball method until the data saturation was reached. The quantitative part included 334 managers at different levels within a hospital, who were selected by a random sampling method based on Cochran's formula.

RESULTS

The hospital evaluation model has 5 dimensions with 20 factors: input (human, financial, physical, information and equipment), process (treatment, para-clinical, prevention, management, and leadership processes), outcome (patient, staff and community outcomes and key performance index), control (internal control, external control), context (hospital culture, hospital status, the role of evaluators and community conditions). The value of chi-square was 4689.154, the degree of freedom was 2385, and the ratio of chi-square to the degree of freedom in the model was 1.966, which is an acceptable value. The values obtained from CFI, GFI, and IFI fit indices were acceptable. The SRMR index was 0.1130.

CONCLUSIONS

Using a performance assessment model along with the IPOCC approach evaluates hospital processes and the output obtained from the proper implementation of these processes in all areas. The areas include the hospital provided services like the control and context, or the traditional perspectives like physical, human, financial, and equipment resources.

Composite Reliability of Workplace Based Assessment of International Medical Graduates

This article was migrated. The article was marked as recommended. Introduction All developed countries depend on International Medical Graduates (IMGs) to complement their workforce. However, the assessment of their fitness to practice and acculturation into the new system can be challenging. To improve this, we introduced Workplace Based Assessment (WBA), using a programmatic philosophy. This paper reports the reliability of this new approach. Method Over the past 10 years, we have assessed over 250 IMGs, each cohort assessed over a 6-month period. We used Mini-Cex, Case Based Discussions (CBD) and Multi-Source Feedback (MSF) to assess them. We analysed the reliability of each tool and the composite reliability of 12 Mini-Cex, 5 CBDs and 12 MSF assessments in the tool kit. Results A reliability coefficient of 0.78 with a SEM of 0.19 was obtained for the sample of 236 IMGs. We found the MSF to be the most reliable tool. By adding one more MSF to the assessment on two occasions, we can reach a reliability of 0.8 and SEM of 0.18. Conclusions The current assessment methodology has acceptable reliability. By increasing the MSF, we can improve the reliability. The lessons from this study are generalisable to IMG assessment and other medical education programs.

Estimating the Lactate Threshold Using Wireless Near-Infrared Spectroscopy and Threshold Detection Analyses

The present investigation examined the ability of two threshold detection analyses (maximum distance, Dmax; modified maximum distance, mDmax) in identifying the near-infrared spectroscopy (NIRS) threshold, a lactate threshold (LT) estimate, from exercising tissue oxygen saturation (StO₂) responses. Additionally, the test-retest reliability of exercising StO₂ and total hemoglobin concentration (THC) responses were examined at moderate and peak cycling intensities. Fourteen healthy, recreationally active participants performed maximal incremental step cycling tests (+25 W / 3 minutes) to volitional fatigue on two separate occasions while StO₂ and THC of the vastus lateralis were monitored. Exercising blood [lactate] was collected during Session One. LT and NIRS thresholds (NIRS1, NIRS2) were then determined using Dmax and mDmax threshold analyses. Significant (p < 0.05), moderate correlations were detected between LT and NIRS1 when using Dmax (LT = 130 ± 49 W, NIRS1 = 136 ± 34 W, r = 0.690), but not for mDmax (r = 0.487). No significant test-retest reliability for the NIRS thresholds were observed for Dmax (ICC = 0.351) or mDmax (ICC = 0.385). Exercising StO₂ responses demonstrated good reliability (ICC = 0.841-0.873) while exercising THC responses demonstrated moderate-good reliability (ICC = 0.720-0.873) at moderate and peak exercise intensities. The results of this study suggest that neither the Dmax nor mDmax threshold analyses should be used to estimate the LT due to the unreliable detection of the NIRS threshold from session to session.