Performance assessment in health care providers: a critical review of evidence and current practice

Barriers Toward the National Program for Prevention and Control of Diabetes in Iran: A Qualitative Exploration

BACKGROUND

Despite the achievements of the national program for the prevention and control of diabetes (NPPCD) over the past two decades, the available evidence indicates a high prevalence of this disease in Iran. This qualitative study aims to investigate barriers to the NPPCD by pursuing the perspectives of relevant policy-makers, planners, and healthcare workers.

METHODS

A grounded theory approach was used to analyze participants' perceptions and experiences. Semi-structured interviews (n=23) and eight focus groups (n=109) were conducted with relevant policy-makers, planners, and healthcare workers in charge of Iran's national diabetes management program. Of the 132 participants, ages ranged from 25 to 56 years, and 53% were female. Constant comparative analysis of the data was conducted manually, and open, axial, and selective coding was applied to the data.

RESULTS

Two main themes emerged from data analysis: implementation barriers and inefficient policy-making/ planning. Insufficient financial resources, staff shortage and insufficient motivation, inadequate knowledge of some healthcare workers, and defects in the referral system were recognized as the NPPCD implementation barriers. Inappropriate program prioritizing, the lack of or poor intersectoral collaboration, and the lack of an effective evaluation system were the inefficient policy-making/planning problems.

CONCLUSION

Current results highlighted that inefficient policy-making and planning have led to several implementation problems. Moreover, the key strategies to promote this program are prioritizing the NPPCD, practical intersectoral collaboration, and utilizing a more efficient evaluation system to assess the program and staff performance.

Big Data Health Care Innovations: Performance Dashboarding as a Process of Collective Sensemaking

Big data is poised to revolutionize health care, and performance dashboards can be an important tool to manage big data innovations. Dashboards show the progress being made and provide critical management information about effectiveness and efficiency. However, performance dashboards are more than just a clear and straightforward representation of performance in the health care context. Instead, the development and maintenance of informative dashboards can be more productively viewed as an interactive and iterative process involving all stakeholders. We refer to this process as dashboarding and reflect on our learnings within a large European Union–funded project. Within this project, multiple big data applications in health care are being developed, piloted, and scaled up. In this paper, we discuss the ways in which we cope with the inherent sensitivities and tensions surrounding dashboarding in such a dynamic environment.

Primary health Centres' performance assessment measures in developing countries: review of the empirical literature

Background

It is universally accepted that primary healthcare is essential for achieving public health and that assessment of its performance is critical for continuous improvement. The World Health Organization’s (WHO’s) framework for performance assessment is a comprehensive global standard, but difficult to apply in developing countries because of financial and data constraints. This study aims to review the empirical literature on measures for Primary Health Centre (PHC) performance assessment in developing countries, and compare them for comprehensiveness with the aspects described by the WHO Framework.

Methods

Research articles published in English scientific journals between January 1979 and October 2016 were reviewed systematically. The reporting quality of the article and the quality of the measures were assessed with instruments adapted for the purpose of this study. Data was categorized and described.

Results

Fifteen articles were included in the study out of 4359 articles reviewed. Nine articles used quantitative methods, one article used qualitative methods exclusively and five used mixed methods. Fourteen articles had a good description of the measurement properties. None of the articles presented validity tests of the measures but eleven articles presented measures that were well established. Mostly studies included components of personnel competencies (skilled/ non-skilled) and centre performance (patient satisfaction/cost /efficiency).

Conclusions

In comparison to the WHO framework, the measures in the articles were limited in scope as they did not represent all service components of PHCs. Hence, PHC performance assessment should include system components along with relevant measures of personnel performance beyond knowledge of protocols. Existing measures for PHC performance assessment in developing countries need to be validated and concise measures for neglected aspects need to be developed.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3423-0) contains supplementary material, which is available to authorized users.

Assessing the Relative Performance of Nurses Using Data Envelopment Analysis Matrix (DEAM)

Assessing employee performance is one of the most important issue in healthcare management services. Because of their direct relationship with patients, nurses are also the most influential hospital staff who play a vital role in providing healthcare services. In this paper, a novel Data Envelopment Analysis Matrix (DEAM) approach is proposed for assessing the performance of nurses based on relative efficiency. The proposed model consists of five input variables (including type of employment, work experience, training hours, working hours and overtime hours) and eight output variables (the outputs are amount of hours each nurse spend on each of the eight activities including documentation, medical instructions, wound care and patient drainage, laboratory sampling, assessment and control care, follow-up and counseling and para-clinical measures, attendance during visiting and discharge suction) have been tested on 30 nurses from the heart department of a hospital in Iran. After determining the relative efficiency of each nurse based on the DEA model, the nurses' performance were evaluated in a DEAM format. As results the nurses were divided into four groups; superstars, potential stars, those who are needed to be trained effectively and question marks. Finally, based on the proposed approach, we have drawn some recommendations to policy makers in order to improve and maintain the performance of each of these groups. The proposed approach provides a practical framework for hospital managers so that they can assess the relative efficiency of nurses, plan and take steps to improve the quality of healthcare delivery.

Assessment formats in dental medicine: An overview

AIM

At the annual meeting of German dentists in Frankfurt am Main in 2013, the Working Group for the Advancement of Dental Education (AKWLZ) initiated an interdisciplinary working group to address assessments in dental education. This paper presents an overview of the current work being done by this working group, some of whose members are also actively involved in the German Association for Medical Education's (GMA) working group for dental education. The aim is to present a summary of the current state of research on this topic for all those who participate in the design, administration and evaluation of university-specific assessments in dentistry.

METHOD

Based on systematic literature research, the testing scenarios listed in the National Competency-based Catalogue of Learning Objectives (NKLZ) have been compiled and presented in tables according to assessment value.

RESULTS

Different assessment scenarios are described briefly in table form addressing validity (V), reliability (R), acceptance (A), cost (C), feasibility (F), and the influence on teaching and learning (EI) as presented in the current literature. Infoboxes were deliberately chosen to allow readers quick access to the information and to facilitate comparisons between the various assessment formats. Following each description is a list summarizing the uses in dental and medical education.

CONCLUSION

This overview provides a summary of competency-based testing formats. It is meant to have a formative effect on dental and medical schools and provide support for developing workplace-based strategies in dental education for learning, teaching and testing in the future.

A competency-based performance appraisal for nurse leaders

Introduction: An employee performance appraisal is used internationally to improve employee performance, productivity, efficiency and effectiveness in nursing care. The purpose of the study was to assess the performance and efficiency of nurse leaders at all levels of leadership, identify deficient areas for each individual leader and propose improvements. Methods: Two surveys were conducted in 15 Slovenian public hospitals. The sample from the first survey included 1311 nurses who assessed the level of competencies of their immediate superiors. The sample from the second survey included 236 nurse leaders. Data envelopment analysis is used in this study to develop a model of practice outputs and inputs to help identify the most efficient nurse leaders. Results: Significant differences exist in the performance appraisal among the leaders in nursing. A total of 203 (86 %) nurse leaders are defined as inefficient and could improve their leadership performance. On average, inefficient leaders could improve their behaviour most in the areas of generic leadership competencies (by 51.7 %) and interprofessional relationships (by 47.7 %). Discussion and conclusion: The areas with the worst performance are generic leadership competencies and interprofessional relationships where also the most significant improvements are possible. Nurse leaders and hospital managers can use the analysis results to determine in which areas the leaders in their organisation should improve the levels of their competencies.

Validation of the Australian Nursing Standards Assessment Tool (ANSAT): A pilot study

BACKGROUND

Workplace-based learning and assessment are essential elements of all nursing education programs. There is, however, limited evidence of validity and reliability available regarding assessment instruments, based on the professional competency standards for nursing in Australia.

OBJECTIVE

The aims of this project were to advance the assessment properties of a new instrument, the Australian Nursing Standards Assessment Tool (ANSAT) and investigate the acceptability of this instrument when applied to the evaluation of the professional competence of nursing students in authentic practice settings.

METHODS

A validation study of ANSAT was conducted by 23 clinical assessors from two universities, completing 220 instruments for second and third year undergraduate nursing students undertaking clinical placement in an authentic practice setting. 'Think Aloud' interviews and a post-test survey provided additional data to determine the validity and acceptability of the ANSAT. The pilot study occurred in parallel with completion of existing university assessment instruments.

RESULTS

Principal components analysis extracted one factor: professional practice competence. Comparison of total instrument scores between year levels demonstrated a significant difference in each of the clinical domains (p=0.000), suggesting that the instrument is sensitive to differing levels of performance across different year levels. The ANSAT demonstrated high internal consistency with a Cronbach's alpha coefficient of 0.976. Post-test evaluation completed by assessors demonstrated high usability and acceptability for use in common practice settings.

DISCUSSION

Results of the pilot study provided preliminary support for the ANSAT instrument. It is recommended that testing of the instrument be undertaken on a larger cohort to ensure the generalisability of the findings to other nursing populations.

CONCLUSION

There is a need for valid and reliable instruments that assess the performance of nursing in the authentic practice setting. With the addition of supportive behavioural cues, the ANSAT enables clarity, consistency and collaboration in workplace-based assessment.

Observation for assessment of clinician performance: a narrative review

BACKGROUND

Video recorded and in-person observations are methods of quality assessment and monitoring that have been employed in high risk industries. In the medical field, observations have been used to evaluate the quality and safety of various clinical processes. This review summarises studies utilising video recorded or in-person observations for assessing clinician performance in medicine and surgery.

METHODS

A search of MEDLINE (PubMed) was conducted using a combination of medical subject headings (MeSH) terms. Articles were included if they described the use of in-person or video recorded observations to assess clinician practices in three categories: (1) teamwork and communication between clinicians; (2) errors and weaknesses in practice; and (3) compliance and adherence to interventions or guidelines.

RESULTS

The initial search criteria returned 3215 studies, 223 of which were identified for full text review. A total of 69 studies were included in the final set of literature. Observations were most commonly used in data dense and high risk environments, such as the emergency department or operating room. The most common use was for assessing teamwork and communication factors.

CONCLUSIONS

Observations are useful for the improvement of healthcare delivery through the identification of clinician lapses and weaknesses that affect quality and safety. Limitations of observations include the Hawthorne effect and the necessity of trained observers to capture and analyse the notes or videos. The comprehensive, subtle and sensitive information observations provided can supplement traditional quality assessment methods and inform targeted interventions to improve patient safety and the quality of care.

Variables that affect the process and outcome of feedback, relevant for medical training: a meta-review

CONTEXT

Feedback is considered important in medical education. The literature is not clear about the mechanisms that contribute to its effects, which are often small to moderate and at times contradictory. A variety of variables seem to influence the impact of feedback on learning. The aim of this study was to determine which variables influence the process and outcomes of feedback in settings relevant to medical education.

METHODS

A myriad of studies on feedback have been conducted. To determine the most researched variables, we limited our review to meta-analyses and literature reviews published in the period from January 1986 to February 2012. According to our protocol, we first identified features of the feedback process that influence its effects and subsequently variables that influence these features. We used a chronological model of the feedback process to categorise all variables found.

RESULTS

A systematic search of ERIC, PsycINFO and MEDLINE yielded 1101 publications, which we reduced to 203, rejecting papers on six exclusion criteria. Of these, 46 met the inclusion criteria. In our four-phase model, we identified 33 variables linked to task performance (e.g. task complexity, task nature) and feedback reception (e.g. self-esteem, goal-setting behaviour) by trainees, and to observation (e.g. focus, intensity) and feedback provision (e.g. form, content) by supervisors that influence the subsequent effects of the feedback process. Variables from all phases influence the feedback process and effects, but variables that influence the quality of the observation and rating of the performance dominate the literature. There is a paucity of studies addressing other, seemingly relevant variables.

CONCLUSIONS

The larger picture of variables that influence the process and outcome of feedback, relevant for medical education, shows many open spaces. We suggest that targeted studies be carried out to expand our knowledge of these important aspects of feedback in medical education.

Assisting allied health in performance evaluation: a systematic review

Background

Performance evaluation raises several challenges to allied health practitioners and there is no agreed approach to measuring or monitoring allied health service performance. The aim of this review was to examine the literature on performance evaluation in healthcare to assist in the establishment of a framework that can guide the measurement and evaluation of allied health clinical service performance. This review determined the core elements of a performance evaluation system, tools for evaluating performance, and barriers to the implementation of performance evaluation.

Methods

A systematic review of the literature was undertaken. Five electronic databases were used to search for relevant articles: MEDLINE, Embase, CINAHL, PsychInfo, and Academic Search Premier. Articles which focussed on any allied health performance evaluation or those which examined performance in health care in general were considered in the review. Content analysis was used to synthesise the findings from individual articles.

Results

A total of 37 articles were included in the review. The literature suggests there are core elements involved in performance evaluation which include prioritising clinical areas for measurement, setting goals, selecting performance measures, identifying sources of feedback, undertaking performance measurement, and reporting the results to relevant stakeholders. The literature describes performance evaluation as multi-dimensional, requiring information or data from more than one perspective to provide a rich assessment of performance. A range of tools or instruments are available to capture various perspectives and gather a comprehensive picture of health care quality.

Conclusions

Every allied health care delivery system has different performance needs and will therefore require different approaches. However, there are core processes that can be used as a framework to evaluate allied health performance. A careful examination of barriers to performance evaluation and subsequent tailoring of strategies to overcome these barriers should be undertaken to achieve the aims of performance evaluation. The findings of this review should inform the development of a standardised framework that can be used to measure and evaluate allied health performance. Future research should explore the utility and overall impact of such framework in allied health service delivery.

Do educational outcomes correspond with the requirements of nursing practice: educators' and managers' assessments of novice nurses' professional competence

Objective

This study evaluated weather educational outcomes of nurse education meet the requirements of nursing practice by exploring the correspondence between nurse educators' and nurse managers' assessments of novice nurses' professional competence. The purpose was to find competence areas contributing to the acknowledged practice–theory gap.

Design

A cross-sectional, comparative design using the Nurse Competence Scale was applied.

Subjects

The sample comprised nurse educators (n = 86) and nurse managers (n = 141).

Methods

Descriptive and inferential statistics were used in the data analysis.

Main outcome measures

Educators assessed novice nurses' competence to a significantly higher level than managers in all competence areas (p < 0.001). The biggest correspondence between educators' and mangers' assessments were in competencies related to immediate patient care, commitment to ethical values, maintaining professional skills and nurses' care of the self. The biggest differences were in competencies related to developmental and evaluation tasks, coaching activities, use of evidence-based knowledge and in activities which required mastering a comprehensive view of care situations. However, differences between educators' and managers' assessments were strongly associated with their age and work experience. Active and improved collaboration should be focused on areas in which the differences between educators' and managers' assessments greatly differ in ensuring novice nurses′ fitness for practice.

Delivering interventions to reduce the global burden of stillbirths: improving service supply and community demand

BACKGROUND

Although a number of antenatal and intrapartum interventions have shown some evidence of impact on stillbirth incidence, much confusion surrounds ideal strategies for delivering these interventions within health systems, particularly in low-/middle-income countries where 98% of the world's stillbirths occur. Improving the uptake of quality antenatal and intrapartum care is critical for evidence-based interventions to generate an impact at the population level. This concluding paper of a series of papers reviewing the evidence for stillbirth interventions examines the evidence for community and health systems approaches to improve uptake and quality of antenatal and intrapartum care, and synthesises programme and policy recommendations for how best to deliver evidence-based interventions at community and facility levels, across the continuum of care, to reduce stillbirths.

METHODS

We systematically searched PubMed and the Cochrane Library for abstracts pertaining to community-based and health-systems strategies to increase uptake and quality of antenatal and intrapartum care services. We also sought abstracts which reported impact on stillbirths or perinatal mortality. Searches used multiple combinations of broad and specific search terms and prioritised rigorous randomised controlled trials and meta-analyses where available. Wherever eligible randomised controlled trials were identified after a Cochrane review had been published, we conducted new meta-analyses based on the original Cochrane criteria.

RESULTS

In low-resource settings, cost, distance and the time needed to access care are major barriers for effective uptake of antenatal and particularly intrapartum services. A number of innovative strategies to surmount cost, distance, and time barriers to accessing care were identified and evaluated; of these, community financial incentives, loan/insurance schemes, and maternity waiting homes seem promising, but few studies have reported or evaluated the impact of the wide-scale implementation of these strategies on stillbirth rates. Strategies to improve quality of care by upgrading the skills of community cadres have shown demonstrable impact on perinatal mortality, particularly in conjunction with health systems strengthening and facilitation of referrals. Neonatal resuscitation training for physicians and other health workers shows potential to prevent many neonatal deaths currently misclassified as stillbirths. Perinatal audit systems, which aim to improve quality of care by identifying deficiencies in care, are a quality improvement measure that shows some evidence of benefit for changes in clinical practice that prevent stillbirths, and are strongly recommended wherever practical, whether as hospital case review or as confidential enquiry at district or national level.

CONCLUSION

Delivering interventions to reduce the global burden of stillbirths requires action at all levels of the health system. Packages of interventions should be tailored to local conditions, including local levels and causes of stillbirth, accessibility of care and health system resources and provider skill. Antenatal care can potentially serve as a platform to deliver interventions to improve maternal nutrition, promote behaviour change to reduce harmful exposures and risk of infections, screen for and treat risk factors, and encourage skilled attendance at birth. Following the example of high-income countries, improving intrapartum monitoring for fetal distress and access to Caesarean section in low-/middle-income countries appears to be key to reducing intrapartum stillbirth. In remote or low-resource settings, families and communities can be galvanised to demand and seek quality care through financial incentives and health promotion efforts of local cadres of health workers, though these interventions often require simultaneous health systems strengthening. Perinatal audit can aid in the development of better standards of care, improving quality in health systems. Effective strategies to prevent stillbirth are known; gaps remain in the data, the evidence and perhaps most significantly, the political will to implement these strategies at scale.