Quality of nursing intensity data: inter-rater reliability of the patient classification after two decades in clinical use

Nurse Managers' Opinions of Information System Support for Performance Management: A Correlational Study

BACKGROUND

 Current information systems do not effectively support nurse managers' duties, such as reporting, resource management, and assessing clinical performance. Few performance management information systems are available and features in many are scattered.

OBJECTIVES

 The purpose of the study was to determine nurse managers' opinions of information system support for performance management.

METHODS

 An online questionnaire was used to collect data from nurse managers (n = 419). Pearson's correlation coefficients and linear regression were used to examine the relationships between variables, which were nurse managers' ability to manage resources, to report and evaluate productivity, and to assess nursing performance and clinical procedures.

RESULTS

 More than half of the managers used performance management systems daily. Managers (60%) felt that they can use information systems to follow the use of physical resources, and in general (63%), they felt that it is easy to perform searches with the systems used for following up activity. Nurse managers' ability to manage resources, to report productivity, and to assess nursing care performance were correlated significantly with each other.

CONCLUSION

 Currently, managers have to collect data from various systems for management purposes, as system integration does not support performance data collection. The availability of continuous in-service training had a positive effect on information system use.

How the reliability and validity of the patient classification system can be ensured in daily nursing work? A follow-up study

AIMS AND OBJECTIVES

This study aimed to determine the reliability and validity of the RAFAELA patient classification system (PCS) for qualified and efficient nurses.

BACKGROUND

The number of patients per nurse or diagnosis-based determination of nursing workload are imprecise measures that do not consider the variation in patients' care needs. Ensuring the reliability and validity of the RAFAELA is important for the efficient allocation of nursing resources.

METHODS

In this study, we investigated how the maintenance (parallel classification measurement and professional assessment of optimal nursing care intensity level measurement) of the RAFAELA was done with 9 years of follow-up data. The results were analysed using quantitative methods supplemented with qualitative audit descriptions. The STROBE checklist was used.

RESULTS

The RAFAELA was used continuously in 44 units (40%). The length of use of the RAFAELA influenced the success of parallel classification measurements. Six per cent of units passed parallel classification measurement over 75% after 1-3 years' use, 42% after 4-6 years and 83% after 7-9 years. Among the units that used the RAFAELA PCS continuously, only four (9%) passed the professional assessment of optimal nursing care intensity level measurement.

CONCLUSIONS

This study shows that ensuring the reliability and validity of the use of the RAFAELA is laborious, requires several years of use and continuous investments in nurses' skills and motivation.

RELEVANCE TO CLINICAL PRACTICE

Qualified use of PCS is challenging, and organisations should invest to maintenance, training, support and user motivation. Each patient should be classified comprehensively, and nursing resources should be calculated correctly. In addition, utilisation of the nursing intensity level should be maximised.

CLINICAL TRIAL REGISTRATION NUMBER

Kuopio University Hospital organisation permit number 73/2014.

PATIENT OR PUBLIC CONTRIBUTION

Information regarding individual patients or nurses was not available to the researchers. All materials are in the form of summary tables.

The role of nurse staffing in the performance of function-preserving processes during acute hospitalization: A cross-sectional study

BACKGROUND

Performance of function-preserving hospitalization processes related to patient mobility, use of continence aids and food intake is significantly associated with outcomes in older adults. Nurses are the front-line personnel responsible for promoting performance of such processes. The degree to which nurse staffing is related to this performance is unclear.

OBJECTIVE

To identify nurse-staffing characteristics and nursing-related care needs associated with older patients' mobility, continence care and food intake during acute hospitalization.

DESIGN

Cross-sectional study using survey data from the Hospitalization Process Effects on Functional Outcomes and Recovery (HoPE-FOR) cohort study combined with day-level administrative nurse staffing data and clinical day-level aggregated data for all patients hospitalized during the HoPE-FOR study period.

SETTING

Internal medicine units in two medical centers in Israel.

PARTICIPANTS

Eight hundred seventy-three older adults.

METHODS

Mobility, continence care and food intake were assessed within 2 days of admission using validated questionnaires. Nurse-to-patient ratios and nursing-skill mix (i.e. registered nurses (RNs), nurse aides, nurses with advanced clinical training and RNs with an academic degree) were assessed using administrative data. Decision trees were developed for mobility, continence care and food intake, applying classification and regression-tree analysis.

RESULTS

The mobility decision tree identified three characteristics subdividing patients into six nodes: pre-admission functioning, pre-admission activity level and percentage of nurses with advanced training. The percentage of nurses with advanced training classified low-functioning patients into those walking in corridors versus walking or sitting only inside the room. The continence-care classification decision tree identified two characteristics that subdivided the patients into four nodes: pre-admission functioning and bladder control. Nurse-to-patient-ratio variables and patients' nursing-related care needs did not contribute to this classification. The food-intake decision tree identified four characteristics-pre-admission functioning, gender, percentage of nurses with advanced training and percentage of nurse aides-subdividing patients into eight nodes. Low-functioning patients exposed to a higher percentage of nurses with advanced training had food-intake scores 14% higher than patients exposed to a lower percentage of nurses with advanced training. Independent men exposed to a higher percentage of nurse aides had a 14% higher habitual daily in-hospital food-intake score than independent men exposed to a lower percentage of nurse aides.

CONCLUSIONS

A higher percentage of nurses with post-graduate education is associated with better performance of mobility and food intake of hospitalized older adults. To maintain the potential benefits of highly trained staff, education levels should be considered in scheduling and assignment decision-making processes in internal medicine units. Tweetable abstract: A higher percentage of nurses with post-graduate education is associated with better mobility and food intake of hospitalized older adults.

Ward staffing guided by a patient classification system: A multi-criteria analysis of "fit" in three acute hospitals

AIMS

To assess how well the Safer Nursing Care Tool (SNCT) predicts staffing requirements on hospital wards, and to use professional judgement to generate hypotheses about factors associated with a "poor fit".

BACKGROUND

The SNCT is widely used in the UK, but there is scant evidence about factors that influence the quality of staffing decisions based upon such patient classification systems.

METHODS

Secondary analysis of data from 69 wards in three acute hospitals to assess the precision of the estimated staffing requirement, variation of estimates, correspondence with professional judgement and achieved staffing levels. Nursing workforce leads suggested factors associated with poor fit, based on the wards that rated worst.

RESULTS

39% of wards were frequently understaffed, while frequent overstaffing was less common (12%). 24% of wards needed a sample of over 182 days to estimate the establishment precisely. Potential reasons identified for poor fit included high turnover, older patients, high levels of 1-to-1 specialing, cancer care, small ward size and high within-day variation in demand.

CONCLUSIONS

Using a staffing tool without applying professional judgement or triangulating against other methods can lead to inaccurate estimates of staffing requirements and unsafe staffing levels.

IMPLICATIONS FOR NURSING MANAGEMENT

Despite the availability of software to calculate staffing requirements, application of professional judgement remains essential.

The Safer Nursing Care Tool as a guide to nurse staffing requirements on hospital wards: observational and modelling study

Background

The Safer Nursing Care Tool is a system designed to guide decisions about nurse staffing requirements on hospital wards, in particular the number of nurses to employ (establishment). The Safer Nursing Care Tool is widely used in English hospitals but there is a lack of evidence about how effective and cost-effective nurse staffing tools are at providing the staffing levels needed for safe and quality patient care.

Objectives

To determine whether or not the Safer Nursing Care Tool corresponds to professional judgement, to assess a range of options for using the Safer Nursing Care Tool and to model the costs and consequences of various ward staffing policies based on Safer Nursing Care Tool acuity/dependency measure.

Design

This was an observational study on medical/surgical wards in four NHS hospital trusts using regression, computer simulations and economic modelling. We compared the effects and costs of a ‘high’ establishment (set to meet demand on 90% of days), the ‘standard’ (mean-based) establishment and a ‘flexible (low)’ establishment (80% of the mean) providing a core staff group that would be sufficient on days of low demand, with flexible staff re-deployed/hired to meet fluctuations in demand.

Setting

Medical/surgical wards in four NHS hospital trusts.

Main outcome measures

The main outcome measures were professional judgement of staffing adequacy and reports of omissions in care, shifts staffed more than 15% below the measured requirement, cost per patient-day and cost per life saved.

Data sources

The data sources were hospital administrative systems, staff reports and national reference costs.

Results

In total, 81 wards participated (85% response rate), with data linking Safer Nursing Care Tool ratings and staffing levels for 26,362 wards × days (96% response rate). According to Safer Nursing Care Tool measures, 26% of all ward-days were understaffed by ≥ 15%. Nurses reported that they had enough staff to provide quality care on 78% of shifts. When using the Safer Nursing Care Tool to set establishments, on average 60 days of observation would be needed for a 95% confidence interval spanning 1 whole-time equivalent either side of the mean. Staffing levels below the daily requirement estimated using the Safer Nursing Care Tool were associated with lower odds of nurses reporting ‘enough staff for quality’ and more reports of missed nursing care. However, the relationship was effectively linear, with staffing above the recommended level associated with further improvements. In simulation experiments, ‘flexible (low)’ establishments led to high rates of understaffing and adverse outcomes, even when temporary staff were readily available. Cost savings were small when high temporary staff availability was assumed. ‘High’ establishments were associated with substantial reductions in understaffing and improved outcomes but higher costs, although, under most assumptions, the cost per life saved was considerably less than £30,000.

Limitations

This was an observational study. Outcomes of staffing establishments are simulated.

Conclusions

Understanding the effect on wards of variability of workload is important when planning staffing levels. The Safer Nursing Care Tool correlates with professional judgement but does not identify optimal staffing levels. Employing more permanent staff than recommended by the Safer Nursing Care Tool guidelines, meeting demand most days, could be cost-effective. Apparent cost savings from ‘flexible (low)’ establishments are achieved largely by below-adequate staffing. Cost savings are eroded under the conditions of high temporary staff availability that are required to make such policies function.

Future work

Research is needed to identify cut-off points for required staffing. Prospective studies measuring patient outcomes and comparing the results of different systems are feasible.

Trial registration

Current Controlled Trials ISRCTN12307968.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full inHealth Services and Delivery Research; Vol. 8, No. 16. See the NIHR Journals Library website for further project information.

Nursing workload, nurse staffing methodologies and tools: A systematic scoping review and discussion

Background

The importance of nurse staffing levels in acute hospital wards is widely recognised but evidence for tools to determine staffing requirements although extensive, has been reported to be weak. Building on a review of reviews undertaken in 2014, we set out to give an overview of the major approaches to assessing nurse staffing requirements and identify recent evidence in order to address unanswered questions including the accuracy and effectiveness of tools.

Methods

We undertook a systematic scoping review. Searches of Medline, the Cochrane Library and CINAHL were used to identify recent primary research, which was reviewed in the context of conclusions from existing reviews.

Results

The published literature is extensive and describes a variety of uses for tools including establishment setting, daily deployment and retrospective review. There are a variety of approaches including professional judgement, simple volume-based methods (such as patient-to-nurse ratios), patient prototype/classification and timed-task approaches. Tools generally attempt to match staffing to a mean average demand or time requirement despite evidence of skewed demand distributions. The largest group of recent studies reported the evaluation of (mainly new) tools and systems, but provides little evidence of impacts on patient care and none on costs. Benefits of staffing levels set using the tools appear to be linked to increased staffing with no evidence of tools providing a more efficient or effective use of a given staff resource. Although there is evidence that staffing assessments made using tools may correlate with other assessments, different systems lead to dramatically different estimates of staffing requirements. While it is evident that there are many sources of variation in demand, the extent to which systems can deliver staffing levels to meet such demand is unclear. The assumption that staffing to meet average need is the optimal response to varying demand is untested and may be incorrect.

Conclusions

Despite the importance of the question and the large volume of publication evidence about nurse staffing methods remains highly limited. There is no evidence to support the choice of any particular tool. Future research should focus on learning more about the use of existing tools rather than simply developing new ones. Priority research questions include how best to use tools to identify the required staffing level to meet varying patient need and the costs and consequences of using tools.

Tweetable abstract

Decades of research on tools to determine nurse staffing requirements is largely uninformative. Little is known about the costs or consequences of widely used tools.

Predicting patient acuity according to their main problem

AIM

To assess the ability of the patient main problem to predict acuity in adults admitted to hospital wards and step-down units.

BACKGROUND

Acuity refers to the categorization of patients based on their required nursing intensity. The relationship between acuity and nurses' clinical judgment on the patient problems, including their prioritization, is an underexplored issue.

METHOD

Cross-sectional, multi-centre study in a sample of 200,000 adults. Multivariate analysis of main problems potentially associated with acuity levels higher than acute was performed. Distribution of patients and outcome differences among acuity clusters were evaluated.

RESULTS

The main problems identified are strongly associated with patient acuity. The model exhibits remarkable ability to predict acuity (AUC, 0.814; 95% CI, 0.81-0.816). Most patients (64.8%) match higher than acute categories. Significant differences in terms of mortality, hospital readmission and other outcomes are observed (p < .005).

CONCLUSION

The patient main problem predicts acuity. Most inpatients require more intensive than acute nursing care and their outcomes are adversely affected.

IMPLICATIONS FOR NURSING MANAGEMENT

Prospective measurement of acuity, considering nurses' clinical judgments on the patient main problem, is feasible and may contribute to support nurse management workforce planning and staffing decision-making, and to optimize patients, nurses and organizational outcomes.

Assessing the relation of the coded nursing care and nursing intensity data: Towards the exploitation of clinical data for administrative use and the design of nursing workload

Patient-care data from the electronic health record systems are increasingly in demand for re-use in administration and resource planning. Nursing documentation with coded concepts is expected to produce more reliable data, fulfilling better requirements for re-use. The aim was to ascertain what kind of relation exist between coded nursing diagnoses, nursing interventions, and nursing intensity and to discuss the possibilities for re-using nursing data for workload design. We analysed the retrospective nursing records of 794 patients documented by the Finnish Care Classification and nursing intensity data assessed by the Oulu Patient Classification over a 15-day period in nine inpatient units at a university hospital. Using the generalised linear mixed model, the clear relationship between the number of coded nursing notes and nursing intensity levels were ascertained. The number of coded nursing notes increases when the nursing intensity increases. The outcomes construct a good basis for continuing elaboration of electronic health record data re-use.

Using a new interrater reliability method to test the modified Oulu Patient Classification instrument in home health care

Aim

To test the interrater reliability of the modified Oulu Patient Classification instrument, using a multiple parallel classification method based on oral case presentations in home health care in Norway.

Design

Reliability study.

Methods

Data were collected at two municipal home healthcare units during 2013-2014. The reliability of the modified OPCq instrument was tested using a new multiple parallel classification method. The data material consisted of 2 010 parallel classifications, analysed using consensus in per cent and Cohen's kappa. Cronbach's alpha was used to measure internal consistency.

Results

For parallel classifications, consensus varied between 64.78-77.61%. Interrater reliability varied between 0.49-0.69 (Cohen's kappa), the internal consistency between 0.81-0.94 (Cronbach's alpha). Analysis of the raw scores showed 27.2% classifications had the same points, 39.1% differed one point, 17.9% differed two points and 16.5% differed ≥3 points.