Applying computerized adaptive testing to the Negative Acts Questionnaire-Revised: Rasch analysis of workplace bullying

The mediating effect of job satisfaction on the relationship between workplace bullying and organizational citizenship behavior in nurses

BACKGROUND: Establishing strategies for improving nursing shortages, which are labor challenges in the current health care industry. OBJECTIVE: This study aimed to examine the correlation between workplace bullying and organizational citizenship behavior (OCB) in nurses and the mediating effects of job satisfaction on this relationship. METHODS: A total of 164 valid samples were obtained. The Negative Acts Questionnaire-Revised, the Minnesota Satisfaction Questionnaire, and an OCB scale were measured. RESULTS: The results indicate that a significantly larger proportion of nurses working in operating rooms (Δ odds ratio, odds = 2.30, p = 0.043), the emergency room, and the ICU (Δ odds = 2.79, p = 0.019) had suffered workplace bullying compared with nurses working in patient wards. No experience of workplace bullying exerted a positive and significant effect on job satisfaction (p < 0.001), and job satisfaction exerted a positive and significant effect on overall OCB (p < 0.001). No experience of workplace bullying exerted a significant mediating effect on the influence of job satisfaction on overall OCB (p < 0.001). CONCLUSIONS: The department of service in which a nurse works influences the occurrence of workplace bullying, previous experience with bullying reduces job satisfaction, and greater job satisfaction promotes higher OCB performance. Based on the study results, we advise that nursing executives address and prevent workplace bullying to increase the job satisfaction of nurses so that nurses are willing to display OCB, apply their expertise, and expand the role and functions of nursing.

An app for predicting nurse intention to quit the job using artificial neural networks (ANNs) in Microsoft Excel

Background:

Numerous studies have identified factors related to nurses’ intention to leave. However, none has successfully predicted the nurse’s intention to quit the job. Whether the intention to quit the job can be predicted is an interesting topic in healthcare settings. A model to predict the nurse’s intention to quit the job for novice nurses should be investigated. The aim of this study is to build a model to develop an app for the automatic prediction and classification of nurses’ intention to quit their jobs.

Methods:

We recruited 1104 novice nurses working in 6 medical centers in Taiwan to complete 100-item questionnaires related to the nurse’s intention to quit the job in October 2018. The k-mean was used to divide nurses into 2 classes based on 5 items regarding leave intention. Feature variables were selected from the 100-item survey. Two models, including an artificial neural network (ANN) and a convolutional neural network, were compared across 4 scenarios made up of 2 training sets (n = 1104 and n = 804 ≅ 70%) and their corresponding testing (n = 300 ≅ 30%) sets to verify the model accuracy. An app for predicting the nurse’s intention to quit the job was then developed as a website assessment.

Results:

We observed that 24 feature variables extracted from this study in the ANN model yielded a higher area under the ROC curve of 0.82 (95% CI 0.80-0.84) based on the 1104 cases, the ANN performed better than the convolutional neural network on the accuracy, and a ready and available app for predicting the nurse’s intention to quit the job was successfully developed in this study.

Conclusions:

A 24-item ANN model with 53 parameters estimated by the ANN was developed to improve the accuracy of nurses’ intention to quit their jobs. The app would help team leaders take care of nurses who intend to quit the job before their actions are taken.

Key Points

Web-Based Skin Cancer Assessment and Classification Using Machine Learning and Mobile Computerized Adaptive Testing in a Rasch Model: Development Study

Background

Web-based computerized adaptive testing (CAT) implementation of the skin cancer (SC) risk scale could substantially reduce participant burden without compromising measurement precision. However, the CAT of SC classification has not been reported in academics thus far.

Objective

We aim to build a CAT-based model using machine learning to develop an app for automatic classification of SC to help patients assess the risk at an early stage.

Methods

We extracted data from a population-based Australian cohort study of SC risk (N=43,794) using the Rasch simulation scheme. All 30 feature items were calibrated using the Rasch partial credit model. A total of 1000 cases following a normal distribution (mean 0, SD 1) based on the item and threshold difficulties were simulated using three techniques of machine learning—naïve Bayes, k-nearest neighbors, and logistic regression—to compare the model accuracy in training and testing data sets with a proportion of 70:30, where the former was used to predict the latter. We calculated the sensitivity, specificity, receiver operating characteristic curve (area under the curve [AUC]), and CIs along with the accuracy and precision across the proposed models for comparison. An app that classifies the SC risk of the respondent was developed.

Results

We observed that the 30-item k-nearest neighbors model yielded higher AUC values of 99% and 91% for the 700 training and 300 testing cases, respectively, than its 2 counterparts using the hold-out validation but had lower AUC values of 85% (95% CI 83%-87%) in the k-fold cross-validation and that an app that predicts SC classification for patients was successfully developed and demonstrated in this study.

Conclusions

The 30-item SC prediction model, combined with the Rasch web-based CAT, is recommended for classifying SC in patients. An app we developed to help patients self-assess SC risk at an early stage is required for application in the future.

An application for classifying perceptions on my health bank in Taiwan using convolutional neural networks and web-based computerized adaptive testing: A development and usability study

BACKGROUND

The classification of a respondent's opinions online into positive and negative classes using a minimal number of questions is gradually changing and helps turn techniques into practices. A survey incorporating convolutional neural networks (CNNs) into web-based computerized adaptive testing (CAT) was used to collect perceptions on My Health Bank (MHB) from users in Taiwan. This study designed an online module to accurately and efficiently turn a respondent's perceptions into positive and negative classes using CNNs and web-based CAT.

METHODS

In all, 640 patients, family members, and caregivers with ages ranging from 20 to 70 years who were registered MHB users were invited to complete a 3-domain, 26-item, 5-category questionnaire asking about their perceptions on MHB (PMHB26) in 2019. The CNN algorithm and k-means clustering were used for dividing respondents into 2 classes of unsatisfied and satisfied classes and building a PMHB26 predictive model to estimate parameters. Exploratory factor analysis, the Rasch model, and descriptive statistics were used to examine the demographic characteristics and PMHB26 factors that were suitable for use in CNNs and Rasch multidimensional CAT (MCAT). An application was then designed to classify MHB perceptions.

RESULTS

We found that 3 construct factors were extracted from PMHB26. The reliability of PMHB26 for each subscale beyond 0.94 was evident based on internal consistency and stability in the data. We further found the following: the accuracy of PMHB26 with CNN yields a higher accuracy rate (0.98) with an area under the curve of 0.98 (95% confidence interval, 0.97-0.99) based on the 391 returned questionnaires; and for the efficiency, approximately one-third of the items were not necessary to answer in reducing the respondents' burdens using Rasch MCAT.

CONCLUSIONS

The PMHB26 CNN model, combined with the Rasch online MCAT, is recommended for improving the accuracy and efficiency of classifying patients' perceptions of MHB utility. An application developed for helping respondents self-assess the MHB cocreation of value can be applied to other surveys in the future.

Workplace bullying among operating room nurses in China: A cross-sectional survey

PURPOSE

To investigate the prevalence and level of severity of workplace bullying among operating room nurses and to identify the risk factors that contribute to workplace bullying in operating room nurses in China.

DESIGN AND METHODS

This descriptive research was conducted on 411 nurses from six medical centers in Harbin using a structured questionnaire.

FINDINGS

The prevalence of workplace bullying was 15.8%. There were significant differences in workplace bullying by sex, hospital level, and marital status. Stepwise multiple regression analysis indicated that gender and marital status were significant determinants of workplace bullying.

PRACTICE IMPLICATIONS

These findings portray a comprehensive landscape of workplace bullying among operating room nurses in China. Understanding the factors that influence workplace bullying may enhance the recognition and management of bullying behaviors.

An App for Classifying Personal Mental Illness at Workplace Using Fit Statistics and Convolutional Neural Networks: Survey-Based Quantitative Study

BACKGROUND

Mental illness (MI) is common among those who work in health care settings. Whether MI is related to employees' mental status at work is yet to be determined. An MI app is developed and proposed to help employees assess their mental status in the hope of detecting MI at an earlier stage.

OBJECTIVE

This study aims to build a model using convolutional neural networks (CNNs) and fit statistics based on 2 aspects of measures and outfit mean square errors for the automatic detection and classification of personal MI at the workplace using the emotional labor and mental health (ELMH) questionnaire, so as to equip the staff in assessing and understanding their own mental status with an app on their mobile device.

METHODS

We recruited 352 respiratory therapists (RTs) working in Taiwan medical centers and regional hospitals to fill out the 44-item ELMH questionnaire in March 2019. The exploratory factor analysis (EFA), Rasch analysis, and CNN were used as unsupervised and supervised learnings for (1) dividing RTs into 4 classes (ie, MI, false MI, health, and false health) and (2) building an ELMH predictive model to estimate 108 parameters of the CNN model. We calculated the prediction accuracy rate and created an app for classifying MI for RTs at the workplace as a web-based assessment.

RESULTS

We observed that (1) 8 domains in ELMH were retained by EFA, (2) 4 types of mental health (n=6, 63, 265, and 18 located in 4 quadrants) were classified using the Rasch analysis, (3) the 44-item model yields a higher accuracy rate (0.92), and (4) an MI app available for RTs predicting MI was successfully developed and demonstrated in this study.

CONCLUSIONS

The 44-item model with 108 parameters was estimated by using CNN to improve the accuracy of mental health for RTs. An MI app developed to help RTs self-detect work-related MI at an early stage should be made more available and viable in the future.

An App for Detecting Bullying of Nurses Using Convolutional Neural Networks and Web-Based Computerized Adaptive Testing: Development and Usability Study

Background

Workplace bullying has been measured in many studies to investigate its effects on mental health issues. However, none have used web-based computerized adaptive testing (CAT) with bully classifications and convolutional neural networks (CNN) for reporting the extent of individual bullying in the workplace.

Objective

This study aims to build a model using CNN to develop an app for automatic detection and classification of nurse bullying-levels, incorporated with online Rasch computerized adaptive testing, to help assess nurse bullying at an earlier stage.

Methods

We recruited 960 nurses working in a Taiwan Ch-Mei hospital group to fill out the 22-item Negative Acts Questionnaire-Revised (NAQ-R) in August 2012. The k-mean and the CNN were used as unsupervised and supervised learnings, respectively, for: (1) dividing nurses into three classes (n=918, 29, and 13 with suspicious mild, moderate, and severe extent of being bullied, respectively); and (2) building a bully prediction model to estimate 69 different parameters. Finally, data were separated into training and testing sets in a proportion of 70:30, where the former was used to predict the latter. We calculated the sensitivity, specificity, and receiver operating characteristic curve (area under the curve [AUC]), along with the accuracy across studies for comparison. An app predicting the respondent bullying-level was developed, involving the model’s 69 estimated parameters and the online Rasch CAT module as a website assessment.

Results

We observed that: (1) the 22-item model yields higher accuracy rates for three categories, with an accuracy of 94% for the total 960 cases, and accuracies of 99% (AUC 0.99; 95% CI 0.99-1.00) and 83% (AUC 0.94; 95% CI 0.82-0.99) for the lower and upper groups (cutoff points at 49 and 66 points) based on the 947 cases and 42 cases, respectively; and (2) the 700-case training set, with 95% accuracy, predicts the 260-case testing set reaching an accuracy of 97. Thus, a NAQ-R app for nurses that predicts bullying-level was successfully developed and demonstrated in this study.

Conclusions

The 22-item CNN model, combined with the Rasch online CAT, is recommended for improving the accuracy of the nurse NAQ-R assessment. An app developed for helping nurses self-assess workplace bullying at an early stage is required for application in the future.

An App Developed for Detecting Nurse Burnouts Using the Convolutional Neural Networks in Microsoft Excel: Population-Based Questionnaire Study

Background

Burnout (BO), a critical syndrome particularly for nurses in health care settings, substantially affects their physical and psychological status, the institute’s well-being, and indirectly, patient outcomes. However, objectively classifying BO levels has not been defined and noticed in the literature.

Objective

The aim of this study is to build a model using the convolutional neural network (CNN) to develop an app for automatic detection and classification of nurse BO using the Maslach Burnout Inventory–Human Services Survey (MBI-HSS) to help assess nurse BO at an earlier stage.

Methods

We recruited 1002 nurses working in a medical center in Taiwan to complete the Chinese version of the 20-item MBI-HSS in August 2016. The k-mean and CNN were used as unsupervised and supervised learnings for dividing nurses into two classes (n=531 and n=471 of suspicious BO+ and BO−, respectively) and building a BO predictive model to estimate 38 parameters. Data were separated into training and testing sets in a proportion 70%:30%, and the former was used to predict the latter. We calculated the sensitivity, specificity, and receiver operating characteristic curve (area under the curve) across studies for comparison. An app predicting respondent BO was developed involving the model’s 38 estimated parameters for a website assessment.

Results

We observed that (1) the 20-item model yields a higher accuracy rate (0.95) with an area under the curve of 0.97 (95% CI 0.94-0.95) based on the 1002 cases, (2) the scheme named matching personal response to adapt for the correct classification in model drives the prior model’s predictive accuracy at 100%, (3) the 700-case training set with 0.96 accuracy predicts the 302-case testing set reaching an accuracy of 0.91, and (4) an available MBI-HSS app for nurses predicting BO was successfully developed and demonstrated in this study.

Conclusions

The 20-item model with the 38 parameters estimated by using CNN for improving the accuracy of nurse BO has been particularly demonstrated in Excel (Microsoft Corp). An app developed for helping nurses to self-assess job BO at an early stage is required for application in the future.

Bullying in Fly-In-Fly-Out employees in the Australian resources sector: A cross-sectional study

Background

Workplace bullying has diverse consequences at both the organisational and individual level. Anecdotal reports indicate that workplace bullying is an issue of particular concern for Australian FIFO workers, which may impact on psychosocial distress. However, no prior studies have examined this issue empirically in a FIFO worker cohort.

Methods and materials

A cross-sectional survey study design was used to establish the prevalence of bullying in Australian FIFO, antecedents of bullying, and its association with psychosocial distress. Responses were received from 580 FIFO workers in the Australian resources sector. Primary outcome measures were Negative Acts Questionnaire-Revised, Beck Depression Inventory II, and Beck Hopelessness Scale. Logistic regression models were constructed to examine the association between bullying, suicide risk, and clinical depression.

Results

Over half of the respondents experienced workplace bullying (55.7%), and about one-third reported moderate or more severe depression (32.3%). Being above the median age (OR = 0.51; 95% CI = 0.31–0.83) and having a supervisor who failed to promote collaboration (OR = 3.04; 95% CI = 1.84–5.04) were both significantly associated with experiencing bullying. Bullying was associated with an almost threefold increase in the likelihood of participants reporting increased suicide risk (OR = 2.70; 95% CI = 1.53–4.76). Bullying was also associated with participants being almost two and a half times more likely to report clinical depression (OR = 2.38; 95% CI = 1.40–4.05).

Conclusion

The incidence of bullying in Australian FIFO workers has reached alarming proportions. Bullying was significantly associated with higher levels of clinical depression and suicide risk. The results highlight the need to implement in the Australian resource sector interventions that reduce workplace bullying.

Classification and Regression Tree and Computer Adaptive Testing in Cardiac Rehabilitation: Instrument Validation Study

Background

There is a need for shorter-length assessments that capture patient questionnaire data while attaining high data quality without an undue response burden on patients. Computerized adaptive testing (CAT) and classification and regression tree (CART) methods have the potential to meet these needs and can offer attractive options to shorten questionnaire lengths.

Objective

The objective of this study was to test whether CAT or CART was best suited to reduce the number of questionnaire items in multiple domains (eg, anxiety, depression, quality of life, and social support) used for a needs assessment procedure (NAP) within the field of cardiac rehabilitation (CR) without the loss of data quality.

Methods

NAP data of 2837 CR patients from a multicenter Cardiac Rehabilitation Decision Support System (CARDSS) Web-based program was used. Patients used a Web-based portal, MyCARDSS, to provide their data. CAT and CART were assessed based on their performances in shortening the NAP procedure and in terms of sensitivity and specificity.

Results

With CAT and CART, an overall reduction of 36% and 72% of NAP questionnaire length, respectively, was achieved, with a mean sensitivity and specificity of 0.765 and 0.817 for CAT, 0.777 and 0.877 for classification trees, and 0.743 and 0.40 for regression trees, respectively.

Conclusions

Both CAT and CART can be used to shorten the questionnaires of the NAP used within the field of CR. CART, however, showed the best performance, with a twice as large overall decrease in the number of questionnaire items of the NAP compared to CAT and the highest sensitivity and specificity. To our knowledge, our study is the first to assess the differences in performance between CAT and CART for shortening questionnaire lengths. Future research should consider administering varied assessments of patients over time to monitor their progress in multiple domains. For CR professionals, CART integrated with MyCARDSS would provide a feedback loop that informs the rehabilitation progress of their patients by providing real-time patient measurements.

Detecting hospital behaviors of up-coding on DRGs using Rasch model of continuous variables and online cloud computing in Taiwan

BACKGROUND

This work aims to apply data-detection algorithms to predict the possible deductions of reimbursement from Taiwan's Bureau of National Health Insurance (BNHI), and to design an online dashboard to send alerts and reminders to physicians after completing their patient discharge summaries.

METHODS

Reimbursement data for discharged patients were extracted from a Taiwan medical center in 2016. Using the Rasch model of continuous variables, we applied standardized residual analyses to 20 sets of norm-referenced diagnosis-related group (DRGs), each with 300 cases, and compared these to 194 cases with deducted records from the BNHI. We then examine whether the results of prediction using the Rasch model have a high probability associated with the deducted cases. Furthermore, an online dashboard was designed for use in the online monitoring of possible deductions on fee items in medical settings.

RESULTS

The results show that 1) the effects deducted by the NHRI can be predicted with an accuracy rate of 0.82 using the standardized residual approach of the Rasch model; 2) the accuracies for drug, medical material and examination fees are not associated among different years, and all of those areas under the ROC curve (AUC) are significantly greater than the randomized probability of 0.50; and 3) the online dashboard showing the possible deductions on fee items can be used by hospitals in the future.

CONCLUSION

The DRG-based comparisons in the possible deductions on medical fees, along with the algorithm based on Rasch modeling, can be a complementary tool in upgrading the efficiency and accuracy in processing medical fee applications in the discernable future.

Application of a multidimensional computerized adaptive test for a Clinical Dementia Rating Scale through computer-aided techniques

BACKGROUND

With the increasingly rapid growth of the elderly population, individuals aged 65 years and above now compose 14% of Taiwanese citizens, thereby making Taiwanese society an aged society. A leading factor that affects the elderly population is dementia. A method of precisely and efficiently examining patients with dementia through multidimensional computer adaptive testing (MCAT) to accurately determine the patients' stage of dementia needs to be developed. This study aimed to develop online MCAT that family members can use on their own computers, tablets, or smartphones to predict the extent of dementia for patients responding to the Clinical Dementia Rating (CDR) instrument.

METHODS

The CDR was applied to 366 outpatients in a hospital in Taiwan. MCAT was employed with parameters for items across eight dimensions, and responses were simulated to compare the efficiency and precision between MCAT and non-adaptive testing (NAT). The number of items saved and the estimated person measures was compared between the results of MCAT and NAT, respectively.

RESULTS

MCAT yielded substantially more precise measurements and was considerably more efficient than NAT. MCAT achieved 20.19% (= [53 - 42.3]/53) saving in item length when the measurement differences were less than 5%. Pearson correlation coefficients were highly consistent among the eight domains. The cut-off points for the overall measures were - 1.4, - 0.4, 0.4, and 1.4 logits, which was equivalent to 20% for each portion in percentile scores. Substantially fewer items were answered through MCAT than through NAT without compromising the precision of MCAT.

CONCLUSIONS

Developing a website that family members can use on their own computers, tablets, and smartphones to help them perform online screening and prediction of dementia in older adults is useful and manageable.

Determining the optimal cut-off scores for the Workplace Bullying Inventory

BACKGROUND

Over the past two decades, there has been considerable research into workplace bullying. One area that remains poorly developed is a tool with the capacity to accurately differentiate between exposed and unexposed employees.

AIM

To determine optimal cut-off scores for the Workplace Bullying Inventory (WBI) that accurately classify cases of exposure to workplace bullying.

DISCUSSION

Secondary analysis of data collected from Australian public sector employees ( n =2,197) was conducted. Receiver operator characteristic (ROC) curve analysis was used with a minimum sensitivity of 80%, to determine those scores on the WBI that corresponded with the highest accuracy of the tool to distinguish cases from non-cases. The results suggest using a cut score of 29 from the total score on the WBI (possible range: 18-90). When compared to a sum-score from a single dichotomous self-report variable, the cut-off score estimated a more conservative bullying rate. The single-item rate was potentially inflated by misconceptions about what constitutes bullying in the workplace.

CONCLUSION

Employing validated cut-off points for exposure provides an objective threshold for establishing exposure to workplace bullying. The results of the analysis provide a more rigorous approach to quantifying exposure to workplace bullying, in a tool that has been designed and tested in the nursing workforce. This is the first such tool with empirically-derived, discriminant accuracy.

IMPLICATIONS FOR PRACTICE

It is common for nurse researchers to employ sum-scores from single items to identify exposure to workplace bullying. By providing reliable cut-off points for exposure, this study offers standardised, diagnostic accuracy for researchers, clinicians and managers.

Development of a Microsoft Excel tool for one-parameter Rasch model of continuous items: an application to a safety attitude survey

Background

Many continuous item responses (CIRs) are encountered in healthcare settings, but no one uses item response theory’s (IRT) probabilistic modeling to present graphical presentations for interpreting CIR results. A computer module that is programmed to deal with CIRs is required.

To present a computer module, validate it, and verify its usefulness in dealing with CIR data, and then to apply the model to real healthcare data in order to show how the CIR that can be applied to healthcare settings with an example regarding a safety attitude survey.

Methods

Using Microsoft Excel VBA (Visual Basic for Applications), we designed a computer module that minimizes the residuals and calculates model’s expected scores according to person responses across items. Rasch models based on a Wright map and on KIDMAP were demonstrated to interpret results of the safety attitude survey.

Results

The author-made CIR module yielded OUTFIT mean square (MNSQ) and person measures equivalent to those yielded by professional Rasch Winsteps software. The probabilistic modeling of the CIR module provides messages that are much more valuable to users and show the CIR advantage over classic test theory.

Conclusions

Because of advances in computer technology, healthcare users who are familiar to MS Excel can easily apply the study CIR module to deal with continuous variables to benefit comparisons of data with a logistic distribution and model fit statistics.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0276-2) contains supplementary material, which is available to authorized users.

Hospital nurses' attitudes, negative perceptions, and negative acts regarding workplace bullying

BACKGROUND

Workplace bullying is a prevalent problem in today's work places that has adverse effects on both bullying victims and organizations. To investigate the predictors of workplace bullying is an important task to prevent bullying victims of nurses in hospitals.

OBJECTIVE

This study aims to explore the relationships among nurses' attitudes, negative perceptions, and negative acts regarding workplace bullying under the framework of the theory of planned behavior (TPB).

METHODS

A total of 811 nurses from three hospitals in Taiwan were surveyed. Nurses' responses to the 201 items of 10 scales were calibrated using Rasch analysis and then subjected to path analysis with partial least-squares structural equation modeling (PLS-SEM).

RESULTS

The instrumental attitude was significant predictors of nurses' negative perceptions to be bullied in the workplace. Instead, the other TPB components of subjective norm and perceived behavioral control were not effective predictors of nurses' negative acts regarding workplace bullying.

CONCLUSIONS

The findings provided hospital nurse management with important implications for prevention of bullying, particularly to them who are tasked with providing safer and more productive workplaces to hospital nurses. Awareness of workplace bullying was recommended to other kinds of workplaces for further studies in future.

A new technique to measure online bullying: online computerized adaptive testing

BACKGROUND

Workplace bullying has been measured in many studies to investigate mental health issues. None uses online computerized adaptive testing (CAT) with cutting points to report bully prevalence at workplace.

OBJECTIVE

To develop an online CAT to examine person being bullied and verify whether item response theory-based CAT can be applied online for nurses to measure exposure to workplace bullying.

METHODS

A total of 963 nurses were recruited and responded to the 22-item Negative Acts Questionnaire-Revised (NAQ-R). All non-adaptive testing (NAT) items were calibrated with the Rasch rating scale model. Three scenarios (i.e., NAT, CAT, and the randomly selected method to NAT) were manipulated to compare their response efficiency and precision by comparing (i) item length for answering questions, person measure, (ii) correlation coefficients, (iii) paired t tests, and (iv) estimated standard errors (SE) between CAT and the random to its counterpart of NAT.

RESULTS

The NAQ-R is a unidimensional construct that can be applied for nurses to measure exposure to workplace bullying on CAT. CAT required fewer items (=8.9) than NAT (=22, an efficient gain of 60% =1-8.9/22). Nursing measures derived from both tests (CAT and the random to NAT) were highly correlated (r = 0.93 and 0.96) and their measurement precisions were not statistically different (the percentage of significant count number less than 5%) as expected, but CAT earns smaller person measure SE than the random scenario. The prevalence rate for nurses was 1.5% (=15/963) when cutting points set at -0.7 and 0.7 logits.

CONCLUSION

The CAT-based NAQ-R reduces respondents' burden without compromising measurement precision and increases endorsement efficiency. The online CAT is recommended for assessing nurses using the criteria at -0.7 and 0.7 (or <30 and <60 in summed score) to identify bully grade as one of the three levels (high, moderate, and low). The bullied nurse can get help from a psychiatrist or a mental health expert at an earlier stage.

Simulation study of activities of daily living functions using online computerized adaptive testing

Background

Computer adaptive testing (CAT) of the activities of daily living (ADL) functions is required (i) to reveal the advantages of using an efficient and accurate estimation method, (ii) to determine the cutpoint for classifying ADL strata in patients with stroke, and (iii) to evaluate the feasibility of online CAT used in clinical settings for smartphones.

Methods

Normally standardized distributions of ADL measurements were simulated using item parameters from published papers. We retrieved item parameters of the combined Barthel Index and Frenchay Activities Index from the literature (the 23-item comprehensive ADL [CADL] and 34-item ADL scales) and simulated three 1000-person measures from a normal standard CAT distribution: [i] CADL (CADL-CAT), [ii] ADL (ADL-CAT), and [iii] NAT (Non-Adaptive Testing). The cutpoints of ADL person strata were determined using a norm-reference method. Maximum a posteriori estimation, expected a posteriori estimation, and maximum likelihood estimation (MAP) were used to compare the Pearson correlation coefficients and different number ratios of paired measures yielded by CAT and NAT. The number of items and the cutpoints for the scale were separately determined.

Results

We found that (i) correlation coefficients for the three CAT-estimated measures were 0.77 (CADL), 0.93 (Male ADL), and 0.93 (Female ADL) compared with their NAT counterparts. Different number ratios of person-paired measures between CAT and NAT for the three scales were all less than 5 %, indicating no difference exists between CAT and NAT. However, CAT might be 66 % more efficient than NAT. (ii) The estimated cutpoints of T scores (i.e., with a mean of 50 and a standard deviation of 10) were 45, 55, and 65 (e.g., separating person ADL function to four strata with not active, fairly active, active, and very active). (iii) An available-for-download online ADL-CAT APP for clinical practice was demonstrated.

Conclusions

An online ADL-CAT APP using the MAP method was created and used on smartphones to classify ADL strata in patients with stroke.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0370-8) contains supplementary material, which is available to authorized users.

Improving Inpatient Surveys: Web-Based Computer Adaptive Testing Accessed via Mobile Phone QR Codes

Background

The National Health Service (NHS) 70-item inpatient questionnaire surveys inpatients on their perceptions of their hospitalization experience. However, it imposes more burden on the patient than other similar surveys. The literature shows that computerized adaptive testing (CAT) based on item response theory can help shorten the item length of a questionnaire without compromising its precision.

Objective

Our aim was to investigate whether CAT can be (1) efficient with item reduction and (2) used with quick response (QR) codes scanned by mobile phones.

Methods

After downloading the 2008 inpatient survey data from the Picker Institute Europe website and analyzing the difficulties of this 70-item questionnaire, we used an author-made Excel program using the Rasch partial credit model to simulate 1000 patients’ true scores followed by a standard normal distribution. The CAT was compared to two other scenarios of answering all items (AAI) and the randomized selection method (RSM), as we investigated item length (efficiency) and measurement accuracy. The author-made Web-based CAT program for gathering patient feedback was effectively accessed from mobile phones by scanning the QR code.

Results

We found that the CAT can be more efficient for patients answering questions (ie, fewer items to respond to) than either AAI or RSM without compromising its measurement accuracy. A Web-based CAT inpatient survey accessed by scanning a QR code on a mobile phone was viable for gathering inpatient satisfaction responses.

Conclusions

With advances in technology, patients can now be offered alternatives for providing feedback about hospitalization satisfaction. This Web-based CAT is a possible option in health care settings for reducing the number of survey items, as well as offering an innovative QR code access.

Estimating Skin Cancer Risk: Evaluating Mobile Computer-Adaptive Testing

Background

Response burden is a major detriment to questionnaire completion rates. Computer adaptive testing may offer advantages over non-adaptive testing, including reduction of numbers of items required for precise measurement.

Objective

Our aim was to compare the efficiency of non-adaptive (NAT) and computer adaptive testing (CAT) facilitated by Partial Credit Model (PCM)-derived calibration to estimate skin cancer risk.

Methods

We used a random sample from a population-based Australian cohort study of skin cancer risk (N=43,794). All 30 items of the skin cancer risk scale were calibrated with the Rasch PCM. A total of 1000 cases generated following a normal distribution (mean [SD] 0 [1]) were simulated using three Rasch models with three fixed-item (dichotomous, rating scale, and partial credit) scenarios, respectively. We calculated the comparative efficiency and precision of CAT and NAT (shortening of questionnaire length and the count difference number ratio less than 5% using independent t tests).

Results

We found that use of CAT led to smaller person standard error of the estimated measure than NAT, with substantially higher efficiency but no loss of precision, reducing response burden by 48%, 66%, and 66% for dichotomous, Rating Scale Model, and PCM models, respectively.

Conclusions

CAT-based administrations of the skin cancer risk scale could substantially reduce participant burden without compromising measurement precision. A mobile computer adaptive test was developed to help people efficiently assess their skin cancer risk.