Estimating Skin Cancer Risk: Evaluating Mobile Computer-Adaptive Testing

Web-Based Skin Cancer Assessment and Classification Using Machine Learning and Mobile Computerized Adaptive Testing in a Rasch Model: Development Study

Background

Web-based computerized adaptive testing (CAT) implementation of the skin cancer (SC) risk scale could substantially reduce participant burden without compromising measurement precision. However, the CAT of SC classification has not been reported in academics thus far.

Objective

We aim to build a CAT-based model using machine learning to develop an app for automatic classification of SC to help patients assess the risk at an early stage.

Methods

We extracted data from a population-based Australian cohort study of SC risk (N=43,794) using the Rasch simulation scheme. All 30 feature items were calibrated using the Rasch partial credit model. A total of 1000 cases following a normal distribution (mean 0, SD 1) based on the item and threshold difficulties were simulated using three techniques of machine learning—naïve Bayes, k-nearest neighbors, and logistic regression—to compare the model accuracy in training and testing data sets with a proportion of 70:30, where the former was used to predict the latter. We calculated the sensitivity, specificity, receiver operating characteristic curve (area under the curve [AUC]), and CIs along with the accuracy and precision across the proposed models for comparison. An app that classifies the SC risk of the respondent was developed.

Results

We observed that the 30-item k-nearest neighbors model yielded higher AUC values of 99% and 91% for the 700 training and 300 testing cases, respectively, than its 2 counterparts using the hold-out validation but had lower AUC values of 85% (95% CI 83%-87%) in the k-fold cross-validation and that an app that predicts SC classification for patients was successfully developed and demonstrated in this study.

Conclusions

The 30-item SC prediction model, combined with the Rasch web-based CAT, is recommended for classifying SC in patients. An app we developed to help patients self-assess SC risk at an early stage is required for application in the future.

An application for classifying perceptions on my health bank in Taiwan using convolutional neural networks and web-based computerized adaptive testing: A development and usability study

BACKGROUND

The classification of a respondent's opinions online into positive and negative classes using a minimal number of questions is gradually changing and helps turn techniques into practices. A survey incorporating convolutional neural networks (CNNs) into web-based computerized adaptive testing (CAT) was used to collect perceptions on My Health Bank (MHB) from users in Taiwan. This study designed an online module to accurately and efficiently turn a respondent's perceptions into positive and negative classes using CNNs and web-based CAT.

METHODS

In all, 640 patients, family members, and caregivers with ages ranging from 20 to 70 years who were registered MHB users were invited to complete a 3-domain, 26-item, 5-category questionnaire asking about their perceptions on MHB (PMHB26) in 2019. The CNN algorithm and k-means clustering were used for dividing respondents into 2 classes of unsatisfied and satisfied classes and building a PMHB26 predictive model to estimate parameters. Exploratory factor analysis, the Rasch model, and descriptive statistics were used to examine the demographic characteristics and PMHB26 factors that were suitable for use in CNNs and Rasch multidimensional CAT (MCAT). An application was then designed to classify MHB perceptions.

RESULTS

We found that 3 construct factors were extracted from PMHB26. The reliability of PMHB26 for each subscale beyond 0.94 was evident based on internal consistency and stability in the data. We further found the following: the accuracy of PMHB26 with CNN yields a higher accuracy rate (0.98) with an area under the curve of 0.98 (95% confidence interval, 0.97-0.99) based on the 391 returned questionnaires; and for the efficiency, approximately one-third of the items were not necessary to answer in reducing the respondents' burdens using Rasch MCAT.

CONCLUSIONS

The PMHB26 CNN model, combined with the Rasch online MCAT, is recommended for improving the accuracy and efficiency of classifying patients' perceptions of MHB utility. An application developed for helping respondents self-assess the MHB cocreation of value can be applied to other surveys in the future.

An App for Identifying Children at Risk for Developmental Problems Using Multidimensional Computerized Adaptive Testing: Development and Usability Study

BACKGROUND

The use of multidomain developmental screening tools is a viable strategy for pediatric professionals to identify children at risk for developmental problems. However, a specialized multidimensional computer adaptive testing (MCAT) tool has not been developed to date.

OBJECTIVE

We developed an app using MCAT, combined with Multidimensional Screening in Child Development (MuSiC) for toddlers, to help patients and their family members or clinicians identify developmental problems at an earlier stage.

METHODS

We retrieved 75 item parameters from the MuSiC literature item bank for 1- to 3-year-old children, and simulated 1000 person measures from a normal standard distribution to compare the efficiency and precision of MCAT and nonadaptive testing (NAT) in five domains (ie, cognitive skills, language skills, gross motor skills, fine motor skills, and socioadaptive skills). The number of items saved and the cutoff points for the tool were determined and compared. We then developed an app for a Web-based assessment.

RESULTS

MCAT yielded significantly more precise measurements and was significantly more efficient than NAT, with 46.67% (=(75-40)/75) saving in item length when measurement differences less than 5% were allowed. Person-measure correlation coefficients were highly consistent among the five domains. Significantly fewer items were answered on MCAT than on NAT without compromising the precision of MCAT.

CONCLUSIONS

Developing an app as a tool for parents that can be implemented with their own computers, tablets, or mobile phones for the online screening and prediction of developmental delays in toddlers is useful and not difficult.

Mobile applications in oncology: A systematic review of health science databases

INTRODUCTION

In recent years there has been an exponential growth in the number of mobile applications (apps) relating to the early diagnosis of cancer and prevention of side effects during cancer treatment. For health care professionals and users, it can thus be difficult to determine the most appropriate app for given needs and assess the level of scientific evidence supporting their use. Therefore, this review aims to examine the research studies that deal with this issue and determine the characteristics of the apps involved.

METHODOLOGY

This study involved a systematic review of the scientific literature on randomized clinical trials that use apps to improve cancer management among patients, using the Pubmed (Medline), Latin America and the Caribbean in Health Sciences (LILACS), and Cochrane databases. The search was limited to articles written in English and Spanish published in the last 10 years. A search of the App Store for iOS devices and Google Play for Android devices was performed to find the apps identified in the included research articles.

RESULTS

In total, 54 articles were found to analyze the development of an application in the field of oncology. These articles were most frequently related to the use of apps for the early detection of cancer (n = 28), particularly melanoma (n = 9). In total, 21 studies reflected the application used. The apps featured in nine articles were located using the App Store and Google Play (n = 9), of which five were created to manage cancer-related issues. The rest of the apps were designed for use in the general population (n = 4).

CONCLUSIONS

There is an increasing number of research articles that study the use of apps in the field of oncology; however, these mobile applications tend to disappear from app stores after the studies are completed.

Assessing disease experience across the life span for individuals with osteogenesis imperfecta: challenges and opportunities for patient-reported outcomes (PROs) measurement: a pilot study

Background

Patient reported outcome (PRO) information is crucial for establishing better patient-provider communication, improving shared decision-making between clinicians and patients, assessing patient responses to therapeutic interventions, and increasing satisfaction with care. We used the Brittle Bones Disease Consortium (BBDC) Contact Registry for People with OI, managed by the Rare Disease Clinical Research Network (RDCRN) to (1) to evaluate the construct validity of the Patient-Reported Outcome Measurement Information System® (PROMIS®) to record important components of the disease experience among individuals with OI; and (2) explore the feasibility of using a registry to recruit individuals with OI to report on health status. Our long-term goal is to enhance communication of health and disease management findings back to the OI community, especially those who do not have access to major OI clinical centers.

Results

We demonstrated the construct validity of PROMIS instruments in OI. Our results confirm that the scores from most domains differ significantly from the general US population: individuals with OI have worse symptom burden and functioning. We found no excessive floor or ceiling effects. Our study demonstrates that the BBDC Contact Registry can be used to recruit participants for online health status surveys. However, there are numerous challenges that must be addressed: lack of self-knowledge of OI type, under-representation of men, limited ethnic diversity, and imperfect questionnaire completion rates.

Conclusion

Our pilot study demonstrated the feasibility of using a contact registry to recruit respondents from the OI community and to obtain analyzable PROMIS data regarding disease experience. Because the results differ from the general population and avoid excessive floor and ceiling effects, PROMIS instruments can be used to assess response to therapeutic interventions in individuals with OI. Future directions will include (1) development and validation of an OI-specific patient-based classification system that aggregates persons with similar clinical characteristics and risks for complications to identify treatment needs; and (2) integrating these PRO tools into routine patient care and research studies.

Electronic supplementary material

The online version of this article (10.1186/s13023-019-1004-x) contains supplementary material, which is available to authorized users.

Application of a multidimensional computerized adaptive test for a Clinical Dementia Rating Scale through computer-aided techniques

BACKGROUND

With the increasingly rapid growth of the elderly population, individuals aged 65 years and above now compose 14% of Taiwanese citizens, thereby making Taiwanese society an aged society. A leading factor that affects the elderly population is dementia. A method of precisely and efficiently examining patients with dementia through multidimensional computer adaptive testing (MCAT) to accurately determine the patients' stage of dementia needs to be developed. This study aimed to develop online MCAT that family members can use on their own computers, tablets, or smartphones to predict the extent of dementia for patients responding to the Clinical Dementia Rating (CDR) instrument.

METHODS

The CDR was applied to 366 outpatients in a hospital in Taiwan. MCAT was employed with parameters for items across eight dimensions, and responses were simulated to compare the efficiency and precision between MCAT and non-adaptive testing (NAT). The number of items saved and the estimated person measures was compared between the results of MCAT and NAT, respectively.

RESULTS

MCAT yielded substantially more precise measurements and was considerably more efficient than NAT. MCAT achieved 20.19% (= [53 - 42.3]/53) saving in item length when the measurement differences were less than 5%. Pearson correlation coefficients were highly consistent among the eight domains. The cut-off points for the overall measures were - 1.4, - 0.4, 0.4, and 1.4 logits, which was equivalent to 20% for each portion in percentile scores. Substantially fewer items were answered through MCAT than through NAT without compromising the precision of MCAT.

CONCLUSIONS

Developing a website that family members can use on their own computers, tablets, and smartphones to help them perform online screening and prediction of dementia in older adults is useful and manageable.

Development of a Microsoft Excel tool for one-parameter Rasch model of continuous items: an application to a safety attitude survey

Background

Many continuous item responses (CIRs) are encountered in healthcare settings, but no one uses item response theory’s (IRT) probabilistic modeling to present graphical presentations for interpreting CIR results. A computer module that is programmed to deal with CIRs is required.

To present a computer module, validate it, and verify its usefulness in dealing with CIR data, and then to apply the model to real healthcare data in order to show how the CIR that can be applied to healthcare settings with an example regarding a safety attitude survey.

Methods

Using Microsoft Excel VBA (Visual Basic for Applications), we designed a computer module that minimizes the residuals and calculates model’s expected scores according to person responses across items. Rasch models based on a Wright map and on KIDMAP were demonstrated to interpret results of the safety attitude survey.

Results

The author-made CIR module yielded OUTFIT mean square (MNSQ) and person measures equivalent to those yielded by professional Rasch Winsteps software. The probabilistic modeling of the CIR module provides messages that are much more valuable to users and show the CIR advantage over classic test theory.

Conclusions

Because of advances in computer technology, healthcare users who are familiar to MS Excel can easily apply the study CIR module to deal with continuous variables to benefit comparisons of data with a logistic distribution and model fit statistics.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0276-2) contains supplementary material, which is available to authorized users.

A new technique to measure online bullying: online computerized adaptive testing

BACKGROUND

Workplace bullying has been measured in many studies to investigate mental health issues. None uses online computerized adaptive testing (CAT) with cutting points to report bully prevalence at workplace.

OBJECTIVE

To develop an online CAT to examine person being bullied and verify whether item response theory-based CAT can be applied online for nurses to measure exposure to workplace bullying.

METHODS

A total of 963 nurses were recruited and responded to the 22-item Negative Acts Questionnaire-Revised (NAQ-R). All non-adaptive testing (NAT) items were calibrated with the Rasch rating scale model. Three scenarios (i.e., NAT, CAT, and the randomly selected method to NAT) were manipulated to compare their response efficiency and precision by comparing (i) item length for answering questions, person measure, (ii) correlation coefficients, (iii) paired t tests, and (iv) estimated standard errors (SE) between CAT and the random to its counterpart of NAT.

RESULTS

The NAQ-R is a unidimensional construct that can be applied for nurses to measure exposure to workplace bullying on CAT. CAT required fewer items (=8.9) than NAT (=22, an efficient gain of 60% =1-8.9/22). Nursing measures derived from both tests (CAT and the random to NAT) were highly correlated (r = 0.93 and 0.96) and their measurement precisions were not statistically different (the percentage of significant count number less than 5%) as expected, but CAT earns smaller person measure SE than the random scenario. The prevalence rate for nurses was 1.5% (=15/963) when cutting points set at -0.7 and 0.7 logits.

CONCLUSION

The CAT-based NAQ-R reduces respondents' burden without compromising measurement precision and increases endorsement efficiency. The online CAT is recommended for assessing nurses using the criteria at -0.7 and 0.7 (or <30 and <60 in summed score) to identify bully grade as one of the three levels (high, moderate, and low). The bullied nurse can get help from a psychiatrist or a mental health expert at an earlier stage.