Investigating latent constructs with item response models: a MATLAB IRTm toolbox

Development and Validation of a Polypharmacy Knowledge Assessment Instrument

Objective. To develop a brief instrument for academic pharmacists or physicians to use in assessing postgraduate residents' knowledge of polypharmacy. Methods. Five clinicians used a modified Delphi process to create a 26-item multiple-choice test to assess knowledge of polypharmacy in geriatric primary care. The test was distributed to 74 participants: 37 internal medicine (MD) residents, six nurse practitioner (NP) residents, nine primary care attendings, 12 pharmacists and pharmacy residents, and 10 geriatrics attendings and fellows. Construct validity was assessed using factor analysis and item response theory. Overall group differences were examined using a Kruskal-Wallis test, and between group differences were assessed using the Wilcoxon rank sum test. Results. The response rate for the survey was 89%. Factor analysis resulted in a one factor solution. Item response theory modeling yielded a 12-item and six-item test. For the 12-item test, the mean scores of geriatricians and pharmacists (88%) were higher than those of MD and NP residents (58%) and primary care attendings (61%). No differences were found between MD and NP residents and primary care attendings. Findings for the six-item test were similar. Conclusion. Both the 12-item and six-item versions of this polypharmacy test showed acceptable internal consistency and known groups validity and could be used in other academic settings. The similar scores between MD and NP residents and primary care attendings, which were significantly lower than scores for pharmacists and geriatricians, support the need for increased educational interventions.

Development of the Informed Choice in Mammography Screening Questionnaire (IMQ): factor structure, reliability, and validity

Background

Informed choice is of ethical and practical importance in mammography screening. To assess the level to which decisions regarding such screening are informed is thus imperative, but no specific instrument has been available to measure informed choice in the German mammography screening programme. The aims of this study were to develop the Informed Choice in Mammography Screening Questionnaire (IMQ) and to find first evidence for the factor structure, reliability and validity of its different components.

Methods

The IMQ was sent to 17.349 women aged 50 in Westphalia-Lippe, Germany. The instrument has been developed after consideration of (1) the results of qualitative interviews on decision making in the mammography screening programme, (2) relevant literature on other informed choice instruments and (3) a qualitative study on influencing factors. The IMQ comprises 3 scales (attitude, norms, and barriers), 1 index (knowledge) and singular items covering intention to participate and sociodemographic variables. To assess the psychometric properties of the components of the IMQ, confirmatory factor and item response theory analyses were conducted. Additionally, reliability, validity and item statistics were assessed.

Results

5.847 questionnaires were returned (response rate 33.7%). For attitude, the confirmatory factor analysis supported a one-factor structure. For norms, the model fit was not acceptable. Reliability levels were good with a Cronbach‘s α of.793 for attitude (4 items) and.795 for norms (5 items). For barriers, 9 items were deleted because of low discrimination indices; 6 items remained. The hypothesised assumption-subscale and the importance-subscale were confirmed, but these subscales showed poor reliabilities with Cronbach‘s α=.525 (4 items) and.583 (2 items). For the knowledge index, item response theory analysis showed that 6 out of 7 items were suitable. Hypotheses concerning the correlations between the different components were confirmed, which supported their convergent and divergent validity.

Conclusion

The results of this study demonstrated that the IMQ is a multidimensional instrument. Further development of the barriers and norms scales is necessary. The IMQ can be utilised to assess the level of informed choices as well as influencing factors.

Electronic supplementary material

The online version of this article (10.1186/s40359-019-0291-2) contains supplementary material, which is available to authorized users.

Treatment-seeking behaviour in low- and middle-income countries estimated using a Bayesian model

BACKGROUND

Seeking treatment in formal healthcare for uncomplicated infections is vital to combating disease in low- and middle-income countries (LMICs). Healthcare treatment-seeking behaviour varies within and between communities and is modified by socio-economic, demographic, and physical factors. As a result, it remains a challenge to quantify healthcare treatment-seeking behaviour using a metric that is comparable across communities. Here, we present an application for transforming individual categorical responses (actions related to fever) to a continuous probabilistic estimate of fever treatment for one country in Sub-Saharan Africa (SSA).

METHODS

Using nationally representative household survey data from the 2013 Demographic and Health Survey (DHS) in Namibia, individual-level responses (n = 1138) were linked to theoretical estimates of travel time to the nearest public or private health facility. Bayesian Item Response Theory (IRT) models were fitted via Markov Chain Monte Carlo (MCMC) simulation to estimate parameters related to fever treatment and estimate probability of treatment for children under five years. Different models were implemented to evaluate computational needs and the effect of including predictor variables such as rurality. The mean treatment rates were then estimated at regional level.

RESULTS

Modelling results suggested probability of fever treatment was highest in regions with relatively high incidence of malaria historically. The minimum predicted threshold probability of seeking treatment was 0.3 (model 1: 0.340; 95% CI 0.155-0.597), suggesting that even in populations at large distances from facilities, there was still a 30% chance of an individual seeking treatment for fever. The agreement between correctly predicted probability of treatment at individual level based on a subset of data (n = 247) was high (AUC = 0.978), with a sensitivity of 96.7% and a specificity of 75.3%.

CONCLUSION

We have shown how individual responses in national surveys can be transformed to probabilistic measures comparable at population level. Our analysis of household survey data on fever suggested a 30% baseline threshold for fever treatment in Namibia. However, this threshold level is likely to vary by country or endemicity. Although our focus was on fever treatment, the methodology outlined can be extended to multiple health seeking behaviours captured in routine national survey data and to other infectious diseases.

A surface EMG test tool to measure proportional prosthetic control

In upper limb amputees, prosthetic control training is recommended before and after fitting. During rehabilitation, the focus is on selective proportional control signals. For functional monitoring, many different tests are available. None can be used in the early phase of training. However, an early assessment is needed to judge if a patient has the potential to control a certain prosthetic set-up. This early analysis will determine if further training is needed or if other strategies would be more appropriate. Presented here is a tool that is capable of predicting achievable function in voluntary EMG control. This tool is applicable to individual muscle groups to support preparation of training and fitting. In four of five patients, the sEMG test tool accurately predicted the suitability for further myoelectric training based on SHAP outcome measures. (P1: "Poor" function in the sEMG test tool corresponded to 54/100 in the SHAP test; P2: Good: 85; P3: Good: 81; P4: Average: 78). One patient scored well during sEMG testing, but was unmotivated during SHAP testing. (Good: 50) Therefore, the surface EMG test tool may predict achievable control skills to a high extent, validated with the SHAP, but requires further clinical testing to validate this technique.

Analysis of Multiple Partially Ordered Responses to Belief Items with Don't Know Option

Understanding beliefs, values, and preferences of patients is a tenet of contemporary health sciences. This application was motivated by the analysis of multiple partially ordered set (poset) responses from an inventory on layman beliefs about diabetes. The partially ordered set arises because of two features in the data-first, the response options contain a Don't Know (DK) option, and second, there were two consecutive occasions of measurement. As predicted by the common sense model of illness, beliefs about diabetes were not necessarily stable across the two measurement occasions. Instead of analyzing the two occasions separately, we studied the joint responses across the occasions as a poset response. Few analytic methods exist for data structures other than ordered or nominal categories. Poset responses are routinely collapsed and then analyzed as either rank ordered or nominal data, leading to the loss of nuanced information that might be present within poset categories. In this paper we developed a general class of item response models for analyzing the poset data collected from the Common Sense Model of Diabetes Inventory. The inferential object of interest is the latent trait that indicates congruence of belief with the biomedical model. To apply an item response model to the poset diabetes inventory, we proved that a simple coding algorithm circumvents the requirement of writing new codes such that standard IRT software could be directly used for the purpose of item estimation and individual scoring. Simulation experiments were used to examine parameter recovery for the proposed poset model.

A probabilistic threshold model: analyzing semantic categorization data with the Rasch model

According to the Threshold Theory (Hampton, 1995, 2007) semantic categorization decisions come about through the placement of a threshold criterion along a dimension that represents items' similarity to the category representation. The adequacy of this theory is assessed by applying a formalization of the theory, known as the Rasch model (Rasch, 1960; Thissen & Steinberg, 1986), to categorization data for eight natural language categories and subjecting it to a formal test. In validating the model special care is given to its ability to account for inter- and intra-individual differences in categorization and their relationship with item typicality. Extensions of the Rasch model that can be used to uncover the nature of category representations and the sources of categorization differences are discussed.