Further development and validation of the Multimorbidity Treatment Burden Questionnaire (MTBQ)

OBJECTIVES

To undertake further psychometric testing of the Multimorbidity Treatment Burden Questionnaire (MTBQ) and examine whether reversing the scale reduced floor effects.

DESIGN

Survey.

SETTING

UK primary care.

PARTICIPANTS

Adults (≥18 years) with three or more long-term conditions randomly selected from four general practices and invited by post.

MEASURES

Baseline survey: sociodemographics, MTBQ (original or version with scale reversed), Treatment Burden Questionnaire (TBQ), four questions (from QQ-10) on ease of completing the questionnaires. Follow-up survey (1-4 weeks after baseline): MTBQ, TBQ and QQ-10. Anonymous data collected from electronic GP records: consultations (preceding 12 months) and long-term conditions. The proportion of missing data and distribution of responses were examined for the original and reversed versions of the MTBQ and the TBQ. Intraclass correlation coefficient (ICC) and Spearman's rank correlation (Rs) assessed test-retest reliability and construct validity, respectively. Ease of completing the MTBQ and TBQ was compared. Interpretability was assessed by grouping global MTBQ scores into 0 and tertiles (>0).

RESULTS

244 adults completed the baseline survey (consent rate 31%, mean age 70 years) and 225 completed the follow-up survey. Reversing the scale did not reduce floor effects or data skewness. The global MTBQ scores had good test-retest reliability (ICC for agreement at baseline and follow-up 0.765, 95% CI 0.702 to 0.816). Global MTBQ score was correlated with global TBQ score (Rs 0.77, p<0.001), weakly correlated with number of consultations (Rs 0.17, p=0.010), and number of different general practitioners consulted (Rs 0.23, p<0.001), but not correlated with number of long-term conditions (Rs -0.063, p=0.330). Most participants agreed that both the MTBQ and TBQ were easy to complete and included aspects they were concerned about.

CONCLUSION

This study demonstrates test-retest reliability and ease of completion of the MTBQ and builds on a previous study demonstrating good content validity, construct validity and internal consistency reliability of the questionnaire.

Analysis of 'One in a Million' primary care consultation conversations using natural language processing

BACKGROUND

Modern patient electronic health records form a core part of primary care; they contain both clinical codes and free text entered by the clinician. Natural language processing (NLP) could be employed to generate these records through 'listening' to a consultation conversation.

OBJECTIVES

This study develops and assesses several text classifiers for identifying clinical codes for primary care consultations based on the doctor-patient conversation. We evaluate the possibility of training classifiers using medical code descriptions, and the benefits of processing transcribed speech from patients as well as doctors. The study also highlights steps for improving future classifiers.

METHODS

Using verbatim transcripts of 239 primary care consultation conversations (the 'One in a Million' dataset) and novel additional datasets for distant supervision, we trained NLP classifiers (naïve Bayes, support vector machine, nearest centroid, a conventional BERT classifier and few-shot BERT approaches) to identify the International Classification of Primary Care-2 clinical codes associated with each consultation.

RESULTS

Of all models tested, a fine-tuned BERT classifier was the best performer. Distant supervision improved the model's performance (F1 score over 16 classes) from 0.45 with conventional supervision with 191 labelled transcripts to 0.51. Incorporating patients' speech in addition to clinician's speech increased the BERT classifier's performance from 0.45 to 0.55 F1 (p=0.01, paired bootstrap test).

CONCLUSIONS

Our findings demonstrate that NLP classifiers can be trained to identify clinical area(s) being discussed in a primary care consultation from audio transcriptions; this could represent an important step towards a smart digital assistant in the consultation room.

Older doctors and progression through specialty training in the UK: a cohort analysis of General Medical Council data

OBJECTIVE

To determine whether older age at graduation is associated with any difference in outcomes from the annual specialty training progression assessment.

DESIGN

An open cohort of 38 308 doctors who graduated from a UK medical school with annual assessments of progression in their specialty training programme with data centrally collected by the General Medical Council between 5 August 2009 to 31 July 2012.

RESULTS

Mature junior doctors (≥29 years at graduation) were more likely to have problems with progression on their annual review of competence progression record of in training assessment (ARCP/RITA) than their younger colleagues (OR 1.34, 95% CI 1.22 to 1.49, p<0.001). This association was, if anything, even stronger (OR 1.57, 95% CI 1.41 to 1.74, p<0.001) after adjustment for gender, ethnicity, type of University and specialty. The same was true when only looking at the most extreme ARCP outcome (4) which is being asked to leave their specialist programme (OR 1.81, 95% CI 1.34 to 2.44, p<0.001).

CONCLUSIONS

Mature doctors are a growing part of the medical workforce and they are likely to broaden the spectrum of doctors by bring different life experience to the profession. These results suggest that they are more likely to have problems with progressing through their specialist training programme. More research is required to determine the reasons behind these associations and how mature doctors can be supported both in choosing the best training programme and in coping with the complex demands of higher training at a later stage in their lives.