Low inter-rater reliability in traditional Chinese medicine for female infertility

Experimental Studies of Inter-Rater Agreement in Traditional Chinese Medicine: A Systematic Review

Objectives: It has been recommended that clinical trials of Traditional Chinese Medicine (TCM) would be more ecologically valid if its characteristic mode of diagnostic reasoning were integrated into their design. In that context, however, it is also widely held that demonstrating a high level of agreement on initial TCM diagnoses is necessary for the replicability that the biomedical paradigm requires for the conclusions from such trials. Our aim was to review, summarize, and critique quantitative experimental studies of inter-rater agreement in TCM, and some of their underlying assumptions. Design: Systematic electronic searches were conducted for articles that reported a quantitative measure of inter-rater agreement across a number of rating choices based on examinations of human subjects in person by TCM practitioners, and published in English language peer-reviewed journals. Publications in languages other than English were not included, nor those appearing in other than peer-reviewed journals. Predefined categories of information were extracted from full texts by two investigators working independently. Each article was scored for methodological quality. Outcome measures: Design features across all studies and levels of inter-rater agreement across studies that reported the same type of outcome statistic were compared. Results: Twenty-one articles met inclusion criteria. Fourteen assessed inter-rater agreement on TCM diagnoses, two on diagnostic signs found upon traditional TCM examination, and five on novel rating schemes derived from TCM theory and practice. Raters were students of TCM colleges or graduates of TCM training programs with 3 or more years experience and licensure. Type of outcome statistic varied. Mean rates of pairwise agreement averaged 57% (median 65, range 19-96) across the 9 studies reporting them. Mean Cohen's kappa averaged 0.34 (median 0.34, range 0.07-0.59) across the seven studies reporting them. Meta-analysis was not possible due to variations in study design and outcome statistics. High risks of bias and confounding, and deficits in statistical reporting were common. Conclusions: With a few exceptions, the levels of agreement were low to moderate. Most studies had significant deficits of both methodology and reporting. Results overall suggest a few design features that might contribute to higher levels of agreement. These should be studied further with better experimental controls and more thorough reporting of outcomes. In addition, methods of complex systems analysis should be explored to more adequately model the relationship between clinical outcomes, and the series of diagnoses and treatments that are the norm in actual TCM practice.

Using a questionnaire among patient, resident doctor and senior supervisor: Are their answers the same?

OBJECTIVES

This study aims to contribute to the development of objective diagnostic standards in Traditional Chinese Medicine (TCM), in order to improve the training of physicians.

METHODS

We devised a questionnaire study to evaluate the accuracy of resident doctors' diagnostic skills by comparing their assessment of patients with those of their senior supervising physician and the patients themselves. We selected 39 patients with systemic lupus erythematosus (SLE) at Chang Gung Memorial Hospital, Taiwan, between November 1, 2008 and June 30, 2012, and had the resident doctors (R), their senior supervisor (S) and their patients (P) fill out questionnaires before treatment (V1), immediately after treatment (V5) and two months after treatment (V6), in order to record their assessments on the patients' condition. The R and S questionnaires covered subjective symptoms, tongue, and pulse, while the P questionnaires only included general symptoms. We then compared the assessment records to determine the level of agreement between them.

RESULTS

The agreements of inquiry during the study for P and S were 0.78 (V1) to 0.84 (V6) and 0.87 (V1) to 0.94 (V6) for R and S, respectively, the agreements between R and S for tongue diagnosis and pulse diagnosis were 0.87 (V1) to 0.90 (V6) and 0.91 (V1) to 0.95 (V6), respectively. All the above agreements improved with time from V1 to V6.

CONCLUSIONS

The results show that the patient input was feasible and effective and that the questionnaire method provided an objective assessment standard to determine how successfully the resident doctor was trained. Furthermore, it facilitated a training process that could help resident doctors improve their skills.

Acupuncture and women's health: an overview of the role of acupuncture and its clinical management in women's reproductive health

BACKGROUND

Acupuncture and other modalities of Chinese/East Asian medicine have been used to treat women's health for many centuries. Gynecology specialties focus particularly on menstrual and reproductive disorders. Both the adoption of the use of acupuncture outside Asia, and the incorporation of scientific analysis in Asia have challenged biomedical conceptions of what can be achieved with this treatment method. The scale of research activity in relation to acupuncture and women's health has increased over the last 20 years.

OBJECTIVE

This review aims to explore the research evidence in relation to acupuncture use for women's reproductive disorders, focusing on both clinical findings and experimental research on acupuncture's mechanisms of action in relation to women's health.

METHODS

A narrative literature search was undertaken using searches of electronic databases and manual searches of journals and textbooks. The search included all literature published prior to June 2013. The literature was assessed as to the nature of the study it was reporting and findings synthesized into a commentary.

RESULTS

For acupuncture's mechanism of action the search resulted in 114 relevant documents; in relation to clinical reports on the use of acupuncture for women's health 204 documents were found and assessed.

CONCLUSION

There is preliminary data indicating acupuncture may improve menstrual health and coping for women experiencing delays falling pregnant. There is experimental data showing that acupuncture can influence female reproductive functioning, although the actual mechanisms involved are not yet clarified. Further well-conducted clinical research would benefit our understanding of the usefulness of acupuncture to women's health.

Acupuncture in practice: investigating acupuncturists' approach to treating infantile colic

Infantile colic is common, but no safe and effective conventional treatment exists. The use of acupuncture has increased despite weak evidence. This practitioner survey explores and discusses how infantile colic is regarded and treated in Traditional Chinese Medicine (TCM). The study is based on personal communication with 24 acupuncturists from nine countries. These acupuncturists specialize in pediatric acupuncture and represent different styles of acupuncture. Their experiences are discussed and related to relevant books and articles. Informants claimed good results when treating infants with colic. The TCM patterns commonly described by informants matched the textbooks to a great extent. The most common syndromes were "stagnation of food" and "Spleen Qi Xu." Regarding treatment, some informants followed the teachers' and the textbook authors' advice on differentiated treatment according to syndrome. The points used most often were LI4, ST36, and Sifeng. Other informants treated all infants alike in one single point, LI4. The results demonstrate the diversity of TCM. The use of acupuncture for infantile colic presents an interesting option, but further research is needed in order to optimize the effects and protect infants from unnecessary or less effective treatment.

Traditional Chinese medicine diagnoses in persons with ketamine abuse

OBJECTIVES

To explore the distribution of TCM patterns in ketamine users and the inter-rater reliability of TCM diagnosis among TCM practitioners.

METHODS

Eighty-four subjects recruited from substance abuse clinics and non-governmental counseling services catering for ketamine abusers were examined by one or two TCM practitioners. The distribution and inter-rater reliability of the TCM diagnoses between the two TCM practitioners were described and measured.

RESULTS

Seven TCM patterns were differentiated. The four most common patterns were the Heart Yin deficiency (29.8%), Kidney Yang deficiency (26.2% Kidney Yin deficiency (19.0%), and Spleen Yang deficiency (14.3%). The agreement between the two TCM practitioners was 59%; the kappa for the three most common patterns was 0.472 (P = 0.003). The proportion of men was lower in the Kidney Yin deficiency group (18.8%), in comparison to the Heart Yin deficiency (60.0%, P = 0.009) and Kidney Yang deficiency group (59.1%, P = 0.013).

CONCLUSION

The differences between the TCM groups in terms of the source of referral and number of drug uses other than ketamine were of borderline significance. Further research should focus on validating diagnostic tools and improving inter-rater reliability in TCM to enhance the quality of clinical trials of TCM treatments for ketamine abuse.

Interrater reliability of chinese medicine diagnosis in people with prediabetes

Background. Achieving reproducibility in research design is challenging when patient cohorts under study are inconsistently defined. Traditional Chinese medicine (TCM) diagnosis is one example where inconsistency between practitioners has been found. We hypothesise that the use of a validated instrument may improve consistency. Biochemical biomarkers may also be used enhance reliability. Methods. Twenty-seven participants with prediabetes were assessed by two TCM practitioners using a validated instrument (TEAMSI-TCM). Inter-rater reliability was summarised using percentage agreement and the kappa coefficient. One-way ANOVA and Tukey's post hoc test were used to test links between TCM diagnosis and biomarkers. Results. The two practitioners agreed on primary diagnosis of 70% of participants. kappa = 0.56 (P < 0.001). The three predominant TCM diagnostic patterns for people with prediabetes were Yin deficiency, Qi and Yin deficiency and Spleen qi deficiency. The Spleen Qi deficiency with Damp cohort had statistically significant higher fasting glucose, higher insulin, higher insulin resistance, higher HbA1c and lower HDL than those with Qi and Yin deficiency. Conclusions. Using the TEAMSI-TCM resulted in moderate interrater reliability between TCM practitioners. This study provides initial evidence of variation in the biomarkers of people with prediabetes according to the different TCM patterns which may suggest a route to further improving interrater reliability.

Validation of the constitution in chinese medicine questionnaire: does the traditional chinese medicine concept of body constitution exist?

The study aims to adapt and validate the Constitution in Chinese Medicine Questionnaire (CCMQ) in Hong Kong Chinese people. 10 patients and 10 Chinese medicine practitioners (CMP) confirmed the content validity (CVI: 50%–100%) of CCMQ. 1084 HK subjects completed a cross-sectional study with 98.6% who could be classified into one or more BC types. Scaling success rates were 85.7%–100% for the 9 BC scales. Construct validity was supported by moderate correlations between CCMQ and SF-12v2 scores. The confirmatory factor analysis showed a reproducible structure as hypothesized. People with gentleness BC type had better health-related quality of life, HRQOL, than those with other (imbalanced) BC types. Internal consistency (reliability) (Cronbach's alpha  >  0.6) and test-retest reliability were also satisfactory (ICC > 0.6) for all scales. However, the sensitivity and specificity in predicting the BC types diagnosed by CMP were only fair, ranging from 42.7% to 82.7%. 27.6% of subjects had a change from the imbalanced BC types to gentleness BC type after 6 months. The CCMQ was adapted for HK Chinese people and proved to be valid, reliable, and responsive. People classified to have imbalanced BC types had significantly lower HRQOL than gentleness BC type, which supported the validity and importance of the TCM concept of the physiological BC type.

Traditional Chinese medicine patterns and recommended acupuncture points in infertile and fertile women

BACKGROUND

Acupuncture is rooted in traditional Chinese medicine (TCM) and emphasises individualised treatment according to TCM patterns. Eight TCM patterns are regarded as typical for female infertility. There are no empirical data comparing these patterns in fertile and infertile women.

OBJECTIVE

To identify and compare the TCM patterns and recommended acupuncture points in infertile and fertile women. The hypothesis that infertile and fertile women should differ in occurrence of TCM patterns described as typical for infertility was examined.

METHODS

A cross-sectional study examined the distribution of TCM patterns and acupuncture points among 24 infertile and 24 fertile women. TCM patterns and points proposed by two acupuncturists were analysed. The odds ratio was used as the effect measure.

RESULTS

Three patterns were found in more than 92% of the women. Five patterns occurred more frequently among infertile (p≤0.03), and four among fertile (p≤0.02) women. Only three of the eight patterns assumed typical for infertile women were more common among infertile women. Two of the eight patterns were more common among fertile women. Acupuncture points KI3, LI3 and SP6 were chosen for almost all women, whereas CV4, CV3 and ST29 were chosen more often for infertile women (p<0.03), and SP3 more often for fertile women (p<0.001).

CONCLUSION

The data indicate that the presence of some, and the absence of other, TCM patterns are associated with infertility. These results are not consistent with the assumed associations between eight specific patterns and infertility, as maintained by TCM textbooks.