Background: The diagnostic framework and clinical reasoning process of Chinese medicine are central to the practice of acupuncture and other related disciplines. There is growing interest in integrating it into clinical trials of acupuncture and Chinese herbal medicine to guide individualized treatment protocols and evaluate outcomes. Strategies that enhance diagnostic reliability may contribute to this integration. Objectives: (1) To evaluate inter-rater reliability among practitioners of Traditional Chinese Medicine (TCM) when assessing women with dysmenorrhea using a structured assessment questionnaire (Traditional East Asian Medicine Structure Interview [TEAMSI]-TCM) compared to using a TCM questionnaire from routine clinical practice, not developed for research purposes (CONTROL); and (2) To evaluate the impact of training in the use of each approach on reliability. Design: Thirty-eight acupuncturists were asked to complete assessments of 10 subjects based on the viewing of a videotape of the initial assessment interview, a picture of the tongue, and a description of the pulse. Acupuncturists were randomized into one of four groups comparing the use of two questionnaires, TEAMSI-TCM versus CONTROL, and comparing training in the use of each versus no training. Analysis: The authors used Cohen's kappa to estimate agreement on TCM diagnostic categories relevant to dysmenorrhea between 2 practitioners with respect to questionnaires and training over all 10 patients and all 10 TCM diagnostic categories. For all analyses, the authors estimated kappa values for questionnaire, training, and experience level. Analysis of variance was used to test agreement among various groupings. Results: Regardless of the questionnaire used or training, analysis of inter-rater reliability indicated overall agreement to be low among practitioners (median 0.26). Kappa varied slightly by questionnaire and training, among 38 practitioners, but the difference was not statistically significant (p = 0.227 and p = 0.126, respectively). Conclusions: A structured assessment interview instrument designed for research purposes with or without training did not significantly improve reliability of TCM diagnosis of dysmenorrhea compared to a commonly used instrument. Challenges in assessing reliability in TCM remain.
Keywords: diagnostic methods; reliability; traditional Chinese medicine; validity.