Interexaminer reliability of thoracic motion palpation using confidence ratings and continuous analysis

Effect of Chiropractic Intervention on Oculomotor and Attentional Visual Outcomes in Young Adults With Long-Term Mild Traumatic Brain Injury: A Randomized Controlled Trial

OBJECTIVE

This study aimed to establish if chiropractic care can improve oculomotor and cognitive symptoms in individuals with persistent postconcussion syndrome (PPCS).

METHODS

A single-blind, randomized controlled intervention study recorded baseline computerized eye-tracker assessment (CEA) outcomes in 40 young adults with PPCS following mild traumatic brain injury. Participants were randomly allocated to either a chiropractic or age-matched active control intervention, and the change in CEA outcomes following intervention was compared between the chiropractic and control groups. A battery of CEAs including egocentric localization, fixation stability, pursuit, saccades, Stroop, and the vestibulo-ocular reflex, were used to assess oculomotor function, visual attention/processing, and selective attention.

RESULTS

Relative to the control group, participants receiving the chiropractic intervention scored better in the Stroop test (P < .001), had improved gaze stability during both vestibulo-ocular reflex (P < .001) and fixation stability (P = .009), and a lower vertical error in egocentric localization (P < .001). However, performance was poorer in pursuits, where they had an increased tracking error (P < .001).

CONCLUSION

Chiropractic care in participants with PPCS significantly improved static and dynamic gaze stability, and performance in the Stroop test, compared with a control intervention. These results suggest that chiropractic care can offer a novel avenue for alleviating certain visual and cognitive symptoms in patients with PPCS. It also adds to the growing evidence that suggests that some longstanding PPCS visual symptoms may have a spinal or proprioceptive basis.

The interrater reliability of static palpation of the thoracic spine for eliciting tenderness and stiffness to test for a manipulable lesion

Background

Despite widespread use by manual therapists, there is little evidence regarding the reliability of thoracic spine static palpation to test for a manipulable lesion using stiffness or tenderness as diagnostic markers. We aimed to determine the interrater agreement of thoracic spine static palpation for segmental tenderness and stiffness and determine the effect of standardised training for examiners. The secondary aim was to explore expert consensus on the level of segmental tenderness required to locate a “manipulable lesion”.

Methods

Two experienced chiropractors used static palpation of thoracic vertebrae on two occasions (pragmatic and standardised approaches). Participants rated tenderness on an 11-point numerical pain rating scale (NPRS) and raters judged segmental stiffness based on their experience and perception of normal mobility with the requested outcomes of hypomobile or normal mobility. We calculated interrater agreement using percent agreement, Cohen’s Kappa coefficients (κ) and prevalence-adjusted bias-adjusted Kappa coefficients (PABAK). In a preliminary study, an expert panel of 10 chiropractors took part in a Delphi process to identify the level of meaningful segmental tenderness required to locate a “manipulable lesion”.

Results

Thirty-six participants (20 female) were enrolled for the reliability study on the 13th March 2017. Mean (SD) age was 22.4 (3.4) years with an equal distribution of asymptomatic (n = 17) and symptomatic (n = 17) participants. Overall, the interrater agreement for spinal segmental stiffness had Kappa values indicating less than chance agreement [κ range − 0.11, 0.53]. When adjusted for prevalence and bias, the PABAK ranged from slight to substantial agreement [0.12–0.76] with moderate or substantial agreement demonstrated at the majority of spinal levels (T1, T2 and T6 to T12). Generally, there was fair to substantial agreement for segmental tenderness [Kappa range 0.22–0.77]. Training did not significantly improve interrater agreement for stiffness or tenderness. The Delphi process indicated that an NPRS score of 2 out of 10 identified a potential “manipulable lesion”.

Conclusion

Static palpation was overall moderately reliable for the identification of segmental thoracic spine stiffness and tenderness, with tenderness demonstrating a higher reliability. Also, an increased agreement was found within the mid-thoracic spine. A brief training intervention failed to improve reliability.

Electronic supplementary material

The online version of this article (10.1186/s12998-018-0218-7) contains supplementary material, which is available to authorized users.

The Chiropractic Vertebral Subluxation Part 9: Complexes, Models, and Consensus From 1979 to 1995

OBJECTIVE

The purpose of this paper is to review and discuss the history of chiropractic vertebral subluxation (CVS) between 1979 and 1995, with an emphasis on complexes, models, and consensus.

DISCUSSION

New models were developed and critiqued during this period. The first subluxation complex model was proposed by Faye. Other theorists such as Lantz and Dishman built upon his model. The complex models were integrated into consensus statements by the International Chiropractic Association and the American Chiropractic Association, and later by profession-wide processes. The plurality of the CVS encompassed known theories and included integration of stress models and biochemistry. The professional consensus around CVS was comprehensive as shown by the process models that included stakeholders in the profession. Calls for more empirical research and changes to terminology increased during this era. Influential articles from this period potentially contained errors and relied on older studies that may no longer be valid.

CONCLUSION

The challenges to researching this complex entity during that time were daunting for many and pointed to challenges of using the word "subluxation" to encompass the multiplicity that defined CVS.

The Chiropractic Vertebral Subluxation Part 10: Integrative and Critical Literature From 1996 and 1997

OBJECTIVE

The purpose of this paper is to review and discuss the history of chiropractic vertebral subluxation (CVS) during 1996 and 1997. The literature during this period offered critical and integrative models emphasized by a need for research into operational and functional definitions.

DISCUSSION

Several integrative approaches emerged, from Rome's 296 synonyms to Bergman's Pain/Tenderness, Asymmetry/Alignment, Range of Motion Abnormality, Tissue Tone, Texture, Temperature Abnormality, and Special Tests (PARTS) analysis adopted by the profession in the United States. Other noteworthy contributions included Ruch's Atlas of Common Subluxations, Epstein's introduction of network spinal analysis, and Kent's review of CVS models. Boone's introduction of the Journal of Vertebral Subluxation Research was accompanied by his 3-part model with Dobson. These years also included the paradigm statement of the Association of Chiropractic Colleges, which was adopted by the American Chiropractic Association, International Chiropractors Association, and World Federation of Chiropractic. Two other papers included Nelson's critique of the CVS paradigm and Keating's 1996 "Hunt for the Subluxation."

CONCLUSION

The CVS reached a new stage of complexity and critique and offered new directions for research, integration, and development.

Chiropractic spinal manipulation alters TMS induced I-wave excitability and shortens the cortical silent period

The objective of this study was to construct peristimulus time histogram (PSTH) and peristimulus frequencygram (PSF) using single motor unit recordings to further characterize the previously documented immediate sensorimotor effects of spinal manipulation. Single pulse transcranial magnetic stimulation (TMS) via a double cone coil over the tibialis anterior (TA) motor area during weak isometric dorsiflexion of the foot was used on two different days in random order; pre/post spinal manipulation (in eighteen subjects) and pre/post a control (in twelve subjects) condition. TA electromyography (EMG) was recorded with surface and intramuscular fine wire electrodes. Three subjects also received sham double cone coil TMS pre and post a spinal manipulation intervention. From the averaged surface EMG data cortical silent periods (CSP) were constructed and analysed. Twenty-one single motor units were identified for the spinal manipulation intervention and twelve single motor units were identified for the control intervention. Following spinal manipulations there was a shortening of the silent period and an increase in the single unit I-wave amplitude. No changes were observed following the control condition. The results provide evidence that spinal manipulation reduces the TMS-induced cortical silent period and increases low threshold motoneurone excitability in the lower limb muscle. These finding may have important clinical implications as they provide support that spinal manipulation can be used to strengthen muscles. This could be followed up on populations that have reduced muscle strength, such as stroke victims.

Comparison of Supine and Prone Methods of Leg Length Inequality Assessment

OBJECTIVE

The primary objective of the current study was to determine the reliability between methods of supine and prone leg length inequality (LLI) assessment. The secondary objective was to determine if the degree of examiner confidence affected the degree of intermethod agreement.

METHODS

Two experienced doctors of chiropractic assessed 43 participants for LLI, one using a prone and the other a supine method. They stated whether they were confident or not confident in their findings.

RESULTS

Kappa values for intermethod agreement were 0.16 for the full data set; 0.00 for the n = 20 subgroup with both examiners confident; 0.24 for the n = 18 subgroup with 1 examiner confident; and 0.55 for the n = 5 subgroup with neither examiner confident. Supine and prone measures exhibited slight agreement for the full data set, but no agreement when both examiners were confident. The moderate agreement with both examiners not confident may be an artifact of small sample size.

CONCLUSIONS

This study found that supine and prone assessments for leg length inequality were not in agreement. Positioning the patient in the prone position may increase, decrease, reverse, or offset the observed LLI that is seen in the supine position.

Effects of 12 Weeks of Chiropractic Care on Central Integration of Dual Somatosensory Input in Chronic Pain Patients: A Preliminary Study

OBJECTIVE

The purpose of this preliminary study was to assess whether the dual somatosensory evoked potential (SEP) technique is sensitive enough to measure changes in cortical intrinsic inhibitory interactions in patients with chronic neck or upper extremity pain and, if so, whether changes are associated with changes in pain scores.

METHODS

The dual peripheral nerve stimulation SEP ratio technique was used for 6 subjects with a history of chronic neck or upper limb pain. SEPs were recorded after left or right median and ulnar nerve stimulation at the wrist. SEP ratios were calculated for the N9, N13, P14-18, N20-P25, and P22-N30 peak complexes from SEP amplitudes obtained from simultaneous median and ulnar stimulation divided by the arithmetic sum of SEPs obtained from individual stimulation of the median and ulnar nerves. Outcome measures of SEP ratios and subjects' visual analog scale rating of pains were recorded at baseline, after a 2-week usual care control period, and after 12 weeks of multimodal chiropractic care (chiropractic spinal manipulation and 1 or more of the following: exercises, peripheral joint adjustments/manipulation, soft tissue therapy, and pain education).

RESULTS

A significant decrease in the median and ulnar to median plus ulnar ratio and the median and ulnar amplitude for the cortical P22-N30 SEP component was observed after 12 weeks of chiropractic care, with no changes after the control period. There was a significant decrease in visual analog scale scores (both for current pain and for pain last week).

CONCLUSION

The dual SEP ratio technique appears to be sensitive enough to measure changes in cortical intrinsic inhibitory interactions in patients with chronic neck pain. The observations in 6 subjects revealed that 12 weeks of chiropractic care improved suppression of SEPs evoked by dual upper limb nerve stimulation at the level of the motor cortex, premotor areas, and/or subcortical areas such as basal ganglia and/or thalamus. It is possible that these findings explain one of the mechanisms by which chiropractic care improves function and reduces pain for chronic pain patients.

The reliability of spinal motion palpation determination of the location of the stiffest spinal site is influenced by confidence ratings: a secondary analysis of three studies

Background

This is a secondary analysis of three previous studies on the cervical, thoracic, and lumbar spines. It uses continuous analysis of the stiffest spinal site rather than more typical level-by-level analysis to assess interexaminer reliability, and the impacts of examiner confidence and spinal region. The primary goal was secondary analysis of the combined data; secondary goal was de novo analysis of combined data emphasizing absolute indices of examiner agreement; and tertiary goal was analysis of actual vs. simulated data to determine to what degree the information provided by motion palpation impacted interexaminer reliability.

Methods

This study emphasized Median Absolute Examiner Differences and Bland-Altman Limits of Agreement to calculate examiner differences, which are immune to subject homogeneity, and de-emphasized intraclass correlation, which is not. It calculated Median Absolute Deviation to determine data dispersion. The study analyzed the entire n = 113 combined dataset, as well as subgroups stratified by examiner confidence and spinal region. Simulations were run using a random number generator to provide chance data for examiners' findings of the stiffest spinal site, the analysis of which was compared with that of the actual data.

Results

Median Absolute Examiner Differences for the combined dataset were 0.7 of one vertebral level, suggesting examiners usually agreed on the stiffest spinal site or the motion segment including it. When both examiners were confident in their findings (53.4%), the median examiner difference decreased to 0.6 levels, increasing to 1.0 levels when one lacked confidence and to 1.8 levels when both lacked confidence. Reliability was greater in the cervical and lumbar spines (each 0.6 vertebral levels examiner differences) than in the thoracic spine (1.1 levels examiner differences). The actual information provided by motion palpation compared to simulated data improved interexaminer reliability by a factor ranging from 1.8 times to 4.7 times, depending on the regional subset analyzed.

Conclusions

Examiner decisions regarding the location of the stiffest spinal site were deemed adequately reliable, especially when the examiners were confident. Researchers and clinicians alike might best design their study protocols and practice methods using the stiffest segment protocol as an alternative to level-by-level spinal analysis.

The reliability of lumbar motion palpation using continuous analysis and confidence ratings: choosing a relevant index of agreement

INTRODUCTION

Most studies show motion palpation unreliable. This study's primary objective was assessing its reliability using a continuous measure methods, most-fixated level paradigm, stratified by examiners' confidence; and the secondary objective was comparing various indices of examiner agreement.

METHODS

Thirty-four minimally symptomatic participants were palpated in side posture by two experienced examiners. Interexaminer differences in identifying the most-fixated level and degree of examiner confidence were recorded. Indices of agreement were: Intraclass correlation coefficient, Mean and Median Examiner Absolute Examiner Differences, Root-Mean-Square Error and Bland-Altman Limits of Agreement.

RESULTS

Three of four reliability indices (excluding intraclass correlation) suggested on average examiners agreed on the most fixated motion segment, and agreement increased with confidence. Statistical measures of data dispersion were low. The analyses of subgroups were "fragile" due to small sample size.

DISCUSSION

Although subject homogeneity lowered ICC levels, the other reliability measures were not similarly impacted. Continuous measures statistical analysis demonstrates examiner agreement in situations where discrete analysis with kappa may not.

CONCLUSION

Continuous analysis for the lumbar most-fixated level is reliable. Future studies will need a larger sample size to properly analyze subgroups based on examiner confidence.

Effect of Spinal Manipulation on Pelvic Floor Functional Changes in Pregnant and Nonpregnant Women: A Preliminary Study

OBJECTIVE

The aim of this study was to investigate whether a single session of spinal manipulation of pregnant women can alter pelvic floor muscle function as measured using ultrasonographic imaging.

METHODS

In this preliminary, prospective, comparative study, transperineal ultrasonographic imaging was used to assess pelvic floor anatomy and function in 11 primigravid women in their second trimester recruited via notice boards at obstetric caregivers, pregnancy keep-fit classes, and word of mouth and 15 nulliparous women recruited from a convenience sample of female students at the New Zealand College of Chiropractic. Following bladder voiding, 3-/4-dimensional transperineal ultrasonography was performed on all participants in the supine position. Levator hiatal area measurements at rest, on maximal pelvic floor contraction, and during maximum Valsalva maneuver were collected before and after either spinal manipulation or a control intervention.

RESULTS

Levator hiatal area at rest increased significantly (P < .05) after spinal manipulation in the pregnant women, with no change postmanipulation in the nonpregnant women at rest or in any of the other measured parameters.

CONCLUSION

Spinal manipulation of pregnant women in their second trimester increased the levator hiatal area at rest and thus appears to relax the pelvic floor muscles. This did not occur in the nonpregnant control participants, suggesting that it may be pregnancy related.

Clinical evaluation tools: a survey of doctors of chiropractic and students at one chiropractic college

INTRODUCTION

The reliability and validity of many evaluation tools leading to clinical decision-making for spinal manipulation are varied. We surveyed senior students and DC employees at one chiropractic college regarding 1) which analysis tools should be used and 2) factors that influence their choices.

METHODS

The survey queried which tools should be used on a routine patient encounter. Clinical evaluation tools included palpation, skin temperature analysis, leg length analysis, and radiographs.

RESULTS

Surveys were collected from 58 doctors of chiropractic (DCs) and 74 students. Respondents from both groups reported to most commonly use static palpation, followed by motion palpation and leg length analysis. DC respondents ranked evidence and personal experience high for rationale; student respondents frequently chose patient preference.

CONCLUSION

DC and student respondents reported use of clinical evaluation tools consistently. However, some variations in rationale were noted. It is important for educators to provide a balanced presentation of the strengths and limitations of clinical analysis procedures to support the development of well-justified evidence-based clinical decision-making skills.

Interrater Reliability of Motion Palpation in the Thoracic Spine

Introduction. Manual therapists commonly use assessments of intervertebral motion to determine the need for spinal manipulation, but the reliability of these procedures demonstrates conflicting results. The objectives of this study were to investigate the interrater reliability of thoracic spine motion palpation for perceived joint restriction and pain. Methods. Twenty-five participants between the ages of 18 and 70, with or without mid-back pain, were enrolled. Two raters motion palpated marked T5–T12 levels using two methods (standardised and pragmatic) and noted any restricted or painful segments. We calculated agreement between two raters by generating raw agreement percentages and Kappa coefficients with 95% confidence intervals. Results. There was poor to low level of agreement between the raters for both joint stiffness and pain localization using both pragmatic and standardized approaches. The results did not improve significantly when we conducted a post hoc analysis where three spinal levels were collapsed as one and right and left sides were also combined. Conclusions. The results for interrater reliability were poor for motion restriction and pain. These findings may have unfavourable implications for all manual therapists who use motion palpation to select patients appropriate for spinal manipulation.

Inter-examiner reliability of the interpretation of paraspinal thermographic pattern analysis

INTRODUCTION

A few spinal manipulation techniques use paraspinal surface thermography as an examination tool that informs clinical-decision making; however, inter-examiner reliability of this interpretation has not been reported. The purpose of this study was to report inter-examiner reliability for classifying cervical paraspinal thermographic findings.

METHODS

Seventeen doctors of chiropractic self-reporting a minimum of 2 years of experience using thermography classified thermographic scans into categories (full pattern, partial +, partial, partial -, and adaptation). Kappa statistics (k) were calculated to determine inter-examiner reliability.

RESULTS

Overall inter-examiner reliability was fair (k=0.43). There was good agreement for identifying full pattern (k=0.73) and fair agreement for adaptation (k=0.55). Poor agreement was noted in partial categories (k=0.05-0.22).

CONCLUSION

Inter-examiner reliability demonstrated fair to good agreement for identifying comparable (full pattern) and disparate (adaptation) thermographic findings; agreement was poor for those with moderate similarity (partial). Further research is needed to determine whether thermographic findings should be used in clinical decision-making for spinal manipulation.

The location of the inferior angle of the scapula in relation to the spine in the upright position: a systematic review of the literature and meta-analysis

Practitioners in several of the health care professions use anatomical landmarks to identify spinal levels, both in order to enhance diagnostic accuracy and to specifically target the site of intervention. Authoritative sources usually state the upright inferior scapular angle (IAS) aligns with the spinous process (SP) of T7, but some specify the T7-8 interspace or the T8 SP. The primary goals of this study were to systematically review the relevant literature; and conduct a meta-analysis of the pooled data from retrieved studies to increase their statistical power. Electronic searching retrieved primary studies relating the IAS to a spinal level, as determined by an imaging reference standard, using combinations of these search terms: scapula, location, landmark, spinous process, thoracic vertebrae, vertebral level, palpation, and spine. Only primary studies were included; review articles and reliability studies related to scapular position but lacking spinal correlations were excluded. Eight-hundred and eighty (880) articles of interest were identified, 43 abstracts were read, 22 full text articles were inspected, and 5 survived the final cut. Each article (with one exception) was rated for quality using the QUADAS instrument. Pooling data from 5 studies resulted in normal distribution in which the upright IAS on average aligns closely with the T8 SP, range T4-T11. Since on average the IAS most closely identifies the T8 SP in the upright position, it is very likely that health professionals, both manual therapists and others, who have been diagnosing and treating patients based on the IAS = T7 SP rule (the conventional wisdom), have not been as segmentally accurate as they may have supposed. They either addressed non-intended levels, or made numeration errors in their charting. There is evidence that using the IAS is less preferred than using the vertebra prominens, and may be less preferred than using the iliac crest for identifying spinal levels Manual therapists, acupuncturists, anesthesiologists, nurses, and surgeons should reconsider their procedures for identifying spinal sites in light of this modified information. Inaccurate landmark benchmark rules will add to patient variation and examiner errors in producing spine care targeting errors, and confound research on the importance of specificity in treating spinal levels.

Changes in H-reflex and V-waves following spinal manipulation

This study investigates whether spinal manipulation leads to neural plastic changes involving cortical drive and the H-reflex pathway. Soleus evoked V-wave, H-reflex, and M-wave recruitment curves and maximum voluntary contraction (MVC) in surface electromyography (SEMG) signals of the plantar flexors were recorded from ten subjects before and after manipulation or control intervention. Dependent measures were compared with 2-way ANOVA and Tukey's HSD as post hoc test, p was set at 0.05. Spinal manipulation resulted in increased MVC (measured with SEMG) by 59.5 ± 103.4 % (p = 0.03) and force by 16.05 ± 6.16 4 % (p = 0.0002), increased V/M max ratio by 44.97 ± 36.02 % (p = 0.006), and reduced H-reflex threshold (p = 0.018). Following the control intervention, there was a decrease in MVC (measured with SEMG) by 13.31 ± 7.27 % (p = 0.001) and force by 11.35 ± 9.99 % (p = 0.030), decreased V/M max ratio (23.45 ± 17.65 %; p = 0.03) and a decrease in the median frequency of the power spectrum (p = 0.04) of the SEMG during MVC. The H-reflex pathway is involved in the neural plastic changes that occur following spinal manipulation. The improvements in MVC following spinal manipulation are likely attributed to increased descending drive and/or modulation in afferents. Spinal manipulation appears to prevent fatigue developed during maximal contractions. Spinal manipulation appears to alter the net excitability of the low-threshold motor units, increase cortical drive, and prevent fatigue.

Review of methods used by chiropractors to determine the site for applying manipulation

Background

With the development of increasing evidence for the use of manipulation in the management of musculoskeletal conditions, there is growing interest in identifying the appropriate indications for care. Recently, attempts have been made to develop clinical prediction rules, however the validity of these clinical prediction rules remains unclear and their impact on care delivery has yet to be established. The current study was designed to evaluate the literature on the validity and reliability of the more common methods used by doctors of chiropractic to inform the choice of the site at which to apply spinal manipulation.

Methods

Structured searches were conducted in Medline, PubMed, CINAHL and ICL, supported by hand searches of archives, to identify studies of the diagnostic reliability and validity of common methods used to identify the site of treatment application. To be included, studies were to present original data from studies of human subjects and be designed to address the region or location of care delivery. Only English language manuscripts from peer-reviewed journals were included. The quality of evidence was ranked using QUADAS for validity and QAREL for reliability, as appropriate. Data were extracted and synthesized, and were evaluated in terms of strength of evidence and the degree to which the evidence was favourable for clinical use of the method under investigation.

Results

A total of 2594 titles were screened from which 201 articles met all inclusion criteria. The spectrum of manuscript quality was quite broad, as was the degree to which the evidence favoured clinical application of the diagnostic methods reviewed. The most convincing favourable evidence was for methods which confirmed or provoked pain at a specific spinal segmental level or region. There was also high quality evidence supporting the use, with limitations, of static and motion palpation, and measures of leg length inequality. Evidence of mixed quality supported the use, with limitations, of postural evaluation. The evidence was unclear on the applicability of measures of stiffness and the use of spinal x-rays. The evidence was of mixed quality, but unfavourable for the use of manual muscle testing, skin conductance, surface electromyography and skin temperature measurement.

Conclusions

A considerable range of methods is in use for determining where in the spine to administer spinal manipulation. The currently published evidence falls across a spectrum ranging from strongly favourable to strongly unfavourable in regard to using these methods. In general, the stronger and more favourable evidence is for those procedures which take a direct measure of the presumptive site of care– methods involving pain provocation upon palpation or localized tissue examination. Procedures which involve some indirect assessment for identifying the manipulable lesion of the spine–such as skin conductance or thermography–tend not to be supported by the available evidence.

Interexaminer reliability of cervical motion palpation using continuous measures and rater confidence levels

INTRODUCTION

Motion palpators usually rate the movement of each spinal level palpated, and their reliability is assessed based upon discrete paired observations. We hypothesized that asking motion palpators to identify the most fixated cervical spinal level to allow calculating reliability at the group level might be a useful alternative approach.

METHODS

Three examiners palpated 29 asymptomatic supine participants for cervical joint hypomobility. The location of identified hypomobile sites was based on their distance from the T1 spinous process. Interexaminer concordance was estimated by calculating Intraclass Correlation Coefficient (ICC) and mean absolute differences (MAD) values, stratified by degree of examiner confidence.

RESULTS

For the entire participant pool, ICC [2,1] = 0.61, judged "good." MAD=1.35 cm, corresponding to mean interexaminer differences of about 75% of one cervical vertebral level. Stratification by examiner confidence levels resulted in small subgroups with equivocal results.

DISCUSSION AND CONCLUSION

A continuous measures study methodology for assessing cervical motion palpation reliability showed more examiner concordance than was usually the case in previous studies using discrete methodology.