Diagnostic agreement in psychiatry. A comparative study between ICD-9, ICD-10 and DSM-III-R

Five-year rehospitalization experience of a state-wide cohort of persons with schizophrenia

PURPOSE

This study has the objective of assessing the psychiatric rehospitalization experience of a large cohort of persons with schizophrenia and modeling the effects of personal and systemic conditions on rehospitalization risk.

METHODS

The study employs a secondary analysis of US data from Massachusetts' casemix database of all patients discharged from acute general hospital units. It focuses on 11,291 patients during 1994-2000 who were discharged from acute adult psychiatric units. Predictors used include basic demographics, length of stay, continuity of care with doctors and facilities, diagnoses, discharge referral, type of insurance, and distance to and selected socioeconomic characteristics of the patient's home zip code. Data are analyzed with descriptive statistics and modeled with the Cox proportional hazard model. The model was assessed through split-half reliability testing, the generalized R², and Harrell's Concordance Index.

RESULTS

Overall, 13.4% of patients were rehospitalized within 1 month; 38.9% within 1 year; and 64.1% within 5 years. Predictors that are most strongly associated with lower rehospitalization rates include continuity of care, discharge to a chronic hospital, and density of home zipcode, whereas discharge to another acute psychiatric unit had the greatest effect on increasing risk of rehospitalization. Overall the Cox model has generalized R² of 0.343 and a Concordance Index of 0.734.

CONCLUSIONS

The results highlight the need to enhance the continuity of the relationships with providers, whether these are with the assigned psychiatrists or other therapists and case managers, as well as workplace issues involving staff turnover and hospital assignment and admitting policies.

Interrater reliability of schizoaffective disorder compared with schizophrenia, bipolar disorder, and unipolar depression - A systematic review and meta-analysis

Schizoaffective disorder is a common diagnosis in clinical practice but its nosological status has been subject to debate ever since it was conceptualized. Although it is key that diagnostic reliability is sufficient, schizoaffective disorder has been reported to have low interrater reliability. Evidence based on systematic review and meta-analysis methods, however, is lacking. Using a highly sensitive literature search in Medline, Embase, and PsycInfo we identified studies measuring the interrater reliability of schizoaffective disorder in comparison to schizophrenia, bipolar disorder, and unipolar disorder. Out of 4126 records screened we included 25 studies reporting on 7912 patients diagnosed by different raters. The interrater reliability of schizoaffective disorder was moderate (meta-analytic estimate of Cohen's kappa 0.57 [95% CI: 0.41-0.73]), and substantially lower than that of its main differential diagnoses (difference in kappa between 0.22 and 0.19). Although there was considerable heterogeneity, analyses revealed that the interrater reliability of schizoaffective disorder was consistently lower in the overwhelming majority of studies. The results remained robust in subgroup and sensitivity analyses (e.g., diagnostic manual used) as well as in meta-regressions (e.g., publication year) and analyses of publication bias. Clinically, the results highlight the particular importance of diagnostic re-evaluation in patients diagnosed with schizoaffective disorder. They also quantify a widely held clinical impression of lower interrater reliability and agree with earlier meta-analysis reporting low test-retest reliability.

Psychiatric disorders in primary care: why so many differences?

Scopo - (I) Operare una revisione critica degli studi che hanno indagato la frequenza dei disturbi psichici tra i pazienti della medicina generale, utilizzando come criterio un'intervista psichiatrica standardizzata; (II) discutere i possibili fattori all'origine delle diverse frequenze riscontrate; (III) mettere in luce il ruolo di primaria importanza che possono assolvere in questo campo gli studi multicentrici condotti grazie a programmi di collaborazione internazionale. Metodo - sono stati selezionati i lavori pubblicati nella letteratura internazionale tra il 1970 ed il 1993. La selezione è stata effettuata grazie ad una ricerca computerizzata, nel corso della quale sono stati incrociati i databases MEDLINE e Psychological Abstracts; questa è stata a sua volta integrata da una ricerca manuale, consistente neU'esame delle bibliografie contenute nei lavori cosi individuati e nella consultazione degli indici analitici delle principal! riviste psichiatriche internazionali. Infine, è stata consultata la bibliografia commentata, curata da Wilkinson (1985), relativa a lavori di interesse e pertinenza psichiatrici condotti nella medicina generale e pubblicati tra il 1977 ed il 1985. Risultati - Sono stati individuati 30 lavori, ma 9 sono stati esclusi, dal momento che replicavano dati già riportati in letteratura o presentavano limiti metodologici. È emerso cosi che indagini sulla frequenza complessiva dei disturbi psichici tra i soggetti che si rivolgevano ai servizi di medicina generale sono state finora condotte secondo procedure standardizzate solo in 13 Paesi. La frequenza dei disturbi psichici variava tra il 10% ed il 50% circa. Le possibili ragioni all'origine dellem diverse frequenze riscontrate sono state discusse alia luce di tre fattori: I) le caratteristiche dell'indagine; II) l'affidabilità e la confrontabilità delle categorie diagnostiche e dei sistemi nosografici utilizzati; e III) la diversa organizzazione dei servizi sanitari nei vari Paesi. Conclusioni - alia luce delle difficoltà che si incontrano quando si tenta di confrontare i risultati forniti da studi indipendenti condotti in Paesi diversi o perfino nello stesso Paese viene suggerito il ruolo di primaria importanza che possono assolvere in questo campo gli studi multicentrici condotti grazie a programmi di collaborazione internazionale.

A comparison of acute psychiatric care under Medicaid carve-outs, HMOs, and fee-for-service

This study compares the use of acute psychiatric hospitalization; selected outcomes, including rehospitalization; as well as costs associated with the health maintenance organization (HMO), carve-out, and fee-for-service models as implemented in the Massachusetts Medicaid program between FY1994 and FY2000. This is a longitudinal analysis that primarily uses unduplicated individual data from the Massachusetts Case Mix database. Analyses focus on 56,518 individuals who were psychiatrically hospitalized on acute units within 57 hospitals. They employ Cox regression to compare rehospitalization among the three programs. The hypotheses were strongly supported: HMOs have the most substantial impacts in minimizing service provision, with the carve-out program having an impact intermediate between the HMO and fee-for-service programs. Lower utilization rates were associated with lower overall rates of hospitalization, shorter lengths of stay, fewer repeated stays, and less geographic access and greater displacement of psychiatric patients to medical units. The final model of rehospitalization has an overall predictive accuracy of 59.6%.

Comparing DISC-IV and clinician diagnoses among youths receiving public mental health services

OBJECTIVES

To compare the prevalence and agreement of diagnoses based on Diagnostic Interview Schedule for Children Version IV (DISC-IV) and clinician assignment for youths receiving public mental health services between 1996 and 1997 and to examine potential predictors of diagnostic agreement.

METHOD

Participants included 240 youths aged 6-18 years. Past-year prevalence rates and kappa statistics were calculated for four diagnostic categories: anxiety, mood, attention-deficit/hyperactivity disorder (ADHD), and disruptive behavior disorders (DBD). Potential predictors of diagnostic agreement were examined with logistic regression analysis.

RESULTS

The prevalence of ADHD, DBD, and anxiety disorders was significantly higher based on the DISC-IV, while the prevalence of mood disorders was significantly higher based on clinician assignment. Diagnostic agreement was poor overall. The kappa values ranged from -0.04 for anxiety disorders to 0.22 for ADHD. Significant predictors of agreement varied by diagnosis and included symptom severity, comorbidity, youth age and gender, and school-based problem identification.

CONCLUSIONS

Consistent with previous findings of poor diagnostic agreement between structured interviews and clinicians, these results call for a better understanding of factors affecting diagnostic assignment across different methods. This is especially important if researchers continue to use structured interviews to determine prevalence, establish diagnosis-based treatment guidelines, and disseminate evidence-based treatments to community mental health settings.

Diagnostic agreement between clinicians and the Diagnostic Interview for Children and Adolescents--DICA-R--in an outpatient sample

The paper examines the diagnostic agreement between clinicians and the Diagnostic Interview for Children and Adolescents. One hundred and thirty-seven outpatients-children and adolescents, and their parents-were diagnosed independently following DSM-III-R criteria by clinicians and by the DICA-R. The diagnostic concordance between clinicians and DICA-R ranged from low to moderate in the majority of the categories. The only exception was Conduct Disorder. Differences depending on the informant and the quality of the information (cognitive vs. observable) were observed. Combining the information from the child/adolescent and their parents ameliorates the concordance. The reasons for the scanty agreement found could be due to the fact that clinicians and structured interviews differ in what they evaluate (conditions on which they focus), how they evaluate (strictness in the criteria application, use of different informants and different information etc.), and when they evaluate (present condition vs. lifespan). After analysing the pros and cons of both, the use of structured interviews is advisable for research purposes. There is a clear need for a variety of informants, and the combination of information from different sources is recommended, depending on the age of the children and the type of disorder.

Inter-rater reliability of ICD-10 in Chinese patients

In an attempt to assess the universal applicability of the International Classification of Disease (ICD-10), two psychiatrists from different socio-cultural backgrounds and training independently performed a chart review of 238 Chinese patients. Inter-rater reliability figures were comparable to those found in the WHO-coordinated ICD-10 field trials. The results suggest that ICD-10 has good ‘universality’ in routine clinical practice.

The ECLW Collaborative Study: III. Training and reliability of ICD-10 psychiatric diagnoses in the general hospital setting--an investigation of 220 consultants from 14 European countries. European Consultation Liaison Workgroup

A comprehensive training program for reliable use of the ICD/10 in Consultation-Liaison (C-L) psychiatry was conducted with 220 psychiatrists and psychologists from 14 European countries. The training included rating of written test cases and development of a coding manual to avoid diagnostic pitfalls not addressed in the ICD-10 manual. Following this training, all consultants rated 13 written case histories. One hundred sixty-seven consultants (76%) had a kappa (kappa) of at least 0.70. Only 13 (6%) had a kappa 0.40. The percentage of high reliability raters was evenly distributed among the different countries. Consultants had some problems in the differentiation between adjustment disorders and depressive disorders, and in the classification of disorders where ICD-10 differs from the DSM-III-R system. National biases in diagnostic practice were found with regard to the "case" concept and the role of alcohol in confusional states. Finnish consultants coded "no psychiatric disorder" significantly more often, whereas German and Italian consultants attributed delirious state more often to alcohol than consultants from other European countries. The study demonstrates that it is possible to achieve acceptable interrater reliability in applying the ICD-10 guidelines, through training programs designed for C-L psychiatrists and psychologists. Nevertheless, this first cross-national study shows the importance of addressing differences in national diagnostic practice.

Time trends in the care-based incidence of schizophrenia

BACKGROUND

Several studies have suggested a declining first-admission rate for schizophrenia. This study examines the care-based incidence of schizophrenia in a Dutch register area.

METHOD

Data from Groningen psychiatric case register were used to compare first-admission rates for schizophrenia over 1976-90 with those of other functional psychoses, and to consider various potential biases.

RESULTS

Diagnostic habits probably affected time trends in incidence rates. Using a broader definition of schizophrenia, no evidence was found for a decrease in the incidence of schizophrenia. Although first admissions to intramural services showed a (non-significant) decrease, this effect seemed to be neutralised when all mental health services were taken into account.

CONCLUSIONS

Care-based studies of time trends in psychiatric disorders should embrace all mental health services. Furthermore, unless a diagnostic classification system with univocal criteria is used, bias caused by changing diagnostic habits cannot be ruled out.