Reliability of Self-Reported Health Service Use: Evidence from the Women with Co-occurring Disorders, and Violence Study

Health Service Utilisation of People Living with Psychosis: Validity of Self-report Compared with Administrative Data in a Randomised Controlled Trial

BACKGROUND

Self-reported service use informs resource utilisation and cost estimates, though its validity for use within economic evaluations is uncertain.

OBJECTIVE

The aim of this study is to assess agreement in health resource-use measurement between self-reported and administrative data across different resource categories, over time and between different recall periods by subgroups among Australians living with psychosis.

METHODS

Data were obtained for 104 participants with psychotic disorders from a randomised controlled trial. Agreement between self-reported resource-use questionnaires and administrative data on community-based services and medication use was assessed through estimating differences of group mean number of visits and medications used and intraclass correlation coefficients (ICC) over multiple time periods.

RESULTS

ICC showed moderate agreement across most time periods for general practitioners, psychiatrists and mental health medications. No clear trends were discernible over time, between varying lengths of recall periods nor across participant subgroups.

CONCLUSION

Despite poor agreement, when measuring visits to psychologists and other health professionals, small overall differences in group mean number of visits indicate that self-reported data may still be valid for use in economic evaluations in people living with psychosis.

Aspects and Challenges of Resource Use Measurement in Health Economics: Towards a Comprehensive Measurement Framework

BACKGROUND

While the methods for conducting health economics research in general are improving, current guidelines provide limited guidance regarding resource use measurement (RUM). Consequently, a variety of methods exists, yet there is no overview of aspects to consider when deciding on the most appropriate RUM methodology. Therefore, this study aims to (1) identify and categorize existing knowledge regarding aspects of RUM, and (2) develop a framework that provides a comprehensive overview of methodological aspects regarding RUM.

METHODS

Relevant articles were identified by enrolling a search string in six databases and handsearching the DIRUM database. Included articles were descriptively reviewed and served as input for a comprehensive framework. Health economics experts were involved during the process to establish the framework's face validity.

RESULTS

Forty articles were included in the scoping review. The RUM framework consists of four methodological RUM domains: 'Whom to measure', addressing whom to ask and whom to measure; 'How to measure', addressing the different approaches of measurement; 'How often to measure', addressing recall period and measurement patterns; and 'Additional considerations', which covers additional aspects that are essential for further refining the methodologies for measurement. Evidence retrieved from the scoping review was categorized according to these domains.

CONCLUSION

This study clustered the aspects of RUM methodology in health economics into a comprehensive framework. The results may guide health economists in their decision making regarding the selection of appropriate RUM methods and developing instruments for RUM. Furthermore, policy makers may use these findings to review study results from an evidence-based perspective.

Cross-national patterns of substance use disorder treatment and associations with mental disorder comorbidity in the WHO World Mental Health Surveys

AIMS

To examine cross-national patterns of 12-month substance use disorder (SUD) treatment and minimally adequate treatment (MAT), and associations with mental disorder comorbidity.

DESIGN

Cross-sectional, representative household surveys.

SETTING

Twenty-seven surveys from 25 countries of the WHO World Mental Health Survey Initiative.

PARTICIPANTS

A total of 2446 people with past-year DSM-IV SUD diagnoses (alcohol or illicit drug abuse and dependence).

MEASUREMENTS

Outcomes were SUD treatment, defined as having either received professional treatment or attended a self-help group for substance-related problems in the past 12 months, and MAT, defined as having either four or more SUD treatment visits to a health-care professional, six or more visits to a non-health-care professional or being in ongoing treatment at the time of interview. Covariates were mental disorder comorbidity and several socio-economic characteristics. Pooled estimates reflect country sample sizes rather than population sizes.

FINDINGS

Of respondents with past-year SUD, 11.0% [standard error (SE) = 0.8] received past 12-month SUD treatment. SUD treatment was more common among people with comorbid mental disorders than with pure SUDs (18.1%, SE = 1.6 versus 6.8%, SE = 0.7), as was MAT (84.0%, SE = 2.5 versus 68.3%, SE = 3.8) and treatment by health-care professionals (88.9%, SE = 1.9 versus 78.8%, SE = 3.0) among treated SUD cases. Adjusting for socio-economic characteristics, mental disorder comorbidity doubled the odds of SUD treatment [odds ratio (OR) = 2.34; 95% confidence interval (CI) = 1.71-3.20], MAT among SUD cases (OR = 2.75; 95% CI = 1.90-3.97) and MAT among treated cases (OR = 2.48; 95% CI = 1.23-5.02). Patterns were similar within country income groups, although the proportions receiving SUD treatment and MAT were higher in high- than low-/middle-income countries.

CONCLUSIONS

Few people with past-year substance use disorders receive adequate 12-month substance use disorder treatment, even when comorbid with a mental disorder. This is largely due to the low proportion of people receiving any substance use disorder treatment, as the proportion of patients whose treatment is at least minimally adequate is high.

Regional differences in the validity of self-reported use of health care in Belgium: selection versus reporting bias

Background

The Health Care Module of the European Health Interview Survey (EHIS) is aimed to obtain comparable information on the use of inpatient and ambulatory care in all EU member states. In this study we assessed the validity of self-reported information on the use of health care, collected through this instrument, in the Belgian Health Interview Survey (BHIS), and explored the impact of selection and reporting bias on the validity of regional differences in health care use observed in the BHIS.

Methods

To assess reporting bias, self-reported BHIS 2008 data were linked with register-based data from the Belgian compulsory health insurance (BCHI). The latter were compared with similar estimates from a random sample of the BCHI to investigate the selection bias. Outcome indicators included the prevalence of a contact with a GP, specialist, dentist and a physiotherapist, as well as inpatient and day patient hospitalisation. The validity of the estimates and the regional differences were explored through measures of agreement and logistic regression analyses.

Results

Validity of self-reported health care use varies by type of health service and is more affected by reporting than by selection bias. Compared to health insurance estimates, self-reported results underestimate the percentage of people with a specialist contact in the past year (50.5 % versus 65.0 %) and a day patient hospitalisation (7.8 % versus 13.9 %). Inversely, survey results overestimated the percentage of people having visited a dentist in the past year: 58.3 % versus 48.6 %. The best concordance was obtained for an inpatient hospitalisation (kappa 0.75). Survey data overestimate the higher prevalence of a contact with a specialist [OR 1.51 (95 % CI 1.33–1.72) for self-report and 1.08 (95 % CI 1.05–1.15) for register] and underestimate the lower prevalence of a contact with a GP [ORs 0.59 (95 % CI 0.51–0.70) and 0.41 (95 % CI 0.39–0.42) respectively] in Brussels compared to Flanders.

Conclusion

Cautiousness is needed to interpret self-reported use of health care, especially for ambulatory care. Regional differences in self-reported health care use may be influenced by regional differences in the validity of the self-reported information.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0198-z) contains supplementary material, which is available to authorized users.

Measuring use of services for mental health problems in epidemiological surveys

The use of services for mental problems is generally reported as being relatively low. However, the methods used for data collection in surveys may have influenced the quality of self-reported service use. This study compares the information on recourse to physicians for mental problems reported in different sections of a survey conducted in six European countries. Thus, 5545 respondents were asked questions on contacts with physicians at least twice: (1) after the symptoms checklist in any completed diagnostic section, and (2) in a section devoted to use of care for mental problems. Of these 39.3% reported contacts with physicians about mental problems in the diagnostic sections, whereas 29.5% did so in the use-of-care section. Inconsistencies concerned 20.1% of participants, among whom those reporting consultations in diagnostic sections without reporting them in the use-of-care section represented the majority (74.4%). Multiple logistic regression analysis revealed that age, marital status, educational level and country were associated with under-reporting in the use-of-care section, as well as having mood or sleep problems. In conclusion, services used for mental health reasons when measured through a question referring to use of care due to the presence of a mental problem may underestimate the care people received for their problems.

Accuracy of self-reports of mental health care utilization and calculated costs compared to hospital records

Assessments of service utilization is often based on self-reports. Concerns regarding the accuracy of self-reports are raised especially in mental health care. The purpose of this study was to analyze the accuracy of self-reports and calculated costs of mental health services. In a prospective cohort study in Germany, self-reports regarding psychiatric inpatient and day-care use collected by telephone interviews based on the Client Socio-Demographic and Service Receipt Inventory (CSSRI) as well as calculated costs were compared to computerized hospital records. The sample consisted of patients with mental and behavioral disorders resulting from alcohol (ICD-10 F10, n=84), schizophrenia, schizophrenic and delusional disturbances (F2, n=122) and affective disorders (F3, n=124). Agreement was assessed using the concordance correlation coefficient (CCC), mean difference (95% confidence intervals (CI)) and the 95% limits of agreement. Predictors for disagreement were derived. Overall agreement of mean total costs was excellent (CCC=0.8432). Costs calculated based on self-reports were higher than costs calculated based on hospital records (15 EUR (95% CI -434 to 405)). Overall agreement of total costs for F2 patients was CCC=0.8651, for F3 CCC=0.7850 and for F10 CCC=0.6180. Depending on type of service, measure of service utilization and costs agreement ranged from excellent to poor and varied substantially between individuals. The number of admissions documented in hospital records was significantly associated with disagreement. Telephone interviews can be an accurate data collection method for calculating mean total costs in mental health care. In the future more standardization is needed.

Changes in treatment content of services during trauma-informed integrated services for women with co-occurring disorders

The experience of trauma is highly prevalent in the lives of women with mental health and substance abuse problems. We examined how an intervention targeted to provide trauma-informed integrated services in the treatment of co-occurring disorders has changed the content of services reported by clients. We found that the intervention led to an increased provision of integrated services as well as services addressing each content area: trauma, mental health and substance abuse. There was no increase in service quantity from the intervention. Incorporation of trauma-specific element in the treatment of mental health and substance abuse may have been successfully implemented at the service level thereby better serve women with complex behavioral health histories.

Validation of self-reported data on assisted conception in The Danish National Birth Cohort

BACKGROUND

An increasing number of children are born after assisted conception and in surveillance programmes information on mode of conception is often achieved via maternal self-report. We assessed the validity of self-reported assisted conception in The Danish National Birth Cohort (DNBC), a prospective pregnancy cohort. Here, the term assisted conception refers to IVF, ICSI, ovulation induction and insemination.

METHODS

We compared self-reported assisted conception in the DNBC to corresponding data from Danish national registers; the IVF Register and Danish Drug Prescription Register, providing method of conception in the entire population. In the DNBC, 101,042 women accepted the invitation in early pregnancy from 1996 to 2002. Our final study population comprised 88,151 DNBC women aged 20 years and older who participated in the first DNBC interview with a pregnancy resulting in a live born child.

RESULTS

In the DNBC, assisted conception was reported with a sensitivity of 83% and positive predictive value of 88%. Misclassification was largely explained by ambiguous phrasing of the DNBC interview question and interview skip patterns. Women with false negative reporting were more often multipara (P < 0.001) and older (P = 0.027 for IVF/ICSI and P = 0.002 for ovulation induction). The risk ratio (RR) for being born preterm in IVF/ICSI children was lower for children identified via the DNBC, RR 3.61 (95% confidence interval (CI) 3.31-3.94), than the IVF Register, RR 4.36 (95% CI 4.02-4.74).

CONCLUSIONS

There was a high positive predictive value of self-reported assisted conception in the DNBC, but the structure of the DNBC interview represented a problem and misclassification could introduce bias.