Identifying primary care datasets and perspectives on their secondary use: a survey of Australian data users and custodians

Measuring what counts in Aboriginal and Torres Strait Islander care: a review of general practice datasets available for assessing chronic disease care

Background Large datasets exist in Australia that make de-identified primary healthcare data extracted from clinical information systems available for research use. This study reviews these datasets for their capacity to provide insight into chronic disease care for Aboriginal and Torres Strait Islander peoples, and the extent to which the principles of Indigenous Data Sovereignty are reflected in data collection and governance arrangements. Methods Datasets were included if they collect primary healthcare clinical information system data, collect data nationally, and capture Aboriginal and Torres Strait Islander peoples. We searched PubMed and the public Internet for data providers meeting the inclusion criteria. We developed a framework to assess data providers across domains, including representativeness, usability, data quality, adherence with Indigenous Data Sovereignty and their capacity to provide insights into chronic disease. Datasets were assessed against the framework based on email interviews and publicly available information. Results We identified seven datasets. Only two datasets reported on chronic disease, collected data nationally and captured a substantial number of Aboriginal and Torres Strait Islander patients. No dataset was identified that captured a significant number of both mainstream general practice clinics and Aboriginal Community Controlled Health Organisations. Conclusions It is critical that more accurate, comprehensive and culturally meaningful Aboriginal and Torres Strait Islander healthcare data are collected. These improvements must be guided by the principles of Indigenous Data Sovereignty and Governance. Validated and appropriate chronic disease indicators for Aboriginal and Torres Strait Islander peoples must be developed, including indicators of social and cultural determinants of health.

Undercounting diagnoses in Australian general practice: a data quality study with implications for population health reporting

BACKGROUND

Diagnosis can often be recorded in electronic medical records (EMRs) as free-text or using a term with a diagnosis code. Researchers, governments, and agencies, including organisations that deliver incentivised primary care quality improvement programs, frequently utilise coded data only and often ignore free-text entries. Diagnosis data are reported for population healthcare planning including resource allocation for patient care. This study sought to determine if diagnosis counts based on coded diagnosis data only, led to under-reporting of disease prevalence and if so, to what extent for six common or important chronic diseases.

METHODS

This cross-sectional data quality study used de-identified EMR data from 84 general practices in Victoria, Australia. Data represented 456,125 patients who attended one of the general practices three or more times in two years between January 2021 and December 2022. We reviewed the percentage and proportional difference between patient counts of coded diagnosis entries alone and patient counts of clinically validated free-text entries for asthma, chronic kidney disease, chronic obstructive pulmonary disease, dementia, type 1 diabetes and type 2 diabetes.

RESULTS

Undercounts were evident in all six diagnoses when using coded diagnoses alone (2.57-36.72% undercount), of these, five were statistically significant. Overall, 26.4% of all patient diagnoses had not been coded. There was high variation between practices in recording of coded diagnoses, but coding for type 2 diabetes was well captured by most practices.

CONCLUSION

In Australia clinical decision support and the reporting of aggregated patient diagnosis data to government that relies on coded diagnoses can lead to significant underreporting of diagnoses compared to counts that also incorporate clinically validated free-text diagnoses. Diagnosis underreporting can impact on population health, healthcare planning, resource allocation, and patient care. We propose the use of phenotypes derived from clinically validated text entries to enhance the accuracy of diagnosis and disease reporting. There are existing technologies and collaborations from which to build trusted mechanisms to provide greater reliability of general practice EMR data used for secondary purposes.

The knowledge and reuse practices of researchers utilising government health information assets, Victoria, Australia, 2008-2020

BACKGROUND

Using government health datasets for secondary purposes is widespread; however, little is known on researchers' knowledge and reuse practices within Australia.

OBJECTIVES

To explore researchers' knowledge and experience of governance processes, and their data reuse practices, when using Victorian government health datasets for research between 2008-2020.

METHOD

A cross-sectional quantitative survey was conducted with authors who utilised selected Victorian, Australia, government health datasets for peer-reviewed research published between 2008-2020. Information was collected on researchers': data reuse practices; knowledge of government health information assets; perceptions of data trustworthiness for reuse; and demographic characteristics.

RESULTS

When researchers used government health datasets, 45% linked their data, 45% found the data access process easy and 27% found it difficult. Government-curated datasets were significantly more difficult to access compared to other-agency curated datasets (p = 0.009). Many respondents received their data in less than six months (58%), in aggregated or de-identified form (76%). Most reported performing their own data validation checks (70%). To assist in data reuse, almost 71% of researchers utilised (or created) contextual documentation, 69% a data dictionary, and 62% limitations documentation. Almost 20% of respondents were not aware if data quality information existed for the dataset they had accessed. Researchers reported data was managed by custodians with rigorous confidentiality/privacy processes (94%) and good data quality processes (76%), yet half lacked knowledge of what these processes entailed. Many respondents (78%) were unaware if dataset owners had obtained consent from the dataset subjects for research applications of the data.

CONCLUSION

Confidentiality/privacy processes and quality control activities undertaken by data custodians were well-regarded. Many respondents included data linkage to additional government datasets in their research. Ease of data access was variable. Some documentation types were well provided and used, but improvement is required for the provision of data quality statements and limitations documentation. Provision of information on participants' informed consent in a dataset is required.

DQAgui: a graphical user interface for the MIRACUM data quality assessment tool

Background

With the growing impact of observational research studies, there is also a growing focus on data quality (DQ). As opposed to experimental study designs, observational research studies are performed using data mostly collected in a non-research context (secondary use). Depending on the number of data elements to be analyzed, DQ reports of data stored within research networks can grow very large. They might be cumbersome to read and important information could be overseen quickly. To address this issue, a DQ assessment (DQA) tool with a graphical user interface (GUI) was developed and provided as a web application.

Methods

The aim was to provide an easy-to-use interface for users without prior programming knowledge to carry out DQ checks and to present the results in a clearly structured way. This interface serves as a starting point for a more detailed investigation of possible DQ irregularities. A user-centered development process ensured the practical feasibility of the interactive GUI. The interface was implemented in the R programming language and aligned to Kahn et al.’s DQ categories conformance, completeness and plausibility.

Results

With DQAgui, an R package with a web-app frontend for DQ assessment was developed. The GUI allows users to perform DQ analyses of tabular data sets and to systematically evaluate the results. During the development of the GUI, additional features were implemented, such as analyzing a subset of the data by defining time periods and restricting the analyses to certain data elements.

Conclusions

As part of the MIRACUM project, DQAgui is now being used at ten German university hospitals for DQ assessment and to provide a central overview of the availability of important data elements in a datamap over 2 years. Future development efforts should focus on design optimization and include a usability evaluation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01961-z.