Challenges in accessing routinely collected data from multiple providers in the UK for primary studies: Managing the morass

The use of routinely collected healthcare records for outcome assessment in clinical trials: a UK perspective

The use of routinely collected electronic healthcare records (EHR) for outcome assessment in clinical trials has been described as a 'disruptive' new technique more than a decade ago. Despite this potential, significant methodological issues and regulatory barriers have hampered the progress in this area. This article discusses the key considerations that trialists should take into account when incorporating EHR into their trials. These include considerations of the clinical relevance of the outcome, data timeliness and quality, ethical and regulatory issues, and some practical considerations for clinical trials units. In addition, this article describes the benefits of using EHR which include cost, reduced trial burden for participants and staff, follow up efficiencies, and improved health economic evaluation procedures. We also describe the major regulatory and start up costs of using EHR in clinical trials. This article focuses on the UK specific EHR landscape in clinical trials and would help researchers and trials units considering the use of this method of outcome data collection in their next trial. If the issues described are mitigated, this method will be a formidable tool for conducting pragmatic clinical trials.

Now Is the Time to Strengthen Government-Academic Data Infrastructures to Jump-Start Future Public Health Crisis Response

During public health crises, the significance of rapid data sharing cannot be overstated. In attempts to accelerate COVID-19 pandemic responses, discussions within society and scholarly research have focused on data sharing among health care providers, across government departments at different levels, and on an international scale. A lesser-addressed yet equally important approach to sharing data during the COVID-19 pandemic and other crises involves cross-sector collaboration between government entities and academic researchers. Specifically, this refers to dedicated projects in which a government entity shares public health data with an academic research team for data analysis to receive data insights to inform policy. In this viewpoint, we identify and outline documented data sharing challenges in the context of COVID-19 and other public health crises, as well as broader crisis scenarios encompassing natural disasters and humanitarian emergencies. We then argue that government-academic data collaborations have the potential to alleviate these challenges, which should place them at the forefront of future research attention. In particular, for researchers, data collaborations with government entities should be considered part of the social infrastructure that bolsters their research efforts toward public health crisis response. Looking ahead, we propose a shift from ad hoc, intermittent collaborations to cultivating robust and enduring partnerships. Thus, we need to move beyond viewing government-academic data interactions as 1-time sharing events. Additionally, given the scarcity of scholarly exploration in this domain, we advocate for further investigation into the real-world practices and experiences related to sharing data from government sources with researchers during public health crises.

A DELPHI study priority setting the remaining challenges for the use of routinely collected data in trials: COMORANT-UK

BACKGROUND

Researchers are increasingly seeking to use routinely collected data to support clinical trials. This approach has the potential to transform the way clinical trials are conducted in the future. The availability of routinely collected data for research, whether healthcare or administrative, has increased, and infrastructure funding has enabled much of this. However, challenges remain at all stages of a trial life cycle. This study, COMORANT-UK, aimed to systematically identify, with key stakeholders across the UK, the ongoing challenges related to trials that seek to use routinely collected data.

METHODS

This three-step Delphi method consisted of two rounds of anonymous web-based surveys and a virtual consensus meeting. Stakeholders included trialists, data infrastructures, funders of trials, regulators, data providers and the public. Stakeholders identified research questions or challenges that they considered were of particular importance and then selected their top 10 in the second survey. The ranked questions were taken forward to the consensus meeting for discussion with representatives invited from the stakeholder groups.

RESULTS

In the first survey, 66 respondents yielded over 260 questions or challenges. These were thematically grouped and merged into a list of 40 unique questions. Eighty-eight stakeholders then ranked their top ten from the 40 questions in the second survey. The most common 14 questions were brought to the virtual consensus meeting in which stakeholders agreed a top list of seven questions. We report these seven questions which are within the following domains: trial design, Patient and Public Involvement, trial set-up, trial open and trial data. These questions address both evidence gaps (requiring further methodological research) and implementation gaps (requiring training and/or service re-organisation).

CONCLUSION

This prioritised list of seven questions should inform the direction of future research in this area and should direct efforts to ensure that the benefits in major infrastructure for routinely collected data are achieved and translated. Without this and future work to address these questions, the potential societal benefits of using routinely collected data to help answer important clinical questions will not be realised.

Identifying researcher learning needs to develop online training for UK researchers working with administrative data: CENTRIC training

BACKGROUND

The use of administrative data in health and social science research continues to expand, with increased availability of data and interest from funders. Researchers, however, continue to experience delays in access, storage and sharing of administrative data. Training opportunities are limited and typically specific to individual data providers or focussed on the analytical aspects of working with administrative data. The CENTRIC study was funded by the Information Commissioners Office, with the aim of developing a broader training curriculum for researchers working with administrative data in the UK.

METHODS

A mixed-methods design informed curriculum content, including surveys with researchers, focus group discussions with data providers and workshops with members of the public. Researchers were identified from relevant administrative data networks and invited to participate in an online survey identifying training needs. Data providers were approached with a request to input to a face-to-face or online meeting with two members of the research team about their experiences of working with researchers. Data were analysed within the broad framework of the interview schedule, free text responses in the survey were analysed thematically.

RESULTS

107 researchers responded to the online survey and four data providers participated in the focus groups. We identified five main themes, relating to research training needs for UK researchers working with administrative data: communication; timelines; changes & amendments; future-proofing applications; and, the availability of training and support. Data providers either provided additional evidence on these learning needs or ways to address identified challenges. Six modules were developed addressing these training needs. Quotes from the survey and focus groups are used anonymously in the online training modules.

CONCLUSION

The CENTRIC online training curriculum was launched in September 2020 and is available, free of charge for UK researchers. CENTRIC specifically addresses commonly identified training needs of researchers working with administrative data.

Early positive approaches to support for families of young children with intellectual disability: the E-PAtS feasibility RCT

Background

Parents of children with intellectual disability are 1.5–2 times more likely than other parents to report mental health difficulties. There is a lack of clinically effective and cost-effective group well-being interventions designed for family carers of young children with intellectual disability.

Aim

To examine the feasibility of a randomised controlled trial of the clinical effectiveness and cost-effectiveness of the Early Positive Approaches to Support (E-PAtS) intervention.

Design

A feasibility study (including randomisation of families into a two-arm trial), questionnaires to assess the feasibility of proposed outcome measures (including resource use and health-related quality of life) and practitioner/family carer interviews. An additional question was included in an online UK survey of families, conducted by the research team to assess usual practice, and a survey of provider organisations.

Setting

Families recruited from community contexts (i.e. third sector, local authority services, special schools) and self-referral. The E-PAtS intervention was delivered by trained community-based providers.

Participants

Families with at least one child aged 1.5–5 years with an intellectual disability. At least one parent had to have English-language ability (spoken) for E-PAtS programme participation and participants had to provide informed consent.

Interventions

E-PAtS intervention – two caregivers from each family invited to eight 2.5-hour group sessions with usual practice. Usual practice – other support provided to the family, including other parenting support.

Objectives

To assess randomisation willingness/feasibility, recruitment of providers/parents, retention, usual practice, adherence, fidelity and feasibility of proposed outcome measures (including the Warwick–Edinburgh Mental Well-Being Scale as the proposed primary outcome measure, and parent anxiety/depression, parenting, family functioning/relationships, child behavioural/emotional problems and adaptive skills, child and parent quality of life, and family services receipt as the proposed secondary outcome measures).

Results

Seventy-four families (95 carers) were recruited from three sites (with 37 families allocated to the intervention). From referrals, the recruitment rate was 65% (95% confidence interval 56% to 74%). Seventy-two per cent of families were retained at the 12-month follow-up (95% confidence interval 60% to 81%). Exploratory regression analysis showed that the mean Warwick–Edinburgh Mental Well-Being Scale well-being score was 3.96 points higher in the intervention group (95% confidence interval –1.39 to 9.32 points) at 12 months post randomisation. High levels of data completeness were achieved on returned questionnaires. Interviews (n = 25) confirmed that (1) recruitment, randomisation processes and the intervention were acceptable to family carers, E-PAtS facilitators and community staff; (2) E-PAtS delivery were consistent with the logic model; and (3) researchers requesting consent in future for routine data would be acceptable. Recorded E-PAtS sessions demonstrated good fidelity (96% of components present). Adherence (i.e. at least one carer from the family attending five out of eight E-PAtS sessions) was 76%. Health-related quality-of-life and services receipt data were gathered successfully. An online UK survey to assess usual practice (n = 673) showed that 10% of families of young children with intellectual disability received any intervention over 12 months. A provider survey (n = 15) indicated willingness to take part in future research.

Limitations

Obtaining session recordings for fidelity was difficult. Recruitment processes need to be reviewed to improve diversity and strategies are needed to improve primary outcome completion.

Conclusions

Study processes were feasible. The E-PAtS intervention was well received and outcomes for families were positive. A barrier to future organisation participation is funding for intervention costs. A definitive trial to test the clinical effectiveness and cost-effectiveness of E-PAtS would be feasible.

Trial registration

Current Controlled Trials ISRCTN70419473.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 10, No. 2. See the NIHR Journals Library website for further project information.

Towards nationally curated data archives for clinical radiology image analysis at scale: Learnings from national data collection in response to a pandemic

The prevalence of the coronavirus SARS-CoV-2 disease has resulted in the unprecedented collection of health data to support research. Historically, coordinating the collation of such datasets on a national scale has been challenging to execute for several reasons, including issues with data privacy, the lack of data reporting standards, interoperable technologies, and distribution methods. The coronavirus SARS-CoV-2 disease pandemic has highlighted the importance of collaboration between government bodies, healthcare institutions, academic researchers and commercial companies in overcoming these issues during times of urgency. The National COVID-19 Chest Imaging Database, led by NHSX, British Society of Thoracic Imaging, Royal Surrey NHS Foundation Trust and Faculty, is an example of such a national initiative. Here, we summarise the experiences and challenges of setting up the National COVID-19 Chest Imaging Database, and the implications for future ambitions of national data curation in medical imaging to advance the safe adoption of artificial intelligence in healthcare.

The road to hell is paved with good intentions: the experience of applying for national data for linkage and suggestions for improvement

BACKGROUND

We can improve healthcare services by better understanding current provision. One way to understand this is by linking data sets from clinical and national audits, national registries and other National Health Service (NHS) encounter data. However, getting to the point of having linked national data sets is challenging.

OBJECTIVE

We describe our experience of the data application and linkage process for our study 'LAUNCHES QI', and the time, processes and resource requirements involved. To help others planning similar projects, we highlight challenges encountered and advice for applications in the current system as well as suggestions for system improvements.

FINDINGS

The study set up for LAUNCHES QI began in March 2018, and the process through to data acquisition took 2.5 years. Several challenges were encountered, including the amount of information required (often duplicate information in different formats across applications), lack of clarity on processes, resource constraints that limit an audit's capacity to fulfil requests and the unexpected amount of time required from the study team. It is incredibly difficult to estimate the resources needed ahead of time, and yet necessary to do so as early on as funding applications. Early decisions can have a significant impact during latter stages and be hard to change, yet it is difficult to get specific information at the beginning of the process.

CONCLUSIONS

The current system is incredibly complex, arduous and slow, stifling innovation and delaying scientific progress. NHS data can inform and improve health services and we believe there is an ethical responsibility to use it to do so. Streamlining the number of applications required for accessing data for health services research and providing clarity to data controllers could facilitate the maintenance of stringent governance, while accelerating scientific studies and progress, leading to swifter application of findings and improvements in healthcare.

A study protocol for the development of a SPIRIT extension for trials conducted using cohorts and routinely collected data (SPIRIT-ROUTINE)

Background: Protocols are an essential document for conducting randomised controlled trials (RCTs). However, the completeness of the information provided is often inadequate. To help improve the content of trial protocols, an international group of stakeholders published the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Initiative in 2013. Presently, there is increasing use of cohorts and routinely collected data (RCD) for RCTs because these data have the potential to improve efficiencies by facilitating recruitment, simplifying, and reducing the cost of data collection. Reporting guidelines have been shown to improve the quality of reporting, but there is currently no specific SPIRIT guidance on protocols for trials conducted using cohorts and RCD. This protocol outlines steps for developing SPIRIT-ROUTINE, which aims to address this gap by extending the SPIRIT guidance to protocols for trials conducted using cohorts and RCD.  Methods: The development of the SPIRIT-ROUTINE extension comprises five stages. Stage 1 consists of a project launch and a meeting to finalise the membership of the steering group and scope of the extension. In Stage 2, a rapid review will be performed to identify possible modifications to the original SPIRIT 2013 checklist. Other key reporting guidelines will be reviewed to identify areas where additional items may be needed, such as the Consolidated Standards of Reporting Trials (CONSORT) extension for trials conducted using cohorts and RCD (CONSORT-ROUTINE). Stage 3 will involve an online Delphi exercise, consisting of two rounds and involving key international stakeholders to gather feedback on the preliminary checklist items. In Stage 4, a consensus meeting of the SPIRIT-ROUTINE steering group will finalise the items to include in the extension. Stage 5 will involve the publication preparation and dissemination of the final checklist.  Conclusion: The SPIRIT-ROUTINE extension will contribute to improving design of trials using cohorts and RCD and transparency of reporting.

Accessing routinely collected health data to improve clinical trials: recent experience of access

BACKGROUND

Routinely collected electronic health records (EHRs) have the potential to enhance randomised controlled trials (RCTs) by facilitating recruitment and follow-up. Despite this, current EHR use is minimal in UK RCTs, in part due to ongoing concerns about the utility (reliability, completeness, accuracy) and accessibility of the data. The aim of this manuscript is to document the process, timelines and challenges of the application process to help improve the service both for the applicants and data holders.

METHODS

This is a qualitative paper providing a descriptive narrative from one UK clinical trials unit (MRC CTU at UCL) on the experience of two trial teams' application process to access data from three large English national datasets: National Cancer Registration and Analysis Service (NCRAS), National Institute for Cardiovascular Outcomes Research (NICOR) and NHS Digital to establish themes for discussion. The underpinning reason for applying for the data was to compare EHRs with data collected through case report forms in two RCTs, Add-Aspirin (ISRCTN 74358648) and PATCH (ISRCTN 70406718).

RESULTS

The Add-Aspirin trial, which had a pre-planned embedded sub-study to assess EHR, received data from NCRAS 13 months after the first application. In the PATCH trial, the decision to request data was made whilst the trial was recruiting. The study received data after 8 months from NICOR and 15 months for NHS Digital following final application submission. This concluded in May 2020. Prior to application submission, significant time and effort was needed particularly in relation to the PATCH trial where negotiations over consent and data linkage took many years.

CONCLUSIONS

Our experience demonstrates that data access can be a prolonged and complex process. This is compounded if multiple data sources are required for the same project. This needs to be factored in when planning to use EHR within RCTs and is best considered prior to conception of the trial. Data holders and researchers are endeavouring to simplify and streamline the application process so that the potential of EHR can be realised for clinical trials.

The Family Nurse Partnership to reduce maltreatment and improve child health and development in young children: the BB:2–6 routine data-linkage follow-up to earlier RCT

Background

The short-term effectiveness (to 24 months post partum) of a preventative home-visiting intervention, the Family Nurse Partnership, was previously assessed in the Building Blocks trial (BB:0–2).

Objectives

The objectives were to establish the medium-term effectiveness of the Family Nurse Partnership in reducing maltreatment and improving maternal health (second pregnancies) and child health, developmental and educational outcomes (e.g. early educational attendance, school readiness); to explore effect moderators and mediators; and to describe the costs of enhancing usually provided health and social care with the Family Nurse Partnership.

Design

Children and their mothers from an existing trial cohort were followed up using routine data until the child was 7 years of age.

Setting

This study was set in 18 partnerships between local authorities and health-care organisations in England.

Participants

The participants were mothers [and their firstborn child(ren)] recruited as pregnant women aged ≤ 19 years, in local authority Family Nurse Partnership catchment areas, at < 25 weeks’ gestation, able to provide consent and able to converse in English. Participants mandatorily withdrawn (e.g. owing to miscarriage) from the BB:0–2 trial were excluded.

Interventions

The intervention comprised up to a maximum of 64 home visits by specially trained family nurses from early pregnancy until the firstborn child was 2 years of age, plus usually provided health and social care support. The comparator was usual care alone.

Main outcome measures

The primary outcome measure was child-in-need status recorded at any time during follow-up. The secondary outcomes were as follows: (1) referral to social services, child protection registration (plan), child-in-need categorisation, looked-after status, recorded injuries and ingestions at any time during follow-up; (2) early child care and educational attendance, school readiness (Early Years Foundation Stage Profile score) and attainment at Key Stage 1; and (3) health-care costs.

Data sources

The following data sources were used: maternally reported baseline and follow-up data (BB:0–2), Hospital Episode Statistics data (NHS Digital), social care and educational data (National Pupil Database) and abortions data (Department of Health and Social Care).

Results

There were no differences between study arms in the rates of referral to social services, being registered as a child in need, receiving child protection plans, entering care or timing of first referral for children subsequently assessed as in need. There were no differences between study arms in rates of hospital emergency attendance, admission for injuries or ingestions, or in duration of stay for admitted children. Children in the Family Nurse Partnership arm were more likely to achieve a good level of development at reception age (school readiness), an effect strengthened when adjusting for birth month. Differences at Key Stage 1 were not statistically different, but, after adjusting for birth month, children in the Family Nurse Partnership arm were more likely to reach the expected standard in reading. Programme effects were greater for boys (Key Stage 1: writing); children of younger mothers (Key Stage 1: writing, Key Stage 1: mathematics); and children of mothers not in employment, education or training at study baseline (Key Stage 1: writing). There were no differences between families who were part of the Family Nurse Partnership and those who were not for any other outcome. The differences between study arms in resource use and costs were negligible.

Limitations

The outcomes are constrained to those available from routine sources.

Conclusions

There is no observable benefit of the programme for maltreatment or maternal outcomes, but it does generate advantages in school readiness and attainment at Key Stage 1.

Future work

The trajectory of longer-term programme benefits should be mapped using routine and participant-reported measures.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 9, No. 2. See the NIHR Journals Library website for further project information.

Reliability of Supervised Machine Learning Using Synthetic Data in Health Care: Model to Preserve Privacy for Data Sharing

BACKGROUND

The exploitation of synthetic data in health care is at an early stage. Synthetic data could unlock the potential within health care datasets that are too sensitive for release. Several synthetic data generators have been developed to date; however, studies evaluating their efficacy and generalizability are scarce.

OBJECTIVE

This work sets out to understand the difference in performance of supervised machine learning models trained on synthetic data compared with those trained on real data.

METHODS

A total of 19 open health datasets were selected for experimental work. Synthetic data were generated using three synthetic data generators that apply classification and regression trees, parametric, and Bayesian network approaches. Real and synthetic data were used (separately) to train five supervised machine learning models: stochastic gradient descent, decision tree, k-nearest neighbors, random forest, and support vector machine. Models were tested only on real data to determine whether a model developed by training on synthetic data can used to accurately classify new, real examples. The impact of statistical disclosure control on model performance was also assessed.

RESULTS

A total of 92% of models trained on synthetic data have lower accuracy than those trained on real data. Tree-based models trained on synthetic data have deviations in accuracy from models trained on real data of 0.177 (18%) to 0.193 (19%), while other models have lower deviations of 0.058 (6%) to 0.072 (7%). The winning classifier when trained and tested on real data versus models trained on synthetic data and tested on real data is the same in 26% (5/19) of cases for classification and regression tree and parametric synthetic data and in 21% (4/19) of cases for Bayesian network-generated synthetic data. Tree-based models perform best with real data and are the winning classifier in 95% (18/19) of cases. This is not the case for models trained on synthetic data. When tree-based models are not considered, the winning classifier for real and synthetic data is matched in 74% (14/19), 53% (10/19), and 68% (13/19) of cases for classification and regression tree, parametric, and Bayesian network synthetic data, respectively. Statistical disclosure control methods did not have a notable impact on data utility.

CONCLUSIONS

The results of this study are promising with small decreases in accuracy observed in models trained with synthetic data compared with models trained with real data, where both are tested on real data. Such deviations are expected and manageable. Tree-based classifiers have some sensitivity to synthetic data, and the underlying cause requires further investigation. This study highlights the potential of synthetic data and the need for further evaluation of their robustness. Synthetic data must ensure individual privacy and data utility are preserved in order to instill confidence in health care departments when using such data to inform policy decision-making.

Depression and Anxiety Symptoms of British Adoptive Parents: A Prospective Four-Wave Longitudinal Study

The mental health of birth parents has gained attention due to the serious negative consequences for personal, family, and child outcomes, but depression and anxiety in adoptive parents remains under-recognized. Using a prospective, longitudinal design, we investigated anxiety and depression symptoms in 96 British adoptive parents over four time points in the first four years of an adoptive placement. Depression and anxiety symptom scores were relatively stable across time. Growth curve analysis showed that higher child internalizing scores and lower parental sense of competency at five months post-placement were associated with higher initial levels of parental depressive symptoms. Lower parental sense of competency was also associated with higher initial levels of parental anxiety symptoms. Parents of older children and those with higher levels of parental anxiety and sense of competency at five months post-placement had a steeper decrease in depressive symptoms over time. Support for adoptive families primarily focuses on child adjustment. Our findings suggest that professional awareness of parental mental health post-placement may be necessary, and interventions aimed at improving parents' sense of competency may be beneficial.

Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future

BACKGROUND

Randomised controlled trials (RCTs) are generally regarded as the "gold standard" for providing quantifiable evidence around the effectiveness and cost-effectiveness of new healthcare technologies. In order to perform the economic evaluations associated with RCTs, there is a need for accessible and good quality resource-use data; for the purpose of discussion here, data that best reflect the care received. Traditionally, researchers have developed questionnaires for resource-use data collection. However, the evolution of routinely collected electronic data within care services provides new opportunities for collecting data without burdening patients or caregivers (e.g. clinicians). This paper describes the potential strengths and limitations of each data collection method and then discusses aspects for consideration before choosing which method to use.

MAIN TEXT

We describe electronic data sources (large observational datasets, commissioning data, and raw data extraction) that may be suitable data sources for informing clinical trials and the current status of self-reported instruments for measuring resource-use. We assess the methodological risks and benefits, and compare the two methodologies. We focus on healthcare resource-use; however, many of the considerations have relevance to clinical questions. Patient self-report forms a pragmatic and cheap method that is largely under the control of the researcher. However, there are known issues with the validity of the data collected, loss to follow-up may be high, and questionnaires suffer from missing data. Routinely collected electronic data may be more accurate and more practical if large numbers of patients are involved. However, datasets often incur a cost and researchers are bound by the time for data approval and extraction by the data holders.

CONCLUSIONS

Owing to the issues associated with electronic datasets, self-reported methods may currently be the preferred option. However, electronic hospital data are relatively more accessible, informative, standardised, and reliable. Therefore in trials where secondary care constitutes a major driver of patient care, detailed electronic data may be considered superior to self-reported methods; with the caveat of requiring data sharing agreements with third party providers and potentially time-consuming extraction periods. Self-reported methods will still be required when a 'societal' perspective (e.g. quantifying informal care) is desirable for the intended analysis.