Defining dimensions of research readiness: a conceptual model for primary care research networks

A "Patient Preference" Model of Recruitment for Research from Primary-Care-Based Memory Clinics: A Promising New Approach

Recruiting persons with dementia for clinical trials can be challenging. Building on a guide initially developed to assist primary-care-based memory clinics in their efforts to support research, a key stakeholder working group meeting was held to develop a standardized research recruitment process, with input from patients, care partners, researchers, and clinicians. Discussions in this half-day facilitated meeting focused on the wishes and needs of patients and care partners, policy and procedures for researchers, information provided to patients, and considerations for memory clinics. Patients and care partners valued the opportunity to contribute to science and provided important insights on how to best facilitate recruitment. Discussions regarding proposed processes and procedures for research recruitment highlighted the need for a new, patient-driven approach. Accordingly, a key stakeholder co-designed "Memory Clinic Research Match" program was developed that has the potential to overcome existing barriers and to increase recruitment for dementia-related research.

System and facility readiness assessment for conducting active surveillance of adverse events following immunization in Addis Ababa, Ethiopia

BACKGROUND

To help distinguish vaccine-related adverse events following immunization (AEFI) from coincidental occurrences, active vaccine pharmacovigilance (VP) prospective surveillance programs are needed. From February to May 2021, we assessed the system and facility readiness for implementing active AEFI VP surveillance in Addis Ababa, Ethiopia.

METHODS

Selected hospitals were assessed using a readiness assessment tool with scoring measures. The site assessment was conducted via in-person interviews within the specific departments in each hospital. We evaluated the system readiness with a desk review of AEFI guidelines, Expanded Program for Immunization Guidelines and Ethiopian Food and Drug Administration and Ethiopian Public Health Institute websites.

RESULTS

Of the hospitals in Addis Ababa, 23.1% met the criteria for our site assessment. During the system readiness assessment, we found that essential components were in place. However, rules, regulations and proclamations pertaining to AEFI surveillance were absent. Based on the tool, the three hospitals (A, B and C) scored 60.6% (94/155), 48.3% (75/155) and 40% (62/155), respectively.

CONCLUSIONS

Only one of three hospitals assessed in our evaluation scored >50% for readiness to implement active AEFI surveillance. We also identified the following areas for improvement to ensure successful implementation: training, making guidelines and reporting forms available and ensuring a system that accommodates paper-based and electronic-based recording systems.

Understanding data provenance when using electronic medical records for research: Lessons learned from the Deliver Primary Healthcare Information (DELPHI) database

Introduction

We set out to assess the impact of Choosing Wisely Canada recommendations (2014) on reducing unnecessary health investigations and interventions in primary care across Southwestern Ontario.

Methods

We used the Deliver Primary Healthcare Information (DELPHI) database, which stores deidentified electronic medical records (EMR) of nearly 65,000 primary care patients across Southwestern Ontario. When conducting research using EMR data, data provenance (i.e., how the data came to be) should first be established. We first considered DELPHI data provenance in relation to longitudinal analyses, flagging a change in EMR software that occurred during 2012 and 2013. We attempted to link records between EMR databases produced by different software using probabilistic linkage and inspected 10 years of data in the DELPHI database (2009 to 2019) for data quality issues, including comparability over time.

Results

We encountered several issues resulting from this change in EMR software. These included limited linkage of records between software without a common identifier; data migration issues that distorted procedure dates; and unusual changes in laboratory test and medication prescription volumes.

Conclusion

This study reinforces the necessity of assessing data provenance and quality for new research projects. By understanding data provenance, we can anticipate related data quality issues such as changes in EMR data over time-which represent a growing concern as longitudinal data analyses increase in feasibility and popularity.

What works to recruit general practices to trials? A rapid review

Background: Recruitment challenges are a barrier to the conduct of trials in general practice, yet little is known about which recruitment strategies work best to recruit practices for randomised controlled trials (RCTs). We aimed to describe the types of strategies used to recruit general practices for trials and synthesize any available evidence of effectiveness. Methods: We conducted a rapid evidence review in line with guidance from Tricco et al. Eligible studies reported or evaluated any strategy to improve practice recruitment to participate in clinical or implementation RCTs. PubMed, Embase, and Cochrane Central Library were searched from inception to June 22 nd, 2021. Reference lists of included studies were screened. Data were synthesized narratively. Results: Over 9,162 articles were identified, and 19 studies included. Most (n=13, 66.7%) used a single recruitment strategy. The most common strategies were: in-person practice meetings/visits by the research team (n=12, 63.2%); phone calls (n=10, 52.6%); financial incentives (n=9, 47.4%); personalised emails (n=7, 36.8%) or letters (n=6, 52.6%) (as opposed to email 'blasts' or generic letters); targeting practices that participated in previous studies or with which the team had existing links (n=6, 31.6%) or targeting of practices within an existing practice or research network (n=6, 31.6%).  Three studies reporting recruitment rates >80%, used strategies such as invitation letters with a follow-up phone call to non-responders, presentations by the principal investigator and study coordinator, or in-person meetings with practices with an existing affiliation with the University or research team.  Conclusions: Few studies directly compared recruitment approaches making it difficult to draw conclusions about their comparative effectiveness. However, the role of more personalised letter/email, in-person, or phone contact, and capitalising on existing relationships appears important. Further work is needed to standardise how recruitment methods are reported and to directly compare different recruitment strategies within one study . PROSPERO registration: CRD42021268140 (15/08/2021).

Recruitment of participants for dementia research: interprofessional perspectives from primary care-based memory clinics

Aim: To understand clinician attitudes and the barriers that impede research recruitment from specialized primary care-based memory clinics. Materials & methods: Clinicians completed a survey on attitudes and barriers to research recruitment from memory clinics. Results: Comfort and willingness to recruit for research were low to moderate and were lower for drug trials than for observational and non-drug trials. Respondents believed that it is important to have a standardized recruitment process. Identified barriers provide some insights into the factors that contribute to discomfort and lack of willingness to recruit research participants. Discussion: Findings can inform future efforts to develop a recruitment process that addresses identified barriers, while also providing an opportunity to increase participant recruitment in dementia research.

Tools for assessing the scalability of innovations in health: a systematic review

BACKGROUND

The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health.

METHODS

We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics.

RESULTS

We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%).

CONCLUSIONS

We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools. Trial registration We registered this review with PROSPERO (identifier: CRD42019107095).

A Conceptual Framework of Data Readiness: The Contextual Intersection of Quality, Availability, Interoperability, and Provenance

BACKGROUND

Data readiness is a concept often used when referring to health information technology applications in the informatics disciplines, but it is not clearly defined in the literature. To avoid misinterpretations in research and implementation, a formal definition should be developed.

OBJECTIVES

The objective of this research is to provide a conceptual definition and framework for the term data readiness that can be used to guide research and development related to data-based applications in health care.

METHODS

PubMed, the National Institutes of Health RePORTER, Scopus, the Cochrane Library, and Duke University Library databases for business and information sciences were queried for formal mentions of the term "data readiness." Manuscripts found in the search were reviewed, and relevant information was extracted, evaluated, and assimilated into a framework for data readiness.

RESULTS

Of the 264 manuscripts found in the database searches, 20 were included in the final synthesis to define data readiness. In these 20 manuscripts, the term data readiness was revealed to encompass the constructs of data quality, data availability, interoperability, and data provenance.

DISCUSSION

Based upon our review of the literature, we define data readiness as the application-specific intersection of data quality, data availability, interoperability, and data provenance. While these concepts are not new, the combination of these factors in a novel data readiness model may help guide future informatics research and implementation science.

CONCLUSION

This analysis provides a definition to guide research and development related to data-based applications in health care. Future work should be done to validate this definition, and to apply the components of data readiness to real-world applications so that specific metrics may be developed and disseminated.

Developing and testing a readiness tool for interpersonal violence prevention partnerships with community-based projects

Community-based projects that serve vulnerable families have the opportunity to identify and respond to interpersonal violence (IPV). We developed a readiness assessment tool to support selection of projects to participate in an initiative that involved implementing a community-based IPV intervention for mothers. The overarching aim of the current study was to describe the development of this tool and examine the reliability of coding, validity, and utility of the tool. After developing and refining the tool, 41 community-based projects completed the tool. Responses were coded and scored; scores were used to select projects for the initiative. Preliminary validation for the tool included (a) expert opinion, (b) uptake/implementation of the intervention, and (c) feedback and responses from service providers in terms of the usefulness and importance of the tool. This tool can be used by both researchers and service providers to assess community project readiness and capacity to provide trauma-informed services for vulnerable families.

Development and evaluation of an osteoarthritis risk model for integration into primary care health information technology

BACKGROUND

We developed and evaluated a prognostic prediction model that estimates osteoarthritis risk for use by patients and practitioners that is designed to be appropriate for integration into primary care health information technology systems. Osteoarthritis, a joint disorder characterized by pain and stiffness, causes significant morbidity among older Canadians. Because our prognostic prediction model for osteoarthritis risk uses data that are readily available in primary care settings, it supports targeting of interventions delivered as part of clinical practice that are aimed at risk reduction.

METHODS

We used the CPCSSN (Canadian Primary Sentinel Surveillance Network) database, which contains aggregated electronic health information from a cohort of primary care practices, to develop and evaluate a prognostic prediction model to estimate 5-year osteoarthritis risk, addressing contextual challenges of data availability and missingness. We constructed a retrospective cohort of 383,117 eligible primary care patients who were included in the cohort if they had an encounter with their primary care practitioner between 1 January 2009 and 31 December 2010. Patients were excluded if they had a diagnosis of osteoarthritis prior to their first visit in this time period. Incident cases of osteoarthritis were observed. The model was constructed to predict incident osteoarthritis based on age, sex, BMI, previous leg injury, and osteoporosis. Evaluation of the model used internal 10-fold cross-validation; we argue that internal validation is particularly appropriate for a model that is to be integrated into the same context from which the data were derived.

RESULTS

The resulting prediction model for 5-year risk of osteoarthritis diagnosis demonstrated state-of-the-art discrimination (estimated AUROC 0.84) and good calibration (assessed visually.) The model relies only on information that is readily available in Canadian primary care settings, and hence is appropriate for integration into Canadian primary care health information technology.

CONCLUSIONS

If the contextual challenges arising when using primary care electronic medical record data are appropriately addressed, highly discriminative models for osteoarthritis risk may be constructed using only data commonly available in primary care. Because the models are constructed from data in the same setting where the model is to be applied, internal validation provides strong evidence that the resulting model will perform well in its intended application.

A basic model for assessing primary health care electronic medical record data quality

Background

The increased use of electronic medical records (EMRs) in Canadian primary health care practice has resulted in an expansion of the availability of EMR data. Potential users of these data need to understand their quality in relation to the uses to which they are applied. Herein, we propose a basic model for assessing primary health care EMR data quality, comprising a set of data quality measures within four domains. We describe the process of developing and testing this set of measures, share the results of applying these measures in three EMR-derived datasets, and discuss what this reveals about the measures and EMR data quality. The model is offered as a starting point from which data users can refine their own approach, based on their own needs.

Methods

Using an iterative process, measures of EMR data quality were created within four domains: comparability; completeness; correctness; and currency. We used a series of process steps to develop the measures. The measures were then operationalized, and tested within three datasets created from different EMR software products.

Results

A set of eleven final measures were created. We were not able to calculate results for several measures in one dataset because of the way the data were collected in that specific EMR. Overall, we found variability in the results of testing the measures (e.g. sensitivity values were highest for diabetes, and lowest for obesity), among datasets (e.g. recording of height), and by patient age and sex (e.g. recording of blood pressure, height and weight).

Conclusions

This paper proposes a basic model for assessing primary health care EMR data quality. We developed and tested multiple measures of data quality, within four domains, in three different EMR-derived primary health care datasets. The results of testing these measures indicated that not all measures could be utilized in all datasets, and illustrated variability in data quality. This is one step forward in creating a standard set of measures of data quality. Nonetheless, each project has unique challenges, and therefore requires its own data quality assessment before proceeding.

Electronic supplementary material

The online version of this article (10.1186/s12911-019-0740-0) contains supplementary material, which is available to authorized users.