Standardizing clinical trials workflow representation in UML for international site comparison

Transforming evidence-based clinical guidelines into implementable clinical decision support services: the CAREPATH study for multimorbidity management

Introduction

The CAREPATH Project aims to develop a patient-centered integrated care platform tailored to older adults with multimorbidity, including mild cognitive impairment (MCI) or mild dementia. Our goal is to empower multidisciplinary care teams to craft personalized holistic care plans while adhering to evidence-based guidelines. This necessitates the creation of clear specifications for clinical decision support (CDS) services, consolidating guidance from multiple evidence-based clinical guidelines. Thus, a co-creation approach involving both clinical and technical experts is essential.

Methods

This paper outlines a robust methodology for generating implementable specifications for CDS services to automate clinical guidelines. We have established a co-creation framework to facilitate collaborative exploration of clinical guidelines between clinical experts and software engineers. We have proposed an open, repeatable, and traceable method for translating evidence-based guideline narratives into implementable specifications of CDS services. Our approach, based on international standards such as CDS-Hooks and HL7 FHIR, enhances interoperability and potential adoption of CDS services across diverse healthcare systems.

Results

This methodology has been followed to create implementable specifications for 65 CDS services, automating CAREPATH consensus guideline consolidating guidance from 25 selected evidence-based guidelines. A total of 296 CDS rules have been formally defined, with input parameters defined as clinical concepts bound to FHIR resources and international code systems. Outputs include 346 well-defined CDS Cards, offering clear guidance for care plan activities and goal suggestions. These specifications have led to the implementation of 65 CDS services integrated into the CAREPATH Adaptive Integrated Care Platform.

Discussion

Our methodology offers a systematic, replicable process for generating CDS specifications, ensuring consistency and reliability across implementation. By fostering collaboration between clinical expertise and technical proficiency, we enhance the quality and relevance of generated specifications. Clear traceability enables stakeholders to track the development process and ensure adherence to guideline recommendations.

A virtual experimenter to increase standardization for the investigation of placebo effects

Background

Placebo effects are mediated by expectancy, which is highly influenced by psychosocial factors of a treatment context. These factors are difficult to standardize. Furthermore, dedicated placebo research often necessitates single-blind deceptive designs where biases are easily introduced. We propose a study protocol employing a virtual experimenter – a computer program designed to deliver treatment and instructions – for the purpose of standardization and reduction of biases when investigating placebo effects.

Methods

To evaluate the virtual experimenter’s efficacy in inducing placebo effects via expectancy manipulation, we suggest a partially blinded, deceptive design with a baseline/retest pain protocol (hand immersions in hot water bath). Between immersions, participants will receive an (actually inert) medication. Instructions pertaining to the medication will be delivered by one of three metaphors: The virtual experimenter, a human experimenter, and an audio/text presentation (predictor “Metaphor”). The second predictor includes falsely informing participants that the medication is an effective pain killer, or correctly informing them that it is, in fact, inert (predictor “Instruction”). Analysis will be performed with hierarchical linear modelling, with a sample size of N = 50. Results from two pilot studies are presented that indicate the viability of the pain protocol (N = 33), and of the virtual experimenter software and placebo manipulation (N = 48).

Discussion

It will be challenging to establish full comparability between all metaphors used for instruction delivery, and to account for participant differences in acceptance of their virtual interaction partner. Once established, the presence of placebo effects would suggest that the virtual experimenter exhibits sufficient cues to be perceived as a social agent. He could consequently provide a convenient platform to investigate effects of experimenter behavior, or other experimenter characteristics, e.g., sex, age, race/ethnicity or professional status. More general applications are possible, for example in psychological research such as bias research, or virtual reality research. Potential applications also exist for standardizing clinical research by documenting and communicating instructions used in clinical trials.

A Design Methodology for Medical Processes

BACKGROUND

Healthcare processes, especially those belonging to the clinical domain, are acknowledged as complex and characterized by the dynamic nature of the diagnosis, the variability of the decisions made by experts driven by their experiences, the local constraints, the patient's needs, the uncertainty of the patient's response, and the indeterminacy of patient's compliance to treatment. Also, the multiple actors involved in patient's care need clear and transparent communication to ensure care coordination.

OBJECTIVES

In this paper, we propose a methodology to model healthcare processes in order to break out complexity and provide transparency.

METHODS

The model is grounded on a set of requirements that make the healthcare domain unique with respect to other knowledge domains. The modeling methodology is based on three main phases: the study of the environmental context, the conceptual modeling, and the logical modeling.

RESULTS

The proposed methodology was validated by applying it to the case study of the rehabilitation process of stroke patients in the specific setting of a specialized rehabilitation center. The resulting model was used to define the specifications of a software artifact for the digital administration and collection of assessment tests that was also implemented.

CONCLUSIONS

Despite being only an example, our case study showed the ability of process modeling to answer the actual needs in healthcare practices. Independently from the medical domain in which the modeling effort is done, the proposed methodology is useful to create high-quality models, and to detect and take into account relevant and tricky situations that can occur during process execution.

Analysis and classification of oncology activities on the way to workflow based single source documentation in clinical information systems

BACKGROUND

Today, cancer documentation is still a tedious task involving many different information systems even within a single institution and it is rarely supported by appropriate documentation workflows.

METHODS

In a comprehensive 14 step analysis we compiled diagnostic and therapeutic pathways for 13 cancer entities using a mixed approach of document analysis, workflow analysis, expert interviews, workflow modelling and feedback loops. These pathways were stepwise classified and categorized to create a final set of grouped pathways and workflows including electronic documentation forms.

RESULTS

A total of 73 workflows for the 13 entities based on 82 paper documentation forms additionally to computer based documentation systems were compiled in a 724 page document comprising 130 figures, 94 tables and 23 tumour classifications as well as 12 follow-up tables. Stepwise classification made it possible to derive grouped diagnostic and therapeutic pathways for the three major classes - solid entities with surgical therapy - solid entities with surgical and additional therapeutic activities and - non-solid entities. For these classes it was possible to deduct common documentation workflows to support workflow-guided single-source documentation.

CONCLUSIONS

Clinical documentation activities within a Comprehensive Cancer Center can likely be realized in a set of three documentation workflows with conditional branching in a modern workflow supporting clinical information system.

Extension of the primary care research object model (PCROM) as clinical research information model (CRIM) for the "learning healthcare system"

Background

Patient data from general practices is already used for many types of epidemiological research and increasingly, primary care systems to facilitate randomized clinical trials. The EU funded project TRANSFoRm aims to create a “Learning Healthcare System” at a European level that is able to support all types of research using primary care data, to recruit patients and follow patients in clinical studies and to improve diagnosis and therapy. The implementation of such a Learning Healthcare System needs an information model for clinical research (CRIM), as an informational backbone to integrate aspects of primary care with clinical trials and database searches.

Methods

Workflow descriptions and corresponding data objects of two clinical use cases (Gastro-Oesophageal Reflux Disease and Type 2 Diabetes) were described in UML activity diagrams. The components of activity diagrams were mapped to information objects of PCROM (Primary Care Research Object Model) and BRIDG (Biomedical Research Integrated Domain Group) and evaluated. The class diagram of PCROM was adapted to comply with workflow descriptions.

Results

The suitability of PCROM, a primary care information model already used for clinical trials, to act as an information model for TRANSFoRm was evaluated and resulted in its extension with 14 new information object types, two extensions of existing objects and the introduction of two new high-ranking concepts (CARE area and ENTRY area). No PCROM component was redundant. Our result illustrates that in primary care based research an important but underestimated portion of research activity takes place in the area of care (e.g. patient consultation, screening, recruitment and response to adverse events). The newly introduced CARE area for care-related research activities accounts for this shift and includes Episode of Care and Encounter as two new basic elements. In the ENTRY area different aspects of data collection were combined, including data semantics for observations, assessment activities, intervention activities and patient reporting to enable case report form (CRF) based data collection combined with decision support.

Conclusions

Research with primary care data needs an extended information model that covers research activities at the care site which are characteristic for primary care based research and the requirements of the complicated data collection processes.

Knowledge translation: a case study on pneumonia research and clinical guidelines in a low- income country

BACKGROUND

The process and effectiveness of knowledge translation (KT) interventions targeting policymakers are rarely reported. In Cambodia, a low-income country (LIC), an intervention aiming to provide evidence-based knowledge on pneumonia to health authorities was developed to help update pediatric and adult national clinical guidelines. Through a case study, we assessed the effectiveness of this KT intervention, with the goal of identifying the barriers to KT and suggest strategies to facilitate KT in similar settings.

METHODS

An extensive search for all relevant sources of data documenting the processes of updating adult and pediatric pneumonia guidelines was done. Documents included among others, reports, meeting minutes, and email correspondences. The study was conducted in successive phases: an appraisal of the content of both adult and pediatric pneumonia guidelines; an appraisal of the quality of guidelines by independent experts, using the AGREE-II instrument; a description and modeling of the KT process within the guidelines updating system, using the Unified Modeling Language (UML) tools 2.2; and the listing of the barriers and facilitators to KT we identified during the study.

RESULTS

The first appraisal showed that the integration of the KT key messages in pediatric and adult guidelines varied with a better efficiency in the pediatric guidelines. The overall AGREE-II quality assessments scored 37% and 44% for adult and pediatric guidelines, respectively. Scores were lowest for the domains of 'rigor of development' and 'editorial independence.' The UML analysis highlighted that time frames and constraints of the involved stakeholders greatly differed and that there were several missed opportunities to translate on evidence into the adult pneumonia guideline. Seventeen facilitating factors and 18 potential barriers to KT were identified. Main barriers were related to the absence of a clear mandate from the Ministry of Health for the researchers and to a lack of synchronization between knowledge production and policy-making.

CONCLUSIONS

Study findings suggest that stakeholders, both researchers and policy makers planning to update clinical guidelines in LIC may need methodological support to overcome the expected barriers.

Time motion studies in healthcare: what are we talking about?

Time motion studies were first described in the early 20th century in industrial engineering, referring to a quantitative data collection method where an external observer captured detailed data on the duration and movements required to accomplish a specific task, coupled with an analysis focused on improving efficiency. Since then, they have been broadly adopted by biomedical researchers and have become a focus of attention due to the current interest in clinical workflow related factors. However, attempts to aggregate results from these studies have been difficult, resulting from a significant variability in the implementation and reporting of methods. While efforts have been made to standardize the reporting of such data and findings, a lack of common understanding on what "time motion studies" are remains, which not only hinders reviews, but could also partially explain the methodological variability in the domain literature (duration of the observations, number of tasks, multitasking, training rigor and reliability assessments) caused by an attempt to cluster dissimilar sub-techniques. A crucial milestone towards the standardization and validation of time motion studies corresponds to a common understanding, accompanied by a proper recognition of the distinct techniques it encompasses. Towards this goal, we conducted a review of the literature aiming at identifying what is being referred to as "time motion studies". We provide a detailed description of the distinct methods used in articles referenced or classified as "time motion studies", and conclude that currently it is used not only to define the original technique, but also to describe a broad spectrum of studies whose only common factor is the capture and/or analysis of the duration of one or more events. To maintain alignment with the existing broad scope of the term, we propose a disambiguation approach by preserving the expanded conception, while recommending the use of a specific qualifier "continuous observation time motion studies" to refer to variations of the original method (the use of an external observer recording data continuously). In addition, we present a more granular naming for sub-techniques within continuous observation time motion studies, expecting to reduce the methodological variability within each sub-technique and facilitate future results aggregation.

Modeling of outpatient prescribing process in iran: a gateway toward electronic prescribing system

Implementation of electronic prescribing system can overcome many problems of the paper prescribing system, and provide numerous opportunities of more effective and advantageous prescribing. Successful implementation of such a system requires complete and deep understanding of work content, human force, and workflow of paper prescribing. The current study was designed in order to model the current business process of outpatient prescribing in Iran and clarify different actions during this process. In order to describe the prescribing process and the system features in Iran, the methodology of business process modeling and analysis was used in the present study. The results of the process documentation were analyzed using a conceptual model of workflow elements and the technique of modeling "As-Is" business processes. Analysis of the current (as-is) prescribing process demonstrated that Iran stood at the first levels of sophistication in graduated levels of electronic prescribing, namely electronic prescription reference, and that there were problematic areas including bottlenecks, redundant and duplicated work, concentration of decision nodes, and communicative weaknesses among stakeholders of the process. Using information technology in some activities of medication prescription in Iran has not eliminated the dependence of the stakeholders on paper-based documents and prescriptions. Therefore, it is necessary to implement proper system programming in order to support change management and solve the problems in the existing prescribing process. To this end, a suitable basis should be provided for reorganization and improvement of the prescribing process for the future electronic systems.

Inter-observer reliability assessments in time motion studies: the foundation for meaningful clinical workflow analysis

Understanding clinical workflow is critical for researchers and healthcare decision makers. Current workflow studies tend to oversimplify and underrepresent the complexity of clinical workflow. Continuous observation time motion studies (TMS) could enhance clinical workflow studies by providing rich quantitative data required for in-depth workflow analyses. However, methodological inconsistencies have been reported in continuous observation TMS, potentially reducing the validity of TMS' data and limiting their contribution to the general state of knowledge. We believe that a cornerstone in standardizing TMS is to ensure the reliability of the human observers. In this manuscript we review the approaches for inter-observer reliability assessment (IORA) in a representative sample of TMS focusing on clinical workflow. We found that IORA is an uncommon practice, inconsistently reported, and often uses methods that provide partial and overestimated measures of agreement. Since a comprehensive approach to IORA is yet to be proposed and validated, we provide initial recommendations for IORA reporting in continuous observation TMS.

Detecting dissonance in clinical and research workflow for translational psychiatric registries

BACKGROUND

The interplay between the workflow for clinical tasks and research data collection is often overlooked, ultimately making it ineffective.

QUESTIONS/PURPOSES

To the best of our knowledge, no previous studies have developed standards that allow for the comparison of workflow models derived from clinical and research tasks toward the improvement of data collection processes.

METHODS

In this study we used the term dissonance for the occurrences where there was a discord between clinical and research workflows. We developed workflow models for a translational research study in psychiatry and the clinic where its data collection was carried out. After identifying points of dissonance between clinical and research models we derived a corresponding classification system that ultimately enabled us to re-engineer the data collection workflow. We considered (1) the number of patients approached for enrollment and (2) the number of patients enrolled in the study as indicators of efficiency in research workflow. We also recorded the number of dissonances before and after the workflow modification.

RESULTS

We identified 22 episodes of dissonance across 6 dissonance categories: actor, communication, information, artifact, time, and space. We were able to eliminate 18 episodes of dissonance and increase the number of patients approached and enrolled in research study trough workflow modification.

CONCLUSION

The classification developed in this study is useful for guiding the identification of dissonances and reveal modifications required to align the workflow of data collection and the clinical setting. The methodology described in this study can be used by researchers to standardize data collection process.

Glocal clinical registries: pacemaker registry design and implementation for global and local integration--methodology and case study

Background

The ability to apply standard and interoperable solutions for implementing and managing medical registries as well as aggregate, reproduce, and access data sets from legacy formats and platforms to advanced standard formats and operating systems are crucial for both clinical healthcare and biomedical research settings.

Purpose

Our study describes a reproducible, highly scalable, standard framework for a device registry implementation addressing both local data quality components and global linking problems.

Methods and Results

We developed a device registry framework involving the following steps: (1) Data standards definition and representation of the research workflow, (2) Development of electronic case report forms using REDCap (Research Electronic Data Capture), (3) Data collection according to the clinical research workflow and, (4) Data augmentation by enriching the registry database with local electronic health records, governmental database and linked open data collections, (5) Data quality control and (6) Data dissemination through the registry Web site. Our registry adopted all applicable standardized data elements proposed by American College Cardiology / American Heart Association Clinical Data Standards, as well as variables derived from cardiac devices randomized trials and Clinical Data Interchange Standards Consortium. Local interoperability was performed between REDCap and data derived from Electronic Health Record system. The original data set was also augmented by incorporating the reimbursed values paid by the Brazilian government during a hospitalization for pacemaker implantation. By linking our registry to the open data collection repository Linked Clinical Trials (LinkedCT) we found 130 clinical trials which are potentially correlated with our pacemaker registry.

Conclusion

This study demonstrates how standard and reproducible solutions can be applied in the implementation of medical registries to constitute a re-usable framework. Such approach has the potential to facilitate data integration between healthcare and research settings, also being a useful framework to be used in other biomedical registries.

A formal representation for numerical data presented in published clinical trial reports

Assessing the quality of and integrating clinical trial reports are necessary to practice evidence-based medicine. In particular, the numerical data is essential to understanding the strength and quality of the clinical trial study. In this paper, we present a formal representation for standardizing numerical data in published clinical trial reports, and our efforts towards developing computational tools to capture and visualize this representation. The approach includes two aspects: a process model used to precisely define experimental context behind the numerical value; and a spreadsheet, an intuitive and familiar tool used to organize numerical data. We demonstrated this representation using clinical trial reports on non-small cell lung cancer (NSCLC). We performed a preliminary evaluation to determine the usefulness of this formalism for identifying the characteristics, quality and significance of a clinical trial. Our initial results demonstrate that the representation is sufficiently expressive to capture reported numerical information in published papers.

Improving the accuracy of therapy descriptions in clinical trials using a bottom-up approach

Randomized clinical trial (RCT) reports commonly have complicated therapy descriptions that are written in free-text. Drug therapy is difficult to describe due to the dynamic nature of how protocols change and the many ways drugs can be administered. Details regarding protocol changes and drug administration must be explained clearly for reproducibility and reliability. A process model supplemented with concept ontologies can clarify the dynamics of how therapies change and make knowledge more explicit. We demonstrated the process to develop a representation model to reveal specific context concerning drug therapies within clinical trial report literature. A PubMed search was conducted to identify RCTs on non-small-cell lung cancer (NSCLC) pertaining to epithelial growth factor receptor (EGFR) mutations. Twenty-seven clinical trials were used to develop the model using a bottom-up approach. This representation describes drug dosage, administration details, and drug cycles within different experimental arms and control groups. We then presented preliminary evaluation of the clarity and understandability of the representation.

Workflow in clinical trial sites & its association with near miss events for data quality: ethnographic, workflow & systems simulation

BACKGROUND

With the exponential expansion of clinical trials conducted in (Brazil, Russia, India, and China) and VISTA (Vietnam, Indonesia, South Africa, Turkey, and Argentina) countries, corresponding gains in cost and enrolment efficiency quickly outpace the consonant metrics in traditional countries in North America and European Union. However, questions still remain regarding the quality of data being collected in these countries. We used ethnographic, mapping and computer simulation studies to identify/address areas of threat to near miss events for data quality in two cancer trial sites in Brazil.

METHODOLOGY/PRINCIPAL FINDINGS

Two sites in Sao Paolo and Rio Janeiro were evaluated using ethnographic observations of workflow during subject enrolment and data collection. Emerging themes related to threats to near miss events for data quality were derived from observations. They were then transformed into workflows using UML-AD and modeled using System Dynamics. 139 tasks were observed and mapped through the ethnographic study. The UML-AD detected four major activities in the workflow evaluation of potential research subjects prior to signature of informed consent, visit to obtain subject́s informed consent, regular data collection sessions following study protocol and closure of study protocol for a given project. Field observations pointed to three major emerging themes: (a) lack of standardized process for data registration at source document, (b) multiplicity of data repositories and (c) scarcity of decision support systems at the point of research intervention. Simulation with policy model demonstrates a reduction of the rework problem.

CONCLUSIONS/SIGNIFICANCE

Patterns of threats to data quality at the two sites were similar to the threats reported in the literature for American sites. The clinical trial site managers need to reorganize staff workflow by using information technology more efficiently, establish new standard procedures and manage professionals to reduce near miss events and save time/cost. Clinical trial sponsors should improve relevant support systems.