Evaluation Considerations for Secondary Uses of Clinical Data: Principles for an Evidence-based Approach to Policy and Implementation of Secondary Analysis

Objectives: To set the scientific context and then suggest principles for an evidence-based approach to secondary uses of clinical data, covering both evaluation of the secondary uses of data and evaluation of health systems and services based upon secondary uses of data. Method: Working Group review of selected literature and policy approaches. Results: We present important considerations in the evaluation of secondary uses of clinical data from the angles of governance and trust, theory, semantics, and policy. We make the case for a multi-level and multi-factorial approach to the evaluation of secondary uses of clinical data and describe a methodological framework for best practice. We emphasise the importance of evaluating the governance of secondary uses of health data in maintaining trust, which is essential for such uses. We also offer examples of the re-use of routine health data to demonstrate how it can support evaluation of clinical performance and optimize health IT system design. Conclusions: Great expectations are resting upon "Big Data" and innovative analytics. However, to build and maintain public trust, improve data reliability, and assure the validity of analytic inferences, there must be independent and transparent evaluation. A mature and evidence-based approach needs not merely data science, but must be guided by the broader concerns of applied health informatics.

Distance-dependent gradient in NMDAR-driven spine calcium signals along tapering dendrites

Neurons receive a multitude of synaptic inputs along their dendritic arbor, but how this highly heterogeneous population of synaptic compartments is spatially organized remains unclear. By measuring N-methyl-d-aspartic acid receptor (NMDAR)-driven calcium responses in single spines, we provide a spatial map of synaptic calcium signals along dendritic arbors of hippocampal neurons and relate this to measures of synapse structure. We find that quantal NMDAR calcium signals increase in amplitude as they approach a thinning dendritic tip end. Based on a compartmental model of spine calcium dynamics, we propose that this biased distribution in calcium signals is governed by a gradual, distance-dependent decline in spine size, which we visualized using serial block-face scanning electron microscopy. Our data describe a cell-autonomous feature of principal neurons, where tapering dendrites show an inverse distribution of spine size and NMDAR-driven calcium signals along dendritic trees, with important implications for synaptic plasticity rules and spine function.

Behavioural adjustments of wild-caught kangaroos to captivity

It is important to understand how wild-caught animals may respond to captivity, and whether their behaviours in captivity are reflective of their wild counterparts. We observed the behaviour of wild-caught western grey kangaroos (Macropus fuliginosus) and red kangaroos (Osphranter rufus; formerly M. rufus) that were transferred to a large naturally vegetated enclosure. Observations were made on the first day of captivity and again after 7–10 days, and were compared with the behaviour of free-living kangaroos at the same locality and over the same period. We quantified feeding, moving, grooming and non-alert behaviours. Of these, grooming was higher in captive kangaroos than in free-living animals on the first day of captivity only, and was no different to that of wild animal by 7–10 days’ captivity. Such self-directed behaviour may be indicative of heightened distress for kangaroos on the first day of captivity, but it may also be indicative of grooming to eliminate contamination of human smells or debris following capture by darting and recovery. Overall, our findings indicate that after a short period of captivity, wild-caught kangaroos adjust to a novel environment relatively quickly, and animals from each species showed behaviour patterns comparable with free-ranging counterparts within 7–10 days after capture and captivity.

Assessing the trend in sustainable remediation: A questionnaire survey of remediation professionals in various countries

Over the past decade, sustainable remediation has grown from an emerging concept into a widely accepted new institutional norm. Scholar literature increased exponentially from nearly none in late 1990s to over 400 publications per year in 2014. The present study used a questionnaire survey conducted in 2012 and 2014 to assess the global trend in the awareness and practice of sustainable remediation. A total of 373 responses were received from survey participants located in 22 countries. The survey found that the US and the UK similarly had the highest level of awareness and adoption rate of sustainable remediation. Asia and other developing countries had much lower awareness levels and/or adoption rates. For all regions, the adoption rates were significantly lower than awareness levels, indicating a large gap between awareness and practice. One specific example is regarding minimizing greenhouse gas emission, which is a focal point in sustainable remediation literature, but with very low adoption rate according to this survey. This study also found that the adoption rates of a few sustainable remediation considerations, such as "minimizing local scale secondary impact", "minimizing national to global scale secondary impact", and "bringing prosperity to disadvantaged community", had decreased between 2012 and 2014. On the other hand, the survey also suggests the remediation community has rendered more expertise, training, and resources in sustainable remediation between 2012 and 2014. The mixed results suggest that in order to enhance sustainable remediation adoption, it is imperative to employ continued effort to enhance the understanding of sustainable remediation by practitioners and to link self-interest and public interest with sustainable remediation considerations.

Improving Evaluation to Address the Unintended Consequences of Health Information Technology:. a Position Paper from the Working Group on Technology Assessment & Quality Development

BACKGROUND AND OBJECTIVES

With growing use of IT by healthcare professionals and patients, the opportunity for any unintended effects of technology to disrupt care health processes and outcomes is intensified. The objectives of this position paper by the IMIA Working Group (WG) on Technology Assessment and Quality Development are to highlight how our ongoing initiatives to enhance evaluation are also addressing the unintended consequences of health IT.

METHODS

Review of WG initiatives Results: We argue that an evidence-based approach underpinned by rigorous evaluation is fundamental to the safe and effective use of IT, and for detecting and addressing its unintended consequences in a timely manner. We provide an overview of our ongoing initiatives to strengthen study design, execution and reporting by using evaluation frameworks and guidelines which can enable better characterization and monitoring of unintended consequences, including the Good Evaluation Practice Guideline in Health Informatics (GEP-HI) and the Statement on Reporting of Evaluation Studies in Health Informatics (STARE-HI). Indicators to benchmark the adoption and impact of IT can similarly be used to monitor unintended effects on healthcare structures, processes and outcome. We have also developed EvalDB, a web-based database of evaluation studies to promulgate evidence about unintended effects and are developing the content for courses to improve training in health IT evaluation.

CONCLUSION

Evaluation is an essential ingredient for the effective use of IT to improve healthcare quality and patient safety. WG resources and skills development initiatives can facilitate a proactive and evidence-based approach to detecting and addressing the unintended effects of health IT.

Submental island flap reconstruction reduces cost in oral cancer reconstruction compared to radial forearm free flap reconstruction: a case series and cost analysis

BACKGROUND

In Canada, 4,400 cases of oral cancer are diagnosed yearly. Surgical resection is a key component of treatment in many of these cancers. Reconstruction of defects, with the goal of preserving function, is of utmost importance. Several choices are possible for reconstruction of larger defects, including both free and pedicled flaps. Free flap reconstruction is reliable and effective, but requires additional personnel and peri-operative resources. Pedicled flaps remain an important alternative to free flaps, and are less resource intensive. This paper reviews our inaugural experience with the submental island flap (SIF) and compares costs incurred to a matched cohort of oral cancer patients reconstructed with forearm free flaps.

METHODS

Charts of patients who underwent SIF and RFFF reconstruction from January 1st 2013 to April 1st 2015 were retrospectively examined. Associated costs were obtained via online database and previously reported costs at the study institution.

RESULTS

Mean length of ICU stay in glossectomy RFFF reconstruction was 4.7 days. Only one patient required ICU stay for one night in the SIF group. Mean length of hospital stay was not significantly different in SIF patients vs RFFF patients (12.4 vs 15.4 days, p > 0.05). Mean operative time was significantly lower in the SIF group compared to the RFFF group (347 vs 552 min, p < 0.05). Total mean intraoperative costs were found to be $4780.59 for RFFF operations, versus $2307.94 for SIF. Total mean cost of post-operative stay was $18158.40 in the SIF group and $43617.60 in the RFFF group. Total cost savings were therefore $27931.85 per patient for the SIF group.

CONCLUSIONS

We have demonstrated the use of the submental island flap as an alternative to radial forearm free flaps, showing both decreased hospital costs and comparable patient outcomes. Pedicled flaps are making a resurgence in head and neck reconstruction, and the submental island flap offers an excellent alternative to more labour intensive and costly free flap alternatives.

Re-exposure to beta cell autoantigens in pancreatic allograft recipients with preexisting beta cell autoantibodies

Re-exposure to beta cell autoantigens and its relevance in the presence of donor-specific antibodies (DSA) in pancreatic allograft recipients is not well known. Thirty-three patients requiring a pancreas transplant were enrolled in an IRB approved study. They underwent prospective monitoring for DSA and beta cell autoantibody (BCAA) levels to GAD65, insulinoma-associated antigen 2 (IA-2), insulin (micro-IAA [mIAA]), and islet-specific zinc transporter isoform-8 (ZnT8). Twenty-five (75.7%) had pre-transplant BCAA. Twenty had a single antibody (mIAA n = 15, GAD65 n = 5); five had two or more BCAA (GAD65 + mIAA n = 2, GAD65 + mIAA+IA-2 n = 2, GA65 + mIAA+IA-2 + ZnT8 = 1). No changes in GAD65 (p > 0.29), IA-2 (>0.16), and ZnT8 (p > 0.07) were observed between pre-transplant and post-transplant at 6 or 12 months. A decrease in mIAA from pre- to post-6 months (p < 0.0001), 12 months (p < 0.0001), and from post-6 to post-12 months (p = 0.0002) was seen. No new BCAA was observed at one yr. Seven (21.0%) developed de novo DSA. The incidence of DSA was 24% in patients with BCAA vs. 25% in patients without BCAA (p = 0.69). Pancreatic allograft function of patients with vs. without BCAA, and with and without BCAA + DSA was comparable until last follow-up (three yr). Re-exposure to beta cell autoantigens by pancreas transplant may not lead to increased levels or development of new BCAA or pancreatic allograft dysfunction.

Patient Portals as a Means of Information and Communication Technology Support to Patient- Centric Care Coordination - the Missing Evidence and the Challenges of Evaluation. A joint contribution of IMIA WG EVAL and EFMI WG EVAL

OBJECTIVES

To review the potential contribution of Information and Communication Technology (ICT) to enable patient-centric and coordinated care, and in particular to explore the role of patient portals as a developing ICT tool, to assess the available evidence, and to describe the evaluation challenges.

METHODS

Reviews of IMIA, EFMI, and other initiatives, together with literature reviews.

RESULTS

We present the progression from care coordination to care integration, and from patient-centric to person-centric approaches. We describe the different roles of ICT as an enabler of the effective presentation of information as and when needed. We focus on the patient's role as a co-producer of health as well as the focus and purpose of care. We discuss the need for changing organisational processes as well as the current mixed evidence regarding patient portals as a logical tool, and the reasons for this dichotomy, together with the evaluation principles supported by theoretical frameworks so as to yield robust evidence.

CONCLUSIONS

There is expressed commitment to coordinated care and to putting the patient in the centre. However to achieve this, new interactive patient portals will be needed to enable peer communication by all stakeholders including patients and professionals. Few portals capable of this exist to date. The evaluation of these portals as enablers of system change, rather than as simple windows into electronic records, is at an early stage and novel evaluation approaches are needed.

Optimising Health Informatics Outcomes--Getting Good Evidence to Where it Matters

This editorial is part of a For-Discussion-Section of Methods of Information in Medicine about the paper "Evidence-based Health informatics: How do we know what we know?", written by Elske Ammenwerth [1]. Health informatics uses and applications have crept up on health systems over half a century, starting as simple automation of large-scale calculations, but now manifesting in many cases as rule- and algorithm-based creation of composite clinical analyses and 'black box' computation of clinical aspects, as well as enablement of increasingly complex care delivery modes and consumer health access. In this process health informatics has very largely bypassed the rules of precaution, proof of effectiveness, and assessment of safety applicable to all other health sciences and clinical support systems. Evaluation of informatics applications, compilation and recognition of the importance of evidence, and normalisation of Evidence Based Health Informatics, are now long overdue on grounds of efficiency and safety. Ammenwerth has now produced a rigorous analysis of the current position on evidence, and evaluation as its lifeblood, which demands careful study then active promulgation. Decisions based on political aspirations, 'modernisation' hopes, and unsupported commercial claims must cease - poor decisions are wasteful and bad systems can kill. Evidence Based Health Informatics should be promoted, and expected by users, as rigorously as Cochrane promoted Effectiveness and Efficiency, and Sackett promoted Evidence Based Medicine - both of which also were introduced retrospectively to challenge the less robust and partially unsafe traditional 'wisdom' in vogue. Ammenwerth's analysis gives the necessary material to promote that mission.

Citation analysis in health care sciences. Innovative investigation or seductive pseudo-science?

BACKGROUND

Citations of scientific papers in health care have become a subject of interest, as in any scientific sector. However, such a metric, while useful in a simple way, can become subject to too much credibility, and therefore to advanced analyses which are not soundly grounded even if methodologically valid.

OBJECTIVES

The objective was to contextualise citation indices in health care science, juxtaposed with a parallel analytic paper.

METHODS

The literature was reviewed and set against the objectives of metrics of literature and of researchers.

RESULTS

The strengths and weaknesses of citation indices in the health sector were identified, with practical examples of perverse effects, and compared with the core objectives of health care development.

CONCLUSIONS

Citation indices in health care science have a value, but this is limited to giving a broad overview. Other measures of effectiveness, including impact on education, on health care development, and on stimulation of applied developments, are needed rather than spuriously scientific advanced analyses of citation counts.

The RICHE taxonomy - an innovative means of classification of child health research

BACKGROUND

Research outputs increase inexorably. Health is now a required element in all policies of the European Union. There is a need for a system that helps to navigate the vast body of children's health research, identify pertinent research institutions, discover ongoing and recently funded research projects and identify gaps where there is little knowledge.

METHODS

The European Commission funded the Research Inventory of Child Health in Europe (RICHE) project through the Framework 7 Programme, to identify gaps in child health research in Europe. A necessary first step was to identify and index current research, for which a website repository was created. As a basis for this task, an innovative taxonomy was necessary to encompass the many arenas of children's health and development, including subjects outside the traditional areas of children's health. Drawing inspiration from existing taxonomies, library systems and other forms of classification, a multi-axial approach was selected as the best way to encompass the many influences on children's health. Six axes were identified and their contents defined. All of the axes can be viewed and searched independently, as well as in relation to each other. The axes encompass factors and service areas that impact on children, including health, education, justice, the environment and others. This has created a system that is consistent and impartial, but adaptable to an enormous variety of uses.

RESULTS

The taxonomy has been tested and validated by a number of well-respected academics, researchers and practitioners across Europe. It forms the basis of an intuitive and accessible database. This allows research knowledge to be easily identified and for networking to take place.

CONCLUSIONS

The RICHE taxonomy facilitates retrieval of knowledge - ongoing research as well as findings - in order to inform researchers and policy makers who wish to include children's health as an element of new policy.