STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics: explanation and elaboration

BACKGROUND

Improving the quality of reporting of evaluation studies in health informatics is an important requirement towards the vision of evidence-based health informatics. The STARE-HI - Statement on Reporting of Evaluation Studies in health informatics, published in 2009, provides guidelines on the elements to be contained in an evaluation study report.

OBJECTIVES

To elaborate on and provide a rationale for the principles of STARE-HI and to guide authors and readers of evaluation studies in health informatics by providing explanatory examples of reporting.

METHODS

A group of methodologists, researchers and editors prepared the present elaboration of the STARE-HI statement and selected examples from the literature.

RESULTS

The 35 STARE-HI items to be addressed in evaluation papers describing health informatics interventions are discussed one by one and each is extended with examples and elaborations.

CONCLUSION

The STARE-HI statement and this elaboration document should be helpful resources to improve reporting of both quantitative and qualitative evaluation studies. Evaluation manuscripts adhering to the principles will enable readers of such papers to better place the studies in a proper context and judge their validity and generalizability, and thus in turn optimize the exploitation of the evidence contained therein.

LIMITATIONS

This paper is based on experiences of a group of editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the details of these principles has to evolve as a function of their use in practice.

An invasive fish and the time-lagged spread of its parasite across the Hawaiian archipelago

Efforts to limit the impact of invasive species are frustrated by the cryptogenic status of a large proportion of those species. Half a century ago, the state of Hawai'i introduced the Bluestripe Snapper, Lutjanus kasmira, to O'ahu for fisheries enhancement. Today, this species shares an intestinal nematode parasite, Spirocamallanus istiblenni, with native Hawaiian fishes, raising the possibility that the introduced fish carried a parasite that has since spread to naïve local hosts. Here, we employ a multidisciplinary approach, combining molecular, historical, and ecological data to confirm the alien status of S. istiblenni in Hawai'i. Using molecular sequence data we show that S. istiblenni from Hawai'i are genetically affiliated with source populations in French Polynesia, and not parasites at a geographically intermediate location in the Line Islands. S. istiblenni from Hawai'i are a genetic subset of the more diverse source populations, indicating a bottleneck at introduction. Ecological surveys indicate that the parasite has found suitable intermediate hosts in Hawai'i, which are required for the completion of its life cycle, and that the parasite is twice as prevalent in Hawaiian Bluestripe Snappers as in source populations. While the introduced snapper has spread across the entire 2600 km archipelago to Kure Atoll, the introduced parasite has spread only half that distance. However, the parasite faces no apparent impediments to invading the entire archipelago, with unknown implications for naïve indigenous Hawaiian fishes and the protected Papahānaumokuākea Marine National Monument.

Evidence Based Health Informatics: 10 Years of Efforts to Promote the Principle. Joint Contribution of IMIA WG EVAL and EFMI WG EVAL

OBJECTIVES

To present the importance of Evidence-based Health Informatics (EBHI) and the ethical imperative of this approach; to highlight the work of the IMIA Working Group on Technology Assessment and Quality Improvement and the EFMI Working Group on Assessment of Health Information Systems; and to introduce the further important evaluation and evidence aspects being addressed.

METHODS

Reviews of IMIA, EFMA and other initiatives, together with literature reviews on evaluation methods and on published systematic reviews.

RESULTS

Presentation of the rationale for the health informatics domain to adopt a scientific approach by assessing impact, avoiding harm, and empirically demonstrating benefit and best use; reporting of the origins and rationale of the IMIA- and EQUATOR-endorsed Statement on Reporting of Evaluation Studies in Health Informatics (STARE-HI) and of the IMIA WG's Guideline for Good Evaluation Practice in Health Informatics (GEP-HI); presentation of other initiatives for objective evaluation; and outlining of further work in hand on usability and indicators; together with the case for development of relevant evaluation methods in newer applications such as telemedicine. The focus is on scientific evaluation as a reliable source of evidence, and on structured presentation of results to enable easy retrieval of evidence.

CONCLUSIONS

EBHI is feasible, necessary for efficiency and safety, and ethically essential. Given the significant impact of health informatics on health systems, care delivery and personal health, it is vital that cultures change to insist on evidence-based policies and investment, and that emergent global moves for this are supported.

Halophyte filter beds for treatment of saline wastewater from aquaculture

The expansion of aquaculture and the recent development of more intensive land-based marine farms require efficient and cost-effective systems for treatment of highly nutrient-rich saline wastewater. Constructed wetlands with halophytic plants offer the potential for waste-stream treatment combined with production of valuable secondary plant crops. Pilot wetland filter beds, constructed in triplicate and planted with the saltmarsh plant Salicornia europaea, were evaluated over 88 days under commercial operating conditions on a marine fish and shrimp farm. Nitrogen waste was primarily in the form of dissolved inorganic nitrogen (TDIN) and was removed by 98.2 ± 2.2% under ambient loadings of 109-383 μmol l(-1). There was a linear relationship between TDIN uptake and loading over the range of inputs tested. At peak loadings of up to 8185 ± 590 μmol l(-1) (equivalent to 600 mmol N m(-2) d(-1)), the filter beds removed between 30 and 58% (250 mmol N m(-2) d(-1)) of influent TDIN. Influent dissolved inorganic phosphorus levels ranged from 34 to 90 μmol l(-1), with 36-89% reduction under routine operations. Dissolved organic nitrogen (DON) loadings were lower (11-144 μmol l(-1)), and between 23 and 69% of influent DON was removed during routine operation, with no significant removal of DON under high TDIN loading. Over the 88-day study, cumulative nitrogen removal was 1.28 mol m(-2), of which 1.09 mol m(-2) was retained in plant tissue, with plant uptake ranging from 2.4 to 27.0 mmol N g(-1) dry weight d(-1). The results demonstrate the effectiveness of N and P removal from wastewater from land-based intensive marine aquaculture farms by constructed wetlands planted with S. europaea.

Personal Health, Person-centred Health and Personalised Medicine – Concepts, Consumers, Confusion and Challenges in the Informatics World

To define and assess ‘Consumer Health Informatics’ and related emergent issues in an era of new media and of personalisation of care, and from this to define what actions need to be taken to optimise benefits and address risks.

Definition of key concepts; review of health personalisation, emergent health information and communication technologies and knowledge sources available to citizens and social media; and identification of unresolved issues threatening optimal use of each.

A structured review supported by citations and examples.

Several new aspects of consumer health informatics are emerging, including new knowledge sources, feedback on treatments and care providers, on-line videos, and a new generation of patient experience sites including those which are for profit and seek to influence treatment paradigms. Not just the information usage, but also the potential social challenges and malicious abuses, are global issues, and also transcend the traditional health community and thus should be addressed in partnership with other global agencies.

Personal Health, Person-centred Health and Personalised Medicine - Concepts, Consumers, Confusion and Challenges in the Informatics World

OBJECTIVES

To define and assess 'Consumer Health Informatics' and related emergent issues in an era of new media and of personalisation of care, and from this to define what actions need to be taken to optimise benefits and address risks.

METHODS

Definition of key concepts; review of health personalisation, emergent health information and communication technologies and knowledge sources available to citizens and social media; and identification of unresolved issues threatening optimal use of each.

RESULTS

A structured review supported by citations and examples.

CONCLUSIONS

Several new aspects of consumer health informatics are emerging, including new knowledge sources, feedback on treatments and care providers, on-line videos, and a new generation of patient experience sites including those which are for profit and seek to influence treatment paradigms. Not just the information usage, but also the potential social challenges and malicious abuses, are global issues, and also transcend the traditional health community and thus should be addressed in partnership with other global agencies.

Health Informatics 3.0 and other increasingly dispersed technologies require even greater trust: promoting safe evidence-based health informatics. Contribution of the IMIA Working Group on Technology Assessment & Quality Development in Health Informatics

BACKGROUND

Health informatics is generally less committed to a scientific evidence-based approach than any other area of health science, which is an unsound position. Introducing the new Web 3.0 paradigms into health IT applications can unleash a further great potential, able to integrate and distribute data from multiple sources. The counter side is that it makes the user and the patient evermore dependent on the 'black box' of the system, and the re-use of the data remote from the author and initial context. Thus anticipatory consideration of uses, and proactive analysis of evidence of effects, are imperative, as only when a clinical technology can be proven to be trustworthy and safe should it be implemented widely - as is the case with other health technologies.

OBJECTIVES

To argue for promoting evidence-based health informatics as systems become more powerful and pro-active yet more dispersed and remote; and evaluation as the means of generating the necessary scientific evidence base. To present ongoing IMIA and EFMI initiatives in this field.

METHODS

Critical overview of recent developments in health informatics evaluation, alongside the precedents of other health technologies, summarising current initiatives and the new challenges presented by Health Informatics 3.0.

RESULTS

Web 3.0 should be taken as an opportunity to move health informatics from being largely unaccountable to one of being an ethical and responsible science-based domain. Recent and planned activities of the EFMI and IMIA working groups have significantly progressed key initiatives.

CONCLUSIONS

Concurrent with the emergence of Web 3.0 as a means of new-generation diffuse health information systems comes an increasing need for an evidence-based culture in health informatics.

STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics

Objective Development of guidelines for publication of evaluation studies of Health Informatics applications.

Methods An initial list of issues to be addressed in reports on evaluation studies was drafted based on experiencesas editorsand reviewers and as authors of systematic reviews , taking into account guidelines for reporting of medical research. This list has been discussed in several rounds by an increasing number of experts in Health Informatics evaluation during conferences and by using e-mail.

ResultsA set of STARE-HI principles to be addressed in papers describing evaluations of Health Informatics interventions is presented. These principles include formulation of title and abstract, of introduction (e.g. scientific background, study objectives), study context (e.g. organizational setting, system details), methods (e.g. study design, outcome measures), results (e.g. study findings, unexpected observations)and discussion and conclusion.

Conclusion Acomprehensivelistofprinciplesrelevantforproperlydescribing Health Informatics evaluations has been developed. When manuscripts submitted to Health Informatics journals and general medical journals adhere to these aspects, readers will be better positioned to place the studies in a proper context and judge their validity and generalisability. STARE-HI may also be used for study planning and hence positively influence the quality of evaluation studies in Health Informatics. We believe that better publication of (both quantitative and qualitative) evaluation studies is an important step toward the vision of evidence-based Health Informatics.

Limitations This study is based on experiences from editors, reviewers, authors of systemati c reviews and readers of the scientific literature. The applicability of the principles has not been evaluated in real practice. Only when authors start to use these principles for reporting, shortcomings in the principles will emerge.

A paediatric telecardiology service for district hospitals in south-east England: an observational study

OBJECTIVES

To compare caseloads of new patients assessed by paediatric cardiologists face-to-face or during teleconferences, and assess NHS costs for the alternative referral arrangements.

DESIGN

Prospective cohort study over 15 months.

SETTING

Four district hospitals in south-east England and a London paediatric cardiology centre.

PATIENTS

Babies and children.

INTERVENTION

A telecardiology service introduced alongside outreach clinics.

MEASUREMENTS

Clinical outcomes and mean NHS costs per patient.

RESULTS

266 new patients were studied: 75 had teleconsultations (19 of 42 newborns and 56 of 224 infants and children). Teleconsultation patients generally were younger (49% being under 1 year compared with 32% seen personally (p = 0.025)) and their symptoms were not as severe. A cardiac intervention was undertaken immediately or planned for five telemedicine patients (7%) and 30 conventional patients (16%). However, similar proportions of patients were discharged after being assessed (32% telemedicine and 39% conventional). During scheduled teleconferences the mean duration of time per patient in sessions involving real-time echocardiography was 14.4 min, and 8.5 min in sessions where pre-recorded videos were transmitted. Mean cost comparisons for telemedicine and face-to-face patients over 14-day and 6-month follow-up showed the telecardiology service to be cost-neutral for the three hospitals with infrequently-held outreach clinics (1519 UK pounds vs 1724 UK pounds respectively after 14 days).

CONCLUSION

Paediatric cardiology centres with small cadres of specialists are under pressure to cope with ever-expanding caseloads of new patients with suspected anomalies. Innovative use of telecardiology alongside conventional outreach services should suitably, and economically, enhance access to these specialists.

STARE-HI -statement on reporting of evaluation studies in health informatics

OBJECTIVE

Development of guidelines for publication of evaluation studies of Health Informatics applications.

METHODS

An initial list of issues to be addressed in reports on evaluation studies was drafted based on experiences as editors and reviewers and as authors of systematic reviews , taking into account guidelines for reporting of medical research. This list has been discussed in several rounds by an increasing number of experts in Health Informatics evaluation during conferences and by using e-mail.

RESULTS

A set of STARE-HI principles to be addressed in papers describing evaluations of Health Informatics interventions is presented. These principles include formulation of title and abstract, of introduction (e.g. scientific background, study objectives), study context (e.g. organizational setting, system details), methods (e.g. study design, outcome measures), results (e.g. study findings, unexpected observations) and discussion and conclusion.

CONCLUSION

A comprehensive list of principles relevant for properly describing Health Informatics evaluations has been developed. When manuscripts submitted to Health Informatics journals and general medical journals adhere to these aspects, readers will be better positioned to place the studies in a proper context and judge their validity and generalisability. STARE-HI may also be used for study planning and hence positively influence the quality of evaluation studies in Health Informatics. We believe that better publication of (both quantitative and qualitative) evaluation studies is an important step toward the vision of evidence-based Health Informatics.

LIMITATIONS

This study is based on experiences from editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the principles has not been evaluated in real practice. Only when authors start to use these principles for reporting, shortcomings in the principles will emerge.

Evaluation – The Cinderella Science of ICT in Health

Information systems are expensive to create, and consume resources in their operation. They are justified in order to achieve clearly defined objectives in health service quality and efficiency. To ensure that these objectives are being met, and in order to build up an evidence-base to support well focused policies, systems, and practice, evaluation would seem to be fully justified and indeed essential. However, there are major factors jeopardising good evaluation practice, ranging from reluctance to commit resources or expose decisions to scrutiny, through to lack of appreciation of the full range of dimensions which should be studied. This paper seeks to create fuller appreciation of the issues and their importance.

Experiential and literature reviews form the basis of an exposition of principles, methods, and key current activities.

A summary has been produced of the key challenges to health ICT evaluation, and its adverse comparison with other sectors. An explanation of the viewpoints and levels of health information evaluation is followed by a short history of principal milestones, current scientific methodology activity, and key collaborative activities.

The need for a stronger commitment to health ICT evaluation is demonstrated if the application of health informatics systems is to receive due recognition as scientifically and empirically justified. Commitment of resources and scientific endeavour are needed, and thus the recent Declaration of Innsbruck, arising out of an event sponsored by the European Science Foundation, is timely and fully justified.

Evaluation--the Cinderella science of ICT in health

OBJECTIVES

Information systems are expensive to create, and consume resources in their operation. They are justified in order to achieve clearly defined objectives in health service quality and efficiency. To ensure that these objectives are being met, and in order to build up an evidence-base to support well focused policies, systems, and practice, evaluation would seem to be fully justified and indeed essential. However, there are major factors jeopardising good evaluation practice, ranging from reluctance to commit resources or expose decisions to scrutiny, through to lack of appreciation of the full range of dimensions which should be studied. This paper seeks to create fuller appreciation of the issues and their importance.

METHODS

Experiential and literature reviews form the basis of an exposition of principles, methods, and key current activities.

RESULTS

A summary has been produced of the key challenges to health ICT evaluation, and its adverse comparison with other sectors. An explanation of the viewpoints and levels of health information evaluation is followed by a short history of principal milestones, current scientific methodology activity, and key collaborative activities.

CONCLUSION

The need for a stronger commitment to health ICT evaluation is demonstrated if the application of health informatics systems is to receive due recognition as scientifically and empirically justified. Commitment of resources and scientific endeavour are needed, and thus the recent Declaration of Innsbruck, arising out of an event sponsored by the European Science Foundation, is timely and fully justified.

Indicators of children's development: considerations when constructing a set of national Child Health Indicators for the European Union

BACKGROUND

In a recent project, commissioned by the European Union, a set of Child Health Indicators were identified and recommended for use in all Member States. In that work, by a group of European experts, children's development was one of the key areas that were reviewed to clarify its role in this set of indicators. This paper deals with neurological (in a broad sense), social and moral aspects of development; other aspects, such as nutrition and physical growth, mental health, quality of life were dealt with in other areas.

METHODS

A number of methods are used to monitor children's neurodevelopment, to identify early deviations that could be treated or alleviated. Few tests fulfil basic criteria of screening and quality of evidence. For Child Health Programmes, developmental surveillance is instead recommended. As a proxy to social development, as part of children's process of adaptation into society, education indicators are recommended, such as suggested by OECD and UNICEF. Moral development is discarded as an indicator, because norms and values vary too much between countries.

CONCLUSION

For now the Child Health Indicators of Life and Development Project found no ground to recommend any particular method or combination of methods for a general monitoring of children's neurological, intellectual, language and moral development, and for including the results as national indicators of children's health and well-being in EU. For social development, indicators in the educational area seem most promising, as they have been proposed and used by OECD and UNESCO. In an ambitious monitoring programme, these indicators could be included as determinants of children's health and development.