Key performance indicators to benchmark hospital information systems - a delphi study

OBJECTIVES

To identify the key performance indicators for hospital information systems (HIS) that can be used for HIS benchmarking.

METHODS

A Delphi survey with one qualitative and two quantitative rounds. Forty-four HIS experts from health care IT practice and academia participated in all three rounds.

RESULTS

Seventy-seven performance indicators were identified and organized into eight categories: technical quality, software quality, architecture and interface quality, IT vendor quality, IT support and IT department quality, workflow support quality, IT outcome quality, and IT costs. The highest ranked indicators are related to clinical workflow support and user satisfaction. Isolated technical indicators or cost indicators were not seen as useful. The experts favored an interdisciplinary group of all the stakeholders, led by hospital management, to conduct the HIS benchmarking. They proposed benchmarking activities both in regular (annual) intervals as well as at defined events (for example after IT introduction). Most of the experts stated that in their institutions no HIS benchmarking activities are being performed at the moment.

CONCLUSION

In the context of IT governance, IT benchmarking is gaining importance in the healthcare area. The found indicators reflect the view of health care IT professionals and researchers. Research is needed to further validate and operationalize key performance indicators, to provide an IT benchmarking framework, and to provide open repositories for a comparison of the HIS benchmarks of different hospitals.

STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics

Objective Development of guidelines for publication of evaluation studies of Health Informatics applications.

Methods An initial list of issues to be addressed in reports on evaluation studies was drafted based on experiencesas editorsand reviewers and as authors of systematic reviews , taking into account guidelines for reporting of medical research. This list has been discussed in several rounds by an increasing number of experts in Health Informatics evaluation during conferences and by using e-mail.

ResultsA set of STARE-HI principles to be addressed in papers describing evaluations of Health Informatics interventions is presented. These principles include formulation of title and abstract, of introduction (e.g. scientific background, study objectives), study context (e.g. organizational setting, system details), methods (e.g. study design, outcome measures), results (e.g. study findings, unexpected observations)and discussion and conclusion.

Conclusion Acomprehensivelistofprinciplesrelevantforproperlydescribing Health Informatics evaluations has been developed. When manuscripts submitted to Health Informatics journals and general medical journals adhere to these aspects, readers will be better positioned to place the studies in a proper context and judge their validity and generalisability. STARE-HI may also be used for study planning and hence positively influence the quality of evaluation studies in Health Informatics. We believe that better publication of (both quantitative and qualitative) evaluation studies is an important step toward the vision of evidence-based Health Informatics.

Limitations This study is based on experiences from editors, reviewers, authors of systemati c reviews and readers of the scientific literature. The applicability of the principles has not been evaluated in real practice. Only when authors start to use these principles for reporting, shortcomings in the principles will emerge.

STARE-HI -statement on reporting of evaluation studies in health informatics

OBJECTIVE

Development of guidelines for publication of evaluation studies of Health Informatics applications.

METHODS

An initial list of issues to be addressed in reports on evaluation studies was drafted based on experiences as editors and reviewers and as authors of systematic reviews , taking into account guidelines for reporting of medical research. This list has been discussed in several rounds by an increasing number of experts in Health Informatics evaluation during conferences and by using e-mail.

RESULTS

A set of STARE-HI principles to be addressed in papers describing evaluations of Health Informatics interventions is presented. These principles include formulation of title and abstract, of introduction (e.g. scientific background, study objectives), study context (e.g. organizational setting, system details), methods (e.g. study design, outcome measures), results (e.g. study findings, unexpected observations) and discussion and conclusion.

CONCLUSION

A comprehensive list of principles relevant for properly describing Health Informatics evaluations has been developed. When manuscripts submitted to Health Informatics journals and general medical journals adhere to these aspects, readers will be better positioned to place the studies in a proper context and judge their validity and generalisability. STARE-HI may also be used for study planning and hence positively influence the quality of evaluation studies in Health Informatics. We believe that better publication of (both quantitative and qualitative) evaluation studies is an important step toward the vision of evidence-based Health Informatics.

LIMITATIONS

This study is based on experiences from editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the principles has not been evaluated in real practice. Only when authors start to use these principles for reporting, shortcomings in the principles will emerge.

The time needed for clinical documentation versus direct patient care. A work-sampling analysis of physicians' activities

OBJECTIVES

Health care professionals seem to be confronted with an increasing need for high-quality, timely, patient-oriented documentation. However, a steady increase in documentation tasks has been shown to be associated with increased time pressure and low physician job satisfaction. Our objective was to examine the time physicians spend on clinical and administrative documentation tasks. We analyzed the time needed for clinical and administrative documentation, and compared it to other tasks, such as direct patient care.

METHODS

During a 2-month period (December 2006 to January 2007) a trained investigator completed 40 hours of 2-minute work-sampling analysis from eight participating physicians on two internal medicine wards of a 200-bed hospital in Austria. A 37-item classification system was applied to categorize tasks into five categories (direct patient care, communication, clinical documentation, administrative documentation, other).

RESULTS

From the 5555 observation points, physicians spent 26.6% of their daily working time for documentation tasks, 27.5% for direct patient care, 36.2% for communication tasks, and 9.7% for other tasks. The documentation that is typically seen as administrative takes only approx. 16% of the total documentation time.

CONCLUSIONS

Nearly as much time is being spent for documentation as is spent on direct patient care. Computer-based tools and, in some areas, documentation assistants may help to reduce the clinical and administrative documentation efforts.

The quality of evidence in health informatics: how did the quality of healthcare IT evaluation publications develop from 1982 to 2005?

OBJECTIVE

To obtain an overview of study designs and study methods used in research evaluating IT in health care, to present a list of quality criteria by which all kinds of reported evaluation studies on IT systems in health care can be assessed, and to assess the quality of reported evaluation studies on IT in health care and its development over time (1982-2005).

METHODS

A generic 10-item list of quality indicators was developed based on existing literature on quality of medical and medical informatics publications. It is applicable to all kind of IT evaluation papers and not restricted to randomized controlled trials. One hundred and twenty explanatory papers evaluating the effects of an IT system in health care published between 1982 and 2005 were randomly selected from PubMed, the study designs and study methods were extracted, and the quality indicators were used to assess the quality of each paper by two independent raters.

RESULTS

The inter-rater variability of scoring the 10 quality indicators as assessed by a pre-test with nine papers was good (K=0.87). There was a trend towards more multi-centre studies and authors coming more frequently from various departments. About 70% of the studies used a design other than a randomized controlled trial (RCT). Forty percent of the studies combined at least two different data acquisition methods. The quality of IT evaluation papers, as defined by the quality indicators, was only slightly improving in time (Spearman correlation coefficient [rs]=0.19). The quality of RCTs publications was significantly higher than the quality of non-RCT studies (p<0.001).

CONCLUSION

The continuous and dominant number of non-RCT studies reflects the various approaches applicable to evaluate IT systems in health care. Despite the increasing discussion on evidence-based health informatics, the quality of published evaluation studies on IT interventions in health care is still insufficient in some aspects. Journal editors and referees should take care that reports of evaluation on IT systems contain all aspects needed for a sufficient understanding and reproducibility of a paper. Publication guidelines should be developed to support more complete and better publications of IT evaluation papers.

Supporting the systematic assessment of clinical processes: the MedFlow method

OBJECTIVES

Healthcare is characterized by complex cooperation between highly specialized healthcare departments. This often leads to inefficient clinical processes. In order to improve these processes, a systematic assessment method is needed. Such methods are still missing. The objective of this paper is to propose and evaluate a method to support the systematic and semi-automatic assessment of clinical processes, with special focus on the quality of information logistics.

METHODS

Criteria for the quality of information logistics were collected based on literature research and system analysis. Appropriate quality checks for these criteria were developed. An extended process modelling notation was developed. The method was evaluated in a pilot study.

RESULTS

An own model integrates four sub-models with each concentrating on distinct process aspects (i.e., control flow, data flow, tool usage, organizational information). In order to assess the quality of a process, selected process details are combined in "views". Weak points are then detected by applying specific rule-sets on these views. Each rule-set represents a pattern of critical cross-points which are searched for in the appropriate view-matrix. The MedFlow method was evaluated in a first pilot study in radiological departments--applying quality checks for the detection of e.g. media cracks or testing the transcription of information objects.

CONCLUSION

The MedFlow method is best used to assess clinical processes regarding their control flow and information handling. The latter directly influences the quality of communication and thus the quality of whole processes. However, this must be evaluated in further studies.

Supporting the Systematic Assessment of Clinical Processes: the MedFlow Method

Objectives: Healthcare is characterized by complex cooperation between highly specialized healthcare departments. This often leads to inefficient clinical processes. In orderto improve these processes, a systematic assessment method is needed. Such methods are still missing. The objective of this paper is to propose and evaluate a method to support the systematic and semi-automatic assessment of clinical processes, with special focus on the quality of information logistics.

Methods: Criteria for the quality of information logistics were collected based on literature research and system analysis. Appropriate quality checks for these criteria were developed. An extended process modelling notation was developed. The method was evaluated in a pilot study.

Results: An own model integrates four sub-models with each concentrating on distinct process aspects (i.e., control flow, data flow, tool usage, organizational information). In orderto assess the quality of a process, selected process details are combined in “views”. Weak points are then detected by applying specific rule-sets on these views. Each rule-set represents a pattern of critical cross-points which are searched for in the appropriate view-matrix. The MedFlow method was evaluated in a first pilot study in radiological departments – applying quality checks for the detection of e.g. media cracks or testing the transcription of information objects.

Conclusion: The MedFlow method is best used to assess clinical processes regarding their control flow and information handling. The latter directly influences the quality of communication and thusthe quality of whole processes. However, this must be evaluated in further studies.

Special Section: Assessing Information Technologies for Health: Health Technology Assessment

To summarize current excellent research in the field of health technology assessment.

Synopsis of the articles selected for the IMIA Yearbook 2006.

Five excellent articles representing the research in four different nations were selected for the IMIA Yearbook 2006 from three international peer reviewed journals.

The best paper selection for the Yearbook section ‘Assessing Information Technologies for Health’ presents papers evaluating the benefit and side-effects of information technology in various settings. They clearly indicate that benefit of IT in health care can be achieved when the systems are appropriately designed, implemented and operated. Besides the presented quantitative studies, also qualitative study designs are of value to find unintended effects of IT, or to better explain found effects. IT evaluation supports a reflective practice on how health informatics influences health care, enabling the emergence of an evidence-based health informatics.

Factors influencing success and failure of health informatics systems--a pilot Delphi study

OBJECTIVES

The aim is to gain information on factors influencing success and failure for Health Informatics applications from a group of medical informaticians.

METHODS

Based on the presentations at a special topic conference on success and failure in Health ICT and analysis of the proceedings, we conducted a Delphi study on success and failure aspects.

RESULTS

A total of 110 success factors and 27 failure criteria were identified, distributed on categories like functional, organizational, behavioral, technical, managerial, political, cultural, legal, strategy, economy, education and user acceptance. These factors and criteria were rated for six different system types. Unanimously it was agreed that "collaboration and co-operation" and "setting goals and courses" are "essential for the success" of clinical systems, and "user acceptance" for educational systems. Similarly, the score "essential in order to avoid a failure" were given unanimously on clinical systems for "response rate and other performance measures" and on administrative systems for "not understanding the organizational context" with "not understanding or foreseeing the extent to which the new IT-system affects the organization, its structure and/or work procedures" as the highest scoring sub-item.

CONCLUSIONS

All success factors and failure criteria were considered relevant by the Delphi expert panel. There is no small set of relevant factors or indicators, but success or failure of a Health ICT depends on a large set of issues. Further, clinical systems and decision support systems depend on more factors than other systems.

Health technology assessment. Findings from the Section on Assessing Information Technologies for Health

OBJECTIVES

To summarize current excellent research in the field of health technology assessment.

METHODS

Synopsis of the articles selected for the IMIA Yearbook 2006.

RESULTS

Five excellent articles representing the research in four different nations were selected for the IMIA Yearbook 2006 from three international peer reviewed journals.

CONCLUSIONS

The best paper selection for the Yearbook section 'Assessing Information Technologies for Health' presents papers evaluating the benefit and side-effects of information technology in various settings. They clearly indicate that benefit of IT in health care can be achieved when the systems are appropriately designed, implemented and operated. Besides the presented quantitative studies, also qualitative study designs are of value to find unintended effects of IT, or to better explain found effects. IT evaluation supports a reflective practice on how health informatics influences health care, enabling the emergence of an evidence-based health informatics.

Evaluation of the electronic transmission of medical findings from hospitals to practitioners by triangulation

OBJECTIVES

The need for regional, cross-institutional electronic networks in health care is steadily growing to support seamless, cooperative health care. The aim of this study is to evaluate the impact of electronic transmission between hospitals and practitioners in a Tyrolean health care network, and to derive technical and organizational points for improvement.

METHODS

Between March and August 2004 we carried out a triangulation-based cross-sectional study, combining a qualitative study based on semi-structured, problem-centric interviews with selected practitioners, with a quantitative study based on a standardized questionnaire survey of all the Tyrolean practitioners that receive electronic messages. The survey was designed to confirm the hypotheses which have been systematically derived from the interviews.

RESULTS AND CONCLUSIONS

The results show high satisfaction and positive impact of electronic communication. The triangulation of quantitative and qualitative methods was found to be useful in order to make the definition and confirmation of the hypotheses more transparent.

Impact of CPOE on mortality rates--contradictory findings, important messages

OBJECTIVE

To analyze the seemingly contradictory results of the Han study (Pediatrics 2005) and the Del Beccaro study (Pediatrics 2006), both analyzing the effect of CPOE systems on mortality rates in pediatric intensive care settings.

METHODS

Seven CPOE system experts from the United States and Europe comment on these papers.

RESULTS

The two studies are not contradictory, but almost non-comparable due to differences in design and implementation. They demonstrate the range of outcomes that can be obtained from introducing informatics applications in complex health care settings. Implementing informatics applications is a sociotechnical activity, which often depends more on the organizational context than on a specific technology. As health informaticians, we must not only learn from failures, but also avoid both uncritical scepticism that may arise from drawing overly general conclusions from one negative trial, as much as uncritical optimism from limited successful ones.

CONCLUSION

The commentaries emphasize the need to promote systematic studies for assessing the socio-technical factors that influence the introduction of increasingly sophisticated informatics applications within complex organizations. The emergence of evidence-based health informatics will be based both on evaluation guidelines and implementation guidelines, both of which increase the chances of successful implementation. In addition, well-educated health informaticians are needed to manage and guide the implementation processes.

Medical informatics education needs information system practicums in health care settings--experiences and lessons learned from 32 practicums at four universities in two countries

OBJECTIVES

To report about the themes and about experiences with practicums in the management of information systems in health care settings (health information management) for medical informatics students.

METHODS

We first summarize the topics of the health information management practicums/projects that the authors organized between 1990 and 2003 for the medical informatics programs at Heidelberg/Heilbronn, Germany, UMIT, Austria, as well as for the informatics program at the University of Leipzig, Germany. Experiences and lessons learned, obtained from the faculty that organized the practicums in the past 14 years, are reported.

RESULTS

Thirty (of 32) health information management practicums focused on the analysis of health information systems. These took place inside university medical centers. Although the practicums were time-intensive and required intensively tutoring students with regard to health information management and project management, feedback from the students and graduates was mainly positive.

DISCUSSION

It is clearly recommended that students specializing in medical informatics need to be confronted with real-world problems of health information systems during their studies.

Process potential screening - an instrument to improve business processes in hospitals

OBJECTIVES

Hospitals are increasingly under pressure to optimize their processes. So far, an instrument to systematically identify the potentials for improvement of a given business process is missing. The aim of this project is to develop such an instrument.

METHODS

Initially, central aspects of the quality of a hospital process were identified on the basis of a systematic literature review. Secondary to that, criteria to measure quality aspects were defined: More than 300 criteria from medical and business informatics, economics and quality management publications were gathered and systematically aggregated.

RESULTS

As a result, the Process Potential Screening (PPS) instrument was developed. The PPS is a matrix containing two axes: Axis I comprises 30 quality aspects referring to results, execution and control of hospital processes. Axis II comprises 16 quality criteria (e.g., customer satisfaction, time). The PPS displays approximately 400 relevant combinations of those quality aspects and quality criteria that help to identify potentials for improvement of a given hospital process. It utilizes different methods for the measurement of the criteria and for application by way of individuals or groups.

CONCLUSIONS

In using the PPS, relevant potentials for improvement were identified in ten typical hospital processes. The instrument's practicability must now be examined in further studies by the final target group (e.g., quality or project managers, and the staff responsible for processes).

The challenge of ubiquitous computing in health care: technology, concepts and solutions. Findings from the IMIA Yearbook of Medical Informatics 2005

OBJECTIVES

To review recent research efforts in the field of ubiquitous computing in health care. To identify current research trends and further challenges for medical informatics.

METHODS

Analysis of the contents of the Yearbook on Medical Informatics 2005 of the International Medical Informatics Association (IMIA).

RESULTS

The Yearbook of Medical Informatics 2005 includes 34 original papers selected from 22 peer-reviewed scientific journals related to several distinct research areas: health and clinical management, patient records, health information systems, medical signal processing and biomedical imaging, decision support, knowledge representation and management, education and consumer informatics as well as bioinformatics. A special section on ubiquitous health care systems is devoted to recent developments in the application of ubiquitous computing in health care. Besides additional synoptical reviews of each of the sections the Yearbook includes invited reviews concerning E-Health strategies, primary care informatics and wearable healthcare.

CONCLUSIONS

Several publications demonstrate the potential of ubiquitous computing to enhance effectiveness of health services delivery and organization. But ubiquitous computing is also a societal challenge, caused by the surrounding but unobtrusive character of this technology. Contributions from nearly all of the established sub-disciplines of medical informatics are demanded to turn the visions of this promising new research field into reality.

An inventory of evaluation studies of information technology in health care trends in evaluation research 1982-2002

OBJECTIVES

During the last years the significance of evaluation studies as well as the interest in adequate methods and approaches for evaluation has grown in medical informatics. In order to put this discussion into historical perspective of evaluation research, we conducted a systematic review on trends in evaluation research of information technology in health care from 1982 to 2002.

METHODS

The inventory is based on a systematic literature search in PubMed. Abstracts were included when they described an evaluation study of a computer-based component in health care. We identified 1035 papers from 1982 to 2002 and indexed them based on a multi-axial classification describing type of information system, study location, evaluation strategy, evaluation methods, evaluation setting, and evaluation focus.

RESULTS AND CONCLUSIONS

We found interesting developments in evaluation research in the last 20 years. For example, there has been a strong shift from medical journals to medical informatics journals. With regard to methods, explanatory research and quantitative methods have dominated evaluation studies in the last 20 years. Since 1982, the number of lab studies and technical evaluation studies has declined, while the number of studies focusing on the influence of information technology on quality of care processes or outcome of patient care has increased. We interpret this shift as a sign of maturation of evaluation research in medical informatics.

Preparing strategic information management plans for hospitals: a practical guideline

OBJECTIVES

Systematic information management in hospitals demands for a strategic information management plan (SIM plan). As preparing a SIM plan is a considerable challenge we provide a practical guideline that is directly applicable when a SIM plan is going to be prepared.

METHODS

The guideline recommends a detailed structure of a SIM plan and gives advice about its content and the preparation process. It may be used as template, which can be adapted to the individual demands of any hospital.

RESULTS

The guideline was used in several hospitals preparing a SIM plan. Experiences showed that the SIM plans could be prepared very efficiently and timely using the guideline, that the proposed SIM plan structure suited well, that the guideline offers enough flexibility to meet the requirements of the individual hospitals and that the specific recommendations of the guideline were very helpful.

CONCLUSIONS

Nevertheless, we must strive for a more comprehensive theory of strategic information management planning which -- in the sense of enterprise architecture planning -- represents the intrinsic correlations of the different parts of a SIM plan to a greater extent.

Bad health informatics can kill--is evaluation the answer?

OBJECTIVE

Health care is entering the age of information society. It is evident that the use of modern information and communication technology (ICT) offers tremendous opportunities to improve health care. However, there are also hazards associated with ICT in health care. We want to present an overview of typical hazards associated with ICT in health care, and to discuss how ICT evaluation can be a solution.

METHODS

We analyze examples of failures and problems associated with ICT in health care. This collection is also made available on a website.

RESULTS AND CONCLUSION

Systematic, continuous evaluation of quality and effects of ICT during the whole life cycle of ICT components seems to be one important approach to detect and prevent possible ICT hazards and failures, supporting a higher quality of patient care. However, empirical studies proving this assumption are needed.

An international course on strategic information management for medical informatics students: international perspectives and evaluation

All over the world, countries more and more take part in the international society and economy. To meet the stringent requirements of this globalization asks for internationally oriented and well-educated graduates. A major challenge of academia thus lies in qualifying graduates for international positions in this new world. A crucial element in the training and education of tomorrow's medical informatics specialists is exposure to health care systems across national borders. In this contribution, we report on the international aspects of and experiences with an inter-university course for medical informatics students on hospital information systems, in particular on their strategic information management. From 2001 onwards, this course was offered jointly for students of the University of Amsterdam, the University of Heidelberg/University of Applied Sciences Heilbronn and the University of Health Informatics and Technology, Tyrol (UMIT). Based on our experiences, future establishment of international courses in the medical and health informatics field is recommended.

An international course on strategic information management for medical informatics students: aim, content, structure, and experiences

We report on a course for medical informatics students on hospital information systems, especially on its strategic information management. Starting as course at the Medical Informatics Program of the University of Heidelberg/University of Applied Sciences Heilbronn, it is now organized as international course in the framework of the International Partnership for Health Informatics Education (http:// www.iphie.org) jointly for medical information science students from the University of Amsterdam, medical informatics students, as well as health information management students from the Universities of Heidelberg/Heilbronn. In 2002, medical informatics students from the Master of Science program of the newly founded University for Health Informatics and Technology Tyrol (UMIT) at Innsbruck, Austria, joined. We report about the aim of this course, its audience, and the educational programs involved, about its content and structure, as well as about our experiences gained so far.

Towards clinical bioinformatics: advancing genomic medicine with informatics methods and tools

OBJECTIVES

To summarize the challenges facing clinical applications in the light of growing research results in genomic medicine and bioinformatics.

METHODS

Analysis of the contents of the Yearbook of Medical Informatics 2004 of the International Medical Informatics Association (IMIA).

RESULTS

The Yearbook of Medical Informatics 2004 includes 32 articles selected from 22 peer-reviewed scientific journals. A special section on clinical bioinformatics highlights recent developments in this field. Several guest editors review the promises and limitations of available methods and resources from biomedical informatics that are relevant to clinical medicine. Integrated data and knowledge resources are generally regarded to be central and key issues for clinical bioinformatics. Further review papers deal with public health implications of bioinformatics, knowledge management and trends in health care education. The Yearbook includes for the first time a section on the history of medical informatics, where the significant impact of the Reisensburg protocol 1973 on international health and medical informatics education is examined.

CONCLUSIONS

Close collaboration between bioinformatics and medical informatics researchers can contribute to new insights in genomic medicine and contribute towards the more efficient and effective use of genomic data to advance clinical care.

Medical informatics and the quality of health: new approaches to support patient care - findings from the IMIA Yearbook of Medical Informatics 2003

OBJECTIVES

The Yearbook of Medical Informatics is published annually by the International Medical Informatics Association (IMIA) and contains a selection of excellent papers on medical informatics research which have been recently published (http://www. yearbook.uni-hd.de). The 2003 Yearbook of Medical Informatics took as its theme the role of medical informatics for the quality of health care. In this paper, we will discuss challenges for health care, and the lessons learned from editing IMIA Yearbook 2003.

RESULTS AND CONCLUSIONS

Modern information processing methodology and information and communication technology have strongly influenced our societies and health care. As a consequence of this, medical informatics as a discipline has taken a leading role in the further development of health care. This involves developing information systems that enhance opportunities for global access to health services and medical knowledge. Informatics methodology and technology will facilitate high quality of care in aging societies, and will decrease the possibilities of health care errors. It will also enable the dissemination of the latest medical and health information on the web to consumers and health care providers alike. The selected papers of the IMIA Yearbook 2003 present clear examples and future challenges, and they highlight how various sub-disciplines of medical informatics can contribute to this.

A requirements index for information processing in hospitals

OBJECTIVES

Reference models describing typical information processing requirements in hospitals do not currently exist. This leads to high hospital information system (HIS) management expenses, for example, during tender processes for the acquisition of software application programs. Our aim was, therefore, to develop a comprehensive, lasting, technology-independent, and sufficiently detailed index of requirements for information processing in hospitals in order to reduce respective expenses.

METHODS

Two-dozen German experts established an index of requirements for information processing in university hospitals. This was done in a consensus-based, top-down, cyclic manner. Each functional requirement was derived from information processing functions and sub-functions of a hospital.

RESULTS

The result is the first official German version of a requirements index, containing 233 functional requirements and 102 function-independent requirements, focusing on German needs. The functional requirements are structured according to the primary care process from admission to discharge and supplemented by requirements for handling patient records, work organization and resource planning, hospital management, research and education. Both the German version and its English translation are available in the Internet.

CONCLUSIONS

The index of requirements contains general information processing requirements in hospitals which are formulated independent of information processing tools, or of HIS architectures. It aims at supporting HIS management, especially HIS strategic planning, HIS evaluation, and tender processes. The index can be regarded as a draft, which must, however, be refined according to the specific aims of a particular project. Although focused on German needs, we expect that it can also be useful in other countries. The high amount of interest shown for the index supports its usefulness.

Medical Imaging Informatics and Medical Informatics: opportunities and constraints. Findings from the IMIA yearbook of Medical Informatics 2002

OBJECTIVES

The Yearbook of Medical Informatics is published annually by the International Medical Informatics Association (IMIA) and contains a selection of recent excellent papers on medical informatics research (http://www.yearbook.uni-hd.de). The 2002 Yearbook of Medical Informatics took as its theme the topic of Medical Imaging Informatics. In this paper, we will summarize the contributions of medical informatics researchers to the development of medical imaging informatics, discuss challenges and opportunities of imaging informatics, and present the lessons learned from the IMIA Yearbook 2002.

RESULTS AND CONCLUSIONS

Medical informatics researchers have contributed to the development of medical imaging methods and systems since the inception of this field approximately 40 years ago. The Yearbook presents selected papers and reviews on this important topic. In addition, as usual, the Yearbook 2002 also contains a variety of papers and reviews on other subjects relevant to medical informatics, such as Bioinformatics, Computer-supported education, Health and clinical management, Health information systems, Knowledge processing and decision support, Patient records, and Signal processing.

Systems analysis in health care: framework and example

OBJECTIVES

Due to the high complexity of structures and processes in health care, thorough systems analyses in health care run the risk of becoming very complex and difficult to handle. Therefore, we aimed to support systematic systems analysis in health care by developing a comprehensive framework that presents and describes potential areas of analysis.

METHODS

A framework for systems analysis in health care was developed and applied in a health care setting. To provide a clear structure, the framework describes the potential views and levels of systems analyses in a health care environment.

RESULTS

The framework comprises five views (roles and responsibilities, information processing and tools, communication, business processes, teams structure and cooperation) and five levels of analysis (overall organization, organizational unit, staff member, role, task). The framework was successfully applied in an analysis of the structures and processes of the Department of Child and Adolescent Psychiatry of the University Medical Center Heidelberg.

CONCLUSIONS

The proposed comprehensive framework aims to structure the views and levels of systems analysis in the complex health care environment. Our first experiences support the usefulness of such a framework.

Systematic evaluation of computer-based nursing documentation

The documentation of the nursing process is an important, but often neglected part of clinical documentation. Paper-based systems have been introduced to support nursing process documentation. Frequently, however, problems, such as low quality, are reported and it is still unclear if computer-based documentation systems can reduce these problems. We therefore introduced a computer-based nursing documentation system on four wards of the University Hospitals of Heidelberg. We systematically evaluated its preconditions and its effects in a pre-test post-test intervention study. We combined objective data (e.g., based on quality checklists) with subjective data drawn from questionnaires and interviews. In this paper, we present preliminary results, focussing on detailed results from the first two wards.

Analysis and modelling of the multi-professional treatment process: preliminary results

This paper presents first results of a research project aimed at improving co-operative work initiatives in hospitals. A holistic analysis of the treatment process is presented as a precondition for process reengineering, quality measurements and improvement of multi-professional co-operation. Treatment process modelling attempts within the last years have concentrated on specialised points of views, such as business process modelling or communication modelling. In contrast, we have developed a framework consisting of several views of the treatment process. We tested our framework in a broad system analysis within the Department of Child and Adolescent Psychiatry of the Heidelberg University Hospitals. Our preliminary results support the framework. Weaknesses were described precisely in both the field of organisational procedures and information management.

Strategic information management plans: the basis for systematic information management in hospitals

Information management in hospitals is a complex task. In order to reduce complexity, we distinguish strategic, tactical, and operational information management. This is essential, because each of these information management levels views hospital information systems from different perspectives, and therefore uses other methods and tools. Since all these management activities deal only in part with computers, but mainly with human beings and their social behavior, we define a hospital information system as a sociotechnical subsystem of a hospital. Without proper strategic planning it would be a matter of chance, if a hospital information system would fulfil the information strategies goals. In order to support strategic planning and to reduce efforts for creating strategic plans, we propose a practicable structure.

Nursing process documentation systems in clinical routine--prerequisites and experiences

Documentation of the nursing process is an important, but often neglected part of clinical documentation. Paper-based systems have been introduced to support nursing process documentation. Frequently, however, problems, such as low quality and high writing efforts, are reported. However, it is still unclear if computer-based documentation systems can reduce these problems. At the Heidelberg University Medical Center, computer-based nursing process documentation projects began in 1998. A computer-based nursing documentation system has now been successfully introduced on four wards of three different departments, supporting all six phases of the nursing process. The introduction of the new documentation system was accompanied by systematic evaluations of prerequisites and consequences. In this paper, we present preliminary results of this evaluation, focusing on prerequisites of computer-based nursing process documentation. We will discuss in detail the creation and use of predefined nursing care plans as one important prerequisite for computer-based nursing documentation. We will also focus on acceptance issues and on organizational and technical issues.

Evaluation of clinical information systems. What can be evaluated and what cannot?

The evaluation of clinical information systems is essential as they are increasingly used in clinical routine and may even influence patient outcome on the basis of reminder functions and decision support. Therefore we try to answer three questions in this paper: what to evaluate; how to evaluate; how to interpret the results. Those key questions lead to the discussion of goals, methods and results of evaluation studies in a common context. We will compare the objectivist and the subjectivist evaluation approach and illustrate the evaluation process itself in some detail, discussing different phases of software development and potential evaluation techniques in each phase. We use four different practical examples of evaluation studies that were conducted in various settings to demonstrate how defined evaluation goals may be achieved with a limited amount of resources. This also illustrates advantages, limitations and costs of the different evaluation methods and techniques that may be used when evaluating clinical information systems.

Digital Libraries and Recent Medical Informatics Research. Findings from the IMIA Yearbook of Medical Informatics 2001

The Yearbook of Medical Informatics is published annually by the International Medical Informatics Association (IMIA) and contains a selection of recent excellent papers on medical informatics research (http://www.med.uni-heidelberg.de/mi/yearbook/index.htm). The special topic of the just published Yearbook 2001 is "Digital Libraries and Medicine". Digital libraries have changed dramatically and will continue to change the way we work with medical knowledge. The selected papers present recent research and new results on digital libraries. As usual, the Yearbook 2001 also contains a variety of papers on other subjects relevant to medical informatics, such as Electronic Patient Records, Health Information Systems, Health and Clinical Management, Decision Support Systems, Education, as well as Image and Signal Processing. This paper will briefly introduce the contributions covering digital libraries and will show how medical informatics research contributes to this important topic.

A randomized evaluation of a computer-based nursing documentation system

A two-month randomized, controlled trial based on 60 patients has been performed on a ward of the Department of Psychiatry at Heidelberg University Medical Center, Germany, to investigate the influence of computer-based nursing documentation on time investment for documentation, quality of documentation and user acceptance. Time measurements, questionnaires, documentation analysis and interviews were used to compare patients documented with the computer-based system (PIK group) with the control group (patients documented with the paper-based system). The results showed the advantages and disadvantages of computer-based nursing documentation. Time needed for nursing care planning was lower in the PIK group. Some formal aspects of quality were considerably better in the PIK group. On the other hand, time required for documentation of tasks and for report writing was greater in the PIK group. User acceptance increased significantly during the study. The interviews indicated a positive influence of PIK on the cooperation between nurses and physicians.

Analysis and modeling of the treatment process characterizing the cooperation within multi-professional treatment teams

We would like to introduce several aspects of the analysis and modeling of the treatment process characterizing the cooperation within multi-professional treatment teams. We will determine what is meant by a treatment process in order to then look at five views and four levels of their description. We will introduce possible methods for surveying and describing it. Currently an extensive analysis of the current state of the treatment process and of the weaknesses is underway in the Department of Child and Adolescent Psychiatry of the Heidelberg University Medical Center.

A compendium of information processing functions in nursing. Development and pilot study

Before introducing computers or other tools to support nursing care, it is important to have a clear picture of the information processing functions required in this area. This article presents a compendium of information processing functions used in nursing. This compendium can be used to support management of information systems in planning and monitoring nursing information systems. The article describes the development of the first version of this compendium and its evaluation in a pilot study. The results of this evaluation show that the compendium can be used to assess the quality of information processing in nursing.

Mobile information and communication tools in the hospital

Mobile information and communication systems in clinical routine have the potential to greatly improve communication, facilitate information access, eliminate double documentation, and increase quality of patient care in the long run. Projects to date have focused, for the most part, on highly specialized applications of the mobile computer. In our research project, 'Cooperative Problem Solving in Health Care', we have, among other things, designed a multifunctional mobile information and communication assistant. A prototype version of this system was implemented. This article outlines the close-to-reality evaluation of our prototype in a 1-week simulation study in a Heidelberg University hospital. We describe methods, aims, design and results of the simulation study, as well as discuss our methodology and the results we have obtained. We argue that the diverse requirements of different professional groups cannot be fulfilled by a single multifunctional device and propose, therefore, a 'multi-device mobile computer architecture'. Finally, we present consequences for the future computing infrastructure.