Combining tabular, rule-based, and procedural knowledge in computer-based guidelines for childhood immunization

Evaluation of a clinical decision support algorithm for patient-specific childhood immunization

OBJECTIVE

To evaluate the effectiveness of a clinical decision support system (CDSS) implementing standard childhood immunization guidelines, using real-world patient data from the Regenstrief Medical Record System (RMRS).

METHODS

Study subjects were age 6-years or younger in 2008 and had visited the pediatric clinic on the campus of Wishard Memorial Hospital. Immunization records were retrieved from the RMRS for 135 randomly selected pediatric patients. We compared vaccine recommendations from the CDSS for both eligible and recommended timelines, based on the child's date of birth and vaccine history, to recommendations from registered nurses who routinely selected vaccines for administration in a busy inner city hospital, using the same date of birth and vaccine history. Aggregated and stratified agreement and Kappa statistics were reported. The reasons for disagreement between suggestions from the CDSS and nurses were also identified.

RESULTS

For the 135 children, a total of 1215 vaccination suggestions were generated by nurses and were compared to the recommendations of the CDSS. The overall agreement rates were 81.3% and 90.6% for the eligible and recommended timelines, respectively. The overall Kappa values were 0.63 for the eligible timeline and 0.80 for the recommended timeline. Common reasons for disagreement between the CDSS and nurses were: (1) missed vaccination opportunities by nurses, (2) nurses sometimes suggested a vaccination before the minimal age and minimal waiting interval, (3) nurses usually did not validate patient immunization history, and (4) nurses sometimes gave an extra vaccine dose.

CONCLUSION

Our childhood immunization CDSS can assist providers in delivering accurate childhood vaccinations.

Decision tables and rule engines in organ allocation systems for optimal transparency and flexibility

Organ allocation systems have become complex and difficult to comprehend. We introduced decision tables to specify the rules of allocation systems for different organs. A rule engine with decision tables as input was tested for the Kidney Allocation System (ETKAS). We compared this rule engine with the currently used ETKAS by running 11,000 historical match runs and by running the rule engine in parallel with the ETKAS on our allocation system. Decision tables were easy to implement and successful in verifying correctness, completeness, and consistency. The outcomes of the 11,000 historical matches in the rule engine and the ETKAS were exactly the same. Running the rule engine simultaneously in parallel and in real time with the ETKAS also produced no differences. Specifying organ allocation rules in decision tables is already a great step forward in enhancing the clarity of the systems. Yet, using these tables as rule engine input for matches optimizes the flexibility, simplicity and clarity of the whole process, from specification to the performed matches, and in addition this new method allows well controlled simulations.

Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population

OBJECTIVES

The objective of this study was to test the hypothesis that clinical alerts for routine pediatric vaccinations within an electronic health record would reduce missed opportunities for vaccination and improve immunization rates for young children in an inner-city population.

METHODS

A 1-year intervention study (September 1, 2004, to August 31, 2005) with historical controls was conducted in 4 urban primary care centers affiliated with an academic medical center. All children who were younger than 24 months were enrolled. Electronic health record-based clinical reminders for routine childhood vaccinations were programmed to appear prominently at every patient encounter with vaccines due. The main outcome measures were rates of captured immunization opportunities and overall immunization rates at 24 months of age.

RESULTS

Immunization alerts appeared at 15,928 visits during the intervention. Alert implementation was associated with increases in captured immunization opportunities from 78.2% to 90.3% at well visits and from 11.3% to 32.0% at sick visits. Adjusted up-to-date immunization rates at 24 months of age increased from 81.7% to 90.1% from the control to intervention period. Children in the intervention group also became up-to-date earlier than control patients. Patient characteristics were stable throughout the study.

CONCLUSIONS

An electronic health record-based clinical alert intervention was associated with increases in captured opportunities for vaccination at both sick and well visits and significant improvements in immunization rates at 2 years of age. As electronic health records become more common in medical practice, such systems may transform immunization delivery to children.

Encoding a clinical practice guideline using guideline interchange format: a case study of a depression screening and management guideline

PURPOSE

Clinical practice guidelines (CPGs) are common tools for clinicians in daily practice. In order to use CPGs effectively at the point of care, representing CPGs into computer-interpretable format is essential. Since computer-interpretable guidelines (CIGs) have been reported to increase clinicians' usage of guidelines and improve patient's outcomes, it is critical to assess health care knowledge translated from CPGs into CIGs. The overall goal of this study was to illustrate the steps involved in encoding a guideline in guideline interchange format-3 (GLIF3) through a case study of a depression screening and management CPG for a nursing decision support system (DSS).

METHODS

This study consisted of five steps: (1) Selection of a CPG; (2) extraction and categorization in GLIF3 of concepts related to depression screening and management tasks; (3) converting GLIF3 steps to a flowchart; (4) creation of scenarios representing possible execution paths; (5) guideline execution engine (GLEE) execution of scenarios.

DISCUSSION

This study contributes to the body of knowledge regarding creation of CIGs and the use of GLEE as an evaluation tool for the encoded CIG in GLIF format for a depression CPG. CPG representation using GLIF3 and its evaluation by GLEE are useful methods to prepare nursing CPGs for implementation in a DSS.

Maintaining and incrementally revalidating a computer-based clinical guideline: a case study

The paper explores the issues involved in maintaining the logic within a complex computer-based clinical guideline, using as a case study IMM/Serve, an operational guideline whose domain is childhood immunization. For a period of more than a year and a half, we have maintained a log of (1) the national changes to the immunization recommendations, (2) the local customizations of IMM/Serve's logic, and (3) certain logic problems that arose in the process of accommodating these changes and customizations. We describe the nature of these changes, customizations, and problems. We also discuss how different types of domain knowledge might assist in the automated process of validating successive versions of the logic. The paper's goal is to use the immunization domain to provide specific examples of the issues and problems that arise in maintaining a computer-based clinical guideline.

Domain-constrained generation of clinical condition sets to help test computer-based clinical guidelines

The paper describes T/Gen, a prototype computer-based tool designed to help maintain the knowledge in a computer-based clinical practice guideline that provides patient-specific recommendations. T/Gen takes as input a set of clinical conditions to which a guideline must react, and allows the user to specify domain-specific constraints as to which combinations of conditions do not make sense or do not need to be exhaustively tested against one another. T/Gen automatically generates constrained sets of combinations of clinical conditions, each corresponding to a clinical case (or to several closely related clinical cases) that can be used to help test the computer-based guideline. The combinations can be used to test the guideline logic using T/Gen's built-in logic interpreter, or to generate a set of test cases for use in testing an operational guideline system. T/Gen has been developed and tested with five pilot guidelines, for two childhood immunization series, for influenza vaccination, for primary thyroid screening, and for embryo transplantation. The paper describes how T/Gen's approach is implemented for the five pilot guidelines, outlines the current status and future directions of the project, and discusses the design issues that arose in the course of carrying out the work.

IMM/Scrub: a domain-specific tool for the deduplication of vaccination history records in childhood immunization registries

IMM/Scrub is a pilot tool developed to assist in the deduplication of vaccination history records in childhood immunization registries. This problem is complicated by a number of factors including that fact that: (1) some doses are numbered and some are not, (2) doses may have different dose numbers, (3) doses may specify different preparations within a vaccine series, (4) one dose may indicate a combination vaccine and the other dose may specify one component of that combination, (5) two doses may have slightly different dates, and (6) combinations of any of these problems may occur together. IMM/Scrub is designed to help detect 10 different types of vaccination dose duplicates and also allows the user to specify flexibly the conditions in which a duplicate dose might be automatically eliminated. In addition, IMM/Scrub is linked to the IMM/Serve immunization forecasting program, which can provide additional assistance in the data cleaning process. The paper describes (1) the design of the current pilot implementation of IMM/Scrub, (2) the lessons learned during its implementation, and (3) our preliminary experience applying it to data from three immunization databases, from a state, a metropolitan area, and an academic medical center.

Tools for immunization guideline knowledge maintenance. I. Automated generation of the logic "kernel" for immunization forecasting

IMM/Def is a prototype computer program designed to facilitate the building and maintenance of a rule-based program which performs childhood immunization forecasting. An immunization forecasting program takes as input a child's immunization history and produces recommendations as to which vaccinations are due and which should be scheduled next. A significant amount of the knowledge required for immunization forecasting can be expressed in tabular form, including the parameters that indicate the minimum age when each dose may be given and the minimum intervals between doses. The choice of which of these sets of parameters apply to a particular case depends upon additional clinical logic. To perform forecasting, this logic must be applied in three temporal contexts: (1) a dose is due now, (2) a dose is not yet due, and (3) a dose must be scheduled to follow a dose which is due now. Building and maintaining this logic by hand is a formidable challenge. IMM/Def demonstrates how this task can be simplified by first defining immunization "definition logic" which can be automatically translated into if-then rules for each of the three contexts. The approach has been applied successfully to the six childhood vaccination series which are routinely administered. A key advantage is that IMM/Def allows one to have two specifications of the logic that can be examined independently and that can be cross-checked to help assure completeness, consistency, and accuracy of the logic. The paper describes how IMM/Def performs its translation and discusses several design issues and lessons learned.

Tools for immunization guideline knowledge maintenance. II. Automated Web-based generation of user-customized test cases

IMM/Test is a prototype software tool built to generate test cases that can be used to help test and verify the internal logic of an immunization forecasting program. A forecasting program takes as input a child's immunization history and produces recommendations as to which vaccinations are due and which should be scheduled next. IMM/Test was developed to test a specific immunization forecasting program, IMM/Serve. In addition, IMM/Test has been incorporated into a broader Web-based tool, IMM/Web, which allows the user (e.g., a member of an immunization registry staff) to customize the parameters used for immunization forecasting (e.g., the minimum ages for each dose and the minimum wait intervals between doses) to reflect local practice. IMM/Web then generates a customized set of test cases that may be used to test the user's immunization forecasting program. The user may also request that the test cases be automatically passed to IMM/Serve to analyze using the newly defined parameters. The paper describes the internal design of IMM/Test and IMM/Web and discusses certain lessons learned in the implementation of the two programs.