ASSESSING REFERRALS AND IMPROVING INFORMATION AVAILABILITY FOR CONSULTATIONS IN AN ACADEMIC ENDOCRINOLOGY CLINIC

A Data-Driven Algorithm to Recommend Initial Clinical Workup for Outpatient Specialty Referral: Algorithm Development and Validation Using Electronic Health Record Data and Expert Surveys

BACKGROUND

Millions of people have limited access to specialty care. The problem is exacerbated by ineffective specialty visits due to incomplete prereferral workup, leading to delays in diagnosis and treatment. Existing processes to guide prereferral diagnostic workup are labor-intensive (ie, building a consensus guideline between primary care doctors and specialists) and require the availability of the specialists (ie, electronic consultation).

OBJECTIVE

Using pediatric endocrinology as an example, we develop a recommender algorithm to anticipate patients' initial workup needs at the time of specialty referral and compare it to a reference benchmark using the most common workup orders. We also evaluate the clinical appropriateness of the algorithm recommendations.

METHODS

Electronic health record data were extracted from 3424 pediatric patients with new outpatient endocrinology referrals at an academic institution from 2015 to 2020. Using item co-occurrence statistics, we predicted the initial workup orders that would be entered by specialists and assessed the recommender's performance in a holdout data set based on what the specialists actually ordered. We surveyed endocrinologists to assess the clinical appropriateness of the predicted orders and to understand the initial workup process.

RESULTS

Specialists (n=12) indicated that <50% of new patient referrals arrive with complete initial workup for common referral reasons. The algorithm achieved an area under the receiver operating characteristic curve of 0.95 (95% CI 0.95-0.96). Compared to a reference benchmark using the most common orders, precision and recall improved from 37% to 48% (P<.001) and from 27% to 39% (P<.001) for the top 4 recommendations, respectively. The top 4 recommendations generated for common referral conditions (abnormal thyroid studies, obesity, amenorrhea) were considered clinically appropriate the majority of the time by specialists surveyed and practice guidelines reviewed.

CONCLUSIONS

 An item association-based recommender algorithm can predict appropriate specialists' workup orders with high discriminatory accuracy. This could support future clinical decision support tools to increase effectiveness and access to specialty referrals. Our study demonstrates important first steps toward a data-driven paradigm for outpatient specialty consultation with a tier of automated recommendations that proactively enable initial workup that would otherwise be delayed by awaiting an in-person visit.

Comparing the content of traditional faxed consultations to eConsults within an academic endocrinology clinic

Objective

To compare the content of traditional faxed referrals and electronic consultations (eConsults) and determine how many questions sent by traditional referral could be successfully addressed using eConsult.

Methods

We conducted a cross-sectional, qualitative study of eConsults and faxed referrals sent to a tertiary diabetes and endocrinology clinic in Ottawa, Ontario. A convenience sample of 300 faxed referrals sent between March and July 2017 and 300 eConsults submitted between January and December 2017 were selected and coded using an established taxonomy to determine question type. Two endocrinologists reviewed the faxed referrals to assess whether they could have been addressed using eConsult. Responses to a mandatory closeout survey were reviewed for all eConsults, assessing the case’s outcome, impact on decision to refer, and educational value.

Results

Most faxed consultations were requests for shared care in diabetes mellitus, whereas most eConsults requested help in diagnostic test interpretation. 25–27% of faxed consults were felt to be potentially amenable to eConsult. Referring provider behaviour was changed in 45.3% of eConsult cases through avoidance of face-to-face consultation.

Conclusion

eConsult is a promising tool for PCPs to improve access to specialist opinion without necessitating a face-to-face visit.

The Impact of Disruption of the Care Delivery System by Commercial Laboratory Testing in a Children's Health Care System

CONTEXT.—

Disruption of outpatient laboratory services by routing the samples to commercial reference laboratories may seem like a cost-saving measure by the payers, but results in hidden costs in quality and resources to support this paradigm.

OBJECTIVE.—

To identify differences when outpatient tests are performed at the Children's Healthcare of Atlanta (Children's) Hospital lab compared to a commercial reference lab, and the financial costs to support the reference laboratory testing.

DESIGN.—

Outpatient testing was sent to 3 different laboratories specified by the payer. Orders were placed in the Children's electronic health record, blood samples were drawn by the Children's phlebotomists, samples were sent to the testing laboratory, and results appeared in the electronic health record. Data comparing the time to result, cancelled samples, and cost to sustain the system of ordering and reporting were drawn from multiple sources, both electronic and manual.

RESULTS.—

The median time from phlebotomy to result was 0.7 hours for testing at the Children's lab and 20.72 hours for the commercial lab. The median time from result posting to caregiver acknowledgment was 5.4 hours for the Children's lab and 18 hours for the commercial lab. The commercial lab cancelled 2.7% of the tests; the Children's lab cancelled 0.8%. The financial cost to support online ordering and reporting for testing performed at commercial labs was approximately $640,000 per year.

CONCLUSIONS.—

Tangible monetary costs, plus intangible costs related to delayed results, occur when the laboratory testing system is disrupted.