What makes a high-quality electronic consultation (eConsult)? A nominal group study

The use of virtual nominal groups in healthcare research: An extended scoping review

INTRODUCTION

The Nominal Group Technique (NGT) is a consensus group method used to synthesize expert opinions. Given the global shift to virtual meetings, the extent to which researchers leveraged virtual platforms is unclear. This scoping review explores the use of the vNGT in healthcare research during the COVID-19 pandemic.

METHODS

Following the Arksey and O'Malley's framework, eight cross-disciplinary databases were searched (January 2020-July 2022). Research articles that reported all four vNGT stages (idea generation, round robin sharing, clarification, voting) were included. Media Synchronicity Theory informed analysis. Corresponding authors were surveyed for additional information.

RESULTS

Of 2,589 citations, 32 references were included. Articles covered healthcare (27/32) and healthcare education (4/32). Platforms used most were Zoom, MS Teams and GoTo but was not reported in 44% of studies. Only 22% commented on the benefits/challenges of moving the NGT virtually. Among authors who responded to our survey (16/32), 80% felt that the vNGT was comparable or superior.

CONCLUSIONS

The vNGT provides several advantages such as the inclusion of geographically dispersed participants, scheduling flexibility and cost savings. It is a promising alternative to the traditional in-person meeting, but researchers should carefully describe modifications, potential limitations, and impact on results.

ACcurate COnsensus Reporting Document (ACCORD) explanation and elaboration: Guidance and examples to support reporting consensus methods

BACKGROUND

When research evidence is limited, inconsistent, or absent, healthcare decisions and policies need to be based on consensus amongst interested stakeholders. In these processes, the knowledge, experience, and expertise of health professionals, researchers, policymakers, and the public are systematically collected and synthesised to reach agreed clinical recommendations and/or priorities. However, despite the influence of consensus exercises, the methods used to achieve agreement are often poorly reported. The ACCORD (ACcurate COnsensus Reporting Document) guideline was developed to help report any consensus methods used in biomedical research, regardless of the health field, techniques used, or application. This explanatory document facilitates the use of the ACCORD checklist.

METHODS AND FINDINGS

This paper was built collaboratively based on classic and contemporary literature on consensus methods and publications reporting their use. For each ACCORD checklist item, this explanation and elaboration document unpacks the pieces of information that should be reported and provides a rationale on why it is essential to describe them in detail. Furthermore, this document offers a glossary of terms used in consensus exercises to clarify the meaning of common terms used across consensus methods, to promote uniformity, and to support understanding for consumers who read consensus statements, position statements, or clinical practice guidelines. The items are followed by examples of reporting items from the ACCORD guideline, in text, tables and figures.

CONCLUSIONS

The ACCORD materials - including the reporting guideline and this explanation and elaboration document - can be used by anyone reporting a consensus exercise used in the context of health research. As a reporting guideline, ACCORD helps researchers to be transparent about the materials, resources (both human and financial), and procedures used in their investigations so readers can judge the trustworthiness and applicability of their results/recommendations.

Healthcare utilization impacts of an eConsult program for headache at an academic medical center

INTRODUCTION

Interprofessional consultations ("eConsults") can reduce healthcare utilization. However, the impact of eConsults on healthcare utilization remains poorly characterized among patients with headache.

METHODS

We performed a retrospective, 1:1 matched cohort study comparing patients evaluated for headache via eConsult request or in-person referral at the Mount Sinai Health System in New York. Groups were matched on clinical and demographic characteristics. Our primary outcome was one or more outpatient headache-related encounters in 6 months following referral date. Secondary outcomes included one or more all-cause outpatient neurology and headache-related emergency department (ED) encounters during the same period. We used univariable and multivariable logistic regression to model associations between independent variables and outcomes.

RESULTS

We identified 74 patients with headache eConsults who were matched to 74 patients with in-person referrals. Patients in the eConsult group were less likely to achieve the primary outcome (29.7% vs 62.2%, P < 0.0001) or have an all-cause outpatient neurology encounter (33.8% vs 79.7%, P < 0.0001) than patients in the comparison group. Both groups did not significantly differ by headache-related ED encounters. In multivariable analyses, patients in the eConsult group had significantly lower odds of having one or more headache-related or all-cause neurology encounters than patients in the comparison group (odds ratio (OR) 0.3, 95% confidence interval (CI) 0.1-0.6; OR 0.1, 95% CI 0.1-0.3, respectively).

DISCUSSION

In comparison to in-person referrals, eConsult requests for headache were associated with reduced likelihood of outpatient neurology encounters in the short-term but not with differential use of headache-related ED encounters. Larger-scale, prospective studies should validate our findings and assess patient outcomes.

Developing an Interprofessional e-Consult Checklist for Family Nurse Practitioner Students

ABSTRACT

Family nurse practitioners (FNPs) are filling the provider gaps in primary care. Primary care patients require referrals to different types of specialty providers that are not always readily available. The use of electronic consultations (e-consults) can fill this gap. FNPS may not be aware of e-consults or may feel they are not beneficial in their practice, likely because of a lack of training. This article details the development of an e-consult checklist for graduate FNPs.

Consensus-Based Development of an Assessment Tool: A Methodology for Patient Engagement in Primary Care and CPD Research

With cardiovascular disease (CVD) posing a significant disease burden in Canada and more broadly, preventative efforts which incorporate best evidence, patient preference, and physician expertise must continue to take place. Primary care providers play a pivotal role in this effort, and a greater understanding of patient perspectives is needed to guide management and inform training. We used a validated consensus method, the nominal group technique (NGT), to identify patient-reported experience measures (PREM) related to CVD prevention deemed most important by both patients and providers. The NGT was used by using structured discussions between patients and providers to bring ideas about PREM CVD outcomes to a consensus. Four patient partners and four primary care providers were selected to participate in an NGT session. Each participant wrote down items/questions they believed important in CVD preventative care. After discussions, all items underwent anonymous ranking on a 5-point scale. Items were included/excluded based on 75% agreement a priori. The panel produced 10 items from a total of 26 after 2 rounds of ranking. The top two items were as follows: "Is your treatment plan tailored to you" and "Was your physician good at giving information about your risk factors?" These results are significantly different compared with existing quality measures because they highlight aspects of patient experience and therapeutic relationship. A questionnaire consisting of prioritized PREM items is valuable in quality improvement and continuous professional development (CPD).

Effect of Peer Benchmarking on Specialist Electronic Consult Performance in a Los Angeles Safety-Net: a Cluster Randomized Trial

BACKGROUND

Since the advent of COVID-19, accelerated adoption of systems that reduce face-to-face encounters has outpaced training and best practices. Electronic consultations (eConsults), structured communications between PCPs and specialists regarding a case, have been effective in reducing face-to-face specialist encounters. As the health system rapidly adapts to multiple new practices and communication tools, new mechanisms to measure and improve performance in this context are needed.

OBJECTIVE

To test whether feedback comparing physicians to top performing peers using co-specialists' ratings improves performance.

DESIGN

Cluster-randomized controlled trial PARTICIPANTS: Eighty facility-specialty clusters and 214 clinicians INTERVENTION: Providers in the feedback arms were sent messages that announced their membership in an elite group of "Top Performers" or provided actionable recommendations with feedback for providers that were "Not Top Performers."

MAIN MEASURES

The primary outcomes were changes in peer ratings in the following performance dimensions after feedback was received: (1) elicitation of information from primary care practitioners; (2) adherence to institutional clinical guidelines; (3) agreement with peer's medical decision-making; (4) educational value; (5) relationship building.

KEY RESULTS

Specialists showed significant improvements on 3 of the 5 consultation performance dimensions: medical decision-making (odds ratio 1.52, 95% confidence interval 1.08-2.14, p<.05), educational value (1.86, 1.17-2.96) and relationship building (1.63, 1.13-2.35) (both p<.01).

CONCLUSIONS

The pandemic has shed light on clinicians' commitment to professionalism and service as we rapidly adapt to changing paradigms. Interventions that appeal to professional norms can help improve the efficacy of new systems of practice. We show that specialists' performance can be measured and improved with feedback using aspirational norms.

TRIAL REGISTRATION

clinicaltrials.gov NCT03784950.

Professional decision making with digitalisation of patient contacts in a medical advice setting: a qualitative study of a pilot project with a chat programme in Sweden

OBJECTIVES

Patient e-services are increasingly launched globally to make healthcare more efficient and digitalised. One area that is digitalised is medical advice, where patients asynchronously chat with nurses and physicians, with patients having filled in a form with predefined questions before the chat. This study aimed to explore how occupational professionalism and the possibility of professional judgement are affected when clinical patient contact is digitalised. The study's overall question concerns whether and how the scope of the healthcare staff's professional judgement and occupational professionalism are affected by digitalisation.

DESIGN AND SETTING

A qualitative study of healthcare professionals working in a pilot project with a chat programme for patients in a medical advice setting in Sweden.

PARTICIPANTS AND ANALYSIS

Contextual inquiries and 17 interviews with nurses (n=9) and physicians (n=8). The interviews were thematically analysed. The analysis was inductive and based on theories of decision making.

RESULTS

Three themes emerged: (1) Predefined questions to patients not tailored for healthcare professionals' work, (2) reduced trust in written communication and (3) reduced opportunity to obtain information through chat communication.

CONCLUSIONS

The results indicate that asynchronous chat with patients might reduce the opportunity for nurses and physicians to obtain and use professional knowledge and discretionary decision making. Furthermore, the system's design increases uncertainty in assessments and decision making, which reduces the range of occupational professionalism.

Optimizing e-Consultations to Adolescent Medicine Specialists: Qualitative Synthesis of Feedback From User-Centered Design

BACKGROUND

e-Consultations between primary care physicians and specialists are a valuable means of improving access to specialty care. Adolescents and young adults (AYAs) face unique challenges in accessing limited adolescent medicine specialty care resources, which contributes to delayed or forgone care. e-Consultations between general pediatricians and adolescent medicine specialists may alleviate these barriers to care. However, the optimal application of this model in adolescent medicine requires careful attention to the nuances of AYA care.

OBJECTIVE

This study aims to qualitatively analyze feedback obtained during the iterative development of an e-consultation system for communication between general pediatricians and adolescent medicine specialists tailored to the specific health care needs of AYAs.

METHODS

We conducted an iterative user-centered design and evaluation process in two phases. In the first phase, we created a static e-consultation prototype and storyboards and evaluated them with target users (general pediatricians and adolescent medicine specialists). In the second phase, we incorporated feedback to develop a functional prototype within the electronic health record and again evaluated this with general pediatricians and adolescent medicine specialists. In each phase, general pediatricians and adolescent medicine specialists provided think-aloud feedback during the use of the prototypes and semistructured exit interviews, which was qualitatively analyzed to identify perspectives related to the usefulness and usability of the e-consultation system.

RESULTS

Both general pediatricians (n=12) and adolescent medicine specialists (n=12) perceived the usefulness of e-consultations for AYA patients, with more varied perceptions of potential usefulness for generalist and adolescent medicine clinicians. General pediatricians and adolescent medicine specialists discussed ways to maximize the usability of e-consultations for AYAs, primarily by improving efficiency (eg, reducing documentation, emphasizing critical information, using autopopulated data fields, and balancing specificity and efficiency through text prompts) and reducing the potential for errors (eg, prompting a review of autopopulated data fields, requiring physician contact information, and prompting explicit discussion of patient communication and confidentiality expectations). Through iterative design, patient history documentation was streamlined, whereas documentation of communication and confidentiality expectations were enhanced.

CONCLUSIONS

Through an iterative user-centered design process, we identified user perspectives to guide the refinement of an e-consultation system based on general pediatrician and adolescent medicine specialist feedback on usefulness and usability related to the care of AYAs. Qualitative analysis of this feedback revealed both opportunities and risks related to confidentiality, communication, and the use of tailored documentation prompts that should be considered in the development and use of e-consultations with AYAs.

eConsult Specialist Quality of Response (eSQUARE): A novel tool to measure specialist correspondence via electronic consultation

High-quality correspondence between healthcare providers is critical for effective patient care. We developed an assessment tool to measure the quality of specialist correspondence to primary care providers (PCPs) via electronic consultation (eConsult), where specialists provide advice without specialist-patient interactions. We incorporated fourteen previously described features of high-quality eConsult correspondence into an assessment tool named the eConsult Specialist Quality of Response (eSQUARE). Six PCPs and two specialists applied the 10-item eSQUARE tool to 30 eConsults of varying quality as informed by PCP survey data. Content, response process, and internal structure validity evidence was gathered. Psychometric properties were calculated using descriptive statistics and generalizability analyses. Mean total score for low-quality eConsults (M = 24 ± 5.6) was significantly lower than moderate-quality eConsults (M = 38 ± 4.7; p<0.001) which was significantly lower than high-quality eConsults (M = 46 ± 3.0; p = 0.002). Reliability measures were high, including generalizability coefficient (0.96), inter-item (≥0.55) and item-total correlations (≥0.68). A decision study demonstrated that a single rater was adequate to achieve a reliability measure of ≥0.70. This study demonstrates initial validity evidence including multiple reliability measures for the eSQUARE. A single rater is adequate to achieve reliability measures for formative feedback. Future studies can apply the eSQUARE when planning educational initiatives aiming to improve specialist-to-PCP correspondence via eConsult.

Specialist Participation in e-Consult and e-Referral Services: Best Practices

Electronic consultations (eConsults) and referrals (eReferrals) are being implemented to improve access to specialist care. As eConsult and eReferral services rely on a roster of engaged specialists for their success, careful attention must be paid to how the term "specialist" is defined, what criteria inform specialists recruitment, and how quality of specialist responses can be monitored and maintained. Key considerations, informed by our personal experiences, review of best practice documents, international frameworks of specialists roles and competencies and a focused small group discussion among providers, health service planners and researchers for each of these important elements is discussed. Individuals participating in services should receive clear expectations around their role and responsibilities and be provided equitable access assuming they meet the necessary requirements. Training and feedback should be provided to ensure timely, quality responses. Paying attention to these key elements will reduce confusion, frustration and disengagement amongst specialists and ensure high quality responses.