Ensuring Best Practice in Genomic Education and Evaluation: A Program Logic Approach

Ensuring best practice in genomics education: A theory- and empirically informed evaluation framework

Implementation of genomic medicine into healthcare requires a workforce educated through effective educational approaches. However, ascertaining the impact of genomics education activities or resources is limited by a lack of evaluation and inconsistent descriptions in the literature. We aim to support those developing genomics education to consider how best to capture evaluation data that demonstrate program outcomes and effectiveness within scope. Here, we present an evaluation framework that is adaptable to multiple settings for use by genomics educators with or without education or evaluation backgrounds. The framework was developed as part of a broader program supporting genomic research translation coordinated by the Australian Genomics consortium. We detail our mixed-methods approach involving an expert workshop, literature review and iterative expert input to reach consensus and synthesis of a new evaluation framework for genomics education. The resulting theory-informed and evidence-based framework encompasses evaluation across all stages of education program development, implementation and reporting, and acknowledges the critical role of stakeholders and the effects of external influences.

Ensuring best practice in genomics education: A scoping review of genomics education needs assessments and evaluations

A health workforce capable of implementing genomic medicine requires effective genomics education. Genomics education interventions developed for health professions over the last two decades, and their impact, are variably described in the literature. To inform an evaluation framework for genomics education, we undertook an exploratory scoping review of published needs assessments for, and/or evaluations of, genomics education interventions for health professionals from 2000 to 2023. We retrieved and screened 4,659 records across the two searches with 363 being selected for full-text review and consideration by an interdisciplinary working group. 104 articles were selected for inclusion in the review-60 needs assessments, 52 genomics education evaluations, and eight describing both. Included articles spanned all years and described education interventions in over 30 countries. Target audiences included medical specialists, nurses/midwives, and/or allied health professionals. Evaluation questions, outcomes, and measures were extracted, categorized, and tabulated to iteratively compare measures across stages of genomics education evaluation: planning (pre-implementation), development and delivery (implementation), and impact (immediate, intermediate, or long-term outcomes). They are presented here along with descriptions of study designs. We document the wide variability in evaluation approaches and terminology used to define measures and note that few articles considered downstream (long-term) outcomes of genomics education interventions. Alongside the evaluation framework for genomics education, results from this scoping review form part of a toolkit to help educators to undertake rigorous genomics evaluation that is fit for purpose and can contribute to the growing evidence base of the contribution of genomics education in implementation strategies for genomic medicine.

A team-based approach to cardiogenomic education

While many genetic professionals are involved in the education of lay and professional audiences, most do not have formal training in education theory and program design. Partnerships with adult education experts can provide additional resources and improve the level of instruction, thereby increasing the impact of an educational intervention. This report discusses the experience of a multidisciplinary team of educators, clinicians, and researchers partnering to develop evidence-based education for cardiology practitioners. It includes practical advice for how clinicians and educators can develop more effective education through collaboration, needs assessment, instructional design, and iterative content development.

A cross-professional competency framework for communicating genomic results

To ensure genomic medicine is delivered safely and effectively, it is crucial that healthcare professionals are able to understand and communicate genomic results. This Education Innovation describes a nationally agreed, cross-professional competency framework outlining the knowledge, skills and behaviors required to communicate genomic results. Using principles of the nominal group technique, consensus meetings with clinical, scientific and educational experts identified six stages in the return of results process, drafted and iterated competencies. Competencies were then mapped across three levels to acknowledge different degrees of experiences and scopes of practice. The framework was open for consultation with healthcare professionals and patient communities before being published. The finalized framework includes six core competency statements required to communicate genomic results. This framework is designed to be a guide for best practice and a developmental tool to support individuals and organizations. It can be used by healthcare professionals, such as genetic counselors, to identify individual learning needs or to structure the development of training for other healthcare professionals who are increasingly involved in requesting and returning results for genomic tests.

Personalising genetic counselling (POETIC) trial: Protocol for a hybrid type II effectiveness-implementation randomised clinical trial of a patient screening tool to improve patient empowerment after cancer genetic counselling

BACKGROUND

Genetic counselling aims to identify, and address, patient needs while facilitating informed decision-making about genetic testing and promoting empowerment and adaptation to genetic information. Increasing demand for cancer genetic testing and genetic counsellor workforce capacity limitations may impact the quality of genetic counselling provided. The use of a validated genetic-specific screening tool, the Genetic Psychosocial Risk Instrument (GPRI), may facilitate patient-centred genetic counselling. The aim of this study is to assess the effectiveness and implementation of using the GPRI in improving patient outcomes after genetic counselling and testing for an inherited cancer predisposition.

METHODS

The PersOnalising gEneTIc Counselling (POETIC) trial is a hybrid type 2 effectiveness-implementation trial using a randomised control trial to assess the effectiveness of the GPRI in improving patient empowerment (primary outcome), while also assessing implementation from the perspective of clinicians and the healthcare service. Patients referred for a cancer risk assessment to the conjoint clinical genetics service of two metropolitan hospitals in Victoria, Australia, who meet the eligibility criteria and consent to POETIC will be randomised to the usual care or intervention group. Those in the intervention group will complete the GPRI prior to their appointment with the screening results available for the clinicians' use during the appointment. Appointment audio recordings, clinician-reported information about the appointment, patient-reported outcome measures, and clinical data will be used to examine the effectiveness of using the GPRI. Appointment audio recordings, health economic information, and structured interviews will be used to examine the implementation of the GPRI.

DISCUSSION

The POETIC trial takes a pragmatic approach by deploying the GPRI as an intervention in the routine clinical practice of a cancer-specific clinical genetics service that is staffed by a multidisciplinary team of genetics and oncology clinicians. Therefore, the effectiveness and implementation evidence generated from this real-world health service setting aims to optimise the relevance of the outcomes of this trial to the practice of genetic counselling while enhancing the operationalisation of the screening tool in routine practice.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry registration number 12621001582842p. Date of registration: 19th November 2021.

Australian Genomics: Outcomes of a 5-year national program to accelerate the integration of genomics in healthcare

Australian Genomics is a national collaborative partnership of more than 100 organizations piloting a whole-of-system approach to integrating genomics into healthcare, based on federation principles. In the first five years of operation, Australian Genomics has evaluated the outcomes of genomic testing in more than 5,200 individuals across 19 rare disease and cancer flagship studies. Comprehensive analyses of the health economic, policy, ethical, legal, implementation and workforce implications of incorporating genomics in the Australian context have informed evidence-based change in policy and practice, resulting in national government funding and equity of access for a range of genomic tests. Simultaneously, Australian Genomics has built national skills, infrastructure, policy, and data resources to enable effective data sharing to drive discovery research and support improvements in clinical genomic delivery.

The implementation of large-scale genomic screening or diagnostic programmes: A rapid evidence review

Genomic healthcare programmes, both in a research and clinical context, have demonstrated a pivotal opportunity to prevent, diagnose, and treat rare diseases. However, implementation factors could increase overall costs and affect uptake. As well, uncertainties remain regarding effective training, guidelines and legislation. The purpose of this rapid evidence review was to draw together the available global evidence on the implementation of genomic testing programmes, particularly on population-based screening and diagnostic programmes implemented at the national level, to understand the range of factors influencing implementation. This review involved a search of terms related to genomics, implementation and health care. The search was limited to peer-reviewed articles published between 2017-2022 and found in five databases. The review included thirty articles drawing on sixteen countries. A wide range of factors was cited as critical to the successful implementation of genomics programmes. These included having policy frameworks, regulations, guidelines; clinical decision support tools; access to genetic counselling; and education and training for healthcare staff. The high costs of implementing and integrating genomics into healthcare were also often barriers to stakeholders. National genomics programmes are complex and require the generation of evidence and addressing implementation challenges. The findings from this review highlight that there is a strong emphasis on addressing genomic education and engagement among varied stakeholders, including the general public, policymakers, and governments. Articles also emphasised the development of appropriate policies and regulatory frameworks to govern genomic healthcare, with a focus on legislation that regulates the collection, storage, and sharing of personal genomic data.

Teaching graduate research skills in genomics via an integrated 'flipped' journal club program

Journal clubs are well regarded as a highly effective means of engaging graduate students with the contemporary research literature, where individual students prepare and deliver presentations on selected research articles to their peers, followed by a group discussion. Regular journal clubs have the advantage of enhancing student scientific reading, assessment and communication skills as well as developing a better understanding of the field. We developed a flipped journal club program as part of the one semester module 'Genomics Research Methods' with the goal of enhancing-and quantifying-individual student ability to engage with the genomics scientific literature. This involves all students and faculty reviewing a given manuscript, with the former submitting research relevant questions they would wish to ask the presenting student at the journal club, and the latter grading them. These questions are then ranked based on their median grade, and subsequently discussed in class. This cycle repeats weekly until all students have presented. Our analysis of question grade data over three consecutive years demonstrated clear improvements in student performance for all students between the start and end of the module. While no difference in performance was noted based on gender over the full semester, improvement in performance was significantly evident for the female cohort between the start and end of the module. Our results are consistent with module survey feedback of overall reported enhanced research self-efficacy. This demonstrates that this flipped journal club implementation is a highly effective means of both assessing and improving individual student learning in genomics research ability. The involvement of the teaching faculty furthermore offers a means to foster a dynamic research community for all participants involved. This methodology is easily transferable to other bioscience graduate/undergraduate programs seeking to effectively teach essential research ability skills and enhance student self-efficacy.

A nationally agreed cross-professional competency framework to facilitate genomic testing

PURPOSE

The study aimed to develop a nationally agreed, cross-professional competency framework outlining the knowledge, skills, and behaviors required to facilitate genomic tests.

METHODS

Using principles of the nominal group technique, a consensus meeting with 25 experts mapped themes to an initial framework and voted on areas of inconsistency. A revised framework was open for consultation with health care professionals and patient communities before being published. An evaluation, using an online survey, was conducted to explore early use and factors to facilitate adoption of the framework.

RESULTS

The framework identified 8 competencies required to facilitate genomic tests. The evaluation (239 survey responses from health care professionals) indicated that the framework addresses a timely need among users and identified ways to improve awareness and accessibility for different health care professional groups.

CONCLUSION

This framework can be used as a guide for best practice by health care professionals who request genomic tests. It can also provide a foundation to identify learning needs and structure training such that conversations about genomic testing can be delivered in a consistent manner across specialties. These competencies can also be used as a reference to evaluate how consent is facilitated in different specialty areas to enhance the responsible delivery of genomic medicine.

Genetic and genomic learning needs of oncologists and oncology nurses in the era of precision medicine: a scoping review

Genetic and genomic data are increasingly guiding clinical care for cancer patients. To meet the growing demand for precision medicine, patient-facing oncology staff will be a part of leading the provision of genomic testing. A scoping review was undertaken to identify the range of genetic and genomic learning needs of oncologists and oncology nurses. Learning needs were reported relating to interpretation of genomic data, clinical decision-making, patient communication and counseling, and fundamentals of genetics and genomics. There was a lack of empirical research specific to oncology nurses and their learning needs in tumor sequencing. Our findings suggest that oncologists and oncology nurses need tailored support, education and training to improve their confidence and skills in adopting genomic testing into clinical practice.

Supporting teachers to use genomics as a context in the classroom: an evaluation of learning resources for high school biology

As genomics becomes embedded into healthcare, public genomic health literacy is critical to support decision-making for personal and family health decisions and enable citizens to engage with related social issues. School science education has the potential to establish the foundations of genetic and genomic literacy. The concept of literacy extends beyond conceptual understanding of biological principles to familiarity with the applications and implications of genetics, critical thinking skills, and socioscientific reasoning. We developed and evaluated a suite of resources for teaching genetics and genomics in the Australian senior biology syllabus for students aged 16-18 years. The aim was to increase teachers' knowledge and confidence to teach genetic and genomic content, and their capacity to develop robust genetic literacy in their students. Resources, including an inquiry-based task and five associated lesson plans, were developed and made freely available to teachers online. Evaluation was undertaken between December 2019 and March 2020 with a post-use survey emailed to teachers who had accessed the resources. The 56 teachers who responded rated the resources as high quality, engaging, and well-aligned with the syllabus. Teachers who used the resources self-reported increases in their knowledge and confidence in teaching. They also perceived positive outcomes in their students, reporting that the resources deepened their students understanding of genetic concepts, helped them to consider social and ethical issues, and developed their higher order thinking skills. Findings may inform future interactions with high schools to improve genetic literacy.

Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE2 Genomics)

PURPOSE

Widespread, quality genomics education for health professionals is required to create a competent genomic workforce. A lack of standards for reporting genomics education and evaluation limits the evidence base for replication and comparison. We therefore undertook a consensus process to develop a recommended minimum set of information to support consistent reporting of design, development, delivery, and evaluation of genomics education interventions.

METHODS

Draft standards were derived from literature (25 items from 21 publications). Thirty-six international experts were purposively recruited for three rounds of a modified Delphi process to reach consensus on relevance, clarity, comprehensiveness, utility, and design.

RESULTS

The final standards include 18 items relating to development and delivery of genomics education interventions, 12 relating to evaluation, and 1 on stakeholder engagement.

CONCLUSION

These Reporting Item Standards for Education and its Evaluation in Genomics (RISE2 Genomics) are intended to be widely applicable across settings and health professions. Their use by those involved in reporting genomics education interventions and evaluation, as well as adoption by journals and policy makers as the expected standard, will support greater transparency, consistency, and comprehensiveness of reporting. Consequently, the genomics education evidence base will be more robust, enabling high-quality education and evaluation across diverse settings.

Mainstreaming genomics: training experience of hospital medical officers at the Royal Melbourne Hospital

The rapid evolution and wide applicability of genomic testing means that medical practitioners outside the field are not appropriately skilled to understand the utility of genomics for their patients. Rotating junior doctors through genomic medicine provides them with the hands-on experience necessary to understand the complexities in this field. In this study, we analysed the training experience of 12 hospital medical officers who rotated through genomic medicine at the Royal Melbourne Hospital. Here, we demonstrate that immersion in clinical genomics aids in mainstreaming genomics knowledge.

Preparing Medical Specialists for Genomic Medicine: Continuing Education Should Include Opportunities for Experiential Learning

With the demand for genomic investigations increasing, medical specialists will need to, and are beginning to, practice genomic medicine. The need for medical specialists from diverse specialties to be ready to appropriately practice genomic medicine is widely recognised, but existing studies focus on single specialties or clinical settings. We explored continuing education needs in genomic medicine of a wide range of medical specialists (excluding genetic specialists) from across Australia. Interviews were conducted with 86 medical specialists in Australia from diverse medical specialties. Inductive content analysis categorized participants by career stage and genomics experience. Themes related to education needs were identified through constant comparison and discussion between authors of emerging concepts. Our findings show that participants believe that experiential learning in genomic medicine is necessary to develop the confidence and skills needed for clinical care. The main themes reported are: tailoring of education to the specialty and the individual; peer interactions contextualizes knowledge; experience will aid in developing confidence and skills. In fact, avenues of gaining experience may result in increased engagement with continuing education in genomic medicine as specialists are exposed to relevant applications in their clinical practice. Participants affirmed the need for continuing education in genomic medicine but identified that it would need to be tailored to the specialty and the individual: one size does not fit all, so a multifaceted approached is needed. Participants infrequently attended formal continuing education in genomic medicine. More commonly, they reported experiential learning by observation, case-review or interacting with a "genomics champion" in their specialty, which contextualized their knowledge. Medical specialists anticipate that genomic medicine will become part of their practice which could lessen demand on the specialist genetic workforce. They expect to look to experts within their own medical specialty who have gained genomics expertise for specific and contextualized support as they develop the skills and confidence to practice genomic medicine. These findings highlight the need to include opportunities for experiential learning in continuing education. Concepts identified in these interviews can be tested with a larger sample of medical specialists to ascertain representativeness.

Development of an Evidence-Based, Theory-Informed National Survey of Physician Preparedness for Genomic Medicine and Preferences for Genomics Continuing Education

Despite some early implementation of genomic medicine globally, there is a lack of rigorous, large-scale assessments of medical specialists' current practice and continuing education needs. As a first step to addressing this gap, we describe the development of a robust, expert-reviewed, survey using a mixed-methods sequential study design. We conducted semi-structured qualitative interviews with 32 education providers and 86 non-genetic medical specialists about current genomic medicine practice and need for continuing education. Key concepts were identified and used as an initial framework for the survey. These were: personal characteristics (medical specialty, years of practice); current practice of genomics in clinical and research settings; perception of how proximal genomic medicine is to practice; perception of preparedness (competence and confidence); and, preferences for future roles and models of care in genomic medicine and for continuing education. Potential survey questions that related to at least one of these concepts were identified from the literature or were created if no suitable question existed. Using a modified, reactive Delphi approach, questions were reviewed by a panel of 22 experts. Experts were selected purposefully representing four areas of expertise: non-genetic medical specialties; clinical genetics; genetic/genomic education and evaluation; and implementation science. Three Delphi rounds assessed relevance, clarity and importance of each question. The questions were also mapped to the behaviour change wheel theoretical framework which encompasses capability, opportunity and motivation (COM-B). The survey (included as supplementary material) was then tested with a small group of non-genetic medical specialists and feedback was written or verbal in 'talk-aloud', cognitive interviews. The final survey was then piloted with a further 29 specialists. We describe the methodology to create a robust, data- and theory-informed survey. The final survey captures not only levels of experience, practice of genomics and preferences for education but also the challenges around engaging with education. Survey data will provide evidence for education providers to inform development of education which meets learner needs and contributes to a medical workforce that is literate in genomics and more confident to competently practice genomic medicine.