1
|
Munce SEP, Wong E, Luong D, Rao J, Cunningham J, Bailey K, John T, Barber C, Batthish M, Chambers K, Cleverley K, Crabtree M, Diaz S, Dimitropoulos G, Gorter JW, Grahovac D, Grimes R, Guttman B, Hébert ML, Henze M, Higgins A, Khodyakov D, Li E, Lo L, Macgregor L, Mooney S, Severino SM, Mukerji G, Penner M, Pidduck J, Shulman R, Stromquist L, Trbovich P, Wan M, Williams L, Yates D, Toulany A. Patient, caregiver and other knowledge user engagement in consensus-building healthcare initiatives: a scoping review protocol. BMJ Open 2024; 14:e080822. [PMID: 38719333 PMCID: PMC11086512 DOI: 10.1136/bmjopen-2023-080822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 04/10/2024] [Indexed: 05/12/2024] Open
Abstract
INTRODUCTION Patient engagement and integrated knowledge translation (iKT) processes improve health outcomes and care experiences through meaningful partnerships in consensus-building initiatives and research. Consensus-building is essential for engaging a diverse group of experienced knowledge users in co-developing and supporting a solution where none readily exists or is less optimal. Patients and caregivers provide invaluable insights for building consensus in decision-making around healthcare, policy and research. However, despite emerging evidence, patient engagement remains sparse within consensus-building initiatives. Specifically, our research has identified a lack of opportunity for youth living with chronic health conditions and their caregivers to participate in developing consensus on indicators/benchmarks for transition into adult care. To bridge this gap and inform our consensus-building approach with youth/caregivers, this scoping review will synthesise the extent of the literature on patient and other knowledge user engagement in consensus-building healthcare initiatives. METHODS AND ANALYSIS Following the scoping review methodology from Joanna Briggs Institute, published literature will be searched in MEDLINE, EMBASE, CINAHL and PsycINFO databases from inception to July 2023. Grey literature will be hand-searched. Two independent reviewers will determine the eligibility of articles in a two-stage process, with disagreements resolved by a third reviewer. Included studies must be consensus-building studies within the healthcare context that involve patient engagement strategies. Data from eligible studies will be extracted and charted on a standardised form. Abstracted data will be analysed quantitatively and descriptively, according to specific consensus methodologies, and patient engagement models and/or strategies. ETHICS AND DISSEMINATION Ethics approval is not required for this scoping review protocol. The review process and findings will be shared with and informed by relevant knowledge users. Dissemination of findings will also include peer-reviewed publications and conference presentations. The results will offer new insights for supporting patient engagement in consensus-building healthcare initiatives. PROTOCOL REGISTRATION https://osf.io/beqjr.
Collapse
Affiliation(s)
- Sarah E P Munce
- KITE Toronto Rehabilitation Institute, University Health Network, Toronto, Ontario, Canada
- Rehabilitation Sciences Institute, University of Toronto, Toronto, Ontario, Canada
| | - Elliott Wong
- University of Toronto Temerty Faculty of Medicine, Toronto, Ontario, Canada
- The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Dorothy Luong
- KITE Toronto Rehabilitation Institute, University Health Network, Toronto, Ontario, Canada
| | - Justin Rao
- University of Toronto Temerty Faculty of Medicine, Toronto, Ontario, Canada
| | - Jessie Cunningham
- Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Katherine Bailey
- University of Toronto Temerty Faculty of Medicine, Toronto, Ontario, Canada
- University of Toronto Institute of Health Policy Management and Evaluation, Toronto, Ontario, Canada
| | - Tomisin John
- The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Claire Barber
- Medicine, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
| | | | - Kyle Chambers
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
| | - Kristin Cleverley
- Centre for Addiction and Mental Health Queen Street Site, Toronto, Ontario, Canada
- Lawrence Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada
| | - Marilyn Crabtree
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Provincial Council for Maternal and Child Health, Toronto, Ontario, Canada
| | - Sanober Diaz
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Provincial Council for Maternal and Child Health, Toronto, Ontario, Canada
| | - Gina Dimitropoulos
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Faculty of Social Work, University of Calgary, Calgary, Alberta, Canada
| | - Jan Willem Gorter
- CanChild Centre for Childhood Disability Research, Department of Pediatrics, McMaster University, Hamilton, Southern Ontario, Canada
- Department of Rehabilitation, Physical Therapy Science & Sports, Wilhelmina Children's Hospital University Medical Centre, Utrecht, The Netherlands
| | - Danijela Grahovac
- National Health Hub in Transition, Children's Healthcare Canada, Hamilton, Southern Ontario, Canada
- CanChild Centre for Childhood Disability Research, McMaster University, Hamilton, Southern Ontario, Canada
| | - Ruth Grimes
- Canadian Pediatric Society, Winnipeg, Manitoba, Canada
| | - Beverly Guttman
- Provincial Council for Maternal and Child Health, Toronto, Ontario, Canada
| | - Michèle L Hébert
- Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, Alberta, Canada
| | - Megan Henze
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Surrey Place Centre, Toronto, Ontario, Canada
| | - Amanda Higgins
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- IWK Health Centre, Halifax, Nova Scotia, Canada
| | | | - Elaine Li
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
| | - Lisha Lo
- University of Toronto Centre for Quality Improvement and Patient Safety, Toronto, Ontario, Canada
| | - Laura Macgregor
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Martin Luther University College, Waterloo, Ontario, Canada
| | - Sarah Mooney
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Stollery Children's Hospital, Edmonton, Alberta, Canada
| | - Samadhi Mora Severino
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- School of Health Policy and Management, York University, Toronto, Ontario, Canada
| | - Geetha Mukerji
- Department of Medicine, Women's College Hospital, Toronto, Ontario, Canada
| | - Melanie Penner
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- Holland Bloorview Kids Rehabilitation Hospital, Toronto, Ontario, Canada
| | - Jacklynn Pidduck
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- IWK Health Centre, Halifax, Nova Scotia, Canada
| | - Rayzel Shulman
- University of Toronto Temerty Faculty of Medicine, Toronto, Ontario, Canada
- Division of Endocrinology and Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada
| | - Lisa Stromquist
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- National Health Hub in Transition, Children's Healthcare Canada, Ottawa, Ontario, Canada
| | - Patricia Trbovich
- University of Toronto Institute of Health Policy Management and Evaluation, Toronto, Ontario, Canada
- Patient Safety and Quality Improvement, North York General Hospital, Toronto, Ontario, Canada
| | - Michelle Wan
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
| | - Laura Williams
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
- University Health Network, Toronto, Ontario, Canada
| | - Darryl Yates
- The Hospital for Sick Children, Toronto, Ontario, Canada
- Integrated Knowledge Translation Panel Member, Toronto, Ontario, Canada
| | - Alene Toulany
- University of Toronto Temerty Faculty of Medicine, Toronto, Ontario, Canada
- Division of Adolescent Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
| |
Collapse
|
2
|
Ramage ER, Burke M, Galloway M, Graham ID, Janssen H, Marsden DL, Patterson AJ, Pollack M, Said CM, Lynch EA, English C. Fit for purpose. Co-production of complex behavioural interventions. A practical guide and exemplar of co-producing a telehealth-delivered exercise intervention for people with stroke. Health Res Policy Syst 2022; 20:2. [PMID: 34980156 PMCID: PMC8722305 DOI: 10.1186/s12961-021-00790-2] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2021] [Accepted: 11/08/2021] [Indexed: 12/12/2022] Open
Abstract
Background Careful development of interventions using principles of co-production is now recognized as an important step for clinical trial development, but practical guidance on how to do this in practice is lacking. This paper aims (1) provide practical guidance for researchers to co-produce interventions ready for clinical trial by describing the 4-stage process we followed, the challenges experienced and practical tips for researchers wanting to co-produce an intervention for a clinical trial; (2) describe, as an exemplar, the development of our intervention package. Method We used an Integrated Knowledge Translation (IKT) approach to co-produce a telehealth-delivered exercise program for people with stroke. The 4-stage process comprised of (1) a start-up planning phase with the co-production team. (2) Content development with knowledge user informants. (3) Design of an intervention protocol. (4) Protocol refinement. Results and reflections The four stages of intervention development involved an 11-member co-production team and 32 knowledge user informants. Challenges faced included balancing conflicting demands of different knowledge user informant groups, achieving shared power and collaborative decision making, and optimising knowledge user input. Components incorporated into the telehealth-delivered exercise program through working with knowledge user informants included: increased training for intervention therapists; increased options to tailor the intervention to participant’s needs and preferences; and re-naming of the program. Key practical tips include ways to minimise the power differential between researchers and consumers, and ensure adequate preparation of the co-production team. Conclusion Careful planning and a structured process can facilitate co-production of complex interventions ready for clinical trial. Graphical Abstract ![]()
Supplementary Information The online version contains supplementary material available at 10.1186/s12961-021-00790-2.
Collapse
Affiliation(s)
- Emily R Ramage
- School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Newcastle, Australia. .,Department of Physiotherapy, Western Health, Furlong Rd, St Albans, Australia. .,Australian Institute for Musculoskeletal Science, Furlong Rd, St Albans, Australia. .,Centre for Research Excellence in Stroke Recovery and Rehabilitation, Florey Institute of Neuroscience and Hunter Medical Research Institute, Parkville, Australia.
| | | | - Margaret Galloway
- School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Newcastle, Australia.,Centre for Research Excellence in Stroke Recovery and Rehabilitation, Florey Institute of Neuroscience and Hunter Medical Research Institute, Parkville, Australia
| | - Ian D Graham
- School of Epidemiology and Public Health and School of Nursing, University of Ottawa, Ottawa, ON, Canada.,Ottawa Hospital Research Institute, Ottawa, ON, Canada
| | - Heidi Janssen
- School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Newcastle, Australia.,Centre for Research Excellence in Stroke Recovery and Rehabilitation, Florey Institute of Neuroscience and Hunter Medical Research Institute, Parkville, Australia.,Hunter Stroke Services, Hunter New England Local Health District, Newcastle, NSW, Australia
| | - Dianne L Marsden
- School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Newcastle, Australia.,Hunter Stroke Services, Hunter New England Local Health District, Newcastle, NSW, Australia
| | - Amanda J Patterson
- School of Health Sciences and Priority Research Centre for Physical Activity and Nutrition, University of Newcastle, Newcastle, Australia
| | - Michael Pollack
- School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Newcastle, Australia.,Hunter Stroke Services, Hunter New England Local Health District, Newcastle, NSW, Australia.,Conjoint Associate Professor School of Health and Medical Sciences, Uni of Newcastle, Newcastle, Australia.,Rehabilitation Medicine, John Hunter Hospital, New Lambton Heights, Australia.,Centre for Rehab Innovations, Uni of Newcastle, Newcastle, Australia
| | - Catherine M Said
- Department of Physiotherapy, Western Health, Furlong Rd, St Albans, Australia.,Australian Institute for Musculoskeletal Science, Furlong Rd, St Albans, Australia.,University of Melbourne, Parkville, Australia
| | - Elizabeth A Lynch
- Research Fellow, Adelaide Nursing School, University of Adelaide, Adelaide, Australia.,Matthew Flinders Research Fellow, College of Nursing and Health Sciences, Flinders University, Adelaide, Australia
| | - Coralie English
- School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Newcastle, Australia.,Centre for Research Excellence in Stroke Recovery and Rehabilitation, Florey Institute of Neuroscience and Hunter Medical Research Institute, Parkville, Australia
| |
Collapse
|
3
|
Moroz I, Archibald D, Breton M, Cote-Boileau E, Crowe L, Horsley T, Hyseni L, Johar G, Keely E, Burns KK, Kuziemsky C, Laplante J, Mihan A, Oppenheimer L, Sturge D, Tuot DS, Liddy C. Key factors for national spread and scale-up of an eConsult innovation. Health Res Policy Syst 2020; 18:57. [PMID: 32493357 PMCID: PMC7268606 DOI: 10.1186/s12961-020-00574-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2019] [Accepted: 05/14/2020] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND Expanding healthcare innovations from the local to national level is a complex pursuit requiring careful assessment of all relevant factors. In this study (a component of a larger eConsult programme of research), we aimed to identify the key factors involved in the spread and scale-up of a successful regional eConsult model across Canada. METHODS We conducted a constant comparative thematic analysis of stakeholder discussions captured during a full-day National eConsult Forum meeting held in Ottawa, Canada, on 11 December 2017. Sixty-four participants attended, representing provincial and territorial governments, national organisations, healthcare providers, researchers and patients. Proceedings were recorded, transcribed and underwent qualitative analysis using the Framework for Applied Policy Research. RESULTS This study identified four main themes that were critical to support the intentional efforts to spread and scale-up eConsult across Canada, namely (1) identifying population care needs and access problems, (2) engaging stakeholders who were willing to roll up their sleeves and take action, (3) building on current strategies and policies, and (4) measuring and communicating outcomes. CONCLUSIONS Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation. Further research is needed to develop and strengthen the conceptual and applied foundations of the spread and scale-up of healthcare innovations, especially in the context of emergent learning health systems across Canada and beyond.
Collapse
Affiliation(s)
- Isabella Moroz
- C.T. Lamont Primary Health Care Research Centre, Department of Family Medicine, University of Ottawa, Ottawa, Canada
- Bruyère Research Institute, Ottawa, Canada
| | - Douglas Archibald
- C.T. Lamont Primary Health Care Research Centre, Department of Family Medicine, University of Ottawa, Ottawa, Canada
- Bruyère Research Institute, Ottawa, Canada
| | - Mylaine Breton
- Faculty of Medicine and Health Sciences Research, University of Sherbrooke, Longueuil, Canada
| | - Elizabeth Cote-Boileau
- Faculty of Medicine and Health Sciences Research, University of Sherbrooke, Longueuil, Canada
| | - Lois Crowe
- C.T. Lamont Primary Health Care Research Centre, Department of Family Medicine, University of Ottawa, Ottawa, Canada
- Bruyère Research Institute, Ottawa, Canada
| | - Tanya Horsley
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | | | - Gina Johar
- South East Local Health Integration Network, Belleville, Canada
| | - Erin Keely
- Department of Medicine, University of Ottawa, Ottawa, Canada
- Division of Endocrinology/Metabolism, The Ottawa Hospital, Ottawa, Canada
- Ontario eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Canada
| | | | | | | | - Ariana Mihan
- C.T. Lamont Primary Health Care Research Centre, Department of Family Medicine, University of Ottawa, Ottawa, Canada
- Bruyère Research Institute, Ottawa, Canada
| | | | | | - Delphine S. Tuot
- Division of Nephrology, University of California San Francisco, San Francisco, United States of America
- Center for Innovation in Access and Quality, University of California San Francisco, San Francisco, United States of America
| | - Clare Liddy
- C.T. Lamont Primary Health Care Research Centre, Department of Family Medicine, University of Ottawa, Ottawa, Canada
- Bruyère Research Institute, Ottawa, Canada
- Ontario eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Canada
| |
Collapse
|
4
|
Scaling up eConsult for access to specialists in primary healthcare across four Canadian provinces: study protocol of a multiple case study. Health Res Policy Syst 2019; 17:83. [PMID: 31511008 PMCID: PMC6739985 DOI: 10.1186/s12961-019-0483-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2019] [Accepted: 08/16/2019] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Canada has been referred to as the land of 'perpetual pilot projects'. Effective innovations often remain small in scale, with limited impact on health systems. Several innovations have been developed in Canada to tackle important challenges such as poor access to services and excessive wait times - one of the most promising innovations that has been piloted is eConsult, which is a model of asynchronous communication that allows primary care providers to electronically consult with specialists regarding their patients' medical issues. eConsult pilot projects have been shown to reduce wait times for specialist care, prevent unnecessary referrals and reduce health system costs. eConsult has been spread throughout Ontario as well as to certain regions in Manitoba, Quebec, and Newfoundland and Labrador. Our aim is to understand and support the scale-up process of eConsult in Ontario, Quebec, Manitoba, and Newfoundland and Labrador. Our specific objectives are to (1) describe the main components of eConsult relevant to the scale-up process in each province; (2) understand the eConsult scale-up process in each province and compare across provinces; (3) identify policy issues and strategies to scaling up eConsult in each province; and (4) foster cross-level and cross-jurisdictional learning on scaling up eConsult. METHODS We will conduct a qualitative multiple case study to investigate the scaling up of eConsult in four Canadian provinces using a grey literature review, key stakeholder interviews (10 interviews/province), non-participant observations, focus groups and deliberative dialogues. We will identify the main components of eConsult to be scaled up using logic models (obj. 1). Scaling up processes will be analysed using strategies adapted from process research (obj. 2). Policy issues and strategies to scale-up eConsult will be analysed thematically (obj. 3). Finally, a symposium will foster pan-Canadian learning on the process of scaling up eConsult (obj. 4). DISCUSSION This study will likely increase learning and support evidence-based policy-making across participating provinces and may improve the capacity for a pan-Canadian scale-up of eConsult, including in provinces where eConsult has not yet been implemented. This work is essential to inform how similar innovations can reshape our health systems in the evolving information age.
Collapse
|
5
|
Liddy C, Bello A, Cook J, Drimer N, Pilon MD, Farrell G, Glassford J, Ireland L, McDonald R, Nabelsi V, Oppenheimer L, Singer A, Keely E. Supporting the spread and scale-up of electronic consultation across Canada: cross-sectional analysis. BMJ Open 2019; 9:e028888. [PMID: 31152043 PMCID: PMC6550029 DOI: 10.1136/bmjopen-2018-028888] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
OBJECTIVE To examine the process of implementing an electronic consultation (eConsult) service and evaluate its impact along key metrics outlined by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework. DESIGN Cross-sectional study. SETTING Clinics using eConsult in four provinces across Canada: Alberta, Manitoba, Quebec and Newfoundland and Labrador. PARTICIPANTS All eConsult cases submitted in four participating provinces were included. INTERVENTION The eConsult service is a secure online application that allows primary care providers and specialists to communicate regarding a patient's care. We measured the impact using system utilisation data and mandatory close-out surveys completed at the end of each eConsult. MAIN OUTCOME MEASURES Implementation progress and impact were examined using the five categories outlined by the RE-AIM framework: reach, effectiveness, adoption, implementation and maintenance. RESULTS Four provinces provided data from different periods, ranging from 4 years (Alberta) to 10 months (Manitoba). Total cases completed ranged from 96 (Manitoba) to 6885 (Alberta). Newfoundland had the largest menu of available specialties (n=35), while Alberta and Quebec had the smallest (n=22). The most frequently requested groups varied across provinces, with only endocrinology appearing in the top five for all provinces. The average specialist response time ranged from 3 days (Manitoba) to 16.7 days (Alberta). Between 54% (Newfoundland) and 66% (Manitoba) of cases resulted in new or additional information. Primary care providers avoided completing referrals they had originally considered in 36% (Newfoundland) to 53% of cases (Manitoba), while only between 27 % (Quebec) and 29% (Newfoundland) of cases resulted in a referral. In every province, services demonstrated higher rates of usage in their last quarter of data than their first. CONCLUSIONS eConsult was successfully implemented in four new provinces across Canada. Implementation strategies and scope varied, but services demonstrated substantial consistency on several key metrics, most notably on whether new information was learnt and impact on decision to refer.
Collapse
Affiliation(s)
- Clare Liddy
- CT Lamont Primary Health Care Research Centre, Department of Family Medicine, University of Ottawa, Ottawa, Ontario, Canada
- Bruyère Research Institute, Ottawa, Ontario, Canada
| | - Aminu Bello
- Department of Medicine, University of Alberta, Calgary, Alberta, Canada
| | - Jean Cook
- Canadian Foundation for Healthcare Improvement, Ottawa, Ontario, Canada
| | - Neil Drimer
- Department of Family Medicine, McGill University, Montreal, Quebec, Canada
| | - Maxine Dumas Pilon
- Newfoundland and Labrador Medical Association, St John’s, Newfoundland and Labrador, Canada
| | - Gerard Farrell
- Department of Family Medicine, Memorial University, St John’s, Newfoundland and Labrador, Canada
| | | | - Laurie Ireland
- Nine Circles Community Health Centre, Winnipeg, Manitoba, Canada
| | - Rana McDonald
- Nine Circles Community Health Centre, Winnipeg, Manitoba, Canada
| | | | - Luis Oppenheimer
- Department of Surgery, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Alex Singer
- Department of Family Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Erin Keely
- Division of Endocrinology/Metabolism, The Ottawa Hospital, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|