1
|
Terheyden JH, Gittel L, Jungblut J, Taylor DJ, Holz FG, Crabb DP, Finger RP. Heterogeneity of patient-reported outcome measures in clinical research. Health Qual Life Outcomes 2024; 22:65. [PMID: 39153981 PMCID: PMC11330056 DOI: 10.1186/s12955-024-02282-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 08/08/2024] [Indexed: 08/19/2024] Open
Abstract
BACKGROUND The use of patient-reported outcome measures (PROMs) in clinical research increases and use of heterogeneous instruments reflects how well diverse traits are captured by a medical specialty. In order to reflect the heterogeneity of current PROM use in ophthalmology, we reviewed the available literature. METHODS The medical literature database Web of Science was searched for the most cited articles in clinical ophthalmology. Titles, abstracts and full text articles were reviewed for the use of PROMs and a list of the 100 most cited articles using PROMs was obtained and stratified by year of publication. RESULTS A total of 1,996 articles were screened. Seventy-seven out of the 100 articles identified included one PROM, and the average number of instruments was 1.5 ± 1.1. The most widely used PROMs were the National Eye Institute Visual Function Questionnaire (33%), the Ocular Surface Disease Index (14%) and the Medical Outcomes Study Short Form (13%). A simulation analysis suggested that the distribution of PROM use in ophthalmology study did not significantly differ from a power law distribution. Twenty-two percent and fifteen percent of articles did not reference and did not specify the PROM used, respectively. This rate decreased in the more recently published articles (p = 0.041). CONCLUSIONS Our data suggest that the heterogeneity of PROMs applied in ophthalmology studies is low. The selection of PROMs for clinical studies should be done carefully, depending on the research goal.
Collapse
Affiliation(s)
- Jan Henrik Terheyden
- Department of Ophthalmology, University Hospital Bonn, Ernst-Abbe-Str. 2, Bonn, 53127, Germany.
- Department of Optometry and Visual Sciences, City, University of London, School of Health and Psychological Sciences, London, UK.
| | - Lisa Gittel
- Department of Ophthalmology, University Hospital Bonn, Ernst-Abbe-Str. 2, Bonn, 53127, Germany
| | - Julie Jungblut
- Department of Ophthalmology, University Hospital Bonn, Ernst-Abbe-Str. 2, Bonn, 53127, Germany
| | - Deanna J Taylor
- Department of Optometry and Visual Sciences, City, University of London, School of Health and Psychological Sciences, London, UK
| | - Frank G Holz
- Department of Ophthalmology, University Hospital Bonn, Ernst-Abbe-Str. 2, Bonn, 53127, Germany
| | - David P Crabb
- Department of Optometry and Visual Sciences, City, University of London, School of Health and Psychological Sciences, London, UK
| | - Robert P Finger
- Department of Ophthalmology, University Hospital Bonn, Ernst-Abbe-Str. 2, Bonn, 53127, Germany
- Department of Ophthalmology, University Hospital Mannheim and Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| |
Collapse
|
2
|
Abegao Pinto L, Sunaric Mégevand G, Stalmans I. European Glaucoma Society - A guide on surgical innovation for glaucoma. Br J Ophthalmol 2023; 107:1-114. [PMID: 38128960 DOI: 10.1136/bjophthalmol-2023-egsguidelines] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2023]
Abstract
PROLOGUE: Glaucoma surgery has been, for many decades now, dominated by the universal gold standard which is trabeculectomy augmented with antimetabolites. Tubes also came into the scene to complement what we use to call conventional or traditional glaucoma surgery. More recently we experienced a changing glaucoma surgery environment with the "advent" of what we have become used to calling Minimally Invasive Glaucoma Surgery (MIGS). What is the unmet need, what is the gap that these newcomers aim to fill? Hippocrates taught us "bring benefit, not harm" and new glaucoma techniques and devices aim to provide safer surgery compared to conventional surgery. For the patient, but also for the clinician, safety is important. Is more safety achieved with new glaucoma surgery and, if so, is it associated with better, equivalent, or worse efficacy? Is new glaucoma surgery intended to replace conventional surgery or to complement it as an 'add-on' to what clinicians already have in their hands to manage glaucoma? Which surgery should be chosen for which patient? What are the options? Are they equivalent? These are too many questions for the clinician! What are the answers to the questions? What is the evidence to support answers? Do we need more evidence and how can we produce high-quality evidence? This EGS Guide explores the changing and challenging glaucoma surgery environment aiming to provide answers to these questions. The EGS uses four words to highlight a continuum: Innovation, Education, Communication, and Implementation. Translating innovation to successful implementation is crucially important and requires high-quality evidence to ensure steps forward to a positive impact on health care when it comes to implementation. The vision of EGS is to provide the best possible well-being and minimal glaucomainduced visual disability in individuals with glaucoma within an affordable healthcare system. In this regard, assessing the changes in glaucoma surgery is a pivotal contribution to better care. As mentioned, this Guide aims to provide answers to the crucial questions above. However, every clinician is aware that answers may differ for every person: an individualised approach is needed. Therefore, there will be no uniform answer for all situations and all patients. Clinicians would need, through the clinical method and possibly some algorithm, to reach answers and decisions at the individual level. In this regard, evidence is needed to support clinicians to make decisions. Of key importance in this Guide is to provide an overview of existing evidence on glaucoma surgery and specifically on recent innovations and novel devices, but also to set standards in surgical design and reporting for future studies on glaucoma surgical innovation. Designing studies in surgery is particularly challenging because of many subtle variations inherent to surgery and hence multiple factors involved in the outcome, but even more because one needs to define carefully outcomes relevant to the research question but also to the future translation into clinical practice. In addition this Guide aims to provide clinical recommendations on novel procedures already in use when insufficient evidence exists. EGS has a long tradition to provide guidance to the ophthalmic community in Europe and worldwide through the EGS Guidelines (now in their 5th Edition). The EGS leadership recognized that the changing environment in glaucoma surgery currently represents a major challenge for the clinician, needing specific guidance. Therefore, the decision was made to issue this Guide on Glaucoma Surgery in order to help clinicians to make appropriate decisions for their patients and also to provide the framework and guidance for researchers to improve the quality of evidence in future studies. Ultimately this Guide will support better Glaucoma Care in accordance with EGS's Vision and Mission. Fotis Topouzis EGS President
Collapse
Affiliation(s)
| | - Gordana Sunaric Mégevand
- Eye Research Centre, Adolphe de Rothschild Hospital, Geneva, Switzerland and Centre Ophtalmologique de Florissant, Geneva, Switzerland
| | - Ingeborg Stalmans
- Ingeborg Stalmans, University Hospitals UZ Leuven, Catholic University KU Leuven
| |
Collapse
|
3
|
Yu CW, Nanji K, Hatamnejad A, Gemae M, Joarder I, Achunair A, Devji T, Phillips M, Zeraatkar D, Steel DH, Guymer RH, Sivaprasad S, Wykoff CC, Chaudhary V. Patient-Reported Outcome Measure Use in Guidelines Published by the American Academy of Ophthalmology: A Review. Ophthalmology 2023; 130:1201-1211. [PMID: 37429499 DOI: 10.1016/j.ophtha.2023.07.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2023] [Revised: 06/13/2023] [Accepted: 07/03/2023] [Indexed: 07/12/2023] Open
Abstract
TOPIC We reviewed the use of patient-reported outcome measures (PROMs) in the treatment of ophthalmologic conditions as recommended by the Clinical Practice Guidelines (CPGs) published by the American Academy of Ophthalmology (AAO). CLINICAL RELEVANCE Patient-reported outcome measures are standardized instruments that provide information regarding a patient's health status or health-related quality of life. Patient-reported outcome measures are increasingly used to inform study end points in ophthalmology studies. However, the extent to which PROMs are ultimately informing patient management recommendations in ophthalmology as part of CPGs remains an area of evidence gap. METHODS We included all CPGs published by the AAO from inception to June 2022. We also included all primary studies and systematic reviews cited in the treatment sections of the CPGs evaluating treatment of an ophthalmic condition. The primary outcome was the frequency of PROMs discussed in CPGs and in cited studies evaluating treatment. Secondary outcomes included frequency of minimal important difference (MID) use to contextualize PROM results and percentage of strong and discretionary recommendations supported by PROMs. We published a study protocol a priori on PROSPERO (CRD42022307427). Reporting followed the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. We assessed risk of bias using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. RESULTS We identified 24 eligible CPGs, providing 2458 cited studies (2191 primary, 267 secondary) evaluating treatment of eye conditions. Ten CPGs (41.7%) reported consideration of PROMs. Of these, 31 of 94 (33%) recommendations were informed by studies evaluating a PROM as an outcome. Across all studies cited in the development of CPGs, 221 (9.0%) used PROMs as a primary or secondary outcome, of which 4 PROM results (1.8%) were interpreted using an empirically determined MID. Overall, the risk of bias was low for all CPGs. CONCLUSIONS Overall, outcomes of PROMs are seldom used in ophthalmology CPGs published by the AAO and in cited primary and secondary research on treatments. When PROMs were considered, their interpretation was seldom based on an MID. To improve patient care, guideline developers may consider incorporating PROMs and applicable MIDs to inform key outcomes when formulating treatment recommendations. FINANCIAL DISCLOSURE(S) Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Caberry W Yu
- Department of Surgery, McMaster University, Hamilton, Ontario, Canada
| | - Keean Nanji
- Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Amin Hatamnejad
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Mohamed Gemae
- School of Medicine, Queen's University, Kingston, Ontario, Canada
| | - Ishraq Joarder
- Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | | | - Tahira Devji
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Mark Phillips
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Dena Zeraatkar
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada
| | - David H Steel
- Bioscience Institute, Newcastle University, Newcastle Upon Tyne, United Kingdom; Sunderland Eye Infirmary, Queen Alexandra Road, Sunderland, United Kingdom
| | - Robyn H Guymer
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia; Department of Surgery, The University of Melbourne, Melbourne, Australia
| | - Sobha Sivaprasad
- NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom
| | - Charles C Wykoff
- Retina Consultants of Texas, Retina Consultants of America, Houston, Texas; Blanton Eye Institute, Houston Methodist Hospital, Houston, Texas
| | - Varun Chaudhary
- Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
| |
Collapse
|
4
|
Taylor DJ, Jones L, Edwards L, Crabb DP. Patient-reported outcome measures in ophthalmology: too difficult to read? BMJ Open Ophthalmol 2021; 6:e000693. [PMID: 34212114 PMCID: PMC8208024 DOI: 10.1136/bmjophth-2020-000693] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2020] [Accepted: 04/30/2021] [Indexed: 12/14/2022] Open
Abstract
Objective Patient-reported outcome measures (PROMs) are commonly used in clinical trials and research. Yet, in order to be effective, a PROM needs to be understandable to respondents. The aim of this cross-sectional analysis was to assess reading level of PROMs validated for use in common eye conditions. Methods and analysis Readability measures determine the level of education a person is expected to have attained to be able to read a passage of text; this was calculated using the Flesch-Kincaid Grade Level, FORCAST and Gunning-Fog tests within readability calculations software package Oleander Readability Studio 2012.1. Forty PROMs, previously validated for use in at least one of age-related macular degeneration, glaucoma and/or diabetic retinopathy, were identified for inclusion via a systematic literature search. The American Medical Association (AMA) and National Institutes of Health (NIH) recommend patient materials should not exceed a sixth-grade reading level. Number of PROMs exceeding this level was calculated. Results Median (IQR) readability scores were 7.9 (5.4-10.5), 9.9 (8.9-10.7) and 8.4 (6.9-11.1) for Flesch-Kincaid Grade Level, FORCAST and Gunning-Fog test, respectively. Depending on metric used, this meant 61% (95% CI 45% to 76%), 100% (95% CI 91% to 100%) and 80% (95% CI 65% to 91%) exceeded the recommended threshold. Conclusion Most PROMs commonly used in ophthalmology require a higher reading level than that recommended by the AMA and NIH and likely contain questions that are too difficult for many patients to read. Greater care is needed in designing PROMs appropriate for the literacy level of a population.
Collapse
Affiliation(s)
- Deanna J Taylor
- Optometry and Visual Sciences, City University of London, London, UK
| | - Lee Jones
- Optometry and Visual Sciences, City University of London, London, UK.,Institute of Ophthalmology, University College London, London, UK
| | - Laura Edwards
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - David P Crabb
- Optometry and Visual Sciences, City University of London, London, UK
| |
Collapse
|
5
|
Puell MC, Contreras I, Pinilla I, Escobar JJ, Soler-García A, Blasco AJ, Lázaro P. Beyond visual acuity: Patient-relevant assessment measures of visual function in retinal diseases. Eur J Ophthalmol 2021; 31:3149-3156. [PMID: 33482694 DOI: 10.1177/1120672121990624] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
PURPOSE To identify patient-reported outcomes (PROs) and other clinical outcome measures (contrast sensitivity (CS), low-luminance visual acuity (LLVA) and reading acuity or reading speed (RA-RS)), relevant to patients with age-related macular degeneration (AMD) or diabetic retinopathy (DR), which would be recommended for use in clinical practice. METHODS The RAND/UCLA Appropriateness Method, based on the synthesis of the scientific evidence and the collective judgment of an expert panel using the two-round Delphi method, was applied. The evidence synthesis was performed by searching for articles on outcome measures for AMD and/or DR published between 2005 and 2018 in English or Spanish. The expert panel consisted of 14 Spanish ophthalmologists, who rated the recommendation degree for each outcome measure on a scale of 1 (extremely irrelevant) to 9 (maximum relevance). The recommended outcome measures were established according to the panel median score and the level of the panelists' agreement. RESULTS Through the evidence search, 33 PRO-specific questionnaires (21 for visual function, six for AMD, three for DR, one for AMD and DR) and two treatment satisfaction questionnaires (one on AMD and one on DR) were identified. In addition, 21 methods were found for measuring CS, five for LLVA, and nine for RA-RS. According to the panel ratings, 11 of the 64 outcome measures evaluated for AMD, and seven of the 61 evaluated for DR were recommended. CONCLUSION The AMD and DR outcome measures recommended will help ophthalmologists choose the outcome measure most appropriate for their patients. Furthermore, the use of PROs will contribute to shifting clinical practice towards patient-centered medicine.
Collapse
Affiliation(s)
- María Cinta Puell
- School of Optics and Optometry, Universidad Complutense de Madrid, Madrid, Spain
| | - Inés Contreras
- Ramón y Cajal University Hospital, Ramón y Cajal Health Research Institute (IRYCIS) and Clínica Rementería, Madrid, Spain
| | - Isabel Pinilla
- Aragon Institute for Health Research (IIS Aragon), Zaragoza, Spain; Department of Ophthalmology, Hospital Clinico Universitario, Zaragoza, Spain
| | | | | | | | - Páblo Lázaro
- Independent Health Services Researcher, Madrid, Spain
| |
Collapse
|
6
|
Uddin D, Jeffrey BG, Flynn O, Wong W, Wiley H, Keenan T, Chew E, Cukras C. Repeatability of Scotopic Sensitivity and Dark Adaptation Using a Medmont Dark-Adapted Chromatic Perimeter in Age-related Macular Degeneration. Transl Vis Sci Technol 2020; 9:31. [PMID: 32832236 PMCID: PMC7414623 DOI: 10.1167/tvst.9.7.31] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2020] [Accepted: 04/04/2020] [Indexed: 12/27/2022] Open
Abstract
Purpose Functional studies of rods in age-related macular degeneration using the Medmont Dark-Adapted Chromatic Perimeter (DACP) have demonstrated impairments in scotopic sensitivities and dark adaptation (DA). We investigated the intersession repeatability of scotopic sensitivity and DA parameters including the rod intercept time recorded from the Medmont DACP. Methods Scotopic thresholds (14 test points) and DA using a 30% photobleach (eight test points) were measured on two separate days from participants 50 years of age or older with a range of age-related macular degeneration severity at loci superior and inferior to the fovea. Repeatability coefficients were calculated for prebleach scotopic sensitivity, and for DA parameters including rod intercept time. Results Twelve participants (mean age, 79.7 ± 8.1 years) repeated Medmont DACP testing within 50 days. Repeatability coefficients for prebleach scotopic sensitivity to long wavelength (red, 625 nm) and short wavelength (cyan, 505 nm) were 5.9 dB and 7.2 dB, respectively. The DA curve-derived repeatability coefficients for cone threshold was 3.9 dB, final threshold 5.3 dB, with an R value of 0.075 decades/min, rod intercept time 7.6 minutes, and RITslope 0.54 min/degree. Conclusions This study establishes repeatability coefficients for scotopic thresholds and multiple DA parameters obtained with the Medmont DACP in patients with age-related macular degeneration. These repeatability coefficients will serve as the basis for determining clinically meaningful change in rod function in future clinical trials. Translational Relevance Measures of repeatability parameters of scotopic thresholds and DA are essential to the accurate interpretation of results in future studies and trials using these measures.
Collapse
Affiliation(s)
- Durin Uddin
- National Eye Institute, NIH, Bethesda, Maryland, USA
| | | | - Oliver Flynn
- National Eye Institute, NIH, Bethesda, Maryland, USA
| | - Wai Wong
- National Eye Institute, NIH, Bethesda, Maryland, USA
| | - Henry Wiley
- National Eye Institute, NIH, Bethesda, Maryland, USA
| | | | - Emily Chew
- National Eye Institute, NIH, Bethesda, Maryland, USA
| | | |
Collapse
|
7
|
Jones PR, Tigchelaar I, Demaria G, Wilson I, Bi W, Taylor DJ, Crabb DP. Refinement and preliminary evaluation of two tablet-based tests of real-world visual function. Ophthalmic Physiol Opt 2019; 40:35-46. [PMID: 31879994 PMCID: PMC7028122 DOI: 10.1111/opo.12658] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2019] [Accepted: 11/09/2019] [Indexed: 01/07/2023]
Abstract
Purpose To describe, refine, evaluate, and provide normative control data for two freely available tablet‐based tests of real‐world visual function, using a cohort of young, normally‐sighted adults. Methods Fifty young (18–40 years), normally‐sighted adults completed tablet‐based assessments of (1) face discrimination and (2) visual search. Each test was performed twice, to assess test‐retest repeatability. Post‐hoc analyses were performed to determine the number of trials required to obtain stable estimates of performance. Distributions were fitted to the normative data to determine the 99% population‐boundary for normally sighted observers. Participants were also asked to rate their comprehension of each test. Results Both tests provided stable estimates in around 20 trials (~1–4 min), with only a further reduction of 14%–17% in the 95% Coefficient of Repeatability (CoR95) when an additional 40 trials were included. When using only ~20 trials: median durations for the first run of each test were 191 s (Faces) and 51 s (Search); test‐retest CoR95 were 0.27 d (Faces) and 0.84 s (Search); and normative 99% population‐limits were 3.50 d (Faces) and 3.1 s (Search). No participants exhibited any difficulties completing either test (100% completion rate), and ratings of task‐understanding were high (Faces: 9.6 out of 10; Search: 9.7 out of 10). Conclusions This preliminary assessment indicated that both tablet‐based tests are able to provide simple, quick, and easy‐to‐administer measures of real‐world visual function in normally‐sighted young adults. Further work is required to assess their accuracy and utility in older people and individuals with visual impairment. Potential applications are discussed, including their use in clinic waiting rooms, and as an objective complement to Patient Reported Outcome Measures (PROMs).
Collapse
Affiliation(s)
- Pete R Jones
- Division of Optometry and Visual Science, School of Health Sciences, City, University of London, London, UK
| | - Iris Tigchelaar
- Department of Ophthalmology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands.,Ocusweep, Turku, Finland.,Doctoral Program in Clinical Research, University of Turku and Turku University Hospital, Turku, Finland
| | - Giorgia Demaria
- Department of Ophthalmology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands.,Graduate School of Medical Sciences (Research School of Behavioral and Cognitive Neurosciences), University of Groningen, Groningen, the Netherlands
| | - Iain Wilson
- Nuffield Laboratory of Ophthalmology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.,Oxford Eye Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Wei Bi
- Division of Optometry and Visual Science, School of Health Sciences, City, University of London, London, UK
| | - Deanna J Taylor
- Division of Optometry and Visual Science, School of Health Sciences, City, University of London, London, UK
| | - David P Crabb
- Division of Optometry and Visual Science, School of Health Sciences, City, University of London, London, UK
| |
Collapse
|
8
|
[ANDROMEDA-an investigation of factors influencing the adherence of patients with neovascular age-related macular degeneration using the newly developed patient questionnaire LAF-IVT]. Ophthalmologe 2019; 117:765-774. [PMID: 31720846 DOI: 10.1007/s00347-019-01005-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
BACKGROUND Lack of adherence to treatment is a widespread problem in the anti-VEGF (Vascular Endothelial Growth Factor) treatment of patients with neovascular age-related macular degeneration (nAMD). In contrast to the extent of the lack of treatment adherence, there is so far insufficient evidence for elucidating the causes of nonadherence. The ANDROMEDA study was initiated to investigate the influencing factors on the adherence of nAMD patients to treatment. The focus of the study was on patient reported endpoints, as the perceptions and experiences of the patients are of enormous importance for the investigation of the various aspects of adherence to treatment. OBJECTIVE This publication presents the design of the ANDROMEDA study as well as the development of a new patient questionnaire for the assessment of barriers to treatment within the design of the study. MATERIAL AND METHODS This prospective noninterventional observational study to assess the compliance of patients with nAMD and anti-VEGF treatment was started at the end of January 2019. It is planned to include 1000 patients in 120 study centers throughout Germany with an observational period of 24 months. Patient interviews on general and vision-related quality of life, treatment satisfaction and possible barriers to treatment will be conducted at the beginning and after 4, 12 and 24 months. All patient visits will be documented by the study centers as part of the clinical routine. To date, there has been no suitable instrument for recording patient-related circumstances and potential barriers to anti-VEGF treatment. Therefore, a specific patient questionnaire for longitudinal assessment of adherence factors to intravitreal (anti-VEGF) therapy (LAF-IVT) was developed as part of the study concept. The questionnaire, developed by an expert panel, was tested via qualitative interviews for its cognitive characteristics ahead of its use. RESULTS The results of the study are expected in early 2023. The cognitive examination of the LAF-IVT confirmed the feasibility of the new questionnaire. The practicability and significance of the new instrument can be assessed after completion of the quantitative data collection. CONCLUSION The symptoms, barriers, burdens and quality of life effects experienced by patients influence the adherence to treatment and thus the outcome. A better understanding of the patient's views and experiences is the basis for long-term optimization of care.
Collapse
|
9
|
Jelin E, Wisløff T, Jørstad ØK, Heiberg T, Moe MC. Patient-reported outcome measures in the management of neovascular age-related macular degeneration: a 1-year prospective study. BMJ Open Ophthalmol 2019; 4:e000353. [PMID: 31673632 PMCID: PMC6797267 DOI: 10.1136/bmjophth-2019-000353] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2019] [Revised: 09/03/2019] [Accepted: 09/14/2019] [Indexed: 11/04/2022] Open
Abstract
Objective To prospectively explore the following patient-reported outcome measures (PROMs) in the management of neovascular age-related macular degeneration (nAMD): (1) self-reported visual function, (2) symptom-state, (3) general-health and (4) satisfaction of treatment. Methods and analysis Corresponding to the four PROMs, participants responded to the following questionnaires: (1) National Eye Institute Visual-Functioning-Questionnaire (NEI-VFQ-25), (2) Patient-Acceptable-Symptom-Status (PASS 5), (3) EuroQol-Group-Questionnaire (EQ-5D-3L) and (4) Dimensions of Importance in Treatment of nAMD (DITAMD). Data were collected at baseline and after 3, 6 and 12 months of intravitreal antivascular endothelial growth factor treatment. Results were evaluated with t-tests and mixed linear regression analyses. Results The study included 197 patients. At baseline NEI-VFQ-25 (79.53±14.52) and EQ-5D (0.74±0.28) had relatively high scores, whereas PASS 5 was below 'acceptable' (3.30±0.80). At 12 months NEI-VFQ-25 and PASS 5 showed significant improvement, whereas EQ-5D and DITAMD remained unchanged. At baseline patients receiving treatment of the better-seeing eye (BSE) (n=52) reported significantly worse NEI-VFQ-25 and PASS 5 than patients for whom treatment only involved the worse-seeing eye (WSE), (n=145). In contrast to BSE patients, there was no improvement of NEI-VFQ-25 for WSE patients at 12 months, despite a significant improvement in best-corrected visual acuity (BCVA). Two independent variables, treatment including the BSE and BCVA for the treated eye, were found to predict both NEI-VFQ-25 and PASS 5. Conclusion After 12 months of nAMD treatment, there was a significant improvement in PASS 5 and NEI-VFQ-25, the latter depending on whether therapy included BSE. EQ-5D and DITAMD remained unaltered.
Collapse
Affiliation(s)
- Elma Jelin
- Department of Ophthalmology, Oslo University Hospital, Oslo, Norway.,Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Torbjørn Wisløff
- Department of Infectious Disease Epidemiology and Modelling, Norwegian Institute of Public Health, Oslo, Norway.,Department of Health and Society, University of Oslo, Oslo, Norway
| | - Øystein Kalsnes Jørstad
- Department of Ophthalmology, Oslo University Hospital, Oslo, Norway.,Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Turid Heiberg
- Regional Research Support, Oslo University Hospital, Oslo, Norway.,Østfold University College, Halden, Norway
| | - Morten Carstens Moe
- Department of Ophthalmology, Oslo University Hospital, Oslo, Norway.,Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| |
Collapse
|
10
|
Lindsley KB, Hutfless S, Hawkins BS, Blim JF, Roberts D, Olsen TW, Lum F, Dickersin K. Evaluation of Clinical Questions and Patient-Important Outcomes Associated With the Treatment of Age-Related Macular Degeneration. JAMA Ophthalmol 2019; 136:1217-1225. [PMID: 30128539 DOI: 10.1001/jamaophthalmol.2018.3456] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Importance Identifying and prioritizing unanswered clinical questions may help to best allocate limited resources for research associated with the treatment of age-related macular degeneration (AMD). Objective To identify and prioritize clinical questions and outcomes for research associated with the treatment of AMD through engagement with professional and patient stakeholders. Design, Setting, and Participants Multiple cross-sectional survey questions were used in a modified Delphi process for panel members of US and international organizations, the American Academy of Ophthalmology (AAO) Retina/Vitreous Panel (n=7), health care professionals from the American Society of Retinal Specialists (ASRS) (n=90), Atlantic Coast Retina Conference (ACRC) and Macula 2017 meeting (n=34); and patients from MD (Macular Degeneration) Support (n=46). Data were collected from January 20, 2015, to January 9, 2017. Main Outcomes and Measures The prioritizing of clinical questions and patient-important outcomes for AMD. Results Seventy clinical questions were derived from the AAO Preferred Practice Patterns for AMD and suggestions by the AAO Retina/Vitreous Panel. The AAO Retina/Vitreous Panel assessed all 70 clinical questions and rated 17 of 70 questions (24%) as highly important. Health care professionals assessed the 17 highly important clinical questions and rated 12 of 17 questions (71%) as high priority for research to answer; 9 of 12 high-priority clinical questions were associated with aspects of anti-vascular endothelial growth factor agents. Patients assessed the 17 highly important clinical questions and rated all as high priority. Additionally, patients identified 6 of 33 outcomes (18%) as most important to them (choroidal neovascularization, development of advanced AMD, retinal hemorrhage, gain of vision, slowing vision loss, and serious ocular events). Conclusions and Relevance Input from 4 stakeholder groups suggests good agreement on which 12 priority clinical questions can be used to underpin research related to the treatment of AMD. The 6 most important outcomes identified by patients were balanced between intended effects of AMD treatment (eg, slowing vision loss) and adverse events. Consideration of these patient-important outcomes may help to guide clinical care and future areas of research.
Collapse
Affiliation(s)
- Kristina B Lindsley
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland.,Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Susan Hutfless
- Division of Gastroenterology & Hepatology, Johns Hopkins School of Medicine, Baltimore, Maryland
| | - Barbara S Hawkins
- Department of Ophthalmology, Johns Hopkins School of Medicine, Baltimore, Maryland
| | - Jill F Blim
- American Society of Retina Specialists, Chicago, Illinois
| | - Dan Roberts
- MD (Macular Degeneration) Support, Grandview, Missouri
| | - Timothy W Olsen
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota.,Preferred Practice Pattern Retina/Vitreous Panel and Secretary for Quality of Care, American Academy of Ophthalmology
| | - Flora Lum
- Quality and Data Science, American Academy of Ophthalmology, San Francisco, California
| | - Kay Dickersin
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland.,Reviews Editor
| |
Collapse
|
11
|
Ophthalmic Drug Discovery and Development: Regulatory Aspects of Patient Focused Drug Development in Ophthalmology. Pharm Res 2019; 36:54. [PMID: 30790065 DOI: 10.1007/s11095-019-2577-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2018] [Accepted: 01/17/2019] [Indexed: 10/27/2022]
Abstract
In 2009, members of the ophthalmic research community held a joint meeting with members of the Food and Drug Administration (FDA) and the National Eye Institute (NEI) to define and describe the types of patient-focused drug development (PFDD) tools used in ophthalmology. Since then numerous reports have been published which indicate that many of the questionnaires used for patient-reported outcomes (PROs) in ophthalmic clinical development lack rigor and reliability according to modern methods. In 2017, the FDA began development of a series of four methodological guidances for sponsors of clinical trials on the significance of PFDD. The new guidances delineate the FDA's thinking and commitments under the Prescription Drug User Fee Act to implement a more structured approach to the assessment of risks and benefits in clinical trials. In these guidances, the FDA provides steps that drug and device manufacturers should follow, not only to obtain, but also to develop reliable and validated tools that measure patients' experience in clinical trials. Subsequent efforts have resulted in the development and validation of PROs specifically for ophthalmology. The purpose of this paper is to assesses the PROs currently used in ophthalmology and to provide practical strategies for incorporating them into clinical trials.
Collapse
|
12
|
Al Jabri S, Kirkham J, Rowe FJ. Development of a core outcome set for amblyopia, strabismus and ocular motility disorders: a review to identify outcome measures. BMC Ophthalmol 2019; 19:47. [PMID: 30736755 PMCID: PMC6368710 DOI: 10.1186/s12886-019-1055-8] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2018] [Accepted: 01/29/2019] [Indexed: 01/01/2023] Open
Abstract
BACKGROUND Core Outcome Sets (COS) are defined as the minimum sets of outcomes that should be measured and reported in all randomised controlled trials to facilitate combination and comparability of research. The aim of this review is to produce an item bank of previously reported outcome measures from published studies in amblyopia, strabismus and ocular motility disorders to initiate the development of COS. METHODS A review was conducted to identify articles reporting outcome measures for amblyopia, strabismus and ocular motility disorders. Using systematic methods according to the COMET handbook we searched key electronic bibliographic databases from 1st January 2011 to 27th September 2016 using MESH terms and alternatives indicating the different subtypes of amblyopia, strabismus and ocular motility disorders in relation to treatment outcomes and all synonyms. We included Cochrane reviews, other systematic reviews, controlled trials, non-systematic reviews and retrospective studies. Data was extracted to tabulate demographics of included studies, primary and secondary outcomes, methods of measurement and their time points. RESULTS A total of 142 studies were included; 42 in amblyopia, 33 in strabismus, and 68 in ocular motility disorders (one study overlap between amblyopia and strabismus). We identified ten main outcome measure domains for amblyopia, 14 for strabismus, and ten common "visual or motility" outcome measure domains for ocular motility disorders. Within the domains, we found variable nomenclature being used and diversity in methods and timings of measurements. CONCLUSION This review highlights discrepancies in outcome measure reporting within published literature for amblyopia, strabismus and ocular motility and it generated an item bank of the most commonly used and reported outcome measures for each of the three conditions from recent literature to start the process of COS development. Consensus among all stakeholders including patients and professionals is recommended to establish a useful COS.
Collapse
Affiliation(s)
- Samia Al Jabri
- Department of Health Services Research, University of Liverpool, Waterhouse Building Block B, 2nd Floor, 1-3 Brownlow Street, L69 3GL Liverpool, UK
| | - Jamie Kirkham
- Department of Biostatistics, University of Liverpool, Liverpool, UK
| | - Fiona J. Rowe
- Department of Health Services Research, University of Liverpool, Waterhouse Building Block B, 2nd Floor, 1-3 Brownlow Street, L69 3GL Liverpool, UK
| |
Collapse
|
13
|
Braithwaite T, Calvert M, Gray A, Pesudovs K, Denniston AK. The use of patient-reported outcome research in modern ophthalmology: impact on clinical trials and routine clinical practice. PATIENT-RELATED OUTCOME MEASURES 2019; 10:9-24. [PMID: 30774489 PMCID: PMC6352858 DOI: 10.2147/prom.s162802] [Citation(s) in RCA: 80] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
This review article considers the rising demand for patient-reported outcome measures (PROMs) in modern ophthalmic research and clinical practice. We review what PROMs are, how they are developed and chosen for use, and how their quality can be critically appraised. We outline the progress made to develop PROMs in each clinical subspecialty. We highlight recent examples of the use of PROMs as secondary outcome measures in randomized controlled clinical trials and consider the impact they have had. With increasing interest in using PROMs as primary outcome measures, particularly where interventions have been found to be of equivalent efficacy by traditional outcome metrics, we highlight the importance of instrument precision in permitting smaller sample sizes to be recruited. Our review finds that while there has been considerable progress in PROM development, particularly in cataract, glaucoma, medical retina, and low vision, there is a paucity of useful tools for less common ophthalmic conditions. Development and validation of item banks, administered using computer adaptive testing, has been proposed as a solution to overcome many of the traditional limitations of PROMs, but further work will be needed to examine their acceptability to patients, clinicians, and investigators.
Collapse
Affiliation(s)
- Tasanee Braithwaite
- Centre for Patient Reported Outcomes Research and NIHR Birmingham Biomedical Research Centre, University of Birmingham, Edgbaston, Birmingham, UK, .,Moorfields Eye Hospital, London, UK,
| | - Melanie Calvert
- Centre for Patient Reported Outcomes Research and NIHR Birmingham Biomedical Research Centre, University of Birmingham, Edgbaston, Birmingham, UK, .,Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, UK
| | - Alastair Gray
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | | | - Alastair K Denniston
- Centre for Patient Reported Outcomes Research and NIHR Birmingham Biomedical Research Centre, University of Birmingham, Edgbaston, Birmingham, UK, .,Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHSFT, Birmingham, UK.,Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.,NIHR Biomedical Research Centre (Moorfields Eye Hospital/UCL), London, UK
| |
Collapse
|
14
|
Saldanha IJ, Lindsley K, Do DV, Chuck RS, Meyerle C, Jones LS, Coleman AL, Jampel HD, Dickersin K, Virgili G. Comparison of Clinical Trial and Systematic Review Outcomes for the 4 Most Prevalent Eye Diseases. JAMA Ophthalmol 2017; 135:933-940. [PMID: 28772305 PMCID: PMC5625342 DOI: 10.1001/jamaophthalmol.2017.2583] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2017] [Accepted: 06/09/2017] [Indexed: 12/24/2022]
Abstract
Importance Suboptimal overlap in outcomes reported in clinical trials and systematic reviews compromises efforts to compare and summarize results across these studies. Objectives To examine the most frequent outcomes used in trials and reviews of the 4 most prevalent eye diseases (age-related macular degeneration [AMD], cataract, diabetic retinopathy [DR], and glaucoma) and the overlap between outcomes in the reviews and the trials included in the reviews. Design, Setting, and Participants This cross-sectional study examined all Cochrane reviews that addressed AMD, cataract, DR, and glaucoma; were published as of July 20, 2016; and included at least 1 trial and the trials included in the reviews. For each disease, a pair of clinical experts independently classified all outcomes and resolved discrepancies. Outcomes (outcome domains) were then compared separately for each disease. Main Outcomes and Measures Proportion of review outcomes also reported in trials and vice versa. Results This study included 56 reviews that comprised 414 trials. Although the median number of outcomes per trial and per review was the same (n = 5) for each disease, the trials included a greater number of outcomes overall than did the reviews, ranging from 2.9 times greater (89 vs 30 outcomes for glaucoma) to 4.9 times greater (107 vs 22 outcomes for AMD). Most review outcomes, ranging from 14 of 19 outcomes (73.7%) (for DR) to 27 of 29 outcomes (93.1%) (for cataract), were also reported in the trials. For trial outcomes, however, the proportion also named in reviews was low, ranging from 19 of 107 outcomes (17.8%) (for AMD) to 24 of 89 outcomes (27.0%) (for glaucoma). Only 1 outcome (visual acuity) was consistently reported in greater than half the trials and greater than half the reviews. Conclusions and Relevance Although most review outcomes were reported in the trials, most trial outcomes were not reported in the reviews. The current analysis focused on outcome domains, which might underestimate the problem of inconsistent outcomes. Other important elements of an outcome (ie, specific measurement, specific metric, method of aggregation, and time points) might have differed even though the domains overlapped. Inconsistency in trial outcomes may impede research synthesis and indicates the need for disease-specific core outcome sets in ophthalmology.
Collapse
Affiliation(s)
- Ian J. Saldanha
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
| | - Kristina Lindsley
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
| | - Diana V. Do
- Byers Eye Institute, Stanford University School of Medicine, Palo Alto, California
| | - Roy S. Chuck
- Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, New York
| | - Catherine Meyerle
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Leslie S. Jones
- Department of Ophthalmology, Howard University Hospital, Washington, DC
| | - Anne L. Coleman
- Frank and Ray Stark Foundation, David Geffen School of Medicine at UCLA, Los Angeles, California
| | - Henry D. Jampel
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Kay Dickersin
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
| | - Gianni Virgili
- Eye Clinic, Department of Translational Surgery and Medicine, University of Florence, Careggi Hospital, Florence, Italy
| |
Collapse
|
15
|
Decreased Visual Function Scores on a Low Luminance Questionnaire Is Associated with Impaired Dark Adaptation. Ophthalmology 2017; 124:1332-1339. [PMID: 28602520 DOI: 10.1016/j.ophtha.2017.05.005] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2016] [Revised: 05/04/2017] [Accepted: 05/08/2017] [Indexed: 12/27/2022] Open
Abstract
PURPOSE We investigate whether responses on a Low Luminance Questionnaire (LLQ) in patients with a range of age-related macular degeneration (AMD) severity are associated with their performance on focal dark adaptation (DA) testing and with choroidal thickness. DESIGN Cross-sectional, single-center, observational study. PARTICIPANTS A total of 113 participants older than 50 years of age with a range of AMD severity. METHODS Participants answered the LLQ on the same day they underwent DA testing using a focal dark adaptometer measuring rod intercept time (RIT). We performed univariable and multivariable analyses of the LLQ scores and age, RIT, AMD severity, subfoveal choroidal thickness [SFCT], phakic status, and best-corrected visual acuity. MAIN OUTCOME MEASURES The primary outcome of this study was the score on the 32-question LLQ. Each item in the LLQ is designated to 1 of 6 subscales describing functional problems in low luminance: driving, emotional distress, mobility, extreme lighting, peripheral vision, and general dim lighting. Scores were computed for each subscale, in addition to a weighted total mean score. RESULTS Responses from 113 participants (mean age, 76.2±9.3 years; 58.4% were female) and 113 study eyes were analyzed. Univariable analysis demonstrated that lower scores on all LLQ subscales were correlated with prolonged DA testing (longer RIT) and decreased choroidal thickness. All associations were statistically significant except for the association of choroidal thickness and "peripheral vision." The strongest association was the LLQ subscale of driving with RIT (r =-0.97, P < 0.001). Multivariable analysis for each of the LLQ subscale outcomes, adjusted for age, included RIT, with total LLQ score, "driving," "extreme lighting," and "mobility" also including choroidal thickness. In all multivariable analyses, RIT had a stronger association than choroidal thickness. CONCLUSIONS This cross-sectional analysis demonstrates associations of patient-reported functional deficits, as assessed on the LLQ, with both reduced DA and reduced choroidal thickness, in a population of older adults with varying degrees of AMD severity and good visual acuity in at least 1 eye. These analyses suggest that local functional measurements of DA testing (RIT) and choroidal thickness are associated with patient-reported functional deficits.
Collapse
|
16
|
Krezel AK, Hogg RE, Krezel S, Fallis R, Azuara-Blanco A. Design characteristic of randomised controlled trials for geographic atrophy in age-related macular degeneration: selection of outcomes and sample size calculation. Eye (Lond) 2015; 29:1458-63. [PMID: 26206532 PMCID: PMC4815658 DOI: 10.1038/eye.2015.132] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2015] [Accepted: 06/05/2015] [Indexed: 01/10/2023] Open
Abstract
PURPOSE The selection of suitable outcomes and sample size calculation are critical factors in the design of a randomised controlled trial (RCT). The goal of this study was to identify the range of outcomes and information on sample size calculation in RCTs on geographic atrophy (GA). METHODS We carried out a systematic review of age-related macular degeneration (AMD) RCTs. We searched MEDLINE, EMBASE, Scopus, Cochrane Library, www.controlled-trials.com, and www.ClinicalTrials.gov. Two independent reviewers screened records. One reviewer collected data and the second reviewer appraised 10% of collected data. We scanned references lists of selected papers to include other relevant RCTs. RESULTS Literature and registry search identified 3816 abstracts of journal articles and 493 records from trial registries. From a total of 177 RCTs on all types of AMD, 23 RCTs on GA were included. Eighty-one clinical outcomes were identified. Visual acuity (VA) was the most frequently used outcome, presented in 18 out of 23 RCTs and followed by the measures of lesion area. For sample size analysis, 8 GA RCTs were included. None of them provided sufficient Information on sample size calculations. CONCLUSIONS This systematic review illustrates a lack of standardisation in terms of outcome reporting in GA trials and issues regarding sample size calculation. These limitations significantly hamper attempts to compare outcomes across studies and also perform meta-analyses.
Collapse
Affiliation(s)
- A K Krezel
- Centre for Experimental Medicine, Queen's University Belfast, Belfast, UK
| | - R E Hogg
- Centre for Experimental Medicine, Queen's University Belfast, Belfast, UK
| | - S Krezel
- Former student of Queen's University Belfast, Belfast, UK
| | - R Fallis
- Medical Library, Queen's University Belfast, Belfast, UK
| | - A Azuara-Blanco
- Centre for Experimental Medicine, Queen's University Belfast, Belfast, UK
| |
Collapse
|