Paper and electronic versions of HM-PRO, a novel patient-reported outcome measure for hematology: an equivalence study

Aim: To determine measurement equivalence of paper and electronic application of the hematological malignancy-patient-reported outcome (HM-PRO), a specific measure for the evaluation of patient-reported outcomes in HMs. Patients & methods: Following International Society of Pharmacoeconomics and Outcomes Research ePRO Good Research Practice Task Force guidelines, a total of 193 adult patients with different HMs were recruited into a multicenter prospective study. The paper and the electronic version of the instrument were completed in the outpatient clinics in a randomized crossover design with a 30 min time interval to minimize the learning effect. Those who completed the paper version first, completed the electronic version after 30 min and vice versa. Instrument version and order effects were tested on total score of the two parts of the HM-PRO (Part A: quality of life and Part B: signs & symptoms) in a two-way ANOVA with patients as random effects. Intraclass correlation coefficients (95% CI) and Spearman's rank correlation coefficients were used to evaluate test–retest reliability and reproducibility. The effects of instrument version and order were tested on total score of the two parts of HM-PRO. Results: The questionnaire version and administration order effects were not significant at the 5% level. There were no interactions found between these two factors for HM-PRO (Part A [quality of life]; p = 0.95); and (part B [signs and symptoms]; p = 0.72]. Spearman's rank correlation coefficients were greater than 0.9, and intraclass correlation coefficients ranged from 0.94 to 0.98; furthermore, the scores were not statistically different between the two versions, showing acceptable reliability indexes. Noteworthy, the difference between the completion time for both paper (mean = 6:38 min) and electronic version (mean = 7:29 min) was not statistically significant (n = 100; p = 0.11). Patients did not report any difficulty in completing the electronic version during cognitive interviews and were able to understand and respond spontaneously. Conclusion: Measurement equivalence has been demonstrated for the paper and electronic application of the HM-PRO.

Factors Influencing Delays in Patient Access to New Medicines in Canada: A Retrospective Study of Reimbursement Processes in Public Drug Plans

Individuals who rely on public health payers to access new medicines can access fewer innovative medicines and must wait longer in Canada compared to major markets around the world. New medicines/indications approved by Health Canada and reviewed for eligibility for reimbursement by the Common Drug Review or the pan-Canadian Oncology Drug Review (CDR/pCODR) from the beginning of 2012 through to the end of December 2016 were analyzed, with data taken from the relevant bodies’ websites and collected by IQVIA. This analysis investigated individual review segments – Notice of Compliance (NOC) to Health Technology Assessment (HTA) submission, HTA review time, pan-Canadian Pharmaceutical Alliance (pCPA) negotiation time, and public reimbursement decision time, and analyzed the trends of each over time and contributions to overall time to listing decisions. Average overall timelines for public reimbursement after NOC were long and most of this time is taken up by HTA and pCPA processes, at 236 and 273 days, respectively. This study confirms that Canadian public reimbursement delays from 2013-2014 to 2015-2016 lengthened from NOC to listing (Quebec + 53%, first provincial listing + 38%, and country-wide listing + 22%), reaching 499, 505, and 571 days, respectively. Over the same period, time from NOC to completion of HTA has increased by 33%, and time from post-HTA to first provincial listing by 44%. The pCPA process appears to be the main contributor to this increasing time trend, and although some provinces could be listing more quickly post-pCPA, they appear to be listing fewer products. Reasons for large delays in time to listing include the many-layered sequential process of reviews conducted before public drug plans decide whether to provide access to new innovative medicines. Although there has been some headway made in certain parts of the review processes (e.g., pre-NOC HTA), total time to listing continues to increase, seemingly due to the pCPA process and other additional review processes by drug plans. More clarity in the pCPA and provincial decision-making processes and better coordination between HTA, pCPA, and provincial decision-making processes is needed to increase predictability in the processes and reduce timelines for Canadian patients and manufacturers.

The South African Medicines Control Council: Comparison of Its Registration Process With Australia, Canada, Singapore, and Switzerland

Introduction: Comparisons between regulatory authorities of similar size and regulatory characteristics facilitate value-added benchmarking and provide insight into regulatory performance. Such comparisons highlight areas for improvement as authorities move toward achieving their regulatory goals and stakeholders’ demands. The aims of this study were to compare the registration process and the regulatory review model of the South African Medicines Control Council (MCC) to that of four other similar-sized regulatory authorities and to identify areas for improvement that may inform recommendations to the South African Health Products Regulatory Authority (SAHPRA) as it looks to re-engineer and enhance the registration process in South Africa.

Methods: A questionnaire describing the organisational structure, the registration process, good review and decision-making practices of the MCC was completed by the author (AK) for the purpose of this study and validated by the Registrar of the MCC. Similar questionnaires were also completed and validated by Australia’s Therapeutic Goods Administration (TGA), Canada’s Health Canada, Singapore’s Health Science Authority (HSA) and Switzerland’s Swissmedic.

Results: A comparison of the MCC regulatory process with the four comparative agencies indicated that they all have similar requirements and employ a full-review model although the timelines for the MCC were considerably longer. However, similar quality measures were implemented by all authorities as part of their good review practices (GRevP) including prioritising transparency, communication, continuous improvement initiatives and training.

Conclusion: Comparisons made through this study provided insight into the areas of the MCC registration process that may be improved and have informed recommendations to SAHPRA including the implementation of facilitated regulatory pathways, definition of targets for key milestones in regulatory review and formal implementation and monitoring of GRevP. In order to build quality into the review process the application of a standardised template for the clinical assessment of medicines such as the Universal Methodology for Benefit-Risk Assessment (UMBRA) could be considered as well as enhancing transparency and communication through the application of an electronic management system and the development of publicly available summaries for the basis of approval.

Haematological malignancy: Are we measuring what is important to patients? A systematic review of quality-of-life instruments

The wide range of health-related quality-of-life (HRQoL) instruments used in haematology makes it challenging for haematologists and other care team members in practice to select, use and understand the scoring system and finally interpret the results. The main objectives of this study were to: (a) provide a comprehensive list of quality-of-life issues important to patients suffering from haematological malignancies, identified through the literature; (b) provide a list of health-related quality-of-life (HRQoL) instruments used in haematological malignancies in both daily clinical practice and research; and (c) evaluate the relevance and comprehensibility of the identified instruments in haematological malignancies. Systematic literature review of two databases, followed by addition of articles by manual searching, was carried out. The articles focusing on the primary studies, which have used semi-structured/structured interviews or surveys to identify issues important to HM patients, and other studies describing the results of testing measurement properties, such as reliability, validity and responsiveness of the instruments currently used to evaluate the HRQoL in different HMs, were included. Fifty-seven studies reported development and validation of 30 HRQoL instruments, which have been used in haematology. Twenty-four studies were identified using qualitative methods to report HRQoL issues and symptoms from a patient's perspective. No identified instrument captured all the issues identified from the qualitative studies. None of the instruments reviewed appeared to have been developed for use in clinical practice and specifically for patients with HM, except MyPOS. Furthermore, measurement properties were established, largely, in clinical trial scenarios. There is a need for development of a new HRQoL instrument entirely based on involvement of patients with haematological malignancies.

Validation of the Thai version of the family reported outcome measure (FROM-16)© to assess the impact of disease on the partner or family members of patients with cancer

BACKGROUND

Cancer not only impairs a patient's physical and psychosocial functional behaviour, but also contributes to negative impact on family members' health related quality of life. Currently, there is an absence of a relevant tool in Thai with which to measure such impact. The aim of this study was to translate and validate the Family Reported Outcome Measure (FROM-16) in Thai cancer patients' family members.

METHODS

Thai version of FROM-16 was generated by interactive forward-backward translation process following standard guidelines. This was tested for psychometric properties including reliability and validity, namely content validity, concurrent validity, known group validity, internal consistency, exploratory and confirmatory factor analysis. Construct validity was examined by comparing the Thai FROM-16 version with the WHOQOL-BREF-THAI.

RESULTS

The internal consistency reliability was strong (Cronbach's alpha = 0.86). A Negative moderate correlation between the Thai FROM-16 and WHOQOL-BREF-THAI was observed (r = - 0.4545, p < 0.00), and known group validity was proved by a statistically significant higher score in family members with high burden of care and insufficient income. The factor analysis supported both 3-factor and 2-factor loading model with slight difference when compared with the original version.

CONCLUSIONS

The Thai FROM-16 showed good reliability and validity in Thai family members of patients with cancer. A slight difference in factor analysis results compared to the original version could be due to cross-culture application.

Occurrence, Chronicity and Intensity of Itch in a Clinical Consecutive Sample of Patients with Skin Diseases: A Multi-centre Study in 13 European Countries

Itch is an unpleasant symptom, affecting many dermatological patients. Studies investigating the occurrence and intensity of itch in dermatological patients often focus on a single skin disease and omit a control group with healthy skin. The aim of this multi-centre study was to assess the occurrence, chronicity and intensity (visual analogue scale 0-10) of itch in patients with different skin diseases and healthy-skin controls. Out of 3,530 dermatological patients, 54.3% reported itch (mean ± standard deviation itch intensity 5.5 ± 2.5), while out of 1,094 healthy-skin controls 8% had itch (3.6 ± 2.3). Chronic itch was reported by 36.9% of the patients and 4.7% of the healthy-skin controls. Itch was most frequent (occurrence rates higher than 80%) in patients with unclassified pruritus, prurigo and related conditions, atopic dermatitis and hand eczema. However, many patients with psychodermatological conditions and naevi also reported itch (occurrence rates higher than 19%).

The Reliability and Relevance of a Quality of Decision Making Instrument, Quality of Decision-Making Orientation Scheme (QoDoS), for Use During the Lifecycle of Medicines

Introduction: The Quality of Decision-Making Orientation Scheme (QoDoS) was developed to provide organisations involved in submission, approval and reimbursement of new medicines with a tool to improve the quality of their decision-making processes and is considered the most promising tool for such purpose. This study aimed to further establish the measurement properties of the QoDoS by evaluating its reliability (internal consistency and test-retest reliability) and relevance in the target population.

Methods: The study participants consisted of 55 individuals recruited from pharmaceutical companies, regulatory and HTA agencies. It was designed as a longitudinal study with participants assessed on two different occasions, at baseline (test 1) and then 7 days later (test 2). Internal consistency reliability was assessed with Cronbach’s alpha and the test-retest reliability was evaluated using the intraclass correlation coefficients (ICC) based on absolute agreement, 2 way mixed-effects model for the four QoDoS domains. The relevance of the QoDoS was evaluated by applying cognitive debriefing using five short feedback questions following test 1.

Results: Test 1 was completed by 44 study participants (80% response rate) and test 2 was completed by 32 of the 44 individuals, resulting in a 73% response rate. Cronbach’s alpha coefficient was greater than 0.7 across all the domains for test 1 and test 2, ranging from 0.71 to 0.79, indicating good consistency of responses. For the overall score across all 47 items, the Cronbach’s alpha coefficient was 0.81 for test 1 and 0.86 for test 2, which is rated as very good. The four QoDoS domains showed moderate to strong reproducibility (ICC range: 0.63–0.86). The outcome of the cognitive debriefing from the 43 respondents (98% response rate) confirmed the relevance (95% agreement), language clarity (95%) and completeness of items (86%); the clarity of the scaling (91%) as well as spontaneity of responses (95%).

Conclusion: These results provide strong support for the relevance and reliability of the QoDoS, which are key properties for future longitudinal and cross-sectional applications of the instrument when evaluating quality of decision making by those involved in the lifecycle of medicines.

The South African Regulatory System: Past, Present, and Future

The drive for improved regulatory systems and the establishment of a more effective regulatory framework in South Africa has been evident for the past two decades but despite political intentions and legislative revisions success has been limited to date. Efforts to address the increasing volume of applications that have been received have to date failed and resources have been stretched to capacity resulting in the development of a significant backlog and extended timelines for product registration. The promulgation of the recently amended Medicines and Related Substance Act of 1965 triggered the establishment of the South African Health Products Regulatory Authority (SAHPRA) as a separate juristic person outside of the National Department of Health to replace the former medicine regulatory authority the Medicines Control Council (MCC). The aim of this review is to provide the historical context supporting the new regulatory environment in South Africa and the transition from the MCC to SAHPRA. Key recommendations to SAHPRA to ensure the full potential of the new regulatory environment in South Africa include: establishing a quality management system to safeguard accountability, consistency and transparency and to streamline the implementation of good review practices including quality decision-making practices and benefit-risk assessment; the measurement and monitoring of regulatory performance, targets for overall approval time and key review milestones to instill a culture of accurate metrics collection and measurement of key performance indicators and their continuous improvement and the employment of a risk-based approach to the evaluation of medical products and codify the use of facilitated regulatory pathways in policy and culture. The application of a risk-based approach to regulatory review commensurate with a product's risk to patients will facilitate the application of increased resources for pharmacovigilance activities and to support the reliance and recognition of reference agencies.

Quality, Non-clinical and Clinical Considerations for Biosimilar Monoclonal Antibody Development: EU, WHO, USA, Canada, and BRICS-TM Regulatory Guidelines

Objective: The aim was to critically evaluate well-established regulatory agencies mAb biosimilar guidelines for development and marketing authorization about quality, efficacy and safety and compare to BRICS-TM regulations to identify challenges. Materials and Methods: The current valid guidelines of EMA, WHO, USFDA, BGTD/HC, ICH, and BRICS-TM were obtained from official websites and comparative qualitative review was performed. Results: The review revealed that Health Canada uses mAb specific guidelines from EMA or USFDA when necessary. The BRICS agencies (except Russia) have incorporated some or most of the WHO SBP TRS and related annexes in similar national biotechnological/biological guidelines; however, gaps or insufficient information have been identified. The Russian Federation has issued general product registration guideline/s with very brief information about mAbs. The TMMDA (Turkey) has published an updated biosimilar guideline which parallels those of the EMA and the ones from WHO; however, no mAb specific guidelines are published. COFEPRIS (Mexico) has published a biotechnological/biological product registration guideline with no information about mAb. The SAHPRA biosimilar guideline has an annex on mAbs which focuses on non-clinical and clinical aspects. The comparative evaluation of BRICS-TM agencies indicates a gap pertaining to clarification for physico-chemical characterization, manufacturing process, overages and compatibility requirements between biological substances and excipients specifically on mAbs. In vitro assay requirements seem quite aligned with those of WHO, whereas in vivo studies mostly have disparity in terms of necessity, type of studies as well as design and criteria. Clinical safety and efficacy studies are indicated in emerging regulatory agencies, however detailed information pertaining to design, size of populations, requirements for primary and secondary endpoints, clarity and evaluation criteria differ. In general, BRICS-TM agencies allow extrapolation of indications provided that pre-defined conditions are met. Interchangeability, switching and substitution of biosimilars are not defined in most of BRIC-TM guidelines whereas South Africa, by law, allows neither interchangeability nor substitution. Pediatric research remains questionable across BRICS-TM. Conclusions: EMA, USFDA guidelines are broadly aligned with WHO and in addition, they also contain specific requirements pertaining to their own region. BRICS-TM has considerably less defined mAb specific biosimilar development and comparability parameters in their published guidelines.

The Regulatory Review Process in South Africa: Challenges and Opportunities for a New Improved System

BACKGROUND

The aims of this study were to assess the regulatory review process in South Africa from 2015 to 2017, identify the key milestones and timelines; evaluate the effectiveness of measures to ensure consistency, transparency, timeliness, and predictability in the review process; and to provide recommendations for enhanced regulatory practices.

METHODS

A questionnaire was completed by the Medicines Control Council (MCC) to describe the organization of the authority, record key milestones and timelines in the review process and to identify good review practices (GRevPs).

RESULTS

Currently, the MCC conducts a full assessment of quality, efficacy, and safety data in the review of all applications. The overall regulatory median approval time decreased by 14% in 2017 (1411 calendar days) compared with that of 2016, despite the 27% increase in the number of applications. However, the MCC has no target for overall approval time of new active substance applications and no targets for key review milestones. Guidelines, standard operating procedures, and review templates are in place, while the formal implementation of GRevPs and the application of an electronic document management system are planned for the near future.

CONCLUSIONS

As the MCC transitions to the newly established South Africa Health Products Regulatory Authority, it would be crucial for the authority to recognize the opportunities for an enhanced regulatory review and should consider models such as abridged assessment, which encompass elements of risk stratification and reliance. It is hoped that resource constraints may then be alleviated and capacity developed to meet target timelines.

A Comparison of Reimbursement Recommendations by European HTA Agencies: Is There Opportunity for Further Alignment?

Introduction: In Europe and beyond, the rising costs of healthcare and limited healthcare resources have resulted in the implementation of health technology assessment (HTA) to inform health policy and reimbursement decision-making. European legislation has provided a harmonized route for the regulatory process with the European Medicines Agency, but reimbursement decision-making still remains the responsibility of each country. There is a recognized need to move toward a more objective and collaborative reimbursement environment for new medicines in Europe. Therefore, the aim of this study was to objectively assess and compare the national reimbursement recommendations of 9 European jurisdictions following European Medicines Agency (EMA) recommendation for centralized marketing authorization.

Methods: Using publicly available data and newly developed classification tools, this study appraised 9 European reimbursement systems by assessing HTA processes and the relationship between the regulatory, HTA and decision-making organizations. Each national HTA agency was classified according to two novel taxonomies. The System taxonomy, focuses on the position of the HTA agency within the national reimbursement system according to the relationship between the regulator, the HTA-performing agency, and the reimbursement decision-making coverage body. The HTA Process taxonomy distinguishes between the individual HTA agency's approach to economic and therapeutic evaluation and the inclusion of an independent appraisal step. The taxonomic groups were subsequently compared with national HTA recommendations.

Results: This study identified European national reimbursement recommendations for 102 new active substances (NASs) approved by the EMA from 2008 to 2012. These reimbursement recommendations were compared using a novel classification tool and identified alignment between the organizational structure of reimbursement systems (System taxonomy) and HTA recommendations. However, there was less alignment between the HTA processes and recommendations.

Conclusions: In order to move forward to a more harmonized HTA environment within Europe, it is first necessary to understand the variation in HTA practices within Europe. This study has identified alignment between HTA recommendations and the System taxonomy and one of the major implications of this study is that such alignment could support a more collaborative HTA environment in Europe.

Evaluating Quality of Decision-Making Processes in Medicines' Development, Regulatory Review, and Health Technology Assessment: A Systematic Review of the Literature

Introduction: Although pharmaceutical companies, regulatory authorities, and health technology assessment (HTA) agencies have been increasingly using decision-making frameworks, it is not certain whether these enable better quality decision making. This could be addressed by formally evaluating the quality of decision-making process within those organizations. The aim of this literature review was to identify current techniques (tools, questionnaires, surveys, and studies) for measuring the quality of the decision-making process across the three stakeholders.

Methods: Using MEDLINE, Web of Knowledge, and other Internet-based search engines, a literature review was performed to systematically identify techniques for assessing quality of decision making in medicines development, regulatory review, and HTA. A structured search was applied using key words and a secondary review was carried out. In addition, the measurement properties of each technique were assessed and compared. Ten Quality Decision-Making Practices (QDMPs) developed previously were then used as a framework for the evaluation of techniques identified in the review. Due to the variation in studies identified, meta-analysis was inappropriate.

Results: This review identified 13 techniques, where 7 were developed specifically to assess decision making in medicines' development, regulatory review, or HTA; 2 examined corporate decision making, and 4 general decision making. Regarding how closely each technique conformed to the 10 QDMPs, the 13 techniques assessed a median of 6 QDMPs, with a mode of 3 QDMPs. Only 2 techniques evaluated all 10 QDMPs, namely the Organizational IQ and the Quality of Decision Making Orientation Scheme (QoDoS), of which only one technique, QoDoS could be applied to assess decision making of both individuals and organizations, and it possessed generalizability to capture issues relevant to companies as well as regulatory authorities.

Conclusion: This review confirmed a general paucity of research in this area, particularly regarding the development and systematic application of techniques for evaluating quality decision making, with no consensus around a gold standard. This review has identified QoDoS as the most promising available technique for assessing decision making in the lifecycle of medicines and the next steps would be to further test its validity, sensitivity, and reliability.

Understanding clinician influences and patient perspectives on outpatient discharge decisions: a qualitative study

OBJECTIVE

To observe the influences on clinicians when discharging patients, to explore patients' perspectives concerning their discharge or follow-up decision and to identify what patients think is important for clinicians to consider when taking a discharge decision.

DESIGN

Qualitative study involving observations of consultations and semistructured interviews with outpatients.

SETTING

National Health Service outpatient clinics at a university hospital secondary referral centre.

PARTICIPANTS

64 consultations were observed followed by 56 interviews with patients aged over 18 years.

MAIN OUTCOME MEASURE

Analysis of patients' perspectives and expectations concerning whether or not they were discharged.

RESULTS

25 types of influences were observed to be influencing the discharge decision process. All 31 discharged patients appeared to accept the clinicians' decision; however, 10 (22%) of those patients later expressed disappointment. Patients' discontent was due to perceived clinicians' uncertainty in diagnosis (patients mentioning=2), poor acceptance of the diagnosis (2), disease not 'cured' (4), differing perception on medical needs (2), lack of concern for job demands (1), felt uninvolved in the decision-making (4), feeling rushed (3), prolonged open appointment (2), pushed to seek private care due to healthcare budget constraints (2), language barrier (1) and not keen to continue follow-up with general practitioner (2). Patients were happy when there was certainty of the diagnosis (19), clear treatment plan (16), advised on treatment side effects (7), given a contact number if symptoms recurred (4), considering their travelling and job demands (3).

CONCLUSIONS

This study highlights the importance of accurately perceiving patients' perspectives in ensuring the appropriateness of outpatient discharge. There was a disparity between patients' and clinicians' perception on what was an appropriate discharge. This included discrepancies concerning diagnostic certainties, private healthcare as an alternative, need for easy reaccess and choice of words surrounding discharge. Medical education should include handling these issues.

Assessing the Quality of Decision Making in the Development and Regulatory Review of Medicines: Identifying Biases and Best Practices

BACKGROUND

Although the quality of decision making (QDM) in the development and regulatory review of medicines influences the delivery of new products, there appears to be no suitable instrument to assess QDM in this area. The aim of this study was to assess differences in QDM using a validated instrument, the Quality of Decision-Making Orientation Scheme (QoDoS), to identify best practices and biases affecting individuals and their organization, as well as to assess differences in decision-making behaviors between pharmaceutical companies and regulatory agencies. QoDoS also enables the measurement against 10 quality decision-making practices (QDMPs) that underpin a quality process.

METHODS

QoDoS, consisting of 47 items that assess individual and organizational decision-making approaches and influences, was completed by 76 participants from regulatory agencies and pharmaceutical companies.

RESULTS

Having a systematic, structured approach to aid in decision making is achieved to a greater extent at an individual level (72%) compared with that of the organization (38%). Key differences between company and agency decision making were uncovered. While it was recognized that both stakeholders felt that their decision making could be improved (100% agencies; 92% companies), training in the science of decision making was rarely provided.

CONCLUSIONS

QoDoS has the ability to measure differences in QDM between individuals and organizations within companies and agencies. The benefits of assessing QDMPs with QoDoS include enabling an increased awareness of biases and best practices that should be incorporated into a decision-making framework; increasing productivity and reducing uncertainty around decision making, thereby resulting in more predictable outcomes for organizations. In addition, it provides a basis for discussion of the issues in decision making within an organization as well as between stakeholders to encourage a level of partnership. Finally, measurements of QDM will enable trust, consistency, transparency, and timeliness to be built into critical decisions that affect medicines' availability.

Health Technology Assessment (HTA) Case Studies: Factors Influencing Divergent HTA Reimbursement Recommendations in Australia, Canada, England, and Scotland

OBJECTIVES

To evaluate the national regulatory, health technology assessment (HTA), and reimbursement pathways for public health care in Australia, Canada, England, and Scotland, to compare initial Canadian national HTA recommendations with the initial decisions of the other HTA agencies, and to identify factors for differing national HTA recommendations between the four HTA agencies.

METHODS

Information from the public domain was used to develop a regulatory process map for each jurisdiction and to compare the HTA agencies' reimbursement recommendations. Medicines that were reviewed by all four agencies and received a negative recommendation from only one agency were selected as case studies.

RESULTS

All four countries have a national HTA agency. Their reimbursement recommendations are guided by both clinical efficacy and cost-effectiveness, and the necessity for patient input. Their activities, however, vary because of different mandates and their unique political, social, and population needs. All have an implicit or explicit quality-adjusted life-year threshold. The seven divergent case studies demonstrate examples in which new medicine-indication pairs have been rejected because of uncertainties surrounding a range of factors including cost-effectiveness, comparator choice, clinical benefit, safety, trial design, and submission timing.

CONCLUSIONS

The four HTA agencies selected for inclusion in this study share common factors, including a focus on clinical efficacy and cost-effectiveness in their decision-making processes. The differences in recommendations could be considered to be due to an individual agency's approach to risk perception, and the comparator choice used in clinical and cost-effectiveness studies.

Emerging Guidelines for Patient Engagement in Research

There is growing recognition that involving patients in the development of new patient-reported outcome measures helps ensure that the outcomes that matter most to people living with health conditions are captured. Here, we describe and discuss different experiences of integrating patients as full patient research partners (PRPs) in outcomes research from multiple perspectives (e.g., researcher, patient, and funder), drawing from three real-world examples. These diverse experiences highlight the strengths, challenges, and impact of partnering with patients to conceptualize, design, and conduct research and disseminate findings. On the basis of our experiences, we suggest basic guidelines for outcomes researchers on establishing research partnerships with patients, including: 1) establishing supportive organizational/institutional policies; 2) cultivating supportive attitudes of researchers and PRPs with recognition that partnerships evolve over time, are grounded in strong communication, and have shared goals; 3) adhering to principles of respect, trust, reciprocity, and co-learning; 4) addressing training needs of all team members to ensure communications and that PRPs are conversant in and familiar with the language and process of research; 5) identifying the resources and advanced planning required for successful patient engagement; and 6) recognizing the value of partnerships across all stages of research. The three experiences presented explore different approaches to partnering; demonstrate how this can fundamentally change the way research work is conceptualized, conducted, and disseminated; and can serve as exemplars for other forms of patient-centered outcomes research. Further work is needed to identify the skills, qualities, and approaches that best support effective patient-researcher partnerships.

Establishing the values for patient engagement (PE) in health-related quality of life (HRQoL) research: an international, multiple-stakeholder perspective

PURPOSE

Active patient engagement is increasingly viewed as essential to ensuring that patient-driven perspectives are considered throughout the research process. However, guidance for patient engagement (PE) in HRQoL research does not exist, the evidence-base for practice is limited, and we know relatively little about underpinning values that can impact on PE practice. This is the first study to explore the values that should underpin PE in contemporary HRQoL research to help inform future good practice guidance.

METHODS

A modified 'World Café' was hosted as a collaborative activity between patient partners, clinicians and researchers: self-nominated conference delegates participated in group discussions to explore values associated with the conduct and consequences of PE. Values were captured via post-it notes and by nominated note-takers. Data were thematically analysed: emergent themes were coded and agreement checked. Association between emergent themes, values and the Public Involvement Impact Assessment Framework were explored.

RESULTS

Eighty participants, including 12 patient partners, participated in the 90-min event. Three core values were defined: (1) building relationships; (2) improving research quality and impact; and (3) developing best practice. Participants valued the importance of building genuine, collaborative and deliberative relationships-underpinned by honesty, respect, co-learning and equity-and the impact of effective PE on research quality and relevance.

CONCLUSIONS

An explicit statement of values seeks to align all stakeholders on the purpose, practice and credibility of PE activities. An innovative, flexible and transparent research environment was valued as essential to developing a trustworthy evidence-base with which to underpin future guidance for good PE practice.

The Development and Validation of a Generic Instrument, QoDoS, for Assessing the Quality of Decision Making

INTRODUCTION

The impact of decision-making during the development and the regulatory review of medicines greatly influences the delivery of new medicinal products. Currently, there is no generic instrument that can be used to assess the quality of decision-making. This study describes the development of the Quality of Decision-Making Orientation Scheme QoDoS(©) instrument for appraising the quality of decision-making.

METHODS

Semi-structured interviews about decision-making were carried out with 29 senior decision makers from the pharmaceutical industry (10), regulatory authorities (9) and contract research organizations (10). The interviews offered a qualified understanding of the subjective decision-making approach, influences, behaviors and other factors that impact such processes for individuals and organizations involved in the delivery of new medicines. Thematic analysis of the transcribed interviews was carried out using NVivo8® software. Content validity was carried out using qualitative and quantitative data by an expert panel, which led to the developmental version of the QoDoS. Further psychometric evaluations were performed, including factor analysis, item reduction, reliability testing and construct validation.

RESULTS

The thematic analysis of the interviews yielded a 94-item initial version of the QoDoS(©) with a 5-point Likert scale. The instrument was tested for content validity using a panel of experts for language clarity, completeness, relevance and scaling, resulting in a favorable agreement by panel members with an intra-class correlation coefficient value of 0.89 (95% confidence interval = 0.56, 0.99). A 76-item QoDoS(©) (version 2) emerged from content validation. Factor analysis produced a 47-item measure with four domains. The 47-item QoDoS(©) (version 3) showed high internal consistency (n = 120, Cronbach's alpha = 0.89), high reproducibility (n = 20, intra-class correlation = 0.77) and a mean completion time of 10 min. Reliability testing and construct validation was successfully performed.

CONCLUSION

The QoDoS(©) is both reliable and valid for use. It has the potential for extensive use in medicines development by both the pharmaceutical industry and regulatory authorities. The QoDoS(©) can be used to assess the quality of decision-making and to inform decision makers of the factors that influence decision-making.