Measuring research influence on drug policy: A case example of two epidemiological monitoring systems

Navigating China's carbon neutrality journey: insights from policy instruments and implementation strategies across provincial regions

China's critical reliance on well-crafted public policies, coupled with the effective execution of central government directives at the local level, drives the achievement of the "dual carbon" goal including the peaking of CO2 emissions and attaining carbon neutrality. Therefore, examining policy records can unveil the holistic strategy for attaining carbon neutrality during the period of peak CO2 emissions; at the same time, it can also highlight the potential obstacles in policy implementation. In this study, we adopt a policy instruments perspective to investigate data related to policies addressing peak CO2 emissions across 29 provincial administrative regions in China. We apply Nvivo12 software to conduct a quantitative literature assessment and content analysis to establish a theoretical framework for the policy process. This framework encompasses dimensions such as political feasibility, regional coordination, attributes of low-carbon initiatives, and policy refinement. Subsequently, we employ the model to carry out a retrospective analysis of policy documents pertaining to peak CO2 emissions in China. Our research findings underscore the pivotal role of political feasibility in shaping policy effectiveness, while also highlighting the facilitative influence of regional coordination, shedding light on the essential synergy between provinces and cities in achieving emissions reduction goals. Similarly, the estimated results highlight the motivating impact of specific attributes within low-carbon initiatives. Moreover, policy enhancements are identified as a critical driver in advancing the path toward carbon neutrality. Consequently, to achieve the objective of carbon neutrality, it is imperative for every province and city to sequentially reach the peak of CO2 emissions. Our research offers a comprehensive "China strategy," providing valuable insights to guide future policy formulation and accelerate progress toward sustainable environmental objectives.

The Meaning of Stigma About Illegal Drug Use and Its Contribution to Public Policymaking in Colombia

The objective of this study is to understand the meaning of the stigma about illegal drug consumption and its contribution to public policymaking in Colombia from the perspective of experts on this topic. Research was carried out based on the methodological perspective of the grounded theory. Twenty participants with extensive experience in the design of regulations, plans, or programs about demand reduction in Colombia were interviewed, using tools such as in-depth interviews and analytical memos. The analysis was performed by means of open, axial, and selective coding techniques and the development of a conditional/consequential matrix. The findings are presented according to the structure of the pattern matrix to show the social process found and include (a) the phenomenon, the fear of being stigmatized as a promoter of more liberal public policies; (b) the context, the Colombia’s history and the policymaking to reduce drug consumption; (c) the conditions, control of governance by influential groups in power; and (d) the strategies to address it, prioritize less sensitive issues, and the consequences, formulation of repressive public policies with the impact on the rights of people who use drugs. From the perspective of the interviewees, the fear of being stigmatized as promoters of more liberal public policies and contrary to the expectations of the social and power classes leads the rulers to have a low-profile discourse on this subject so as not to assume the costs politicians that this fact would imply.

Health Policies Based on Patient Satisfaction: A Bibliometric Study

Healthcare decision-makers increasingly face a changing and ever-evolving landscape, forcing them to formulate public policies based on the results from different scientific investigations. This article evaluates the field of research on patient satisfaction as a basis for health policies. The analysis was carried out with a sample of 621 articles published between 2000 and 2020 in the Scopus database. The world's largest producer and research co-operator on patient satisfaction and health policy was the United States. However, the most prolific authors, institutions, and journals are of British origin. Regarding the themes, we find that, in economic and management matters, scientific production is scarce. To study the evolution of keywords, we divided the study period into two periods of an equal number of years. In both sub-periods, the keyword "Human" stands out. In the second sub-period, the word "Perception" stands out, which indicates the current attention paid to the patient's opinion.

Mobile supervised consumption services in Rural British Columbia: lessons learned

Background

In 2016, a public health emergency was declared in British Columbia due to an unprecedented number of illicit drug overdose deaths. Injection drug use was implicated in approximately one third of overdose deaths. An innovative delivery model using mobile supervised consumption services (SCS) was piloted in a rural health authority in BC with the goals of preventing overdose deaths, reducing public drug use, and connecting clients to health services.

Methods

Two mobile SCS created from retrofitted recreational vehicles were used to serve the populations of two mid-sized cities: Kelowna and Kamloops. Service utilization was tracked, and surveys and interviews were completed to capture clients’, service providers’, and community stakeholders’ attitudes towards the mobile SCS.

Results

Over 90% of surveyed clients reported positive experiences in terms of access to services and physical safety of the mobile SCS. However, hours of operation met the needs of less than half of clients. Service providers were generally dissatisfied with the size of the space on the mobile SCS, noting constraints in the ability to respond to overdose events and meaningfully engage with clients in private conversations. Additional challenges included frequent operational interruptions as well as poor temperature control inside the mobile units. Winter weather conditions resulted in cancelled shifts and disrupted services. Among community members, there was variable support of the mobile SCS.

Conclusions

Overall, the mobile SCS were a viable alternative to a permanent site but presented many challenges that undermined the continuity and quality of the service. A mobile site may be best suited to temporarily provide services while bridging towards a permanent location. A needs assessment should guide the stop locations, hours of operation, and scope of services provided. Finally, the importance of community engagement for successful implementation should not be overlooked.

Does citation matter? Research citation in policy documents as an indicator of research impact - an Australian obesity policy case-study

Background

Citation of research in policy documents has been suggested as an indicator of the potential longer-term impacts of research. We investigated the use of research citations in childhood obesity prevention policy documents from New South Wales (NSW), Australia, considering the feasibility and value of using research citation as a proxy measure of research impact.

Methods

We examined childhood obesity policy documents produced between 2000 and 2015, extracting childhood obesity-related references and coding these according to reference type, geographical origin and type of research. A content analysis of the policy documents examined where and how research was cited in the documents and the context of citation for individual research publications.

Results

Over a quarter (28%) of the policy documents (n = 86) were not publicly available, almost two-thirds (63%) contained references, half (47%) cited obesity-related research and over a third (41%) of those containing references used unorthodox referencing styles, making reference extraction laborious. No patterns, in terms of the types of documents more likely to cite research, were observed and the number of obesity research publications cited per document was highly variable. In total, 263 peer-reviewed and 94 non-peer-reviewed obesity research publications were cited. Research was most commonly cited to support a policy argument or choice of solution. However, it was not always possible to determine how or why individual publications were cited or whether the cited research itself had influenced the policy process. Content analysis identified circumstances where research was mentioned or considered, but not directly cited.

Conclusions

Citation of research in policy documents in this case did not always provide evidence that the cited research had influenced the policy process, only that it was accessible and relevant to the content of the policy document. Research citation across these public health policy documents varied greatly and is unlikely to be an accurate reflection of actual research use by the policy agencies involved. The links between citation and impact may be more easily drawn in specific policy areas or types of documents (e.g. clinical guidelines), where research appraisal feeds directly into policy recommendations.

Looking both ways: a review of methods for assessing research impacts on policy and the policy utilisation of research

Background

Measuring the policy and practice impacts of research is becoming increasingly important. Policy impacts can be measured from two directions – tracing forward from research and tracing backwards from a policy outcome. In this review, we compare these approaches and document the characteristics of studies assessing research impacts on policy and the policy utilisation of research.

Methods

Keyword searches of electronic databases were conducted in December 2016. Included studies were published between 1995 and 2016 in English and reported methods and findings of studies measuring policy impacts of specified health research, or research use in relation to a specified health policy outcome, and reviews reporting methods of research impact assessment. Using an iterative data extraction process, we developed a framework to define the key elements of empirical studies (assessment reason, assessment direction, assessment starting point, unit of analysis, assessment methods, assessment endpoint and outcomes assessed) and then documented the characteristics of included empirical studies according to this framework.

Results

We identified 144 empirical studies and 19 literature reviews. Empirical studies were derived from two parallel streams of research of equal size, which we termed ‘research impact assessments’ and ‘research use assessments’. Both streams provided insights about the influence of research on policy and utilised similar assessment methods, but approached measurement from opposite directions. Research impact assessments predominantly utilised forward tracing approaches while the converse was true for research use assessments. Within each stream, assessments focussed on narrow or broader research/policy units of analysis as the starting point for assessment, each with associated strengths and limitations. The two streams differed in terms of their relative focus on the contributions made by specific research (research impact assessments) versus research more generally (research use assessments) and the emphasis placed on research and the activities of researchers in comparison to other factors and actors as influencers of change.

Conclusions

The Framework presented in this paper provides a mechanism for comparing studies within this broad field of research enquiry. Forward and backward tracing approaches, and their different ways of ‘looking’, tell a different story of research-based policy change. Combining approaches may provide the best way forward in terms of linking outcomes to specific research, as well as providing a realistic picture of research influence.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0310-4) contains supplementary material, which is available to authorized users.

Public support for safe consumption sites and syringe services programs to combat the opioid epidemic

We examine Americans' support for two evidence-based harm reduction strategies - safe consumption sites and syringe exchange programs - and their attitudes about individuals who use opioids. We conducted a web-based survey of a nationally representative sample of U.S. adults in July-August 2017 (N = 1004). We measured respondents' support for legalizing safe consumption sites and syringe services programs in their communities and their attitudes toward people who use opioids. We used ordered logistic regression to assess how stigmatizing attitudes toward people who use opioids, political party identification, and demographic characteristics correlated with support for the two harm reduction strategies. Twenty-nine percent of Americans supported legalizing safe consumption sites and 39% supported legalizing syringe services programs. Respondents reported high levels of stigmatizing attitudes toward people who use opioids: 16% of respondents were willing to have a person using opioids marry into their family and 28% were willing to have a person using opioids start working closely with them on a job, and 27% and 10% of respondents rated persons who use opioids as deserving (versus worthless) and strong (versus weak). Stigmatizing attitudes were associated with lower support for legalizing safe consumption sites and syringe services programs. Democrats and Independents were more likely than Republicans to support both strategies. Stigmatizing attitudes toward people who use opioids are a key modifiable barrier to garnering the public support needed to fully implement evidence-based harm reduction strategies to combat the opioid epidemic. Dissemination and evaluation of stigma reduction campaigns are a public health priority.

Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme

BACKGROUND

This report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review.

OBJECTIVES

(1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme.

DATA SOURCES

We searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014.

REVIEW METHODS

This narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015.

RESULTS

The literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers.

DISCUSSION

The findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence's remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities' research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish^® (researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established.

LIMITATIONS

There were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme.

CONCLUSIONS

Research funders can benefit from continuing to monitor and evaluate the impacts of the studies they fund. They should also review the contribution of case studies and expand work on linking trials to meta-analyses and to guidelines.

FUNDING

The National Institute for Health Research HTA programme.

Uses of Research Evidence by State Legislators Who Prioritize Behavioral Health Issues

OBJECTIVE

Disseminating behavioral health (BH) research to elected policy makers is a priority, but little is known about how they use and seek research evidence. This exploratory study aimed to identify research dissemination preferences and research-seeking practices of legislators who prioritize BH issues and to describe the role of research in determining policy priorities. The study also assessed whether these legislators differ from those who do not prioritize BH issues.

METHODS

A telephone-based survey was conducted with 862 state legislators (response rate, 46%). A validated survey instrument assessed priorities and the factors that determined them, research dissemination preferences, and research-seeking practices. Bivariate analyses were used to characterize and compare the two groups.

RESULTS

Legislators who prioritized BH issues (N=125) were significantly more likely than those who did not to identify research evidence as a factor that determined policy priorities (odds ratio=1.91, 95% confidence interval=1.25-2.90, p=.002). Those who prioritized BH issues also attributed more importance to ten of 12 features of research, and the difference was significant for four features (unbiased, p=.014; presented in a concise way, p=.044; delivered by someone known or respected, p=.033; and tells a story, p=.030). Those who prioritized BH issues also engaged more often in eight of 11 research-seeking and utilization practices, and a significance difference was found for one (attending research presentations, p=.012).

CONCLUSIONS

Legislators who prioritized BH issues actively sought, had distinct preferences for, and were particularly influenced by research evidence. Testing legislator-focused BH research dissemination strategies is an area for future research.

Ambivalence about supervised injection facilities among community stakeholders

Background

Community stakeholders express a range of opinions about supervised injection facilities (SIFs). We sought to identify reasons for ambivalence about SIFs amongst community stakeholders in two Canadian cities.

Findings

We used purposive sampling methods to recruit various stakeholder representatives (n = 141) for key informant interviews or focus group discussions. Data were analyzed using a thematic process. We identified seven reasons for ambivalence about SIFs: lack of personal knowledge of evidence about SIFs; concern that SIF goals are too narrow and the need for a comprehensive response to drug use; uncertainty that the community drug problem is large enough to warrant a SIF(s); the need to know more about the “right” places to locate a SIF(s) to avoid damaging communities or businesses; worry that a SIF(s) will renew problems that existed prior to gentrification; concern that resources for drug use prevention and treatment efforts will be diverted to pay for a SIF(s); and concern that SIF implementation must include evaluation, community consultation, and an explicit commitment to discontinue a SIF(s) in the event of adverse outcomes.

Conclusions

Stakeholders desire evidence about potential SIF impacts relevant to local contexts and that addresses perceived potential harms. Stakeholders would also like to see SIFs situated within a comprehensive response to drug use. Future research should determine the relative importance of these concerns and optimal approaches to address them to help guide decision-making about SIFs.

Increasing public support for supervised injection facilities in Ontario, Canada

AIM

To determine the level and changes in public opinion between 2003 and 2009 among adult Canadians about implementation of supervised injection facilities (SIFs) in Canada.

DESIGN

Population-based, telephone survey data collected in 2003 and 2009 were analysed to identify strong, weak, and intermediate support for SIFs.

SETTING

Ontario, Canada

PARTICIPANTS

Representative samples of adults aged 18 years and over.

MEASUREMENTS

Analyses of the agreement with implementation of SIFs in relation to four individual SIF goals and a composite measure.

FINDINGS

The final sample sizes for 2003 and 2009 were 1212 and 968, respectively. Between 2003 and 2009, there were increases in the proportion of participants who strongly agreed with implementing SIFs to: reduce neighbourhood problems (0.309 versus 0.556, respectively); increase contact of people who use drugs with health and social workers (0.257 versus 0.479, respectively); reduce overdose deaths or infectious disease among people who use drugs (0.269 versus 0.482, respectively); and encourage safer drug injection (0.213 versus 0.310, respectively). Analyses using a composite measure of agreement across goals showed that 0.776 of participants had mixed opinions about SIFs in 2003, compared with only 0.616 in 2009. There was little change among those who strongly disagreed with all SIF goals (0.091 versus 0.113 in 2003 and 2009, respectively).

CONCLUSIONS

Support for implementation of supervised injection facilities in Ontario, Canada increased between 2003 and 2009, but at both time-points a majority still held mixed opinions.