Learning from the challenges of undertaking an evaluation of a multi-partner housing support initiative delivered within a hospital setting

Objectives

We present learning from a mixed-methods evaluation of a housing support initiative for hospital inpatients.

Study design

A mixed-methods process evaluation.

Methods

A social housing provider delivered a housing support service in two hospitals (mental health unit and general hospital). Healthcare providers, the social housing provider and academic researchers designed and undertook a co-produced, mixed-methods process evaluation of the intervention. The evaluation included questionnaires, semi-structured interviews, analysis of routinely collected data and economic analysis. Despite commitment from the partners, the evaluation faced challenges. We reflect on the lessons learnt within our discussion paper.

Results

Despite the commitment of the partners, we faced several challenges.

We took an iterative approach to the design and processes of the evaluation to respond to arising challenges. Recruitment of service-users was more difficult than anticipated, requiring additional staff resources. Given the small-scale nature of the intervention, and the quality of data recorded in hospital records, the planned economic analysis was not feasible. Positive factors facilitating evaluation included involvement of staff delivering the intervention, as well as managers. Being able to offer payment to partner organisations for staff time also facilitated ongoing engagement.

Conclusions

Multi-partner evaluations are useful, however, researchers and partners need to be prepared to take an iterative, resource intensive approach. Both availability and quality of routine data, and the resources required to support data collection, may limit feasibility of specific methods when evaluating small-scale cross-sector initiatives. Thus, this necessitates a flexible approach to design and analysis.

OC-090 Heterotopic Ossification in a Laparostomy Wound Complicating Reconstruction– a Case Report

Introduction

Heterotopic Ossification (HO) is a benign process of osseous matrix deposition in extra-skeletal tissues; it is well recognised in the orthopaedic setting, especially after blast injury, but is a rare phenomenon following abdominal surgery and is likely under reported. The aetiology is unclear but may be due to differentiation of local mesenchymal cells into osteoblasts leading to calcification of the extracellular matrix, or damage to xiphoid or pubis causing liberation of bone-forming cells into the wound.

Case Report

A 46-year-old man, presented with perforated diverticulitis and underwent an emergency Hartmann's procedure with temporary abdominal closure. Sequential returns to theatre failed to achieve fascial closure and he was managed as a laparostomy. Two years later he underwent reversal of Hartmann's with complex abdominal wall reconstruction (CAWR) – the procedure was complicated by an unrecognised ring of HO around the edge of his laparostomy wound, which necessitated additional excision of fascia. Only partial fascial closure could be achieved leaving an 8×4cm bridge over polypropylene mesh. After a complicated post-operative course, he was discharged home with intestinal continuity and healed abdominal wounds.

Discussion

Pre-operative review of the 2-D axial CT images failed to identify the almost complete ring of HO clearly evident on 3-D reconstructions which impacted on the outcome of the CAWR; 3-D reconstructions should be routinely reviewed. It is commoner in males, those with prolonged ICU intubation, hollow viscus injury and non-closure of the fascia. Restoration of intestinal continuity can be safely performed with complex abdominal wall reconstruction.

EP.TU.157Can Twitter Attention Predict Citation Metrics? A Machine Learning Aided Analysis

Aims

Surgical journals have developed social media profiles to increase engagement though it remains unclear as to whether social media attention indices for publications act as a surrogate or predictor of traditional citation metrics. This study used machine learning to determine if there is a relationship between Twitter mentions and number of citations for surgical publications.

Methods

We identified all original research and review papers published in Annals of Surgery, BJS and JAMA Surgery in 2019. Citations data and Twitter mentions were retrieved and the Spearman rank coefficient was used to determine degree of correlation between the two variables. An unsupervised machine-learning hierarchical clustering algorithm was used to define clusters of outlying papers. Quantitative and qualitative analysis of the clusters was completed.

Results

413 papers were selected. Median number of citations was 7 (IQR 3-14), median number of Twitter mentions was 40 (IQR 15-79). No correlation between Twitter mentions and number of citations was observed (Spearman’s rho 0.076 p-value 0.124). Cluster analysis identified one large (cluster 2, 367/413 papers) and six small clusters. Analysis of cluster 2 revealed a weak but significant correlation between citations and Twitter mentions (Spearman’s rho 0.107 p-value 0.041). The remaining six clusters were characterised by an out of proportion number of Twitter mentions compared to citations or vice versa.

Conclusions

Twitter mentions should not be used as a surrogate or predictor of traditional citation metrics. In our database the relationship between social media attention and citations was skewed by a small number of outlying papers.

SP2.2.3How to make the perfect tweet: exploring factors which affect retweetability of surgical research

Aims

Twitter is a powerful platform for sharing information. Retweets allow users to reach a wider audience quickly. Our aim was to determine what tweet and twitter-account related factors affect the “retweetability” of surgical publications.

Methods

All papers published (in print) in Annals of Surgery, BJS and JAMASurgery in 2019 were selected. We identified each paper’s first appearance on Twitter and analysed characteristics of the posting twitter account and the tweet itself. The association between such characteristics and the number of retweets was investigated.

Results

413 papers were selected. Median number of retweets was 3 (range, 0-121). The first tweet created about a paper was usually from a reader (42%) with only a minority created by the journal (17%) or the authors (20%).

Use of visual abstracts (22/413, p<0.001), pictures (145/413, p<0.001), hashtags (149/413, p<0.001), mentions (183/413, p<0.001), number of followers (p < 0.001), age of twitter account (p = 0.003) and authors as creators of the first tweet (p < 0.001) were associated with higher retweets. When the journal created the first tweet, retweets were also increased (P < 0.001). Posting the title of the paper alone attracted fewer retweets (p < 0.001).

Multivariate analysis confirmed visual abstracts (p < 0.001), hashtags (p=p.008), pictures (p < 0.001) and authors as creators of the first tweet (p < 0.001) were predictive of increased retweetability.

Conclusions

Authors and journals should create the first presence on social media to improve engagement with their papers. Such tweets should use visual abstracts or pictures, relevant hashtags and avoid solely posting the paper title to make their research more widely shared.

EP.WE.372Has COVID-19 affected the attention surgical papers are receiving?

Aim

Unsurprisingly, much of the medical profession has focussed on treating covid-19 over the past year often to the detriment of other pathologies. Engagement with academic literature may follow a similar trend; this paper used standard social media attention and citations metrices to assess whether the pandemic has affected engagement with surgical literature.

Method

Twitter Mentions and Mendeley Readers Data were retrieved for all papers published in Annals of Surgery, BJS and JAMASurgery between January 2019 and October 2020. The non-parametric Mann-Whitney U test was used to compare Twitter Mentions and Mendeley Readers for COVID surgical vs non-COVID surgical publications (in 2020) and all surgical papers published before and after the advent of COVID-19. A control database of all papers published in NEJM, BMJ and Lancet over the same period of time was also created.

Results

There was no difference in Twitter mentions between COVID-19 and non-COVID-19 papers (p-value 0.604); however there were significantly more Mendeley readers of COVID-19 papers than non-COVID-19 papers in 2020 (55 vs 5 median readers, p < 0.001). Surgical papers published in 2020 received significantly fewer Twitter mentions than those published in 2019 (15.5 vs 27, p < 0.001).

Analysis of the non-surgical control dataset revealed the opposite trend; papers published in 2020 received significantly more attention than those published in 2019 (39 vs 30 median Twitter citations, p < 0.001).

Conclusion

Surgical papers published during the COVID-19 pandemic received significantly fewer social media mentions. Such reduced visibility has the potential to affect future citation metrics and dissemination of surgical knowledge