Quality of Reporting in Poster versus Oral Presentations at the American Society of Plastic Surgeons 2008 Conference in Chicago:

Standardized methodological assessment of research presentations (SHARP): development of a new instrument

BACKGROUND

There are currently no validated guidelines to assess the quality of the content and the delivery style of scientific podium surgical presentations. We have developed a simple, short, and reliable instrument to objectively assess the overall quality of scientific podium presentations.

METHODS

A simple and efficient rating instrument was developed to assess the scientific content and presentation style/skills of the surgical residents' presentations from 1996 to 2013. Absolute and consistency agreement for the different sections of the instrument was determined and assessed overtime, by stage of the project and study design. Intraclass correlation coefficients with 95% confidence intervals were calculated and reported using a mixed-effects model.

RESULTS

Inter-rater reliability for both absolute and consistency agreement was substantial for total score and for each of the 3 sections of the instrument. The absolute agreement for the overall rating of the presentations was .87 (.63 to .98) and .78 (.50 to .95), and the consistency agreement was .90 (.70 to .99) and .87 (.67 to .97) for the 2012 and 2013 institutional research presentations, respectively. Rater agreement for evaluating project stage and different study designs varied from .70 to .81 and was consistent over the years. The consistency agreement in rating of the presentation was .77 for both faculty and resident raters.

CONCLUSIONS

Standardized methodological assessment of research presentations (SHARP) instrument rates the scientific quality of the research and style of the delivered presentation. It is highly reliable in scoring the quality of the all study designs regardless of their stage. We recommend that researchers focus on presenting the key concepts and significant elements of their evidence using visually simple slides in a professionally engaging manner for effective delivery of their research and better communication with the audience.

Developing the art of scientific presentation

PURPOSE

Few guidelines exist regarding the most effective approach to scientific oral presentations. Our purpose is to (1) develop a standardized instrument to evaluate scientific presentations based on a comprehensive review of the available literature regarding the components and organization of scientific presentations and (2) describe the optimal characteristics of scientific presentations.

METHODS

At the Sixty-sixth (2011) Annual Meeting of the American Society for Surgery of the Hand, 69 presentations were evaluated by at least 2 independent observers. A rating instrument was developed a priori to examine presentation content (background, methods, results, and conclusions), presentation style (speech, structure, delivery, slide aesthetics), and overall quality. We examined correlations between reviewers' ratings of each component as well as overall perceived quality of the presentation using regression analysis. Intraclass correlation coefficients were calculated to measure the degree of variation because of reviewer disagreement and identify the aspects of presentations that contribute to overall quality.

RESULTS

Reviewer agreement was high for presentation content, and less than 1% of variation was caused by reviewer disagreement for background, methods, and conclusions. With respect to presentation style, reviewers agreed most frequently regarding speech and slide appearance, and only 9% and 13%, respectively, of the variation was caused by reviewer disagreement. Disagreement was higher for delivery and presentation structure, and 21% of the variation was attributable to reviewer disagreement. Speaker delivery and slide appearance were the most important predictors of presentation quality, followed by the quality of the presentation of conclusions and background information. Presentation of methods and results were not associated with overall presentation quality.

CONCLUSIONS

Distinct aspects of presentation content and style correlate with quality, which can be reliably and objectively measured. By focusing on selected concepts with visually simple slides, speakers can enhance their delivery and may potentially improve the audience's comprehension of the study findings.