Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?

Amazon's Mechanical Turk (MTurk) is a relatively new website that contains the major elements required to conduct research: an integrated participant compensation system; a large participant pool; and a streamlined process of study design, participant recruitment, and data collection. In this article, we describe and evaluate the potential contributions of MTurk to psychology and other social sciences. Findings indicate that (a) MTurk participants are slightly more demographically diverse than are standard Internet samples and are significantly more diverse than typical American college samples; (b) participation is affected by compensation rate and task length, but participants can still be recruited rapidly and inexpensively; (c) realistic compensation rates do not affect data quality; and (d) the data obtained are at least as reliable as those obtained via traditional methods. Overall, MTurk can be used to obtain high-quality data inexpensively and rapidly.

Bringing Internet-based education and intervention into mental health practice: afterdeployment.org

BACKGROUND

Internet-facilitated interventions may offer numerous advantages in reaching the large numbers of military service men and women exposed to traumatic events. The Internet is now a primary source of health-related information for consumers and research has shown the effectiveness of web-based interventions in addressing a range of mental health problems.

OBJECTIVE

Clinicians can learn how to bring Internet education and intervention into routine care, to help clients better understand mental health issues and learn skills for self-management of problems.

METHOD

The Afterdeployment.org (AD) Internet site can be used by health care professionals serving U.S. military personnel returning from Iraq and Afghanistan, and their families. The site currently addresses 18 key domains of functioning, including post-traumatic stress, sleep, anger, alcohol and drugs, and military sexual trauma. It provides an extensive amount of client and family education that is suitable for immediate use by clients and providers, as well as the kinds of interactive workshop content and self-assessment tools that have been shown to be helpful in other treatment contexts.

RESULTS

AD CAN BE UTILIZED IN CLINICAL PRACTICE IN A VARIETY OF WAYS: as an adjunct to treatment for PTSD, to supplement existing treatments for a range of post-deployment problems, or as the primary focus of treatment for a client.

CONCLUSIONS

AD represents a kind of service that is likely to become increasingly available in coming years and that is important for mental health providers to actively explore as a tool for extending their reach, improving their efficiency, and improving quality of care.

Marketing to increase participation in a Web-based continuing medical education cultural competence curriculum

INTRODUCTION

CME providers may be interested in identifying effective marketing strategies to direct users to specific content. Online advertisements for recruiting participants into activities such as clinical trials, public health programs, and continuing medical education (CME) have been effective in some but not all studies. The purpose of this study was to compare the impact of 2 marketing strategies in the context of an online CME cultural competence curriculum (www.c-comp.org).

METHODS

In an interrupted time-series quasi-experimental design, 2 marketing strategies were tested: (1) wide dissemination to relevant organizations over a period of approximately 4 months, and (2) Internet paid search using Google Ads (5 consecutive 8-week periods--control 1, cultural/CME advertisement, control 2, hypertension/ content advertisement, control 3). Outcome measures were CME credit requests, Web traffic (visits per day, page views, pages viewed per visit), and cost.

RESULTS

Overall, the site was visited 19,156 times and 78,160 pages were viewed. During the wide dissemination phase, the proportion of visits requesting CME credit decreased between the first (5.3%) and second (3.3%) halves of this phase (p = .04). During the Internet paid search phase, the proportion of visits requesting CME credit was highest during the cultural/CME advertisement period (control 1, 1.4%; cultural/CME ad, 4.3%; control 2, 1.5%; hypertension/content ad, 0.6%; control 3, 0.8%; p < .001). All measures of Web traffic changed during the Internet paid search phase (p < .01); however, changes were independent of the advertisement periods. The incremental cost for the cultural advertisement per CME credit requested was US $0.64.

DISCUSSION

Internet advertisement focusing on cultural competence and CME was associated with about a threefold increase in requests for CME credit at an incremental cost of under US $1; however, Web traffic changes were independent of the advertisement strategy.

Social media policies at US medical schools

BACKGROUND/PURPOSE

Today's medical students are learning in a social media era in which patient confidentiality is at risk yet schools' social media policies have not been elucidated. The purpose of this study is to describe the presence of medical schools on top social media sites and to identify whether student policies for these schools explicitly address social media use.

METHOD

Websites of all 132 accredited US medical schools were independently assessed by two investigators for their presence (as of March 31, 2010) on the most common social networking and microblogging sites (Facebook and Twitter) and their publicly available policies addressing online social networking. Key features from these policies are described.

RESULTS

100% (n=132) of US medical schools had websites and 95.45% (126/132) had any Facebook presence. 25.76% (34/132) had official medical school pages, 71.21% (94/132) had student groups, and 54.55% (72/132) had alumni groups on Facebook. 10.6% of medical schools (14/132) had Twitter accounts. 128 of 132 medical schools (96.97%) had student guidelines or policies publicly available online. 13 of these 128 schools (10.16%) had guidelines/policies explicitly mentioning social media. 38.46% (5/13) of these guidelines included statements that defined what is forbidden, inappropriate, or impermissible under any circumstances, or mentioned strongly discouraged online behaviors. 53.85% (7/13) encouraged thoughtful and responsible social media use.

CONCLUSIONS

Medical schools and their students are using social media. Almost all US medical schools have a Facebook presence, yet most do not have policies addressing student online social networking behavior. While social media use rises, policy informing appropriate conduct in medical schools lags behind. Established policies at some medical schools can provide a blueprint for others to adopt and adapt.

Impact of online lecture-capture on student outcomes in a therapeutics course

OBJECTIVES

To examine the correlation between students accessing recorded lecture files (audio and slides) online and course grades and class attendance.

METHODS

Second professional year (of 6-year program) students in a therapeutics course had access to recorded online lectures for 72 hours following live lectures. The number and duration of lecture accessions were compared to final course grades and class attendance. Course grades were compared to those of a historical control group. At the end of the semester, students completed a brief survey instrument regarding their use and perceptions of online lectures.

RESULTS

No correlation was found between final course grades and the number of lecture accessions (r = 0.0014) or total number of minutes lectures were viewed (r = 0.033), nor between class attendance and minutes viewed (r = 0.2158). Students with access to recorded lectures outperformed the historical control group on the final examination (p < 0.002). Seventy-two percent of students reported no influence of online files on class attendance.

CONCLUSIONS

Posting lectures online did not affect student outcomes, but students did score higher on the final examination.

Continuing medical education methodology: current trends and applications in wound care

The field of professional medical continuing education is changing rapidly. The traditional format for continuing education for health care professionals has long been classroom learning. This form of learning has been found to be relatively ineffective in changing learners' practice patterns. The reasons most often cited are that there is little interaction and that the learning does not occur when the learner is ready to learn. Interaction greatly enhances retention and the tendency for the learner to later apply the learning in actual clinical encounters. There are a growing number of online continuing medical education (CME) Web sites that combine on-demand didactic presentations with the ability to discuss the content with peers. The easy access and interaction potential of this new CME media promises to improve the effectiveness of future CME.

Validation of a web-based dietary questionnaire designed for the DASH (dietary approaches to stop hypertension) diet: the DASH online questionnaire

OBJECTIVE

With the upsurge in online dietary modification programmes, online dietary assessment tools are needed to capture food intake. Although the DASH (Dietary Approaches to Stop Hypertension) diet is recommended by the US Department of Agriculture, there are no online instruments that capture DASH food servings. Our objective was to assess the validity of a new, short, online dietary questionnaire developed to capture intake of DASH food servings. The DASH Online Questionnaire (OLQ) was validated against the well-known Block Food Frequency Questionnaire (FFQ).

DESIGN

This was a cross-sectional validation of the DASH OLQ, which contained eleven food groups (breakfast cereals; dairy; drinks; fats and oils; fruits; grains and snacks; meat, fish and poultry; mixed dishes; sweets; vegetables; and nuts, seeds and legumes). Each subject completed a DASH OLQ once weekly for four weeks and one 98.2 Block FFQ (110 questions) between weeks 2 and 4. DASH OLQ were averaged and then compared with the Block FFQ for nutrient intakes as well as intakes of DASH food groups.

SETTING

Boston University Medical Center, Boston, Massachusetts.

SUBJECTS

One hundred and ninety-one faculty and staff at Boston University Medical Center aged 20-70 years.

RESULTS

There were significant positive correlations between the Block FFQ and the DASH OLQ for all food groups ranging from r = 0.8 for the nuts/seeds/legumes category to r = 0.3 for vegetables and mixed dishes. A comparison of nutrient intakes found strong positive correlations in all nutrient categories. Of particular interest in the DASH diet and the web-based nutrition and physical activity programme were total fat (r = 0.62), total carbohydrate (r = 0.67), total K (r = 0.68), total Ca (r = 0.69), total vitamin C (r = 0.60) and total energy intake (r = 0.68).

CONCLUSIONS

The DASH OLQ captures food and nutrient intake well in relation to the more established Block FFQ.

Recommendations for Performing Internet-Based Research on Sensitive Subject Matter with "Hidden" or Difficult-to-Reach Populations

Since the mid-1990s, the rapidly increasing popularity of the Internet has contributed to a situation in which many men turn to Web sites to find sex partners with whom they can engage in risky behaviors. Scholars only recently began to examine the role of the Internet in harm-seeking and help-seeking behaviors. They are just now beginning to study and understand how to apply public health promotion principles to people using the Internet. Due in part to the relative newness of the Internet on the public health landscape, scholars wishing to conduct research or to implement health promotion programs online should consider a variety of challenges to doing such work-challenges that differ from those typically faced when undertaking similar work in other types of venues offline. The purpose of this article is to address several of these research considerations. In particular, the present authors wish to provide researchers and health care specialists with key considerations when developing their own Internet-based research or health promotion programs. We also wish to furbish readers with some experience-based suggestions about how to avoid the potential pitfalls of conducting Internet-based studies. Moreover, our emphasis is on how to develop such programs when they are targeting hard-to-reach or "hidden" populations and/or when they deal with sensitive subject matter. Recommendations pertaining to the planning, recruitment, implementation, and evaluation stages of doing professional work online are provided.

A pharmacy preregistration course using online teaching and learning methods

OBJECTIVE

To design and evaluate a preregistration course utilizing asynchronous online learning as the primary distance education delivery method.

DESIGN

Online course components including tutorials, quizzes, and moderated small-group asynchronous case-based discussions were implemented. Online delivery was supplemented with self-directed and face-to-face learning.

ASSESSMENT

Pharmacy graduates who had completed the course in 2004 and 2005 were surveyed. The majority felt they had benefited from all components of the course, and that online delivery provided benefits including increased peer support, shared learning, and immediate feedback on performance. A majority of the first cohort reported that the workload associated with asynchronous online discussions was too great. The course was altered in 2005 to reduce the online component. Participant satisfaction improved, and most felt that the balance of online to face-to-face delivery was appropriate.

CONCLUSION

A new pharmacy preregistration course was successfully implemented. Online teaching and learning was well accepted and appeared to deliver benefits over traditional distance education methods once workload issues were addressed.

Availability of a remote online hemodynamic monitoring system during treatment in a private dental office for medically high-risk patients

The importance of systemic management to prevent accidents is increasing in dentistry because co-morbid illnesses in an aging society and invasive surgical procedures are increasing. In this prefecture, a new medical system called the remote online hemodynamic monitoring system (ROHMs) was started in 2001. Eight private dental offices participated in this trial. When dental practitioners feel the risk of a dental procedure, they can contact via ROHMs to this hospital. Then, the hemodynamic data (blood pressure, heart rate, ECG, SpO₂, and RPP) of the patient in the clinic can be transmitted here via the internet, and the images and the voice can be transmitted as well. The availability of this system was assessed in 66 patients (98 cases). The most frequent complications were hypertension, heart disease, and diabetes mellitus. Systemic management included monitoring during the dental procedure (71.4%), checking vital signs after an interview (15.3%), and monitoring under sedation (13.3%). There were 35.7% of all cases where an unscheduled procedure was necessary for the systemic management. Based on a questionnaire, the majority of the patients felt relieved and safe. This system creates a situation where a specialist is almost present during the procedure. This system will provide significant assistance for future medical cooperation for risk management.

Principles-based learning design for an online postgraduate psychiatric pharmacy course

OBJECTIVES

To implement an interactive, Web-based postgraduate course aimed at improving the knowledge, skills, and confidence of health care practitioners with interest in providing pharmaceutical care to people with mental disorders.

DESIGN

The Web-based course was created in which authentic tasks and multiple modes of presentation and problem-solving were used to develop understanding of core concepts. Participants formulated pharmaceutical care plans for authentic case scenarios and cases selected from practice. Participants developed management plans using audiovisual lectures, videotaped demonstration patient interviews, and links to mental health practice tools and the literature. A combination of online discussion, collaborative and individual preparation of care plans, and peer and instructor feedback were used.

ASSESSMENT

Responses on postcourse surveys showed that participants' comfort levels with providing pharmaceutical care for common mental disorders increased moderately. Three-quarters of respondents indicated that the course had deepened their understanding and enhanced their mental health practice.

SUMMARY

A Web-based postgraduate course in pharmaceutical care for patients with mental disorders was successfully implemented and resulted in a template which can be used in the development of similar postgraduate courses.

Dual format course design: neuroanatomy and neurophysiology for adult learners

Adult learners require alternative learning opportunities to enable them to work and still be able to upgrade their education. This dual format course, combining online and face-to-face components, was developed to meet the need of students to complete a prerequisite undergraduate level neuroanatomy and neurophysiology course while attending a program that included fly-in weekends (seven) and online instruction (15 weeks). A combination of online lectures and on-campus lectures were used to teach pre-requisite neuroanatomy and neurophysiology course topics. The article presents the teaching material division between online and on-campus lectures and presentation of data collected. Survey data collected included student preferred: online media presentation, online activities developed to facilitate learning of the online material, online case study discussion, and testing formats. Students also were asked to comment on whether the class should move to a 100% online format and what their concerns would be moving to an all online format. Additional qualitative data on student input related to the course, adult learning and the learning environment will be presented. Blackboard data include: student daily access patterns, media and documents access and download patterns, and case study participation. Additionally, descriptive statistics from in class quizzes versus online quizzes includes: student patterns of test taking in an unlimited retake environment, scores on retakes and final scores (highest of retakes), in-class quiz scores, and comparison of comprehensive final exam scores from online versus face-to-face lecture material and testing. Findings provide valuable information for online course formatting, revisions and additional course development.

An online debate series for first-year pharmacy students

OBJECTIVES

This article describes an online debate series that was developed as a new component to an introductory core course for first-professional year pharmacy students. Objectives were to facilitate the group process, introduce controversial issues related to the US healthcare system, improve critical thinking and communication skills, enable students' ability to analyze and evaluate evidence, help develop skills in formulating written arguments, and encourage tolerance of diverse points of view.

DESIGN

One hundred sixty-two students were assigned to 40 teams (half assigned to argue as "Pros" and half as "Cons") and paired into 20 debating groups. The paired teams posted 3 arguments in an online forum alternatively over a 12-week period. The winning teams were determined by a panel of 3 judges.

ASSESSMENT

Feedback from the judges was posted online and summarized in an in-class discussion. Thematic analysis of qualitative data from students and faculty members demonstrated the effectiveness of the online debate component in helping students work together in a group, learn alternative sides of complex issues, and write persuasive arguments.

CONCLUSION

This novel online-debate forum was a feasible teaching and learning strategy, which helped pharmacy students improve their communication skills and critical thinking, expanded their scope of knowledge, and provided a platform for group process.

Similarity-based interference during language comprehension: Evidence from eye tracking during reading

The nature of working memory operation during complex sentence comprehension was studied by means of eye-tracking methodology. Readers had difficulty when the syntax of a sentence required them to hold 2 similar noun phrases (NPs) in working memory before syntactically and semantically integrating either of the NPs with a verb. In sentence structures that placed these NPs at the same linear distances from one another but allowed integration with a verb for 1 of the NPs, the comprehension difficulty was not seen. These results are interpreted as indicating that similarity-based interference occurs online during the comprehension of complex sentences and that the degree of memory accessibility conventionally associated with different types of NPs does not have a strong effect on sentence processing.

An online lab manual for neurophysiology

A combination of student interest, a grant for computers, and a suggestion from a colleague led me to convert the printed lab manual for my neurophysiology course into an online version. The web-based version incorporates videos of procedures, color photos, diagrams, links to articles, and other supplementary materials. This paper discusses designing for the web, retaining compatibility with print, layering information, making videos in digital format, and sharing self-published resources. The labs themselves (appendix) are traditional "wet" experiments with crayfish that have been used in many neurobiology courses.