Impact of a Regional Grant Program Through the Lens of Social Cognitive Career Theory: A Mixed-Method Evaluation

PURPOSE

Evaluations of educational grant programs have focused on research productivity, with few examining impacts on grantees or effective program characteristics. This evaluation examined the regional grant program sponsored by Group on Educational Affairs to examine if and how grantees' careers were affected by funding, and if these experiences aligned with program goals.

METHOD

In this concurrent, mixed-methods theory-driven evaluation, quantitative and qualitative data were analyzed independently and then integrated to examine complementarity. Quantitative data examined differences among 4 geographic regions and included proposal and grantee characteristics abstracted from administrative records of 52 funded proposals from 2010-2015 grant cycles. Qualitative data from 23 interviews conducted from 2018 to 2019 explored the impact on grantees, with Social Cognitive Career Theory (SCCT) serving as a framework for deductive thematic analysis. To facilitate integration of findings, quantitative data were layered onto each interview to permit exploration of associations between the 2 data types.

RESULTS

Although significant regional differences existed in project length and amount of funding, there were few regional differences in grantee experiences. Despite small funding amounts, grants were perceived as career launching pads. The SCCT framework accounted for grantee experiences, including researcher identity formation and subsequent research, but did not capture collaboration phenomena. Integration of the 2 data types identified experience patterns unique to different groups of grantees (e.g., more or less research experience). The diversity among grantees suggests that clarification of program goals and stronger alignment with criteria for funding may be warranted.

CONCLUSIONS

This evaluation illuminates why small educational grant programs may or may not impact interest and productivity in research. Implications exist for funders, including clarifying program goals and providing support for less experienced grantees. Future research should explore grantee subsets (e.g., underrepresented in medicine) to further identify what fosters or inhibits careers of medical education scholars.

Research Methodologies in Health Professions Education Publications: Breadth and Rigor

PURPOSE

Research methodologies represent assumptions about knowledge and ways of knowing. Diverse research methodologies and methodological standards for rigor are essential in shaping the collective set of knowledge in health professions education (HPE). Given this relationship between methodologies and knowledge, it is important to understand the breadth of research methodologies and their rigor in HPE research publications. However, there are limited studies examining these questions. This study synthesized current trends in methodologies and rigor in HPE papers to inform how evidence is gathered and collectively shapes knowledge in HPE.

METHOD

This descriptive quantitative study used stepwise stratified cluster random sampling to analyze 90 papers from 15 HPE journals published in 2018 and 2019. Using a research design codebook, the authors conducted group coding processes for fidelity, response process validity, and rater agreement; an index quantifying methodological rigor was developed and applied for each paper.

RESULTS

Over half of research methodologies were quantitative (51%), followed by qualitative (28%), and mixed methods (20%). No quantitative and mixed methods papers reported an epistemological approach. All qualitative papers that reported an epistemological approach (48%) used social constructivism. Most papers included participants from North America (49%) and Europe (20%). The majority of papers did not specify participant sampling strategies (56%) or a rationale for sample size (80%). Among those reported, most studies (81%) collected data within 1 year.The average rigor score of the papers was 56% (SD = 17). Rigor scores varied by journal categories and research methodologies. Rigor scores differed between general HPE journals and discipline-specific journals. Qualitative papers had significantly higher rigor scores than quantitative and mixed methods papers.

CONCLUSIONS

This review of methodological breadth and rigor in HPE papers raises awareness in addressing methodological gaps and calls for future research on how the authors shape the nature of knowledge in HPE.

What Is an Innovation Article? A Systematic Overview of Innovation in Health Professions Education Journals

PURPOSE

Innovation articles have their own submission category and guidelines in health professions education (HPE) journals, which suggests innovation might be a unique genre of scholarship. Yet, the requirements for innovation submissions vary among journals, suggesting ambiguity about the core content of this type of scholarship. To reduce this ambiguity, the researchers conducted a systematic overview to identify key features of innovation articles and evaluate their consistency in use across journals. Findings from this review may have implications for further development of innovation scholarship within HPE.

METHOD

In this systematic overview, conducted in 2020, the researchers identified 13 HPE journals with innovation-type articles and used content analysis to identify key features from author guidelines and publications describing what editors look for in innovation articles. The researchers then audited a sample of 39 innovation articles (3/journal) published in 2019 to determine presence and consistency of 12 innovation features within and across HPE journals. Audit findings informed the researchers' evaluation of innovation as a genre in HPE.

RESULTS

Findings show variability of innovation feature presence within and across journals. On average, articles included 7.8 of the 12 innovation features (SD 2.1, range 3-11). The most common features were description of: how the innovation was implemented (92%), a problem (90%), what was new or novel (79%), and data or outcomes (77%). On average, 5.5 (SD 1.5) out of 12 innovation features were consistently used in articles within each journal.

CONCLUSIONS

The authors identified common features of innovation article types based on journal guidelines, but there was variability in presence and consistency of these features, suggesting HPE innovations are in an emerging state of genre development. The authors discuss potential reasons for variability within this article type and highlight the need for further discussion among authors, editors, and reviewers to improve clarity.

Challenges to assessing professional identity in medical students: a tale of two measures

Background: Professional identity formation (PIF), a foundational process in becoming a physician, includes establishment of values, moral principles, and self-awareness. The purpose of this report is to examine challenges in establishing the validity of measures of identity fusion as one facet of PIF. Method: Utilizing the modern approach of validity as a unitary concept, the authors generated six hypotheses to examine the evidence for the construct validity of the scores of Physician Professional Identity (PPI) and Identity Integration (IdIn), considering relationships of these measures with each other, year of training and data from a larger survey. Results: Responses from 3473 students at 8 medical schools revealed a weak association between the measures with distributions varying by cohort. PPI had a stronger relationship to cohort and IdIn was moderately associated with students' attitudes relevant to social media use. Responses were independent of response format and evidence supported the interpretation of scores for IdIn as indications of integration of identity. Discussion : Sufficient evidence was found to suggest that these measures assess aspects of PIF. Use of these measures as part of a multidimensional, longitudinal approach to refining understanding of the construct of PIF and developing effective assessment strategies.

A Review of Continuous Quality Improvement Processes at Ten Medical Schools

The Liaison Committee on Medical Education now expects all allopathic medical schools to develop and adhere to a documentable continuous quality improvement (CQI) process. Medical schools must consider how to establish a defensible process that monitors compliance with accreditation standards between site visits. The purpose of this descriptive study is to detail how ten schools in the Association of American Medical Colleges' (AAMC) Southern Group on Educational Affairs (SGEA) CQI Special Interest Group (SIG) are tackling practical issues of CQI development including establishing a CQI office, designating faculty and staff, charging a CQI committee, choosing software for data management, if schools are choosing formalized CQI models, and other considerations. The information presented is not meant to certify that any way is the correct way to manage CQI, but simply present some schools' models. Future research should include defining commonalities of CQI models as well as seeking differences. Furthermore, what are components of CQI models that may affect accreditation compliance negatively? Are there "worst practices" to avoid? What LCME elements are most commonly identified for CQI, and what are the successes and struggles for addressing those elements? What are identifiable challenges relating to use of standard spreadsheet software and engaging information technology for support? How can students be more engaged and involved in the CQI process? Finally, how do these major shifts to a formalized CQI process impact the educational experience?

An Interprofessional Standardized Patient Case for Improving Collaboration, Shared Accountability, and Respect in Team-Based Family Discussions

Introduction

The science of patient safety demonstrates that good communication is essential for effective interprofessional collaboration.

Methods

We created a low-stakes, formative assessment with which medical students, pharmacy students, and nursing students could practice several of the Interprofessional Education Collaborative competencies. We aimed to enable students to practice collaborative care, respect for other disciplines, and shared accountability. Senior students from medicine, nursing, and pharmacy worked in teams to disclose a medical error to a standardized patient. The activity began with an icebreaker exercise wherein students learned about each other. Next, each team planned a strategy for error disclosure and collaboratively disclosed the error. Standardized patients evaluated the team's performance. Subsequently, students regrouped for a debriefing. The participating institutions administered a survey to their students.

Results

In total, 1,151 students participated: 464 fourth-year students from the University of Houston College of Pharmacy, 450 third- and fourth-year students from Baylor College of Medicine, and 237 fourth-year students from Texas Woman's University Nelda C. Stark College of Nursing, all in Houston, Texas. Postsession survey data showed that students thought they achieved the relevant competencies. Students' understanding of the perspectives of the other two disciplines improved. Students found the simulation encounter and debriefing effective in helping them consider the contributions of other disciplines to patient care.

Discussion

This interprofessional standardized patient activity enabled collaborative problem solving. The debriefing discussion broadened students' understanding of the expertise of the other disciplines and promoted shared accountability. Students found this activity engaging and effective.

Patient Safety Interprofessional Training for Medical, Nursing, and Pharmacy Students

INTRODUCTION

Patient safety education is required in medical, nursing, and pharmacy training, and interprofessional education offers an ideal format for teaching the core concepts of patient safety. This training activity was developed to fulfill interprofessional education core competencies for communication and teamwork and was nested within a required patient safety course taught at a medical school. However, the activity can easily be adapted as a stand-alone offering that can be included in a preclinical doctoring course, offered as an elective, or hosted at a college of nursing or pharmacy. Our goal was to prepare learners for the clinical environment by providing a context for patient safety, communication, and teamwork.

METHODS

Students participate in a 1.5-hour large-group activity that explores a case from the perspectives of each discipline. Faculty from all three disciplines sequentially present and debrief the case using focused questions to guide students' reflections and interactions between team members.

RESULTS

We have presented this activity for 4 consecutive years. Students complete a questionnaire with retrospective pre-post ratings of their perspectives on the activity and its impact on their awareness of disciplinary roles and responsibilities, communication errors, and strategies for addressing interdisciplinary conflicts. Results show statistically significant increases in the items of interest.

DISCUSSION

This interprofessional education offering is effective in terms of increasing awareness and knowledge among members of three health care disciplines, improving awareness of potential kinds of communication errors, and helping students consider the role of interdisciplinary interactions.

Near-peers improve patient safety training in the preclinical curriculum

BACKGROUND

Accrediting bodies require medical schools to teach patient safety and residents to develop teaching skills in patient safety. We created a patient safety course in the preclinical curriculum and used continuous quality improvement to make changes over time.

OBJECTIVE

To assess the impact of resident teaching on student perceptions of a Patient Safety course.

DESIGN

Using the Institute for Healthcare Improvement patient safety curriculum as a frame, the course included the seven IHI modules, large group lectures and small group facilitated discussions. Applying a social action methodology, we evaluated the course for four years (Y1-Y4).

RESULTS

In Y1, Y2, Y3 and Y4, we distributed a course evaluation to each student (n = 184, 189, 191, and 184, respectively) and the response rate was 96, 97, 95 and 100%, respectively. Overall course quality, clarity of course goals and value of small group discussions increased in Y2 after the introduction of residents as small group facilitators. The value of residents and the overall value of the course increased in Y3 after we provided residents with small group facilitation training.

CONCLUSIONS

Preclinical students value the interaction with residents and may perceive the overall value of a course to be improved based on near-peer involvement. Residents gain valuable experience in small group facilitation and leadership.

Leaders' Ambivalence about Reciprocity Obligations in Self-Help Groups

This study explored the extent to which self-help group leaders endorsed beliefs that group members should primarily take care of themselves (a need-based norm) and that group members have an obligation to help others (an equity-based nonn). Leadersfrom 63 different groups were asked to respond to a fictional vignette describing "Betty," who had been attending a cancer survivors group for 4 months. Leaders 'endorsements of both norms were interpreted as evidence that leaders experienced pressures to respond to both individual members 'needs and the group 's survival needs. Group-level variables, rather than dispositional variables, were most strongly associated with their endorsements of needand equity-based norms. Leaders were less likely to endorse the need-based norm and more likely to endorse the equity-based norm when (a) there was more diversified leadership in their own group, (b) their group was affiliated with other groups rather than stand-alone groups, and (c) the leader was a successor rather than a founder.

Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review

PURPOSE

Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies.

METHOD

The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality.

RESULTS

Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills.

CONCLUSIONS

These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.

Meaningful Learning Moments on a Family Medicine Clerkship: When Students Are Patient Centered

BACKGROUND AND OBJECTIVES

Reflection after patient encounters is an important aspect of clinical learning. After our medical school instituted a reflection paper assignment for all clerkships, we wanted to learn about the types of encounters that students found meaningful on a family medicine clerkship and how they impacted students' learning.

METHODS

Family and Community Medicine Clerkship students completed a reflection paper after the clerkship, based on guidelines that were used for all clerkship reflection papers at our medical school. Two reviewers independently organized student responses into themes and then jointly prioritized common themes and negotiated any initial differences into other themes.

RESULTS

A total of 272 reflection papers describing an actual learning moment in patient care were submitted during the study period of January 2011--December 2012. In describing actions performed, students most frequently wrote about aspects of patient-centered care such as listening to the patient, carefully assessing the patient's condition, or giving a detailed explanation to the patient. In describing effects of those actions, students wrote about what they learned about the patient-physician interaction, the trust that patients demonstrated in them, the approval they gained from their preceptors, and the benefits they saw from their actions.

CONCLUSIONS

An important contribution of a family medicine clerkship is the opportunity for students to further their skills in patient-centered care and realize the outcomes of providing that type of care.

A guiding framework to maximise the power of the arts in medical education: a systematic review and metasynthesis

CONTEXT

A rich literature describes many innovative uses of the arts in professional education. However, arts-based teaching tends to be idiosyncratic, depending on the interests and enthusiasm of individual teachers, rather than on strategic design decisions. An overarching framework is needed to guide implementation of arts-based teaching in medical education. The objective of this study was to review and synthesise the literature on arts-based education and provide a conceptual model to guide design, evaluation and research of the use of the arts in medical education.

METHODS

A systematic literature review using the PubMed and ERIC databases. Search terms included humanism, art, music, literature, teaching, education, learning processes, pedagogy and curriculum. We selected empirical studies and conceptual articles about the use of creative arts, imagery and symbolism in the context of professional education. Data synthesis involved a qualitative content analysis of 49 included articles, identifying themes related to educational characteristics, processes and outcomes in arts-based education.

RESULTS

Four common themes were identified describing (i) unique qualities of the arts that promote learning, (ii) particular ways learners engage with art, (iii) documented short- and long-term learning outcomes arising from arts-based teaching and (iv) specific pedagogical considerations for using the arts to teach in professional education contexts.

CONCLUSIONS

The arts have unique qualities that can help create novel ways to engage learners. These novel ways of engagement can foster learners' ability to discover and create new meanings about a variety of topics, which in turn can lead to better medical practice. At each of these steps, specific actions by the teacher can enhance the potential for learners to move to the next step. The process can be enhanced when learners participate in the context of a group, and the group itself can undergo transformative change. Future work should focus on using this model to guide process design and outcome measurement in arts-based education.

The utility of reflective writing after a palliative care experience: can we assess medical students' professionalism?

BACKGROUND

Medical education leaders have called for a curriculum that proactively teaches knowledge, skills, and attitudes required for professional practice and have identified professionalism as a competency domain for medical students. Exposure to palliative care (PC), an often deeply moving clinical experience, is an optimal trigger for rich student reflection, and students' reflective writings can be explored for professional attitudes.

OBJECTIVE

Our aim was to evaluate the merit of using student reflective writing about a PC clinical experience to teach and assess professionalism.

METHODS

After a PC patient visit, students wrote a brief reflective essay. We explored qualitatively if/how evidence of students' professionalism was reflected in their writing. Five essays were randomly chosen to develop a preliminary thematic structure, which then guided analysis of 30 additional, randomly chosen essays. Analysts coded transcripts independently, then collaboratively, developed thematic categories, and selected illustrative quotes for each theme and subtheme.

RESULTS

Essays revealed content reflecting more rich information about students' progress toward achieving two professionalism competencies (demonstrating awareness of one's own perspectives and biases; demonstrating caring, compassion, empathy, and respect) than two others (displaying self-awareness of performance; recognizing and taking actions to correct deficiencies in one's own behavior, knowledge, and skill).

CONCLUSIONS

Professional attitudes were evident in all essays. The essays had limited use for formal summative assessment of professionalism competencies. However, given the increasing presence of PC clinical experiences at medical schools nationwide, we believe this assessment strategy for professionalism has merit and deserves further investigation.

Fostering students' reflection about bias in healthcare: cognitive dissonance and the role of personal and normative standards

BACKGROUND

To reduce cognitive dissonance about one's beliefs or behavior, individuals may compare their behavior to personal and/or normative standards. The details of this reflection process are unclear.

AIMS

We examined how medical students compare their behavior or beliefs to standards in discussions about implicit bias, and explored if and how different reflective pathways (preserving vs. reconciling) are associated with each standard.

METHODS

Third-year students engaged in a small-group discussion about bias. Some students and group facilitators also participated in a debriefing about the experience. Using qualitative methods, the transcripts from these 11 sessions were analyzed for evidence of student comparison to a standard and of reflection pathways.

RESULTS

Of 557 text units, 75.8% could be coded with a standard and/or a path of reflection. Students referenced personal and normative standards about equally, and preserved or reconciled existing beliefs about equally. Uses of normative standards were associated with preservation-type reflection, and uses of personal standards with reconciliation-type reflection.

CONCLUSIONS

Normative expectations of physicians are sometimes used to provoke students' consideration of implicit biases about patients. To encourage critical reflection and reconciliation of biased beliefs or behavior, educators should frame reflective activities as a personal exercise rather than as a requirement.

Measuring the quality of patients' goals and action plans: development and validation of a novel tool

BACKGROUND

The purpose of this study is to develop and test reliability, validity, and utility of the Goal-Setting Evaluation Tool for Diabetes (GET-D). The effectiveness of diabetes self-management is predicated on goal-setting and action planning strategies. Evaluation of self-management interventions is hampered by the absence of tools to assess quality of goals and action plans. To address this gap, we developed the GET-D, a criteria-based, observer rating scale that measures the quality of patients' diabetes goals and action plans.

METHODS

We conducted 3-stage development of GET-D, including identification of criteria for observer ratings of goals and action plans, rater training and pilot testing; and then performed psychometric testing of the GET-D.

RESULTS

Trained raters could effectively rate the quality of patient-generated goals and action plans using the GET-D. Ratings performed by trained evaluators demonstrated good raw agreement (94.4%) and inter-rater reliability (Kappa = 0.66). Scores on the GET-D correlated well with measures theoretically associated with goal-setting, including patient activation (r=.252, P<.05), diabetes specific self-efficacy (r=.376, P<.001) and inverse relationship with depression (r= -.376, P<.01). Significant between group differences (P<.01) in GET-D scores between goal-setting intervention (mean = 7.33, standard deviation = 4.4) and education groups (mean = 4.93, standard deviation = 3.9) confirmed construct validity of the GET-D.

CONCLUSIONS

The GET-D can reliably and validly rate the quality of goals and action plans. It holds promise as a measure of intervention fidelity for clinical interventions that promote diabetes self-management behaviors to improve clinical outcomes.

TRIAL REGISTRATION

Clinicaltrials.gov Identifier: NCT00481286.

A standards-based, peer-reviewed teaching award to enhance a medical school's teaching environment and inform the promotions process

The authors provide the rationale, design, and description of a unique teaching award that has enhanced Baylor College of Medicine's teaching environment and become highly valued by the promotions and tenure (P&T) committee in determining a faculty member's readiness for promotion. This award is self-nominating and standards based. The primary purpose for development of the award was to provide the Baylor community and the P&T committee a method to understand and value the scholarship of teaching to the same degree that they understand and value the scholarship of discovery.The authors also present results from an internal evaluation of the program that included a survey and interviews. Between the inception of the award in 2001 and the internal review conducted in 2010, the award could have had an influence on the promotion of 130 of the recipients. Of the 130, 88 (65.6%) received this award before gaining their current rank (χ (1) = 16.3, P < .001). Stakeholders, including department chairs and members of the P&T committee, agreed that this award is valuable to those seeking promotion. Individual recipients stated that the award is good for the institution by encouraging reflection on teaching; increasing the recognition, importance, and value of teaching; encouraging the improvement of teaching skills; and providing a better understanding to others about what medical teachers really do. Of the 214 open-ended responses to survey questions of award recipients, more than half the comments were about the value of the award and its positive effect on promotion.

Improving quality of care through improved audit and feedback

Background

The Department of Veterans Affairs (VA) has led the industry in measuring facility performance as a critical element in improving quality of care, investing substantial resources to develop and maintain valid and cost-effective measures. The External Peer Review Program (EPRP) of the VA is the official data source for monitoring facility performance, used to prioritize the quality areas needing most attention. Facility performance measurement has significantly improved preventive and chronic care, as well as overall quality; however, much variability still exists in levels of performance across measures and facilities. Audit and feedback (A&F), an important component of effective performance measurement, can help reduce this variability and improve overall performance. Previous research suggests that VA Medical Centers (VAMCs) with high EPRP performance scores tend to use EPRP data as a feedback source. However, the manner in which EPRP data are used as a feedback source by individual providers as well as service line, facility, and network leadership is not well understood. An in-depth understanding of mental models, strategies, and specific feedback process characteristics adopted by high-performing facilities is thus urgently needed.

This research compares how leaders of high, low, and moderately performing VAMCs use clinical performance data from the EPRP as a feedback tool to maintain and improve quality of care.

Methods

We will conduct a qualitative, grounded theory analysis of up to 64 interviews using a novel method of sampling primary care, facility, and Veterans Integrated Service Network (VISN) leadership at high-, moderate-, and low-performing facilities. We will analyze interviews for evidence of cross-facility differences in perceptions of performance data usefulness and strategies for disseminating performance data evaluating performance, with particular attention to timeliness, individualization, and punitiveness of feedback delivery.

Discussion

Most research examining feedback to improve provider and facility performance lacks a detailed understanding of the elements of effective feedback. This research will highlight the elements most commonly used at high-performing facilities and identify additional features of their successful feedback strategies not previously identified. Armed with this information, practices can implement more effective A&F interventions to improve quality of care.

Helping medical learners recognise and manage unconscious bias toward certain patient groups

CONTEXT

For the last 30 years, developments in cognitive sciences have demonstrated that human behaviour, beliefs and attitudes are shaped by automatic and unconscious cognitive processes. Only recently has much attention been paid to how unconscious biases based on certain patient characteristics may: (i) result in behaviour that is preferential toward or against specific patients; (ii) influence treatment decisions, and (iii) adversely influence the patient-doctor relationship. Partly in response to accreditation requirements, medical educators are now exploring how they might help students and residents to develop awareness of their own potential biases and strategies to mitigate them.

METHODS

In this paper, we briefly review key cognition concepts and describe the limited published literature about educational strategies for addressing unconscious bias.

DISCUSSION

We propose a developmental model to illustrate how individuals might move from absolute denial of unconscious bias to the integration of strategies to mitigate its influence on their interactions with patients and offer recommendations to educators and education researchers.

Knowing the ABCs: a comparative effectiveness study of two methods of diabetes education

OBJECTIVE

To test an active-learning, empowerment approach to teaching patients about the "diabetes ABCs" (hemoglobin A(1)C, systolic blood pressure, and low density lipoprotein cholesterol).

METHODS

84 (97%) diabetic patients who participated in a randomized effectiveness trial of two clinic-based group educational methods and completed a post-intervention assessment. The empowerment arm participated in a group session that incorporated two educational innovations (a conceptual metaphor to foster understanding, and team-based learning methods to foster active learning). The traditional diabetes education arm received a didactic group session focused on self-management and educational materials about the diabetes ABCs. Participants in both arms received individual review of their current ABC values.

RESULTS

A questionnaire evaluated knowledge, understanding, and recall of the diabetes ABCs was administered three months after enrollment in the study. At three months, participants in the empowerment group demonstrated greater understanding of the diabetes ABCs (P<0.0001), greater knowledge of their own values (P<0.0001), and greater knowledge of guideline-derived target goals for the ABCs compared with participants in the traditional arm (P<0.0001).

CONCLUSION

An active-learning, empowerment-based approach applied to diabetes education can lead to greater understanding and knowledge retention.

PRACTICE IMPLICATIONS

An empowerment approach to education can facilitate informed, activated patients and increase performance of self-management behaviors.

Following the clues: teaching medical students to explore patients' contexts

OBJECTIVE

Physicians often overlook important contextual clues that patients give during an encounter. The objective of our study was to increase medical students' knowledge and skills in identifying contextual issues.

METHODS

Six consecutive learning experiences, including a standardized patient (SP) encounter and activities designed to trigger reflection, were implemented within a first-year Introduction to Clinical Medicine course. Evaluation of the intervention was measured through self-confidence, attitudes, SP history checklist, and student and small group facilitator evaluations.

RESULTS

Standardized patient encounters, coupled with activities designed to trigger reflection, can help students identify patients' contextual clues. Students' confidence in eliciting patient clues significantly increased after the intervention. Our results suggest that some contextual clues were more difficult for students to elicit.

CONCLUSION

Multi-faceted approaches that include activities to trigger reflection are effective in teaching students to recognize and respond to contextual clues, however, more research is needed.

PRACTICE IMPLICATIONS

While students elicited most clues in this study, they struggled with identifying some clues. These results suggest the need for additional research and educational development in this area.

Ideals, activities, dissonance, and processing: a conceptual model to guide educators' efforts to stimulate student reflection

PURPOSE

Medical schools are increasingly incorporating opportunities for reflection into their curricula. However, little is known about the cognitive and/or emotional processes that occur when learners participate in activities designed to promote reflection. The purpose of this study was to identify and elucidate those processes.

METHOD

In 2008, the authors analyzed qualitative data from focus groups that were originally conducted to evaluate an educational activity designed to promote reflection. These data afforded the opportunity to explore the processes of reflection in detail. Transcripts (94 pages, single-spaced) from four focus groups were analyzed using a narrative framework. The authors spent approximately 40 hours in group and 240 hours in individual coding activities.

RESULTS

The authors developed a conceptual model of five major elements in students' reflective processes: the educational activity, the presence or absence of cognitive or emotional dissonance, and two methods of processing dissonance (preservation or reconciliation). The model also incorporates the relationship between the student's internal ideal of what a doctor is or does and the student's perception of the teacher's ideal of what a doctor is or does. The model further identifies points at which educators may be able to influence the processes of reflection and the development of professional ideals.

CONCLUSIONS

Students' cognitive and emotional processes have important effects on the success of educational activities intended to stimulate reflection. Although additional research is needed, this model-which incorporates ideals, activities, dissonance, and processing-can guide educators as they plan and implement such activities.

Reconsidering the team concept: educational implications for patient-centered cancer care

Patient-centered cancer care has become a priority in the oncology field. Increasing efforts to train oncologists in communication skills have led to a growing literature on patient-centered cancer education. In addition, systems approaches have led to an increased emphasis on the concept of teams as an organizing framework for cancer care. In this essay, we examine issues involved in educating teams to provide patient-centered cancer care. In the process, we question the applicability of a tightly coordinated 'team' concept, and suggest the concept of a 'care community' as a more achievable ideal for the way that cancer care is commonly delivered. We discuss the implications that this has for cancer communication education, and propose three principles to guide the development of educational interventions aimed at increasing patient-centeredness in cancer care delivery systems.

Development and validation of Patient Reported Impact of Spasticity Measure (PRISM)

Persons with spinal cord injury (SCI) may experience a range of symptoms typically labeled "spasticity." Previous efforts to develop assessment tools that measure spasticity have failed to represent the experiences of persons who live with the condition. The purpose of this multicenter study was to develop an instrument that measures the impact of spasticity on quality of life. Based on 24 semistructured interviews, a developmental form of the Patient Reported Impact of Spasticity Measure (PRISM) was constructed. The developmental PRISM was administered to 180 persons at five sites. Subscales were developed based on factor analytic results. Evidence for the reliability and validity of the scores was evaluated. Seven subscales were developed, including one that measures the positive effects of spasticity. Results of reliability and validity assessments indicate that the PRISM subscale scores effectively measure the impact of spasticity in the population of veterans with SCI.

Does poorer familiarity with Medicare translate into worse access to health care?

OBJECTIVES

To examine the relationship between a global measure of Medicare program familiarity and a broad set of measures of actual and perceived healthcare access.

DESIGN

Mailed survey in fall of 2004 (2,997 completed surveys; 53% response rate).

SETTING

Metropolitan and nonmetropolitan areas across the United States.

PARTICIPANTS

White, black, and Hispanic Medicare beneficiaries.

MEASUREMENTS

Familiarity with Medicare and self-reported measures of health status, healthcare use, and perceived access to care.

RESULTS

Reported poorer familiarity with Medicare is associated with a greater likelihood of delayed care due to cost, multiple emergency department visits, lack of prescription medication use, poorer perceived access to care, poorer overall health, and a greater reported decline in health from the prior year. Black and Hispanic respondents were more likely to be unfamiliar than whites, although the relationship between familiarity and healthcare access persisted after adjusting for race or ethnicity, Medicare health plan enrollment status, supplemental insurance status, age, sex, income level, education, geographic area, and general healthcare use.

CONCLUSION

Poorer familiarity with Medicare may affect beneficiaries' ability to access needed care effectively, may lead them to delay or avoid seeking care, and ultimately may have negatively affect the quality of the health care that they receive and their outcomes.

Critical elements of culturally competent communication in the medical encounter: a review and model

Increasing the cultural competence of physicians is one means of responding to demographic changes in the USA, as well as reducing health disparities. However, in spite of the development and implementation of cultural competence training programs, little is known about the ways cultural competence manifests itself in medical encounters. This paper will present a model of culturally competent communication that offers a framework of studying cultural competence 'in action.' First, we describe four critical elements of culturally competent communication in the medical encounter--communication repertoire, situational awareness, adaptability, and knowledge about core cultural issues. We present a model of culturally competent physician communication that integrates existing frameworks for cultural competence in patient care with models of effective patient-centered communication. The culturally competent communication model includes five communication skills that are depicted as elements of a set in which acquisition of more skills corresponds to increasing complexity and culturally competent communication. The culturally competent communication model utilizes each of the four critical elements to fully develop each skill and apply increasingly sophisticated, contextually appropriate communication behaviors to engage with culturally different patients in complex interactions. It is designed to foster maximum physician sensitivity to cultural variation in patients as the foundation of physician-communication competence in interacting with patients.

Conceptual challenges and practical approaches to screening capacity for self-care and protection in vulnerable older adults

Identifying impairments in the capacity to make and execute decisions is critical to the assessment and remediation of elder self-neglect. Few capacity assessment tools are available for use outside of healthcare settings, and none have been validated in the context of elder self-neglect. Health and social services professionals are in need of validated tools to assess capacity for self-care and self-protection (SC&P) during initial evaluations of older adults with suspected self-neglect syndrome. Currently, legal and medical declarations of incapacity and guardianship rely on clinical evaluations and instruments developed to assess only decision-making capacity. This article first describes the conceptual and methodological challenges to assessing the capacity to make and execute decisions regarding safe and independent living. Second, the article describes the pragmatic obstacles to developing a screening tool for the capacity for SC&P. Finally, the article outlines the process for validation and field testing of the screening tool. Social services professionals can then use a valid and feasible screening tool during field assessments to screen for potential impairments in the capacity for SC&P in vulnerable older adults.

IRT health outcomes data analysis project: an overview and summary

BACKGROUND

In June 2004, the National Cancer Institute and the Drug Information Association co-sponsored the conference, "Improving the Measurement of Health Outcomes through the Applications of Item Response Theory (IRT) Modeling: Exploration of Item Banks and Computer-Adaptive Assessment." A component of the conference was presentation of a psychometric and content analysis of a secondary dataset.

OBJECTIVES

A thorough psychometric and content analysis was conducted of two primary domains within a cancer health-related quality of life (HRQOL) dataset.

RESEARCH DESIGN

HRQOL scales were evaluated using factor analysis for categorical data, IRT modeling, and differential item functioning analyses. In addition, computerized adaptive administration of HRQOL item banks was simulated, and various IRT models were applied and compared.

SUBJECTS

The original data were collected as part of the NCI-funded Quality of Life Evaluation in Oncology (Q-Score) Project. A total of 1,714 patients with cancer or HIV/AIDS were recruited from 5 clinical sites.

MEASURES

Items from 4 HRQOL instruments were evaluated: Cancer Rehabilitation Evaluation System-Short Form, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire, Functional Assessment of Cancer Therapy and Medical Outcomes Study Short-Form Health Survey.

RESULTS AND CONCLUSIONS

Four lessons learned from the project are discussed: the importance of good developmental item banks, the ambiguity of model fit results, the limits of our knowledge regarding the practical implications of model misfit, and the importance in the measurement of HRQOL of construct definition. With respect to these lessons, areas for future research are suggested. The feasibility of developing item banks for broad definitions of health is discussed.

Evidence-based practice for equating health status items: sample size and IRT model

BACKGROUND

In the development of health outcome measures, the pool of candidate items may be divided into multiple forms, thus "spreading" response burden over two or more study samples. Item responses collected using this approach result in two or more forms whose scores are not equivalent. Therefore, the item responses must be equated (adjusted) to a common mathematical metric.

OBJECTIVES

The purpose of this study was to examine the effect of sample size, test size, and selection of item response theory model in equating three forms of a health status measure. Each of the forms was comprised of a set of items unique to it and a set of anchor items common across forms.

RESEARCH DESIGN

The study was a secondary data analysis of patients' responses to the developmental item pool for the Health of Seniors Survey. A completely crossed design was used with 25 replications per study cell.

RESULTS

We found that the quality of equatings was affected greatly by sample size. Its effect was far more substantial than choice of IRT model. Little or no advantage was observed for equatings based on 60 or 72 items versus those based on 48 items.

CONCLUSIONS

We concluded that samples of less than 300 are clearly unacceptable for equating multiple forms. Additional sample size guidelines are offered based on our results.

Geriatric depression, medical diagnosis, and functional recovery during acute rehabilitation

OBJECTIVE

To examine and compare the prevalence and functional impact of depressive symptoms for older adult stroke and nonstroke rehabilitation inpatients.

DESIGN

Case-control study examining functional outcome using a 2 (stroke, nonstroke) by 2 (depression, no depression) design.

SETTING

Urban hospital rehabilitation unit.

PARTICIPANTS

A total of 509 rehabilitation inpatients (age, > or = 60 y) were included and grouped by diagnosis of stroke (n=207) and nonstroke (n=302).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Geriatric Depression Scale and FIM instrument. Analysis of covariance procedures examined the impact of depressive symptoms on discharge functional ability controlling for age, sex, admission functional ability, and hospital length of stay.

RESULTS

Prevalence of depressive symptoms was similar for stroke (31.8%) and nonstroke (31.5%) and negatively associated with functional ability at discharge for both groups. Overall, the stroke and nonstroke groups did not differ significantly with respect to functional recovery.

CONCLUSIONS

Depression, and its impact on acute rehabilitation, is significantly related to functional recovery but does not differ in its frequency or impact for stroke patients. Because depressive symptoms do not appear to discriminate across diagnostic groups, routine screening for depression is recommended for all rehabilitation inpatients.

Medicare beneficiary knowledge: measurement implications from a qualitative study

Medicare beneficiary knowledge about fee-for-service (FFS) Medicare versus managed care alternatives (MCA) has been studied extensively. However, these efforts might be compromised by lack of familiarity with common Medicare terminology. We used qualitative methods to examine beneficiaries' familiarity with Medicare Programs (FFS and MCA) and terminology. Twenty-one indepth, semi-structured beneficiary interview transcripts were analyzed through iterative review. Across sex, race/ethnicity, and benefits programs, participants found interview questions with Medicare terminology difficult to answer, potentially causing missing, incorrect, and inaccurate responses to interview questions. Assessment of beneficiary knowledge may be fundamentally impacted by absence of basic familiarity with Medicare Programs terminology.

Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function

BACKGROUND AND OBJECTIVE

To test unidimensionality and local independence of a set of shoulder functional status (SFS) items, develop a computerized adaptive test (CAT) of the items using a rating scale item response theory model (RSM), and compare discriminant validity of measures generated using all items (theta(IRT)) and measures generated using the simulated CAT (theta(CAT)).

STUDY DESIGN AND SETTING

We performed a secondary analysis of data collected prospectively during rehabilitation of 400 patients with shoulder impairments who completed 60 SFS items.

RESULTS

Factor analytic techniques supported that the 42 SFS items formed a unidimensional scale and were locally independent. Except for five items, which were deleted, the RSM fit the data well. The remaining 37 SFS items were used to generate the CAT. On average, 6 items were needed to estimate precise measures of function using the SFS CAT, compared with all 37 SFS items. The theta(IRT) and theta(CAT) measures were highly correlated (r = .96) and resulted in similar classifications of patients.

CONCLUSION

The simulated SFS CAT was efficient and produced precise, clinically relevant measures of functional status with good discriminating ability.

Measurement in Veterans Affairs Health Services Research: veterans as a special population

OBJECTIVE

To introduce this supplemental issue on measurement within health services research by using the population of U.S. veterans as an illustrative example of population and system influences on measurement quality.

PRINCIPAL FINDINGS

Measurement quality may be affected by differences in demographic characteristics, illness burden, psychological health, cultural identity, or health care setting. The U.S. veteran population and the VA health system represent a microcosm in which a broad range of measurement issues can be assessed.

CONCLUSIONS

Measurement is the foundation on which health decisions are made. Poor measurement quality can affect both the quality of health care decisions and decisions about health care policy. The accompanying articles in this issue highlight a subset of measurement issues that have applicability to the broad community of health services research. It is our hope that they stimulate a broad discussion of the measurement challenges posed by conducting "state-of-the-art" health services research.

Ethnicity-related variation in breast cancer risk factors

BACKGROUND

A variety of factors are predictors of breast cancer risk. However, the studies conducted to establish these risk factors have rarely included African American women. The few studies with sufficient numbers of African-American women suggest that risk factors for breast cancer among African-American women are similar to those of white women. Although risk factors may be similar for African-American and white women, differences in the prevalence of risk factors may explain the differences in patterns of incidence.

METHODS

The authors reviewed the epidemiologic studies of breast cancer among African-American women and identified resources with information regarding the prevalence of risk factors among African American and white women.

RESULTS

Considerable variation exists in the studies of breast cancer risk factors among African American women. Because few studies have included sufficient numbers of African-American women, no firm conclusions can be drawn regarding whether risk estimates for African American women differ from those of white women. Estimates of the prevalence of breast cancer risk factors indicate that African American and white women differ in terms of their ages at menarche, menstrual cycle patterns, birth rates, lactation histories, patterns of oral contraceptive use, levels of obesity, frequency of menopausal hormone use, physical activity patterns, and alcohol intake.

CONCLUSIONS

The risk factor profile of African-American women appears to differ from that of white women. This may explain in part, the higher incidence rates for African Americans before age 45 years and the lower incidence rates at older ages. Discussions of these data at a workshop highlighted the need for future research on breast cancer risk among African Americans. This research should acknowledge the heterogeneous heritage, cultural beliefs, and cultural knowledge of African-American women. Studies conducted in collaboration with the African-American community of women and with the breast cancer advocacy community can benefit from assistance in the design of questionnaires and recruitment of participants.