Modeling the problem-based learning preferences of McMaster University undergraduate medical students using a discrete choice conjoint experiment

Modelling the case-based learning preferences of undergraduate nursing students using a discrete choice experiment in China

PURPOSE

To investigate the preferences for case-based learning programmes among undergraduate nursing students.

METHOD

A questionnaire was designed based on a discrete choice experiment, and 227 undergraduate nursing students were investigated. In STATA 15.0 software, the data were statistically analysed using a mixed logit model.

RESULT

All attributes in our study were found to have a significant influence on undergraduate nursing students' preferences for case-based learning programmes. The students' preference for the CBL programme was influenced by the clinical internship experience and type of university. Furthermore, the most ideal scenario was found to be video case modality, unfolding delivery, provided by academic experts and clinical instructors, group size 9-11, adequate feedback, and fragmented case content.

CONCLUSION

The undergraduate nursing students' preferences for case-based learning programmes were affected by the provider, case modality, modality, group size, feedback, and case content. Our results can provide useful information for nursing educators to gain insight into student preferences and formulate case-based learning programs.

Exploring doctors' trade-offs between management, research and clinical training in the medical curriculum: a protocol for a discrete choice experiment in Southern Africa

INTRODUCTION

Medical curricula should prepare doctors for roles that extend beyond that of a clinician. But the formal inclusion of both management and research training still appear to be neglected. It is important to understand what the profession would be willing to give up in terms of clinical training time for management and research content teaching prior to making any changes in a medical curriculum.

METHODS AND ANALYSIS

A discrete choice experiment will elicit the preferences and trade-offs that medical doctors in Southern Africa are prepared to make about the management, research and clinical training. Attention will also be given to the teaching method and placement of the content. DCE data will be collected using an online survey with an estimated sample size of 368 medical doctors. Data regarding participants' preference for a traditional or revised curriculum will be assessed using the Resistance to Change-Beliefs (RC-B) scale and demographic information will also be collected to assess preference heterogeneity.Analysis of the DCE data will be based on the Random Utility Maximisation framework using variants of the multinomial logit model. Data quality will be assessed. Value will be estimated in terms of clinical time, that is, how much clinical training time medical doctors are willing to give up to have research and management training within a curriculum that has a maximum of 40 hours per week. Observed preference heterogeneity will be assessed using the RC-B scale data and characteristics of respondents. Latent class models will be used to test for unobserved heterogeneity.

ETHICS AND DISSEMINATION

The research ethics and institutional committees of the sites have approved the study. The survey includes an informed consent section. Study findings will be reported to the medical schools and papers will be submitted to peer-reviewed, accredited journals and higher education and health economic conferences.

Current Practices for Accounting for Preference Heterogeneity in Health-Related Discrete Choice Experiments: A Systematic Review

BACKGROUND

Accounting for preference heterogeneity is a growing analytical practice in health-related discrete choice experiments (DCEs). As heterogeneity may be examined from different stakeholder perspectives with different methods, identifying the breadth of these methodological approaches and understanding the differences are major steps to provide guidance on good research practices.

OBJECTIVES

Our objective was to systematically summarize current practices that account for preference heterogeneity based on the published DCEs related to healthcare.

METHODS

This systematic review is part of the project led by the Professional Society for Health Economics and Outcomes Research (ISPOR) health preference research special interest group. The systematic review conducted systematic searches on the PubMed, OVID, and Web of Science databases, as well as on two recently published reviews, to identify articles. The review included health-related DCE articles published between 1 January 2000 and 30 March 2020. All the included articles also presented evidence on preference heterogeneity analysis based on either explained or unexplained factors or both.

RESULTS

Overall, 342 of the 2202 (16%) articles met the inclusion/exclusion criteria for extraction. The trend showed that analyses of preference heterogeneity increased substantially after 2010 and that such analyses mainly examined heterogeneity due to observable or unobservable factors in individual characteristics. Heterogeneity through observable differences (i.e., explained heterogeneity) is identified among 131 (40%) of the 342 articles and included one or more interactions between an attribute variable and an observable characteristic of the respondent. To capture unobserved heterogeneity (i.e., unexplained heterogeneity), the studies largely estimated either a mixed logit (n = 205, 60%) or a latent-class logit (n = 112, 32.7%) model. Few studies (n = 38, 11%) explored scale heterogeneity or heteroskedasticity.

CONCLUSIONS

Providing preference heterogeneity evidence in health-related DCEs has been found as an increasingly used practice among researchers. In recent studies, controlling for unexplained preference heterogeneity has been seen as a common practice rather than explained ones (e.g., interactions), yet a lack of providing methodological details has been observed in many studies that might impact the quality of analysis. As heterogeneity can be assessed from different stakeholder perspectives with different methods, researchers should become more technically pronounced to increase confidence in the results and improve the ability of decision makers to act on the preference evidence.

What can discrete choice experiments do for you?

CONTEXT

In everyday life, the choices we make are influenced by our preferences for the alternatives available to us. The same is true when choosing medical education, training and jobs. More often than not, those alternatives comprise multiple attributes and our ultimate choice will be guided by the value we place on each attribute relative to the others. In education, for example, choice of university is likely to be influenced by preferences for institutional reputation, location, cost and course content; but which of these attributes is the most influential? An understanding of what is valued by applicants, students, trainees and colleagues is of increasing importance in the higher education and medical job marketplaces because it will help us to develop options that meet their needs and preferences.

METHODS

In this article, we describe the discrete choice experiment (DCE), a survey method borrowed from economics that allows us to quantify the values respondents place on the attributes of goods and services, and to explore whether and to what extent they are willing to trade less of one attribute for more of another.

CONCLUSIONS

To date, DCEs have been used to look at medical workforce issues but relatively little in the field of medical education. However, many outstanding questions within medical education could be usefully addressed using DCEs. A better understanding of which attributes have most influence on, for example, staff or student satisfaction, choice of university and choice of career, and the extent to which stakeholders are prepared to trade one attribute against another is required. Such knowledge will allow us to tailor the way medical education is provided to better meet the needs of key stakeholders within the available resources.

Using Latent Class Analysis to Model Preference Heterogeneity in Health: A Systematic Review

BACKGROUND

Latent class analysis (LCA) has been increasingly used to explore preference heterogeneity, but the literature has not been systematically explored and hence best practices are not understood.

OBJECTIVE

We sought to document all applications of LCA in the stated-preference literature in health and to inform future studies by identifying current norms in published applications.

METHODS

We conducted a systematic review of the MEDLINE, EMBASE, EconLit, Web of Science, and PsycINFO databases. We included stated-preference studies that used LCA to explore preference heterogeneity in healthcare or public health. Two co-authors independently evaluated titles, abstracts, and full-text articles. Abstracted key outcomes included segmentation methods, preference elicitation methods, number of attributes and levels, sample size, model selection criteria, number of classes reported, and hypotheses tests. Study data quality and validity were assessed with the Purpose, Respondents, Explanation, Findings, and Significance (PREFS) quality checklist.

RESULTS

We identified 2560 titles, 99 of which met the inclusion criteria for the review. Two-thirds of the studies focused on the preferences of patients and the general population. In total, 80% of the studies used discrete choice experiments. Studies used between three and 20 attributes, most commonly four to six. Sample size in LCAs ranged from 47 to 2068, with one-third between 100 and 300. Over 90% of the studies used latent class logit models for segmentation. Bayesian information criterion (BIC), Akaike information criterion (AIC), and log-likelihood (LL) were commonly used for model selection, and class size and interpretability were also considered in some studies. About 80% of studies reported two to three classes. The number of classes reported was not correlated with any study characteristics or study population characteristics (p > 0.05). Only 30% of the studies reported using statistical tests to detect significant variations in preferences between classes. Less than half of the studies reported that individual characteristics were included in the segmentation models, and 30% reported that post-estimation analyses were conducted to examine class characteristics. While a higher percentage of studies discussed clinical implications of the segmentation results, an increasing number of studies proposed policy recommendations based on segmentation results since 2010.

CONCLUSIONS

LCA is increasingly used to study preference heterogeneity in health and support decision-making. However, there is little consensus on best practices as its application in health is relatively new. With an increasing demand to study preference heterogeneity, guidance is needed to improve the quality of applications of segmentation methods in health to support policy development and clinical practice.

Segmenting patients and physicians using preferences from discrete choice experiments

People often form groups or segments that have similar interests and needs and seek similar benefits from health providers. Health organizations need to understand whether the same health treatments, prevention programs, services, and products should be applied to everyone in the relevant population or whether different treatments need to be provided to each of several segments that are relatively homogeneous internally but heterogeneous among segments. Our objective was to explain the purposes, benefits, and methods of segmentation for health organizations, and to illustrate the process of segmenting health populations based on preference coefficients from a discrete choice conjoint experiment (DCE) using an example study of prevention of cyberbullying among university students. We followed a two-level procedure for investigating segmentation incorporating several methods for forming segments in Level 1 using DCE preference coefficients and testing their quality, reproducibility, and usability by health decision makers. Covariates (demographic, behavioral, lifestyle, and health state variables) were included in Level 2 to further evaluate quality and to support the scoring of large databases and developing typing tools for assigning those in the relevant population, but not in the sample, to the segments. Several segmentation solution candidates were found during the Level 1 analysis, and the relationship of the preference coefficients to the segments was investigated using predictive methods. Those segmentations were tested for their quality and reproducibility and three were found to be very close in quality. While one seemed better than others in the Level 1 analysis, another was very similar in quality and proved ultimately better in predicting segment membership using covariates in Level 2. The two segments in the final solution were profiled for attributes that would support the development and acceptance of cyberbullying prevention programs among university students. Those segments were very different-where one wanted substantial penalties against cyberbullies and were willing to devote time to a prevention program, while the other felt no need to be involved in prevention and wanted only minor penalties. Segmentation recognizes key differences in why patients and physicians prefer different health programs and treatments. A viable segmentation solution may lead to adapting prevention programs and treatments for each targeted segment and/or to educating and communicating to better inform those in each segment of the program/treatment benefits. Segment members' revealed preferences showing behavioral changes provide the ultimate basis for evaluating the segmentation benefits to the health organization.

A comparison of classroom and online asynchronous problem-based learning for students undertaking statistics training as part of a Public Health Masters degree

This case-study compared traditional, face-to-face classroom-based teaching with asynchronous online learning and teaching methods in two sets of students undertaking a problem-based learning module in the multilevel and exploratory factor analysis of longitudinal data as part of a Masters degree in Public Health at Maastricht University. Students were allocated to one of the two study variants on the basis of their enrolment status as full-time or part-time students. Full-time students (n = 11) followed the classroom-based variant and part-time students (n = 12) followed the online asynchronous variant which included video recorded lectures and a series of asynchronous online group or individual SPSS activities with synchronous tutor feedback. A validated student motivation questionnaire was administered to both groups of students at the start of the study and a second questionnaire was administered at the end of the module. This elicited data about student satisfaction with the module content, teaching and learning methods, and tutor feedback. The module coordinator and problem-based learning tutor were also interviewed about their experience of delivering the experimental online variant and asked to evaluate its success in relation to student attainment of the module's learning outcomes. Student examination results were also compared between the two groups. Asynchronous online teaching and learning methods proved to be an acceptable alternative to classroom-based teaching for both students and staff. Educational outcomes were similar for both groups, but importantly, there was no evidence that the asynchronous online delivery of module content disadvantaged part-time students in comparison to their full-time counterparts.

Evaluation of an e-PBL model to promote individual reasoning

AIM

Medical educators should promote the development of student clinical reasoning toward independence in clinical settings. The objective of this study was to evaluate an online problem-based learning (e-PBL) model designed to promote student individual reasoning in supplement to traditional PBL.

METHODS

Twelve e-PBL modules were added to the fully problem-based curriculum for Year 2 at Sungkyunkwan University School of Medicine (SKKUSOM). In this e-PBL, students worked on the problems individually in an online setting, followed by face-to-face discussions in a colloquium. The cases were presented using interactive multimedia to enhance the authenticity of the case and stimulate student interest in learning. A formative evaluation study was conducted to determine student satisfaction with e-PBL and its effectiveness as perceived by the students using both quantitative and qualitative methods. A cohort of Year 2 students at SKKUSOM (n = 38) took part in this study.

RESULTS

Students perceived e-PBL significantly more positively after they had taken a module in terms of its ability to foster problem-solving skills and its ability to allow them to learn in ways suited to individual learning styles. Additionally, student survey and interview revealed that a vast majority of students were satisfied with the overall learning process in e-PBL and perceived it positively in fostering knowledge acquisition and clinical reasoning. Moreover, students found the cases realistic and engaging.

CONCLUSIONS

The results show the potential of e-PBL to enhance traditional PBL by promoting the development of individual reasoning in a flexible online-learning environment and offering cases in an interactive multimedia format, which warrants further investigation into its impact on student learning outcomes.

Adaptive choice-based conjoint analysis: a new patient-centered approach to the assessment of health service preferences

Conjoint analysis (CA) has emerged as an important approach to the assessment of health service preferences. This article examines Adaptive Choice-Based Conjoint Analysis (ACBC) and reviews available evidence comparing ACBC with conventional approaches to CA. ACBC surveys more closely approximate the decision-making processes that influence real-world choices. Informants begin ACBC surveys by completing a build-your-own (BYO) task identifying the level of each attribute that they prefer. The ACBC software composes a series of attribute combinations clustering around each participant's BYO choices. During the Screener section, informants decide whether each of these concepts is a possibility or not. Probe questions determine whether attribute levels consistently included in or excluded from each informant's Screener section choices reflect 'Unacceptable' or 'Must Have' simplifying heuristics. Finally, concepts identified as possibilities during the Screener section are carried forward to a Choice Tournament. The winning concept in each Choice Tournament set advances to the next choice set until a winner is determined.A review of randomized trials and cross-over studies suggests that, although ACBC surveys require more time than conventional approaches to CA, informants find ACBC surveys more engaging. In most studies, ACBC surveys yield lower standard errors, improved prediction of hold-out task choices, and better estimates of real-world product decisions than conventional choice-based CA surveys.

Blended learning positively affects students' satisfaction and the role of the tutor in the problem-based learning process: results of a mixed-method evaluation

Problem-based learning (PBL) is an established didactic approach in medical education worldwide. The impact of PBL depends on the tutors' quality and the students' motivation. To enhance students' motivation and satisfaction and to overcome the problems with the changing quality of tutors, online learning and face-to-face classes were systematically combined resulting in a blended learning scenario (blended problem-based learning--bPBL). The study aims at determining whether bPBL increases the students' motivation and supports the learning process with respect to the students' cooperation, their orientation, and more reliable tutoring. The blended PBL was developed in a cooperation of students and teachers. The well-established Seven Jump-scheme of PBL was carefully complemented by eLearning modules. On the first training day of bPBL the students start to work together with the online program, but without a tutor, on the final training day the tutor participates in the meeting for additional help and feedback. The program was evaluated by a mixed-method study. The traditional PBL course was compared with the blended PBL by means of qualitative and quantitative questionnaires, standardized group interviews, and students' test results. In addition log-files were analyzed. A total of 185 third-year students and 14 tutors took part in the study. Motivation, subjective learning gains and satisfaction achieved significantly higher ratings by the bPBL students compared with the students learning by traditional PBL. The tutors' opinion and the test results showed no differences between the groups. Working with the web-based learning environment was assessed as very good by the students. According to the log-file analysis, the web-based learning module was frequently used and improved the cooperation during the self-directed learning.

Modeling the information preferences of parents of children with mental health problems: a discrete choice conjoint experiment

Although materials informing parents about children's mental health (CMH) problems can improve outcomes, we know relatively little about the design factors that might influence their utilization of available resources. We used a discrete choice conjoint experiment to model the information preferences of parents seeking mental health services for 6 to 18 year olds. Parents completed 30 choice tasks presenting experimentally varied combinations of 20 four-level CMH information content, transfer process, and outcome attributes. Latent class analysis revealed three segments with different preferences. Parents in the Action segment (43%) chose materials providing step-by-step solutions to behavioral or emotional problems. They preferred weekly meetings with other parents and coaching calls from a therapist. The Information segment (41%) chose materials helping them understand rather than solve their child's problems. These parents were more sensitive to logistical factors such as receiving information in groups, the location where information was available, the modality in which the information was presented, and the time required to obtain and use the information. The Overwhelmed segment (16%) reported more oppositional and conduct problems, felt their children's difficulties exerted a greater adverse impact on family functioning, and reported higher personal depression scores than those in the Action or Information segments. Nonetheless, they did not choose information about, or solutions to, the problems their children presented. Simulations predicted that maximizing utilization and realizing the potential benefits of CMH information would require knowledge transfer strategies consistent with each segment's preferences.

Problem-based learning: where are we now?

The definition of problem-based learning (PBL) as an educational concept is as elusive in 2008 as it has been since the concept was first expressed over forty years ago. A definitive guide to the practice of PBL is equally elusive. Like all worthwhile educational ideas, PBL has proved attractive to those teachers who seek improvements for their courses. Its appeal has transcended the traditional boundaries in formal education so that there are examples of PBL from primary to tertiary education, and across many disciplines within these. Dissemination, however, has wrought confusion in understanding and practice, and consequent difficulties for researchers in evaluating its efficacy, and lack of clear advice for those who would like to adopt PBL. Rather than attempting to be definitive, this Guide explores the various interpretations and practices that claim the label PBL, and critiques these against the original concept and practice. The primary aim is to provide insight into the causes of the confusion about PBL in 2008. The second aim is to point a feasible way forward so that, where appropriate, the potential of PBL as a whole-of-curriculum concept may be realised; and, where it is not possible to implement the whole concept, worthwhile educational principles that have been associated more or less with PBL may be recognised as such and given value in their own right.