Effect of the framing of questionnaire items regarding satisfaction with training on residents' responses

Development and validation of the 25-item Hikikomori Questionnaire (HQ-25)

AIM

Hikikomori, a form of severe social withdrawal, is an emerging issue in mental health, for which validated measurement tools are lacking. The object was to develop a self-report scale of hikikomori, and assess its psychometric properties and diagnostic accuracy.

METHODS

A sample of 399 participants from clinical and community settings completed measures. Psychometric properties were assessed with factor analysis; diagnostic accuracy was compared against a semi-structured diagnostic interview.

RESULTS

The Hikikomori Questionnaire contained 25 items across three subscales representing socialization, isolation, and emotional support. Internal consistency, test-retest reliability, and convergent validity were all satisfactory. The area under the curve was 0.86 (95% confidence interval, 0.80-0.92). A cut-off score of 42 (out of 100) was associated with a sensitivity of 94%, specificity of 61%, and positive predictive value of 17%.

CONCLUSION

The 25-item Hikikomori Questionnaire (HQ-25) possesses robust psychometric properties and diagnostic accuracy in an initial sample of Japanese adults. Additional research on its psychometric properties and ability to support clinical assessment of hikikomori is warranted.

Pediatric Resident Burnout and Attitudes Toward Patients

BACKGROUND AND OBJECTIVES

Burnout occurs in up to 75% of resident physicians. Our study objectives were to: (1) determine the prevalence of burnout, and (2) examine the association between burnout and self-reported patient care attitudes and behaviors among pediatric residents.

METHODS

A total of 258 residents (53% response rate) from 11 pediatric residency programs completed a cross-sectional Web-based survey. Burnout was measured with 2 items from the Maslach Burnout Inventory. Patient care attitudes and behaviors were measured with 7 questions from a standardized qualitative survey. χ² and logistic regression tested the association between burnout and self-reported patient care attitudes and behavior.

RESULTS

A total of 39% of respondents (mean age, 29.4 years ± 2.3 SD; 79% female; 83% white; 35% postgraduate year [PGY] -1, 34% PGY-2, and 31% PGY-3), endorsed burnout. Residents with burnout had significantly greater odds (P < .01) of reporting suboptimal patient care attitudes and behaviors, including: discharging patients to make the service more manageable (adjusted odds ratio [aOR] 4.2; 95% confidence interval [CI], 1.6-11.1), not fully discussing treatment options or answering questions (aOR 3.5; 95% CI, 1.7-7.1), making treatment or medication errors (aOR 7.1; 95% CI, 2.0-25.8), ignoring the social or personal impact of an illness (aOR 9.6; 95% CI, 3.2-28.9), and feeling guilty about how a patient was treated (aOR 6.0; 95% CI, 1.6-22.1).

CONCLUSIONS

Burnout is highly prevalent among pediatric residents and is associated with self-reported negative patient care attitudes and behaviors. Residency programs should develop interventions addressing burnout and its potential negative impact on patient care.

Evaluating the quality, clinical relevance, and resident perception of the radiation oncology in-training examination: A national survey

PURPOSE

The yearly radiation oncology in-training examination (ITE) by the American College of Radiology is a widely used, norm-referenced educational assessment, with high test reliability and psychometric performance. We distributed a national survey to evaluate the academic radiation oncology community's perception of the ITE.

METHODS AND MATERIALS

In June 2014, a 7-question online survey was distributed via e-mail to current radiation oncology residents, program directors, and attending physicians who had completed residency in the past 5 years or junior attendings. Survey questions were designed on a 5-point Likert scale. Sign test was performed with P ≤ .05 considered statistically different from neutral.

RESULTS

Thirty-one program directors (33.3%), 114 junior attendings (35.4%), and 225 residents (41.2%) responded. Junior attendings and program directors reported that the ITE directly contributed to their preparation for the American Board of Radiology written certification (P = .050 and .004, respectively). Residents did not perceive the examination as an accurate assessment of relevant clinical and scientific knowledge (P < .0001) and feel the quality assurance is insufficient in its current form (P < .0001). Residents and junior attendings agree that there are factual errors, and unclear questions/answers (P < .0001 and .04, respectively). Free response suggestions included: less questions on rare disease sites (16.4%), more relevance to clinical practice (15.4%), avoiding questions that discriminate between a few percentage points (11.8%), and designing the test similar to the written certification examination (9.2%).

CONCLUSIONS

Despite high examination reliability and psychometric performance, resident and attending physicians report a need for improved quality assurance and clinical relevance in the ITE. Although the current examination allows limited feedback, establishing a venue for individualized feedback may allow continual and timely improvement of the ITE. Adopting a criterion-referenced examination may further increase resident investment in and utilization of this valuable learning tool.

Dutch paediatrician's opinions about acute care for critically ill children in general hospitals

Paediatricians in general hospitals have limited experience with critically ill children, due to the low incidence and their diversity in age, pathology and presentation. Consequently, adequate organization, training and materials and medication are of major importance. This voluntary and anonymous survey-based study was conducted to gain insight in the current status of these aspects. In June 2012, all 687 paediatricians employed at 84 general hospitals in The Netherlands received a hardcopy questionnaire with questions relating to demographics, organization, training and materials and medication concerning the acute care for critically ill children. Of the sent questionnaires, 41.3% were eligible for analysis. According to the organization of the acute care of critically ill children, 73.9% of the respondents indicated verbal agreements were made, of which 77.0% stated that these were recorded in written protocols. Taskforces were present according to 64.5% of our respondents. Of the respondents, 64.4% were Advanced Paediatric Life Support (APLS) certified. Of the stated training scenarios, 90.8% were available in their hospital, which were followed on a regular basis by 63.9% of the paediatricians. Paediatric resuscitation carts were present on both emergency department and paediatric ward according to 95.1%. Materials (37.7%) and medication (45.3%) were frequently lacking.

CONCLUSION

Paediatricians from general hospitals in The Netherlands consider that acute care for critically ill children has to be improved in terms of organization, training and teamwork, and medication and materials. National guidelines concerning the organization and training may contribute to this improvement, as well as a standardized inventory list for paediatric resuscitation carts.

Confounding factors in using upward feedback to assess the quality of medical training: a systematic review

PURPOSE

Upward feedback is becoming more widely used in medical training as a means of quality control. Multiple biases exist, thus the accuracy of upward feedback is debatable. This study aims to identify factors that could influence upward feedback, especially in medical training.

METHODS

A systematic review using a structured search strategy was performed. Thirty-five databases were searched. Results were reviewed and relevant abstracts were shortlisted. All studies in English, both medical and non-medical literature, were included. A simple pro-forma was used initially to identify the pertinent areas of upward feedback, so that a focused pro-forma could be designed for data extraction.

RESULTS

A total of 204 articles were reviewed. Most studies on upward feedback bias were evaluative studies and only covered Kirkpatrick level 1-reaction. Most studies evaluated trainers or training, were used for formative purposes and presented quantitative data. Accountability and confidentiality were the most common overt biases, whereas method of feedback was the most commonly implied bias within articles.

CONCLUSION

Although different types of bias do exist, upward feedback does have a role in evaluating medical training. Accountability and confidentiality were the most common biases. Further research is required to evaluate which types of bias are associated with specific survey characteristics and which are potentially modifiable.

Measuring risk perceptions of skin cancer: reliability and validity of different operationalizations

OBJECTIVE

Risk perceptions play a pivotal role in health behaviour theories. Accurate measurement is essential in order to investigate the explanatory value and effectiveness of interventions influencing these beliefs. This study investigated the reliability and predictive validity of different risk perception operationalizations related to skin cancer and sunscreen use in order to explain the inconsistent findings in literature regarding the relationship between risk perceptions and (cancer related) behaviours.

DESIGN AND METHODS

Two on-line surveys were conducted. Study 1 (N= 175) was conducted among university students to investigate the test-retest reliability of different operationalizations of perceived likelihood and perceived severity and to assess their correlations with sunscreen use. Study 2 (N= 418) was a prospective study among Dutch adults and assessed longitudinal correlations between the different operationalizations and sunscreen use.

RESULTS

Internal consistency (alpha) ranged between .13 and .90 for likelihood measures and between .37 and .88 for severity measures. Test-retest correlations (r/ICC) ranged between .51 and .82 for the likelihood measures and between .58 and .75 for the severity measures. Conditional likelihood estimates, particularly when phrased affectively (feeling at risk), and comparative severity questions were the strongest correlates of behaviour.

CONCLUSION

Conditional likelihood and comparative severity might be better predictors of health behaviour than commonly used operationalizations of risk perception. These measures may be relevant for use in the development and evaluation of intervention programmes, and should be acknowledged by health behaviour theories. Suggestions for future research are discussed.

Extreme and acquiescence bias in a bi-ethnic population

BACKGROUND

Extreme and acquiescence biases are the tendency to give a positive or extreme answer regardless of the 'true' answer. These biases may compromise comparisons of attitudes regarding health between population groups. The aim of the study was to measure the extent of extreme and acquiescence biases and identify factors associated with them in two ethnic groups: Jews and Arabs in Israel.

METHODS

A random telephone survey was conducted during 2006, interviewing 2322 Jews and 809 Arabs. Three attitude questions were presented twice with opposite wording to measure extreme and acquiescence biases in these two groups.

RESULTS

Extreme bias ranged from 2 to 14% among Jews and from 6 to 29% among Arabs, depending on the question. Acquiescence bias ranged from 2 to 10% among Jews and 5-19% among Arabs. The less educated respondents gave more extreme biased responses for all items. The older respondents gave more extreme answers for two out of the three questions tested. After adjusting for age and education the odds ratio (OR) of giving more extreme biased answers was higher among Arabs compared with Jews for all three questions [OR = 2.49, confidence interval (CI) = 1.87, 3.31; OR = 2.33, CI = 1.75, 3.10; and OR = 2.94, CI = 1.83-4.71, respectively, for each question].

CONCLUSIONS

Levels of response biases are higher in the Arab minority population compared with the majority Jewish population and depended on the subject, age and education.

The use of expert elicitation in environmental health impact assessment: a seven step procedure

BACKGROUND

Environmental health impact assessments often have to deal with substantial uncertainties. Typically, the knowledge-base is limited with incomplete, or inconsistent evidence and missing or ambiguous data. Consulting experts can help to identify and address uncertainties.

METHODS

Formal expert elicitation is a structured approach to systematically consult experts on uncertain issues. It is most often used to quantify ranges for poorly known parameters, but may also be useful to further develop qualitative issues such as definitions, assumptions or conceptual (causal) models. A thorough preparation and systematic design and execution of an expert elicitation process may increase the validity of its outcomes and transparency and trustworthiness of its conclusions. Various expert elicitation protocols and methods exist. However, these are often not universally applicable, and need customization to suite the needs of a specific study. In this paper, we set out to develop a widely applicable method for the use of expert elicitation in environmental health impact assessment.

RESULTS

We present a practical yet flexible seven step procedure towards organising expert elicitation in the context of environmental health impact assessment, based on existing protocols. We describe how customization for specific applications is always necessary. In particular, three issues affect the choice of methods for a particular application: the types of uncertainties considered, the intended use of the elicited information, and the available resources. We outline how these three considerations guide choices regarding the design and execution of expert elicitation. We present signposts to sources where the issues are discussed in more depth to give the newcomer the insights needed to make the protocol work. The seven step procedure is illustrated using examples from earlier published elicitations in the field of environmental health research.

CONCLUSIONS

We conclude that, despite some known criticism on its validity, formal expert elicitation can support environmental health research in various ways. Its main purpose is to provide a temporary summary of the limited available knowledge, which can serve as a provisional basis for policy until further research has been carried out.

Effect of order of presentation of a generic and a specific health-related quality of life instrument in knee and hip osteoarthritis: a randomized study

OBJECTIVE

Current guidelines recommend using both a generic and a specific instrument to measure quality of life (QoL) among people with chronic diseases. However, the two questionnaires may not be independent, which raises the issue of whether the order in which they are completed influences their value. We aimed to assess the effect of order of presentation of a generic (SF36) and a specific (Osteoarthritis Knee and Hip QoL [OAKHQOL]) QoL instrument administered to patients with knee and hip osteoarthritis (OA).

METHODS

We recruited 341 outpatients from rheumatology and orthopaedic surgery clinics. Demographic and clinical data and responses to the SF36 and OAKHQOL self-administered questionnaires were collected at baseline and 10 days after inclusion; surgical patients were also assessed 6 months post-surgery. The order of presentation of the instruments was randomized at inclusion.

RESULTS

The order of instrument presentation had no significant effect on response rate, number of dimensions with a floor or a ceiling effect, or questionnaire scores. In one of 13 dimensions (social support dimension of the OAKHQOL), test-retest reliability was slightly better when the generic SF36 questionnaire was presented first (intra-class correlation coefficient (ICC) 0.84 vs 0.55). The analysis of sensitivity to change and discriminant ability did not favour one group over the other.

CONCLUSION

The order of presentation of a generic and a specific QoL questionnaire to patients with OA had an only marginal effect on the quality of responses and the QoL scores obtained.

Perceptions of stress among pathology residents: survey results and some strategies to reduce them

We conducted a survey to attempt to identify stressors perceived by pathology residents. A comparison of survey results from pathology residents with stressors perceived by residents in other specialties was done to evaluate whether the stressors in pathology are unique. A confidential e-mail survey of residency programs in the United States generated a response rate from program directors of 23.3% and a resident response rate of 6.4%. Although the study is limited by response rates and the data are primarily from university-based residency programs, the concordance between the identifiable risk factors for stress by the resident and program director groups validates the importance of this issue among pathology residency programs. Both groups identified variability in faculty expectations and work overload that inhibits optimal learning as the top 2 stressors for residents. We discuss various factors that contribute to stress and some strategies that pathology residency programs can implement to address the stressors.

Linguistic adaptation and validation of the General Oral Health Assessment Index (GOHAI) in an elderly Japanese population

OBJECTIVES

This study sought to develop a Japanese version of the General Oral Health Assessment Index (GOHAI) and validate its use in an elderly Japanese population.

METHODS

The GOHAI scale was translated in a standardized way consisting of forward translation, pilot study, and backward translation. Psychometric properties of the final version were assessed on 175 participants (mean age: 70.0 years, SD 6.4).

RESULTS

The response rate was more than 98% for each item. The reliability in terms of a Cronbach's alpha value was 0.89. The mean GOHAI scores were significantly lower for participants with poor perceived oral health (P<0. 001) or poor perceived general health (P<0.01). Significant correlations were observed between the GOHAI scores and the general health or physical functioning scores of the Medical Outcome Study Short Form 36 (P<0.01).

CONCLUSIONS

The present study showed that the Japanese version of the GOHAI was psychometrically valid in an elderly sample population.

Mortality predictions in the intensive care unit: comparing physicians with scoring systems

OBJECTIVE

Risk-prediction models offer potential advantages over physician predictions of outcomes in the intensive care unit (ICU). Our systematic review compared the accuracy of ICU physicians' and scoring system predictions of ICU or hospital mortality of critically ill adults.

DATA SOURCE

MEDLINE (1966-2005), CINAHL (1982-2005), Ovid Healthstar (1975-2004), EMBASE (1980-2005), SciSearch (1980-2005), PsychLit (1985-2004), the Cochrane Library (Issue 1, 2005), PubMed "related articles," personal files, abstract proceedings, and reference lists.

STUDY SELECTION

We considered all studies that compared physician predictions of ICU or hospital survival of critically ill adults to an objective scoring system, computer model, or prediction rule. We excluded studies if they focused exclusively on the development or economic evaluation of a scoring system, computer model, or prediction rule.

DATA EXTRACTION AND ANALYSIS

We independently abstracted data and assessed study quality in duplicate. We determined summary receiver operating characteristic curves and areas under the summary receiver operating characteristic curves+/-se and summary diagnostic odds ratios.

DATA SYNTHESIS

We included 12 observational studies of moderate methodological quality. The area under the summary receiver operating characteristic curves for seven studies was 0.85+/-0.03 for physician predictions compared with 0.63+/-0.06 for scoring system predictions (p=.002). Physicians' summary diagnostic odds ratios derived from the area under the summary receiver operating characteristic curves were significantly higher (12.43; 95% confidence interval 5.47, 27.11) than scoring systems' summary diagnostic odds ratios (2.25; 95% confidence interval 0.78, 6.52, p=.001). Combined results of all 12 studies indicated that physicians predict mortality more accurately than do scoring systems: ratio of diagnostic odds ratios (95% confidence interval) 1.92 (1.19, 3.08) (p=.007).

CONCLUSIONS

Observational studies suggest that ICU physicians discriminate between survivors and nonsurvivors more accurately than do scoring systems in the first 24 hrs of ICU admission. The overall accuracy of both predictions of patient mortality was moderate, implying limited usefulness of outcome prediction in the first 24 hrs for clinical decision making.

Well-being in residency training: a survey examining resident physician satisfaction both within and outside of residency training and mental health in Alberta

BACKGROUND

Despite the critical importance of well-being during residency training, only a few Canadian studies have examined stress in residency and none have examined well-being resources. No recent studies have reported any significant concerns with respect to perceived stress levels in residency. We investigated the level of perceived stress, mental health and understanding and need for well-being resources among resident physicians in training programs in Alberta, Canada.

METHODS

A mail questionnaire was distributed to the entire resident membership of PARA during 2003 academic year. PARA represents each of the two medical schools in the province of Alberta.

RESULTS

In total 415 (51 %) residents participated in the study. Thirty-four percent of residents who responded to the survey reported their life as being stressful. Females reported stress more frequently than males (40% vs. 27%, p < 0.02). Time pressure was reported as the number one factor contributing to stress (44% of males and 57% of females). A considerable proportion of residents would change their specialty program (14%) and even more would not pursue medicine (22%) if given the opportunity to relive their career. Up to 55% of residents reported experiencing intimidation and harassment. Intimidation and harassment was strongly related to gender (12% of males and 38% of females). Many residents (17%) rated their mental health as fair or poor. This was more than double the amount reported in the Canadian Community Health Survey from the province (8%) or the country (7%). Residents highly valued their colleagues (67%), program directors (60%) and external psychiatrist/psychologist (49%) as well-being resources. Over one third of residents wished to have a career counselor (39%) and financial counselor (38%).

CONCLUSION

Many Albertan residents experience significant stressors and emotional and mental health problems. Some of which differ among genders. This study can serve as a basis for future resource application, research and advocacy for overall improvements to well-being during residency training.

Modalités de l’évaluation de l’enseignement de l’anesthésie–réanimation en France : point de vue des enseignants et des internes

OBJECTIVE

The objective of this study was to provide informations about French practice of anaesthesiology and intensive care graduation.

STUDY DESIGN

Prospective study.

MATERIAL AND METHODS

[corrected] Surveys relative to evaluation tools were sent to professors, and students in anaesthesiology and intensive care in France.

RESULT

About 95% of French students in anaesthesiology and intensive care want to be evaluated all over their cursus, by using "Carnet de stage", tutor-student agreement or written examinations. Pedagogical evaluation is hardly wished by students and professors despite that "Carnet de stage" and written examinations are frequently used before professional certification. In order to improve the professional competence, the patronage and use of simulator are frequently quoted by professors.

CONCLUSION

Despite the fact that "Carnet de stage" and written examinations are frequently used, pedagogic dimension of evaluation need to be improved.

Accuracy of responses from postal surveys about continuing medical education and information behavior: experiences from a survey among German diabetologists

BACKGROUND

Postal surveys are a popular instrument for studies about continuing medical education habits. But little is known about the accuracy of responses in such surveys. The objective of this study was to quantify the magnitude of inaccurate responses in a postal survey among physicians.

METHODS

A sub-analysis of a questionnaire about continuing medical education habits and information management was performed. The five variables used for the quantitative analysis are based on a question about the knowledge of a fictitious technical term and on inconsistencies in contingency tables of answers to logically connected questions.

RESULTS

Response rate was 52%. Non-response bias is possible but seems not very likely since an association between demographic variables and inconsistent responses could not be found. About 10% of responses were inaccurate according to the definition.

CONCLUSION

It was shown that a sub-analysis of a questionnaire makes a quantification of inaccurate responses in postal surveys possible. This sub-analysis revealed that a notable portion of responses in a postal survey about continuing medical education habits and information management was inaccurate.

[Residents' assessment of education received in hospital departments: a monitoring tool]

OBJECTIVE

[corrected] The Spanish postgraduate medical education system has made great contributions to the development of the National Health Service. Despite recent regulations on critical aspects of this education system, there still remains a need for a global assessment process. Hospital services evaluation by the residents should play a part in this process.

METHODS

Administration over three years in a general teaching hospital of a specific questionnaire devoted to measure residents' perceptions of tthe medical education provided by their own service or department. Only residents who had stayed at least one year in their service were allowed to participate. Multivariable analyses using Multiple Correspondence Analysis (MCA) and Automatic Classification (AC) methods were performed.

RESULTS

The overall response rate was 66.6% (325/488). 84,7% of the respondents deemed the overall education received as either adequate or excellent. The better scored aspects were patient management education, ethics education and residents' patient care supervision (more than 80% responses rating them as adequate/excellent). Performance of interdepartamental and bibliographic sessions along with research education were the worse perceived aspects by residents (less than 50% of responses as adequate/excellent). A factorial plane that explained 95% of overall response variability and allowed to rank the residents according to their assessment of education was obtained. Services with utmost ratings were found. AC results showed that were three different groups on the basis of overall peception of education received: The first group (18.2%) deemed it as excellent, the second group (61.5%) as adequate, while the thrid group (20.3%) considered it as inadequate.

CONCLUSIONS

The administration of this questionnaire to hospital's residents and the analysis of its results using specific multivariable techniques provides useful information in order to monitor postgraduate medical education programmes and detect areas of improvement.

Adapting to heart conditions: a test of the hedonic treadmill

This paper tests the hypothesis of hedonic adaptation by analyzing the role that a history of heart problems has on the ability to deal with future heart conditions. The results show that those who have had a heart condition in the past are less likely to report worse self-assessed health and emotional health due to the onset of a new condition than those who have not previously had exposure to heart trouble. The results are fairly supportive of the notion of a hedonic treadmill.