Nonequivalence of on-line and paper-and-pencil psychological tests: The case of the prospective memory questionnaire

A scoping review of the utility of self-report and informant-report prospective memory measures

There has been increased use of self-report prospective memory (PM) scales in recent years, despite uncertainty about their validity. This study reviewed how self-and informant-report PM questionnaires have been used in the assessment of PM. We evaluated relationships between self-report, informant-report, and performance-based PM measures, and the validity of using self-report measures in detecting PM impairments and monitoring intervention outcomes. The scoping review methodology of Arksey and O'Malley (2005. Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32) was used. Database searches yielded 488 published studies that used the Prospective Memory Questionnaire (PMQ), Prospective and Retrospective Memory Questionnaire (PRMQ), Comprehensive Assessment of PM (CAPM), and Brief Assessment of PM (BAPM). The self-report and informant-report measures of PM had weak- to moderate-strength relationships with performance-based PM measures. Some self-report PM scales could detect PM impairments and monitor intervention outcomes, however few studies had investigated this. The findings indicated that self- and informant-report scales measure different constructs to performance-based measures of PM. It is recommended that these scales be used alongside performance-based measures to provide complementary and comprehensive assessments of PM. Further research into assessment of PM using self-report measures will aid assessment and treatment choices.

Video-Based Remote Administration of Cognitive Assessments and Interventions: a Comparison with In-Lab Administration

While remote data collection is not a new concept, the quality and psychometric properties of data collected remotely often remain unclear. Most remote data collection is done via online survey tools or web-conferencing applications (i.e., Skype or Zoom) and largely involves questionnaires, interviews, or other self-report data. Little research has been done on the collection of cognitive assessments and interventions via web-conferencing that requires multiple sessions with or without the assistance of an experimenter. The present paper discusses limitations and challenges of studies administered remotely, and outlines methods used to overcome such challenges while effectively collecting cognitive performance data remotely via Zoom. We further discuss relative recruitment, retention rates, compliance, and performance findings between in-lab and remotely administered cognitive assessment and intervention studies, as well as limitations to remote data collection. We found that while it was necessary to recruit more participants in remote studies to reach enrollment goals, compliance and performance were largely comparable between in-lab and remotely administered studies, illustrating the opportunities of conducting this type of experimental research remotely with adequate fidelity.

Validation of the Prospective Memory Concerns Questionnaire (PMCQ)

Prospective memory (PM), the ability to remember to complete intended tasks, is essential for successfully completing activities of daily living. PM impairments are common in people with neuropathology such as acquired brain injury and dementia. These PM impairments affect individuals’ capabilities in key aspects of daily living including their health, safety, and independence. The Prospective Memory Concerns Questionnaire (PMCQ) was designed as a self-report measure to understand individuals’ concerns about their memory. This questionnaire may help identify issues with PM which in turn may assist clinicians in the targeted implementation of memory compensation strategies. The PMCQ was developed using Rasch and classical test methodologies, with subscales measuring frequency of forgetting behaviors, memory concerns, and retrieval failures. The current study aimed to confirm the factor structure of the PMCQ for use in adults in the general population. The study also aimed to examine relationships between the PMCQ and naturalistic performance-based measures of PM to determine how the self-report PMCQ could be used in conjunction with performance-based measures. A community dwelling sample of 558 adults completed the PMCQ, an event-based naturalistic PM task, and time-based naturalistic PM task. Confirmatory factor analyses (CFAs) indicated that a higher order model with three subscales containing 35 items produced acceptable fit [RMSEA = 0.056 (90% CI 0.054, 0.060), SRMR = 0.062, CFI = 0.915, TLI = 0.909] The PMCQ demonstrated good internal consistency (total α = 0.95, subscales: α = 0.88–0.89). The Forgetting Behaviors subscale significantly correlated with performance on the event-based naturalistic PM task (r = −0.14, p < 0.01). The Memory Concerns and Retrieval Failures subscales did not correlate significantly with performance-based PM tasks. These findings suggest that the PMCQ may be best suited for assessing individuals’ concerns about their forgetting behaviors and identifying appropriate compensation strategies or support services. It is recommended that the PMCQ be used alongside performance-based PM tasks and other cognitive measures to comprehensively assess PM. It was concluded that the PMCQ is a suitable measure for use in adults in the general population. Further validation research of the PMCQ in general population and clinical samples will determine the measures’ sensitivity and specificity in identifying PM impairments.

Comparison of the printed and online administration of the Behavioral Regulation in Exercise Questionnaire (BREQ-2)

OBJECTIVE

To compare the traditional printed form of the Behavioral Regulation in Exercise Questionnaire with a proposed online form in terms of validity, reliability, and applicability.

METHODS

A crossover design study was conducted with 157 undergraduate students. Half of the sample answered the printed questionnaire first and then answered the online questionnaire 7 days later, while the other half of the sample did the inverse. Cronbach's alpha was used to analyze the internal consistency of both the online and printed questionnaires. The construct validity was analyzed by confirmatory factor analysis, using a weighted least square mean and adjusted variance estimation and oblique rotation. The quality of the model was tested with fit indices.

RESULTS

The confirmatory factor analysis showed the 19-item structure with five factors: χ2 of 230.718; degrees of freedom of 142; χ2/degrees of freedom of 1.625; comparative fit index of 0.978 and root mean square error of approximation of 0.073. All items presented factorial loads above 0.5. There was also excellent consistency between the formats of administration in all dimensions, with Cronbach's alpha values above 0.70. The stability between the formats of administration varied between 0.78 (95%CI: 0.69-0.85) and 0.84 (95%CI: 0.77-0.89), suggesting desirable confidence between both formats of administration.

CONCLUSION

The five-factor model of the online Behavioral Regulation in Exercise Questionnaire shows internal consistency both in terms of the scale dimensions as well as in terms of the total items.

Improving the assessment of functional impairment in tinnitus sufferers: validation of the German version of the Tinnitus Functional Index using a confirmatory factor analysis

OBJECTIVES

Chronic tinnitus negatively impacts daily functioning. To specifically assess this impairment, the Tinnitus Functional Index (TFI) was developed. The current study investigated the hierarchical, eight-factorial structure for the German TFI and examined its psychometric properties.

DESIGN

In an online assessment, the TFI and other validated health-related measurements were completed. Confirmatory factor analysis (CFA) was conducted to investigate the factorial structure by testing two competing models: (1) a general factor model, and (2) a hierarchical second-order factor model.

STUDY SAMPLE

316 research volunteers (59.8% female) with low to moderate tinnitus distress were included.

RESULTS

CFA revealed an insufficient fit of the data to the general factor model. For the hierarchical second-order factor model, an acceptable model fit was shown (χ²/df ratio = 2.74, RMSEA = 0.07, SRMR = 0.05, CFI = 0.95, TLI = 0.95). Correlational analyses between the TFI and measures assessing tinnitus distress, depression, sleeping difficulties, subjective well-being, and personality dimensions indicated high convergent and moderate discriminant validity. Internal consistency reliability was excellent.

CONCLUSIONS

The results confirm the hierarchical, eight-factorial structure of the German TFI. The TFI is a promising inventory that should be used on a regular basis.HighlightsThe results of our study confirm the hierarchical eight-factorial structure of the German TFI.Confirmatory factor analysis revealed an acceptable model fit of the data.Convergent validity of the German TFI was high.Discriminant validity of the German TFI was moderate.The German TFI is a reliable questionnaire to assess tinnitus functional impairment.

Psychometric Properties and Factor Structure of the French Version of the Behavioral Activation for Depression Scale (BADS) in Non-Clinical Adults

Behavioral activation (BA) is a well-established empirical treatment for depression that aims to improve depressive mood by increasing activation and reducing avoidance. Therefore, it is essential to evaluate activation and avoidance when a BA treatment is applied. The Behavioral Activation for Depression Scale (BADS) was developed to measure the changes in activation and avoidance over the course of BA treatment of depression. This study aims to validate the French version of this scale. In a first study, 131 bilingual adults were recruited to explored internal consistency, test-retest reliability and construct validity of the final French version. In a second study, 409 non-clinical adults completed an online survey assessing concurrent measures. Results of the first study suggested good internal consistency, test-retest reliability and construct validity. The second study revealed a confirmatory factor analysis supporting the original four-factor structure, with Activation, Avoidance/Rumination, Work/School Impairment, and Social Impairment subscales. Results also revealed that a 5-factor model distinguishing Behavioral Avoidance and Rumination had a better fit than the original four-factor structure. All subscales showed adequate internal consistency and good construct validity with evidence of convergent validity with depressive symptoms, brooding, psychological flexibility, negative automatic thought, behavioral inhibition and activation system. Furthermore, the French BADS total scale and subscales showed a good ability to predict depressive symptoms. The French version of the BADS appears to be a reliable tool for clinician and researchers to assess mechanisms of change in BA interventions.

Validation of a Mobile Version of the American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form: An Observational Randomized Crossover Trial

Background

The American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASES) questionnaire is an effective tool for evaluating shoulder joint function. The development and usage of a mobile version of the ASES questionnaire has the potential to save time, money, and effort.

Objective

The aim of this study is to assess the equivalence between the paper and mobile versions of the ASES questionnaire and their acceptability among patients.

Methods

The paper and mobile versions of the ASES questionnaire were used to evaluate the shoulder joint function of 50 patients with shoulder pain. This study included patients from the shoulder clinic of Sun Yat-sen Memorial Hospital. The intraclass correlation coefficient (ICC) and Bland-Altman method were used to evaluate the agreement (reliability) of the scores obtained by the two methods (paper versus mobile).

Results

Of the 50 patients recruited from March 2018 to May 2019, 46 (92%) completed the study. There was a high agreement between the paper and mobile versions of the ASES questionnaire (ICC=0.979, 95% CI 0.943-0.987; P<.001). The mean difference between the scores of the mobile and paper versions was 1.0, and only 1/46 (2%) had a difference greater than the minimal clinically important difference of 12 points. About 75% of patients preferred the mobile version to the paper version.

Conclusions

Our study shows that the mobile version of the ASES questionnaire is comparable to the paper version, and has a higher patient preference. This could prove to be a useful tool for epidemiological studies and patient follow-up over longer periods of time.

Replicability of the Psychometric Properties of Trait-Levels Measures of Spontaneous and Deliberate Mind Wandering

Abstract. Recent research on individual differences in mind wandering has shown that two different forms, spontaneous and deliberate, can be distinguished and measured with the Mind Wandering-Spontaneous (MW-S) and the Mind Wandering-Deliberate (MW-D) scales. In this study we tested whether the psychometric properties of Italian versions of these scales replicated across two common administration methods (paper-and-pencil and online survey). We also investigated their construct validity with respect to other self-report measures of MW and daydreaming, and measures of attentional control (AC). These measures were completed by 123 psychology students using the paper-and-pencil versions and by 165 online participants. The factor structure of the MW scales and their correlations with the other measures were replicated almost perfectly across administration methods. A confirmatory factor analysis (CFA) performed on data from 270 community participants further supported the replicability of the factor structure of the MW and AC scales. Albeit moderately correlated, the MW-S and the MW-D scales showed discriminant validity, since the former had significantly higher correlations with the other MW and AC measures, but not with daydreaming. These findings further supported the distinctiveness of the two forms of MW and suggested that in correlational studies the administration method is unlikely to affect results.

An examination of the equivalency of self-report measures obtained from crowdsourced versus undergraduate student samples

Increasingly, researchers have begun to explore the potential of the Internet to reach beyond the traditional undergraduate sample. In the present study, we sought to compare the data obtained from a conventional undergraduate college-student sample to data collected via two online survey recruitment platforms. In order to examine whether the data sampled from the three populations were equivalent, we conducted a test of equivalency using inferential confidence intervals-an approach that differs from the more traditional null hypothesis significance testing. The results showed that the data obtained via the two online recruitment platforms, the Amazon Mechanical Turk crowdsourcing site and the virtual environment of Second Life, were statistically equivalent to the data obtained from the college sample, on the basis of means of standardized measures of psychological stress and sleep quality. Additionally, correlations between the sleep and stress measures were not statistically different between the groups. These results, along with practical considerations for the use of these recruitment platforms, are discussed, and recommendations for other researchers who may be considering the use of these platforms are provided.

Software version of Roland Morris Disability questionnaire for outcome assessment in low back pain

BACKGROUND

Developing and using the software version of existing validated paper version of patient-related outcome can go a long way in saving cost, time and effort. However, the equivalence of paper version and software versions cannot be assumed. The aim of the study is to test the equivalence between paper version and software version of Roland Morris Disability Questionnaire and its acceptability among patients.

METHODS

This is a within-subject cross over equivalence study. Fifty-five patients with back pain were asked to complete the paper and software version of RMDQ in random order. Patients were included from the Neuro Spinal surgery outpatient department of Lilavati Hospital and Research Center.

RESULTS

Statistical analysis of 52 patients who completed the study showed high agreement between the paper and software version of the questionnaire (intraclass correlation coefficient 0.994, 95% confidence interval (0.989-0.996)). High sensitivity and specificity of 84 and 88% of the software version was noted. About 69.2% patients preferred software version over paper version.

CONCLUSION

Our study shows that software version is comparable to the paper version. It may prove to be a useful tool for epidemiological studies and patient follow-up over longer period.

Comparing Two Short Forms of the Hewitt-Flett Multidimensional Perfectionism Scale

Hewitt and Flett's 45-item Multidimensional Perfectionism Scale is a widely used instrument to assess self-oriented, other-oriented, and socially prescribed perfectionism. With 45 items, it is not overly lengthy, but there are situations where a short form is useful. Analyzing data from four samples, this article compares two frequently used 15-item short forms of the Multidimensional Perfectionism Scale-Cox et al.'s and Hewitt et al.'s-by examining to what degree their scores replicate the original version's correlations with various personality characteristics (e.g., traits, social goals, personal/interpersonal orientations). Regarding self-oriented and socially prescribed perfectionism, both short forms performed well. Regarding other-oriented perfectionism, however, Cox et al.'s short form (exclusively composed of negatively worded items) performed less well than Hewitt et al.'s (which contains no negatively worded items). It is recommended that researchers use Hewitt et al.'s short form to assess other-oriented perfectionism rather than Cox et al.'s.

Internal consistency and measurement equivalence of the cannabis screening questions on the paper-and-pencil face-to-face ASSIST versus the online instrument

Background

Validated Internet-based screening tools for cannabis use and abuse are needed. The present study aimed to establish equivalence between the previously validated Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) as a paper-and-pencil (PaP)-administered questionnaire and its online use.

Methods

Two groups of cannabis users took part in this study and the results were analyzed using structural equation modeling. One group consisted of 150 participants and was assessed with the ASSIST PaP questionnaire in a face-to-face interview (the PaP group). They were recruited from three settings: a primary health care outpatient clinic, a general psychiatric facility, and an ambulatory specialized addiction treatment facility. The other group (the Web group) comprised 1382 persons who answered the online version of the same questionnaire. This sample was drawn from people who naturalistically visited a website dedicated to helping people with cannabis addiction.

Results

The internal consistency was good for the online questionnaire (0.74) and high for the already validated PaP questionnaire (0.91). The Web group, however, had higher scores on cannabis use than did the PaP group. The results show support for configural invariance, meaning that the one-factor structure was preserved across groups, although measurement equivalence between these two survey modes was not achieved. However, when the Web group was split into two random subsamples, measurement invariance was demonstrated between them by cross-validation.

Conclusions

Measurement equivalence was not achieved between the two survey modes. Nonetheless, subanalyses of the Web group demonstrated that the cannabis screening questions of the ASSIST can be used for online screening. Differences in ASSIST scores between samples may be due to the sensitive nature of the information surveyed, with possible underreporting in face-to-face interviews, or to the different characteristics of the Web group because of the specialized nature of the website.

Examination of the factor structure of the Schizotypal Personality Questionnaire among British and Trinidadian adults

Much debate in schizotypal research has centred on the factor structure of the Schizotypal Personality Questionnaire (SPQ), with research variously showing higher-order dimensionality consisting of two to seven dimensions. In addition, cross-cultural support for the stability of those factors remains limited. Here, we examined the factor structure of the SPQ among British and Trinidadian adults. Participants from a White British subsample (n = 351) resident in the UK and from an African Caribbean subsample (n = 284) resident in Trinidad completed the SPQ. The higher-order factor structure of the SPQ was analysed through confirmatory factor analysis, followed by multiple-group analysis for the model of best fit. Between-group differences for sex and ethnicity were investigated using multivariate analysis of variance in relation to the higher-order domains. The model of best-fit was the four-factor structure, which demonstrated measurement invariance across groups. Additionally, these data had an adequate fit for two alternative models: (a) 3-factor and (b) modified 4-factor model. The British subsample had significantly higher scores across all domains than the Trinidadian group, and men scored significantly higher on the disorganised domain than women. The four-factor structure received confirmatory support and, importantly, support for use with populations varying in ethnicity and culture.

Evaluation of magical thinking: validation of the Illusory Beliefs Inventory

Magical thinking has been related to obsessive-compulsive disorder; yet, little research has examined this construct in other anxiety disorders. The Illusory Beliefs Inventory (IBI) is a recently developed measure of magical thinking. The aim of this study was to investigate the psychometric properties of this new measure and to determine if magical thinking accounts for pathological worry beyond the well-researched constructs of intolerance of uncertainty (IU) and perfectionism. A sample of 502 participants completed an online survey. Confirmatory factor analysis identified a three-factor solution for the IBI, and the measure had good internal consistency (α = .92), test-retest reliability (r = .94) and discriminant validity. Magical thinking, IU, and perfectionism all predicted pathological worry; however, magical thinking accounted for less than 1% of unique variance in worry, suggesting that it is not strongly related to worry. Further investigation regarding the validity and clinical utility of the IBI is required.

Does assessment type matter? A measurement invariance analysis of online and paper and pencil assessment of the Community Assessment of Psychic Experiences (CAPE)

Background

The psychometric properties of an online test are not necessarily identical to its paper and pencil original. The aim of this study is to test whether the factor structure of the Community Assessment of Psychic Experiences (CAPE) is measurement invariant with respect to online vs. paper and pencil assessment.

Method

The factor structure of CAPE items assessed by paper and pencil (N = 796) was compared with the factor structure of CAPE items assessed by the Internet (N = 21,590) using formal tests for Measurement Invariance (MI). The effect size was calculated by estimating the Signed Item Difference in the Sample (SIDS) index and the Signed Test Difference in the Sample (STDS) for a hypothetical subject who scores 2 standard deviations above average on the latent dimensions.

Results

The more restricted Metric Invariance model showed a significantly worse fit compared to the less restricted Configural Invariance model (χ²(23) = 152.75, p<0.001). However, the SIDS indices appear to be small, with an average of −0.11. A STDS of −4.80 indicates that Internet sample members who score 2 standard deviations above average would be expected to score 4.80 points lower on the CAPE total scale (ranging from 42 to 114 points) than would members of the Paper sample with the same latent trait score.

Conclusions

Our findings did not support measurement invariance with respect to assessment method. Because of the small effect sizes, the measurement differences between the online assessed CAPE and its paper and pencil original can be neglected without major consequences for research purposes. However, a person with a high vulnerability for psychotic symptoms would score 4.80 points lower on the total scale if the CAPE is assessed online compared to paper and pencil assessment. Therefore, for clinical purposes, one should be cautious with online assessment of the CAPE.

A qualitative review of the psychometric properties and feasibility of electronic headache diaries for children and adults: where we are and where we need to go

BACKGROUND

While paper headache pain diaries have been used to determine the effectiveness of headache treatments in clinical trials, recent advances in information and communication technologies have resulted in the burgeoning use of electronic diaries (e-diaries) for headache pain.

OBJECTIVE

To qualitatively review headache e-diaries, assess their measurement properties, examine measurement components and compare these components with recommended reporting guidelines.

METHODS

The databases Medline, the Cumulative Index to Nursing and Allied Health Literature, Embase, PsychInfo, the Education Resources Information Centre and ISI Web of Science were searched for self-report headache e-diaries for children and adults. A total of 21 publications that involved e-diaries were found; five articles reported on the development of an e-diary and 16 used an e-diary as an outcome measure in randomized controlled trials or observational studies. The diary measures' components, features and psychometric properties, as well as the quality of evidence of their psychometric properties, were evaluated.

RESULTS

Five headache e-diaries met the a priori criteria and were included in the final analysis. None of these e-diaries had well-developed evidence of reliability and validity. Three e-diaries showed evidence of feasibility. E-diaries with ad hoc measures developed by the study investigators were most common, with little to no supportive evidence of reliability and⁄or validity. Compliance with the reporting guidelines was variable, with only one-half of the e-diaries measuring the recommended primary outcome of headache frequency.

CONCLUSIONS

Specific recommendations regarding the development (including essential components) and testing of headache e-diaries are discussed. Further research is needed to strengthen the measurement of headache pain in clinical trials using headache e-diaries.

Can we trust the internet to measure psychotic symptoms?

Online studies are increasingly utilized in applied research. However, lack of external diagnostic verification in many of these investigations is seen as a threat to the reliability of the data. The present study examined the robustness of internet studies on psychosis against simulation. We compared the psychometric properties of the Community Assessment of Psychic Experiences scale (CAPE), a self-report instrument measuring psychotic symptoms, across three independent samples: (1) participants with a confirmed diagnosis of schizophrenia, (2) participants with self-reported schizophrenia who were recruited over the internet, and (3) clinical experts on schizophrenia as well as students who were asked to simulate a person with schizophrenia when completing the CAPE. The CAPE was complemented by a newly developed 4-item psychosis lie scale. Results demonstrate that experts asked to simulate schizophrenia symptoms could be distinguished from real patients: simulators overreported positive symptoms and showed elevated scores on the psychosis lie scale. The present study suggests that simulated answers in online studies on psychosis can be distinguished from authentic responses. Researchers conducting clinical online studies are advised to adopt a number of methodological precautions and to compare the psychometric properties of online studies to established clinical indices to assert the validity of their results.

Assessing public speaking fear with the short form of the Personal Report of Confidence as a Speaker scale: confirmatory factor analyses among a French-speaking community sample

BACKGROUND

The main aim of this study was to assess the reliability and structural validity of the French version of the 12-item version of the Personal Report of Confidence as Speaker (PRCS), one of the most promising measurements of public speaking fear.

METHODS

A total of 611 French-speaking volunteers were administered the French versions of the short PRCS, the Liebowitz Social Anxiety Scale, the Fear of Negative Evaluation scale, as well as the Trait version of the Spielberger State-Trait Anxiety Inventory and the Beck Depression Inventory-II, which assess the level of anxious and depressive symptoms, respectively.

RESULTS

Regarding its structural validity, confirmatory factor analyses indicated a single-factor solution, as implied by the original version. Good scale reliability (Cronbach's alpha = 0.86) was observed. The item discrimination analysis suggested that all the items contribute to the overall scale score reliability. The French version of the short PRCS showed significant correlations with the Liebowitz Social Anxiety Scale (r = 0.522), the Fear of Negative Evaluation scale (r = 0.414), the Spielberger State-Trait Anxiety Inventory (r = 0.516), and the Beck Depression Inventory-II (r = 0.361).

CONCLUSION

The French version of the short PRCS is a reliable and valid measure for the evaluation of the fear of public speaking among a French-speaking sample. These findings have critical consequences for the measurement of psychological and pharmacological treatment effectiveness in public speaking fear among a French-speaking sample.

The online version of the Dutch Penn State Worry Questionnaire: factor structure, predictive validity and reliability

The Penn State Worry Questionnaire (PSWQ) is a widely used self report scale to assess pathological worry. Several studies have shown that the paper-and-pencil version of the PSWQ shows satisfactory psychometric properties. Yet, it is unknown whether the PSWQ is suited to be administered on Internet. As the amount of assessments conducted online is vastly increasing, we examined the factor structure, predictive validity and reliability of the online version of the PSWQ in a large community sample (N = 1025). Confirmatory factor analyses demonstrated that the best fit to the data was provided by a one factor model wherein the reverse-worded items also loaded on a separate method factor. In a second part of the study a self-selected subsample of the total sample (n = 189) kept a log of the frequency and duration of worry episodes for six consecutive days. Worry frequency and worry duration were both predicted by the PSWQ, thereby lending support to its predictive validity. These findings provide preliminary evidence that the online version of the PSWQ possesses psychometric properties that are similar to the paper-and-pencil version.

Developing a standardized approach to the assessment of pain in children and youth presenting to pediatric rheumatology providers: a Delphi survey and consensus conference process followed by feasibility testing

BACKGROUND

Pain in children with rheumatic conditions such as arthritis is common. However, there is currently no standardized method for the assessment of this pain in children presenting to pediatric rheumatologists. A more consistent and comprehensive approach is needed to effectively assess, treat and monitor pain outcomes in the pediatric rheumatology population. The objectives of this study were to: (a) develop consensus regarding a standardized pain assessment tool for use in pediatric rheumatology practice and (b) test the feasibility of three mediums (paper, laptop, and handheld-based applications) for administration.

METHODS

In Phase 1, a 2-stage Delphi technique (pediatric rheumatologists and allied professionals) and consensus meeting (pediatric pain and rheumatology experts) were used to develop the self- and proxy-report pain measures. In Phase 2, 24 children aged 4-7 years (and their parents), and 77 youth, aged 8-18 years, with pain, were recruited during routine rheumatology clinic appointments and completed the pain measure using each medium (order randomly assigned). The participant's rheumatologist received a summary report prior to clinical assessment. Satisfaction surveys were completed by all participants. Descriptive statistics were used to describe the participant characteristics using means and standard deviations (for continuous variables) and frequencies and proportions (for categorical variables)

RESULTS

Completing the measure using the handheld device took significantly longer for youth (M = 5.90 minutes) and parents (M = 7.00 minutes) compared to paper (M = 3.08 and 2.28 minutes respectively p = 0.001) and computer (M = 3.40 and 4.00 minutes respectively; p < 0.001). There was no difference in the number of missed responses between mediums for children or parents. For youth, the number of missed responses varied across mediums (p = 0.047) with the greatest number of missed responses occurring with the handheld device. Most children preferred the computer (65%, p = 0.008) and youth reported no preference between mediums (p = 0.307). Most physicians (60%) would recommend the computer summary over the paper questionnaire to a colleague.

CONCLUSIONS

It is clinically feasible to implement a newly developed consensus-driven pain measure in pediatric rheumatology clinics using electronic or paper administration. Computer-based administration was most efficient for most users, but the medium employed in practice may depend on child age and economic and administrative factors.

Concurrent validity of an online version of the Timeline Followback assessment

The Timeline Followback (TLFB) interview has been used extensively in the assessment of alcohol and other substance use. While this methodology has been validated in multiple formats for multiple behaviors, to date no systematic comparisons have been conducted between the traditional interview format and online versions. The present research employed a randomized within-subjects design to compare interview versus online-based TLFB assessments of alcohol and marijuana use among 102 college students. Participants were randomly assigned to receive either the online version first or the in-person interview format first. Participants subsequently completed the second format within 3 days. While we expected few overall differences between formats, we hypothesized that differences might emerge to the extent that participants are more comfortable and willing to answer honestly in an online format, which provides a degree of anonymity. Results were consistent with expectations in suggesting relatively few differences between the online version and the in-person version. Participants did report feeling more comfortable in completing the online version. Moreover, greater discomfort during the in-person assessment was associated with reporting more past-month marijuana use on the online assessment, but reported discomfort did not moderate differences between formats in reported alcohol consumption.

Maternal smoking cessation and reduced academic and behavioral problems in offspring

BACKGROUND

There is some debate whether smoking during pregnancy causes or is only a risk factor for negative academic outcomes and increased risk of psychopathology in offspring. This study evaluated whether maternal smoking cessation would reduce the risk of adverse outcomes in school-aged children.

METHODS

Women completed an online survey that included items about child scholastic performance and the Child Behavior Checklist (CBCL). Mothers were divided based on pre-pregnancy and pregnancy smoking status into: (1) Nonsmokers (N=320); (2) Women that smoked in the 3 months preceding and throughout pregnancy (Smokers, N=83); and (3) Smoking before, but not during pregnancy (Quitters, N=72).

RESULTS

The Smokers and Quitters groups each had lower education and incomes compared to Nonsmokers but were indistinguishable from each other on these measures. The offspring of Smokers were more likely (p<.05) to be behind their peers on standardized tests in math (27.8%) relative to both Nonsmokers (17.4%) and Quitters (13.0%) with similar findings for reading. Smokers reported more behavioral problems by their children in several areas including Hyperactivity and Impulsivity, Social problems, and Externalizing problems including Aggression and Rule-Breaking. Further, the children of Quitters had significantly fewer Attention and Externalizing problems than Smokers. These outcomes were observed even after accounting for the variance attributable to maternal education and several other potential confounds.

CONCLUSIONS

Together, these findings indicate that smoking cessation is associated with reduced risk of having children with academic and neuropsychological difficulties. These outcomes are discussed within the framework that nicotine may be a neurobehavioral teratogen.

The impact of stressors on second generation Indian Residential School survivors

From 1863 to 1996, many Aboriginal children in Canada were forced to attend Indian Residential Schools (IRSs), where many experienced neglect, abuse, and the trauma of separation from their families and culture. The present study examined the intergenerational impact of IRS exposure on depressive symptomatology in a convenience sample of 143 First Nations adults. IRS experiences had adverse intergenerational effects in that First Nations adults who had a parent attend IRS (n = 67) reported greater depressive symptoms compared to individuals whose parents did not attend (n = 76). Parental IRS attendance moderated the relations between stressor experiences (adverse childhood experiences, adult traumas, and perceived discrimination) and depressive symptoms, such that second generation Survivors exhibited greater symptomatology. Adverse childhood experiences partially mediated the relation between parental IRS attendance and both adult trauma and perceived discrimination. Moreover, both of these adulthood stressors partially mediated the relation between adverse childhood experiences and depressive symptoms. Finally, all three stressors demonstrated a unique mediating role in the relation between parental IRS attendance and depressive symptoms. Although alternative directional paths could not be ruled out, offspring of IRS Survivors appeared at increased risk for depression, likely owing to greater sensitivity to and experiences of childhood adversity, adult traumas, and perceived discrimination.

Learning to live well with chronic fatigue: the personal perspective

BACKGROUND

Persistent fatigue is recognized as an integral and significantly disabling aspect of the experience of living with a long-term health condition. Acute medical care models have limited applicability when seeking to provide health care to meet the needs of those living with chronic fatigue.

AIMS

This article is a report of a study that sought to understand how people can live well in spite of the presence of chronic fatigue.

METHODS

A thematic analysis was conducted on 43 narratives provided by people living with chronic fatigue during 2007 as part of an internet-based study.

FINDINGS

This report focuses on the two themes that appeared of most relevance to participants: managing energy and redefining self. Two particular approaches to energy management were noted. In the first, the focus is upon moderating and avoiding excess to manage energy. In the second approach, the body was conceptualized as a machine so that energy becomes a limited resource to be managed. Work to redefine the sense of self appeared to be focused upon the desire to seek normality, to see oneself as rational and come to terms with change.

CONCLUSION

People can and do find ways to live well with chronic fatigue. Understanding how the person with chronic fatigue has come to conceptualize his/her experiences will be a more fruitful starting point than providing recipes for successful living if nurses are to work effectively with this group of people.

The effects of an Internet based self-help course for reducing panic symptoms--Don't Panic Online: study protocol for a randomised controlled trial

Background

Internet based self-help for panic disorder (PD) has proven to be effective. However, studies so far have focussed on treating a full-blown disorder. Panic symptoms that do not meet DSM-IV criteria are more prevalent than the full-blown disorder and patients with sub-clinical panic symptoms are at risk of developing PD. This study is a randomised controlled trial aimed to evaluate an Internet based self-help intervention for sub-clinical and mild PD compared to a waiting list control group.

Methods

Participants with mild or sub-clinical PD (N = 128) will be recruited in the general population. Severity of panic and anxiety symptoms are the primary outcome measures. Secondary outcomes include depressive symptoms, quality of life, loss of production and health care consumption. Assessments will take place on the Internet at baseline and three months after baseline.

Discussion

Results will indicate the effectiveness of Internet based self-help for sub-clinical and mild PD. Strengths of this design are the external validity and the fact that it is almost completely conducted online.

Trial registration

Netherlands Trial Register (NTR): NTR1639 The Netherlands Trial Register is part of the Dutch Cochrane Centre.

Using the Internet to prescreen participants for research on interpersonal violence: experimental design considerations

Internet-based testing has become increasingly popular in recent years due largely to the availability of computers and the Web to research participants. For researchers studying interpersonal violence, Internet-based methodologies can also be used as a prescreening device to identify a population of interest, such as individuals reporting violence in their relationships. However, several challenges exist with this approach. This article evaluates the use of an Internet-based prescreening device to identify potential participants for a study on interpersonal violence, highlighting challenges encountered with this methodology, and offers suggestions for improvement in experimental design when conducting Internet-based screenings of interpersonal violence.

Smoking and everyday prospective memory: a comparison of self-report and objective methodologies

AIMS

To examine whether persistent smoking leads to impairments in self-reported and objective measures of prospective memory (PM: the cognitive ability to remember to carry out activities at some future point in time).

METHODS

An opportunity sample of 18 existing smokers and 22 who had never smoked were compared. An existing-groups design was utilised, comparing a smoking group with a never-smoked control group as the independent factor. Scores on the sub-scales of the Prospective and Retrospective Memory Questionnaire (PRMQ) and scores on the Cambridge Prospective Memory Test (CAMPROMPT) constituted the dependent factors. Age, mood, other drug use, strategy scores and IQ were also measured. Each participant was tested in a laboratory setting. Self-reported PM lapses were measured using the PRMQ. The CAMPROMPT was used as an objective measure of PM. Alcohol and other drug use were assessed by a Recreational Drug Use Questionnaire. The Hospital Anxiety and Depression Scale gauged levels of anxiety and depression. A strategy scale measured the number of strategies used to aid memory. The National Adult Reading Test measured IQ.

RESULTS

After observing no between-group differences on age, mood, alcohol use, strategy use, and IQ, smokers and the never-smoked did not differ on the self-reported lapses measured on the PRMQ. However, smokers recalled significantly fewer items on the CAMPROMPT than the never-smoked group.

CONCLUSION

The results of the present study suggest that persistent smoking leads to impairments in everyday PM.

Avaliação de escrita na dislexia do desenvolvimento: tipos de erros ortográficos em prova de nomeação de figuras por escrita

OBJETIVO: avaliar o padrão de resposta de disléxicos em uma tarefa de nomeação de figuras por escrita, por meio da análise dos tipos de erros ortográficos cometidos. MÉTODOS: o desempenho de um grupo de 15 disléxicos foi comparado ao de dois grupos controles, pareados por idade e por nível de leitura. RESULTADOS: os grupos dislexia e controle por leitura não diferiram quanto ao número de acertos, mas ambos acertaram menos que o grupo controle por idade. Com relação aos tipos de erros, foram observadas diferenças significantes com maior número de erros entre disléxicos para erros de correspondência unívoca grafema-fonema, omissão de segmentos e correspondência fonema-grafema independente de regras. CONCLUSÃO: conclui-se que a análise dos erros ortográficos é útil para a compreensão das estratégias utilizadas e dos processos lingüísticos subjacentes às dificuldades de escrita em indivíduos com dislexia.

The Dutch version of the Behavioral Activation for Depression Scale (BADS): psychometric properties and factor structure

In a community sample of 402 respondents, the authors examined the psychometric properties of the Dutch version of the Behavioral Activation for Depression Scale (BADS; Kanter, Mulick, Busch, Berlin, & Martell, 2006). Results of confirmatory factor analysis supported the original 4-factor structure, providing the following subscales: Activation, Avoidance/Rumination, Work/School Impairment, and Social Impairment. All subscales showed adequate internal consistency, evidence of convergent validity with concurrent measures of depressive symptoms, rumination, psychological flexibility, and avoidance behavior, and evidence of differential validity between currently, formerly, and never depressed respondents.

A within-subjects trial to test the equivalence of online and paper outcome measures: the Roland Morris disability questionnaire

BACKGROUND

Augmenting validated paper versions of existing outcome measures with an equivalent online version may offer substantial research advantages (cost, rapidity and reliability). However, equivalence of online and paper questionnaires cannot be assumed, nor can acceptability to respondents. The aim was to test whether online and written versions of the Roland Morris Disability Questionnaire (RMDQ), a standard measure of functional disability in back pain, are equivalent at both group and individual levels to establish whether they can be used interchangeably.

METHODS

This is a within-participants equivalence study. 167 participants with back pain fully completed both the paper and online versions of the RMDQ in random order. Participants were recruited from a chiropractic clinic and patient support groups in Southern England. Limits of equivalence were pre-defined as 0.5 RMDQ points, the Bland-Altman range was calculated, and participants' comments were examined using content analysis.

RESULTS

The mean score difference was 0.03 (SD = 1.43), with the 95% Confidence Interval falling entirely within our limits of equivalence (-0.19 to 0.25). The Bland-Altman range was -2.77 to 2.83 RMDQ points. Participants identified unique advantages and disadvantages associated with each version of the RMDQ.

CONCLUSIONS

The group and individual level data suggest that online and paper versions of the RMDQ are equivalent and can be used interchangeably. The Bland-Altman range appears to reflect the known measurement properties of the RMDQ. Furthermore, participants' comments confirmed the potential value to be had from offering them the choice of completing the RMDQ online or on paper.

Does cannabis use affect prospective memory in young adults?

The aim of the present study was to examine prospective memory impairments associated with cannabis use in young adults. An independent measures design utilising pre-existing groups of users and non-users was employed in which an opportunity sample of 90 undergraduates studying at universities in the north east of England participated. The number of prospective memory failures reported on the Prospective Memory Questionnaire and the number of location-action combinations correctly recalled during a video-based prospective memory task were measured. The number of strategies used to assist memory, level of anxiety and depression, and use of alcohol, nicotine and any other recreational drugs in addition to cannabis were also measured and controlled during the analysis. Analysis revealed no significant differences in the number of self-reported prospective memory failures; however, cannabis users recalled significantly fewer location-action combinations than non-users in the video-based prospective memory task. The findings from the present study suggest that cannabis use has a detrimental effect on prospective memory ability in young adults but users may not be aware of these deficits.

Psychometric evaluation of the Revised Professional Practice Environment (RPPE) scale

OBJECTIVE

The purpose was to examine the psychometric properties of the Revised Professional Practice Environment (RPPE) scale.

BACKGROUND

Despite renewed focus on studying health professionals' practice environments, there are still few reliable and valid instruments available to assist nurse administrators in decision making.

METHODS

A psychometric evaluation using a random-sample cross-validation procedure (calibration sample [CS], n = 775; validation sample [VS], n = 775) was undertaken.

RESULTS

Cronbach alpha internal consistency reliability of the total score (r = 0.93 [CS] and 0.92 [VS]), resulting subscale scores (r range: 0.80-0.87 [CS], 0.81-0.88 [VS]), and principal components analyses with Varimax rotation and Kaiser normalization (8 components, 59.2% variance [CS], 59.7% [VS]) produced almost identical results in both samples.

CONCLUSIONS

The multidimensional RPPE is a psychometrically sound measure of 8 components of the professional practice environment in the acute care setting and sufficiently reliable and valid for use as independent subscales in healthcare research.

Methodological challenges in online trials

Health care and health care services are increasingly being delivered over the Internet. There is a strong argument that interventions delivered online should also be evaluated online to maximize the trial's external validity. Conducting a trial online can help reduce research costs and improve some aspects of internal validity. To date, there are relatively few trials of health interventions that have been conducted entirely online. In this paper we describe the major methodological issues that arise in trials (recruitment, randomization, fidelity of the intervention, retention, and data quality), consider how the online context affects these issues, and use our experience of one online trial evaluating an intervention to help hazardous drinkers drink less (DownYourDrink) to illustrate potential solutions. Further work is needed to develop online trial methodology.