The dependence of kappa on attribute prevalence when assessing the repeatability of questionnaire data

A Survey Bias Index Based on Unmanned Aerial Vehicle Imagery to Review the Accuracy of Rural Surveys

Field surveys and questionnaires are a cornerstone of rural socioeconomic research, providing invaluable firsthand data regarding on-the-ground situations. However, cost-effective and efficient methods for validating the accuracy of self-reported data in such questionnaires are lacking. Biased data are likely to lead to incorrect conclusions. In this study, we propose a new index, the survey bias index (SBI), for evaluating the degree of survey bias in field surveys. This index was obtained by comparing the data recorded in questionnaires with those from portable unmanned aerial vehicles (UAVs). In a case study, we employed SBI to reveal the degree of survey bias of questionnaires in field surveys on rural homesteads. The SBI of self-reported areas of rural homesteads reached 0.439, implying that 43.9% of data were significantly different from those collected using UAVs. A greater SBI was obtained in the pre-urban zone (0.515) than in the pure rural zone (0.258). These results indicate that homestead areas in the pre-urban zone have more incentive to expand than those in the pure rural zone. UAV remote sensing can strongly support research in the field of social economy, which reveals key information hidden in field surveys and questionnaires.

Inter-observer agreement of canine and feline paroxysmal event semiology and classification by veterinary neurology specialists and non-specialists

Background

Advances in mobile technology mean vets are now commonly presented with videos of paroxysmal events by clients, but the consistency of the interpretation of these videos has not been investigated. The objective of this study was to investigate the level of agreement between vets (both neurology specialists and non-specialists) on the description and classification of videos depicting paroxysmal events, without knowing any results of diagnostic workup. An online questionnaire study was conducted, where participants watched 100 videos of dogs and cats exhibiting paroxysmal events and answered questions regarding: epileptic seizure presence (yes/no), seizure type, consciousness status, and the presence of motor, autonomic and neurobehavioural signs. Agreement statistics (percentage agreement and kappa) calculated for each variable, with prevalence indices calculated to aid their interpretation.

Results

Only a fair level of agreement (κ = 0.40) was found for epileptic seizure presence. Overall agreement of seizure type was moderate (κ = 0.44), with primary generalised seizures showing the highest level of agreement (κ = 0.60), and focal the lowest (κ =0.31). Fair agreement was found for consciousness status and the presence of autonomic signs (κ = 0.21–0.40), but poor agreement for neurobehavioral signs (κ = 0.16). Agreement for motor signs ranged from poor (κ = ≤ 0.20) to moderate (κ = 0.41–0.60). Differences between specialists and non-specialists were identified.

Conclusions

The relatively low levels of agreement described here highlight the need for further discussions between neurology experts regarding classifying and describing epileptic seizures, and additional training of non-specialists to facilitate accurate diagnosis. There is a need for diagnostic tools (e.g. electroencephalogram) able to differentiate between epileptic and non-epileptic paroxysms.

Electronic supplementary material

The online version of this article (doi:10.1186/s12917-015-0356-2) contains supplementary material, which is available to authorized users.

Parent-reported ISAAC written questionnaire may underestimate the prevalence of asthma in children aged 10-12 years

OBJECTIVES

The aim of the present study was to compare the validity of the International Study of Asthma and Allergies in Childhood (ISAAC) written (WQ) and audiovisual questionnaires (AVQ 3.0) in two age-groups (10-12 and 13-15 years, respectively).

METHODS

The 13-15 year olds performed the self-completed the WQ and AVQ on the same day. The 10-12 year olds performed the self-completed the AVQ and the parent-completed WQ was completed by their parents. The methacholine challenge test was conducted in 10-12 year olds from one elementary school.

RESULTS

In 10-12 year olds, the AVQ detected a generally higher prevalence of asthma symptoms than WQ. In 13-15 year olds, this was reversed. In 10-12 year olds, poor agreement was found between the parent-completed WQ and the self-reported AVQ. In 13-15 year olds, moderate agreement was found between the self-reported WQ and AVQ. Low sensitivity was found, in predicting bronchial hyper-responsiveness (BHR) for all questions of both WQ and AVQ in 10-12 year olds. However, the AVQ had slightly higher sensitivity than WQ, with the exception of wheeze ever, although it was not statistically significant.

CONCLUSION

The ISAAC AVQ may be another effective instrument for assessing the prevalence of asthma symptoms in children aged 10-12 years, whereas the parent-reported-WQ may underestimate the prevalence of asthma symptoms in this age-group.

Development and validation of a reverse transcription quantitative PCR for universal detection of viral hemorrhagic septicemia virus

Viral hemorrhagic septicemia virus (VHSV) infects over 70 fish species inhabiting marine, brackish or freshwater environments throughout the Northern Hemisphere. Over its geographic range, 4 VHSV genotypes and multiple subtypes exist. Here, we describe the development and validation of a rapid, sensitive and specific real-time reverse transcription quantitative PCR assay (RT-qPCR) that amplifies sequence from representative isolates of all VHSV genotypes (I, II, III and IV). The pan-specific VHSV RT-qPCR assay reliably detects 100 copies of VHSV nucleoprotein RNA without cross-reacting with infectious hematopoietic necrosis virus, spring viremia of carp virus or aquatic birnavirus. Test performance characteristics evaluated on experimentally infected Atlantic salmon Salmo salar L. revealed a diagnostic sensitivity (DSe) > or = 93% and specificity (DSp) = 100%. The repeatability and reproducibility of the procedure was exceptionally high, with 93% agreement among test results within and between 2 laboratories. Furthermore, proficiency testing demonstrated the VHSV RT-qPCR assay to be easily transferred to and performed by a total of 9 technicians representing 4 laboratories in 2 countries. The assay performed equivalent to the traditional detection method of virus isolation via cell culture with the advantage of faster turnaround times and high throughput capacity, further suggesting the suitability of the use of this VHSV RT-qPCR in a diagnostic setting.

Quantitative epidemiology: progress and challenges

This manuscript is derived from a presentation at the 2006 AVEPM--Schwabe Symposium which honoured the 2006 recipient of the Calvin Schwabe Award--Dr. S. Wayne Martin. Throughout his career, Dr. Martin was instrumental in furthering the development of quantitative epidemiology. This manuscript highlights some of the recent advances in quantitative methods used in veterinary epidemiology and identifies some of the challenges we still face in the use of these methods. Areas discussed are: methods for analyzing hierarchical data, Bayesian methods, survival analysis and integrating causal thinking and statistical analyses.

Intracluster correlation coefficients from the 2005 WHO Global Survey on Maternal and Perinatal Health: implications for implementation research

Cluster-based studies involving aggregate units such as hospitals or medical practices are increasingly being used in healthcare evaluation. An important characteristic of such studies is the presence of intracluster correlation, typically quantified by the intracluster correlation coefficient (ICC). Sample size calculations for cluster-based studies need to account for the ICC, or risk underestimating the sample size required to yield the desired levels of power and significance. In this article, we present values for ICCs that were obtained from data on 97,095 pregnancies and 98,072 births taking place in a representative sample of 120 hospitals in eight Latin American countries. We present ICCs for 86 variables measured on mothers and newborns from pregnancy to the time of hospital discharge, including 'process variables' representing actual medical care received for each mother and newborn. Process variables are of primary interest in the field of implementation research. We found that overall, ICCs ranged from a minimum of 0.0003 to a maximum of 0.563 (median 0.067). For maternal and newborn outcome variables, the median ICCs were 0.011 (interquartile range 0.007-0.037) and 0.054 (interquartile range 0.013-0.075) respectively; however, for process variables, the median was 0.161 (interquartile range 0.072-0.328). Thus, we confirm previous findings that process variables tend to have higher ICCs than outcome variables. We demonstrate that ICCs generally tend to increase with higher prevalences (close to 0.5). These results can help researchers calculate the required sample size for future research studies in maternal and perinatal health.

Comparison of rapid enzyme-linked immunosorbent assay and immunomagnetic separation methods for detection of Escherichia coli O157 in fecal, hide, carcass, and ground beef samples

Rapid enzyme-linked immunosorbent assays (ELISAs) are approved for detection of Escherichia coil O157 in beef products. However, these kits have also been used in the industry to detect this pathogen on hides or in feces of cattle, although this use has not been validated. The objective of this study was to compare commercially available ELISAs (E. coli Now, Reveal, and VIP) with immunomagnetic separation along with selective media to detect E. coli O157 on hides, in feces, and in medium- and low-level-inoculated ground beef and carcasses (simulated by using briskets) samples. Naturally infected hide and fecal samples were subjected to both the immunomagnetic separation method and ELISAs for the detection of E. coli O157. Additionally, E. coli O157 inoculated and noninoculated ground beef and beef briskets were used to simulate meat and carcass samples. When comparing the detection results from the ELISAs (E. coli Now, Reveal, and VIP) to the immunomagnetic separation method, poor agreement was observed for fecal samples (kappa = 0.10, 0.02, and 0.03 for E. coli Now, Reveal, and VIP, respectively), and fair-to-moderate agreement was observed for hide samples (kappa = 0.30, 0.51, and 0.29 for E. coli Now, Reveal, and VIP, respectively). However, there was near-perfect agreement between the immunomagnetic separation method and ELISAs for ground beef (kappa = 1, 1, and 0.80 for E. coli Now, Reveal, and VIP, respectively) and brisket (kappa = 1, 1, and 1 for E. coli Now, Reveal, and VIP, respectively) samples. Assuming immunomagnetic separation is the best available method, these data suggest that the ELISAs are not useful in detecting E. coli O157 from hide or fecal samples. However, when ELISAs are used on ground beef and beef brisket samples they can be used with a high degree of confidence.

Naming and recognition of six foot lesions of sheep using written and pictorial information: a study of 809 English sheep farmers

In 2005, 3000 questionnaires were sent to a random sample of English sheep farmers from a list kept by the English Beef and Lamb Executive (EBLEX) to investigate whether farmers could correctly name six common foot lesions in sheep from a characteristic picture and a written description. The lesions were interdigital dermatitis (ID), footrot (FR), contagious ovine digital dermatitis (CODD), shelly hoof, foot abscess and toe granuloma. In addition, farmers were asked to report the total percent of lame sheep in their flock in 2004 and the percent of this lameness attributable to each of the six lesions listed above. The overall response percentage was 44 with a useable response of 32%. Fifty-nine farmers out of 262 (23%) who answered all six questions named all six lesions correctly. This was greater than expected by chance. The same questionnaire of six lesions was presented at a meeting of specialist sheep advisors, primarily veterinarians, 37/47 (79%) responders named all six lesions correctly. From the six lesions listed above, the percent correctly named by farmers was approximately 83%, 85%, 36%, 28%, 65% and 43% and the percent incorrectly attributed to another lesion was 5%, 47%, 10%, 13%, 35% and 7%, respectively. The most commonly used incorrect name was FR, with farmers tending to name any hoof horn lesion as FR. A comparison of the distribution of sheep lame by a lesion correctly named compared with the same lesion incorrectly named as FR suggested that farmers recognised lesions but did not name them correctly; the distribution of lameness fitted the pattern for the correctly named lesion rather than the pattern of lameness attributed to FR. The results were validated with farm visits and a repeatability study of the questionnaire. The mean farmer-estimated prevalence for all lameness was 10.4%; with 6.9%, 3.7%, 2.4%, 1.9%, 0.9% and 0.8% of the sheep lame with ID, FR, CODD, shelly hoof, foot abscess and toe granuloma respectively from respondents who correctly named these lesions. Whilst ID and FR were the most prevalent causes of lameness in most flocks it is possible that in up to 17% flocks the primary cause of lameness was a different lesion.

Comparison of a new ESAT-6/CFP-10 peptide-based gamma interferon assay and a tuberculin skin test for tuberculosis screening in a moderate-risk population

Screening for latent tuberculosis infection (LTBI) with Mantoux tuberculin skin test (TST) has many limitations, including false-positive results due to exposure to Mycobacterium other than tuberculosis (TB) and BCG vaccination. A total of 474 adult inmates in a county jail were screened for LTBI using TST and a new ESAT-6/CFP-10 peptide-based whole-blood gamma interferon (IFN-gamma) assay. LTBI prevalence was 9.0 and 5.4% as determined by TST and IFN-gamma assay, respectively. Overall, agreement between test results was 90% (kappa = 0.25). Positive TST results were significantly associated with increased age (odds ratio [OR], 1.04; 95% confidence interval [CI], 1.01 to 1.08), African-American ethnicity (OR, 4.97; 95% CI, 1.58 to 15.68), foreign birth (OR, 20.20; 95% CI, 4.21 to 97.02) and prior incarceration (OR, 6.19; 95% CI, 1.48 to 25.95). Positive IFN-gamma assay results were significantly associated with African-American ethnicity (OR, 5.58; 95% CI, 1.16 to 26.74). Factors associated with statistically significant discordance between TST and IFN-gamma assay results were African-American ethnicity (OR, 0.29; 95% CI, 0.11 to 0.77), foreign birth (OR, 0.23; 95% CI, 0.07-0.80), and prior incarceration (OR, 0.06; 95% CI, 0.01-0.50). Among subjects born in the United States, African-American ethnicity was the only variable significantly associated with positive test results for both TST (OR, 4.26; 95% CI, 1.38 to 13.16) and IFN-gamma assay (OR, 5.74; 95% CI, 1.19 to 27.75) and remained associated with statistically significant discordance between TST and IFN-gamma assay results. The reactivity of the new IFN-gamma assay is unaffected by prior BCG vaccination or serial TSTs but may be diminished in African-Americans. Future longitudinal studies are needed to assess the sensitivity and specificity of this new assay in detecting LTBI.

Problems translating a questionnaire in a cross-cultural setting

Questionnaires are widely used in veterinary epidemiological studies. With careful design, assessment and administration, questionnaires can collect accurate data. In cross-cultural settings, questionnaire design is complicated by the added step of translation. As part of a cross-sectional study of smallholder pig raisers in the Philippines, we assessed the importance of translation as a source of error in our questionnaire. The questionnaire contained 120 questions that collected data for entry into 361 separate data-entry fields in a computerised database. The translation was verified using the methods of back-translation and pilot-testing to identify and quantify errors of literal translation, omission and mistranslation. Errors were identified in 44 (37%) questions that obtained data for 147 (41%) data-entry fields. Although errors might have remained in the final translated version of the questionnaire, we have no doubt that the verification procedures substantially improved the internal validity of the questionnaire. It is important that veterinary epidemiologists use such procedures to check the internal validity of translated questionnaires.