A graphical judgmental aid which summarizes obtained and chance reliability data and helps assess the believability of experimental effects

Structured Solicitation: A Standardized Method for Gaining Travel Information

Travelers must be able to obtain locational information if they are to travel successfully to unfamiliar places. Three subjectives were asked to call a number of businesses to pinpoint their location. After establishing a baseline rate, the subjects were introduced to a structured solicitation procedure, which consisted of asking seven specific questions in a particular order. The completeness of information obtained increased dramatically with the use of this procedure.

Consistent Methodological Errors Observed in Single-Case Studies: Suggested Guidelines

This article reviews current and consistent methodological problems that occur in research articles using single-case experimental designs. Problems discussed include (1) the use of pre- and quasi-experimental single-case designs, (2) the presentation and analysis of data, (3) the lack or inadequate demonstration of experimental reliability, and (4) the improper or ineffective use of generalization. The authors suggest ways of solving these problems and provide guidelines for avoiding their recurrence.

Rates of Teachers' Verbal Approval and Disapproval

Existing rates of teacher approval and disapproval were assessed in 35 classrooms (grades K-4, 9, 12) during various activities. Teachers'mean rates of both approval and disapproval were highest in first grade and lower in upper grades, with the partial exception of the ninth. These data lend certain generality to those reported by White (1975), but there are also substantial differences between the two data sets. White's conclusions-that disapproval predominated after grade 2 and may account for disillusionment of children in grades thereafter-are questioned. Transition was the only activity in which disapprovals exceeded approvals. Approval and disapproval rates were not related to teachers' ages, years of experience, or recency of training. One implication is that teacher training should focus more on skill at verbal approval, particularly in classroom management, and that such training should be conducted by behavior analysts in classrooms.

Behavioral and facial thermal variations in 3-to 4-month-old infants during the Still-Face Paradigm

Behavioral and facial thermal responses were recorded in twelve 3- to 4-month-old infants during the Still-Face Paradigm (SFP). As in the usual procedure, infants were observed in a three-step, face-to-face interaction: a normal interaction episode (3 min); the “still-face” episode in which the mother became unresponsive and assumed a neutral expression (1 min); a reunion episode in which the mother resumed the interaction (3 min). A fourth step that consisted of a toy play episode (5 min) was added for our own research interest. We coded the behavioral responses through the Infant and Caregiver Engagement Phases system, and recorded facial skin temperature via thermal infrared (IR) imaging. Comparing still-face episode to play episode, the infants’ communicative engagement decreased, their engagement with the environment increased, and no differences emerged in self-regulatory and protest behaviors. We also found that facial skin temperature increased. For the behavioral results, infants recognized the interruption of the interactional reciprocity caused by the still-face presentation, without showing upset behaviors. According to autonomic results, the parasympathetic system was more active than the sympathetic, as usually happens in aroused but not distressed situations. With respect to the debate about the causal factor of the still-face effect, thermal data were consistent with behavioral data in showing this effect as related to the infants’ expectations of the nature of the social interactions being violated. Moreover, as these are associated to the infants’ subsequent interest in the environment, they indicate the thermal IR imaging as a reliable technique for the detection of physiological variations not only in the emotional system, as indicated by research to date, but also in the attention system. Using this technique for the first time during the SFP allowed us to record autonomic data in a more ecological manner than in previous studies.

Relationships between regulatory temperament dimensions and self-regulatory behaviors at 4 and 6 months of age

The present study focused on relationships between temperament and behavior in early regulation development. Unlike most studies on the topic, we observed infant behavior in a naturalistic playful situation rather than in experimental stressful procedure, and employed temperament measures uniquely reflecting regulatory dispositions rather than a global measure of reactivity. The infant's self-regulatory behaviors were observed at 4 and 6 months during face-to-face interactions and regulatory dimensions were assessed at 4 months. We found that low intensity pleasure and soothability dimensions, related to the infant physical and social experience, respectively, significantly affected regulatory behavior and their influence showed to depend on the infant's age, with the former dimension being influential at the earlier age and the latter being influential when the behavior was observed at the later age. Results are interpreted on the light of a dynamic view of regulation development.

A Comparison of Reliability Measures for Continuous and Discontinuous Recording Methods: Inflated Agreement Scores With Partial Interval Recording and Momentary Time Sampling for Duration Events

The authors evaluated the extent to which interobserver agreement (IOA) scores, using the block-by-block method for events scored with continuous duration recording (CDR), were higher when the data from the same sessions were converted to discontinuous methods. Sessions with IOA scores of 89% or less with CDR were rescored using 10-s partial interval recording (PIR) and 10-s momentary time sampling (MTS). Results indicated that IOA scores for 10-s PIR and 10-s MTS were consistently higher than IOA scores based on CDR for the same sessions. Specifically, 10-s MTS provided higher overestimations for low-duration events, whereas 10-s PIR produced higher overestimations for moderate- and high-duration events. Implications for researchers and clinicians are briefly discussed.

Attachment patterns and emotion regulation strategies in the second year

With the aim of studying the relationship between methods of emotion regulation and quality of attachment we examined 39 infants with different patterns of attachment, of whom 20 were classified as secure (B), 12 as avoidant (A) and 7 as resistant (C), assessing the regulatory strategies adopted by them during the Strange Situation at 13 months. Secure infants used strategies of positive social engagement more than insecure avoidant infants, while resistant infants displayed greater negative social engagement and less object orientation than the other two groups. Avoidant infants adopted positive and negative hetero-regulatory strategies less than the other groups, also differing from resistant infants in their greater use of object regulatory strategies. There were no significant differences as regards self-comforting regulation. Thus, the findings showed how the most significant differences to emerge between the groups concerned hetero-regulatory strategies, developed by the infant in interaction with attachment figures, and regulatory strategies oriented towards objects. Further analysis showed how the use by part of each attachment group of the emotion regulation strategies varies, differentiating the episodes of the SSP according to their level of stress.

The nature of behavioral assessment: A commentary

This special issue of the Journal of Applied Behavior Analysis provides the reader with a sample of current work in behavioral assessment. The purpose of this paper is to present an overview of behavioral assessment and to place the other articles in context of this developing area.

Back to basics: Percentage agreement measures are adequate, but there are easier ways

Percentage agreement measures of interobserver agreement or "reliability" have traditionally been used to summarize observer agreement from studies using interval recording, time-sampling, and trial-scoring data collection procedures. Recent articles disagree on whether to continue using these percentage agreement measures, and on which ones to use, and what to do about chance agreements if their use is continued. Much of the disagreement derives from the need to be reasonably certain we do not accept as evidence of true interobserver agreement those agreement levels which are substantially probable as a result of chance observer agreement. The various percentage agreement measures are shown to be adequate to this task, but easier ways are discussed. Tables are given to permit checking to see if obtained disagreements are unlikely due to chance. Particularly important is the discovery of a simple rule that, when met, makes the tables unnecessary. If reliability checks using 50 or more observation occasions produce 10% or fewer disagreements, for behavior rates from 10% through 90%, the agreement achieved is quite improbably the result of chance agreement.

A3

HCI studies assessing nonverbal individuals (especially those who do not communicate through traditional linguistic means: spoken, written, or sign) are a daunting undertaking. Without the use of directed tasks, interviews, questionnaires, or question-answer sessions, researchers must rely fully upon observation of behavior, and the categorization and quantification of the participant’s actions. This problem is compounded further by the lack of metrics to quantify the behavior of nonverbal subjects in computer-based intervention contexts. We present a set of dependent variables called A3 (pronounced A-Cubed) or Annotation for ASD Analysis, to assess the behavior of this demographic of users, specifically focusing on engagement and vocalization. This paper demonstrates how theory from multiple disciplines can be brought together to create a set of dependent variables, as well as demonstration of these variables, in an experimental context. Through an examination of the existing literature, and a detailed analysis of the current state of computer vision and speech detection, we present how computer automation may be integrated with the A3 guidelines to reduce coding time and potentially increase accuracy. We conclude by presenting how and where these variables can be used in multiple research areas and with varied target populations.

The standard deviation as an informative measure of variability in reporting interobserver agreement means

The range is the measure of variability typically presented with the means of interobserver agreement scores of behavioral data. However, the range by itself yields little information about the individual interobserver agreement scores before they were averaged. The standard deviation, on the other hand, is a measure of variability that provides the researcher and reader with information regarding the dispersion of scores, allowing a more thorough interpretation of the interobserver agreement mean. The case is presented for the adoption of the standard deviation as an adjunct to the range in reporting the mean of interobserver agreement scores.

Using statistical process control to make data-based clinical decisions

Applied behavior analysis is based on an investigation of variability due to interrelationships among antecedents, behavior, and consequences. This permits testable hypotheses about the causes of behavior as well as for the course of treatment to be evaluated empirically. Such information provides corrective feedback for making data-based clinical decisions. This paper considers how a different approach to the analysis of variability based on the writings of Walter Shewart and W. Edwards Deming in the area of industrial quality control helps to achieve similar objectives. Statistical process control (SPC) was developed to implement a process of continual product improvement while achieving compliance with production standards and other requirements for promoting customer satisfaction. SPC involves the use of simple statistical tools, such as histograms and control charts, as well as problem-solving techniques, such as flow charts, cause-and-effect diagrams, and Pareto charts, to implement Deming's management philosophy. These data-analytic procedures can be incorporated into a human service organization to help to achieve its stated objectives in a manner that leads to continuous improvement in the functioning of the clients who are its customers. Examples are provided to illustrate how SPC procedures can be used to analyze behavioral data. Issues related to the application of these tools for making data-based clinical decisions and for creating an organizational climate that promotes their routine use in applied settings are also considered.

The reliability of observational data: I. Theories and methods for speech-language pathology

Much research and clinical work in speech-language pathology depends on the validity and reliability of data gathered through the direct observation of human behavior. This paper reviews several definitions of reliability, concluding that behavior observation data are reliable if they, and the experimental conclusions drawn from them, are not affected by differences among observers or by other variations in the recording context. The theoretical bases of several methods commonly used to estimate reliability for observational data are reviewed, with examples of the use of these methods drawn from a recent volume of the Journal of Speech and Hearing Research (35, 1992). Although most recent research publications in speech-language pathology have addressed the issue of reliability for their observational data to some extent, most reliability estimates do not clearly establish that the data or the experimental conclusions were replicable or unaffected by differences among observers. Suggestions are provided for improving the usefulness of the reliability estimates published in speech-language pathology research.

Comparison of Interjudge Reliabilities in Scoring TAT Protocols as a Measure of NACH

Several indices of interjudge reliability are discussed with respect to the TAT-scoring methodology. Some improvements are suggested for the indices proposed by McClelland, et al. (15). Two indices of proportion of agreement are proposed. A comparison of McClelland, et al.'s indices and those of proportion of agreement shows that the proportion of agreement in scoring subcategories is more severe and suggests that it should be added to evaluate more tightly interjudge reliability in scoring the TAT.

The use of human observers in psychopharmacological research

During 1974-1978, over 40% of the nonhuman drug studies that appeared in Psychopharmacology, Pharmacology Biochemistry and Behavior, and Journal of Pharmacology and Experimental Therapeutics involved human observers; far fewer studies published in Journal of the Experimental Analysis of Behavior did so. In all of these journals, measures of interobserver agreement seldom were provided. The great majority of studies also failed to utilize one or more "blind" observers, unaware of experimental conditions. These findings are of interest in light of reports that observational data are affected by a wide range of factors and often provide an inaccurate index of behavior. The believability of observational data seemingly is enhanced by careful descriptions of recording procedures coupled with the use of two or more blind observers whose concordance in rating behavior has been determined. These procedures characteristically are followed in some behavioral sciences, such as applied behavior analysis, but not to the same degree in psychopharmacology.