Expert and novice performance in solving physics problems

The internal representation of arithmetical word problem sentences: frontal and posterior-injured patients compared

Patients with frontal and left posterior brain lesions have severe difficulties in arithmetical word problem solving. In the present study the origin of these difficulties is investigated from an information-processing perspective. Following this perspective the first stage in word problem solving consists of a translation of individual sentences to an internal representation. This translation process is examined in 30 frontal patients, 10 left posterior-injured patients, and 10 healthy controls with a recognition and a sentence-picture matching task. In addition, the relationship between sentence representation and arithmetical word problem solving is studied. The results suggest that error rates in the translation of different types of arithmetical word problem propositions differ substantially in our three groups. A relationship between translation skills and arithmetical word problem solving ability is also found.

A closer look at the probabilities of the notorious three prisoners

The "problem of three prisoners", a counterintuitive teaser, is analyzed. It is representative of a class of probability puzzles where the correct solution depends on explication of underlying assumptions. Spontaneous beliefs concerning the problem and intuitive heuristics are reviewed. The psychological background of these beliefs is explored. Several attempts to find a simple criterion to predict whether and how the probability of the target event will change as a result of obtaining evidence are examined. However, despite the psychological appeal of these attempts, none proves to be valid in general. A necessary and sufficient condition for change in the probability of the target event, following observation of new data, is proposed. That criterion is an extension of the likelihood-ratio principle (which holds in the case of only two complementary alternatives) to any number of alternatives. Some didactic implications concerning the significance of the chance set-up and reliance on analogies are discussed.

A cooperative approach to decision support in the differential diagnosis of breast disease

The problem of differential diagnosis in breast histopathology is described and areas in need of decision support identified. It is argued that the conventional role played by decision support systems, i.e. that of the 'autonomous problem-solver', is unacceptable in this domain and that support must be given to the actual cognitive functions of potential system users. Decision-making processes of pathologists are examined with particular reference to Rector's category of 'broad experts'. Pathologists possess a rich set of general diagnostic concepts for their domain, but their knowledge of detailed aspects of differential diagnosis may contain gaps and errors. A cooperative model of decision support, which utilizes the strengths and complements the weaknesses of pathologists' problem-solving skills, is described and the operation of this model within a functioning system illustrated.

Medical problem solving and uncertainty in the emergency department

STUDY OBJECTIVE

To compare the diagnostic processes of experienced emergency physicians with those of novices.

DESIGN

Prospective, convenience sample of patients.

SETTING

Emergency department of a county university medical center in a large southwestern urban community.

PARTICIPANTS

Experienced emergency physicians (attending and senior residents) and novice clinicians (junior residents and senior medical students).

INTERVENTIONS

Participants developed initial diagnostic impressions after reviewing the chief complaint, nurse triage notes, and vital signs. Tests were then selected, and a final diagnostic impression was identified after results were known. Clinicians also marked a visual analog scale corresponding to their estimate that each diagnostic possibility was correct.

RESULTS

Experienced physicians increased their certainty more than novices (P = .014). They deviated from a standard history-physical-laboratory sequence more often than novices (P = .008).

CONCLUSION

Expertise in medical decision making is characterized by a moderate initial level of certainty concerning a diagnosis that significantly increases as the experienced clinician follows a flexible strategy of testing to arrive at a final diagnosis.

Conditional reasoning and causation

An experiment was conducted to investigate the relative contributions of syntactic form and content to conditional reasoning. The content domain chosen was that of causation. Conditional statements that described causal relationships (if mean value of cause, then mean value of effect) were embedded in simple arguments whose entailments are governed by the rules of truth-functional logic (i.e., modus ponens, modus tollens, denying the antecedent, and affirming the consequent). The causal statements differed in terms of the number of alternative causes and disabling conditions that characterized the causal relationship. (A disabling condition is an event that prevents an effect from occurring even though a relevant cause is present). Subjects were required to judge whether or not each argument's conclusion could be accepted. Judgements were found to vary systematically with the number of alternative causes and disabling conditions. Conclusions of arguments based on conditionals with few alternative causes or disabling conditions were found to be more acceptable than conclusions based on those with many.

A model for designing intelligent tutoring systems

A model for the design of an intelligent tutoring system is presented using artificial intelligence techniques and cognitive processing theories. The model of cognitive processing known as the Knowledge Acquisition and Recall Theory is derived from Anderson's Adaptive Control of Thought Theory. The model is used as a basis for a methodology for the design of an intelligent tutoring system that teaches problem-solving strategies for blood grouping discrepancies. The system developed was tested to determine the effectiveness of the methodology developed and to provide support for the concepts in the model of cognition developed. The preliminary results provide some evidence that an intelligent tutoring system designed using the methodology developed may aid in the knowledge acquisition process. (knowledge acquisition, artificial intelligence, computer-assisted instruction, cognition, intelligent tutoring system).

Decision support system for the differential diagnosis of breast disease

The histopathological diagnosis of breast disease is representative of many problems of differential diagnosis encountered in the medical domain. It requires highly trained and experienced experts and is characterized by a large number of features whose presence or absence involves much uncertainty. Computer-based decision support systems intended to function in a consultative capacity during differential diagnosis have had limited success for two fundamental reasons. Firstly, they take an autonomous role and assume that the user has no contribution to make to the problem-solving process. Secondly, the established techniques for representing and reasoning with medical knowledge are of limited suitability in this domain. Such systems are unable to reach a correct diagnosis quickly and often subject the user to a cumbersome dialogue. These are not tolerated by pathologists working under severe time constraints. We first look at the problem-solving methods employed by pathologists in this domain and examine the functionality of traditional expert system methodologies. We then present a cooperative design which allows the pathologists to express his or her ideas within a decision support system whilst gaining assistance in required areas. A novel inference technique based upon the set partitioning technique in hypergraphs is also described. This mathematical method has the ability to cope with the incomplete or inadequate knowledge which is a characteristic of breast disease, whilst directing data gathering in a meaningful manner. In particular this approach can significantly reduce the amount of irrelevant data which the pathologist must enter before a conclusion is reached. Thus it can potentially improve the efficiency and user acceptability of medical expert systems.

Medical expertise as a function of task difficulty

This paper is concerned with factors that disrupt the pattern of forward reasoning characteristic of experts with accurate performance. Two experiments are described. In the first, the performances of cardiologists, psychiatrists, and surgeons in diagnostic explanation of a clinical problem in cardiology were examined. In the second, the performances of cardiologists and endocrinologists in diagnostic explanation of clinical problems within and outside their domains of expertise were examined. The performances of researchers and practicing physicians are also compared. The results of Experiment 1 replicated earlier results regarding the relationship between forward reasoning and accurate diagnosis. There were no differences in recall as a function of expertise. Experts did not show any bias toward using specific knowledge from their own areas of expertise. The results of Experiment 2 showed that the breakdown of forward reasoning was related to the structure of the task. In particular, nonsalient cues induced some backward reasoning even in subjects with accurate diagnoses. Some differences were also found between the types of explanation used by researchers and practitioners. The practitioners referred more to clinical components in their explanations, whereas the researchers focused more on the biomedical components.

Knowledge acquisition and strategic action in "Mastermind" problems

In two experiments, the strategies used by subjects playing the logical-deduction game, Mastermind, were examined. In the first experiment, subjects showed improvement resulting from the continued use of a particular strategic action, and the data suggested that the subjects learned the strategy from their transactions with the task. In the second experiment, the question of changes in underlying strategic knowledge of Mastermind was examined. The accuracy and complexity of the subjects' deductions and their use of the previously identified strategy were used to generate a model of the cognitive operations involved in Mastermind. Although there were improvements in the accuracy and complexity of the subjects' deductions resulting from continued play, these improvements were unrelated to the use of the strategy. Moreover, the likelihood of making accurate and complex deductions was well accounted for by a Markovian model, suggesting that the deployment of the strategy was not driven by any change in the subject's underlying knowledge structures. Rather, the subjects seemed to use the strategy to create Mastermind situations whose interpretation was fairly easy. The implications for previous work on the issue of Mastermind strategies and the development of logical-deduction strategies are discussed.

Domain knowledge and hypothesis generation in diagnostic reasoning

The role of domain knowledge in the process of hypothesis generation during diagnostic reasoning was examined. Subjects were given a clinical case presented one segment at a time on a microcomputer. They were prompted to think aloud after presentation of each segment of the clinical case. A combination of discourse and protocol analysis techniques was used to investigate the problem solving process in two groups of experts working on an endocrine problem. The groups consisted of high-domain-knowledge subjects (HDK), endocrinologists, and low-domain-knowledge subjects (LDK), cardiologists. The results showed no significant differences between the groups in terms of selection of relevant and critical cues from the case. However, specific differences were found in the links or relations between the cues, with the HDK subjects using more relations to connect important information. The HDK subjects generated accurate diagnostic hypotheses early in the problem encounter and spent the rest of the time confirming the hypotheses by explaining the given cues. The LDK subjects also generated accurate diagnostic hypotheses but were unable to discriminate between and eliminate alternative hypotheses. A two-stage problem solving process and its relationship to domain specific knowledge are proposed.

Clinical judgment and therapeutic decision making

Clinical decision making is under increased scrutiny due to concerns about the cost and quality of medical care. Variability in physician decision making is common, in part because of deficiencies in the knowledge base, but also due to the difference in physicians' approaches to clinical problem solving. Evaluation of patient prognosis is a critical factor in the selection of therapy, and careful attention to methodology is essential to provide reliable information. Randomized controlled clinical trials provide the most solid basis for the establishment of broad therapeutic principles. Because randomized studies cannot be performed to address every question, observational studies will continue to play a complementary role in the evaluation of therapy. Randomized studies in progress, meta analyses of existing data, and increased use of administrative and collaborative clinical data bases will improve the knowledge base for decision making in the future.

The quality of expertise: implications of expert-novice differences for knowledge acquisition

Recognizing that the knowledge of experts is qualitatively and not just quantitatively different from the knowledge of novices is an important pre-requisite to conducting effective knowledge acquisition. This article reviews the current cognitive research on expertise and proposes seven ways in which the knowledge of experts is different from the knowledge of novices, including such aspects as underlying schema, goal-orientation, practical focus, categorical chunking, cognitive complexity, automaticity of expert problem solving, and finally, the episodic nature of expert memory. The paper concludes with an outline of several implications that follow from this research including showing why knowledge acquisition will remain problematic without knowledge of experts' knowledge, as well as making a number of specific suggestions for those who need to elicit expert knowledge for knowledge-based systems.