Validation of intelligent systems: a critical study and a tool

Web Accessibility Assessment Through Multi-Agent Support for Visually Impaired Users

In general, users who access Web sites do not necessarily have the same abilities in terms of cognitive, sensory and physical conditions. An accessible Web site is one whose content can be used by as many users as possible; and to improve accessibility, the use of evaluation tools has been incorporated among other resources. However, from the point of view of the end user, the evaluation obtained by most of these tools does not fit to the individual needs and abilities of the users. On the other hand, there is a greater understanding of the need to automate the support to reach higher levels of accessibility.

Promoting this type of support on both the developer and the end-user sides is a complex procedure that can be done by incorporating intelligent features, such as the collaborative participation of intelligent agents. In this paper, we introduce a multi-agent system based on visually impaired user’s profiles to improve Web Accessibility assessment. The proposal is validated through a proof of concept, by contrasting results against 10 wide-known assessment tools. Our results show that evaluations are more focused, and therefore, unnecessary assessments are avoided improving efficiency.

Developing a web-based advisory expert system for implementing traffic calming strategies

Lack of traffic safety has become a serious issue in residential areas. In this paper, a web-based advisory expert system for the purpose of applying traffic calming strategies on residential streets is described because there currently lacks a structured framework for the implementation of such strategies. Developing an expert system can assist and advise engineers for dealing with traffic safety problems. This expert system is developed to fill the gap between the traffic safety experts and people who seek to employ traffic calming strategies including decision makers, engineers, and students. In order to build the expert system, examining sources related to traffic calming studies as well as interviewing with domain experts have been carried out. The system includes above 150 rules and 200 images for different types of measures. The system has three main functions including classifying traffic calming measures, prioritizing traffic calming strategies, and presenting solutions for different traffic safety problems. Verifying, validating processes, and comparing the system with similar works have shown that the system is consistent and acceptable for practical uses. Finally, some recommendations for improving the system are presented.

A multi-agent-based, semantic-driven system for decision support in epidemic management

Issues in epidemiology are truly multidisciplinary, requiring knowledge from diverse disciplines such as sociology, medicine, biology, geography and information science. Such inherent complexity has led to a challenge in developing decision support systems for epidemic information management, especially when data are from heterogeneous origins. In order to achieve a solution, an integrative framework is proposed. The Semantic Web is introduced in the context of enriching meaningful and machine-readable descriptions of epidemiological data. Software agents are utilised to achieve automation in semantic discovery, composition of data and process services. The objective is to enhance the performance in information retrieval in a dynamic decision-making environment while concealing technical complexity from inexperienced users. We illustrate how a prototype system can be developed by considering an epidemiology management scenario in which spatio-temporal analysis is undertaken of a specified epidemic.

Empirical evaluation of a hybrid intelligent monitoring system using different measures of effectiveness

The validation of a software product is a fundamental part of its development, and focuses on an analysis of whether the software correctly resolves the problems it was designed to tackle. Traditional approaches to validation are based on a comparison of results with what is called a gold standard. Nevertheless, in certain domains, it is not always easy or even possible to establish such a standard. This is the case of intelligent systems that endeavour to simulate or emulate a model of expert behaviour. This article describes the validation of the intelligent system computer-aided foetal evaluator (CAFE), developed for intelligent monitoring of the antenatal condition based on data from the non-stress test (NST), and how this validation was accomplished through a methodology designed to resolve the problem of the validation of intelligent systems. System performance was compared to that of three obstetricians using 3450 min of cardiotocographic (CTG) records corresponding to 53 different patients. From these records different parameters were extracted and interpreted, and thus, the validation was carried out on a parameter-by-parameter basis using measurement techniques such as percentage agreement, the Kappa statistic or cluster analysis. Results showed that the system's agreement with the experts is, in general, similar to agreement between the experts themselves which, in turn, permits our system to be considered at least as skillful as our experts. Throughout our article, the results obtained are commented on with a view to demonstrating how the utilisation of different measures of the level of agreement existing between system and experts can assist not only in assessing the aptness of a system, but also in highlighting its weaknesses. This kind of assessment means that the system can be fine-tuned repeatedly to the point where the expected results are obtained.