Problematic smartphone use and academic achievement: A systematic review and meta-analysis

Background and aims

The present study aimed to synthesize existing quantitative evidence on the relationship between problematic smartphone use (PSU) and academic achievement with a focus on quantifying its magnitude and examining its potential moderators.

Methods

Eligible studies were searched for up to February 10, 2023 in six different databases (i.e., MEDLINE, Current Contents Connect, PsycINFO, Web of Science, SciELO, and Dissertations & Theses Global). Studies were considered eligible if they provided information derived from self-report instruments that allowed statistical calculation of the relationship between PSU and academic achievement. Pooled effect sizes (r) were computed using a random-effects model. Meta-regressions were conducted to test the influence of study-level moderators on the relationship of interest. Influence analyses and a three-parameter selection model (3PSM) were conducted to examine the robustness of the results and publication bias, respectively.

Results

A total of 33 effect sizes from 29 studies (n = 48,490) were retrieved. Results showed a small effect size (r = -0.110), which tended to be larger in samples consisting of students from elementary and middle schools.

Discussion and Conclusions

Findings from the present study contribute to the understanding of a potential determinant of decreased academic achievement by providing evidence that PSU may be one of them.

Chatbots for future docs: exploring medical students' attitudes and knowledge towards artificial intelligence and medical chatbots

Artificial intelligence (AI) in medicine and digital assistance systems such as chatbots will play an increasingly important role in future doctor - patient communication. To benefit from the potential of this technical innovation and ensure optimal patient care, future physicians should be equipped with the appropriate skills. Accordingly, a suitable place for the management and adaptation of digital assistance systems must be found in the medical education curriculum. To determine the existing levels of knowledge of medical students about AI chatbots in particular in the healthcare setting, this study surveyed medical students of the University of Luebeck and the University Hospital of Tuebingen. Using standardized quantitative questionnaires and qualitative analysis of group discussions, the attitudes of medical students toward AI and chatbots in medicine were investigated. From this, relevant requirements for the future integration of AI into the medical curriculum could be identified. The aim was to establish a basic understanding of the opportunities, limitations, and risks, as well as potential areas of application of the technology. The participants (N = 12) were able to develop an understanding of how AI and chatbots will affect their future daily work. Although basic attitudes toward the use of AI were positive, the students also expressed concerns. There were high levels of agreement regarding the use of AI in administrative settings (83.3%) and research with health-related data (91.7%). However, participants expressed concerns that data protection may be insufficiently guaranteed (33.3%) and that they might be increasingly monitored at work in the future (58.3%). The evaluations indicated that future physicians want to engage more intensively with AI in medicine. In view of future developments, AI and data competencies should be taught in a structured way during the medical curriculum and integrated into curricular teaching.

Functional and neuromuscular changes induced via a low-cost, muscle-computer interface for telerehabilitation: A feasibility study in chronic stroke

Stroke is a leading cause of adult disability in the United States. High doses of repeated task-specific practice have shown promising results in restoring upper limb function in chronic stroke. However, it is currently challenging to provide such doses in clinical practice. At-home telerehabilitation supervised by a clinician is a potential solution to provide higher-dose interventions. However, telerehabilitation systems developed for repeated task-specific practice typically require a minimum level of active movement. Therefore, severely impaired people necessitate alternative therapeutic approaches. Measurement and feedback of electrical muscle activity via electromyography (EMG) have been previously implemented in the presence of minimal or no volitional movement to improve motor performance in people with stroke. Specifically, muscle neurofeedback training to reduce unintended co-contractions of the impaired hand may be a targeted intervention to improve motor control in severely impaired populations. Here, we present the preliminary results of a low-cost, portable EMG biofeedback system (Tele-REINVENT) for supervised and unsupervised upper limb telerehabilitation after stroke. We aimed to explore the feasibility of providing higher doses of repeated task-specific practice during at-home training. Therefore, we recruited 5 participants (age = 44-73 years) with chronic, severe impairment due to stroke (Fugl-Meyer = 19-40/66). They completed a 6-week home-based training program that reinforced activity of the wrist extensor muscles while avoiding coactivation of flexor muscles via computer games. We used EMG signals to quantify the contribution of two antagonistic muscles and provide biofeedback of individuated activity, defined as a ratio of extensor and flexor activity during movement attempt. Our data suggest that 30 1-h sessions over 6 weeks of at-home training with our Tele-REINVENT system is feasible and may improve individuated muscle activity as well as scores on standard clinical assessments (e.g., Fugl-Meyer Assessment, Action Research Arm Test, active wrist range of motion) for some individuals. Furthermore, tests of neuromuscular control suggest modest changes in the synchronization of electroencephalography (EEG) and EMG signals within the beta band (12-30 Hz). Finally, all participants showed high adherence to the training protocol and reported enjoying using the system. These preliminary results suggest that using low-cost technology for home-based telerehabilitation after severe chronic stroke is feasible and may be effective in improving motor control via feedback of individuated muscle activity.

Wearable super-resolution muscle-machine interfacing

Muscles are the actuators of all human actions, from daily work and life to communication and expression of emotions. Myography records the signals from muscle activities as an interface between machine hardware and human wetware, granting direct and natural control of our electronic peripherals. Regardless of the significant progression as of late, the conventional myographic sensors are still incapable of achieving the desired high-resolution and non-invasive recording. This paper presents a critical review of state-of-the-art wearable sensing technologies that measure deeper muscle activity with high spatial resolution, so-called super-resolution. This paper classifies these myographic sensors according to the different signal types (i.e., biomechanical, biochemical, and bioelectrical) they record during measuring muscle activity. By describing the characteristics and current developments with advantages and limitations of each myographic sensor, their capabilities are investigated as a super-resolution myography technique, including: (i) non-invasive and high-density designs of the sensing units and their vulnerability to interferences, (ii) limit-of-detection to register the activity of deep muscles. Finally, this paper concludes with new opportunities in this fast-growing super-resolution myography field and proposes promising future research directions. These advances will enable next-generation muscle-machine interfaces to meet the practical design needs in real-life for healthcare technologies, assistive/rehabilitation robotics, and human augmentation with extended reality.

Comparing the Effects of Stroke-Appearing and Stroke-Disappearing on Learning the Order of Strokes in Chinese Characters

Different approaches to stimulating perceptions in learning can be easily designed with technology-enhanced learning systems. This study aimed to explore how different approaches can influence learners' perceptions that may negatively or positively affect their learning performance of writing Chinese characters using the correct Chinese order of strokes (COS). We therefore designed an e-learning system which was subdivided into two modes: stroke-appearing (i.e., using red to mark incorrect strokes) and stroke-disappearing (i.e., using blanks to mark incorrect strokes) to indicate strokes written in the incorrect order. We then investigated the modes that would facilitate a higher level of attention and better learning outcomes. A total of 10 third-grade elementary school students participated in the experiment, divided into two test groups. Their EEG data were collected, and time series analysis and t-tests were utilized to analyze the differences. The results indicated that: (1) there was a significant difference in the attention levels of the students practicing with the stroke-appearing and stroke-disappearing modes when learning COS, and (2) there was a significant difference in the learning outcomes of the students practicing with the stroke-appearing and stroke-disappearing modes when learning COS. These findings support the specific role of stroke order knowledge in learning Chinese characters and the need for the design of an effective method for teaching children to learn Chinese characters.

An assessment of ergonomic issues in the home offices of university employees sent home due to the COVID-19 pandemic

BACKGROUND

As millions of workers have shifted to telework, special accommodations for workers with respect to ergonomics may be required to ensure the workforce remains healthy.

METHODS

A survey about home office ergonomics and discomfort was sent to faculty, staff, and administrators by email and was completed by 843 individuals.

RESULTS

Over 40%of the participants reported moderate to severe discomfort (severe low/middle back pain, moderate discomfort in eyes/neck/head, and discomfort in the upper back/shoulders). Laptops (always and often) were widely used (85%) with most using the laptop monitor (55%) of all respondents. Further, less than 45%of the seating conditions were reported as having adjustable arm rests.

CONCLUSION

As teleworking in makeshift offices becomes more common, the risk of significant discomfort and potentially more serious musculoskeletal disorders may result from poor static postures. Companies may need to accommodate workers by allowing them to take home office chairs, external monitors, keyboards, and mice as laptops are insufficient, ergonomically.

Development of a Low-Cost, Modular Muscle-Computer Interface for At-Home Telerehabilitation for Chronic Stroke

Stroke is a leading cause of long-term disability in the United States. Recent studies have shown that high doses of repeated task-specific practice can be effective at improving upper-limb function at the chronic stage. Providing at-home telerehabilitation services with therapist supervision may allow higher dose interventions targeted to this population. Additionally, muscle biofeedback to train patients to avoid unwanted simultaneous activation of antagonist muscles (co-contractions) may be incorporated into telerehabilitation technologies to improve motor control. Here, we present the development and feasibility of a low-cost, portable, telerehabilitation biofeedback system called Tele-REINVENT. We describe our modular electromyography acquisition, processing, and feedback algorithms to train differentiated muscle control during at-home therapist-guided sessions. Additionally, we evaluated the performance of low-cost sensors for our training task with two healthy individuals. Finally, we present the results of a case study with a stroke survivor who used the system for 40 sessions over 10 weeks of training. In line with our previous research, our results suggest that using low-cost sensors provides similar results to those using research-grade sensors for low forces during an isometric task. Our preliminary case study data with one patient with stroke also suggest that our system is feasible, safe, and enjoyable to use during 10 weeks of biofeedback training, and that improvements in differentiated muscle activity during volitional movement attempt may be induced during a 10-week period. Our data provide support for using low-cost technology for individuated muscle training to reduce unintended coactivation during supervised and unsupervised home-based telerehabilitation for clinical populations, and suggest this approach is safe and feasible. Future work with larger study populations may expand on the development of meaningful and personalized chronic stroke rehabilitation.

An investigation of in-ear sensing for motor task classification

Our study aims to investigate the feasibility of in-ear sensing for human-computer interface. We first measured the agreement between in-ear biopotential and scalp-EEG signals by channel correlation and power spectral density analysis. Then we applied EEG compact network (EEGNet) for the classification of a two-class motor task using in-ear electrophysiological signals. The best performance using in-ear biopotential with global reference reached an average accuracy of 70.22\% (cf. 92.61\% accuracy using scalp-EEG signals), but the performance in-ear biopotential with near-ear reference was poor. Our results suggest in-ear sensing would be a viable human-computer interface for movement prediction, but careful consideration should be paid to the position of the reference electrodes.

A Review of Force Myography Research and Development

Information about limb movements can be used for monitoring physical activities or for human-machine-interface applications. In recent years, a technique called Force Myography (FMG) has gained ever-increasing traction among researchers to extract such information. FMG uses force sensors to register the variation of muscle stiffness patterns around a limb during different movements. Using machine learning algorithms, researchers are able to predict many different limb activities. This review paper presents state-of-art research and development on FMG technology in the past 20 years. It summarizes the research progress in both the hardware design and the signal processing techniques. It also discusses the challenges that need to be solved before FMG can be used in an everyday scenario. This paper aims to provide new insight into FMG technology and contribute to its advancement.

Multi-Scale Coordination of Distinctive Movement Patterns During Embodied Interaction Between Adults With High-Functioning Autism and Neurotypicals

Autism Spectrum Disorder (ASD) can be understood as a social interaction disorder. This requires researchers to take a “second-person” stance and to use experimental setups based on bidirectional interactions. The present work offers a quantitative description of movement patterns exhibited during computer-mediated real-time sensorimotor interaction in 10 dyads of adult participants, each consisting of one control individual (CTRL) and one individual with high-functioning autism (HFA). We applied time-series analyses to their movements and found two main results. First, multi-scale coordination between participants was present. Second, despite this dyadic alignment and our previous finding that individuals with HFA can be equally sensitive to the other’s presence, individuals’ movements differed in style: in contrast to CTRLs, HFA participants appeared less inclined to sustain mutual interaction and instead explored the virtual environment more generally. This finding is consistent with social motivation deficit accounts of ASD, as well as with hypersensitivity-motivated avoidance of overstimulation. Our research demonstrates the utility of time series analyses for the second-person stance and complements previous work focused on non-dynamical and performance-based variables.

A Novel Feature Optimization for Wearable Human-Computer Interfaces Using Surface Electromyography Sensors

The novel human-computer interface (HCI) using bioelectrical signals as input is a valuable tool to improve the lives of people with disabilities. In this paper, surface electromyography (sEMG) signals induced by four classes of wrist movements were acquired from four sites on the lower arm with our designed system. Forty-two features were extracted from the time, frequency and time-frequency domains. Optimal channels were determined from single-channel classification performance rank. The optimal-feature selection was according to a modified entropy criteria (EC) and Fisher discrimination (FD) criteria. The feature selection results were evaluated by four different classifiers, and compared with other conventional feature subsets. In online tests, the wearable system acquired real-time sEMG signals. The selected features and trained classifier model were used to control a telecar through four different paradigms in a designed environment with simple obstacles. Performance was evaluated based on travel time (TT) and recognition rate (RR). The results of hardware evaluation verified the feasibility of our acquisition systems, and ensured signal quality. Single-channel analysis results indicated that the channel located on the extensor carpi ulnaris (ECU) performed best with mean classification accuracy of 97.45% for all movement’s pairs. Channels placed on ECU and the extensor carpi radialis (ECR) were selected according to the accuracy rank. Experimental results showed that the proposed FD method was better than other feature selection methods and single-type features. The combination of FD and random forest (RF) performed best in offline analysis, with 96.77% multi-class RR. Online results illustrated that the state-machine paradigm with a 125 ms window had the highest maneuverability and was closest to real-life control. Subjects could accomplish online sessions by three sEMG-based paradigms, with average times of 46.02, 49.06 and 48.08 s, respectively. These experiments validate the feasibility of proposed real-time wearable HCI system and algorithms, providing a potential assistive device interface for persons with disabilities.

Sensitivity to Social Contingency in Adults with High-Functioning Autism during Computer-Mediated Embodied Interaction

Autism Spectrum Disorder (ASD) can be understood as a social interaction disorder. This makes the emerging "second-person approach" to social cognition a more promising framework for studying ASD than classical approaches focusing on mindreading capacities in detached, observer-based arrangements. According to the second-person approach, embodied, perceptual, and embedded or interactive capabilities are also required for understanding others, and these are hypothesized to be compromised in ASD. We therefore recorded the dynamics of real-time sensorimotor interaction in pairs of control participants and participants with High-Functioning Autism (HFA), using the minimalistic human-computer interface paradigm known as "perceptual crossing" (PC). We investigated whether HFA is associated with impaired detection of social contingency, i.e., a reduced sensitivity to the other's responsiveness to one's own behavior. Surprisingly, our analysis reveals that, at least under the conditions of this highly simplified, computer-mediated, embodied form of social interaction, people with HFA perform equally well as controls. This finding supports the increasing use of virtual reality interfaces for helping people with ASD to better compensate for their social disabilities. Further dynamical analyses are necessary for a better understanding of the mechanisms that are leading to the somewhat surprising results here obtained.

Human-Centered Design Study: Enhancing the Usability of a Mobile Phone App in an Integrated Falls Risk Detection System for Use by Older Adult Users

BACKGROUND

Design processes such as human-centered design (HCD), which involve the end user throughout the product development and testing process, can be crucial in ensuring that the product meets the needs and capabilities of the user, particularly in terms of safety and user experience. The structured and iterative nature of HCD can often conflict with the necessary rapid product development life-cycles associated with the competitive connected health industry.

OBJECTIVE

The aim of this study was to apply a structured HCD methodology to the development of a smartphone app that was to be used within a connected health fall risk detection system. Our methodology utilizes so called discount usability engineering techniques to minimize the burden on resources during development and maintain a rapid pace of development. This study will provide prospective designers a detailed description of the application of a HCD methodology.

METHODS

A 3-phase methodology was applied. In the first phase, a descriptive "use case" was developed by the system designers and analyzed by both expert stakeholders and end users. The use case described the use of the app and how various actors would interact with it and in what context. A working app prototype and a user manual were then developed based on this feedback and were subjected to a rigorous usability inspection. Further changes were made both to the interface and support documentation. The now advanced prototype was exposed to user testing by end users where further design recommendations were made.

RESULTS

With combined expert and end-user analysis of a comprehensive use case having originally identified 21 problems with the system interface, we have only seen and observed 3 of these problems in user testing, implying that 18 problems were eliminated between phase 1 and 3. Satisfactory ratings were obtained during validation testing by both experts and end users, and final testing by users shows the system requires low mental, physical, and temporal demands according to the NASA Task Load Index (NASA-TLX).

CONCLUSIONS

From our observation of older adults' interactions with smartphone interfaces, there were some recurring themes. Clear and relevant feedback as the user attempts to complete a task is critical. Feedback should include pop-ups, sound tones, color or texture changes, or icon changes to indicate that a function has been completed successfully, such as for the connection sequence. For text feedback, clear and unambiguous language should be used so as not to create anxiety, particularly when it comes to saving data. Warning tones or symbols, such as caution symbols or shrill tones, should only be used if absolutely necessary. Our HCD methodology, designed and implemented based on the principles of the International Standard Organizaton (ISO) 9241-210 standard, produced a functional app interface within a short production cycle, which is now suitable for use by older adults in long term clinical trials.

A Randomized Trial Comparing Classical Participatory Design to VandAID, an Interactive CrowdSourcing Platform to Facilitate User-centered Design

BACKGROUND

Early involvement of stakeholders in the design of medical software is particularly important due to the need to incorporate complex knowledge and actions associated with clinical work. Standard user-centered design methods include focus groups and participatory design sessions with individual stakeholders, which generally limit user involvement to a small number of individuals due to the significant time investments from designers and end users.

OBJECTIVES

The goal of this project was to reduce the effort for end users to participate in co-design of a software user interface by developing an interactive web-based crowdsourcing platform.

METHODS

In a randomized trial, we compared a new web-based crowdsourcing platform to standard participatory design sessions. We developed an interactive, modular platform that allows responsive remote customization and design feedback on a visual user interface based on user preferences. The responsive canvas is a dynamic HTML template that responds in real time to user preference selections. Upon completion, the design team can view the user's interface creations through an administrator portal and download the structured selections through a REDCap interface.

RESULTS

We have created a software platform that allows users to customize a user interface and see the results of that customization in real time, receiving immediate feedback on the impact of their design choices. Neonatal clinicians used the new platform to successfully design and customize a neonatal handoff tool. They received no specific instruction and yet were able to use the software easily and reported high usability.

CONCLUSIONS

VandAID, a new web-based crowdsourcing platform, can involve multiple users in user-centered design simultaneously and provides means of obtaining design feedback remotely. The software can provide design feedback at any stage in the design process, but it will be of greatest utility for specifying user requirements and evaluating iterative designs with multiple options.

Case study: Head orientation and neck electromyography for cursor control in persons with high cervical tetraplegia

We evaluated the ability of an individual with a high cervical spinal cord injury (SCI) to control a cursor on a computer screen using two different user interfaces: (1) head movements measured via a head-mounted orientation sensor and (2) electromyography (EMG) signals from four head and neck muscles acquired using a 4-channel implanted upper-limb neuroprosthesis that had been deployed in an earlier study. The subject moved the cursor to a set of targets on the screen in a two-dimensional, center-out, target-acquisition task, and his performance was evaluated with a variety of performance measures to assess both position and velocity control accuracy. The subject's performance with both command sources was also compared with the performance of a group of nondisabled subjects. Head orientation provided more accurate performance but was less responsive than EMG. Both command sources showed some directionally dependent performance, with movement to diagonally located targets being performed by a series of sequential motions rather than via straight paths. Overall, the SCI subject's performance with each command source was similar to that reported for a nondisabled population using the same interfaces and performing the same task.

A Framework for the Development of Context-Adaptable User Interfaces for Ubiquitous Computing Systems

This paper addresses the problem of developing user interfaces for Ubiquitous Computing (UC) and Ambient Intelligence (AmI) systems. These kind of systems are expected to provide a natural user experience, considering interaction modalities adapted to the user abilities and preferences and using whatever interaction devices are present in the environment. These interaction devices are not necessarily known at design time. The task is quite complicated due to the variety of devices and technologies, and the diversity of scenarios, and it usually burdens the developer with the need to create many different UIs in order to consider the foreseeable user-environment combinations. Here, we propose an UI abstraction framework for UC and AmI systems that effectively improves the portability of those systems between different environments and for different users. It allows developers to design and implement a single UI capable of being deployed with different devices and modalities regardless the physical location.

A Kinect-Based Sign Language Hand Gesture Recognition System for Hearing- and Speech-Impaired: A Pilot Study of Pakistani Sign Language

Sign language provides hearing and speech impaired individuals with an interface to communicate with other members of the society. Unfortunately, sign language is not understood by most of the common people. For this, a gadget based on image processing and pattern recognition can provide with a vital aid for detecting and translating sign language into a vocal language. This work presents a system for detecting and understanding the sign language gestures by a custom built software tool and later translating the gesture into a vocal language. For the purpose of recognizing a particular gesture, the system employs a Dynamic Time Warping (DTW) algorithm and an off-the-shelf software tool is employed for vocal language generation. Microsoft(®) Kinect is the primary tool used to capture video stream of a user. The proposed method is capable of successfully detecting gestures stored in the dictionary with an accuracy of 91%. The proposed system has the ability to define and add custom made gestures. Based on an experiment in which 10 individuals with impairments used the system to communicate with 5 people with no disability, 87% agreed that the system was useful.

Rehabilitation Program Integrating Virtual Environment to Improve Orientation and Mobility Skills for People Who Are Blind

This paper presents the integration of a virtual environment (BlindAid) in an orientation and mobility rehabilitation program as a training aid for people who are blind. BlindAid allows the users to interact with different virtual structures and objects through auditory and haptic feedback. This research explores if and how use of the BlindAid in conjunction with a rehabilitation program can help people who are blind train themselves in familiar and unfamiliar spaces. The study, focused on nine participants who were congenitally, adventitiously, and newly blind, during their orientation and mobility rehabilitation program at the Carroll Center for the Blind (Newton, Massachusetts, USA). The research was implemented using virtual environment (VE) exploration tasks and orientation tasks in virtual environments and real spaces. The methodology encompassed both qualitative and quantitative methods, including interviews, a questionnaire, videotape recording, and user computer logs. The results demonstrated that the BlindAid training gave participants additional time to explore the virtual environment systematically. Secondly, it helped elucidate several issues concerning the potential strengths of the BlindAid system as a training aid for orientation and mobility for both adults and teenagers who are congenitally, adventitiously, and newly blind.

An approach for representing sensor data to validate alerts in Ambient Assisted Living

The mainstream of research in Ambient Assisted Living (AAL) is devoted to developing intelligent systems for processing the data collected through artificial sensing. Besides, there are other elements that must be considered to foster the adoption of AAL solutions in real environments. In this paper we focus on the problem of designing interfaces among caregivers and AAL systems. We present an alert management tool that supports carers in their task of validating alarms raised by the system. It generates text-based explanations—obtained through an argumentation process—of the causes leading to alarm activation along with graphical sensor information and 3D models, thus offering complementary types of information. Moreover, a guideline to use the tool when validating alerts is also provided. Finally, the functionality of the proposed tool is demonstrated through two real cases of alert.

A robust Kalman algorithm to facilitate human-computer interaction for people with cerebral palsy, using a new interface based on inertial sensors

This work aims to create an advanced human-computer interface called ENLAZA for people with cerebral palsy (CP). Although there are computer-access solutions for disabled people in general, there are few evidences from motor disabled community (e.g., CP) using these alternative interfaces. The proposed interface is based on inertial sensors in order to characterize involuntary motion in terms of time, frequency and range of motion. This characterization is used to design a filtering technique that reduces the effect of involuntary motion on person-computer interaction. This paper presents a robust Kalman filter (RKF) design to facilitate fine motor control based on the previous characterization. The filter increases mouse pointer directivity and the target acquisition time is reduced by a factor of ten. The interface is validated with CP users who were unable to control the computer using other interfaces. The interface ENLAZA and the RKF enabled them to use the computer.

Evaluation of an architecture for intelligent query and exploration of time-oriented clinical data

OBJECTIVE

Evaluate KNAVE-II, a knowledge-based framework for visualization, interpretation, and exploration of longitudinal clinical data, clinical concepts and patterns. KNAVE-II mediates queries to a distributed temporal-abstraction architecture (IDAN), which uses a knowledge-based problem-solving method specializing in on-the-fly computation of clinical queries.

METHODS

A two-phase, balanced cross-over study to compare efficiency and satisfaction of a group of clinicians when answering queries of variable complexity about time-oriented clinical data, typical for oncology protocols, using KNAVE-II, versus standard methods: both paper charts and a popular electronic spreadsheet (ESS) in Phase I; an ESS in Phase II. The measurements included the time required to answer and the correctness of answer for each query and each complexity category, and for all queries, assessed versus a predetermined gold standard set by a domain expert. User satisfaction was assessed by the Standard Usability Score (SUS) tool-specific questionnaire and by a "Usability of Tool Comparison" comparative questionnaire developed for this study.

RESULTS

In both evaluations, subjects answered higher-complexity queries significantly faster using KNAVE-II than when using paper charts or an ESS up to a mean of 255 s difference per query versus the ESS for hard queries (p=0.0003) in the second evaluation. Average correctness scores when using KNAVE-II versus paper charts, in the first phase, and the ESS, in the second phase, were significantly higher over all queries. In the second evaluation, 91.6% (110/120) of all of the questions asked within queries of all levels produced correct answers using KNAVE-II, opposed to only 57.5% (69/120) using the ESS (p<0.0001). User satisfaction with KNAVE-II was significantly superior compared to using either a paper chart or the ESS (p=0.006). Clinicians ranked KNAVE-II superior to both paper and the ESS.

CONCLUSIONS

An evaluation of the functionality and usability of KNAVE-II and its supporting knowledge-based temporal-mediation architecture has produced highly encouraging results regarding saving of physician time, enhancement of accuracy of clinical assessment, and user satisfaction.