Development and Validity of Western University's On-Road Assessment

Performance Analysis of Driving Ability (P-Drive): Investigating Construct Validity and Concordance of Australasian Data

On-road assessment is optimal for determining medical fitness-to-drive but unreliable if determined by global pass/fail decisions alone. Occupational therapists need standardized, psychometrically sound on-road scoring procedures. Performance Analysis of Driving Ability (P-Drive) is a promising on-road test developed in Sweden, but it has not been standardized for Australasia. We investigated the psychometric properties and concordance with an on-road decision of data gathered with the Australasian version of P-Drive. P-Drive was administered to older and cognitively impaired drivers (N = 134) aged 18 to 91 years (mean age 68) who were referred to 10 driving clinics in Australia and New Zealand to determine driving performance. Rasch analysis provided evidence for construct validity and concordance of the data gathered. An optimal cut-off score of 85 was set, yielding evidence of good sensitivity at 88% and specificity at 88%. The Australasian version of P-Drive produces valid and reliable data regarding on-road driving performance.

Driving Errors Predicting Pass/Fail On-Road Assessment Outcomes Among Cognitively Impaired Older Drivers

Older drivers with cognitive impairment (CI)/dementia make significantly more driving errors than healthy controls; however, whether driving errors are predictive of pass/fail outcomes in older drivers with CI/dementia are unclear. This study determined the driving errors that predicted failing an on-road assessment in drivers with CI. We retrospectively collected comprehensive driving evaluation data of 80 participants (76.1 ± 9.3 years) from an Ontario driving assessment center. Adjustment to stimuli (area under the curve [AUC] = 0.88), lane maintenance (AUC = 0.84), and speed regulation errors (AUC = 0.85) strongly predicted pass/fail outcomes. Worse performance on the Trails B (time) and Useful Field of View® (Subtest 2, Subtest 3, and risk index) were significantly correlated with adjustment to stimuli (p < .05), lane maintenance (p < .05), and speed regulation errors (p < .05). Adjustment to stimuli, lane maintenance, and speed regulation errors may be critical indicators of failing an on-road assessment in older drivers with CI. Prioritizing these errors may help identify at-risk drivers.

Feasibility of automated in-vehicle technologies on volunteers' driving performance

BACKGROUND

Automated in-vehicle technologies, specifically in-vehicle information systems (IVIS) and advanced driver assistance systems (ADAS), are increasingly common in today's cars. Previous studies illustrate benefits of using IVIS and ADAS to improve safety, convenience, and comfort in healthy older drivers. However, research is sparse on the feasibility of such technologies for medically at-risk drivers, such as those with Parkinson's disease (PD).

OBJECTIVE

This study enrolled healthy volunteer drivers to examine the feasibility of the procedures and measures for evaluating the effects of IVIS and ADAS on their driving performance.

METHODS

During this feasibility study researchers compared drives completed with and without support of IVIS and ADAS, as participants drove a 2019 Toyota Camry XLE. The test vehicle was equipped with IVIS, ADAS, cameras, a telematics system, and sensors. Participants drove the road course supervised by a Driver Rehabilitation Specialist (DRS).

RESULTS

Overall study procedures and vehicle equipment were feasible and provided sufficient data collection for measuring the impact of IVIS and ADAS on driving performance. Data observation by the DRS combined with data captured from cameras and telematics, facilitated comparisons to increase data reliability and validity.

CONCLUSIONS

Feasibility study findings informed a randomized clinical trial, examining the use of IVIS and ADAS technologies as an intervention to support drivers with Parkinson's disease.

RCT protocol for driving performance in people with Parkinson's using autonomous in-vehicle technologies

Introduction

Driving is an essential facilitator of independence, community participation, and quality of life. Drivers with Parkinson's Disease (PD) make more driving errors and fail on-road evaluations more than healthy controls. In-vehicle technologies may mitigate PD-related driving impairments and associated driving errors. Establishing a rigorous study protocol will increase the internal validity and the transparency of the scientific work.

Methods

We present a protocol to assess the efficacy of autonomous in-vehicle technologies (Level 1) on the driving performance of drivers with PD via a randomized crossover design with random allocation. Drivers with a PD diagnosis based on established clinical criteria (N = 105), referred by neurologists, are exposed to two driving conditions (technology activated or not) on a standardized road course as they drove a 2019 Toyota Camry. The researchers collected demographic, clinical, on-road data observational and kinematic, and video data to understand several primary outcome variables, i.e., number of speeding, lane maintenance, signaling, and total driving errors.

Discussion

The protocol may enhance participant adherence, decrease attrition, provide early and accurate identification of eligible participants, ensure data integrity, and improve the study flow. One limitation is that the protocol may change due to unforeseen circumstances and assumptions upon implementation. A strength is that the protocol ensures the study team executes the planned research in a systematic and consistent way.

Following, adapting, and refining the protocol will enhance the scientific investigation to quantify the nuances of driving among those with PD in the era of automated in-vehicle technologies.

Trial registration

ClinicalTrials.gov NCT04660500.

Development of a weighted scoring system for the Electronic Driving Observation Schedule (eDOS)

The electronic Driving Observation Schedule (eDOS) is a novel approach to assessing older drivers' performance in their everyday driving environment on their chosen routes. The original eDOS total score is generated using the count of driving errors, which does not account for distinct risk levels of different types of driving errors made in different complexity of driving environments. This study was conducted to create one score to represent the complexity of driving route during each eDOS observation and one weighted eDOS total score to represent older drivers' performance accounting for the risk of driving errors by their type and the complexity of maneuvers in their corresponding environments. A literature review, a two-round survey with 13 experts in driving evaluation, and iterative discussions between primary investigators were conducted for generating these scores. Two formulae were created to calculate a weighted maneuver/environmental complexity score and a weighted eDOS total score. •An advanced weighted score is created to represent one's on-road driving performance in their everyday driving environment not only using the count of driving errors, but also accounting for the risk level of each error.•The complexity of driving maneuver and environment in each on-road driving trip can be systematically rated.

Standardized on-road tests assessing fitness-to-drive in people with cognitive impairments: A systematic review

Objective

The on-road assessment is the gold standard because of its ecological validity. Yet existing instruments are heterogeneous and little is known about their psychometric properties. This study identified existing on-road assessment instruments and extracted data on psychometric properties and usability in clinical settings.

Method

A systematic review identified studies evaluating standardized on-road evaluation instruments adapted for people with cognitive impairment. Published articles were searched on PubMed, CINHAL, PsycINFO, Web of Science, and ScienceDirect. Study quality and the level of evidence were assessed using the COSMIN checklist. The collected data were synthetized using a narrative approach. Usability was subjectively assessed for each instrument by extracting information on acceptability, access, cost, and training.

Results

The review identified 18 published studies between 1994 and 2016 that investigated 12 different on-road evaluation instruments: the Performance-Based Driving Evaluation, the Washington University Road Test, the New Haven, the Test Ride for Practical Fitness to Drive, the Rhode Island Road Test, the Sum of Manoeuvres Score, the Performance Analysis of Driving Ability, the Composite Driving Assessment Scale, the Nottingham Neurological Driving Assessment, the Driving Observation Schedule, the Record of Driving Errors, and the Western University’s On-road Assessment. Participants were mainly male (64%), between 48 and 80 years old, and had a broad variety of cognitive disorders. Most instruments showed reasonable psychometric values for internal consistency, criterion validity, and reliability. However, the level of evidence was poor to support any of the instruments given the low number of studies for each.

Conclusion

Despite the social and health consequences of decisions taken using these instruments, little is known about the value of a single evaluation and the ability of instruments to identify expected changes. None of the identified on-road evaluation instruments seem currently adapted for clinical settings targeting rehabilitation and occupational priorities rather than road security alone.

Study registration

PROSPERO registration number CRD42018103276.

Reliability and validity of on-road driving tests in vulnerable adults: a systematic review

The on-road driving test is considered a ‘gold standard’ evaluation; however, its validity and reliability have not been sufficiently reviewed. This systematic review aimed to map out and synthesize literature regarding on-road driving tests using the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. Cochrane Library, PubMed, CINAHL, and Web of Science databases were searched from initiation through February 2018. All articles addressing reliability or validity of on-road driving tests involving adult rehabilitation patients were included. The search output identified 513 studies and 36 articles, which were included in the review. The Washington University Road Test/Rhode Island Road Test, performance analysis of driving ability, test ride for investigating practical fitness-to-drive, and K-score demonstrated high reliability and validity in regard to the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. The Washington University Road Test/Rhode Island Road Test and test ride for investigating practical fitness-to-drive were analyzed based on Classical Test Theory techniques, and performance analysis of driving ability and K-score were analyzed based on Item Response Theory techniques. The frequency of studies were Washington University Road Test/Rhode Island Road Test (n=9), Test Ride for Investigating Practical fitness-to-drive (n=8), performance analysis of driving ability (n=4), and K-score (n=1). From the viewpoint of accuracy and generalization, the Washington University Road Test/Rhode Island Road Test, test ride for investigating practical fitness-to-drive, and performance analysis of driving ability were identified as highly qualified concerning on-road driving tests. However, the ability to assess real-world driving depends on various environmental conditions.

Insufficient sleep and fitness to drive in shift workers: A systematic literature review

BACKGROUND

Insufficient sleep, <6.5 h per night, majorly affects shift workers, placing them at higher risk for motor vehicle crash related injury or fatality. While systematic reviews (SLRs) examine the effects of insufficient sleep and driving, to date, no SLR focuses on driver fitness or performance in shift workers.

OBJECTIVES

Determine the class of evidence (Class I-highest to Class IV-lowest), and level of confidence (Level A-high, to Level U-insufficient) in the determinants of driver fitness and performance in shift workers. Next, consider evidence-based recommendations for clinical practice, research, and policy.

METHODS

A protocol was registered on PROSPERO (#CRD42018052905) using an established SLR methodology: a comprehensive electronic database search, study selection, data extraction, critical appraisal, analysis, and interpretation using published guidelines.

RESULTS

Searches identified 1226 unique records with 11(2 on-road, 9 simulator) meeting final inclusion criteria. Class III to IV evidence identified that exposure to overnight shift work possibly predicts (Level C confidence) drivers at risk for adverse on-road outcomes and likely predicts (Level B) drivers at risk for adverse driving simulator outcomes. Higher ratings of subjective sleepiness and extended time driving possibly predict (Level C) drivers at risk for adverse driving simulator outcomes.

CONCLUSIONS

This study demonstrates a low to moderate level of confidence in the determinants of driving in shift workers. A critical need exists for gold-standard on-road assessments integrating complex driving environments representative of real-world demands, targeting tactical and strategic outcomes in a broad spectrum of shift workers.

Insufficient sleep and fitness to drive in shift workers: a systematic literature review protocol

BACKGROUND

The majority of shift workers experience insufficient sleep as a result of their employment. Insufficient sleep is associated with impaired neurocognitive functioning, affecting key skills required for driving, resulting in shift workers experiencing a disproportionate burden of RTC injuries and fatalities. Yet, to our knowledge, no systematic literature review (SLR) exists to critically appraise and synthesise evidence on the determinants of fitness to drive (assessed on-road) and driving performance (assessed in a driving simulator) in shift workers with insufficient sleep.

OBJECTIVES

A SLR protocol is established to conduct analysis and synthesis of the level of evidence and confidence in the determinants of fitness to drive and driving performance, among shift workers with insufficient sleep.

METHODS

This study follows Cooper and Hedges' established SLR methodology: formulate the problem, locate and select studies, collect data, appraise critically, analyse and present data, interpret results and disseminate information. Critical appraisal and analysis follows the 2017 American Academy of Neurology guidelines determining the level of evidence and the level of confidence for each determinant identified in the literature. Protocol and results reporting adhere to the Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocols guidelines.

CONCLUSIONS

This SLR contributes to research evidence examining the impact of insufficient sleep and driver sleepiness on fitness to drive and driving performance. Analysis of the level of evidence and level of confidence in the existing literature will advance evidence-informed prevention strategies and critical decision-making, to mitigate adverse effects of insufficient sleep for improving road safety.

On-road assessment of fitness-to-drive in persons with MS with cognitive impairment: A prospective study

Background:

Cognitive impairment is common in multiple sclerosis (MS). In other populations, cognitive impairment is known to affect fitness-to-drive. Few studies have focused on fitness-to-drive in MS and no studies have solely focused on the influence of cognitive impairment.

Objective:

To assess fitness-to-drive in persons with MS with cognitive impairment and low physical disability.

Methods:

Persons with MS, aged 18–59 years with EDSS ⩽ 4.0, impaired processing speed, and impairment on at least one measure of memory or executive function, were recruited. Cognition was assessed using the Minimal Assessment of Cognitive Function battery. A formal on-road driving assessment was conducted. Chi-square analysis examined the association between the fitness-to-drive (pass/fail) and the neuropsychological test results (normal/impaired). Bayesian statistics predicting failure of the on-road assessment were calculated.

Results:

Of 36 subjects, eight (22.2%) were unfit to drive. Only the BVMTR-IR, measuring visual-spatial memory, predicted on-road driving assessment failure ( X2 ( df = 1, N = 36) = 3.956; p = 0.047) with a sensitivity of 100%, but low specificity (35.7%) due to false positives (18/25).

Conclusion:

In persons with MS and impaired processing speed, impairment on the BVMTR-IR should lead clinicians to address fitness-to-drive.

Driving Errors That Predict On-Road Outcomes in Adults With Multiple Sclerosis

Driving errors that predict on-road outcomes for persons with multiple sclerosis (PwMS) are not well studied. The objective of this study was to determine whether adjustment-to-stimuli and gap acceptance errors significantly predict passing/failing a standardized on-road assessment of PwMS. Thirty-seven participants completed visual ability and visual attention assessments, and participated in an on-road assessment, where seven types of driving errors and pass/fail outcomes were determined. Adjustment-to-stimuli (No.) and gap acceptance errors (commit/did not commit) significantly predicted passing/failing the on-road assessment, with an area under the curve of 91.6% ( p < .0001). With no gap acceptance errors committed, five adjustment-to-stimuli errors optimally determined pass/fail outcomes in PwMS. Furthermore, with no adjustment to stimuli errors committed, committing any gap acceptance errors also optimally determined pass/fail outcomes in PwMS. Further research may focus on visual, cognitive, and/or motor impairments underlying adjustment-to-stimuli and gap acceptance errors for eventual development of rehabilitation strategies for PwMS.