Objective measurement of attention deficit hyperactivity disorder symptoms outside the clinic using the QbCheck: Reliability and validity

Practitioner Review: Clinical utility of the QbTest for the assessment and diagnosis of attention-deficit/hyperactivity disorder - a systematic review and meta-analysis

BACKGROUND

Several computerised cognitive tests (e.g. continuous performance test) have been developed to support the clinical assessment of attention-deficit/hyperactivity disorder (ADHD). Here, we appraised the evidence-base underpinning the use of one of these tests - the QbTest - in clinical practice, by conducting a systematic review and meta-analysis investigating its accuracy and clinical utility.

METHODS

Based on a preregistered protocol (CRD42022377671), we searched PubMed, Medline, Ovid Embase, APA PsycINFO and Web of Science on 15th August 2022, with no language/type of document restrictions. We included studies reporting accuracy measures (e.g. sensitivity, specificity, or Area under the Receiver Operating Characteristics Curve, AUC) for QbTest in discriminating between people with and without DSM/ICD ADHD diagnosis. Risk of bias was assessed with the Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2). A generic inverse variance meta-analysis was conducted on AUC scores. Pooled sensitivity and specificity were calculated using a random-effects bivariate model in R.

RESULTS

We included 15 studies (2,058 participants; 48.6% with ADHD). QbTest Total scores showed acceptable, rather than good, sensitivity (0.78 [95% confidence interval: 0.69; 0.85]) and specificity (0.70 [0.57; 0.81]), while subscales showed low-to-moderate sensitivity (ranging from 0.48 [0.35; 0.61] to 0.65 [0.52; 0.75]) and moderate-to-good specificity (from 0.65 [0.48; 0.78] to 0.83 [0.60; 0.94]). Pooled AUC scores suggested moderate-to-acceptable discriminative ability (Q-Total: 0.72 [0.57; 0.87]; Q-Activity: 0.67 [0.58; 0.77); Q-Inattention: 0.66 [0.59; 0.72]; Q-Impulsivity: 0.59 [0.53; 0.64]).

CONCLUSIONS

When used on their own, QbTest scores available to clinicians are not sufficiently accurate in discriminating between ADHD and non-ADHD clinical cases. Therefore, the QbTest should not be used as stand-alone screening or diagnostic tool, or as a triage system for accepting individuals on the waiting-list for clinical services. However, when used as an adjunct to support a full clinical assessment, QbTest can produce efficiencies in the assessment pathway and reduce the time to diagnosis.

Objective Parameters in Attention Deficit Hyperactivity Disorder: Eye and Head Movements

OBJECTIVE

We aimed to evaluate eye and head movements, which are objective parameters in ADHD. Method: While the children were watching the course video task, which included the relevant (teacher and smart board) and irrelevant (any regions outside the relevant area) areas of interest, their eye movements were evaluated through eye tracking, and video recordings were made simultaneous. Head position estimation was made using through video recordings. The proportion of total fixation duration on areas of interest (PFDAOI) and saccade count, amplitude, velocity for eye movements, number of total head movements and angular change of head movement in x-y-z axes for head movements were compared.

RESULTS

Children with ADHD had lower PFDAOI on the relevant area, and had more saccade and head movements The angular change of head movement in the x-axis was higher in the ADHD group.

CONCLUSION

In the assessment of ADHD, the eye and head movements may be particulary useful.

Effects of one single-dose methylphenidate compared to one single-dose placebo on QbTest performance in adults with untreated ADHD: a randomized controlled trial

BACKGROUND

Treatment of adults with attention-deficit/hyperactivity disorder (ADHD) primary involves methylphenidate (MPH). Earlier studies have identified placebo responders to increase toward the end of the treatment periods. However, little is known about the immediate effects of placebo on the core symptoms of ADHD in adults. The present study aimed to examine the effects of one single-dose MPH compared to one single-dose placebo during clinical assessments with continuous performance tests (CPT).

METHODS

In a randomized study with cross-over design, 40 adults between 19 and 64 years (72.5% women) with untreated ADHD were consecutively enrolled. The study comprised two trial days with four days in between. The QbTest was performed twice on the same day, before and 80 min after intake of one single-dose 20 mg immediate release methylphenidate (IR-MPH) and with one single-dose placebo, in randomized order.

RESULTS

Performance improved in QbInattention, F (3, 117) = 38.25, p < 0.001, after given IR-MPH (mean diff = 1.14) and after placebo (mean diff = 0.60) with the effect sizes 1.17 and 0.63 respectively. IR-MPH improved performance in QbActivity (mean diff = 0.81, p < 0.001) and QbImpulsivity (mean diff = 0.46, p < 0.04). The proportion of improvements (a decrease by ≥ 0.5 Qb-score) in the parameters QbInattention, QbActivity and QbImpulsivity were 90%, 60% and 52.5%, respectively. After given placebo, corresponding proportions were 60%, 30% and 35%, respectively.

CONCLUSIONS

There seems to be an immediate placebo response in the core symptom inattention. The effect of placebo cannot be ruled out and must be taken in consideration during drug trials with continuous performance tests (CPTs).

TRIAL REGISTRATION

ClinicalTrials.gov; Identifier: NCT02473185.

The Structure and Diagnostic Accuracy of the QbTest in Pediatric ADHD: A Retrospective Clinical Study

OBJECTIVE

The QbTest that combines a continuous performance task (CPT) with a motion-tracking system may help identify attention deficit hyperactivity disorder (ADHD). This study examined the structure and diagnostic ability of the QbTest in children and adolescents.

METHOD

Retrospective data from 1,274 children and adolescents were analyzed. The study assessed data on a principal component analysis (PCA), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

The QbActivity component included the variables micro-events, distance, area, and time active; the QbImpulsivity included normalized commissions and commissions (anticipatory errors were added in a version for 6-12-year-olds only); and the QbInattention included omissions, reaction time, and reaction time variation. Sensitivity ranged between 22% and 50%, specificity 79% and 96%, PPVs 40% and 95%, and NPVs 24% and 66%.

CONCLUSION

The structure of the QbTest with three cardinal parameters and nine/ten CPT and motion analysis variables was supported. The diagnostic accuracy was found to be poor to moderate. Given that this is a retrospective study, the interpretation of diagnostic accuracy should be considered within this context.

It Takes Time: Vigilance and Sustained Attention Assessment in Adults with ADHD

Objectives: The present study compares the utility of eight different tests of vigilance and sustained attention in the neuropsychological examination of adults with Attention-deficit/hyperactivity disorder (ADHD). Methods: Thirty-one adults diagnosed with ADHD performed eight tests for vigilance and sustained attention, spread over three assessment days. Results: Adults with ADHD showed cognitive impairments in most tests and test variables, even though their sensitivity differed greatly. No specific type of test variable stands out to be the most sensitive, and no evidence for a differential deterioration of performance over time was observed. Conclusion: This study underscores the role of vigilance and sustained attention tests in the assessment of adult ADHD. It is further concluded that summary scores over the entire test duration are sufficient, but that all variables of a test should be considered. Finally, we hypothesize that reassessment on a different day may benefit a more accurate clinical assessment of adults with ADHD, in order to adequately take intraindividual fluctuations and limitations regarding test reliability into account.

A review of objective assessments for hyperactivity in attention deficit hyperactivity disorder

Hyperactivity is one of the three core symptoms of attention deficit hyperactivity disorder (ADHD) that is a common childhood mental disorder. Objective assessments of hyperactivity are seldom utilized compared with measures of inattention and impulsivity during clinical diagnosis and evaluation. Acceleration-sensitive devices (e.g., Actigraph) and motion tracking systems (e.g., QbTest) are two main groups of devices that can be used to objectively measure hyperactivity. The Actigraph and QbTest have good discriminant validity, convergent validity, and sensitivity to the effects of stimulants. Furthermore, the assessment setting (i.e., research laboratory, school, or home) can greatly influence the presence and severity of hyperactivity. Nevertheless, objective assessments for hyperactivity have poor ability to distinguish ADHD from other disorders, or among the three types of ADHD. Thus, further studies are needed to assess objective measurements of hyperactivity in terms of discriminant and convergent validity, test-retest reliability in different settings, and correlations with brain activity.

Movement Patterns in Students Diagnosed with ADHD, Objective Measurement in a Natural Learning Environment

Attention deficit hyperactivity disorder is the most common neuropsychological disorder in childhood and adolescence, affecting the basic psychological processes involved in learning, social adaptation and affective adjustment. From previous research, the disorder is linked to problems in different areas of development, with deficiencies in psychological processes leading to the development of the most common characteristics of the disorder such as inattention, excess of activity and lack of inhibitory control. As for the diagnosis, in spite of being a very frequent disorder, there are multiple controversies about which tools are the most suitable for evaluation. One of the most widespread tools in the professional field is behavior inventories such as the Strengths and Difficulties Questionnaires for Parents and Teachers or the ADHD Rating Scale-V. The main disadvantage of these assessment tools is that they do not provide an objective observation. For this reason, there are different studies focused on recording objective measures of the subjects' movement, since hyperkinesia is one of the most characteristic symptoms of this disorder. In this sense, we have developed an application that, using a Kinect device, is capable of measuring the movement of the different parts of the body of up to six subjects in the classroom, being a natural context for the student. The main objective of this work is twofold, on the one hand, to investigate whether there are correlations between excessive movement and high scores in the inventories for the diagnosis of ADHD, Rating Scale-V and Strengths and Difficulties Questionnaire (SDQ) and, on the other hand, to determine which sections of the body present the most significant mobility in subjects diagnosed with ADHD. Results show that the control group, composed of neurotypical subjects, presents less kinaesthetic activity than the clinical group diagnosed with ADHD. This indicates that the experimental group presents one of the main characteristics of the disorder. In addition, results also show that practically all the measured body parts present significant differences, being higher in the clinical group, highlighting the head as the joint with the highest effect size.

The Impact of Methylphenidate on QbTest Performance of Children with ADHD: A Retrospective Clinical Study

INTRODUCTION

The Quantified behavior Test (QbTest), which combines a continuous performance task (CPT) and motion tracking, provides data for the core signs of attention-deficit/hyperactivity disorder (ADHD): attention, hyperactivity, and impulsivity. This study aimed to evaluate the performance of children and adolescents with ADHD on the QbTest before and after a single methylphenidate (MPH) dose.

SUBJECTS AND METHODS

This retrospective chart review study included data from 149 children and 215 adolescents who completed the QbTest. A summary index of the CPT and motion capture data on the QbTest is provided by three cardinal parameters: QbActivity, QbImpulsivity, and QbInattention. The test was performed twice on the same day, before and up to three hours after MPH intake. A decrease by ≥ 0.5 in a cardinal parameter score was considered an improvement, whereas an increase by ≥ 0.5 a deterioration.

RESULTS

QbActivity improvement after MPH intake was present in 71.7% and 76.2% of the children and adolescents, respectively. QbImpulsivity improvement was observed in 50.4% of the children and 44.7% of the adolescents, and QbInattention improvement in 85.1% and 91.1% of the children and adolescents, respectively. All three parameters improved simultaneously in 27.7% of the children and 28.7% of the adolescents. The likelihood that one parameter deteriorated after MPH use was greater if that parameter was within the normal range before medication. This was most pronounced for QbImpulsivity. Among male adolescents, QbInattention improvement was often accompanied by QbImpulsivity deterioration.

CONCLUSION

The QbTest inattention and motor activity parameters improved markedly after a single MPH dose in children and adolescents with ADHD, while less so for impulsivity. Improvement of one parameter is not necessarily associated with improvement of the other two, and deterioration, especially regarding impulsivity, may occur. If confirmed, these results highlight the need for optimization and individualization of MPH treatment, while monitoring all aspects of the ADHD symptomatology based on the QbTest performance.

The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database

At the beginning of the artificial intelligence (AI)/machine learning (ML) era, the expectations are high, and experts foresee that AI/ML shows potential for diagnosing, managing and treating a wide variety of medical conditions. However, the obstacles for implementation of AI/ML in daily clinical practice are numerous, especially regarding the regulation of these technologies. Therefore, we provide an insight into the currently available AI/ML-based medical devices and algorithms that have been approved by the US Food & Drugs Administration (FDA). We aimed to raise awareness of the importance of regulatory bodies, clearly stating whether a medical device is AI/ML based or not. Cross-checking and validating all approvals, we identified 64 AI/ML based, FDA approved medical devices and algorithms. Out of those, only 29 (45%) mentioned any AI/ML-related expressions in the official FDA announcement. The majority (85.9%) was approved by the FDA with a 510(k) clearance, while 8 (12.5%) received de novo pathway clearance and one (1.6%) premarket approval (PMA) clearance. Most of these technologies, notably 30 (46.9%), 16 (25.0%), and 10 (15.6%) were developed for the fields of Radiology, Cardiology and Internal Medicine/General Practice respectively. We have launched the first comprehensive and open access database of strictly AI/ML-based medical technologies that have been approved by the FDA. The database will be constantly updated.

Objective measurement of attention deficit hyperactivity disorder symptoms outside the clinic using the QbCheck: Reliability and validity

Objective measurements of ADHD symptom levels can be a highly valuable complement to ratings. However, sometimes it is not feasible to bring patients into the clinic/lab for assessment. The aim of the present study was therefore to evaluate the psychometric properties of the QbCheck, an online computerized test that measures errors and reaction time as well as activity during testing using the computer's built-in web camera. Study I (n = 27 adolescents/adults) investigated test-retest reliability and concurrent validity of the QbCheck. Study II included 142 adolescents/adults (69 with ADHD/73 controls) and investigated convergent and diagnostic validity, as well as usability, of the QbCheck. In Study I, the QbCheck showed high test-retest reliability and high concurrent validity. In Study II, high convergent validity was observed when studying associations between the QbCheck performed in the home and the QbTest performed at the clinic. In addition, the QbCheck discriminated well between patients with ADHD and controls, with a sensitivity of 82.6 and a specificity of 79.5. The QbCheck appears to be a valuable test with good psychometric properties and will thereby enable assessment of ADHD symptom levels in adolescents and adults outside the clinic in the home setting.