Does generative artificial intelligence pose a risk to performance validity test security?

OBJECTIVE

We examined the performance validity test (PVT) security risk presented by artificial intelligence (AI) chatbots asking questions about neuropsychological evaluation and PVTs on two popular generative AI sites.

METHOD

In 2023 and 2024, multiple questions were posed to ChatGPT-3 and Bard (now Gemini). One set started generally and refined follow-up questions based on AI responses. A second set asked how to feign, fake, or cheat. Responses were aggregated and independently rated for inaccuracy and threat. Responses not identified as inaccurate were assigned a four-level threat rating (no, mild, moderate, or high threat). Combined inaccuracy and threat ratings were examined cross-sectionally and longitudinally.

RESULTS

Combined inaccuracy rating percentages were 35 to 42% in 2023 and 16 to 28% in 2024. Combined moderate/high threat ratings were observed in 24 to 41% of responses in 2023 and in 17 to 31% of responses in 2024. More ChatGPT-3 responses were rated moderate or high threat compared to Bard/Gemini responses. Over time, ChatGPT-3 responses became more accurate with a similar threat level, but Bard/Gemini responses did not change in accuracy or threat. Responses to how to feign queries demonstrated ethical opposition to feigning. Responses to similar queries in 2024 showed even stronger ethical opposition.

CONCLUSIONS

AI chatbots are a threat to PVT test security. A proportion of responses were rated as moderate or high threat. Although ethical opposition to feigning guidance increased over time, the natural language interface and the volume of AI chatbot responses represent a potentially greater threat than traditional search engines.

Differentiating maximal and submaximal voluntary strength measures for the purposes of medico-legal assessments and para sport classification: A systematic review

Measurement of maximum voluntary muscle contractions are effort-dependent - valid measurement requires maximal voluntary effort (MVE) from participants. Submaximal efforts (SMEs) yield invalid and potentially misleading results. This is particularly problematic in medico-legal and Para sport assessments where low strength scores may confer a personal advantage. Therefore, objective methods for accurately differentiating MVE and SME are required. This systematic review aimed to identify, appraise and synthesise evidence from scientific studies evaluating the validity of objective methods for differentiating MVE from SME during maximal voluntary contractions. Four electronic databases were searched for original research articles published in English and secondary references appraised for relevance yielding 25 studies for review. Methods were categorised based on eight distinct underlying theories. For isokinetic strength assessment, methods based on two theories - Strength-measure Ratios and Inter-Trial Strength Consistency - correctly classified 100% MVE and > 92% SME. Consequently, research evaluating the relative suitability of these methods for translation into practice is warranted. During isometric strength assessments, methods based on Deceptive Visual Feedback and Force-length properties warrant further investigation. Both methods yielded statistically significant differences between MVE and SME, with minimal overlap in values, but their sensitivity and specificity have not been evaluated.