1
|
Clayson PE. The psychometric upgrade psychophysiology needs. Psychophysiology 2024; 61:e14522. [PMID: 38228400 PMCID: PMC10922751 DOI: 10.1111/psyp.14522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 12/13/2023] [Accepted: 12/19/2023] [Indexed: 01/18/2024]
Abstract
Although biological measurements are constrained by the same fundamental psychometric principles as self-report measurements, these essential principles are often neglected in most fields of neuroscience, including psychophysiology. Potential reasons for this neglect could include a lack of understanding of appropriate measurement theory or a lack of accessible software for psychometric analysis. Generalizability theory is a flexible and multifaceted measurement theory that is well suited to handling the nuances of psychophysiological data, such as the often unbalanced number of trials and intraindividual variability of scores of event-related brain potential (ERP) data. The ERP Reliability Analysis Toolbox (ERA Toolbox) was designed for psychophysiologists and is tractable software that can support the routine evaluation of psychometrics using generalizability theory. Psychometrics can guide task refinement, data-processing decisions, and selection of candidate biomarkers for clinical trials. The present review provides an extensive treatment of additional psychometric characteristics relevant to studies of psychophysiology, including validity and validation, standardization, dimensionality, and measurement invariance. Although the review focuses on ERPs, the discussion applies broadly to psychophysiological measures and beyond. The tools needed to rigorously assess psychometric reliability and validate psychophysiological measures are now readily available. With the profound implications that psychophysiological research can have on understanding brain-behavior relationships and the identification of biomarkers, there is simply too much at stake to ignore the crucial processes of evaluating psychometric reliability and validity.
Collapse
Affiliation(s)
- Peter E. Clayson
- Department of Psychology, University of South Florida, Tampa, FL, USA
| |
Collapse
|
2
|
Clayson PE. Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology. Int J Psychophysiol 2024; 197:112311. [PMID: 38296000 DOI: 10.1016/j.ijpsycho.2024.112311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 01/02/2024] [Accepted: 01/24/2024] [Indexed: 02/10/2024]
Abstract
Psychophysiological research is an inherently complex undertaking due to the nature of the data, and its analysis is characterized by many decision points that shape the final dataset and a study's findings. These decisions create a "multiverse" of possible outcomes, and each decision from study conceptualization to statistical analysis can lead to different results and interpretations. This review describes the concept of multiverse analyses, a methodological approach designed to understand the impact of different decisions on the robustness of a study's findings and interpretation. The emphasis is on transparently showcasing different reasonable approaches for constructing a final dataset and on highlighting the influence of various decision points, from experimental design to data processing and outcome selection. For example, the choice of an experimental task can significantly impact event-related brain potential (ERP) scores or skin conductance responses (SCRs), and different tasks might elicit unique variances in each measure. This review underscores the importance of transparently embracing the flexibility inherent in psychophysiological research and the potential consequences of not understanding the fragility or robustness of experimental findings. By navigating the intricate terrain of the psychophysiological multiverse, this review serves as an introduction, helping researchers to make informed decisions, improve the collective understanding of psychophysiological findings, and push the boundaries of the field.
Collapse
Affiliation(s)
- Peter E Clayson
- Department of Psychology, University of South Florida, Tampa, FL, USA.
| |
Collapse
|
3
|
Xu XJ, Liu TL, He L, Pu B. Changes in neurotransmitter levels, brain structural characteristics, and their correlation with PANSS scores in patients with first-episode schizophrenia. World J Clin Cases 2023; 11:5215-5223. [PMID: 37621579 PMCID: PMC10445057 DOI: 10.12998/wjcc.v11.i22.5215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 06/15/2023] [Accepted: 07/10/2023] [Indexed: 08/04/2023] Open
Abstract
BACKGROUND In patients with schizophrenia, the brain structure and neurotransmitter levels change, which may be related to the occurrence and progression of this disease. AIM To explore the relationships between changes in neurotransmitters, brain structural characteristics, and the scores of the Positive and Negative Symptom Scale (PANSS) in patients with first-episode schizophrenia. METHODS The case group comprised 97 patients with schizophrenia, who were evaluated using the Canadian Neurological Scale and confirmed by laboratory tests at Ningbo Mental Hospital from January 2020 to July 2022. The control group comprised 100 healthy participants. For all participants, brain structural characteristics were explored by measuring brain dopamine (DA), glutamic acid (Glu), and gamma-aminobutyric acid (GABA) levels, with magnetic resonance imaging. The case group was divided into negative and positive symptom subgroups using PANSS scores for hierarchical analysis. Linear correlation analysis was used to analyze the correlations between neurotransmitters, brain structural characteristics, and PANSS scores. RESULTS Patients in the case group had higher levels of DA and lower levels of Glu and GABA, greater vertical and horizontal distances between the corpus callosum and the inferior part of the fornix and larger ventricle area than patients in the control group (P < 0.05). Patients with positive schizophrenia symptoms had significantly higher levels of DA, Glu, and GABA than those with negative symptoms (P < 0.05). In patients with positive schizophrenia symptoms, PANSS score was significantly positively correlated with DA, vertical and horizontal distances between the corpus callosum and the infrafornix, and ventricular area, and was significantly negatively correlated with Glu and GABA (P < 0.05). In patients with negative schizophrenia symptoms, PANSS score was significantly positively correlated with DA, vertical distance between the corpus callosum and the infrafornix, horizontal distance between the corpus callosum and the infrafornix, and ventricular area, and was significantly negatively correlated with Glu and GABA (P < 0.05). CONCLUSION In patients with first-episode schizophrenia, DA levels increased, Glu and GABA levels decreased, the thickness of the corpus callosum increased, and these variables were correlated with PANSS scores.
Collapse
Affiliation(s)
- Xian-Jia Xu
- The Fifth Ward, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| | - Tang-Long Liu
- Department of Science and Education, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| | - Liang He
- The Sixteenth Ward, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| | - Ben Pu
- The Twelfth Ward, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| |
Collapse
|
4
|
Xu XJ, Liu TL, He L, Pu B. Changes in neurotransmitter levels, brain structural characteristics, and their correlation with PANSS scores in patients with first-episode schizophrenia. World J Clin Cases 2023; 11:5209-5217. [DOI: 10.12998/wjcc.v11.i22.5209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 06/15/2023] [Accepted: 07/10/2023] [Indexed: 08/03/2023] Open
Abstract
BACKGROUND In patients with schizophrenia, the brain structure and neurotransmitter levels change, which may be related to the occurrence and progression of this disease.
AIM To explore the relationships between changes in neurotransmitters, brain structural characteristics, and the scores of the Positive and Negative Symptom Scale (PANSS) in patients with first-episode schizophrenia.
METHODS The case group comprised 97 patients with schizophrenia, who were evaluated using the Canadian Neurological Scale and confirmed by laboratory tests at Ningbo Mental Hospital from January 2020 to July 2022. The control group comprised 100 healthy participants. For all participants, brain structural characteristics were explored by measuring brain dopamine (DA), glutamic acid (Glu), and gamma-aminobutyric acid (GABA) levels, with magnetic resonance imaging. The case group was divided into negative and positive symptom subgroups using PANSS scores for hierarchical analysis. Linear correlation analysis was used to analyze the correlations between neurotransmitters, brain structural characteristics, and PANSS scores.
RESULTS Patients in the case group had higher levels of DA and lower levels of Glu and GABA, greater vertical and horizontal distances between the corpus callosum and the inferior part of the fornix and larger ventricle area than patients in the control group (P < 0.05). Patients with positive schizophrenia symptoms had significantly higher levels of DA, Glu, and GABA than those with negative symptoms (P < 0.05). In patients with positive schizophrenia symptoms, PANSS score was significantly positively correlated with DA, vertical and horizontal distances between the corpus callosum and the infrafornix, and ventricular area, and was significantly negatively correlated with Glu and GABA (P < 0.05). In patients with negative schizophrenia symptoms, PANSS score was significantly positively correlated with DA, vertical distance between the corpus callosum and the infrafornix, horizontal distance between the corpus callosum and the infrafornix, and ventricular area, and was significantly negatively correlated with Glu and GABA (P < 0.05).
CONCLUSION In patients with first-episode schizophrenia, DA levels increased, Glu and GABA levels decreased, the thickness of the corpus callosum increased, and these variables were correlated with PANSS scores.
Collapse
Affiliation(s)
- Xian-Jia Xu
- The Fifth Ward, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| | - Tang-Long Liu
- Department of Science and Education, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| | - Liang He
- The Sixteenth Ward, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| | - Ben Pu
- The Twelfth Ward, Ningbo Psychiatric Hospital, Ningbo 315000, Zhejiang Province, China
| |
Collapse
|
5
|
Clayson PE, Joshi YB, Thomas ML, Sprock J, Nungaray J, Swerdlow NR, Light GA. Click-evoked auditory brainstem responses (ABRs) are intact in schizophrenia and not sensitive to cognitive training. Biomark Neuropsychiatry 2022. [DOI: 10.1016/j.bionps.2022.100046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
|
6
|
The data-processing multiverse of event-related potentials (ERPs): A roadmap for the optimization and standardization of ERP processing and reduction pipelines. Neuroimage 2021; 245:118712. [PMID: 34800661 DOI: 10.1016/j.neuroimage.2021.118712] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Revised: 09/27/2021] [Accepted: 11/04/2021] [Indexed: 11/20/2022] Open
Abstract
In studies of event-related brain potentials (ERPs), numerous decisions about data processing are required to extract ERP scores from continuous data. Unfortunately, the systematic impact of these choices on the data quality and psychometric reliability of ERP scores or even ERP scores themselves is virtually unknown, which is a barrier to the standardization of ERPs. The aim of the present study was to optimize processing pipelines for the error-related negativity (ERN) and error positivity (Pe) by considering a multiverse of data processing choices. A multiverse analysis of a data processing pipeline examines the impact of a large set of different reasonable choices to determine the robustness of effects, such as the impact of different decisions on between-trial standard deviations (i.e., data quality) and between-condition differences (i.e., experimental effects). ERN and Pe data from 298 healthy young adults were used to determine the impact of different methodological choices on data quality and experimental effects (correct vs. error trials) at several key stages: highpass filtering, lowpass filtering, ocular artifact correction, reference, baseline adjustment, scoring sensors, and measurement procedure. This multiverse analysis yielded 3,456 ERN scores and 576 Pe scores per person. An optimized pipeline for ERN included a 15 Hz lowpass filter, ICA-based ocular artifact correction, and a region of interest (ROI) approach to scoring. For Pe, the optimized pipeline included a 0.10 Hz highpass filter, 30 Hz lowpass filter, regression-based ocular artifact correction, a -200 to 0 ms baseline adjustment window, and an ROI approach to scoring. The multiverse approach can be used to optimize pipelines for eventual standardization, which would support efforts toward establishing normative ERP databases. The proposed process of analyzing the data-processing multiverse of ERP scores paves the way for better refinement, identification, and selection of data processing parameters, ultimately improving the precision and utility of ERPs.
Collapse
|
7
|
Clayson PE, Molina JL, Joshi YB, Thomas ML, Sprock J, Nungaray J, Swerdlow NR, Light GA. Evaluation of the frequency following response as a predictive biomarker of response to cognitive training in schizophrenia. Psychiatry Res 2021; 305:114239. [PMID: 34673326 DOI: 10.1016/j.psychres.2021.114239] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Revised: 10/01/2021] [Accepted: 10/09/2021] [Indexed: 11/28/2022]
Abstract
Neurophysiological biomarkers of auditory processing show promise predicting outcomes following auditory-based targeted cognitive training (TCT) in schizophrenia, but the viability of the frequency following response (FFR) as a biomarker has yet to be examined, despite its ecological and face validity for auditory-based interventions. FFR is an event-related potential (ERP) that reflects early auditory processing. We predicted that schizophrenia patients would show acute- and longer-term FFR malleability in the context of TCT. Patients were randomized to either TCT (n = 30) or treatment as usual (TAU; n = 22), and electroencephalography was recorded during rapid presentation of an auditory speech stimulus before treatment, after one hour of training, and after 30 h of training. Whereas patients in the TCT group did not show changes in FFR after training, amplitude reductions were observed in the TAU. FFR was positively associated with performance on a measure of single word-in-noise perception in the TCT group, and with a measure of sentence-in-noise perception in both groups. Psychometric reliability analyses of FFR scores indicated high internal consistency but low one-hour and 12-week test-rest reliability. These findings support the dissociation between measures of speech discriminability along the hierarchy of cortical and subcortical early auditory information processing in schizophrenia.
Collapse
Affiliation(s)
- Peter E Clayson
- Department of Psychology, University of South Florida, University of California San Diego, 9500 Gilman Drive #0804 La Jolla, Tampa, CA 92093, USA
| | - Juan L Molina
- VISN 22 Mental Illness Research, Education and Clinical Center (MIRECC), San Diego VA Healthcare System, San Diego, CA, USA
| | - Yash B Joshi
- VISN 22 Mental Illness Research, Education and Clinical Center (MIRECC), San Diego VA Healthcare System, San Diego, CA, USA; Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| | - Michael L Thomas
- Department of Psychology, Colorado State University, Fort Collins, CO, USA
| | - Joyce Sprock
- Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| | - John Nungaray
- Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| | - Neal R Swerdlow
- Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| | - Gregory A Light
- VISN 22 Mental Illness Research, Education and Clinical Center (MIRECC), San Diego VA Healthcare System, San Diego, CA, USA; Department of Psychiatry, University of California San Diego, San Diego, CA, USA.
| |
Collapse
|