1
|
Kessels R, Moerbeek M. A comparison of the multilevel MIMIC model to the multilevel regression and mixed ANOVA model for the estimation and testing of a cross-level interaction effect: A simulation study. Biom J 2023; 65:e2200112. [PMID: 37068180 DOI: 10.1002/bimj.202200112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 02/24/2023] [Accepted: 03/18/2023] [Indexed: 04/19/2023]
Abstract
When observing data on a patient-reported outcome measure in, for example, clinical trials, the variables observed are often correlated and intended to measure a latent variable. In addition, such data are also often characterized by a hierarchical structure, meaning that the outcome is repeatedly measured within patients. To analyze such data, it is important to use an appropriate statistical model, such as structural equation modeling (SEM). However, researchers may rely on simpler statistical models that are applied to an aggregated data structure. For example, correlated variables are combined into one sum score that approximates a latent variable. This may have implications when, for example, the sum score consists of indicators that relate differently to the latent variable being measured. This study compares three models that can be applied to analyze such data: the multilevel multiple indicators multiple causes (ML-MIMIC) model, a univariate multilevel model, and a mixed analysis of variance (ANOVA) model. The focus is on the estimation of a cross-level interaction effect that presents the difference over time on the patient-reported outcome between two treatment groups. The ML-MIMIC model is an SEM-type model that considers the relationship between the indicators and the latent variable in a multilevel setting, whereas the univariate multilevel and mixed ANOVA model rely on sum scores to approximate the latent variable. In addition, the mixed ANOVA model uses aggregated second-level means as outcome. This study showed that the ML-MIMIC model produced unbiased cross-level interaction effect estimates when the relationships between the indicators and the latent variable being measured varied across indicators. In contrast, under similar conditions, the univariate multilevel and mixed ANOVA model underestimated the cross-level interaction effect.
Collapse
Affiliation(s)
- Rob Kessels
- Department of Biometrics, Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Mirjam Moerbeek
- Department of Methodology and Statistics, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
2
|
Kessels R, Moerbeek M, Bloemers J, van der Heijden PG. A multilevel structural equation model for assessing a drug effect on a patient-reported outcome measure in on-demand medication data. Biom J 2021; 63:1652-1672. [PMID: 34270801 PMCID: PMC9292391 DOI: 10.1002/bimj.202100046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Revised: 06/07/2021] [Accepted: 06/19/2021] [Indexed: 11/08/2022]
Abstract
We analyze data from a clinical trial investigating the effect of an on-demand drug for women with low sexual desire. These data consist of a varying number of measurements/events across patients of when the drug was taken, including data on a patient-reported outcome consisting of five items measuring an unobserved construct (latent variable). Traditionally, these data are aggregated prior to analysis by composing one sum score per event and averaging this sum score over all observed events. In this paper, we explain the drawbacks of this aggregating approach. One drawback is that these averages have different standard errors because the variance of the underlying events differs between patients and because the number of events per patient differs. Another drawback is the implicit assumption that all items have equal weight in relation to the latent variable being measured. We propose a multilevel structural equation model, treating the events (level 1) as nested observations within patients (level 2), as alternative analysis method to overcome these drawbacks. The model we apply includes a factor model measuring a latent variable at the level of the event and at the level of the patient. Then, in the same model, the latent variables are regressed on covariates to assess the drug effect. We discuss the inferences obtained about the efficacy of the on-demand drug using our proposed model. We further illustrate how to test for measurement invariance across grouping covariates and levels using the same model.
Collapse
Affiliation(s)
- Rob Kessels
- Emotional Brain BVAlmereThe Netherlands
- Department of BiometricsNetherlands Cancer InstituteAmsterdamThe Netherlands
| | - Mirjam Moerbeek
- Department of Methodology and StatisticsUtrecht UniversityUtrechtThe Netherlands
| | - Jos Bloemers
- Emotional Brain BVAlmereThe Netherlands
- Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of NeuroscienceUtrecht UniversityUtrechtThe Netherlands
| | - Peter G.M. van der Heijden
- Department of Methodology and StatisticsUtrecht UniversityUtrechtThe Netherlands
- Department of Social Statistics and DemographyUniversity of SouthamptonSouthamptonUnited Kingdom
| |
Collapse
|
3
|
Lim-Watson MZ, Hays RD, Kingsberg S, Kallich JD, Murimi-Worstell IB. A Systematic Literature Review of Health-related Quality of Life Measures for Women with Hypoactive Sexual Desire Disorder and Female Sexual Interest/Arousal Disorder. Sex Med Rev 2021; 10:23-41. [PMID: 34481749 DOI: 10.1016/j.sxmr.2021.07.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2021] [Revised: 07/19/2021] [Accepted: 07/23/2021] [Indexed: 11/30/2022]
Abstract
INTRODUCTION Hypoactive Sexual Desire Disorder (HSDD) / Female Sexual Interest/Arousal Disorder (FSIAD) impacts health-related quality of life (HRQoL) of women and their partners, yet existing measures fail to adequately capture relevant concepts (ie, what is essential to measure including symptoms/impacts) important to women with HSDD/FSIAD. OBJECTIVES To identify HRQoL tools used to assess women with HSDD/FSIAD, and to evaluate their psychometric properties (ie, reliability, validity, and responsiveness). METHODS We conducted searches in PubMed, Embase and PsychINFO from June 5, 1989 to September 30, 2020 for studies in women with HSDD/FSIAD and psychometric analyses (English only). Principles of the Preferred Reporting Items for Systematic reviews and Meta-Analyses, the COnsensus-based Standards for the selection of health Measurement INstruments Risk of Bias Checklist and other psychometric criteria were applied. Based on this search, 56 papers were evaluated including 15 randomized-controlled trials, 11 observational/single arm/open label studies, and 30 psychometric studies. RESULTS Of the 18 measures identified, the Female Sexual Function Index (FSFI) and Female Sexual Distress Scale-Revised (FSDS-R) were included in most studies (> 50%). General HRQoL instruments were not used in any of the clinical trials; the SF-12, SF-36 and EQ-5D-5L were reported in two observational studies. No instruments achieved positive quality ratings across all psychometric criteria. The FSFI, FSDS-R, Sexual Event Diary (SED) and the Sexual Desire Relationship Distress Scale (SDRDS), were the only measures to receive a positive rating for content validity. CONCLUSION Reliable and valid HRQoL measures that include sexual desire and distress are needed to provide a more systematic and comprehensive assessment of HRQoL and treatment benefits in women with HSDD/FSIAD. While inferences about HRQoL are limited due to the lack of uniformity in concepts assessed and limited psychometric evaluation of these measures in women with HSDD/FSIAD, opportunities exist for the development of reliable and validated tools that comprehensively measure the most relevant and important concepts in women with HSDD/FSIAD. Lim-Watson MZ, Hays RD, Kingsberg S, et al. A systematic literature review of health-related quality of life measures for women with Hypoactive Sexual Desire Disorder and Female Sexual Interest/Arousal Disorder. Sex Med Rev 2021;XX:XXX-XXX.
Collapse
Affiliation(s)
- Michelle Z Lim-Watson
- Department of Pharmacoeonomics and Policy, Massachusetts College of Pharmacy and Health Sciences University, Boston, MA, USA.
| | - Ron D Hays
- Department of Health Policy and Management, School of Public Health, University of California, Los Angeles, CA, USA; Division of General Internal Medicine and Health Services Research, Department of Medicine, University of California, Los Angeles, CA, USA; RAND Corporation, Santa Monica, CA, USA
| | - Sheryl Kingsberg
- OB/GYN Behavioral Medicine, University Hospitals Cleveland Medical Center, Cleveland, OH, USA
| | - Joel D Kallich
- Department of Pharmacoeonomics and Policy, Massachusetts College of Pharmacy and Health Sciences University, Boston, MA, USA
| | - Irene B Murimi-Worstell
- Department of Pharmacoeonomics and Policy, Massachusetts College of Pharmacy and Health Sciences University, Boston, MA, USA
| |
Collapse
|
4
|
Höhle D, van Rooij K, Bloemers J, Pfaus JG, Michiels F, Janssen P, Claassen E, Tuiten A. A survival of the fittest strategy for the selection of genotypes by which drug responders and non-responders can be predicted in small groups. PLoS One 2021; 16:e0246828. [PMID: 33667227 PMCID: PMC7935233 DOI: 10.1371/journal.pone.0246828] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2019] [Accepted: 01/27/2021] [Indexed: 11/25/2022] Open
Abstract
Phenotype Prediction Scores (PPS) might be powerful tools to predict traits or the efficacy of treatments based on combinations of Single-Nucleotide Polymorphism (SNPs) in large samples. We developed a novel method to produce PPS models for small samples sizes. The set of SNPs is first filtered on those known to be relevant in biological pathways involved in a clinical condition, and then further filtered repeatedly in a survival strategy to select stabile positive/negative risk alleles. This method is applied on Female Sexual Interest/Arousal Disorder (FSIAD), for which two subtypes has been proposed: 1) a relatively insensitive excitatory system in the brain for sexual cues, and 2) a dysfunctional activation of brain mechanisms for sexual inhibition. A double-blind, randomized, placebo-controlled cross-over experiment was conducted on 129 women with FSIAD. The women received three different on-demand drug-combination treatments during 3 two-week periods: testosterone (0.5 mg) + sildenafil (50 mg), testosterone (0.5 mg) + buspirone (10 mg), or matching placebos. The resulted PPS were independently validated on patient-level and group-level. The AUC scores for T+S of the derivation set was 0.867 (95% CI = 0.796-0.939; p<0.001) and was 0.890 (95% CI = 0.778-1.000; p<0.001) on the validation set. For T+B the AUC of the derivation set was 0.957 (95% CI = 0.921-0.992; p<0.001) and 0.869 (95% CI = 0.746-0.992; p<0.001) for the validation set. Both formulas could reliably predict for each drug who benefit from the on-demand drugs and could therefore be useful in clinical practice.
Collapse
Affiliation(s)
| | - Kim van Rooij
- Emotional Brain B.V., Almere, The Netherlands
- Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands
| | - Jos Bloemers
- Emotional Brain B.V., Almere, The Netherlands
- Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands
| | - James G Pfaus
- Centro de Investigaciones Cerebrales, Xalapa, Mexico
| | - Frits Michiels
- Chemistry and Life Sciences, V.O. Patients & Trademarks, Amsterdam, The Netherlands
| | - Paddy Janssen
- Department of Clinical Pharmacy and Toxicology, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Hospital Pharmacy, VieCuri Medical Center Venlo, Venlo, The Netherlands
| | - Eric Claassen
- Emotional Brain B.V., Almere, The Netherlands
- Athena Institute, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | | |
Collapse
|
5
|
Multilevel analyses of on-demand medication data, with an application to the treatment of Female Sexual Interest/Arousal Disorder. PLoS One 2019; 14:e0221063. [PMID: 31415608 PMCID: PMC6695215 DOI: 10.1371/journal.pone.0221063] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2019] [Accepted: 07/29/2019] [Indexed: 12/13/2022] Open
Abstract
Data from clinical trials investigating on-demand medication often consist of an intentionally varying number of measurements per patient. These measurements are often observations of discrete events of when the medication was taken, including for example data on symptom severity. In addition to the varying number of observations between patients, the data have another important feature: they are characterized by a hierarchical structure in which the events are nested within patients. Traditionally, the observed events of patients are aggregated into means and subsequently analyzed using, for example, a repeated measures ANOVA. This procedure has drawbacks. One drawback is that these patient means have different standard errors, first, because the variance of the underlying events differs between patients and second, because the number of events per patient differs. In this paper, we argue that such data should be analyzed by applying a multilevel analysis using the individual observed events as separate nested observations. Such a multilevel approach handles this drawback and it also enables the examination of varying drug effects across patients by estimating random effects. We show how multilevel analyses can be applied to on-demand medication data from a clinical trial investigating the efficacy of a drug for women with low sexual desire. We also explore linear and quadratic time effects that can only be performed when the individual events are considered as separate observations and we discuss several important statistical topics relevant for multilevel modeling. Taken together, the use of a multilevel approach considering events as nested observations in these types of data is advocated as it is more valid and provides more information than other (traditional) methods.
Collapse
|
6
|
Tuiten A, Michiels F, Böcker KB, Höhle D, van Honk J, de Lange RP, van Rooij K, Kessels R, Bloemers J, Gerritsen J, Janssen P, de Leede L, Meyer JJ, Everaerd W, Frijlink HW, Koppeschaar HP, Olivier B, Pfaus JG. Genotype scores predict drug efficacy in subtypes of female sexual interest/arousal disorder: A double-blind, randomized, placebo-controlled cross-over trial. ACTA ACUST UNITED AC 2018; 14:1745506518788970. [PMID: 30016917 PMCID: PMC6052493 DOI: 10.1177/1745506518788970] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Attempts to develop a drug treatment for female sexual interest/arousal disorder
have so far been guided by the principle of ‘one size fits all’, and have failed
to acknowledge the complexity of female sexuality. Guided by personalized
medicine, we designed two on-demand drugs targeting two distinct hypothesized
causal mechanisms for this sexual disorder. The objective of this study was to
design and test a novel procedure, based on genotyping, that predicts which of
the two on-demand drugs will yield a positive treatment response. In a
double-blind, randomized, placebo-controlled cross-over experiment, 139 women
with female sexual interest/arousal disorder received three different on-demand
drug-combination treatments during three 2-week periods: testosterone
0.5 mg + sildenafil 50 mg, testosterone 0.5 mg + buspirone 10 mg, and matching
placebo. The primary endpoint was change in satisfactory sexual events.
Subjects’ genetic profile was assessed using a microarray chip that measures
300,000 single-nucleotide polymorphisms. A preselection of single-nucleotide
polymorphisms associated with genes that are shown to be involved in sexual
behaviour were combined into a Phenotype Prediction Score. The Phenotype
Prediction Score demarcation formula was developed and subsequently validated on
separate data sets. Prediction of drug-responders with the Phenotype Prediction
Score demarcation formula gave large effect sizes (d = 0.66 through 1.06) in the
true drug-responders, and medium effect sizes (d = 0.51 and d = 0.47) in all
patients (including identified double, and non-responders). Accuracy,
sensitivity, specificity, positive predictive value, and negative predictive
value of the Phenotype Prediction Score demarcation formula were all between
0.78 and 0.79, and thus sufficient. The resulting Phenotype Prediction Score was
validated and shown to effectively and reliably predict which women would
benefit from which on-demand drug, and could therefore also be useful in
clinical practice, as a companion diagnostic establishing the way to a true
personalized medicine approach.
Collapse
Affiliation(s)
| | - Frits Michiels
- 2 Chemistry and Life Sciences, V.O. Patients & Trademarks, Amsterdam, The Netherlands
| | | | - Daniël Höhle
- 3 Alan Turing Institute Almere, Almere, The Netherlands
| | - Jack van Honk
- 4 Department of Experimental Psychology, Utrecht University, Utrecht, The Netherlands.,5 Institute of Infectious Disease and Molecular Medicine (IDM), University of Cape Town, Cape Town, South Africa.,6 Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
| | | | - Kim van Rooij
- 1 Emotional Brain BV, Almere, The Netherlands.,7 Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands
| | - Rob Kessels
- 1 Emotional Brain BV, Almere, The Netherlands
| | - Jos Bloemers
- 1 Emotional Brain BV, Almere, The Netherlands.,7 Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands
| | - Jeroen Gerritsen
- 1 Emotional Brain BV, Almere, The Netherlands.,7 Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands
| | - Paddy Janssen
- 8 Division of Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.,9 Department of Central Hospital Pharmacy, Viecuri Hospital, Venlo, The Netherlands
| | - Leo de Leede
- 10 Exelion Bio-Pharmaceutical Consultancy B.V., Waddinxveen, The Netherlands
| | - John-Jules Meyer
- 3 Alan Turing Institute Almere, Almere, The Netherlands.,11 Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands
| | - Walter Everaerd
- 12 Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Henderik W Frijlink
- 13 Research Group of Pharmaceutical Technology and Biopharmacy, University of Groningen, Groningen, The Netherland
| | | | - Berend Olivier
- 7 Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands.,14 Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA.,15 Groningen Institute for Evolutionary Life Sciences, University of Groningen, Groningen, The Netherlands
| | - James G Pfaus
- 16 Department of Psychology, Centre for Studies in Behavioral Neurobiology, Concordia University, Montreal, QC, Canada
| |
Collapse
|