Development and validation of a machine learning-supported strategy of patient selection for osteoarthritis clinical trials: the IMI-APPROACH study

Objectives

To efficiently assess the disease-modifying potential of new osteoarthritis treatments, clinical trials need progression-enriched patient populations. To assess whether the application of machine learning results in patient selection enrichment, we developed a machine learning recruitment strategy targeting progressive patients and validated it in the IMI-APPROACH knee osteoarthritis prospective study.

Design

We designed a two-stage recruitment process supported by machine learning models trained to rank candidates by the likelihood of progression. First stage models used data from pre-existing cohorts to select patients for a screening visit. The second stage model used screening data to inform the final inclusion. The effectiveness of this process was evaluated using the actual 24-month progression.

Results

From 3500 candidate patients, 433 with knee osteoarthritis were screened, 297 were enrolled, and 247 completed the 2-year follow-up visit. We observed progression related to pain (P, 30%), structure (S, 13%), and combined pain and structure (P ​+ ​S, 5%), and a proportion of non-progressors (N, 52%) ∼15% lower vs an unenriched population. Our model predicted these outcomes with AUC of 0.86 [95% CI, 0.81-0.90] for pain-related progression and AUC of 0.61 [95% CI, 0.52-0.70] for structure-related progression. Progressors were ranked higher than non-progressors for P ​+ ​S (median rank 65 vs 143, AUC = 0.75), P (median rank 77 vs 143, AUC = 0.71), and S patients (median rank 107 vs 143, AUC = 0.57).

Conclusions

The machine learning-supported recruitment resulted in enriched selection of progressive patients. Further research is needed to improve structural progression prediction and assess this strategy in an interventional trial.

Machine-learning predicted and actual 2-year structural progression in the IMI-APPROACH cohort

In the Innovative Medicine's Initiative Applied Public-Private Research enabling OsteoArthritis Clinical Headway (IMI-APPROACH) knee osteoarthritis (OA) study, machine learning models were trained to predict the probability of structural progression (s-score), predefined as >0.3 mm/year joint space width (JSW) decrease and used as inclusion criterion. The current objective was to evaluate predicted and observed structural progression over 2 years according to different radiographic and magnetic resonance imaging (MRI)-based structural parameters. Radiographs and MRI scans were acquired at baseline and 2-year follow-up. Radiographic (JSW, subchondral bone density, osteophytes), MRI quantitative (cartilage thickness), and MRI semiquantitative [SQ; cartilage damage, bone marrow lesions (BMLs), osteophytes] measurements were obtained. The number of progressors was calculated based on a change exceeding the smallest detectable change (SDC) for quantitative measures or a full SQ-score increase in any feature. Prediction of structural progression based on baseline s-scores and Kellgren-Lawrence (KL) grades was analyzed using logistic regression. Among 237 participants, around 1 in 6 participants was a structural progressor based on the predefined JSW-threshold. The highest progression rate was seen for radiographic bone density (39%), MRI cartilage thickness (38%), and radiographic osteophyte size (35%). Baseline s-scores could only predict JSW progression parameters (most P>0.05), while KL grades could predict progression of most MRI-based and radiographic parameters (P<0.05). In conclusion, between 1/6 and 1/3 of participants showed structural progression during 2-year follow-up. KL scores were observed to outperform the machine-learning-based s-scores as progression predictor. The large amount of data collected, and the wide range of disease stage, can be used for further development of more sensitive and successful (whole joint) prediction models. Trial Registration: Clinicaltrials.gov number NCT03883568.

Predicted and actual 2-year structural and pain progression in the IMI-APPROACH knee osteoarthritis cohort

OBJECTIVES

The IMI-APPROACH knee osteoarthritis study used machine learning (ML) to predict structural and/or pain progression, expressed by a structural (S) and pain (P) predicted-progression score, to select patients from existing cohorts. This study evaluates the actual 2-year progression within the IMI-APPROACH, in relation to the predicted-progression scores.

METHODS

Actual structural progression was measured using minimum joint space width (minJSW). Actual pain (progression) was evaluated using the Knee injury and Osteoarthritis Outcomes Score (KOOS) pain questionnaire. Progression was presented as actual change (Δ) after 2 years, and as progression over 2 years based on a per patient fitted regression line using 0, 0.5, 1 and 2-year values. Differences in predicted-progression scores between actual progressors and non-progressors were evaluated. Receiver operating characteristic (ROC) curves were constructed and corresponding area under the curve (AUC) reported. Using Youden's index, optimal cut-offs were chosen to enable evaluation of both predicted-progression scores to identify actual progressors.

RESULTS

Actual structural progressors were initially assigned higher S predicted-progression scores compared with structural non-progressors. Likewise, actual pain progressors were assigned higher P predicted-progression scores compared with pain non-progressors. The AUC-ROC for the S predicted-progression score to identify actual structural progressors was poor (0.612 and 0.599 for Δ and regression minJSW, respectively). The AUC-ROC for the P predicted-progression score to identify actual pain progressors were good (0.817 and 0.830 for Δ and regression KOOS pain, respectively).

CONCLUSION

The S and P predicted-progression scores as provided by the ML models developed and used for the selection of IMI-APPROACH patients were to some degree able to distinguish between actual progressors and non-progressors.

TRIAL REGISTRATION

ClinicalTrials.gov, https://clinicaltrials.gov, NCT03883568.

GaitSmart motion analysis compared to commonly used function outcome measures in the IMI-APPROACH knee osteoarthritis cohort

Background

There are multiple measures for assessment of physical function in knee osteoarthritis (OA), but each has its strengths and limitations. The GaitSmart® system, which uses inertial measurement units (IMUs), might be a user-friendly and objective method to assess function. This study evaluates the validity and responsiveness of GaitSmart® motion analysis as a function measurement in knee OA and compares this to Knee Injury and Osteoarthritis Outcome Score (KOOS), Short Form 36 Health Survey (SF-36), 30s chair stand test, and 40m self-paced walk test.

Methods

The 2-year Innovative Medicines Initiative—Applied Public-Private Research enabling OsteoArthritis Clinical Headway (IMI-APPROACH) knee OA cohort was conducted between January 2018 and April 2021. For this study, available baseline and 6 months follow-up data (n = 262) was used. Principal component analysis was used to investigate whether above mentioned function instruments could represent one or more function domains. Subsequently, linear regression was used to explore the association between GaitSmart® parameters and those function domains. In addition, standardized response means, effect sizes and t-tests were calculated to evaluate the ability of GaitSmart® to differentiate between good and poor general health (based on SF-36). Lastly, the responsiveness of GaitSmart® to detect changes in function was determined.

Results

KOOS, SF-36, 30s chair test and 40m self-paced walk test were first combined into one function domain (total function). Thereafter, two function domains were substracted related to either performance based (objective function) or self-reported (subjective function) function. Linear regression resulted in the highest R² for the total function domain: 0.314 (R² for objective and subjective function were 0.252 and 0.142, respectively.). Furthermore, GaitSmart® was able to distinguish a difference in general health status, and is responsive to changes in the different aspects of objective function (Standardized response mean (SRMs) up to 0.74).

Conclusion

GaitSmart® analysis can reflect performance based and self-reported function and may be of value in the evaluation of function in knee OA. Future studies are warranted to validate whether GaitSmart® can be used as clinical outcome measure in OA research and clinical practice.

Relationship between motion, using the GaitSmartTM system, and radiographic knee osteoarthritis: an explorative analysis in the IMI-APPROACH cohort

Objectives

To assess underlying domains measured by GaitSmart^TMparameters and whether these are additional to established OA markers including patient reported outcome measures (PROMs) and radiographic parameters, and to evaluate if GaitSmart analysis is related to the presence and severity of radiographic knee OA.

Methods

GaitSmart analysis was performed during baseline visits of participants of the APPROACH cohort (n = 297). Principal component analyses (PCA) were performed to explore structure in relationships between GaitSmart parameters alone and in addition to radiographic parameters and PROMs. Logistic and linear regression analyses were performed to analyse the relationship of GaitSmart with the presence (Kellgren and Lawrence grade ≥2 in at least one knee) and severity of radiographic OA (ROA).

Results

Two hundred and eighty-four successful GaitSmart analyses were performed. The PCA identified five underlying GaitSmart domains. Radiographic parameters and PROMs formed additional domains indicating that GaitSmart largely measures separate concepts. Several GaitSmart domains were related to the presence of ROA as well as the severity of joint damage in addition to demographics and PROMs with an area under the receiver operating characteristic curve of 0.724 and explained variances (adjusted R²) of 0.107, 0.132 and 0.147 for minimum joint space width, osteophyte area and mean subchondral bone density, respectively.

Conclusions

GaitSmart analysis provides additional information over established OA outcomes. GaitSmart parameters are also associated with the presence of ROA and extent of radiographic severity over demographics and PROMS. These results indicate that Gaitsmart^TM may be an additional outcome measure for the evaluation of OA.

Expression of granulocyte macrophage-colony stimulating factor and its receptor in the synovium of osteoarthritis patients is negatively correlated with pain

OBJECTIVES

The crosstalk between the immune and nervous system in the regulation of OA pain is increasingly becoming evident. GM-CSF signals in both systems and might be a new treatment target to control OA pain. Anti GM-CSF treatment has analgesic effects in OA without affecting synovitis scores, suggesting that treatment effects are not caused by local anti-inflammatory effects. We aimed to evaluate whether expression of GM-CSF and its receptor GM-CSFrα in synovial tissue is linked to synovial inflammation and/or knee pain in knee OA patients.

METHODS

Cartilage and synovial tissue of knee OA patients (n = 20) was collected during total knee replacement. Cartilage damage was evaluated by histology and ex vivo matrix proteoglycan turnover. Synovial inflammation was evaluated by histology and ex vivo synovial production of TNF-α, (PGE2) and nitric oxide (NO). Numbers of synovial tissue cells expressing GM-CSF or GM-CSFrα were determined by immunohistochemistry. Pain was evaluated using WOMAC questionnaire and visual analogue scale (VAS) knee pain.

RESULTS

Collected cartilage and synovial tissue had a typical OA phenotype with enhanced cartilage damage and synovial inflammation. GM-CSF and GM-CSFrα expressing cells in the synovial sublining correlated negatively with knee pain. Cartilage damage and synovial inflammation did not correlate with knee pain.

CONCLUSION

Unanticipated, the negative correlation between synovial tissue cells expressing GM-CSF(r) and OA knee pain suggests that local presence of these molecules does not promote pain, and that the role of GM-CSFr in OA pain is unrelated to local inflammation.

TRIAL REGISTRATION

ToetsingOnline.nl NL18274.101.07.

Cohort profile: The Applied Public-Private Research enabling OsteoArthritis Clinical Headway (IMI-APPROACH) study: a 2-year, European, cohort study to describe, validate and predict phenotypes of osteoarthritis using clinical, imaging and biochemical markers

PURPOSE

The Applied Public-Private Research enabling OsteoArthritis Clinical Headway (APPROACH) consortium intends to prospectively describe in detail, preselected patients with knee osteoarthritis (OA), using conventional and novel clinical, imaging, and biochemical markers, to support OA drug development.

PARTICIPANTS

APPROACH is a prospective cohort study including 297 patients with tibiofemoral OA, according to the American College of Rheumatology classification criteria. Patients were (pre)selected from existing cohorts using machine learning models, developed on data from the CHECK cohort, to display a high likelihood of radiographic joint space width (JSW) loss and/or knee pain progression.

FINDINGS TO DATE

Selection appeared logistically feasible and baseline characteristics of the cohort demonstrated an OA population with more severe disease: age 66.5 (SD 7.1) vs 68.1 (7.7) years, min-JSW 2.5 (1.3) vs 2.1 (1.0) mm and Knee injury and Osteoarthritis Outcome Score pain 31.3 (19.7) vs 17.7 (14.6), except for age, all: p<0.001, for selected versus excluded patients, respectively. Based on the selection model, this cohort has a predicted higher chance of progression.

FUTURE PLANS

Patients will visit the hospital again at 6, 12 and 24 months for physical examination, pain and general health questionnaires, collection of blood and urine, MRI scans, radiographs of knees and hands, CT scan of the knee, low radiation whole-body CT, HandScan, motion analysis and performance-based tests.After two years, data will show whether those patients with the highest probabilities for progression experienced disease progression as compared to those wit lower probabilities (model validation) and whether phenotypes/endotypes can be identified and predicted to facilitate targeted drug therapy.

TRIAL REGISTRATION NUMBER

NCT03883568.