Transforming clinical practice guidelines and clinical pathways into fast-and-frugal decision trees to improve clinical care strategies

Converting IMPROVE bleeding and VTE risk assessment models into a fast-and-frugal decision tree for optimal hospital VTE prophylaxis

ABSTRACT

Current hospital venous thromboembolism (VTE) prophylaxis for medical patients is characterized by both underuse and overuse. The American Society of Hematology (ASH) has endorsed the use of risk assessment models (RAMs) as an approach to individualize VTE prophylaxis by balancing overuse (excessive risk of bleeding) and underuse (risk of avoidable VTE). ASH has endorsed IMPROVE (International Medical Prevention Registry on Venous Thromboembolism) risk assessment models, the only RAMs to assess short-term bleeding and VTE risk in acutely ill medical inpatients. ASH, however, notes that no RAMs have been thoroughly analyzed for their effect on patient outcomes. We aimed to validate the IMPROVE models and adapt them into a simple, fast-and-frugal (FFT) decision tree to evaluate the impact of VTE prevention on health outcomes and costs. We used 3 methods: the "best evidence" from ASH guidelines, a "learning health system paradigm" combining guideline and real-world data from the Medical University of South Carolina (MUSC), and a "real-world data" approach based solely on MUSC data retrospectively extracted from electronic records. We found that the most effective VTE prevention strategy used the FFT decision tree based on an IMPROVE VTE score of ≥2 or ≥4 and a bleeding score of <7. This method could prevent 45% of unnecessary treatments, saving ∼$5 million annually for patients such as the MUSC cohort. We recommend integrating IMPROVE models into hospital electronic medical records as a point-of-care tool, thereby enhancing VTE prevention in hospitalized medical patients.

Improving methods of clinical practice guidelines: From guidelines to pathways to fast-and-frugal trees and decision analysis to develop individualised patient care

BACKGROUND

Current methods for developing clinical practice guidelines have several limitations: they are characterised by the "black box" operation-a process with defined inputs and outputs but an incomplete understanding of its internal workings; they have "the integration problem"-a lack of framework for explicitly integrating factors such as patient preferences and trade-offs between benefits and harms; they generate one recommendation at a time that typically are not connected in a coherent analytical framework; and they apply to "average" patients, while clinicians and their patients seek advice tailored to individual circumstances.

METHODS

We propose augmenting the current guideline development method by converting evidence-based pathways into fast-and-frugal decision trees (FFTs) and integrating them with generalised decision curve analysis to formulate clear, individualised management recommendations.

RESULTS

We illustrate the process by developing recommendations for the management of heparin-induced thrombocytopenia (HIT). We converted evidence-based pathways for HIT, developed by the American Society of Hematology, into an FFT. Here, we consider only thrombotic complications and major bleeding. We leveraged the predictive potential of FFTs to compare the effects of argatroban, bivalirudin, fondaparinux, and direct oral anticoagulants (DOACs) using generalised decision curve analysis. We found that DOACs were superior to other treatments if the FFT-predicted probability of HIT exceeded 3%.

CONCLUSIONS

The proposed analytical framework connects guidelines, pathways, FFTs, and decision analysis, offering risk-tailored personalised recommendations and addressing current guideline development critiques.

A Call to Action on Assessing and Mitigating Bias in Artificial Intelligence Applications for Mental Health

Advances in computer science and data-analytic methods are driving a new era in mental health research and application. Artificial intelligence (AI) technologies hold the potential to enhance the assessment, diagnosis, and treatment of people experiencing mental health problems and to increase the reach and impact of mental health care. However, AI applications will not mitigate mental health disparities if they are built from historical data that reflect underlying social biases and inequities. AI models biased against sensitive classes could reinforce and even perpetuate existing inequities if these models create legacies that differentially impact who is diagnosed and treated, and how effectively. The current article reviews the health-equity implications of applying AI to mental health problems, outlines state-of-the-art methods for assessing and mitigating algorithmic bias, and presents a call to action to guide the development of fair-aware AI in psychological science.

Sensitivity and Specificity of Different Prognostic Systems in Guiding Surveillance for Metastases in Uveal Melanoma

Uveal melanoma (UM) metastasises in ~50% of patients, most frequently to the liver. Surveillance imaging can provide early detection of hepatic metastases; however, guidance regarding UM patient risk stratification for surveillance is unclear. This study compared sensitivity and specificity of four current prognostic systems, when used for risk stratification for surveillance, on patients treated at the Liverpool Ocular Oncology Centre (LOOC) between 2007-2016 (n = 1047). It found that the Liverpool Uveal Melanoma Prognosticator Online III (LUMPOIII) or Liverpool Parsimonious Model (LPM) offered greater specificity at equal levels of sensitivity than the American Joint Committee on Cancer (AJCC) system or monosomy 3 alone, and suggests guidance to achieve 95% sensitivity and 51% specificity (i.e., how to detect the same number of patients with metastases, while reducing the number of negative scans). For example, 180 scans could be safely avoided over 5 years in 200 patients using the most specific approach. LUMPOIII also offered high sensitivity and improved specificity over the AJCC in the absence of genetic information, making the result relevant to centres that do not perform genetic testing, or where such testing is inappropriate or fails. This study provides valuable information for clinical guidelines for risk stratification for surveillance in UM.

Decomposing clinical practice guidelines panels' deliberation into decision theoretical constructs

RATIONALE, AIMS AND OBJECTIVES: The development of clinical practice guidelines (CPG) suffers from the lack of an explicit and transparent framework for synthesising the key elements necessary to formulate practice recommendations. We matched deliberations of the American Society of Haematology (ASH) CPG panel for the management of pulmonary embolism (PE) with the corresponding decision-theoretical constructs to assess agreement of the panel recommendations with explicit decision modelling.

METHODS

Five constructs were identified of which three were used to reformulate the panel's recommendations: (1) standard, expected utility threshold (EUT) decision model; (2) acceptable regret threshold model (ARg) to determine the frequency of tolerable false negative (FN) or false positive (FP) recommendations, and (3) fast-and-frugal tree (FFT) decision trees to formulate the entire strategy for management of PE. We compared four management strategies: withhold testing versus d-dimer → computerized pulmonary angiography (CTPA) ('ASH-Low') versus CTPA→ d-dimer ('ASH-High') versus treat without testing.

RESULTS

Different models generated different recommendations. For example, according to EUT, testing should be withheld for prior probability PE < 0.13%, a clinically untenable threshold which is up to 15 times (2/0.13) below the ASH guidelines threshold of ruling out PE (at post probability of PE ≤ 2%). Three models only agreed that the 'ASH low' strategy should be used for the range of pretest probabilities of PE between 0.13% and 13.27% and that the 'ASH high' management should be employed in a narrow range of the prior PE probabilities between 90.85% and 93.07%. For all other prior probabilities of PE, choosing one model did not ensure coherence with other models.

CONCLUSIONS

CPG panels rely on various decision-theoretical strategies to develop its recommendations. Decomposing CPG panels' deliberation can provide insights if the panels' deliberation retains a necessary coherence in developing guidelines. CPG recommendations often do not agree with the EUT decision analysis, widely used in medical decision-making modelling.

Integration of artificial intelligence in lung cancer: Rise of the machine

The goal of oncology is to provide the longest possible survival outcomes with the therapeutics that are currently available without sacrificing patients' quality of life. In lung cancer, several data points over a patient's diagnostic and treatment course are relevant to optimizing outcomes in the form of precision medicine, and artificial intelligence (AI) provides the opportunity to use available data from molecular information to radiomics, in combination with patient and tumor characteristics, to help clinicians provide individualized care. In doing so, AI can help create models to identify cancer early in diagnosis and deliver tailored therapy on the basis of available information, both at the time of diagnosis and in real time as they are undergoing treatment. The purpose of this review is to summarize the current literature in AI specific to lung cancer and how it applies to the multidisciplinary team taking care of these complex patients.

Evaluation of a fast-and-frugal clinical decision algorithm ('pathways') on clinical outcomes in hospitalised patients with COVID-19 treated with anticoagulants

RATIONALE, AIMS AND OBJECTIVES

Critics have charged that evidence-based medicine (EBM) overemphasises algorithmic rules over unstructured clinical experience and intuition, but the role of structured decision support systems in improving health outcomes remains uncertain. We aim to assess if delivery of anticoagulant prophylaxis in hospitalised patients with COVID-19 according to an algorithm based on evidence-based clinical practice guideline (CPG) improved clinical outcomes compared with administration of anticoagulant treatment given at individual practitioners' discretion.

METHODS

An observational design consisting of the analysis of all acutely ill, consecutive patients (n = 1783) with confirmed COVID-19 diagnosis admitted between 10 March 2020 to 11 January 2022 to an US academic center. American Society of Haematology CPG for anticoagulant prophylaxis in hospitalised patients with COVID-19 was converted into a clinical pathway and translated into fast-and-frugal decision (FFT) tree ('algorithm'). We compared delivery of anticoagulant prophylaxis in hospitalised patients with COVID-19 according to the FFT algorithm with administration of anticoagulant treatment given at individual practitioners' discretion.

RESULTS

In an adjusted analysis, using combination of Lasso (least absolute shrinkage and selection operator) and propensity score based weighting [augmented inverse-probability weighting] statistical techniques controlling for cluster data, the algorithm did not reduce death, venous thromboembolism, or major bleeding, but helped avoid longer hospital stay [number of patients needed to be treated (NNT) = 40 (95% CI: 23-143), indicating that for every 40 patients (23-143) managed on FFT algorithm, one avoided staying in hospital longer than 10 days] and averted admission to intensive-care unit (ICU) [NNT = 19 (95% CI: 13-40)]. All model's selected covariates were well balanced. The results remained robust to sensitivity analyses used to test the stability of the findings.

CONCLUSIONS

When delivered using a structured FFT algorithm, CPG shortened the hospital stay and help avoided admission to ICU, but it did not affect other relevant outcomes.

Formulating Management Strategies Using Fast-and-Frugal Trees (A Decision Tool to Transform Clinical Practice Guidelines and Clinical Pathways into Decision Support at the Point of Care)

Clinical management is rarely based on the collection of one data item. Instead, it is typically characterized by the continuous collection and evaluation of clinical data (symptoms, signs, laboratory, imaging tests, etc.) to establish a platform for further management decisions.

Evidence and Decision-Making

Today, every country struggles to provide adequate health care to its citizens. Globally, an average of $8.3 trillion or 10% of gross domestic product (GDP) is annually spent on health services. In 2019, the USA spent nearly 18% ($3.2 trillion) of its GDP on health care, projected to reach $6.2 trillion by 2028.

Utilization of model-agnostic explainable artificial intelligence frameworks in oncology: a narrative review

Background and Objective

Machine learning (ML) models are increasingly being utilized in oncology research for use in the clinic. However, while more complicated models may provide improvements in predictive or prognostic power, a hurdle to their adoption are limits of model interpretability, wherein the inner workings can be perceived as a "black box". Explainable artificial intelligence (XAI) frameworks including Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) are novel, model-agnostic approaches that aim to provide insight into the inner workings of the "black box" by producing quantitative visualizations of how model predictions are calculated. In doing so, XAI can transform complicated ML models into easily understandable charts and interpretable sets of rules, which can give providers with an intuitive understanding of the knowledge generated, thus facilitating the deployment of such models in routine clinical workflows.

Methods

We performed a comprehensive, non-systematic review of the latest literature to define use cases of model-agnostic XAI frameworks in oncologic research. The examined database was PubMed/MEDLINE. The last search was run on May 1, 2022.

Key Content and Findings

In this review, we identified several fields in oncology research where ML models and XAI were utilized to improve interpretability, including prognostication, diagnosis, radiomics, pathology, treatment selection, radiation treatment workflows, and epidemiology. Within these fields, XAI facilitates determination of feature importance in the overall model, visualization of relationships and/or interactions, evaluation of how individual predictions are produced, feature selection, identification of prognostic and/or predictive thresholds, and overall confidence in the models, among other benefits. These examples provide a basis for future work to expand on, which can facilitate adoption in the clinic when the complexity of such modeling would otherwise be prohibitive.

Conclusions

Model-agnostic XAI frameworks offer an intuitive and effective means of describing oncology ML models, with applications including prognostication and determination of optimal treatment regimens. Using such frameworks presents an opportunity to improve understanding of ML models, which is a critical step to their adoption in the clinic.

Ethical considerations for precision psychiatry: A roadmap for research and clinical practice

Precision psychiatry is an emerging field with transformative opportunities for mental health. However, the use of clinical prediction models carries unprecedented ethical challenges, which must be addressed before accessing the potential benefits of precision psychiatry. This critical review covers multidisciplinary areas, including psychiatry, ethics, statistics and machine-learning, healthcare and academia, as well as input from people with lived experience of mental disorders, their family, and carers. We aimed to identify core ethical considerations for precision psychiatry and mitigate concerns by designing a roadmap for research and clinical practice. We identified priorities: learning from somatic medicine; identifying precision psychiatry use cases; enhancing transparency and generalizability; fostering implementation; promoting mental health literacy; communicating risk estimates; data protection and privacy; and fostering the equitable distribution of mental health care. We hope this blueprint will advance research and practice and enable people with mental health problems to benefit from precision psychiatry.

Mind the gap: Mapping variation between national and local clinical practice guidelines for acute paediatric asthma from the United Kingdom and the Netherlands

BACKGROUND

Clinical practice guidelines (CPGs) aim to standardize clinical care. Increasingly, hospitals rely on locally produced guidelines alongside national guidance. This study examines variation between national and local CPGs, using the example of acute paediatric asthma guidance from the United Kingdom and the Netherlands.

METHODS

Fifteen British and Dutch local CPGs were collected with the matching national guidance for the management of acute asthma in children under 18 years old. The drug sequences, routes and methods of administration recommended for patients with severe asthma and the tone of recommendation across both types of CPGs were schematically represented. Deviations from national guidance were measured. Variation in recommended doses of intravenous salbutamol was examined. CPG quality was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II.

RESULTS

British and Dutch national CPGs differed in the recommended drug choices, sequences, routes and methods of administration for severe asthma. Dutch national guidance was more rigidly defined. Local British CPGs diverged from national guidance for 23% of their recommended interventions compared to 8% for Dutch local CPGs. Five British local guidelines and two Dutch local guidelines differed from national guidance for multiple treatment steps. Variation in second-line recommendations was greater than for first-line recommendations across local CPGs from both countries. Recommended starting doses for salbutamol infusions varied by more than tenfold. The quality of the sampled local CPGs was low across all AGREE II domains.

CONCLUSIONS

Local CPGs for the management of severe acute paediatric asthma featured substantial variation and frequently diverged from national guidance. Although limited to one condition, this study suggests that unmeasured variation across local CPGs may contribute to variation of care more broadly, with possible effects on healthcare quality.

Managing Boundary Uncertainty in Diagnosing the Patients of Rural Area Using Fuzzy and Rough Set

People of rural India often suffer from acute health conditions like diarrhea, flu, lung congestion, and anemia, but they are not receiving treatment even at primary level due to scarcity of doctors and health infrastructure in remote villages. Health workers are involved in diagnosing the patients based on the symptoms and physiological signs. However, due to inadequate domain knowledge, lack of expertise, and error in measuring the health data, uncertainty creeps in the decision space, resulting many false cases in predicting the diseases. The paper proposes an uncertainty management technique using fuzzy and rough set theory to diagnose the patients with minimum false-positive and false-negative cases. We use fuzzy variables with proper semantic to represent the vagueness of input data, appearing due to measurement error. We derive initial degree of belonging of each patient in two different disease class labels (YES/NO) using the fuzzified input data. Next, we apply rough set theory to manage uncertainty in diagnosing the diseases by learning approximations of the decision boundary between the two class labels. The optimum lower and upper approximation membership functions for each disease class label have been obtained using Non-dominated Sorting Genetic Algorithm-II (NSGA-II). Finally, using the proposed disease_similarity_factor, new patients are diagnosed precisely with 98% accuracy and minimum false cases.

Mind the Differences: How Diagnoses and Hospital Characteristics Influence Coordination in Cancer Patient Pathways

Integrated care pathway (ICP) is a prevailing concept in health care management including cancer care. Though substantial research has been conducted on ICPs knowledge is still deficient explaining how characteristics of diagnose, applied procedures, patient group and organizational context influence specific practicing of ICPs. We studied how coordination takes place in three cancer pathways in four Norwegian hospitals. We identified how core contextual variables of cancer pathways affect complexity and predictability of the performance of each pathway. Thus, we also point at differences in core preconditions for accomplishing coordination of the cancer pathways. In addition, the findings show that three different types of coordination dynamics are present in all three pathways to a divergent degree: programmed chains, consultative hubs and problem-solving webs. Pathway coordination also depends on hierarchical interaction. Lack of corresponding roles in the medical–professional and the administrative–institutional logics presents a challenge for coordination, both within and between hospitals. We recommend that further improvement of specific ICPs by paying attention to what should be standardized and what should be kept flexible, aligning semi-formal and formal structures to pathway processes and identify the professional cancer related background and management style required by the key-roles in pathway management.

Professional judgement in clinical practice (part 2): knowledge into practice

RATIONALE, AIMS, AND OBJECTIVES

Though strong evidence-based medicine is assertive in its claims, an insufficient theoretical basis and patchwork of arguments provide a good case that rather than introducing a new paradigm, EBM is resisting a shift to actually revolutionary complexity theory and other emergent approaches. This refusal to pass beyond discredited positivism is manifest in strong EBM's unsuccessful attempts to continually modify its already inadequate previous modifications, as did the defenders of the Ptolemaic astronomical model who increased the number of circular epicycles until the entire epicycle-deferent system proved untenable.

METHODS

Narrative Review.

RESULTS

The analysis in Part 1 of this three part series showed epistemological confusion as strong EBM plays the discredited positivistic tradition out to the end, thus repeating in a medical sphere and vocabulary the major assumptions and inadequacies that have appeared in the trajectory of modern science. Paper 2 in this series examines application, attending to strong EBM's claim of direct transferability of EBM research findings to clinical settings and its assertion of epistemological normativity. EBM's contention that it provides the "only valid" approach to knowledge and action is questioned by analyzing the troubled story of proposed hierarchies of the quality of research findings (especially of RCTs, with other factors marginalized), which falsely identifies evaluating findings with operationally utilizing them in clinical recommendations and decision-making. Further, its claim of carrying over its normative guidelines to cover the ethical responsibilities of researchers and clinicians is questioned.

Clinical pathways and patient-related outcomes in hospital-based settings: a systematic review and meta-analysis of randomized controlled trials

Clinical pathways represent a multi-disciplinary approach to translate clinical practice guidelines into practical interventions. The literature from 2010 onward regarding the efficacy of adopting a clinical pathway on patient-related outcomes within the in-hospital setting has not been synthesized yet. For this reason, this systematic review and meta-analysis of randomized controlled trials aimed to critically synthesize the literature from 2010 onward about the efficacy of clinical pathways, compared with standard of care, on patient-related outcomes in different populations and to determine the effects of clinical pathways on patient outcomes. We searched PubMed, Scopus, CINAHL, and reference lists of the included studies. Two independent reviewers screened the 360 identified articles and selected fifteen eligible articles, which were evaluated for content and risk of bias. Eleven studies were finally included. Given the commonalities of the measured outcomes, a meta-analysis including eight studies was performed to evaluate the effect size of the associations between clinical pathways and quality of life (OR=1.472 [0.483–4.486]; p=0.496), and two meta-analyses, including four studies, were performed to evaluate the effect sizes of the associations between clinical pathways with satisfaction (OR=2.226 [0.868–5.708]; p=0.096) and length of stay (OR=0,585 [0.349–0.982]; p=0.042). Reduced length of stay appeared to be associated with clinical pathways, while it remains unclear whether adopting clinical pathways could improve levels of quality of life and satisfaction. More primary research is required to determine in specific populations the efficacy of clinical pathways on patient-related outcomes. (www.actabiomedica.it)

Practicing Integrated Care Pathways in Norwegian Hospitals: Coordination through Industrialized Standardization, Value Chains, and Quality Management or an Organizational Equivalent to Improvised Jazz Standards

The goal of coordinating pathways for cancer patients through their diagnostic and treatment journey is often approached by borrowing strategies from traditional industries, including standardization, process redesign, and variation reduction. However, the usefulness of these strategies is sometimes limited in the face of the complexity and uncertainty that characterize these processes over time and the situation at both patient and institutional levels. We found this to be the case when we did an in-depth qualitative study of coordination processes in patient pathways for three diagnoses in four Norwegian hospitals. What allows these hospitals to accomplish coordination is supplementing standardization with improvisation. This improvisation is embedded in four types of emerging semi-formal structures: collegial communities, networks, boundary spanners, and physical proximity. The hierarchical higher administrative levels appear to have a limited ability to manage and support coordination of these emerging structures when needed. We claim that this can be explained by viewing line management as representative of an economic-administrative institutional logic while these emerging structures represent a medical-professional logic that privileges proximity to the variation and complexity in the situations. The challenge is then to find a way for emergent and formal structures to coexist.

Improving management of hospitalised patients with COVID-19: algorithms and tools for implementation and measurement

BACKGROUND

The COVID-19 pandemic represents an unprecedented challenge to healthcare systems and nations across the world. Particularly challenging are the lack of agreed-upon management guidelines and variations in practice. Our hospital is a large, secondary-care government hospital in Kuwait, which has increased its capacity by approximately 28% to manage the care of patients with COVID-19. The surge in capacity has necessitated the redeployment of staff who are not well-trained to manage such conditions. There was a great need to develop a tool to help redeployed staff in decision-making for patients with COVID-19, a tool which could also be used for training.

METHODS

Based on the best available clinical knowledge and best practices, an eight member multidisciplinary group of clinical and quality experts undertook the development of a clinical algorithm-based toolkit to guide training and practice for the management of patients with COVID-19. The team followed Horabin and Lewis' seven-step approach in developing the algorithms and a five-step method in writing them. Moreover, we applied Rosenfeld et al's five points to each algorithm.

RESULTS

A set of seven clinical algorithms and one illustrative layout diagram were developed. The algorithms were augmented with documentation forms, data-collection online forms and spreadsheets and an indicators' reference sheet to guide implementation and performance measurement. The final version underwent several revisions and amendments prior to approval.

CONCLUSIONS

A large volume of published literature on the topic of COVID-19 pandemic was translated into a user-friendly, algorithm-based toolkit for the management of patients with COVID-19. This toolkit can be used for training and decision-making to improve the quality of care provided to patients with COVID-19.

A Symptom-Based Rule for Diagnosis of COVID-19

SARS-CoV-19 PCR testing has a turn-around time that makes it impractical for real-time decision-making, and current point-of-care tests have limited sensitivity, with frequent false negatives. The study objective was to develop a clinical prediction rule to use with a point-of-care test to diagnose COVID-19 in symptomatic outpatients. A standardized clinical questionnaire was administered prior to SARS-CoV-2 PCR testing. Data was extracted by a physician blinded to the result status. Individual symptoms were combined into 326 unique clinical phenotypes. Multivariable logistic regression was used to identify independent predictors of COVID-19, from which a weighted clinical prediction rule was developed, to yield stratified likelihood ratios for varying scores. A retrospective cohort of 120 SARS-CoV-2-positive cases and 120 SARS-CoV-2-negative matched controls among symptomatic outpatients in a Connecticut HMO was used for rule development. A temporally distinct cohort of 40 cases was identified for validation of the rule. Clinical phenotypes independently associated with COVID-19 by multivariable logistic regression include loss of taste or smell (olfactory phenotype, 2 points) and fever and cough (febrile respiratory phenotype, 1 point). Wheeze or chest tightness (reactive airways phenotype, − 1 point) predicted non-COVID-19 respiratory viral infection. The AUC of the model was 0.736 (0.674–0.798). Application of a weighted C19 rule yielded likelihood ratios for COVID-19 diagnosis for varying scores ranging from LR 15.0 for 3 points to LR 0.1 for − 1 point. Using a Bayesian diagnostic approach, combining community prevalence with the evidence-based C19 rule to adjust pretest probability, clinicians can apply a point of care test with limited sensitivity across a range of clinical scenarios to differentiate COVID-19 infection from influenza and respiratory viral infection.

Evaluation of Omics-Based Strategies for the Management of Advanced Lung Cancer

PURPOSE

Omic-informed therapy is being used more frequently for patients with non-small-cell lung cancer (NSCLC) being treated on the basis of evidence-based decision-making. However, there is a lack of a standardized framework to evaluate those decisions and understand the association between omics-based management strategies and survival among patients. Therefore, we compared outcomes between patients with lung adenocarcinoma who received omics-driven targeted therapy versus patients who received standard therapeutic options.

PATIENTS AND METHODS

This was a retrospective study of patients with advanced NSCLC adenocarcinoma (N = 798) at City of Hope who received genomic sequencing at the behest of their treating oncologists. A thoracic oncology registry was used as a clinicogenomic database to track patient outcomes.

RESULTS

Of 798 individuals with advanced NSCLC (median age, 65 years [range, 22-99 years]; 60% white; 50% with a history of smoking), 662 patients (83%) had molecular testing and 439 (55%) received targeted therapy on the basis of the omic-data. A fast-and-frugal decision tree (FFT) model was developed to evaluate the impact of omics-based strategy on decision-making, progression-free survival (PFS), and overall survival (OS). We calculated that the overall positive predictive value of the entire FFT strategy for predicting decisions regarding the use of tyrosine kinase inhibitor-based targeted therapy was 88% and the negative predictive value was 96%. In an adjusted Cox regression analysis, there was a significant correlation with survival benefit with the FFT omics-driven therapeutic strategy for both PFS (hazard ratio [HR], 0.56; 95% CI, 0.42 to 0.74; P < .001) and OS (HR, 0.51; 95% CI, 0.36 to 0.71; P < .001) as compared with standard therapeutic options.

CONCLUSION

Among patients with advanced NSCLC who received care in the academic oncology setting, omics-driven therapy decisions directly informed treatment in patients and was correlated with better OS and PFS.

Complex Oncological Decision-Making Utilizing Fast-and-Frugal Trees in a Community Setting-Role of Academic and Hybrid Modeling

Non-small cell lung cancer is a devastating disease and with the advent of targeted therapies and molecular testing, the decision-making process has become complex. While established guidelines and pathways offer some guidance, they are difficult to utilize in a busy community practice and are not always implemented in the community. The rationale of the study was to identify a cohort of patients with lung adenocarcinoma at a City of Hope community site (n = 11) and utilize their case studies to develop a decision-making framework utilizing fast-and-frugal tree (FFT) heuristics. Most patients had stage IV (N = 9, 81.8%) disease at the time of the first consultation. The most common symptoms at initial presentation were cough (N = 5, 45.5%), shortness of breath (N = 3, 27.2%), and weight loss (N = 3, 27.2%). The Eastern Cooperative Oncology Group (ECOG) performance status ranged from 0-1 in all patients in this study. Distribution of molecular drivers among the patients were as follows: EGFR (N = 5, 45.5%), KRAS (N = 2, 18.2%), ALK (N = 2, 18.2%), MET (N = 2, 18.2%), and RET (N = 1, 9.1%). Seven initial FFTs were developed for the various case scenarios, but ultimately the decisions were condensed into one FFT, a molecular stage IV FFT, that arrived at accurate decisions without sacrificing initial information. While these FFT decision trees may seem arbitrary to an experienced oncologist at an academic site, the simplicity of their utility is essential for community practice where patients often do not get molecular testing and are not assigned proper therapy.

A unifying framework for improving health care

The quality health care around world is suboptimal. To improve the quality of contemporary health care delivery, advocates have proposed a number of scientific and technical initiatives. All these initiatives, however, have arisen and continue to operate in siloes, resulting in confusion and incommensurability among those concerned with health care improvement. Participants in the quality improvement (QI) space typically stress their own, often narrow, perspective, failing to place QI in context or to acknowledge other approaches. In order to improve delivery of health care, the following is required: Provide a unifying framework for improving health care. We argue this is best done under a Health System Science (HSS) framework but with full understanding that the fundamental principles of HSS are rooted in evidence-based medicine (EBM) and decision sciences. Understand that QI initiatives are fundamentally local activities. Hence, incentivizing bottom-up, local QI initiatives would improve health care delivery to a far greater extent than the current top-down initiatives undertaken in a response to various regulatory mandates. Akin to the "Choosing Wisely" initiative, which challenged professional societies, each institution should identify (a) the extent to which its practices are evidence-based and (b) the top 5 health care practices or interventions that, at a given institution, represent overuse, underuse, or misuse/error or undermine clinicians' efforts to deliver kind and empathic care. Providing a framework that can unify the current patchwork of the initiatives would help create a common basis to help align all the existing QI efforts. In addition, thinking small (at local level) may lead to health care quality improvements that national initiatives (thinking big), focused on regulation, competition, or legal requirements, have failed to achieve.

The threshold model revisited

BACKGROUND

The threshold model represents one of the most significant advances in the field of medical decision-making, yet it often does not apply to the most common class of clinical problems, which include health outcomes as a part of definition of disease. In addition, the original threshold model did not take a decision-maker's values and preferences explicitly into account.

METHODS

We reformulated the threshold model by (1) applying it to those clinical scenarios, which define disease according to outcomes that treatment is designed to affect, (2) taking into account a decision-maker's values.

RESULTS

We showed that when outcomes (eg, morbidity) are integral part of definition of disease, the classic threshold model does not apply (as this leads to double counting of outcomes in the probabilities and utilities branches of the model). To avoid double counting, the model can be appropriately analysed by assuming diagnosis is certain (P = 1). This results in deriving a different threshold-the threshold for outcome of disease (M_t ) instead of threshold for probability of disease (P_t ) above which benefits of treatment outweigh its harms. We found that M_t  ≤ P_t , which may explain differences between normative models and actual behaviour in practice. When a decision-maker values outcomes related to benefit and harms differently, the new threshold model generates decision thresholds that could be descriptively more accurate.

CONCLUSIONS

Calculation of the threshold depends on careful disease versus utility definitions and a decision-maker's values and preferences.

Treating real people: Science and humanity

Something important is happening in applied, interdisciplinary research, particularly in the field of applied health research. The vast array of papers in this edition are evidence of a broad change in thinking across an impressive range of practice and academic areas. The problems of complexity, the rise of chronic conditions, overdiagnosis, co-morbidity, and multi-morbidity are serious and challenging, but we are rising to that challenge. Key conceptions regarding science, evidence, disease, clinical judgement, and health and social care are being revised and their relationships reconsidered: Boundaries are indeed being redrawn; reasoning is being made "fit for practice." Ideas like "person-centred care" are no longer phrases with potential to be helpful in some yet-to-be-clarified way: Theorists and practitioners are working in collaboration to give them substantive import and application.

From MARS to MAGIC: The remarkable journey through time and space of the Grading of Recommendations Assessment, Development and Evaluation initiative

For over 30 years, "evidence-based" clinical guidelines remained entrenched in an oversimplified, design-based, framework for rating the strength of evidence supporting clinical recommendations. The approach frequently equated the rating of evidence with that of the recommendations themselves. "Grading Recommendations Assessment, Development and Evaluation (GRADE)" has emerged as a proposed antidote to obsolete guideline methodology. GRADE sponsors and collaborators are in the process of attempting to amplify and extend the framework to encompass implementation and adaptation of guidelines, above and beyond the evaluation and rating of clinical research. Alternative schemes and models for such extensions are beginning to appear. This commentary reviews the strengths and weaknesses of GRADE with reference to other recent critiques. It considers the GRADE Working Group's "evidence-to-decision" extension of the evidence rating framework, together with proposed alternatives. It identifies pitfalls of the GRADE system's cooptation of relational processes necessary to the interpretation and uptake of recommendations that properly belong to end-users. It also identifies dangers inherent in blurring important boundaries between clinical and policy applications of guidelines. Finally, it addresses criticisms regarding the lack of a theoretical framework supporting the different facets of the GRADE approach and proposes a social constructivist orientation to clinical guideline development and use. Recommendations are offered to potential guideline developers and users regarding how to draw upon the strengths of the GRADE framework without succumbing to its pitfalls.

How do we know if a clinical practice guideline is good? A response to Djulbegovic and colleagues' use of fast-and-frugal decision trees to improve clinical care strategies

Clinical practice guidelines (CPGs) and clinical pathways have become important tools for improving the uptake of evidence-based care. Where CPGs are good, adherence to the recommendations within is thought to result in improved patient outcomes. However, the usefulness of such tools for improving patient important outcomes depends both on adherence to the guideline and whether or not the CPG in question is good. This begs the question of what it is that makes a CPG good? In this issue of the Journal, Djulbegovic and colleagues offer a theory to help guide the development of CPGs. The "fast-and-frugal tree" (FFT) heuristic theory is purported to provide the theoretical structure needed to quantitatively assess clinical guidelines in practice, something that the lack of theory to guide CPG development has precluded. In this paper, I examine the role of FFTs in providing an adequate theoretical framework for developing CPGs. In my view, positioning guideline development within the FFT framework may help with problems related to adherence. However, I believe that FTTs fall short in providing panel members with the theoretical basis needed to justify which factors should be considered when developing a CPG, how information on those factors derived from research studies should be interpreted, and how those factors should be integrated into the recommendation.

The evolution of GRADE (part 1): Is there a theoretical and/or empirical basis for the GRADE framework?

RATIONALE, AIMS, AND OBJECTIVES

The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework has been presented as the best method available for developing clinical recommendations. GRADE has undergone a series of modifications. Here, we present the first part of a three article series examining the evolution of GRADE. Our purpose is to explore if (and if so, how) GRADE provides: (1) a justification (ie, theoretical and/or empirical) for why the criteria/components under consideration in the system are included (and other factors excluded), as well as why some criteria/components where added/modified in the evolution process, (2) clear and functional (ie, how to operationalize them) definitions of the included criteria/components, and (3) instruction and justification for how all the criteria/components are to be integrated when determining a recommendation. In part 1 of the series, we examine the first two versions of GRADE.

METHODS

Narrative review.

RESULTS

The justification scheme that sustains GRADE is not articulated in the first two versions of the framework. Why some criteria/components were included, and others excluded, is not justified theoretically nor is empirical support provided to suggest that the framework as presented includes that which is needed to produce valid recommendations. The first two versions of GRADE show a lack of clear instruction on how to operationalize the criteria for assessing the quality of evidence and the components for making a recommendation (including how to integrate the criteria/components at each step), which leaves substantial room for judgement on the part of the user of GRADE for guideline development.

CONCLUSIONS

This article revealed an absence of a justification (theoretical and/or empirical) to support important aspects of the GRADE framework, as well as a lack of clear instruction on how to operationalize the criteria and components in the framework. These issues limit one's ability to scientifically assess the appropriateness of GRADE for determining clinical recommendations.

Heuristic value-based framework for lung cancer decision-making

Heuristics and the application of fast-and-frugal trees may play a role in establishing a clinical decision-making framework for value-based oncology. We determined whether clinical decision-making in oncology can be structured heuristically based on the timeline of the patient's treatment, clinical intuition, and evidence-based medicine. A group of 20 patients with advanced non-small cell lung cancer (NSCLC) were enrolled into the study for extensive treatment analysis and sequential decision-making. The extensive clinical and genomic data allowed us to evaluate the methodology and efficacy of fast-and-frugal trees as a way to quantify clinical decision-making. The results of the small cohort will be used to further advance the heuristic framework as a way of evaluating a large number of patients within registries. Among the cohort whose data was analyzed, substitution and amplification mutations occurred most frequently. The top five most prevalent genomic alterations were TP53 (45%), ALK (40%), LRP1B (30%), CDKN2A (25%), and MYC (25%). These 20 cases were analyzed by this clinical decision-making process and separated into two distinctions: 10 straightforward cases that represented a clearer decision-making path and 10 complex cases that represented a more intricate treatment pathway. The myriad of information from each case and their distinct pathways was applied to create the foundation of a framework for lung cancer decision-making as an aid for oncologists. In late-stage lung cancer patients, the fast-and-frugal heuristics can be utilized as a strategy of quantifying proper decision-making with limited information.