Outcome Rating Scale and Session Rating Scale in psychological practice: Clinical utility of ultra-brief measures

Construct Validity of Outcome Rating Scale (ORS) Scores in Clinical Samples: Extension of Harris, Murphy, and Rakes' (2019) Narrative Review

PURPOSE

A recent review of 24 studies primarily evaluated psychometric properties of the Outcome Rating Scale (ORS) related to coefficient alpha and convergent validity.

METHOD

To provide a fuller picture, this review examined effect size (ES) estimates and the full range of correlations between ORS scores and related measures.

RESULTS

Mean ORS ESs equaled 1.00 for 44 comparison periods. ORS scores correlated highest with measures of depression and evidenced larger ESs in depression treatments.

DISCUSSION

The ORS functions as a robust measure for detecting client progress in clinical samples that were primarily young, female, and Caucasian. Some results suggest ORS scores primarily reflect a rapid response to interventions.

CONCLUSION AND LIMITATIONS

Future research should assess ORS scores' psychometric properties in response to interventions with more heterogeneous client samples as well as its capacity to monitor change beyond initial sessions.

Anxiety Gremlins: mixed methods sequential explanatory evaluation of a CBT group intervention for children

Although research evidence supports the efficacy of cognitive behavioural therapy (CBT) for anxiety in children, it is important to examine practice-based evidence of effectiveness in typical clinical contexts. This study evaluated a CBT group intervention – ‘Anxiety Gremlins’ – for childhood anxiety. Participants were 36 children (19 boys, 17 girls) aged 8–13, referred for anxiety symptoms at a UK NHS service. The 8-session intervention included six child sessions (2 h) and two parent sessions (1 h). Self-report outcome measures of anxiety symptoms, life functioning and therapeutic relationships were used to measure change pre- and post-intervention. Semi-structured interviews were conducted with group facilitators and analysed through deductive content analysis to identify barriers and facilitators to change. No substantive differences were found between aggregated scores on pre- versus post-intervention outcome measures. Reliable change in anxiety symptoms was identified in 10 children (31%), with five improvers and five deteriorators. Interviews with facilitators identified disruption in group flow, lack of facilitator time to prepare and reflect, and the complexity of clients as hindering factors. Children meeting like-minded peers to share their stories and high engagement in the therapeutic process were helpful factors. Anxiety Gremlins did not demonstrate effectiveness on outcome measures, and this contrasted with clinical opinion. Recommendations were made for the service to revisit the intervention content and the method for recruiting children to the group – as complexity/co-morbidity was linked to poorer outcomes. Future research could explore fidelity to an adapted intervention and include interviews with children and their parents.

Key learning aims

After reading this paper the reader should be able to: (1)

Changes in clients’ well-being (ORS) and state hope (SHS) during inpatient substance abuse treatment

Aims:

During inpatient substance abuse treatment, professionals seek to help clients to achieve improvement in various life domains affected by substance abuse. Progress monitoring during substance abuse treatment has been called for, and in this study we examine how clients’ levels of well-being and hope change during inpatient substance abuse treatment.

Methods:

Clients ( N = 168) entered the treatment facility between January 1, 2017 and August 31, 2018. The prospective data were gathered from one treatment unit located in central Finland on entry into treatment, during and at the end of treatment. Using the Outcome Rating Scale (ORS) as a therapeutic tool was standard procedure in the treatment unit. Demographic information on participants, information on their previous substance use and scores on the State Hope Scale (SHS) were collected through structured questionnaires. Non-parametrical tests (Mann–Whitney U test; Kruskal–Wallis χ2; Spearman’s rho) were used to study changes in indicators of well-being and hope.

Results:

As hypothesised, inpatient treatment had a positive effect on both ORS and SHS and they were interrelated. Differences in gender, living situation and previous admissions were found when studying how the background information of the participants was related to the change achieved in ORS and SHS.

Conclusions:

Inpatient substance abuse treatment seemed to enhance both well-being and hope of the clients of inpatient substance abuse treatment.

Adherence and acceptability of a robot-assisted Pivotal Response Treatment protocol for children with autism spectrum disorder

The aim of this study is to present a robot-assisted therapy protocol for children with ASD based on the current state-of-the-art in both ASD intervention research and robotics research, and critically evaluate its adherence and acceptability based on child as well as parent ratings. The robot-assisted therapy was designed based on motivational components of Pivotal Response Treatment (PRT), a highly promising and feasible intervention focused at training “pivotal” (key) areas such as motivation for social interaction and self-initiations, with the goal of establishing collateral gains in untargeted areas of functioning and development, affected by autism spectrum disorders. Overall, children (3–8 y) could adhere to the robot-assisted therapy protocol (Mean percentage of treatment adherence 85.5%), showed positive affect ratings after therapy sessions (positive in 86.6% of sessions) and high robot likability scores (high in 79.4% of sessions). Positive likability ratings were mainly given by school-aged children (H(1) = 7.91, p = .005) and related to the movements, speech and game scenarios of the robot. Parent ratings on the added value of the robot were mainly positive (Mean of 84.8 on 0–100 scale), while lower parent ratings were related to inflexibility of robot behaviour.

The Psychometric Properties of the Session Rating Scale: A Narrative Review

Background: The Session Rating Scale (SRS) is a measure used in psychotherapy practice. Given its application and frequent use in clinical practice settings, several studies including primary validation studies as well as outcome studies reporting psychometric properties have examined the validity and reliability of the SRS. Because the SRS is designed to be used in clinical practice, demonstration of reliability and validity is ideal with clinical populations instead of non-clinical populations. To our knowledge, there have been no reviews evaluating the psychometric properties of the SRS with clinical populations.Purpose: The aim of this study was to review and synthesize the psychometric properties of the SRS by examining all relevant outcome and validation studies.Methods: We completed an electronic literature search of several online databases in order to identify relevant articles. We identified 12 articles that met our inclusion and analyzed the reliability and validity evidence for the SRS with clinical populations.Results: Findings showed high internal consistency for the SRS and support evidence that the underlying dimension is a measure of therapeutic alliance. Test-retest reliability estimates were reported in 4 of the 12 studies included in this review (ranging from .54 to .70). The SRS had low to moderate concurrent validity with other longer measures of therapeutic alliance (.29 with ORS; .48 with HAQ II; .57-.65 with WAI).Discussion: Clinicians should be aware of the limitations in validity evidence for the SRS that may result from its brevity. However, clinicians should also consider the relative stability of the SRS as a pragmatic global measure of the construct of therapeutic alliance. Future research efforts should focus on validation of the SRS with larger, more diverse samples and specific clinical populations with a wider range of symptom severity.

A pilot cross-over randomised controlled trial of child-focused process and outcome evaluation of arts therapies at primary mainstream schools: study protocol

Background: Arts therapies are increasingly being adopted within schools to alleviate and prevent a wide range of children's difficulties. Such therapies help to facilitate personal change and growth through arts media; visual arts, music, drama, dance and movement.Methods: This study aims to investigate the child-focused process and outcomes of an arts therapies intervention through child-reported questionnaires, interviews, participant observations, arts-work and biomarkers. The participants are 64 primary school children with mild emotional and behavioural difficulties.Expected results: To explore a) whether all components of the study (i.e. recruitment, randomisation, follow-up) can work together in a larger trial; b) the children's evaluations of arts therapies;c) the impact on children's wellbeing, quality of life and sleep; d) the sample size calculations and cost evaluations for future research. This is the first study to embrace all arts therapies as one research domain with a clear focus on children's verbal, non-verbal and physical responses.

A minimum evaluation protocol and stepped-wedge cluster randomized trial of ACCESS Open Minds, a large Canadian youth mental health services transformation project

BACKGROUND

Many Canadian adolescents and young adults with mental health problems face delayed detection, long waiting lists, poorly accessible services, care of inconsistent quality and abrupt or absent inter-service transitions. To address these issues, ACCESS Open Minds, a multi-stakeholder network, is implementing and systematically evaluating a transformation of mental health services for youth aged 11 to 25 at 14 sites across Canada. The transformation plan has five key foci: early identification, rapid access, appropriate care, the elimination of age-based transitions between services, and the engagement of youth and families.

METHODS

The ACCESS Open Minds Research Protocol has multiple components including a minimum evaluation protocol and a stepped-wedge cluster randomized trial, that are detailed in this paper. Additional components include qualitative methods and cost-effectiveness analyses. The services transformation is being evaluated at all sites via a minimum evaluation protocol. Six sites are participating in the stepped-wedge trial whereby the intervention (a service transformation along the key foci) was rolled out in three waves, each commencing six months apart. Two sites, one high-population and one low-population, were randomly assigned to each of the three waves, i.e., randomization was stratified by population size. Our primary hypotheses pertain to increased referral numbers, and reduced wait times to initial assessment and to the commencement of appropriate care. Secondary hypotheses pertain to simplified pathways to care; improved clinical, functional and subjective outcomes; and increased satisfaction among youth and families. Quantitative measures addressing these hypotheses are being used to determine the effectiveness of the intervention.

DISCUSSION

Data from our overall research strategy will help test the effectiveness of the ACCESS Open Minds transformation, refine it further, and inform its scale-up. The process by which our research strategy was developed has implications for the practice of research itself in that it highlights the need to actively engage all stakeholder groups and address unique considerations in designing evaluations of complex healthcare interventions in multiple, diverse contexts. Our approach will generate both concrete evidence and nuanced insights, including about the challenges of conducting research in real-world settings. More such innovative approaches are needed to advance youth mental health services research.

TRIAL REGISTRATION NUMBER

Clinicaltrials.gov, ISRCTN23349893 (Retrospectively registered: 16/02/2017).

Enhancement of self compassion in psychotherapy: The role of therapists' interventions

Aim: Self Compassion (SC) has been consistently linked to decreased emotional distress and is offered as a mechanism of change in several therapeutic approaches. The current study aimed to identify therapists' interventions that enhance clients' SC within individual psychodynamic psychotherapy. We examined a diverse set of interventions as predictors of clients' SC, on treatment and session levels. We hypothesized that improvement in SC will be associated with greater use of directive or common factor interventions. Method: Client/therapist (N = 89) dyads from a university-based community clinic participated in the study. Therapists' interventions and changes in clients' SC level were monitored at each psychotherapy session. Results: Clients' SC in a given session was not predicted by therapist use of interventions from any of the three clusters in the previous session. However, positive change in SC across treatment was predicted by greater use of directive interventions. Furthermore, among clients with low pretreatment SC, a positive change in SC across treatment was predicted by lesser use of common factor interventions. Discussion: The results highlight the importance of understanding clients' pretreatment characteristics when selecting therapeutic interventions and suggest that the integration of directive interventions into the psychodynamic therapeutic practice may be beneficial in enhancing clients' SC.

Protocol for a randomised controlled feasibility study examining the efficacy of brief cognitive therapy for the Treatment of Anxiety Disorders in Adolescents (TAD-A)

Background

Anxiety disorders affect a quarter of the population during their lifetime, and typically emerge in childhood or adolescence. Anxiety disorders disrupt young people’s social, emotional and academic development and in the absence of treatment, often follow a chronic course. Although effective treatments, such as Cognitive Behaviour Therapy (CBT), exist, only a small proportion of adolescents with anxiety disorders who need treatment receive them. Barriers to treatment provision include the fact that CBT typically requires 14–16 sessions by a highly qualified therapist and services are stretched – resulting in lengthy waiting lists and limited access to treatment. This highlights the importance of developing new ways of providing effective treatments for adolescent anxiety disorders. This study aims to assess the feasibility of a future, large-scale trial. This will give a clear indication of the likely success of running a randomised controlled trial to compare a new, brief cognitive therapy treatment to an existing CBT group therapy for adolescents with anxiety disorders.

Methods/design

The study will examine whether a definitive trial can be conducted on the basis of a feasibility RCT using a number of well-defined criteria. The feasibility RCT is a single-centre, randomised control trial. Forty-eight Young people (age 11–17.5 years) attending a university research clinic, who meet the diagnostic criteria for a DSM-5 anxiety disorder, will be randomly allocated to receive either (1) Adolescent Cognitive Therapy for Anxiety (ACTA), which involves six 60–90-min sessions and a booster session or (2) group CBT, which involves eight 2-h sessions and a booster session. As part of the feasibility indicators, patient outcomes, expectations and experiences, as well as health economic factors, will be assessed before, at the end of treatment and at a 3-month follow-up.

Discussion

The successful delivery of a future, definitive trial has the potential to bring direct benefits to young people and their families, adolescent mental health service providers, as well as benefits to adult mental health services and society more broadly by disrupting the negative trajectory commonly associated with adolescent anxiety disorders.

Trial registration

ISRCTN, ID: ISRCTN86123204. Retrospectively registered on 23 November 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3295-6) contains supplementary material, which is available to authorized users.

The ART of conversation: feasibility and acceptability of a pilot peer intervention to help transition complex HIV-positive people from hospital to community

OBJECTIVES

To pilot a peer-based intervention for people living with HIV who used substances, had challenges with antiretroviral adherence and would be discharged from hospital to community.

STUDY DESIGN

A community-based, quasi-experimental pilot intervention study designed to assess feasibility, acceptability and connection to a community-based HIV organisation.

SETTING

This study was conducted in Toronto, Canada, at Casey House (CH; hospital for people living with HIV) in collaboration with the AIDS Committee of Toronto (ACT; community-based HIV organisation).

PARTICIPANTS

People living with HIV who were CH inpatient between 1 April 2017 and 31 March 2018, struggled with antiretroviral adherence, actively used substances and would be discharged to community were eligible. Forty people met criteria, 19 were approached by an inpatient nurse and 17 consented. Average age was 48.8 years (SD=11.4), 58.8% were male and participants averaged 7.8 physical and mental health comorbidities (SD=3.1).

INTERVENTION

Titled 'The ART of Conversation', the three-pronged personalised intervention was developed through input from CH clients and ACT volunteers, all living with HIV. Intervention components were (a) predischarge goal-setting (adherence, substance use and self-identified goal) with the study nurse; (b) predischarge meeting with an HIV+ peer volunteer (PV) and (c) nine postdischarge phone calls between PV and participant, once per day for 3 days, then once per week for 6 weeks.

PRIMARY OUTCOMES

Feasibility was measured through proportion of eligible participants recruited and PV availability. Acceptability was assessed through participant interviews at three times (preintervention, post-intervention and 6 weeks follow-up) and through PV call logs. Client records determined connection to ACT within the study timeframe.

RESULTS

Twelve participants completed the intervention and nine connected with ACT. Predischarge goal-setting and PV meeting were both feasible and acceptable. Postdischarge phone calls were a challenge as half of completers missed at least one call.

CONCLUSIONS

Although predischarge goal-setting and PV meeting were feasible, methods to maintain connection following discharge require further investigation.

Becoming an Evidence-Based Practitioner

Mental health and substance use disorders co-occur frequently, and are associated with poorer outcomes in life domains including housing, employment, health, and recovery. Finding evidence-based interventions for engagement and recovery can be a challenge for practitioners and organizations, as it involves accepting new interventions, and then implementing and measuring the results. However, practitioners frequently use their opinions or non-generalizable experiences rather than evidence-based findings to guide their practice. Medication-assisted therapy programs, especially for individuals with co-occurring mental health and substance use disorders, is an area of treatment where there are solid evidence-based outcome findings and where, nonetheless, many practitioners continue to use less-, or non-effective treatment approaches. Conflict between groups of staff using two different approaches can have serious negative impact on treatment outcome. These can be effectively addressed through a combination of education and interventions aimed at resolving intra-staff conflict.

An Internet-based Acceptance and Commitment Therapy intervention for older adults with anxiety complaints: study protocol for a cluster randomized controlled trial

Background

Anxiety is among the most prevalent and disabling mental health problems in older adults. Few older adults with mild to moderately severe anxiety symptoms receive adequate interventions, putting them at risk for developing anxiety disorders, depression, and various somatic problems. Effective, low-threshold interventions should be developed. Blended care, in which a web-based intervention is combined with a limited amount of face-to-face contacts with a mental healthcare counselor at the general practice, is a promising option. The online self-help intervention “Living to the Full”—an Acceptance and Commitment Therapy (ACT) intervention—has been proven to reduce depression and anxiety in several patient groups, but has not yet been investigated in older adults. The aim of this study is to evaluate the (cost-)effectiveness of a blended form of “Living to the Full” in reducing anxiety symptoms in adults aged 55 to 75 years. Furthermore, moderators and mediators of the treatment effect are investigated.

Methods/design

The (cost-)effectiveness of the ACT intervention will be investigated in a cluster single-blind randomized controlled trial (RCT). The blended intervention will be compared to treatment-as-usual. Thirty-six mental health counselors working at general practices in the Netherlands will be randomized to deliver blended care or treatment as usual. A total of 240 participants (aged 55–75 years) with mild to moderately severe anxiety complaints (defined as a total score of 5–15 on the GAD-7) will be recruited. There are four measurements consisting of online questionnaires (primary outcome: GAD-7) and a telephone interview: before the start of the intervention; directly following the intervention (14 weeks after baseline); and six and twelve months after baseline. Possible mediator variables will be assessed multiple times basis during the intervention.

Discussion

This RCT will evaluate the effectiveness of a blended ACT intervention for older adults with anxiety symptoms. If the intervention is shown to be effective, it will be implemented, thereby improving the accessibility and quality of preventive interventions for older adults with anxiety problems.

Trial registration

Netherlands Trial Register, NTR6270. Registered on 21 March 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2731-3) contains supplementary material, which is available to authorized users.

Guided self-help to manage binge eating in a dietetic-led community weight management service

An estimated 30% of patients accessing community weight management services experience symptoms of binge eating disorder (BED). Guided self-help (GSH) is the recommended first line of treatment for BED. This study is a preliminary investigation into the effectiveness of GSH delivered by dietitians for patients with binge eating within a weight management service and a consideration of the association between wellbeing, therapeutic relationship and outcomes. The study was conducted as a single group, pre- and post-intervention study with 24 patients reporting symptoms of binge eating who completed the self-help manual with guidance from a trained community dietitian. Primary outcomes were eating disorder psychopathology and behaviours (Eating Disorder Evaluation Questionnaire), depression and anxiety. Principle results showed a significant reduction on all subscales of eating disorder psychopathology, anxiety and depression. There was a reduction in loss of control over eating but the 40% reduction in binge episodes was not statistically significant. Mid-treatment sessional ratings were positively associated with outcome. In conclusion, the GSH intervention was appropriate for dietitian delivery to patients with obesity and binge eating behaviour. This research indicates potential for other dietetic-led weight management services to deliver such interventions and support patients with binge eating accessing their service.

Psychometric Properties of the Outcome Rating Scale (ORS) in a Spanish Clinical Sample

The Outcome Rating Scale (ORS) is an ultra-brief measure of well-being designed to track outcome in psychotherapy. This research studied the psychometric properties of the ORS in a Spanish clinical sample. One-hundred and sixty-five adult participants from different primary care centers of the city of Barcelona were recruited. The psychometric properties of the ORS in the sample were explored and described, comparing them to the properties of other instruments already validated in Spain. Our results showed good reliability (α = .91 [.88, .93]; α = .96; test re-test correlations from .61 to .84), good validity (convergent validity correlations with distress and symptoms measures from -.32 to -.76), and good sensitivity to change (pre-post comparison through Wilcoxon signed-rank test, Mdnpre = 31.0, Mdnpost = 19.6, z = -7.38, p < .05, r = .42). These results are consistent with previous findings in other countries. We conclude suggesting that the instrument can be applied to monitor outcome in psychotherapy and to test the effectiveness of treatments imparted with Spanish speaking clients. However, further research with the ORS in Spanish could provide more evidence of its psychometric properties.

The EIS Model: A Pilot Investigation of a Multidisciplinary Sex Therapy Treatment

Marital couples not experiencing a satisfying sexual relationship are less satisfied in their overall relationship. There is, however, a paucity of controlled outcome studies researching the effectiveness of sex therapy interventions. This study was a pilot investigation of a new manual-based sex therapy called the EIS (empathy, intimacy, and sexual satisfaction) model. Results showed significant improvement in sexual and marital satisfaction, marital intimacy, and verbal sexual intimacy for men and women, and sexual function for women. Effect sizes were large for these variables and posttreatment differences. Results provide initial support for the theory and interventions of the EIS model. Treatment should include addressing vulnerable communication, relational intimacy, conflict resolution, sexual functioning, sexual medicine care, and sensual and sexual skills.

Evaluating Integrated Mental Health Care Programs for Children and Youth

Evaluations of integrated care programs share many characteristics of evaluations of other complex health system interventions. However, evaluating integrated care for child and adolescent mental health poses special challenges that stem from the broad range of social, emotional, and developmental problems that need to be addressed; the need to integrate care for other family members; and the lack of evidence-based interventions already adapted for primary care settings. Integrated care programs for children's mental health need to adapt and learn on the fly, so that evaluations may best be viewed through the lens of continuous quality improvement rather than evaluations of fixed programs.

Recovering Process from Child Sexual Abuse During Adulthood from an Integrative Approach to Solution-Focused Therapy: A Case Study

In recent times, strengths-based recovery approaches that focus on the present and build strategies that look toward the future have become popular. However, some cases require the consideration of experiences from previous stages of the clients' development. This single-case study explores the psychotherapeutic process of a middle-aged woman who presented with a history of child sexual abuse (incest) and a long-term adult diagnosis of depression that was treated in public health services. This psychotherapy involved an integrative approach to solution-focused therapy; specifically, the approach proposed by Yvonne Dolan to work with adult survivors of sexual abuse, in conjunction with techniques and strategies from the transtheoretical model. Measures incorporating therapeutic working alliance and outcomes were administered over sessions. Results showed positive outcomes from this therapeutic intervention, which remained at 3-month and 12-month follow-ups. Implications for practitioners' specialist practice in health services are discussed, given the complexity of comorbid mental health conditions with a history of child sexual abuse.

The Solving Problems in Everyday Living Model: Toward a Demedicalized, Education-Based Approach to “Mental Health”

We argue that human existential pain and threat may usefully be helped by a noncoercive educational approach that also resonates with many interpersonally focused psychological approaches, rather than by the widely touted current medical model of “mental health” treatment (using psychoactive drugs and supportive psychotherapy). First, the “progress” leading to the latest Diagnostic and Statistical Manual of Mental Disorders is briefly reviewed, highlighting the scientific limitations of the medical model. Next, an educational model of self-understanding and change, based on Popper’s fallibilism, Freire’s critical pedagogy, and Miller’s feedback-informed treatment is explicated. Finally, some options for funding and testing the model are discussed. We hope this offers mental health clinicians another important alternative to conceptualize the helping encounter to ameliorate personal problems in living.

Beyond Critique: The Partners for Change Outcome Management System as an Alternative Paradigm to Psychiatric Diagnosis

Critics claim that current psychiatric diagnostic systems lack reliability, validity, and clinical utility; are incompatible with known evidence of how change occurs in psychotherapy; are compromised by bias; and risk harmful effects for clients. This article argues that the Partners for Change Outcome Management System (PCOMS), a transparent, egalitarian process that collects and utilizes client feedback at each session, convincingly addresses these concerns. Furthermore, it suggests that PCOMS offers a viable alternative to the reimbursement and administrative functions of the Diagnostic and Statistical Manual of Mental Disorders and the International Classification of Diseases. The authors propose that PCOMS represents a radical realignment of the practitioner/client relationship via full, dialogical partnership at every level of psychotherapy practice and thus constitutes a step toward a new paradigm that reconnects psychotherapy and humanistic psychology with its core relational values.

Strengthening the Planning Process of Motivational Interviewing Using Goal Attainment Scaling

Client-centered, outcomes-based mental health counseling is driving the behavioral health field toward identifying mechanisms to facilitate specific, measurable goal setting and tracking with clients. Motivational interviewing (MI) is an evidence-based, collaborative, goal-oriented style of counseling; however, the fourth process of MI, planning, is not as well defined as the other MI processes, especially related to goal setting. Goal attainment scaling (GAS) is a method for setting measurable goals and assessing clinical progress. The combination of MI and GAS has potential to be a valuable clinical tool to establish client-centered goals, monitor goal attainment, and provide feedback within clinical mental health settings. Therefore, we propose that the MI planning process can be enhanced by incorporating GAS. However, research is needed to substantiate the feasibility of the proposed integration. A brief case study is provided to illustrate key concepts.

Casting a wider net in behavioral health screening in primary care: a preliminary study of the Outcome Rating Scale

Introduction The integration of behavioral health services into primary care has led to enhanced use of brief screening measures to identify mental health problems. Although useful, such instruments are largely symptom based and diagnosis specific. This narrow focus can potentially limit the identification of broader social or relational distress in patients that affect medical outcomes, as well as present feasibility challenges using a multi-measure approach in identifying mental health comorbidities.

METHOD

This exploratory study of adult primary care patients compared an ultra-brief, and widely used measure of global distress across life functioning, the Outcome Rating Scale (ORS), with the Patient Health Questionnaire (PHQ-9 and PHQ-2).

RESULTS

Correlations between the ORS and the PHQ-9 and PHQ-2 indicated agreement between the measures in classifying patients, and the ORS identified significantly more patients in the clinical range. Discussion Although results are preliminary, the ORS may cast a wider net in identifying patients with significant distress in primary care.

Feasibility and Preliminary Efficacy of a Novel RDoC-Based Treatment Program for Adolescent Depression: "Training for Awareness Resilience and Action" (TARA)-A Pilot Study

BACKGROUND

The novel group treatment program Training for Awareness, Resilience, and Action (TARA) was developed to target specific mechanisms based on neuroscience findings in adolescent depression and framed within the National Institute of Mental Health Research Domain Criteria. TARA contains training of autonomic and emotional self-regulation, interoceptive awareness, relational skills, and value-based committed action.

METHODS

We performed a single-arm trial to test the feasibility and preliminary efficacy of TARA in reducing depression and anxiety levels and assessed whether the specific targeted domains of function reflected the hypothesized symptom change. Twenty-six adolescents (14-18 years old, 7 males and 19 females) participated in the 12-week group program. Assessment was performed before (T0), immediately after (T1), and 3 months after the end of TARA (T2).

RESULTS

Significant improvement was seen in depression symptoms (Reynolds Adolescent Depression Scale Second Edition) between T0-T1 (t-value = -3.56, p = 0.002, CI = -6.64, -1.77) and T0-T2 (t-value = -4.17, p < 0.001, CI = -11.20, -3.75) and anxiety symptoms (Multidimensional Anxiety Scale for Children) between T0-T1 (t-value = -2.26, p = 0.033, CI = -4.61, -0.21) and T0-T2 (t-value = -3.06, p = 0.006, 95% confidence interval = -9.02, -1.73). Significant improvements in psychological flexibility, sleep, and mindfulness skills were also found between T0 and T2.

LIMITATIONS

The sample size was small without a control condition. The pilot design did not allow for testing the hypothesized brain changes and effect of TARA on relevant systemic biomarkers.

CONCLUSION

TARA is feasible in a sample of clinically depressed and/or anxious adolescents and preliminary efficacy was demonstrated by reduced depression and anxiety symptoms. The specific symptom and behavioral outcomes corresponded well with the hypothesized mechanisms of change.

EARLY CHILDHOOD MENTAL HEALTH CONSULTATION: AN EVALUATION OF EFFECTIVENESS IN A RURAL COMMUNITY

Little research has been done to evaluate the effectiveness of early childhood mental health consultation (ECMHC) in rural, applied settings. In this mixed-methods study, we evaluated an approach to ECMHC used in rural Southwest Kansas with individualized services for childcare providers. Twenty-nine home-based and center-based childcare providers completed measures on provider growth, perceptions of child outcomes, and satisfaction with sessions. In total, 162 data points were collected and analyzed using multilevel growth models. In addition, 16 providers participated in qualitative interviews. Both home-based and center-based providers reported very high satisfaction with consultation sessions which increased with time, although home-based providers showed significantly higher satisfaction than did center-based providers. Provider growth, encompassing personal well-being, scheduling and transitions, connections with parents, and positive discipline strategies increased significantly over time. Child outcomes, encompassing prosocial behavior, resilience, and overall well-being also improved significantly in providers' perception. ECMHC as conducted in Southwest Kansas appears to have a positive effect on childcare providers and the children in their care.

Enhancing student occupational therapists' client-centred counselling skills

BACKGROUND

Client-centred practice is the cornerstone of the occupational therapy profession. However, there has been little focus on how to teach students to be client-centred practitioners while engaged in counselling.

PURPOSE

The purpose of this study was to examine the impact of the use of a client-completed rating scale on student occupational therapists' client-centred counselling skills.

METHOD

A time-series design was used to measure the changes in students' counselling skills over time. Demographic information was collected prior to time one. An online questionnaire was administered after study completion to explore students' experiences of using the Session Rating Scale.

FINDINGS

The impact of using the Session Rating Scale as a measure of students' client-centred counselling skills performance significantly improved over time. Most students valued using the rating scale and would recommend its use for future students.

IMPLICATIONS

The process of supporting students to learn how to engage clients in providing timely feedback and using this feedback to design treatment sets the stage for integration and application of client-centred practice.

The alliance in psychiatric rehabilitation: Client characteristics associated with the initial alliance in a supported employment program

BACKGROUND

The therapeutic alliance is associated with outcomes for individuals in psychiatric rehabilitation programs. However, there are no known studies examining factors associated with the alliance in these programs.

OBJECTIVE

To determine whether client characteristics associated with the alliance across multiple populations and program types were associated with the alliance for individuals enrolling in a supported employment program.

METHODS

Previously collected data were available for 70 individuals served by the single employment specialist in the program. Potential independent variables and the strength of their relationship to the alliance were identified, then further explored using a forward stepwise logistic regression model.

RESULTS

Seventy-three percent of the sample had the highest possible alliance rating. Being older and having a smaller social network size were significantly correlated with a high alliance, but neither remained a predictor of the alliance, controlling for the effects of the other.

CONCLUSIONS

Client age and social network size were associated with the initial alliance, though not in a model containing both variables. Low power may have hindered the ability to show additional relationships, and the uniqueness of the program structure limits the generalizability to other programs. Future research should investigate the variables examined further.

Assessing Brief Changes in Adolescents' Mood: Development, Validation, and Utility of the Fast Assessment of Children's Emotions (FACE)

Mood states of youth have a strong influence on their cooperation, comfort, and engagement in many health care and educational settings. Children who are fearful, angry, or sad are more likely to have difficulty learning new skills or connecting with others. Many interventions are used in hospital and school settings to help youth, but it is difficult to assess their effectiveness without appropriate assessment tools that are easy to administer, age appropriate, and psychometrically sound. We examined the validity and reliability of the Fast Assessment of Children's Emotions (FACE). After obtaining parental consent and youth assent, 61 patients ages 12 to 17 years were recruited from the psychiatry services at a large children's hospital. Participants completed the FACE, the Brunel Mood Scale (BRUMS), and a measure of satiety at three time points-before and after a 60-minute psychotherapeutic intervention and after lunch. The FACE measure was significantly correlated with the BRUMS (r(2) = 0.85; p < .001) and not correlated with the satiety measure (r(2) = -0.17; not significant). Cronbach's α for the FACE was 0.7734. The FACE showed significant changes in mood from before to after the therapeutic intervention for all patients. For general psychiatry patients, the FACE did not change significantly after lunch, although for patients with eating disorders, the FACE did indicate an increase in distressed emotions after lunch. This finding indicates sensitivity to change in a clinically meaningful manner. The FACE is easy to use and may be used quickly to assess mood changes in adolescents.

Measurement Issues: Review of four patient reported outcome measures: SDQ, RCADS, C/ORS and GBO - their strengths and limitations for clinical use and service evaluation

BACKGROUND

There is an international drive for routine use of Patient Reported Outcome Measures (PROMs) across all health services including in relation to Child and Adolescent Mental Health Services (CAMHS). A number of reviews have summarized the validity and reliability of well-being and mental health measures for children but there are fewer attempts to consider utility for routine use.

METHOD

This review considers four child self-report measures: the Strengths and Difficulties Questionnaire (SDQ), the Revised Child Anxiety and Depression Scale (RCADS), (Child) Outcomes Rating Scale (C/ORS) and Goals Based Outcomes (GBOs). It explores the strengths and limitations of each and considers how they can be used to support both clinical practice and service evaluation.

RESULTS

There is evidence for the clinical utility of RCADS, C/ORS and GBOs, although the utility of the SDQ as a feedback measure remains unclear. For service evaluation, the SDQ has the greatest evidence for norms making it useful for comparison and there is evidence that the RCADS may be the most sensitive to change of the measures reviewed; C/ORS has issues around ceiling effect, data error and data manipulation. More research is required around GBOs before their use for service evaluation can be determined.

CONCLUSIONS

In summary, these different measures may be viewed as complementary tools and determining the best way to make use of them severally and individually in clinical and community settings is a current focus for child mental health practitioners.

Development and Validation of the Schedule for Nonadaptive and Adaptive Personality Brief Self-Description Rating Form (SNAP-BSRF)

The Schedule for Nonadaptive and Adaptive Personality full-length (SNAP) and short versions (SNAP-SRF and SNAP-ORF) were developed as measures of normal-range and more pathological personality traits. This study investigated the validity of the SNAP Brief Self-Description Rating Form (SNAP-BSRF), an alternative version of the SNAP Self-Description Rating Form (SNAP-SRF) revised for further brevity. The scales of the SNAP-BSRF showed good convergence with the SNAP-SRF and the SNAP Other-Description Rating Form (SNAP-ORF) scales. A three-factor structure consistent with extant literature was found for the SNAP-BSRF using an exploratory structural equation modeling approach. Scales from the SNAP-BSRF showed meaningful associations with self-reported internalizing symptoms. Results suggest that this new version is a reasonable substitute for the SNAP-SRF and will be useful when a very brief measure of adaptive and maladaptive personality is needed.

Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment

This study investigated whether routine monitoring of client progress, often called "client feedback," via an abbreviated version of the Partners for Change Outcome Management System (PCOMS) resulted in improved outcomes for soldiers receiving group treatment at an Army Substance Abuse Outpatient Treatment Program (ASAP). Participants (N = 263) were active-duty male and female soldiers randomized into a group feedback condition (n = 137) or a group treatment-as-usual (TAU) condition (n = 126). Results indicated that clients in the feedback condition achieved significantly more improvement on the outcome rating scale (d = 0.28), higher rates of clinically significant change, higher percentage of successful ratings by both clinicians and commanders, and attended significantly more sessions compared to the TAU condition. Despite a reduced PCOMS protocol and a limited duration of intervention, preliminary results suggest that the benefits of client feedback appear to extend to group psychotherapy with clients in the military struggling with substance abuse.

Accounting for therapist variability in couple therapy outcomes: what really matters?

This study examined whether therapist gender, professional discipline, experience conducting couple therapy, and average second-session alliance score would account for the variance in outcomes attributed to the therapist. The authors investigated therapist variability in couple therapy with 158 couples randomly assigned to and treated by 18 therapists in a naturalistic setting. Consistent with previous studies in individual therapy, in this study therapists accounted for 8.0% of the variance in client outcomes and 10% of the variance in client alliance scores. Therapist average alliance score and experience conducting couple therapy were salient predictors of client outcomes attributed to therapist. In contrast, therapist gender and discipline did not significantly account for the variance in client outcomes attributed to therapists. Tests of incremental validity demonstrated that therapist average alliance score and therapist experience uniquely accounted for the variance in outcomes attributed to the therapist. Emphasis on improving therapist alliance quality and specificity of therapist experience in couple therapy are discussed.

Measuring Feedback From Clients

Treatment results can be improved by obtaining feedback from clients concerning their progress during therapy and the quality of the therapeutic relationship. This feedback can be rated using short instruments such as the Outcome Rating Scale (ORS) and the Session Rating Scale (SRS), which are being increasingly used in many countries. This study investigates the validity and reliability of the Dutch ORS and SRS in a large sample of subjects (N = 587) drawn from the clients of an outpatient mental healthcare organization. The results are compared to those of previous Dutch and American studies. While both the ORS and the SRS exhibited adequate test-retest reliability and internal consistency, their concurrent validity was limited (more for the SRS than for the ORS). New standards are proposed for the Dutch ORS and SRS. The scores obtained with these standards are interpreted differently than those obtained using American standards. The clinical implications of the limited validity of the ORS and the SRS are discussed, as is the use of different standards in conjunction with these instruments.

Effect size calculations for the clinician: methods and comparability

OBJECTIVE

The measurement of clinical change via single-group pre-post effect size has become increasingly common in psychotherapy settings that collect practice-based evidence and engage in feedback-informed treatment. Different methods of calculating effect size for the same sample of clients and the same measure can lead to wide-ranging results, reducing interpretability.

METHOD

Effect sizes from therapists-including those drawn from a large web-based database of practicing clinicians-were calculated using nine different methods.

RESULTS

The resulting effect sizes varied significantly depending on the method employed. Differences between measurement methods routinely exceeded 0.40 for individual therapists.

CONCLUSIONS

Three methods for calculating effect sizes are recommended for moderating these differences, including two equations that show promise as valid and practical methods for use by clinicians in professional practice.

Outcome Orientated Child and Adolescent Mental Health Services (OO-CAMHS): a whole service model

The international evidence base on factors that most influence outcomes in mental health care finds that matching therapeutic intervention to diagnosis has a clinically insignificant impact on outcomes. Decades of outcome research into treatment of psychiatric disorders shows that, despite the development of many new techniques, the outcomes being achieved in studies 30 years ago are similar to those being achieved now. In the last few years, new service models that incorporate systems of feedback on progress and alliance have emerged and show promise with regards improving overall outcomes for mental health service users. Growing familiarity with this outcome literature, together with a desire to be part of a service that can continue to improve patient outcomes, led a small community Child and Adolescent Mental Health Services team to develop a new whole service model - Outcome Orientated Child and Adolescent Mental Health Services (OO-CAMHS). OO-CAMHS incorporates key aspects of the evidence base on what could make a differential positive impact on outcomes and relinquishes those aspects that do not. In this paper, we outline the evidence base on which OO-CAMHS is built, describe the key features of the approach and present some of the early findings on its impact.

Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach

OBJECTIVE

In the modern healthcare system, rapidly expanding costs/complexity, the growing myriad of treatment options, and exploding information streams that often do not effectively reach the front lines hinder the ability to choose optimal treatment decisions over time. The goal in this paper is to develop a general purpose (non-disease-specific) computational/artificial intelligence (AI) framework to address these challenges. This framework serves two potential functions: (1) a simulation environment for exploring various healthcare policies, payment methodologies, etc., and (2) the basis for clinical artificial intelligence - an AI that can "think like a doctor".

METHODS

This approach combines Markov decision processes and dynamic decision networks to learn from clinical data and develop complex plans via simulation of alternative sequential decision paths while capturing the sometimes conflicting, sometimes synergistic interactions of various components in the healthcare system. It can operate in partially observable environments (in the case of missing observations or data) by maintaining belief states about patient health status and functions as an online agent that plans and re-plans as actions are performed and new observations are obtained. This framework was evaluated using real patient data from an electronic health record.

RESULTS

The results demonstrate the feasibility of this approach; such an AI framework easily outperforms the current treatment-as-usual (TAU) case-rate/fee-for-service models of healthcare. The cost per unit of outcome change (CPUC) was $189 vs. $497 for AI vs. TAU (where lower is considered optimal) - while at the same time the AI approach could obtain a 30-35% increase in patient outcomes. Tweaking certain AI model parameters could further enhance this advantage, obtaining approximately 50% more improvement (outcome change) for roughly half the costs.

CONCLUSION

Given careful design and problem formulation, an AI simulation framework can approximate optimal decisions even in complex and uncertain environments. Future work is described that outlines potential lines of research and integration of machine learning algorithms for personalized medicine.

Teaching accountability: using client feedback to train effective family therapists

The AAMFT Task Force on Core Competencies (Nelson et al., 2007) proposed that marriage and family therapy (MFT) educators teach and provide evidence of trainee competence beyond coursework and accrued clinical hours. This article describes the integration of a systematic client feedback protocol into an MFT-accredited program's curricula to address the call for outcome-based learning. Outcome management (OM) provides a framework for teaching and assessing trainee effectiveness. Continuous incorporation of client feedback embodies collaborative, strengths-based, integrative, and diversity-centered program values. Students learn a system for being accountable to clients, the profession, and service communities.