Research in Action May 2016
Collaborative Partnership with Boston College
ServiceNet’s partnership with Boston College researcher enhances our analysis of evidence-based practices
Over the past ten years, the Outpatient Services program at ServiceNet has been working with Dr. Thomas O’Hare, Associate Professor with the Boston College Graduate School of Social Work and creator of the PsychoSocial Wellbeing Scale (PSWS). This easy-to-use questionnaire measures clients’ cognitive functioning, emotional functioning, trauma-related symptoms, impulse control, coping skills, social relationships, recreational activities, material resources, substance use, health, self-care, work/role functioning, and legal problems. Dr. O’Hare also created the Practice Skills Inventory (PSI), which is designed to measure the frequency with which practitioners use core clinical skills that are closely related to a wide range of Evidence Based Practices (EBP).
In collaboration with Dr. O’Hare, ServiceNet has implemented these tools to record practitioner interventions and client outcomes so that we might further analyze the use of EBP in clinical care. The benefit of this work can be seen in the evolution of ServiceNet’s electronic medical record (EMR) for outpatients, which has included the PSWS since 2009. The PSWS is free, and can easily be added to a normal clinical assessment in the EMR to measure many domains of client well-being. It also serves as a useful tool for assessment, treatment planning, and evaluation. Based on the data we have collected with this tool, ServiceNet has added a summary graph of PSWS scores to our EMR, allowing clinicians to respond to assessment data in real time and to provide higher quality care.
In addition to the PSWS, Dr. O’Hare helped ServiceNet’s Outpatient Services team develop a list of the most frequently used clinical interventions and divided them into five distinct types: supportive skills, insight-oriented skills, cognitive-bahavioral coping skills, Acceptance and Commitment Therapy or Dialectical Behavior Therapy, and child and family treatment. This list was added to our EMR in 2010, which has enabled ServiceNet’s clinicians to save time by easily reporting standard interventions.
The culmination of our partnership with Dr. O’Hare was an article he co-authored with Jennifer Geertsma, Director of Outpatient Research for ServiceNet, in the journal Best Practices in Mental Health (O’Hare & Geertsma, 2013). Their article focused on the value of the Practice Skills Inventory as an evaluative tool for clinicians to use during therapy sessions in the field. They found separate patterns of skills were used to address different psychological and social problems; and that the skill patterns were consistent with published guidelines for evidence-based practices used to treat these problems. In combination, then, the PSWS and the PSI provide excellent means to evaluate the clinical impact of evidence-based practice for both client and clinician, as well as to guide clinicians in effective treatment implementation.
In collaboration with Dr. Ce Shen of the Boston College Graduate School of Social Work, Dr. O’Hare has now begun a longitudinal assessment of client change over time as measured by the PSWS. Thanks to the implementation of PSWS at ServiceNet across a wide variety of clinical programs, Doctors O’Hare and Shen have access to completed inventories spanning more than five years. This will provide their work with significant statistical power to draw conclusions about effective treatment based on a large amount of available evidence.
O’Hare, T., PhD, & Geertsma, J. H. (2013). Using the Practice Skills Inventory in Real Time: Implications for Evaluating Evidence-Based Practices” [Abstract]. Best Practices in Mental Health.
“The current study examined the reliability, construct validity, and concurrent validity of a modified version of the Practice Skills Inventory (PSI) with 916 outpatient mental health clients with various psychiatric disorders in order to test its practical and psychometric value in a naturalistic mental health outpatient treatment setting. The PSI showed very good internal consistency and good construct and concurrent validity for measuring supportive, insight-oriented, cognitive, and behavioral skills. One-way ANOVA demonstrated significant differences in skill use in patterns congruent with published guidelines for evidence-based practices, and regression demonstrated a differential use of skills for psychological and social problems. The PSI is recommended for use as part of routine evaluation of mental health practice.”
Reflections of an Undergraduate Research Intern
During the spring 2016 semester, I completed an undergraduate internship at ServiceNet, and it was the most hands-on and eye-opening experience of my academic career.
Through this internship placement, I worked with Jen Geertsma, Director of Applied Research; Seth Dunn, Director of Quality Management; and Ren Yao, Director of Operations for Mental Health Recovery Services. Each of these mentors gave me the chance to examine and analyze data from a variety of programs, including Dialectical Behavioral Therapy, Prevention and Recovery in Early Psychosis (PREP), Housing Plus, and Mental Health Recovery Services (MHRS).
My internship at ServiceNet left me with such a feeling of accomplishment. Everyone I worked with entrusted me with projects that were important to them, and which were incredible learning opportunities for me. For example, working with the Housing Plus program helped me realize that the case management ServiceNet does affects the lives of people in a variety of ways-with their mental health issues, and also with basics such as groceries, housing assistance, and legal services. Working with Jen Geertsma on a Substance Abuse and Mental Health Services Administration (SAMHSA) grant application was another positive and informative opportunity for me. If awarded, this grant will fund a program focused on helping members of the community who are chronically homeless: improving their lives, and hopefully integrating them back into society. I was so proud to be a part of that project.
Whether I was analyzing the results of quality control surveys, creating graphs and charts to assist the MRHS program, or entering data for the DBT program, I was never bored during my internship with ServiceNet. The opportunities I had there instilled many skills in me, which will no doubt benefit my future studies and career in Psychology. It was an incomparable experience, which I will always value.
– Rebecca Murrin, UMass Amherst, Psychology Department, Class of 2016
