—by Nicole Clevenger and Paul M. Kubek
Cleveland, OH—The Ohio SE CCOE is now using a revised fidelity scale to assist mental health service organizations in Ohio that are implementing the evidence-based Supported Employment (SE) model. Evaluators from the CCOE have already conducted fidelity reviews using the new scale at several organizations across the state and plan to conduct at least one review with the new scale at all evidence-based SE programs in Ohio by the end of the year. The consultants are also helping staff at each mental health organization compare current and previous fidelity results. The Ohio SE CCOE is an initiative of the Center for Evidence-Based Practices.
Why Change the Fidelity Scale?
The evidence-based SE model and fidelity scale were originally designed by researchers at the New Hampshire-Dartmouth Psychiatric Research Center (PRC). The researchers have been tracking fidelity and outcomes data for the model over time from multiple sites across the United States and have used this knowledge to revise the fidelity scale. The evidence-based SE model's core principles and components remain unchanged.
The revised fidelity scale was released by the New Hampshire-Dartmouth PRC in January 2008 and is helping organizations implement the principles and practices of SE in a way that enhances elements of the practice that are believed to improve competitive employment outcomes for consumers in recovery from severe mental illness. Some notable enhancements to the current 25-item instrument include the following:
- The scale measures the quantity and quality of face-to-face contacts with employers made by employment specialists during job-development activities in the community.
- The scale measures the level of collaboration between employment specialists at the mental-health service organization that is providing evidence-based SE services and counselors at the local vocational rehabilitation (VR) office.
- The scale emphasizes that each consumer's preference for disclosure to employers about his or her mental illness is a "key element" to client choice.
Review the new SE fidelity scale online.
What Is Supported Employment?
In behavioral healthcare settings, the generic term supported employment is often used to describe a wide variety of vocational supports and services. However, Supported Employment, the evidence-based practice, is a very specific way of providing employment services to people diagnosed with severe mental illness. Research demonstrates that providing evidence-based employment services with high fidelity to the model generates improved consumer outcomes consistently over time. The evidence-based SE model has seven distinct principles that distinguish it from traditional vocational services.
What Is Fidelity?
In the context of implementing an evidence-based practice (EBP), the term fidelity refers to how closely a service organization follows the EBP model in everyday practice. Research shows that organizations which maintain fidelity to the original design of an EBP like Supported Employment achieve and sustain the best outcomes for the intended population(s) and, thus, are the most effective at helping improve the quality of life for consumers and their families.
What Is A Fidelity Scale?
A fidelity scale is a measurement and evaluation instrument that serves several purposes:
- Represents the organizational, service, and staffing characteristics (e.g., philosophy, policy, professional behavior) that are believed to embody the core components of the EBP model in the delivery of services.
- Guides organizations with the implementation of all components of the model.
- Provides evaluators with an instrument to examine implementation and offer feedback to each service organization about the progress of their SE implementation.
- Allows for consistency of SE implementation across all organizations, which are located in communities worldwide: such consistency is essential for comparing treatment outcomes across multiple sites and attributing outcomes to the intervention.
- Provides evaluators with a consultation tool.
What Is A Fidelity Assessment?
A fidelity-assessment team comprised of reviewers external to the agency uses the fidelity scale to conduct a fidelity review (or assessment), which provides a formal mechanism for evaluation of EBP services. The fidelity review is not an audit or accreditation process. It is strictly a quality-improvement process that provides organizations with information to make decisions about next steps. After the fidelity review is completed, the following typically occur:
- External fidelity reviewers evaluate all the data that they collect from multiple sources (e.g., chart reviews; interviews with service-team members, administrators, and consumers; real-time observations of team meetings and service team-members at work, etc.) and generate fidelity ratings for all items outlined in the fidelity scale.
- The reviewers present a comprehensive report to the organization: the report contains all fidelity ratings and recommendations for specific steps to improve fidelity and, thus, related service outcomes.
- Based upon recommendations from the fidelity report, the organization writes a fidelity action-plan, outlining manageable next steps that it identifies as priorities for enhancement of SE services.
- The evaluators/consultants work with the organization and its stakeholders to develop the fidelity action-plan.
Fidelity-Review Process
For more information about the fidelity-review/assessment process, consult these resources:
Ric Kruszynski, et. al. (2006). Implementing IDDT: A step-by-step guide to organizational change for evidence-based practices, p24-25, 30. Cleveland, OH: Center for Evidence-Based Practices.
Deborah R. Becker, et. al. (2008). "Evidence-Based Supported Employment Fidelity Review Manual", Chapter 3, page 15.
Contact Us
For more information about implementing Supported Employment, the evidence-based practice, contact the following:
Patrick E. Boyle, MSSA, LISW-S, LICDC, director of implementation services
patrick.boyle@case.edu