Take a look at the Recent articles

Abstract

Objectives: The aim was to describe the initial phase of the implementation process in changing to evidence-based practices within a child and adolescent mental health service.

Method: Prior to the five-year project, an external service evaluation was carried out. The employees expressed a need for a clear direction from management to guide their clinical practice. Employees participated in the process of developing clinical standards during the first phase of implementation.

Results: Fixsen’s four stage model and the PSDA circle were used to guide the implementation process. The employees developed a template for a clinical standard based on national and international clinical guidelines. During the period, 17 clinical standards were established and 10 new evidence-based methods were implemented. All service leads and a group of senior clinicians were invited to participate in an evaluation five years after the initial service evaluation. There was overall agreement that the mental health service was developing positively providing high quality services for children and adolescents.

Conclusion: Involving employees in the implementation process seemed to be an important factor in changing a mental health service.

Keywords

clinical guidelines, healthcare, evidence-based medicine, clinical safety

Introduction

In line with an expansion in research and proven methods during recent decades, expectations of delivering evidence-based practice are increasing. In many ways this has challenged existing culture within mental health services where clinical autonomy has long been cherished and a disregard for clinical guidelines reported [1]. The difficulty in integrating research findings into clinical care has been described as the “research practice gap” [2,3].

Barriers to changing practice have been reported to include a number of factors such as administrative constraints (eg. lack of time), negative staff attitudes, anxiety about changing practice, information overload as well as a lack of culture and leadership [1,4]. There continues to be uncertainty about clinically meaningful and sustainable effects of clinical guidelines on patient outcomes and how best to implement such guidelines for maximal benefit [5,6]. While several promising implementation strategies are described [7], the evidence of their effectiveness remains less evaluated [8-12]. The general advice is therefore to use low cost approaches when planning organisational or clinical changes [4,7].

The use of evidence-based practice in mental health services is associated with proficient organizational cultures [13-16]. Clinicians who work in proficient cultures report that they are expected to be effective and there is evidence that mental health organizations with proficient cultures provide higher quality service and better outcomes [14,17-19].

Alongside organisational cultures the concept of employee-driven innovation (EDI) is receiving increasing attention [20,21]. Proponents of EDI suggest that in the course of their working days employees capture significant information and insight into the running of an organisation and its potential for improvement. Furthermore, they claim that EDI provides higher self-esteem to employees through empowerment. Alignment between employee-driven ideas and management priorities is important to ensure that ideas are implemented, to improve morale and showing employees that they are being listened to. In support of this claim, it appears that communication that makes use of active participation and involvement, using for instance audits and feedback systems, have greater effect than passive dissemination [10,11].

Training and performance management also have positive impact on patient and staff outcome [6,22]. Fixsen, et al. has identified that good implementation includes at least four stages; exploration, installation, first implementation and full implementation [8]. Although the initial exploration stage is considered particularly important, this is often neglected by managers [10].

With respect to the more practical ways to facilitate the change process, Deming’s PDSA circle (Plan-Do-Study-Act) is frequently used as a practical feedback system tool during all stages of implementation [23].

In summary, factors that facilitate the implementating of EBP include developing a culture that is positive to change as well as alignment between managers and staff in priorities for the service. Further, change requires good exploration, planning, communication as well as a commitment to training and involvement of employees. There continues to be a call for research to improve methods for implementation [4,7,13].

The present paper describes and evaluates the first stage of a five-year service quality improvement project. The overall aim of the project was to optimize clinical care through introducing a clear direction for the service, disseminating a range of evidence-based methods and identifying effective organisational models for low incidence client groups.

Methods

Background

The process of change took place in a child and adolescent mental health service (CAMHS) consisting of seven outpatient teams, two specialized teams and three hospital wards (Haukeland university hospital). This change process was part of a five-year project following a critical service evaluation.

Prior to the five-year project, an external service evaluation had been carried out by the hospital’s human resource (HR) department in 2012. The aim of this evaluation was to examine to what extent the department’s organizational structure and clinical practice were experienced as beneficial. The evaluation consisted of interviews with 40 individuals; service leads, key senior clinicians, trade union and safety representatives, service users and staff from collaborating units.

The results of the service evaluation highlighted satisfaction with levels of staff resources and expertise. But a perceived exclusive focus on operational and administrative issues, a failure to address the wide variety of approaches practiced within the service, and the absence of a clear strategy specifically to guide clinical practice were sources of significan disatisfaction.

CAMHS vison and strategy

Based on the recommendations from the evaluation carried out by HR in 2012, a working group consisting of managers, clinicians, union as well as user representatives, developed a vision and service strategy. The conclusion was a vision; “to develop excellence in service delivery, equality in access across the service, a patient centred approach and evidence based practice” and a service strategy which identified a range of clinical and organisational areas in need of development. The role of the key champion for implementing the vision and strategy was passed onto a member of staff who had previous experience in the area of implementation.

Implementing the first stage of the CAMHS vision and strategy

In addition to develop a plan for implementating new methods, consideration was given to the process of change. It was decided to involve employees in the development of the service and to use Fixsen’s four stages of implementation to structure the process [8]. The first stage, the exploration stage, involves creating a team, assessing needs, exploring evidence and usability of interventions. The process described below belongs to the exploration stage.

A steering group consisting of the director of service and two consultant psychiatrists, coordinated activities during the exploration stage (two members were part of the management team and all had background as researchers). The details of the involvement of the broader staff group is described below.

Clinical standards

The goal was to develop a tool that provided staff with quick access to relevant research literature and recommendations for evidence based practice. These tools would serve as clinical standards and contribute to identify the requested direction of the service. Success metrics would involve clinicians perceiving the standards as useful and that later service development adhered to recommendations.

In order to provide easy access to relevant research literature, it was decided to use Norwegian, English and American clinical guidelines as the main source of information. The term clinical guidelines refers to evidence based recommendations for health and care with the aim of optimizing patient care, how to diagnose and treat a range of medical conditions.

These are informed by a systematic reviews of evidence. The purpose of guidelines is to improve clinical effectiveness by implementation of evidence based care in daily practice [24,25].

Pairs of clinicians were asked to summarize relevant clinical guidelines. The summary of each guideline gave rise to discussions highlighting service gaps and needs. Key information from guidelines was operationalized to increase the feasability for clinical care. Clinicians created drafts of clinical standards. Five drafts of clinical standards underwent piloting before the final clinical standard template was established. The piloting took place within the service during a 6 months period.

Evaluation of the service development and process of change

In 2017, five years after the initial service evaluation, a review of achievements and a structured evaluation of employees’ perception of the service developments and process of change was conducted.

Evaluation of employees’ perception of service develompment and process of change

The HR department developed a structured questionnaire in collaboration with the CAMHS management team. The questions were consistently worded in a positive manner. There is a risk of acquiescence or extreme response bias when using only positively worded questions [26,27]. However it is well-known that including a mix of both positively and negatively worded questions can also create new challenges (such as confusion and incorrect answers) [28]. The questionnaire was distributed and collected electronically, asking about areas of implementation and clinical management.

The seven questions are presented in Table 1. Each question had a score where 1 was “totally disagree”, 2 “disagree”, 3 “both disagree and agree”, 4 “agree”, and 5 “completely agree”.

Table 1. Evaluation of implementation and Clinical management. A process of developing clinical standards in a CAMHS¹. Employees (45/60) participated answering the following seven questions as follow²

¹CAMHS: Child and adolescent mental health service (CAMHS) consisting of seven outpatient teams, three specialized teams and three hospital wards (Haukeland university hospital).

²Agreement score: Each question had a score where 1 was “totally disagree”, 2 “disagree”, 3 “both disagree and agree”, 4 “agree”, and 5 “completely agree”.

All employees or their equivalent replacements (if left the service) that were interviewed in 2012 were included (service leads and senior clinicians).

Statistical analyses

Descriptive analyses were used to describe the outcome measures with mean, standard deviation (Table 1, seven answers). We compared mean values of the seven answers for the senior clinicians (N=32) and the service leads (N=13) using Paired Sampled t-test. Secondly, a linear regression analysis was carried out using one question “CAMHS is developing in the right direction» as the dependent variable and two other questions as independent variables.

This was done to explore possible factors facilitating the development of the CAMHS. A p-value of <0.05 was considered statistically significant. The SPSS statistical package version 24.0 (IBM SPSS Statistics, IBM Corporation) was used for all analyses [29].

Results

Clinical standards

Initially the draft clinical standards provided by clinicians varied from 10 to 15 pages. As stated previously, the goal was to develop a tool that would provide staff with quick access to relevant research literature and recommendations for evidence based practice. To succeed, it was considered that standards should be possible to read by busy clinicians, perceived as clinically useful and that subsequent staff training adhered to recommendations.

A template limited to a two page, operationalized description included the following elements: When to use, diagnostic criteria, assessment, and treatment methods. In addition, there were notes recommending case formulation and regular points for evaluation. The standards included references to the clinical guidelines that formed the basis of the standards [24,25]. Clinicians responsible for summaries and the steering group negotiated the final content of the standards.

During the five-year period, 34 senior clinicians participated in developing altogether 17 clinical standards. Examples of the content of clinical standards included categories of disorders such as eating disorders, anxiety disorders, mood disorders, ADHD, Autism.

Clinical standards became available for all staff in the hospital “electronic quality control handbook” and updated as new guidelines have become available.

Although it could be argued that training and implementation of new methods belong to the implementation stages [8] it is, for the purpose of highlighting the effects of the clinical standards, considered relevant to report that between 2013 and 2017 staff received training in 10 new evidence based methods, all recommended in clinical guidelines and thus local clinical standards. The process of implementing these methods as well as the organisation of the service is not addressed in this paper.

Evaluation of perception of change and process

Evaluation of employee’s perception of change took place in 2017, five years following the initial evaluation and was performed asking seven questions. Results are given in Table 1. For the invited employees 45 of 60 (75%) completed the questionnaire in 2017. All service leads participated (13 (29%)). More than one third (18/45, 40%) of the respondents were also interviewed in the 2012 evaluation.

The results of the evaluation expressed that both service leads and clinicians tend to agree that the mental health service was developing positively, providing a high quality service for children and adolescents (Table 1). Furthermore, both groups agreed that the introduction of clinical standards was particularly important in ensuring quality care. The highest score was found for the statement “the development of clinical standards has contributed to clarify the clinical direction of the service” with a score of 4.5 (SD: 0.6). There were no differences in ratings between service leads and clinicians on this item.

On the question about how “they perceived themselves to be involved in developing clinical standards” there was a statistically significant difference between service leads and clinicians with the clinicians rating themselves as having been slightly less involved than service leads.

In order to explore employee involvement in the process, a linear regression analysis was carried out. The score of the question “CAMHS is developing in the right direction» was used as the dependent variable and the two variables; question 1) “The clinical standards have contributed to clarify the professional direction of CAMHS”, and question 2) “Employees in CAMHS have been positively involved in the development of clinical standards”, as independent variables. Only employee experience of involvement was significant with 14% of the explained variance of CAMHS development in the right direction (adjusted R: 0.14; 95% CI 0.1 to 0.7; p < 0.05).

Discussion

During a five-year period, managers and clinicians were involved in developing clinical standards as the first stage of implementing a CAMHS vision and strategy. The content of the standards was based on the advice given in clinical guidelines and contained operationalized and clinically relevant information. In addition, recommendations effectuated a staff-training plan. 17 standards were developed and 10 evidenced based methods were implemented during the time period.

All service leads and a group of senior clinicians participated in a structured evaluation of the process. There was an overall contentment with the service development and staff agreed that the introduction of clinical standards provided a clear direction for the service and quality of care.

Involving employees in the change process seemed to be one of the key factors in successfully changing the service. The knowledge that passive dissemination of information has limited effect on staff behaviour made us actively use EDI in the implementation process. We suggest that the participation of service leads and clinicians created ownership, engagement and a sense of responsibility for developing a shared culture, variables recognised as important in facilitating implementation [20]. The 2017 evaluation outcomes appears to support this understanding, where staff partialy attributed the positive service development to the involvement of employees. This finding is also in line with the notion of EDI [20].

An important aspect of the strategy was to establish evidence based knowledge in the clinical setting. One of the barriers to implementing clinical guidelines include information overload [1]. By creating clinical standards that presented the content of clinical guidelines succinctly, focusing only on key aspects of assessment and interventions, research information was transported into the clinical setting and as such contributed to reduce the “research practice gap” [2]. The fact that the recommendations set out in clinical standards were followed up with training and implementation of recommended methods, has most likely added to the positive perception of change [6].

Managers often ignore and disregard the stage of exploration as unnecessary and too time-consuming [5]. However, in this project the exploration stage was seen as essential in clarifying the strategy and to facilitate employees’ motivation for developing and adopting changes.

This is in line with previous reported findings that lack of time is often a barrier to changing practice [5].

Strength and limitations

The strength of this study was the focus on the challenging area of implementing evidence based practice in the real world of clinical care. This is important in order to maximise the quality and effectiveness of health care services.

There are however limitations to this report. A low cost approach together with a focus on service development has been employed which has affected the amount of data, eg. a lack of comparison group, collected in this study.

Furthermore, the relatively positive response from employees may be an overestimation of actual satisfaction. Satisfaction levels were however only one factor in evaluating service improvements which was a limitation.

Conclusion

In conclusion, the initial step of implementing clinical guidelines has been achieved through involving employees in developing succinct clinical standards as a vehicle for the communication of evidence based practice. We attribute this to leadership, sufficient time spent to plan and communicate the strategy as well as involving employees in the process. Following up recommendations with a training plan and implementation of new methods has also been vital to keep up a momentum of change.

Abbreviations

AACAP: American Academy of Child and Adolescent Psychiatry; CAMHS: Child and Adolescent Mental Health Service; EDI: Employee-Driven Innovation; NICE: National Institute for Clinical Excellence, PDSA: Plan, Do, Study, and Act

Declarations

Ethics

The project is a service evaluation project and according to Norwegian law does not require ethics approval (see link). Consent to participate in the study was ensured through the participation process which was voluntary and anonymous. https://helseforskning.etikkom.no/reglerogrutiner/soknadsplikt/sokerikkerek?p_dim=34999&_ikbLanguageCode=n

Consent for publication

Not applicable

Availability of data and material

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

No funding was obtained for this study.

Authors’ contributions

First author, LK, has designed the study in collaboration with all other authors and written the paper. Second author, LSS, conducted the evaluation in collaboration with all other authors and participated in writing the paper. Third author, IE, analyzed the data and participated in structuring and writing of the paper.

References

1. Forsner T, Hansson J, Brommels M, Wistedt AÅ, Forsell Y (2010) Implementing clinical guidelines in psychiatry: a qualitative study of perceived facilitators and barriers. BMC psychiatry 10: 1-0.
2. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, et al. (1998) Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. Bmj 317: 465-468.
3. Flottorp S, Aakhus E (2013) Implementeringsforskning: vitenskap for forbedring av praksis. Norsk epidemiologi 23: 187-196.
4. Baker R, Camosso‐Stefinovic J, Gillies C, Shaw EJ, Cheater F, et al. (2010) Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2010: Cd005470.
5. Bighelli I, Ostuzzi G, Girlanda F, Cipriani A, Becker T, et al. (2016) Implementation of treatment guidelines for specialist mental health care. Cochrane Database Syst Rev 12: Cd009780
6. Patterson M, Rick J, Wood SA, Carroll C, Balain S, et al. (2010) Systematic review of the links between human resource management practices and performance. Health Technol Assess 14: 1-334.
7. Weinmann S, Koesters M, Becker T (2007) Effects of implementation of psychiatric guidelines on provider performance and patient outcome: systematic review. Acta Psychiatr Scand 115: 420-433.
8. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F, et al. (2005) Implementation research: A synthesis of the literature. USF University of South Florida.
9. Gagliardi AR, Alhabib S (2015) Trends in guideline implementation: a scoping systematic review. Implement Sci 10: 54.
10. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, et al. (2001) Changing provider behavior: an overview of systematic reviews of interventions. Medical Care 39: Ii2-45.
11. Grimshaw J, Thomas R, MacLennan G, Fraser CR, Ramsay CR, et al. (2004) Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 8: 1-72.
12. Powell BJ, Proctor EK, Glisson CA, Kohl PL, Raghavan R, et al. (2013) A mixed methods multiple case study of implementation as usual in children’s social service organizations: study protocol. Implement Sci 8: 92.
13. Aarons GA, Sommerfeld DH, Walrath-Greene CM (2009) Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci 4: 83.
14. Glisson C, Williams NJ, Hemmelgarn A, Proctor E, Green P (2016) Increasing clinicians' EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC. Behav Res Ther 76: 40-46.
15. Novins DK, Green AE, Legha RK, Aarons GA (2013) Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. J Am Acad Child Adolesc Psychiatry 52: 1009-1025.
16. Rohrbach LA, Graham JW, Hansen WB (1993) Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Prev Med 22: 237-260.
17. Glisson C, Williams NJ (2015) Assessing and changing organizational social contexts for effective mental health services. Annu Rev Public Health 36: 507-523.
18. Olin SS, Williams N, Pollock M, Armusewicz K, Kutash K, et al. (2014) Quality indicators for family support services and their relationship to organizational social context. Adm Policy Ment Health 41: 43-54.
19. Williams NJ, Glisson C (2013) Reducing turnover is not enough: The need for proficient organizational cultures to support positive youth outcomes in child welfare. Child Youth Serv Rev 35: 1871-1877.
20. Høyrup S, Bonnafous-Boucher M, Hasse C, Møller K, Lotz M (2012) Employee-driven innovation: A new approach. Palgrave Macmillan.
21. Smith P, Ulhøi JP, Kesting P (2012) Mapping key antecedents of employee-driven innovations. International Journal of Human Resources Development and Management 12: 224-454
22. Proctor EK, Landsverk J, Baumann AA, Mittman BS, Aarons GA, et al. (2013) The implementation research institute: training mental health implementation researchers in the United States. Implementation Science 8: 105.
23. Donnelly P, Kirk P (2015) Use the PDSA model for effective change management. Education for primary care: an official publication of the Association of Course Organisers, National Association of GP Tutors. World Organisation of Family Doctors 26: 279-281.
24. https://www.aacap.org/AACAP/About_AACAP/Home.aspx
25. When designing usability questionnaires, does it hurt to be positive? Proceedings of the International Conference on Human Factors in Computing Systems, CHI; 2011; Vancouver, BC, Canada.
26. Salazar MS (2015) The dilemma of combining positive and negative items in scales. Psicothema 27: 192-199.
27. Sauro J, Lewis JR (2011) When designing usability questionnaires, does it hurt to be positive? Proceedings of the International Conference on Human Factors in Computing Systems, CHI; Vancouver, BC, Canada.
28. van Sonderen E, Sanderman R, Coyne JC (2013) Ineffectiveness of reverse wording of questionnaire items: let's learn from cows in the rain. PloS one 8: e68967.
29. IBM Corp (2017) IBM SPSS Statistics for windows. 24 ed. Armonk, NY: IBM Corp.