Supporting evidence-based practice: Changes in service provision and practitioners’ attitudes following EBP Service Centre consultation – a qualitative study / Evidenzbasierte Praxis unterstützen: Veränderungen der Arbeitsweise und der Einstellungen von Ergotherapeuten/-innen durch die Nutzung eines EBP Service Centers - eine qualitative Studie

Based on the knowledge about facilitators of EBP, various implementation strategies have been developed and evaluated. EBP has become an integrative element in health care education. EBP workshops aim to inform therapists about the scope and the steps of EBP and train the skills in searching and appraising the literature. Other activities are EBP journal clubs where practitioners search and discuss relevant literature and their impact on practice (Scott, Del Mar, Hoffmann, & Bennett, 2013; Straus et al., 2011). Brangan, Quinn, & Spirtos (2015) investigated the impact of an EBP course on the therapists’ attitude and perceived level of confidence concerning EBP. McCluskey and Lovarini (2005) showed that EBP education and outreach support improved the knowledge but did not change the participants’ practice regarding the frequency of searching and appraising literature. Novak and McIntyre (2010) found that a workshop with workplace support improved EBP implementation. The workplace support included supervision, incentives, resource allocation and working groups. Other studies focused on the system-level: Caldwell, Whitehead, Fleming and Moes (2008) developed an organisational model of practice, and Bennett et al. (2016) explored the influence of an organisational initiative to support EBP. All mentioned strategies lead to some improvement in the implementation of EBP, but no strategy on its own can be considered sufficient or consummate.

An Austrian survey asking occupational therapists about their attitudes, skills and barriers concerning EBP showed a similar picture of surveys in other countries. There is a positive attitude towards EBP; however, evidence from research is rarely used in clinical practice (Ritschl et al., 2015). The results displayed the typical barriers concerning lack of time and lack of access to research literature. Only 55% of the therapists reported to have the skills to develop a clinical question, 53% stated to be able to search for literature in databases and 54% to be able to critically appraise a paper.

To promote and evaluate the implementation of EBP, we designed a new format to support EBP in practice and established an EBP Service Centre at the University of Applied Sciences FH Campus Wien. Occupational therapy practitioners working in or in the surroundings of Vienna were invited to participate in this pilot project. The therapists attended a 9-hour EBP workshop delivered on two afternoons within one month. The participants learned about the definitions and stages of EBP and received basic EBP training in skills, for example, asking a clinical question, searching literature, and appraising the literature. Additionally, the format of a critically appraised topic (CAT) was introduced. After completing the workshop, the therapists were invited to send their clinical questions to the EBP Service Centre, and research staff trained in EBP produced German CATs for these questions. Whenever a clinical question was unclear or not specific enough, the practitioner was contacted to clarify the request. Databases and journals subscribed to by the university provided rich resources for searching and accessing literature. McCluskey’s CAT template (2003) was used and the CATs were sent to the person who had posed the question via email. Additionally, the CATs were accessible for all participants via Dropbox.

This format was designed to overcome the barriers concerning the practitioners’ lack of time and access to literature, as well as lack of skills in searching, reading and appraising scientific English articles. The main objective of the workshop was to inform about EBP and the whole process behind a CAT. Step 4 in the EBP process, the integration of research knowledge into all other relevant information, remained the therapists’ responsibility.

The first aim of the study was to explore occupational therapists’ changes in service provision, in professional viewpoints and attitudes, and in job satisfaction through EBP Service Centre consultation. The second aim was to receive feedback on the EBP Service Centre for its further development.

13 therapists participated in the project. The therapists represented a great variety in terms of years of working experience, fields of practice and work settings (Tab.1). During the five-month pilot phase, the EBP Service Centre received 56 questions and produced 42 CATs. Due to limited resources of the CAT service team, 14 questions could not be dealt with during the project time. Two of the participants did not send any question but the other 11 participants sent between 1 to 10 questions. Reasons for not posing a question were difficult working situations at the departments. Participant (J) reported a very high workload as they were short of a full-time occupational therapist at the department, and participant (I) explained that she had to deal with general issues about service provision during the time of the pilot project, and therefore, did not use the CAT service.

Data analysis resulted in the following four major categories:

Changed way of working

In contrast to preliminary theoretical considerations, which indicated an improved efficacy of the occupational therapy intervention, the findings in this study didn’t effectively show improved efficacy. Nevertheless, several participants explained changes in their working habits, which could lead to increased efficacy.

Many aspects mentioned by the participants show that the occupational therapists are doing things in a different way, or an improved quality.

As expressed by a participant, scientific evidence had the impact that ‘I try to use methods, where I know, that evidence exists’ (G:273). And clear decisions were made in terms of ‘I definitively don’t use some methods any more… this has changed a lot!’ (G:275) or about particular assessments.

‘I asked for a specific assessment and it [the CAT] has a very concrete impact on the decision on which assessment I will purchase.’ (M:275)

Examples of decisions were reported in terms of interventions, usage of assessments and individual decisions concerning extended vocational training

‘[in the CAT] I received assessment tools; including the one I already used – that validated my work. I said “Fine, it’s still up to date – I can continue to use it.”’ (K:328)

One change of working routines was that participants reported the implementation of newly developed scientific skills like ‘… one is considerably less inhibited to search the literature.’ (C:1216)

‘Now, if I have a question, or when I think “is what I’m doing good?” I’m calmer, I search the literature and I look if there is OT specific evidence or not.’ (D:979)

Even if participants didn’t explicitly change their way of working, they experienced increased knowledge through the CATs because ‘it showed me partly, how I could do it differently’ (D:181). However, some participants mentioned that, at present, the usual EBP barriers are reasons for not applying this new knowledge.

Several participants emphasised that they developed a considerably better client-centred attitude.

‘What really changed my work is that I am more courageous to work with the adolescents in a client-centred way.’(F:395)

Additionally, some participants expressed an increased occupation-based approach through EBP Service Centre consultation.

‘Now, for example, I go swimming with the child. I didn‘t do that before!’ (G:271)

Some participants reported that ‘talking with colleagues about evidence – I’m doing this more now, since the EBP Service Centre’ (C:548). This interprofessional dialogue among professionals from the healthcare and social as well as the educational system was mentioned by employed colleagues and freelancers.

All participants reported the opinion that EBP strengthens the credibility and professionalization of their profession. They think ‘that EBP is politically important for our profession and the financing of therapy.’ (L:988). Furthermore, they value EBP for their own professional self-image, for maintaining jobs and as an argumentation aid. The positive impact on professionalization was also found in the increased will to ‘present ourselves even better’ (G:860).

Participant B encapsulated the central topic of this subcategory by quoting ‘it aroused such an eagerness, to re-examine’ (B:831). Several others expressed the increased interest with wordings like ‘I’m curious about EBP, which originated from using the EBP Service Centre’ (E:1498).

Several participants stated that ‘it motivated me to conduct research projects on my own’ (M:214). The participants’ newly emerged willingness to participate in research projects became evident.

‘One suddenly feels like doing research – if you see that so many OT topics have not been researched yet – I have so many clients and would really like to do something.’ (K:601)

Within category 2, the strongest evidence for the impact of EBP Service Centre usage on the participants was the aspect of increased reflexivity. The changes of reflexivity were particularly strong with regard to self-reflection and were expressed by all participants.

‘I‘m questioning my own doing more now.’ (A:318)

This subcategory could be seen as a counterpart to the subcategories 2.1 and 3.1.

Despite a lot of positive changes, some participants also experienced insecurity when using the EBP Service Centre. The main reasons for that were the lack of evidence and the fear of overvalued scientific evidence while degrading individual professional expertise and intuition.

Participants attributed their increased job satisfaction to widely different aspects.

For example, the view of strengthened credibility and professionalization of OT, which is described in subcategory 2.1, led directly to greater job satisfaction, particularly the knowledge about existing current evidence ‘made me proud of our profession again. We OTs are really a qualified profession!’ (M:705)

The implementation of new skills acquired through EBP Service Centre usage was explicitly expressed as a factor for increase in job satisfaction.

‘There [in the CAT] was the motivational interviewing – it was proposed there. And I read it – and then – I was satisfied, because it changed something in my case.’ (F:442)

On the other hand, some participants expressed increased job satisfaction because the missing scientific evidence made them more confident in their own expert knowledge.

‘When it became clear that there is no evidence so far – it enhanced the satisfaction – or encouraged us to confide in our own expertise.’ (A:1072)

Some participants expressed intensive emotions associated with EBP Service Centre usage. These emotions ranged from ‘absolutely unsatisfied’ to ‘everything is fine again’ and from disillusionment to uncertainty.

‘Wow – it’s great, we [OTs] are really special – and there are studies proving it – and on the other hand – there are only very few studies – this discouraged me again – because I thought, there is not very much proof about our great work.’ (M:715)

For few participants, the ‘individual job satisfaction didn’t change as a consequence of the EBP Service Centre’(J:774). This was mainly argued by the fact that ‘other [factors] are far too important and far too good as to be affected’ (L:736).

In the focus group, the participants were asked to discuss the aspects of training that were helpful and the aspects that were not helpful, as well as suggestions for further development of the EBP workshop. Feedback concerning helpful aspects can be condensed to learning outcome and methodical-didactical aspects.

All participants expressed the value of ‘learning how to formulate a good research question’ as a need for proper use of the CAT service.

‘For me, the workshop was very helpful to understand the topic of EBP and to develop and formulate questions.’ (A:1235)

In total, some participants expressed that ‘EBP didn’t mean much to me. Only through the workshop did I get a feel for this’ (E:726), whereas approximately one third of the participants indicated that for them, the workshop was a very good revision.

Concerning methodical-didactical aspects, most participants stated the comprehensive content and the quality of explanations as helpful. Furthermore, the script was explicitly mentioned as helpful. Also, the structure of two half-day workshops few weeks apart was experienced as conducive to the whole learning process.

In contrast to the helpful aspects, responses on not helpful aspects were mainly given from individual participants and were considerably less comprehensive. The feedback in this respect focused on the tempo, the technical jargon and the size of the learning group.

Central among the suggestions for further development of the EBP workshop was the request ‘to practice together using a concrete example; this would have been helpful …’ (F:1324). Additionally, some participants argued for a third workshop or refreshers on different levels.

Like the feedback for the EBP workshop, the CAT service was also discussed with participants in terms of helpful or not helpful aspects and suggestions for further development of the CAT service.

In general, the CAT service was experienced as ‘awesome’ (B:127), which was evident in the statements of several participants, who sent in one or more clinical questions.

‘I was happy that there was this CAT service. Because, I’m up to my ears with my kids at home, my English is poor – I have never been motivated to search the Internet for hours, to translate everything – I was absolutely overwhelmed. I’m very happy – and would be very happy, if a CAT service existed permanently.’ (G:93)

Most of the participants emphasised the high quality of the CATs. For some participants, the German language of the CATs was particularly helpful. It allowed them to concentrate on specific content and the practical usability, instead of struggling with words.

‘The CAT has been worth gold! And it was in German language … you could really focus on the specific content.’ (K:900)

Despite only very few hindering aspects mentioned by participants, there were several suggestions for further development of the CAT service. The need for intensive communication, at least in the early phase of the CAT service, became evident.

‘When it comes to the clarification of the clinical question, the communication could be more intensive before you start the literature search, to clarify if it’s really exactly what you want to know.’ (L:1122)

During the five-month pilot phase 42 CATs were created. 14 more questions were posed, but could not be handled during the project time, due to restricted resources of the CAT team. This can be interpreted as excellent acceptance of the service.

In an Australian EBP project, the occupational therapists were trained in EBP skills and learned how to produce CATs themselves. The 114 therapists produced 15 CATs in an eight-month period (McCluskey, 2004). A statistically significant increase of CAT production could be reached by another Australian study (Novak & McIntyre, 2010), in which 88 allied health professions were trained and received different kinds of workplace support. During the 18-month period, 24 CATs were produced. Given the 42 CATs in the current study with only 13 participants in a five-month period, the EBP Service Centre seems to fulfil the practitioners’ needs. Results from a survey by Bennett et al. (2003) showed that more therapists are interested in reading brief summaries of evidence or clinical guidelines than in searching and appraising literature themselves. Also, Lin et al. (2010, p. 164) claimed that ‘clinicians must have readily available, relevant, and concisely summarized evidence’ to embrace EBP.

The participating therapists seemed to be very eager for knowledge, but one can expect that they would not have produced so many CATs on their own. Therefore, the CAT service might be seen as an efficient use of resources, as someone who is trained in searching and appraising will take much less time to produce a CAT than someone who does it once in a while. McCluskey (2004) concluded that using services of health librarians that assist with searching ‘may be a better use of staff time and resources’ (p.49) but acknowledged that critically appraising the literature is even more difficult to learn than searching the literature. Additionally, the production of CATs was based on the university’s library resources, which offered broad access to databases and journals, which are not accessible to practitioners.

All participating therapists knew that the current EBP-Service Centre was a pilot project offering the CAT service for a limited period. This might have stimulated sending in questions immediately. This is in consensus with the study by McCluskey (2004) in which the ‘presence of deadlines’ emerged as one of the facilitating factors for becoming more evidence-based. Therefore, at present, we only know the acceptance of this short period service; the acceptance of an ongoing service has to be studied during a longer project.

11 therapists sent between 1 to 10 questions, whereas two colleagues only participated in the workshop but did not make use of the CAT service. Both showed a positive attitude towards EBP and the project but explained facing difficult situations at their workplace during the time of the project. Being reflective, asking clinical questions and being eager to develop quality is an approach that also needs time and energy. Therefore, not using a service like this should not immediately be interpreted as ‘no interest in EBP’ but may suggest considering the organisational and systemic levels as well.

EBP aims to support the delivery of more effective and efficient service. In the short period of the pilot project, none of the therapists mentioned that they offered a more efficient therapy, but some presented examples of changes in therapy according to the information received through the CAT. This included offering new therapeutic approaches, not delivering a special method any more, or the decision to buy a specific assessment. In the project of McCluseky et al. (2004), which focused on training EBP skills and the production of CATs by therapists, no changes in practice were reported, whereas Novak and McIntyre (2010) documented changes in service provision after offering an EBP workshop with workplace support. In the specific literature, the process of changing service according to new evidence is most often called ‘implementation of evidence’ or ‘knowledge translation’ and receives much attention because step 4 in EBP –

‘making a decision and applying new knowledge’

Theories that explain or predict behaviour change are used to support the implementation of evidence (McCluskey, 2014; Law & MacDermid, 2014b).

It is clear that not every new knowledge can be applied within a short period of time, as new approaches might need special training, special equipment or commitment of the whole department. In such cases, a change management process and a longer period of time has to be scheduled. The changes reported in the focus group interviews were in the therapists’ scope of decision-making and therefore could immediately be transformed into practice. Maybe this positive uptake of new knowledge was encouraged by the strategy that the therapists asked their questions. They wanted to receive an answer and therefore ‘pulled’ information. This approach might complement strategies that provide new evidence, such as guidelines, without a current question.

Some therapists stressed that they started to work in a more client-centred way and increasingly occupation-based. This result was quite unexpected and disproved the myth that EBP is ‘cookie-cutter’ care or ‘devoid of the need for individual clinical judgement’ (Law & MacDermid, 2014a, p. 7). There are two explanations why this process might have started. On the one hand, some CATs favoured occupation-based therapy as being more effective than working at component level. On the other hand, it seems that learning to pose a clinical question, for example, the PICO question (Person – Intervention – Comparison – Outcome) might have stimulated a process to focus more on the goal of therapy and the client’s aim, and therefore, might have triggered a client-centred approach.

The statements concerning personal and professional viewpoints and attitudes built the richest category in the analysis. All therapists acknowledged the importance and the necessity of research evidence to show the effects of occupational therapy, to strengthen their own professional self-perception, as well as to legitimate occupational therapy services. Such awareness of the importance of research led to another unexpected effect. Many therapists stated their willingness to participate in research and thereby to contribute to the building of evidence. Additionally, some participants stated to be interested in analysing their documentations and data or conducting small research studies, like case studies on their own. According to Tomlin and Douthery (2014), these activities of systematic gathering and analysis of therapy outcomes in practice contribute to the internal evidence, which complements the external evidence. Practitioners’ experiences thereby contribute to the profession’s body of knowledge and might stimulate research as well.

A positive attitude and mutual recognition of practitioners and researchers is vital. It is important that practitioners have a positive attitude towards information from research to be willing to integrate this knowledge into clinical decisions. Whenever a researcher tries to recruit practitioners to participate in a project and ask them to follow the study plan, a positive attitude and commitment are prerequisites. However, mutual recognition is also a mandate for researchers to focus research on topics, which are clinically relevant (Lin et al., 2010; Kielhofner, 2005; Sudsawad, 2005).

Participating in the EBP project and learning about EBP seemed to stimulate critical thinking and professional reasoning. Maybe this was due to a similar effect as the increase of client-centredness. Asking a clinical question is the first step in the EBP process but also seems to stimulate questioning one’s own therapeutic behaviour and enhance the professional reasoning skills (Schell & Schell, 2008; Turpin & Higgs, 2013). This result is consistent with Tse, Lloyd, Penman, King and Bassett’s (2004) assumptions that EBP ‘will assist therapists to be more reflective, analytical, while remaining creative practitioners’ (p. 270). Additionally, the EBP training appears to encourage the therapists to question other professionals about their evidence or to decide more critically regarding which continuing education workshop to subscribe to.

The participants gained better confidence in their skills of searching and appraising the literature, and a few used these skills. This is a similar result to the study of McCluskey and Lovarini (2005) whose participants improved their knowledge and skills; but only the group of proactive participants reported searching and appraising literature themselves. One participant explained that she started to appraise an article, but soon got lost in the scientific language. Another therapist stated that these steps were very time consuming and asked for additional training or supervision. These results confirm the assumption that a CAT service, where searching and appraising the literature is ‘outsourced’, may bridge the gap between the practitioners’ questions and their willingness to integrate research knowledge.

For some of the questions posed, a great deal of literature was included in the CAT, and therapists felt proud when their approach was confirmed by research or felt enriched when they received new ideas for treatment. In contrast to this situation, no evidence could be found for other questions. This situation was mentioned several times in the focus group interviews and led to many different feelings. Some therapists reported a feeling of relief because then they knew that there is no research knowledge out there, which they might have neglected so far in their clinical decisions. Other colleagues reacted differently; they felt quite ashamed of themselves and the profession as parts of their service is delivered without any evidence from research.

It is a known phenomenon that dealing with little or no evidence might reduce the confidence in EBP (Law & MacDermid, 2014c). Therefore, these different reactions might be an indicator that the situation of no or little evidence should be covered in the workshop in more detail. It would be sensible to emphasise that not having any evidence does not imply that a treatment is ineffective or should not be offered. The situation of no evidence has to be distinguished from the situation where there is evidence that a treatment or an approach is not effective or causes harm. Novak’s traffic light grading system for the summary of evidence (Novak & McIntyre, 2010) offers excellent visualisation and might be helpful for explanations.

The feedback to the offerings of the EBP Service Centre was very positive and highly appreciative of the quality of the service. Participants approved the length of the workshop and the format of delivery: two parts with about one month in between. This feedback is consistent with the feedback from participants of the EBP project by McCluskey (2004) in Australia. Due to huge travel distances, a two-day workshop was offered in a block, but participants and providers suggested afterwards that the workshop should be split to reduce cognitive overload. This experience led to the suggestion that when providing a workshop in rural regions, where the participants have to travel long distances, the pros and cons for providing the workshop either at a single time point or in two parts have to be considered carefully. Additionally, innovative didactic methods for e-learning and distance learning should be considered.

In general, the participants were satisfied with the content of the workshop. Depending on the pre-existing knowledge, some stated that largely it was a good revision of topics they had learned during their education, whereas for others the seminar was quite demanding because most of the content was new to them. Some participants asked for consolidation workshops to revise the content and practice the skills learned. When offering such consolidation days, different levels could be offered. They may either focus on a revision of the content or more on practising under supervision.

There was a consensus that learning about EBP and getting to know the CAT form in the workshop was essential for subsequent use of the CAT service. Without the workshop, most participants would not have been skilled to ask (answerable) clinical questions. Additionally, it was essential that therapists had the opportunity to clarify their question with the researcher who produced the CAT. The participants strongly recommended continued provision of personal communication between the researcher and the practitioner.

Practitioners highly appreciated the CATs as a concise appraisal and summary of the evidence. An essential factor was that the CATs were produced in German language, which made them much easier to read. Practitioners suggested that questions could be labelled with different priority status so that urgent questions are answered first. The place to provide the CATs was also mentioned in the feedback. During the project, the CATs were sent to the participant who had asked the question, via e-mail and they were also put into a dropbox. Due to security reasons, some participants had no access to the dropbox at their workplace, and therefore, only received the CATs they asked for. Having the possibility to view all CATs was appreciated, as the therapists made use of this opportunity. When providing such a service, the place to store files has to be reconsidered. It should be a secure but accessible place. Depending of the structure and resources of a future service, the CATs could be made accessible even for a broader community. Professional associations as well as other stakeholders could be considered for offering resources.

In summary, practitioners considered the EBP Service Centre providing an EBP workshop and the CAT service ‘a really good package’ (F) and expressed their desire for an ongoing service. This new implementation strategy for EBP seemed to overcome many barriers. Outsourcing the steps of searching and appraising seemed to stimulate critical thinking and questioning but also the willingness to integrate new knowledge in practice. Strategies that support practitioners whenever their questions arise might have another effect than strategies that inform without a question asked, for example, providing information about guidelines. To support EBP, different approaches have to be considered. Strategies may focus on different stakeholders: clinicians, educators, researchers, managers and/or associations (Lin et al., 2010). Using skills and knowledge as well as librarian resources from academia to run the EBP Service Centre might be a new, effective and efficient strategy to implement EBP.

As the project took place within a time period of five months, the results only offer short-term outcomes. At the moment, there is no evidence about the sustainability of the effects and results.

All participants were aware that the EBP workshop leader as well as the moderator and co-moderator of the focus group interview are lecturers at the university. As a lecturer, they have a working relationship with the majority of the participants, as most of them are fieldwork educators for university students. This relationship might have had an influence on the participants’ openness during the interview.

The analysis approach of Gläser und Laudel was considered appropriate to answer the study questions. The main reason to choose this approach was that it allowed the definition of preliminary categories based on existing knowledge as well as the possibility to create new categories during the data analysis process. However, we are aware that other qualitative methods of analysis might have triggered different findings.

The project included occupational therapists only. As the topic is relevant to all health professionals, a future study might include other health professionals as well. It would be interesting to see if the acceptance and effects are similar to the experiences described in this paper.