Implementing a psychosocial intervention DIALOG+ for patients with psychotic disorders in low and middle income countries in South Eastern Europe: protocol for a hybrid effectiveness-implementation cluster randomized clinical trial (IMPULSE)

Nikolina Jovanovic 1 , Jill Francis 2 , Nadja P. Maric 3 , Aliriza Arenliu 4 , Stojan Barjaktarov 5 , Alma Dzubur Kulenovic 6 , Lidija Injac 7 , Yan Feng 8  and Antoni Novotni 5
  • 1 Unit for Social and Community Psychiatry, WHO Collaborating Centre for Mental Health Services Development, Bart’s and The London School of Medicine and Dentistry, Queen Mary University of London
  • 2 School of Health Sciences, City University of London
  • 3 Faculty of Medicine, University of Belgrade, Belgrade, Serbia
  • 4 Department of Psychology, University of Prishtina
  • 5 University Psychiatry Clinic, , Skopje
  • 6 Clinical Center University of Sarajevo, , Sarajevo, Bosnia and Herzegovina
  • 7 Psychiatric Clinic, University of Montenegro, Podgorica, Montenegro
  • 8 Centre for Primary Care and Public Health, Barts and The London School of Medicine and Dentistry, Queen Mary University of London
Nikolina Jovanovic
  • Corresponding author
  • Unit for Social and Community Psychiatry, WHO Collaborating Centre for Mental Health Services Development, Bart’s and The London School of Medicine and Dentistry, Queen Mary University of London
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Jill Francis, Nadja P. Maric, Aliriza Arenliu, Stojan Barjaktarov, Alma Dzubur Kulenovic, Lidija Injac, Yan Feng
  • Centre for Primary Care and Public Health, Barts and The London School of Medicine and Dentistry, Queen Mary University of London
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Antoni Novotni

Abstract

Objectives

Psychotic disorders have large treatment gap in low- and middle-income countries (LMICs) in South-Eastern Europe, where up to 45% of affected people do not receive care for their condition. This study will assess the implementation of a generic psychosocial intervention called DIALOG+ in mental health care services and its effectiveness at improving patients’ clinical and social outcomes.

Methods

This is a protocol for a multi-country, pragmatic, hybrid effectiveness–implementation, cluster-randomised, clinical trial. The trial aims to recruit 80 clinicians and 400 patients across 5 South-Eastern European LMICs: Bosnia and Herzegovina, Kosovo*, Montenegro, Republic of North Macedonia and Serbia. Clusters are clinicians working with patients with psychosis, and each clinician will deliver the intervention to five patients. After patient baseline assessments, clinicians will be randomly assigned to either the DIALOG+ intervention or treatment as usual, with an allocation ratio of 1:1. The intervention will be delivered six times over 12 months during routine clinical meetings. TThe primary outcome measure is the quality of life at 12 months [Manchester Short Assessment of Quality of Life (MANSA)]; the secondary outcomes include mental health symptoms [Brief Psychiatric Rating Scale (BPRS), Clinical Assessment Interview for Negative Symptoms (CAINS), Brief Symptom Inventory (BSI)], satisfaction with services [Client Satisfaction Questionnaire (CSQ-8)] and economic costs at 12 months [based on Client Service Receipt Inventory (CSRI), EQ-5D-5L and Recovering Quality of Life (ReQOL-10)]. The study will assess the intervention fidelity and the experience of clinicians and patients’ about implementing DIALOG+ in real-life mental health care settings. In the health economic assessment, the incremental cost-effectiveness ratio is calculated with effectiveness measured by quality-adjusted life year. Data will also be collected on sustainability and reach to inform guidelines for potentially scaling up and implementing the intervention widely. Conclusion: The study is expected to generate new scientific knowledge on the treatment of people with psychosis in health care systems with limited resources. The learning from LMICs could potentially help other countries to expand the access to care and alleviate the suffering of patients with psychosis and their families.

Trial registration: ISRCTN 11913964

INTRODUCTION

Severe mental disorders such as psychotic disorders affect 3–5% of the population. The symptoms can vary but often include hearing voices, not being able to discriminate between what is real and what is not, thinking that family or friends are part of a conspiracy and social and emotional withdrawal (Sadock & Kaplan, 2014). Stigma and discrimination of people with psychotic disorders are still common. Their life expectancy is lower by 15–20 years compared to the general population because of underdiagnosed physical illnesses, poor access to health care and suicide (Sadock & Sadock, 2014). Psychotic disorders have a particularly large treatment gap in low- and middle-income countries (LMICs) in South Eastern Europe, where up to 45% of affected people do not receive care for their condition (WHO, 2008, 2014; McDaid et al., 2005). This fosters further social exclusion and inequality of this vulnerable group. Health care systems in high-income countries provide a combination of care, medication and psychosocial interventions, which helps a number of people affected by psychotic disorders to find employment and lead a productive life. However, LMICs in South Eastern Europe have neither the funding nor enough qualified staff to provide such specialised services (WHO, 2014; Maric et al., 2019) Therefore, there is an urgent need to implement low-cost, easily deliverable psychosocial interventions to expand the access to care and alleviate the suffering of these patients and their families.

This study aims to assess the effectiveness and the implementation of a psychosocial intervention called DIALOG+ in community-based mental health care services in five South Eastern European LMICs. The study is designed as an effectiveness–implementation hybrid pragmatic trial that simultaneously aims to assess the clinical and social outcomes of the DIALOG+ intervention relative to treatment as usual (TAU), while also assessing the potential utility of the implementation strategy in facilitating the implementation of DIALOG+ (Curran et al., 2012). As suggested by Curran and colleagues, hybrid trials can provide significant benefits such as more rapid translational gains, more effective implementation strategies and more useful information for decision makers (Curran et al., 2012). The DIALOG+ intervention was shown effective in the United Kingdom (Priebe et al., 2015); however, the uncertainties around the service context in South Eastern Europe do not allow its implementation to be simply broadened to these countries. The participating countries shared similar socioeconomic and political background before 1990s, which will facilitate the research project and mutual learning across sites. However, their individual developments in the post-communist period have led to significant differences in the organisation of mental health care systems. For that reason, this study will enable us to explore the implementation of DIALOG+ within different contexts that are potentially representative of other LMICs, thus increasing the generalisability and broadening impact of the results. Mental health care in the participating countries is still largely hospital based because of historical, political, professional and service organisation characteristics that impede the development of community psychiatry. As indicated by colleagues in Slovenia, a high-income country in South-Eastern Europe, these barriers are to be addressed through coordinated action involving primary care professionals, non-governmental organisations with service users and carers, health insurance agencies and politicians involved in the planning of health services (Svab V and Svab, 2013). We hope that this study will contribute to the development of community-based mental health care for patients with psychosis in South Eastern Europe.

DIALOG+ is a technology-assisted intervention based on patient-centred communication and solution-focused therapy. DIALOG+ was designed to make routine meetings between clinicians and patients more therapeutically effective. The intervention is available as an app and makes use of a tablet computer within routine clinical meetings. In the United Kingdom, DIALOG+ has been shown effective in long-term patients with psychosis in the community (Priebe et al., 2015). It led to reduced clinical symptoms, better quality of life and lower treatment costs. DIALOG+ uses already existing clinical relationships and does not require establishment of new services or referral to other clinicians, so the intervention is well suited for health care systems with limited resources. With only minimal training (3 hours), a range of clinical staff (e.g. nurses, psychologists and psychiatrists) in the United Kingdom was able to successfully implement the intervention in routine practice (Priebe et al., 2015). As a cost-saving intervention that does not rely on any one professional group, it allows for affordable management and for expanding patients’ access to care. DIALOG+ provides a comprehensive screening of mental, physical and social problems and leads to plans for actions in all these areas, thus avoiding an inefficient fragmentation of care planning. Figure 1 shows key components, proposed mechanisms of action and outcomes of DIALOG+ (Omer et al., 2016).

Figure 1
Figure 1

Key components, proposed mechanisms of action and outcomes of DIALOG+

Citation: Global Psychiatry 3, 1; 10.2478/gp-2019-0020

This protocol outlines the design and methods for a multicentre, cluster-randomised, controlled trial to evaluate the implementation of DIALOG+ in mental health care services and its effectiveness at improving patients’ clinical and social outcomes.

STUDY AIMS

1) Aim 1 (effectiveness aim) is to explore the clinical effectiveness and cost-effectiveness of the DIALOG+ intervention (intervention arm) relative to TAU (control arm) on patient-level clinical and social outcomes in LMICs in South-Eastern Europe. The primary outcome measure is the quality of life at 12 months; the secondary outcome measures include mental health, satisfaction with services and economic costs at 12 months. We hypothesise that the use of DIALOG+ is associated with improved quality of life [score on Manchester Short Assessment of Quality of Life (MANSA) at 12 months (Priebe et al., 1999)], reduced clinical symptoms [scores on Brief Psychiatric Rating Scale (BPRS) (Ventura et al., 1993), Clinical Assessment Interview for Negative Symptoms (CAINS) (Kring et al., 2013) and Brief Symptom Inventory (BSI) (Derogatis, 1993), and improved satisfaction with services (score on Client Satisfaction Questionnaire [CSQ-8])]. We hypothesise that overall treatment costs in the intervention arm will be equal or lower compared to the control arm. Please find more details below under Aim 1.

2) Aim 2 (Implementation aim) is to explore the factors associated with successful implementation of DIALOG+ based on patient–clinician engagement with and adherence to the DIALOG+ intervention (intervention fidelity). We hypothesise that clinicians will adhere to the study protocol and deliver the DIALOG+ intervention with high fidelity. Please find more details below under Aim 2.

This study protocol has the potential to contribute to the scientific psychiatric literature by outlining hybrid implementation–effectiveness methodology that is not often used in psychiatric literature. The work is guided by the implementation strategy developed based on the qualitative data from the pilot stage and is informed by behavioural theory, which is often neglected in the intervention literature. This study is the first of its kind conducted in South Eastern Europe and one of very few trials in psychosocial mental health care provided to patients with psychosis globally.

PRE-TRIAL WORK

Before the trial, we conducted a series of activities to inform the strategy for implementing DIALOG+ in each country. We conducted individual interviews and focus groups with all relevant stakeholders (patients, carers, clinicians, service providers and policy makers) to understand the local context. We also analysed policy documents to better understand local regulations in mental health care and their framework for the implementation of psychosocial interventions for patients with psychosis. Next, five clinician–patient pairs in each country met to use DIALOG+ up to three times for 3 months. After this pilot study, we organised theoretical domains framework (TDF)-based focus groups with patients and clinicians from the pilot study (Francis et al., 2012; Cane et al., 2012; Phillips et al., 2015). These focus groups were conducted to elicit clinicians’ and patients’ perceptions about the barriers and facilitators of engaging with the DIALOG+ intervention. The TDF is a useful and flexible framework for the assessment of barriers and targeting resources to influence behaviour change for the implementation projects. The TDF consists of 12 domains that are proposed to influence behaviour and stimulate behaviour change. The discussion was concentrated on how participants received and engaged with the intervention and barriers and facilitators to engagement. Using the published methods, appropriate behaviour change techniques (BCTs) (Michie et al., 2013) were identified to address important barriers that were assessed for feasibility and usability by researchers in each country. On the basis of these steps, an implementation strategy was developed iteratively within the team (this preliminary research is published separately).

METHODS

This is a multi-country, pragmatic, hybrid effectiveness–implementation, cluster-randomized, clinical trial. The trail aims to recruit 80 clinicians and 400 patients across 5 South-Eastern European LMICs: Bosnia and Herzegovina, Kosovo*, Montenegro, Republic of North Macedonia, and Serbia. The coordinating centre is Queen Mary University of London in the United Kingdom, where the DIALOG+ intervention was developed (Priebe et al., 2015). A cluster-randomisation design is used to avoid potential contamination of the practice of clinicians when treating patients in both the groups. We will test the effectiveness and the cost-effectiveness of DIALOG+ for patients with psychosis. The trial will last for 12 months. The primary outcome measure is the quality of life at 12 months (as measured by MANSA) (Priebe et al., 1999); the secondary outcome measures include mental health, satisfaction with services and economic costs at 12 months. We hypothesise that the use of DIALOG+ is associated with improved patient outcomes in these areas. The study will also assess the intervention fidelity and clinicians and patients’ experience of implementing DIALOG+ in real-life mental health care settings. Data will be collected on sustainability and reach to inform guidelines for potentially scaling up and implementing the intervention widely. The study started with a preparatory phase that included exploring socio-cultural context for implementing DIALOG+ and developing an implementation strategy. Recruited patients will undergo baseline assessments before their clinicians are randomised to the intervention or standard care. The patients are assessed again at 6 and 12 months. Figure 2 presents the study flow chart.

Figure 2
Figure 2

Study flow chart

Citation: Global Psychiatry 3, 1; 10.2478/gp-2019-0020

ELIGIBILITY CRITERIA

In each participating country, all outpatient psychiatric services with caseloads of more than 200 patients with psychosis were identified and invited to join the study. A total of 17 services were identified. The goal was to include at least two services per country. Finally, 11 services were included, 3 in Montenegro and 2 in each of the other 4 countries. In 4 countries (Bosnia and Herzegovina, Montenegro, Republic of North Macedonia and Serbia), these services are hospital based. Only services in Kosovo* are community based and not linked to psychiatric hospitals. In this trial, clusters are clinicians working with patients with psychosis in outpatient clinics in the participating countries. All eligible clinicians will be invited to participate in the study with the goal to recruit a minimum of 16 clinicians per country. Inclusion and exclusion criteria are listed in Table 1.

Table 1

Inclusion and exclusion criteria for clinicians and patients

CliniciansPatients
Inclusion criteria
Having a professional qualification in mental health care (e.g. psychiatrists, nurses and psychologists)

More than 6 months’ experience of working in mental health care

No plans to leave their post within the study period (12 months)
Primary diagnosis of psychosis or related disorder in remission (i.e. ICD-10 F20-29, F31) (ICD-10, 1990)

Aged 18 years or older

Attending the outpatient clinic or day hospital

History of at least one hospital admission in their lifetime

There should not be plans to leave mental health care services for the next 12 months

Capacity to provide informed consent
Exclusion criteria
N/AHaving a diagnosis of organic brain disorders

Having severe cognitive deficits (unable to provide information to study instruments). This is based on the clinical judgement

RANDOMISATION, ALLOCATION AND CONCEALMENT

Eligible clinicians will be identified from outpatient psychiatric services and consented. Their caseloads will be screened to identify patients who met the inclusion criteria. Patients who agree to be involved in the study will meet with researchers who will double check if patients meet the eligibility criteria and will invite them to sign a consent form. Once a patient gives informed consent, researchers can proceed to completing the baseline assessment, which includes socio-demographic and clinical characteristics, subjective quality of life, mental health symptoms, satisfaction with services, use of mental health services, and health economic assessment. To minimise selection bias within clusters, clinicians will be randomised once all patients from their caseload had been recruited and all baseline assessments were completed. Clinicians will be randomised 1:1 to intervention or control. Randomization will be conducted by an independent statistician who used sequential computer-generated random numbers to determine allocation. The randomisation sequence will be created using random block sizes of 4 and 6. An unblinded research assistant in each country will assign a unique study ID to each clinician before the statistician undertakes the randomization of these IDs. To prevent an unequal allocation across both groups, the participants will be stratified before randomisation on two factors: gender (e.g. male vs female) and professional status (e.g. psychiatrists vs non-psychiatrists). Baseline data collection will be completed before clinicians and the study team were made aware of the allocation. The allocation of clusters will be concealed from outcome assessors, and clinicians are asked to keep their treatment allocation concealed from their colleagues and managers.

BLINDING

Owing to the nature of the intervention, clinicians and patients in this study cannot be blinded to their allocation to the experimental or control group. Researchers who conduct the outcome assessments are blinded to the allocation of patients. If and when blinding is broken, it is recorded. We will also ensure blinding of the data analyst until the entire analysis has been completed.

CLINICIANS’ TRAINING

All clinicians randomised to the intervention arm will receive face-to-face training to deliver DIALOG+ to patients from a member of the local research team. Training materials were adapted during the pre-trial work, and appropriate BCTs (Michie et al., 2013) were used to classify active ingredients of the intervention and to address the identified barriers to implementation. The core training lasts for 3 h and enables clinicians to implement DIALOG+ in real-world practice settings. After the first DIALOG+ session with patients, clinicians are offered top-up training sessions that are delivered flexibly, tailored to the individual needs.

OVERVIEW OF THE DIALOG+ INTERVENTION

The intervention group will receive DIALOG+. The intervention consists of up to 6 sessions for 12 months (for the first 3 months once a month and then every 3 months) during routine meetings between patients and clinicians. Each session begins with the patient using the tablet to rate their satisfaction with eight life domains (mental health, physical health, job situation, accommodation, leisure activities, friendships, relationship with family/partner and personal safety) and three treatment aspects (medication, practical help and meetings with professionals). The tablet allows patients to be more actively involved in the meeting, with the tablet easily shared between the clinician and patient. Each satisfaction item is rated on a scale from 1 (‘totally dissatisfied’) to 7 (‘totally satisfied’) and followed by a question on whether the patient wants additional help with that domain. The ratings are summarised on screen, allowing for comparisons with ratings from previous meetings. Clinicians are instructed to offer positive feedback on any improving or high-scoring domains. The ratings are followed by a four-step solution-focused approach to identify the patient’s existing resources that can be used to address the concerns raised. The four steps are Understanding (Why is the patient dissatisfied? What nevertheless went well?), Looking Forward (What is the best case scenario? What is the smallest step forward?), Exploring Options (What can the patient, the clinician or others do?) and, finally, Agreeing on Actions (e.g. homework and referrals). Each session should last from 30 to 60 min. It is advisable that first session (or even first 3 sessions) last for 60 min to allow enough time for clinicians and patients to adapt to using tablets and DIALOG+. After rating domains, clinicians and patients need to choose domains for further discussion using the four-step solution-focused approach. As a general rule, the clinician is advised to select no more than one domain in the first session and no more than three domains in subsequent sessions, depending on time constraints and focus on domains where satisfaction is below 4. Clinicians are asked to encourage patients to attend the next session even if they did not manage to complete agreed actions. More details can be found in DIALOG+ Manual (Priebe et al., 2016). Clinicians randomised to the intervention will receive a computer tablet with the intervention software (DIALOG+) uploaded.

CONTROL GROUP

The control group will receive standard care that includes routine meetings offering TAU, following the same delivery schedule as the intervention arm. This includes consultations on medication, psychological support and discussion on other aspects of care. At the beginning of the study, clinicians are asked not to change their usual approach when working with patients in this study. More information on standard outpatient mental health care in recruited services in each country is available in Table 2. As part of the IMPULSE study, we will also audiotape and analyse sessions from the control arm (one per clinician) to improve our understanding of the content and form of TAU sessions.

Table 2

Standard outpatient mental health care in recruited services

CountryDescription of routine outpatient mental health care
Bosnia and HerzegovinaHospital-based outpatient services: patients are primarily seen by psychiatrists for every 1–3 months; the duration of routine meetings is 15–30 min. Other staffs include nurses, psychologists, social workers, occupational therapists and trainees. Interventions offered include medication, psychotherapy and occupational therapy.
Kosovo*Community-based services: patients are primarily seen by nurses one to four times per month, the duration of routine meetings is 15–20 min. Psychiatrists see patients once per month or less frequently mainly to review medication or provide psychotherapy. Other professionals in services include psychologists, psychotherapists, social workers and trainees. Interventions offered include medication, family intervention, occupational therapy and home visits.
Republic of North MacedoniaHospital-based services: patients are seen predominately by psychiatrists or psychologists once per month or less frequently; the duration of routine meetings is 30 min. Other staffs include nurses, social workers, psychotherapists, occupational therapists, pharmacists and psychiatric trainees. Interventions offered include medication, psychotherapy, occupational therapy, psychoeducation and supportive psychotherapy.
MontenegroHospital-based services: patients are seen predominately by psychiatrists once per month or less frequently; the duration of routine meetings 15–45 min. Other staffs include nurses, psychologists, social workers, occupational therapists, psychiatric trainees and defectologists. Interventions offered include medication, psychotherapy and occupational therapy.
SerbiaHospital-based services: patients are seen predominately by psychiatrists and psychologists every 2–3 months; the duration of routine meetings is 20–40 min. Other staffs include nurses, social workers, occupational therapists and trainees. Interventions offered include medication, psychotherapy, occupational therapy and needs-based supportive psychotherapy.

IMPLEMENTATION STRATEGY

The implementation strategy is a data-driven and theory-informed document that aims to guide implementation of DIALOG+ in this study, if DIALOG+ is shown to be effective, to help adapt and scale-up the implementation of DIALOG+ to health systems in LMICs in Europe. Such a strategy is vital to maximise desired behaviour change and optimise implementation of DIALOG+ in LMICs in South-eastern Europe. This implementation strategy was designed for use by clinicians who deliver DIALOG+, service managers and the relevant policy makers. It provides detailed descriptive information of each potential barrier and the actions (and resources) to overcome it. For example, the pre-trial work identified that the most common barrier is clinicians’ perceived lack of capability and skills to deliver DIALOG+ to psychotic patients. As this barrier could be overcome with sufficient training, DIALOG+ training was modified to include proposed actions and resources to overcome identified barriers. The implementation strategy is updated at the end of the trial based on additional findings from the main study and is published separately.

AIM 1: EFFECTIVENESS AND COST-EFFECTIVENESS OF DIALOG+

Data are collected using a standardised Case Report Form, and each partner site is collecting the same measures for all recruited patients. All measures were translated into the local languages, and all researchers were trained in patient assessments. At baseline, we obtained socio-demographic and clinical characteristics including gender, age, ethnicity, highest level of education, marital status, employment, mental and physical diagnosis, previous psychiatric admissions, prescribed medication and received psychosocial interventions.

Data collection

Study measures include the following:

  1. Subjective quality of life using the MANSA (Priebe et al., 1999). The MANSA has been widely used in mental health research and contains satisfaction items with 12 life domains that are rated by the patient between 1 (most negative score) and 7 (most positive score). The mean score of those 12 items is used to reflect subjective quality of life.
  2. Mental Health Symptoms are assessed using three questionnaires. First, they are observer rated on the 24-item Brief Psychiatric Rating Scale (Ventura et al., 1993). Each symptom is assessed and rated between 1 (not present) and 7 (extremely severe) using a scoring guide. In addition, the participants are assessed for negative symptoms using the CAINS (Kring et al., 2013). The CAINS is an empirically developed and evaluated measure of negative symptoms; it covers several constructs such as motivation, pleasure, social engagement and affective expression. The CAINS ratings combine assessments of behavioural engagement in relevant activities and reported experiences of motivation and emotion, enabling comprehensive assessment of negative symptoms. Lastly, BSI (Derogatis, 1993) is administered. The BSI consists of 53 items covering 9 symptom dimensions, such as Somatization, Obsession-Compulsion, Interpersonal Sensitivity, Depression, Anxiety, Hostility, Phobic anxiety, Paranoid ideation and Psychoticism, and three global indices of distress, such as Global Severity Index, Positive Symptom Distress Index and Positive Symptom Total. The global indices measure current or past level of symptomatology, intensity of symptoms and number of reported symptoms, respectively.
  3. Satisfaction with services are assessed using the CSQ-8 (Attkisson & Greenfield, 2004), which is a self-report statement of satisfaction with health and human services. The questionnaire has been used in a wide variety of settings and contains 8 items, each of the items is rated on a 4-point Likert scale.
  4. Use of mental health services are documented on a simplified version of the Client Service Receipt Inventory (CSRI) (Beecham, 2001). Information on medication use, contact with mental health professionals and any instances of hospitalisation in the last 6 months are recorded. Research team in each country will provide a list of costs for commonly used medicines, health care services and inpatient stays.
  5. Health Economic Assessment will include Recovering Quality of Life (ReQoL-10) (Keetharuth et al., 2018), EQ-5D-5L (Herdman et al, 2011) and the Health Economic Assessment Inventory Form. ReQoL-10 is a patient-reported outcome measure that assesses the quality of life for people with various mental health conditions. EQ-5D-5L is a measure of health-related quality of life for clinical and economic assessment.

All questionnaires are administered at baseline, month 6 and month 12 in both the groups. The exception is the CAINS, which is administered at baseline and month 12. The Health Economic Assessment Inventory Form is completed at the end of the study by unblinded researchers.

Process evaluation is conducted to identify how and why the intervention was effective or ineffective and the effect of context. Quantitative data are collected to provide implementation feedback, for example, number, duration and content of delivered DIALOG+/TAU session. Researchers will collect these data after each scheduled session. In-depth interviews/focus groups with the participating clinicians, patients, service providers and policy makers are conducted at the end of the study to explore their experience with DIALOG+ intervention.

For more details about the study schedule, refer to Table 3.

Table 3

SPIRIT Diagram

Table 3

DATA ANALYSIS

The study will use both qualitative and quantitative methods. The statistical analysis follows guidance for mixed-methods research. Descriptive statistics is computed to compare the two groups at baseline. Our primary analysis is conducted on an intention-to-treat basis, with participants being analysed in the group to which they were randomised. The analysis is carried out using a repeated measure linear mixed model with unstructured covariance matrix to identify whether the groups differ in the quality of life at follow-up. The significance level is set at 5%. All data analyses are conducted using SPSS. We planned to use baseline MANSA as a covariate to account for different starting points amongst study patients. Two stratifying factors (male/female sex and professional status psychiatrist/non-psychiatrist) are considered as covariates. Complete case analysis is used as the primary analysis if the proportions of missing data are below approximately 5% (Jakobsen et al., 2017). The last observation carried forward (LOCF) imputation method is used to impute 6-month data (if available) into 12-month data. Details about qualitative assessments and analysis of qualitative data are provided under Aim 2. Quantitative and qualitative data are integrated to develop possible explanations for the trial outcomes (O’Cathain et al., 2010). In this process, we will first identify potentially useful explanatory themes that will be further discussed by the research team in order to build more generic explanatory theory. The study was powered to look at the effect across the whole sample (400 patients). We will perform exploratory analysis of differences and similarities between countries included in the study hoping that this work could provide additional insights into studied topics and possibly generate hypotheses for future studies.

SAMPLE SIZE

Preliminary sample size calculations were based on the data from the original DIALOG+ trial for patients with psychosis in which the effect size was 0.35 (Priebe et al., 2015). This effect size reflects improved the quality of life ratings for 5 of 12 life domains in DIALOG+ intervention that indicate a clinically and socially relevant improvement. To detect the effect size of 0.35, the trial would need 260 patients. The number was inflated to account for cluster design and 10% dropout of clinicians. As mentioned, clusters are clinicians working with patients with psychosis, and each clinician will deliver the intervention to five patients. The intracluster correlation coefficient (ICC) was 0.05, and the design effect (DE) due to cluster randomised trial was 1.2 (Kerry et al., 1998). Assuming a standard deviation of 1.0, with 80% power at the 5% significance level, the study requires a minimum of 36 clinicians in each arm (72 in total) at the start of the study. To allow for equal number of clinicians per country, we aim to recruit a minimum of 16 clinicians per country, 8 in the intervention group and 8 in the control group. In total, we aim to recruit 80 clinicians and 400 patients.

HEALTH ECONOMIC ASSESSMENT

Health Economic Assessment will be conducted from health and wider societal perspectives to explore the cost-effectiveness of the DIALOG+ intervention for patients with psychotic disorders. The evaluation will include the 12-month period with data collected at baseline, month 6 and month 12. Data for clinical outcomes will be collected for both the control and the intervention group. The cost of providing the DIALOG+ intervention and TAU will be estimated. Resource use associated with delivery of the intervention will be documented by the study team over the 12-month period using inventory forms developed by a health economist. It includes clinician’s salary, costs for documents and materials, and administrative costs. The costs for DIALOG+ intervention will also include the one-off costs such as staff training, and costs of purchasing tablets. In addition to the costs of delivery the intervention/treatment, data on the use of health and social care services will be collected at baseline, month 6, and month 12 by researchers using the CSRI (Beecham J, 2001). Unit costs collected from each of the five countries will be applied to resource use to estimate patient level costs. Costs will be calculated by combining resource use of delivering the intervention with respective costs.

Cost-effectiveness analysis will be conducted to illustrate the relation between costs and outcomes for DIALOG+ intervention. The cost-effectiveness analyses will compare the DIALOG+ intervention with TAU by pooling data from the five countries together. Sensitive analysis will be conducted with cost-effectiveness analysis on individual country. Furthermore, it will be conducted to investigate how uncertainty in the input parameters of the cost-effectiveness analyses can affect the results. The impact of missing data on the cost-effectiveness outcome will be addressed using multiple imputation procedures should issues associated with an incomplete dataset arise.

AIM 2: IMPLEMENTATION – CLINICIANS AND PATIENTS’ USE OF DIALOG+ AND FACTORS ASSOCIATED WITH ITS USE

The study focuses on assessing the intervention fidelity and the experience of clinicians and patients about implementing DIALOG+ in real-life mental health care settings. Data will also be collected on sustainability and reach to inform guidelines for scaling up and implementing the intervention widely.

INTERVENTION FIDELITY

Intervention fidelity refers to the degree to which an intervention is delivered as intended (Caroll et al., 2007). The evaluation of implementation fidelity is needed because this variable may not only moderate the relationship between an intervention and its outcomes but its assessment may also prevent potentially false conclusions from being drawn about an intervention’s effectiveness (Caroll et al., 2007). The study uses the framework for intervention fidelity proposed by Carroll and colleagues, which includes exposure or dose, adherence to an intervention, quality of delivery, participant responsiveness and intervention differentiation (Caroll et al., 2007).

A series of activities will be undertaken to assess intervention fidelity. Throughout the study unblinded research assistants (RAs) will contact clinicians after each scheduled DIALOG+/TAU session to collect information if the session happened as well as duration and content of the delivered session. For DIALOG+ sessions RAs will collect data on domains covered during the sessions and on agreed actions for patients and clinicians. IN addition, clinicians will be asked about any problems with delivering the intervention and how the problems or obstacles resolved. RAs will provide ongoing support and supervision to clinicians. To assess fidelity and adherence to study protocol, DIALOG+ manual and training materials, as well as quality of delivery, we aim to audiotape one DIALOG+ session per clinician. An operationalized adherence scale outlining core components of the intervention will be used to assess staff competence and the extent to which they comply with the study protocol. Content of DIALOG+ sessions will be analysed from tablet computers to provide additional information and insights. We will also audiotape and analyse sessions from the control arm (one per clinician) to assess contamination. Data on participant responsiveness (e.g. acceptance, perceived usefulness and engagement) and intervention differentiation (e.g. how different the active intervention is from the control condition) will be collected from RAs notes and from end-of-study qualitative interviews/focus group with study participants.

CLINICIANS’ AND PATIENTS’ EXPERIENCE OF IMPLEMENTING/USING DIALOG+

At the end of the study we will conduct in-depth interviews/focus groups with all clinicians participating in the intervention arm and with up to 15 patients per country who received DIALOG+. Purposive sampling will be used to capture different patients’ characteristics such as age, gender, and varying levels of engagement. The interviews will focus on: a) the experience of implementing/using the intervention; b) perceived benefits, barriers and facilitators of intervention use; c) participants’ engagement and responsiveness; and d) suggested adaptations. Data will be used for process evaluation.

SUSTAINABILITY AND REACH

In order to assess sustainability of the intervention in each country, we will conduct up to five individual in-depth interviews with service providers (e.g. service manager, clinical director, lead nurse) and policy makers. These interviews will focus on a) barriers and facilitators on the organisational level (infrastructure, clinical leadership engagement, ongoing monitoring, policy makers engagement), b) sustainability of the intervention (based on the Dynamic Sustainability Framework) (Chambers DA et al., 2013); and c) suggestions for scaling up and implementing the intervention widely. Three months after the last DIALOG+ sessions, we will contact all clinicians in each country to collect data on a) number of clinicians who continued to use DIALOG+ in their practice (intervention arm), b) number of clinicians who are delivering DIALOG+ in the control arm; and c) their reasons for using or not using DIALOG+ in their practice. Reach will be assessed based on number of clinicians and services that expressed interest and/or completed training in DIALOG+ throughout the study.

DATA COLLECTION

Table 4 offers details on data collected to evaluate intervention fidelity, participants’ experience of the intervention and sustainability and reach.

Table 4

Evaluation of implementation outcomes

OutcomeDescriptionData SourceData collection
Intervention fidelityIntervention fidelity framework: adherence to an intervention, exposure or dose, quality of delivery, participant responsiveness and programme differentiation (Carroll, 2007)Staff interviews Researchers’ notes 20% of sessions are audiotaped and analysed using content analysis

Content of DIALOG+ sessions is analysed from tablet computers
Throughout the study, end of the study
Clinicians’ and patients’ experience of using DIALOG+Acceptability and feasibility of the intervention, including acceptability of technology (e.g. using a computer tablet) to staffs and patients, participant responsiveness and engagement with the intervention and overall experience with the interventionPatient and staff interviews/focus groupsEnd of the study
Sustainability and reachOrganisational level: Sustainability (infrastructure, clinical leadership engagement, monitoring, factors to promote maintenance) and reach (number of additional sites using DIALOG+)

Individual level: Sustainability [number and characteristics of participants (staff and patients) who are using DIALOG+ after the competition of the active implementation, formative evaluations on reasons for using DIALOG+]; Reach (number of additional clinicians using DIALOG+)
Interviews with key informants at each site (site champion, medical director) based on the Dynamic Sustainability Framework 49 Staff and patient interviews Site visitsThroughout the study, end of the study

DATA ANALYSIS

As mentioned under Aim 1, the study uses both qualitative and quantitative methods. Data from focus groups and in-depth individual interviews are collected, transcribed and analysed. Data from the transcripts are coded in each of the participating centres and analysed using the thematic analysis framework proposed by Braun and Clarke (2006). Data from audiotaped sessions are transcribed and used to assess the adherence to the DIALOG+ manual using an operationalised adherence scale. Content of DIALOG+ sessions, including identified quality of life domains (first part) and agreed actions (second part), is collected from tablet computers after each delivered DIALOG+ session. Descriptive quantitative analysis determines the frequency of identified life domains as well as the frequency of actions agreed. Descriptive statistics determines the proportion of clinicians who will use the intervention 3 months after the study and the mean number of patients receiving the intervention.

DISCUSSION

This study will test the implementation of a generic psychosocial intervention called DIALOG+ in community-based mental health care services in five South European countries and its effectiveness at improving patients’ quality of life, mental health and satisfaction with care.

The study has several strengths and limitations. To the best of our knowledge, this trial is the first of its kind conducted in South Eastern Europe and one of very few trials in psychosocial mental health care provided to patients with psychosis globally. Although most international guidelines suggest combined-therapy approach including antipsychotic medication, talking therapy and family support, most research in this area is still focused on advances in pharmacotherapy. This trial is guided by the implementation strategy that is initially driven by the qualitative data from the pilot stage and is informed by behavioural theory, which is often neglected in the intervention literature. Further strengths are that patients and their families are actively involved in all phases of the study, the outcome assessors will be blinded to the allocation of patients and the inclusion criteria for patients are wide. The study also has several limitations. Clinicians could not be blinded towards their own allocation, which raises the possibility of performance bias. The dropout rate might be higher than expected because patients with psychosis have been reported as particularly difficult to engage and retain in research trials (Gilbody et al., 2002). Despite many similarities, the participating countries differ in the provision of standard outpatient mental health care as indicated in Table 2, and this may have an impact on the final study findings. The trial was not powered to detect differences in main outcomes amongst the participating countries; however, we plan to conduct an exploratory analysis to gain additional insights into main findings.

Overall, the study has the potential to provide considerable insight into whether DIALOG+ is feasible and effective in mental health care in South Eastern Europe and the wider LMICs context. Some of the challenges shared across LMICs include low national mental health care budgets and large treatment gap for psychotic disorders as previously mentioned. What is specific for countries included in this study and what might make the provision of mental health care different and more complex are the recent influx of refugees and migrants (after 2015) and the long-term psychological and social consequences of the war in 1990s. The main study findings are disseminated through a variety of mediums, including national and international meetings with clinicians, policy makers and researchers, national workshops, end-of-study workshops, peer-reviewed journal articles, social media platforms and updates on the project’s own website (http://impulse.qmul.ac.uk/home/).

ETHICAL APPROVAL AND INFORMED CONSENT

All procedures described in this protocol are in accordance with the ethical standards of the institutional and/or national research committee and with the 1975 Helsinki declaration and its later amendments or comparable ethical standards. All procedures were approved by the following ethics committees (in alphabetical order): Bosnia and Herzegovina (Klinicki Centar Univerziteta u Sarajevu – Eticki Komitet 03-02-4216, Eticki komitet JU Psihijatriska bolnica Kantona Sarajevo & JU Zavod za bolesti ovisnosti Kantona Sarajevo 02.8 – 408/19), Kosovo* (Hospital and University Clinical Service of Kosovo – Ethics Committee 2019-85), Montenegro (Javna Zdravstvena Ustanova Klinicki Centar Crne Gore – Eticki komitet 03/01 – 29304/1, ZU Specijalna Bolnica za Psihijatriju ‘Dobrota’ Kotor – Eticki komitet, Eticki Komitet JZU Dom Zdravlja ‘DR Nika Labovic’ Berane 01-47), Republic of North Macedonia (Eticka Komisija za istrazuvanje na luge, Medicinski Fakultet pri UKIM vo Skopje 03-24219), and Serbia (Eticka komisija Medicinskog fakulteta u Beogradu 2650/XII-20 and Eticka komisija Specijalne bolnice ‘Dr Slavoljub Bakalovic’ Vrsac 01-36/1) and the United Kingdom (Queen Mary University of London QMREC2204a, 16 October 2018).

All study participants will be asked to sign informed consent form prior to their participation in the study. The patients who agree to be involved in the study will meet with researchers who will check if patients meet the eligibility criteria and will invite them to sign a consent form. Researchers will explain the study to the patient and provide all relevant information, including risks, benefits and confidentiality. Once a patient signs informed consent, researchers can proceed to completing the baseline assessment. Participating patients will receive their routine treatment form the mental health care systems in their area and/or according their local health insurance arrangements. We don’t envisage any risks from participating in the study, however some participants taking part in assessment and interviews might be upset due to recalling distressing personal experiences, mental, physical health and/or social functioning problems. In the unexpected event that any patient appears highly stressed or upset, the research activity will be terminated and a clinician contacted.

FUNDING

This study has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779334. All authors received funding support through this grant for their effort related to this study.

ACKNOWLEDGEMENTS

We would like to thank clinicians and patients who expressed interest in participating in the study. We are also thankful to all research assistants on the Impulse project for their support during setting up and conducting the study. We would like to thank Professor Stefan Priebe, Dr Victoria Bird and Ms Aleksandra Matanov for their contributions during the initial phase of preparing the study and to Professor Borislava Mihaylova for supervising the health economic aspect of the study.

REFERENCES

  • Attkisson, C.C., Greenfield, T.K. (2004). The UCSF Client Satisfaction Scales: I. The Client Satisfaction Questionnaire-8. In: Maruish ME, ed. The use of psychological testing for treatment planning and outcomes assessment (3rd Edition). Mahwah (NJ, USA): Lawrence Erlbaum Associates; p. 799–811.

  • Beecham, J. (2001). Costing psychiatric interventions, in Measuring Mental Health Needs, T. G, ed. London: Gaskell.

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 32:77–101.

  • Cane, J., O’Connor, D., Michie, S. (2012). Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 24;7:37

  • Carroll, C., Patterson. M., Wood. S., Booth. A., Rick. J., Balain. S. (2007) A conceptual framework for implementation fidelity. Implement Sci 30;2:40.

  • Chambers, D.A., Glasgow, R.E., Stange, K.C. (2013). The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement. Sci. 8:117.

  • Curran, G.M., Bauer, M., Mittman, B., Pyne, J.M., Stetler, C. (2012). Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care;50:217–26.

  • Derogatis, L.R. (1993). BSI Brief Symptom Inventory: Administration, Scoring, and Procedure Manual. 4th ed. Minneapolis: National Computer Systems

  • Francis, J.J., O’Connor, D., Curran, J. (2012). Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci 7:35.

  • Gilbody, S., Wahlbeck, K., Adams, C. (2002). Randomized controlled trials in schizophrenia: a critical perspective on the literature. Acta Psychiatr Scand. 105: 243–251.

  • Herdman, M., Gudex, C., Lloyd, A., Janssen, M. F., Kind, P., Parkin, D., & Badia, X. (2011). Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Quality of life research; 20:1727–36.

  • Jakobsen, J.C., Gluud, C., Wetterslev, J., Winkel, P. (2017). When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts. BMC Med Res Methodol 17:162.

  • Keetharuth, A.D., Brazier, J., Connell, J., Bjorner, J. B., Carlton, J., Buck, E. T., & Barkham, M. (2018) Recovering Quality of Life (ReQoL): a new generic self-reported outcome measure for use with people experiencing mental health difficulties. The British Journal of Psychiatry 212:42–49.

  • Kerry, S.M., Bland, J.M. (1998) The intracluster correlation coefficient in cluster randomisation. BMJ;316:1455.

  • Kring, A.M., Blanchard, J.J., Horan, W.P., Reise, S.P. (2013) The Clinical Assessment Interview for Negative Symptoms (CAINS): final development and validation. American Journal of Psychiatry 170:165–72.

  • Maric, N.P., Andric Petrovic, S., Rojnic-Kuzman, M., Riecher-Rössler, A. (2019). Implementation of early detection and intervention services for psychosis in Central and Eastern Europe: Current status. Early Interv Psychiatry 13:1283–1288.

  • McDaid, D., Knapp, M., & Curran, C. (2005) Funding mental health in Europe. Copenhagen: European Observatory on Health Systems and Policies.

  • Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J. and Wood, C. E. (2013). The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Annals of Behavioral Medicine 46:81–95.

  • O’Cathain, A., Murphy, E., Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. BMJ 17;341:c4587.

  • Omer, S., Golden, E., Priebe, S. (2016). Exploring the Mechanisms of a Patient-Centred Assessment with a Solution Focused Approach (DIALOG+) in the Community Treatment of Patients with Psychosis: A Process Evaluation within a Cluster-Randomised Controlled Trial. PLoS One;11:e0148415.

  • Phillips, C.J., Marshall, A.P., Chaves, N.J., Jankelowitz, S.K., Lin, I.B., Loy, C.T., Rees, G., Sakzewski, L., Thomas, S., To, T.P., Wilkinson, S.A., Michie, S. (2015). Experiences of using the Theoretical Domains Framework across diverse clinical environments: a qualitative study. J Multidiscip Healthc 18;8:139–46.

  • Priebe S, H.P., Knight S, Evans S. (1999). Application and results of the Manchester Short Assessment of Quality of Life (MANSA). The International journal of social psychiatry 45:7–12.

  • Priebe S, K.L., Omer S, Golden E, Walsh S, Khanom H, Kingdon D, Rutterford C, McCrone P, McCabe R. (2015). The Effectiveness of a Patient-Centred Assessment with a Solution-Focused Approach (DIALOG+) for Patients with Psychosis: A Pragmatic Cluster-Randomised Controlled Trial in Community Care. Psychotherapy and psychosomatics 84:304–13.

  • Priebe, S., Golden, E., Katevas, H., Healey, P., McCabe, R. (2016) DIALOG+ Manual. Available at: https://dialog.elft.nhs.uk/file_download.aspx?id=3306 [Accessed 25 Sept 2019].

  • Sadock, B.J. & Sadock, V. (2014) Kaplan and Sadock’s Synopsis of Psychiatry: Behavioral Science/Clinical Psychiatry. 11th edition, Wolters Kluwer.

  • Svab, V., Svab, I. (2013). Barriers and errors in the implementation of community psychiatry in Slovenia. Ment Health Fam Med 10:23–8.

  • Ventura, J., Lukoff, D., Nuechterlein, K.H., Liberman, R.P., Green, M.F., Shaner, A. (1993). Manual for the Expanded Brief Psychiatric Rating Scale. International Journal of Methods in Psychiatric Research 3:177–43.

  • WHO (1990) ICD-10. International classification of disease. 10th ed. Geneva

  • WHO (2008). MhGAP Mental Health Gap Action Programme. Scaling up care for mental, neurological, and substance use disorders.

  • WHO (2014). Mental Health Atlas. Available at: http://www.who.int/mental_health/evidence/atlas/mental_health_atlas_2014/en/ [Accessed 25 Sept 2019].

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Attkisson, C.C., Greenfield, T.K. (2004). The UCSF Client Satisfaction Scales: I. The Client Satisfaction Questionnaire-8. In: Maruish ME, ed. The use of psychological testing for treatment planning and outcomes assessment (3rd Edition). Mahwah (NJ, USA): Lawrence Erlbaum Associates; p. 799–811.

  • Beecham, J. (2001). Costing psychiatric interventions, in Measuring Mental Health Needs, T. G, ed. London: Gaskell.

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 32:77–101.

  • Cane, J., O’Connor, D., Michie, S. (2012). Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 24;7:37

  • Carroll, C., Patterson. M., Wood. S., Booth. A., Rick. J., Balain. S. (2007) A conceptual framework for implementation fidelity. Implement Sci 30;2:40.

  • Chambers, D.A., Glasgow, R.E., Stange, K.C. (2013). The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement. Sci. 8:117.

  • Curran, G.M., Bauer, M., Mittman, B., Pyne, J.M., Stetler, C. (2012). Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care;50:217–26.

  • Derogatis, L.R. (1993). BSI Brief Symptom Inventory: Administration, Scoring, and Procedure Manual. 4th ed. Minneapolis: National Computer Systems

  • Francis, J.J., O’Connor, D., Curran, J. (2012). Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci 7:35.

  • Gilbody, S., Wahlbeck, K., Adams, C. (2002). Randomized controlled trials in schizophrenia: a critical perspective on the literature. Acta Psychiatr Scand. 105: 243–251.

  • Herdman, M., Gudex, C., Lloyd, A., Janssen, M. F., Kind, P., Parkin, D., & Badia, X. (2011). Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Quality of life research; 20:1727–36.

  • Jakobsen, J.C., Gluud, C., Wetterslev, J., Winkel, P. (2017). When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts. BMC Med Res Methodol 17:162.

  • Keetharuth, A.D., Brazier, J., Connell, J., Bjorner, J. B., Carlton, J., Buck, E. T., & Barkham, M. (2018) Recovering Quality of Life (ReQoL): a new generic self-reported outcome measure for use with people experiencing mental health difficulties. The British Journal of Psychiatry 212:42–49.

  • Kerry, S.M., Bland, J.M. (1998) The intracluster correlation coefficient in cluster randomisation. BMJ;316:1455.

  • Kring, A.M., Blanchard, J.J., Horan, W.P., Reise, S.P. (2013) The Clinical Assessment Interview for Negative Symptoms (CAINS): final development and validation. American Journal of Psychiatry 170:165–72.

  • Maric, N.P., Andric Petrovic, S., Rojnic-Kuzman, M., Riecher-Rössler, A. (2019). Implementation of early detection and intervention services for psychosis in Central and Eastern Europe: Current status. Early Interv Psychiatry 13:1283–1288.

  • McDaid, D., Knapp, M., & Curran, C. (2005) Funding mental health in Europe. Copenhagen: European Observatory on Health Systems and Policies.

  • Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J. and Wood, C. E. (2013). The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Annals of Behavioral Medicine 46:81–95.

  • O’Cathain, A., Murphy, E., Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. BMJ 17;341:c4587.

  • Omer, S., Golden, E., Priebe, S. (2016). Exploring the Mechanisms of a Patient-Centred Assessment with a Solution Focused Approach (DIALOG+) in the Community Treatment of Patients with Psychosis: A Process Evaluation within a Cluster-Randomised Controlled Trial. PLoS One;11:e0148415.

  • Phillips, C.J., Marshall, A.P., Chaves, N.J., Jankelowitz, S.K., Lin, I.B., Loy, C.T., Rees, G., Sakzewski, L., Thomas, S., To, T.P., Wilkinson, S.A., Michie, S. (2015). Experiences of using the Theoretical Domains Framework across diverse clinical environments: a qualitative study. J Multidiscip Healthc 18;8:139–46.

  • Priebe S, H.P., Knight S, Evans S. (1999). Application and results of the Manchester Short Assessment of Quality of Life (MANSA). The International journal of social psychiatry 45:7–12.

  • Priebe S, K.L., Omer S, Golden E, Walsh S, Khanom H, Kingdon D, Rutterford C, McCrone P, McCabe R. (2015). The Effectiveness of a Patient-Centred Assessment with a Solution-Focused Approach (DIALOG+) for Patients with Psychosis: A Pragmatic Cluster-Randomised Controlled Trial in Community Care. Psychotherapy and psychosomatics 84:304–13.

  • Priebe, S., Golden, E., Katevas, H., Healey, P., McCabe, R. (2016) DIALOG+ Manual. Available at: https://dialog.elft.nhs.uk/file_download.aspx?id=3306 [Accessed 25 Sept 2019].

  • Sadock, B.J. & Sadock, V. (2014) Kaplan and Sadock’s Synopsis of Psychiatry: Behavioral Science/Clinical Psychiatry. 11th edition, Wolters Kluwer.

  • Svab, V., Svab, I. (2013). Barriers and errors in the implementation of community psychiatry in Slovenia. Ment Health Fam Med 10:23–8.

  • Ventura, J., Lukoff, D., Nuechterlein, K.H., Liberman, R.P., Green, M.F., Shaner, A. (1993). Manual for the Expanded Brief Psychiatric Rating Scale. International Journal of Methods in Psychiatric Research 3:177–43.

  • WHO (1990) ICD-10. International classification of disease. 10th ed. Geneva

  • WHO (2008). MhGAP Mental Health Gap Action Programme. Scaling up care for mental, neurological, and substance use disorders.

  • WHO (2014). Mental Health Atlas. Available at: http://www.who.int/mental_health/evidence/atlas/mental_health_atlas_2014/en/ [Accessed 25 Sept 2019].