Specialty training in family medicine in Montenegro – An evaluation of the programme by the first generation of trainees

Open access

Abstract

Introduction

One of the aims of health care reform in Montenegro is to strengthen primary care. An important step forward is the implementation of specialty training in family medicine (FM). The aim of this article is to evaluate the implementation of specialty training in family medicine in Montenegro, regarding the content, structure and methods, by the first generation of trainees and the coordinator of the training.

Methods

A questionnaire was sent by mail in July and August 2017 to all 26 eligible trainees who started specialty training in 2013. Twenty-two of the 26 trainees (84.6%) responded. The questionnaire consisted of closed and open-ended questions related to the evaluation of the training. A descriptive quantitative and qualitative analysis with predefined themes and a semi-structured interview with the coordinator were carried out.

Results

The process of training in FM was assessed positively by both trainees and the coordinator. The positive assessment included that the specialisation course offered modern design through modules and practice, and trainees both improved their existing knowledge and skills and acquired new ones necessary for everyday work. The coordinator emphasised the importance of the introduction of new teaching methods and formative assessment, the important role of mentors, and the involvement of Slovenian colleagues in the teaching process and supervision of the programme.

Conclusions

The implementation of speciality training in FM in Montenegro was successful. Several assessment methods were used that can be further developed in individual structured feedback, which could stimulate the continual improvement of trainees’ knowledge and competencies.

1 Introduction

For decades, the World Health Organisation has underlined the importance of strong primary health care for better health of the population, for reducing inequality, for improving access to health care, and for lower costs (1, 2). Systems which are oriented towards secondary and tertiary health care are both more expensive and less effective (3).

In Montenegro, one of the former Yugoslavian countries, primary health care is community-oriented (4) and declared a priority by law (5, 6). The Ministry of Health of Montenegro prepared health care reform in 2003 and introduced new roles and responsibilities for primary health care teams. One of the main changes was that each patient must select a (personal) doctor who has a gate-keeping role and responsibility for the quality of service. One of the goals of health care reform was also to implement a curriculum of family medicine (FM) into undergraduate education, and to develop and implement the specialisation of family medicine. This process has been described in detail elsewhere (7).

The first step in implementing a specialisation in family medicine was an intensive 4-month course for almost all the primary care teams in the country, which started in 2005 with the help of Slovenian educators. From 2005 to 2011, eleven courses were organised, and 222 primary health care teams finished the training. The first results showed that the quality of work increased; primary care was more accessible, the organisation of the service improved, and patient satisfaction was higher (7).

The next step was the organisation of an additional one-year training programme for the most experienced General Practitioners (GPs) who were motivated towards teaching; these became the first specialists in family medicine. In 2012, twenty-four GPs completed this programme. It was intended as a short-term solution to create mentors for future trainees in family medicine. The programme was led by the Department of Family Medicine at the Faculty of Medicine, of the University of Ljubljana, as a part of a project by the World Bank (8).

The first generation of trainees to start the full four-year specialty training began their course in January 2013. The specialisation course in family medicine was led by a coordinator from the Faculty of Medicine of the University of Montenegro, in Podgorica, who is responsible for specialty training in family medicine under the supervision of the Department of Family Medicine of the Faculty of Medicine in Ljubljana. The specialisation course is based on modern concepts of teaching, described in the EURACT Educational Agenda (9). The Agenda was conceived to define which specific competences can be expected from the discipline of general practice/family medicine, and to enable the harmonisation of the different learning programmes in Europe. The Educational Agenda represents the basis for preparing relevant training programmes that are mainly competence-driven (10).

The specialisation course has practical and theoretical elements, which run side by side during both the family practice component and the hospital-based component. Its total length is 48 months; 24 months take place in family medicine practices working with a direct mentor (the family practice component), and the other 24 months take place in different clinical wards (the hospital-based component). The family practice component is further divided into two parts, namely: the first part lasts 5 months and the second part 19 months, of which 4 months are spent in the primary care paediatric offices and 15 months in family medicine practices. During the family practice component, 20 two-day modules of theoretical teaching are organised on the specific knowledge, skills and attitudes of family medicine (e.g., the organisation of the practice, the quality of care, the management of patients with chronic diseases, and drug prescribing).

The hospital-based component of the specialisation course takes place at the Clinical Centre of Montenegro, which represents the teaching basis of the Faculty of Medicine, and in some general hospital wards. The clinical rotations are as follows: 6 months of internal medicine, 3 months of surgery and paediatrics, and 2 months of gynaecology and psychiatry, while other specialties, e.g., dermatology, orthopaedics, etc., have one month each. The detailed programme of the specialisation course has been published elsewhere (11).

Before the implementation of the training programme, several obstacles in organisation, negative public and professional opinion, and political indifference had to be overcome. Many problems were exacerbated by the fact that in Montenegro specialisation in family medicine is not mandatory (7).

In general, doctors were not interested in applying for specialty training for financial reasons – the salary of trainees was smaller than what they had been able to earn before, and even after they finished the programme, it would still only be similar to that of doctors who had not completed the specialisation course.

Doctors on the training course temporarily lost their autonomy – before entering the programme, they worked independently, but during the training, they had to work under the supervision of mentors in the mentors’ own practices. In addition, primary health care managers were not interested in sending their experienced GPs (most of them had at least five years of experience), as they lost part of their workforce and had to pay expenses.

The Faculty of Medicine did not fully support the development of the training course; they allocated only one lecturer (who was also involved in the teaching of another subject at the university) and two assistants. There was no Department of Family Medicine with administrative staff and devoted space, and financial resources for the coordination of the training course were very limited.

There was also no political will to legalise the specialisation in family medicine as obligatory for all GPs.

However, despite these many barriers, the specialisation course was carried out according to the programme, and in February and June 2017, trainees from the first generation completed the programme. Some of them have already passed the final examination. The evaluation of the programme is the first step in the quality improvement process to improve the process of this training. The aim of this article is to evaluate the implementation of specialty training in family medicine in Montenegro, regarding the content, structure and methods, by the first generation of trainees and the coordinator of the training.

2 Participants and methods

2.1 Participants

All the 26 trainees in the first generation of the family medicine training, who began their training in 2013, were invited to participate in the study. Twenty-two (84.6%) of the trainees participated in the study, of whom 11 had already passed the final exam, while 10 trainees were prolonging their training (9 due to maternity leave) and one failed the exam.

2.2 Methods

A questionnaire was sent by post to all 26 eligible trainees in July and August 2017.

The questionnaire was developed based on the literature by researchers from both Faculties of Medicine involved in the training as educators. Specifically, the questionnaire was composed of one on the satisfaction of trainees with the training programme, prepared by the Irish College of General Practitioners (12), and the Job Evaluation Survey tool, a simple, validated tool to evaluate the satisfaction of trainees with their training (13). A consensus of all the researchers on the content of the questionnaire was reached, considering the aim of the study. Additionally, a former trainee who had recently passed the speciality exam was also involved in the preparation of the final version of the questionnaire.

The questionnaire consisted of the following combination of closed and open-ended questions: the demographic characteristics of the trainees, and questions related to the evaluation of the process of the training and the specialty exam. Evaluation of the training was made by using a Likert scale from 1 (very poor) to 5 (excellent) to assess the training in terms of usefulness, quality, learning methods, mentorship and organisation. The final questions asked the trainees how the training had influenced their work in the practice and how their expectations had been fulfilled.

A semi-structured interview was carried out by the two researchers (MPŠ and DP) with the coordinator of the training, to discover the obstacles in the implementation of the training and to evaluate the programme.

2.3 Statistical Analyses

In the quantitative part of the analyses, descriptive statistical methods, with mean and standard deviation in the case of a normal distribution of data, and range and median in the case of a non-normal distribution of data, were used.

Simple thematic qualitative analyses with predefined themes were carried out: the assessment of the specialty training; the role of mentors; the organisation of the specialty training; the influence on the work in practice; and the fulfilment of expectations. The themes for analysis of the semi-structured interview with the coordinator were: teaching methods, assessment of knowledge, the final exam, mentors, and foreign advisors. Open coding of the text quotations was carried out separately by two researchers (DP and MPŠ); where there were differences between the researchers, the final decision was reached by a consensus. The sub-themes which emerged from the coding were checked as to whether they fitted into the predefined themes. If any new themes emerged, they were added to the predefined themes.

3 Results

3.1 Description of the Population

The basic demographical data of the 22 participating trainees are in Table 1. The trainees were from all regions of Montenegro and were predominantly female.

Table 1

Description of the participating trainees.

CharacteristicN or Mean% or distribution of values, SD
Gender (Female)1986.4%
Age39.9from 35-57, SD 5.0 years
Age at the beginning of training35.7from 31-52, SD 4.7 years
Average grade of undergraduate education8.1from 7.0-9.7, SD 0.8
Years of working experience in FM before starting the training6.3from 2-20 years, SD 4.4 years

The trainees evaluated different aspects of training and the impact of the training on their daily practice. The evaluation of the training by Likert scale is set out in Table 2.

Table 2

Evaluation of the training.

Aspects of evaluationLowest/highest rate (range)N or Mean
Were your expectations of the specialty training fulfilled? (Usefulness)3-5 (2)4.41 (0.73)
How would you assess the overall quality of the training (usefulness, quality, teaching methods)? (Quality)3-5 (2)4.41 (0.67)
Did you find the teaching methods to be appropriate to reaching the teaching goals? (Teaching methods)3-5 (2)4.62 (0.59)
How would you assess the role of your main mentor? (Mentorship)3-5 (2)4.55 (0.74)
How would you assess the organisation of the training? (Organisation)2-5 (3)4.68 (0.72)
Did the training have an impact on your daily practice? (Impact on daily practice)3-5 (2)4.43 (0.75)

The trainees assessed the process of training with high scores. The highest mean was given for ‘organisation,’ but it also received the highest range – from poor to excellent. The teaching methods were also assessed very highly (between very good and excellent).

3.2 Formative Assessment

In each year of the training, the trainees had to take a written test, called the ‘progress test.’ It consisted of 100 multiple choice questions from a database of 1400 questions prepared for the final exam and gave the trainees feedback about the required and obtained knowledge. Table 3 represents the results of the progress test; 60 or more points were required to pass the final exam.

Table 3

Results of the progress test.

Test yearNMinimum number of pointsMaximum number of pointsMean (SD)The number (percentage) of trainees with at least 60 points (pass/fail criteria)
201319317457.710 (52.6%)
201421639480.421 (100%)
201514537565.911 (78.6%)
201617568471.715 (88.2%)

The percentage of trainees who obtained at least 60% of positive answers increased from the first to the last year of training, except for 2014, in which all the trainees successfully passed the progress test.

3.3 The Final Exam

Of the 22 participants in the evaluation, 12 had already taken the final exam, which consisted of 120 multiple-choice questions, 12 OSCE stations (Objective Structured Clinical Examination), two MEQs (Multiple Essay Questions) and an oral exam. The multiple-choice questions assess the trainee’s knowledge, while the OSCE stations assess the trainee’s skills. The assessment criteria for the OSCE stations were predefined to reduce the subjectivity of the observer. The MEQs assessed the student’s ability to identify and resolve clinical problems by applying their existing knowledge. The answers to the MEQs were assessed by two independent assessors from the assessing committee, who arrived at a consensus.

Only those candidates who scored at least 60% of the points in the written exam (72 points), passed 9 out of the 12 OSCE stations, and satisfactorily answered both MEQs had the right to continue to the final oral exam.

Table 4 presents data on the grades in the different elements of the exam and in the final exam.

Table 4

Grades in the different elements of the exam.

Elements of the examPass criteriaThe number (%) of participants who passed this part of the examMean number points (N=12)Range, median
Written test (max. number of points: 120)72 points (60%)12 (100)105.8From 96 to 117, median 104

OSCE station (max. 12)9 (75%)12 (100)11.8From 10 to 12, median 12

MEQ questionsExcellent, very good or good12 (100)

Final examPass11 (91.7)

A comparison of the results of the progress test and of the final exam shows that additional learning was needed to succeed in the final exam. One candidate was not successful in the final exam.

3.4 Qualitative Analyses of Data

3.4.1 Evaluation of the Programme and Exam by the Trainees

The content analysis with predefined themes revealed 64 codes that were attributed to the following themes: the assessment of the specialty training, the role of the mentors, the organisation of the specialty training, the influence on the work in practice, and the fulfilment of expectations. Most of the themes were presented by positive and negative aspects. To protect the anonymity of the trainees, the quotations are only identified by the trainee number.

Assessment of specialisation

There were 14 different codes for positive assessment, and 6 for negative. The positive assessment stated that the specialisation course offered teaching modules and practice which were based on modern educational theory, improved existing and acquired new knowledge, and offered practical useful knowledge and skills necessary for everyday work. It helped the trainees not only to master the principles of modern medical trends, but also improved professionalism, increased self-confidence, and improved the scientific approach to everyday work using guidelines and evidence-based medicine (EBM).

Fulfilment of expectations was unanimously high, from enough/sufficient to completely fulfilled and ‘more than expected.’

Negative assessment was exclusively associated with the clinical components of rotations, where mentors were not adequately acquainted with the scope of clinical knowledge for FM, the lack of influence of the Department of Family Medicine on training in clinical rotations, and some clinical lectures being too theoretical. One opinion was also that the training programme was too ambitious for the existing working conditions.

I learned how to deal with science, to analyse my work, to defend my opinion, to accept mistakes, and not to be ashamed of presenting. Enforcing EBM is a great thing and a step forward in Montenegro. (Trainee 1)

The part of the module dealing with clinical branches was not well enough adapted to what we will be dealing with as family medicine specialists in the future. (Trainee 2)

The role of the main and direct mentors

To a great extent, the experience was very positive; the mentors were motivated, committed, enthusiastic, selfless, collaborative, good role models and friendly.

There were some negative comments regarding clinical mentors showing variable dedication and accessibility, and also that some were unprepared and insufficiently engaged, showing a lack of knowledge of the content and the specialisation course plan.

The clinical mentors were unprepared, mainly due to a lack of knowledge of the content and the specialisation course plan, i.e. the scope of work that future specialists should undertake. (Trainee 3)

The organisation of the specialty training

The trainees stated that the organisation was very good, offering continuity in learning with teaching organisations, and that the modules and practice were well-balanced throughout the course. Negative comments included that the practical and theoretical elements of the same field should be given simultaneously, and that more time should be spent in outpatient clinics. There were criticisms of the organisation of the clinical mentors and the quality of some lecturers at the beginning of the specialisation course.

Technical, accurate, concise, clear, open principles and without autocracy. (Trainee 1)

Perfect. (Trainee 4)

It would be much more beneficial to carry out a certain element practically and follow a theoretical module in the same field at the same time. (Trainee 2)

The influence of the training on work in the practice

The trainees stated that they had acquired a new approach to patients, new knowledge and skills, and more EBM and scientific reasoning. They also found the influence of the training in their changed attitudes towards work and medicine, specific methods of work (e.g., time as diagnostic criterion), and improved communication and self-confidence in their everyday work. The training influenced their organisation of teamwork, and the realisation of how important the satisfaction of both patients and health workers is as a quality criterion.

Through the specialisation course I have deepened the knowledge and skills necessary for working in the clinic, as well as the attitude that it is necessary to treat the patient rather than the disease, and to use time as a diagnostic criterion. (Trainee 5)

The assessment of the final exam

The exam was described as well-conceived, and appropriate for testing the necessary knowledge and skills for practical work. A human approach towards the trainees was mentioned. It was an important moment in the lives of the trainees. It was suggested that it should be organised over two days instead of one.

One of the most beautiful moments in my life. (Trainee 6)

All parts of the exam are intended for acquiring the practical knowledge and skills necessary for working in an outpatient clinic. (Trainee 5)

3.4.2 The evaluation of the programme and exam by the coordinator

During the interview, the coordinator assessed the process of the training and the final exam positively, and expressed the main points for the success, described below.

New teaching methods

Mixed teaching methods were introduced: fewer plenary lectures and more work in small groups; interactive learning and project work; and teaching of clinical skills on models. Small group work led by experienced group leaders fostered the changing of attitudes and changes in clinical practice. The professional integrity of trainees increased, and they were able to implement the new knowledge and attitudes in their practice.

The assessment of knowledge

Modern teaching and assessment methods, including formative assessment with progress tests, encouraged the upgrading of knowledge and continuity in learning.

The final exam

The complex structure of the final exam gave a comprehensive assessment of the candidate.

Mentors

The number of competent mentors and their distribution all over the country were not adequate to the needs of the training. There were some enthusiastic mentors without trainees and some mentors in the capital with more than three trainees at the same time. Nevertheless, the mentors, especially the main mentors, had an extremely important role in the whole process of the training; most of the trainees recognised a role model in their mentor.

Foreign advisors

The involvement of our Slovenian colleagues in the teaching process, their supervision of the programme, and their participation in the final exam set the whole process at a higher level and gave it a better reputation at the national and international level.

4 Discussion

4.1 Main Findings

A strong will and efforts of the coordinator and team involved in the process of the training brought the project to a successful conclusion. The first generation of specialist family medicine trainees in Montenegro have successfully finished their specialty training.

The evaluation of the programme by the trainees and the coordinator showed that it did not only increase their knowledge and skills, but also increased their selfconfidence and improved professionalism and scientific approach in their everyday work, using guidelines and evidence-based medicine.

4.2 Comparison with the Existing Literature

4.2.1 The Evaluation of the Process of Training and Formative Assessment

Training in family medicine, taking into account the EURACT Educational Agenda (9) and modern teaching methods, and also including constant feedback from mentors according to the ‘Slovenian model’ of training (14), was well accepted by the trainees in Montenegro. There is one important difference between the two programmes (namely, Slovenian and Montenegrin) - in Montenegro the trainees lose contact with general practice and with their main mentors for two years during their clinical rotations, because, at that time, all the teaching activities take place in hospitals.

The organisation of the training was in general assessed very highly. Most of the criticisms came from lower satisfaction with clinical rotations, mainly because the clinical mentors were not well acquainted with the training programme and goals.

During the training of the first group of Montenegro trainees, progress tests were used annually to evaluate the level of knowledge, and we found that the differences in knowledge reduced through the process of training (15). We did not use the OSCE method for formative purposes assessment because it has lower psychometric standards (16). Longitudinal and competence-based assessment were also found to be the currently preferred approach for FM specialty training in other studies, such as in a survey based on a convenient sample in five European countries (Denmark, Germany, Poland, the Netherlands and the United Kingdom) (17).

One of the challenges of teaching medical trainees is to choose an assessment method that is directed towards enhancing learning in addition to assessing clinical competence. Workplace-based assessments allow trainees to continually gather evidence of learning and formative feedback (18).

Feedback in workplace-based clinical settings often relies on expert trainers’ judgements of the directly observed trainee (15, 18). Close contact between the mentor and the trainee working in the same practice during the family practice component of the rotation enables constant feedback. According to the results of our study, the trainees assessed their mentor’s feedback as valuable and often took their mentors as a role model. However, this assessment was not structured, and it relied on the personal approach of the mentor. In the future, the feedback of mentors will need to be further developed and structured according to the developing competencies and progress of the trainees, so that it is useful for the trainees and feasible for the mentors at the same time (19, 20).

One of the important achievements of the training was the participants’ feelings that the training helped them to improve the quality of their work and patient safety. The trainees mentioned that they felt more competent in decision-making and felt more self-confident and less vulnerable to potential medical errors and complaints (21). A unique goal of the training programme was to give trainees an understanding of the holistic and generalist approach, on the one hand, and the usefulness of EBM, on the other, and to see possibilities for future development of the discipline (22, 23).

4.2.2 The Final Exam

The use of OSCEs for the assessment of clinical skills in a standard setting has been shown to be normative in all high stake exams (16). The differences in clinical skills between the trainees were far smaller than the differences in the written part of the exam. One of the reasons for a good performance in the OSCEs was the limited number of OSCE stations and that the trainees could relatively easily gain the skills at least at a moderate level. In the future, the set of OSCE stations should be increased to enable the testing of various skills which a FM specialist must master. In the MEQs, and especially in the oral exam, we tested the level of critical thinking based on clinical knowledge. According to the findings of Ross and co-authors, there is a significant positive correlation between critical thinking skills and performance on knowledge tests (24). In addition, good critical thinking skills have been found to predict success in family medicine certification examinations. An assessment of critical thinking in the progress tests during the training may help to identify applicants more likely to be successful in the final certification exam (25).

Finally, an important emotional aspect of the exam was that it presented a positive experience, even ‘a beautiful moment in life,’ and gave the colleagues necessary confidence for their future professional careers and positive incentives for future learning.

Mentorship

Mentorship was shown to be extremely important in specialty training. Mentors have different roles, from organisational ones to very personal ones (being a friend); for most of the trainees, the mentor also became a role model. According to Hesketh et al., being a good mentor means being a good clinician, knowing the programme and goal of the training, but also having appropriate personal characteristics (26).These expectations have been found in other literature – the mentor’s ability to be a personal ‘role model’ and coaching is coupled with their clinical and coordinative work with family medicine trainees (27). From the trainees’ feedback it seems that the main mentors fulfilled all the expectations. This is very encouraging, especially if we consider that there was a severe shortage of appropriate main mentors, and that some family physicians were mentoring up to three trainees. The main shortcoming of the clinical mentors was their lack of knowledge of the content and specialisation plan. The clinical mentors did not have previous experience with mentoring family medicine trainees and sometimes had unrealistic expectations (e.g., they expected too much of the trainees’ clinical knowledge, and did not understand the role of the family medicine specialist in the system). Evidently, the problem is not only local, as we can find the same opinion in the literature, where Bulc et al. and Švab et al. emphasise that all teachers participating in the training of family medicine should be familiar with the basic characteristics of the discipline and theoretical framework of family medicine (28, 29). For the future, it seems there is a need to organise training for clinical mentors as well, so that they can become familiar with the aims, content, process and methods of teaching and giving feedback to the trainees (27). Further development of the programme and content for all clinical rotations is also necessary.

In Slovenia, which has almost 400 trainees at the moment, comments from the trainees are similar to those in Montenegro. To improve satisfaction with the clinical rotations and clinical mentorship, several regional coordinators have been implemented in the training programme. They represent a coordinative and pedagogical structure of working in the local environment, where most of the clinical rotations are performed, to make practical improvements in the training process (27, 30).

The recognition of family medicine as an academic discipline, and the support of the University and the Faculty of Medicine of Montenegro in establishing a Department of Family Medicine, would overcome many obstacles in the implementation of the programme. A department with professional personnel, an organisational structure and financial resources is necessary for a stable situation, which can offer conditions for improvement.

4.3 Limitations of the Study

In this study, we assessed the process of implementation of speciality training in family medicine. However, our study has several limitations. We included trainees who had not yet passed the specialty exam; these trainees (10 out of 22) may have assessed the process of training more positively, in order to try to present themselves positively to their future examination committee.

Due to the reasons of feasibility (trainees living in different parts of the country, where some areas are not very well connected to the capital), we sent a questionnaire with open-ended questions to participants by post and asked them for written answers. Some of the participants gave us no answers or very general statements in reply to the open questions, mainly to the questions related to the organisation of the specialty course and the assessment of the exam.

As the study was not anonymous, the participants tended not to answer the open-ended questions or gave relatively general, not very critical answers. Other methods, e.g., a focus group study or more in-depth one-to-one interview with all the trainees, could have given us a more detailed and accurate view of the satisfaction of the trainees with the programme, the process of training and exam, but these two methods were not feasible on this occasion.

The assessment of quality is a complex process; many subjects can assess quality, including patients, colleagues, professional organisations, health care authorities and society in general, which assesses social acceptance. In our study, the evaluation of the implementation of the specialty training was made only by the trainees and the coordinator of training. The view of the coordinator of training is subjective and, from that point of view, biased.

5 Conclusion

New teaching methods, mentorship and formative assessment were introduced into the education process for the first time. The implementation of the evidence into the medical training, considering the feedback of the trainees and teachers, helps to change educational practice towards the goal. This study represents the first attempt to evaluate the process, results and satisfaction with the training programme. In the future, other methods should be implemented, such as a more structured evaluation of the progress of the trainee.

Conflicts of interest: The authors declare that no conflicts of interest exist.
Ethical approval: Ethical approval was not necessary according to practice in Montenegro, as only anonymous data of the trainees who voluntarily participated in the survey were used.
Acknowledgments

We would like to thank all the trainees and colleagues who participated in the research.

References

  • 1

    World Health Organization. Declaration of Alma-Ata: International Conference on Primary Health Care Alma-Ata USSR 6–12 September 1978. Geneva: WHO 1978. Accessed January 28th 2018 at: http://www.who.int/hpr/NPH/docs/declaration_almaata.pdf.

  • 2

    Starfield B Shi L Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83:457-502.

    • Crossref
    • Export Citation
  • 3

    Kringos D Boerma W Bourgueil Y Cartier T Dedeu T Hasvold T et al. The strength of primary care in Europe: an international comparative study. Br J Gen Pract. 2013; 63(616):e742-50. .

    • Crossref
    • Export Citation
  • 4

    Klančar D Svab I. Primary care principles and community health centres in the countries of former Yugoslavia. Health Policy. 2014;118:166-72.

    • Crossref
    • Export Citation
  • 5

    Saric M Rodwin V. The once and future health system in the former Yugoslavia: myths and realities. J Public Health Policy. 1993;14:220-37.

    • Crossref
    • Export Citation
  • 6

    Svab I Rotar Pavlic D Radic S Vainiomaki P. General practice east of Eden: an overview of general practice in Eastern Europe. Croat Med J. 2004;45:53-42.

  • 7

    Cvejanov Kezunović L Drecun M Švab I. Primary care reform in Montenegro. Zdr Varst. 2013;52:247-54. .

    • Crossref
    • Export Citation
  • 8

    Petek Šter M Švab I Kolšek M Kersnik J. Zaključek uvajanja projekta Svetovne banke uvajanja specializacije iz družinske medicine v Črni Gori – projekt Svetovne banke. Isis. 2013:22:48-9.

  • 9

    Heyrman J editor. The EURACT Educational Agenda. European Academy on Teachers in General Practice. EURACT Leuven 2005. Accessed September 25th 2017 at: http://euract.woncaeurope.org/sites/euractdev/files/documents/publications/official-documents/euract-educationalagenda.pdf.

  • 10

    Klemenc Ketis Z Svab I. Using movies in family medicine teaching: a reference to EURACT educational agenda. Zdr Varst. 2017;56:99-106. .

    • Crossref
    • Export Citation
  • 11

    Pravilnik o specializacijama („Sl. List CG“ 17/14 od 04.04.2014). Accessed September 25th 2017 at: http://www.sluzbenilist.me/PravniAktDetalji.aspx?tag=%7B1777FE38-9C46-42E9-9BC3-174A275D4208%7D.

  • 12

    O’Kelly Fergus M O’Kelly DarachÓC. A National Survey of GP Trainees 2012. TCD/HSE Higher Specialist Training Scheme in General Practice. Dublin: Trinity College Department of Public Health & Primary Care 2012.

  • 13

    Wall D Goodyear H Singh B Whitehouse A Hughes E Howes J. A new tool to evaluate postgraduate training posts: the Job Evaluation Survey Tool (JEST). BMC Med Educ. 2014:14:210. .

    • Crossref
    • Export Citation
  • 14

    Bulc M Svab I Pavlic DR Kolsek M. Specialist training of Slovene family physicians. Eur J Gen Pract. 2006;12:128-32. .

    • Crossref
    • Export Citation
  • 15

    Pelgrim EA Kramer AW Mokkink HG van der Vleuten CP. The process of feedback in workplace-based assessment: organisation delivery continuity. Med Educ. 2012;46(6):604-12. .

    • Crossref
    • Export Citation
  • 16

    Kreptul D Thomas RE. Family medicine resident OSCEs: a systematic review. Educ Prim Care. 2016;27(6):471-7. .

    • Crossref
    • Export Citation
  • 17

    Van der Vleuten CPM Driessen EW. What would happen to education if we take education evidence seriously? Perspect Med Educ. 2014;3:222-32. .

    • Crossref
    • Export Citation
  • 18

    Flum E Maagaard R Godycki-Cwirko M Scarborough N Scherpbier N Ledig T et al. Assessing family medicine trainees - what can we learn from the European neighbours? GMS Z Med Ausbild. 2015 13;32(2): :Doc21. .

    • Crossref
    • Export Citation
  • 19

    Govaerts M van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47(12): 1164-74. .

    • Crossref
    • Export Citation
  • 20

    de Jong J Visser M Van Dijk N van der Vleuten C Wieringa-de Waard M. A systematic review of the relationship between patient mix and learning in work-based clinical settings: a BEME systematic review: BEME Guide No. 24. Med Teach. 2013;35:e1181-96. .

    • Crossref
    • Export Citation
  • 21

    Greenhalgh T Howick J Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. .

    • Crossref
    • Export Citation
  • 22

    Premji K Upshur R Légaré F Pottie K. Future of family medicine: role of patient-centred care and evidence-based medicine. Can Fam Physician. 2014;60:409-12.

  • 23

    Ivetić V PaŠić K Selič P. The effect of an educational intervention in family phisicians on self-rated quality of life in patients with medically unexplained symptoms. Zdr Varst. 2017;56:91-8. .

    • Crossref
    • Export Citation
  • 24

    Ross D Schipper S Westbury C Linh Banh H Loeffler K Allan GM et al. Examining critical thinking skills in family medicine residents. Fam Med. 2016;48(2):121-6.

  • 25

    Al-Wassia H Al-Wassia R Shihata S Park YS Tekian A. Using patients’ charts to assess medical trainees in the workplace: a systematic review. Med Teach. 2015 ;37(Suppl 1):S82-7. .

    • Crossref
    • Export Citation
  • 26

    Hesketh EA Laidlaw JM. Developing the teaching instinct 5: mentoring. Med Teach. 2003;25:9-12. .

    • Crossref
    • Export Citation
  • 27

    Petek D Vidič Hudobivnik P Jančar V Petek B Klemenc-Ketiš Z. Regional coordinators: a new teaching opportunity in family medicine training. BMC Med Educ. 2016; 16:141. .

    • Crossref
    • Export Citation
  • 28

    Bulc M Svab I Radic S de Sousa JC Yaphe J. Faculty development for teachers of family medicine in Europe: reflections on 16 years’ experience with the international Bled course. Eur J Gen Pract. 2009;15:69-73. .

    • Crossref
    • Export Citation
  • 29

    Švab I Allen J Žebiene E Petek Šter M Windak A. Training experts in family medicine teaching. Eur J Gen Pract. 2016;22:58-63. .

    • Crossref
    • Export Citation
  • 30

    Klemenc-Ketis Z Svab I Petek-Ster M Bulc M Buchanan J Finnegan H et al. Twenty-five years of the international Bled course for teachers of family medicine in Europe: glancing back and looking forward. Eur J Gen Pract. 2016;22:262-6. .

    • Crossref
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • 1

    World Health Organization. Declaration of Alma-Ata: International Conference on Primary Health Care Alma-Ata USSR 6–12 September 1978. Geneva: WHO 1978. Accessed January 28th 2018 at: http://www.who.int/hpr/NPH/docs/declaration_almaata.pdf.

  • 2

    Starfield B Shi L Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83:457-502.

    • Crossref
    • Export Citation
  • 3

    Kringos D Boerma W Bourgueil Y Cartier T Dedeu T Hasvold T et al. The strength of primary care in Europe: an international comparative study. Br J Gen Pract. 2013; 63(616):e742-50. .

    • Crossref
    • Export Citation
  • 4

    Klančar D Svab I. Primary care principles and community health centres in the countries of former Yugoslavia. Health Policy. 2014;118:166-72.

    • Crossref
    • Export Citation
  • 5

    Saric M Rodwin V. The once and future health system in the former Yugoslavia: myths and realities. J Public Health Policy. 1993;14:220-37.

    • Crossref
    • Export Citation
  • 6

    Svab I Rotar Pavlic D Radic S Vainiomaki P. General practice east of Eden: an overview of general practice in Eastern Europe. Croat Med J. 2004;45:53-42.

  • 7

    Cvejanov Kezunović L Drecun M Švab I. Primary care reform in Montenegro. Zdr Varst. 2013;52:247-54. .

    • Crossref
    • Export Citation
  • 8

    Petek Šter M Švab I Kolšek M Kersnik J. Zaključek uvajanja projekta Svetovne banke uvajanja specializacije iz družinske medicine v Črni Gori – projekt Svetovne banke. Isis. 2013:22:48-9.

  • 9

    Heyrman J editor. The EURACT Educational Agenda. European Academy on Teachers in General Practice. EURACT Leuven 2005. Accessed September 25th 2017 at: http://euract.woncaeurope.org/sites/euractdev/files/documents/publications/official-documents/euract-educationalagenda.pdf.

  • 10

    Klemenc Ketis Z Svab I. Using movies in family medicine teaching: a reference to EURACT educational agenda. Zdr Varst. 2017;56:99-106. .

    • Crossref
    • Export Citation
  • 11

    Pravilnik o specializacijama („Sl. List CG“ 17/14 od 04.04.2014). Accessed September 25th 2017 at: http://www.sluzbenilist.me/PravniAktDetalji.aspx?tag=%7B1777FE38-9C46-42E9-9BC3-174A275D4208%7D.

  • 12

    O’Kelly Fergus M O’Kelly DarachÓC. A National Survey of GP Trainees 2012. TCD/HSE Higher Specialist Training Scheme in General Practice. Dublin: Trinity College Department of Public Health & Primary Care 2012.

  • 13

    Wall D Goodyear H Singh B Whitehouse A Hughes E Howes J. A new tool to evaluate postgraduate training posts: the Job Evaluation Survey Tool (JEST). BMC Med Educ. 2014:14:210. .

    • Crossref
    • Export Citation
  • 14

    Bulc M Svab I Pavlic DR Kolsek M. Specialist training of Slovene family physicians. Eur J Gen Pract. 2006;12:128-32. .

    • Crossref
    • Export Citation
  • 15

    Pelgrim EA Kramer AW Mokkink HG van der Vleuten CP. The process of feedback in workplace-based assessment: organisation delivery continuity. Med Educ. 2012;46(6):604-12. .

    • Crossref
    • Export Citation
  • 16

    Kreptul D Thomas RE. Family medicine resident OSCEs: a systematic review. Educ Prim Care. 2016;27(6):471-7. .

    • Crossref
    • Export Citation
  • 17

    Van der Vleuten CPM Driessen EW. What would happen to education if we take education evidence seriously? Perspect Med Educ. 2014;3:222-32. .

    • Crossref
    • Export Citation
  • 18

    Flum E Maagaard R Godycki-Cwirko M Scarborough N Scherpbier N Ledig T et al. Assessing family medicine trainees - what can we learn from the European neighbours? GMS Z Med Ausbild. 2015 13;32(2): :Doc21. .

    • Crossref
    • Export Citation
  • 19

    Govaerts M van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47(12): 1164-74. .

    • Crossref
    • Export Citation
  • 20

    de Jong J Visser M Van Dijk N van der Vleuten C Wieringa-de Waard M. A systematic review of the relationship between patient mix and learning in work-based clinical settings: a BEME systematic review: BEME Guide No. 24. Med Teach. 2013;35:e1181-96. .

    • Crossref
    • Export Citation
  • 21

    Greenhalgh T Howick J Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. .

    • Crossref
    • Export Citation
  • 22

    Premji K Upshur R Légaré F Pottie K. Future of family medicine: role of patient-centred care and evidence-based medicine. Can Fam Physician. 2014;60:409-12.

  • 23

    Ivetić V PaŠić K Selič P. The effect of an educational intervention in family phisicians on self-rated quality of life in patients with medically unexplained symptoms. Zdr Varst. 2017;56:91-8. .

    • Crossref
    • Export Citation
  • 24

    Ross D Schipper S Westbury C Linh Banh H Loeffler K Allan GM et al. Examining critical thinking skills in family medicine residents. Fam Med. 2016;48(2):121-6.

  • 25

    Al-Wassia H Al-Wassia R Shihata S Park YS Tekian A. Using patients’ charts to assess medical trainees in the workplace: a systematic review. Med Teach. 2015 ;37(Suppl 1):S82-7. .

    • Crossref
    • Export Citation
  • 26

    Hesketh EA Laidlaw JM. Developing the teaching instinct 5: mentoring. Med Teach. 2003;25:9-12. .

    • Crossref
    • Export Citation
  • 27

    Petek D Vidič Hudobivnik P Jančar V Petek B Klemenc-Ketiš Z. Regional coordinators: a new teaching opportunity in family medicine training. BMC Med Educ. 2016; 16:141. .

    • Crossref
    • Export Citation
  • 28

    Bulc M Svab I Radic S de Sousa JC Yaphe J. Faculty development for teachers of family medicine in Europe: reflections on 16 years’ experience with the international Bled course. Eur J Gen Pract. 2009;15:69-73. .

    • Crossref
    • Export Citation
  • 29

    Švab I Allen J Žebiene E Petek Šter M Windak A. Training experts in family medicine teaching. Eur J Gen Pract. 2016;22:58-63. .

    • Crossref
    • Export Citation
  • 30

    Klemenc-Ketis Z Svab I Petek-Ster M Bulc M Buchanan J Finnegan H et al. Twenty-five years of the international Bled course for teachers of family medicine in Europe: glancing back and looking forward. Eur J Gen Pract. 2016;22:262-6. .

    • Crossref
    • Export Citation
Search
Journal information
Impact Factor

IMPACT FACTOR 2018: 1.074
5-year IMPACT FACTOR: 0.728

CiteScore 2018: 0.94

SCImago Journal Rank (SJR) 2018: 0.564
Source Normalized Impact per Paper (SNIP) 2018: 0.478

Metrics
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 443 278 35
PDF Downloads 86 41 1