Open Access

A systematic review of assessments for procedural skills in physiotherapy education / Assessment von prozeduralen Fähigkeiten in der physiotherapeutischen Ausbildung: Ein systematischer Review

Martin Sattelmayer
Martin Sattelmayer
, 
Roger Hilfiker
Roger Hilfiker
 and 
Gillian Baer
Gillian Baer
   | Jun 30, 2017
International Journal of Health Professions's Cover Image
About this article
Previous Article
Next Article
Cite
Published Online: Jun 30, 2017
Page range: 53 - 65
Received: Aug 07, 2016
Accepted: Feb 27, 2017
DOI: https://doi.org/10.1515/ijhp-2017-0008
Keywords
© 2017 Martin Sattelmayer, Roger Hilfiker, Gillian Baer, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Figure 1

Study flow.
Study flow.

Figure 2

Internal consistency estimates.Nb. The statistical method from Boulet et al. (2004) was not available.
Internal consistency estimates.Nb. The statistical method from Boulet et al. (2004) was not available.

Figure 3

Interrater reliability estimates.
Interrater reliability estimates.

In-and exclusion criteria

CategoryCriteria
PopulationStudies with physiotherapists or physiotherapy students were included.
Studies with health professionals or health professions students were included when they practiced procedures that can be used in physiotherapy (i.e., when medical students were evaluated on their ability to perform a musculoskeletal examination)
Studies with health professionals or health professions students were excluded when they practiced procedures that cannot be practiced by physiotherapists (such as surgery)
Educational assessmentsThe assessment could be either a procedure specific measurement instrument (i.e., the assessment is designed exclusively for one procedure) or a procedure unspecific measurement (i.e., the assessment is designed to measure procedures in physiotherapy education but can be used for more than one procedure)
The assessment should measure procedures in reality. Assessments based on virtual reality were excluded.
The assessment should be feasible in various settings. Therefore, assessments that require expensive equipment were excluded.
Data must be available for a specific assessment. Studies with summary data of several assessments were excluded (e.g., summary scores of a complete OSCE).
OutcomeThe aim of assessment should be to measure procedural skills. Assessments of similar constructs such as clinical skills or psychomotor skills [defined as ‘... motor skill, some manipulation of material, or some act which requires a neuromuscular action’ Simpson (1966, p. 17)] were included.
Assessments that aimed to exclusively evaluate other outcomes such as communication skills or professionalism were excluded.
When assessments were designed to measure multiple outcomes, it was evaluated whether the focus was based on procedural skills (e.g., more than 50% of the items concentrate on procedural skills). The assessments with focus on procedural skills were included.
Measurement propertiesStudies had to report the measurement properties of an educational assessment (e.g., reliability or validity)

Search strategy

PopulationAssessmentOutcome
medical education OR education, medical[Mesh] OR physiotherapy education OR physical therapy education OR health professions education OR healthcare education OR allied health care educationscale OR global rating scale OR GRS OR checklistpractical skill* OR psychomotor skill* OR procedural skill* OR clinical skill*

Characteristics of included studies and assessments.

StudyCountrySettingSampleAssessed procedureScale and itemsDurationPatientsAssessorsPurpose
Beran et al. (2012) AMPEUSAOrthopaedic department24 orthopaedic residentsPSMI: Musculoskeletal physical examination; Inspection, palpation, joint range of motion, strength testing and any special tests pertinent to the clinical scenarioFour 12-15 items checklists for clinical scenarios (upper extremity, lower extremity, trauma and spine) on dichotomous scales (yes or no).10 MinutesStandardised patients are required (120 minutes training)Pool of experienced ratersHigh stakes purpose
Boulet et al.(2004) OMTUSAOsteopathic college121 osteopathic students (4th year)PUMI: Osteopathic manipulative treatment of three clinical cases (low back pain, frozen shoulder and asthmatic with cough)OMT (Osteopathic Manipulative Treatment) assessment tool with 15 items; Every item is scored on a 0 to 2 scale (0 = done incorrectly or not done, 1 = not performed optimally and 2 = done proficiently)13 minutesStandardised patients with 8 hours of formal training16 osteopathic physicians (5 hours of formal training)High stakes examination (OSCE)
Herbers et al. (2003) PES-CUSAUniversity Medical Centre72 internal medicine residentsPSMI: Pelvic examination29 item dichotomous checklist (yes = when the behaviour was observed; no = when the behaviour was not observed); Includes some items about communication skillsNot specifiedGynaecologic teaching trainer required; 1 trainer was being examined and the second trainer rated the student's skills.Gynaecologic teaching trainer requiredNot specified
Herbers et al. (2003) PES-RUSAUniversity Medical Centre72 internal medicine residentsPUMI: Pelvic examinationGlobal rating scale evaluating the overall performance of the pelvic examination (five-point ordinal scale between 1 = inadequate and 5 = excellent)Not specifiedGynaecologic teaching trainer required; 1 trainer was being examined and the second trainer rated the student's skillsGynaecologic teaching trainer requiredNot specified
Ladyshewsky et al. (2000) PhyESAustraliaPhysiotherapy department12 undergraduate physiotherapy students 4 physiotherapists (at least 2 years of experience)PSMI: Musculoskeletal physical examination of a patient with a rotator cuff problemPhysical examination checklist (3-point scale: 0 = not done, 1 = done poorly or incompletely and 2 = done well), number of items not availableMean 30 minutes (range: 20 - 46 minutes)Standardised patients are requiredAssessors with 30 hours of trainingHigh stakes examination (OSCE)
Nothnagle et al. (2010) GPSEUSAFamily medicine department5 faculty members and 5 students (semi structured interviews); Focus groups: 7 experienced family medicine educators, 5 residents and 5 faculty membersPUMI: Eligible for all procedures in family medicineGlobal Procedural Skills Evaluation Form, 4-point scale, amount of assistance is documented ranging from significant guidance is provided to performed independently; communication skills etc. are included; Student's self-assessment is included; Difficulty of the procedures is rated as wellNot specifiedNot requiredNot specifiedLow stakes examination (formative feedback)
Swift et al. (2013)

It was only possible to use data from a small pilot study. The follow up study evaluated a 6 station OSCE. Single values for a specific scale were not available

 mO-S3		USAPhysiotherapy department12 undergraduate 1st year physiotherapy studentsPSMI: Examination skills in musculoskeletal physiotherapy (shoulder tests)Checklist for a musculoskeletal OSCE station; 6 items checklist (5 dichotomous items and 1 ordinal item)6 minutesSimulation patients with 2 hours of supervised training and 1 week of independent trainingClinical instructors (2 - 20 years of experience)Low stakes examination (mid-term)
Yudkowsky et al. (2004) HTTPEUSAUniversity Medical Centre369 medical students (2nd year)PSMI: Head to toe physical examination138 item checklist; three-point scale (0 = incorrect, 1 = correct after prompt, 2 = correct without prompting); Test duration: 2 h; high stakes summative assessment or low stakes formative assessment2 hours (45 minutes unprompted exam, remaining 1:45 hours are used for scoring, feedback, and teaching)Trained patient instructors with 25 hours of trainingTrained patient instructors with 25 hours of trainingHigh stakes summative assessment and low stakes formative assessment
eISSN:
2296-990X
Languages:
English, German
Publication timeframe:
Volume Open
Journal Subjects:
Medicine, Clinical Medicine, other
Journal RSS Feed