Research

Evaluation of assessment marks in the clinical years of an undergraduate medical training programme: Where are we and how can we improve?

H Brits, G Joubert, J Bezuidenhout, L van der Merwe

Abstract


Background. In high-stakes assessments, the accuracy and consistency of the decision to pass or fail a student is as important as the reliability of the assessment. Objective. To evaluate the reliability of results of high-stakes assessments in the clinical phase of the undergraduate medical programme at the University of the Free State, as a step to make recommendations for improving quality assessment. Methods. A cohort analytical study design was used. The final, end-of-block marks and the end-of-year assessment marks of both fourth-year and final-year medical students over 3 years were compared for decision reliability, test-retest reliability, stability and reproducibility. Results. 1 380 marks in 26 assessments were evaluated. The G-index of agreement for decision reliability ranged from 0.86 to 0.98. In 88.9% of assessments, the test-retest correlation coefficient was <0.7. Mean marks for end-of-block and end-of-year assessments were similar. However, the standard deviations of differences between end-of-block and end-of-year assessment marks were high. Multiple-choice questions (MCQs)  and objective structured clinical examinations (OSCEs) yielded good reliability results. Conclusion. The reliability of pass/fail outcome decisions was good. The test reliability, as well as stability and reproducibility of individual student marks, could not be accurately replicated. The use of MCQs and OSCEs are practical examples of where the number of assessments can be increased to improve reliability. In order to increase the number of assessments and to reduce the stress of high-stake assessments, more workplace-based assessment with observed clinical cases is recommended.

Authors' affiliations

H Brits, Department of Family Medicine, School of Clinical Medicine, University of the Free State, Bloemfontein, South Africa

G Joubert, Department of Biostatistics, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa

J Bezuidenhout, Department of Health Sciences Education, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa

L van der Merwe, Undergraduate Programme Management, School of Clinical Medicine, University of the Free State, Bloemfontein, South Africa

Full Text

PDF (198KB)

Cite this article

African Journal of Health Professions Education 2021;13(4):222. DOI:10.7196/AJHPE.2021.v13i4.137

Article History

Date submitted: 2022-01-28
Date published: 2022-01-28

Article Views

Abstract views: 1070
Full text views: 574

Comments on this article

*Read our policy for posting comments here