Background: To examine the artificial intelligence (AI) tools currently being studied in modern medical education, and critically evaluate the level of validation and the quality of evidence presented in each individual study.
Methods: This review (PROSPERO ID: CRD42023410752) was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. A database search was conducted using PubMed, Embase, and Cochrane Library. Articles written in the English language between 2000 and March 2023 were reviewed retrospectively using the MeSH Terms "AI" and "medical education" A total of 4642 potentially relevant studies were found.
Results: After a thorough screening process, 36 studies were included in the final analysis. These studies consisted of 26 quantitative studies and 10 studies investigated the development and validation of AI tools. When examining the results of studies in which Support vector machines (SVMs) were employed, it has demonstrated high accuracy in assessing students' experiences, diagnosing acute abdominal pain, classifying skilled and novice participants, and evaluating surgical training levels. Particularly in the comparison of surgical skill levels, it has achieved an accuracy rate of over 92%.
Conclusion: AI tools demonstrated effectiveness in improving practical skills, diagnosing diseases, and evaluating student performance. However, further research with rigorous validation is required to identify the most effective AI tools for medical education.
Keywords: artificial intelligence; deep learning; machine learning; medical education.