Christopher See

How does a humanoid AI compare to human teachers in bioethics teaching? A Quasi-Experimental Study

Christopher See

The Chinese University of Hong Kong

christophersee@cuhk.edu.hk

Charlotte Tien Lan Lin

Bernice Ya Rui Fong

Background: Artificial intelligence (AI) agents are increasingly being deployed in medical education, but their effectiveness compared to traditional human-led instruction remains underexplored, particularly in more discursive subjects such as bioethics. This study examines the learning outcomes and beliefs of medical students learning from our human-sized interactive AI avatars as class facilitators versus human-led bioethics instruction.

Methods: A post-test-only, quasi-experimental non-inferiority study was conducted with 122 medical students, divided into two groups of 61. One group received AI-led classroom teaching and the other received human-led instruction on the bioethics implications of AI in healthcare. We developed AI Avatars specifically for bioethics teaching, using generative AI for voice, face and with Large Language Model (LLM) Integration. They were deployed as 2 metre tall human-like touchscreens, each leading a small groups of students in 1 hour tutorials. Learning outcomes were assessed using a standardized 10-item multiple-choice knowledge test. Secondary outcomes on acceptability of AI-led classes and learner satisfaction were evaluated with a 10-item survey, combining Likert-scale and open-ended questions. Statistical analysis via independent t-test compared learning outcomes for both groups to evaluate non-inferiority with predefined margin of 5%.

Results: AI-led instruction was found to be non-inferior to human instruction in learning outcomes (mean difference: -0.8, 95% CI: -2.9 to +1.6, p < 0.05). Students in both groups demonstrated comparable performance in knowledge and application of bioethics principles to questions. Survey findings revealed that 68% of students believed AI could effectively teach complex topics, felt more free to discuss in peer discussions without a human teacher present and highlighted the bilingual nature of the AI as an advantage. However 48% expressed a preference for human-led instruction, citing better interpersonal interaction and lesser perceived ability of AI to adapt to individual student needs and potential biases.

Conclusions: AI-led bioethics instruction achieves learning outcomes comparable to human-led approaches, supporting its potential use in medical education. However, given students’ mixed beliefs on purely AI-led classes, a role as an in-class teaching assistant or co-teacher may enable the benefits of human and AI educators to be combined in bioethics education.

BIOGRAPHY

Dr. Christopher See is a medical educator, receiving his medical degree from the University of Cambridge (Trinity College), a PGCE from the University of Edinburgh and Ph.D. from HKU in Medical Education. Currently a lecturer in anatomy at the School of Biomedical Sciences, he has received teaching excellence awards from both the University of Manchester and CUHK. His educational research centres on Artificial Intelligence (AI) in medical education, receiving over 5.6 million HKD of grants and multiple publications in this area. He leads both the AI for Education Community of Practice in CUHK and a Hong Kong-wide 1.6 million HKD TDLEG initiative “AI for Education” with 6 partner institutions. He has taught modules on AI in Healthcare, introduction to Machine Learning, Neural Networks and Deep Learning in several medical schools’ curricula.

Ms. Charlotte Lin is a 2nd year medical student at the Chinese University of Hong Kong, member of the 'Med Ed Lab Hong Kong' and spent her gap year as a medical education research assistant. Primarily working on AI chatbot and Avatar projects, she has received the Gold Award for Educational Technology Innovation at CUHK and had her educational work covered in the South China Morning Post Newspaper.

Ms. Bernice Fong is a first year medical student at St. Andrew's University, Scotland and previously full time researcher in medical education. Her research interests have ranged from gamification of learning anatomy via escape game development and educational AI development via Avatar systems for anatomy, Obstetrics and Gynaecology and most recently bioethics. She has received an award for Pedagogical Innovation in 2023 for her contributions to gamification of learning.