Experimental Study on an AI Chatbot for Learning Support
Keywords:
Artificial Intelligence (AI), Chatbot, Instruction, Algorithms and Programming, Educational ExperimentationAbstract
Artificial intelligence (AI) technology is bringing innovation to the education sector, significantly influencing both teaching and learning processes. This study aimed to evaluate the effectiveness of an AI-based chatbot in supporting student learning in the course "Introduction to Algorithms and Programming," involving 60 students from the Software Engineering program at the Mongolian National University of Education. The experimental design divided participants into an experimental group and a control group to analyze academic performance outcomes. The experimental group used an AI chatbot that employed Socratic dialogue to support students in understanding course content and completing tasks, while the control group followed traditional instructional methods. To assess the impact of the intervention, a t-test analysis was conducted on pre-test and post-test data. The results indicated a statistically significant improvement (p < 0.05) in the experimental group, demonstrating the educational value of AI chatbot integration.
References
Badamsuren, B., & Munkhtuya, L. (2019). Сургалтын цахим агуулга хөгжүүлэх арга зүйн асуудалд. Lavai - Олон улсын боловсролын сэтгүүл, 22, 77–83.
Battsetseg, S., Tungalagtuya, Kh., Munkhtuya, L., (2024). A Solution to Challenges Faced by Teachers in
Distance Learning to Improve the Quality and Accessibility of Education. Lavai - Олон улсын боловсролын сэтгүүл, 31, 35-46. https://doi.org/10.5564/lavai.v20i31.3874
Chang, D. H., Lin, M. P. C., Hajian, S., & Wang, Q. Q. (2023). Educational design principles of using AI chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization. Sustainability, 15(17), 12921. https://doi.org/10.3390/su151712921
Chen, X., Xie, H., & Hwang, G. J. (2020). A multi-perspective study on artificial intelligence in education: Grants, conferences, journals, software tools, institutions, and researchers. Computers & Education: Artificial Intelligence, 1, 100005. https://doi.org/10.1016/j.caeai.2020.100005
Danzan, T., Sambuu, U., Chuluunbat, M., Avkhinsukh, T., & Sandag, E. (2025). ICT readiness of secondary school teachers: Mongolian case study. Contemporary Educational Technology, 17(3), ep586. https://doi.org/10.30935/cedtech/16531
Deng, X., & Yu, Z. (2023). A meta-analysis and systematic review of the effect of chatbot technology use in sustainable education. Sustainability, 15(4), 2940. https://doi.org/10.3390/su15042940
Goel, A., & Polepeddi, L. (2018). Jill Watson’s Terrific Twos. Georgia Tech News. https://mlatgt.blog
Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching & learning. Center for Curriculum Redesign. https://curriculumredesign.org/
Kim, J., & Lim, K. (2022). Artificial intelligence tutors in programming education: Effects on learning performance and self-efficacy. Journal of Educational Computing Research, 60(1), 57–78. https://doi.org/10.1177/0735633121995662
Lkhagvasuren, M., Chuluunbat, M., Tsendsuren, N., & Batsuuri, B. (2024). Analysis of e-learning materials in pre-service teacher training: In case of the Mongolian National University of Education (MNUE). In Proceedings of the Conference on Quality Assurance in Higher Education: Transforming education—new generation of learners (QAHE 2023) (pp. 80–88). Atlantis Press. https://doi.org/10.2991/978-94-6463-382-5_12
Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed: An argument for AI in education. Pearson. https://www.pearson.com
Монгол Улсын Их Хурал. (2023). Боловсролын тухай хууль (шинэчилсэн найруулга). Монгол Улсын хууль тогтоомжийн мэдээллийн сан. https://legalinfo.mn/
Ng, D. T. K., Tan, C. W., & Leung, J. K. L. (2024). Empowering student self-regulated learning and science education through ChatGPT: A pioneering pilot study. British Journal of Educational Technology, 55(4), 1328–1353. https://doi.org/10.1111/bjet.13341
Popenici, S. A. D., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1), 22. https://doi.org/10.1186/s41039-017-0062-8
Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26(2), 582–599. https://doi.org/10.1007/s40593-016-0110-3
Wu, R., & Yu, Z. (2024). Do AI chatbots improve students’ learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13309
Zimmerman, B. J., & Moylan, A. (2009). Self-regulated learning: Where we are and where we need to go.
In D. H. Schunk & B. J. Zimmerman (Eds.), Handbook of self-regulation of learning and performance (pp. 1–19). Routledge.
