TY - GEN
T1 - A Review of Large Language Model Evaluation Metrics in Education
AU - Paoprasert, Naraphorn
AU - Chui, Kwok Tai
AU - Yip, Tik Yung
AU - Liu, Jiaqi
AU - Ng, Kwan Keung
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2026.
PY - 2026
Y1 - 2026
N2 - The versatility of large language models (LLMs) leads to an increasing penetration rate and market share of LLMs. Many new LLMs have launched, challenging the selection and evaluation of an appropriate LLM for target applications. In this research, attention is drawn to education as one of the most remarkable areas. Evaluating LLMs using different evaluation metrics is both subjective and objective. Using the advanced search tool in Scopus, 231 relevant studies have been shortlisted for the review of LLM evaluation metrics in education. Some basic analyses, including the number of publications by year, a frequency table of the subject areas, distribution of document types, distribution of source titles, a word cloud summary of the top 50 keywords, and distribution of regions, were provided. It was followed by an in-depth discussion of the top 10 highly cited publications. Then, five challenges in LLM evaluation were analyzed.
AB - The versatility of large language models (LLMs) leads to an increasing penetration rate and market share of LLMs. Many new LLMs have launched, challenging the selection and evaluation of an appropriate LLM for target applications. In this research, attention is drawn to education as one of the most remarkable areas. Evaluating LLMs using different evaluation metrics is both subjective and objective. Using the advanced search tool in Scopus, 231 relevant studies have been shortlisted for the review of LLM evaluation metrics in education. Some basic analyses, including the number of publications by year, a frequency table of the subject areas, distribution of document types, distribution of source titles, a word cloud summary of the top 50 keywords, and distribution of regions, were provided. It was followed by an in-depth discussion of the top 10 highly cited publications. Then, five challenges in LLM evaluation were analyzed.
KW - educational research
KW - educational technology
KW - evaluation metrics
KW - generative artificial intelligence
KW - large language model
UR - https://www.scopus.com/pages/publications/105023586003
U2 - 10.1007/978-981-95-4499-8_33
DO - 10.1007/978-981-95-4499-8_33
M3 - Conference contribution
AN - SCOPUS:105023586003
SN - 9789819544981
T3 - Communications in Computer and Information Science
SP - 402
EP - 415
BT - Technology in Education. Smart and Innovative Learning - International Conference on Technology in Education, ICTE 2025, Proceedings
A2 - Cheung, Simon K. S.
A2 - Liu, Xiaojun
A2 - Xu, Guoai
A2 - Kwok, Lam-For
T2 - 8th International Conference on Technology in Education, ICTE 2025
Y2 - 10 December 2025 through 12 December 2025
ER -