TY - GEN
T1 - Artificial Intelligence (AI) Literacy Questionnaire with Confirmatory Factor Analysis
AU - Kit Ng, Davy Tsz
AU - Wu, Wenjie
AU - Lok Leung, Jac Ka
AU - Wah Chu, Samuel Kai
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - In recent years, schools started to incorporate artificial intelligence (AI) into computer science/STEAM curricula. However, few validated measurements have been designed to examine how secondary students develop AI literacy and perceive their learning outcomes. AI literacy has been measured from students' knowledge and skill acquisition, and behavior and attitudes. This research aims to develop and validate an instrument to assess AI literacy for secondary students. A questionnaire with 25 items measured in a 5-point Likert scale was created. In a pilot study, the questionnaire was administered to a sample of 363 secondary school students from two different schools in Hong Kong. Confirmatory factor analysis was conducted and grouped into six factors: (1) intrinsic motivation, (2) self-efficacy, (3) behavioral intention, (4) behavioral engagement, (5) know and understand, and (6) use and apply AI. The questionnaire showed a good fit model to support internal consistency reliability in most of the factors, with Cronbach's Alpha levels ranging from.58 to.88. A less-parsimonious model was proposed that help educators measure a wider AI literacy skill set with an acceptable model fit, with Cronbach's Alpha levels ranging from.91 to.94 based on affective, behavorial, cognitive and ethical (ABCE) learning framework. Further studies are needed to confirm the factor structure.
AB - In recent years, schools started to incorporate artificial intelligence (AI) into computer science/STEAM curricula. However, few validated measurements have been designed to examine how secondary students develop AI literacy and perceive their learning outcomes. AI literacy has been measured from students' knowledge and skill acquisition, and behavior and attitudes. This research aims to develop and validate an instrument to assess AI literacy for secondary students. A questionnaire with 25 items measured in a 5-point Likert scale was created. In a pilot study, the questionnaire was administered to a sample of 363 secondary school students from two different schools in Hong Kong. Confirmatory factor analysis was conducted and grouped into six factors: (1) intrinsic motivation, (2) self-efficacy, (3) behavioral intention, (4) behavioral engagement, (5) know and understand, and (6) use and apply AI. The questionnaire showed a good fit model to support internal consistency reliability in most of the factors, with Cronbach's Alpha levels ranging from.58 to.88. A less-parsimonious model was proposed that help educators measure a wider AI literacy skill set with an acceptable model fit, with Cronbach's Alpha levels ranging from.91 to.94 based on affective, behavorial, cognitive and ethical (ABCE) learning framework. Further studies are needed to confirm the factor structure.
KW - AI
KW - AI literacy
KW - machine learning
KW - psychometric measurement
UR - http://www.scopus.com/inward/record.url?scp=85170045172&partnerID=8YFLogxK
U2 - 10.1109/ICALT58122.2023.00074
DO - 10.1109/ICALT58122.2023.00074
M3 - Conference contribution
AN - SCOPUS:85170045172
T3 - Proceedings - 2023 IEEE International Conference on Advanced Learning Technologies, ICALT 2023
SP - 233
EP - 235
BT - Proceedings - 2023 IEEE International Conference on Advanced Learning Technologies, ICALT 2023
A2 - Chang, Maiga
A2 - Chen, Nian-Shing
A2 - Kuo, Rita
A2 - Rudolph, George
A2 - Sampson, Demetrios G
A2 - Tlili, Ahmed
T2 - 23rd IEEE International Conference on Advanced Learning Technologies, ICALT 2023
Y2 - 10 July 2023 through 13 July 2023
ER -