Automatic generation of matching rules for programming exercise assessment

Automatic programming exercise assessment aims at determining the correctness of the attempts of programming exercises submitted by students. Automation allows students to receive instant and customized feedback which are important to enhance the learning of novice students. Educators can benefit from saving time and effort in marking students’ attempts, making teaching large, online classes or Massive Open Online Courses (MOOC) possible and effective. Recently, we modelled program outputs using Hierarchical Program Output Structure (HiPOS), which allows instructors to design matching rules to determine correct or partially correct programs depending on the teaching and learning needs. This paper extends our previous work by automating the matching rule construction process through developing a machine learning method for generalizing program outputs from students’ attempts. To achieve this, our approach firstly employs natural language processing techniques to create a HiPOS from a set of students’ program outputs. A greedy algorithm is then applied to generalize the HiPOS and create the associated matching rules. We conducted a case study to illustrate how to apply our proposed method in automated programming exercise assessment and demonstrated the usefulness and effectiveness of our approach.