Validating Plagiarism Detection Systems with Metamorphic Testing

Context: Plagiarism is a severe issue in the academic world, and plagiarism detection systems are commonly used to find plagiarism violations. Although advanced technologies have improved plagiarism detection systems, there is a lack of focus on assessing their effectiveness and performance. Objective: Plagiarism detection systems give a local optimum rather than a global optimum, which leads to a test oracle problem. Therefore, a feasible and novel approach to validate those systems is required. Method: We propose Metamorphic Testing (MT) to validate plagiarism detection systems. Metamorphic relations (MRs) validate the relations between inputs and outputs, which are vital components of MT. Based on the concept of Metamorphic relations patterns (MRP), three MRs are proposed to generate follow-up test inputs while characteristics of the source test outputs are preserved. If the proposed MRs are violated, faults in the system will be detected. Results: The experimental results show that the proposed MRs can validate four plagiarism detection systems. Conclusion: The result shows that MT can systematically and effectively validate plagiarism detection systems. Subsequent studies can further extend MT applications in validating other plagiarism detection systems by applying proposed MRs or exploring other MRs related to the semantics of source text corpus.