TY - JOUR
T1 - OdeBERT
T2 - One-stage Deep-supervised Early-exiting BERT for Fast Inference in User Intent Classification
AU - Liu, Yuanxia
AU - Hao, Tianyong
AU - Liu, Hai
AU - Mu, Yuanyuan
AU - Weng, Heng
AU - Wang, Fu Lee
N1 - Publisher Copyright:
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
PY - 2023/5/9
Y1 - 2023/5/9
N2 - User intent classification is a vital task for analyzing users' essential requirements from the users' input query in information retrieval systems, question answering systems, and dialogue systems. Pre-Trained language model Bidirectional Encoder Representation from Transformers (BERT) has been widely applied to the user intent classification task. However, BERT is compute intensive and time-consuming during inference and usually causes latency in real-Time applications. To improve the inference efficiency of BERT for the user intent classification task, this article proposes a new network named one-stage deep-supervised early-exiting BERT as one-stage deep-supervised early-exiting BERT (OdeBERT). In addition, a deep supervision strategy is developed to incorporate the network with internal classifiers by one-stage joint training to improve the learning process of classifiers by extracting discriminative category features. Experiments are conducted on publicly available datasets, including ECDT, SNIPS, and FDQuestion. The results show that the OdeBERT can speed up original BERT 12 times faster at most with the same performance, outperforming state-of-The-Art baseline methods.
AB - User intent classification is a vital task for analyzing users' essential requirements from the users' input query in information retrieval systems, question answering systems, and dialogue systems. Pre-Trained language model Bidirectional Encoder Representation from Transformers (BERT) has been widely applied to the user intent classification task. However, BERT is compute intensive and time-consuming during inference and usually causes latency in real-Time applications. To improve the inference efficiency of BERT for the user intent classification task, this article proposes a new network named one-stage deep-supervised early-exiting BERT as one-stage deep-supervised early-exiting BERT (OdeBERT). In addition, a deep supervision strategy is developed to incorporate the network with internal classifiers by one-stage joint training to improve the learning process of classifiers by extracting discriminative category features. Experiments are conducted on publicly available datasets, including ECDT, SNIPS, and FDQuestion. The results show that the OdeBERT can speed up original BERT 12 times faster at most with the same performance, outperforming state-of-The-Art baseline methods.
KW - BERT
KW - OdeBERT
KW - deep supervision
KW - inference
KW - user intent classification
UR - http://www.scopus.com/inward/record.url?scp=85162138772&partnerID=8YFLogxK
U2 - 10.1145/3587464
DO - 10.1145/3587464
M3 - Article
AN - SCOPUS:85162138772
SN - 2375-4699
VL - 22
JO - ACM Transactions on Asian and Low-Resource Language Information Processing
JF - ACM Transactions on Asian and Low-Resource Language Information Processing
IS - 5
M1 - 129
ER -