OdeBERT: One-stage Deep-supervised Early-exiting BERT for Fast Inference in User Intent Classification

Yuanxia Liu, Tianyong Hao, Hai Liu, Yuanyuan Mu, Heng Weng, Fu Lee Wang

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)

Abstract

User intent classification is a vital task for analyzing users' essential requirements from the users' input query in information retrieval systems, question answering systems, and dialogue systems. Pre-Trained language model Bidirectional Encoder Representation from Transformers (BERT) has been widely applied to the user intent classification task. However, BERT is compute intensive and time-consuming during inference and usually causes latency in real-Time applications. To improve the inference efficiency of BERT for the user intent classification task, this article proposes a new network named one-stage deep-supervised early-exiting BERT as one-stage deep-supervised early-exiting BERT (OdeBERT). In addition, a deep supervision strategy is developed to incorporate the network with internal classifiers by one-stage joint training to improve the learning process of classifiers by extracting discriminative category features. Experiments are conducted on publicly available datasets, including ECDT, SNIPS, and FDQuestion. The results show that the OdeBERT can speed up original BERT 12 times faster at most with the same performance, outperforming state-of-The-Art baseline methods.

Original languageEnglish
Article number129
JournalACM Transactions on Asian and Low-Resource Language Information Processing
Volume22
Issue number5
DOIs
Publication statusPublished - 9 May 2023

Keywords

  • BERT
  • OdeBERT
  • deep supervision
  • inference
  • user intent classification

Fingerprint

Dive into the research topics of 'OdeBERT: One-stage Deep-supervised Early-exiting BERT for Fast Inference in User Intent Classification'. Together they form a unique fingerprint.

Cite this