GLUE is a training, evaluation, and analyzing benchmark platform for natural language understanding services. Developers of the technology can test their models on GLUE to gain better understanding of performance. Microsoft’s MT-DNN-SMART has been the standout natural language processing system. If has topped GLUE, pushing Google’s T3 system into second place. However, there’s a new winner in town as Baidu now tops the standings. The company’s ERNIE (Enhanced Representation through kNowledge IntEgration) system managed a benchmark score of 90.1. In a blog, Baidu promised to continue development to further prefect natural language understanding: “While language understanding remains a difficult challenge, our results on GLUE indicate that pre-training language models with continual training and multi-task learning are a promising direction for NLP research. We will keep improving the performance of the ERNIE model via the continual pre-training framework,” wrote Baidu Research team.
Development
ERNIE has already been successful in real-world scenarios. Baidu uses it in its search engine for answering spoken searches. The company claims the model has driven a 16% increase in customer satisfaction in terms of search results. “ERNIE is a continual pre-training framework that builds and learns incrementally by pre-training tasks through sequential multi-task learning. We introduced ERNIE 1.0 early this year and released the improved ERNIE 2.0 model in July. The latter outperformed Google’s BERT and Carnegie Mellon University’s XLNet – competing pre-training models – in 16 NLP tasks in both Chinese and English.”
