Noise-Robust Keyword Spotting Through Self-supervised Pretraining

Voice assistants are now widely available, and to activate them a keywordspotting (KWS) algorithm is used. Modern KWS systems are mainly trained usingsupervised learning methods and require a large amount of labelled data toachieve a good performance. Leveraging unlabelled data through self-supervisedlearning (SSL) has been shown to increase the accuracy in clean conditions.This paper explores how SSL pretraining such as Data2Vec can be used to enhancethe robustness of KWS models in noisy conditions, which is under-explored. Models of three different sizes are pretrained using different pretrainingapproaches and then fine-tuned for KWS. These models are then tested andcompared to models trained using two baseline supervised learning methods, onebeing standard training using clean data and the other one being multi-styletraining (MTR). The results show that pretraining and fine-tuning on clean datais superior to supervised learning on clean data across all testing conditions,and superior to supervised MTR for testing conditions of SNR above 5 dB. Thisindicates that pretraining alone can increase the model's robustness. Finally,it is found that using noisy data for pretraining models, especially with theData2Vec-denoising approach, significantly enhances the robustness of KWSmodels in noisy conditions.