Automated cleft speech evaluation using speech recognition

Perceptual evaluation remains the gold-standard evaluation of cleft speech, but with any human interpretation, there can be bias. Eliminating bias, allowing comparison of speech data between units, is labor and time intensive. Globally, there is a shortage of listeners. We have developed a computer learning system to evaluate cleft speech. Our automated cleft speech evaluator interprets resonance and articulatory cleft speech errors. Speech recognition engines typically ignore voice characteristics and speech errors of the speaker, but in cleft speech evaluation, these features are paramount. Our evaluator targets these to distinguish between normal speech, velopharyngeal dysfunction and articulatory speech errors. Speech samples from our Craniofacial Team clinic were recorded and rated independently by two experienced speech pathologists: 60 patients were used to train the evaluator, and the evaluator was tested on the 13 subsequent patients. The inter-speech pathologist agreement rate was 79%. Our cleft speech evaluator achieved 77% on its best sentence and a median of 65% for all sentences. This automated cleft speech evaluator has applications for global cleft speech evaluation when no speech pathologist is available, and for unbiased evaluation, facilitating collaboration between teams. We anticipate that as the training samples increase, the accuracy will match human listeners.