Standard Presentation (25-minute) Automated Transcription and Measures of Lexical Diversity in L2 Spoken Texts more

Sun, Sep 17, 14:15-14:40 Asia/Tokyo

While there has been an increase in the use of transformer-based Automated Speech Recognition (ASR) models like OpenAI’s Whisper for transcribing L1 speech (e.g. Lin, 2023; Seyedi et al., 2022) using these models to transcribe speech produced by L2 English learners can be difficult due to factors such as pronunciation errors, disfluencies, and atypical grammatical constructions (Wang et al., 2021). The presenters’ previous research (Brooks & Jordan, 2023) showed that these issues can result in word error rates (WER) that are higher than those of a professional transcriber. This presentation builds on this previous research and examines the viability of using ASR programs to transcribe texts being used to investigate lexical diversity. This is done by investigating the differences in the lexical diversity scores of the automated version of 100 texts (50 presentations and 50 discussions) compared to the professionally transcribed versions of the same texts. Results show that the accuracy of ASR transcriptions comes close to professional transcriptions and that the areas where ASR is likely to make mistakes tend not to impact measures of lexical diversity. The presenters will highlight areas where ASR-based transcribers struggle and best practices for using models such as Whisper for transcribing L2 texts.