The impact of speech type on listening effort and intelligibility for native and non-native listeners

Olympia Simantiraki; Anita E Wagner; Martin Cooke

doi:10.3389/fnins.2023.1235911

The impact of speech type on listening effort and intelligibility for native and non-native listeners

Front Neurosci. 2023 Sep 28:17:1235911. doi: 10.3389/fnins.2023.1235911. eCollection 2023.

Authors

Olympia Simantiraki¹, Anita E Wagner², Martin Cooke³

Affiliations

¹ Institute of Applied and Computational Mathematics, Foundation for Research & Technology-Hellas, Heraklion, Greece.
² Department of Otorhinolaryngology/Head and Neck Surgery, University Medical Center Groningen, University of Groningen, Groningen, Netherlands.
³ Ikerbasque (Basque Science Foundation), Vitoria-Gasteiz, Spain.

Abstract

Listeners are routinely exposed to many different types of speech, including artificially-enhanced and synthetic speech, styles which deviate to a greater or lesser extent from naturally-spoken exemplars. While the impact of differing speech types on intelligibility is well-studied, it is less clear how such types affect cognitive processing demands, and in particular whether those speech forms with the greatest intelligibility in noise have a commensurately lower listening effort. The current study measured intelligibility, self-reported listening effort, and a pupillometry-based measure of cognitive load for four distinct types of speech: (i) plain i.e. natural unmodified speech; (ii) Lombard speech, a naturally-enhanced form which occurs when speaking in the presence of noise; (iii) artificially-enhanced speech which involves spectral shaping and dynamic range compression; and (iv) speech synthesized from text. In the first experiment a cohort of 26 native listeners responded to the four speech types in three levels of speech-shaped noise. In a second experiment, 31 non-native listeners underwent the same procedure at more favorable signal-to-noise ratios, chosen since second language listening in noise has a more detrimental effect on intelligibility than listening in a first language. For both native and non-native listeners, artificially-enhanced speech was the most intelligible and led to the lowest subjective effort ratings, while the reverse was true for synthetic speech. However, pupil data suggested that Lombard speech elicited the lowest processing demands overall. These outcomes indicate that the relationship between intelligibility and cognitive processing demands is not a simple inverse, but is mediated by speech type. The findings of the current study motivate the search for speech modification algorithms that are optimized for both intelligibility and listening effort.

Keywords: cognitive load; growth curve analysis; listening effort; non-native listeners; pupillometry; speech perception.

Grants and funding

We acknowledge support from the European Commission under the Marie Curie European Training Network ENRICH (675324) for enabling the initial experiments and data analysis, and funding from the Hellenic Foundation for Research and Innovation (HFRI) through the Second Call for HFRI Research Projects to support Faculty Members and Researchers under Project 4753 for supporting further data analysis and the preparation of the manuscript.