Automatic Evaluation of In-Car Communication Systems

   
  Contents:  
   
   
   
   
   
 

In-car communication systems can improve the communication within a passenger compartment. To accomplish this, the speech signal of the talking passenger is recorded by the already installed hands-free microphones and played back via the loudspeakers located close to the listener. Further information about ICC systems can be found here.

After the implementation of an ICC system two main questions appear:

  • What does improve the communication for the passengers?
  • What does impair the communication for the passengers?

Due to the fact that at least two persons are involved in a conversation these two questions have to be answered for the listening and the talking passenger within in the passenger compartment.In addition, quality is a rather subjective concept, which depends on a variety of factors.

A first, intuitive approach to evaluate ICC systems would be to form a group of test subjects which have to evaluate the system in terms of speech-quality and speech-intelligibility. This evaluation can be perfomed by diagnostic, modified rhyme or comparison mean opinion score tests, so-called subjective test methods. These tests are quite expensive due to their duration and the large number of necessary subjects. To reduce costs and to produce more reliable and of course reproducible test results, objective tests are highly desirable.

Due to the fact, that the term quality is a complex concept, a sophisticated ICC system evaluation strategy is required. By our group a first concept consisting of three evaluation groups of different evaluation methods has been published (see literature hint below). Within each of these groups different measures should be utilized and combined to an overall quality result. First instrumental evaluation methods, such as the ICC system delay estimation, ICC system support, the signal-to-noise ratio (SNR) difference or spectral distance, have been published. All these mentioned evaluation methods do have in common, that they utilize a black-box testing, which allows the evaluation of an arbitrary ICC system.

 

Corresponding Publications

     

T. Hübschen, M. Gimm, G. Schmidt: A Background Noise and Impulse Response Corpus for Research in Automotive Speech and Audio Processing, Proc. DAGA, Germany, 2022

   

A. Namenas, G. Schmidt: Acoustic Ambiance Simulation Using Orthogonal Loudspeaker Signals, Proceedings of the 14th ITG Conference on Speech Communication, September 2021

   

T. Hübschen, R. Al-Mafrachi, G. Schmidt: Impact of a Speaker Head Rotation on the Far-end Listening Situation, Proceedings of the 14th ITG Conference on Speech Communication, September 2021

   

A. Theiß, A. Namenas, T. Hübschen G. Schmidt: Spectral Distance Analysis for Quality Estimation of In-Car Communication Systems,  in H.Abut, J. H. L. Hansen, G. Schmidt and K. Takeda (eds.), Intelligent Vehicles and Transportation, de Gruyter, 2020.

   

G. Schmidt, and A. Theiß: Automatic Evaluation of In-Car Communication Systems, Proc. DAGA, Kiel, Germany, open access, 2017

   

R. Landgraf, G. Schmidt, J. Köhler-Kaeß, O. Niebuhr, and T. John: More Noise, Less Talk - The Impact of Driving Noise and ICC Systems on Acoustic-prosodic Parameters in Dialogue, Proc. DAGA, Kiel, Germany, open access, 2017

   

A. Theiß, G. Schmidt: Spectral Distance Analysis for Quality Estimation of In-Car Communication Systems, 7th Biennial Workshop on DSP for In-Vehicle Systems and Safety 2015, Berkeley, CA, USA

   

R. Landgraf, O. Niebuhr, G. Schmidt, T. John, C. Lüke, A. Theiß: Von der Straße ins Labor: Die Modifikation der Sprachproduktion bei lauten Fahrgeräuschen, DAGA 2015, Nürnberg, Germany

   

S. Graf, A. Theiß, T. Herbig, G. Schmidt: Listening Test to Determine the Mismatch Between Signal-To-Noise Ratio and Human Perception, DAGA 2015, Nürnberg, Germany

   

C. Lüke, A. Wolf, M. Brodersen, G. Schmidt: Digitale Simulation der Fahrzeuginnenraumakustik zur Unterstützung der Entwicklung und Evaluierung von Innenraum-Kommunikationssystemen, DAGA 2015, Nürnberg, Germany

   

T. John, R. Landgraf, C. Lüke, S. Rohde, G. Schmidt, A. Theiß, J. Withopf: Über die Verbesserung der Sprachkommunikation in geräuschbehafteten Umgebungen, in O. Niebuhr (ed.): Formen des Nichtverstehens, Peter Lang, December 2014 (in German)

   

A. Theiß, G. Schmidt, J. Withopf, C. Lüke: Instrumental Evaluation of In-Car Communication Systems, Proc. ITG Fachtagung Sprachkommunikation 2014, Erlangen, Germany

   

A. Theiß, G. Schmidt: Investigation of Self-Masking Effects for the Evaluation of In-Car Communication Systems, Proc. International Workshop on Acoustic Signal Enhancement IWAENC 2014, Antibes, France

   

C. Lüke, G. Schmidt, A. Theiß, J. Withopf: In-Car Communication, in G. Schmidt, H. Abut, K. Takeda, J. Hansen (eds.), Smart Mobile In-Vehicle Systems, Springer, January 2014

   

T. John, O. Niebuhr, G. Schmidt and A. Theiß: Phonetic Analyses vs. Dirty Signals: Fixing the Paradox, Proc. ESSV, Bielefeld , Germany, 2013

   

C. Lüke, A. Theiß, G. Schmidt, O. Niebuhr, T. John: Creation of a Lombard Speech Database using an Acoustic Ambiance Simulation with Loudspeakers, 6th Biennial DSP Workshop for In-Vehicle Systems 2013, Seoul, Korea

   

G. Schmidt, A. Theiß, J. Withopf, A. Wolf: Evaluation of In-Car Communication Systems, in J. Hansen, P. Boyraz, K. Takeda, H. Abut (eds.), Digital Signal Processing for In-Vehicle Systems and Safety, Springer, January 2012