The MuMoSD Corpus

   
  Contents:  
   
   
   
 

The "MuMoSD" (MultiModal Swallowing Detection) is a corpus that contains various signals from volunteers performing different tasks, including swallowing. Swallowing issues can lead to choking, coughing, nasal regurgitation, pain, malnutrition, and more. This is why developing techniques for analyzing swallowing is crucial for early symptom detection.

MuMoSD includes tasks such as swallowing different liquids, coughing, and speech, recorded using various sensors:

  • Throat mic
  • Condenser mic
  • Magnetic sensors
  • Camera

A recording protocol with specific instructions was designed to ensure systematic data collection. The recordings were made using the KiRAT software. More details about the recordings can be found in the Recording section.

 

Key Features

The corpus comprises 20 volunteers, consisting of 16 males and 4 females. It contains over 1000 segments, which include activities such as swallowing saliva, water, yogurt, coughing, throat clearing, text reading, sustaining the vowel 'A' for three seconds or more, and saying the phrase "Guten morgen, wie geht es Ihnen?" twice.

 

Recordings

The following segments describes the recording protocol. The volunteer is placed in front of a camera that is pointing to their throat, first the throat mic is placed around the top part of their neck. A total of 8 tasks are performed:

  1. Swallow saliva three times: during this task the volunteer is asked to swallow saliva three times in the same recording.
  2. Swallow water three times: during this task the volunteer is given 3 cups that contains 10 ml of water poured with a syringe, the volunteer is asked to swallow the content of the cups.
  3. Swallow yogurt three times: during this task the volunteer is asked to swallow a spoon that contains 10 ml of yogurt. This is done 3 times.
  4. Coughing ten times: Volunteers are asked to cough ten times in the same recording.
  5. Throat clearing ten times: Volunteers are instructed to clear their throats ten times in the same recording.
  6. Text reading: Volunteers read a text that includes speech portions and instructions, such as swallowing or coughing during the text.
  7. Sustaining the vowel 'A': Volunteers are asked to sustain the vowel 'A' for three seconds or more.
  8. Phrase: Volunteers are requested to say a German phrase twice.

All eight tasks are repeated for each combination of sensors: first with only the throat mic, then with the throat mic and magnetic sensors, and finally with only the magnetic sensors. This is done to eliminate sensor interference during measurements.

After the recordings, all the audios are listened to give labels to different segments of the audios depending on the task, this label as well as the start and finish time of the segment are stored in a text file in the same folder as the recordings. The different labels that can be found are:

  • S: Swallowing saliva
  • SW: Swallowing Water
  • SY: Swallowing Yogurt
  • C: Coughing
  • T: Throat clearing

Notice that we can use all five classes and perform the detection of liquids swallowed or we can take just the first letter of the label, this will leave us with just three different classes and we will be able to detect whether the volunteer was swallowing or not.

 

Download

The corpus can be used for non-commercial research purposes. Details can be found here.

 

Examples (More examples)

Text reading with throat mic.

 

 

Creators of the Data Base

The data base was created in a joint work between Kiel University (CAU) and the University of Antioquia (UdeA). Involved researchers are:

  • Nestor Calvo (UdeA)
  • Tobias Schmidt (CAU)
  • Juan Rafael Orozco Arroyave (UdeA)
  • Gerhard Schmidt (CAU)

 

Corresponding Publications

No publications yet