Interactive Website for Parkinson’s Speech Therapy

Overview

A speech therapy to take home or on the go – that’s what four electrical engineering students are aiming at to support the therapy of Parkinson’s patients.

Parkinson’s disease is a degenerative disorder affecting more than 4 million people worldwide. In addition to the well-known symptoms such as trembling and difficulty of movement, the affected also struggle with articulation. Despite the patients’ existing cognitive abilities, they tend to mumble and speak too quietly. Unfortunately the therapy in this field is not a priority.

Therefore this project is about creating a possibility for patients to control the volume of their voice. Regarding this we cooperate with local speech therapists and the clinic for neurology. The goal is to set up a Website that patients can access at any time. We use microphones to determine the volume and provide visual feedback.

A preliminary website of the project can be found here.

Motivation

The motivation for us in realizing this project is to help facilitate everyday life for Parkinson patients specifically and patients with similar disease patterns as well. Especially in a field where the therapy is not prioritized even though it plays a vital part in a patient’s life. That’s why we hope to provide a website for speech therapy that is easy to use and accessible anytime, anywhere and for free.

Status Quo

At the beginning the members of our team were introduced to GitLab and Visual Studio Code where we would implement our code. We then divided the setup of the website into front end and back end design.

The goal of the back end programming was the design of a suitable signal processing for the speech level analysis. This was introduced and elucidated to us by Professor Schmidt with a MATLAB code example. It included an A-weighting filter to pre-process the microphone input signal, a short term power estimation, a speech-level estimation and a voice activity detection comparing the speech level to the background noise.

We started implementing this using WebAssembly first but ended up switching to JavaScript later on for better compatibility.

A challenge we came across was the linkage of front end and back end. We had to determine which data to provide and how to provide it to get an accurate representation of a user’s volume.

The front end design’s main goal is usability. This includes a clear menu navigation which we hope to achieve by having the tabs for the homepage, the about page and the speech trainer visible in the upper left corner. Another feature we included was a timer that counts down the time of your session once you start it.

The representation of the verbal input was one of the biggest challenges in the front end design. We decided to display the volume of a person’s speech visually using a candle flame. The flame is visualized with the help of Bezier curves. On one hand side if a user’s speech level is too low the flame begins to die out. On the other hand side if it is too high the candle starts to flicker thus giving the user the visual feedback on their volume.

One thing we noticed while testing the speech trainer was the difference in people’s speaking volume, background noises and microphone sensibility which affected the response of the candle flame. As a solution we set up a slider to individually regulate your microphone sensibility.

Our plans for the future are to include texts to read during your sessions and to set up a database to save user profiles with corresponding settings and training results.

Participating Students

Patrick Dietrich
Jonas Rädisch
Carlotta Schwertfeger
Sophie Thiele

Supervisors

Erik Engelhardt
Johannes Hoffmann
Patricia Piepjohn
Gerhard Schmidt

We are delighted to announce that Marten Finck has been awarded the GaS prize for his Master's thesis on "Evaluation of Speech Quality Using Machine Learning".

In his work, he investigated possible side effects of magnetic resonance-guided focussed ultrasound treatment (MRgFUS) on speech quality in patients with neurological diseases such as Parkinson's disease and essential tremor. This innovative and non-invasive treatment method is used to reduce tremor, i.e. the shaking of limbs.

To evaluate the effects on speech, pre- and post-operative speech recordings of 56 patients were analysed: Speech quality was assessed in five categories (monotony, clarity, fluency, auditory effort and vocal tremor) through a survey with 21 participants and used to create a convolutional neural network (CNN). This generates an objective assessment of speech quality using mel frequency cepstral coefficients (MFCCs), fundamental frequency and loudness. A mean absolute error (MAE) of 0.81 on a scale of 0 to 10 was achieved, compared to a standard deviation of 0.67 for the survey participants compared to a reference assessment. The evaluation of the survey revealed minor positive and negative changes in speech quality, which were balanced overall and tended to indicate natural speech variations in connection with the patients' underlying illnesses.

Congratulations on this well-deserved honour!