Brain-Computer Interface for home-use application

There are several diseases, such as amyotrophic lateral sclerosis (ALS), which can lead to a loss of the ability to communicate. As a healthy person one cannot imagine what it feels like to be trapped in one’s own body - mentally present, but unable to communicate with relatives, this is called locked-in syndrome. However, a distinction must be made between locked-in and complete locked-in syndrome (CLIS). For the former, those affected can still voluntarily control certain muscles, above all the eye muscles, which in turn can be used for communication. Brain-computer interfaces (BCIs), i.e. systems that allow to control a computer by pure brain activity, have proven to be a helpful method for restoring the ability to communicate. However, all recent BCI systems are almost exclusively used in research, since all previous methods are not suitable for real-world applications. This is mainly due to the fact that for a meaningful and independent use, the recognition of the user’s intention (to control the system or not) must be highly accurate. Otherwise, this leads to random classifications/commands, which can be dangerous depending on the application, for example when controlling an electric wheelchair.

We have tackled this issue and developed a self-paced BCI based on complex visual stimulation patterns (see publications below). It was tested in the context of a spelling application, for this a virtual keyboard is shown on a computer monitor. While the user gazes a specific target (letter) which is modulated with its own unique blinking (black-white) stimulation pattern, the underlying model predicts the stimulation pattern based on the recorded Electroencephalogram (EEG). The predicted stimulation pattern is compared to all possible stimulation patterns using a similarity measure to classify the correct letter. This approach allows to spell up to 49 error-free letters per minute, whereby a target classification accuracy of almost constant 100% was achieved. In average, the method is 3 times faster than the previously fastest method. In addition, the intention to control the system was always recognized for all subjects. The intention not to control the system could always be detected for 80% of all subjects. Compared to the previously best method, the recognition rate is improved by an average factor of 6.5. Furthermore, the method allows an almost infinite number of choices to be distinguished, whereby this was shown for up to 500,000 choices by a simulation. Even with this enormous classification problem, accuracies of up to 100% were achieved. These results could only be achieved by a deep learning approach, which illustrates the importance of machine learning in this research area.

However, there are still three key points for further optimization required to make the BCI suitable for home-use, (1) subject specific adaptivity, (2) elimination of unintended commands, and (3) distinct stimulation patterns. These points should be addressed in the scope of this project:

  • (1) Subject specific adaptivity is necessary for a BCI to react to changing conditions, like the fact that brain activities are non-stationary over time, even within the same user. Without an adaptive learning approach, the performance would drop over time. Due to the very high classification accuracy, the underlying model could be trained adaptively in a so-called semi-supervised procedure. This means that the brain signal that accumulates over time are continuously used for training by labeling it with the predicted classes.
  • (2) Elimination of unintended commands: As mentioned, the state when the user does not intent to control the BCI, called non-control state, is as important as the control state or even more important for the reasons given. For this, a user specific threshold is required, which has so far been more or less arbitrarily determined. Although the results outperform recent methods by far, the error-rate must be reduced to 0 in order to ensure safe use. To achieve this, the threshold value should be determined using a machine learning approach which also adapts over time. In the scope of this project, several approaches should be evaluated.
  • (3) Distinct stimulation patterns: The BCI is based on complex (but random) visual stimulation patterns. In a recent work [2] it was shown that the stimulation patterns can be optimized to improve the performance. In the scope of this work, an automatic machine learning approach should be developed which automatically “finds” a set of optimal stimulation patterns. Whereby the resultant set of stimulation pattern should be continuously improved.

By solving these points, a robust, accurate and user friendly BCI for communication in daily live application can be achieved. In a later phase of the project, the resultant BCI shall be evaluated with at least one potential end-user, like an ALS patient, over a period of 6 months to ensure a robust and reliable use.

Participating Team Members

Nagel, Sebastian

Blöck, Alexander

Rosenstiel, Wolfgang

Publications

Thesis Topics