Brain-Computer Interface for home-use application
There are several diseases, such as amyotrophic lateral sclerosis (ALS), which can lead to a loss of the ability to communicate. As a healthy person one cannot imagine what it feels like to be trapped in one’s own body - mentally present, but unable to communicate with relatives, this is called locked-in syndrome. However, a distinction must be made between locked-in and complete locked-in syndrome (CLIS). For the former, those affected can still voluntarily control certain muscles, above all the eye muscles, which in turn can be used for communication. Brain-computer interfaces (BCIs), i.e. systems that allow to control a computer by pure brain activity, have proven to be a helpful method for restoring the ability to communicate. However, all recent BCI systems are almost exclusively used in research, since all previous methods are not suitable for real-world applications. This is mainly due to the fact that for a meaningful and independent use, the recognition of the user’s intention (to control the system or not) must be highly accurate. Otherwise, this leads to random classifications/commands, which can be dangerous depending on the application, for example when controlling an electric wheelchair.
We have tackled this issue and developed a self-paced BCI based on complex visual stimulation patterns (see publications below). It was tested in the context of a spelling application, for this a virtual keyboard is shown on a computer monitor. While the user gazes a specific target (letter) which is modulated with its own unique blinking (black-white) stimulation pattern, the underlying model predicts the stimulation pattern based on the recorded Electroencephalogram (EEG). The predicted stimulation pattern is compared to all possible stimulation patterns using a similarity measure to classify the correct letter. This approach allows to spell up to 49 error-free letters per minute, whereby a target classification accuracy of almost constant 100% was achieved. In average, the method is 3 times faster than the previously fastest method. In addition, the intention to control the system was always recognized for all subjects. The intention not to control the system could always be detected for 80% of all subjects. Compared to the previously best method, the recognition rate is improved by an average factor of 6.5. Furthermore, the method allows an almost infinite number of choices to be distinguished, whereby this was shown for up to 500,000 choices by a simulation. Even with this enormous classification problem, accuracies of up to 100% were achieved. These results could only be achieved by a deep learning approach, which illustrates the importance of machine learning in this research area.
However, there are still three key points for further optimization required to make the BCI suitable for home-use, (1) subject specific adaptivity, (2) elimination of unintended commands, and (3) distinct stimulation patterns. These points should be addressed in the scope of this project:
- (1) Subject specific adaptivity is necessary for a BCI to react to changing conditions, like the fact that brain activities are non-stationary over time, even within the same user. Without an adaptive learning approach, the performance would drop over time. Due to the very high classification accuracy, the underlying model could be trained adaptively in a so-called semi-supervised procedure. This means that the brain signal that accumulates over time are continuously used for training by labeling it with the predicted classes.
- (2) Elimination of unintended commands: As mentioned, the state when the user does not intent to control the BCI, called non-control state, is as important as the control state or even more important for the reasons given. For this, a user specific threshold is required, which has so far been more or less arbitrarily determined. Although the results outperform recent methods by far, the error-rate must be reduced to 0 in order to ensure safe use. To achieve this, the threshold value should be determined using a machine learning approach which also adapts over time. In the scope of this project, several approaches should be evaluated.
- (3) Distinct stimulation patterns: The BCI is based on complex (but random) visual stimulation patterns. In a recent work [2] it was shown that the stimulation patterns can be optimized to improve the performance. In the scope of this work, an automatic machine learning approach should be developed which automatically “finds” a set of optimal stimulation patterns. Whereby the resultant set of stimulation pattern should be continuously improved.
By solving these points, a robust, accurate and user friendly BCI for communication in daily live application can be achieved. In a later phase of the project, the resultant BCI shall be evaluated with at least one potential end-user, like an ALS patient, over a period of 6 months to ensure a robust and reliable use.
Participating Team Members
Publications
2019
Towards a home-use BCI: fast asynchronous control and robust non-control state detection
by Sebastian NagelPhD thesis. University of Tübingen, 2019. [BIB] [DOI] [ABSTRACT]
@phdthesis{SNT2019,
author = {Nagel, Sebastian},
title = {Towards a home-use BCI: fast asynchronous control and robust non-control state detection},
school = {University of Tübingen},
year = {2019},
month = {dec},
doi = {10.15496/publikation-37739},
month_numeric = {12}
}
World’s fastest brain-computer interface: Combining EEG2Code with deep learning
by Sebastian Nagel and Martin SpülerIn PLOS ONE 14(9): e0221909, 2019. [BIB] [DOI] [ABSTRACT]
@article{SM092019,
author = {Nagel, Sebastian and Spüler, Martin},
title = {World’s fastest brain-computer interface: Combining EEG2Code with deep learning},
journal = {PLOS ONE},
year = {2019},
month = {sep},
volume = {14},
number = {9},
pages = {e0221909},
doi = {10.1371/journal.pone.0221909},
month_numeric = {9}
}
Asynchronous non-invasive high-speed BCI speller with robust non-control state detection
by Sebastian Nagel and Martin SpülerIn Scientific Reports 9(1): 8269, 2019. [BIB] [DOI] [ABSTRACT]
@article{SM062019,
author = {Nagel, Sebastian and Spüler, Martin},
title = {Asynchronous non-invasive high-speed BCI speller with robust non-control state detection},
journal = {Scientific Reports},
year = {2019},
month = {jun},
volume = {9},
number = {1},
pages = {8269},
doi = {10.1038/s41598-019-44645-x},
month_numeric = {6}
}
2018
Modelling the brain response to arbitrary visual stimulation patterns for a flexible high-speed Brain-Computer Interface
by Sebastian Nagel and Martin SpülerIn PLoS one 13(10): e0206107, 2018. [BIB] [DOI] [ABSTRACT]
@article{SM102018,
author = {Nagel, Sebastian and Spüler, Martin},
title = {Modelling the brain response to arbitrary visual stimulation patterns for a flexible high-speed Brain-Computer Interface},
journal = {PLoS one},
year = {2018},
month = {oct},
volume = {13},
number = {10},
pages = {e0206107},
doi = {10.1371/journal.pone.0206107},
month_numeric = {10}
}
Finding optimal stimulation patterns for BCIs based on visual evoked potentials
by Sebastian Nagel, Wolfgang Rosenstiel, and Martin SpülerIn Proceedings of the 7th International BCI Meeting 2018, pages 164-165, 2018. [BIB] [PDF]
@inproceedings{SWM052018,
author = {Nagel, Sebastian and Rosenstiel, Wolfgang and Spüler, Martin},
title = {Finding optimal stimulation patterns for BCIs based on visual evoked potentials},
booktitle = {Proceedings of the 7th International BCI Meeting 2018},
year = {2018},
month = {may},
pages = {164-165},
address = {Asilomar, CA},
month_numeric = {5}
}
The effect of monitor raster latency on VEPs, ERPs and Brain-Computer Interface performance
by Sebastian Nagel, Werner Dreher, Wolfgang Rosenstiel, and Martin SpülerIn Journal of Neuroscience Methods 295: 45-50, 2018. [BIB] [DOI] [ABSTRACT]
@article{SWWM012018,
author = {Nagel, Sebastian and Dreher, Werner and Rosenstiel, Wolfgang and Spüler, Martin},
title = {The effect of monitor raster latency on VEPs, ERPs and Brain-Computer Interface performance},
journal = {Journal of Neuroscience Methods},
year = {2018},
month = {jan},
volume = {295},
pages = {45-50},
doi = {10.1016/j.jneumeth.2017.11.018},
month_numeric = {1}
}
2017
Random Visual Evoked Pontentials (RVEP) for Brain-Computer Interface (BCI) Control
by Sebastian Nagel, Wolfgang Rosenstiel, and Martin SpülerIn Proceedings of the 7th Graz Brain-Computer Interface Conference 2017, pages 349-354. Verlag der TU Graz, 2017. [BIB] [DOI] [ABSTRACT]
@inproceedings{SWM092017,
author = {Nagel, Sebastian and Rosenstiel, Wolfgang and Spüler, Martin},
title = {Random Visual Evoked Pontentials (RVEP) for Brain-Computer Interface (BCI) Control},
booktitle = {Proceedings of the 7th Graz Brain-Computer Interface Conference 2017},
publisher = {Verlag der TU Graz},
year = {2017},
month = {sep},
pages = {349-354},
doi = {10.3217/978-3-85125-533-1-64},
month_numeric = {9}
}