Abstract: Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Their main purpose is to restore several functionalities of motor disabled people, for example, to restore the communication ability. Recent BCIs based on visual evoked potentials (VEPs), which are brain responses to visual stimuli, have shown to achieve high-speed communication. However, BCIs have not really found their way out of the lab yet. This is mainly because all recent high-speed BCIs are based on synchronous control, which means commands can only be executed in time slots controlled by the BCI. Therefore, the user is not able to select a command at his own convenience, which poses a problem in real-world applications. Furthermore, all those BCIs are based on stimulation paradigms which restrict the number of possible commands. To be suitable for real-world applications, a BCI should be asynchronous, or also called self-paced, and must be able to identify the user’s intent to control the system or not. Although there some asynchronous BCI approaches, none of them achieved suitable real-world performances. In this thesis, the first asynchronous high-speed BCI is proposed, which allows using a virtually unlimited number of commands. Furthermore, it achieved a nearly perfect distinction between intentional control (IC) and non-control (NC), which means commands are only executed if the user intends to. This was achieved by a completely different approach, compared to recent methods. Instead of using a classifier trained on specific stimulation patterns, the presented approach is based on a general model that predicts arbitrary stimulation patterns. The approach was evaluated with a "traditional" as well as a deep machine learning method. The resultant asynchronous BCI outperforms recent methods by a multi-fold in multiple disciplines and is an essential step for moving BCI applications out of the lab and into real life. With further optimization, discussed in this thesis, it could evolve to the very first end-user suitable BCI, as it is effective (high accuracy), efficient (fast classifications), ease of use, and allows to perform as many different tasks as desired.

@phdthesis{SNT2019, author = {Nagel, Sebastian}, title = {Towards a home-use BCI: fast asynchronous control and robust non-control state detection}, school = {University of Tübingen}, year = {2019}, month = {dec}, doi = {10.15496/publikation-37739}, month_numeric = {12} }

Abstract: We present a novel approach based on deep learning for decoding sensory information from non-invasively recorded Electroencephalograms (EEG). It can either be used in a passive Brain-Computer Interface (BCI) to predict properties of a visual stimulus the person is viewing, or it can be used to actively control a BCI application. Both scenarios were tested, whereby an average information transfer rate (ITR) of 701 bit/min was achieved for the passive BCI approach with the best subject achieving an online ITR of 1237 bit/min. Further, it allowed the discrimination of 500,000 different visual stimuli based on only 2 seconds of EEG data with an accuracy of up to 100%. When using the method for an asynchronous self-paced BCI for spelling, an average utility rate of 175 bit/min was achieved, which corresponds to an average of 35 error-free letters per minute. As the presented method extracts more than three times more information than the previously fastest approach, we suggest that EEG signals carry more information than generally assumed. Finally, we observed a ceiling effect such that information content in the EEG exceeds that required for BCI control, and therefore we discuss if BCI research has reached a point where the performance of non-invasive visual BCI control cannot be substantially improved anymore.

@article{SM092019, author = {Nagel, Sebastian and Spüler, Martin}, title = {World’s fastest brain-computer interface: Combining EEG2Code with deep learning}, journal = {PLOS ONE}, year = {2019}, month = {sep}, volume = {14}, number = {9}, pages = {e0221909}, doi = {10.1371/journal.pone.0221909}, month_numeric = {9} }

Abstract: Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Recent BCIs based on visual evoked potentials (VEPs) have shown to be suitable for high-speed communication. However, all recent high-speed BCIs are synchronous, which means that the system works with fixed time slots so that the user is not able to select a command at his own convenience, which poses a problem in real-world applications. In this paper, we present the first asynchronous high-speed BCI with robust distinction between intentional control (IC) and non-control (NC), with a nearly perfect NC state detection of only 0.075 erroneous classifications per minute. The resulting asynchronous speller achieved an average information transfer rate (ITR) of 122.7 bit/min using a 32 target matrix-keyboard. Since the method is based on random stimulation patterns it allows to use an arbitrary number of targets for any application purpose, which was shown by using an 55 target German QWERTZ-keyboard layout which allowed the participants to write an average of 16.1 (up to 30.7) correct case-sensitive letters per minute. As the presented system is the first asynchronous high-speed BCI speller with a robust non-control state detection, it is an important step for moving BCI applications out of the lab and into real-life.

@article{SM062019, author = {Nagel, Sebastian and Spüler, Martin}, title = {Asynchronous non-invasive high-speed BCI speller with robust non-control state detection}, journal = {Scientific Reports}, year = {2019}, month = {jun}, volume = {9}, number = {1}, pages = {8269}, doi = {10.1038/s41598-019-44645-x}, month_numeric = {6} }

Abstract: Visual evoked potentials (VEPs) can be measured in the EEG as response to a visual stimulus. Commonly, VEPs are displayed by averaging multiple responses to a certain stimulus or a classifier is trained to identify the response to a certain stimulus. While the traditional approach is limited to a set of predefined stimulation patterns, we present a method that models the general process of VEP generation and thereby can be used to predict arbitrary visual stimulation patterns from EEG and predict how the brain responds to arbitrary stimulation patterns. We demonstrate how this method can be used to model single-flash VEPs, steady state VEPs (SSVEPs) or VEPs to complex stimulation patterns. It is further shown that this method can also be used for a high-speed BCI in an online scenario where it achieved an average information transfer rate (ITR) of 108.1 bit/min. Furthermore, in an offline analysis, we show the flexibility of the method allowing to modulate a virtually unlimited amount of targets with any desired trial duration resulting in a theoretically possible ITR of more than 470 bit/min.

@article{SM102018, author = {Nagel, Sebastian and Spüler, Martin}, title = {Modelling the brain response to arbitrary visual stimulation patterns for a flexible high-speed Brain-Computer Interface}, journal = {PLoS one}, year = {2018}, month = {oct}, volume = {13}, number = {10}, pages = {e0206107}, doi = {10.1371/journal.pone.0206107}, month_numeric = {10} }

@inproceedings{SWM052018, author = {Nagel, Sebastian and Rosenstiel, Wolfgang and Spüler, Martin}, title = {Finding optimal stimulation patterns for BCIs based on visual evoked potentials}, booktitle = {Proceedings of the 7th International BCI Meeting 2018}, year = {2018}, month = {may}, pages = {164-165}, address = {Asilomar, CA}, month_numeric = {5} }

Abstract: BACKGROUND: Visual neuroscience experiments and Brain–Computer Interface (BCI) control often require strict timings in a millisecond scale. As most experiments are performed using a personal computer (PC), the latencies that are introduced by the setup should be taken into account and be corrected. As a standard computer monitor uses a rastering to update each line of the image sequentially, this causes a monitor raster latency which depends on the position, on the monitor and the refresh rate. NEW METHOD: We technically measured the raster latencies of different monitors and present the effects on visual evoked potentials (VEPs) and event-related potentials (ERPs). Additionally we present a method for correcting the monitor raster latency and analyzed the performance difference of a code-modulated VEP BCI speller by correcting the latency. COMPARISON WITH EXISTING METHODS: There are currently no other methods validating the effects of monitor raster latency on VEPs and ERPs. RESULTS: The timings of VEPs and ERPs are directly affected by the raster latency. Furthermore, correcting the raster latency resulted in a significant reduction of the target prediction error from 7.98% to 4.61% and also in a more reliable classification of targets by significantly increasing the distance between the most probable and the second most probable target by 18.23%.

@article{SWWM012018, author = {Nagel, Sebastian and Dreher, Werner and Rosenstiel, Wolfgang and Spüler, Martin}, title = {The effect of monitor raster latency on VEPs, ERPs and Brain-Computer Interface performance}, journal = {Journal of Neuroscience Methods}, year = {2018}, month = {jan}, volume = {295}, pages = {45-50}, doi = {10.1016/j.jneumeth.2017.11.018}, month_numeric = {1} }

Abstract: Brain-Computer Interfaces (BCIs) enable users to control devices or communicate by using brain activity only. While BCIs based on visual evoked potentials (VEPs) have been shown to achieve high performance, we present a different paradigm for BCI control: random VEP (rVEP). We designed a regression model, trained on VEPs of fully random bit codes. Afterwards, the model is able to perform a bit-wise prediction of a previously unseen stimulation sequence, which in turn can be used for BCI control. In an offline study, the model predicts unknown stimulation sequences with an average ITR of 94.5 bits per minute (bpm) and up to 281 bpm on a single-trial level. In a copy-spelling task, the model achieved an average ITR of 64.3 bpm and up to 115.5 bpm.

@inproceedings{SWM092017, author = {Nagel, Sebastian and Rosenstiel, Wolfgang and Spüler, Martin}, title = {Random Visual Evoked Pontentials (RVEP) for Brain-Computer Interface (BCI) Control}, booktitle = {Proceedings of the 7th Graz Brain-Computer Interface Conference 2017}, publisher = {Verlag der TU Graz}, year = {2017}, month = {sep}, pages = {349-354}, doi = {10.3217/978-3-85125-533-1-64}, month_numeric = {9} }

Abstract: In this paper, we present a spiking neuronal model that learns to perform a motor control task. Since the long-term goal of this project is the application of such a neuronal model to study the mutual adaptation between a Brain-Computer Interface (BCI) and its user, neurobiological plausibility of the model is a key aspect. Therefore, the model was trained using reinforcement learning similar to that of the dopamine system, in which a global reward and punishment signal controlled spike-timing dependent plasticity (STDP). Based on this method, the majority of the randomly generated models were able to learn the motor control task. Although the models were only trained on two targets, they were able to reach arbitrary targets after learning. By introducing structural synaptic plasticity (SSP), which dynamically restructures the connections between neurons, the number of models that successfully learned the task could be significantly improved.

@inproceedings{MSW072015, author = {Spüler, Martin and Nagel, Sebastian and Rosenstiel, Wolfgang}, title = {A Spiking Neuronal Model Learning a Motor Control Task by Reinforcement Learning and Structural Synaptic Plasticity}, booktitle = {Proceedings of the 2015 Internation Joint Conference on Neuronal Networks}, year = {2015}, month = {jul}, pages = {1652-1659}, doi = {10.1109/IJCNN.2015.7280521}, month_numeric = {7} }