This paper presents our progress in development of a brain computer interface. We used existing system which was extended to support synchronous experiments with feedback. We use the simplest possible methods as we focus on feedback influence. We examined influence of the way how the feedback is visualized. The abstract feedback, the feedback based on computer game, and the feedback showing real photographs were tested. The classification score 88.5% was achieved with one subject even with simple classification method.
Keywords: BCI, Real-time processing, feedback
Our group has been dealing with a research in the
field of movement-related EEG recognition towards
developing a Brain-Computer Interface (BCI). We
have shown that off-line single trial classification of
extension and flexion movements of right index finger
is possible; classification score of up to 95 % was
achieved . The logical next step was to move
towards a real-time processing; therefore we have been
designing a modular real time processing system ,
 and .
In the paper  we examined how a BCI system is
influenced by the presence of feedback. We decided to
advance to the online classification of left and right
hand movement imagery with visual feedback. This
contribution examines the influence of the way how the
feedback is presented to the user.
First movement related BCIs
We started with a research on methods used in the
oldest movement related BCIs. The first movement
related BCIs used synchronous experimental protocol
and movements on the opposite side of the body.
Synchronous setup means that the user is provided with
instruction when and which mental activity he has to
perform, in opposite to asynchronous when user
performs activities in his own pace.
Works ,,, and  use the following
experimental protocol: at first, the fixation cross is
presented to let the subject concentrate, then there is an
acoustic beep prior to the instruction of movement to
imagine (arrow). Then, the feedback in a form of
extending bar is presented, and finally a randomized
resting interval is used to avoid adaptation of the
subject. Based on these findings we decided to adopt
experimental protocol as shown in Figure 1.
Electrode placement was adopted from work . We
recorded 3 differential channels connected to the
following electrodes (positive - negative): FC3 - CP3,
FCZ - CPZ and FC4 - CP4. These channels are simply
called C3, C4, and CZ in further text. Linked ears were
used as reference.
Basic design of our distributed modular EEG
Processing Pipeline (EPP) was presented in works 
and . The system is composed of independent
standalone modules connected by a network interface,
see Figure 2.
The modules are independent on used operation
system and hardware as implemented in the Java
The communication protocol is based on Real-time
Transport Protocol (RTP) to achieve real-time data
transfer over the network. Intended purposes of RTP
are real-time audio and video streaming. These have
very similar requirements as EEG data transfer. And
RTP packets are usually prioritized in network devices
such as routers.
The latency is most critical parameter for real-time
applications. The latency of the pipeline was tested in
 and was found sufficient for the applications with
The communication protocol enables to transfer
optional parameters defined by the common
configuration system. Logging system can store all the
communication including all states, messages and
Following modules were implemented:
BIOPAC Bridge module provides support for a EEG device manufactured by the BIOPAC Company which is available in our department. All the experiments presented in this paper were conducted using this device with a sampling rate of 200 Hz.
Control module serves for real-time control of the running system, including switching states of the system and sending command to other modules.
Visualization module serves for visual representation of classification to the experimental subject.
Perceptron module implements a real-time classification algorithm – a simple one layer Perceptron.
Feature Extraction module implements a general FIR filtration and a short time power estimation by a leaking integrator filter.
Detection module computes the power in the same way as the Feature Extraction module and compares the value with a threshold.
The Data Flow Monitoring module was extended to be capable of showing given experiment setting and all the inter-module communication.
Following modules were implemented in order to
support synchronous experiments:
The Trigger module serves for time synchronization. The module generates sequence of states from the predefined experimental protocol, see Figure 1.
Feedback module provides instructions and feedback for the experimental subject. Current state of the feedback is picked up from the data flow. The feedback can be displayed as extending horizontal bar, moving image or animation.
Arkanoid present feedback in the style based on the popular computer game Arkanoid.
General purpose Generator module, which loads data from log file of any other module and simulates its behavior.
Threshold module was implemented for the purposes of online classification experiments with asymmetric ratio . The ratio A is defined as
where R is the feature extracted from a
channel recorder over right hemisphere and L
over left hemisphere.
We found out the ratio is usually biased.
The bias is subject dependent and differs with
each experiment even with same subject. It is
caused by electrode placement, contact quality
or by asymmetry of brain signal itself. We
decided to make normalization of features
before computing the ratio. We use balance
constant b and we multiply one feature with
1 + b and the other with 1 − b
The constant can be set by the configuration
file or it can be automatically computed. For automatic balancing we presume long term
averages of the normalized features should be
We found out that the modular architecture concept
of the system as published in  was the right choice
because we could easily extend it for new types of
Synchronous experiments followed the experimental
protocol as shown in Figure 1. Modules were
connected as shown in Figure 3. The first synchronous
experiments were with simulated feedback and the
classification was performed offline. The purpose of
these experiments was to find simple classification
algorithm. Movement imagery of only right hand was
used at first as we want to investigate the movement
related activity. Stronger movement related activity
compared to asynchronous experiment was detected
which is in compliance with work . Then two
classification algorithms which used signal filtered by
an 8-40 Hz band pass filter were tested.
The first algorithm used the Trigger module to
compare power from left and right hemisphere. The
second algorithm used Support Vector Machine (SVM)
and Hjorth features (Activity, Mobility and
Complexity). The achieved scores (ratio of correctly
classified trials and all trials) were 73 % for power
comparison and 75 % for SVM. As the achieved score
for SVM was not significantly better we decided to use
the Trigger module for feedback comparison
The results with simulated feedback were promising
therefore we have attempted to use real feedback based
on an online classification. Threshold classification
module was activated during the movement period thus
the classification results were presented directly to the
subject. Subjects reported that it was hard to
concentrate on the movement imagery while watching
the feedback as the classification results were quite frequently changing during the single movement
imagery. The subjects felt more like they are
responding to the presented feedback rather than that
the system is responding to their imagery. This
degraded the signal as offline classification scores have
fallen to around 64 %.
We examined the influence of the feedback behavior
. The same setting as in previous experiments was
used. The Threshold module was used for
classification. The feedback was operating in two
modes. One was realistic when the feedback bar was
extending according to the classification result. The
second one was so called optimistic it was expanding
when the result of classification was the same as the
command and it was still when the result was different.
The optimistic feedback was found easier to control
and achieved score was better .
The next experiment examined the feedback
visualization itself. We were using three different
feedback types (Fig. 4).
We were using optimistic feedback as it achieved
best results in previous experiment. The experiment
was performed with three subjects. These subjects were
healthy and had no previous experience with the BCI.
Each subject was measured in three sessions in
different days. Session consists of 4 to 5 minutes long
measurements with short breaks between them. One
feedback type was tested in one measurement. There
were nine measurements in each session (three of each
feedback type). The videos from experiments can be
found on research group website .
Each cycle command-feedback was considered right
or wrong. Right cycle had more samples classified
according to the command. The score was computed as
ratio of right cycles and all cycles. The score for each
feedback type can be found in Table 1.
The best results were with feedback based on
Arkanoid probably because it is much more motivating
than the others. The Arkanoid feedback shows
whatever the user was successful and it even displays
score. The subjective opinion about the feedback
visualizations differs among subjects. The subject one
claimed there is not big difference between feedback
visualizations. The subject two founds the Bar
feedback most easy to control but he has better results
for the Arkanoid. He seemed very motivated when
controlling the Arkanoid. The subject three claimed she
does not like the Arkanoid feedback because it is much
more stressful than the others. But the results for
Arkanoid was also better than for other feedback types.
We found out the score is very subject dependent.
Our second subject have significantly better results
than other two subjects (Table 2).
Conclusion and next steps
We found out that even the simple methods can be
used for left and right hand imagined movement
classification of EEG. We proven the BCI can be
operated even with relatively simple equipment (only
two channels EEG) and without need of very special
environment such as shielded room. The novel method
of asymmetric ratio automatic balancing was achieving
high successful rates. This method was used to
compare different feedback visualization types. The
best results were with feedback which was based on
computer game. The second was feedback based on
real photographs and last was abstract visualization.
The large differences between individual subjects was
found. The average score of the best subject was 88.5%
but the worst average score was only 56.3% (50% is a
The significant influence of momental psychical state
of subject was also observed. For example stress or
insufficient sleep can lower classification score.
Subjects also easily lose focus or motivation. This can
be partially avoided by right experiment setup and
communication with subject.
The subjects were able to use the system almost
without training. The results close to average were
achieved after less than ten minutes from start of their
first experience with BCI.
Because the system is relatively simple it provides
large space for future improvements. More advanced
classification methods can be used. The modular
processing system enables easy adaptation of the new
classification methods. The data from realized
experiments can be used for design and testing of these
methods. The first step in future improvements will
probably be adding new class to classification. Thus
the classification classes will be left, right, and neutral.
This can be done with existing classification with
asymmetric ratio even without changes in the source
Jakub Šťastný is supported by the Grant Agency of
the Czech Republic through project P-102/11/1795:
Novel selective transforms for non-stationary signal
processing and along with the rest of the team by Grant
Agency of the Czech Technical University in Prague,
grant No. SGS12/143/OHK3/2T/13.
Biological Signal Lab.,
Faculty of Electrical
Czech Technical University in Prague,
Technická 2, Prague 6, 166 27 Czech Republic
Phone: +420 603 512 421
 J. Doležal, J. Šťastný, P. Sovka. Recognition of Direction of Finger Movement From EEG Signal Using Markov Models. In 3rd European Medical and Biological Conference on Biomedical Engineering EMBEC 2005, Prague, pp. 336-341, 2005.
 J. Šťastný, J. Doležal, V. Černý, J. Kubový. Design of a modular brain-computer interface. In Applied Electronics, pp. 319-322, 2010.
 J. Doležal, J. Šťastný, V. Černý, J. Kubák. Real Time EEG processing. 57. společný sjezd české a slovenské společnosti pro klinickou neurofyziologii, proceedings, pp. 49, 2010.
 J. Doležal, V. Černý, J. Šťastný. Constructing a Brain- Computer Interface. In Applied Electronics,. 99-102, 2011.
 B. Obermaier, C. Guger, C. Neuper, G. Pfurtscheller. Hidden Markov models for online classification of single trial EEG data. In Pattern Recognition Letters, vol 22, pp 1299-1309, 2001.
 B. Obermaier, C. Munteanu, A. Rosa, G. Pfurtscheller. Asymmetric Hemisphere Modeling in an Offline Brain– Computer Interface. In IEEE Trans. On Systems, Man and Cyber. Part C, vol. 32, pp. 536-540, 2001.
 G. Pfurtscheller and C. Neuper. Motor Imagery and Direct Brain–Computer Communication. Proceedings of the IEEE, vol. 89, no. 7, pp. 1123-1134, 2001
 A. Schloegl, C. Neuper and G. Pfurtscheller. Subject specific EEG patterns during motor imagery. In procceding of 19th International conference IEEG/EMGS, pp. 1530-1532, 1997.
 H. Ehrlichman, M. S. Wiener, EEG asymmetry during covert mental activity. In Psychophysiol, vol. 17, pp. 228-235, 1980
 C. Neuper. Feedback-Regulated Mental Imagery in BCI Applications: Using Non-Invasive EEG and NIRS Signals. In BBCI Workshop 2009 – Advances in Neurotechnologies, Berlin, 2009.
 J. Doležal, V. Černý, J. Šťastný. Online motor-imagery based BCI Study on feedback traning. submitted for publication In Applied Electronics. 2012
 J. Štastný. Brain-Computer Interface Research Group website, online: http://amber.feld.cvut.cz/fpga/studenti_kolegove.html