BCI Potential Currently Unrealized
The ability to control an external device via thought has the potential to enhance humanity greatly. Applications nearing commercial viability include control of neuroprosthetics, computers (for spelling, research, communication…), phones, drones and VR worlds. Despite great progress however much work remains. BCIs are not yet a viable commercial technology (Chaudhary et. al., 2021). They are scarcely used outside laboratories for practical applications, and effective translation from proof-of-concept prototypes into reliable applications remains elusive. (Chavarriaga et. al., 2016).
The Mind is Critical
Signal decoding and classification is central to BCI. What does the user’s brain signal mean? What mental command does it represent? Making full use of the subjective mind would greatly improve this. However the mind is currently all but ignored. During a BCI task or mental command, only one (ex: “imagine the cursor moving to the left”) or two (ex: “imagine… with high cognitive workload”) mental states or processes are considered. Due to this very narrow account of the user’s mind, classifiers which decode and classify the brain signal are similarly limited.
A classifier representing a single aspect of mind narrows the user’s mind, and brain signal, to be considered. The classifier can decode and classify (explicitly) only those mental components. However even in a controlled laboratory setting, the user activates a dozen or more mental states and processes during a single task. With the above example, mind includes not just imagination but perception, emotion (excitement, confidence, frustration, anxiety), motivation, goals, attention, cognitive workload, intention, and prediction as well as hunger, thirst, fatigue, and much more.
As the % of the mind included in the classifier (or set thereof) increases, signal classification accuracy increases accordingly. Why? Because the brain signal represents the state and function of the mind. It mirrors the mind. Human movement clearly demonstrates this. The efferent signal can respond, directly or indirectly, to any aspect of mind. This includes not only the signal directly causing movement, sent from the motor cortex. Movement also responds to paths leading to the motor cortex (movement intentions such as “raise my right arm”), and to paths associated with intentions (perception, recognition, meaning, emotion, prediction, goals…). If brain activity doesn’t mirror that of the user’s mind, then how could he possibly control his movement with skill and precision, in real time? How could perception, thought, and intention instantly affect movement?
In fact, the idea the brain’s activity & signal mirrors the mind is the entire basis of the field of cognitive neuroscience. It’s well-known a particular type or instance of stimulus, thought, emotion, imagination, intention and the rest are closely connected to (large scale, coordinated) patterns of neural activity. Distinct brain oscillations underlie specific cognitive functions (Cox et. al., 2018)..
The key to unlocking the mind’s potential is to first take it seriously. Then understand it — well enough to define its contents, and their function through space and time. In other words a mind model is needed. Using an accurate one, any set of mental states and processes can be defined, connected and weighted. This “functional mind map” can then be connected to brain activity and signal. From here, a mind/brain signal, “signature” can be developed. This can be used as the basis of a classifier set, to decode and classify matching brain signals (signal features). The more precise and accurate the subjective map, the more accurate the classifier, and subsequent classification.
If on the other hand strongly-active mental components aren’t accounted for, the classifier can’t include them (explicitly). Most of the signal’s features will remain unlabeled, and unclassified (as such). Or if labeled, done so in a very vague way — as a single paradigm or task (ex: “move my right arm forward”).
Taking advantage of the user’s subjective mind in BCI is supported by other researchers. They argue the field would benefit greatly from a more “user-centered” approach. The user’s role in device control and performance has been minimized, and taking into account his or her mental states and skills could have a substantial impact in improving BCI efficiency, effectiveness and usability (Lotte, Jeunet, Mladenovic et al., 2018).
I agree, but argue the user-centered approach should be extended beyond user states and traits, to include the entire range of mental states and processes. Those can then be narrowed to the components most strongly and consistently activated during device use, within a given context — environment, situation, activity, task, recent performance etc.
Brain Signal Classifiers
As a mind, and corresponding brain, signal is expressed across task trials, a signature of activity can be identified. This is a range of (mind/brain) expression. This signature can then be used to develop a brain signal classifier. A classifier could be built that is most likely to be expressed — given what is known about the user, the context of the BCI task, and the user’s likely state of mind.
Developing a mind-based classifier involves the following: (1) acknowledge the mind exists (the mind is not just “the brain” but a subjective phenomenon in its own right), (2) understand and define it using a mind model, (3) define the mind’s components active during a mental command, within a mind/brain/body/environment context, (4) identify the brain signal characteristics corresponding to these mind signatures, to create brain signal signatures, and classifiers, and (5) identify those most interesting, motivating and relevant to the user, that he or she can activate with consistency.
Standing in the way of mind-based classifier development however is the lack of a mind model. The human mind is poorly understood (Poldrack & Yarkoni, 2016). Currently no accurate or systematic method of defining a mental state or process, in a given task context, exists. Because of this shortcoming even a sharp turn toward a user-centered approach, though helpful, will have limited effect. The lack of a mind model creates a distorted view of not only it, but the brain to which it connects.
User Empowerment
Mind-based classifiers empower the BCI user. A shift in focus from brain to mind allows the user to see he is in control. He controls his own mind — and brain signal. This can be done with conscious intent. However the mind is manipulated, the brain follows. This shift in perspective helps the user control her brain signal, and external devlce, more naturally yet intentionally and precisely.
Mind-based classifiers also encouraged their personalization. In consultation with others, the user can research and define the states & processes easy for her to achieve, strongly and consistently. These are the mind “targets” she can then strive to “hit” during BCI use. These could include aspects of the user’s mind most strongly aligned with her lifestyle, interests, personal and professional goals and so on.
In short, mind-based classifiers increase the user’s sense of control, and enable personalized mind “targets” that are easier for the user to activate — with strength and specificity.
The Core Problem: The Mind is Ignored
The idea the human mind strongly connects to brain activity & signal is obvious. It’s the basis of the entire field of cognitive neuroscience. Yet mental phenomena are underappreciated within brain science. This neglect trickles down to the field of BCI. Although a specific (mental or behavioral) task may be described in great detail, the mind that executes it is largely ignored.
Minimizing the subjective causes the mind, and corresponding brain signal, to be defined (encoded and decoded) far from optimally. Signal classification suffer accordingly. Many positive states of mind — feeling calm, confident, happy, content, motivated etc. remain unacknowledged. So do unwanted states — feeling frustrated, impatient, anxious, hungry, thirsty etc. If these mental categories are not defined or accounted for subjectively, they won’t be included as part of the classifier, and won’t be able to classify the signal features which represent those categories (at least not purposefully).
The good news is any mental phenomena can be included in a classifier, wanted or unwanted. If consistently activated, a state can be represented by a signal classifier. Or if sporadically and unpredictably activated, it could be labeled and classified as noise, and filtered out of the signal (as such). It can be labeled as noise or as part of the signal — but has to be acknowledged as part of the mind/brain system in the first place.
Summary
Once the user’s mind is defined accurately, connected to the brain, and used to build brain signal signatures, higher-performing classifiers and “mind targets” for the BCI designer, trainer and user to benefit from can be developed.
References
Chaudhary, U., Chander, B. S., Ohry, A., Jaramillo-Gonzalez, A., Lule, D., Birbaumer, N. (2021). Brain Computer Interfaces for Assisted Communication in Paralysis and Quality of Life. International Journal of Neural Systems v. 31. https://doi.org/10.1142/S0129065721300035
Chavarriaga, R., Fried, O., Kleih, S., Lotte, F., Scherer, R. (2016). Heading for new shores! Overcoming pitfalls in bci design. Brain-Computer Interfaces, 4,60.
Cox, R., Schapiro, A., Stickgold, R. (2018). Variability and stability of large-scale cortical oscillation patterns. Network Neuroscience, 2(4),481. doi: 10.1162/netn_a_00046
Lotte, F., Jeunet, C., Mladenovic, J., N’Kaoua, B., Pillette, L. (2018). A BCI challenge for the signal processing community: considering the user in the loop. Signal Processing and Machine Learning for Brain-Machine Interfaces, IETpp.1-2
Poldrack, R.A., Yarkoni, T. (2016). From brain maps to cognitive ontologies: informatics and the search for mental structure. Annual Review of Psychology, 67, 587.