Recent theoretical and technological advances in neuro-imaging techniques allowing brain electrical activity recording, both on hardware and software packages, lead to affordable and user-friendly equipment for the non-scientists end-users. More and more artists start to use electroencephalogramm (EEG) as an input to directly produce multimedia content based on concurently occuring EEG events or specific EEG characteristics, or to jointly produce and/or model, modulate, live artistic content. Art productions, as sensitive and emotional experiences, have always been a window on human‘s heart and mind. Indeed, for the spectator, it opens doors to questioning and appeals curiosity through interpretation. This embodied phenomenon can be used as a pedagogical tool, each one of the participant building his own experience of the sensitive world through sensory stimulation, to bring out, explain and ease the understanding through direct concretisation (using sensory-emotional experience) of underlying scientific facts/principles.
I introduce here a new concept of brain computer interface which allows artistical interactive work, constitutes a pedagogical tool to deliver and illustrate scientific knowledge, as well as a new setup for scientific exploration/investigation. The “Brainarium” (originally “Cerveaurium” in French) is an integrative work based on the convertion of a portable planetarium device into a brain metaphor which allows to play classical multimedia content, but above all which can display EEG data recorded from a subject in real time using brain-computer interface (BCI) technologies.
The system has been demonstrated at several performances in national museums. We believe this tool offers a tremendous potential to innovative applications in the field of neuroscience and more generally art, entertainment, education and health.
The Brainarium inflating at the Natural History Museum of Toulouse
Brainarium's presentation vidéo
The idea of the Brainarium came from crossing of several disciplines and technical skills I built up allong past years, gathering my academical background and personnal subjects of interests. During my master degree and my Phd, I had the opportunity to start studying a field of research I had heard off in the past and which I was really curious about, namely Neurofeedback and Brain Computer Interfaces. I could start playing with neurofeedback and BCI solutions, mainly using very basic applications as feedback elements, but this allowed me to discover the first bricks of the systems, EEG recordings, Signal Processing, Features extractions, Classification, Machine Learning...
Since I am interested since my teenage in computer graphics, 3D modeling and animation, as well as game development and virtual reality, and as they are at the final edge of the closed loop used by neurofeedback and BCI, I did my first trials experimenting with existing OpenSource solutions for Gaming such as the amazing peace of software which Blender is. Browsing tutorials and the Blender's website I noticed there was an great plugin which allowed projection on hemispheric surfaces. So I though that would be cool to combine BCI with a kind of immersive environment. And I think this is where the idea started to arrise... On the neuroscientific part, as I was spending at the beginning a lot of time starring and trying to understand and interpret EEG or fMRI activation or statistical maps, the analopgy between hemispherical surfaces, and more specifically fulldomes, the scalp surface or the brain, and these maps, fused into many different new ideas gathering real time or offline representation and visualisation of brain acitivity data. It appeared to me that the dome could be a nice metaphorical representation of the brain, in which we could place ourself, diving into the brain, looking up at what's recorded at the surface of the skull, or even at the surface of the cortex...and as we are at the 3D visualisation aera, why not trying doing it in stereoscopy. Of course there are many technological issues to overcome before reaching this stage.
However the idea seemed worth trying building a first version. I looked for a formal frame to bring it to life and that's were I started collaborating with two associations from Toulouse, "Incognu" and "Les Chemins Buissonniers", and later a third one: "La ligue de l'enseignement Midi-Pyrénées"...With EEG recording devices, BCI softwares, visualization tools, hemispheric projection system and fulldome projection surface, everything was there to start stitching the whole loop...
The complete system is based on combining together an hemispheric projection system such as the one used in a planetarium, an hemispheric projection surface, and a BCI system. Various solutions can be developed depending on how much funding and materials are available, and every functional block of the system can be changed by an equivalent element.
For projection, we used a transportable planetarium system which is composed of a Digitarium® Delta Portable Digital Planetarium System ($30,000) and a Digitalis™ Portable Dome with a diameter of 7 meters ($12,400). However, both hemispheric projection system as well as projection surface can be made at a lesser cost using custom made tools (see Yves Lhoumeau excellent website at Lhoumeau Sky- System). The control console was composed of a classic personal computer equipped with a dual screen graphic card and two LCD monitors. A video splitter was used to send the display signal to both a control monitor and the video projector. This can be purchased at a low cost (less than 100 dollars) on the Internet. For the EEG signal acquisition, Emotiv Epoc headset was purchased in its Research edition package in order to have access to raw data for 750 dollars. Except from Emotiv software suite and Mind Your OSC, the software used to do signal processing and visualisation are mostly part of the Open Source community. The freely available OpenVibe software was used for signal acquisition, signal processing and visualisation of the EEG data. The interactive fractal video was displayed using the VVVV software. The 3D brain model application has been developed using Blender.
I first formulated the project aiming at presenting the device at "The Brain's Week", a national event around the brain. I add chance to benefit from administrative support and material means to design the first prototype, as well as creative inputs to help elaborate the pedagogical content. Incognu brought their recently acquired Emotiv Headset to make things smoother than using the lab EEG caps, La Ligue de l'Enseignement let us borrow his inflatable planetarium to build the system which saved us from building our own dome and projection system, and made the whole realisation much easier. Finally, Les Chemin Buissonniers handled the administrative work, financial prospection, production of the performance and communication, they build up contact and collaboration with artists and are now developping new perpectives for the Brainarium.