The typical image that comes to mind when we discuss studying time-series data, such as EEG Signals, is poring over a slew of graphs and charts in various formats. But have you thought about exploring the data using audio? Sounds weird, right?
Sonification is the process of using audio or sound to convey the information a dataset carries. Auditory perception makes use of spatial, temporal, frequency, and amplitude resolution as opposed to visual perception, which only uses spatial resolution for interpretation. The advantage of using sound to explore the data is that you get to “feel” the data rather than just visualizing it. It gives life to the data and thereby leads to a comprehensive understanding, especially when the data is highly dense and complex.
The history of sonification dates back to 1908, when the Geiger Counter was invented for measuring the level of radiation in the surrounding area. It makes a clicking sound every time it detects radiation, and the more it clicks, the more radiation is present.
Sonification finds many applications in disparate fields, like studying patterns in air quality indexes over several years can be a useful tool for raising awareness of climate change since people can sense danger looming as the index rises. To demonstrate that, computer scientist Brian Foo utilized Beijing’s air pollution data over a period of three years to generate a musical composition. This piece grows harsher, more percussion-heavy, and has faster-paced tones to convey a sense of danger and emergency; conversely, as the index drops, the composition becomes softer and more relaxing.
How amazing it would be to have this set up in every city so that people could learn about the air quality in their area and enjoy the music while they strive to improve it!
Morse code, used for encoding secret messages, uses sonification to convert letters (dots and dashes) into audio (short and long tone pulses), which can then be broadcast by radio. Sonification is also being utilized in medical technology as an assistive device, and for reviewing patient records. It is being employed in the financial realm as well for inspecting stock prices.
Let us now explore the notion of sonifying our brain signals!
EEG Sonification
Sonification has now entered the world of neuroscience too. The signals acquired from the electrical activity of the brain are temporally quite dense and complex, having a mix of large number of frequencies changing repeatedly. As a result, sonification occasionally outperforms visualization as a method of signal analysis. Sonification of brain activity has been around since brain imaging when in 1930, Prof. Edgar Adrian listened to his own EEG signal.
Various techniques are being employed to convert brain signals into audio depending on the task we need to accomplish. The most widely used sonification method at the moment is Parameter Mapping, wherein EEG parameters like frequency bands are mapped to audio features like pitch, intensity, or duration to create a new audio signal.
Another method being used for sonification is mathematical models or deep learning models, which can take in processed EEG features and output audio signals that can subsequently be used for diverse applications. The generative music technique is used primarily in the Brain-Computer Music Interface (BCMI), wherein spectral information from the EEG signals is used to activate generative music rules and compose music on the fly. Researchers who have a solid grasp of both EEG signal processing and music technology and composition are using this technology to produce relaxing and enjoyable musical compositions.
Applications
EEG sonification has a wide range of applications in healthcare, entertainment, and research. Here is a glimpse of some of those applications:
- EEG Sonification can be used to determine the current emotional state of the person, for example, if a person is fatigued.
- In addition to locating seizure foci during epilepsy surgery, it can also be utilized to detect seizures in patients’ EEG recordings.
- Sonification also finds many applications in the neurofeedback domain. For example, we can listen to our own brain signals while meditating in order to reach the state of calmness and relaxation.
- We can even develop a model to generate our own “music of the brain”, calming tunes from our brains! Researchers are also aiming to create an instrument that can be solely played by modulating brain impulses.
Let us know in the comments section if you have listened to your brain!
Follow The Brainy Bits for more interesting content