Improving the experience of music for people with cochlear implants

Cochlear implants are one of the most effective neuroprosthetic technologies ever devised. Hundreds of thousands of deaf people have received cochlear implants, which have enabled them to hear. But despite this incredible success and the fact that the technology has advanced rapidly over the last decade, there are still major issues yet to be resolved. For example, cochlear implant recipients find it particularly difficult to process complex sounds such as speech where there is a lot of background noise, or music. The reason for this is that the tiny wire threaded into the cochlea has a limited number of electrodes. This significantly limits the frequency information that can be relayed to the brain.

The research project 'Tactile Listening' is a collaboration between students and scientists from a range of different disciplines seeking to improve cochlear implant technology. They are currently developing software and hardware that will allow cochlear implant recipients to get more enjoyment from music through tactile listening.

The project leaders are Rúnar Unnþórsson, professor and Head of the Faculty of Industrial Engineering, Mechanical Engineering and Computer Science, and Árni Kristjánsson, professor at the Faculty of Psychology. They have worked together for many years to find solutions aimed at improving the experiences of those with sensory impairments. 'Tactile Listening' has received a grant from the Technology Development Fund. "We have been improving and testing a solution that we developed for the EU project Sound of Vision, the Sensory Belt, which is a belt with motors that was used to transmit information using vibrations around the waist,“ says Rúnar. The belt won an Innovation Radar Prize in 2018 and came second in the 2017 University of Iceland Applied Science Prize.
 

The solution has attracted significant interest

Two students were involved in the Tactile Listening project this summer, supported by the Student Innovation Fund: Elvar Atli Ævarsson, Master's student in electrical and computer engineering at the University of Iceland, and Þórhildur Ásgeirsdóttir, Master's student in interactive media technology at the KTH Royal Institute of Technology in Sweden. They have been planning and implementing sensory tests with the new equipment. 

They report that there is a huge demand for this solution, since music can have a significant and positive impact on people's health. "We know this from our own experience as music lovers. On the other hand, many people who have lost their sense of hearing as adults and regained it through a cochlear implant have been disappointed with their ability to enjoy music. We want to help them regain the joy of listening to music," says Þórhildur, adding that the aim is to develop a new approach based on surmounting the limitations of the cochlear implant by integrating electrical and tactical stimulation, i.e. adding vibrations on the skin to the information from the cochlear implant. 

The main challenge this summer is to user test the prototype equipment currently being developed. "At this stage we are mainly investigating the sensitivity threshold in users for different vibration frequencies. This will help us determine what frequency range it is best to use in the final product, how best to match strength and frequency settings and so forth. We will then conduct further vital research into user experiences."

They have already designed a prototype of the software and multi-channel equipment that divides the audio signal into several vibration outlets. It can be used to manage the strength and frequency of vibration at each individual outlet. The vibration then transmits to the skin as a new feed of information, which the brain learns to interpret. "Our final goal is for the brain to use the extra information from our equipment to fill in the gaps in the information from the cochlear implant, thus improving the musical enjoyment of CI recipients," says Þórhildur.

It is painstaking work to interpret information from musical signals, so the team have to actually rethink the vibration technology from the ground up. They say that they hope to gain a better insight into how users interact with the equipment in order to aid the next stages of development. It is important in innovation and technological development to work in line with the users' needs so that the final product is functional and serves as a genuine solution. If the project is successful, it should have a really positive impact on the deaf community and may even benefit people with unimpaired hearing.

The project is conducted in partnership with Oticon, manufacturer of hearing aids and cochlear implants, DTU (Danmarks Tekniske Universitet), and the University of Southampton. "Oticon has expressed a lot of interest in tactile solutions and if the project is successful, the company will want to take the technology, develop it and market it along with its cochlear implants. The gaming company CCP has also expressed an interest in using this technology for its virtual reality games because it should be possible to use it to make players even more immersed in the game."