Facebook Wants to Build AR Glasses that Gives Users Hearing Superpowers
Oculus’ parent company Facebook has been working on developing its own pair of AR glasses for quite some time now, and a new report from the Facebook Reality Labs Research (FRL) team reveals how sound could play a significant role in augmented reality and future AR/VR devices.
In an official blog post, the audio team at Facebook Reality Labs Research shared two advanced audio technologies that are currently in the works. The first, which the team refers to as “Perceptual Superpowers,” will allow the AR system to identify the source of the sound that a person is trying to hear, and then will be able to enhance it while also dampening any unwanted background noise in the real world.
The technology works by utilizing an array of microphones built into the device to capture the sounds around you within the physical environment. It will then take contextual cues using the pattern of your head and eye movements to figure out exactly the sound you are trying to listen to and isolate it from any background noise in the same space.
While the lab’s “perceptual superpowers” is somewhat similar to the functionality of existing hearing aids, which also amplify sound and reduce background noise for the hearing impaired, Facebook says the majority of its work focuses on transforming communication for all people everywhere. This technology will allow those with or without hard of hearing to take full advantage of its enhanced auditory capabilities in any social setting or situation.
The FRL Research team is also developing what it calls “Audio Presence,” which will recreate how sound waves travel and bounce off surfaces in the real world, allowing you to feel as though the source of the virtual sound that you hear is physically present in the same space as you. It is so realistic that it would be “indistinguishable” from reality.
The researchers are working on advanced algorithms and software processing techniques to make it possible to personalize spatialized audio in a way that is unique to how each person hears sound differently in the real world based on the shape of their head and ears. This is also known as a head-related transfer function (HRTF).
With today’s VR and computer gaming, the HRTF is generic and considered limited by a one-size-fits-all approach to audio, but with personalized HRTF, however, the sounds are uniquely individualized for each person, allowing everyone to truly hear virtual sounds in the same way that they would perceive sounds to be in the real world. The FRL audio research team is currently looking into several cost-effective and scalable methods that would accurately produce a personalized HRTF, such as from using a simple photograph of a user’s ear.
Facebook aims to make human communication through advanced audio innovation so convincing that a person would not distinguish between the virtual sound from headphones and real sound from the world around them.
For the time being, these advances in AR audio are still in early development stages. There are no details regarding when or if this technology will make its way to a consumer product, but we do expect to hear more about Facebook’s AR/VR future ambitions at the upcoming Facebook Connect (formerly Oculus Connect) on September 16th.