Oculus-parent company Facebook is aiming to redefine the future of communication with new groundbreaking 3D capture technology and AI systems that can create lifelike avatars of people.

In an Oculus blog post today, Facebook Reality Labs (FRL) chief scientist Michael Abrash highlighted the company’s research and development into creating more realistic and personalized virtual avatars, which internally the project is called ‘Codec Avatars’ and is being worked on by the team in FPL’s Pittsburgh office.

According to Yaser Sheikh, Director of Research at Facebook Reality Labs in Pittsburgh, the ‘Codec Avatars’ project seeks to overcome the challenges of physical distance between people and make social connections in virtual reality feel as “natural and common” as possible to those that people experience and share with others in the real world.

“Codec Avatars is an active research project today, but it could radically change the way we connect through VR headsets and AR glasses tomorrow,” says Sheikh. “It’s not just about cutting-edge graphics or advanced motion tracking. It’s about making it as natural and effortless to interact with people in virtual reality as it is with someone right in front of you. The challenge lies in creating authentic interactions in artificial environments.”

In order to achieve this level of real-time, ultra-realistic virtual avatars it requires a sophisticated, state-of-the-art setup. FRL constructed two capture studios—one for capturing faces, and another for capturing full bodies—both of which are considered too “large and impractical” for use outside of the research facility. However, the company says its goal is to be able to achieve the same realistic results through the use of lightweight headsets in the future.

“Codec Avatars need to capture your three-dimensional profile, including all the subtleties of how you move and the unique qualities that make you instantly recognizable to friends and family,” the company says. “And, for billions of people to use Codec Avatars every day, making them has to be easy and without fuss.”

The capture system FRL Pittsburgh has developed is capable of gathering 180 gigabytes of data per second utilizing hundreds of cameras fitted around a dome structure – each camera captures data at a rate of 1 GB per second from a small group of research participants in an effort to create a database of physical traits. A proprietary algorithm then uses the data to create a unique avatar for the individual scanned. The Pittsburgh team’s goal is to build models that will one day allow people to easily create their own lifelike avatars based on just a few images or videos.

There are plenty of challenges ahead that come along with this technology, particularly the risks involved when dealing with safety and the potential misuse of a person’s digital likeness to deceive others in the virtual world. “Trust is a critical component when talking to people in real life, and it shouldn’t be any different in virtual reality,” says Sheikh.

Sheikh goes on to further explain, “FRL Pittsburgh is thinking pragmatically about safeguards to keep avatar data safe. For example, the team is exploring the idea of securing future avatars through an authentic account. How we work with real identities on our platforms will be a key part of this, and we have discussed several security and identity verification options for future devices. We’re still years away from this type of technology reaching consumer headsets, but FRL is already working through possible solutions.”

To learn more about FRL’s ‘Codec Avatar’ research, check out Facebook’s technical post here.