Advancing AR, VR and MR with AI

AI in the New Reality

Extended Reality (XR) is an umbrella term encapsulating Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), and everything in between. While each of these simulated realities offers a wide range of revolutionary experiences, all require similar underlying technologies, including Artificial Intelligence (AI). As the demand for richer user experiences and more natural interactions grows, complex neural networks increasingly come into play. Neural networks in XR applications must operate in real-time and require high-performance Neural Processing Units (NPUs). Still, even with higher processing needs, XR applications must balance performance needs with power efficiency, as battery life is an essential user requirement. Therefore, OEMs and chipmakers require NPU architectures that balance performance and power consumption to meet the unique requirements of their application.

Neural Network Diversity
NPUs used in XR need to perform at tens of TOPS or more. They also must run multiple concurrent Neural Networks (NN) efficiently—something not typically seen in other markets. For example, a small consumer device may run a single NN with a small resolution, but NPUs used in XR need to run multiple networks concurrently at much higher resolutions. Additionally, many OEMs have developed unique custom and proprietary NNs. Consequently, NPUs in XR applications must have the ability to run multiple and diverse high-resolution networks concurrently.

Reducing AI Power Consumption in AR/VR/MR Headsets
One of the key performance metrics of any XR device is battery life. NPUs can consume significant power—that is, if the wrong NPU is chosen. Comparing the power efficiency of NPUs can be complicated. Ours isn't—Expedera's Origin™ E6 family averages a market-leading 18 TOPS/W. Origin has repeatedly been cited as the most power-efficient NPU available by third parties and customers alike.

AI in AR/VR/MR Devices - Use Cases
AI unlocks new capabilities and features in XR headsets. Image-based neural networks can support enhanced use cases such as facial and hand detection/recognition, eye gaze tracking, object labeling and detection, and many others. In addition to image and video capabilities, the use of AI in audio applications, including speech recognition and audio de-noising is growing.

Best AI for AR/VR/MR Uses
While there are many general-purpose NPUs available, a one-size-fits-all solution is rarely the most efficient, especially for performance- and power-sensitive consumer devices. General-purpose AI processors are often larger than needed and consume more power than necessary. Expedera's E6 IP cores are optimized for XR use cases. The Origin™ E6 will always provide optimal PPA (power, performance, area) and typically achieve superior performance in about half the silicon area compared to other NPUs. By embedding Expedera Origin NPU IP into XR applications, manufacturers of AR/VR/MR headsets and SoCs/ASICs can reduce latency, increase performance, decrease DRAM requirements, and extend battery life.

Learn More About Expedera Products