X

Origin Evolution for Edge

Edge-friendly LLM and CNN AI Inference processing

Edge devices are increasingly equipped with advanced AI processing capabilities that enhance functionality and improve the user experience. While many of these devices previously depended on cloud-based inference, manufacturers are now shifting towards on-device inference. This transition helps to lower latency, reduce overall power consumption, and minimize the need for cloud processing, thereby cutting costs.

DOWNLOAD PRODUCT BRIEF

Perfect-Fit Solutions

Origin Evolution™ for Edge offers out-of-the-box compatibility with today's most popular LLM and CNN networks. Attention-based processing optimization and advanced memory management ensure optimal AI performance across a variety of networks and representations. Featuring a hardware and software co-designed architecture, Origin Evolution for Edge scales to 32 TFLOPS in a single core to address the most advanced edge inference needs.

Bringing AI to the Edge

Always-sensing cameras continuously sample and analyze visual data to identify specific triggers relevant to the user experience. They enable a seamless, more natural user experience. However, always-sensing data requires specialized AI processing due to the quantity and complexity of data generated. OEMs are turning to specialized AI engines like Expedera’s LittleNPU. The LittleNPU is optimized to process the low-power, high-quality neural networks used by leading OEMs in always-sensing applications. It runs at low power—often as low as 10-20mW—and keeps all camera data securely within the LittleNPU subsystem to preserve user privacy.

Innovative Architecture

Origin Evolution uses Expedera’s unique packet-based architecture to achieve unprecedented NPU efficiency. Packets, which are contiguous fragments of neural networks, are an ideal way to overcome the hurdle of large memory movements and differing network layer sizes, which are exacerbated by LLMs. Packets are routed through discrete processing blocks, including Feed Forward, Attention, and Vector, which accommodate the varying operations, data types, and precisions required when running different LLM and CNN networks. Origin Evolution includes a high-speed external memory streaming interface that is compatible with the latest memory standards.

Choose the Features You Need
Customization brings many advantages, including increased performance, lower latency, reduced power consumption, and eliminating dark silicon waste. Expedera works with edge device customers to understand their use case(s), PPA goals, and deployment needs during their design stage. Using this information, we configure Origin Evolution to create a customized solution that perfectly fits the application.

Unique Packet Architecture

Ultra-Efficient Neural Network Processing

Accepting standard, custom, and black box networks in a variety of AI representations, Origin Evolution offers a wealth of user features such as mixed precision quantization. Expedera’s unique packet-based processing reduces much larger networks into smaller, contiguous fragments, overcoming the hurdle of large memory movements and offering much higher processor utilization. Packets are routed through discrete processing blocks, including Feed Forward, Attention, and Vector, which accommodate the varying operations, data types, and precisions required when running different types of networks. Internal memory handles intermediate needs, while the memory streaming interface interfaces with off-chip storage.

  • 32 TFLOPS performance
  • Support for standard, custom, and proprietary neural networks
  • Readily customized to specific use cases and deployment needs
  • Full software stack provided, including compiler, estimator, scheduler, and quantizer
  • Runs LLM, CNN, RNN, DNN, LSTM, and other network types
  • Delivered as Soft IP (RTL) or GDS

Download our White Papers

White Papers

Get in Touch With Us

Contact