WebGPU and WebNN Drive 3x Faster Browser AI in LiteRT.js

Google has officially launched LiteRT.js, a high-performance JavaScript binding for the LiteRT edge AI runtime. The release allows developers to execute machine learning models directly in web browsers with near-native performance. By leveraging modern web standards like WebGPU and WebNN, LiteRT.js shifts computational loads to client-side hardware, reducing server costs and eliminating network latency for real-time applications.

Hardware Acceleration and Benchmarks

LiteRT.js manages execution across different hardware profiles using a dynamic CompiledModel API. This implementation automates accelerator selection and handles I/O buffers asynchronously to keep latency low.

Backend	Technology	Target Hardware
GPU	ML Drift via WebGPU	High-throughput parallel processing graphics units
NPU	WebNN API	Dedicated Neural Processing Units for low-power efficiency
CPU	XNNPACK mapped to WebAssembly	Legacy devices via multi-threading and relaxed SIMD

In controlled browser environments running on a 2024 Apple MacBook Pro with M4 Apple Silicon, Google recorded up to a 3x increase in inference throughput across both CPU and GPU backends compared to previous web runtimes. The gains were particularly pronounced in classical computer vision and audio processing workloads.

Generative AI and Gemma 4 Integration

Alongside the core library, Google introduced LiteRT-LM.js, a specialized API built specifically for browser-based Large Language Models. This module handles streaming token output and conversation state management.

It ships with native support for web-optimized variants of Gemma 4. Developers can target specific configurations like gemma-4-E2B-it-web.litertlm and gemma-4-E4B-it-web.litertlm. If you plan to run Gemma 4 on-device, this specialized web variant avoids the memory bottlenecks typically associated with loading billion-parameter models into a browser tab.

Ecosystem Interoperability

LiteRT.js utilizes the standard .tflite flatbuffer format. This ensures that models deployed to web environments remain consistent with those compiled for Android and iOS.

For teams working outside the Google ecosystem, the release includes a litert-torch converter. Developers can write their models in PyTorch, JAX, or standard TensorFlow, export them via standard tooling, and run the resulting .tflite files directly in the browser. The package also integrates with existing TensorFlow.js pipelines, meaning you can swap out the underlying AI inference engine without rewriting your application’s data preprocessing or rendering logic.

The core package is available via npm as @litertjs/core. If you are building client-side AI features, you should test your existing .tflite models in the new LiteRT.js model tester application to measure the exact throughput gains on your target hardware before migrating your production dependencies.