Google Releases LiteRT.js: A JavaScript Binding of LiteRT That Runs .tflite Models in Browsers via WebGPU
Google released LiteRT.js, a JavaScript binding of LiteRT. LiteRT is Google’s on-device inference library, previously called TensorFlow Lite.
LiteRT.js runs .tflite models directly inside the browser. Because inference stays local, Google cites enhanced user privacy, zero server costs, and ultra-low latency.
What is LiteRT.js?
It is not a new model format. Rather, Google compiled its existing native runtime to WebAssembly and exposed it to JavaScript.
Earlier web AI solutions, including TensorFlow.js, relied on JavaScript-based kernels. Google describes those as less performant. LiteRT.js instead ships the native cross-platform runtime with its optimizations intact.
Consequently, web apps inherit work done elsewhere. Performance upgrades, quantization improvements, and hardware optimizations built for Android, iOS, and desktop arrive on the web too.
How It Works: One Runtime, Three Backends
Under that runtime, LiteRT.js targets three backends:
CPU uses XNNPACK, Google’s optimized CPU library, with multi-thread support and a relaxed SIMD build.
GPU uses ML Drift, Google’s on-device GPU solution, running through WebGPU.
NPU uses the WebNN API, currently experimental in Chrome and Edge.
Two related rules govern dispatch. First, LiteRT.js does not support partial delegation. A graph cannot split across CPU and GPU.
Second, delegation is all-or-nothing per model. If a model cannot be fully delegated to the chosen accelerator, LiteRT falls back to wasm execution. The CPU path has the widest operator coverage.
Performance
Given those backends, Google team reports two distinct results.
Against other web runtimes, LiteRT.js is up to 3x faster across CPU and GPU inference. That figure covers classical computer vision and audio processing models.
Against its own CPU execution, GPU or NPU delivers a 5–60x speedup. That applies to demanding real-time work like object tracking and audio transcription.
Both benchmarks ran in a controlled browser environment on a 2024 MacBook Pro with M4 Apple Silicon. Google notes results vary with local GPU, thermal throttling, and driver optimization. A “10x” figure circulating alongside the launch does not appear in the announcement.
Getting a PyTorch Model In
LiteRT Torch converts PyTorch models to .tflite in a single step.
However, the prerequisites are strict. Your model must be exportable with torch.export.export, meaning TorchDynamo-exportable. It cannot contain Python conditional branches that depend on runtime tensor values. It also cannot have dynamic input or output dimensions, including the batch dimension.
For size, AI Edge Quantizer configures quantization schemes across different model layers. Pretrained .tflite models are also available on Kaggle and the LiteRT Hugging Face Community.
The Minimal Pipeline
Once converted, the runtime code is short. This is the WebGPU path, verified against @litertjs/core v2.5.2:
// Wasm files live in node_modules/@litertjs/core/wasm/ or on a CDN.
await loadLiteRt(‘path/to/wasm/directory/’);
const model = await loadAndCompile(‘path/to/model.tflite’, {
accelerator: ‘webgpu’, // ‘wasm’ | ‘webgpu’ | ‘webnn’
});
const input = new Tensor(new Float32Array(1 * 3 * 224 * 224), [1, 3, 224, 224]);
const results = await model.run(input);
// Accelerator results live off-heap. Move to CPU, then convert.
const cpuTensor = await results[0].moveTo(‘wasm’);
const output = cpuTensor.toTypedArray();
// LiteRT.js uses manual memory management. Delete every tensor.
input.delete();
for (const t of results) t.delete();
cpuTensor.delete();
That last block deserves attention. LiteRT.js does not garbage-collect tensors. Every Tensor must be deleted explicitly, or the app leaks device memory. The snippet in Google’s announcement post omits this step.
WebNN needs one extra flag. LiteRT.js requires JSPI, which bridges synchronous kernel scheduling with asynchronous device polling:
const model = await loadAndCompile(‘model.tflite’, {
accelerator: ‘webnn’,
webNNOptions: {devicePreference: ‘npu’}, // ‘cpu’ | ‘gpu’ | ‘npu’
});
Before writing pre-processing, test with fake inputs. Run npm i @litertjs/model-tester, then npx model-tester. It runs your model on WebNN, WebGPU, and CPU using random inputs. Use model.getInputDetails() to read input names and shapes.
Use Cases With Examples
Those APIs back four demos Google shipped at launch:
LiteRT.js vs TensorFlow.js
Those demos raise an obvious question for existing web ML teams.
Importantly, the two are not mutually exclusive. Google positions LiteRT.js as a replacement for TF.js Graph Models specifically, not the whole library.
TensorFlow.js remains the recommended tool for pre- and post-processing. The @litertjs/tfjs-interop package passes tensors between them via runWithTfjsTensors. Avoid tensor.dataSync, which carries a significant penalty on the WebGPU backend.
Interactive Explainer
The embed below animates the six pipeline stages across each backend.
Key Takeaways
LiteRT.js runs .tflite models in-browser using Google’s native runtime compiled to WebAssembly.
Reported gains: up to 3x over other web runtimes; 5–60x for GPU/NPU over its own CPU path.
Three backends: XNNPACK on CPU, ML Drift over WebGPU, WebNN for NPUs. No partial delegation; falls back to wasm.
Tensors are manually managed. Call .delete() or leak device memory.
WebNN remains experimental. WebGPU is the practical acceleration target today.
Sources
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.
