Google Releases LiteRT.js: A JavaScript Binding of LiteRT That Runs .tflite Models in Browsers via WebGPU

Google has released LiteRT.js, a JavaScript binding that allows .tflite models to run directly in browsers using WebGPU. The tool is the web version of LiteRT, previously known as TensorFlow Lite.

By keeping inference on the client side, Google claims the approach improves privacy, removes server costs, and reduces latency.

What is LiteRT.js

This release is not a new model format. It compiles Google’s existing native runtime into WebAssembly and exposes it to JavaScript.

Previous web AI tools, including TensorFlow.js, relied on JavaScript-based kernels. Google states those were less performant. LiteRT.js ships the native cross-platform runtime with its optimisations intact.

Web applications now inherit work done for mobile and desktop. Performance upgrades, quantisation improvements, and hardware optimisations built for Android, iOS, and desktop arrive on the web.

One runtime, three backends

Under that runtime, LiteRT.js targets three backends:

• CPU uses XNNPACK, Google’s optimised CPU library, with multi-thread support and a relaxed SIMD build.
• GPU uses ML Drift, Google’s on-device GPU solution, running through WebGPU.
• NPU uses the WebNN API, currently experimental in Chrome and Edge.

Two rules govern dispatch. First, LiteRT.js does not support partial delegation. A graph cannot split across CPU and GPU.

Second, delegation is all-or-nothing per model. If a model cannot be fully delegated to the chosen accelerator, LiteRT falls back to wasm execution. The CPU path has the widest operator coverage.

Performance

Google reports two distinct results based on those backends.

Against other web runtimes, LiteRT.js is up to 3x faster across CPU and GPU inference. That figure covers classical computer vision and audio processing models.

Against its own CPU execution, GPU or NPU delivers a 5–60x speedup. That applies to demanding real-time work like object tracking and audio transcription.

Both benchmarks ran in a controlled browser environment on a 2024 MacBook Pro with M4 Apple Silicon. Google notes results vary with local GPU, thermal throttling, and driver optimisation. A 10x figure circulating alongside the launch does not appear in the announcement.

Getting a PyTorch model in

LiteRT Torch converts PyTorch models to .tflite in a single step.

However, the prerequisites are strict. Your model must be exportable with torch.export.export, meaning TorchDynamo-exportable. It cannot contain Python conditional branches that depend on runtime tensor values. It also cannot have dynamic input or output dimensions, including the batch dimension.

For size, AI Edge Quantizer configures quantisation schemes across different model layers. Pretrained .tflite models are also available on Kaggle and the LiteRT Hugging Face Community.

The minimal pipeline

Once converted, the runtime code is short. This is the WebGPU path, verified against @litertjs/core v2.5.2:

import {loadLiteRt, loadAndCompile, Tensor} from '@litertjs/core';

// Wasm files live in node_modules/@litertjs/core/wasm/ or on a CDN.
await loadLiteRt('path/to/wasm/directory/');

const model = await loadAndCompile('path/to/model.tflite', {
  accelerator: 'webgpu', // 'wasm' | 'webgpu' | 'webnn'
});

const input = new Tensor(new Float32Array(1 * 3 * 224 * 224), [1, 3, 224, 224]);
const results = await model.run(input);

// Accelerator results live off-heap. Move to CPU, then convert.
const cpuTensor = await results[0].moveTo('wasm');
const output = cpuTensor.toTypedArray();

// LiteRT.js uses manual memory management. Delete every tensor.
input.delete();
for (const t of results) t.delete();
cpuTensor.delete();

That last block deserves attention. LiteRT.js does not garbage-collect tensors. Every Tensor must be deleted explicitly, or the app leaks device memory. The snippet in Google’s announcement post omits this step.

WebNN needs one extra flag. LiteRT.js requires JSPI, which bridges synchronous kernel scheduling with asynchronous device polling:

await loadLiteRt('path/to/wasm/', {jspi: true});

const model = await loadAndCompile('model.tflite', {
  accelerator: 'webnn',
  webNNOptions: {devicePreference: 'npu'}, // 'cpu' | 'gpu' | 'npu'
});

Before writing pre-processing, test with fake inputs. Run npm i @litertjs/model-tester, then npx model-tester. It runs your model on WebNN, WebGPU, and CPU using random inputs. Use model.getInputDetails() to read input names and shapes.

Use cases with examples

Those APIs back four demos Google shipped at launch:

LiteRT.js vs TensorFlow.js

Those demos raise an obvious question for existing web ML teams.

Importantly, the two are not mutually exclusive. Google positions LiteRT.js as a replacement for TF.js Graph Models specifically, not the whole library.

TensorFlow.js remains the recommended tool for pre- and post-processing. The @litertjs/tfjs-interop package passes tensors between them via runWithTfjsTensors. Avoid tensor.dataSync, which carries a significant penalty on the WebGPU backend.

What it means

Developers building browser apps can now use the same .tflite models they deploy on Android or iOS. This removes the need to maintain separate code paths for web and mobile inference. The trade-off is manual memory management for tensors, which requires explicit cleanup to prevent leaks.