JavaScript
// Install and setup
npm install @litertjs/core

// Basic model inference
import {loadLiteRt, loadAndCompile, Tensor} from '@litertjs/core';

await loadLiteRt('https://cdn.jsdelivr.net/npm/@litertjs/core/wasm/');
const model = await loadAndCompile('/path/to/model.tflite', {
  accelerator: 'webgpu' // or 'wasm' for CPU
});

// Run inference
const inputData = new Float32Array(224 * 224 * 3).fill(0);
const inputTensor = new Tensor(inputData, [1, 3, 224, 224]);
const outputs = await model.run(inputTensor);

// Clean up
inputTensor.delete();
const result = await outputs[0].data();
outputs[0].delete();

Progress:

• Install LiteRT.js dependencies
• Convert existing model to .tflite format
• Set up LiteRT.js runtime with appropriate accelerator
• Load and compile the model
• Create input tensors with correct shape
• Run inference and handle outputs
• Clean up tensor resources
• Test performance vs existing solution

1. Installation and Setup

Bash
npm install @litertjs/core
# For TensorFlow.js integration:
npm install @litertjs/tfjs-interop

Copy wasm files from node_modules/@litertjs/core/wasm/ to your server.

2. Model Conversion (PyTorch)

Python
import litert_torch
import torchvision

# Load your model
model = torchvision.models.resnet18(pretrained=True)
sample_inputs = (torch.randn(1, 3, 224, 224),)

# Convert to LiteRT
edge_model = litert_torch.convert(model.eval(), sample_inputs)
edge_model.export('model.tflite')

3. Runtime Configuration

JavaScript
import {loadLiteRt} from '@litertjs/core';

// Load runtime with CDN or local hosting
await loadLiteRt('https://cdn.jsdelivr.net/npm/@litertjs/core/wasm/');
// or locally: await loadLiteRt('your/path/to/wasm/');

4. Model Loading and Inference

JavaScript
const model = await loadAndCompile('/model.tflite', {
  accelerator: 'webgpu' // Best performance on Chromium browsers
});

// Multiple input formats supported:
const outputs1 = await model.run(inputTensor);
const outputs2 = await model.run([inputTensor1, inputTensor2]);
const outputs3 = await model.run({'input_name': inputTensor});

Example 1: Image Classification

JavaScript
// Input: 224x224 RGB image
const imageData = new Float32Array(224 * 224 * 3);
// ... fill with normalized pixel values
const inputTensor = new Tensor(imageData, [1, 3, 224, 224]);

const outputs = await model.run(inputTensor);
const predictions = await outputs[0].data();
// Output: Float32Array of class probabilities

Example 2: TensorFlow.js Pipeline Integration

JavaScript
import {runWithTfjsTensors} from '@litertjs/tfjs-interop';
import * as tf from '@tensorflow/tfjs';

// Use existing TensorFlow.js tensors
const tfInput = tf.randomNormal([1, 224, 224, 3]);
const tfOutputs = await runWithTfjsTensors(liteRtModel, [tfInput]);
// Returns TensorFlow.js tensors compatible with existing pipeline

Example 3: Multiple Accelerator Fallback

JavaScript
let accelerator = 'webgpu';
try {
  const model = await loadAndCompile('/model.tflite', {accelerator});
} catch (e) {
  console.log('WebGPU not available, falling back to WASM');
  accelerator = 'wasm';
  const model = await loadAndCompile('/model.tflite', {accelerator});
}