Web inference.js

inferencejs is a JavaScript package that enables real-time inference via the browser using models trained on Roboflow.

Note

See the inferencejs reference here.

For most business applications, the Serverless API is suitable. But for many consumer applications and some enterprise use cases, having a server-hosted model is not workable (for example, if your users are bandwidth constrained or need lower latency than you can achieve using a remote API).

Learning Resources

Try Your Model With a Webcam: You can try out a webcam demo of a hand-detector model here (it is trained on the public EgoHands dataset).
Interactive Replit Environment: We have published a "Getting Started" project on Repl.it with an accompanying tutorial showing how to deploy YOLOv8 models using our Repl.it template.
GitHub Template: The Roboflow homepage uses inferencejs to power the COCO inference widget. The README contains instructions on how to use the repository template to deploy a model to the web using GitHub Pages.
Documentation: If you would like more details regarding specific functions in inferencejs, check out our documentation page.

Supported Models

inferencejs currently supports these model architectures:

RF-DETR
Roboflow 3.0 (YOLOv8-compatible)
YOLOv5
Gaze Detection

Installation

To add inferencejs to your project, install using npm or add the script tag reference to your page's <head> tag.

npm install inferencejs

<script src="https://cdn.jsdelivr.net/npm/inferencejs"></script>

Initializing inferencejs

Authenticating

You can obtain your publishable_key from the Roboflow workspace settings.

Warning

Your publishable_key is used with inferencejs, not your private API key (which should remain secret).

Start by importing InferenceEngine and creating a new inference engine object.

Note

inferencejs uses webworkers so that multiple models can be used without blocking the main UI thread. Each model is loaded through the InferenceEngine, our webworker manager that abstracts the necessary thread management for you.

import { InferenceEngine } from "inferencejs";
const inferEngine = new InferenceEngine();

Now we can load models from Roboflow using your publishable_key and the model metadata (model name and version) along with configuration parameters like confidence threshold and overlap threshold.

const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]");

inferencejs will now start a worker that runs the chosen model. The returned worker id corresponds with the worker id in InferenceEngine that we will use for inference. To infer on the model we can invoke the infer method on the InferenceEngine.

Let's load an image and infer on our worker.

const image = document.getElementById("image"); // get image element with id `image`
const predictions = await inferEngine.infer(workerId, image); // infer on image

Note

This can take in a variety of image formats (HTMLImageElement, HTMLVideoElement, ImageBitmap, or TFJS Tensor).

This returns an array of predictions (as a class, in this case RFObjectDetectionPrediction).

Configuration

If you would like to customize and configure the way inferencejs filters its predictions, you can pass parameters to the worker on creation.

const configuration = {scoreThreshold: 0.5, iouThreshold: 0.5, maxNumBoxes: 20};
const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]", configuration);

Or you can pass configuration options on inference:

const configuration = {
    scoreThreshold: 0.5,
    iouThreshold: 0.5,
    maxNumBoxes: 20
};
const predictions = await inferEngine.infer(workerId, image, configuration);