inferencejs Reference

Installation

This library is designed to be used within the browser, using a bundler such as Vite, webpack, Parcel, etc. Assuming your bundler is set up, you can install by executing:

npm install inferencejs

Getting Started

Begin by initializing the InferenceEngine. This will start a background worker which is able to download and execute models without blocking the user interface.

import { InferenceEngine } from "inferencejs";

const PUBLISHABLE_KEY = "rf_a6cd..."; // replace with your own publishable key from Roboflow

const inferEngine = new InferenceEngine();
const workerId = await inferEngine.startWorker("[PROJECT URL SLUG]", [VERSION NUMBER], PUBLISHABLE_KEY);

//make inferences against the model
const result = await inferEngine.infer(workerId, img);

API

InferenceEngine

new InferenceEngine()

Creates a new InferenceEngine instance.

startWorker(modelName: string, modelVersion: number, publishableKey: string): Promise<number>

Starts a new worker for the given model and returns the workerId. publishableKey is required and can be obtained from Roboflow in your project settings folder.

infer(workerId: number, img: CVImage | ImageBitmap): Promise<Inference>

Infer on an image using the worker with the given workerId. img can be created using new CVImage(HTMLImageElement | HTMLVideoElement | ImageBitmap | TFJS.Tensor) or createImageBitmap.

stopWorker(workerId: number): Promise<void>

Stops the worker with the given workerId.

YOLOv8 / YOLOv5

The result of making an inference using the InferenceEngine on a YOLOv8 or YOLOv5 object detection model is an array of the following type:

type RFObjectDetectionPrediction = {
    class?: string;
    confidence?: number;
    bbox?: {
        x: number;
        y: number;
        width: number;
        height: number;
    };
    color?: string;
};

GazeDetections

The result of making an inference using the InferenceEngine on a Gaze model. An array with the following type:

type GazeDetections = {
    leftEye: { x: number; y: number };
    rightEye: { x: number; y: number };
    yaw: number;
    pitch: number;
}[];

• leftEye.x — The x position of the left eye as a floating point number between 0 and 1, measured in percentage of the input image width.
• leftEye.y — The y position of the left eye as a floating point number between 0 and 1, measured in percentage of the input image height.
• rightEye.x — The x position of the right eye as a floating point number between 0 and 1, measured in percentage of the input image width.
• rightEye.y — The y position of the right eye as a floating point number between 0 and 1, measured in percentage of the input image height.
• yaw — The yaw of the visual gaze, measured in radians.
• pitch — The pitch of the visual gaze, measured in radians.

CVImage

A class representing an image that can be used for computer vision tasks. It provides various methods to manipulate and convert the image.

Constructor

The CVImage(image) class constructor initializes a new instance of the class. It accepts one image of one of the following types:

• ImageBitmap: An optional ImageBitmap representation of the image.
• HTMLImageElement: An optional HTMLImageElement representation of the image.
• tf.Tensor: An optional tf.Tensor representation of the image.
• tf.Tensor4D: An optional 4D tf.Tensor representation of the image.

Methods

• bitmap() — Returns a promise that resolves to an ImageBitmap representation of the image.
• tensor() — Returns a tf.Tensor representation of the image.
• tensor4D() — Returns a promise that resolves to a 4D tf.Tensor representation of the image.
• array() — Returns a promise that resolves to a JavaScript array representation of the image.
• dims() — Returns an array containing the dimensions of the image. If the image is a bitmap, it returns [width, height]. If the image is a tensor, it returns the shape of the tensor.
• dispose() — Disposes of the tensor representations of the image to free up memory.
• static fromArray(array: tf.TensorLike) — Creates a new CVImage instance from a given tensor-like array.