iOS SDK

The Roboflow Mobile iOS SDK is a great option if you are developing an iOS application where having a model running on the edge (iPad or iPhone) is needed for faster inference or to unlock a new suite of features, capabilities, and use cases (like augmented reality).

Task Support

Deploy a Model to an iOS Device

Supported Hardware and Software

All iOS devices support on-device inference, but those older than the iPhone 8 (A11 Bionic Processor) will fall back to the less energy efficient GPU engine.

Roboflow requires a minimum iOS version of 15.4 (18.0 for instance segmentation models).

Prototyping

You can develop against the Roboflow Hosted Inference API. It uses the same trained models as on-device inference.

Installation

• Install CocoaPods | Troubleshooting Guide

Check that CocoaPods is successfully installed by entering pod --version in your Terminal.

Installing the Roboflow CocoaPod

First, run pod init in your project directory.

Make sure the Podfile specifies the platform :ios, '15.4'

Next, add pod 'Roboflow' to your Podfile.

If you do not have the XCode Command Line Tools installed, run xcode-select --install in your Terminal.

Lastly, run pod install and open the generated .xcworkspace file in XCode.

• If it returns this error: "You may have encountered a bug in the Ruby interpreter or extension libraries," then first run brew install cocoapods, and then run pod install and open the generated .xcworkspace file in XCode.

Using Roboflow in Swift

Navigate to the .xcworkspace file in XCode.

Next, import Roboflow by adding import Roboflow to the .xcworkspace file.

Then, create an instance of the Roboflow API with let rf = Roboflow(apiKey: "API_KEY"). For modelVersion, replace YOUR-MODEL-VERSION-# with the integer value of your model's version number.

Completion Handler Usage:

import Roboflow
...
//initialize with your API Key
let rf = RoboflowMobile(apiKey: "API_KEY")
var model: RFModel?
...

//model is your model's project name
rf.load(model: "YOUR-MODEL-ID", modelVersion: YOUR-MODEL-VERSION-#) { [self] model, error, modelName, modelType in
    if error != nil {
        print(error?.localizedDescription as Any)
    } else {
        model?.configure(threshold: threshold, overlap: overlap, maxObjects: maxObjects
                            processingMode: .performance or .balanced or .quality, // instance seg
                            maxNumberPoints: max number of mask processing points) // instance seg
        self.model = model
    }

}
...

//model?.detect takes a UIImage and runs inference on it
let img = UIImage(named: "example.jpeg") // or CVPixelBuffer
model?.detect(image: img!) { predictions, error in
    if error != nil {
        print(error)
    } else {
        print(predictions)
    }
}

Asynchronous Usage:

import Roboflow
...
//initialize with your API Key
let rf = RoboflowMobile(apiKey: "API_KEY")
...

//model is your model's project name
let (model, loadingError, modelName, modelType) = await rf.load(model: "YOUR-MODEL-ID", modelVersion: YOUR-MODEL-VERSION-#)
model!.configure(threshold: threshold, overlap: overlap, maxObjects: maxObjects)
...

//model?.detect takes a UIImage and runs inference on it
let img = UIImage(named: "example.jpeg")
let (predictions, predictionError) = await model!.detect(image: img!)
print(predictions)

Predictions Format:

x:Float //center of object x
y:Float //center of object y
width:Float
height:Float
className:String
confidence:Float
color:UIColor
box:CGRect
points:[CGPoint] // instance segmentation models only

Native Swift Example

The roboflow-ios-starter app is a great starting point for creating a realtime iOS app with roboflow models. It includes camera setup, model loading and process, and output drawing code for both object-detection and instance segmentation models.

React Native Expo App Example

We also provide an example of integrating this SDK into an expo app with React Native. You may find this useful when considering the construction of your own downstream application.

Be sure that you have both Expo and CocoaPods installed.

• RoboflowExpoExample on GitHub

Example iOS Application - CashCounter

Download CashCounter, our example iOS app that counts US coins and bills, as an example of how you could deploy a computer vision model to an iPhone.