Creating Inferencing System

Once the training or simulation is completed, you can the create and scale the inferencing system for production in a single click through ‘Create API’ button on model show page.

Creating Inferencing system

Create a inferencing system after training the model

API Profile

While creating the API profile, you can choose to launch the inferencing system on any epoch. This gives complete freedom of launching the inferencing server on any desired epoch.

API Profile

Fill the API profile

API Type

You can specify any number of layers from the network from for your API output.

Classification
API Type: classification

API type: Classification

Select this type if your network is trained for classification. You can select any or all the final layers (layers connected to Loss) from the network. The output of the API will be a dictionary with keys as layer names and value as a list in which each element will contain a label and its probability.For example, if the network is trained to classify between bags and wallets, and the final layer is “final_fc”, the output will look like:

{‘final_fc’: [{‘label’:‘bags’,‘prob’: 0.8536},{‘label’:‘wallet’,‘prob’:0.1464}]}
Basic
API Type: basic

API type: Basic

Select this type for all other use cases. You can select any or all layers from the network. The output of the API will be a dictionary with keys as layer names and value as a list or list of lists containing output of that layer. For example, if you select a layer “layer_n” with output dimensions 2x3, the output will look like:

{‘layer_n’: [[ 0.2349, 0.6134, 0.3897 ],[ 0.7892, 0.2398, 0.3478 ]]}

Choice of server

You can choose to launch on servers as per the production requirement. To know more about the Servers and cost, refer to Servers and Pricing;

Servers

Servers

Inferencing/API System

After launching the server, it’ll take 10 to 15mins depending on the type of server to start the server.

Usage Analytics

Once the server is up, you will be navigated to API Analytics page. This dashboard outlines the usage of the API Calls and the latency details. It can also display the real time metrics.

Usage Analytics

Usage Analytics

Demo

You can access the demo of the API from the dashboard.

Demo

Demo

Token

To access the API details, click on the ‘API Token’.

Token

Token