Getting Started with Scale

As part of Scale’s mission to democratize access to intelligent data, we’re committed to making high quality, human-powered training data accessible for anyone in the AI development community. That’s why we provide two ways for customers to work with us: On-Demand and Enterprise engagements. The On-Demand model means no commitment requirements, no platform fees, and you pay-as-you-go. Furthermore, with On-Demand, you can just sign up and send in tasks via our developer-friendly API. After a combination of human work and review, smart tools, and statistical confidence checks and machine learning checks, we’ll return scalable, accurate training data. It’s a great way to get started and experience Scale’s ease-of-use and quality.

Ready to get started? Follow the 4 steps below and you’ll be well on your way to building a robust training dataset for your next AI application.

Sign Up

If you haven’t already, create a Scale account. When prompted, input your credit card information to activate your account. You’ll only be charged for completed tasks under our On-Demand rates (your first 5 tasks are free), which are based on the task type, volume of data and number of annotations with each task.

Note: On-demand tasks are completed best-effort for quality and response urgency. For quality & turnaround SLAs, chat with us about our Enterprise engagements.

Design Your Task

To maximize the quality of your task, we recommend writing clear and concise instructions that include:

Step-by-step directions

Rules and example images (i.e. minimum label size, when to not label an object)

Label classes/fields with definitions and example images

Screenshots of correct and incorrect tasks for comparison

Tip: To allow you to collaborate on instructions with your team and embed formatting and images, create your instructions in a shared document via Google Docs and embed it into the API call via markdown:

View a more granular list of your submitted tasks, filterable by status, project, and batch. If you click a specific task, you can view the original API call. For completed tasks, there are additional options that will allow you Audit the task, Resend Callback, and download the JSON response. You can also mass export the JSON responses for all your tasks via the Export JSON link.

Tip: Use our Teams feature to allow multiple users to have access to the same set of tasks.

Audit a random sample of tasks for quality and view stats for tasks you’ve reviewed. When you audit a task, we give you access to our annotation application where you can see the original source data with completed labels. You can Approve or Reject the task, and provide feedback if the task was incorrect.

If you any questions, feel free to join our Slack channel here or ping us on our web chat.
If you’re interested in learning more about our Enterprise engagements, get in touch with us.

Client Libraries / SDKs

To install the client library, use the following command:

pipinstall--upgradescaleapi

npminstallscaleapi--save

geminstallscaleapi

Currently we have client libraries available for the following languages:

You must replace {{ApiKey}} with your personal API key. If you sign up or log in, your API key will be automatically filled in the docs.Your test API key {{ApiKey}} is included in all the examples on this page, so you can test any example right away. Only you can see this value.

Scale expects for the API key to be included in all API requests to the server via HTTP Basic Auth. Provide your API key as the basic auth username value. You do not need to provide a password. You can do this using the -u flag:

-u "{{ApiKey}}:"

You must replace {{ApiKey}} with your personal API key. If you sign up or log in, your API key will be automatically filled in the docs.
Your test API key {{ApiKey}} is included in all the examples on this page, so you can test any example right away. Only you can see this value.

Test and Live Modes

To make the API as explorable as possible, accounts have test mode and live mode API keys. There is no “switch” for changing between modes, just use the appropriate key to perform a live or test API requests.

Requests made with test mode credentials are not completed by a human, and therefore have incorrect test responses. Requests made with live mode credentials are always completed by a human, and will incur a charge past your first 5 API requests.

Callback Authentication

If you’d like to authenticate our callbacks, we set a scale-callback-auth HTTP header on each of our callbacks. The value will be equal to your Live Callback Auth Key shown on your dashboard. If this header is not set, or it is set incorrectly, the callback is not from Scale.

Task Object

The task object represents a single task that you create with Scale and is completed by a worker.

Attributes

{"task_id":"576ba74eec471ff9b01557cc","completed_at":"2016-06-23T09:10:02.798Z","created_at":"2016-06-23T09:09:34.752Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"completed","instruction":"Would you say this item is big or small?","params":{"attachment_type":"text","attachment":"car","categories":["big","small"]},"callback_succeeded":true,"response":{"category":"big"},"metadata":{}}

Attribute

Type

Description

task_id

string

The task_id is the unique identifier for the task.

type

string

The type of the task. Currently, we support categorization, transcription, audiotranscription, comparison, annotation, and datacollection.

instruction

string

A markdown-enabled string explaining the instructions for the task. You can use markdown to show example images, give structure to your instructions, and more. HTML tags are unsupported.

params

object

An object with the parameters of the task based on the type. For categorization, for example, this will include attachment_type, attachment, and categories.

response

object

An object corresponding to the response once the task is completed. For categorization, it will have the attribute category, corresponding to the chosen category.

callback_url

string

A string of the URL that should be POSTed once the task is completed for the response data. See the Callback section for more details.

status

string

The status of the task, one of pending, completed, canceled or error.

created_at

timestamp

A string of the UTC timestamp of when the task was created.

completed_at

timestamp

A string of the UTC timestamp of when the task was completed. This will only be filled in after it is completed.

callback_succeeded

boolean

A boolean stating whether or not the callback succeeded. If the callback returns with a 2xx status code, the value will be true. If the callback fails to return a 2xx status code through all retries, then the value will be false.

customer_review_status

string

The status of the QA’d task, one of pending, accepted, fixed or rejected.

metadata

object, default {}

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Metadata

Tasks objects have a metadata parameter. You can use this parameter to attach key-value data to tasks.

Metadata is useful for storing additional, structured information on an object. As an example, you could store a video’s unique identifier in your system on its content moderation categorization task. You can also use it to denote the end use case for the task, as “content moderation” or “data categorization” for example. Metadata is not used by Scale (e.g., to affect how the task is done).

Attachments

Callback example for bad attachment URL:

{"error":"One or more attachments could not be downloaded.","attachments":[{"statusCode":500,"url":"https://your-s3-bucket.s3.amazonaws.com/attachment-1.png"},{"statusCode":500,"url":"https://your-s3-bucket.s3.amazonaws.com/attachment-2.png"}]}

Tasks often require a file associated with them that Scale calls an attachment. For example, an annotation task requires an image file to show to labelers. These attachments are not limited to images and may also be audio, video, or pdf files, or a website, or even plain text. For all attachment types except plain text, the attachment must be specified via a URL. See the specific task documentation below for some examples.

Currently we support fetching URLs over HTTP or HTTPS. Make sure that the URL is a direct link to the file you wish to use as an attachment and not a document showing a preview of the file. For certain storage providers we may be able to rewrite URLs that are previews of the file you are trying to attach to the correct direct download URL. HTTPS is preferred over HTTP, as HTTPS guarantees data integrity, preventing against corruption.

In addition to fetching using HTTP/HTTPS urls, we can also directly retrieve attachments from an S3 or Google Cloud Storage bucket. See S3 IAM Access or GCP IAM Access for more information.

Scale will attempt to fetch the attachment via the URL you provide. If we do not receive an HTTP 200 response when attempting to fetch your attachment(s), we will send your callback_url an error with the bad HTTP codes we received, and the task status will be error.

For audio and video attachments we support the containers and formats listed on this Chromium project page; for image attachments we support JPEG, PNG, BMP, GIF, and TIFF file formats up to a maximum of 10000 pixels in width or height.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_annotation_task(callback_url='http://www.example.com/callback',instruction='Draw a box around each **car** and **pedestrian**',attachment_type='image',attachment='http://i.imgur.com/XOJbalC.jpg',objects_to_annotate=['car','pedestrian'],with_labels=True,min_width='30',min_height='30')

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createAnnotationTask({'callback_url':'http://www.example.com/callback','instruction':'Draw a box around each **car** and **pedestrian**','attachment_type':'image','attachment':'http://i.imgur.com/XOJbalC.jpg','objects_to_annotate':['car','pedestrian'],'with_labels':true,'min_width':'30','min_height':'30'},(err,task)=>{// do something with task});

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"annotation","status":"pending","instruction":"Draw a box around each **car** and **pedestrian**","params":{"with_labels":true,"min_width":30,"min_height":30,"objects_to_annotate":["car","pedestrian"],"attachment_type":"image","attachment":"http://i.imgur.com/XOJbalC.jpg"},"metadata":{}}

This endpoint creates an annotation task. In this task, one of our labelers will view the given image and draw bounding boxes around the specified objects, returning the positions and sizes of these boxes.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

It is recommended, but not required, for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell labelers to only annotate objects whose bounding boxes are of dimension at least min_width x min_height.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/annotation

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

[string]

An array of strings describing which objects you’d like bounding boxes to be drawn around. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 7 objects. If you need more objects, try using nested annotation labels.

attachment

string

A URL to the image you’d like to be annotated with bounding boxes.

with_labels (optional, default true)

boolean

Specifies whether you’d like labels for each bounding box in the response. Each label will be a member of the objects_to_annotate array.

min_height (optional)

integer, default 0

The minimum height in pixels of the bounding boxes you’d like to be made.

min_width (optional)

integer, default 0

The minimum width in pixels of the bounding boxes you’d like to be made.

instruction (optional)

string

A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type (optional, default image)

string

Describes what type of file the attachment is. We currently only support image for the annotation endpoint.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have either an error field or an annotations field.

If the annotation was completed successfully, the annotations field will contain an array of annotations. Each annotation will have the following values:

left: The distance, in pixels, between the left border of the bounding box and the left border of the image.

top: The distance, in pixels, between the top border of the bounding box and the top border of the image.

width: The width, in pixels, of the bounding box.

height: The height, in pixels, of the bounding box.

label (if with_labels is true): The label for the bounding box, which will be one of the specified task.params.objects_to_annotate.

Visual representations of the task responses are accessible via the Scale dashboard. In these, the border is drawn inside the bounding box. The example response body (at right) corresponds to the image below.

If the attachment was invalid, the error will be detailed in the error field.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_pointannotation_task(callback_url='http://www.example.com/callback',instruction='Draw a point on every **headlight** and **brakelight** of a car in the image.',attachment_type='image',attachment='http://i.imgur.com/XOJbalC.jpg',objects_to_annotate=['headlight','brakelight'],with_labels=True)

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createPointannotationTask({'callback_url':'http://www.example.com/callback','instruction':'Draw a point on every **headlight** and **brakelight** of a car in the image.','attachment_type':'image','attachment':'http://i.imgur.com/XOJbalC.jpg','objects_to_annotate':['headlight','brakelight'],'with_labels':true},(err,task)=>{// do something with task});

require'scale'scale=Scale.new(api_key: '{{ApiKey}}')scale.create_pointannotation_task({callback_url: 'http://www.example.com/callback',instruction: 'Draw a point on every **headlight** and **brakelight** of a car in the image.',attachment_type: 'image',attachment: 'http://i.imgur.com/XOJbalC.jpg',objects_to_annotate: ['headlight','brakelight'],with_labels: true})=>#<Scale::Api::Tasks::Pointannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="pointannotation", @instruction="Draw a point on every **headlight** and **brakelight** of a car in the image.", @params={"with_labels"=>true, "objects_to_annotate"=>["headlight", "brakelight"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"},@response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"pointannotation","status":"pending","instruction":"Draw a point on every **headlight** and **brakelight** of a car in the image.","params":{"with_labels":true,"objects_to_annotate":["headlight","brakelight"],"attachment_type":"image","attachment":"http://i.imgur.com/XOJbalC.jpg"},"metadata":{}}

This endpoint creates a pointannotation task. In this task, one of our labelers will view the given image and draw points at the specified locations, returning the locations of these points.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated. You may include at most 6 objects in the objects_to_annotate parameter. If you need more objects, try using nested annotation labels.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/pointannotation

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

[string]

An array of strings describing which objects you’d like points to be drawn on. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 6 objects. If you need more objects, try using nested annotation labels.

attachment

string

A URL to the image you’d like to be annotated with points.

with_labels (optional, default true)

boolean

Specifies whether you’d like labels for each point in the response. Each label will be a member of the objects_to_annotate array.

instruction (optional)

string

A markdown-enabled string explaining how to draw the points. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type (optional, default image)

string

Describes what type of file the attachment is. We currently only support image for the point annotation endpoint.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_lineannotation_task(callback_url='http://www.example.com/callback',instruction='Annotate lines over all of the **lane lines** in the street-level image.',attachment_type='image',attachment='http://i.imgur.com/XOJbalC.jpg',objects_to_annotate=['solid line','dashed line'],with_labels=True)

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createLineannotationTask({'callback_url':'http://www.example.com/callback','instruction':'Annotate lines over all of the **lane lines** in the street-level image.','attachment_type':'image','attachment':'http://i.imgur.com/XOJbalC.jpg','objects_to_annotate':['solid line','dashed line'],'with_labels':true},(err,task)=>{// do something with task});

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"lineannotation","status":"pending","instruction":"Annotate lines over all of the **lane lines** in the street-level image.","params":{"with_labels":true,"objects_to_annotate":["solid line","dashed line"],"attachment_type":"image","attachment":"http://i.imgur.com/XOJbalC.jpg"},"metadata":{}}

This endpoint creates a lineannotation task. In this task, one of our labelers will view the given image and draw segmented lines along each object, returning the vertices of these segmented lines.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

If you’d prefer splines as opposed to segmented lines, then you may specify the splines flag as true.

You can also optionally set min_vertices and/or max_vertices which specify the minimum and maximum number of vertices that you would want on a line. Note the range is inclusive of both bounds. For example, if min_vertices is 2 and max_vertices is 2, then Scale will only return line segments.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/lineannotation

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

[string]

An array of strings describing which objects you’d like segmented lines to be drawn along. Each string should be singular and self-descriptive (ex: “lane line”, “crop line”). You may include at most 3 objects. If you need more objects, try using nested annotation labels.

attachment

string

A URL to the image you’d like to be annotated with segmented lines.

with_labels (optional, default true)

boolean

Specifies whether you’d like labels for each segmented line in the response. Each label will be a member of the objects_to_annotate array.

instruction (optional)

string

A markdown-enabled string explaining how to draw the segmented lines. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type (optional, default image)

string

Describes what type of file the attachment is. We currently only support image for the line annotation endpoint.

splines (optional, default false)

boolean

Specifies whether or not you’d like your lines drawn as cardinal splines instead of segmented lines

min_vertices (optional, default 1)

number

An optional parameter defining the minimum number of vertices in a valid line annotation for your request.

max_vertices (optional, default null)

number

An optional parameter defining the maximum number of vertices in a valid line annotation for your request. Must be at least min_vertices.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_cuboidannotation_task(callback_url='http://www.example.com/callback',instruction='Draw a cuboid around each car or truck.',attachment_type='image',attachment='http://i.imgur.com/v4cBreD.jpg',objects_to_annotate=['headlight','brakelight'],with_labels=True)

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCuboidannotationTask({'callback_url':'http://www.example.com/callback','instruction':'Draw a cuboid around each car or truck.','attachment_type':'image','attachment':'http://i.imgur.com/v4cBreD.jpg','objects_to_annotate':['car','truck'],},(err,task)=>{// do something with task});

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"cuboidannotation","status":"pending","instruction":"Draw a cuboid around each car or truck.","params":{"objects_to_annotate":["car","truck"],"attachment_type":"image","attachment":"http://i.imgur.com/XOJbalC.jpg"},"metadata":{}}

This endpoint creates a cuboidannotation task. Given a 2D image, and camera intrinsics and extrinsics as optional parameters, Scale will annotate the image with perspective cuboids and return the vertices in the image of these cuboids. If camera intrinsics and extrinsics are provided as well, Scale will return scale-invariant 3D coordinates with respect to the camera, i.e. assuming the camera is at the origin.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachment is a URL to an image you’d like to be annotated. objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell labelers to only annotate objects whose cuboids are of dimension at least min_width x min_height.

Please refer to the camera_rotation_quaternion and camera_intrinsics parameters if you’re interested in getting perspective cuboids and 3D coordinates.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/cuboidannotation

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

Array<string>

An array of strings describing which objects you’d like cuboids to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects. If you need more objects, try using nested annotation labels.

attachment

string

A URL to the image you’d like to be annotated with perspective cuboids.

instruction (optional)

string

A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more.

camera_intrinsics (optional)

object

Object defining camera intrinsics, in format {fx: 1300, fy: 1300, cx: 600, cy: 300, scalefactor: 2.75, skew: 0} (skew defaults to 0, scalefactor defaults to 1). scalefactor is used if the image sent is of different dimensions from the original photo (if the attachment is half the original, set scalefactor to 2) to correct the focal lengths, offsets. Use in conjunction with camera_rotation_quaternion to get perspective-corrected cuboids and 3d points.

camera_rotation_quaternion (optional)

object

Object defining the rotation of the camera in relation to the world. Expressed as a quaternion, in format {w: 0.5, x: 0.5, y: 0.5, z: 0.5}. Use in conjunction with camera_intrinsics to get perspective-corrected cuboids and 3d points. Note that the z-axis of the camera frame represents the camera’s optical axis.

camera_height

number

Height of camera above the ground, in meters.

min_height (optional, default 0)

integer

The minimum height in pixels of the bounding cuboids you’d like to be made.

min_width (optional, default 0)

integer

The minimum width in pixels of the bounding cuboids you’d like to be made.

min_annotations (optional, default 0)

integer

The minimum number of cuboids that will be returned. Should only be used when one is certain that the image contains at least this many annotatable cuboids.

max_annotations (optional)

default no limit

The maximum number of cuboids that can be annotated in the image. A value of 0 (default) means no limit.

attachment_type (optional, default image)

string

Describes what type of file the attachment is. We currently only support image for the annotation endpoint.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

Response & Callback Format

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain only an annotations field.

The annotations field will contain an array of Cuboid objects. We define the Vertex and Cuboid objects below:

Definition: Vertex objects

Vertex objects are represented as dictionaries with the following structure:

Key

Description

x

The distance, in pixels, between the vertex and the left border of the image.

y

The distance, in pixels, between the vertex and the top border of the image.

type

This will be “vertex” for all points

description

An plaintext description of this vertex

This is the list of all possible descriptions for a vertex:

face-topleft

face-bottomleft

face-topright

face-bottomright

side-topcorner

side-bottomcorner

Definition: Edge objects

Edge objects are represented as dictionaries with the following structure:

Key

Description

x1

The distance, in pixels, between the first vertex and the left border of the image.

y1

The distance, in pixels, between the first vertex and the top border of the image.

x2

The distance, in pixels, between the second vertex and the left border of the image.

y2

The distance, in pixels, between the second vertex and the top border of the image.

type

This will be “edge” for all edges

description

An plaintext description of this edge

This is the list of all possible descriptions for an edge:

face-top

face-bottom

face-left

face-right

side-top

side-bottom

Definition: Cuboid objects

Points on cuboid are returned in this order for both points_2d and points_3d:
3-------2
/| /|
/ | / |
0-------1 |
| 7----|--6
| / | /
4-------5

Cuboid objects are represented as dictionaries with the following structure:

Key

Description

label

string belonging to objects_to_annotate which describes what kind of object the cuboid annotates

vertices

A list of Vertex objects defining all visible vertices of the cuboid.

edges

A list of Edge objects defining the edges of the cuboid.

points_2d (optional)

If camera_rotation_quaternion, camera_intrinsics were provided, gives 2d points of all 8 vertices of the cuboid after perspective correction.

points_3d (optional)

If camera_rotation_quaternion, camera_intrinsics were provided, gives 3d coordinates (arbitrarily scaled, relative to the camera location) of all 8 vertices of the cuboid after perspective correction.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_polygonannotation_task(callback_url='http://www.example.com/callback',instruction='Draw a tight polygon around every **car** in the image.',attachment_type='image',attachment='http://i.imgur.com/XOJbalC.jpg',objects_to_annotate=['car','truck'],with_labels=True)

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createPolygonannotationTask({'callback_url':'http://www.example.com/callback','instruction':'Draw a tight polygon around every **car** in the image.','attachment_type':'image','attachment':'http://i.imgur.com/XOJbalC.jpg','objects_to_annotate':['car','truck'],'with_labels':true},(err,task)=>{// do something with task});

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"polygonannotation","status":"pending","instruction":"Draw a tight polygon around every **car** in the image.","params":{"with_labels":true,"objects_to_annotate":["car","truck"],"attachment_type":"image","attachment":"http://i.imgur.com/XOJbalC.jpg"},"metadata":{}}

This endpoint creates a polygonannotation task. In this task, one of our labelers will view the given image and draw bounding polygons around the specified objects, returning the vertices of these polygons.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

You can also optionally set min_vertices and/or max_vertices which specify the minimum and maximum number of vertices that you’d like on a polygon. Note the range is inclusive of both bounds. For example, if min_vertices is 3 and max_vertices is 5, then Scale will only return triangles, quadrilaterals, and pentagons.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/polygonannotation

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

[string]

An array of strings describing which objects you’d like bounding polygons to be drawn around. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 6 objects. If you need more objects, try using nested annotation labels.

attachment

string

A URL to the image you’d like to be annotated with bounding polygons.

with_labels (optional, default true)

boolean

Specifies whether you’d like labels for each bounding polygon in the response. Each label will be a member of the objects_to_annotate array.

instruction (optional)

string

A markdown-enabled string explaining how to draw the bounding polygons. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type (optional, default image)

string

Describes what type of file the attachment is. We currently only support image for the polygon annotation endpoint.

min_vertices (optional, default 1)

number

An optional parameter defining the minimum number of vertices in a valid polygon annotation for your request.

max_vertices (optional, default null)

number

An optional parameter defining the maximum number of vertices in a valid polygon annotation for your request. Must be at least min_vertices.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_segmentannotation_task(callback_url='http://www.example.com/callback',instruction='Please segment the image using the given labels.',attachment_type='image',attachment='http://i.imgur.com/XOJbalC.jpg',labels=['background','road','vegetation','lane marking'],instance_labels=['vehicle','pedestrian'],allow_unlabeled=False)

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createSegmentannotationTask({callback_url:'http://www.example.com/callback',instruction:'Please segment the image using the given labels.',attachment_type:'image',attachment:'http://i.imgur.com/XOJbalC.jpg',labels:['background','road','vegetation','lane marking'],instance_labels:['vehicle','pedestrian'],allow_unlabeled:false},(err,task)=>{// do something with task});

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"segmentannotation","status":"pending","instruction":"Please segment the image using the given labels.","params":{"allow_unlabeled":false,"labels":["background","road","vegetation","lane marking"],"instance_labels":["vehicle","pedestrian"],"attachment_type":"image","attachment":"http://i.imgur.com/XOJbalC.jpg"},"metadata":{}}

This endpoint creates a segmentannotation task. In this task, one of our labelers will view the given image and classify every pixel of the image according to the labels provided. You will get a full semantic, pixel-wise, dense segmentation of the image.

We also support instance-aware semantic segmentations, also called panoptic segmentation, via the instance_labels parameter.

The required parameters for this task are callback_url, attachment, and labels. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be segmented.

labels is an array of strings describing the different types of objects you’d like to be used to segment the image.

You can also optionally set allow_unlabeled to true, which will allow the existence of unlabeled pixels in the segmentation response - otherwise, all pixels in the image will be classified (in which case it’s important that there are labels for everything in the image, to avoid misclassification).

The response you will receive will be a series of images where each pixel’s value corresponds to the label, either via a numerical index or a color mapping. You will also get separate masks for each label for convenience.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/segmentannotation

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

labels

Array<string>

An array of strings describing the different types of objects you’d like to be used to segment the image. Each string should be singular and descriptive (ex: car, background, pole). You may include at most 50 labels.

instance_labels (optional, default [])

Array<string>

An array of strings describing the different types of objects you’d like to be segmented on a per-instance basis. In the case of instance_labels, the objects will get different masks / colors for each instance. For example, if you defined car as an instance_label, each individual car would get a separate mask in the image, allowing you to distinguish between them. Instance labels cannot overlap with standard labels. You may include at most 10 instance labels. If you need more instance labels, try using nested annotation labels.

attachment

string

A URL to the image you’d like to be segmented.

allow_unlabeled (optional, default false)

boolean

Specifies whether you’ll allow unlabeled pixels in the segmentation. If left with the default value of false, all pixels in the image will be classified using the labels you provided.

instruction (optional)

string

A markdown-enabled string explaining how carry out the semantic segmentation. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type (optional, default image)

string

Describes what type of file the attachment is. We currently only support image for the semantic segmentation endpoint.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a labelMapping and an annotations field.

labelMapping is a dictionary where the keys are each label name, and the value is an object with the index and color used to represent each label in the response images. In the case of instance_labels, the value will be an array of such objects with index and color keys, representing each instance.

annotations is an object that contains URLs of PNG images describing the segmentation result in different ways:

combined: an object containing the URLs of two PNG images describing the segmentation response:

image: a 4-channel PNG image URL where each non-transparent colored pixel represents a pixel annotated with a given label, and each transparent pixel represents an unlabeled one. The color values in the labelMapping object are used.

indexedImage: a 1-channel PNG image URL where each positive integer value represents a pixel annotated with a given label, and each 0 value represents an unlabeled pixel. The index values in the labelMapping object are used.

labeled: a dictionary where the keys are each label name, and the values are either:

a 4-channel PNG image URL where each non-transparent colored pixel (using the label’s color) represents a pixel annotated with that label in the segmentation response, or

null, if there is no pixel in the image annotated with that label in the segmentation response

In the case of an instance_label, an array of 4-channel PNG image URLs where each non-transparent colored pixel (using the label’s color) represents a pixel annotated with that label in the segmentation response. Each element of the array will represent a different instance of the label.

unlabeled:

a 4-channel PNG image URL where each non-transparent black (#000000) pixel represents a pixel left unlabeled in the segmentation response, or

null, if there is no pixel left unlabeled in the segmentation response

importrequestsimportjsonpayload={'callback_url':'http://www.example.com/callback','instruction':'Please track each object','attachment_type':'image','attachments':['https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png',],'objects_to_annotate':['car','pedestrian','cyclist'],'annotation_attributes':{'location':{'conditions':{'label_condition':{'label':['cyclist']}},'description':'Where is the cyclist?','choices':['Sidewalk','Bike lane','Shared lane','Road','Other']}},'with_labels':True,'min_width':'30','min_height':'30',}headers={"Content-Type":"application/json"}task_request=requests.post("https://api.scale.com/v1/task/videoboxannotation",json=payload,headers=headers,auth=('YOUR_SCALEAPI_KEY',''))printtask_request.json()

// please see the Python example

# please see the Python example

The above command returns an object structured like this:

{"callback_url":"http://www.example.com/callback","created_at":"2018-03-02T23:45:17.821Z","instruction":"Please track each object","is_test":false,"metadata":{},"params":{"annotation_attributes":{"location":{"choices":["Sidewalk","Bike lane","Shared lane","Road","Other"],"conditions":{"label_condition":{"label":["cyclist"]}},"description":"Where is the cyclist?"}},"attachment_type":"image","attachments":["https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png"],"min_height":30,"min_width":30,"objects_to_annotate":["car","pedestrian","cyclist"],"with_labels":true},"status":"pending","task_id":"5a99e20de50d4979ce6d291e","type":"videoboxannotation"}

This endpoint creates a videoboxannotation task. Given a series of images sampled from a video, (which we will now refer to as “frames”), Scale will annotate each frame with bounding boxes.

The required parameters for this task are callback_url, attachments, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachments are a list of URLs to the frames you’d like to be annotated. objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell labelers to only annotate objects whose boxes are of dimension at least min_width x min_height.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.com/v1/task/videoboxannotation

Parameters

Parameter

Type

Description

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

Array<string>

An array of strings describing which objects you’d like boxes to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects.

instruction (optional)

string

A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more.

min_height (optional, default 0)

integer

The minimum height in pixels of the bounding boxes you’d like to be made.

min_width (optional, default 0)

integer

The minimum width in pixels of the bounding boxes you’d like to be made.

Response & Callback Format

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a link to the annotations file containing a list of annotations per frame. Each annotation object contains its unique id to be used for tracking an object as well as its position, label, and attributes.

The distance, in pixels, between the vertex and the left border of the image.

top

number

The distance, in pixels, between the vertex and the top border of the image.

width

number

The width, in pixels, of the bounding box.

height

number

The height, in pixels, of the bounding box.

The annotations file will be stored on S3 and be a JSON encoded list of annotation objects per frame. The index of each annotation list will correspond to the frame defined in the original response.

The order of the annotations lists will correspond directly to the order of the frame attachments that you sent in your original request. That is, the first annotation list will correspond to the first frame in the attachments array, etc.

Once the JSON file is reviewed and uploaded, we will send a response to your callback URL.

importrequestsimportjsonpayload={'callback_url':'http://www.example.com/callback','instruction':'Please track each object','attachment_type':'image','attachments':['https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png','https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png',],'objects_to_annotate':['car','pedestrian','cyclist'],'annotation_attributes':{'location':{'conditions':{'label_condition':{'label':['cyclist']}},'description':'Where is the cyclist?','choices':['Sidewalk','Bike lane','Shared lane','Road','Other']}},'with_labels':True,'min_width':'30','min_height':'30',}headers={"Content-Type":"application/json"}task_request=requests.post("https://api.scale.com/v1/task/videocuboidannotation",json=payload,headers=headers,auth=('YOUR_SCALEAPI_KEY',''))printtask_request.json()

// please see the Python example

# please see the Python example

The above command returns an object structured like this:

{"callback_url":"http://www.example.com/callback","created_at":"2018-03-02T23:45:17.821Z","instruction":"Please track each object","is_test":false,"metadata":{},"params":{"annotation_attributes":{"location":{"choices":["Sidewalk","Bike lane","Shared lane","Road","Other"],"conditions":{"label_condition":{"label":["cyclist"]}},"description":"Where is the cyclist?"}},"attachment_type":"image","attachments":["https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png","https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png"],"min_height":30,"min_width":30,"objects_to_annotate":["car","pedestrian","cyclist"],"with_labels":true},"status":"pending","task_id":"5a99e20de50d4979ce6d291e","type":"videocuboidannotation"}

This endpoint creates a videocuboidannotation task. Given a series of images sampled from a video, (which we will now refer to as “frames”), Scale will annotate each frame with cuboids.

The required parameters for this task are callback_url, attachment or attachments, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachments are a list of URLs to the frames you’d like to be annotated. The attachment parameter is a URL pointing to the video file you’d like th be annotated. objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell labelers to only annotate objects whose cuboids are of dimension at least min_width x min_height.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

Parameters

Parameter

Type

Description

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

objects_to_annotate

Array<string>

An array of strings describing which objects you’d like cuboids to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects.

instruction (optional)

string

A markdown-enabled string explaining how to draw the bounding cuboids. You can use markdown to show example images, give structure to your instructions, and more.

min_height (optional, default 0)

integer

The minimum height in pixels of the bounding cuboids you’d like to be made.

min_width (optional, default 0)

integer

The minimum width in pixels of the bounding cuboids you’d like to be made.

Object defining camera intrinsics, in format {fx: 1300, fy: 1300, cx: 600, cy: 300, scalefactor: 2.75, skew: 0} (skew defaults to 0, scalefactor defaults to 1). scalefactor is used if the image sent is of different dimensions from the original photo (if the attachment is half the original, set scalefactor to 2) to correct the focal lengths, offsets. Use in conjunction with camera_rotation_quaternion to get perspective-corrected cuboids and 3d points. This value will be applied for all frames in the video.

camera_height

number

Height of camera above the ground, in meters.

camera_rotation_quaternion (optional)

object

Object defining the rotation of the camera in relation to the world. Expressed as a quaternion, in format {w: 0.5, x: 0.5, y: 0.5, z: 0.5}. Use in conjunction with camera_intrinsics to get perspective-corrected cuboids and 3d points. Note that the z-axis of the camera frame represents the camera’s optical axis. This value will be applied for all frames in the video.

camera_rotation_quaternion_per_frame (optional)

Array<object>

Array of objects defining the rotation of the camera per frame. The format of each object should be the same as for the camera_rotation_quaternion property. The index of each object list will correspond to the frame number.

camera_height_per_frame

Array<number>

Height of camera above the ground per frame, in meters.

Response & Callback Format

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a link to the annotations file containing a list of annotations per frame. Each annotation object contains its unique id to be used for tracking an object as well as its position, label, and attributes.

The annotations file will be stored on S3 and be a JSON encoded list of annotation objects per frame. The index of each annotation list will correspond to the frame defined in the original response.

The order of the annotations lists will correspond directly to the order of the frame attachments that you sent in your original request. That is, the first annotation list will correspond to the first frame in the attachments array, etc.

Once the JSON file is reviewed and uploaded, we will send a response to your callback URL.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_categorization_task(callback_url='http://www.example.com/callback',instruction='Categorize this movie trailer',attachment_type='website',attachment='https://www.youtube.com/embed/6ZfuNTqbHE8',taxonomies={'watch':{'type':'category','description':'Are you going to watch the full movie?','choices':['yes','no']},'genre':{'type':'category','description':'What genre does it belong to?','choices':['Action','Adventure','Comedy','Crime','Drama'],'allow_multiple':True}})

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCategorizationTask({callback_url:'http://www.example.com/callback',instruction:'Categorize this movie trailer',attachment_type:'website',attachment:'https://www.youtube.com/embed/6ZfuNTqbHE8',taxonomies:{watch:{type:'category',description:'Are you going to watch the full movie?',choices:['yes','no']},genre:{type:'category',description:'What genre does it belong to?',choices:['Action','Adventure','Comedy','Crime','Drama'],allow_multiple:true}}},(err,task)=>{// do something with task});

{"task_id":"5c3f9c25744e7d005052e319","created_at":"2019-01-16T21:03:33.166Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"pending","instruction":"Categorize this movie trailer","params":{"attachment_type":"website","attachment":"https://www.youtube.com/embed/6ZfuNTqbHE8","taxonomies":{"watch":{"type":"category","description":"Are you going to watch the full movie?","choices":["yes","no"]},"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure","Comedy","Crime","Drama"],"allow_multiple":true}}},"is_test":false,"metadata":{},"processed_attachments":[]}

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and select the correct answer for every taxonomy/question. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabledinstruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally taxonomies. The format for taxonomies is an object whose key-value pairs specify taxonomies/questions for the attachment.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment and taxonomies will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/categorization

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.

attachment

string

The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

taxonomies

object

An object whose key-value pairs specify taxonomies/questions for the attachment.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

Categorical Taxonomies

Example definition of a categorical taxonomy.

{..."taxonomies":{"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure","Comedy","Crime","Drama"],"allow_multiple":true}}}

Each key-value defining a categorical taxonomy should have the following structure:

key: a short string that serves as the identifier for the taxonomy.

value: an object that should have the following keys:

description: a human-readable string describing the taxonomy to a categorizer.

choices: a list of strings or objects of nested choices.

type: a string describing the type of taxonomy. Must be category for categorical taxonomies.

allow_multiple (optional): allow multiple categories to be chosen.

condition (optional): an object that specifies that the current taxonomy will only be present to worker if another taxonomy has certain answers.

Categorical Taxonomies: nested choices

There are often categorization tasks that have too many choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested choices, where choices may have subchoices within them. A simple example illustrated in the example API call on the right is when selecting a movie genre. While there may be a large amount of total choices, using subchoices a worker can first categorize a movie as adventure, and based on that choice select the specific type of adventure.

For example, for a choices array of ["Action", "Adventure"], you could instead add a nested choice by passing an array like ["Action", { "choice": "Adventure", "subchoices": ["Anime", "Superhero"] }]. Then, if a worker selected Adventure, they would be further prompted to choose one of the corresponding subchoices.

These hierarchies can themselves be nested, so that the subchoices array may itself specify further nested choices.

Numerical Taxonomies

Example definition of a numerical taxonomy.

{..."taxonomies":{"genre":{"type":"number","description":"Rate the movie","min":0,"max":10,"step":1}}}

Each key-value defining a numerical taxonomy should have the following structure:

key: a short string that serves as the identifier for the taxonomy.

value: an object that should have the following keys:

description: a human-readable string describing the taxonomy to a categorizer.

type: a string describing the type of taxonomy. Must be number for numerical taxonomies.

min (optional): The minimum number that may be chosen as the attribute.

max (optional): The maximum number that may be chosen as the attribute.

step (optional, default 1): allow multiple categories to be chosen.

Conditional taxonomies

Sometimes a categorization taxonomy should only be present to the worker if specific choices are select for another taxonomy. In those cases, specify a dependent taxonomy and the required choice(s). Only if one of the choices are select by the worker, will the worker see the current taxonomy.

The condition property defining a conditional taxonomy should have the following structure:

taxonomy: the dependent taxonomy that can activate the current taxonomy.

choices: the required choices for the dependent taxonomy. At least one of them needs to be chosen before the current taxonomy is activated.

Example definition of a conditional taxonomy. The “adventure-genre” taxonomy will only be present to the worker, if “Adventure” is chosen for taxonomy/question “genre”.

{..."taxonomies":{"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure"]},"adventure-genre":{"type":"category","description":"What sub genre of adventure does it belong to?","choices":["Anime","Superhero"],"condition":{"taxonomy":"genre","choices":["Adventure"]}}}}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a taxonomies field.

For categorical taxonomies, the value will be an array of choices, each one being one of the original choices.

For numerical taxonomies, the value will be a number.

Nested choices are only intended for grouping a large set of choices together. Thus, the response will be the same as with plain choices, where only the final “leaf” choice that the worker selected is reported. In the example above, if a worker selected Adventure and then Superhero, the choice would be Superhero.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_categorization_task(callback_url='http://www.example.com/callback',instruction='Is this company public or private?',attachment_type='website',attachment='http://www.google.com/',categories=['public','private'])

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCategorizationTask({'callback_url':'http://www.example.com/callback','instruction':'Is this company public or private?','attachment_type':'website','attachment':'http://www.google.com/','categories':['public','private']},(err,task)=>{// do something with task});

{"task_id":"576ba74eec471ff9b01557cc","created_at":"2016-06-23T09:09:34.752Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"pending","instruction":"Is this company public or private?","params":{"attachment_type":"website","attachment":"http://www.google.com/","categories":["public","private"]},"metadata":{}}

Categorization single mode is deprecated. Multiple mode with a single taxonomy is preferred.

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and choose a category for it according to the instruction. You may allow multiple categories to be chosen by setting allow_multiple to true. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabledinstruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally a list of categories.

There is an optional category_ids parameter, which you can use to impose an id system over the categories. The format of this parameter should be a dictionary, where the keys are the ids (as strings), and then the values are the category values provided in categories. An example is:

"category_ids": {
"123": "Blue Cross",
"124": "Red Cross"
}

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment, categories, and category_ids (optional) will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/categorization

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.

attachment

string

The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

categories

[string]

An array of strings for the categories which you’d like the object to be sorted between.

category_ids (optional)

dictionary

An optional dictionary where the keys are the optional ids, and the values are the category values provided in categories.

allow_multiple (optional)

boolean

Default is false. Determines whether you allow multiple categories to be chosen for the attachment

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

Computer Vision Layers

All of these input formats have the same formatting as our responses, so passing one Scale tasks’ data to another Scale task is really easy.

// This example uses express with body-parservarscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');app.post('/polygon_task_callback_handler',function(req,res){// Validate callback auth key...varpolygons=req.body.response.annotations;varattachment=req.body.task.params.attachment;client.createAnnotationTask({'callback_url':'http://www.example.com/annotation_task_callback_handler','instruction':'Draw a box around each **car** and **pedestrian**','attachment_type':'image','attachment':attachment,'objects_to_annotate':['car','pedestrian'],'with_labels':true,'min_width':'30','min_height':'30','layers':{'polygons':polygons}},(err,task)=>{// do something with task});});

# This example uses Flaskimportscaleapifromflaskimportrequest,Flaskapp=Flask(__name__)client=scaleapi.ScaleClient('{{ApiKey}}')@app.route('/polygon_task_callback_handler',methods=['POST'])defcreate_annotation_task():# Validate callback auth key...polygons=request.json['response']['annotations']attachment=request.json['task']['params']['attachment']task=client.create_annotation_task(callback_url='http://www.example.com/annotation_task_callback_handler',instruction='Draw a box around each **car** and **pedestrian**',attachment_type='image',attachment=attachment,objects_to_annotate=['car','pedestrian'],with_labels=True,min_width='30',min_height='30',layers={'polygons':polygons})# do something with the task

# This example uses Railsrequire'scale'classExampleController<ActionController::Basedefinitialize@scale=Scale.new(api_key: '{{ApiKey}}')end# This should be mapped to a POST routedefpolygon_task_callback_handler# Validate callback auth key...polygons=params['response']['annotations']attachment=params['task']['params']['attachment']task=@scale.create_annotation_task({callback_url: 'http://www.example.com/annotation_task_callback_handler',instruction: 'Draw a box around each **car** and **pedestrian**',attachment_type: 'image',attachment: attachment,objects_to_annotate: ['car','pedestrian'],with_labels: true,min_width: '30',min_height: '30',layers: {polygons: polygons}})# do something with the taskendend

“Layers” can be used to specify existing (read-only) boxes, lines, polygons, and/or cuboids to be pre-drawn on an image.

For instance, you could specify boxes around the cars of an image, for a task that requires drawing polygons around the currently boxed cars. Or you could create a task for drawing boxes around all cars which you hadn’t already recognized.

To specify layers, you can pass an optional layers parameter in the request for annotation, lineannotation, polygonannotation, or pointannotation tasks. They can also be used in categorization tasks if the attachment_type is image.

The layers parameter can contains fields for boxes, lines, polygons, and cuboids, which are arrays of the corresponding elements. Each of these elements are specified in the same format as the responses for their respective endpoints.

Lines & Splines

lines are objects with an optional label field, and a vertices field which contains a list of objects with x and y attributes, similar to polygons. Additionally, line objects have an optional spline flag, which determines whether the curve used is linear (if the value is false or the param is not passed) or cardinal (if the value is true).

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_cuboidannotation_task(callback_url="http://www.example.com/callback",attachment="https://i.imgur.com/VDPoOZE.jpg",attachment_type="image",metadata={"some_metadata":"some metadata"},instruction="Annotate the cars and pedestrians",min_width=30,min_height=30,objects_to_annotate=[{"choice":"Vehicle","subchoices":["Car","Truck","Train","Motorcycle"]},{"choice":"Pedestrian","subchoices":["Animal","Adult","Child"]}],annotation_attributes={"occlusion":{"description":"What percent of the object is occluded?","choices":["0%","25%","50%","75%"],"conditions":{"label_condition":{"label":["Car","Truck","Motorcycle"]}}}})

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCuboidannotationTask({callback_url:'http://www.example.com/callback',attachment:'https://i.imgur.com/VDPoOZE.jpg',attachment_type:'image',metadata:{some_metadata:'some metadata'},instruction:'Annotate the cars and pedestrians',min_width:30,min_height:30,objects_to_annotate:[{choice:'Vehicle',subchoices:['Car','Truck','Train','Motorcycle']},{choice:'Pedestrian',subchoices:['Animal','Adult','Child']}],annotation_attributes:{occlusion:{description:'What percent of the object is occluded?',choices:['0%','25%','50%','75%'],conditions:{label_condition:{label:['Car','Truck','Motorcycle']}}}}},(err,task)=>{// do something with task});

{"task_id":"5774cc78b01249ab09f089dd","created_at":"2016-9-03T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"annotation","status":"pending","instruction":"Annotate the cars and pedestrians","params":{"attachment":"https://i.imgur.com/VDPoOZE.jpg","attachment_type":"image","objects_to_annotate":[{"choice":"Vehicle","subchoices":["Car","Truck","Train","Motorcycle"]},{"choice":"Pedestrian","subchoices":["Animal","Adult","Child"]}],"with_labels":false,"min_width":30,"min_height":30,"examples":[],"annotation_attributes":{"occlusion":{"description":"What percent of the object is occluded?","choices":["0%","25%","50%","75%"],"conditions":{"label_condition":{"label":["Car","Truck","Motorcycle"]}}}}},"metadata":{}}

Nested labels may be used with any 2D image annotation task, including annotation, lineannotation, pointannotation, cuboidannotation, polygonannotation, segmentannotation.

There are often annotation tasks that have too many label choices for a worker to efficiently sort through them all at once.
In those cases, a solution is to use nested labels, where labels may have subcategories within them.
A simple example illustrated in the example API call on the right is when labeling several different kinds of pedestrians and vehicles.
While there may be a large amount of total labels, using subchoices a worker can first categorize an object as pedestrian or vehicle,
and based on that choice select the specific type of pedestrian or vehicle.

Nested labels may be specified both for the object labels (the objects_to_annotate array parameter), as well as in the choices array of a
categorical annotation attribute. In both cases, you would specify a nested label by using an object instead of a string.

For example, for an objects_to_annotate array of ["Vehicle", "Pedestrian"], you could instead add a nested label by passing an array like
["Vehicle", {"choice": "Pedestrian", "subchoices": ["Animal", "Adult", "Child"]}]. Then, if a worker selected “Pedestrian” for an annotation,
they would be further prompted to choose one of the corresponding subchoices for that annotation.

These hierarchies can themselves be nested, so that the subchoices array of a nested label may itself specify further nested labels.

Response Format

Nested labels are only intended for grouping a large set of labels together. Thus, the response will be the same as with unnested labels, where
only the final “leaf” label that the worker selected is reported. In the example above, if a worker selected “Pedestrian” and then “Animal” for an
annotation, that annotation’s label would be “Animal”.

Conditional attributes

Nested labels can be used when specifying conditional attributes as well.

As with the response object for a nested label, the condition should specify the “leaf” labels that it is conditional on.

For example, if in the above example you wanted an “occlusion” attribute for some mix of the “leaf” labels, you could specify
the label condition for the attribute as e.g. { "label": ["Car", "Truck", "Motorcycle", "Adult"] }. If you wanted the attribute to be
conditional on all the subchoices of a given label (e.g., all “Vehicle"s), you would simply specify all of those subchoices in the array,
e.g. { "label": ["Car", "Truck", "Train", "Motorcycle"] }.

Annotation Attributes

Annotation attributes may be used with any image annotation task, including annotation, lineannotation, pointannotation, cuboidannotation, polygonannotation, segmentannotation, and lidarannotation.

In many cases, it is useful to have more human-judged metadata on top of each annotation for a 2D image annotation task, for example, measuring the occlusion-level of all vehicles in an image.

To achieve this, we support annotation_attributes, an object representing additional attributes that you’d like to capture per image annotation.

You may use annotation_attributes to define categorical attributes, numerical attributes, angle attributes, or text attributes for each annotation. You define the type of the attribute using the type property of the attribute, which is either category, number, angle, or text. It defaults to category.

The format for annotation_attributes is an object whose key-value pairs all specify attributes of each annotation that you want to capture. The schema differs slightly based on the type of the attribute:

Categorical Attributes

Example definition of a categorical attribute.

{..."annotation_attributes":{"example_category":{"type":"category","description":"What is the answer to this example question?","choices":["Answer","Other"]}}}

Each key-value defining a categorical attribute should have the following structure:

key: a short string that serves as the identifier for the attribute

value: an object that should have the following keys:

description: a human-readable string describing the attribute to a labeler.

choices: a list of strings corresponding to the categorical choices.

type (default category): a string describing the type of attribute. Must be category for categorical attributes.

conditions (optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information.

Numerical Attributes

Example definition of a numerical attribute.

{..."annotation_attributes":{"example_number":{"type":"number","description":"What is the answer to this example question?","min":0,"max":100}}}

Each key-value defining a numerical attribute should have the following structure:

key: a short string that serves as the identifier for the attribute

value: an object that should have the following keys:

description: a human-readable string describing the attribute to a labeler.

type: a string describing the type of attribute. Must be number for numerical attributes.

min (optional): The minimum number that may be chosen as the attribute.

max (optional): The maximum number that may be chosen as the attribute.

step (optional, default 1): The legal number intervals for the attribute.

conditions (optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information

Angle Attributes

Example definition of a angle attribute.

{..."annotation_attributes":{"example_angle":{"type":"angle","description":"What is the angle of this object?",}}}

An angle attribute allows you to receive some angular information of the attribute, such as the heading yaw angle or heading pitch angle. This will return a number from 0 to 360, which is the value of the angle in degrees.

Each key-value defining an angle attribute should have the following structure:

key: a short string that serves as the identifier for the attribute

value: an object that should have the following keys:

description: a human-readable string describing the attribute to a labeler.

type: a string describing the type of attribute. Must be angle for angle attributes.

conditions (optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information

Text Attributes

Example definition of a text attribute.

{..."annotation_attributes":{"example_text":{"type":"text","description":"What does this sign say?",}}}

A text attribute allows you to receive freeform text, such as a transcription of a road sign or a car’s license plate. This will return a string.

Each key-value defining a text attribute should have the following structure:

key: a short string that serves as the identifier for the attribute

value: an object that should have the following keys:

description: a human-readable string describing the attribute to a labeler.

type: a string describing the type of attribute. Must be text for text attributes.

conditions (optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information

Conditionality

The conditions JSON describes the set of conditions under which the attribute should be collected. Each of the conditions described in the JSON are AND’d together, meaning that adding more conditions is more restrictive.

The conditions JSON can have up to two key-values: label_condition and attribute_conditions.

A Condition is an object defined by a single key and a value. The key describes the semantic string that you want to check, whether it is 'label' for the label_condition, or the attribute identifier for attribute_conditions.

The value is either a string or an array of strings. In the case of a string, the label or attribute will be checked for strict equality with the string. In the case of an array of strings, the label or attribute will be checked for membership within the array.

Condition Parameters

The label_condition key-value will describe the label or labels that you want to collect this attribute for. The value is a Condition object, which must have key equal to 'label'.

The attribute_conditions key-value will describe under what conditions with the other attributes you’d like to collect this attribute for. The value should a Condition object or a list of Condition objects.

attribute_conditions are automatically verified for circular dependencies, and tasks will be rejected if any circular dependencies are found.

Request Format

{"callback_url":"http://www.example.com/callback","instruction":"Draw boxes around the vehicles in the image.","attachment_type":"image","attachment":"http://i.imgur.com/v4cBreD.jpg","objects_to_annotate":["car","bus"],"with_labels":true,"annotation_attributes":{"parked":{"description":"Is the car currently parked?","choices":["Yes","No"]},"heading":{"description":"Which direction is the car heading","choices":["left","right","back","front"],"conditions":{"label_condition":{"label":"car"},"attribute_conditions":[{"parked":"No"}]}}}}

To create a task with attributes, simply add the annotation_attributes parameter to your task creation request using the format described above.

In addition to the standard attributes for the annotation response, if you specified annotation_attributes in the request, each annotation object will contain an attributes JSON. It will contain keys for each of the attributes you defined in the annotation_attributes schema, and the values will be the categorical value or numerical value chosen by a labeler.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_categorization_task(callback_url='http://www.example.com/callback',instruction='Categorize this movie trailer',attachment_type='website',attachment='https://www.youtube.com/embed/6ZfuNTqbHE8',taxonomies={'watch':{'type':'category','description':'Are you going to watch the full movie?','choices':['yes','no']},'genre':{'type':'category','description':'What genre does it belong to?','choices':['Action','Adventure','Comedy','Crime','Drama'],'allow_multiple':True}})

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCategorizationTask({callback_url:'http://www.example.com/callback',instruction:'Categorize this movie trailer',attachment_type:'website',attachment:'https://www.youtube.com/embed/6ZfuNTqbHE8',taxonomies:{watch:{type:'category',description:'Are you going to watch the full movie?',choices:['yes','no']},genre:{type:'category',description:'What genre does it belong to?',choices:['Action','Adventure','Comedy','Crime','Drama'],allow_multiple:true}}},(err,task)=>{// do something with task});

{"task_id":"5c3f9c25744e7d005052e319","created_at":"2019-01-16T21:03:33.166Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"pending","instruction":"Categorize this movie trailer","params":{"attachment_type":"website","attachment":"https://www.youtube.com/embed/6ZfuNTqbHE8","taxonomies":{"watch":{"type":"category","description":"Are you going to watch the full movie?","choices":["yes","no"]},"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure","Comedy","Crime","Drama"],"allow_multiple":true}}},"is_test":false,"metadata":{},"processed_attachments":[]}

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and select the correct answer for every taxonomy/question. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabledinstruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally taxonomies. The format for taxonomies is an object whose key-value pairs specify taxonomies/questions for the attachment.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment and taxonomies will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/categorization

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.

attachment

string

The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

taxonomies

object

An object whose key-value pairs specify taxonomies/questions for the attachment.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

Categorical Taxonomies

Example definition of a categorical taxonomy.

{..."taxonomies":{"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure","Comedy","Crime","Drama"],"allow_multiple":true}}}

Each key-value defining a categorical taxonomy should have the following structure:

key: a short string that serves as the identifier for the taxonomy.

value: an object that should have the following keys:

description: a human-readable string describing the taxonomy to a categorizer.

choices: a list of strings or objects of nested choices.

type: a string describing the type of taxonomy. Must be category for categorical taxonomies.

allow_multiple (optional): allow multiple categories to be chosen.

condition (optional): an object that specifies that the current taxonomy will only be present to worker if another taxonomy has certain answers.

Categorical Taxonomies: nested choices

There are often categorization tasks that have too many choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested choices, where choices may have subchoices within them. A simple example illustrated in the example API call on the right is when selecting a movie genre. While there may be a large amount of total choices, using subchoices a worker can first categorize a movie as adventure, and based on that choice select the specific type of adventure.

For example, for a choices array of ["Action", "Adventure"], you could instead add a nested choice by passing an array like ["Action", { "choice": "Adventure", "subchoices": ["Anime", "Superhero"] }]. Then, if a worker selected Adventure, they would be further prompted to choose one of the corresponding subchoices.

These hierarchies can themselves be nested, so that the subchoices array may itself specify further nested choices.

Numerical Taxonomies

Example definition of a numerical taxonomy.

{..."taxonomies":{"genre":{"type":"number","description":"Rate the movie","min":0,"max":10,"step":1}}}

Each key-value defining a numerical taxonomy should have the following structure:

key: a short string that serves as the identifier for the taxonomy.

value: an object that should have the following keys:

description: a human-readable string describing the taxonomy to a categorizer.

type: a string describing the type of taxonomy. Must be number for numerical taxonomies.

min (optional): The minimum number that may be chosen as the attribute.

max (optional): The maximum number that may be chosen as the attribute.

step (optional, default 1): allow multiple categories to be chosen.

Conditional taxonomies

Sometimes a categorization taxonomy should only be present to the worker if specific choices are select for another taxonomy. In those cases, specify a dependent taxonomy and the required choice(s). Only if one of the choices are select by the worker, will the worker see the current taxonomy.

The condition property defining a conditional taxonomy should have the following structure:

taxonomy: the dependent taxonomy that can activate the current taxonomy.

choices: the required choices for the dependent taxonomy. At least one of them needs to be chosen before the current taxonomy is activated.

Example definition of a conditional taxonomy. The “adventure-genre” taxonomy will only be present to the worker, if “Adventure” is chosen for taxonomy/question “genre”.

{..."taxonomies":{"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure"]},"adventure-genre":{"type":"category","description":"What sub genre of adventure does it belong to?","choices":["Anime","Superhero"],"condition":{"taxonomy":"genre","choices":["Adventure"]}}}}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a taxonomies field.

For categorical taxonomies, the value will be an array of choices, each one being one of the original choices.

For numerical taxonomies, the value will be a number.

Nested choices are only intended for grouping a large set of choices together. Thus, the response will be the same as with plain choices, where only the final “leaf” choice that the worker selected is reported. In the example above, if a worker selected Adventure and then Superhero, the choice would be Superhero.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_categorization_task(callback_url='http://www.example.com/callback',instruction='Is this company public or private?',attachment_type='website',attachment='http://www.google.com/',categories=['public','private'])

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCategorizationTask({'callback_url':'http://www.example.com/callback','instruction':'Is this company public or private?','attachment_type':'website','attachment':'http://www.google.com/','categories':['public','private']},(err,task)=>{// do something with task});

{"task_id":"576ba74eec471ff9b01557cc","created_at":"2016-06-23T09:09:34.752Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"pending","instruction":"Is this company public or private?","params":{"attachment_type":"website","attachment":"http://www.google.com/","categories":["public","private"]},"metadata":{}}

Categorization single mode is deprecated. Multiple mode with a single taxonomy is preferred.

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and choose a category for it according to the instruction. You may allow multiple categories to be chosen by setting allow_multiple to true. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabledinstruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally a list of categories.

There is an optional category_ids parameter, which you can use to impose an id system over the categories. The format of this parameter should be a dictionary, where the keys are the ids (as strings), and then the values are the category values provided in categories. An example is:

"category_ids": {
"123": "Blue Cross",
"124": "Red Cross"
}

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment, categories, and category_ids (optional) will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/categorization

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.

attachment

string

The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

categories

[string]

An array of strings for the categories which you’d like the object to be sorted between.

category_ids (optional)

dictionary

An optional dictionary where the keys are the optional ids, and the values are the category values provided in categories.

allow_multiple (optional)

boolean

Default is false. Determines whether you allow multiple categories to be chosen for the attachment

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_categorization_task(callback_url='http://www.example.com/callback',instruction='Categorize this movie trailer',attachment_type='website',attachment='https://www.youtube.com/embed/6ZfuNTqbHE8',taxonomies={'watch':{'type':'category','description':'Are you going to watch the full movie?','choices':['yes','no']},'genre':{'type':'category','description':'What genre does it belong to?','choices':['Action','Adventure','Comedy','Crime','Drama'],'allow_multiple':True}})

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCategorizationTask({callback_url:'http://www.example.com/callback',instruction:'Categorize this movie trailer',attachment_type:'website',attachment:'https://www.youtube.com/embed/6ZfuNTqbHE8',taxonomies:{watch:{type:'category',description:'Are you going to watch the full movie?',choices:['yes','no']},genre:{type:'category',description:'What genre does it belong to?',choices:['Action','Adventure','Comedy','Crime','Drama'],allow_multiple:true}}},(err,task)=>{// do something with task});

{"task_id":"5c3f9c25744e7d005052e319","created_at":"2019-01-16T21:03:33.166Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"pending","instruction":"Categorize this movie trailer","params":{"attachment_type":"website","attachment":"https://www.youtube.com/embed/6ZfuNTqbHE8","taxonomies":{"watch":{"type":"category","description":"Are you going to watch the full movie?","choices":["yes","no"]},"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure","Comedy","Crime","Drama"],"allow_multiple":true}}},"is_test":false,"metadata":{},"processed_attachments":[]}

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and select the correct answer for every taxonomy/question. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabledinstruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally taxonomies. The format for taxonomies is an object whose key-value pairs specify taxonomies/questions for the attachment.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment and taxonomies will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/categorization

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.

attachment

string

The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

taxonomies

object

An object whose key-value pairs specify taxonomies/questions for the attachment.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

Categorical Taxonomies

Example definition of a categorical taxonomy.

{..."taxonomies":{"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure","Comedy","Crime","Drama"],"allow_multiple":true}}}

Each key-value defining a categorical taxonomy should have the following structure:

key: a short string that serves as the identifier for the taxonomy.

value: an object that should have the following keys:

description: a human-readable string describing the taxonomy to a categorizer.

choices: a list of strings or objects of nested choices.

type: a string describing the type of taxonomy. Must be category for categorical taxonomies.

allow_multiple (optional): allow multiple categories to be chosen.

condition (optional): an object that specifies that the current taxonomy will only be present to worker if another taxonomy has certain answers.

Categorical Taxonomies: nested choices

There are often categorization tasks that have too many choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested choices, where choices may have subchoices within them. A simple example illustrated in the example API call on the right is when selecting a movie genre. While there may be a large amount of total choices, using subchoices a worker can first categorize a movie as adventure, and based on that choice select the specific type of adventure.

For example, for a choices array of ["Action", "Adventure"], you could instead add a nested choice by passing an array like ["Action", { "choice": "Adventure", "subchoices": ["Anime", "Superhero"] }]. Then, if a worker selected Adventure, they would be further prompted to choose one of the corresponding subchoices.

These hierarchies can themselves be nested, so that the subchoices array may itself specify further nested choices.

Numerical Taxonomies

Example definition of a numerical taxonomy.

{..."taxonomies":{"genre":{"type":"number","description":"Rate the movie","min":0,"max":10,"step":1}}}

Each key-value defining a numerical taxonomy should have the following structure:

key: a short string that serves as the identifier for the taxonomy.

value: an object that should have the following keys:

description: a human-readable string describing the taxonomy to a categorizer.

type: a string describing the type of taxonomy. Must be number for numerical taxonomies.

min (optional): The minimum number that may be chosen as the attribute.

max (optional): The maximum number that may be chosen as the attribute.

step (optional, default 1): allow multiple categories to be chosen.

Conditional taxonomies

Sometimes a categorization taxonomy should only be present to the worker if specific choices are select for another taxonomy. In those cases, specify a dependent taxonomy and the required choice(s). Only if one of the choices are select by the worker, will the worker see the current taxonomy.

The condition property defining a conditional taxonomy should have the following structure:

taxonomy: the dependent taxonomy that can activate the current taxonomy.

choices: the required choices for the dependent taxonomy. At least one of them needs to be chosen before the current taxonomy is activated.

Example definition of a conditional taxonomy. The “adventure-genre” taxonomy will only be present to the worker, if “Adventure” is chosen for taxonomy/question “genre”.

{..."taxonomies":{"genre":{"type":"category","description":"What genre does it belong to?","choices":["Action","Adventure",""]},"adventure-genre":{"type":"category","description":"What sub genre of adventure does it belong to?","choices":["Anime","Superhero"],"condition":{"taxonomy":"genre","choices":["Adventure"]}}}}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a taxonomies field.

For categorical taxonomies, the value will be an array of choices, each one being one of the original choices.

For numerical taxonomies, the value will be a number.

Nested choices are only intended for grouping a large set of choices together. Thus, the response will be the same as with plain choices, where only the final “leaf” choice that the worker selected is reported. In the example above, if a worker selected Adventure and then Superhero, the choice would be Superhero.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_categorization_task(callback_url='http://www.example.com/callback',instruction='Is this company public or private?',attachment_type='website',attachment='http://www.google.com/',categories=['public','private'])

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createCategorizationTask({'callback_url':'http://www.example.com/callback','instruction':'Is this company public or private?','attachment_type':'website','attachment':'http://www.google.com/','categories':['public','private']},(err,task)=>{// do something with task});

{"task_id":"576ba74eec471ff9b01557cc","created_at":"2016-06-23T09:09:34.752Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"pending","instruction":"Is this company public or private?","params":{"attachment_type":"website","attachment":"http://www.google.com/","categories":["public","private"]},"metadata":{}}

Categorization single mode is deprecated. Multiple mode with a single taxonomy is preferred.

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and choose a category for it according to the instruction. You may allow multiple categories to be chosen by setting allow_multiple to true. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabledinstruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally a list of categories.

There is an optional category_ids parameter, which you can use to impose an id system over the categories. The format of this parameter should be a dictionary, where the keys are the ids (as strings), and then the values are the category values provided in categories. An example is:

"category_ids": {
"123": "Blue Cross",
"124": "Red Cross"
}

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment, categories, and category_ids (optional) will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/categorization

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.

attachment

string

The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

categories

[string]

An array of strings for the categories which you’d like the object to be sorted between.

category_ids (optional)

dictionary

An optional dictionary where the keys are the optional ids, and the values are the category values provided in categories.

allow_multiple (optional)

boolean

Default is false. Determines whether you allow multiple categories to be chosen for the attachment

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

layers (optional)

object

A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_transcription_task(callback_url='http://www.example.com/callback',instruction='Transcribe the given fields.',attachment_type='website',attachment='http://news.ycombinator.com/',fields={'title':'Title of Webpage','top_result':'Title of the top result'})

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createTranscriptionTask({'callback_url':'http://www.example.com/callback','instruction':'Transcribe the given fields.','attachment_type':'website','attachment':'http://news.ycombinator.com/','fields':{'title':'Title of Webpage','top_result':'Title of the top result'}},(err,task)=>{// do something with task});

{"task_id":"576de9dc1ea5f917d56fc2a0","created_at":"2016-06-25T02:18:04.248Z","callback_url":"http://www.example.com/callback","type":"transcription","status":"pending","instruction":"Transcribe the given fields.","params":{"fields":{"title":"Title of Webpage","top_result":"Title of the top result"},"attachment":"http://news.ycombinator.com/","attachment_type":"website"},"metadata":{}}

This endpoint creates a transcription task. In this task, one of our workers will read an attachment and arbitrarily transcribe any information you’d like. Example use cases could be transcribing information from PDFs, manually scraping a web page for information, etc.

This task involves a markdown-enabledinstruction about how to transcribe the attachment, an attachment of what you’d like to transcribe, an attachment_type, fields, and repeatable_fields.

fields is a dictionary which describes items you’d like transcribed for the attachment. Examples are phone numbers, names, etc. repeatable_fields is a dictionary which describes items which you’d like transcribed for the attachment which appear in the attachment multiple times. Examples are the row-by-row items of an invoice or purchase order.

In addition, we now support choices for repeatable_fields where you can specify an array of choices for the repeatable field.

At least one of fields or repeatable_fields is required. Both fields and repeatable_fields are dictionaries where the keys are the identifiers you’d like the results to be returned using, and values are plaintext descriptions you’d like to show labelers as they complete the task.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment, fields, and repeatable_fields will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/transcription

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to transcribe the attachment. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of image, pdf, or website. Describes what type of file the attachment is.

attachment

string

The attachment to be transcribed. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.

fields (optional if using repeatable_fields)

object

A dictionary corresponding to the fields to be transcribed. Keys are the identifiers you’d like the fields to be returned using, and values are descriptions to be shown to the labelers as they complete the task.

repeatable_fields (optional if using fields)

object

If your task requires a transcription of items which might be repeated within the attachment, such as rows of an invoice, then this dictionary describes those fields. With the addition of choices, there are now two acceptable formats. The first acceptable format is the same format as fields to generate repeatable text fields. The second acceptable format is a dictionary with keys of description (identifiers) and choices (the list of choices for that input). In addition, for repeatable_fields with the same format as fields, our internal task model will represent each repeatable field as a dictionary object with key description to the original value.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Callback Format

Example callback response sent on completion with fields

{"response":{"fields":{"title":"Some Title","top_result":"The Top Result or Something"}},"task_id":"5774cc78b01249ab09f089dd","task":{//populatedtaskforconvenience...}}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a fields field.

fields will have keys corresponding to the keys you provided in the parameters, with values equal to the transcribed value.

If you requested repeatable_fields, repeatable_fields in the response will be an array of such dictionaries, with keys corresponding to the keys you provided in the repeatable_fields parameter, and values corresponding to the transcribed value. Each element of the array will correspond to one transcribed value in the attachment.

importscaleapiclient=scaleapi.ScaleClient('{{ApiKey}}')client.create_comparison_task(callback_url='http://www.example.com/callback',instruction='Do the objects in these images have the same pattern?',attachment_type='image',attachments=['http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG','http://images.wisegeek.com/checkered-tablecloth.jpg'],choices=['yes','no'])

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');client.createComparisonTask({'callback_url':'http://www.example.com/callback','instruction':'Do the objects in these images have the same pattern?','attachment_type':'image','attachments':['http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG','http://images.wisegeek.com/checkered-tablecloth.jpg'],'choices':['yes','no']},(err,task)=>{// do something with task});

{"task_id":"5774cc78b02487c424f089dd","created_at":"2016-06-30T07:38:32.368Z","callback_url":"http://www.example.com/callback","type":"comparison","status":"pending","instruction":"Do the objects in these images have the same pattern?","params":{"choices":["yes","no"],"attachment_type":"image","attachments":["http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG","http://images.wisegeek.com/checkered-tablecloth.jpg"]},"metadata":{}}

This endpoint creates a comparison task. In this task, one of our workers view the given attachments and do any comparison requested.

This task involves a markdown-enabledinstruction, an array of attachments, and an attachment_type.

At least of the fields or choices parameters must specified for the data to be returned. choices is an array of strings from which the worker to choose, and fields is useful for free-text response.

fields is a dictionary where the keys are the keys you’d like the results to be returned under, and values are the descriptions you’d like to show the human worker.

If successful, Scale will immediately return the generated task object, of which you should store the task_id.

The parameters attachment_type, attachments, choices, and fields will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.com/v1/task/comparison

Parameters

Parameter

Type

Description

project (optional)

string

The name of the project to associate this task with. See Projects section for more details.

batch (optional)

string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.

callback_url

string

The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.

instruction

string

A markdown-enabled string explaining how to compare the attachments. You can use markdown to show example images, give structure to your instructions, and more.

attachment_type

string

One of text, image, video, audio, website, or pdf. Describes what type of file the attachments are.

attachments

array

An array of attachments to compare. If attachment_type is text, then each attachment should be plaintext. Otherwise, they should be URLs pointing to the attachments.

fields (optional)

dictionary

A dictionary corresponding to the fields to be recorded. Keys are the keys you’d like the fields to be returned under, and values are descriptions to be shown to human workers.

choices (optional)

[string]

An array of strings for the choices to be given to the worker. One of choices or fields must be specified.

metadata (optional, default {})

object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Secure Attachment Access

IP Whitelisting

Scale uses a consistent set of IP addresses to fetch data and send callbacks, allowing for IP whitelisting of attachments sent to us, as well as for callback endpoints, to increase data security.
If you are enabling IP whitelisting, we request that you whitelist access to your data to all 5 listed IP addresses below, and we will only fetch content using these IP addresses. In this way, you can secure your content from the public while still allowing Scale to access it.

Scale static IP addresses

52.38.24.56
35.160.30.43
35.167.66.86
52.11.250.38
54.203.55.239

Our static IP addresses are:

52.38.24.56

35.160.30.43

35.167.66.86

52.11.250.38

54.203.55.239

If you are using AWS S3, then you can add these IPs to your S3 Bucket Policy using these instructions.

S3 IAM Access

If you use AWS S3 to store data, you can also submit tasks with attachments as s3: protocol URLs, rather than http: or https:. We will directly fetch attachments from your S3 bucket, using AWS account ID 307185671274 (canonical ID ae2259599e139df6cedb60b6300bcafa1c652aff129aa3d887477b6d4abf2e47), which you can grant access to on a per-object or a per-bucket basis.

GCP IAM Access

If you use Google Cloud Storage to store data, you can also submit tasks with attachments as gs: protocol URLs, rather than http: or https:. We will directly fetch attachments from your GCS bucket, using the GCP role [email protected]unt.com.

Projects

Projects are a way of organizing similar tasks, so that one can share parameters among tasks, or control what workers are allowed to work on certain classes of tasks.

The parameters associated with a project will be inherited by tasks created under that project. Currently the only project parameter supported is instruction; instructions defined in a project will be concatenated to the end of the instructions of tasks created under the project.

To use a project for a given task, add a project parameter when creating the task that references the project name; e.g., to associate a task with project my_project, add project: my_project to the task creation request. Note that projects created using your test API key can only be used for creating test tasks; likewise with the live API key and live tasks.

importrequestsimportjsonpayload={'type':'annotation','name':'kitten_labeling','params':{'instruction':'please label the kittens in this image'},}headers={"Content-Type":"application/json"}create_project_request=requests.post("https://api.scale.com/v1/projects",json=payload,headers=headers,auth=('{{ApiKey}}',''))printcreate_project_request.json()

constrequest=require('request');constpayload={type:'annotation',name:'kitten_labeling',params:{instruction:'please label the kittens in this image'}};request.post({url:'https://api.scale.com/v1/projects',json:true,auth:{user:'{{ApiKey}}',pass:''},body:payload,},(err,response,body)=>{console.log(body);});

{"type":"annotation","name":"kitten_labeling","param_history":[{"instruction":"please label the kittens in the image","version":0,"created_at":"2018-04-20T07:38:32.368Z"}],"created_at":"2018-04-20T07:38:32.368Z"}

HTTP Request

POST https://api.scale.com/v1/projects

Parameters

Parameter

Type

Description

type

string

The task type of all the tasks belonging to this project

name

string

Name identifying this project. Must be unique among all your projects. When creating tasks for this project, you should add a project: <name> parameter to the task creation request to associate the task with this project.

params (optional)

object

Default parameters for a task created under this project. Currently only supports params.instruction, which get appended to the instructions of any task created under this project.

constrequest=require('request');constpayload={instruction:'please label the kittens and the cats in this image'};request.get({url:'https://api.scale.com/v1/projects/kitten_labeling',json:true,auth:{user:'{{ApiKey}}',pass:''},},(err,response,body)=>{console.log(body);});

{"type":"annotation","name":"kitten_labeling","param_history":[{"instruction":"please label the kittens in the image","version":0,"created_at":"2018-04-20T07:38:32.368Z"}],"created_at":"2018-04-20T07:38:32.368Z"}

HTTP Request

GET https://api.scale.com/v1/projects/:projectName returns the information for a project with name :projectName.

GET https://api.scale.com/v1/projects returns information for all projects owned by a user.

List Tasks

To retrieve a list of tasks that belong to a particular project, use the project option in the List Tasks Endpoint.

importrequestsimportjsonpayload={'instruction':'please label the kittens and the cats in this image'}headers={"Content-Type":"application/json"}set_params_request=requests.post("https://api.scale.com/v1/projects/kitten_labeling/setParams",json=payload,headers=headers,auth=('{{ApiKey}}',''))printset_params_request.json()

constrequest=require('request');constpayload={instruction:'please label the kittens and the cats in this image'};request.post({url:'https://api.scale.com/v1/projects/kitten_labeling/setParams',json:true,auth:{user:'{{ApiKey}}',pass:''},body:payload},(err,response,body)=>{console.log(body);});

require"net/http"require"uri"uri=URI.parse("https://api.scale.com/v1/projects/kitten_labeling/setParams")req=Net::HTTP::Post.new(uri,'Content-Type'=>'application/json')req.basic_auth('{{ApiKey}}','')req.body={instruction: 'please label the kittens and the cats in this image'}.to_jsonresponse=http.request('request')printresponse.body

The above command returns an object structured like this:

{"type":"annotation","name":"kitten_labeling","param_history":[{"instruction":"please label the kittens in the image","version":0,"created_at":"2018-04-20T07:38:32.368Z"},{"instruction":"please label the kittens and the cats in the image","version":1,"created_at":"2018-04-20T10:38:32.368Z"}],"created_at":"2018-04-20T07:38:32.368Z"}

HTTP Request

POST https://api.scale.com/v1/projects/:projectName/setParams

Projects keep a history of the parameters that they were set with. Tasks created under a project inherit the latest params of the project (the last entry in param_history), one can also specify project_param_version when creating a task to inherit from an older set of params (or use -1 to skip parameter inheritance altogether.)

Currently we only support params.instruction, which get appended to the instructions of any task created under this project.

Batches

Batches are a way to logically group together tasks under the same project.

Note that the API key you use to create batches/projects/tasks with must all match: test projects can only be used with test batches, and test tasks; likewise with live projects, live batches, and live tasks.

To use a batch for a given task, add a batch parameter when creating the task that references the batch name; e.g., to associate a task with batch my_batch, add batch: my_batch to the task creation request. Note that batchs created using your test API key can only be used for creating test tasks; likewise with the live API key and live tasks.

This endpoint returns the status of a batch with name :batchName, as well as the counts of its tasks grouped by task status.

HTTP Request

GET https://api.scale.com/v1/batches/:batchName/status

List All Batches

This is a paged endpoint for all of your batches. Batches will be returned in descending order based on created_at. Pagination is based off limit and offset parameters, which determine the page size and how many results to skip.

HTTP Request

POST https://api.scale.com/v1/batches/:batchName/finalize

Callbacks

The callback_url will be POSTed with application/json data of the following object form:

{"task":{"task_id":"576c41bf13e36b0600b02b34","completed_at":"2016-06-23T21:54:44.904Z","response":{"category":"red"},"created_at":"2016-06-23T20:08:31.573Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"completed","instruction":"Is this object red or blue?","params":{"attachment_type":"text","attachment":"tomato","categories":["red","blue"]},"metadata":{}},"response":{"category":"red"},"task_id":"576c41bf13e36b0600b02b34"}

On your tasks, you will be required to supply a callback_url, a fully qualified URL that we will POST with the results of the task when completed. The data will be served as a JSON body (application/json). Alternately, you can set a default callback URL in your profile, which will be used for tasks that do not specify one.

Additionally, in order to simplify testing and add support for email automation pipelines, you may provide an email address as the callback_url. In this case, each completed task will result in an email sent with the body as the task’s JSON payload.

You should respond to the POST request with a 2xx status code. If we do not receive a 2xx status code, we will continue to retry up to 20 times over the course of the next 24 hours.

If we receive a 2xx status code, the task will be populated with a true value for the callback_succeeded parameter. Otherwise, if we do not receive a 2xx status code on any retry, the task will be populated with a false value for the callback_succeeded parameter.

Getting Started

We have sample callback server implementations, which you can deploy to Heroku in seconds, in the following languages:

If you’re just testing and want to try a few requests, the easiest way to get started is to use a RequestBin and send requests using your https://someHash.x.pipedream.net URL as the callback_url. You can also use ngrok to expose a local server to the internet for fast prototyping.

Authentication

If you’d like to authenticate our callbacks, we set a scale-callback-auth HTTP header on each of our callbacks. The value will be equal to your Live Callback Auth Key shown on your dashboard. If this header is not set, or it is set incorrectly, the callback is not from Scale.

POST Data

Attribute

Type

Description

task_id

string

The task_id is the unique identifier for the task.

status

string

The status of the task when it was completed. Normally completed, but can also be error in the case that a task failed processing.

response

object

The response object of the completed request. For categorization, it will contain a category attribute of the assigned category.

{"task_id":"576ba74eec471ff9b01557cc","completed_at":"2016-06-23T09:10:02.798Z","created_at":"2016-06-23T09:09:34.752Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"completed","instruction":"Would you say this item is big or small?","params":{"attachment_type":"text","attachment":"car","categories":["big","small"]},"response":{"category":"big"},"metadata":{}}

This endpoint retrieves a specific task.

HTTP Request

GET https://api.scale.com/v1/task/{TASK_ID}

URL Parameters

Parameter

Description

task_id

The task_id of the task to retrieve

Returns

Returns a task if a valid identifier was provided, and returns a 404 error otherwise.

{"task_id":"576ba74eec471ff9b01557cc","created_at":"2016-06-23T09:09:34.752Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"canceled","instruction":"Would you say this item is big or small?","params":{"attachment_type":"text","attachment":"car","categories":["big","small"]},"metadata":{}}

This endpoint cancels a task so that it will not be completed.

You may only cancel pending tasks, and the endpoint will return a 500 error code if you attempt to cancel a completed task.

HTTP Request

POST https://api.scale.com/v1/task/{TASK_ID}/cancel

URL Parameters

Parameter

Description

task_id

The task_id of the task to cancel

Returns

Returns the canceled task if a valid identifier for a pending task was provided, and returns a 404 error or 500 error otherwise.

List All Tasks

varscaleapi=require('scaleapi');varclient=scaleapi.ScaleClient('{{ApiKey}}');// specify URL parameters as properties in the params objectvarparams={};client.tasks(params,(err,tasklist)=>{// do something with tasklist});

{"docs":[{"task_id":"576b998b4628d1bfaed7d3a4","created_at":"2016-06-23T08:10:51.032Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"completed","instruction":"Is this object big or small?","params":{"attachment_type":"text","attachment":"ant","categories":["big","small"]},"completed_at":"2016-06-23T19:36:23.084Z","response":{"category":"small"},"metadata":{}},{"task_id":"576ba301eed30241b0e9bbf7","created_at":"2016-06-23T08:51:13.903Z","callback_url":"http://www.example.com/callback","type":"categorization","status":"completed","instruction":"Is this object big or small?","params":{"attachment_type":"text","attachment":"T-Rex","categories":["big","small"]},"completed_at":"2016-06-23T09:09:10.108Z","response":{"category":"big"},"metadata":{}}],"total":2,"limit":100,"offset":0,"has_more":false}

This is a paged endpoint retrieves a list of your tasks. The tasks will be returned in descending order based on created_at time. The pagination is based on the limit and offset parameters, which determine the page size and how many results to skip.

HTTP Request

GET https://api.scale.com/v1/tasks

URL Parameters

Parameter

Type

Description

Required

start_time

ISO 8601 Date

The minimum value of created_at for tasks to be returned

optional

end_time

ISO 8601 Date

The maximum value of created_at for tasks to be returned

optional

status

string

The status of the task - can be: completed, pending, or canceled

optional

type

string

The type of the task - can be: transcription, categorization, comparison, annotation, datacollection, audiotranscription, or any other task type.

optional

project

string

The name of the project that the returned tasks must belong to.

optional

batch

string

The name of the batch that the returned tasks must belong to.

optional

limit

integer

A number between 1 and 100, the maximum number of results to display per page

Idempotent Requests

Scale supports idempotency for task creation endpoints. This allows you to resubmit requests that may have failed in transit without risk of creating duplicate tasks.

To use this feature, provide a header Idempotency-Key: <key>. The client is responsible for ensuring the uniqueness of your chosen keys. We recommend using V4 UUIDs.

The results of requests specifying an idempotency key are saved. If we later receive a matching request with the same idempotency key, the saved response will be returned, and no additional task will be created. Note that this behavior holds even when the response is an error. Keys are removed after 24 hours.

If an incoming request has the same idempotency key as a saved request, but the two requests do not match in parameters, or the users associated with the two requests are different, we will return a 409 error.

In rare situations, we may return a 429 error if two matching requests with identical idempotency keys are made simultaneously. In this case, it is safe to retry.

Errors

//exampletask.responsewithfailedattachment{"error":"One or more attachments could not be downloaded.",//reasonforerror"attachments":[{"statusCode":403,//HTTPcodereceivedwhenfetchingattachment"url":"http://example.com/kitten.png",//attachmentURL}]}

Scale uses the following HTTP codes:

Error Code

Meaning

200

OK – Everything worked as expected.

400

Bad Request – The request was unacceptable, often due to missing a required parameter.

Conflict – The provided idempotency key is already in use for a different request.

429

Too Many Requests – Too many requests hit the API too quickly.

500

Internal Server Error – We had a problem with our server. Try again later.

In the event of one or more task attachments having invalid data, the task callback will be invoked with an error response detailing the problems, and the task’s status will be set to error. Additional detail about the error and failed attachments will be stored in the task response.