Dataset API¶

Methods¶

Below are available methods in the Newton client, grouped by the relevant context.

Dataset¶

Below methods operate within the Dataset context

1.create_dataset()

2.get_dataset()

3.update_dataset()

4.list_datasets()

5.delete_dataset()

create_dataset(body)¶

Creates a new dataset based on configuration settings provided in the JSON body.

Sample Invocation

from pprint import pprint
from objectways import newton
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")
body = {
'dataset_name': 'Test Dataset Demo',
'dataset_description': 'This is a test Dataset',
'dataset_items': 0,
'dataset_type': 'Image',
....
}
# check API docs for the full JSON body structure https://api.tensoract.com/docs/#/projects/upload_project
pprint(client.create_dataset(body))

type body:: JSON Object
param body:: JSON object with elements defined as here

get_dataset(dataset_id)¶

Fetches configuration details of an existing dataset in the workbench based on the provided dataset_id parameter.

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

# Invoke get_dataset method
response = client.get_dataset(dataset_id="449354de1168469a8229f605")
pprint(response)

Parameters:: dataset_id – Internal dataset_id assigned to the Dataset within Newton Workbench.

Response

The requested dataset’s configuration is returned in JSON format.

{
'dataset_name': 'Test Dataset Demo',
'dataset_id': '5c1d6892f4712fde11d8e25a',
'dataset_description': 'This is a test Dataset',
'dataset_items': 0,
'dataset_type': 'Image'
.....
}

update_dataset(dataset_id, body)¶

Updates configuration of an existing dataset in the workbench based on the provided dataset_id and body JSON parameter.

Sample Invocation

  from pprint import pprint
  from objectways import newton

  # Initialize Newton client
  client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

  # Set the JSON config in body
  body = {
'dataset_name': 'Test Dataset Demo',
'dataset_description': 'This is a test Dataset',
'dataset_items': 0,
'dataset_type': 'Image'
  ......
}

# Invoke update_dataset method
  response = client.update_dataset(dataset_id="449354de1168469a8229f605",body=body)
  pprint(response)

Parameters:

dataset_id – Internal dataset_id assigned to the Dataset within Newton Workbench.
body (JSON Object) –
JSON object with elements defined as here

Response

The updated dataset’s new configuration is returned in JSON format.

 {
   'dataset_name': 'Test Dataset Demo',
   'dataset_description': 'This is a test Dataset',
   'dataset_items': 0,
   'dataset_type': 'Image'
     ......
}

get_datasets(dataset_id, dataset_name, active)¶

Fetches a list of datasets that meet the criteria defined by the parameters (dataset_id or dataset_name or active status)

Sample Invocation
from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

# Invoke get_datasets method
response = client.get_datasets(dataset_id=None,dataset_name="Test Dataset Demo",active=True)
pprint(response)
type project_id:

String

param dataset_id:

Internal dataset_id assigned to the Dataset within Newton Workbench.

type dataset_name:

String

param dataset_name:

Dataset Name defined for this dataset in Newton workbench

type active:

Boolean

param active:

Dataset state is active or not.

All the above three parameters can be applied as filters with AND criteria. When all three parameters are set to None, then no filter is applied and all datasets are returned

Response

JSON list of datasets is returned as below

[
    {'company': '60f38299446de8dabe9207e5',
    'company_name': 'Objectways',
    'dataset_id': '2dfc93fb1723921e8bad8027',
    'dataset_name': 'Test Dataset Demo',
    'dataset_type': 'Image',}

]

delete_dataset(dataset_id)¶

Deletes an existing dataset in the workbench based on the provided dataset_id parameter.

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

# Invoke delete_dataset method
response = client.delete_dataset(dataset_id="8bc05ec26ab161e218a0e842")
pprint(response)

Parameters:

dataset_id (String) –

Internal dataset_id assigned to the Dataset within Newton Workbench.

Response

Status of the delete operation is returned as a JSON

{
'message': 'dataset deleted',
'status': 1
}

Dataset_Items¶

Below methods operate within the Dataset Items context

1.add_new_items_to_dataset_item()

2.get_dataset_item()

3.delete_dataset_item()

4.list_dataset_items()

create_dataset_item(body)¶

Add dataset items to a given dataset

Admin can post the metadata with the items as below

Sample Invocation

from pprint import pprint
from objectways import newton

body = {
"dataset_id": "208eb554d3fa1e161b2c6a44",
"dataset_items": [
{
  "meta_data": {
    "batch": "10",
    "test": "cell"
  },
  "source": "s3://mybucket/test-dataset/file1.tiff"
},
{
  "meta_data": {
    "batch": "1",
    "test": "cell"
  },
  "source": "s3://mybucket/test-dataset/file2.tiff"
  }
  ....
  ....
]
}
# Invoke create_dataset_item method
pprint(client.add_task(body))

Parameters:

body (JSON Object) –

JSON Object with following elements

dataset_id - daatset to which the items needs to be added

Metadata - Metadata can de addedwith dataset item

source - link to S3 source of the document/image.

Response

Status of operation is returned as a JSON

{'inserted_items': 11, 'status': 1}

get_dataset_item(item_id)¶

Retrieve a dataset item

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

data = client.get_dataset_item(item_id="449354de1168469a8229f605")
pprint(data)

Parameters:

item_id (String) –

Internal item_id for the given dataset item in Newton workbench

Response

Dataset Item details returned as a JSON

[{'company': '60f38299446de8dabe9207e5',
'company_name': 'Objectways',
'dataset_id': '2ed8316fe3691cea93b8d70c',
'file_name': '10_2a68e073-b0b2-eb11-80cb-484d7e9f90c1.tiff',
'file_type': 'image/tiff',
'intermediate_source': 's3://newton-demo-examples/sample.png',
'item_id': '0d1f56964171e372afa90323',
'source': 'ss3://newton-demo-examples/sample.tiff',
'state': 0,
'version': [0]}]

delete_dataset_item(item_id)¶

Delete the given dataset item

Sample Invocation

    from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

ata = client.delete_dataset_item(item_id="6bae961fcebe81cbacbedec6")
pprint(data)

Parameters:: item_id (String) – Internal item_id for the given dataset item in Newton workbench

Response

Status of the operation is returned as a JSON
{
 "message": "dataset item deleted",
"status": 1
}

list_dataset_items(dataset_id, dataset_version)¶

Retrieve a list of dataset items matching the filter criteria

Sample Invocation

dataset_items = client.list_dataset_items( dataset_id=”2ed8316fe3691cea93b8d70c”, dataset_version=”0”)

Parameters:

dataset_id – Internal dataset_id for the given dataset in Newton workbench
dataset_version (String) – Internal dataset_version for the given datasetk in Newton workbench

Response

JSON List of tasks matching the filter criteria.

{'company': '60f38299446de8dabe9207e5',
'company_name': 'Objectways',
'dataset_id': '3aa168a963f1d4dbe0b1f549',
'file_name': 'violet.tiff',
'file_type': 'image/tiff',
'item_id': '6bae961fcebe81cbacbedec6',
'meta_data': {'color': 'violet'},
'source': 'file:3aa168a963f1d4dbe0b1f549/6bae961fcebe81cbacbedec6',
'state': 0,
'tags': {'a dataset tag': 1},
'version': [0]}

Export-Project¶

Below methods operate within the Project Export context

1.export_project_tasks()

2.list_project_exports()

3.download_project_export()

export_project_tasks(project_id)¶

Export all dataset items from a given project as JSON list

Sample Invocation

 from pprint import pprint
 from objectways import newton

 # Initialize Newton client
 client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

projects = client.export_project_tasks(project_id="1e80e389c318d5746169c5b6"),
export_note="This is test export via API",
pprint(projects)

Parameters:: project_id (String) – Internal project_id for the given project in Newton workbench

Response

Export status

{'message': 'The export is being generated, please try again in a few seconds '
            'to check the export status',
            'progress': '0%'}


{'progress': '100%'}

list_project_exports(project_id)¶

This method enlisits the exports exported in the project.

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")
response = client.list_project_exports( project_id="449354de1168469a8229f605")

Parameters:: project_id (String) – Internal project_id for the given project in Newton workbench

Response

[{ 'created': 1667897879,
   'createdBy': 'abc@example.com',
   'exportId': '**************',
   'filter': '',
   'items': 41,
   'note': 'This is test export via API',
   'project': 'd2ff8489f14f0b1b59ec1df7',
   'projectName': 'test10',
   'selection': False}]

download_project_export(export_id)¶

Export status with url

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

response = client.download_project_export(export_id="449354de1168469a8229f605")

Response

{'export_url': 'https://devdocuments.objectways.com/export_2022-10-20T09:54:14.7355669_project1.manifest',
 'message': 'Use the export_url to download the project export file',
 'progress': '100%',
 'status': 'COMPLETED'}

Export-Dataset¶

Below methods operate within the Project Export context

1.export_dataset()

2.list_dataset_exports()

3.download_dataset_export()

export_dataset(dataset_id)¶

Export all dataset items from a given dataset as JSON list.

Dataset can be exported with GroudTruth filter

Sample Invocation

 from pprint import pprint
 from objectways import newton

 # Initialize Newton client
 client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

 response = client.export_dataset(
 dataset_id="3aa168a963f1d4dbe0b1f549",
 export_note="This is test export via API",
 project_id="449354de1168469a8229f606")

:type dataset_id: String
:param project_id: project_id is optional

       Internal project_id for the given project in Newton workbench

Response

Export status
{'progress': '100%'}

list_dataset_exports(dataset_id)¶

This method enlisits the exports exported in the project.

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

response = client.list_dataset_exports(dataset_id="449354de1168469a8229f605")

Parameters:: project_id (String) – Internal project_id for the given project in Newton workbench

Response

[
    {'created': 1667901440,
    'createdBy': 'abc@example.com',
    'datasetId': '3aa168a963f1d4dbe0b1f549',
    'exportId': '***************',
    'filter': '',
    'items': 44,
    'note': 'This is test export via API',
    'project': '',
    'projects': [],
    'selection': False}]

download_dataset_export(export_id)¶

Export status with url

Sample Invocation

from pprint import pprint
from objectways import newton

# Initialize Newton client
client = newton.Newton(api_url="YOUR_API_URL",api_key="YOUR_API_KEY")

response = client.download_dataset_export(
export_id="449354de1168469a8229f605")

Response

{'export_url': 'https://devdocuments.objectways.com/export_2022-08-20T14:55:09.5593106_goldenset1.manifest',
'message': 'Use the export_url to download the project export file',
'progress': '100%',
'status': 'COMPLETED'}