Data Connectors

APIs for creating and manipulating data connectors

Connectors are a main component of the BitBroker system. You will find more details about connectors, within the key concepts section of this documentation.

Connectors are always created within the context of housing entity types, which can be created and manipulated using other part of this API, described earlier in this documentation.

Creating a New Connector

New connectors can be created by issuing an HTTP/POST to the /entity/:eid/connector/:cid end-point.

Connectors are always created within the context of a housing entity type and, hence, you must know its ID (eid). In order to create a connector, you must select a unique connector ID (cid) for it.

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --request POST \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Wikipedia", \
         "description": "Country information from Wikipedia", \
         "webhook": "http://my-domain.com/connectors/1", \
         "cache": 0 \
     }
EOF

This will result in a response as follows:

HTTP/1.1 201 Created
Location: http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia

The body of this response will contain the connector ID and connector authorization token, which the new data connector should utilize to make its data contributions. For example:

{
    "id":"9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "token":"ef6ba361-ef55-4a7a-ae48-b4ecef9dabb5.5ab6b7a2-6a50-4d0a-a6b4-f43dd6fe12d9.7777df3f-e26b-4f4e-8c80-628f915871b4"
}

The following validation rules will be applied to the body of a new connector request.

Attribute Necessity Validation Rules
cid
required
String between 3 and 32 characters long
Consisting of lowercase letters, numbers and dashes only
Starting with a lowercase letter
Conforming to the regex expression ^[a-z][a-z0-9-]+$
name
required
String between 1 and 64 characters long
description
required
String between 1 and 2048 characters long
webhook
optional
String between 1 and 1024 characters long
Must conform to URI format
cache
optional
Integer between 0 and 31536000

Details about the meaning of these attributes can be found within the key concepts section of this documentation.

Updating a connector

Existing connectors can have their profile updated by issuing an HTTP/PUT to the /entity/:eid/connector/:cid end-point.

In order to update a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --request PUT \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Wikipedia", \
         "description": "A new description for the Wikipedia connector", \
         "webhook": "http://my-domain.com/connectors/1", \
         "cache": 0 \
     }
EOF

This will result in a response as follows:

HTTP/1.1 204 No Content

The validation rules for updated connector information, are the same as that for creating new connectors.

List of Existing Connectors

You can obtain a list of all the existing connectors housed within a parent entity type by issuing an HTTP/GET to the /entity/:eid/connector end-point.

curl http://bbk-coordinator:8001/v1/entity/country/connector \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

[
    {
        "id": "wikipedia",
        "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
        "name": "Wikipedia",
        "description": "A new description for the Wikipedia connector"
    }
    // ... other connectors here
]

Each connector, housed within a parent entity type, will be returned within this array. Note: There is currently no paging on this API.

Details of an Existing Connector

You can obtain the details of an existing connector by issuing an HTTP/GET to the /entity/:eid/connector/:cid end-point.

In order to obtain details of a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON object as follows:

{
    "id": "wikipedia",
    "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
    "name": "Wikipedia",
    "description": "A new description for the Wikipedia connector",
    "entity": {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country"
    },
    "contribution_id": "9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "webhook": "http://my-domain.com/connectors/1",
    "cache": 0,
    "is_live": false,
    "in_session": false
}

Other sections of this document will explain what the is_live and in_session attributes refer to.

Promoting a Connector to Live

Connectors can be promoted to live status by issuing an HTTP/POST to the /entity/:eid/connector/:cid/live end-point.

In order to promote a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia/live \
     --request POST \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

When getting details for promoted connectors, their is_live status will be reflected in the corresponding attribute:

{
    "id": "wikipedia",
    "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
    "name": "Wikipedia",
    "description": "A new description for the Wikipedia connector",
    "entity": {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country"
    },
    "contribution_id": "9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "webhook": "http://my-domain.com/connectors/1",
    "cache": 0,
    "is_live": true,
    "in_session": false
}

Demoting a Connector from Live

Existing connectors can be demoted from live status by issuing an HTTP/DELETE to the /entity/:eid/connector/:cid/live end-point.

In order to demote a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia/live \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

When getting details for such users, their is_live status will be reflected in the corresponding attribute:

{
    "id": "wikipedia",
    "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
    "name": "Wikipedia",
    "description": "A new description for the Wikipedia connector",
    "entity": {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country"
    },
    "contribution_id": "9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "webhook": "http://my-domain.com/connectors/1",
    "cache": 0,
    "is_live": false,
    "in_session": false
}

Deleting a connector

Existing connectors can be deleted from the system by issuing an HTTP/DELETE to the /entity/:eid/connector/:cid end-point.

In order to delete a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content