Consumer API

The outward facing API which consumers will use to access data from BitBroker

The Consumer API is the main outward facing API for the BitBroker system. It is the API used by consumers to access data from the system, but always in the context of a data sharing policy.

Paging Lists

It is possible that record lists returned from APIs may run to a great many records. There is a hard limit to how many records will be returned in a single call to any Consumer API.

It is possible to page access to record lists by using a set of URL query parameters, as follows:

Attribute Necessity Description
An integer number of data records, between 1 and 250
A positive integer index of the earlier desired record

If the paging parameters are incorrect, the API will respond with a standard validation error containing details of the violation.

If the specified offset is greater than the count of records available, the API will return an empty list. Using limit and offset you can page your way through a long list, getting the entire list a page at a time.

Rate and Quota Limits

All Consumer API calls happen within the context of a data sharing policy. Amongst other things, policy defines a rate and quota limit on calls you can make with the Consumer API. These should have been communicated to you by the coordinator user who gave you your consumer authorization token.

  • Rate - the maximum calls-per-second that you are allowed to make (e.g. 250)
  • Quota - the total calls you can make in a given period (e.g. 86,400 per day)

If you breach a rate or quota limit then calls to any part of the Consumer API will respond as follows:

HTTP/1.1 429 Too Many Requests

This response will persist until the breach has expired.

All Consumer API calls happen within the context of a data sharing policy. Amongst other things, policy defines the legal context under which data access is permitted. A legal notice will be present at the end of every entity instance record returned by any part of the Consumer API.

These legal notices are in the form of a JSON array of objects, each with three attributes:

Attribute Presence Description
The type of the legal notice
The description of the legal notice
A link for more information about the legal notice

Examples of types of legal notices which may be present are:

Type Description
attribution How the data should be attributed within your application
contact Who to contact about the data and it’s use
license The licenses under which this data sharing operates
note Ad hoc notes about the data and/or it’s use
source Information about the source or origination of the data
terms The terms and conditions of use for the data

It is possible that the coordinator user who gave you your consumer authorization token will have more information about the legal basis of use of the data. They may require you to perform additional legal steps in order to be given a consumer authorization token.

Accessing “Staged” Records

It is important to understand that data connectors might be in a live or staged state. That is, their contribution might be approved for the live catalog, or might be being held back into a staging space only.

When staged, any records they contributed will not be visible in any part of the Consumer API. This concept is outlined in more detail in the key concepts documentation.

When developing a new connector, it is often useful to be able to see data from it alongside other public data. There is a mechanism available in the Consumer API which allows data connectors to see how their records will look alongside other existing public records.

In order to achieve this, connectors authors can send in a list of connector IDs in a private HTTP header attribute to any part of the Consumer API. Only connector authors will be aware of their connector ids. They can send up to 16 connectors in a single request header. The API will then include in responses, records contributed by those connectors as if they were live.

The header they should use for this is x-bbk-connectors. This should be a string value which is a comma separated list of connector IDs without any spaces.

Catalog API

The catalog API enabling search and discovery services for entity instances

Entity API

The entity API enabling direct access to entity types and entity instances

Time Series API

The time series API enabling access to time-value pair datasets within entity instances