Consumer API

The Consumer API is the main outward facing API for the BitBroker system. It is the API used by consumers to access data from the system, but always in the context of a data sharing policy.

Paging Lists

It is possible that record lists returned from APIs may run to a great many records. There is a hard limit to how many records will be returned in a single call to any Consumer API.

It is possible to page access to record lists by using a set of URL query parameters, as follows:

If the paging parameters are incorrect, the API will respond with a standard validation error containing details of the violation.

If the specified offset is greater than the count of records available, the API will return an empty list. Using limit and offset you can page your way through a long list, getting the entire list a page at a time.

Rate and Quota Limits

All Consumer API calls happen within the context of a data sharing policy. Amongst other things, policy defines a rate and quota limit on calls you can make with the Consumer API. These should have been communicated to you by the coordinator user who gave you your consumer authorization token.

• Rate - the maximum calls-per-second that you are allowed to make (e.g. 250)
• Quota - the total calls you can make in a given period (e.g. 86,400 per day)

If you breach a rate or quota limit then calls to any part of the Consumer API will respond as follows:

HTTP/1.1 429 Too Many Requests

This response will persist until the breach has expired.

Legal Notices

All Consumer API calls happen within the context of a data sharing policy. Amongst other things, policy defines the legal context under which data access is permitted. A legal notice will be present at the end of every entity instance record returned by any part of the Consumer API.

These legal notices are in the form of a JSON array of objects, each with three attributes:

Examples of types of legal notices which may be present are:

It is possible that the coordinator user who gave you your consumer authorization token will have more information about the legal basis of use of the data. They may require you to perform additional legal steps in order to be given a consumer authorization token.

Accessing “Staged” Records

It is important to understand that data connectors might be in a live or staged state. That is, their contribution might be approved for the live catalog, or might be being held back into a staging space only.

When staged, any records they contributed will not be visible in any part of the Consumer API. This concept is outlined in more detail in the key concepts documentation.

When developing a new connector, it is often useful to be able to see data from it alongside other public data. There is a mechanism available in the Consumer API which allows data connectors to see how their records will look alongside other existing public records.

In order to achieve this, connectors authors can send in a list of connector IDs in a private HTTP header attribute to any part of the Consumer API. Only connector authors will be aware of their connector ids. They can send up to 16 connectors in a single request header. The API will then include in responses, records contributed by those connectors as if they were live.

The header they should use for this is x-bbk-connectors. This should be a string value which is a comma separated list of connector IDs without any spaces.