Catalog API

The catalog API enabling search and discovery services for entity instances

All the entity instances being managed by BitBroker are recorded with the catalog. This Catalog API is used to search this index and to discover entity instances which are needed by applications. The Catalog API returns a list of entity instances, which are then accessible via the Entity API.

In this section, we will explore the capabilities of the Catalog API in depth.

A quick way to get going with building your own applications is to adapt the example apps which use this and the other Consumer APIs.

All Consumer API calls happen within the context of a data sharing policy. The policy defines a data segment which you are allowed to access. Any queries which you make using the Catalog API can only operate within the data segment you are permitted access to.

In our sample calls, we use the standard server name and port format outlined for local installations and demos. If you have an alternative API host base URL, then enter it in the box below to update all the sample calls on this page :

Your API Host Base URL

The Consumer API can be run in production or development mode; each requiring a different header. If you are running a local installation of BitBroker, then you can switch the sample calls on this page to development mode by activating the toggle below. If you are unsure, leave this toggle off.

Apply development mode

All API calls in BitBroker require authorization. The sample calls below contain a placeholder string where you should insert your consumer API authorization token. This authorization token should have been provided to you by the coordinator user who administers the BitBroker instance. If you already have a token, enter it in the box below to update all the sample calls on this page:

Your Consumer API Authorization Token

When installing BitBroker locally, authorization is bypassed. In this scenario, you need to specify the policy you are using in a development header. Enter your desired policy in the box below to update all the sample calls on this page:

Your Current Policy

Querying the Catalog

You can query the catalog by issuing an HTTP/GET to the /catalog end-point.

curl http://bbk-consumer:8003/v1/catalog \
     --header "x-bbk-auth-token: your-header-goes-here"

This will return an empty JSON array.

The catalog API will return a list of entity instances which match the submitted query string. The query string is submitted by adding a q URL parameter to the call. Submitting no query string, results in an empty array (as opposed to all items).

Lists retrieved from this API will be returned in pages

For example, using the query string:

q={"name" : "India"}

Would return the following JSON array:

[
    {
        "id": "917d0311c687e5ffb28c91a9ea57cd3a306890d0",
        "url": "http://bbk-consumer:8003/v1/entity/country/917d0311c687e5ffb28c91a9ea57cd3a306890d0",
        "type": "country",
        "name": "India",
        "legal": []
    }
]

Here we have one matching entity instance. The API returns Entity API links to matching instances.

In the examples here we show the query strings in readable JSON format, but when actual calls are made, the query string is required to be url encoded.

Querying Options

Here, we will go through each available query option one-by-one, giving an example of each.

All query strings are url encoded JSON objects. The query string syntax used is loosely based on the query language used within Mongo DB.

Implicit Equality

This query format is a shorthand for using the $eq operator

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "name": "India" }'

Equals / Not Equals

The $eq and $ne operators work for a range of data types such as integers, floats and strings.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.capital": { "$eq": "London" } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.capital": { "$ne": "Paris" } }'

Less Than (or Equal to)

The $lt and $lte operators work for a range of data types such as integers, floats and strings.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.population": { "$lt": 100000 } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.population": { "$lte": 100000 } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "name": { "$lt": "China"} }'

Greater Than (or Equal to)

The $gt and $gte operators work for a range of data types such as integers, floats and strings.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.population": { "$gt": 100000 } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.population": { "$gte": 100000 } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "name": { "$gt": "Mexico"} }'

In / Not In

The $in and $nin operators are for searching with array parameters.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "name": {
         "$in": ["United Kingdom", "India", "France"]
     } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "name": {
         "$nin": ["United Kingdom", "India", "France"]
     } }'

Contains

The $contains operator is for searching within array values.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.languages": { "$contains": [ "Japanese" ] } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.languages": { "$contains": [ "Japanese", "English" ] } }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "entity.location.coordinates": { "$contains": [ -3.435973, 55.378051 ] } }'

Logical Operators

The $and, $or and $nor operators can be combined in any combination.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "$and": [
         { "entity.continent": "Europe" },
         { "entity.population": { "$gt": 50000000 } }
     ] }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "$or": [
         { "entity.continent": "Europe" },
         { "entity.population": { "$gt": 50000000 } }
     ] }'

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "$nor": [
         { "name": "United Kingdom" },
         { "entity.population": { "$gt": 50000000 } }
     ] }'

Regular Expressions

The $regex operator allows for the use of regular expressions to query any string attribute.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "name":
         { "$regex": "United .*", "$options": "i" }
     }'

Geo Spatial Queries

The $near operator is used to find entity instances close to a GeoJSON geometry. The $min and $max parameters are specified in meters. You must specify either one of $min or $max or both together. The $geometry attribute must be present.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={
         "entity.location": {
             "$near": {
                 "$max": 750000,
                 "$min": 0,
                 "$geometry": {
                     "type": "Point",
                     "coordinates": [
                         -3.435973,
                         55.378051
                     ]
                 }
             }
         }
     }'

The $within operator is used to find entity instances inside a closed GeoJSON geometry. The $geometry attribute must be present.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={
        "entity.location": {
          "$within": {
            "$geometry": {
              "type": "Polygon",
              "coordinates": [
                [
                  [ -12.386341, 59.062341 ],
                  [ -12.386341, 49.952269 ],
                  [   2.500282, 49.952269 ],
                  [   2.500282, 59.062341 ],
                  [ -12.386341, 59.062341 ]
                ]
              ]
            }
          }
        }
      }'

Querying for Timeseries

You can also search for entity instances which have timeseries associated with them, by using catalog query calls. For example:

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "timeseries.population.value": "people" }'

This call will find all entity instances which have a timeseries called population with values of people.

curl http://bbk-consumer:8003/v1/catalog \
     --get \
     --header "x-bbk-auth-token: your-header-goes-here" \
     --data-urlencode 'q={ "timeseries.noise.unit": "decibel" }'

This call will find all entity instances which have a timeseries called noise with a unit of decibel.