Data Connectors

Data connectors which allow contribution of data into a BitBroker instance

BitBroker is a contribution based system, meaning that data contributed by a community of users. In some cases, these contributors will be people you have direct control over (and may well be other roles you yourself are playing). However, in other instances, you maybe relying upon second and third parties to be contributing data into your BitBroker instance.

The vector through which data contribution is managed is the concept of data connectors. Each entity type within the system should have at least one data connector. However, it is entirely possible for an entity type to be sharing multiple data connectors, all contributing entity instance records for that type.

A data connector can only be contributing data to one entity type. If a party wants to contribute data to multiple entity types, you must create a different connector for each one. These can, in practice, point back to one data connector implementation.

Managing Data Contribution

Whether or not the contribution is coming from internal or external players, the first step in managing the process is to create a data connector - housed within the entity type for which it has permission to submit records. Creating and managing data connectors can only be done by coordinator users.

As part of the creation process for connectors, the system will generate a connector ID and an authorization token. These will be returned to the coordinator user in response to the creation request. The coordinator should communicate these items in a secure manner to the party responsible for implementing the data connector. Here is an example of these items:

{
    "id":"9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "token":"ef6ba361-ef55-4a7a-ae48-b4ecef9dabb5.5ab6b7a2-6a50-4d0a-a6b4-f43dd6fe12d9.7777df3f-e26b-4f4e-8c80-628f915871b4"
}

Armed with these two items, data connectors can go ahead and contribute entity instance records into the catalog, by using the Contributor API. You should direct new data connectors to the contributor documentation in order to get them started with their implementation.

Live vs Staging Connectors

By default, newly created connectors are not “live”. This means that data which they contribute will not be visible within the Consumer API. Implementing a data connector can be a tricky operation and may require a few attempts before data is being delivered to a sufficient quality. Hence, it is useful to isolate contributions from certain connectors into a “staging space” - so that it doesn’t pollute the public space visible to consumers.

In order to make a connector’s data visible, it must be promoted to live status. Only coordinator users can promote connectors in this way. They can also demote connectors, if they believe their data should no longer be publically visible.

When developing a new connector, it is often useful to be able to see data from it alongside other public data. There is a mechanism available in the Consumer API which allows data connectors to see how their records will look alongside other existing public records.