In the end, it has similar goals to those of FIWARE's STH Comet. However, Comet does not yet support NGSIv2, it's strongly tied to MongoDB, and some of the conditions and constraints under which it was developed are no longer hold. That being said, there is nothing wrong with it; this is just an exploration on a new way to provide historical data for FIWARE NGSIv2 with different time-series databases as backend.
The idea is to keep the time-series database swappable so as to look forward to having support for different ones. We started testing InfluxDB, RethinkDB and CrateDB. However, we have decided for now to focus the development on the translator for CrateDB because of the following advantages:
- Easy scalability with containerised database cluster out of the box.
- Geo-queries support out of the box
- Nice SQL-like querying language to work with
- Supported integration with visualisation tools like Grafana
Typical Usage and How it works
The typical usage scenario for QuantumLeap would be the following (notice the numbering of the events)...
The idea of QuantumLeap is pretty straightforward. By leveraging on the NGSIv2 notifications mechanism, clients first create an Orion subscription (1) to notify QuantumLeap of the changes in the entities they care about. This can be done either through QuantumLeap's API or directly talking to Orion. Details of this process are explained in the Orion Subscription part of the User Manual.
Then, new values arrive in Orion Context Broker (2) for the entities of interest, for example from a whole IoT layer governed by 1 or more IoT Agents pushing data in NGSI format. Consequently, notifications will arrive from Orion to QuantumLeap's API /v2/notify endpoint (3).
QuantumLeap's Reporter submodule will parse and validate the received notification. If configured, it can do some preprocessing, for example supported by the Geocoder such as harmonising the location representation of the different entities. Eventually, the Reporter will pass a validated NGSI representation of received entities to the configured Translator. The Translator is ultimately responsible for persisting the NGSI information to the configured times-series database cluster. In the same way, the Translator supports the Reporter at the time to respond to queries of historical records.
The current API includes some endpoints for raw and aggregated data retrieval (4) for clients to query historical data. It also supports deletion of historical records. Please note not all endpoints are currently implemented in QL. For more info about the API, you can refer to the NGSI-TSDB specification.
For the visualisation of data (5), at the time being we are using Grafana, complemented with open source plugins for the databases (known as data-sources). In the future, we could envision a Grafana plugin for direct interaction with QL's API.
- Refer to the Admin Guide to learn more about installing QuantumLeap and getting it running.
- Refer to the User Manual to learn more about how to use it and connect it to other complementary services.
- Have a look at the SmartSDK guided tour for more examples of QuantumLeap usage.
- Have a look at the FIWARE Step by Step