Presentation by Tomas Aliaga Sr. Software Developer, Martel Innovate FIWARE Tech Summit 28-29 November, 2017 Malaga, Spain

1. QuantumLeap: a FIWARE time-series DB
Tomas Aliaga
tomas.aliaga@martel-innovate.com

2. Some background
1

3. 2
Orion: Review
▪ Main NGSI Concepts:
▪ Entity Id
▪ Entity Type
▪ Attributes
▪ By design, Orion CB persists only
the last pushed value of each attribute.
{ https://github.com/Fiware/dataModels/blob/master/Weather/
WeatherObserved/example.json
"id": "Spain-WeatherObserved-Valladolid-2016-11-30T07:00:00.00Z",
"type": "WeatherObserved",
"address":
{
"addressLocality": "Valladolid",
"addressCountry": "ES"
},
"barometricPressure": 938.9,
"dataProvider": "TEF",
"dateObserved": "2016-11-30T07:00:00.00Z",
"location":
{
"type": "Point",
"coordinates":
[
-4.754444444,
41.640833333
]
},
"precipitation": 0,
"pressureTendency": 0.5,
"relativeHumidity": 1,
"source": "http://www.aemet.es",
"stationCode": "2422",
"stationName": "Valladolid",
"temperature": 3.3,
"windDirection": -45,
"windSpeed": 2,
"refDevice": "device-0A3478"
}

4. 3
Comet: Review
▪ Comet brings Historical Data to FIWARE in two forms:
• Raw Queries
http://fiware-sth-comet.readthedocs.io/en/latest/raw-data-retrieval/index.html
http://<sth-host>:<sth-port>/STH/v1/
contextEntities/type/<entityType>/id/<entityId>/attributes/<attrName>?
hLimit=3&hOffset=0&dateFrom=2016-01-01T00:00:00.000Z&dateTo=2016-01-31T23:59:59.999Z
• Aggregated Queries
http://fiware-sth-comet.readthedocs.io/en/latest/aggregated-data-retrieval/index.html
http://<sth-host>:<sth-port>/STH/v1/
contextEntities/type/<entityType>/id/<entityId>/attributes/<attrName>?
aggrMethod=sum&aggrPeriod=second&dateFrom=2016-02101T00:00:00.000Z&dateTo=2016-01-
01T23:59:59.999Z

5. 4
Comet: What’s missing
▪ Support for NGSI v2.
• Discussed in GH #323
▪ Support for Cross-Entities Queries.
• E.g: Avg temperature of all weather station meters in Madrid?
• Discussed in GH #430
• /STH/v1/contextEntities/type/<entityType>/id/<entityId>/attributes/<attrName>?
▪ Support for other than MongoDB. Why?
• Difficult to scale it properly.
• Schemaless -> no joins -> not easy cross-document queries.
• Many new options for efficient and powerful TimeSeries data handling.

6. QuantumLeap’s Approach
5

7. 6
Typical Scenario
▪ 1: NGSI Subscription
▪ 2: NGSI Insert/Update
▪ 3: Orion Notification (/notify)
▪ 4: Client Queries
▪ 5: Client Visualisations

8. 7
1: NGSI v2 Subscription

9. 8
3: The API proposal: INPUT

10. 9
4: The API proposal: QUERIES

11. 10
4: The API proposal: PARAMETERS
Query
parameter
Example Value
type WeatherObserved,
id station01,station02,
idPattern .*
attrs temperature,pressure
Query
parameter
Example Value
aggrMethod avg
aggrPeriod month
options see NGSIv2
Query
parameter
Example Value
from
2016-01-
01T00:00:00.000Z
to
2016-03-
01T00:00:00.000Z
lastN 100
limit 100

12. 11
The API proposal: META
/config endpoint to bring optimization opportunities through the customization of:
▪ Number of replicas
▪ Number of shards
▪ Partitioning by index

13. Live Example
12

14. How to get started
13

15. 14
Docs Available!
DOCS: https://smartsdk.github.io/ngsi-timeseries-api/
REPO : https://github.com/smartsdk/ngsi-timeseries-api

16. 15
Report Issues!
https://github.com/smartsdk/ngsi-timeseries-api/issues
Don’t forget to include QL’s output log.
▪ When Deployed as a Service
• $ docker service ls
• Get ID of QuantumLeap service
• $ docker service logs QL_SERVICE_ID
▪ When Deployed with Compose
• $ docker ps
• Get ID of container
• $ docker logs QL_CONTAINER_ID

17. Thank you!
http://fiware.org
Follow @FIWARE on Twitter
16
https://smartsdk.github.io/ngsi-timeseries-api/
Don’t miss Today!
- Docker Swarm secrets … (14:30 @ ROOM 1)
- Mastermind ... (15:00 @ ROOM 1)

19. 18

20. 19

21. 20

22. 21

23. Thank you!
http://fiware.org
Follow @FIWARE on Twitter