The expansion of hydrogen refueling stations in Germany and Europe is progressing rapidly. But how to maintain the overview of all relevant parameters and fault messages? How can the refuelling processes be analysed in realtime in order to improve the systems together with the manufacturers? And how does the data from the system actually get into the monitoring system? This presentation will show how a powerful IoT monitoring solution was developed based on the hardware of the RevPi platform and open source software such as Ansible, Icinga2, InfluxDB and Grafana, which helps to ensure the availability of the hydrogen infrastructure and optimize it further.

1. HRS-Connect
Opensource IIoT-Monitoring

2. 05.11.2019 OSMC 2019 2 / 40
Nicolai Buchwitz
●
CTO @ ENDA KG, Berlin
●
Monitoring solutions for EU, federal agencies and industry
●
Opensource developer
●
Monitoring & M2M consulting
 @NicolaiBuchwitz  nbuchwitz

3. 05.11.2019 OSMC 2019 3 / 40
HRS Connect – Introduction
●
Monitoring of hydrogen refuelling stations (HRS*) in realtime
●
Analysis of refuellings and other metrics
●
Predictive maintenance
●
Maintenance / task scheduler
●
HRS availability API and app for customers
●
(KPI) Reporting
* not the hotel platform :-)

4. 05.11.2019 OSMC 2019 4 / 40
How does a HRS work?

5. 05.11.2019 OSMC 2019 5 / 40
H2 infrastructure in Germany
●
4 HRS manufacturers
●
Liquid hydrogen (LH2) and compressed gas (GH2)
●
15 manufacturer / modell combinations
– … extras like bus extensions
●
H2.live app with live information for customers
H2.live

6. 05.11.2019 OSMC 2019 6 / 40
H2 infrastructure in Germany
HRS in Nuremberg
HRS in Fürth

7. 05.11.2019 OSMC 2019 7 / 40
HRS-Connect – the system

8. 05.11.2019 OSMC 2019 8 / 40
Revolution PI – the industrial Swiss Army Knife
●
Industrial hardware platform based on raspberry pi 3 compute module
●
Lots of extension modules (IO, fieldbus gateways, maker kit, …)
●
RT kernel patches
●
Access data via process image (piControl kernel driver)
●
Open-source hardware specs and source code (GitHub)

9. 05.11.2019 OSMC 2019 9 / 40
HRS-Connect
preprocessing &
transmission via
MQTT (TLS)
sensors data
from HRS PLC

10. 05.11.2019 OSMC 2019 10 / 40
Data collection – hardware
●
Dedicated control cabinet (visu
box)
●
Independent of other HRS
infrastructure
●
Industrial LTE router with IO
●
Self healing (reset relais)

11. 05.11.2019 OSMC 2019 11 / 40
Data collection – agent on RevPi
●
Communication via MQTT (TLS!)
●
Telegraf & InfluxDB
●
Collection and preprocessing of Ø 200 metrics / HRS @ 100-500ms
– Pressure, temperature, ophours, load cycles, alarms, ...
●
Recording of aggregated refuelling logs
●
Monitoring of fill levels (automatic ordering of supplies)

12. 05.11.2019 OSMC 2019 12 / 40
●
Replaced Icinga2 agent for
metrics
●
Python3 & MQTT
●
Easy usage with RevPiModIO2
●
Cyclic and event based collection
of process data
Data collection – RevPiModIO2 & MQTT

13. 05.11.2019 OSMC 2019 13 / 40
Visualization
●
All data is visualized in Grafana ( … and even used by our supervisors!)
●
Custom plugins (e.g. trend panel, ...)
●
Various datasources
– InfluxDB
– PostgresSQL (meta data / non volatile metrics like refuelling logs)
– Custom datasource with metric specifications (unit, tag name,
location, ...)

14. 05.11.2019 OSMC 2019 14 / 40
Visualization - use cases
●
One base dashboard per HRS type
●
Events via annotations (e.g. refueling or alarm from PLC)
●
Special dashboard (e.g. refuelling logs or KPI reporting)
●
„Management dashboard“ @ office

15. 05.11.2019 OSMC 2019 15 / 40
Example: compressor

16. 05.11.2019 OSMC 2019 16 / 40
Refueling logs – archive

17. 05.11.2019 OSMC 2019 17 / 40
Refueling logs – detail

18. 05.11.2019 OSMC 2019 18 / 40
Custom (P&ID) diagrams
https://grafana.com/grafana/plugins/pierosavi-imageit-panel

19. 05.11.2019 OSMC 2019 19 / 40
Monitoring with Icinga2
●
Monitoring of „usual“ metrics on the RevPi
(CPU, memory, disk, temperature, ...)
●
RevPi module configuration and errors
(e.g. wrong wiring or missing module)
●
Pending firmware updates
●
Frozen process image or missing fieldbus link
●
Notification via Telegram (groups / individual)

20. 05.11.2019 OSMC 2019 20 / 40
Monitoring with Icinga2

21. 05.11.2019 OSMC 2019 21 / 40
Monitoring with Icinga2 – fieldbus gateways
All checks for RevPi platform will be released on GitHub shortly!

22. 05.11.2019 OSMC 2019 22 / 40
Monitoring with Icinga2 – the map
●
Where is the station?
Search for name or address
●
Visualization of ongoing
maintenance work
●
Is there a geographic pattern?
Mobile network or power outage
https://github.com/nbuchwitz/icingaweb2-module-map

23. 05.11.2019 OSMC 2019 23 / 40
We dashboards❤️

24. 05.11.2019 OSMC 2019 24 / 40
Monitoring with Icinga2 – dashboards
Is the agent
running?

25. 05.11.2019 OSMC 2019 25 / 40
Monitoring with Icinga2 – dashboards
PLC connected via
fieldbus?

26. 05.11.2019 OSMC 2019 26 / 40
Monitoring with Icinga2 – dashboards
Key metrics and
customer
availability

27. 05.11.2019 OSMC 2019 27 / 40
Monitoring with Icinga2 – dashboards
Current
maintenance and
other downtimes

28. 05.11.2019 OSMC 2019 28 / 40
Monitoring with Icinga2 – dashboards

29. 05.11.2019 OSMC 2019 29 / 40
Monitoring with Icinga2 – mobile devices

30. 05.11.2019 OSMC 2019 30 / 40
Monitoring with Icinga2 – mobile devices

31. 05.11.2019 OSMC 2019 31 / 40
Monitoring with Icinga2 – Integrations

32. 05.11.2019 OSMC 2019 32 / 40
Provisioning & Deployment

33. 05.11.2019 OSMC 2019 33 / 40
Provisioning - Hardware
●
„provisioning desk“
●
Simultaneously deployment
2x9 RevPi
●
Flexible connection sets for
different hardware
configuration
●
Unit testing

34. 05.11.2019 OSMC 2019 34 / 40
Provisioning - Ansible
●
Infrastructure as Code
●
Dynamic inventory with
PostgreSQL CMDB
●
Plug & play in case of hardware
replacement

35. 05.11.2019 OSMC 2019 35 / 40
Provisioning - Ansible
●
Custom facts
– RevPi type
– Module specs and status
●
Coming soon @ GitHub
 nbuchwitz

36. 05.11.2019 OSMC 2019 36 / 40
Provisioning - Ansible low level
●
Flashing images via USB
●
Automatic configuration of
profinet settings
●
Firmware updates on gateways
●
Ansible module & standalone
# update firmware on 10.10.12.98
$ update.py
--hostname 10.10.12.98
--firmware mGateProfinetTps1.kfu
# configure all attached PN devices
$ pnio.py
-i enp0s20f0u4u4u1
-m configure
--station-name hrs-gw2
--ip-address 10.10.12.98
--netmask 255.255.255.0
--all

37. 05.11.2019 OSMC 2019 37 / 40
HRS-Connect in numbers
4
vendors
4
vendors
80
HRS
80
HRS
15
HRS types
15
HRS types
>3.200
refuellings / month
>3.200
refuellings / month
~20.000
metrics / second
~20.000
metrics / second
<= 500 ms
resolution
<= 500 ms
resolution

38. 05.11.2019 OSMC 2019 38 / 40
Opensource keeps the world spinning!
offical armhf packages
(Sponsoring)

39. 05.11.2019 OSMC 2019 39 / 40
Questions?
 @NicolaiBuchwitz  nbuchwitz

40. 05.11.2019 OSMC 2019 40 / 40
Thanks!