In H2020 EU project symbIoTe (symbiosis of smart objects across IoT environments) we have been building IoT middleware based on microservices programmed in Java with Spring Boot and Spring Cloud components. Here I will present our experiences in developing such services in distributed team across EU and employed by 15 organizations. I will present organizational and technical advantages and drawbacks as well as our choices in building such system.

1. © 2019 – The symbIoTe Consortium
Building IoT Middleware with Microservices
Mario Kušek, University of Zagreb, FER
Javantura v6, 23 Feb 2019, Zagreb
Grant Agreement No 688156

2. © 2019 – The symbIoTe Consortium2
Who am I?
@MarioKusek
mario.kusek@fer.hr
University of Zagreb
Faculty of Electrical Engineering and Computing
Department of Telecommunications
IoT Laboratory

3. © 2019 – The symbIoTe Consortium3
• What is ”thing”?
– Object from physical world (physical object with
build in sensors and/or actuators) or virtual object
– Internet Connected Object (ICO)
• Has unique identifier and is connected to the Internet
• Communicates and generate data (reading from
environment)
• Can receive data/commands from network
• Can execute commands – actuate (electrical or
mechanical)
• Can receive data from other ICO, process them and
send for processing to cloud
Internet of Things - IoT

4. © 2019 – The symbIoTe Consortium4
• IoT platforms integrates „things” and
continuously acquire data
– Large distributed system
– Processing large amount of data (often in real time)
– Integrates and saves data from different sources
– For application developers offer:
• Searching for things (sensors/actuators)
• Access to data
• There are more then 400 platforms
– Mostly specialised for one area
How to integrate „things” and provide user applications?

5. © 2019 – The symbIoTe Consortium5
Architecture of IoT System
User
IoT
Platform
Gateway
IoT
Application
Smart spaceCloudUser

6. © 2019 – The symbIoTe Consortium6
• Can not easy create cross domain applications?
• Interoperability:
– Data structures are different
– Different measurement units
– Platforms vertically oriented
• Project: symbIoTe
– Symbiosis of smart objects across IoT environments
– H2020 project: 2016-2018 (3 years)
– 15 EU partners (universities, institutes, SMEs)
Some Challenges

7. © 2019 – The symbIoTe Consortium7
The symbIoTe approach
Core Services
IoT
Platform A
IoT
Platform B
AdaptorAdaptor
Domain
Enabler
Application
Adaptor
IoT
Platform A
IoT
Platform C
Adaptor
Adaptor
IoT
Platform D
Adaptor
IoT
Platform B
Adaptor
Application
Agent
IoT
Gateway A
IoT
Gateway B Smart
Device
Smart Space Gateway
Agent Agent
L1
L2
L3L4
roaming
Smart Space A
Core Services
Smart Device
Smart Space B
Smart Space GatewaySmart Space Gateway
Smart Device

8. © 2019 – The symbIoTe Consortium8
• Development
– Agile process (iterations) – planned 4 week releases
– Weekly meetings of all developers (video conference)
– One or two weeks for internal component releases
– Programming language is Java
– Unit testing: >70% code coverage
• Tools:
– Attlasian Jira as the feature planning tool
– github.com as code repository tool
– Git as versioning and code revisioning tool
– Travis CI as Continuous Integration server
Project organisation

9. © 2019 – The symbIoTe Consortium9
High-level Architecture

10. © 2019 – The symbIoTe Consortium10
Putting it all together: L1 compliance
Components for
semantic and
syntactic
interoperability +
security

11. © 2019 – The symbIoTe Consortium11
• Each component
– Have component owner
– One microservice (mostly in one organisation)
– One repository
• Frameworks:
– SpringBoot, Spring Cloud
– Communication between microservices:
RabbitMQ
Microservices – Decisions

12. © 2019 – The symbIoTe Consortium12
• Spring Cloud Config – configuration
• Zipkin – distributed tracing system
• Spring Cloud Gateway – did not exist then
– We used NGINX
• Spring Cloud Netflix:
– Eureka – service registration
– Ribbon – REST client that uses Eureka
– Feign – declarative HTTP client
• For accessing core services
Spring Cloud

13. © 2019 – The symbIoTe Consortium13
• Microservices were not divided by domains
• Frequent changing of messages
– Hard integration testing, lots of communication
between teams
• RabbitMQ
– Most of communication is request/response
• Architecture not designed for messaging
– No messages (API) versioning
– Late knowledge about Spring Cloud Contract
(v1.0.0 Sep 2016)
• Not so good support for messages (documentation and
examples)
Problems

14. © 2019 – The symbIoTe Consortium14
• Lots of microservices need to implement the
same data objects:
– Put data objects in library: SymbIoTeLibraries
• Put all libraries in Jitpack (https://jitpack.io)
• Problems with versions of library:
– Start using semantic versioning (https://semver.org)
• All components need to contact AAM
(Authentication Authorization Manager)
component
– Created SymbIoTeSecurity library which is dependency
of SymbIoTeLibraries
Problems/decisions

15. © 2019 – The symbIoTe Consortium15
• Each IoT platform needs to deploy cloud
components
– No automatic deployment
• 3 stages/ways of deployment:
– 1. From sources:
• Support web page (github wiki)
• Problems:
– Complex → automatization
– Slow → automatization
– hard to configure → configuration generator (web page
similar to https://start.spring.io)
Deployment (1)

16. © 2019 – The symbIoTe Consortium16
• 2. From jars:
– Created script for downloading, configuring,
starting and stopping
– Problems:
• Microservices have startup dependencies
– Solved by putting small class in main method to wait for some
service (host:port)
• If doesn’t work first time you need to dive into details
– Similar to SpringBoot
• Bash script - problem on windows if someone just want
to try it
Deployment (2)

17. © 2019 – The symbIoTe Consortium17
• 3. From dockers
– Easy startup
– Starting with docker-compose or docker swarm
– Config git repo is in docker image
• Problem of custom configuration
– Putting config in volume
– Different documentation for using docker directly
(linux) or in docker machine
• Problems with port mapping
Deployment (3)

18. © 2019 – The symbIoTe Consortium18
• For testing, hackathons you run components
on machine that is in local network
• Cloud components need to have public IP,
DNS, and certificate (for https)
– Workaround is ngrok tunnelling
• Different NGINX configuration
Cloud components on dev machine

19. © 2019 – The symbIoTe Consortium19
Memory Consumption – Problem
• 1.5GB on startup
• 2-3GB when working
• Solution:
– Java 11: new GC, String internals, netty, …
– Problem: late release of SpringBoot 2.1 (Oct 2018)
and Spring Cloud (Greenwich)(Jan 2019)

20. © 2019 – The symbIoTe Consortium20
• Distributed teams require lots of
communication
• Be careful in choosing technologies
• Divide to microservices that are independent
and scalable (up/down)
• Versioning of API is important
• Dedicate time for building/testing automation
– To it in the beginning of the project and maintain
Conclusions

21. © 2019 – The symbIoTe Consortium
www.symbiote-h2020.eu
middleware.symbiote-h2020.eu
info@symbiote-h2020.eu
@symbiote_h2020
H2020 symbIoTe
github.com/symbiote-h2020
Member of
Thank you!
Questions?
@MarioKusek
mario.kusek@fer.hr