As Open Data Hub acquires partners and clients, its architecture has to change to adapt to the growing number of requests, the need for a more resilient system, and a more reactive response. In the past year, Animshon worked alongside the Open Data Hub team to design and build the new core of Open Data Hub. A real-time, event-driven, integrable Open Data Hub.

1. Open Data Hub
ARCHITECTURE DESIGN BY ANIMESHON

2. 01 – Open Data Hub
02 – Technologies
03 – A new way to communicate
04 – Reactivity
05 – Plug & Research
00 – Introduction
AGEND
A
06 – Conclusion

3. 00
INTRODUCTIO
N

4. Like to a post
send notification to
users
weather
forecast
model
improve future
forecasts
notify consumers
recommendation
engine
recommend ads
update feed
update profiling
information
Resolve Captcha
IoT sensor
sending pressure
feedback to
AI models
protect from
unauthorized actors
collect data for
models training

5. Open Data hub
A modern system which already serves
thousands of citizens while processing
data from dozens of providers
Open Data Hub v2
A system designed to meet future
requirements while reaching more people
with more and faster information
State of the art
technologies.
International standards
enforcement

6. 01
OPEN DATA HUB

7. A distributed, event-driven,
zero-downtime system
Open Data Hub:

8. distributed
microservices
Open Data Hub is a cluster of countless
microservices
single responsibility
Each service has only one responsibility which
can be carried out in the most efficient way

9. event-driven
async communication
The subprocess chain communicates
asynchronously, making each component
independent from the others
service self-healing
Events are thrown in the system, and the
consumer can read them when they are ready.
If something goes down, events are preserved
until it comes back online
agnostic communication
Services do not know who will be the next
process to take control, they just inform the
system about their own results

10. zero-downtime
no data loss
Once an information enters the system, it will be
preserved and backed up to ensure data integrity
and data durability
update rollout
Service orchestration allows the team to rollout
updates with different strategies zero or minimal
downtime
fault tolerant
Even if multiple services are offline, the system
can still accept data and serve results

11. 02
TECHNOLOGIES

12. no vendor lock-in
All technologies are provider
agnostic and can be migrated to
any private or public cloud provider
open source
All technologies are open source
easy to contribute
Open Data Hub’s public repository
allows anyone to start a local
cluster of the core architecture
and start contributing

13. anyone can locally instantiate an Open Data
Hub cluster in just 20 minutes and start
contributing to the project
https://github.com/noi-techpark/odh-infrastructure-v2

14. 02
A NEW WAY TO
COMMUNICATE

15. Open Data Hub uses a standard communication protocol to both
serve and process incoming data in alignment with internet 2.0
common practices
standard communication protocol
Speaking a common language makes it more appealing for third
parties to integrate with Open Data Hub
easy to integrate
A more structured and standardized communication protocol allows
for a reduction of custom solutions and more precise specifications
open specifications

16. RestApi
Open Data Hub enforces communication standards
with providers.
MQTT Websocket

17. 04
REACTIVITY

18. easy to maintain
Changing the order of the
blocks, or substitute some of
them will not affect the rest of
the system
The whole architecture is
composed by small
lego-blocks
modular design

19. All lego-blocks can be
rearranged in whatever form
to satisfy new requirements
adaptability
The system is designed to
react to future needs and
evolve
evolution by design

20. 05
PLUG & RESEARCH

21. Open Data Hub allows for
isolated environments to
utilize historical raw data
Thanks to no-code tools
researches could create
data processing pipeline
without IT knowledge
Allowing institutions to
experiment with both
processed and raw data
enables an exponential
grow of the system itself
sandbox
semplicity
grow

22. 06
CONCLUSION

23. Open Data Hub promotes open source by
adopting only open technologies while
publishing it’s core infrastructure
open
State-of-the-art technologies and modern paradigms
ensure an easy maintenance and integrability
innovative
Designed to meet any present and future requirement
scalable

24. Thank you