This document discusses integrating the Presto data platform with the Consul service mesh. It begins by introducing Presto and Consul and their purposes. It then describes initial attempts to integrate them using Consul Connect which became complex due to additional requirements. The document outlines refactoring to instead use a native integration approach between Presto and Consul that is simpler. It concludes by emphasizing the importance of simplifying state-of-the-art technologies and being bold in experimenting with open-source tools.

1. A Consul Service Mesh
Integration Case Study with
Presto
Arturo Viveros
Fredrik Grelland

2. What makes a state-of-the-art data
platform?

3. Hello Presto

4. Hello Consul

5. ➔ Software Development
➔ Data Engineering
➔ Service Integration
Arturo Viveros (@gugalnikov)
Fredrik Grelland (@grelland)
https://sysco.no/

7. What makes a state-of-the-art data
platform?

10. Consul Connect integration is quite
straightforward

11. Consul Connect: Service Mesh w/ sidecar proxies

12. ...until it isn’t

13. Consul Connect: Introducing an opinionated application

14. Consul Connect: securing proprietary internal communication

15. Consul Connect: 1...n coordinators bring even more complexity

16. ...so, back to the drawing board

17. Native Integration isn’t scary

18. Native Integrating Presto

19. Consul Connect: Native integration

20. Don’t be afraid of the bleeding edge

21. Final Demo
https://asciinema.org/a/jMSEPX5ms8NuyA37swzEGQJs5

22. We should be stopped by policy, not by
technology

23. ➔ State-of-the-art should be “simple as that”
➔ Refactor to reduce, redesign to refine
➔ Fortune favors the bold: compile, fork & run your open-source components
Our Personal Takeaways

24. Thanks!!!