Principal Engineer – Workday’s Grid Cloud Master team.
This is Simon Wardley, as the CEO of Canonical, was one of the few people to predicted the cloud and the disruption it would cause.
He is why Ubuntu is the most deployed Linux distribution in the Cloud.
He uses Pioneers, Settlers and Town planners as a way to think about the roles and attitudes required for technological innovation and change
https://irishamericancivilwar.com/2016/04/09/the-voices-of-californias-irish-pioneers/
Pioneers are able to explore new concepts. Sometimes their ideas are crazy. They work quickly, they bend rules and cut corners. You don’t always trust what they build, but they invent the future.
http://www.sfgate.com/news/article/The-Great-Quake-1906-2006-Days-before-the-2520536.php
Settlers turn these half baked things into something useful. They discover patterns, and figure out how to apply them. They build trust. They make the possible, happen. They turn the idea, the prototype, the MVP into a product or service people want use.
https://www.vice.com/en_au/article/watch-dogs-2-embraces-san-franciscos-long-weird-history-as-a-hackers-paradise
Town planners take products or services and industrialise them. They find ways to make them faster, better, smaller, more efficient, profitable.
These are different attitudes, different roles and sometimes even different people
I want you to keep these three concepts in mind, now as we discuss microservices
What is full stack developer? What does it have to do with microservices
Full stack developers have a mix of skills, language skills like java or ruby to write the service, maybe some Docker, Casandra for storage and Chef or Ansible for deployment
A lot us in Workday are or have been full stack developers. In one of my first roles here I had to mixing Java, Rabbitmq, and Ruby with Chef, to build our Messaging service
http://www.wordclouds.com/
1 ElasticSearch
1 JavaScript
1 Promethius
1 Wavefront
1 OpenStack
1 LogStash
1 Casandra
1 Graphite
1 RabbitMq
1 Postgres
1 Riemann
1 Grafana
1 Ansible
1 VMWare
1 Kibana
1 Docker
1 Puppet
1 Hadoop
1 Python
1 Neo4j
1 Redis
1 Spark
1 Kafka
1 Scala
1 Mysql
1 Chef
1 Ruby
1 Java
1 Logs
1 SDN
1 AWS
But why do we do it? Why does one person take on so many rolls?
https://www.flickr.com/photos/photolaphoto/12127113904
Agility – we want to deliver to production early, but we don’t know what its going to look like yet.
We want to own both pieces of the puzzle, because we have not figured out where to implement the solution.
Describing what we want in Jira will take longer than just doing it.
If there is a problem we want to fix it quickly – we don’t want a bun fight of where or how to fix it.
This is Creation, Invention, Innovation
If we think about, these are all pioneer activities.
Happily I am saying that the pioneer days of microserices are over, the technology and practices are starting to settle.
But why?
Interfaces
We needed this agility because we didn’t know what to put above or below the line, we didn’t know where the line was, so we assumed it didn’t exist
But now micoservices are maturing the industry is coalescing around a set of interfaces, we are agreeing where to draw these lines
Just like operating systems provide a neat hardware abstraction to processes. Enabling the operating system and the processes we write to evolve independently.
So…
… for microservices the line is between container and some form of orchestration like Kubernetes / Mesos
The services is implemented in a docker container, they describe the process that has to be launched and monitored, encapsulate its dependencies and specify the the ports it uses to communicate with the outside world
Next you describe where and how you want your services to be deployed and managed with either Kubernetes or Mesos
But is this it? Well
Externally Workday looks like an application company, but ERP is actually a suite of very large applications
Internally we are platform company we are an ERP application factory
So how does “Platform Thinking” change the way we think about microservices?
We look for the patterns, this is another property of settlers, we see what works and plan how to replicate it.
Not all companies are the same. And not all microservices are the same.
We are looking at the patterns in our services and what makes them specific to us? What’s our magic sauce? Is it how we scale Zookeeper, collect logs and metrics mange tenant lifecycle. James will talk about how we discovered and documented these things.
We are currently building on Docker and Kubernetes, encapsulating what we learned into …
Customised Orchestration
For general services: provide common Command and Control to service teams, this will guide the pipeline that delivers their service (in a container) to production, and ensure their service is easy for our ops team to operate.
Other services are more specific their lifecycle is bound to the application and how workday manages tenants, here we automate tenant level orchestration and scalability while at the same time ensuring tenant isolation/fairness and safety.
Other companies have taken this path: Google have the Borg, Netflix have just released Titan. And I have heard of projects in other companies implementing similar customised orchestration.
I am currently building the second of these orchestration services, and I am not a full stack developer any more I am not a pioneer I am a settler
I am building on the interfaces provided to me by Jonathon, Owen and Andrea here today and many others. In turn I will be providing a clean container based interface to my customers.
Just like working with lego, these interfaces are constraints. But more importantly they enable my service to concentrate on delivering value to my customers, while they concentrate on delivering value to theirs. Then when join it all together we can all innovate and deliver regular continuous improvements to all our customers.
Full stack developers and other microservice pioneers helped us all to discover these interfaces through their successes and their mistakes.
Microservice infrastructure is largely a solved problem this reenables developer specialisation again, now is the time to innovate on the customer value our services provide
Principal Engineer – Workday’s Grid Cloud Master team.