Microservices are hot! A lot of companies are experimenting with this architectural pattern that greatly benefits the software development process. When adopting new patterns we always encounter that moment where we think 'if only I knew this three months ago'. This talk will be a sneak peak into the world of microservices at Atlassian and reveal what we've learned about microservices: how to arrange, configure and build your code efficiently; deployment and testing; and how to operate effectively in this environment. In this talk you will learn how to immediately apply five simple strategies.

1. Microservices; 5 things I wish I’d known
Vincent Kok
AMSTERDAM 16 - 17 MAY 2017

2. Lives in Sydney
Move to the other side of the
world to join Atlassian
About me: @vincentkok
Confluence
Development manager on
the Confluence team
Dutch
Lived most of my live 30 mins from
Amsterdam

3. Microservices
Everybody seems to want them. Do we
really know the impact of our choices?
Why do we want them so badly?
Microservices are messy!
https://flic.kr/p/9u5pDA

4. http://geek-and-poke.com/geekandpoke/2013/7/13/foodprints

5. Grow Fat
Code base grows. All
the things slow
down.
Age
Your code base will
become a jurassic
park introducing new
tech becomes hard
Ownership
Who is responsible
for which part and
more important: who
has the pager
Economies of
Scale
The bigger the team
the more they
interrupt each other
Monolithical issues

7. 81000
Build jobs ran last week

8. 31992
Automated tests

9. Cause of issues can be extremely hard

10. INCIDENT RESPONSE
Who is having
the pager?

11. Remember, we’re not all
webscale

13. Optimise for rapid and
sustainable flow of value
DAN NORD

14. Small
The size will be
reasonable and
manageable
Independent
lifecycle
Nothing will hold the
team back. Go as
fast as you can
Optimise for
the problem
Pick solution and
tech based on the
problem at hand
Replaceable
It is easier to replace
if there is a need for
it
The microservice promise

15. CONFLUENCE EXAMPLES

16. CONFLUENCE EXAMPLES
Scheduler
Attachments
Operational
Transformation
Platform Services

17. CONFLUENCE EXAMPLES
Scheduler
Attachments
Operational
Transformation
Platform Services
Front end

18. CONFLUENCE EXAMPLES
Core functionality
Scheduler
Attachments
Operational
Transformation
Platform Services
Front end

19. 5 patterns
Basics
Deployments
Testing
Security
Operations
https://flic.kr/p/9t2138

20. #1: Basics
https://flic.kr/p/5E9ZF

21. Creating a call-out
Watch the tutorial in the
Presentation Guidelines to learn
how to create call-outs on
screenshots within this template.

22. Treat them as cattle, not
pets
BILL BAKER

23. A MINIMAL SERVICE
Health check
200 app is alive. 500 app is unhealthy,
destroy the node
Stateless*
Run as many nodes as you need
Expose a port
Only access to the service

24. DEEPCHECK
Deep check
Quickly discover if a service
fails to connect to a dependency

25. DEEPCHECK EXAMPLE
{
"avatar": {
"details": {
"avatarRepository": {
"isHealthy": true
},
"crowd": {
"isHealthy": true
},
"deadlock": {
"isHealthy": true

26. CODE & BUILDS
1 repository 1 build

27. Strict separation of config
from code
12 FACTOR APP

28. Redeploy
Part of the service
configuration.
Configuration lifecycles
Instant change
Switches you would like to
enable/disable straight away
Rebuild
Rebuild to apply changes

29. #2: Deployments
https://flic.kr/p/qP31Tf

30. Only one person
There is only one person in
the team that owns it
Deployment smells
Takes more then 15
mins
Setting it up should be quick
and initial deployment should
quick
Requires a ticket
A ticket for the deployment
team

31. Always deploy an empty
service into production…
ME; AND PROBABLY OTHERS

32. Developers in control
Artifact
What is the artifact we’re running.
We’re mostly standardising on Docker
Resources
What resources are requires: RDS,
SQS, Dynamo etc..
Compute
What EC2 instance do we want how
many of those and when to scale
Alarms
What are the alarm thresholds for this
service
Ownership
Who is owning the service
Configuration
We will be adding more icons as need
arises. Speak up if in need!

33. DECLARATIVE DEPLOYMENT
name: Confluence
description: Confluence Vertigo
links:
binary:
type: docker
name: docker.atlassian.io/confluence
tag: latest
healthcheck:
uri: /wiki/internal/healthcheck
deepcheck:
uri: /wiki/internal/deepcheck
semanticCheck:

34. CONFIGURATION
config:
environmentVariables:
ASAP_AUDIENCE: "foo"
ASAP_ISSUER: "foo"
CONFLUENCE_VERTIGO_SMTP_HOST: "smtp.foo.com"
CONFLUENCE_VERTIGO_SMTP_PORT: "587"
LOG4J_EXTRA_RULES: "log4j.logger.org.hiberate=DEBUG"
environmentOverrides:
staging:
config:
environmentVariables:
ASAP_PUBLIC_KEY_FALLBACK_REPOSITORY_URL:

35. RESOURCES
resources:
- type: sqs
name: default
attributes:
MaxReceiveCount: 20
VisibilityTimeout: 60
scaling:
instance: m3.xlarge
min: 7

36. 500
Services in production

37. #3: Testing
https://flic.kr/p/hn4K4b

38. Testing microservices

39. Testing microservices

40. TESTING MONOLITHS IS EASY

41. Unit
Integration
UI

42. TESTING
Live service
Test agains a real service

43. TESTING
Mock service
Test against a mock service

44. In process
A local implementation of
your client
Out of process
Use tools like WireMock and
MockServer
Two options

45. MOCKING SERVICES - IN PROCESS
<beans profile=“integration-test">
<bean id="attachmentService"
class=“c.a.attachment.AttachmentStub”/>
</beans>

46. MOCKING SERVICES - WIREMOCK
{
"request": {
"url": “/rest/api/content“,
"method": “POST”
"Accept": {
"matches": “application/json”
}
},
"response": {
"status": 200
}
}

47. Stable API
If it is external it already
should have a CTK so rely on
it
How to trust your mock?
Contract testing
Internal fast moving API’s an
benefit from this
Rely on monitoring
Small service, low MTTR
therefore low impact

48. Semantic Check
Automated test that runs against a
node before it will be added to the
load balancer

49. #4: Security
https://flic.kr/p/7LcF2W

50. OAuth 2.0
Grant a client access to
resources based on a newly
created set of credentials
Common standards
OpenID Connect
Identity on top of OAuth 2
OpenID
Allows identity and some
metadata only

51. SECURING SERVICES
How to secure a
set of many
services

52. ASAP
Atlassian Service Authentication Protocol

53. HOW DOES IT WORK
Foo BarJWT

54. WHATS INSIDE?
Foo Bar
{
"typ": "JWT",
"kid": "foo/key1",
"alg": "RS256"
}
{
"sub": “32769:87e…”
"aud": "bar",
"nbf": 1494284564,
"iss": "foo",
"exp": 1494284624,
"iat": 1494284564,
"jti": “961253cf-ac…”
}

55. s2sauth.bitbucket.io
AVAILABLE ON BITBUCKET

56. #5: Operations
https://flic.kr/p/npbxAm

57. 100 kg
99% water
dehydrate 98%
Guess the weight!
https://flic.kr/p/npbxAm

58. 50
kg

59. Uptime of a system with 30
services of 99.99
WHAT A MICROSERVICE ARCHITECTURE

60. 2 hours
99.99 = 99.7
30

61. RESILIENCE
Failure is
imminent!

62. Circuit breakers
Write code with failure in
mind
Three must haves
Request tracing
Don’t spend hours debugging
Log aggregations
Stream all logs into one
place.

63. DO YOU KNOW YOUR SYSTEM?

64. CREATE INSIGHT: AGGREGATED LOGGING

65. Response times
How much time do spend
calling other services.
Back pressure
Stop putting pressure on a
system that is in trouble
and fail fast
Fallback
How do you handle failure. A
mandatory step in the
programming model.
Circuit breakers

66. CREATE INSIGHT: CIRCUIT BREAKERS

67. Request Tracing

68. Request TracingX-B3-TraceId : 1
X-B3-SpanId : 1

69. Request TracingX-B3-TraceId : 1
X-B3-SpanId : 1
X-B3-TraceId : 1
X-B3-SpanId : 2
X-B3-ParentSpanId : 1

70. Request TracingX-B3-TraceId : 1
X-B3-SpanId : 1
X-B3-TraceId : 1
X-B3-SpanId : 2
X-B3-ParentSpanId : 1
X-B3-TraceId : 1
X-B3-SpanId : 3
X-B3-ParentSpanId : 2

71. Request TracingX-B3-TraceId : 1
X-B3-SpanId : 1
X-B3-TraceId : 1
X-B3-SpanId : 2
X-B3-ParentSpanId : 1
X-B3-TraceId : 1
X-B3-SpanId : 3
X-B3-ParentSpanId : 2
X-B3-TraceId : 1
X-B3-SpanId : 4
X-B3-ParentSpanId : 3

72. TRACE ID’S

74. You Build It You Run It
The team who builds it looks after it.
Ops Team
Handover your services and let them
deal with the fun. Don’t do this.

75. What should you take home?
Basics
Services are cattle not pets.
Testing
Testing a monolith is “easy” what’s
think about your service testing
strategy
Deployment
Deploying a service shouldn’t take
longer then 15 minutes
Operations
You build it you run it.
Security
Think how you would like to secure
service to service communications
Focus on value
Optimise for rapid and sustainable
flow of value

76. VINCENT KOK - ATLASSIAN - @VINCENTKOK
Thank you
CodeMotion Amsterdam 2017