The document discusses common mistakes made when prioritizing speed over quality, such as de-emphasizing testing, releases, operations, insights, security, and knowledge. It recommends focusing on system design, configurations, limits, growth, processes, resources, and building resilience through redundancies and documentation. Testing the full system, having playbooks, and minding assumptions and dependencies are emphasized.

1. the Fallacy
Of Fast

2. INES
Sombra
@Randommood

4. Obligatory Disclaimer
Thingsyou will see in thistalk
Fast Talking & Opinions TM
Un-tweetable moments
Rantifestos TM
Questionable language
A therapy llama
and ZERO Kevin Bacons
@Randommood

5. Common
Mistakes
What
Matters
@Randommood

6. SPOILER alert
@Randommood

7. @Randommood

8. Reason about design
choices in terms of
trade-offs
@Randommood

9. chosen trade-offs
make the foundation
ofyour system
@Randommood

10. @Randommood

11. @Randommood

12. @Randommood

13. @Randommood

14. Common
Mistakes

15. Accidentally
de-emphasizing
long term quality&
system Stability
@Randommood

16. De-prioritizing Testing
Cutting corners on testing
carries a hidden cost
Test the full system: client,
code, & provisioning code
Code reviews != tests.
Have both
Continuous Integration
(CI) is critical to velocity,
quality, & transparency

17. De-prioritizing Releases
Release stability is tied to system
stability
Iron out your deploy process!
Dependencies on other systems
make this even more important
Canary testing, dark launches,
feature flags, etc are good
@Randommood

18. Automation shortcuts taken while
in a rush will come back to haunt
you
Playbooks are a must have
Localhost is the devil
Sloppy operational work is the
mother of all evils
De-prioritizing Ops
!
@Randommood

19. “Future you monitoring” is
bad, make it part of MVP
Alert fatigue has a high
cost, don’t let it get that far
Link alerts to playbook
Routinely test your
escalation paths
De-prioritizing Insight
✨
@Randommood

20. The inner workings of data
components matter.
Learn about them
System boundaries ought
to be made explicit
Deprecate your go-to
person
De-prioritizing Knowledge
@Randommood

21. The internet is an awful
place
Expect DoS/DDoS
Think about your system, its
connections, and their
dependencies
Having the ability to turn off
features/clients helps
De-prioritizing Security

22. Service ownership implies
leveling-up operationally
Architectural choices
made in a rush can have
a long shelf life
Don’t sacrifice tests.
Test the FULL system
Whatwe learned
✨
@Randommood

23. RANTIFESTO!
@Randommood

24. building shrines of Agile
Assuming a given
methodology will solve
everything is naive at best
Magical thinking leads to
misaligned expectations
All tools are terrible, avoid
religious wars
#

25. Anarchy
Complex
Complicated
Complicated
Simple
Closetoagreementfarfromagreement
Close to certainty far from certainty
* “When to Scrum?” stolen from Angela Druckman
Requirements
Technology
Scrum

26. @Randommood

27. @Randommood

28. @Randommood

29. @Randommood
$%&%
'''''''' '

30. ( (
@Randommood

31. “In truth a range of approaches, a
hybrid mix, of management methods is
required to succeed in today's
enterprise IT environment. That customer
enterprise environment never was like
the simplified product development
environment where Agile software
development was conceived…”
@Randommood

32. “In truth a range of approaches, a
hybrid mix, of management methods is
required to succeed in today's
enterprise IT environment. That customer
enterprise environment never was like
the simplified product development
environment where Agile software
development was conceived…”
@Randommood
DUH

33. Agile Gotchas
Uncertainty in problem
domain (and company
size) will challenge your
ability to adhere to it
Has a cost but it’s different
Nihilism FTW?

34. )
?
@Randommood

35. What
Matters
@Randommood

36. Mind system Design
Simple & utilitarian design
takes you a long way
Use well understood
components
NIH is a double edged
sword
Use feature flags & on/off
switches (test them!)
@Randommood

37. Meet
AliceI’mwaytoo
coolforthis
outfit

38. Alice’s Testing Areas
Correctness Error Performance Robustness
Good output
from good
inputs
Reasonable
reaction to
incorrect
input
Time to Task (TTT) for Behavior after
Goal
Single node
Multi node
Clustered
Cache enabled
Given # of
input/outputs
Given uptime
@Randommood

39. a Testing Harness
Is a fantastic thing to have
Invest in QA automation
engineers
Adding support for
regressions & domain-
specific testing pays off
@Randommood

40. Mind system Configs
System assumptions are
dangerous, make them
explicit
Standardize system
configuration (data bags,
config file, etc)
Hardcoding is the devil

41. Mind system Limits
Rate limit your API calls
especially if they are public or
expensive to run
Instrument / add metrics to
track them
Rank your services & data
(what can you drop?)
Capacity analysis is not dead
✨

42. Mind system Growth
Watch out for initial over-
architecting
“The application that takes
you to 100k users is not the
same one that takes you to
1M, and so on…” @netik
Expect changes & refactors
@Randommood

43. Mind Process
Architectural reviews FTW
Request flow, API shape,
Failure conditions, Reliability,
Data Model, Threat
modeling, Testing strategy,
Operations, Monitor logging
& Alerting, Pricing/Billing,
Supported clients, etc
@Randommood

44. Mind Resources
Redundancies of resources,
execution paths, checks,
replication of data, replay of
messages, anti-entropy build
resilience
Mechanisms to guard system
resources are good to have
Your system is also tied to the
resources of its dependencies

45. Distrust is healthy
Distrust client behavior, even
if they are internal
Decisions have an expiration
date. Periodically re-
evaluate them as past
you was much dumber
A revisionist culture produces
more resilient systems
✨
@Randommood

46. about Resilience
Traditional
engineering
Reactive
ops unk-unk
* Stolen from Paul Borrill
Cascading or catastrophic
failures & you don’t know
where they will come from!
Same area as other 2 combined
Probability
of failure
Rank

47. classical
Engineering
reactive
Operations
unk-unk
Building Resilience
Code standards
Programming patterns
Testing (full system!)
Metrics & monitoring
Convergence to good
state
Hazard inventories
Redundancies
Feature flags
Dark deploys
Runbooks & docs
Canaries
System verification
Formal methods
Fault injection
The goal is to build
failure domain
independence

48. Keep track of your
technical debt &
repay it regularly
It’s about lowering
the risk of change
with tools & culture
Mind assumptions
Whatwe learned
✨
@Randommood

49. TL;DR
Easy to sacrifice things
may be harder to
correct later
Think in terms of
tradeoffs
TESTING MATTERS!
Not all process is evil
Keep in Mind
Make system
boundaries &
dependencies explicit
Playbooks are your
friends, have them
Use kill switches & limits
Prioritize your services
Distributed systems

50. github.com/Randommood/FallacyOfFast
@Randommood