presented at Web Unleashed 2019
For more info see https://fitc.ca/event/webu19/
Kevin Daly RBC Ventures
Every developer has faced the difficult choice of deciding what tech stack they should use for a new project. Should you use the latest tech or something that everyone knows? Which framework is the best for your team? To survive your tech stack, developers must make trade-offs with developing on new tech stacks and the ability to maintain and scale their applications.
In this presentation, you’ll learn how to evaluate your tech stack and understand the pros and cons of using bleeding edge technology. Using his past experiences, Kevin will also share his lessons learned and how his team tackles managing their tech stack today.
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Surviving your Stack: Choosing the Right Tech Stack
1. Surviving your
StackMAGING
RETIREMENTWeb Unleashed 2019 – September
Kevin Daly - Lead Digital Architect – RBC Ventures
kevin.daly@rbc.com
https://www.linkedin.com/in/kevin-daly-50247a1b/
@therealkedaly
2. 2
Who am I?
Lead Digital Architect for RBC Ventures
• Architecture team is responsible for advising all Ventures on architecture
• Encouraging and influencing best practices for Ventures
Experienced cloud architect with over 20 years experience
I’ve been building software and systems since I was 15
3. 3
Why should you listen to me?
• Because I know everything! I’ve been doing this for years!
8. RBC Ventures
goes beyond banking
to deliver unique value
for all Canadians
BUSINESS
MOBILITY
CONSUMER
HOME
$
$
$
DIVERSIFIED
DISTRIBUTION
STRATEGIC
PARTNERSHIPS
PERSONALIZED
ADVICE
DATA AND
TECHNOLOGY
SCALE
PROPRIETARY
REWARDS
INNOVATIVE
PRODUCT LEADER
8
WELLNESS
Now, we’re moving beyond
traditional banking
9. Keeping you in the know
on the little things
Making renting hassle free
for renters and landlords
Making moving
hassle free
Connecting consumers to
trusted contractors
Helping Canadian SMEs
sell in China
Helps entrepreneurs start
their business
Digital glovebox and
servicing for car owners
9
Home search,
reimagined
BUSINESS
MOBILITY
CONSUMER
HOME
WELLNESS
Tracking taxes for the
self-employed
BOOMERANG
Helping retirees make
connections
Securing sale-by-owner
payments
Snapshot of ventures
Helping Canadians make
better spending decisions
Helping you get
more value
Helping newcomers be
successful in Canada
Finding dream jobs for
recent graduates
Amplifying value for
Canadians
Making living with friends
stress free
Earn cashback at
local merchants
Public transit arrival times
and locations
12. 12
So what is a stack?
A software stack is a group of programs that work
in tandem to produce a result or achieve a common
goal.
https://www.techopedia.com/definition/27268/software-stack
13. 13
My view of a stack
Users
•Build/Test
•Develop•Commit
•Runtime Environment• • ••Commit •Build/Test •Deploy ••Runtime
Environmen
t
20. 20
akka is state of the art!
• Cool languages like Scala
• Functional!
• Reactive!
• Concurrent!
• Distributed!
• We could run it on our Kubernetes cluster
• Build out an AI System maybe
• Kafka Cluster
• maybe even build a Block Chain !
21. 21
But…
• Distributed Applications are hard
• Actors are a pretty new pattern
• Clusters have some maintenance and support issues
• Nobody at this company understood how this thing worked and the guy who
wrote it left the company.
22. 22
So Why?
• 100’s per second
100 * 60 seconds is = 6,000 per minute
X 60 minutes
360,000 per hour
X 24 hours
8,640,000 documents per day
40. 40
Why did we choose this architecture?
•People•Process•Technology
• Hire talent easily
Identify talent and hire from larger talent pool
• Transferable internal skills
Developers can support and switch teams as
needed.
• Simple
Easily understood development stack
• Rapid development cycle
• Able to leverage open source and tech
community
•People
•Technology•Process
• Use leading technology
AWS Cloud
• Be scalable and versatile
PostGres is an extremely versatile database.
• Full Stack Development
• Find support and knowledge
NodeJS has a great community
41. 41
Lessons Learned “The good stuff”
•Productivity + Cost •Job Candidate Availability•Consistency•Agility •Flexibility
42. 42
Lessons Learned “Gotchas”
WTF Is a Full Stack Developer?
What Happens with a partial product market fit?
Technical Debt?
Node Stack doesn’t fit every use case?
43. 43
Takeaways
Understand your scale from a most pessimistic to a most
optimistic view
Don’t over engineer!
Know the size of your market
Avoid the hype machine!
44. 44
And the most Important Takeaway
•People
•Technology•Process
45. 45
Well Almost… Final Thought
Dogmatically pursing a stack as a single
source of truth is a recipe for disaster!
46. 46
Kevin Daly
Lead Digital Architect for RBC Ventures
Connect with me
Email - kevin.daly@rbc.com
LinkedIn - https://www.linkedin.com/in/kevin-daly-50247a1b/
@therealkedaly
Notes de l'éditeur
I’ve got years of experience
I’ve worked as a Principal Engineer in the valley
I’ve been part of successful startups
<insert pause>
- Pretty much everything in this presentation is common sense
Not enough decisions we make are data driven in tech
- which is surprising for “engineers”
- fashion is king (MongoDB, NoSQL, Cassandra, “Full Stack” – all things that promise Valhalla
- often decisions are made on “My favorite tool”
I personally fell in love with In Memory databases
- tried to apply them everywhere
- let’s all go for drinks! I’m done
- Pretty much everything in this presentation is common sense
Not enough decisions we make are data driven in tech
- which is surprising for “engineers”
- fashion is king (MongoDB, NoSQL, Cassandra, “Full Stack” – all things that promise Valhalla
- often decisions are made on “My favorite tool”
I personally fell in love with In Memory databases
- tried to apply them everywhere
- let’s all go for drinks! I’m done
These are all areas where decisions need to be made
Your stack is an important part of your corporate culture
It goes beyond technology
I can build quickly in all of these different tech stacks
React / ReactNative / NodeJS / PostGres
Java / Flutter / Dart / PostGres
.Net core
Python
Choosing a stack is one of the most important strategic things a startup can do.
You don’t have to choose your stack forever
But you need to understand all of the ramifications of your choices
But first…
- Engineering teams like to think that everything is a big problem to solve.
Over engineering is a huge risk, building things that are way too complex to fit the use case
Making sure you understand what you are trying to accomplish
We’re going to start with a story about epic over engineering
Who here has ever over engineered something here?
<pause to question audience>
Started a new job as the Lead Engineer for a company, my first task was to look into their document ingestion pipeline
Some new features had to be added
There were some bugs and problems
And to my horror, no documentation, and nobody knew how it worked.
So what was required was to ingest documents from our vendors, and build an output document that would provide metadata to our servers about the content that we were publishing on behalf of our vendors.
So I dug in!
Upon examination the pipeline was an Akka cluster
In a nutshell Akka allows you to build massively distributed applications using something called the “Actor” pattern
The previous architect had chosen akka for this ingestion engine
Everybody loves distributed applications
Akka is “Web scale!”
- Pause for the end of the slide
Pause at end of slide…
Why was this built this way?
Began to question the team and the answer was scale… Our current cluster can scale up to massive amounts of documents.. How many?
Very small cluster, we could scale this up to 1000’s per second easily
So how many documents did we process a day?
10,000 documents a day could have been handled with a very simple function that would be
Easy to understand
Easy to deploy
The takeaway here is don’t over engineer things
Add that Akka is a great tool if you need it, and have the deepest respect for Lightbend
10,000 documents a day could have been handled with a very simple function that would be
Easy to understand
Easy to deploy
The takeaway here is don’t over engineer things
Add that Akka is a great tool if you need it, and have the deepest respect for Lightbend
10,000 documents a day could have been handled with a very simple function that would be
Easy to understand
Easy to deploy
The takeaway here is don’t over engineer things
Add that Akka is a great tool if you need it, and have the deepest respect for Lightbend
we’ve all probably seen this.
It somewhat holds true for building products
We need to balance the choice of moving quickly with building quality.
So how much time do you have?
Time to market
Time to MVP
Don’t worry about the details, move to MVP
Don’t worry about technical debt?
Prove product market fit at all costs
Throw it out once you prove product market fit
Are there tools / stacks that you could use to build fast?
Google App Engine
FireBase
AWS Lambda
Test driven development & TDD Environments
Planning and roadmapping
Continuous Delivery and Build
You can but it takes strategy
Do a little bit of planning
Identify uncertainties and put a box around them
Is Build fast and break things a good idea?
You may never get back to fix them.
It’s a risk
What skills do you have?
Who can you hire?
What will it cost to hire these people?
What are the side effects?
A full stack developer might not have the depth of knowledge.
You might not have funding to have specialized developers
Agility
AWS+Infra as code via Terraform enables agility, experimentation at low cost.
This is instrumental in shaping and continuously evolving reference architecture.
Consistency
Declarative reference architecture has led to consistency across many projects.
Consistency allows us to effectively and efficiently communicate our architecture to other departments.
Flexibility
Many options available to make minor customizations to infrastructure, for example RDS vs Dynamo.
Productivity And Cost
We manage over a dozen Ventures projects, numerous AWS accounts, networks with a very small IT operations team.
Job Candidate and Availability
AWS is leader in public cloud infrastructure
Many job seekers list AWS experience along with other skills.
This slide is hard to read, but love this painting
Take a heuristic view of your stack
it’s not all about technology, it’s about people
You need to align your stack with your people and processes