5. Most Startups Fail But it doesn’t have to be that way. We can do better. This talk is about how.
6. What is a startup? A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty. Nothing to do with size of company, sector of the economy, or industry
7. Product / Market Fit “Do whatever is required to get to product/market fit. Including: changing out people, rewriting your product, moving into a different market, telling customers no when you don’t want to, telling customers yes when you don’t want to, raising that fourth round of highly dilutive venture capital —whatever is required.” — Marc Andreessen http://blog.pmarca.com/2007/06/the-pmarca-gu-2.html
8. The Pivot What do successful startups have in common? They started out as digital cash for PDAs, but evolved into online payments for eBay. They started building BASIC interpreters, but evolved into the world's largest operating systems monopoly. They were shocked to discover their online games company was actually a photo-sharing site. Pivot: change directions but stay grounded in what we’ve learned. http://startuplessonslearned.blogspot.com/2009/06/pivot-dont-jump-to-new-vision.html
9. Speed Wins if we can reduce the time between major iterations we can increase our odds of success
14. A good plan? Start a company with a compelling long-term vision. Raise plenty of capital. Hire the absolute best and the brightest. Hire an experienced management team with tons of startup experience. Focus on quality. Build a world-class technology platform. Build buzz in the press and blogosphere.
15. Achieving Failure Company failed utterly, $40MM and five years of pain. Crippled by “shadow beliefs” that destroyed the effort of all those smart people.
19. A good plan? Start a company with a compelling long-term vision. Raise plenty of capital. Hire the absolute best and the brightest. Hire an experienced management team with tons of startup experience. Focus on quality. Build a world-class technology platform. Build buzz in the press and blogosphere.
23. New plan Shipped in six months – a horribly buggy beta product Charged from day one Shipped multiple times a day (by 2008, on average 50 times a day) No PR, no launch Results: 2007 revenues of $10MM
24. Lean Startups Go Faster Commodity technology stack, highly leveraged (free/open source, user-generated content, SEM). Customer development – find out what customers want before you build it. Agile (lean) product development – but tuned to the startup condition.
25. Commodity technology stack Leverage = for each ounce of effort you invest in your product, you take advantage of the efforts of thousands or millions of others. It’s easy to see how high-leverage technology is driving costs down. More important is its impact on speed. Time to bring a new product to market is falling rapidly.
34. Agile Product Development Unit of Progress: A line of Working Code “Product Owner” or in-house customer Problem: known Solution: unknown
35. Product Development at Lean Startup Unit of Progress: Validated Learning About Customers ($$$) Customer Development Hypotheses, Experiments, Insights Problem: unknown Data, Feedback, Insights Solution: unknown
37. How to build a Lean Startup Let’s talk about some specifics. Small Batches Continuous deployment Split-test (A/B) experimentation Minimum Viable Product Five why’s
38. Small Batches IDEAS LEARN BUILD Learn Faster Customer Development Five Whys Build Faster Continuous Deployment Small Batches Continuous Integration Refactoring DATA CODE MEASURE Measure Faster Split Testing Actionable Metrics Net Promoter Score SEM
39. Benefits of Small Batches Faster feedback Problems are instantly localized Reduce risk Reduce overhead
40. Continuous Deployment IDEAS LEARN BUILD Learn Faster Customer Development Five Whys Build Faster Continuous Deployment Small Batches Continuous Integration Refactoring DATA CODE MEASURE Measure Faster Split Testing Actionable Metrics Net Promoter Score SEM
41.
42. At IMVU time from check-in to production = 20 minutes
43. Tell a good change from a bad change (quickly)
61. Split-testing all the time A/B testing is key to validating your hypotheses Has to be simple enough for everyone to use and understand it Make creating a split-test no more than one line of code: if( setup_experiment(...) == "control" ) { // do it the old way } else { // do it the new way }
62. The AAA’s of Metrics Actionable Accessible Auditable
63. Measure the Macro Always look at cohort-based metrics over time Split-test the small, measure the large
65. Why do we build products? Delight customers Attract lots of them Make a lot of money Big vision: change the world
66. Possible Approaches “Maximize chances of success” Build a great product with many features that increase the odds that customers will want it Problem: no feedback until the end, might be too late to adjust “Release early, release often” Get as much feedback as possible, as soon as possible Problem: run around in circles, chasing what customers think they want
67. Minimum Viable Product The minimum set of features needed to learn from earlyvangelists – visionary early adopters Avoid building products that nobody wants Maximize the learning per dollar spent Get the facts before it’s too late Probably much more minimum than you think!
68. Minimum Viable Product Visionary customers can “fill in the gaps” on missing features, if the product solves a real problem Allows us to achieve a big vision in small increments without going in circles Requires a commitment to iteration
70. Techniques Smoke testing with landing pages, AdWords SEM on five dollars a day In-product split testing Paper prototypes Customer discovery/validation Removing features (“cut and paste”)
71. Fears False negative: “customers would have liked the full product, but the MVP sucks, so we abandoned the vision” Visionary complex: “but customers don’t know what they want!” Too busy to learn: “it would be faster to just build it right, all this measuring distracts from delighting customers”
72. Five Whys IDEAS Code Faster Learn Faster BUILD LEARN Continuous Deployment Five Whys Root Cause Analysis CODE DATA Measure Faster MEASURE Rapid Split Tests
The premise of the lean startup is simple: if we can reduce the time between these major iterations, we can increase the odds of success.
Even though some aspects of the product were eventually vindicated as good ones, the underlying architecture suffered from hard-to-change assumptions. After years of engineering effort, changing these assumptions was incredibly hard. Without conscious process design, product development teams turn lines of code written into momentum in a certain direction. Even a great architecture becomes inflexible. This is why agility is such a prized quality in product development.
Do our actions live up to our ideals?
After our crushing failure, the founders of my next company decided to question every single assumption for how a startup should be built. Failure gave us the courage to try some radical things.
After our crushing failure, the founders of my next company decided to question every single assumption for how a startup should be built. Failure gave us the courage to try some radical things.
Based on that experience, and the experience of the other startups I have worked for, I now strongly believe there is a better way to create startups. I’ve called this vision the Lean Startup. It combines three key trends.
Let’s look at those changes schematically.
Run tests locally:-- Sandbox includes as much of production as humanly possible (db, memcached, Solr, Apache).-- Write tests in every language. We use 8 different test frameworks for different environs. Otherwise you get fear and brittle.-- Example kind of problem is that AJAX updater for site header. Seemingly innocuous change would break shopping experience.CIT/BuildBot:-- Simply don’t push with red tests. Even if the site is in trouble. Example Christmas site outage with memcache sampling.-- Give an idea of the scale. 20 machine cluster, runs 10000 tests and 100,000’s of thousand of assertions on every change.Incremental deploy:-- Catch performance bugs and gaps in test coverageSlow query in free tags. This started to drive database load higher on one MySQL instance due to contention and data size. Detected and rolled back before it affected users and before the database was hosed due to high load.Changed transaction commit logic in foundation of the system. This passed all tests but caused registrations to fail in production due to subtle difference between sandbox and production. System detected drop in business metric in 1 minute and reverted the changeAlerting and Predictive MonitoringExample: Second tier ISP to block our outbound emailExample: Rooms list performance time bombExample: Registration quality, second tier payment methods, invite mail success rates by serviceStory: Anything that can go wrong will, so just catch it then fast.
Webcast: May 1Workshop: May 29Fliers up frontDiscussion in web2open
