Punctuated Equilibrium, Celestial Navigation,and API Strategy

Punctuated Equilibrium,Celestial Navigation,and APIs Competing through dynamic adaptation #W2EAPI Sam Ramji @sramji Dan Jacobson @daniel_jacobson Michael Hart @michaelhart

Darwin formulated his theory of evolution about 150 years ago

Based on observations he made in the Galapagos Islands 15 years earlier

A wild diversity of creatures existed in a new environment

Starting from an ancestor which looked like this

Finches evolved that looked like this

For many years the belief was that this change happened slowly and gradually.

In 1972, Stephen Jay Gould and Niles Eldredge proposed a new idea

that evolution is not slow and gradual

Applying this view to the observations of finches

G. Scandens C. Pallidus G. Magnirostris G. Conirostris C. Heliobates G. Fortis G. Fulginosa C. Psittacula C. Pauper C. Olivacea C. Parvulus Certhidea Camarhynchus Geospiza

“Thus, from the war of nature, from famine and death, the most exalted object of which we are capable of conceiving, namely the production of higher animals directly follows.” Charles Darwin On Origin of Species

So while it may look slow and gradual in hindsight

Evolution is experienced in punctuated bursts.

If you’re living in a punctuated burst of evolution it feels like a revolution

Navigation Navigation While it’s nice to think that evolutionary change happens in a planned way, this is not the case – it only looks smooth in hindsight. Not even the voyage of Darwin’s HMS Beagle was smoothly planned. Looking back he seems to have been predestined to arrive at the Galapagos islands, but this is not the case. Planning is hard enough and at best “no plan survives contact with the enemy” (Patton) – so navigation becomes a crucial skill. Knowing where you mean to go is one thing. Knowing where you are and how to move forward is quite another. Realizing that you may in fact be making progress towards a different and better goal, rather than failing to meet your original goal, can be the key to success. Columbus may have set out to find India but as a result Spain fared extremely well through the discovery of the Americas. Dead reckoning – the art of knowing where you are based observing your speed and direction over time. Very difficult and subject to cascading error. “coasting” like the ancient Greeks Word of mouth and story-based navigation “books” - portolani Blue-water sailing – takes instruments, maps, goals. Necessary when there are no landmarks in sight. Shaping of the hull, multiple masts, the rudder – enabling the ship to sail closer to the wind and cross the oceans. (AKA Technology). But this was not enough – a way to manage this was needed in order to achieve the goal: compass and sea-chart navigation. Pole Star – Stella Maris Astrolabe Columbus – set out to make landfall at 24 degrees north and succeeded, although in America and not China Started with a map like p18 and p20, but we’ll give you one like p35 or p52 Sea charts transformed the world view APIs Are a bit like sailing off to find new lands and opportunity. But while the first to travel may have had to suffer through early tools, poor maps, and hostile natives, you can learn from their experience, borrow their maps and the tools they had to invent to succeed. Military advantage came from percentage-point improvements in navigational accuracy – and time became a military technology. The more precisely you could measure time itself – and keep the measuring devices in sync across your fleet – the more effectively you could compete and win.

only looks smooth in hindsight

Living through it is usually chaotic

Karen James The Beagle Project Blog

To navigate, you need a map and instruments

Maps exist for transferring knowledge

and they too have evolved over time.

They started as oral traditionand were written down in a form called a periplus

Periplus of Hanno Courtesy of Heidelberg University

Periplus of Hanno Courtesy of Cornell University

Far less efficient knowledge transfer than a modern map of the same journey

Map of Hanno’s Journey Courtesy of Bourrichon/Wikipedia

Exploration was dramatically held back for want of a map

In the two thousand years between Hanno’s journey on a Phoenician trireme

And the Mediterranean caravel of the 15th century

Maps had only evolved to be graphical descriptions of coastlines

That was a map published a few years before Columbus crossed the Atlantic to find India

After his crossing, his expedition shared their knowledge in a new map

Still far from perfectbut much improved.

The biggest challenge in this kind of exploration was determining their location on the Earth

Instruments for measuring latitude had beenused and improved for centuries

Longitude was the hard problem.

You needed to know not just the angle of the sun and stars

you also needed to know the precise time.

Regardless of your sailing technology without the proper measurement you were lost

We are not promising a perfect map of the new world

But it should be more like this

and we will show you what we know how to measure.

There are more niches today than we’ve seen before, so we need to borrow from nature

If we start with an API we can explore all the niches around our business

The leaders of today’s Internet

clearly understand this mechanism.

They understand that the distribution model for value has changed in the Internet era.

Packaged Goods Consumer Retail Store Producer Internet Services App Developer Consumer Provider

Developers took their APIs and explored the niches for them

The providers and the developers both benefited from this adaptation

Suddenly this seems obvious to everyone.

And developers are racing to pack the niches.

This is a sudden, material shift in competition.

It only looks gradual if you’re losing.

We’ve seen punctuated equilibrium in business models over the last hundred years

App Developer Device App Big Box Retailer Mobile App National Chain Web Retail Department Store Web Catalog Specialty Store Indirect Sales Direct Sales

What’s the environmental stress driving the current rapid change?

Hardt’s Theorem: The Internet Power Law The first wave of the Internet demonstrated the economic impact of web-based business models. 99:1 95:5 The API-driven Internet is demonstrating the next concentration of power and is reflecting a “99:1” distribution (examples: Twitter, Facebook) due to the high switching costs and effective lock-in through software. The HTML-driven Internet showed new business models that focused revenue in the 3rd standard deviation (examples: Amazon, EBay). Reality reflected a “95:5” rule where 5% of companies dominated the transactions and profits. 80:20 Previous eras of business showed a normal distribution for revenue, with most firms getting most of the revenue. In the later half of the 20th century, business model innovations focused revenue in the 2nd standard deviation above the mean. The “80:20” rule became conventional wisdom. The next wave of the Internet is demonstrating the economic impact of API-based business models.

But you need to tackle it in a way that fits your business

1st Party Open Open Open Platform Partner

These are complementary and distinct. Open is different for each one.

1st Party Apps Partner Apps Platform Ecosystem

1st party 1st party is about offering direct access to your core business via apps that you make or contract out

1st party Here open means all the business is accessible to internal developers and contract specialists

partner Partner is about enabling directed development of apps that extend your business model towards your business partners

partner Here open means existing partners have access to your business via APIs and can innovate asynchronously

platform Platform is about enabling unknown developers to build brand new apps and businesses that will surprise and inform you

platform Here open means enabling business models and allowing developers to support each other at massive scale

open Open lets you navigate across the possible business models when your first model doesn’t work as planned

To get your API strategy properly grounded

“ John Musser Programmable Web

Establish Target Segments Engage Developer Channel Set Industry Goal

target segments An API should extend your core business into a new part of the market

target segments Your core business already has key performance indicators

target segments So apply your KPIs to the new market segment you’re targeting with your APIs

target segments What is the market impact you need to create in order to succeed as a business?

target segments What does the targetsegment need that it is not getting from you today?

target segments The answer will be the foundation of your API strategy.

developer channel In most cases the channel for your API will be developers, but what do they need?

developer channel A profit motive.

developer channel Here are the leading profit models for developers today

developer channel In-app purchases

developer channel Affiliate royalty

developer channel Your advertising spend

developer channel Market awareness of their offering

industry goal If you don’t know where you’re going, you definitely won’t get there

industry goal Partnerships and platform businesses are very different things.

industry goal Partnerships are formed to serve a known set of entities

industry goal A partner API should be traceable to each partner’s relationship

industry goal And support end-to-end business processes

industry goal A platform exists to create massive and unpredictable opportunities

industry goal All your technology, support, and community decisions will be about surviving the scale of adoption

That’s the strategy dimension.

The execution dimension is what you already know.

Planning. Management. Organization.

Putting all this in context gives us a map for our API strategy

The instruments will be your KPIs and your core API metrics: performance and adoption

Connected Devices Smartphone Web App N-tier Personal Computer DCOM CORBA Website Minicomputer Client/Server Mainframe Integrated Distributed Computing

“ The Web is Dead. Long Live the Internet. Chris Anderson Wired Magazine

The Web is Dead. Long Live APIs!

Netflix traffic distribution is nearly the same

The majority of traffic comes from API-drivenconnected devices.

Like Columbus, Netflix started with a map of the coastline

Build an open API as a platform and let a thousand flowers bloom

But they had left the coastline far behind

And the instruments indicated that there were fewer flowers than expected

Netflix API Requests by Segment

But partners started building apps for connected devices and the business took off

Wii iPad PS3 iPhone Apple TV LG TVs Samsung TVs Roku XBox

Architecture should reflect the business model

So Netflix has drawn the following map

Wii Roku PS3 iPhone iPad Google TV Apple TV XBox LG TVs Samsung TVs

Instruments show that API traffic has grown tremendously in a short time

20,000,000,000 API requests per month. Is that a cause for celebration? Or for concern?

When you’re navigating uncharted waters, speed is not your friend. Perhaps it’s time to slow down and avoid risking unknown reefs.

Navigating this growth challenge means that the next API revision will focus on reducing overall traffic.

Part of this redesign is reviewing conventions

Punctuation: REST ProgrammableWeb

REST, as long as it makes sense for the business Don’t be afraid to break convention

Tiered architecture helps you navigate different problems with agility

Controller Model View iPhone Wrapper Recommendations R12n Service Xbox Wrapper UNIFIED LIST/TITLE API Wii Wrapper User Info User Service PS3 Wrapper Similar Movies Similar Movie Service SHARED API INTERFACES Roku Wrapper Movie Metadata DataNormalization & Resiliency iPad Wrapper USER API Viewing History Apple TV Wrapper Ratings PC / Mac Wrapper TiVo Wrapper API Repository Layer Wrapper Layer Web Service Layer Shared Layer Source Data Layer App Layer API Layer

Stable Flexible Agile iPhone Wrapper Recommendations R12n Service Xbox Wrapper UNIFIED LIST/TITLE API Wii Wrapper User Info User Service PS3 Wrapper Similar Movies Similar Movie Service SHARED API INTERFACES Roku Wrapper Movie Metadata DataNormalization & Resiliency iPad Wrapper USER API Viewing History Apple TV Wrapper Ratings PC / Mac Wrapper TiVo Wrapper API Repository Layer Wrapper Layer Web Service Layer Shared Layer Source Data Layer App Layer API Layer

Server architecture should support both crests and troughs of the waves of demand

Instance Architecture Based on Specialization List Creation Dependency Service ELASTIC INSTANCE LAYER API METADATA CACHING LAYER Meta Data Dependency Service METADATA SERVICE

Instance Architecture Based on Specialization Handles Request/Response List Creation Dependency Service ELASTIC INSTANCE LAYER Caches Dependency Data API METADATA CACHING LAYER Meta Data Dependency Service Populates and Manages Cache METADATA SERVICE

Map out your usage patterns and cache your data accordingly

Vertical Document-Based Caching

Vertical Caching Netflix Full Movie

Horizontal Caching: Netflix Basic Data

Combining horizontal and vertical caching may be the best approach when building for multiple geographies

Design for The Audience You Want If your current audience is this size Then design your system for an audience that is this size

You don’t have to implement for the dream audience

But you should design systems to scale to the dream audience

Versioning APIs is the default approach In many cases, it is appropriate

Versioning means supporting multiple applications All of which do the same thing, basically

Version-less API? 1.0 1.5 2.0 3.0? 4.0? 5.0? api.NEXT Today

Extend your API by extending data types Addition is not version-worthy Better to be incomplete than inaccurate Withhold implementation if you are unsure

Don’t forget to support your API With technology, people and management

With APIs emerging, we need better tools help us navigate

Service Level Agreements (SLA) Set reasonable SLAs Will vary based on API Communicate them to customers Set up monitors Monitor average latencies at various percentiles Monitor error rates and types Trigger alerts when SLAs get broken Establish escalation and response policies Who responds? What is the escalation plan?

Use metrics to guide your decisions They provide better navigation than anything else

Metrics / Business Intelligence Request-Based Metrics (default) What endpoints were called? What parameters were passed in? Response-Based Metrics What was delivered? If applicable, what was presented? Was the response valid? Request Trace What underlying systems were called? How did they respond? System Health Are the systems meeting uptime SLAs? Are they performing with reasonable latencies?

Metrics (continued) They can affect the future design of the API They can inform strategic business decisions They can reveal rogue usage They can reveal errors in the system

DataFrom Silos to Social Silos Better than a random collection of flat files! At least it’s all in one place! Shared (in the enterprise) History: Data warehousing and insights (extra-enterprise) Now: OData shared with communities according to business model Pubsub - Events & Async – better for the provider, better for the app Relevance/Collaborative Filtering/Community Modeling From aggregating data or mixing data to produce a contextual recommendation or view Domain-specific database monopolies Social Bootstrapping social context within your own data via the Facebook Graph Surround your customer by understanding their context and delight them Understand Domain-specific database monopolies – location, messaging, more Look for experts in other domains that you need Is the verb “enrich” instead of deliver? From content to experiences

Domain-specific Data APIs Private Cloud DBs Caching DBs Data API Data Warehousing Mainframe RDBMS Flat file Shared Silos Data

Sharing your data, from silos to social

evolution in data sharing App Org Cross-org Cross-business Mainframe Databases Middleware APIs

challenges in data sharing loss of control (once it’s out there) only recourse is legal enforcement is expensive (time and reputation)

case study: opening up movie metadata

optimize data delivery enable larger downloads for fewer queries more liberal retention policies fewer refreshes, less breakage push incremental updates limited access to richer queries

Highest rated titles from the 80's and their awards: http://odata.netflix.com/Catalog/Titles?$filter=ReleaseYear le 1989 and ReleaseYear ge 1980 and AverageRating gt 4&$expand=Awards

[{ "id": null, "name": null, "type": "/dining/restaurant", "/business/business_location/address": [{ "street_address": [], "citytown": { "id": "/en/toronto" } }], "cuisine": [{ "/dining/cuisine/region_of_origin": [{ "!/film/film/featured_film_locations": [{ "id": "/en/the_italian_job" }] }] Find Toronto restaurants with cuisine from a filming location of "The Italian Job"

What data APIs should your business be using?

Open your database via an API and let others access + download RPC-like access to a database is not a great idea SQL-like access makes more sense – which indicates OData (but beware being tied to a Microsoft infrastructure)

Punctuated Equilibrium, Celestial Navigation,and API Strategy

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (7)

Similaire à Punctuated Equilibrium, Celestial Navigation,and API Strategy

Similaire à Punctuated Equilibrium, Celestial Navigation,and API Strategy (20)

Plus de Apigee | Google Cloud

Plus de Apigee | Google Cloud (20)

Dernier

Dernier (20)