This document summarizes an agenda for a Pinterest Engineering meeting. It includes discussions on mobile growth and monetization, deploying and shipping code. Specific topics that will be covered include scaling user education on mobile, growth strategies like user education, monetization through data, and how Pinterest deploys and ships code. Speakers will discuss mobile features, how user growth is driven through education, monetizing user data, and ensuring smooth code deployment.
28. What are nags?
•Important information which we
occasionally display at the top of
a user’s feed
•Can be a Call to Action “Confirm
your email” or an announcement
“You can now add a map to any
board!”
30. Enter: The Experience Framework
•Each time we reload the home
feed, ask the experience
framework: What should I show
in this nag?
31. Single Nag Manager that relies on the experience
framework to give it the contents to render
Nag Manager
•Experiment with nag messaging, call to actions, images
•Add any new nag dynamically, controlled from the backend.
•Cool-down management - should not see more than one nag in a set
period of time
32. What does that nag data look like?
Handling actions
"bg_img_url_2x" = "http://mobile-
assets.pinterest.com/iphone/nags/invite-
mail@2x.png"
"title_text" = "Pinspire your friends!"
"detailed_text" = "Know someone who'd like
Pinterest? Invite them along."
"button1_text" = "No, Thanks”!
"button1_uri" = ""!
"button2_text" = "Invite Friends"!
"button2_uri" = "pinterest://invite_friends"
33. Handling Actions
•All initialization and presentation of view controllers is handled through a
central “Navigation Manager.”
• Centralizes code to create and present view controllers
• Consistency to other platforms for deep links
•Allows dynamic insertion of nags from the backend without having to write
new client code and submit a new release
40. Experiment between classic user ed and NUX
•When doing experiments where we need to call the network to get the
user’s treatment group, need to make sure we’re not adding perceived
latency
• Structure view controllers in a way that you can asynchronously load in the modules
dependent on the treatment group
• If need to transition to different view controllers, set a time out in which we transition
to the control treatment
• “Be Fast or Fail Fast”
41. 1 step vs. 2 steps vs. 3 steps
Experiment with different versions of NUX
42. Experiment with different versions of NUX
•Backend controls all strings, allowing us to dynamically experiment with
different text (Messaging, titles, calls-to-action)
43. steps = (
{
"continue_button_text" = Continue;
"detailed_text" = "Tap a network to find people who share your interests.";
"follow_button_text" = "Follow selected people";
step = 1;
"title_text" = "First things first";
"total_steps" = 2;
},
{
"completion_message" = "Finding Pins for you...";
"continue_button_text" = "Tap at least {0} more to continue";
"detailed_text" = "Tap whatever you're interested in these days.";
"finish_text" = Finish;
"num_interests" = 5;
"skip_text" = "Pinterest is much more interesting when you tell us what you like.";
step = 2;
"title_text" = "Pick 5 interests";
"total_steps" = 2;
}
);
After signup, request all data for NUX
Enter: Experience Framework
44. Supports dynamic number and order of steps
NUXViewController : UINavigationController
•Maps an array of display data to an array of view controllers
•Protocol method advanceToNextStep called by each child view controller
• Checks the array it keeps for the next view controller to push
JSON dict for Intro
JSON dict for Friend Selector
JSON dict for Interests Selector
NUXIntroViewController
NUXConnectViewController
NUXInterestsViewController
45. Wins from Experience Framework
•Single place in the backend that manages all experiences for all platforms
•Dynamically trigger display of content
•Conflict resolution for educations that touch the same views
•Experiment with flows, messaging, and images
48. Pinterest is a data driven company
•Data matters
• 100+ experiments active at a given point in time
• 1500+ tracked metrics
• 200+ log types
49. We produce a lot of data
•We produce a lot of data
• PBs of data in S3, growing by Tens of TB a day
• Hundreds of production hadoop jobs, processing about half a PB of data each day
52. Kafka 101
•Distributed pub-sub service
•Designed for high throughput
Producer Producer Producer
Consumer Consumer Consumer
Kafka
cluster
53. Anatomy of a topic
•Topic is a category to which messages are published
•Partition is a ‘shard’ of a topic controlling the level of consumption parallelism
•Messages are assigned unique identifiers called offsets
1 2 3 4 5 6 7 8 9 10 11 12
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9 10 11 12
Writes
Partition 0
Partition 1
Partition 2
54. Save the day
•Kafka is optimized for local writes
•Local disk capacity is good for a few
days worth of data
•Data needs to be saved (at least) daily
to long term storage - Amazon S3
55. How soon is “eventually”
•Amazon S3 is a cloud file system
•Eventual consistency model
• No guarantees on when uploaded data will become visible to the readers
• No monotonicity - data available in the past may magically disappear
56. Secor design guidelines
•Objectives:
• Persist Kafka logs to S3
• Cause no data loss
• Work properly with eventual consistency model
•Properties:
• Horizontal scalability
• Fault tolerance
• Customizability
57. No-S3-reads principle
•Secor never reads data from S3
• Lightweight metadata is stored in strongly
consistent state repository
•Strategic choice of file names
• s3n://logs/<topic>/
<generation>_<partition>_<start_offset>
• <generation> represents software compatibility
version
• Inconsistencies introduced by consumer failures
get fixed automatically by file overwrites
58. Date clustering
•Data processing tools rely on date-
partitioned directory structure
• s3n://logs/event/dt=2014-04-04/
•Timestamps extracted from messages on
the fly
•Support for pluggable parsers for thrift,
json, etc.
63. Developing - Ideal state
•able to iterate quickly
•easy errors caught automatically
•easy-to-understand and powerful abstractions
64. Fast iteration
Developing
•Build tasks and dependencies modeled as a graph
•“cumberbatch” watches for changes of file contents
•“orchestrator” knows tasks and dependencies
•build the minimum tasks to heal damage in the graph
•maximize parallelization of tasks
•built on Grunt - access to large library of build tasks
75. Abstractions
Developing
•component framework
• styles are scoped to the component
• DOM access is scoped to the component’s DOM
• “events up, methods down”
• scaffolding script
•live component catalog - discovery of existing components
•autoprefixer, spriting - remove boilerplate
83. Integration problems are caught
Reviewing
•PRs trigger a build and tests - 3 minutes
•latest.pinterest.com is continuously deployed from “head”
•Selenium integration tests run against every deploy
87. Deploying - Ideal state
•code deploys are invisible to users
•frequent and non-disruptive to developers
•immediate rollback when there’s a problem
88. User experience
Deploying
•stickiness to a version
•flip nearly instantaneously between builds
• reduce version thrashing
• worry less about (style mismatches, JS errors due to data format mismatch with
server)
•asset versioning
99. Results
•5 engineers on web team
•all teams at Pinterest developing their own web features on our platform
•components re-used across teams
•2 scheduled deploys a day
•anomalies in key metrics surfaced immediately
•100s of simultaneous experiments
