Video and slides synchronized, mp3 and slide download available at http://bit.ly/12BvrmR. Marc Frons discusses the New York Times’ digital subscription model. Rajiv Pant shares their experiences transitioning to continuous delivery, and using NodeJS, Scala, cloud and big data. Filmed at qconnewyork.com. Marc Frons was named Senior Vice President and Chief Information Officer of The New York Times Company in March 2012. Rajiv Pant is Chief Technology Officer & VP at The New York Times.

1. Marc Frons | @marcfrons | CIO, New York Times
Rajiv Pant | @rajivpant | CTO, New York Times
Can Technology Innovation Save
The New York Times?

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/innovation-new-york-times

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. Core Purpose
Enhance society by creating, collecting
and distributing high quality news,
information and entertainment

5. In the Internet era, technology not only
enables great journalism; it is integral to
the creation of great journalism

6. The Internet has forever redefined
how journalists tell stories, how they
discover news, how they interact with
their audience – and how their
audience interacts with them

7. Journalism & technology
have become inexorably
intertwined

8. And yet…
Until very recently, most
media companies viewed
technology as a cost center,
and as a service organization

9. But at the most successful Internet
companies, the engineers are in
charge

11. How do you create a
successful technology
culture in a non-technology
company?

12. It’s a lot easier to create a
culture than it is to change one

13. The State of NYT Digital 7 Years Ago
 Less than 20
engineers
 Separated from
the company &
print newspaper
 Worked in a
separate physical
location
 Little interaction
with newsroom

14. Separate from
legacy business?
How do you create a
successful technology
culture in a non-
technology company?

15. Separate from
legacy business?
Integrate into
the business?
or…
How do you create a
successful technology
culture in a non-
technology company?

16. We chose to integrate into the business
Initially, merged print and digital ad sales forces
We knew the day would come when we would need to
unify our tech teams as well…
Digital Technology remained independent from the
rest of the company - philosophies of the Digital
team and their print counterparts were too
divergent
Eventually merged print and digital marketing and
circulation

17. Growing NYT Digital

18. …we built our own CMS

19. …we built our own CMS

20. …we Built the Interactive News Team

22. Aron Pilhofer

23. Jeremy Ashkenas

25. …we created a new culture

26. Hack Days
Monthly, on Fridays
Internal AND
external participants
Pick whatever you
want to work on
Hack together with
journalists,
engineers
Present to the
group

27. 100% Days

29. Times Open
Times-hosted
monthly
technology
forum

33. Developer NYTimes.com

34. code.nytimes.com

35. NYT Open Blog

36. Open Source Science Fair

37. Beta620

41. …we built our own
User Generated Content systems

42. Stuffy

43. Community

44. Recommendation Engine

45. Evaluating Rec. Algorithms
Precision Recall F1 Factor of Impr.
(Logged In)
Factor of Impr.
(Anonymous)
Rec Eng 2.0 0.07% 0.32% 0.11% 1.00x 1.00x
UBCF 0.14% 0.65% 0.24% 2.05x 2.00x
Harry
Potter
(Naïve)
0.26% 1.19% 0.43% 3.76x 14.00x
Dataset = Event tracker, Apr 1, 2013 – Apr 10, 2013
Precision = % of recommendations that were hits
Recall = % of all possible hits that were recommended
F1 = 2*P*R/(P+R)
UBCF = User based collaborative filtering
Harry Potter = Give 10 most popular to everyone

46. …we built our own ad systems

48. Custom Ad Units

49. Integrated E-Commerce
Links to live e-commerce and purchases

50. …we built advanced data
visualization capabilities

51. Mike
Bostock

53. Nate Silver

55. Debunking the Story

56. …and when the time came, built
Digital subscriptions commerce and
marketing platform

58. Where this all puts us today

60. Android

61. Windows 8

62. HTML5 Web App

63. iOS

64. Live Video

65. Live
Updates

66. Google Glass
Breaking News Alerts
delivered in real time with
audio alerts
Top News article headlines
delivered in hourly batches
Top News article summaries
that can be read aloud using
Text-to-Speech

67. 2003
Content Management / Delivery
2013 2015
Vendor developed CMS
Capabilities
• Limited web-based
content management
• Site templating
Tech
• Some Java/JSP
Challenges
• Siloed print & digital
content management
• FTP-based image
management
In-house content management
systems
Achievements Unlocked
• Multimedia content support
• REST/JSON content API
• External Syndication platform
• Editing workflows
• Some print integration
• Faceted search
New Tech
• Java
• MySQL
• jQuery
• ElasticSearch
Achievements Unlocked
• Unified digital & print
editing
• APIs for everything

68. 2003
Advertising
2013 2015
In-house advertising engine
Capabilities
• Dynamic ad content
• Ad campaign management
Tech
• C++
• Oracle
• “Context”
Challenges
• Memory & hardware
constraints
• ADX not really well funded
In-house advertising engine
Achievements Unlocked
• Sales cycle planning, media
spend planning
• Refactored codebase
(Context to Java)
• JSON/REST Ad APIs to
front end
New Tech
• Java
• C++/Autotools
Achievements Unlocked
• Real time ad intelligence &
placement
• Refactored sales orders
system (about.com
codebase)

69. 2003
Web Front End
2013 2015
In-house WWW;
Vendor developed HTML
mobile web
Capabilities
• Limited web
development
Tech
• “Context”
• XSLT/JSP
• Some PHP
Challenges
• Pre-transformation
In-house WWW;
In-house mobile web
Achievements Unlocked
• Mature front end delivery
capability
New Tech
• “Modern client” (HTML5)
• Fledgling node.js
• From prototype to jquery
Achievements Unlocked
• Radically more touch-
friendly site
• Flexibility to respond
to a highly volatile
environment
New Tech
• HTML5
• Keep on your toes…

70. 2003
Mobile Apps
2013 2015
No mobile apps in 2003
First iPhone app in 2008,
vendor developed
Capabilities
• 1st generation iPhone
app
• Offline sync
Tech
• Native iOS
Challenges
• Crashing, crashing,
crashing
• Constrained content
In-house developed mobile
apps on iOS, Android
Vendor developed apps for
Windows, BlackBerry
Achievements Unlocked
• Good release cadence,
stable products!
• iPad app
• Unified, highly product
oriented team
New Tech
• Still a lot of native
• HTML5, sparingly
Achievements Unlocked
• “Sensor Driven Web”
• Wearable computing
• Mobile-specific
journalism

71. 2003
Tools / Infrastructure / Operations
2013 2015
Own
Capabilities
• Local Data Center
Tech
• CVS
• Solaris
• Apache
• Akamai CDN
Challenges
• `scp *.tar.gz` deploys
• Manual backups
• Shared passwords
Built out mobile team
Achievements Unlocked
• SVN, sometimes Git
• Sane branch & release
• EC2 data center
• Test & build automation
• Dev sandboxes
• Still Akamai CDN
Vendors for
• Cloud infrastructure (AMZ)
New Tech
• Nginx
• Hudson
Achievements Unlocked
• Continuous
deployment
• From CDN to in-
house built
messaging
architecture
• WebSockets

72. 2003
Business Intelligence
2013 2015
Buy, with in-house analytics
Capabilities
• Web log reporting (buy)
• Traditional data warehouse
Tech
• Omniture
• SAS
• Oracle
• Informatica (ETL)
Challenges
• Vendor controls log data
• Incomplete data
• Data quality
Added In-house business
intelligence
New Capabilities
• Fine grained reporting
• Unified event tracking
(petabytes) for all
platforms
• Content tagging
• Real-time analytics
New Tech
• R
• Pentaho
• Hadoop/S3/Cassandra
• D3.js
New Capabilities
• Full integration of
business intelligence
into product
development cycle
• More real time
analytics driving
content
• Improved resolution
through improved
performance

73. Just as the Web disrupted Print, mobile is disrupting the
traditional Web.
Our best defense is to innovate.
Planned major redesign of existing mobile apps,
launching new mobile paid products next year.
All of them will give our readers and advertisers
the best possible user experience on all major
platforms and on all devices.