1. Live Coverage at
The New York Times
WordPress VIP Developer Workshop
Napa, California - May 5, 2015
2. Scott Taylor
• Core Developer, WordPress
open source project
• Release Lead for 4.4
• Sr. Software Engineer, The
New York Times
• @wonderboymusic on Twitter/
Instagram/Swarm
• I like Music, NYC, and
Mexican food
3. Blogs at the New York Times
• The 00s were the glory days
for Blogs and WordPress
• Separate from the rest of the
codebase (NYT4)
• Global NYTimes CSS and JS,
CSS for all Blogs, custom CSS
per-blog
• A universe that assumed
jQuery AND Prototype were
loaded on the page in global
scope
5. • Inline HTML from 2008 that assumes Prototype will
still be a thing in 2015, stored in post_content
• Widgets and inline code that add their own version
of jQuery/Prototype, because YOLO
• Even better: widgets/modules from other teams that
use a different version of jQuery … at times there
could be 4 jQuerys on the page (4 diff versions)
6. No shared modules
• Code/HTML markup can get out of sync with other
projects regularly (header/footer/nav)
• The CSS and JS files split across multiple SVN
repos - changes to global assets can affect us
without us knowing. Fixing the code requires
scouring through multiple repos.
• The blogs team divided front-end and back-end
responsibilities. Typically, no one had a full view of
features or the architecture as a whole
7. At the NYT, we don’t use …
• WordPress Comments: There is an entire team that deals with
these for the site globally, in a different system called CRNR
• Media: There is another CMS at the Times, Scoop, which
stores the images, videos, slideshows, etc
• WordPress native post-locking: This only landed in
WordPress core in version 3.6, we have yet to reconcile the
differences
• There is layer for Bylines which is separate from Users:
Our users are employees authenticated via LDAP, most post
authors don’t actually enter the content themselves
9. • A Blog would create a post and check “Start Live
Blogging”
• the updates related to the post were stored in
custom tables in the database
• the APIs for interacting with these tables duplicated
tons of WordPress functionality
• Custom Post Types didn’t exist until WordPress 3.0
(June 2010) - the NYT code was never rewritten to
leverage them (would have required porting the
content as well)
11. The Next Iteration:
Dashboards/Dashblogs
• A Live Blog would be its own blog in the network, its own
set of tables
• A special dashboard theme that had hooks to add
custom JS/CSS for each individual blog, without baking
them into the theme
• Making an entirely new site in the network for a 4-hour
event is overkill
• For every 10 or so new blogs that are added, you are
adding 100 new database tables - gross!
12. 2013: The New Frontier
I arrived at the New York Times
13.
14.
15.
16. NYT5
• NYT5 “apps” are Git repos that are transformed via
Grunt into a new directory structure. You can’t run
your repo as a website: it has to be built
• Impossible to create a “theme” this time with
shared JS and CSS - CSS is SASS, JS is Require.
• PHP for shared components has Composer
dependencies and uses namespaces - the
directories are expanded via Grunt in accordance
with PSR-0 Autoloading Standard
17. NYT5 Dealbreakers
• We can’t just point at WordPress on every request
and have our code figure out routing. Apache
points to app.php in NYT5
• Because PHP Namespaces are used, WP has to
load early and outside of them (global scope)
• On the frontend, WP cannot exit prematurely before
hitting the framework, which returns the response
to the server via
SymfonyHttpFoundation
18. NYT5 Advantages
• “shared” modules - we inherit the “shell” of the page,
which includes: navigation, footer, login, etc.
• our nyt5 theme doesn’t need to produce an entire
HTML document, just the “content” portion
• With WP in global scope, all of its code is available even
when we hit the MVC parts of the NYT5 framework.
• WP output is captured via an output buffer on load - it’s
accessible downstream when the app logic is running.
19. • We can inherit the global Varnish setup and ditch
Batcache
• Instead of rendering media HTML, our [nytmedia]
shortcodes can just output “markers”
• The NYT5 logic for articles already knows to replace
these markers with markup that is generated from shared
modules - and the data is fetched using parallel HTTP
from internal asset stores (JSON).
• At the end of the day, we were able to eradicate a lot of
old, terrible code and lean on code from foundation
and shared repos
20. What about the old code that
used inline Prototype JS code?
• I wrote a filter on ‘the_content’ that does a quick string
scan to see if one of the old slideshows exists in post’s
content. If it does, I wrote RegEx that replaces the
slideshow blob on the fly with a new blob that leverages
our Require code.
I am not proud of this, but it works, and it replaces the
need to migrate potentially 100s or 1000s of posts in the
database.
• Whenever I can, I choose to not ignore legacy or archival
content - since it’s all still searchable on the net, why let it
die a death by Prototype?
21. Bad News for Blogs
• Blogs were duplicating Section Fronts, Columns:
Mark Bittman has column in the paper.
The column also exists on the web as an article.
He contributes to the Diner’s Journal blog.
There is a section front for dining.
He also has his own NYTimes blog. Why?
• Blogs and WordPress were combined in everyone’s
mind. So whenever WordPress was mentioned as a
solution for anything, the response was: aren’t blogs
going away? #dark
25. What if…
• Instead of using those custom tables
and all of that weird code, we could
actually create new object types:
events and updates!
• To create a new “Live Blog”: you only
need to create an event, then go to a
Backbone-powered screen to add
updates
• Even if WordPress isn’t desired for the
front end, it could be the backend for
anything that wants a JSON feed for
live event data
• Because we would be using custom
post types, building a Live Event UI
that looks like the NYT5 theme would
be nominal
26. • Built an admin interface with Backbone to quickly
produce content - which in turn could be read from
JSON feeds.
• When saving, the updates post into a service we
have called Invisible City
• Our first real foray into using the JSON API (WP-
API)
• Our plan was just to be admin to produce data via
self-service URLs
What we did
28. Complete Rewrite of 2008 code
• nytimes.com/live/{event} and nytimes.com/live/{event}/
{update}
• Brand new admin interface: Backbone app that uses the
WP-API. Constantly updated filterable stream - Backbone
collections that re-fetch on Heartbeat tick
• Custom JSON endpoints that handle processes that need
to happen on save
• Front end served by WordPress for SEO, but data is
received by web socket from Invisible City and rendered
via React
35. HTTP is time-consuming
• Admin needs to be fast
• The front end is typically cached, but page generation
shouldn’t be bogged down by HTTP requests
• It is easy to lose track of how many things are
happening on the ‘save_post’ hook
• Anything which is time-consuming should be
offloaded to a separate “process” or request who
response you don’t need to handle
37. Custom JSON Endpoints for POST
• Use fire-and-forget technique on save, instead of
waiting for responses inline. You can still log/
handle/re-try responses in the separate request.
• Most things that happen on ‘save_post’ only
need to know $post_id for context, the endpoint
handler can call get_post() from there
41. Custom JSON Endpoints for GET
• We do not hit these endpoints on the front-end
• We have a storage mount that is fronted via Varnish
and Akamai
• JSON feeds can show up on the homepage of the
NYT to dynamically render “promos” - these have
to massively scale