Moving to the Cloud with NYTimes.com

Ben Gerst (The New York Times), Vadim Jelezniakov (The New York Times)
Average rating: ***..
(3.40, 5 ratings)

The community platform at The New York Times is a PHP-based, multi-tiered application that utilizes MySQL, Apache and Memcached. It was originally hosted on our own internal servers. We recently moved the platform to the cloud in order to increase its flexibility and to better handle high-load events (like links from the Yahoo home page).

Moving an existing application to the cloud presented both organizational and technical challenges. Here are some of the points we plan to cover:

Cloud Management

  • Existing cloud management tools did not meet our needs in the areas of existing processes and organizational structure.
  • To address these needs, we built our own tool and are in the process of open-sourcing it.
  • Cloud management tools make it easy to manage infrastructures, even if you’re unfamiliar with the cloud.

Auto-Scaling Issues

  • Work in progress
  • Scaling back when traffic is low and scaling up to handle traffic spikes

Load Balancing and Proxying

  • Using nginx
  • Balancing the front-end tier and APIs

Deployment

  • We needed a very simple, straightforward way to deploy on the cloud.
  • We developed our own tool (called Cloud Source) based on Servermattic scripts.
  • We further improved Servermattic by introducing better support for revisions, via the use of SVN externals.
  • We also added support for notifications and Solaris.

Monitoring

  • Currently, we’re using Nagios and Munin. We use dynamically generated lists of instances to configure Nagios and Munin.
  • This solution provides visibility into our full stack.

Security Considerations

  • We were missing a way of auditing our firewall rules, so we also added this functionality to our cloud management tool.

Q&A / Demo

  • We’ll encourage questions from the attendees and look at how our approach can apply to the needs of other organizations.
  • We’ll demo our cloud management tools if time allows.

Ben Gerst

The New York Times

Ben Gerst is a Manager of Platform Technologies and Product Technologist for The New York Times. At the Times, he leads the User Generated Content Platform, which includes Comments on Articles, Rate and Review of Movies, Travel, Dining and Theater. He also worked as a developer on My Alerts, E-Commerce and the Most Popular applications.

Photo of Vadim Jelezniakov

Vadim Jelezniakov

The New York Times

Vadim Jelezniakov is a Lead Software Engineer and Infrastructure Evangelist at The New York Times. At the Times, he leads the creation of processes and tools that allow NYTimes.com to grow and manage its computing infrastructure in a controlled and sane way. He is also the master mind behind NYTimes.com’s own Cloud Manager and a contributor to several open source projects including CloudMaster and MRToolkit. When his head is not in the clouds, he teaches Aikido and plays Wii with his daughter.

  • Intel
  • Microsoft
  • Google
  • Facebook
  • Rackspace Hosting
  • (mt) Media Temple, Inc.
  • ActiveState
  • CommonPlaces
  • DB Relay
  • FireHost
  • GoDaddy
  • HP
  • HTSQL by Prometheus Research
  • Impetus Technologies Inc.
  • Infobright, Inc
  • JasperSoft
  • Kaltura
  • Marvell
  • Mashery
  • NorthScale, Inc.
  • Open Invention Network
  • OpSource
  • Oracle
  • Parallels
  • PayPal
  • Percona
  • Qualcomm Innovation Center, Inc.
  • Rhomobile
  • Schooner Information Technology
  • Silicon Mechanics
  • SourceGear
  • Symbian
  • VoltDB
  • WSO2
  • Linux Pro Magazine

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Sharon Cordesse at scordesse@oreilly.com

Download the OSCON Sponsor/Exhibitor Prospectus

Media Partner Opportunities

Download the Media & Promotional Partner Brochure (PDF) for information on trade opportunities with O'Reilly conferences or contact mediapartners@ oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

OSCON Newsletter

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the OSCON Newsletter (login required)

OSCON 2.0 Ideas

Have an idea for OSCON to share? oscon-idea@oreilly.com

Contact Us

View a complete list of OSCON contacts