Moving to the Cloud with NYTimes.com

The community platform at The New York Times is a PHP-based, multi-tiered application that utilizes MySQL, Apache and Memcached. It was originally hosted on our own internal servers. We recently moved the platform to the cloud in order to increase its flexibility and to better handle high-load events (like links from the Yahoo home page).

Moving an existing application to the cloud presented both organizational and technical challenges. Here are some of the points we plan to cover:

Cloud Management

• Existing cloud management tools did not meet our needs in the areas of existing processes and organizational structure.

• To address these needs, we built our own tool and are in the process of open-sourcing it.

• Cloud management tools make it easy to manage infrastructures, even if you’re unfamiliar with the cloud.

Auto-Scaling Issues

• Work in progress

• Scaling back when traffic is low and scaling up to handle traffic spikes

Load Balancing and Proxying

• Using nginx

• Balancing the front-end tier and APIs

Deployment

• We needed a very simple, straightforward way to deploy on the cloud.

• We developed our own tool (called Cloud Source) based on Servermattic scripts.

• We further improved Servermattic by introducing better support for revisions, via the use of SVN externals.

• We also added support for notifications and Solaris.

Monitoring

• Currently, we’re using Nagios and Munin. We use dynamically generated lists of instances to configure Nagios and Munin.

• This solution provides visibility into our full stack.

Security Considerations

• We were missing a way of auditing our firewall rules, so we also added this functionality to our cloud management tool.

Q&A / Demo

• We’ll encourage questions from the attendees and look at how our approach can apply to the needs of other organizations.

• We’ll demo our cloud management tools if time allows.