A Completely Open Source Django Website

Hosting open source documentation was a mess. The best-of-class solution for the Python world as uploading a tarball of html to packages.python.org or doing similar to upload to github pages. If you wanted to self-host it, that generally meant having a cron job that ran a shell script to pull your source code nightly. We set out to solve this problem using the current best of class tools that Django has to offer.

Read the Docs is the official documentation host for many open source Python projects. It is built around the Sphinx documentation toolkit. In the simplest form, we are a hosting provider for Sphinx documentation. However, we have added a lot of features to make this useful. These include:

• Support for svn, hg, git, and bzr.
• Post-commit hooks to automatically build documentation on commit
• A custom Read the Docs styled Sphinx theme.
• Full-text search across all projects.
• Support for VCS tags and branches. (branches git only for now)
• PDF generation for all documentation.
• Editing of documentation that results in a pull request on github. (Bitbucket doesn’t have a pull request API)

Read the Docs has a lot of the standard parts of any website, and also some other intersting parts that are relatively unique. These include:

• Subdomains
• CNAME Support
• Search using Solr and Haystack
• Delayed task execution with Celery
• Front end caching with Varnish
• Deployment with Chef
• Multi-server architecture
• Monitoring with Nagios and Munin

I will provide a brief overview of each part of these systems, with code snippets where appropriate to show exactly how it’s done.

This talk will consist of three parts. The first part is the origin story of the site, how and why it was created over a weekend by 3 people. Then I’ll talk about the technology involved as the site has grown. It started out as a very simple site, but as features have been added, it has gotten more complex. Finally I will discus some of the interesting outcomes that come from having a completely open source site, including security and community contributions.