Free QA! What FOSS can Learn from CPAN Testers

If you’ve heard of Perl, you’ve probably heard of CPAN. But did you know that CPAN comes with a free QA team? CPAN Testers is a large-scale, distributed, grass-roots project to continuously collect and syndicate Perl library tests from over 80 different platforms and every version of Perl since 5.6.

Over 150 volunteers test the 20,000 libraries already on CPAN plus 1,000 new uploads each month. Over 6.5 million test reports have been collected to date and over 400,000 more arrive each month. Within days of a library arriving on CPAN, it will be tested dozens of times. And it’s all free—free as in speech and free as in beer.

Where did this project come from? Why did it succeed? How is it benefiting Perl users and developers? And what can other FOSS projects learn (or steal) from CPAN Testers?

This talk uses CPAN Testers as a case study to explore four important practices for large-scale, volunteer QA projects.

• Building for your culture: why a culture of testing matters and how not to alienate people with (unsolicited) QA
• Getting people hooked: how to attract new participants and how to keep them motivated through feedback and competition
• Freeing the data: why public QA benefits the community and how innovative mashups are changing the way developers use QA
• Designing for change: how volunteer projects can plan for growth and how to use cheap cloud services as a force multiplier