I've just updated the "quick-start" documentation on the CPAN Testers wiki: Quick HowTo for CPAN Testers 2.0

It's not perfect, but it's a lot better than it used to be. If you are still submitting reports by email, please read and follow the instructions before the end of August. If you have trouble, please ask for help on the cpan-testers-discuss mailing list or on #cpantesters-discuss on irc.perl.org.

• Free QA! -- a non-technical talk about the history and social architecture choices of the CPAN Testers project
• Perl 5, Version 13 -- a lightning talk summarizing notable changes in the Perl 5.13 development series

If you are already a CPAN Tester, a "profile file" has been pre-generated for you and you should submit a request that it be emailed to you. This pre-generated profile will identify your new CT 2.0 reports in a way that will make it easier to link them to the CT 1.0 old reports as those are converted.

If you are new to CPAN Testing, you will need to generate a profile file. Install Metabase::Fact and then run the metabase-profile program from the command line. It will prompt you to provide some information and output a profile file. The first time you use this profile file to submit a report, your profile will be automatically registered on the CT 2.0 server.

While the CT 2.0 server has launched, the clients that submit reports are, for now, the same ones used for CT 1.0 -- either CPAN::Reporter or CPANPLUS (or one of the smoke frameworks written for them like minismokebox or CPAN::Reporter::Smoker). The transport framework originally intended for email gets hijacked to send reports via http to CT 2.0 instead.

To send reports to CT 2.0, you will need to install the Test::Reporter::Transport::Metabase module and then configure your client to use that transport instead of the default one. Instructions are available on the CPAN Testers Wiki. While the docs are a bit sparse as I write this, I expect they will continue to evolve and improve as more people contribute to them.

If you're really stuck, please check the FAQ or ask for help on the CPAN Testers Discuss mailing list. If you are a CPAN Tester, I highly recommend joining the list to keep up on the latest issues and solutions in the migration to CT 2.0. If you do ask for help, please be sure to include the version number of all the relevant testing modules you're using along with your perl -V output.

Another good place to go for quick questions is the #cpantesters-discuss channel on irc.perl.org.

In other news, I'll be speaking about CPAN Testers at OSCON in my Free QA! talk. If you or someone you know will be there, please check it out or recommend it.

Apocalypse then helpfully batched several thousand reports and tried to overwhelm the server with parallel submissions. At about 450 reports per minute we finally started seeing some submission errors and now have some work to do to use the Amazon SimpleDB more efficiently or to queue submission peaks and process them offline.

In other interesting news, the CT2.0 server now supports Test::Smoke::Metabase reports, which Tux implemented during the Vienna QA hackathon and refined in the last couple weeks. He uploaded 150,000 or so historical Perl smoke reports as a test and is exploring using the CT2.0 server instead of the existing email system that supports Perl smoke testing. It's the exact same problem (at somewhat lower volumes) that CT2.0 was designed to address and it's nice to see some synergy in the work to date.

In the end, I redirected to a new Amazon "bucket" and reset the server. So the old results are offline (but should be safe) and I'll migrate them over when I get some free time.

Thank you to the testers who reported issues and were patient while I fixed things.

Reports per Minute

Total Report Count

Reports have been received from most major operating systems and versions of Perl.

So far, I've fixed several issue relating to character encoding, removed a number of dependencies and fixed a few other oddball bugs. Chris Williams developed a new "zero dependency" report proxy, Test::Reporter::Transport::Socket and metabase-relayd to allow CT2.0 testing without any Metabase prerequisites on the target perl.

What to look forward to:

• Barbie has started working on extracting report data from the beta site. This is the critical path before we can plan the full launch of CT2.0
• I plan to start working on the conversion of the 6.9 million reports from CT1.0
• Once I have a chance to deploy an app that gives legacy testers an ID file to associate their new CT2.0 identity with their legacy CT1.0 reports, I'll open up the beta to anyone who wants to participate

That's all for now. Thank you, again, to all the beta testers for their efforts and thank you to Perl NOC administrators for their forebearance and support.

Some of the immediate discoveries and improvements include:

• Chris Williams released a new version of CPANPLUS to provide an additional data field needed for CT2.0.
• I revised the Metabase::Fact distribution to avoid some test bugs in bleadperl and downgrade several prereqs so it can be used on Perls back to 5.006.
• I released a couple new versions of Test::Reporter. 1.55 was a patch from Dan Collins to improve how reports are saved/loaded for testers that save files to disk, move them to another machine and then upload. 1.56 attempts to fix line-ending issues moving report files between Windows and Unix (reported by Serguei Trouchelle)
• I released a new version of Metabase::Client::Simple with Net::SSL as a prerequisite. This may be a temporary fix (and it makes it not so "simple"), but for now it helps beta testers avoid annoying and hard-to-diagnose errors from LWP

I want to particularly thank all the intrepid CT2.0 beta test volunteers: Andreas Koenig, Apocalypse, Dan Collins, MW487, Nigel Horne, Serguei Trouchelle, and Shlomi Fish. While I'm sure they might feel more like crash-test dummies, I greatly appreciate the time they are taking and the excellent feedback they've been giving. Keep up the great work!

After another week or so of shakedown, I hope to open the beta test to anyone interested in participating.

I should note that the original deadline for finishing CT2.0 was March 1 and clearly we're not there yet. I've discussed the situation with Robert and Ask at the Perl NOC, and they've been willing to extend the deadline for cutting off CT1.0 for a while longer. Thank you, Robert and Ask, for your understanding!

Progress in the last couple weeks:

• I did some alpha testing of the CT2.0 Metabase hosted on Amazon. Based on that, I revised yet again (sigh) the user registration/credentials approach to minimize the hassle for old and new registrants.
• Florian Ragwitz wrote a Catalyst app to help distribute new credentials files to legacy CT1.0 users. I haven't deployed it yet (since I now need to regenerate all the credentials), but greatly appreciate his quick turnaround.
• I implemented the Metabase search capabilities that Barbie will need to update the CPAN Testers statistics database. This will be based on the excellent SQL::Abstract approach to WHERE clause construction.
• I wrote several helper modules to simplify configuration of a CPAN Testers metabase in preparation for deployment. The first of these has already been uploaded to CPAN: Net::Amazon::Config
• I finalized the "version 0" API for the Metabase web service and revised the interface between the Catalyst app and the Metabase to reflect the latest changes to the library.

In the last several weeks, members the #catalyst and #moose IRC channels were very, very helpful and patiently answered my many stupid questions. Thank you in particular to confound, perigrin, rafl, rjbs, stevan and t0m.

Coming up:

• Deploy all my new code onto the server for "beta" testing
• Release all the code to CPAN that people will need to configure their clients for beta testing
• Regenerate user profiles and deploy rafl's app to distribute them to legacy users
• Whip the server into production shape (e.g. proper boot scripts to auto-start the CT2.0 apps on restart)
• Get back to work on legacy report migration

It's at the point now where I suspect the "hard thinking" part is pretty much done and it's a lot of grotty but straightforward tasks to go. Hopefully, beta testing won't reveal any major issues and the end of March update will be focused on planning on orderly transition from CT1.0 to CT2.0.

There are some tweaks I need to make before I'm ready to open it up to beta testing, and all the updated module are still in git and not yet CPAN, but there is a light at the end of the tunnel and (so far) I don't think it's a train.

• I successfully created a test Metabase backed by Amazon S3+SimpleDB and have created, retrieved and deleted some user profile facts. This built on some of the earlier work by Leon, but reflected the work I've been doing to revise the guts of the Metabase libraries.
• I wrote a program that converts a given NNTP report into a CPAN::Testers::Report object (which is itself a Metabase Fact subclass.) This wound up being trickier than I expected due to potential for ambiguous mapping of distribution names to actual distribution files. (Thank you to Tokuhirom, Andreas, Takesako and Offer Kaye for reviewing and fixing my exception list.)
• I generated 900+ Metabase user profiles for all the known CPAN Testers, which will be used to linked with their reports during the conversion process. We still need to figure out a way to distribute these to testers so that new records will link up and other logistics, but this was a necessary step to prepare for conversion.
• I've started configuring Amazon EC2 instances -- one for the parallel conversion of reports and one for the CT2.0 server itself. I'm still coming up the learning curve on EC2, but see no obstacles (except time)

My immediate goals are (1) to get a test CT2.0 server up and receiving reports so we can start testing it and (2) prep and run the parallel conversion process. I've already gotten some help on the #catalyst channel for #1 (thank you, hobbs) and #2 is now mostly a matter of programming it up now that all the foundational prep work is done.

Hitting the March 1 deadline is going to be tight. We're still holding at about 2 weeks behind the original estimate (where we've been since mid-January). But I think progress is coming fast and furious and I hope to get a solid beta launch by March 1 and negotiate with the Perl NOC for an orderly transition as we ramp up the new service.