I have redone my Perl 5 release cycle graph again with a few changes. First, for the v5.4 through v5.8 series, I have broken the line to the final release, which I consider to be "outliers". I think the Perl community was lucky to get those releases — was lucky that someone stepped up and made them — and that they don't reflect a "normal development" or support cycle.

Second, I have projected an estimated lifecycle under the official support policy for v5.12, v5.14 and the not-yet-released v5.16. This represents an expectation for the normal support lifetime of these releases and I think shows a better contrast of expectations resulting from the support policy introduced with v5.14 compared to historical releases.

click for larger view

My observations (ignoring outliers):

• Prior to v5.14, there was a (sometimes lengthy) gap between the end of one stable series and the start of the next.
• The actively maintained periods of v5.4, v5.5 and v5.6 were shorter than the proposed support windows under the new policy
• v5.8 had two different support paradigms. Between v5.8.0 and v5.8.1 was a long gap similar to the v5.6 series. v5.8.1 to v5.8.8 had a more regularly-spaced series of support releases.
• v5.10 had the longest gap between initial release (v5.10.0) and the subsequent support release (v5.10.1)
• v5.12 has had the most consistent pattern of support releases after the initial release, and is the only stable Perl 5 to have a (regular, not outlier) support release after the release of the next stable version
• v5.14 was the first stable Perl 5 released under the new annual-release cycle

The new support policy most resembles a return to the best support period seen historically (v5.8.1 to v5.8.8), but without the subsequent gap to the next stable release.

Why do I think this new policy is a positive step forward? Here are some reasons:

• The support policy is actually written down. What expectations did anyone have prior? I don't know. But if I'm using Perl 5, I'd rather know what to expect that have to guess and hope for the best.
• The new policy offers a support window longer in practice than any Perl 5 except v5.8 and more regular than any period except between v5.8.1 and v5.8.8
• The new stable Perl 5 is available for migration testing mid-way through the support window of the prior stable release. If there are issues with migration, users can be confident of support for their existing version for an additional year (and emergency security support for a year after that).
• The annual release cycle means the change between one stable release and the next will be smaller, lowering migration risk

The new policy does cut off the "long tail" of expectations for an outlier release. I can understand that for some companies or OS packagers, a two-year support window (three for security) might feel too short, even if that is longer than was typically seen historically.

Here is where the annual stable release cycle and monthly development release cycle offer a huge side benefit: there is now a well-documented, frequently-used, regularly-updated release manager's guide for Perl 5. Now, the release process is so easy that a moderately-skilled Perl software engineer without much prior exposure to the Perl source repository can make a Perl 5 release tarball in about a day.

This means that even if the core Perl 5 development team isn't supporting, say, v5.12 anymore, a motivated company or community group could do the work necessary to prepare their own "outlier" release and either petition the Perl 5 core team to release it or could release their own stable "micro-fork" for others with long-term support needs. (There might even be a profitable business opportunity selling support for Perl versions past the official support window.)

Previously, Perl 5 development used to be bursty, with long delays between stable releases and with unclear expectations for support. Now, Perl 5 development happens like clockwork, and has a clear, written support policy.

[Note: this post represents my individual opinion and was not reviewed by the Perl 5 Porters core development team; it may or may not represent the views of other core developers; it is certainly not an "official" statement of the Perl 5 Porters in any way]

1. done_testing instead of no_plan

If you don't know how many tests you are going to run (or don't want to keep count yourself), you used to have to specify 'no_plan' at the start of your tests. That can lead to surprises if your tests exit prematurely. Instead, put the done_testing function at the end of your tests. This ensures that all tests actually run.

use strict; use warnings;
use Test::More 0.88;

ok(1, "first test");
ok(1, "second test");

done_testing;

2. new_ok for object creation

You used to have to create an object and then call isa_ok on it. Now those two can be combined with new_ok. It will also let you pass arguments in an arrayref to be used in the call to new.

use strict; use warnings;
use Test::More 0.88;

require Foo;
my $obj = new_ok("Foo");
# ... use $obj in testing ...

done_testing();

Changed "require_ok" to "require" per Ovid's comment, below.

3. Add diagnostics only in verbose testing

The old diag function always prints to stderr. Particularly for debugging notes, that can clutter up the output when run under a harness. You can now use the note() function to add diagnostics that are only seen in verbose output.

use strict; use warnings;
use Test::More 0.88;

note("Testing on perl $]");
ok(1, "first test");

done_testing();

4. Explain data structures in diagnostics

I often find myself wanting to dump a data structure in diagnostics, and wind up loading Data::Dumper to do that. Now Test::More can do that for you with the explain() function. The output is a string that you can pass to diag or note.

use strict; use warnings;
use Test::More 0.88;

my $want = { pi => 3.14, e => 2.72, i => -1 };
my $have = get_data();

is_deeply($have, $want) or diag explain $have;

done_testing();

5. Encapsulate related tests in a subtest (0.96)

use strict; use warnings;
use Test::More 0.96;

pass("First test");

subtest 'An example subtest' => sub {
  pass("This is a subtest");
  pass("So is this");
};

pass("Third test");

done_testing();

Subtests can have their own plan, but if they don't have one, Test::More acts like there was an implicit done_testing at the end of the code reference. That means you don't have to keep count of tests in a subtest and things still work safely.

You can use a 'skip_all' plan in a subtest, too, which is a useful way of constructing a SKIP block without having to count how many tests are being skipped the way you would with the skip() function.

use strict; use warnings;
use Test::More 0.96;

pass("First test");

subtest 'Like a SKIP block' => sub {
  plan 'skip_all' unless $required_condition;
  pass("This is a subtest");
  # ... many more tests that you don't have to count ...
};

pass("Third test");

done_testing();

The big advantage of Git::Wrapper is that it wraps the binary version of git, which keeps your perl and git independent. Upgrade perl and not git? Add Git::Wrapper and you're done. Upgrade git while keeping your existing perl? No problem. If you get your git from an OS distribution package, but roll your own perl, then Git::Wrapper is for you!

Git::Wrapper just makes it really easy to pass git commands and get back useful data. It really shines on "git log" commands, as it parses each commit message into Git::Wrapper::Log objects with accessors for id, author, date, and message. For everything else, it's not much more than a wrapper around qx(), but it returns arrays of lines and throws exceptions when things go wrong, which just saves a little time and code.

Here is a short example that I whipped up recently to automate the process of adding a github repository remote for someone when they send me a pull request. (It also uses my Getopt::Lucid module, but as the name suggests, it should be clear what that does.)

#!/usr/bin/env perl
use 5.010;
use strict;
use warnings;
use autodie;
use Git::Wrapper;
use Getopt::Lucid qw/:all/;

# USAGE: github-remote [-r REPO] [NAME]
#     NAME -- a github user name (or defaults to 'dagolden')
#     REPO -- a github repository path (or is parsed out of my private origin repo)

my $opts = Getopt::Lucid->getopt([
  Param('repo|r'),
]);

my $who = shift @ARGV;

# Get a wrapper for the current directory
my $git = Git::Wrapper->new(".");

# Locate the origin repository URL
my ($origin) = grep { /origin/ } $git->remote("-v");
die "Couldn't determine origin\n" unless $origin;
$origin =~ s/^origin\s+//;
$origin =~ s/\s+\(.*$//;

# Parse out the repository path
my $repo = $opts->get_repo;
unless ( $repo ) {
  # me@git.example.com:my-repo-name.git
  ($repo) = $origin =~ m/^[^:]+:(.+)$/;
}

# Set up someone else's github repo
if ( $who ) {
  $git->remote("add", $who, "git://github.com/${who}/${repo}");
}
# Or set up my own github mirror
else {
  $git->remote("add", "github", "git\@github.com:dagolden/${repo}");
}

That's it. I admit the program is a little crufty because of how I manage my repositories with a private origin that I mirror to a github repository, but there's almost no cruft around interactions with git. Consider this:

  $git->remote("add", $who, "git://github.com/${who}/${repo}");

If the command fails (if there's already $who as a remote name), an exception will be thrown. That means I don't have to write any error handling myself, which is perfect for a simple automation program like this one.

• CPAN Meta Spec version 2 -- I released the final spec and will soon be looking to rally toolchain developers to implement it.
• CPAN Testers 2.0 -- I launched CPAN Testers 2.0 Beta site, the eventual replacement for the current practice of sending CPAN Testers reports over email and I'm continuing to monitor it, tweak it and fix little bugs
• Perl 5 core development -- I'm scheduled to be the release-engineer for Perl 5.13.3 in July. In addition, my goals for Perl 5.13 include getting HTTP support for CPAN clients into the core, simplifying CPAN configuration further and implementing CPAN Meta.
• Drafting a formal description of the expectations for Perl build systems that use Build.PL. Right now it's only Module::Build, but it could be other things in the future. I've started a very rough draft of what cpan clients should expect Build.PL (and the resulting Build) to support.
• Dist::Zilla -- I've been converting my distributions over to Dist::Zilla as I work on them. It's a huge time saver for me (something I hope to blog about more later). I've also implemented some plugins recently: DZP::CheckExtraTests, DZP::BumpVersionFromGit and DZP::Twitter
• I converted the Module::Build repository to git to encourage more contributions and to make it easier to apply them. I released a development version, 0.36_08, with some recent additions.

Here are several other things released to CPAN in the last couple months not mentioned above:

• File-chdir-0.1003 -- maintenance release for 5.13.X
• Acme-Module-Build-Tiny-0.04 -- a half-serious replacement for Module::Build
• Version-Next-0.001 -- increment module version numbers simply and correctly
• ExtUtils-CBuilder-0.2703 -- better Windows support for MSVC
• Net-Amazon-Config-0.001 -- simple utility to manage Amazon Web Services credentials
• CPAN-Visitor-0.001 -- generic traversal of distributions in a local CPAN or minicpan repository
• Capture-Tiny-0.07 -- the only output capturing program you should ever need

It's a remarkable achievement, not only technically, but in the reaction it has sparked. As one of the (sometimes reluctant) maintainers of CPAN and CPANPLUS, I've realized that I both love and hate cpanminus.

• I love that Miyagawa has done so much with so little and in such a short span of time
• I hate that fanboy-types flocked to it and trashed the older clients without noting cpanminus' limitations
• I love that Perl toolchain maintainers have rallied around Miyagawa and contributed their wisdom to make cpanminus better instead of rejecting it
• I hate that one of Perl's great strengths (CPAN) has legacy clients that are so unwieldy, hated and difficult to maintain

Miyagawa graciously acknowledges standing on the shoulders of giants. Still, I can't shake the nagging thought that cpanminus should never have been necessary in the first place.

What I've come to realize is that cpanminus is another example of the power of not being all things to all people. Miyagawa doesn't promise that it works for all of CPAN or that it works everywhere that Perl does. He doesn't have to. Making it work for 99% of CPAN for 99% of people is more than good enough.

I've been co-maintaining various parts of the Perl toolchain for a while now. It's a frustrating challenge needing to make thing work everywhere, for everything, and trying as hard as possible not to break backwards compatibility. Plus, I don't even get to use CPAN to make life easier. I don't get to use handy tools like Moose or DateTime or Regexp::Common or SQLite or anything in the Config::* namespace or even basic tools like Archive::Zip. Nearly everything is done by hand.

Things have to work with just core Perl on a diverse set of platforms and with an incredibly limited set of assumptions. For example, the Perl core still doesn't come with an HTTP client, so CPAN has to rely on FTP or command line programs to bootstrap LWP. (This is something I personally plan to tackle during the Perl 5.13 development series later this year.)

I think this is an ongoing challenge for core Perl development in general. It's a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice. (Anyone else for use strict by default? I hope that finally comes to pass in Perl 5.14.) chromatic writes about this topic often in his Modern Perl blog and I usually tend to agree with the points he makes. (October February 2009 had a particularly good series of posts.)

In the meantime, I look at cpanminus with greed and envy. Miyagawa++

What happened next was the inevitable result of the proverb: "good, fast, or cheap -- pick two". CT2.0 was designed to be a good replacement for CT1.0, and it was being done by (cheap) all volunteer labor. So, despite some progress towards a proof of concept a year later at the Birmingham QA hackathon, there has been no real end in sight for CT1.0.

That all changed last week. With a firm deadline to hit, it's time to reconsider the good, fast or cheap tradeoff. Fast is now critical. I think the design is good enough. What can be done quickly won't replace all of the CT1.0 ecosystem right away but just the core transport and report repository parts.

I think cheap is what is going to change. It's still volunteer labor, but I think there's a way to need less of it. My current hypothesis for a plan to hit the deadline is to implement the Metabase framework on top of Amazon Web Services (AWS). That offloads scalability and reliability concerns, changing those technical and administrative challenges into resource challenges.

I've already successfully demonstrated a proof of concept that the existing Test::Reporter based testing clients can feed a CT2.0 Metabase. How well the framework scales remains a big unknown, but by implementing on top of AWS, we can throw resources at the problem in the short-run by deploying more EC2 instances to deal with bottlenecks. If the SQLite databases that drive the CPAN Testers statistics sites have to be regenerated from scratch each night, that's just a MapReduce job. If the Metabase web app is too slow to deal with the test report volume, we just deploy more instances and stick a load balancer in front.

Having that flexibility simplifies the job of getting CT2.0 off the ground to a handful of to-do's:

• Implement a Metabase backend on top of AWS
• Create an AWS virtual machine to accept reports and publish to the CT2.0 Metabase on AWS
• Migrate existing NNTP reports to the CT2.0 Metabase
Implement a web app to serve up new and legacy reports from the CT2.0 Metabase
• Design a process to update the CPAN Testers stats database with newly uploaded reports (from scratch if necessary)
• Update CPAN Testers websites to link to the new CT2.0 reports archive site
• Get testers to switch to Test::Reporter::Transport::Metabase

I think that gets most of what we need by March 1 and without a whole lot of new code to write and test and without a lot of sysadmin or DBA time and attention required. It's a limited implementation, but will solve the need of the Perl NOC to get CPAN Testers reports off their email infrastructure.

I'll be writing up more details and plans over the next week.