I have redone my Perl 5 release cycle graph again with a few changes. First, for the v5.4 through v5.8 series, I have broken the line to the final release, which I consider to be "outliers". I think the Perl community was lucky to get those releases — was lucky that someone stepped up and made them — and that they don't reflect a "normal development" or support cycle.

Second, I have projected an estimated lifecycle under the official support policy for v5.12, v5.14 and the not-yet-released v5.16. This represents an expectation for the normal support lifetime of these releases and I think shows a better contrast of expectations resulting from the support policy introduced with v5.14 compared to historical releases.

click for larger view

My observations (ignoring outliers):

• Prior to v5.14, there was a (sometimes lengthy) gap between the end of one stable series and the start of the next.
• The actively maintained periods of v5.4, v5.5 and v5.6 were shorter than the proposed support windows under the new policy
• v5.8 had two different support paradigms. Between v5.8.0 and v5.8.1 was a long gap similar to the v5.6 series. v5.8.1 to v5.8.8 had a more regularly-spaced series of support releases.
• v5.10 had the longest gap between initial release (v5.10.0) and the subsequent support release (v5.10.1)
• v5.12 has had the most consistent pattern of support releases after the initial release, and is the only stable Perl 5 to have a (regular, not outlier) support release after the release of the next stable version
• v5.14 was the first stable Perl 5 released under the new annual-release cycle

The new support policy most resembles a return to the best support period seen historically (v5.8.1 to v5.8.8), but without the subsequent gap to the next stable release.

Why do I think this new policy is a positive step forward? Here are some reasons:

• The support policy is actually written down. What expectations did anyone have prior? I don't know. But if I'm using Perl 5, I'd rather know what to expect that have to guess and hope for the best.
• The new policy offers a support window longer in practice than any Perl 5 except v5.8 and more regular than any period except between v5.8.1 and v5.8.8
• The new stable Perl 5 is available for migration testing mid-way through the support window of the prior stable release. If there are issues with migration, users can be confident of support for their existing version for an additional year (and emergency security support for a year after that).
• The annual release cycle means the change between one stable release and the next will be smaller, lowering migration risk

The new policy does cut off the "long tail" of expectations for an outlier release. I can understand that for some companies or OS packagers, a two-year support window (three for security) might feel too short, even if that is longer than was typically seen historically.

Here is where the annual stable release cycle and monthly development release cycle offer a huge side benefit: there is now a well-documented, frequently-used, regularly-updated release manager's guide for Perl 5. Now, the release process is so easy that a moderately-skilled Perl software engineer without much prior exposure to the Perl source repository can make a Perl 5 release tarball in about a day.

This means that even if the core Perl 5 development team isn't supporting, say, v5.12 anymore, a motivated company or community group could do the work necessary to prepare their own "outlier" release and either petition the Perl 5 core team to release it or could release their own stable "micro-fork" for others with long-term support needs. (There might even be a profitable business opportunity selling support for Perl versions past the official support window.)

Previously, Perl 5 development used to be bursty, with long delays between stable releases and with unclear expectations for support. Now, Perl 5 development happens like clockwork, and has a clear, written support policy.

[Note: this post represents my individual opinion and was not reviewed by the Perl 5 Porters core development team; it may or may not represent the views of other core developers; it is certainly not an "official" statement of the Perl 5 Porters in any way]

This is a significant change from the history of Perl, so I though it would be interesting to see how recent release cycles have compared to historic ones. The chart below shows releases over time since Perl 5, version 4 when releases were more officially split between "stable" and intermediate releases.

click for larger view

(Note: Starting with version 7, odd numbered versions were reserved for development releases and are omitted above. Versions 13 and 15 moved to a monthly release cycle for easier community testing of incremental development.)

I think the overall change to a shorter development cycle will benefit both users and maintainers of Perl 5. For users, each new release will be a smaller change from the previous, lowering upgrade risk, plus they can have confidence in an ongoing process of improvement. For maintainers, it avoids taking away effort from mainline development to retro-fit patches into a Perl that is many years old and might have substantially different guts.

The counterfactual is this: "If Perl 6 never existed, then Perl 5 could bump itself to Perl 6 NOW in order to signal a backwards-incompatible break from the past."

If the counterfactual were true, I believe this could happen, even with the limitations of the Perl 5 development process we have today. Perl 6 (as it actually does exist today) is revolutionary design, not evolutionary design, but that doesn't preclude an evolution of Perl 5 that significantly breaks backwards compatibility.

Without recommending any particular change, here are some examples of things that I think could be possible, just to give a sense of what evolutionary (rather than revolutionary) changes could attempt:

• Replacing eval BLOCK syntax with try BLOCK
• Removing of Unix-specific user/group functions and similar cruft
• Eliminating of indirect syntax
• Replacing method call arrow -> with the shorter (and more widely recognized) dot/period character (and finding a new concatenation operator)
• Eliminating version objects and v-strings
• Replacing (frequently misunderstood) function prototypes with function signatures

Those are just ideas off the top of my head, but I hope it's easy to see than any of those break enough existing code as to make them very difficult to envision being added to any future release of Perl 5. However, if the counterfactual were true, then there would always be the option of making a clean break to "Perl 6", ensuring that there is no automatic expectation of backwards compatibility.

Even if the counterfactual were true, making such evolutionary changes would not necessarily be easy. It would require a Pumpking (other than Larry Wall) to play language designer for Perl 5 and to push the Perl core developer community into a rough consensus. But hard does not mean impossible.

However, with the reality of Perl 6 as it stands, even a name change by Perl 6 ("Camelia" was one suggestion) doesn't help Perl 5's evolution. Given the expectations that have been set for Perl 5, it seems unlikely to me that there would be broad community support for Perl 5 evolving into Perl 6 if Perl 6 were renamed. "Hi, we promised you Perl 6 ten years ago, but it's not ready for production, so we're going to call this other thing Perl 6 instead." I don't think that's great for external perceptions, either. What then? Jump over Perl 6 straight to Perl 7? I don't think that would be well received, either.

For Perl 5 to evolve in ways that are significantly backwards-incompatible, I see only two realistic options:

• Rename Perl 5 to something else, with all the associated branding and marketplace traction issues that would cause, but with the ability to make a major version number bump to manage compatibility expectations.
• Increase the speed of mutation of Perl 5 so that Versions 16, 18, 20, etc. become bigger departures cumulatively than we ever saw with Versions 6, 8, 10, etc., and become much more ruthless about setting future compatibility expectations.

(I suppose an interesting variation on the first way would be for Perl 5 and Perl 6 to be renamed simultaneously. I could see that having some positive opportunities for branding and marketing.)

The status quo alternative is for Perl 5 and Perl 6 to continue as they have been, slowly chugging along on their respective tracks. Perl 5 will continue to put backwards compatibility over evolution and won't see substantial new features added or old mis-features cleaned up. Perl 6 will continue to push towards some sort of production readiness.

For some (perhaps many), the status quo is ideal. For me, I would find it rather boring to be a part of.

So -- let that be the real debate in the Perl 5 community: compatibility or evolution? Names and version numbers are just the means to an end.

Barring any show-stopping bugs, the forthcoming Perl 5.13.9 will let a non-privileged user bootstrap access to CPAN, even if FTP is blocked and they have no command line HTTP clients installed. That means that if you can run perl, and you can use port 80, then you can use CPAN.

Imagine someday when Perl 5.14 becomes standard in linux distributions. Log-in to a new machine (or perhaps a shared host), type "cpan", and start installing stuff from CPAN. Easy!

To show you how it works, I installed a snapshot of Perl from the git repository and then switched to a clean, unprivileged user account to try installing something. All I typed at the keyboard was this:

/opt/perl/fromgit/bin/cpan5.13.8[ENTER]
[ENTER]
[ENTER]
[ENTER]
[ENTER]
install File::Marker[ENTER]

The log below is long and verbose (the way CPAN.pm just is, unfortunately) but you can see how it uses HTTP::Tiny, bootstraps local::lib, updates my .bashrc, installs File::Marker in the local library and even reminds me to restart my shell. Isn't that user-friendly?

$ /opt/perl/fromgit/bin/cpan5.13.8 

CPAN is the world-wide archive of perl resources. It consists of about
300 sites that all replicate the same contents around the globe. Many
countries have at least one CPAN site already. The resources found on
CPAN are easily accessible with the CPAN.pm module. If you want to use
CPAN.pm, lots of things have to be configured. Fortunately, most of
them can be determined automatically. If you prefer the automatic
configuration, answer 'yes' below.

If you prefer to enter a dialog instead, you can answer 'no' to this
question and I'll let you configure in small steps one thing after the
other. (Note: you can revisit this dialog anytime later by typing 'o
conf init' at the cpan prompt.)

Would you like me to configure as much as possible automatically? [yes] 

 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib] 

Autoconfigured everything but 'urllist'.

Now you need to choose your CPAN mirror sites.  You can let me
pick mirrors for you, you can select them from a list or you
can enter them by hand.

Would you like me to automatically choose some CPAN mirror
sites for you? (This means connecting to the Internet) [yes] 

Trying to fetch a mirror list from the Internet
Fetching with HTTP::Tiny:
http://www.perl.org/CPAN/MIRRORED.BY

Looking for CPAN mirrors near you (please be patient)
..............................................................
..............................................................
..............................................................
..................................... done!

New urllist
  http://cpan.erlbaum.net/
  http://cpan.mirror.clemson.edu/
  http://mirrors.servercentral.net/CPAN/

Autoconfiguration complete.

Attempting to bootstrap local::lib...

Writing /home/cleanroom/.cpan/CPAN/MyConfig.pm for bootstrap...
commit: wrote '/home/cleanroom/.cpan/CPAN/MyConfig.pm'
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/01mailrc.txt.gz
Going to read '/home/cleanroom/.cpan/sources/authors/01mailrc.txt.gz'
............................................................................DONE
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/modules/02packages.details.txt.gz
Going to read '/home/cleanroom/.cpan/sources/modules/02packages.details.txt.gz'
  Database was generated on Sat, 15 Jan 2011 20:57:19 GMT
  HTTP::Date not available
............................................................................DONE
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/modules/03modlist.data.gz
Going to read '/home/cleanroom/.cpan/sources/modules/03modlist.data.gz'
............................................................................DONE
Going to write /home/cleanroom/.cpan/Metadata
Running make for A/AP/APEIRON/local-lib-1.008001.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/A/AP/APEIRON/local-lib-1.008001.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/A/AP/APEIRON/CHECKSUMS
Checksum for /home/cleanroom/.cpan/sources/authors/id/A/AP/APEIRON/local-lib-1.008001.tar.gz ok

  CPAN.pm: Going to build A/AP/APEIRON/local-lib-1.008001.tar.gz

Attempting to create directory /home/cleanroom/perl5

*** Module::AutoInstall version 1.03
*** Checking for Perl dependencies...
*** Since we're running under CPAN, I'll just let it take care
    of the dependency's installation later.
[Core Features]
- ExtUtils::MakeMaker ...loaded. (6.57_05 >= 6.31)
- ExtUtils::Install   ...loaded. (1.55 >= 1.43)
- Module::Build       ...loaded. (0.3607 >= 0.36)
- CPAN                ...loaded. (1.9463 >= 1.82)
*** Module::AutoInstall configuration finished.
Checking if your kit is complete...
Looks good
Writing Makefile for local::lib
Writing MYMETA.yml
cp lib/POD2/PT_BR/local/lib.pod blib/lib/POD2/PT_BR/local/lib.pod
cp lib/lib/core/only.pm blib/lib/lib/core/only.pm
cp lib/local/lib.pm blib/lib/local/lib.pm
cp lib/POD2/DE/local/lib.pod blib/lib/POD2/DE/local/lib.pod
Manifying blib/man3/POD2::PT_BR::local::lib.3
Manifying blib/man3/lib::core::only.3
Manifying blib/man3/local::lib.3
Manifying blib/man3/POD2::DE::local::lib.3
  APEIRON/local-lib-1.008001.tar.gz
  /usr/bin/make -- OK
'YAML' not installed, will not store persistent state
Running make test
PERL_DL_NONLAZY=1 /opt/perl/fromgit/bin/perl5.13.8 -I/home/cleanroom/perl5/lib/perl5/x86_64-linux-thread-multi -I/home/cleanroom/perl5/lib/perl5 "-MExtUtils::Command::MM" "-e" "test_harness(0, 'inc', 'blib/lib', 'blib/arch')" t/classmethod.t t/coderefs_in_inc.t t/de-dup.t t/install.t t/lib-core-only.t t/pipeline.t
t/classmethod.t ...... Name "File::Spec::rel2abs" used only once: possible typo at t/classmethod.t line 20.
t/classmethod.t ...... 1/? Attempting to create directory t/var/splat
t/classmethod.t ...... ok
t/coderefs_in_inc.t .. ok
t/de-dup.t ........... ok
t/install.t .......... skipped: Install Capture::Tiny to test installation
t/lib-core-only.t .... ok
t/pipeline.t ......... ok
All tests successful.
Files=6, Tests=10,  1 wallclock secs ( 0.04 usr  0.01 sys +  0.19 cusr  0.01 csys =  0.25 CPU)
Result: PASS
  APEIRON/local-lib-1.008001.tar.gz
  /usr/bin/make test -- OK
Running make install
Manifying blib/man3/POD2::PT_BR::local::lib.3
Manifying blib/man3/lib::core::only.3
Manifying blib/man3/local::lib.3
Manifying blib/man3/POD2::DE::local::lib.3
Installing /home/cleanroom/perl5/lib/perl5/local/lib.pm
Installing /home/cleanroom/perl5/lib/perl5/POD2/PT_BR/local/lib.pod
Installing /home/cleanroom/perl5/lib/perl5/POD2/DE/local/lib.pod
Installing /home/cleanroom/perl5/lib/perl5/lib/core/only.pm
Installing /home/cleanroom/perl5/man/man3/POD2::PT_BR::local::lib.3
Installing /home/cleanroom/perl5/man/man3/lib::core::only.3
Installing /home/cleanroom/perl5/man/man3/local::lib.3
Installing /home/cleanroom/perl5/man/man3/POD2::DE::local::lib.3
Appending installation info to /home/cleanroom/perl5/lib/perl5/x86_64-linux-thread-multi/perllocal.pod
  APEIRON/local-lib-1.008001.tar.gz
  /usr/bin/make install  -- OK

local::lib is installed. You must now add the following environment variables
to your shell configuration files (or registry, if you are on Windows) and
then restart your command line shell and CPAN before installing modules:

export PERL_LOCAL_LIB_ROOT="/home/cleanroom/perl5"
export PERL_MB_OPT="--install_base /home/cleanroom/perl5"
export PERL_MM_OPT="INSTALL_BASE=/home/cleanroom/perl5"
export PERL5LIB="/home/cleanroom/perl5/lib/perl5/x86_64-linux-thread-multi:/home/cleanroom/perl5/lib/perl5:$PERL5LIB"
export PATH="/home/cleanroom/perl5/bin:$PATH"

Would you like me to append that to /home/cleanroom/.bashrc now? [yes] 

commit: wrote '/home/cleanroom/.cpan/CPAN/MyConfig.pm'
Terminal does not support AddHistory.

cpan shell -- CPAN exploration and modules installation (v1.9463)
Enter 'h' for help.

cpan[1]> install File::Marker
Running install for module 'File::Marker'
Running make for D/DA/DAGOLDEN/File-Marker-0.13.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/D/DA/DAGOLDEN/File-Marker-0.13.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/D/DA/DAGOLDEN/CHECKSUMS
Checksum for /home/cleanroom/.cpan/sources/authors/id/D/DA/DAGOLDEN/File-Marker-0.13.tar.gz ok

  CPAN.pm: Going to build D/DA/DAGOLDEN/File-Marker-0.13.tar.gz

Creating new 'MYMETA.yml' with configuration results
Creating new 'Build' script for 'File-Marker' version '0.13'
Building File-Marker
  DAGOLDEN/File-Marker-0.13.tar.gz
  ./Build -- OK
Running Build test
t/01_File_Marker.t ... ok
t/02_fork.t .......... ok
t/03_thread.t ........ ok
t/04_save_load.t ..... ok
t/98_pod.t ........... skipped: Skipping author tests
t/99_pod_coverage.t .. skipped: Skipping author tests
All tests successful.
Files=6, Tests=58,  1 wallclock secs ( 0.04 usr  0.02 sys +  0.22 cusr  0.02 csys =  0.30 CPU)
Result: PASS
  DAGOLDEN/File-Marker-0.13.tar.gz
  ./Build test -- OK
Running Build install
Building File-Marker
Installing /home/cleanroom/perl5/lib/perl5/File/Marker.pm
Installing /home/cleanroom/perl5/man/man3/File::Marker.3
  DAGOLDEN/File-Marker-0.13.tar.gz
  ./Build install  -- OK

cpan[2]> q
Terminal does not support GetHistory.
Lockfile removed.

*** Remember to restart your shell before running cpan again ***

I want to thank Andreas Koenig (CPAN.pm) and Chris Nehren (local::lib) for quickly turning around new releases of their modules to make this work.

Here's how it works. As of Perl 5.12, internally, Perl 5 file handles are represented as IO::File objects:

$ perl -E 'say(ref(*STDERR{IO}))'
IO::File
$ perl -E 'open my $fh, ">", "/dev/null"; say(ref(*{$fh}{IO}))'
IO::File
$ perl -E 'my $str; open my $fh, ">", \$str ; say(ref(*{$fh}{IO}))'
IO::File

However, trying to call methods on them fails if you haven't loaded IO::File (or a parent class like IO::Handle):

$ perl -E 'STDOUT->binmode(":utf8"); say "\x{263a}"'
Can't locate object method "binmode" via package "IO::File" at -e line 1.

With the patch I wrote, just before such a method call would die, if the call is on an IO::File object and $INC{'IO/File.pm'} doesn't exist, Perl attempts to load IO::File and tries to resolve the method again:

$ bleadperl -E 'STDOUT->binmode(":utf8"); say "\x{263a}"'
☺

This has zero performance impact on code that doesn't call methods on file handles or that loads IO::File (or a parent class) before calling a method on a file handle. It only kicks in when Perl would otherwise die on you.

Barring any issues emerging, this feature will be available in the development release of Perl 5.13.8 on or around December 20, just in time for Christmas.

Some of the highlights:

• New regex modifiers to force unicode or locale-specific semantics (5.13.6)
• Regular expression retain original semantics when interpolated later (5.13.6)
• Array and hash functions (push, pop, keys, etc.) work on array and hash references (5.13.7)
• Set default regular expression modifiers in a lexical scope (5.13.7)
• Non-destructive transliteration (y///r) just like substitution (s///r) (5.13.7)

I've just finished some work to add HTTP::Lite support to CPAN.pm and Andreas has merged it into the master branch. The next step will be to test it out in a development release. If all goes well, the plan is to merge the latest CPAN.pm into the Perl core and bring in HTTP::Lite as well. Then, finally, a mere 20 years after the invention of HTTP, Perl 5.14 will be able to speak HTTP out of the box and bootstrap access to all of CPAN.

Keep your fingers crossed.

Add single-term prototype

The C<+> prototype is a special alternative to C<$> that will act like
C<\[@%]> when given a literal array or hash variable, but will otherwise
force scalar context on the argument. This is useful for functions which
should accept either a literal array or an array reference as the argument

I've finished most of the hard work on my next patch, which would allow push/pop/keys/values/etc to act directly on references of an appropriate type. Now, I'm drafting the email that explains the change and the rationale and hope to submit it to the p5p list later this week.

If you liked what I was proposing in some of my recent posts on the topic, keep your fingers crossed that any debate on it is mild and constructive and we'll have some new, cool features in Perl for either Thanksgiving or Christmas.

Here is a simple example:

sub foo($) { say shift }

The '$' prototype says that foo takes a single argument in scalar context. Consider this:

$ perl -wE 'sub foo($){say shift} @array=("a".."z"); foo(@array)'
26

If your reaction is, "What?!?", you need to remember than an array in scalar context is the length of the array. And just to drive home the point that prototypes can be surprising, consider these two:

$ perl -wE 'sub foo($){say shift} foo("a","b","c")'
Too many arguments for main::foo at -e line 1, near ""c")"
Execution of -e aborted due to compilation errors.

$ perl -wE 'sub foo($){say shift} foo(qw/a b c/)'
Useless use of a constant (a) in void context at -e line 1.
Useless use of a constant (b) in void context at -e line 1.
c

In the former case, the error is clear. In the latter case, the qw/a b c/ is equivalent to a parenthesized expression ("a", "b", "c") which in scalar context evaluates throws away the left values and returns the right-most argument. (See documentation for the Comma Operator.)

These kinds of unexpected errors are why you generally shouldn't use prototypes. However, they are absolutely necessary to override or mimic the effect of a built-in keyword.

For my work on push() and keys(), I want to be able to represent a prototype that takes an array or an array reference or a hash or a hash reference. You can get perl to pass a reference to an array instead of the array itself using the \@ prototype like this:

$ perl -wE 'sub foo(\@){say shift} @array=("a".."z"); foo(@array)'
ARRAY(0x1156588)

But that doesn't let you use an array reference as an argument -- only a literal array variable can be used, even though a reference is what is passed to the function when called as foo(@array).

$ perl -wE 'sub foo(\@){say shift} @array=("a".."z"); foo(\@array)'
Type of arg 1 to main::foo must be array (not reference constructor) at -e line 1 ...

$ perl -wE 'sub foo(\@){say shift} foo(["a".."z"])'
Type of arg 1 to main::foo must be array (not anonymous list ([])) at -e line 1 ...

Since the prototype I want doesn't exist, I've started working on a patch that would allow something like this to work:

sub foo(+) { say shift }

foo( @array );   # passed as \@array
foo( %hash );    # passed as \%hash
foo( $arrayref );
foo( $hashref );
foo( bar() );    # bar() called in scalar context

The "+" prototype passes literal array or hash variables using a reference, but passes any other argument in scalar context. I call it a "single term" prototype. With luck and favorable reviews, I hope it will make it into the November development release of Perl.

• Reviewing and finalizing 'package NAME VERSION' patches for the Perl core (hopefully before the code freeze)
• CPAN Testers 2.0 migration project -- volunteers are jumping in, which is good since I've barely had time to do anything but answer questions and help give people direction. I haven't even had a chance to blog about first week progress
• CPAN Meta Spec 2.0 -- delayed once and will be delayed again. This is really, really close to done, but just doesn't have the urgency of everything else
• Assorted bug reports and module patches -- and only if they take a few minutes or less

That's my Iron Man post for the week. Stay tuned next week for actual progress reports, I hope.