Real $VERSIONs on CPAN

I've been looking at patterns of $VERSION definition in the wild. And, wow. There's some crazy stuff out there.

Remember that the way PAUSE (and other tools) parse a $VERSION definition line is not by loading the module, but by extracting it as a standalone line and essentially running it through eval. PAUSE says this must work:

perl -MExtUtils::MakeMaker -le 'print MM->parse_version(shift)' path/to/file.pm

PAUSE actually does something more sophisticated and safe, but if parse_version works, so will PAUSE.

Keeping that in mind, listed below are some of the more surprising/amusing/horrifying things I've seen. Not all are broken, but even the ones that work are, well, unusual.

I'm not naming any names — and I'll obscure them if it would be too revealing — but these are all from real .pm files in tarballs currently indexed on CPAN.

$VERSION = '';

$VERSION = '0.O1';

$VERSION = "0.01c";

$VERSION = VERSION;

$VERSION = '0.10E0';

our $VERSION = -722;

my $VERSION = '0.01';

$VERSION = 0xdeadbeef;

our $VERSION = '1.0-1';

$VERSION = 2006_08_16.1;

our $VERSION = '0.8.1-2';

$VERSION = $VERSION = "0.1";

$Foo::Bar::VERSION |= '2.6';

my $VERSION = 'OMG-04-05-01';

our $VERSION = '1.4.A8UG1gG';

our $VERSION = 'set-when-loading';

our $VERSION = '$Revision: 1.2 $';

our $VERSION = $Foo::Bar::VERISON;

our $VERSION=$Foo::VERSION; use Foo;

local $Other::Module::VERSION = 666;

$Foo::Bar::VERSION="1.23" unless $Foo::Bar::VERSION;

$Foo::Bar::VERSION='1.00' unless $INC{'Foo/Bar0.pm'};

$VERSION=eval 'use version;1' ? 'version'->new('0.33') : '0.33';

Some of these are so nuts that unless they are modified by a subsequent line into a standard version format, then a version check will throw an error on any recent perl:

# Foo.pm
package Foo;
our $VERSION = "1.2-trailing-junk";
$ perl -I. -we 'use Foo 0;'
Invalid version format (non-numeric data) at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

Moral of this story: make sure your $VERSION definition parses cleanly on a line by itself and make sure it's a valid version number. If you're not sure, check it with the is_lax function from version.pm.

Posted in perl programming | Tagged , , | Comments closed

Previewing POD before shipping

I always find typos after I ship to CPAN.

There is something about reading my docs in nicely-formatted HTML on a web site that makes my mistakes jump out at me. And then I feel stupid for not having caught it earlier.

I used to use the search.cpan.org pod renderer to preview my POD, but I stopped after a while, probably when I stopped using search.cpan.org for everything else as well.

What I really wanted was something local, quick and pretty. So I ripped off a bit of what MetaCPAN was doing to render POD and came up with my own utility program: podpreview.

$ podpreview path/to/whatever.pm

That renders the POD from the file into HTML with an embedded stylesheet, saves it in a temporary file and opens up that file in my browser. There I can happily proofread and find my typos before shipping.

If you'd like to try it out and adapt it to your own style, here it is:

#!/usr/bin/env perl
use v5.10;
use strict;
use warnings;
use Browser::Open qw/open_browser/;
use Path::Tiny;
use Pod::Simple::XHTML;

my $file = shift @ARGV
  or die "Usage: $0 <file>";
$file = path($file);

my $psx = Pod::Simple::XHTML->new;
$psx->output_string( \my $html );
$psx->html_charset('UTF-8');
$psx->html_encode_chars('&<>">');
$psx->perldoc_url_prefix("https://metacpan.org/module/");
$psx->html_header( my_header() );
$psx->html_footer( my_footer() );
$psx->parse_string_document( $file->slurp_utf8 );

my $temp = path( $ENV{TMPDIR}, 'podpreview', $file->relative . '.html' );
$temp->touchpath;
$temp->spew_utf8($html);
open_browser("file:///$temp");

sub my_css {
    return <<'CSS';
body { background: snow; font-family: sans-serif; }
div#main { width: 70%; margin: 5% auto; }
h1 { font-size: 1.5em; margin: .83em 0 }
h2 { font-size: 1.17em; margin: 1em 0 }
h3 { margin: 1.33em 0 }
h4 { font-size: .83em; line-height: 1.17em; margin: 1.67em 0 }
h5 { font-size: .67em; margin: 2.33em 0 }
h1, h2, h3, h4, h5 { font-weight: bolder; color: #36c }
a:link { color: #36c }
code { font-size: 1.2em }
CSS
}

sub my_header {
    my $css = my_css();
    return <<"HEADER";
<html>
<head>
<title></title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<style>
$css
</style>
</head>
<body>
<div id="main">
HEADER
}

sub my_footer {
    return <<'FOOTER';
</div>
</body>
</head>
FOOTER
}
Posted in perl programming | Tagged , , | Comments closed

How do you manage your DarkPAN?

Hi, folks.

I'm doing research for a talk about how people manage and deploy proprietary Perl code and CPAN code and how they manage their dependency chains over time.

If your current (or past) company does something you think is particularly good (or bad!), I'd love to swap emails or do a 15 or 20 minute phone call with you. (You and/or your company can be anonymous in the final talk if you desire.)

Please email me at "dagolden" at "CPAN dot org" if you're willing to share.

Thank you very much!

David

Posted in perl programming | Tagged , , | Comments closed

Nothing to see here

This is a test to see if the ironman feed is picking up this category/tag.

Posted in devops | Tagged , | Comments closed

Paying respect to Module::Build

Earlier this week, I proposed that Module::Build be deprecated from the Perl 5 core. After some discussion, this proposal has been accepted by the Pumpking.

I want to take a moment to discuss what this means, why I suggested it, and what I think Module::Build trail blazed for Perl 5.

Deprecation means a warning in v5.19 and removal in v5.22

Soon, in Perl 5 version 19 (the current development series), the Module::Build included in the core library path will issue a warning when used. If Module::Build is then installed from CPAN, it will be installed into the "sitelib" path and the deprecation warning will stop.

Sometime in Perl 5 version 21 (starting late Q2 2014), Module::Build will most likely no longer ship with the core Perl and will need to be installed from CPAN.

Fortunately, CPAN clients already recognize a 'configure requires' directive in CPAN distribution meta files (META.yml or META.json) and can bootstrap Module::Build before Build.PL runs.

So... the biggest impact on end-users will be that when Perl 5 version 20 is released in Q2 2014, people will have to install Module::Build from CPAN to squelch the deprecation warning.

Update: Miyagawa reminded me on IRC that the major CPAN clients will install from CPAN automatically if they see a deprecated module in a prereq list, so actually, most people won't even notice. Shiny!

Module::Build was good, but not good enough, and it plateaued

Before Module::Build, we only had ExtUtils::MakeMaker and it sucked. Michael Schwern, its long-time maintainer, even wrote a presentation called MakeMaker is DOOMED! that encouraged people to switch to Module::Build. In hindsight, this was premature.

For all the many problems that Module::Build fixed, it introduced some of its own, built up its own technical debt, and suffered a crisis of maintenance.

I tried to roughly estimate the amount of effort going into Module::Build during four phases of its life. I used lines of Changes file as my metric (though I think "git diff --stats" would be fairly similar):

  • 2001-2007: Ken Williams author and maintainer → 2397 lines of Changes
  • 2007-2009: Ken and Eric Wilhelm tag-team → 310 lines of Changes
  • 2009-2011: David Golden maintainer → 1033 lines of Changes
  • 2011-now: Leon Timmermans maintainer → 95 lines of Changes

After my announcement that I was stepping down as Module::Build maintainer, no one volunteered for seven months until Leon kindly offered to be a "caretaker" and shepherd some patches and releases -- partly as a side effect of his work on a Module::Build replacement called Module::Build::Tiny, which itself was a serious spin off of a half-joke of my own called Acme::Module::Build::Tiny.

Module::Build innovated things now taken for granted

The best thing that Module::Build did was define a de facto specification for using a Build.PL to drive a perl-based (rather than Makefile-based) install program. That work has been formalized into a Build.PL Spec, so other Perl-based builders can be developed.

Module::Build also introduced the META.yml file that evolved into the CPAN::Meta::Spec that is in widespread use today. The META.yml file also helped solve a tricky bootstrapping problem: by specifying configure_requires dependencies within the META file, CPAN clients could install whatever modules were necessary to run Build.PL.

With the release of Perl v5.10, both CPAN and CPANPLUS supported configure_requires, meaning that the groundwork for future Build.PL-based alternatives was already in place!

Module::Build also introduced the install_base parameter as a way to specify a custom install location. It was much easier to understand than PREFIX from ExtUtils::MakeMaker, and was subsequently adopted by ExtUtils::MakeMaker as INSTALL_BASE. This is a critical part of the magic behind tools like local::lib and Carton.

The other crucial innovation was that — for the first time — customizing the build, test and install process could be done by writing only Perl code rather than writing Perl code to spit out Makefile fragments. It made building complex modules much easier — particularly Alien modules like Alien::wxWidgets. That then made projects like Padre possible.

Module::Build also spawned a counter-reaction in the form of Module::Install, which tried to make the easy Perl customization possible, while shielding users from pure ExtUtils::MakeMaker and avoiding the bootstrap problems of Module::Build by bundling itself in inc/. Module::Install then triggered a counter-reaction in the form of Dist::Zilla, which then led to Dist::Milla and Minilla.

Module::Build was the trail blazer for the tools that came after.

Module::Build made its own unique mistakes

People have complained that Module::Build was bloated. In lines of code, it's actually comparable to ExtUtils::MakeMaker. The bigger problem is that it puts 4,200 of its 5,800 lines of code in just one file: Module::Build::Base. (ExtUtils::MakeMaker split similar functionality across three mega files.)

More than just size, Module::Build is complex. The Build.PL file runs configuration and serializes the results into some files, which the Build file uses to reconstruct the original Module::Build object. Arguments can modify properties at any stage. And since Build.PL might really be a subclass, there's a lot of meta object stuff going on just to manage the configuration before ever getting around to the real business of building and installing modules.

It also suffered from feature creep. Instead of just being an install tool, it became a swiss-army-knife author's tool, with release-time features never needed by end-users, but which forced end-users to upgrade Module::Build just to run Build.PL without error. It added new concepts, like "optional features", which were poorly specified and have never achieved much traction.

One of the big, valid complaints is that it never incorporated a proper dependency system. Actions (build, test, etc.) could depend on each other, but there was nothing like Makefile's ability to detect that since file "A" changed, then action "B" had to run.

My personal pet peeve — possibly one of the big reasons I got discouraged doing maintenance — was that it also included Module::Build::Compat, which was used to generate a Makefile.PL from the Build.PL. While this seemed like a benefit to ease transition, it meant that Module::Build needs to maintain feature-compatibility — and in many cases bug-compatibility — with ExtUtils::MakeMaker effectively forever.

Module::Build promised easy subclassing and this was mostly true. But re-use and sharing was nearly impossible. If you had a subclass to do one thing and I had a subclass that did something else and you wanted to combine them, you pretty much had to copy and paste code. Contrast that with Dist::Zilla's incredible plugin ecosystem — where just about anything you want to do has been written up into a plugin that you can just drop in.

Module::Build will live on as a CPAN distribution

Module::Build never became the uncontested successor to ExtUtils::MakeMaker. It's not used as part of the Perl 5 core build process. It originally went in at least in part to ease adoption, but now all CPAN clients can bootstrap it on demand.

It's not a bad module, but it has no reason to live in the Perl 5 core any more.
Removing it means one less thing for the already-stretched Perl 5 porters to maintain, update and support.

Module::Build helped us through a critical transition away from purely Makefile based installers. It will continue to live on CPAN and will continue to support the thousands of distributions that rely on it. If a motivated maintainer came along, it might even start to innovate again, or pay down its technical debt.

I give it — and its creator, Ken Williams — my respect for what it accomplished, even while I bid it farewell from the core.

Posted in p5p, perl programming, toolchain | Tagged , , , | Comments closed

© 2009-2014 David Golden All Rights Reserved