My first attempts at replication failed, so after a few emails back and forth, Chad sent me detailed instructions for replication. Here is an excerpt from his email (used with his permission):

I setup a new user, and deleted his .bashrc, leaving only the stock
Ubuntu 10.10 /etc/bash.bashrc in the environment. Then I created this
three-line ~/.bashrc :

perl5=/tmp/tmplib
lib="$perl5/lib/perl5"
eval $(perl -I"$lib" -Mlocal::lib="$perl5")

So, the environment now resolves to:

PERL5LIB=/tmp/tmplib/lib/perl5/x86_64-linux-gnu-thread-multi:/tmp/tmplib/lib/perl5
PERL_LOCAL_LIB_ROOT=/tmp/tmplib
PERL_MB_OPT='--install_base /tmp/tmplib'
PERL_MM_OPT=INSTALL_BASE=/tmp/tmplib

Notice that Chad describes the setup in detail, and even went to the trouble of replicating it with a "clean" user. This made it very easy to set up exactly the same situation on my development machine.

Next, Chad walked me through how to replicate the issue and even confirmed it on a fresh virtual machine!

Then I installed Parse::CPAN::Meta 1.42 (with CPAN 1.9402 and
local::lib 1.008), then I tested MB 0.37_04 which gave the same
errors. Then I quit the cpan shell, restart it, first do an explit
test of PCM 1.4401 before running a test of MB and all tests pass, as
before.

I also verified the same behavior on a virtual machine with a
fresh ubuntu 10.10 (with updates) and the same three-line .bashrc and
got the same behavior. I'm surprised that you cannot reproduce this.
At this point I don't believe there is anything left that is specific
to my environment.

With those clear instructions, I was able to replicate the issue.

Chad then took me through variations and outcomes:

I then upgraded CPAN to 1.94_65 but have the same errors on MB 0.37_04
unless I explicitly test PCM 1.4401 first.

And I have the same problems with MB version 0.37_05 as well, which
also works after an explicit test of PCM 1.4401

I tried to start working backwards a bit, MB 0.3624 is fine,
presumably because it doesn't depend on PCM.
However, MB 0.3701 fails, despite the fact that it only depends on PCM
1.42, which is the one that's installed, according to pmvers.  It
looks to be the same issue in each case: the existing PCM is not always detected.

Now I have a fact base that I can test and confirm. Finally, Chad did some further digging into the problem:

Then I looked at t/mymeta.t and traced back to CPAN::Meta and looked
at it dependencies in Makefile.PL to find that PCM 1.44 is listed both
as prereq and as a build prereq, which, being a novice, looked a bit
funny to me. Taking out the build prereq and leaving the prereq then
allowed me to test MB without errors.

And with that, I've got a decent workaround solution, as well as a hint as to what's going wrong in CPAN.pm. I haven't fixed CPAN.pm yet, but with this start, it's going to be much, much easier.

I want to thank Chad for his responsiveness and the incredible detail of his report. I hope it can be a lesson to anyone reporting bugs or responding to questions about failure:

1. Describe your environment in detail. If you can, replicate the error with a "clean" user or even on a clean install of the OS.
2. Describe the exact steps you took to replicate the failure. Take notes as you do it, so you can be as specific as possible.
3. Describe any variations you tried that did work, or any workarounds you used to deal with the problem.

That seems like common sense, but few bug reports or failure investigations I get are as thorough and constructive as the example above. Thank you, Chad!

Please give it a try! From the command line:

$ cpan DAGOLDEN/CPAN-1.94_65.tar.gz

Thank you to everyone contributing patches or commits (according to the git log): Andreas Koenig, David Golden, Frank Wiegand, Nick Patch, Robert Bohne, Tomas Doran, brian d foy and burak. Thank you as well to anyone else who contributed to RT tickets or sent in patches outside git.

Here is a summary of major changes and bug fixes since 1.9402.

New features

• Major simplification of the FirstTime experience for new users, including auto-pick of CPAN mirrors
• Added support for bootstrapping local::lib when the user does not have write access to perl's site library directories
• Added support for and prerequisite on HTTP::Tiny for pure-perl HTTP bootstrapping
• Added support for META/MYMETA.json files if CPAN::Meta is installed
• Quieter user interface: made lots of '$module missing' type warnings only warn once; eliminated 'no YAML' warnings for distroprefs if there are no distroprefs.
• Allows Foo/Bar.pm on the commandline to mean Foo::Bar
• Allows calling make/test/install with regexp if unambiguous
• bzip2 support should now be on par with gzip

New configuration options

• added 'atexit' option for scan_cache
• new config option prefer_external_tar (RT#64037)
• new config variable version_timeout used in CPAN::Module::parse_version()

RT Tickets closed

• RT #63357: use Dumpvalue when dumping potential crap
• RT #62986: original config directories will be found even if File::HomeDir is later installed
• RT #62064: build_requires_install_policy set to "no" did not work correctly
• RT #61607: make the FTP download code more robust
• RT #59216: make sure $builddir exists before calling tempdir
• RT #57482 and RT #57788 revealed that configure_requires implicitly assumed build_requires instead of normal requires.
• RT #55093: no_proxy doesn't work with more then one entries
• RT #55091: don't ask the proxy credentials if proxy_user empty
• RT #53305: amended lib/App/Cpan.pm because of a regression bugfix: Non-English locales got no diagnostics on a failed locking due to permissions
• RT #51018: do not switch to default sites when we have a user-configured urllist
• RT #48803: avoid 'unreached' if not following configure_requires bugfix: treat modules correctly that are deprecated in perl 5.12. improved support for Perl core module deprecation
• RT #47774: allow duplicate mention of modules in Makefile prelude
• Fixed rt.perl.org#72362: CPAN ignoring configure_requires. Also fixed (MY)META.yml processing to always prefer Parse::CPAN::Meta, if available.
• Fixed rt.perl.org#72348: missing CPAN::HandleConfig::output;

Other bug fixes

• Adds HOMEDRIVE/HOMEPATH or USERPROFILE as home directory options on Windows
• Fixed several recent regressions related to external transport tools (ncftp, lynx, curl, etc)
• Fixed quoting for downloading into directories containing whitespace
• Solaris tar gets more handholding to avoid solaris tar errors
• Portability fix: By-pass alarm() calls if we're running under perl 5.6.x && $OS is Windows.
• Work around win32 URI::file volume bug
• Prerequisites declared with the string "==" now supported

It's not completely silent, so new users of Perl and CPAN.pm aren't entirely mystified, but you can see that it's down to just under 40 lines of text if someone chooses to auto-configure everything.

$ cpan

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes] 

Autoconfigured everything but 'urllist'.

Now you need to choose your CPAN mirror sites.  You can let me
pick mirrors for you, you can select them from a list or you
can enter them by hand.

Would you like me to automatically choose some CPAN mirror
sites for you? (This means connecting to the Internet) [yes]
Trying to fetch a mirror list from the Internet
Fetching with HTTP::Tiny:
http://www.perl.org/CPAN/MIRRORED.BY

Looking for CPAN mirrors near you (please be patient)
................................. done!

New urllist
  http://mirrors.24-7-solutions.net/pub/CPAN/
  http://mirrors.ccs.neu.edu/CPAN/
  http://www.stathy.com/CPAN/

Autoconfiguration complete.

commit: wrote '/home/david/.cpan/CPAN/MyConfig.pm'

You can re-run configuration any time with 'o conf init' in the CPAN shell

cpan shell -- CPAN exploration and modules installation (v1.9464)
Enter 'h' for help.

cpan[1]>

This change should be part of the next stable release of CPAN and should also be in Perl 5.14, coming in April.

Barring any show-stopping bugs, the forthcoming Perl 5.13.9 will let a non-privileged user bootstrap access to CPAN, even if FTP is blocked and they have no command line HTTP clients installed. That means that if you can run perl, and you can use port 80, then you can use CPAN.

Imagine someday when Perl 5.14 becomes standard in linux distributions. Log-in to a new machine (or perhaps a shared host), type "cpan", and start installing stuff from CPAN. Easy!

To show you how it works, I installed a snapshot of Perl from the git repository and then switched to a clean, unprivileged user account to try installing something. All I typed at the keyboard was this:

/opt/perl/fromgit/bin/cpan5.13.8[ENTER]
[ENTER]
[ENTER]
[ENTER]
[ENTER]
install File::Marker[ENTER]

The log below is long and verbose (the way CPAN.pm just is, unfortunately) but you can see how it uses HTTP::Tiny, bootstraps local::lib, updates my .bashrc, installs File::Marker in the local library and even reminds me to restart my shell. Isn't that user-friendly?

$ /opt/perl/fromgit/bin/cpan5.13.8 

CPAN is the world-wide archive of perl resources. It consists of about
300 sites that all replicate the same contents around the globe. Many
countries have at least one CPAN site already. The resources found on
CPAN are easily accessible with the CPAN.pm module. If you want to use
CPAN.pm, lots of things have to be configured. Fortunately, most of
them can be determined automatically. If you prefer the automatic
configuration, answer 'yes' below.

If you prefer to enter a dialog instead, you can answer 'no' to this
question and I'll let you configure in small steps one thing after the
other. (Note: you can revisit this dialog anytime later by typing 'o
conf init' at the cpan prompt.)

Would you like me to configure as much as possible automatically? [yes] 

 <install_help>

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib] 

Autoconfigured everything but 'urllist'.

Now you need to choose your CPAN mirror sites.  You can let me
pick mirrors for you, you can select them from a list or you
can enter them by hand.

Would you like me to automatically choose some CPAN mirror
sites for you? (This means connecting to the Internet) [yes] 

Trying to fetch a mirror list from the Internet
Fetching with HTTP::Tiny:
http://www.perl.org/CPAN/MIRRORED.BY

Looking for CPAN mirrors near you (please be patient)
..............................................................
..............................................................
..............................................................
..................................... done!

New urllist
  http://cpan.erlbaum.net/
  http://cpan.mirror.clemson.edu/
  http://mirrors.servercentral.net/CPAN/

Autoconfiguration complete.

Attempting to bootstrap local::lib...

Writing /home/cleanroom/.cpan/CPAN/MyConfig.pm for bootstrap...
commit: wrote '/home/cleanroom/.cpan/CPAN/MyConfig.pm'
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/01mailrc.txt.gz
Going to read '/home/cleanroom/.cpan/sources/authors/01mailrc.txt.gz'
............................................................................DONE
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/modules/02packages.details.txt.gz
Going to read '/home/cleanroom/.cpan/sources/modules/02packages.details.txt.gz'
  Database was generated on Sat, 15 Jan 2011 20:57:19 GMT
  HTTP::Date not available
............................................................................DONE
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/modules/03modlist.data.gz
Going to read '/home/cleanroom/.cpan/sources/modules/03modlist.data.gz'
............................................................................DONE
Going to write /home/cleanroom/.cpan/Metadata
Running make for A/AP/APEIRON/local-lib-1.008001.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/A/AP/APEIRON/local-lib-1.008001.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/A/AP/APEIRON/CHECKSUMS
Checksum for /home/cleanroom/.cpan/sources/authors/id/A/AP/APEIRON/local-lib-1.008001.tar.gz ok

  CPAN.pm: Going to build A/AP/APEIRON/local-lib-1.008001.tar.gz

Attempting to create directory /home/cleanroom/perl5

*** Module::AutoInstall version 1.03
*** Checking for Perl dependencies...
*** Since we're running under CPAN, I'll just let it take care
    of the dependency's installation later.
[Core Features]
- ExtUtils::MakeMaker ...loaded. (6.57_05 >= 6.31)
- ExtUtils::Install   ...loaded. (1.55 >= 1.43)
- Module::Build       ...loaded. (0.3607 >= 0.36)
- CPAN                ...loaded. (1.9463 >= 1.82)
*** Module::AutoInstall configuration finished.
Checking if your kit is complete...
Looks good
Writing Makefile for local::lib
Writing MYMETA.yml
cp lib/POD2/PT_BR/local/lib.pod blib/lib/POD2/PT_BR/local/lib.pod
cp lib/lib/core/only.pm blib/lib/lib/core/only.pm
cp lib/local/lib.pm blib/lib/local/lib.pm
cp lib/POD2/DE/local/lib.pod blib/lib/POD2/DE/local/lib.pod
Manifying blib/man3/POD2::PT_BR::local::lib.3
Manifying blib/man3/lib::core::only.3
Manifying blib/man3/local::lib.3
Manifying blib/man3/POD2::DE::local::lib.3
  APEIRON/local-lib-1.008001.tar.gz
  /usr/bin/make -- OK
'YAML' not installed, will not store persistent state
Running make test
PERL_DL_NONLAZY=1 /opt/perl/fromgit/bin/perl5.13.8 -I/home/cleanroom/perl5/lib/perl5/x86_64-linux-thread-multi -I/home/cleanroom/perl5/lib/perl5 "-MExtUtils::Command::MM" "-e" "test_harness(0, 'inc', 'blib/lib', 'blib/arch')" t/classmethod.t t/coderefs_in_inc.t t/de-dup.t t/install.t t/lib-core-only.t t/pipeline.t
t/classmethod.t ...... Name "File::Spec::rel2abs" used only once: possible typo at t/classmethod.t line 20.
t/classmethod.t ...... 1/? Attempting to create directory t/var/splat
t/classmethod.t ...... ok
t/coderefs_in_inc.t .. ok
t/de-dup.t ........... ok
t/install.t .......... skipped: Install Capture::Tiny to test installation
t/lib-core-only.t .... ok
t/pipeline.t ......... ok
All tests successful.
Files=6, Tests=10,  1 wallclock secs ( 0.04 usr  0.01 sys +  0.19 cusr  0.01 csys =  0.25 CPU)
Result: PASS
  APEIRON/local-lib-1.008001.tar.gz
  /usr/bin/make test -- OK
Running make install
Manifying blib/man3/POD2::PT_BR::local::lib.3
Manifying blib/man3/lib::core::only.3
Manifying blib/man3/local::lib.3
Manifying blib/man3/POD2::DE::local::lib.3
Installing /home/cleanroom/perl5/lib/perl5/local/lib.pm
Installing /home/cleanroom/perl5/lib/perl5/POD2/PT_BR/local/lib.pod
Installing /home/cleanroom/perl5/lib/perl5/POD2/DE/local/lib.pod
Installing /home/cleanroom/perl5/lib/perl5/lib/core/only.pm
Installing /home/cleanroom/perl5/man/man3/POD2::PT_BR::local::lib.3
Installing /home/cleanroom/perl5/man/man3/lib::core::only.3
Installing /home/cleanroom/perl5/man/man3/local::lib.3
Installing /home/cleanroom/perl5/man/man3/POD2::DE::local::lib.3
Appending installation info to /home/cleanroom/perl5/lib/perl5/x86_64-linux-thread-multi/perllocal.pod
  APEIRON/local-lib-1.008001.tar.gz
  /usr/bin/make install  -- OK

local::lib is installed. You must now add the following environment variables
to your shell configuration files (or registry, if you are on Windows) and
then restart your command line shell and CPAN before installing modules:

export PERL_LOCAL_LIB_ROOT="/home/cleanroom/perl5"
export PERL_MB_OPT="--install_base /home/cleanroom/perl5"
export PERL_MM_OPT="INSTALL_BASE=/home/cleanroom/perl5"
export PERL5LIB="/home/cleanroom/perl5/lib/perl5/x86_64-linux-thread-multi:/home/cleanroom/perl5/lib/perl5:$PERL5LIB"
export PATH="/home/cleanroom/perl5/bin:$PATH"

Would you like me to append that to /home/cleanroom/.bashrc now? [yes] 

commit: wrote '/home/cleanroom/.cpan/CPAN/MyConfig.pm'
Terminal does not support AddHistory.

cpan shell -- CPAN exploration and modules installation (v1.9463)
Enter 'h' for help.

cpan[1]> install File::Marker
Running install for module 'File::Marker'
Running make for D/DA/DAGOLDEN/File-Marker-0.13.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/D/DA/DAGOLDEN/File-Marker-0.13.tar.gz
Fetching with HTTP::Tiny:
http://cpan.erlbaum.net/authors/id/D/DA/DAGOLDEN/CHECKSUMS
Checksum for /home/cleanroom/.cpan/sources/authors/id/D/DA/DAGOLDEN/File-Marker-0.13.tar.gz ok

  CPAN.pm: Going to build D/DA/DAGOLDEN/File-Marker-0.13.tar.gz

Creating new 'MYMETA.yml' with configuration results
Creating new 'Build' script for 'File-Marker' version '0.13'
Building File-Marker
  DAGOLDEN/File-Marker-0.13.tar.gz
  ./Build -- OK
Running Build test
t/01_File_Marker.t ... ok
t/02_fork.t .......... ok
t/03_thread.t ........ ok
t/04_save_load.t ..... ok
t/98_pod.t ........... skipped: Skipping author tests
t/99_pod_coverage.t .. skipped: Skipping author tests
All tests successful.
Files=6, Tests=58,  1 wallclock secs ( 0.04 usr  0.02 sys +  0.22 cusr  0.02 csys =  0.30 CPU)
Result: PASS
  DAGOLDEN/File-Marker-0.13.tar.gz
  ./Build test -- OK
Running Build install
Building File-Marker
Installing /home/cleanroom/perl5/lib/perl5/File/Marker.pm
Installing /home/cleanroom/perl5/man/man3/File::Marker.3
  DAGOLDEN/File-Marker-0.13.tar.gz
  ./Build install  -- OK

cpan[2]> q
Terminal does not support GetHistory.
Lockfile removed.

*** Remember to restart your shell before running cpan again ***

I want to thank Andreas Koenig (CPAN.pm) and Chris Nehren (local::lib) for quickly turning around new releases of their modules to make this work.

Here's a quick look at how you would use it for the two common tasks I mentioned:

    use HTTP::Tiny;
    my $http = HTTP::Tiny->new;
    my $response;

    # get a single page
    $response = $http->get('http://example.com/');
    die "Failed!\n" unless $response->{success};
    print $response->{content};

    # mirror a file
    $response = $http->mirror('http://example.com/file.tar.gz', 'file.tar.gz');
    die "Failed!\n" unless $response->{success};
    print "Unchanged!\n" if $reponse->{status} eq '304';

The example above is almost the same as you'd get using LWP::UserAgent with one big exception. HTTP::Tiny returns the response as a hash reference rather than as an object. Just like LWP::UserAgent, the mirror method will send an If-Modified-Since header for an existing file to skip downloading if the file is unchanged. HTTP::Tiny doesn't (yet) handle query parameters -- you have to prepare those yourself, but for simple downloads, you don't generally need those anyway.

Where did HTTP::Tiny come from? When I was working on getting CPAN.pm to support a pure-Perl HTTP bootstrap, I started with HTTP::Lite. When I discussed it on #p5p, Christian Hansen pointed out a number of serious shortcomings and decided that it would be easier for him to write a new, lightweight HTTP/1.1 client from scratch rather than try to redo the plumbing in HTTP::Lite.

The result is HTTP::Tiny and I've been collaborating with Christian to get it ready for use by CPAN.pm and ready for the Perl core. Unlike HTTP::Lite, HTTP::Tiny is a conditionally conforming HTTP/1.1 client. It supports both redirection and mirroring, which HTTP::Lite does not, both of which are important features for a CPAN client.

As a "Tiny" module, HTTP::Tiny achieves its HTTP/1.1 conformance in a just a fraction the code required for LWP::UserAgent and its non-core dependents. Don't get me wrong -- LWP::UserAgent is a great piece of software. But sometimes -- like for the Perl core or for a fatpacked application -- something much smaller and simpler will do just as well.

Let's see what a difference there is. CPANdeps shows us all of LWP::UserAgent's dependencies and highlights the ones that are non-core (as of Perl 5.10.1):

• URI
• HTML::Parser
• HTML::Tagset

I used David A. Wheeler's SLOCCOUNT to count lines of code in each of the distributions containing these modules as well as for HTTP::Lite and HTTP::Tiny (based on the "soon-to-be" 0.007 release). In all cases, I excluded files in t/ and examples directories. (Lines of Perl includes any programs distributed with the distribution.)

Here are the results:

Distribution        .pm files Lines (Perl) Lines (C)  Total Lines
-----------------   --------- ------------ ----------  -----------
libwww-perl-5.837       52        10258          0       10258
HTML-Parser-3.68         7          883       1972        2855
HTML-Tagset-3.20         1          139          0         139
URI-1.56                52         2715          0        2715
                     -----        -----      -----       -----
TOTAL LWP & friends    112        13995       1972       15967

versus

Distribution        .pm files Lines (Perl) Lines (C)  Total Lines
-----------------   --------- ------------ ----------  -----------
HTTP-Lite-2.3            1          634          0         634
HTTP-Tiny-0.007          1          603          0         603

Wow! Both HTTP::Lite and HTTP::Tiny accomplish simple HTTP tasks with less than 4% of the SLOC of LWP and its non-core dependencies.

What about memory usage? Here's the "null" case that shows memory usage of just loading the three client modules:

  VSZ   RSS COMMAND
30440  5512 perl -MLWP::UserAgent -e 1 while 1
26700  3960 perl -MHTTP::Tiny -e 1 while 1
26040  3168 perl -MHTTP::Lite -e 1 while 1

HTTP::Lite is smallest, but it also doesn't have the features or conformance I need.

That's not much of a real-world test, so let's try something else -- downloading the 950K CPAN 02packages.details.txt.gz file. Here is the test code for HTTP::Tiny and LWP::UserAgent side by side:

# http-tiny-mirror.pl                  # lwp-useragent-mirror.pl
#!/usr/bin/env perl                    #!/usr/bin/env perl
use strict;                            use strict;
use warnings;                          use warnings;
use HTTP::Tiny;                        use LWP::UserAgent;                                 

my ($url, $file) = @ARGV;              my ($url, $file) = @ARGV;
die unless $url & $file;               die unless $url & $file;                            

my $http = HTTP::Tiny->new;            my $http = LWP::UserAgent->new;
my $res = $http->mirror($url, $file);  my $res = $http->mirror($url, $file);
die "Failed!\n"                        die "Failed!\n"
  unless $res->{success};                unless $res->is_success;                          

1 while 1;                             1 while 1;

What about HTTP::Lite? It doesn't have a mirror method and doesn't do redirection, which I get for free with HTTP::Tiny and LWP::UserAgent. I'd have to write dozens of lines of code to emulate that for a "fair" test, so I skipped it.

Let's add those two programs (after downloading the 02packages file to files with different names) to our memory benchmarks:

  VSZ   RSS COMMAND
50916  9824 perl ./lwp-useragent-mirror.pl [...]
35376  4348 perl ./http-tiny-mirror.pl [...]
30440  5512 perl -MLWP::UserAgent -e 1 while 1
26700  3960 perl -MHTTP::Tiny -e 1 while 1
26040  3168 perl -MHTTP::Lite -e 1 while 1

HTTP::Tiny wins -- not by a huge amount in absolute terms, admittedly, but if you don't need the extra features that LWP::UserAgent offers, HTTP::Tiny might just be all you need.

Thank you, Christian, for HTTP::Tiny!

The background for this particular situation involves the role that YAML plays in the Perl 5 module toolchain. For many years, now, CPAN distributions have included META.yml files containing distribution metadata. For CPAN clients to read this metadata, the Parse::CPAN::Meta module (P::C::M) was added to the Perl 5 core in Perl 5.10.1. P::C::M is (unfortunately) a read-only copy of YAML::Tiny.

The situation became more complex after the Oslo Consensus, which agreed on using the META.yml format as a standard way for module toolchain components to communicate post-configuration dependencies. This means module builders like Module::Build and ExtUtils::MakeMaker need to be able to write metadata files, not just read them. Thus, Module::Build::YAML was added, which is a full-fledged copy of YAML::Tiny that allows Module::Build to bootstrap if YAML::Tiny is not installed. Likewise, ExtUtils::MakeMaker needs ExtUtils::MakeMaker::YAML for its own bootstrap.

Thus, we seemed destined for three copies of YAML::TIny of various degrees of fidelity to be tucked away in the Perl 5 core under alternate names.

This situation is, frankly, nuts. Fortunately, the Perl 5 pumpking agrees.

Thus, it seemed that YAML::Tiny was destined for the Perl 5 core, instead. But a wrinkle emerged. YAML::Tiny is not really YAML, but only a subset. It is sufficient for META.yml files, but whatever goes is core to support that should not be considered a 'preferred' YAML library. A compromise was struck, to call it CPAN::Meta::YAML instead.

Now, I needed to create CPAN::Meta::YAML and make it easy to track YAML::Tiny.

The first Dist::Zilla plugin I wrote was Doppelgaenger, which downloads the latest stable release of a module and uses it as the basis for a new, renamed distribution. It also strips the Pod and version number from the source. The second plugin is AppendExternalData, which I use to add new Pod that is specific to CPAN::Meta::YAML.

The final CPAN::Meta::YAML repository contains only four files:

Changes
dist.ini
pod/lib/CPAN/Meta/YAML.pm
MANIFEST.SKIP

And here is the dist.ini file:

name = CPAN-Meta-YAML
author = Adam Kennedy <adamk@cpan.org>
author = David Golden <dagolden@cpan.org>
license = Perl_5
copyright_holder = Adam Kennedy
copyright_year = 2010

[Doppelgaenger]
source_module = YAML::Tiny
strip_pod = 1
strip_version = 1

[AppendExternalData]
source_dir = pod

[MetaResources]
;repository.url = git://github.com/dagolden/cpan-meta-yaml.git
;repository.web = http://github.com/dagolden/cpan-meta-yaml/
;bugtracker.web = http://rt.cpan.org/NoAuth/Bugs.html?Dist=CPAN-Meta-YAML

[@Filter]
-bundle = @DAGOLDEN
-remove = Prepender
-remove = PodCoverageTests
-remove = PortabilityTests
git_remote = github

[RemovePrereqs]
remove = YAML
remove = YAML::Perl
remove = YAML::Syck
remove = YAML::XS
remove = t::lib::Test

With that, all I need to do is run dzil release and -- tada! -- a new CPAN::Meta::YAML is created, tested and released to CPAN. And whenever a new YAML::Tiny is released, I just run dzil release again. It's that easy.

Hooray for Dist::Zilla!

Once this is done and released as part of CPAN and core Perl (hopefully for 5.14 if I can get it done quickly), it will be possible to have a working CPAN setup for an ordinary user with a system Perl with about this much effort:

$ cpan
[... welcome blah blah ... do you want automatic configuration? ]
y
[... do you want local::lib? ]
y
[... auto config ... ]
[... auto mirror detection ... ]
[... bootstrapping local::lib ... ]
[... fixing up your shell config with PERL_* variables ... ]

cpan[1]> install Whatever::Module::You::Want

And done! A non-privileged user could have a working CPAN and local::lib with "cpan\n", "y\n", "y\n". Spiffy!

Keep your fingers crossed and I'll report back when I have it done.

Since I first released a distribution to CPAN in the early 2000's, I've gotten familiar with the gory details of most of the toolchain. I maintain Module::Build, I've contributed to Dist::Zilla, and I know the guts of ExtUtils::MakeMaker (pretty well) and Module::Install (at least a little). Based on that experience, I think I can offer a (hopefully) objective view of the tradeoffs in using these different tools.

• ExtUtils::MakeMaker (aka EU::MM) -- this is the "standard" but it is annoying to work with and tricky to customize (unless you are a Makefile expert). Prolific CPAN authors seem to prefer Module::Install or Dist::Zilla to avoid dealing with EU::MM directly.
• Module::Build (aka M::B) -- this is the 'pure perl' alternative. It is very easy to customize compared to EU::MM but you can't always rely on users having a new enough M::B for any given feature unless you target Perl 5.10.1 (which added the ability for CPAN/CPANPLUS to bootstrap tools you need *before* running Build.PL)
• Module::Install (aka M::I) -- this gives a nicer API than EU::MM, but it uses EU::MM behind the scenes to do the work. It is different in that it bundles itself with the distribution, avoiding bootstrap problems. This is viewed as a benefit by some and as a burden or design defect by others. If existing M::I plugins don't meet your needs, it is a bit harder to understand how to customize M::I compared to M::B.
• Dist::Zilla (aka D::Z) -- this is a complete distribution management tool, not just a builder. It tries to let authors write as little repetitive scaffolding and documentation as possible and then generates a "release ready" distribution from templates and other configuration. Think of it as a "distribution compiler". In the generation process, it will create your Makefile.PL (EU::MM) or your Build.PL (M::B) for you. (It's your choice which one you prefer.)

The original question was about "creating CPAN packages". All four tools will let you "create" a distribution tarball that you can upload to CPAN. For the first three, you have to create a Makefile.PL or Build.PL and then use the tool you select to create the tarball. With D::Z, you create a "dist.ini" file and D::Z creates the tarball itself (generating the Makefile.PL or Build.PL) without relying on EU::MM or M::B to do the work.

D::Z also can create a new "boilerplate" distribution, similar to Module::Starter or h2xs (or any of various other distribution boilerplate tools on CPAN). The other three do not.

I have switched to Dist::Zilla and many other prolific CPAN authors have as well. (It was written by RJBS, the most prolific CPAN author of all, so he has some perspective on what is the easiest thing for managing CPAN distributions). If you are interested in learning Dist::Zilla, the online tutorial is the way to go.

If you don't want a complete solution, the question is how much customization do you need (to the test process, distribution packaging process, etc.)? If the answer is "a lot", then I would recommend M::B. If the answer is "very little", then I would recommend M::I. Avoid direct use of EU::MM entirely.

I've just finished some work to add HTTP::Lite support to CPAN.pm and Andreas has merged it into the master branch. The next step will be to test it out in a development release. If all goes well, the plan is to merge the latest CPAN.pm into the Perl core and bring in HTTP::Lite as well. Then, finally, a mere 20 years after the invention of HTTP, Perl 5.14 will be able to speak HTTP out of the box and bootstrap access to all of CPAN.

Keep your fingers crossed.

Here's how you can install the development release from the command line with cpan, cpanm or cpanp:

$ cpan DAGOLDEN/Module-Build-0.36_17.tar.gz
$ cpanm DAGOLDEN/Module-Build-0.36_17.tar.gz
$ cpanp -i DAGOLDEN/Module-Build-0.36_17.tar.gz

Thank you very much!

A summary of changes since 0.3607 follows:

0.36_17 - Wed Oct 27 18:08:36 EDT 2010

  Enhancements:

  - Added 'distinstall' action to run 'Build install' inside the
    generated distribution directory [Jeff Thalhammer]

0.36_16 - Thu Aug 26 12:44:07 EDT 2010

  Bug fixes:

  - Better error message in case package declaration is not found
    when searching for version. [Alexandr Ciornii]

  - Skips 'release_status' tests on perl < 5.8.1 due to buggy
    treatment of dotted-decimal version numbers [David Golden]

0.36_15 - Wed Aug 25 10:41:28 EDT 2010

  Bug fixes:

  - Added a mock Software::License to prevent t/properties/license.t
    from failing.

0.36_14 - Sun Aug 22 22:56:50 EDT 2010

  Enhancements:

  - Adds 'release_status' and 'dist_suffix' properties in preparation
    for adding CPAN Meta Spec 2 support.  'dist_suffix' will be set
    to 'TRIAL' automatically when necessary. [David Golden]

  - Makes 'license' more liberal.  You can now specify either a license
    key from the approved list (c.f. Module::Build::API) or just a
    Software::License subclass name (e.g. 'Perl_5').  This should
    provide better support for custom or proprietary licenses.
    [David Golden]

0.36_13 - Wed Jul 28 22:40:25 EDT 2010

 Bug-fixes:

 - Bundled Module::Build::Version updated to bring into sync with CPAN
   version.pm 0.82 [David Golden]

0.36_12 - Tue Jul 27 00:08:51 EDT 2010

  Enhancements:

  - Module::Build::Compat will now convert dotted-decimal prereqs into
    decimal rather than dying (and will warn about this). [Apocalypse]

  Bug fixes:

  - Caches case-sensitivity checks to boost performance, fixes
    RT#55162 and RT#56513 [Reini Urban]

  - Won't try to use ActivePerl doc generation tools without confirming
    that they are indeed installed. [David Golden]

  - Sets temporary $ENV{HOME} in testing to an absolute path, which fixes
    some issues when tested as part of the Perl core [Nicholas Clark]

  - Module::Build::ModuleInfo now warns instead of dying when a module
    has an invalid version.  ->version now just returns undef
    (RT#59593) [David Golden]

  Changes:

  - When authors do not specify Module::Build in configure_requires and
    Module::Build is automatically added, a warning will be issued
    showing the added prerequisite [David Golden]

  - Moved automatic configure_requires generation into get_metadata()
    and added an 'auto' argument to toggle it (on for META and off
    for MYMETA) [David Golden]

0.36_11 - Thu May 27 09:41:23 EDT 2010

  Bug fixes:

  - Handle META/MYMETA reading and writing within Module::Build to ensure
    utf8 mode on filehandles.  Now passes/gets only strings to YAML::Tiny
    or Module::Build::YAML

0.36_10 - Wed May 19 18:36:06 EDT 2010

  Bug fixes:

  - Fix failing t/manifypods.t on Windows from 0.36_09 changes [Klaus
    Eichner]

0.36_09 - Tue May 11 09:19:12 EDT 2010

  Bug fixes:

  - Improve HTML documentation generation on ActivePerl (RT#53478)
    [Scott Renner and Klaus Eichner]

0.36_08 - Mon Apr 26 08:00:15 EDT 2010

 Enhancements:

 - Give a list of valid licenses when given one we don't recognize
   (RT#55951) [Yanick Champoux]

 - Added 'Build manifest_skip' action to generate a default MANIFEST.SKIP
   [David Golden]

 Changes:

 - When temporarily generating a MANIFEST.SKIP when none exists, it will
   be removed on exit instead of hanging around until 'Build clean'.  This
   is less surprising/confusing and the 'Build manifest_skip' action
   is now available instead to bootstrap the file [David Golden]

 Bug fixes:

 - Fixed runtime error on cygwin when searching for an executable command
   during installdeps testing [David Golden]