MongoDB Perl driver v1.0.0 RC 2 available – please test!

Yesterday, I released the MongoDB Perl Driver v1.0.0 RC2 to CPAN as MongoDB-v0.999.999.5-TRIAL. As I mentioned in my RC1 announcement last week that summarized the rationale and changes, everyone who uses MongoDB is strongly encouraged to download it and test it with their code.

In particular, please read the MongoDB::Upgrading document, which describes the major changes from the "v0" series to v1.0.0. Here is the table of contents so you have a sense of what you'll get from it:

    • Moo instead of Moose
    • SSL and SASL
    • MongoClient configuration
    • Lazy connections and reconnections on demand
    • Exceptions are the preferred error handling approach
    • Cursors and query responses
    • Aggregation API
    • Read preference objects and the read_preference method
    • Write concern objects and removing the safe argument
    • Authentication based only on configuration options
    • Bulk API
    • GridFS
    • Low-level functions removed
    • MongoDB::Connection removed
    • BSON encoding changes
    • DBRef objects

Testing the release candidate

You can install RC2 from CPAN like this:

$ cpan MONGODB/MongoDB-v0.999.999.5-TRIAL.tar.gz

If you encounter problems or have questions, please open tickets on the MongoDB Perl driver bug tracker.

Barring any show-stoppers or urgent changes, the stable v1.0.0 MongoDB Perl driver will be shipped to CPAN around the end of August.

Posted in mongodb, perl programming | Tagged , , , | Comments closed

Review a git branch in the terminal

When I review a pull-request with many commits, I want to be able to see all the commits and step back and forth through them. I used to use a git GUI (or github's web pages), but I hated having to leave the terminal to do that.

Today I whipped up a Perl program to stage all the commit information into tempfiles and fire up vim on them. I saved it in my bin path as 'git-review', so I can call it as "git review". No promises of quality, but I'm sharing it in case anyone wants to steal/adapt it.

#!/usr/bin/env perl
use v5.10;
use strict;
use warnings;
use Path::Tiny;
use Capture::Tiny qw/capture_stdout/;

my ( $branch, $trunk ) = @ARGV;

die "Usage: $0 <branch> [trunk]"
  unless $branch;

$trunk //= 'master';

my $fork_point = qx{git merge-base --fork-point $trunk $branch};
chomp $fork_point;

die "Couldn't locate fork point for $branch from $trunk"
  unless $fork_point;

my @commits = map { chomp; $_ } qx/git rev-list $fork_point..$branch/;

my $tempdir = Path::Tiny->tempdir;

my @files;
for my $c (@commits) {
    push @files, my $file = $tempdir->child($c);
    $file->spew( scalar capture_stdout { system( qw/git show -p --stat/, $c ) } );

my $index = $tempdir->child("log");
$index->spew( scalar capture_stdout { system( qw/git log/, q[--pretty=%h %s%d], "$fork_point..$branch" ) } );
$index->append( "\ngit diff --stat\n", scalar capture_stdout { system( 'git', 'diff', '--stat', "$fork_point..$branch" ) } );

system( "vim", $index, reverse @files );

Now, since I use ingy's git-hub command line tool, my pull-request review is as easy as this:

$ git hub pr-fetch 27
$ git review PR/27
Posted in git, perl programming | Tagged , , , , | Comments closed

Announcing the MongoDB Perl driver v1.0.0 Release Candidate

I'm very happy to announce the release of the MongoDB Perl driver v1.0.0 Release Candidate 1 to CPAN!

This next-generation driver is a substantial rewrite of the original and has been my primary project as lead engineer for the Perl driver since last summer.

Rationale for a rewrite

Over the last year, MongoDB engineers rewrote most of the language drivers maintained in-house. As mentioned in the next-generation drivers announcement, we've built the next-generation drivers on a common set of specification documents. Whereas previous drivers were written idiosyncratically, each driver now aims to deliver similar underlying behaviors through similar APIs, while still striving to be idiomatic for its language.

For the Perl driver, however, we had several goals that went above and beyond cross-driver consistency. In particular, we wanted to address some fundamental deficiencies in the "v0.x" series of drivers:

  • intra-driver consistency – many parts of the v0 API were inconsistent, behaving differently from method to method; the v1 API minimizes developer surprises by improving consistency in return types and exception mechanisms.
  • encapsulation – too many low-level, internal operations were exposed as part of the API, which complicates maintenance work; the v1 API aims to minimize the "public surface" available to developers, allowing faster future development keeping up with MongoDB server enhancements with less risk of breakage.
  • abstraction – many v0 methods returned raw server documents for end-user code to inspect, which is brittle in the face of changes in server responses over time; the v1 API uses result classes to abstract the details behind standardized accessors.
  • server compatibility – some new features and behavior changes in the MongoDB server no longer fit the old driver design; the v1 driver transparently supports both old and new servers.
  • portability – the v0 driver had a large dependency tree and substantial non-portable C code; the v1 driver removes some dependencies and uses widely-used, well-tested CPAN modules in place of custom C code where possible; it lays the groundwork for a future "pure-Perl optional" driver.
  • round-trippable data – the v0 BSON implementation could easily change data types when round-tripping documents; the v1 driver is designed to round-trip data correctly whenever possible (within the limits of Perl's dynamic typing).

Summary of changes

As you might imagine, there are an enormous number of changes, listed in detail in the MongoDB::Upgrading document.

Here is a summary of some of the more substantial changes:

  • Configuration – MongoDB::MongoClient gains a number of new configuration options to control server selection and timeouts; configurations are now always read-only; some existing options are deprecated; others are removed entirely where they no longer fit the new paradigms of the client
  • Lazy connection – creating a MongoDB::MongoClient object no longer connects to a server right away. This is the new standard for all official MongoDB drivers, but might break code that expected an immediate error from new!
  • Failover – when a networking error occurs or a server goes away, an error is throw. If handled, the next attempt at communicating with the server will automatically reconnect. For a replica set, this means it will failover to a new primary when a new primary is ready.
  • Exceptions – All errors now throw exceptions rather than relying on inconsistent ways of returning errors from methods
  • Read preferences and write concerns – these are now expressed as objects; they can be set at the client, database or collection object level
  • Authentication – this is now based only on configuration options and occurs immediately when any server connection is made. This is another change for all official drivers that harmonizes the behavior of MongoDB's different authentication mechanisms, some of which could only happen at connection time and some of which could be done later.
  • Deprecations – pretty much the entire MongoDB::Collection API was deprecated and replaced with the new official-driver-wide CRUD API. The deprecated methods are now undocumented, but still mostly work as they used to. A handful of methods in other classes were deprecated as well.

BSON encoding changes

BSON encoding was substantially overhauled. The various $MongoDB::BSON::... global variables have been removed, as BSON encoding is now encapsulated and can be set per-client, per-database or per-collection.

Integers will now be encoded to the smallest BSON integer type that fits rather than always taking up a fixed size (equal to the compiled integer size of the interpreter). This means that storing zero no longer takes up 64 bits on a 64-bit perl.

All common JSON boolean classes will now encode correctly. Time::Moment is supported for both encoding and decoding datetimes.

Testing the release candidate

If you use the MongoDB Perl driver, I strongly encourage you to read MongoDB::Upgrading and test your code with the release candidate driver.

You can install it from CPAN like this:

$ cpan MONGODB/MongoDB-v0.999.999.4-TRIAL.tar.gz

If you encounter problem or have questions, please open tickets on the MongoDB Perl driver bug tracker.

Barring any show-stoppers or urgent changes, the stable v1.0.0 MongoDB Perl driver will be shipped to CPAN in mid-to-late August.

Posted in mongodb, perl programming | Tagged , , | Comments closed

Visualizing Perl 5 Release History – 2015 edition

I've update my Perl 5 release history chart through the release of Perl 5.22.0. As I've pointed out before, the steady march of annual releases sets clear expectations for future development.

Perl Release History 2015

Perl Release History 2015

Looking at this chart, I'm always struck by how the Perl 5.10 series differs from the rest. Perls 5.4, 5.5, 5.6 and 5.8 arrived and delivered new features (and breaking changes) not far off the current annual release cycle.

Perhaps the people who think Perl is changing too fast were conditioned by the 5½ year gulf from 5.8.0 to 5.10.0 or the 2½ year gap from 5.10.0 to 5.12.0. Together, that's 8 years of infrequent change, particularly for those who skipped 5.10 and stuck with 5.8.

I recently wrote about why you shouldn't waste time on Perl 5.6. The Perl toolchain targets Perl 5.8.1 and I suspect a large portion of CPAN still targets Perl 5.8, released 13 years ago. Is it any wonder that we wind up with presentations like Stevan's "Perl is not Dead, It's a Dead End"?

I have a different hypothesis.

Perl isn't dead, it's just living in the past.

At some point, the community needs to shed its attachment to Perl 5.8.

Or maybe we'll all just start using Perl 6, instead.

Posted in perl programming | Tagged , , , | Comments closed

Why you shouldn't waste your time on Perl 5.6

This is a sort-of response to mst's On Perl 5.6 post. Mostly, I agree with Matt about not gratuitously breaking 5.6 compatibility (i.e. to "force people to upgrade").

However, in recent years, I've never received a single bug report or patch from a person actually using 5.6 for anything except smoke testing things on 5.6.

The argument seems to be: "because I think it's important to test things on 5.6, you should make it possible for me to do so". This seems circular.

Testing on 5.6 is pointless if no one is using 5.6 for anything except testing 5.6.

There is an argument sometimes made that if there are users on 5.6, then they should just upgrade. That seems rather obviously wrong, since if there really are such people, there are probably good reasons why they can't. But at the same time, it seems entirely inconsistent for someone to insist that they can't possibly upgrade their perl to something released in the last decade, yet they want the latest and greatest from CPAN.

If there really are people using 5.6 for real, do they really expect CPAN modules to just work? Are they really put out if they don't? Or have they learned to work around it, just like they've worked around the myriad of feature gaps and bug fixes in the last dozen years?

As far as accepting patches for 5.6 support goes, I have very mixed feelings about it. Reviewing patches takes time. Patches accepted might then implicitly indicate support for 5.6 going forward. And if there are bugs, the original patch author might not be around.

So a simple patch that restores 5.6 compatibility might be fine. E.g. "hey, I removed your 'use 5.008' and tested it on 5.6.2 and it passed tests just fine!". But a complex, possibly fragile workaround burdens the receiver if accepted.

If a real person actually using 5.6 asked me to do that, I might accept, depending on how ugly the patch is. But I'm far, far less inclined to do so when sent a gnarly patch by some self-appointed compatibility police telling me 5.6 support is still important "just because".

While it's not for me to tell people what to do with their volunteer time, I suspect that whatever time people are spending writing 5.6 patches and doing 5.6 smoke testing could be far more valuably spent on other things that affect orders of magnitude more real users.

Posted in perl programming | Tagged , | Comments closed