Optimizing Modern Perl

No, I'm not talking about code (but it made you look, didn't it!) — I'm talking about optimizing the search for "modern perl", because right now it sucks.

Give it a try:

Sure, chromatic's blog and excellent Modern Perl book show up a lot. The Modern::Perl module is there. Depending on the search engine, some Modern Perl SlideShare presentations show up, many of which are pretty technical.

On the other hand, Stevan Little's Modern Perl presentation is a pretty good history, but is hard to find. Nor is it concise (though it wasn't meant to be). And I've entirely run out of patience to track down other good examples.

There is no obvious web page that introduces "Modern Perl" style to someone. If there's no page like that, we can't link to it, which means we can't optimize search for it, either.

So here is my challenge to the Perl community:

Find the best material you can to explain what "Modern Perl" means. Comment about it below. Blog about it. Twitter about it. Link to it. If you can't find something you like, try writing a presentation on it yourself. If some person or organization has the means and motivation, draft a page or a white paper on Modern Perl for perl.org or go buy modernperl.com and put it up.

For whatever reason, the phrase "Modern Perl" has caught on and it's a good catch-phrase to differentiate the latest best style, patterns and libraries for Perl. But a catch phrase is only helpful if there's depth behind it. chromatic's book — great as it is — shouldn't be the only reference.

Posted in perl programming | Tagged , | 3 Comments

Is Javascript the new Perl?

I saw a Javascript/JQuery presentation, The JQuery Divide, by Rebecca Murphey on Hacker News that struck me as remarkably similar to complaints I've heard about Perl development.

Look at this graphic from p. 34 -- rewind the clock to the late 1990's and replace Javascript with Perl and JQuery with CGI:

Here's are some telling quotes:

  • people who don't know JS write non-trivial JS apps whether we like it or not (p. 11)
  • [ignorant developers writing bad JS] has actual effects on perceptions of Javascript and on our ability to do interesting things with it. (p.32)

And consider this humorous fake quote from p.4:

"We need you to help us organize our JQuery-based application. It's a steaming pile of unmaintainable crap..."

Sound familiar?

By now, the active core of the Perl community seems to have figured out the need to promote better practices and the Modern Perl movement actively seeks to both to teach how to write good code and change perceptions of Perl that still linger from the dot.com 90's.

I hope that the Javascript community can learn similar lessons without taking a whole decade to get around to it.

Posted in perl programming | Tagged , | 5 Comments

How not to recommend a Perl module

The first comments in response to my Anatomy of ylastic-costagent article were recommendations to consider other modules than the ones I used. Frankly, these module recommendations sucked, and I'm annoyed enough to explain why.

If you recommend a module, make sure it solves my problem

The first recommendation was to consider Date::Simple instead of Time::Piece + Time::Piece::Month, but Date::Simple doesn't appear to solve my problem.

I was looking for a way to find the first day of the next month (without manually incrementing the month and potentially the year). Here's how I did it:

# returns a Time::Piece object
Time::Piece::Month->new( Time::Piece->new() )->next_month->start()

Note that Time::Piece::Month has a next_month() method. Does Date::Simple offer a way to do that? Not that I can see in the documentation, so the recommendation is useless to me.

If you recommend a module, make sure I can see how to use it

The second recommendation was to use Log::Any::App instead of Log::Dispatchouli. Leaving aside that I said the reason I was using Log::Dispatchouli was because I was explicitly trying to learn it, the recommendation suggested that I could have "1 instead of 6" lines of code.

Here's my "six" (actually about 8) lines:

local $ENV{DISPATCHOULI_PATH} = $opts->get_logpath
  if $opts->get_logpath;

my $logger = Log::Dispatchouli->new({
  ident     => basename($0),
  facility  => $opts->get_syslog ? $opts->get_syslog : undef,
  to_file   => $opts->get_logpath ? 1 : undef,
  log_pid   => 0,
  debug     => $opts->get_debug,
});

Note that I'm setting a name, optionally logging to syslogd, optionally logging to a file of a user-defined path (and potentially doing both), and toggling a debug mode.

Looking at Log::Any::App's usage synopsis, here's what I see:

use Log::Any::App qw($log);

Yes, that's one line. But does it provide the same options? Obviously not in just that line. If I read to the end of the documentation, I find there is an init() function and I can do something like this (N.B. untested):

require Log::Any::App;
Log::Any::App::init([
  '$log',
  ($opts=>get_logpath ? (-file => $opts->get_logpath) : () ),
  ($opts=>get_syslog ? (-syslog => { facility => $opts->get_syslog) }) : () ),
  ($opts=>get_debug ? (-level => 'debug') : () ),
]);

I don't know if that really does exactly the same thing -- it looks like it would get close, but it's certainly not "1 line versus 6".

In evaluating the two modules, I compare whether documentation makes it easy to figure out how to use each module. Here are the two synopses together for comparison:

# Log::Any::App
use Log::Any::App qw($log);

# Log::Dispatchouli
my $logger = Log::Dispatchouli->new({
    ident     => 'stuff-purger',
    facility  => 'daemon',
    to_stdout => $opt->{print},
    debug     => $opt->{verbose}
})

One of those shows me how to configure it to suit my needs. The other doesn't. Log::Dispatchouli also documents its new() method and all arguments within half a page after the synopsis. Log::Any::App buries its init() documentation towards the end of the documentation (and also doesn't mention that it will take a variable name to export to, just like on a "use" line).

If the recommendation had said, "if you want a logger that gives you decent defaults without configuration", I might have found the recommendation more useful, but that is not what it said. Unless it's obvious from the docs, a recommendation should tell me how to do the same thing easier/faster/better than what I've already done. If it's not obvious, write an example yourself, or don't bother with the recommendation.

Log::Any::App actually seems decent, but if I wasn't annoyed enough to write this article, I probably wouldn't have bothered to read past its terrible synopsis.

Posted in perl programming | Tagged , | 9 Comments

Perl whipuptitude: Anatomy of ylastic-costagent

Many people love Perl for how easily and quickly one can write useful little programs, but many of these one-off programs are never seen by anyone but the original author. Recently, I wrote ylastic-costagent to help process Amazon Web Services (AWS) data for analysis. Rather than leave it hidden on my hard drive, I decided to share my program on CPAN and write about some of the techniques and modules I used.

Why I wrote it

I've been using the Ylastic dashboard to help monitor AWS usage and costs. Ylastic provides some very nice features for managing AWS cloud services and some great visualizations. Unfortunately, their only option for gathering the data was a poorly documented, poorly structured Python program that scrapes Amazon's site data.

When I tried it, I had problems satisfying the dependencies. I had to install dependencies manually and even then it turns out that Ubuntu 10.10 has libraries that are too old to work. I realized that in the time it would take me (a Python non-user) to figure out how to get newer Python libraries installed, I could just whip up an equivalent program in Perl.

Two parts: a module and an executable

I followed a CPAN trend of putting most of the guts of the program into a module, App::Ylastic::CostAgent, and wrote a simpler executable that just managed the command line options, created an object, and dispatched to a run method. I like this approach because modules are a bit easier to test than executables and author tools for managing modules are a bit better developed.

In this case, I did the command line option processing in the executable with my own Getopt::Lucid. I've also seen option processing put into the module as well. I haven't decided which I like better, though I slightly favor seeing options code in the executable because I think it should be documented there as well and I like keeping code and documentation close.

Five modules inside App::Ylastic::CostAgent

App::Ylastic::CostAgent contains less than 200 lines of code validating inputs and coordinating the interactions of five modules that do all the real work:

  • Config::Tiny -- the original Python program required users to edit global variables to configure it. (Horrible!) Whenever I need a simple configuration file, I start with Config::Tiny. It even supports both top-level data and named sections, which was a good semantic fit for global Ylastic user data and named sections for each AWS account from which to collect data.
  • Object::Tiny -- I didn't expect this program to manage a lot of state, but I didn't want to pass around a raw data structure either. Wrapping data as an object and calling accessors helps me avoid annoying typo errors accessing hash-keys directly. Since I didn't need a full-featured OO system like Moose, I opted for Object::Tiny instead to give me a simple constructor and some accessors.
  • WWW::Mechanize -- this is the workhorse module that does the actual screen scraping and form posting. I spent more time fiddling with it that I would have liked, but I haven't used it in years and I had to relearn it.
  • Archive::Zip -- Ylastic wants a zip file containing all the CSV files downloaded from AWS. Instead of saving them to disk and then running an external tool, I chose to use Archive::Zip to build the zip file in memory. I also wasted a bit of time here discovering that Archive::Zip wants you to set compression levels explicitly instead of doing it by default. (I should have RTFM more closely.) I don't really like the API and I wish someone would write an "Archive::Zip::DWIM" that makes it easier to use for a simple case like mine.
  • Log::Dispatchouli -- the original Python program did some debug logging to a file. I didn't really need a powerful logging library, but I was looking for an excuse to learn Ricardo Signes' Log::Dispatchouli. With about half a dozen lines of code, I gave ylastic-costagent the ability to log either to a file or syslogd or both. Cool!

What is the first day of the next month?

The original Python program requested custom data ranges from AWS that ended on the first day of the next month. I'm not sure why they didn't just set an end date one day in the future, but I wanted to replicate their logic in ylastic-costagent.

I could have copied the algorithm from the Python program (grab date components, increment month with a modulus, etc.) but I expected an answer on CPAN already. As sometimes happens when searching CPAN, I blundered around for longer than it would have taken to just copy the logic, but I came up with this gem using Time::Piece and Time::Piece::Month:

# returns a Time::Piece object
Time::Piece::Month->new( Time::Piece->new() )->next_month->start()

I'm sure there are better ways to do it, but I didn't really want to spend any more time on it.

Bootstrapping installation

Since dependencies were what stopped me from using the original Python program, I decided to release ylastic-costagent to CPAN so the regular Perl module toolchain could automate dependency resolution for anyone wanting to use it. That wouldn't help a Perl novice, so I included a five-line recipe in the documentation for anyone who didn't know how to configure CPAN. (Including how how to install the OpenSSL development library -- for a Debian-based system in this example.)

$ sudo apt-get install libssl-dev
$ curl -L http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
$ eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`
$ echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.bashrc
$ cpanm App::Ylastic::CostAgent

Conclusion

In the end, this took me less than a day's work, spread out over a couple actual days. Could I have sorted out my Python dependencies in that time? Possibly. But I did spent some extra time I didn't really need to being fancy with logging and the date manipulation as well as writing decent documentation. In the end, I created an easily-installable program with automatic dependency resolution -- which is exactly what I didn't have before.

It's now working for me, and I hope it might save some time for future AWS/Ylastic customers as well.

Posted in perl programming | Tagged , , | 4 Comments

A short Perl break

My usual weekly Perl post has been preempted by the arrival of my second child on March 22.

Naomi

I'll be back next week (I hope) with a discussion of the modules I used to whip up ylastic-costagent, a program to download Amazon Web Services usage data and upload them to Ylastic for analysis and charting.

Posted in perl programming | Tagged , | 5 Comments

© 2009-2012 David Golden All Rights Reserved