How to mass-favorite modules on MetaCPAN

I love that MetaCPAN lets me "++" modules (distributions, really) to mark them as favorites, but going distribution by distribution to click things is not my idea of a good time.

Time for laziness, impatience, hubris.

Click favorites? No way! Let's automate that shit!

(First, thanks to Moritz for giving me the pieces of the MetaCPAN API puzzle.)

Step 1: Get a list of distributions I use

Here's how I did it:

  • Use Path::Iterator::Rule (PIR) to find .pm files in @INC that I've used in the last two weeks
  • Convert file names into module names
  • Use the CPAN 02packages.details.txt file to map module names to unique distributions (tarballs)
  • Spew it out so I can save it to a file

Here's the code:

#!/usr/bin/env perl
use v5.14;
use strict;
use warnings;
use PIR;
use Parse::CPAN::Packages;
use HTTP::Tiny;

# how many days history to consider
my $days = shift || 7;

# get 02packages index
say STDERR "Loading 02packages.details";
my $index = "02packages.details.txt.gz";
my $res = HTTP::Tiny->new->mirror("http://www.cpan.org/modules/$index", $index);
if ( ! $res->{success} ) {
    die "Couldn't update $index\n";
}
my $packages = Parse::CPAN::Packages->new($index);

# find recently used .pm files
say STDERR "Finding modules used in within $days day(s)";
my $rule=PIR->new->perl_module->accessed("<$days");
my $iter = $rule->iter( { relative => 1 }, @INC );

my %dists;
while ( my $file = $iter->() ) {
    my $mod = $file =~ s{/}{::}gr =~ s{\.pm$}{}r;
    my $dist = eval { $packages->package($mod)->distribution };
    next unless $dist;
    $dists{$dist->dist} = $dist;
}

say join("     ", $dists{$_}->dist, $dists{$_}->cpanid, $dists{$_}->distvname) for sort keys %dists;

Then I ran it and captured the output:

$ perl recent-mods.pl 14 > dists.txt
Loading 02packages.details
Finding modules used in within 14 day(s)

Step 2: Prune the list

Lots of stuff in that file is deep in the dependency tree weeds.

Unless I recognize the name and think it's actually worth people checking out, I don't want to favorite it.

So I edited the dists.txt file and deleted out lines that I didn't want to favorite.

Step 3: Find my MetaCPAN API token

To favorite dists from a command-line app, I needed my user access token from this URL (you need to be logged into MetaCPAN first): https://api.metacpan.org/user

I looked for a stanza like this:

   "access_token" : [
      {
         "client" : "metacpan.org",
         "token" : "EfABjkq5qU0lXXXb_Be9TdwqB2I"
      }
   ],

Note: that's not my real token. If you don't see yours, log in with one of the social media links (not PAUSE) at https://api.metacpan.org/login

I copied the long "token" value for use in the next section.

Step 4: Post favorites using the API

With the token and list of distribution data, favoriting is just a matter of posting to the API with this program. (I pasted the token in the code, so if you copy this, update it with your own.)

#!/usr/bin/env perl
use v5.14;
use strict;
use warnings;
use HTTP::Tiny;
use JSON;
use Path::Tiny;

my $list = shift
  or die "Usage: $0 <file>\n";

my $token = "EfABjkq5qU0lXXXb_Be9TdwqB2I";

my $ua = HTTP::Tiny->new;

for my $d ( reverse path($list)->lines( { chomp => 1 } ) ) {
    my ($dist, $author, $release) = split ' ', $d;
    my $post = to_json( { distribution => $dist, author => $author, release => $release } );
    my $res = $ua->post(
        "https://api.metacpan.org/user/favorite?access_token=$token",
        { content => $post, headers => {'content-type' => 'application/json' }},
    );
    if ( $res->{success} ) {
        say "Favorited $dist";
    }
    else {
        warn "Could not favorite $dist ($res->{status} $res->{reason})\n";
    }
}

Then I run that with the distribution data:

$ perl vote-favorite.pl dist.txt
...

200 or so distributions later, I was done.

Step 5: Link my MetaCPAN account to my PAUSE account

To get my favorites to show up on my CPAN author's page, I had to link my PAUSE account to my MetaCPAN account on the identities page.

Step 6: Profit! Well... share, anyway

If you're not a CPAN author, I don't think there's yet a place to show off your favorites, but that's OK. Your vote still counts!.

Places like the weekly metacpan report blog will include your votes in their summary.

Now go vote for your favorite CPAN modules!

This entry was posted in cpan, hacks, perl programming and tagged , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

10 Comments

  1. Posted March 21, 2013 at 7:47 am | Permalink

    Awesome!
    Now, could you be kind to check if the recommendations provided by http://cpan-u.sysd.org/ for your PAUSE/MetaCPAN account really matters?

    • Posted March 21, 2013 at 11:57 am | Permalink

      Cool project. The recommendations were ok, but not great. AnyEvent stuff which I don't use but might. A couple IDEs, Padre and "Farabi", which I probably wouldn't use. Some App modules. Bot::BasicBot? The best recommendation is Starman, which I do use but didn't favorite.

      • Posted March 21, 2013 at 12:19 pm | Permalink

        Thanks for the feedback!
        I hope your mass-favorite trick will encourage Perl people to populate their tastes. Less sparse database will mean more meaningful recommendations :)

  2. hdp
    Posted March 21, 2013 at 10:00 am | Permalink

    You could instead generate a list of modules directly depended on by code for which you are the author. That might be a better starting list.

    • Posted March 21, 2013 at 11:59 am | Permalink

      Maybe, but then I have to do that analysis. "Everything I've used recently" is fast and picks up things from random utility scripts I use, too. The first pass gave me about 500 or so distributions and it took maybe 5 minutes (with dinner in between) to cut it down.

  3. brok
    Posted March 21, 2013 at 1:31 pm | Permalink

    Be careful, because you likely won't be able to undo any accidental favoriting. There's a long-standing bug that prevents many people from unfavoriting modules on metacpan: https://github.com/CPAN-API/metacpan-web/issues/572

  4. Posted March 25, 2013 at 3:43 am | Permalink

    This is cool! Thanks.

  5. Posted March 25, 2013 at 3:45 am | Permalink

    Very nice way! Good job.

  6. Glen
    Posted March 25, 2013 at 8:48 am | Permalink

    I've logged in on MetaCPAN. How do I get an access token?

    • Posted March 25, 2013 at 12:53 pm | Permalink

      Read Step 3 again. There is a URL you have to follow to get it. If that doesn't work, there's another URL for logging in (but if you're already logged into MetaCPAN, the first URL should work.)

© 2009-2014 David Golden All Rights Reserved