What are your top binary library dependencies?

On the #toolchain IRC channel, jawnsy and I were discussing how cool it would be if there were Alien::* packages for the top 10 or so binary libraries that people use with Perl. I.e. things like libssl, libxml2, libexpat, etc.

I don't know what it would take to make that happen, because getting Alien packages right across platforms is tricky, but a first step is figuring out what libraries people wind up needing the most.

If you install libraries/headers by hand, what do you wind up using most often? Please answer in the comments.

This entry was posted in perl programming and tagged , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

18 Comments

  1. Posted July 26, 2011 at 11:25 pm | Permalink

    I think the Gnu Scientific Library (aka GSL, libgsl) would be good to have, as Alien::GSL I guess. I know that PDL and Math::GSL want/need this (and rely on `gsl-config` or heuristics to find it. I have a small module (Math::GSLx::ODEIV2) which needs the newest version.

    I wouldn't mind trying to work on this; I can look at some other modules in the Alien:: namspace, but are there some other documents/webpages that could help me get started?

    • Jakub Narebski
      Posted July 27, 2011 at 7:28 am | Permalink

      If GSL (GNU Scientific Library), then perhaps also FFTW?

  2. kmx
    Posted July 27, 2011 at 2:28 am | Permalink

    Perhaps the set of libs included in strawberry perl is a good selection - see http://strawberryperl.com/package/kmx/32_libs/5.14/

    --
    kmx

  3. Christian Walde
    Posted July 27, 2011 at 2:44 am | Permalink

    I'm fairly sure the two topmost ones are mysql and postgres dev stuff.

  4. Bill
    Posted July 27, 2011 at 3:15 am | Permalink

    libssl

    libxml

  5. Hank
    Posted July 27, 2011 at 3:37 am | Permalink

    libgd

  6. bingos
    Posted July 27, 2011 at 5:26 am | Permalink

    I always make sure the following are installed on CPAN smoke boxen (either as -dev packages or from ports/pkgsrc/etc):

    curl (well libcurl-dev on such linuxy beasts)
    expat
    gdbm
    gtk2 (this pulls in hideous amounts of other -dev)
    libgmp
    libmm (this was requested by a CPAN author)
    libxml2
    mysql-client
    postgres-client
    ssl (usually in 'base' on *BSD, libssl-dev on linuxen)
    subversion

  7. Steffen Mueller
    Posted July 27, 2011 at 6:10 am | Permalink

    The suggestions in the comments seem like a pretty good selection. Highest on my list would be ssl. SQLite is another common one, but we have that embedded in DBD::SQLite already.

    What would be *even* more useful than just a fixed set of new Alien:: libraries would be having really good tools for creating new Alien:: extensions. I don't have a good overview of what it would take and the approaches to this are manifold, but it's always felt to me like it'll just take an afternoon of the right set of people talking and thinking to come up with a useful toolkit AND a standardized base API for getting library information. For extra kicks, add in versioning of libraries and figure out how to do that in a portable way. Oh my.

    The approaches I've seen many times are:

    a) Detect the system library (and headers!) and use them if available. Clearly, this must be the default. It must be possible to provide this functionality in a reusable, semi-abstract way.

    b) Fetch the library via HTTP/FTP and build it.

    c) Ship it and build it.

    d) Require the user to download the library and then build it.

    b), c), d) have the disadvantage of making distributors angry at you for duplicating code which is primarily a security/bug fixing issue. This means that it would be important to provide the distributors with an easy way to prevent that. Linux folks could just set a dependency on the right packages and set an environment variable that totally suppresses any attempt to fetch/build anything.

    Furthermore, b/c/d would not install the library into the standard system location, which can be both good and bad. As an example, for Alien::ROOT, I use approach a) with fallback to b) and provide the information whether we're using a "Perl-private" copy of the library via the Alien::ROOT API.

    If you plan to work on this general problem or need help with any particular bit or a sanity check, feel free to poke me.

    • Posted July 28, 2011 at 9:22 am | Permalink

      What would be *even* more useful than just a fixed set of new Alien:: libraries would be having really good tools for creating new Alien:: extensions.

      I agree. I've been thinking about creating such a framework but my toolchain todo-list is longish.

      The approaches I've seen many times are

      There's another approach that I haven't seen but that may be best in many cases: using the platform's native package manager.

  8. Posted July 27, 2011 at 7:47 am | Permalink

    1) openssl
    2) GNU readline
    3) libxml2
    4) expat (unfortunately...)
    5) gd (with libjpeg, libpng and the others)
    6) libidn (although there is one from IBM that might be better now)
    7) mysql-client
    8) libevent (although EV as libev which seems to be better now)
    9) libmcrypt

    Others that I used to need:

    a) subversion;
    b) apr
    c) libfaac, libmp3lame, libmp4v2, libx264
    d) tokyocabinet
    e) sphinx

  9. Posted July 27, 2011 at 9:15 am | Permalink

    I agree with Steffen, tools for makeing Alien:: packages (or even some good boilerplate) would encourage more people to contribute Alien modules when a module they need could use one.

    For example I have no idea how to build GSL on a windows computer. I know that Strawberry comes with MinGW, can that be used? What if Cygwin is present? etc.

    Further I would have no way to test on Win/Mac let alone HPUX or other OSes.

  10. nyet
    Posted July 27, 2011 at 9:24 pm | Permalink

    I am using win32, so this should be a longer list:
    gnu-coreutils and gnu-etc
    openssl
    gpg
    libxml2
    sqlite
    db_file
    mysql
    pg
    jdk/jre
    pkg-config & glib

  11. Alexander Hartmaier
    Posted July 28, 2011 at 4:24 am | Permalink

    in order of importance:

    build-essential for XS support
    libssl-dev for https support
    libaio1 for the Oracle Instant Client and DBD::Oracle
    libgd2-xpm-dev for GD::Graph
    libxml2-dev for XML::LibXML
    libwxgtk2.8-dev for Padre
    libssh2-1-dev for Net::SSH2

  12. Jeremy Leader
    Posted July 28, 2011 at 2:58 pm | Permalink

    We're a Linux-only shop, here's an approximate list off the top of my head:

    mysql (for DBD::mysql)
    npsclient (for Netezza data warehouse appliance, via ODBC)
    unixODBC (for DBD::ODBC)
    cdb (for CDB_File)
    db4 (for BerkeleyDB and/or DB_File)
    bzip2 (for Compress-Bzip2)
    protobuf-perlxs, which depends on protobuf
    subversion
    some of our older code uses nagios-nsca-client, but newer code uses Net::Nsca, which I think is pure Perl.

    Yeah, we use too many different databases, got a problem with that? :)

  13. Posted July 29, 2011 at 4:08 pm | Permalink

    Request for comment: on Alien::GSL
    see: https://github.com/jberger/Alien-GSL

    Ok so I have been working on an Alien::GSL module. Since the magic is supposed to go in the Makefile.PL, I worry that I use the functionality without any testing first. However it is a simple module so not much testing is needed perhaps (none is written yet). How do people find the workflow? Suggestions are welcome, though possibly they should be made on my blog post, at http://joelslinux.blogspot.com/2011/07/request-for-comment-on-aliengsl.html

  14. Posted July 29, 2011 at 8:05 pm | Permalink

    I'd really like to have Alien modules for:

    OpenSSL (for custom-built Perls on Windows, so it's a low priority)
    Image::Magick
    MySQL
    Apache (because building mod_perl can be a pain sometimes)

    libxml2/libxslt/sablotron may be good to have too, even if I use them only on occasion.

  15. Paul "LeoNerd" Evans
    Posted August 4, 2011 at 2:07 pm | Permalink

    I've never been sufficiently clear on what an Alien module is supposed to do. There seems to be quite a variety of differences between them, to the point that I couldn't work out how to make one.

    Should it provide a file called Alien/Whatever.pm?

    Should it contain the upstream source or merely link to it somehow?

    Should it be installed by its dependant's Build.PL? If so, how?

    How do you get the dependant to ensure it is installed and working if required, before Build.PL has finished?

    How do you handle non-root (uid!=0) installation? In particular, what part of the PERL5LIB layout should .h include files go on?

    These are all questions I came up with trying to write an Alien::TermKey for my libtermkey; I still haven't really answered any to the point I feel confident enough in releasing such a module.

    • Posted August 4, 2011 at 4:30 pm | Permalink

      I don' t think there's a real "standard" yet. Steffen's comment about better Alien builder tools is apropos.

      Should it provide a file called Alien/Whatever.pm?

      Yes, if the library is somehow "installed" and "available". This is what lets dependent modules know that the library is found.

      Should it contain the upstream source or merely link to it somehow?

      Either is fine with me. It probably depends on the size of the source and the stability of a download link.

      Should it be installed by its dependant's Build.PL? If so, how?

      I suspect it should usually be a 'configure_requires' item as the Alien::Whatever can provide routines to determine library/header directories used to configure Build.PL dynamically.

      How do you get the dependant to ensure it is installed and working if required, before Build.PL has finished?

      If it's a configure_requires item, you just load it in Build.PL. E.g.

      # Start of Build.PL
      eval { require Alien::Whatever } or do {
          warn "Alien::Whatever not installed.  Aborting install."; 
          exit 0; # graceful exit avoids CPAN Testers FAIL reports
      }
      

      How do you handle non-root (uid!=0) installation? In particular, what part of the PERL5LIB layout should .h include files go on?

      I'm not sure. I suspect the (ugly) answer is to install headers and libraries into site_perl with File::ShareDir, as that should work for non-root users. I've never tried it, though.

      -- David

© 2009-2014 David Golden All Rights Reserved