#perl #programming

Menu

Category Archives: perl

Post navigation

My contribution to celebrating CPAN Day this year is releasing at least one new distribution each day in August from day 1 to the culminating 16. Yeah, all of the distros are trivial and most of them classify as crap, but perhaps one or two are crap that you need 🙂 (A large proportion of them do solve my itch.)

Yesterday more or less completed the migration of my CPAN modules from using Log::Any to Log::ger. Sorry for the noise in the releases news channels due to the high number of my CPAN modules (particularly to Chase Whitener).

I did not have anything against Log::Any, to be honest. It was lightweight and so easy to use in modules as it encourages separation of log producers and consumers. Sure, I wish Log::Any had some features that I want but I was okay with using it for all my logging needs.

Until Log::Any 1.00 came along in late 2014, when its startup overhead jumped from ~2ms (version 0.15) to ~15ms. The new version ballooned from just loading strict, warnings, and Log::Any::Adapter::Null to over a dozen modules. Later versions of Log::Any improve somewhat on the startup overhead front but after the introduction of Log::Any::Proxy I thought it probably will not get back to the previous lightweight level. So I planned to write a more lightweight alternative along with probably implementing my wishlist. But this would require some time and in the mean time I wrote a hackish workaround called Log::Any::IfLOG that will load Log::Any only if environment variable like LOG or DEBUG or TRACE is set to true. I hated that workaround and regretted having created it.

By the way, why do I fuss over a few milliseconds? My major interest in using Perl is for building CLI applications. I am simply annoyed if my CLIs show a noticeable (~50-100ms or more) delay before responding with output (major offenders include Moose-based CLIs like dzil) when the fact is that Perl can be much more responsive than that. Log::Any is but one of several (sometimes many) modules I must load in a CLI so if a few ms is added here and a few more there, it could quickly add up. Also my CLIs feature shell tab completion and this is implemented by running the CLIs themselves for getting the completion answers so I always prefer responsive CLIs.

The recent Eid al-Fitr holiday finally made it possible for me to write a replacement for Log::Any: Log::ger in the course of a couple of weeks, along with all the log outputs and plugins to match all the features that I needed. So in what ways is Log::ger different than Log::Any or the other logger libraries?

First of all, for the low startup overhead goal, I've managed to keep use Log::ger having an overhead of just 0.5-1ms (without loading any extra modules). This is even less than use warnings and certainly less than use Log::Any (8-10ms) or the much heavier use Log::Log4perl ':easy' (35ms). This means, adding logging with Log::ger to your modules now incurs a negligible startup overhead. You can add logging to most of your modules without worrying about startup overhead.

What about null-logging (stealth-logging) overhead? Log::ger also manages to be the fastest here. It's about 1.5x faster than Log::Any, 3x faster than Log4perl, and 5x faster than Log::Fast (included here because the name claims something about speed). This is thanks to using procedural style logging (log_warn("foo")) instead of OO ($log->warn("foo")) and just using an empty subroutine sub {0} as the null default. If you don't want that tiny runtime overhead too, you can eliminate it with Log::ger::Plugin::OptAway. This plugin uses some B magic to turn your logging statements into a constant so they are removed during run-time.

As a bonus, due to the modular and flexible design of Log::ger, you can also: log using OO-style, use Log::Any style (method names and formatting rule), use Log::Log4perl style (method names and formatting rule), use Log::Contextual style (block style), or mimic other interface that you want. And mix different styles in different modules of your application. And as another bonus, writing a Log::ger output is also simpler and significantly shorter than writing a Log::Any adapter. Compare Log::Any::Adapter::Callback with Log::ger::Output::Callback, or Log::Any::Adapter::Syslog with Log::ger::Output::Syslog.

To keep this post short, instead of explaining how Log::ger works or the details of its features here I welcome you to look at the documentation.

Today I decided to add csv-grep to App::CSVUtils, as an alternative to NEILB's csvgrep (which Neil announced about a week ago). I find csvgrep too simplistic for my taste or future needs. It's basically equivalent to:

% ( head -n1 FILE.CSV; grep PATTERN FILE.CSV ) | csv2asciitable

I also think csvgrep's -d option does not belong. It's not relevant to grepping as well as too case-specific. What if user wants the oldest file in the directory? The biggest? The find or ls command should be able to do that for you:

% csvgrep PATTERN "`ls *.csv –sort=t | head -n1`"

In csv-grep, you specify Perl code instead of regex pattern. Your Perl code receives the CSV row in $_ as an arrayref (or hashref, if you specify -H). So you can filter based on some particular fields and use the full expressive Power of Perl. csv-grep outputs CSV, but you can convert it to other formats by the abovementioned csv2asciitable, or to JSON with csv2json, or to Perl data structure with csv2dd, or what have you.

Aside from csv-grep, App::CSVUtils also includes a bunch of other CSV utilities which I wrote when I needed to munge CSV files a few months back. Check it out.

This morning I decided to purge my 5.24 installation on my development PC and move to 5.26. Bad mistake. Many distributions still fail to build without PERL_USE_UNSAFE_INC=1, I might as well put the environment on my ~/.bashrc. These are the ones that hit me (after a few of these, I just built with PERL_USE_UNSAFE_INC=1): Compiler::Lexer (GOCCY), OrePAN (TOKUHIROM), Task::Weaken (ADAMK), Test::SubCalls (ADAMK), Test::Object (ADAMK), Term::Encoding (MIYAGAWA). Apparently, for many of these dists tickets/issues have been posted since 3 months ago but the authors/maintainers have simply not responded. Many authors have perhaps moved on to greener pastures, we need more adopters.