Perl source filters: evil or not?

I showed my program to a Perl hacker and got a lot of good feedback on it. He didn’t like my tedious table parsing and general reinventing of core Perl features with mediocre code. He suggested I look at Filter::Simple.

I have to say that Active State’s package manager program for CPAN is a pleasure to use. It’s exactly the sort of painless no-hacking type of tool that Windows development practices breed addictions to in its developers. No hacking around in weird script files. No posting questions on forums. Yea! So getting Filter::Simple is not a problem… and dashing out a quick test project is a piece of cake thanks to the good documentation on CPAN:

package SQUIG;
use Filter::Simple;

    s/(\w+)~(\w+)~(\w+)/try('\1', '\2', '\3')/g;

use SQUIG;

sub try {
    my ($a, $b, $c) = @_;
    print "$a...$b...$c...\n";


I think this easy-to-use tool can solve about 80% of my syntactic sugar issues. The only hitch I see is that I’d like to go beyond a simple sed-like command and call subroutines or functions on the replacement side so I can get different results depending on what turns up in the regexp match. For instance, I’d want to adapt the above “squiggle” filter to put single quotes around text, but leave scalars plain. I might want to transform Foo~Bar~$baz into $self->a_function_call(‘Foo’, ‘Bar’, $baz). (I’m not sure I’d want to see a regex that could do that….) This could help me clean up my object code and eliminate some of the ugliness in the method routines. (Although I may be using Moose in a hamfisted way to begin with….) Turning this technique to my data file parsing of table data… I could create simple functions that build up my ‘environment’ objects. Instead of parsing text and ‘manually’ loading up Moose objects, I could use the sourcefilter transform my table definitions directly into perl code wherever that makes sense.

This also brings me back to my original ‘dream’ project. I could create my own shorthand for a certain class of objects. I could create a set of source filters that get rid of the boiler plate. Instead of hacking up my own evalutator or abusing eval, I could rearrange functions however I like based on the keywords I tag with them.

My spider sense is tingling, though. I suspect this is only marginally less evil than my original inclination. This technique is not covered in any of my Perl books. Larry mentions it in passing in the Camel book, the Perl Cookbook mentions it only in the context of a switch command hack, and Mastering Perl explicitly avoids it. It’s strange to me that Perl literature is so far removed from how I actually use Perl day-to-day. Without word-of-mouth type tips from ‘real’ Perl hackers, I would never have thought to try either Moose or source filters… but that’s what I’m leaning on the most in my architecture. Is there really a gap or do I want bizarre things from my programming environment?


4 Responses to “Perl source filters: evil or not?”

  1. Aristotle Pagaltzis Says:

    Source filters are definitely evil and the online Perl community will generally try to dissuade you from using them. In some extremely limited cases they are useful, but as a regular code structuring tool they are evil evil evil. That is why you won’t find much mention of it in the literature. It is, for all intents and purposes, impossible to transform Perl code correctly.

    Translating a non-Perl format that has bits of Perl embedded inside it to Perl is OK, so long as you take care never to try to parse the Perl bits, which mainly means parsing the file while considering the Perl bits opaque. (Eg. if you put Perl inside XML: an XML parser won’t care that the text nodes it finds contain Perl, and if there’s any need to use XML metacharacters inside Perl code you have to escape them in the Perl code according to the XML rules. There is then never any ambiguity.)

    As for Moose, its popularity is simply too new for the module to have found its way into any printed Perl literature.

    You are definitely trying to do moderately unusual things and based on the fact that you say they are part of a “dream” project, this should really not be surprising to you. 🙂

    I don’t really get why you aren’t just writing a proper parser for an external DSL instead of translating to Perl, though.

  2. Mark Says:

    Why are you using backslashes in the shebang line at the top? Does that even do anything? Won’t your code work fine without that line?

  3. lispy Says:

    Sorry for the irritating error, Mark. I use Perl in three different environments, so I generally forget which way the slashes go until I get shell complaint. Yeah, it works without it as I normally don’t take the time to chmod stuff.

  4. draegtun Says:

    Well if you use Source Filters in ones own programs then we’re all old enough to accept the trade off & consequences! If you’re using it in programs that get released to the wild then make sure it does come with a government health warning attached 😉

    I think one of the problems with Source Filters is that it screws up the Perl debugger. Thus the slippery slope starts!

    In terms of literature, Simon Cozens does have a small section on Source Filters in “Advanced Perl Programming”.

    The modern alternative being pushed instead of Source Filters is Devel::Declare. This works on the bytecode instead of the source code. Its early doors at the moment but it does show great promise.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: