Archive for the ‘OOP’ Category

Why UML Fails to Add Value to the Design and Development Process

October 29, 2008

While attending the Domain Specific Modeling workshop at OOPSLA 2008, I heard many pointed criticisms of UML. No one went into detail, so I bought a book on DSM by Steven Kelly and Juha-Pekka Tolvanen. The book was not cheap– over ninety bucks when tax was added. (Doh!) So far I’ve read the first four chapters, but they cover the problems of UML fairly well in those early sections and also outline the basic tenets of the DSM philosophy as well. I’ve synthesized the gist of their points below. All the good ideas about modeling architecture below are imperfect summaries of Kelly and Tolvanen’s work. The opinionated ideas about programming languages and tools are my own.

UML is applying an abstraction at the wrong end of the problem. It is primarily used to sketch object models for inferior languages. As such, it tends to explode into incomprehensible patterns of accidental complexity in order to accommodate the various “design patterns” that are used work around the lack of essential language features. Because the UML models cannot be compiled, executed, or interpreted, they are reduced to the level of mere documentation. As such, it is generally not even worth keeping in sync– the manual round trip from the code to the model and back is just too expensive for something that adds no more value to a project than an elaborate code comment. (Slides of elaborate UML diagrams, on the other hand, are nevertheless great for impressing the uninitiated in a presentation, of course– that goes without saying!) Efforts to “fix” UML tend to not to gain traction: the specification is itself too broad and coding environments and applications are themselves too diverse.

A modeling language needs to do three things on order to become useful:

First, it should map directly to domain problem concepts. UML is designed to map to coding architectures– and because of this it fails to raise the level of abstraction. The jump from assembler to C gave an order of magnitude increase in productivity because of its corresponding increase in abstraction. OOP languages and UML have not given the increase in productivity gains that they should have– in some cases they may even hurt productivity. Modelers should not even think about implementation details when they develop their language. Instead they need to focus on mapping their ideas directly to the domain concepts. (Note: this is not a new idea. Ableson and Sussman popularized this approach in their SICP lectures at MIT where they demonstrated how Scheme would allow them to define programs that called functions that didn’t even exist yet. They would start with the right solution… and then gradually build up until it would run. Michael Harrison talked about this when he described the ‘wishful thinking’ approach to programming from SICP.)

Second, the modeling language must be formalized. The modeling language must be a first-class citizen of the development process rather than just make-work for architects and project managers. It must be possible to generate useful executable code from the models made within the language. The language needs to add value not just in communicating with domain experts and helping them validate the “business logic”– it needs to add value at all levels of the development process. It needs to raise the level of abstraction for the code maintainers by allowing them to stop thinking about the underlying frameworks and libraries. It needs to contribute to testing efforts by eliminating the need for implementing certain classes of tests and also by providing a basis for generating other classes of tests automatically from the models. It needs to possible to generate documentation automatically from the models. The models should be useful in and of themselves and should be significantly useful to all development tasks downstream from them.

Finally, the modeling language should have first-class tooling support. The tools should not be thought of as IDE extensions for programmers. These tools are not “wizards” to generate ugly code or partial stubs for developers to flesh out. The tooling should stand alone for the domain experts; if they have to think about the code at all then the tools are developing in the wrong direction. Expert developers aware of the intricacies of the domain problem need to organize their frameworks and libraries in such a way that the models can be “compiled” into fully functioning code. The correct analogy for this is, again, to think in terms of compiling C to machine code. The C compilers are created by machine language experts. C programmers do not modify the compiled machine code– they use the results in practically all cases. The C programming environment allows the programmer to work at a higher level– without having to think in terms of the underlying machine code or hardware. C is, in effect, a modeling language for machine code.

So, to get a modeling language that is actually useful, you have to go “Domain Specific” on both sides of the problem: the modeling language has to map directly to the problem domain and the generated code has to map directly to the target environment. There are two linguistic abstraction barriers that must be implemented in order to make this work: 1) the modeling language between the models and the generated code and 2) the framework between the generated code and the target libraries. You must build up from your core code components to the framework… and you must build down from the models to the generated code. If the code generation process is too complicated, you may need better abstractions at the framework level. If the code generation process is impossible, then the modeling language may not be providing a detailed enough description of the requirements. If there is too much repetition in the models, then the modeling language will need to be extended to cover additional concepts.

As Steven Kelly said in a 2006 article, “To make model-driven development work this way, you cannot use a general-purpose design language like UML and a modeling tool’s built-in code generator. The people that created UML did not design it for describing applications in your domain, or for generating code other than skeletons. Despite the efforts of many to make it suitable for generation, no one has ever, or will ever, make it happen in a way which is still smart and convenient. An application’s behavior and the rules it has to adhere to are domain-specific. To capture that effectively and completely in a graphical design, you need a Domain-Specific Modeling language.”

Update 10/30/08:

I have discovered some interesting ideas from the opposite point of view. Here is some remarks from Franco Civello– someone that has used UML successfully in model driven development:

“… having produced informal use cases to clarify requirements, and a domain model to get an initial understanding of the subject area, the analyst produces a precise specification model, in UML, in which the system to be developed is represented as an object, belonging to a type (note, not a class, as the system is an abstraction used to define visible behaviour, not a software entity to be directly implemented in e.g. Java).”

Notice the key similarity there with the Kelly/Tolvanen Tolvanen approach. Civello is not using UML to describe the code architecture– he is mapping the UML more toward the problem domain.

“Steps in the use case flows are then formalised as operations on the system type, with a declarative specification of behaviour based on the notion of functional contract, written as pre- and post-conditions expressed on an underlying model (the system type model, derived closely from the domain model).”

Again, this point reflects back to my second point: there must be a formalization of the modeling language at some point. Note also, we have (possibly) an analog to the declarative approaches that I saw in the ModelTalk presentation last week.

“UML static modelling gives you the language to represent the state of the system in abstract and yet precise terms. Just don’t think of your classes as software things with methods and member data, but as specification types that give you the vocabulary to specify the business outcome of a system operation. These types can have attributes, associations, queries, constraints and definitions, all powerful UML concepts, but no state-changing operations. The only state-changing operations are defined at the system level, and are not elaborated into message-based solutions, but just specified declaratively in terms of business logic, steering clear of any design decisions.”

OOPSLA 2008 or Bust!

October 16, 2008

Sadly, I have to cut the conference short for work reasons.  (Grumble grumble….)  Much of the stuff I’m missing appears pretty academic, though– way over my head. (But I really was hoping to take in the whole thing…. Sigh.)

Here’s what I’m catching:

SUNDAY:

The 8th OOPSLA Workshop on Domain-Specific Modeling
(Workshop)
Juha-Pekka Tolvanen, Jonathan Sprinkle, Jeff Gray, Matti Rossi
Room: 210
8:30 – 12:00

DesignFest
13:30 – 17:00

MONDAY:

Lisp50 – The 50th Birthday of Lisp at OOPSLA 2008
(Workshop)
Pascal Costanza, Richard P. Gabriel, Robert Hirschfeld, Guy L. Steele Jr.
Room: 204
8:30 – 17:00

Bodging alternatives to full-on compilers– and the true meaning of OOPness

October 16, 2008

“You say, well if we were using Haskell for this then we would just do this thing; can we adapt that to what we’re doing here? Not saying you have to use Haskell to solve it, but could you use that kind of approach?” — Damian Conway

I’ve been experimenting with parsing for a while now. I’m on the verge of digging into tokenizing and grammars. Before I hunker down and educate myself, I just want to see how far you can go with less industrial-strength approaches. I think it’s pretty cool how combining a single good regex with a closure gets you what is for all intents and purposes a mini-compiler. This is a pretty handy trick; filing away such dynamically generated functions in a hash table can lead to some pretty powerful techniques, too. A dispatch table like that can even be used to create a simple programming language on the fly. Tokenizing a string is just another algorithm… and on first glance, grammars appear (in a way) to be fancy dispatch tables. How do you bring them all together? Some kind of deep magic from the dawn of time– something I would never figure out by trial and error or intuition. I’m getting there. But just for fun, let’s generalize some of these ideas and (at the same time) temporarily ignore them. Let’s go entirely orthogonal to this plane of thinking for a moment.

OOP as it gets applied in ‘the real world’ is primarily a defensive exercise: “I don’t like you and I don’t like the way you code. Let me just wall off on my little corner and do my thing. Let me work with your stuff without having to understand your code. Don’t break the contract, either, bub. Somehow, I don’t think this is where Alan Kay intended us to go with this….

Think about the difference between Checkers and Chess for a second. The permutations of moves are effectively infinite in both games. But the space of possibilities have different ‘shapes’. If you were, like some sort of Paul Atreides, to look out at the limitless patterns in the games, for chess you’d see an expansive plane of cascading options. Checkers would look more like an infinitely deep well.

The way we tend to do OOP is more like Checkers than it is like Chess. The calls between and among the various objects seems brittle and constricted. Like talking to someone through an incredibly long pipe instead of face to face…. So how to open things up and make things more fluid and fluent? My idea is to swing the emphasis in OOP back much more toward the message passing concept, steal a few idioms from Lisp, and generalize heavily on the overall theme of DSL’s. (I always thought it was cool how in Lisp practically everything had a straight-forward text representation. The reader transforms what you type into a tidier, more consistent representation… and then shows it to you as the response.)

What if in designing a programming language, you divided it up into various objects or components. To extend the language you just drop in new objects. The classes that the objects derive from each come (after a fashion) with their own DSL. Each object would ‘know’ how to generate its own source code. While working with an object via a REPL, you could modify the object… then view and/or save the modified source without dropping back to the IDE or text editor. The object source code should understand the concept of testing and incorporate test code into its specification. When you talk to code via a REPL, you end up doing lots of little tests anyway. You should be able to incorporate those off-the-cuff tests into the code without thinking about it. (Any inconvenience in automated testing development just makes it that much more likely that it doesn’t get done. Get rid of everything that stands in the way of it.)

So each object in the system would know how to do stuff like read, source, save, and load. Each object would have its own mini-language that you can focus on separately without having to think of all of the parsing at once. Your collection of objects would get wrapped in a unified reader. The main reader would pass messages to each object-group in the deck until something understood the message. (And just like “library lists” on the AS/400, you could rearrange the reading order on the fly.) Simple! Now you can build a programming language without having to think about the whole thing at once– and you get modularity and testability benefits in the process. If you don’t have time to write ‘real’ compilers and DSL’s, this might be enough to get you by for a while. Of course, it might be enough to make you buckle down and learn some real computing techniques, too. Judge for yourself.

If you’d like to play with this (or see more specifically part of what I’m trying to describe), I’ve implemented the gist of the idea in Perl. There’s nothing fancy in it; I’m just using the bottom 30% of the language for the most part. There’s plenty of places the overall idea could be further generalized and tightened up. (I think I’m part way through reinventing Smalltalk, but this is just for fun anyway.)

Here’s a couple of sample screen shots plus the code:

Do what I mean; Who cares about joins?

Do what I mean; Who cares about joins?

Some Lame Syntactic Sugar Hacks

Some Lame Syntactic Sugar Hacks

###################################################################################################
{
    
package Functions;
use Moose;

has Functions => (isa => 'HashRef', is => 'rw', default => sub { {} } );
has Source => (isa => 'HashRef', is => 'rw', default => sub { {} } );
has Environment => (isa => 'Environment', is => 'rw'); ### required only if implementing syntactic sugar for table refernces

sub set {
    my ($self, $key, $text) = @_;
    my ($ref, $code, $r);
    $code = $text;
    $code =~ s/(\d+[dD]\d+)/\$self->roll_maker('$1')->()/g; ### the regex could be improved-- missing some cases....
    $code =~ s/!(\w+)/\$self->Functions->{$1}->()/g; ### should this call Environment or the local deck?   [YES!]
    $r = '[^~\s;]';
    $code =~ s/($r+)~($r+)~($r+)/\$self->Environment->Tables->cell($1, $2, $3)/g;
    $code = "sub { $code }";
    print "The actual code is $code\n";
    $ref = eval $code;
    if ($@){
        print "Error setting $key: $@" if $@;
    } else {
        $self->Source->{$key} = $text;
        $self->Functions->{$key} = $ref;
    }
}

sub call {
    my ($self, $key) = @_;
    if ($self->Functions->{$key}){
        eval { $self->Functions->{$key}->(); };
        print "Error calling $key: $@" if $@;
    } else {
        print "No such function '$key'.\n";
    }
}

sub show {
    my ($self, $key) = @_;
    print "\n\n";
    if ($key){
        if ($self->Source->{$key}) {
            my $source = $self->Source->{$key};
            print "set $key: $source\n";
        } else {
            print "No source available for $key.\n";
        }
    } else {
        foreach $key (sort keys %{$self->Source}) {
            my $source = $self->Source->{$key};
            print "set $key: $source\n";
        }
    }
    print "\n";
}

sub read {
    my ($self, $line) = @_;
    if ($line =~ /^set (\w+):\s*(.+)/){
        if (defined($self->set($1, $2))) {
            print "Function $1 set.\n";
        }
    } elsif ($line =~ /^call (\w+)$/){
        $self->call($1);
        print "\n";
    } elsif ($line =~ /^show\s*(\w*)/){
        if ($1) {
            $self->show($1);
        } else {
            $self->show();
        }
    } elsif ($line =~ /^load\s+([\w.]+)/){
        $self->load($1);
    } elsif ($line =~ /^save\s+([\w.]+)/){
        $self->save($1);
    } else {
        return 0;
    }
    return 1;
}

sub load {
    my ($self, $file) = @_;
    open FILE, "< $file";
    while (<FILE>){
        chomp;
        $self->read($_);
    }
    close FILE;
}

sub save {
    my ($self, $file) = @_;
    open FILE, "> $file";
    foreach my $key (sort keys %{$self->Source}) {
        my $source = $self->Source->{$key};
        print FILE "set $key: $source\n";
    }
    close FILE;
}

sub roll_maker ($) {
    my($self, $line, $num, $type);
    $self = shift;
    $line = shift;
    
    return undef unless $line =~ m/^(\d+)?[dD](\d+|%)/;

    $num    = $1 || 1;
    $type   = $2;

    $type  = 100 if $type eq '%';

    return sub {
        my $result;
        for( 1 .. $num ) {
            $result += int (rand $type) + 1;
        }
        return $result;
    }
}

}
###################################################################################################
{    

package Callback;
use Moose;

sub choose {
    my ($self, $text) = @_;
    my @options = split /;/, $text;
    #print "the text is $text\n";
    my $i = 0;
    print "\n";
    foreach(@options){
        $i++;
        print "$i: $_\n";
    }
    
    my $answer;
    until ($answer) {
        print ">> ";
        chomp(my $line = <STDIN>);
        if ($line =~ /^\d+$/) {
            $answer = $options[$line - 1];
        } else {
            foreach(@options){
                if(/^$line/i){
                    $answer = $_;
                    last;
                }
            }
        }
    }
    return $answer;
}

}
###################################################################################################
{

package Table;
use Moose;

my $show_debug2 = 0;

has Name => (is => 'rw', isa => 'Str', default => 'Bob');
has 'Rows' => (isa => 'HashRef', is => 'rw', default => sub { {} } ); 
has 'Columns' => (isa => 'ArrayRef[Str]', is => 'rw', default => sub { my @a; return \@a; } );
has 'ColumnRegexes' => (isa => 'ArrayRef[Str]', is => 'rw', default => sub { my @a; return \@a; } );

sub initialize {
    my ($self, $table, $text) = @_;
    $self->Name($table);
    my ($columns, $column_regexes) = parse_column_header($text);
    $self->Columns($columns);
    $self->ColumnRegexes($column_regexes);
    my %rows;
    $self->Rows(\%rows);
}

sub load_line {
    my ($self, $text) = @_;
    my ($rowkey, $row) = parse_row_detail($self->Name(), $text, $self->Columns());
    $self->Rows->{$rowkey} = $row;
}

sub cell {
    my ($self, $row, $column) = @_;
    return $self->Rows->{$row}->{$column};
}

### pass a comma delimited header line from a table definition
### and get two array references describing the table structure
sub parse_column_header {
    my ($line) = @_;
    my @fields = split /,/, $line;
    my $column_number = 0;
    my @columns;
    my @regexes;
    print "reading columns to table: $line\n" if $show_debug2;
    foreach(@fields){
        my $field = $_;
        $field =~ s/^\s+|\s+$//g; # trim field
        if($field =~ /^([^\/]*)\/([^\/]*)\//){
            $field = $1;
            $regexes[$column_number] = $2;
        }
        $columns[$column_number] = $field;
        $column_number++;
    }
    return (\@columns, \@regexes);
}

### pass a table name and a comma delimited header line from a table definition
### and also pass a reference to an array of column names...
### and get the row's key and a hash of detail data
sub parse_row_detail {
    my ($table, $line, $columns) = @_;
    my @fields = split /,/, $line;
    print "reading rows to table $table: $line\n" if $show_debug2;
    my %row;
    my $column_number = 0;
    my $rowkey;
    my $didit;
    foreach(@fields){
        my $field = $_;
        $field =~ s/^\s+|\s+$//g; # trim field
        # Need to allow some keys to be '0'!
        if ($didit){
            $row{$columns->[$column_number]} = $field;
        } else {
            $rowkey = $field;
        }
        $column_number++;
        $didit = 1;
    }
    return ($rowkey, \%row);
}

sub show_keys {
    my ($self) = @_;
    my $key;
    my $table = $self->Name();
    my $rs = $self->Rows();
    foreach $key (sort keys %{$rs}){
        print "$table row    : $key\n";
    }
    my $cs = $self->Columns();
    foreach (@{$cs}){
        print "$table column : $_\n";
    }
}

sub show {
    my ($self) = @_;
    print $self->text();
}

sub text {
    my ($self) = @_;
    my $text;
    my $x;
    
    $text =  "table $self->{Name}:\n";
    $text .= "head: ";
    foreach(@{$self->Columns}){
        $text .=  ", " if $x;
        $text .=  $_;
        $x = 1;
    }
    $text .=  "\n";
    foreach my $key (sort keys %{$self->Rows}){
        $x = 0;
        $text .=  "row: ";
        foreach(@{$self->Columns}){
            $text .=  ", " if $x;
            $text .=  $key unless $x;
            $text .=  $self->cell($key, $_) if $x;
            $x = 1;
        }
        $text .=  "\n";
    }

    return $text;
}

}
###################################################################################################
{

package Tables;
use Moose;

has Tables => (isa => 'HashRef[Table]', is => 'rw', default => sub { {} } );
has ReadAsTable => (isa => 'HashRef[Str]', is => 'rw', default => sub { {} } );
has Current => (isa => 'Str', is => 'rw');
has Callback => (isa => 'Callback', is => 'rw', default => sub { Callback->new(); } );

sub cur {
    my ($self, $table) = @_;
    $self->Current($table);
    if ($self->Tables->{$table}) {
        print "Table $table is the active table.\n";
    } else {
        print "Ready to initialize table $table.\n";
    }
}

sub cell {
    my ($self, $table, $row, $column) = @_;
    my $T = $self->Tables->{$table};
    if ($T){
        my $value = $T->cell($row, $column);
        $value = $self->private_check_other_tables($table, $row, $column, $T) unless $value;        
        if ($value){
            $value = $self->ReadAsTable()->{$value} if $self->ReadAsTable()->{$value};
            if ($value =~ /;/){
                $value = $self->Callback->choose($value);
            }
            return $value;
        } else {
            print "Row $row Column $column not found in table $table.\n";
            return undef;
        }
    } else {
        print "Table $table does not exist.\n";
        return undef;
    }        
}

### if value not found, loop through each table
### if a table contains the column *and* the key is contained in the specified table
### then return the foreign value!
sub private_check_other_tables {
    my ($self, $table, $row, $column, $T) = @_;
    foreach my $key (sort keys %{$self->Tables}) {
        unless ($key eq $table){
            my $OTHER = $self->Tables->{$key};
            foreach(@{$OTHER->Columns}){
                #print "col-- $_\n";
                if ($_ eq $column){
                    ### does the first column match the table we're looking at?
                    foreach(@{$T->Columns}){
                        #print "mycol-- $_\n";
                        if ($_ eq $OTHER->Columns->[0]){
                            my $foreign_key = $T->cell($row, $_);
                            my $foreign_cell = $OTHER->cell($foreign_key, $column);
                            #print "Found it! (foreign key $foreign_key) (value $foreign_cell)\n";
                            return $foreign_cell;
                            #last; ### this doesn't really *last* it.
                        }
                    }
                }
            }
        }
    }           
}

sub show {
    my ($self, $key) = @_;
    print $self->text($key);
}

sub text {
    my ($self, $key) = @_;
    my $text;
    
    $text = "\n\n";
    if ($key){
        $text .= $self->Tables->{$key}->text();
    } else {
        foreach $key (sort keys %{$self->Tables}) {
            $text .= $self->Tables->{$key}->text();
            $text .= "\n\n";
        }
    }
    $text .= "\n";

    return $text;
}

sub read {
    my ($self, $line) = @_;
    my $table = $self->Current();
    if ($line =~ /^table (\w+):?/){
        $self->cur($1);
        #print "Table $1 is now active.\n";
    } elsif ($line =~ /head: (.*)/) {
        my $T = new Table();
        $T->initialize($table, $1);
        $self->Tables->{$table} = $T;
        print "Table $table initialized.\n";
    } elsif ($line =~ /row: (.*)/) {
        my $T = $self->Tables->{$table};
        if ($T) {
            $T->load_line($1);
            print "Table $table line loaded.\n";
        } else {
            print "Table not found.\n";
        }
    } elsif ($line =~ /^(\w+)~(\w+)~(\w+)/){
        my $cell = $self->cell($1, $2, $3);
        if (defined($cell)) {
            print "$cell\n";
        } else {
            print "Cell not found.\n";
        }
    } elsif ($line =~ /^readas ([\w\s]+)\s+->\s+(.+)/){
        $self->ReadAsTable->{$1} = $2;
        print "Reading '$1' as '$2'.\n";
    } elsif ($line =~ /^show\s*(\w*)/){
        if ($1){
            if ($self->Tables->{$1}){
                $self->show($1);
            } else {
                print "Table not found.\n";
            }
        } else {
            $self->show();
        }
    } elsif ($line =~ /^show/){
        $self->show();
    } elsif ($line =~ /^load\s+([\w.]+)/){
        $self->load($1);
    } elsif ($line =~ /^save\s+([\w.]+)/){
        $self->save($1);
    } else {
        return 0;
    }
    return 1;
}

### same routine as in Functions class
sub load {
    my ($self, $file) = @_;
    open FILE, "< $file";
    while (<FILE>){
        chomp;
        $self->read($_);
    }
    close FILE;
}

sub save {
    my ($self, $file) = @_;
    open FILE, "> $file";
    print FILE $self->text();
    close FILE;
}

}
###################################################################################################
{

package Environment;
use Moose;

has Functions => (isa => 'Functions', is => 'rw', default => sub { new Functions() } );
has Tables => (isa => 'Tables', is => 'rw', default => sub { new Tables() } );
has Settings => (isa => 'HashRef', is => 'rw', default => sub { my %hash; $hash{WorkingWith} = 'Functions'; return \%hash; } );

### This should be in the constuctor
sub hack {
    my ($self) = @_;
    $self->Functions->Environment($self);
}

sub read {
    my ($self, $line) = @_;

    if ($line =~ /^work functions/){
        print "Working with functions.\n";
        $self->Settings->{WorkingWith} = 'Functions';
        return;
    }

    if ($line =~ /^work tables/){
        print "Working with tables.\n";
        $self->Settings->{WorkingWith} = 'Tables';
        return;
    }

    ### Refactor Me Sometime
    if ($self->Settings->{WorkingWith} eq 'Tables'){
        unless ($self->Tables->read($line)){
            unless ($self->Functions->read($line)){
                print "Unknown command.\n";
            }
        }        
    } else {
        unless ($self->Functions->read($line)){
            unless ($self->Tables->read($line)){
                print "Unknown command.\n";
            }
        }
    }
}

}
###################################################################################################
{
    
package Test;
use strict;
use warnings;
my $T = new Tables();
my $F = new Functions();
my $E = new Environment(Tables => $T, Functions => $F);

print "\n>> ";
while(<STDIN>){
    chomp;
    $E->read($_);
    print "\n>> "
}

}
###################################################################################################

Looks like we’re rolling with Moose…

September 26, 2008

After the nth person telling me to try Moose, I finally gave in.

The first thing I wanted to know was how to handle private variables, hash table properties, and object type properties. The Moose Quick-Ref Card lists the built-in Data Type constraints. I think Randal L. Schwartz has written the definitive introduction, though. (His follow up goes into more detail about custom types….)

We can see below that the read-only field on the Example object needed to have a writer defined for it. Even though the method is named private_set_x, it can still be called from the outside world. Also, we can set the “read only” member via the constructor hash! Hash and object property types seem to work just fine. The only real trick there was to use an in-line sub to initialize them.

#!/usr/bin/perl
package Example;
use Moose;

has 'x' => (isa => 'Int', is => 'ro', writer => 'private_set_x', default => 21);

sub test {
    my $self = shift;
    my $x = $self->x();
    print "My x is currently a $x\n";
}

sub setit {
    my $self = shift;
    $self->private_set_x(42);
}

package Another;
use Moose;
extends 'Example';

has 'hashtest' => (isa => 'HashRef', is => 'rw', default => sub { my %hash; return \%hash; } );
has 'exampletest' => (isa => 'Example', is => 'rw', default => sub { Example->new(); } );

after 'setit' => sub {
    my $self = shift;
    print "The after method is getting called now.\n";
};



my $thing = Example->new();
$thing->test();  #My x is currently a 21
$thing->setit();
$thing->test();  #My x is currently a 42
$thing->private_set_x(84);
$thing->test();  #My x is currently a 84

my $otherthing = Example->new(x => 168);
$otherthing->test();  # My x is currently a 168

my $another = Another->new();
$another->setit();  # The after method is getting called now.

my $h = $another->hashtest();
$h->{"One"} = "Hello";
my $v = $another->hashtest()->{"One"};
print "The hash value is $v\n";  # The hash value is Hello

my $e = $another->exampletest();
$e->test();  # My x is currently a 21

How Studying SICP Made Me a Better Programmer, part II

October 27, 2007

Okay, I told “Jon” that I would follow this up, so here goes.

The problem I had with the code last time was that I was leaning too much on bad habits I developed staring at too much relational database code. I used a hash table to store everything about the data structures. In all my functions for manipulating the data, I had to tag the arguments with an additional reference to to the hash. This was pointlessly cumbersome. What I really needed to do was eliminate my dependency on a data store and just use cons, car, and cdr to construct everything. This allows me to treat my data structures (and pieces of my data structures) as primitives in a linguistic abstraction layer. As a bonus, all of the standard common lisp operators work with them without any “crutches” or extraneous code getting in the way.

One thing I learned going down this road is that the Visitor Pattern is hardly anything more than an application of mapcar. I remember trying to learn about Design Patterns a few years ago and I just couldn’t pick many of them up. The code examples for them and the UML diagrams seem to me to do a good job of obfuscating the whole point. Your brain has to “compile” so much information just to get to the point– its no wonder so many developers mangle them up in the application of them. After reading SICP, it’s much easier to parse them. In a lot of cases they just do things that are very similar to what I want to do in my more functional language anyway.

Another thing I picked up in this exercise was that dynamic variables and hash tables store references to the “conses”. This means you have a lot of control over the lists that are stored in them. It also means that I don’t have to work too hard to write code to manipulate or transform them. Life certainly gets easier coding in Lisp once you think in terms of “pairs” so fluently that they are more natural than anything else. The example code for this post demonstrates some implications of this in detail, so folks that are just beginning their studies with Lisp will want to take note of what’s going on there when we start doing the “forbidden” destructive operations….  (The output of the Common Lisp code should look like this.)

Finally, after having slogged through most of the first two chapters of SICP, I can finally understand much better how bad a developer I’ve been. On the one hand, I can comprehend a much wider range of architectural approaches… but as a consequence it’s sinking in more and more just how limited my mental programming vocabulary was up until now. I really had no clue how ignorant I was. I knew I was struggling with things that should have had tidier answers, but I really thought I could only ever get just a little bit better than what I was. Now I can sense all those several orders of magnitudes that I have to go and I wish I could consume a dozen more books before going on to the next project– but life and learning don’t work quite like that….

The main thing is that I now see that I never really understood OOP near as well as I thought that I did. A lot of people objected to the recent Golf ball example that a Perl hacker recently used to criticize typical object oriented “design,” but that guy sure had me pegged. Of course, good OOP design is going to be almost indistinguishable from functional programming approaches in some cases– even if it’s a little more verbose– but I don’t think I would ever have understood that so well without my many hours spent with Lisp and SICP.

What You Don’t Know About Closures Can Hurt You

October 25, 2007

Some of you out there may still be wondering why it is that closures and macros are so great. “What exactly can you do with them that I can’t do in my more widely used Turing-complete language,” you ask. That’s reasonable. Well, I don’t think you’ll fully appreciate closures until you grasp some of the basics of functional programming, so let’s start there.

Functional programming languages accord a first-class status to procedures:

* They may be named by variables.
* They may be passed as arguments to procedures.
* They may be returned as the results of procedures.
* They may be included in data structures.

In other words, you can treat procedures as primitives. In the same way that you operate on integers, juggle them, shuffle them, combine them, transform lists of them, do anything you want with them… a functional programmer does the exact same thing with procedures.

Now if you’ve never thought much about this or experimented with these ideas, this will sound bizarre. You might have a hard time imagining how this can possibly be so useful. The only way I can think of to describe it is to imagine that you lived in a world where there was no such thing as subroutines, yet– and then suddenly some experimental language came out that had that feature. People would not admit that it was particularly useful and they would point to their terrible hacks and workarounds as evidence that such “ivory tower” concepts were useless in the Real World. Of course, you’ve seen almost the exact same thing in your workplace already, more than likely. When you see code repeated across the code base and you abstract it out into a class that is referenced by multiple projects, suddenly your coworkers are peeved at you because they can’t understand a section of code that they’re looking at without hitting “right-click — Definition” to jump to your new class file. This is why the default code architecture for a crummy business app is to open up a hundred separate form files and freely mix, match, and repeat business logic, data access code, and gui instructions all together. It’s a wonder that these guys even use subroutines…. (Oh wait, you see that, too: gigantic subs that go on for screen after screen that you can’t understand….)

Anyways, when you get procedures being treated as first-class citizens of the language, new abstractions are possible. You can create procedures that take procedures as an argument and that return procedures as a value. This means you can take several similar functions and abstract out the basic process that they all share. The procedures that you use as arguments can be used customize as many other procedures as you like. So on the one hand, you get code reuse via the basic “skeleton” procedures… and on the other, you get opportunities to reuse the code that is being used to customize the “skeletons.” Anonymous functions can be used to in-line the customizations when they’re not going to be reused, so you end up with cleaner, more expressive code.

Before I looked much into it, functional programming sounded like a step backward. I thought that the whole idea of separating code and data out and encapsulating it into classes was the answer to everything. Of course, you don’t have to give up on classes altogether. You can use functional style techniques to expand the configurability of your objects. In some cases this might be a more appropriate choice than inheritance, for example.

Now, one thing I’ve noticed with my OOP designs is that I end up spending a lot of time tinkering with my interfaces. It seems like I spend more time fiddling with the properties and methods lists and avoiding breaking the Law of Demeter than I do actually working on real code. And then I go back and break apart classes and rework the object model some more. With first class procedures, you can often skip the tedious object instantiation ritual and the maintenance of a separate class file and just jump right to what you’re trying to do. And if you can apply your procedure to a wide variety of data structures, so much the better.

Along into this scene arrives the closure. Again, if you aren’t jumping up and down with excitement for what first class procedures can do for you, you won’t really appreciate what closures add in to the mix. A closure is just a procedure that carries an environment with it. It’s like a class that has just one method and as many private variables as you’d want to cram into it. If you’re entrenched in an OOP mindset, you might think that’s ridiculous– you can make such things in your current language, you think to yourself. But this is one of those cases where less is more… and when mixed with other functional techniques you get some powerful options at your disposal.

So a closure turns out to be just a procedure with state. It’s sort of like an extremely stripped down object. The first thing people tend to show off when they’re going over closures is how you can add a counter to a procedure. Not very exciting, eh? But yeah, you can actually use that simple counter to package all kinds of logging and diagnostic functionality with your procedures. You can make a procedure that takes a procedure as an argument and returns that exact same function with your diagnostic code bolted onto the inside of it. The rest of your code doesn’t care if you’re using the plain vanilla procedure or the one with the extra logging utilities running with it.

Another thing that attaching a counter to a procedure makes possible is the creation of iterators. You might have seen an article recently that talked about how one of these days you just might write your last for loop. This, my friends, is how you get rid of for loops: you write a procedure that takes a procedure and an integer as arguments and returns a closure as its value. Each time you call the procedure, the counter gets incremented and it returns the next value from the procedure. When the counter reaches the same value as the integer argument, the closure returns a null value. Then what you need is a means of applying your other procedures to the values that are kicked out of your iterators– and with that you are now set free from the tedium of cryptic for-next loop code.

Now you may be thinking that there’s nothing wrong with your trusty for-next loop. And if you’re an OOPy type of person, you might think that your for-each loop does the job just fine. But how would you say “for each prime number from 200 to 300, execute the following function with that number as an argument.” Functional programming gives you a very expressive way to code exactly that. Another situation where iterators are useful is where you’re working with infinite lists or loops that repeat back on themselves. You might not be able to store an entire list of objects in a collection all at once– but with iterators you can simulate that very thing. You probably already use something very similar to an iterator when you open up a text file and operate on it on a line by line basis.

It occurs to me as I write this, that there was a time when I’d write “do while not rs.eof” at least a dozen times a day. I’d crash my IDE at least once every week or so because I forgot to put in “rs.movenext” at the back end of loop. That redundant code peppered throughout the application was a major liability– especially when we ended up having to move to a different data access paradigm. Oh the pain! What we needed was an iterator that could be instantiated with a SQL string. Then we could have written procedures to operate on those iterators. No more annoying do-loops! As a special bonus, if we had been writing unit tests back then, we could have made dummy iterators that don’t even call a live database– they just ould have pulled in a stream of test data instead!

There’s a lot more we could delve into on this topic, but the point is that with even just a smidgen of functional programming techniques, we could have made our code much more concise. We would have had additional means of separating out redundant code, and it would have been much easier to “rewire” the code to change its behavior.

Now, I mentioned briefly already that in functional programming we’ll strive for making our procedures able to operate on a wide range of data structures. It turns out that the key to making your functions really powerful in a wide range of situations is to use recursion. In your Imperative/OOP world, you may have only used recursion in a few rare cases– like searching through directories or setting up a tree view. Well, in functional programming, the default approach to just about everything is to use recursion. Your data structures are often defined recursively, so your functions that operate on them are consequently recursive as well. Recursion becomes your meat and drink and even the air that you breath… and you become intimate with every variety of it.

And just as closures are always introduced with a lame “counter” example, so recursion is introduced with fibonacci numbers. This is a really good bad example, because it shows not only how recursion can be the easiest way to express something, but also that recursion can spin dangerously out of control if it is mishandled.

Anyways, as you study recursion, it often takes a lot of trial and error to get it right. You test your function with several arguments to make sure it’s coming out right. You add in code to print stuff out so that you can get a feel for what’s going on. You take out the text code when you think you’re done and you put it back in when you find a new problem you need to fix. There has to be a better way. Wouldn’t it be great if we could write code to help us write code? Sounds like a job for a macro!

What we’ve got for show and tell in today’s code example is a tool called memoization. When you memoize a procedure, you create a closure that contains the procedure and a hashtable to go with it. Each time you call it, the closure checks to see if the argument is already in the hash table. If it is, it returns what’s in the hash table instead of running the potentially time consuming code to determine the value in the usual way. If the argument is in the hashtable, the closure finds the value using your procedure and then stores it in the hash table. In the case of the typical fibonacci function, this results in a dramatic performance increase.

Looking at the code, we have two defuns, a defmacro, and a macro call. The testing-memoize function takes a function as an argument and returns a closure. The print statements we use to show us what our recursive function is doing are here. We also added a “back door” to allow us access to the hash table that is attached to the function. The other function, tree-substitutes, is a recursive function that searches a hierarchical list for a specific symbol and splices in a list of things in its place. The macro takes the same arguments as a defun, but sets up a parameter to hold the closure, defines the a slightly modified version of the function using the tree-substitutes function, stores the closure in the parameter, and then redefines the function to call the closure instead. Finally, we have an example of how to apply our memoization macro with our ubiquitous fibonacci procedure.

If we like the behavior of our recursive function, we can turn off the memoization by changing “memoize” to “defun” in our procedure definition. If we like the performance gains and want to keep them, we can create our closures with a different momoization technique that doesn’t print out diagnostic text or expose a back door.

So, we can see that closures give us a means of abstracting away any extraneous or temporary behavior in our procedures. These techniques don’t replace subroutines and libraries, but they do provide an additional means of eliminating redundant code. We also see that macros can be used to set up all kinds of behind-the-scenes code by giving us a means of establishing additional ways of defining functions. We could potentially set up “testing” and “production” variants of our definition macros to radically change the behavior of our application by switching between which package we reference.

Hopefully you now have a better idea of what you can gain from studying functional programming, closures and macros. We’ve only scratched the surface, of course. If you’d like to know more, I highly recommend the classic Structure and Interpretation of Computer Programs. If you’d rather learn from a cool Perl hacker (instead of a “smug Lisp weenie”) then you might try Mark Jason Dominus’s Higher Order Perl instead.

(By the way, if you take a peek at the hash table after running the fibonacci code, it makes a pretty picture for you at the REPL prompt.)

The Future of Programming

October 4, 2007

“84 months worth of work was reduced to 2 months, and the results were error free.” It’s stories like this that fire our imaginations. Whether it’s Paul Graham fixing bugs while the client is still on the phone or Peter Siebel’s dad finishing a project with only half a budget, we want to know the secret.

And what were some of the attributes of this  latest successful project? “I didn’t have to code the changes for each machine; it would create what was needed from the machine specifications…! I didn’t need a new release, all I needed to do was apply my new business rules to the existing system…! This is what development was supposed to do for us.”

I could see some of this. Configuring systems with tables stored in a database or in XML files is not enough. Each installation is different… and if enough are them are different enough that we’re forced into making changes in the actual code, we’re hosed. We get sucked into an endless treadmill of patching, redeploying, gathering more requirements, putting out fires…. Source Control, Unit Testing, Agile techniques, and good coding style all contribute to making this somewhat manageable. But they don’t address the core issues. They will eventually fail us on the interesting problems.

My own little toy project, though fun, is deficient not only because of its amateur technique, but rather because it works against the grain of the language. It did achieve a moderate level of configurability via a human readable configuration language, but it was accomplished in a brute force manner… and extending the new language is not a lot of fun. While we managed to abstract away the essence of the generation definitions, we nevertheless violate the closure principle: we’re frozen at a single “pretty good” level of abstraction. And unlike a true embedded language, my custom language does not benefit so much from the features of the parent language.

How can the success story be recreated? Quoth the hacker, we need “grammars to read and execute specification files….” This, of course, points back to Norvig’s deceptively simple PAIP chapter 2… a theme that sets the tone for his entire book.

To say that we are going to invent custom languages on a problem by problem basis is misleading. We’re going to be extending existing languages in expressive ways– without burning the bridge back to the parent language’s core idiom. As pico explains, “a DSL isn’t really writing a new language, but rather manipulating an existing language to define your problem, or domain, in a more natural form. It’s designing objects and writing methods that isolate the problem and illuminate your business rules.”

We are not “true believers” in any single programming language, but we recognize that some languages are friendlier to our creativity than others. As far as is possible, we will not allow any language to limit our imaginations. And we will code solutions to problems that chafe at the constraints imposed by relational and object-oriented assumptions. Still, the question is not whether or not to write DSL’s or embedded languages, but when.

Closures, Objects, Forklifts, and the quest for truth

July 23, 2007

My first post on Closures has so far netted 2,235 hits. My second one took in 1,346 while my CodeMunger post has picked up a similar 1,338. The majority of this traffic has come in from Reddit with a smaller chunk coming in from YCombinator’s Reddit-like “Startup News” page.

I wrote these posts because the subject matter was interesting to me. I really had no idea that they’d be read this extensively. (At the time I wrote them I was planning on increasing my readership by doing a few more Emacs articles in order to pick up an extra 10 or 20 hits a day from Google!) The response was largely positive, due in no small part to the timing of Joel’s recent piece on the general “dribble of morbid, meaningless, thoughtless comments” that blog entries so often provoke. There was one guy, however, that was so angered by my overall stupidity (and the fact that I’ve introduced a lot of noise into the search engines) that he needed to go into graphic detail about the violent things he wanted to do to me. (Dude… I’m not the one that posted the article on Reddit. Sheesh.) I have felt pretty lame for posting my ideas to the web anonymously– it’s rather cowardly– but given my unanticipated infamy I think I’ll continue to hang back for a while.

At the same time, traffic such as this does not necessarily equate to readers. It certainly doesn’t mean thoughtful attention. Even Sean Ross’s balanced and well stated post seems to miss the things that I think are key to the discussion:

1) The entire point of the closure/OOP articles is to explore what Paul Graham meant when he wrote on page 2 of ANSI Common Lisp, “With macros, closures, and run-time typing, Lisp transcends object oriented programming.” So yes, I’m writing more from a explorative standpoint. I haven’t yet come to a conclusion about what I think about these things– but these first insights and ah-ha moments that I’m getting by pursuing this line of thought have been more fun than any other programming exercise I’ve ever undertaken.

2) And no, it’s not about closures vs. CLOS or anything like that: closures and objects appear to have an unusual tao-like relationship. And OOP itself is such a slippery concept that the idea that there is “one true implementation/approach” is… well… unrealistic. (And it’s pretty cool that you could, if you wanted to, implement in Lisp whatever vision of OOP you prefer. You don’t currently have the freedom to go the other way in the current reigning OOP languages.)

3) My 7 years of professional life have been predicated on the assumption that OOP should be the fundamental to all that you do in your programming labors. Some really smart people have done just fine without OOP… and see no need to incorporate it into their designs. How is it that folks like that are solving problems with closures and so forth? Are they imitating OOP techniques when they solve real problems? Or is it the other way around? Do advanced OOP programs tend toward having a veneer covering a hacked-out typeless stew of ugliness that belies the core premises built into the design of their languages?  What’s going on here?

And this brings me to Miklos Hollender’s quiet plea for sanity. Am I missing something important in my current quest for beauty and truth? Yes our tools might suck and our languages might hobble us… but shouldn’t I find a zen like contentment in my professional life in spite of all this? My answer to this is two part. One: if you are a professional programmer and you lack a fluency in the core concepts of real computer science, you ought to be doing something to remedy your ignorance. (And if ignorance doesn’t make you feel stupid, then there’s not much anyone can do for you.) Two: normal people in any other line of work would be angry if they were forced to do a job with markedly inferior tools. Programmers should not be content to carry water with pails and a yoke when forklifts and fifty gallon drums are available. While indeed there are many zen-like pleasures to be had in the simple things of life, maintaining badly architected code is not one of them. (At the same time, Miklos probably has a good point that meeting our clients’ needs with tools that are reasonably suitable for the task may not be “fulfilling”… but there’s no reason it can’t be a pretty cool job that we programmers execute well in spite of some of these “academic” issues.)

Nevertheless… there are questions that seem to have gotten lost in the curriculum and tool-kits and language wars. I, for one, would like to look into them. Of course… the next time I post something that difficult and controversial that I can expect to get page views in the thousands… well… I just might take the time to clean it up a little more. (And maybe avoid such shameless hyperbole, too.)

Closures + Lambda = The key to OOP in Lisp

July 18, 2007

As Truls Becken remarked in a comment on a previous post, “when it comes to encapsulation, the very foundation is that all state must be accessible only from inside the object’s methods.” On the one hand, it was loads of fun for us to be able to add methods to our “obi” on the fly at run-time. But on the other hand… those methods would have no clue about the state of the object. It seemed like a fatal blow to my approach. Maybe closures weren’t the key to home-grown object orientation in Lisp after all….

I puzzled over this for a while and tried to think about how I could solve this. I’ll never know if I could have figured it out on my own or not because I happened to pick up Peter Norvig’s Paradigms of Artifical Intelligence Programming last night. Chapter thirteen is all about objects and it just so happens that closures are the key to making things happen. Unlike Paul Graham, who began his OOP example from ANSI Common Lisp with inheritance (and multiple inheritance!), Norvig emphasizes encapsulation (via closures) as the cornerstone to his OOP design. If you ever wondered about the phrase “Lambda the Ultimate,” well… this is one example that illustrates why lambda really is the ultimate… uh… thing.

Here is a proof of concept of from the REPL prompt demonstrating that we’ve solved Truls’ problem:

CL-USER> (setf example (create-obi))
#<COMPILED-CLOSURE CREATE-OBI-1>
CL-USER> (setf setter (funcall example ‘set-size ‘() ‘()))
#<COMPILED-CLOSURE CREATE-OBI-1-1>
CL-USER> (funcall setter ‘huge)
HUGE
CL-USER> (setf shower (funcall example ‘show-size ‘() ‘()))
#<COMPILED-CLOSURE CREATE-OBI-1-2>
CL-USER> (funcall shower)
The size is HUGE
NIL
CL-USER> (funcall setter ‘extra-large)
EXTRA-LARGE
CL-USER> (funcall shower)
The size is EXTRA-LARGE

As you can see, the variables referenced in the set-size and show-size methods are the same… and they are protected from general access by the closure.

And here is my current code file. You’ll see there that I’ve been experimenting with reflection and a recursive approach to evaluating long chains of object references. Nothing earth-shattering, though. Yeah, I know… I still need to add class definitions and inheritance into it. And I know that I’ve got at least three conflicting strands of thought emerging there right now and that a lot will have to change in the coming iterations. If you really want to see how to do things right, go read Graham and Norvig.

But what I really wanted to say about all of this is that I’ve been puzzling over some of Paul Graham’s remarks for some time. How is it that closures can be so powerful? When I wrote my last post on closures and OOP… it had just come to me in a flash of insight that you could do stuff with closures. (It wasn’t really written to change the world or tell people how they should do things… and I was shocked that it was even read at all, much less taken seriously by anybody.)

And macros. Yeah… I could see that you can define looping constructs and so forth… and that such things couldn’t be done with a mere function. But my question always was, when will I ever want to do this? I hear lots and lots of people on the web droning on about their lisp-illumination, but I don’t see so much practical advice on how to get illuminated and what you can do with your new powers once you have them. I know what the Blub paradox is… but… how do you get out of it? Well for me it began when I realized that I could implement my own vision of OOP in Lisp if I wanted to. (That’s something that’s impossible in my professional toolset, now. And as a result, I think of that as a sort of thing that only ‘real’ programmers with hairy chests can ever even think about futzing with.) And as I first started experimenting, I saw that there was going to end up being a lot of repetitive code to get classes to work right. And then it sunk in… that I could write some code to write that code for me. And it would of course be macros that I would use to do that. And I could make things as elaborate as I wanted and all the work would be done at compile time… my constructs would hardly impact performance at run time.

And that leads inexorably to a third Ah-ha moment. With lambda, closures, and macros… I can build not just my own implementation of OOP… but I can build anything. I finally see what Alan Kay meant when he said, “Lisp isn’t a language, it’s a building material.” Lisp is amazing not because of what the Paul Grahams of the world do with it, but because it allows average programmers to solve problems that they thought were only solvable by geniuses.

So anyway… if you want to get illuminated… what do you do? Stay away from the web a bit. A lot of it will just confuse you unnecessarily. Stick with the very best books on Lisp… and tackle the hardest problems you can think of. You know, the one’s you’ve always felt were above your abilities. It will be like starting to learn programming all over again. But the sinking feeling in your stomach that you get at that realization will be offset by the fact that programming will again be as fun as it was when you were first learning.

If you are bored with your current set of tools, languages, and projects… that’s a sign that it’s time for you to start learning Lisp.

—————————————

Related Articles:

Closures + Hash Tables = As Much OOP as You’ll Ever Need

Closures, Objects, Forklifts, and the quest for truth

Closures + Hash Tables = As Much OOP as You’ll Ever Need

July 9, 2007

“In another thirty years people will laugh at anyone who tries to invent a language without closures, just as they’ll laugh now at anyone who tries to invent a language without recursion.” — Mark Jason Dominus

You might be wondering what all of the talk about closures is about, really. You might be scratching your head at the boring examples that just run a simple little counter inside a function. You might be wondering how functional programming can be more fundamental than (and even able to subsume) object oriented programming. You might be disappointed that Mark Jason Dominus hasn’t gotten his book up on-line, yet. You might be willing to settle for a quick dumb example in Common Lisp written by an average developer. If any of the previous guesses are right, you came to the right place.

In this text file, you’ll find a mere 16 13 lines of code that define a small chunk of OOP in Common Lisp. The code should not only be understandable to beginners, but it was actually written by a beginner– so it should also hopefully give an indication of how easy it is to build “new” fundamental-ish programming concepts onto Common Lisp.

An “obi,” as I define it here, is somewhat like an object, but there are some key differences. (I call my version of an object an “obi” because I’m implementing just a very small piece of OOP.) An “obi” is a hash table wrapped up in a closure. The closure provides one of the main pillars of OOP: encapsulation. Nothing in your program can mess with the variables inside of closure without interacting through the function call that the closure provides. The hash table provides a place to store all of your properties and methods. Because Lisp is dynamically typed and because functions are first-class objects, there’s really no need to make a difference between properties and methods. In an “obi,” functions are just one of the many information types that can be stored in the hash table.

In the example below, we see an obi being used from the REPL prompt. Its color and shape properties are set and also a function called add is put in and called. The “.” is used in Common Lisp to notate a dotted list, so I used ~ to represent “get”, ~> to represent the create/set a property/method, and ~! is used to represent a call to an obi’s function:

; SLIME: The Superior Lisp Interaction Mode for Emacs
CL-USER> (setf f (create-obi))
#<COMPILED-CLOSURE CREATE-OBI-1>
CL-USER> (~> f ‘color ‘red)
RED
CL-USER> (~> f ‘shape ‘square)
SQUARE
CL-USER> (~ f ‘color)
RED
T
CL-USER> (~ f ‘shape)
SQUARE
T
CL-USER> (~> f ‘add #’+)
#<SYSTEM-FUNCTION +>
CL-USER> (~! f ‘add ‘(1 2 3 4 5 6 7))
28

Obi’s lack any form of “type”– in fact, you can add additional properties and methods to them at any time. Even if you don’t intend to use Common Lisp in a production application, this is an option that should be in your mental tool box. You may not always need to resort to using inheritance or subclassing to create a variation of an object you defined. If you think a little bit more functionally, you can abstract out the bulk of the design and then allow other developers to customize its use by setting or overriding some of the key functions. Of course, in a wide-open typeless setting, that’s a feature you get whether you think you want it or not!

For a much more elegant and substantive example of how to implement OOP in Common Lisp with an insanely small amount of code, see Paul Graham’s Ansi Common Lisp.

PS If anyone knows how to trick WordPress into letting me display Lisp code without destroying the indenting, please let me know.

PPS Looking at the search engine traffic coming in here, I see that (besides all of the people trying to figure out carriage returns in Emacs) there are a few coming to the site for stuff like “lisp closures,” “oop in lisp” and “lisp hash tutorial.” Hopefully this post will provide a little more substance for those folks than the previous drivel I’ve spewed forth here in my clumsy efforts to get used to some of these new ideas.

PPPS Please forgive the hyperbole in the title.

——————————

Related Articles:

Closures + Lambda = The key to OOP in Lisp

Closures, Objects, Forklifts, and the quest for truth


Follow

Get every new post delivered to your Inbox.