Archive for January, 2008

A Quick and Dirty Look at Exploratory Programming in Common Lisp

January 31, 2008

As the uproar subsides over the recent release of Arc, I can’t help but conclude that the folks that are most outraged by the lack of Unicode support would indeed have little interest in the results of Paul Graham’s labors. Their core values and world view are so antithetical to Paul’s, that they simply cannot see the point. You don’t believe that to be true, of course. You just think Paul Graham is stupid. It doesn’t matter that he wrote On Lisp or that guys like Peter Norvig have a few nice things to say about him. You know he’s insane… and somehow, you’re hurt, insulted, and angry all at once. This is understandable, really. Lisp really is sort of an alien technology. The more you use it, the more it infects your brain. You get all wrapped up in what suddenly appears to be the right way of doing things. But this explanation is meaningless to you. You shrug it off and continue to justify your rage.

But at least give me a chance to make one brief analogy. You know how it is with your wife/girlfriend. You want to be nice, so you offer to take her out to eat. You ask her where she wants to go. She says she doesn’t know. You say you’ll take her anywhere she wants. She insists that you pick somewhere that you want to go. You pick a place that you think would be nice– and she immediately says she doesn’t want to go there. You get frustrated and annoyed. Her feelings get hurt because you can’t hide the sharpness in your words… and suddenly you’re stomping off to avoid the litany of complaints that are sure to follow. “You never do x; you always do y; blah blah blah.” As you mope in your computing dungeon, you feel that something significant has transpired. You feel like there was no way you could have won, yet you’re sure you missed something. You, my friend, are now in the dog house indefinitely.

And that’s the truth of it: you live with an “alien.” You look at the same events and view them differently than she does. You think differently. You have different needs. The “alien’s” rationalizations seem incredibly illogical to you… and you just can’t understand it. The same thing is going on between you and Paul. (Not the living together part. You know… the misunderstanding part. It’s an analogy. Stay with me here for a second.) More than likely you’d rather justify your confusion and disappointment. Certainly, sifting through the kibble to find the meat is the last thing on your mind. But I’m going to give you the benefit of the doubt. Perhaps if you had a little more concrete explanation of how Lisp enables exploratory programming… maybe then you would cut Paul some slack. (Just a little, maybe.) Here goes….

Lisp allows you to define symbols that reference atoms and lists. This allows you to create a shorthand for whatever it is that your working on. Below we create a symbol that represents an x-y coordinate and then reference that list in another symbol:

CL-USER> (setf a '(4 2)) 
(4 2) 
CL-USER> (setf b (list 42 a)) 
(42 (4 2))

Now one thing you should remember… that really is a reference to ‘a’ “sitting” in b’s list. If I make a change to ‘a’, then ‘b’ will change, too!

CL-USER> (cadr a) 
CL-USER> (setf (cadr a) 
(4 7) 
(42 (4 7))

I don’t know about you, but in my mainstream OOP language of choice, my tendency is to start with my own Point class in its own code file. Next I’m coding up a seperate Widget class in its own class file. It has an integer property and point property. I code a variety of constructors in both files and code the getters and setters for the properties of each. Then I start feeling guilty about my Widget class because it violates the Law of Demeter, and I want to go back and put a wrapper on the point property so that no user of a Widget object will reach through to the methods and properties on its internal Point object reference.

Notice above, that we could define a getter function with the cadr operater to retrieve the y coordinate of our point. Setf could then be used with that getter function to make an instant setter. So in lisp… once you define a getter, you automatically define a setter, too. This may seem trivial, but as your object model evolves, I sure get tired of mucking around with property definitions in a half dozen files. In Lisp, all of that stuff is optional. You can just jump in and get started. Note also that because lists can reference other lists, you get a “quick and dirty” form of inheritance for free. Sort of. But even better, you don’t have to take any extra effort to write special code for setting up collections and arrays. The list’s hierarchical structure gives that to you for free.

CL-USER> (setf c (list '(x y z) 42 a)) 
((X Y Z) 42 (4 7)) 
CL-USER> (defun tags (x) 
           (car x)) 
CL-USER> (defun weight (x) 
           (cadr x)) 
CL-USER> (defun point (x) 
           (caddr x)) 
CL-USER> (tags c) 
(X Y Z) 
CL-USER> (weight c) 
CL-USER> (point c) 
(4 7)

So above, we demonstrate the creation of property “getter” functions… and also show how one of those properties can be a list of items with practically no extra effort. I know, I know. Strongly typed collections that take advantage of those new fangled generics features are great fun, when you’re prototyping you’re not all that interested in that sort of paranoid defensive restrictiveness. You may not even know that “types” each of your properties should be, yet!

Here we define a symbol that represents a list of our objects. Notice that the list operator is sort of a universal constructor:

CL-USER> (setf d (list (list '(x x) 17 a) 
                       (list '(y z q) 56 a) 
                       (list '(z z z) 73 '(2 1)))) 
(((X X) 17 (4 7)) ((Y Z Q) 56 (4 7)) ((Z Z Z) 73 (2 1)))

We need to instantiate three of our objects and store them in a list? No problem! You just _do_ it…. The dirty thing here is that you don’t have to define any constructors for your objects. Further… you don’t have to have “all” of your object defined or set up in order to test parts of it.

Let’s set up a simple function designed to operate on a list of our objects. (This is a completely arbitrary example.) But wait… we’re not sure what we want to do as we operate on each item. Well… let’s just skip that part and let the caller of the function tell us what to do each step:

CL-USER> (defun do-something (fn &rest args) 
           (funcall fn (car args)) 
           (if (not (null (cdr args))) 
               (apply #'do-something fn (cdr args)))) 

CL-USER> (apply #'do-something #'(lambda (x) (format t ">> ~a~%" (weight x))) 
>> 17 
>> 56 
>> 73

And instead of setting up a variable to hold our list… let’s just use the function with some objects we define on the spur of the moment:

CL-USER> (do-something #'(lambda (x) (format t ">> ~a~%" (weight x))) 
           '((X X) 17 (4 7)) 
           '((Y Z Q) 56 (4 7)) 
           '((Z Z Z) 73 (2 1))) 
>> 17 
>> 56 
>> 73

This may seem weird, but the more you use Lisp, the more natural this becomes. Lists are fundamentally recursive: they’re defined that way, so you tend to write recursive functions to operate on them. Many built in functions will take functions for their arguments, so you tend to do the same thing with your own code. This not only lets you defer some design decisions, but it also lets you build reusable skeleton functions that are broadly applicable. Finally, “rest” parameters give you even more flexibility… and *another* excuse to write recursive functions. It’s trivial to modify a function to utilize the rest parameters, so you end up using it in places you wouldn’t have thought to.

The syntax for what we just did may seem a little hard to keep up with. When do you you use funcall and when do you use apply? When do you use neither and just call the function? It can be frustrating at times, but that’s the sort of thing that motivated some of the changes in the Scheme dialect.

(I know… I could have used a built-in mapping function for the above. But that’s one cool thing about recursion. If you’ve forgotten the syntax or the name of a built-in function, you can generally write a recursive substitute for it in a pinch. This keeps you rolling along even when you are hacking without reference materials or an internet connection handy.)

Anyways, the main thing here is that you don’t have to waste time juggling class files. You don’t waste time writing properties and constructors and overloaded versions of your constructors. You’ll use “getters” as an elementary abstraction barrier of course, but you don’t have to endure constant object instantiation rituals. You just type out the list that represents your object’s state whenever you need a new one. (This means you can write unit tests that are very clean and concise, by the way… and you can even cut and paste them directly into the REPL prompt without near so much fuss and bother as you might have on other platforms.)

Also, you know how annoyed you get with having an IDE reformat your code even when you don’t want it to? Sometimes the strictly enforced “one class to a file rule” can be equally bothersome. I like to group all of my boring “getter” functions together in one place. This helps document the classes’ read/write fields. Also, related methods are kept together even if they belong to different classes. This makes it easier to think about related parts of the program… which leads to better utilities “bubbling” out of the prototyping process. (By the way, the REPL is just plain great. In terms of its significance, I’d say it’s easily up there with Source Control and Unit Tests in terms of how much it impacts your programming. The amount of time developers waste in the compile-crap loop is insane… and it may be that an entire strata of unit tests are necessitated by the lack of a REPL. But that’s a different story.)

Hopefully this little article can give you some insights into some of the “quick and dirty” type things Paul was talking about. This is not advanced voodoo rocket science by any means, but this is stuff you can master fairly quickly as you start programming in the Lisp dialect of your choice.