Closures + Lambda = The key to OOP in Lisp

As Truls Becken remarked in a comment on a previous post, “when it comes to encapsulation, the very foundation is that all state must be accessible only from inside the object’s methods.” On the one hand, it was loads of fun for us to be able to add methods to our “obi” on the fly at run-time. But on the other hand… those methods would have no clue about the state of the object. It seemed like a fatal blow to my approach. Maybe closures weren’t the key to home-grown object orientation in Lisp after all….

I puzzled over this for a while and tried to think about how I could solve this. I’ll never know if I could have figured it out on my own or not because I happened to pick up Peter Norvig’s Paradigms of Artifical Intelligence Programming last night. Chapter thirteen is all about objects and it just so happens that closures are the key to making things happen. Unlike Paul Graham, who began his OOP example from ANSI Common Lisp with inheritance (and multiple inheritance!), Norvig emphasizes encapsulation (via closures) as the cornerstone to his OOP design. If you ever wondered about the phrase “Lambda the Ultimate,” well… this is one example that illustrates why lambda really is the ultimate… uh… thing.

Here is a proof of concept of from the REPL prompt demonstrating that we’ve solved Truls’ problem:

CL-USER> (setf example (create-obi))
#<COMPILED-CLOSURE CREATE-OBI-1>
CL-USER> (setf setter (funcall example ‘set-size ‘() ‘()))
#<COMPILED-CLOSURE CREATE-OBI-1-1>
CL-USER> (funcall setter ‘huge)
HUGE
CL-USER> (setf shower (funcall example ‘show-size ‘() ‘()))
#<COMPILED-CLOSURE CREATE-OBI-1-2>
CL-USER> (funcall shower)
The size is HUGE
NIL
CL-USER> (funcall setter ‘extra-large)
EXTRA-LARGE
CL-USER> (funcall shower)
The size is EXTRA-LARGE

As you can see, the variables referenced in the set-size and show-size methods are the same… and they are protected from general access by the closure.

And here is my current code file. You’ll see there that I’ve been experimenting with reflection and a recursive approach to evaluating long chains of object references. Nothing earth-shattering, though. Yeah, I know… I still need to add class definitions and inheritance into it. And I know that I’ve got at least three conflicting strands of thought emerging there right now and that a lot will have to change in the coming iterations. If you really want to see how to do things right, go read Graham and Norvig.

But what I really wanted to say about all of this is that I’ve been puzzling over some of Paul Graham’s remarks for some time. How is it that closures can be so powerful? When I wrote my last post on closures and OOP… it had just come to me in a flash of insight that you could do stuff with closures. (It wasn’t really written to change the world or tell people how they should do things… and I was shocked that it was even read at all, much less taken seriously by anybody.)

And macros. Yeah… I could see that you can define looping constructs and so forth… and that such things couldn’t be done with a mere function. But my question always was, when will I ever want to do this? I hear lots and lots of people on the web droning on about their lisp-illumination, but I don’t see so much practical advice on how to get illuminated and what you can do with your new powers once you have them. I know what the Blub paradox is… but… how do you get out of it? Well for me it began when I realized that I could implement my own vision of OOP in Lisp if I wanted to. (That’s something that’s impossible in my professional toolset, now. And as a result, I think of that as a sort of thing that only ‘real’ programmers with hairy chests can ever even think about futzing with.) And as I first started experimenting, I saw that there was going to end up being a lot of repetitive code to get classes to work right. And then it sunk in… that I could write some code to write that code for me. And it would of course be macros that I would use to do that. And I could make things as elaborate as I wanted and all the work would be done at compile time… my constructs would hardly impact performance at run time.

And that leads inexorably to a third Ah-ha moment. With lambda, closures, and macros… I can build not just my own implementation of OOP… but I can build anything. I finally see what Alan Kay meant when he said, “Lisp isn’t a language, it’s a building material.” Lisp is amazing not because of what the Paul Grahams of the world do with it, but because it allows average programmers to solve problems that they thought were only solvable by geniuses.

So anyway… if you want to get illuminated… what do you do? Stay away from the web a bit. A lot of it will just confuse you unnecessarily. Stick with the very best books on Lisp… and tackle the hardest problems you can think of. You know, the one’s you’ve always felt were above your abilities. It will be like starting to learn programming all over again. But the sinking feeling in your stomach that you get at that realization will be offset by the fact that programming will again be as fun as it was when you were first learning.

If you are bored with your current set of tools, languages, and projects… that’s a sign that it’s time for you to start learning Lisp.

—————————————

Related Articles:

Closures + Hash Tables = As Much OOP as You’ll Ever Need

Closures, Objects, Forklifts, and the quest for truth

About these ads

18 Responses to “Closures + Lambda = The key to OOP in Lisp”

  1. Mark Smith Says:

    Very nice read. Some good positive insight that I’m sure will be useful to all those new to Lisp. But, what if you’re board of Lisp? I know, it sounds impossible. But after 3 years of thinking Lisp is the end of the line, you realize what tripe that is.

    Lisp is great yes, but you should be weary. After a while everything you’ll think of with be a Lisp. (From a language design point of view.)

  2. lispy Says:

    Thanks, Mark. I’ll try not to drink the Kool-Aide. :)

  3. Peter Seibel Says:

    And macros. Yeah… I could see that you can define looping constructs and so forth… and that such things couldn’t be done with a mere function. But my question always was, when will I ever want to do this? I hear lots and lots of people on the web droning on about their lisp-illumination, but I don’t see so much practical advice on how to get illuminated and what you can do with your new powers once you have them.

    Instead of Chapter 8 (which you linked to) check out Chapters 9 and 24 for two realistic examples of what you might actually want to do with macros:

    http://www.gigamonkeys.com/book/practical-building-a-unit-test-framework.html

    http://www.gigamonkeys.com/book/practical-parsing-binary-files.html

    -Peter

    P.S. It’s also probably worth mentioning that while you *can* build an OO system in Lisp using closures and macros, it’s not necessary because Common Lisp has a wonderful object system built in.

  4. Wynand Says:

    http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg03277.html

    The venerable master Qc Na was walking with his student, Anton. Hoping to prompt the master into a discussion, Anton said “Master, I have heard that objects are a very good thing – is this true?” Qc Na looked pityingly at
    his student and replied, “Foolish pupil – objects are merely a poor man’s
    closures.”

    Chastised, Anton took his leave from his master and returned to his cell,
    intent on studying closures. He carefully read the entire “Lambda: The
    Ultimate…” series of papers and its cousins, and implemented a small
    Scheme interpreter with a closure-based object system. He learned much, and looked forward to informing his master of his progress.

    On his next walk with Qc Na, Anton attempted to impress his master by
    saying “Master, I have diligently studied the matter, and now understand
    that objects are truly a poor man’s closures.” Qc Na responded by hitting
    Anton with his stick, saying “When will you learn? Closures are a poor man’s object.” At that moment, Anton became enlightened.

  5. lispy Says:

    Hi Peter,

    Sorry if I slammed your book there by implication. I’m pretty sure it was your work (and the link to LispBox) that covinced me to give Lisp a try. I grabbed your page there because it was representative (and freely available) not because it was somehow inherently bad.

    To clarify: it’s not about the existing tutorials being bad or anything. It’s that a really good explanation of something that you’ve never wanted to do (and can barely conceive of doing) just won’t connect somehow. It was in the course of working on this OOP exercise that I, for the first time, *wanted* to use macros to solve a problem that I was faced with. It was just an “ah-ha” moment that escaped me for weeks.

    There was something missing in my brain that might have caused me not to “get it” when I started out with your book. I spent a lot of time with Shapiro’s book after that. I think if I take a look at some of your “harder” chapters now, I’d probably get a lot more out of them. It’s hard to pay attention to difficult material when your “core self” is not yet convinced that your brain can apply it to practical problems. Yeah, you’ve got “practical” written all over your book and all… but for some weird reason I couldn’t “hear” the message back then.

  6. Top Posts « WordPress.com Says:

    […] Closures + Lambda = The key to OOP in Lisp As Truls Becken remarked in a comment on a previous post, “when it comes to encapsulation, the very foundation is […] […]

  7. Esteban Says:

    And all this to protect yourself from yourself? The benefits of encapsulation are reaped more simply with lower tech data abstraction:

    http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-14.html#%_sec_2.1.2

    “If you really want to see how to do things right, go read Graham and Norvig.”

    Yes, but start with their recommendations (see the first two reviews):

    Structure and Interpretation of Computer Programs - 2nd Edition (MIT Electrical Engineering and Computer Science)

    Structure and Interpretation of Computer Programs – 2nd Edition (MIT Electrical Engineering and Computer Science)

    Buy from Amazon

  8. Justin Says:

    Hi, can you elaborate a bit on what you mean by “protected from general access by the closure”? In what way does a closure ‘protect’ anything?

    I don’t see how the code examples you’ve provided have much to do with closures. Could you be more precise and explain /what/ is protected from *what*?

    If you mean to say that the insides of ‘example’ are protected, then I don’t see what that has to do with closures.

    Thanks.

  9. lispy Says:

    Justin,

    We have a hash table wrapped in a closure. We can send “messages” to to the closure and get back fields, functions, etc. In my first proto-type I was only returning items from the hashtable when messages were sent to/through the closure. In the example of this post, we see that by using lambda to create functions and return functions that reference the same hash-table… then we can make our own object that can respond to messages by kicking out a family of methods that all access the same variable.

    The variable(s) that the methods reference is not like the usual dynamic or lexically scoped variables that you get with lisp, but the variables are instead protected from being accessed in the usual ways: you can only “talk to” the variables by sending messages to the object.

    Closures are what make it possible to do this in Common Lisp.

  10. Eli Bendersky Says:

    Nice article.

  11. Closures + Hash Tables = As Much OOP as You’ll Ever Need « Learning Lisp Says:

    […] Closures + Lambda = The key to OOP in Lisp […]

  12. Mikael Jansson Says:

    I tried modifying your run, so the hash would be passed on as the first parameter, doing this: (run (apply (gethash field h) (cons ‘h value)))))))

    …which alas, I cannot get to work. How would you do it?

  13. DenkZEIT » Blog Archive » Object Obsessed Programming Says:

    […] I like Closures (somehow reifying’objectifying’ executional state) and the fun and enlightening things you you can do with them (e.g. read Closures + Lambda = The key to OOP in Lisp […]

  14. joseph gutierrez Says:

    Thanks,
    I was able to take your code and create a publish/subscribe. I need to add an array to for adding closures so I can create broadcasts.


    (defvar *event-aggregator* nil)

    (defun create-event-aggregator ()
    (let ((event-aggregator (make-hash-table)))
    (lambda (command event &optional event-handler)
    (case command
    (:can-fire (setf (gethash event event-aggregator) ()))
    (:when-fire (setf (gethash event event-aggregator) event-handler))
    (:fire (apply (gethash event event-aggregator) nil))
    (:listeners (gethash event event-aggregator))))))

    (defun make-event-aggregator ()
    (setf *event-aggregator* (create-event-aggregator)))

    (defun listeners? (event)
    (not (eq nil (funcall *event-aggregator* :listeners event))))

    (defun can-fire (event)
    (funcall *event-aggregator* :can-fire event))

    (defun when-fire (event event-handler)
    (funcall *event-aggregator* :when-fire event event-handler))

    (defun fire (event)
    (funcall *event-aggregator* :fire event))

    Thanks Again!

  15. joseph gutierrez Says:

    oops! here is a unit tests:

    (define-test create-event
    (make-event-aggregator)
    (can-fire :whatever-event)
    (assert-equal nil (listeners? :whatever-event))
    (when-fire :whatever-event (lambda () (+ 1 1)))
    (assert-equal t (listeners? :whatever-event))
    (assert-equal 2 (fire :whatever-event))
    (when-fire :whatever-event (lambda () (+ 1 2)))
    (assert-equal 3 (fire :whatever-event)))

  16. Closures in F# « Angel “Java” Lopez on Blog Says:

    […] Closures + Lambda = The key to OOP in Lisp « Learning Lisp […]

  17. Closures en F# - Angel "Java" Lopez Says:

    […] Closures + Lambda = The key to OOP in Lisp « Learning Lisp […]

  18. Closures en F# | Buanzolandia Says:

    […] Closures + Lambda = The key to OOP in Lisp « Learning Lisp […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: