Lisp Isn’t Really a Programming Language

In Lisp, DSL’s are free. In fact, practically any program written in Lisp could be considered to be a DSL. First Class Procedures are a big part of this. Any procedure you write becomes a part of the programming language– your procedures are indistinguishable from the built-in ones. You don’t write Lisp code so much as you extend the Lisp language until it becomes a custom language for whatever it is you’re trying to do. Its a subtle difference, but a significant one.

Car, cons, and cdr can be used to define almost any data structure. You create an interface for your data structure with procedures– just like the constructors you write for objects in that OOP language you use at work. Recursive functions using car, cons, and cdr can perform just about any manipulation or transformation with your custom defined data structures.

A dispatch table is a list of names and their corresponding function objects. If you were to create one for another programming language, you could read a text file and instantiate variables and functions from it. You could set up functions that would allow you to load other text files. With a dispatch table of function names (and their arguments) that’s read into your program sequentially, you get an imperative programming customization language for free that allows your users to customize the program environment as much as they like. Languages that have first class functions make implementing dispatch tables trivial.

And Lisp of course has a dispatch table that’s wide open for you to play with however you like. That’s where all the functions you write live… and their names are symbols that allow you to look them up and reference them. And because Lisp allows you to mess with its dispatch table, you can redefine it even while its running. I mean you can actually redefine the language itself as your application is running. It is this feature that allows people to debug programs running on satellites travelling to distant planets… and also to debug programs before a client can hang up on the phone to report it to you.

Now there’s an interesting thing about those data structures you make with car, cons, and cdr. They’re an abstract syntax tree. Now in most other languages, if someone wants to do something cool, they convert what they’re working with to an abstract syntax tree and then go to town with it. If you want to convert something from in-fix notation to something more lisp-like, its trivial once you can get that original notation into an abstract syntax tree. But the thing is, if you’re using lisp, you’re using lists. And Lisp’s lists are essentially abstract syntax trees by definition. So in Lisp, a lot hard things are easy… by default.  Finally, your Lisp code is itself written in terms of list data structures. This means its easy to write code to execute transformations of your code. This is what macros do and this is why other languages can’t do this.

Now, you may not like the conventions of the Lisp language. But the thing is, the Reader itself can be redefined. It’s just as wide open as the dispatch table is that you store all of your functions in. This means you can program the reader to transform just about any programming language text into actual Lisp code… and from there do anything else you can imagine.

So Lisp isn’t really a programming language. It’s a consistent system for defining programming languages in general. That is why learning it is difficult– you’re going up into a level abstraction that you never imagined could exist. But if you can master it… you can code in the environment that all the other languages “hack” into when they need to get real work done. So in some sense, Lisp is the Platonic ideal that all other languages fall short of.   It is the truth that most programmers glimpse only in brief moments of lucidity. All the other languages are shadows cast on the wall of the cave… but Lisp is is what lies behind the veneer of what we think of as “reality” in programming.


7 Responses to “Lisp Isn’t Really a Programming Language”

  1. gerel Says:

    Well, I remember reading Graham noting that (that what you actually write in lisp is its own ASTs), it’s true.
    It took me some time to see the power given by the data=code metaphor, and that is actually the highest abstraction level we can get I think. That makes lisp be lisp.

    Lisp is not a programming language, is THE abstraction language.

    BTW, I’d like to se more lisp code in your posts, let’s be more objectives 🙂


  2. lispy Says:

    I probably read that once, too. But it’s different when you “discover” it on your own. It’s the difference between repeating someone else’s words and having Lisp itself reprogram your brain. 🙂 Months back, Graham’s comments sounded impossible to me… now I’m just begining to understand enough to see that he knows what he’s talking about.

    If you’re looking for more code, please check out my sizzlin’ hot memoization example in the next post. It’s probably mediocre by lisp-hacker standards, but even mediocre lisp code can be fun!

  3. gerel Says:

    Yes , I read that memozation code it´s fun.

    BTW, my second “brain reprogramming” was when I learnt Haskell 🙂

  4. Daniel Says:

    Now, don’t take this as a flamebait, but lisp has it’s own restrictions. You can’t really control the interpreter, the way you can with Forth. It’s a very flexible framework, but you are required to play inside the framework.

    With Forth, you can effectively switch to lisp, syntax-perfect, halfway through the program.

    Please, understand this: I’m not advocating Forth. I’m just trying to show there are further levels of abtraction.

    • William O. Yates Says:

      Daniel, you appear to know how to do, what I need to know.
      I wish to use Dr. Moore’s chips, (which have forth machine instructions), but do most code in lisp.

  5. Domain Specific Language in Ruby used as Virtual Machine | Robert Gawron Says:

    […] Lisp Isn’t Really a Programming Language, […]

  6. Samantha Atkins Says:

    Very nice writing. The trouble for me and I expect many is that it is not that we don’t get what is cool, really cool, about Lisp The problem is that unless you are a known Lisp guru or have been at it for a while, it is difficult to program in it for money. Se one gets dragged back to mundane language land. Less happy than before because now one knows that the job hot software technology du jour is really quite primitive and needlessly waste scads of time, brain cells, money and who knows how much brilliant innovation there just was not time for. We were too busy babysitting tons of xml worse than any Cobol baby DSLs fed to some relatively inflexible language with a bunch of clever dumb hackery to make the weaker by design language act half intelligent. Pity the poor slob that has to maintain it (probably you for a while). So your learning Lisp led to dissatisfaction? Naw, I was already unsatisfied. Lisp shows how much better is possible.

    All that said, it does have its real warts. Mainly in the very common now things that are not part of the standard, especially threads/concurrency and sockets and a few other lower level things. Programming in the large is harder to see how to do in Lisp. Not as many libraries is a problem for many. But we can at least pick one Lisp, ideally open source if we want a lot of people to do likewise, that has some of these non-standard bits. If open source and you are inclined dive under the hood to make the core better. Or work on some libraries that you would have liked to have had. Or help test and debug existing ones.

    The world of programming cannot improve all that much imho without much broader use of Lisp or equivalent powered languages (Scheme, not much else).

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: