Lisp Isn’t Really a Programming Language

In Lisp, DSL’s are free. In fact, practically any program written in Lisp could be considered to be a DSL. First Class Procedures are a big part of this. Any procedure you write becomes a part of the programming language– your procedures are indistinguishable from the built-in ones. You don’t write Lisp code so much as you extend the Lisp language until it becomes a custom language for whatever it is you’re trying to do. Its a subtle difference, but a significant one.

Car, cons, and cdr can be used to define almost any data structure. You create an interface for your data structure with procedures– just like the constructors you write for objects in that OOP language you use at work. Recursive functions using car, cons, and cdr can perform just about any manipulation or transformation with your custom defined data structures.

A dispatch table is a list of names and their corresponding function objects. If you were to create one for another programming language, you could read a text file and instantiate variables and functions from it. You could set up functions that would allow you to load other text files. With a dispatch table of function names (and their arguments) that’s read into your program sequentially, you get an imperative programming customization language for free that allows your users to customize the program environment as much as they like. Languages that have first class functions make implementing dispatch tables trivial.

And Lisp of course has a dispatch table that’s wide open for you to play with however you like. That’s where all the functions you write live… and their names are symbols that allow you to look them up and reference them. And because Lisp allows you to mess with its dispatch table, you can redefine it even while its running. I mean you can actually redefine the language itself as your application is running. It is this feature that allows people to debug programs running on satellites travelling to distant planets… and also to debug programs before a client can hang up on the phone to report it to you.

Now there’s an interesting thing about those data structures you make with car, cons, and cdr. They’re an abstract syntax tree. Now in most other languages, if someone wants to do something cool, they convert what they’re working with to an abstract syntax tree and then go to town with it. If you want to convert something from in-fix notation to something more lisp-like, its trivial once you can get that original notation into an abstract syntax tree. But the thing is, if you’re using lisp, you’re using lists. And Lisp’s lists are essentially abstract syntax trees by definition. So in Lisp, a lot hard things are easy… by default.  Finally, your Lisp code is itself written in terms of list data structures. This means its easy to write code to execute transformations of your code. This is what macros do and this is why other languages can’t do this.

Now, you may not like the conventions of the Lisp language. But the thing is, the Reader itself can be redefined. It’s just as wide open as the dispatch table is that you store all of your functions in. This means you can program the reader to transform just about any programming language text into actual Lisp code… and from there do anything else you can imagine.

So Lisp isn’t really a programming language. It’s a consistent system for defining programming languages in general. That is why learning it is difficult– you’re going up into a level abstraction that you never imagined could exist. But if you can master it… you can code in the environment that all the other languages “hack” into when they need to get real work done. So in some sense, Lisp is the Platonic ideal that all other languages fall short of.   It is the truth that most programmers glimpse only in brief moments of lucidity. All the other languages are shadows cast on the wall of the cave… but Lisp is is what lies behind the veneer of what we think of as “reality” in programming.