I ran across this recently:
“In fact the problem of ‘boilerplate design pattern code’ says nothing about the quality of the language; it says only that the programmer are working at the wrong level of abstraction and are the victims of that all too common sickness: bad design. Picking another language is not the cure for this disease. (More than likely it’s only going to make things worse!)”
This is just plain wrong– except for the part about the programmer working at the wrong level of abstraction and being a victim of “bad design.” The question is, how much of that has been foisted onto the programmer by the design of the language he’s writing in? And if much of it is due to the language, then it’s certainly not off base to go shopping for a new one!
If you’re not clear on this issue, then you need to watch SICP Lecture 3a: Henderson Escher Example. Early in the lecture (00:05:45) he points out the importance of closure in any means of combination in a programming language. (Now this isn’t about “closures”, now… this is a more general mathematical principle.) Anyways, Abelson says, “Closure was the thing that allowed us to start building up complexity…. The things that we make… those things themselves can be combined…. to make more complicated things…. When you look at a means of combination, you should be asking yourself whether things are closed under that means of combination.”
Obviously, it’s much better to be able to make an array of arrays than it is to be restricted to only storing numbers or strings in them. Whenever the principle of closure is violated, you’re going to be limited in your ability to formulate useful abstractions. Just like you lose the ability to express certain ideas cleanly (or even at all) when functions are anything other than first class citizens in your programming language. Programming languages really do have significant differences; they’re not all the same.
After going through his example step by step (and incorporating everything covered so far in the course), he (01:02:37) sums up why the ability to create embedded languages is so important: “That’s the important point: the difference between merely implementing something in a language and embedding something in a language so that you don’t lose the original power of the language. Lisp is a lousy language for doing any particular problem; what it’s good for is figuring out the right language that you want and embedding that in Lisp. That’s the real power to this approach to design.”
I think a lot of people get the heeby jeebies when they hear Lisp programmers talking about creating new languages to solve problems. They’re thinking, “if people start writing their own macros, how will I understand the code when I have to maintain it?!” Well, yeah. That’s only going to be a problem if you don’t know how to use macroexpand to show what’s going on “under the hood.”
But when you look at what’s going on in the example from the video, you can see that the process of implementing languages to describe what’s going on actually makes the code much easier to understand. The languages are implemented at each level of abstraction. Each language doesn’t “care” how the lower level languages are actually implemented– you can think at each level in terms of that level without being confused by unnecessary detail. And each level can be manipulated with and integrated with all of the usual idioms of the Lisp language itself. It’s not like you’re randomly inventing a new Python or Perl depending on the problem you’re solving. You’re making languages that are fully embedded in Lisp… and there’s nothing keeping you from utilizing other Lisp tools, techniques, and elements with these language layers.
So don’t let all the talk about DSL’s and embedded languages scare you. Expressive code can actually be easier to understand, extend and maintain.
Update 10/2/07: “discipline and punish” responds with Programming Languages, DSLs, Static Typing, and the Answer to Life, the Universe and Everything.
Learning Lisp 
October 1, 2007 at 7:38 pm |
This is quite interesting, because if you look at the discussion of how to use abstraction, inspired by the BSD kernel, on the “Beautiful Code” website (http://beautifulcode.oreillynet.com/2007/09/abstraction_and_variation.php) you can see what happens when you don’t have the power to do this kind of macro-based abstraction…
October 1, 2007 at 8:42 pm |
Just to clarify: no macros are being used to create new language elements at this point. It’s all just recursion, higher order procedures, and constructors/selectors implemented with cons, car, and cdr.
October 1, 2007 at 8:43 pm |
I agree that embedded languages make code better. The challenge is naming things well and using relavent terminology to the language. Having been trained in traditional CS I still feel “brain damaged” by the emphasis on how the machine works. There is a strong temptation to adapt myself to the data, API, and language system I’m dealing with. That’s how I was implicitly taught. I don’t just blame my formal education for this. All my years of programming have essentially taught me the same thing. The challenge is in thinking “What language do I want to create?”, and only thinking in those terms, almost considering the existing language and API as background noise not worth considering, unless it’s semantically relavent.
Since I’m working in Smalltalk I’m also wondering “What classes should I create?” According to Message-Oriented Programming, what Alan Kay would’ve preferred to call it instead of OOP, each class will basically have its own language semantics via. messages. It can use the same language, but it may interpret it differently.
Something embedded languages emphasize is knowledge of the problem domain. From looking at Smalltalk code I noticed it’s sometimes hard for me to understand what’s going on because I’m not familiar with the domain perspective that the original developer was coming from. I’d have to steep myself in the domain to understand what they were talking about. This is probably no different than looking at just traditional code. Even though the terminology would be pretty familiar (the language itself, like C, C++, etc.) I wouldn’t understand what was going on unless I understood the domain, anyway.
October 2, 2007 at 2:42 am |
Section 2.1 in SICP gets very OOP like– just without any classes. The overloaded constructors and multiple “property” interfaces they train you to do in the exercises there are very similar to what I reflexively do in C#. I see that my OOP habits are in some cases preventing me from formulating more broadly defined abstraction layers, though. I know that many of my OOP designs have violated the closure principle descibed above, and I’m itching to re-implement some complex code in the SICP approach to take advantage of my new found combinatorial freedom– but I know I’m only just now scratching the surface in what can be done. I suspect I really need to have my mind blown several more times before I do that. I’ve got to keep an open mind and let them follow their point-of-view a few more levels down so that I don’t waste more precious development time on mediocre approaches.
But I do see a better way to do things that I’ve tried and failed in the past…. I knew I was struggling with things that were over my head– and SICP is the only resource I’ve yet found that can actually train you to tackle “hard” stuff. It is packed with terminology and techniques that help you break apart difficult things into beautiful solutions.
October 2, 2007 at 6:27 pm |
I can see part of ocean’s argument: that tons of horrible boilerplate code is evidence not of language failure but of design failure.
One of the things I hated about Java, besides how often I seemed to have an int when I needed an Integer or vice versa, was how hard it was (how much keyboard typing it took) to do something simple like instantiate an object.
FactoryConfigurator factoryConfigurator = (FactoryConfigurator) ConfigurationEnvironment.getFactoryConfigurator;
factoryConfigurator.configure(configurationObject);
DesiredObjectFactory desiredObjectFactory = factoryConfigurator.getDesiredObjectFactory;
DesiredObject desiredObject = (DesiredObject)factoryConfigurator.getInstance();
But in a way, this isn’t Java’s fault. This is a problem caused by someone who decided that whether or not you needed a factory, or a configurable factory, or some system-wide abstraction for resources defining configuration, those were exactly what you were getting. That’s the problem. Looking back, why couldn’t I just type DesiredObject.new(); if that’s all I wanted? All the crap above could get implemented in some other class method if I really wanted it.
When I wasted a year of my life trying to hack a J2EE application together, I cursed the way all the application frameworks were desined to so that everything had to be typed three times, twice in code and once again in XML, and how even when I didn’t have a layer for something, I had to create a phantom layer that did nothing but relay messages across the void. But again, is this horrible verbosity Java’s fault, or is it a failure of imagination on the part of the framework designers?
I have to say, though, that I’m not going back to Java any time soon.
October 8, 2007 at 3:05 am |
[…] Lisp (notes from an average programmer studying the hard stuff) « Boilerplate code says a lot about a language… We are in an echo chamber… […]