Alright. I misunderstood you. It just seemed like you went right from the ad to implying a host of problems with the job. It left me wondering, “Hmm. Perhaps Lispy is a bit jaded and is venting?” I didn’t follow the links. Perhaps that would’ve filled me in.

I agree with you about ownership of the code and trying to make improvements to the architecture (like using your library) over time. My main obstacle with some jobs has been I don’t have the time to do the project right. The budget is constrained by the customer, so even if I wanted to do it up right and had ownership, the budget doesn’t allow it unless I feel like doing it gratis. On bigger corporate projects I’ve been able to sneak in refactoring as part of bug fixing, but I agree, it has to be innocuous.

I think the reason employers get queezy about developers creating their own languages is the prospect that from then on every developer they hire will have to learn it. I think that’s an expectation most employers should have anyway, but in this age when knowing a popular language is the expected norm in business it can be difficult to get over that hump. I think something that can set their mind at ease is if you make a commitment (this may require stressing the importance of this with your employer) to write documentation for the language, and do it well, so that others can learn it easily. The reason I say “stress the importance” is it needs to be a project task. Unless it’s real simple it’s not something that can be written up in a half-hour.

I also don’t have any problem with the ad or with the people that run shops like that. I was looking for a specific example to give a real context to the discussion. I chose that one in particular after reading though about 6 or 8 postings because it sounded sorta like the kind of pressures that gave rise to Ruby on Rails– and as a bonus it was more or less the same problem that Paul Graham solved back in the nineties up in his attic with a couple of other Lisp hackers. It sounded to me like a situation where custom tool construction even at the lone developer level would pay dividends in a matter of months.

If I was applying for the job, my main question would be who would have “ownership” of the code. Would I be allowed to refactor the code or not?

If I took the job, I would stay in maintenance mode for a while. If I saw repetitive patterns accross the code base, I’d start making a library. I’d review existing tools and libraries to make sure I wasn’t reinvinting the wheel. I’d look at ways of increasing the configurability/adaptability of the classes/routines.

I’d start using the new library on all new development work… and I would consider applying it to the older web pages if I ever had to go into them for a set of major changes or additions. If it’s possible to write Emacs macros to make automate some of the rewriting, I’d do it even if it takes a little longer.

If the library was a big success and if even doing minor maintenance on the legacy pages was a pain, I’d consider sneaking in and changing them some time. If my library eliminated a certain class of bugs altogether, I’d have all the more reason to do this. If I didn’t think I could pull the change off without anyone noticing, I’d hesitate to do this.

At this point I’d look at the code base and start trying to imagine the tool that would make my work even easier. The place to look is… after I done all of the easy refactoring and made my library as powerful as I can… what’s left? Where’s the muck? It’s at this point that we might be entering DSL and/or code generation territory. (If we drink the .Net cool aide, we may even want to look at Workflow foundation here.)

Then we have the phase of dreaming about every aspect of the problem, reading books on the weekends, and building small proof of concept applications. When the idea is hammered out enough, you have a business proposal on your hands: we can hire a “junior” programer to maintain all the old sites if we have this framework– he’ll be able to do so much without very much programming. All the hard stuff is done. I get to work on extending the framework so that we can expand the domain of people that we’re selling to… and we take this business to the next level. Something like that.

The tool vs. “not tool” dilemma is not really a big problem. We should always have a “tool” mindset even when we’re just trying to get stuff done. The framework can slowly grow over time without a huge investment of time all at once… and it doesn’t have to be implemented until it can justify its existence.

Though I’m not really in the mood for defending .Net I feel the need to correct your premise that a job that requires a programmer to work in C# and VB.Net somehow makes it impossible to integrate the two. This is not the case. So long as they’re compiled into separate assemblies one language can use the other’s stuff very easily. That’s one of the things .Net was designed for.

Your examples of “what are we going to tell the developer?” are based on reality for many shops, but I don’t see the connection to the ad. A lot of programming ads are like this, even if they encourage best practices.

I think your argument boils down to computing architecture (in software); use what’s appropriate. ocean’s basic point is “don’t reinvent the wheel”, I think. Both are valid points, IMO. The thing is our industry has a screwed up way of determining what’s the best architecture for a job. By architecture I mean the language, the VM, and any frameworks that are used with it. Often what gets picked is out of what’s popular, and that’s the sole criteria, not what’s most appropriate for the technical problem. You know as well as I do that the web architectures most places use is overly complex and I think it’s fair to say “rickety”. It’s improved from the days of Perl and C++, but the challenge is still how to keep track of session state. Seaside running in Squeak/Smalltalk handles this well by almost totally eliminating the need for the programmer to keep track of session state. Ruby on Rails does a good job of removing the complexity of modeling a database app. on the web. Though I haven’t reviewed them, I’m sure there are Lisp web frameworks that do a good job as well of managing these things. Here’s the rub–hardly anyone uses these technologies. So what you get is the result of the “worse is better” mentality. So while “don’t reinvent the wheel” is valid, I think we need to be looking at the quality of the wheel before we make that argument. If the “wheel” we’re using is squares on axles, and we’re using that because that’s what everyone else uses, then we deserve what we get…

I think that a developer’s job is to evaluate existing tools and create custom tools as needed, taking into account the scope and lifetime of the project. There’s a lot of leverage that can be had by creating a language or a framework, and I think history has shown that we shouldn’t leave tool creation solely to the tool companies.

I’m sorry if I took your comments out of context, but I’m not so sure that I did.

You’re repeating the idea that making tools is about “solving problems that will change tommorow doesn’t make much sense.” If people find themselves doing the same task a hundres times a day, they will come to us developers for help. If we do the same task fifty times in a row, we start working on tools to help take the work out of doing that type of task. That’s far from “willy-nilly” and not a case of YAGNI– especially not when your trying to sell a hundred more such solutions to people with similar problems.

Yes, it took a huge amount of effort to come up with Java and VB. Java was made to “solve” certain problems with C++. VB was made to make it easy to write Windows applications without having to use massive numbers of obscure API calls. Huge undertakings. But that doesn’t mean that writing an spreadsheet style expression language for customizing one of your apps is going to be so tough. It doesn’t mean that your only choice for configuring your application is to have a crapload of tables in the database. And it doesn’t mean that there aren’t programming languages, tools, and environments that make constructing new language elements easier.

If we aren’t investing in creating new tools and languages, we’re painting ourselves into a corner and creating a maintenance nightmare in the process. We need every abstraction technique we can get our hands on to stay on top of this.

The question here is not about the stupid strawman of whether developers should *ever* create their own tools and languages. I even point out the conditions under which it makes sense to write your own DSL. The question is about whether creating new tools and language is the sole purpose or “highest dream” of software development as Warfield suggests. And, as I suggest, the reason we don’t go about creating tools willy nilly is because (1) common sense. Spending a lot of time solving problems that will change tommorow doesn’t make much sense. (2) As engineers our principle tactic is to take new problems and reduce them to previously solved problems. This is what engineers actually do and in practice it has very little to do with inventing new paradigms and exercising our creative juices.