When I first got started programming I was shocked by the cluelessness of some of the Comp Sci grads that trickled through as interns and entry level coders. A couple were really brilliant, but some never really got up to speed. How could it be that I, with no real training, was coding circles around these guys on the job– even with only my very small amount of experience? Well, one answer to that is that the smart Comp Sci grads were probably all somewhere else doing much more interesting work (and making more money) than what I got stuck with in doing crappy business apps. Another answer is that I was learning a single tool 24/7 and using it to accomplish the sort of things my corporate masters wanted me to do. But Computer Science classes are not vocational training for that kind of work, so don’t make the mistake of assuming that it should prepare people for it.
David Chisnell spells this out for us in a recent article:
“A lot of dissatisfaction with computer science comes from the misplaced expectation that a computer science graduate will be a good programmer. Such a graduate should have been exposed to at least half a dozen languages, but won’t necessarily have done anything particularly complicated with those languages. She almost certainly won’t have any kind of deep understanding of the standard library available for a given platform. This kind of comprehension comes only with experience. She may have picked it up from other jobs or open source work, but not from her degree.”
“Computer science and software engineering are very different disciplines, and a lot of people seem to confuse the two. Software engineering teaches the process of developing software, in terms of both tools and processes. A computer science course briefly touches on these topics, in the same way that a materials physicist may learn something of mechanical engineering. This doesn’t make a computer scientist a software engineer, any more than it makes a physicist the best candidate for building a bridge.”
For all the time and money you spend getting your degree, I sure hope you bypass the kind of creeps us “street programmers” run into when we’re first getting our start. You know… the sort of people that think the answer to everything is to waltz through the internet picking up pirated copies of somebody else’s gigantic business application…. Once you get established, you are still stuck with the terror of out-of-control development way too often. At best you’ll be stuck working with people that refuse to make tools to help them do their jobs– they’re just always under the gun. “The shoemaker’s children have no shoes,” they blithely say. Bob Warfield reiterates what’s going on in these kinds of companies:
“Many times what has happened is a company started out with a great idea and some knowledge of the domain. They built a piece of software that is a very literal embodiment of their view of the domain. So long as the whole domain and all the customers fit their view, life is good. However, for most non-trivial (i.e. interesting market size) domains, nobody knows the whole domain. You learn as you go. Customers throw curve balls. If the only way to adapt to that is to either keep adding tons of features to the software, or do expensive custom work on each deal, that’s way too much friction to scale. A domain specific language makes it possible to maneuver a bit at low cost and give customers what they want.”
There are many ways to kill projects. Easily, the number one killer is going to be scope creep. (The most depressing sort is going to be scope creep caused by management that want a completely new system and then incrementally force you to make it just like their old system.) Well… scope creep wouldn’t be that bad. It’s unanticipated shifts of scope that occur in a direction that’s orthogonal to anything you ever anticipated. Teams will implement awful hacks to keep a client happy enough to keep the money coming in. Then, boom! Their money dries up and you’re suddenly dealing with people in a different region that have subtly different industries that they focus on. And your team can’t adapt to the new needs fast enough to either swing the sale or keep them happy enough to stay with you.
There are few things more depressing than the pain of failure. One of those things is– surprisingly– the pain of success…. What do you do when you’ve got lots of clients running your software– but they’ve all got different versions and different modifications done to it? Untrained “street programmers” can get this far with luck and hard work, but they (and their companies) cannot grow beyond this point without developing tools to help them write their tools. To do that, they need some serious Computer Science know-how. And that’s the bitter irony: the hell of real world development has, on the surface, little to do with what goes on in the Computer Science classroom. But, to solve the problems we face here, we need the help of people that understand the implications of what gets taught there.
Learning Lisp 
December 2, 2007 at 6:05 pm |
Street programmers have access to the same Computer Science know-how as computer science grads. On the Internet and in books.
December 3, 2007 at 5:09 pm |
I think I’m hitting a point where I had a good systematic overview of the major topics would do me good– and also having a prof keep me focused on things even when I get bored with them and start to move on to another random idea. I hear so many bad remarks concerning the current scene in the Comp Sci departments, I wonder if I’m not missing so much… but it is hard being an autodidact….
December 3, 2007 at 7:52 pm |
“The significant problems we face cannot be solved at the same level of thinking we were at when we created them.”
– Albert Einstein.
Nick, you’re right, in that the material is there. You can buy and download books, and get free implementations of most languages and platforms. What “street programmers” often lack is a map through all that. They need to know why they should read books with boring names like “Structure and Interpretation of Computer Programs” and “How to Design Programs,” instead of learning Java. That’s basically the role a professor or mentor plays, someone to tell you where to focus, when you don’t know enough to see for yourself.
Now, all that said…lispy, like you said, many CS departments don’t teach you the real stuff. There are lots of java schools out there; I went to one. I think being an autodidact is preferable to paying money for that kind of education. There -are- schools that still start with SICP, but they seem rarer.
December 10, 2007 at 10:50 am |
Hi Lispy. I can kind of relate to this. I don’t think I’ve worked for a company that’s pirated someone else’s stuff and then added on to it, but I did work for a small software consulting/IT shop more than a year ago that clearly needed some CS expertise. In one case a project had been delayed by a couple years because the company didn’t get the necessary expertise on it earlier. I finally solved the problem and the customer loved it.
One of the problems I had with the job is I gradually realized there was a disconnect between what was expected of me by the company, and what was actually required to get the job done well. The company expected me to use VS’s tools to basically drag-and-drop my way to a solution. In a couple cases this was not possible. They required custom code and a deep understanding of what was being created.
I got onto my track of going back and forth between Lisp and Smalltalk by reading some of Paul Graham’s articles, and reading Eric Raymond’s quote that learning Lisp will make you a better programmer. I think I was attracted to this because I realized that I needed to learn more about software architecture. I had gotten bitten by this in a big way on one project when I got lulled into letting the dev. tool design the app. for me. I figured out the architectural problem eventually, but I saw that I had a blind spot that I needed to rectify.
December 26, 2007 at 8:27 pm |
I agree with your premises, but not with your conclusions. The final state you mention is exactly what you need business and software development skills – the things explicitly not taught to comp sci graduates – to manage through. I don’t think anyone really teaches this in any significant way, it’s something you can really understand only through experience and mentorship, and it’s equally accessible to “street programmers” and graduates. “Tools to write tools” is a hacker mindset, not a compsci one – people with the mentality to do this will do it with or without a graduate degree and indeed, did it before compsci was even identified as a skill.
You might be conflating “street programmer” with “Java codemonkey”, which isn’t really fair – the Java codemonkey only barely qualifies as a programmer, works from a checklist and through the IDE, and gives over all interesting decision making to an “architect”. You can be this person even if you have a comp sci degree (but it’s unlikely, since you probably would have majored in business information systems instead).
If your comp sci major really did include classes on how to manage feature scope, code maintenance, requirements gathering, agile development, scheduling, version control, resource management, deployment, and all the other issues that need solving when you’re in the position you’re describing I’d be interested in seeing a link to the syllabus.
If this was a thinly veiled “compsci grads learn lisp and lisp is the panacea for development problems” post…. well, it really doesn’t matter.
December 26, 2007 at 8:39 pm |
arkanes,
You’re right. It may be that comp sci departments don’t adress the critical “hard” components of real software development– though I really would have preferred to live in a world where people understood this stuff and taught it. Probably too many departmants have gone the route of producing the “Java codemonkies” you describe. So perhaps the title should be… “Why Comp Sci grads can’t hack… and why you need people that understand the implications of the principles that underlie what used to be taught in Comp Sci departments.”
And as to the whole Lisp-panacea thing… I don’t care so much about the language wars– but I do think that looking at the off-beat stuff can give you valuable insights into how to structure things in the “uncool” langauges. (See the recent Ruby vs PHP debacle with the CDBaby site for what I mean there.)
December 26, 2007 at 9:35 pm |
lispy, I think you’re missing arcane’s point. The truth is that computer science (rightly) has nothing to do with software development. CS is basically mathematics with a thin layer of application. CS dictates the architecture of large software systems mostly at a theoretical level.
A good CS curriculum is primarily useful to software engineers as a background of all the important algorithms. In practice, most programmers only write simple algorithms, yet it’s very valuable to have an abstract understanding of how all of the libraries and compilers do their jobs.
The hard part of CS is figuring out the math, after that a few reference implementations are written and these things become library components no one thinks about any more.
The hard part of software engineering is managing the human interface, and I think arkanes is right in suggesting that this can’t be taught. There’s no scientific principles to turn to here. Every domain and project and team has unique challenges and requirements. To be a successful lead architect requires navigating a shifting sea of unknowns that can only be estimated through years of experience. I suppose a curriculum could be developed that better tested some of these skills, but frankly that’s not academia’s forte.
December 26, 2007 at 9:53 pm |
I consider myself a “street programmer” according to your definition as I have no formal education outside a couple programming classes for my associates degree. My brother however is a comp sci major. When we discuss implementation and approach to real world applications, he is all theory, and unfortunately for him and I think many comp sci’s, theory doesn’t get you far in the real world. If you can’t readily adapt to inevitable change, all the theory in the world can’t finish a project.
I think the advantage goes to the street programmer who gets a comp sci degree after the fact.
December 26, 2007 at 11:30 pm |
As a CS professor at a reasonable university somewhere in the south of the USA, I find that the CS students generally can’t program. When I bring this up at faculty meeting the response is “CS is not programming”. This is partially true -trivially CS is the study of algorithms – and as such the actual implementations are not really critical. (Think about the Platonic idea of an “ideal” vs a real item). However, without good programming skills the students are highly limited in what they can do. I find my graduate students are often unable to finish projects in a timely manner (if at all) when any coding is required. They are often very good at finding an application (say SVMlite) and varying the parameters, but this doesn’t work for performing truly innovative research.
And my chair wonders why I’m picky about students!
December 27, 2007 at 3:12 am |
I really believe that engineering (non computer) provide a much better background than CS. Software development is first and foremost about solving problems which is what engineers traditionally train for. The tools are necessary to get there but they aren’t the goal of the software. That being said, a team of developers should be a mix of people with different skills who can learn from each other and make the overall product better.
December 27, 2007 at 4:49 am |
I think you’re making this too complicated. IMO there are only 3 primary skills a software engineer needs to master:
1. how to program (or, how to represent a conceptual problem in a programming language)
2. generalization (or, how not to copy & paste)
3. abstraction (or, how to find the most dumbed down way of communicating with some external module(s) that still allows us to achieve everything we want without being affected by major refactoring)
Now, these 3 problems take about 10 years-ish for a really smart dev. to fully crack. Let’s say, for arguments sake, a year for the first, a few years for the second, and the rest of the time for the third. The reason for that is because it requires much longer to get burned by screwing up abstraction then it does by screwing up generalization, and so there is a quicker feedback loop, and so learning happens at a quicker pace.
Now, there are shit loads of secondary skills — too many to iterate, but some include:
visually representing problem domains — knowing when to use them and which diagram types to use
time complexity analysis — being able to immediately appreciate when one algorithim is significantly faster than other (i.e. the difference between seconds, hours, and infinity) and learning not to waste time with insignificant optimizations except where they are shown to be on the critical path
For me, even though CompSci doesn’t create good programmers (and it certainly doesn’t have the 10 years to create great programmers), it is still a great curriculum because it encourages people to think pure thoughts (even if they’re not great at implementing them) and leaves the dirt and grime of real development for the on job. If you can remember the higher aspirational goals the CompSci people taught you, you’ll be less like to miss a trick, and forge ahead with an approach when a much more elegant one is available.
That’s basically it. The perfect example of this is all the donuts that use regexps for modifying code, and of course it’s full of bugs!!! If they didn’t remember their parser class at this point then WTF!
December 27, 2007 at 9:02 am |
@Brian
I don’t believe that’s true. There’s no reason a mechanical engineer would be a better programmer than a CS major. If both get their degrees and then learn software development, most likely the CS major will blow the MechE out of the water.
I like the analogy of physics:bridges as CS:software. You could train a physicist to be a great bridge builder, but he won’t come out of his undergraduate degree being one! The same goes for the CS grad.
December 27, 2007 at 12:22 pm |
I’ve been coding and learning and taking courses since the 1960’s. There was no undergraduate major in CS back then.
Being a good ‘street programmer’ or hacker is an excellent background for understanding CS courses. I found that I understood the questions automatically and the answers reasonable soon. It also helped me get through my projects more quickly.
I also found that CS courses made me a much better programmer, and wish I could have taken a graduate degree in CS, but that just wasn’t practical.
Nothing, however, prepares you for dealing with truly clueless management. The only solution is to be the manager. I’ve worked for people who actually did not know what a computer is. Period. This was extremely difficult. Nowadays most managers have some computer experience (even if they’re managing a shoe factory) or software development experience (if it’s a software or web house).
Often overlooked is the need for an application specialist. This can be a person who knows next to nothing about computers but is a real expert on the domain area, be it mathematics, accounting, the telephone network, aviation, whatever. Getting questions answered and the requirements right is more than half the battle.
This is because CS types, street programmers and hackers are often totally clueless about mathematics, accounting, the phone network, aviation, and so on.
In the US, but not Europe, Registered Nurses make more money than CS grads. I think the RN’s may have a harder job, too. And certainly more responsibility. I just thought you’d like to know.
Engineering and physics have turned out to be good backgrounds for software developers, but I think a CS or SE degree is better.
One area that does not help much is the part of “Software Engineering” that studies methods, methodology, CASE tools, flowchart generators, UML, CMM, waterfall versus agile, and on and on. They should, but they don’t. One reason is that in business, the structure of a project is dominated by budgets and schedules, and anything that contradicts that tends to get crushed. This especially true in aerospace firms and the like. It is less true in advanced software organizations like Google and Microsoft. I think tools and solutions in these areas are needed, I just think we don’t have them yet. Of course, they’ve been selling them for over 20 years.
Another area that hasn’t helped is Verificationism. This is the set of ideas to “Prove programs correct”. Someday we’ll have tools in the bigger IDE’s to help us do this, but that day is not here. And bigger IDE’s may require enormous computing resources. 100 cores, anyone? In a laptop?
Large teams of developers are hard to organize and manage. Communication is harder. Higher level languages enable development with smaller teams. Ruby, Python and Lisp lead the way, Perl too, but there has to be a next step, a higher level language that will enable even more functionality from manageable human resources.
December 27, 2007 at 1:52 pm |
If you try to teach people how to program you need to teach them something that looks like Pascal. If you don’t do that, you will spend endless time trying to rescue people lost in weird syntax badlands. And they will have problems finishing their tasks.
Any language that uses curly braces will mess up the students minds and they should not be exposed to those languages before they can program.
In ancient times C++ shops would recruit people who started out with Ada because they were far better programmers than people who started out with C++.
A 50% reduction in time spent to finish a task was never an argument for Comp Sci graduates to change programming language. So much for science.
greetings,
December 27, 2007 at 4:35 pm |
JFred is right about clueless management. The dotcom era brought in a lot of phonies who claimed to be IT pros, web developers, etc. It also has affected reality of what it really takes to create a scalable system. These phonies some squirmed their way into management positions without any real IT experience. They have to schedule meetings to avoid decisions. A few of these so-called IT managers have CS degrees, some even masters. But generally, after 25 years in IT at all levels. The CS people fail. Some over analyze, some implement so many layers, even to talented can’t save the projects. Don’t forget to add the insecurity factor in the mix. I’ve seen many projects fail because the manager is afraid to have anyone “smarter” than he on the project. Duh. If course, then there is the common sense factor. Probably, the most important skill over any IT, or CS or EE skill. Sorry for the vent. It just happens so often.
December 30, 2007 at 1:32 am |
@Brian, Re “That being said, a team of developers should be a mix of people with different skills who can learn from each other and make the overall product better.”
I agree, but I think this leads to the point that we’re really missing here: a singular, good programmer is someone who can wear many hats (cs, seng, engr, math, writing and probably a few more for the right brain side of things). Realistically, no one graduates with all of this and I think schools realize this. Instead, they set out to give you a choice of degrees to start with as the basis for a good programming career. The rest of the way is left to the individual in cooperation with whoever they work for or with.
January 5, 2008 at 1:13 am |
[…] are likely to occur and the product requires more and more customization. As highlighted by Lispy, many teams “implement awful hacks" to adapt rapidly to new needs and "to keep a client happy enough to keep the money coming in.” This is what he refers […]
January 16, 2008 at 9:51 am |
[…] Why Comp Sci Grads Can’t Hack (and why you need them anyway…) – Its refreshing seeing this in words, from someone else, other than me. Reading this, and the article that spurred it: Is Computer Science Dying? > Computer Scientists Can’t Program! can be pretty useful. A 3-4 year bachelors teaches you an array of languages and concepts, that in the end, one becomes something of a “know it all”, yet not an expert in any domain. There is just no room for you to become an expert, because the curriculum just isn’t free-flowing. Writing a library management system over and over again, in a different language even, just makes no sense at all, does it? (I picked on a library management system because Ditesh, Prabu and I had a good laugh about this yesterday). […]
March 1, 2008 at 12:59 am |
I am a “street programmer” and i liked your article. It is definitely interesting how most of the CS grads we come into contact with are clueless when it comes to the specifics of a lanuage or the real way to implement a solution.