Linux Kernel Design and Why Python is Rad

[Jonathan Gardner]

The following is a lot of wind. Just read the article at kerneltrap.org, 
you'll probably enjoy hearing it from the horse's mouth rather than my own 
interpretation.

On Tuesday 04 December 2001 02:14 am, Andrew Dalke wrote:
> Jonathan Gardner:
> > Linus Torvalds on design: "Nothing successful has every been
> > designed. It has evolved."
> >
> > Everyone else: "No wait a minute, that's not what this book
> > says! And that's not what my teacher said!"
>
> Now, which books say that?  "Rapid Development" is very
> explicit that all the design models, even the classic
> waterfall design, have a way to back up from mistakes.
> It might be slower than other development models, but
> it's even useful in some cases, like writing the Nth
> iteration of a salary accounting package where all the
> requirements and dependencies are well known.
>

"Code Complete", and other great classics of Computer Black Magic (I refuse 
to call it a science yet...) emphasized that you had to spend something like 
4 times as much time designing than coding, with that factor increasing for 
larger projects. it also emphasized strict testing procedures and stricter 
coding standards and such. There was (and still is!) this idea that before 
you touch the keyboard with a line of code, you should have everything 
figured out one way or another.

The sad truth is that no one knows what they are doing, not even Linus, not 
even Bill Gates. You have to go out and implement it before you can decide 
whether it was worth doing.

"Rapid Development" (if what I understand is true, not having bought the tome 
yet) emphasizes small design steps, quick implementation, and solid 
evaluation. This is more along the lines of evolution. The one thing that 
Linus stressed that I understand RD misses is the idea of trying many things 
out simultaneously. On the Kernel Project, you have several independent 
teams, some of which are working on similar parts of the Kernel. They are all 
writing code, sometimes making their own forks, and overall what you get is a 
community effort to try all these things out, and you as the user get the 
cream of the crop.

For instance, the VM discussion they had a while ago. Well, some guys (I know 
it wasn't just one, and there may have been several more attempts) just up 
and went and wrote a new VM machine. Linus decided to pick the one that he 
thought worked the best. However, the old VM machine is still alive in the ac 
fork.

> Coming from the other side, all designs have been built
> from knowledge gained in previous work.  "Evolved" in
> its most precise sense means "changed over time."
> (NOT "improved over time".)  So by definition I agree
> that everything successful has been evolved.  Then
> again, so has everything unsuccessful.
>
>
> Conclusion: this argument is based on a false dichotomy.
> Design helps minimize the chance of being unsuccessful,
> and goes hand-in-hand with the evolution of software.
>

Maybe you should read the posts at kerneltrap before you are so sure about 
this. Sun is dying precisely because of strict design. They only have one 
"species" and that is a very specialized one that can only survive in odd 
situations. Their species cannot adapt because they cannot try out many 
different things at once. Microsoft built their species by trial and error. 
They completely overhauled the kernel and structure of windows how many 
times? While they satisfy many people, they cannot satisfy all people.

The Linux Kernel however, has the ability to try many things at once. Some of 
the results are good, and some are bad. You throw out the bad (or rework 
them) and you try to figure out why the good is so good. Then you do it all 
again. What you end up with is satisfying everyone with a superior product 
because you have actually tried out several things at once.

Design is not totally useless - it is uselful for small projects or small 
changes, but it can easily get out of hand and become a burden not worth its 
weight if it is used for large projects or large schemes. Linus never 
designed anything for the kernel on a large scale - he claims he can barely 
understand what it does himself, so how can he be expected to say where it is 
going or what it should be able to do? And if he were to offer such a 
statement, what are the chances of him being right? So design becomes 
pointless at a certain point.

Jonathan

>                     Andrew
>                     dalke at dalkescientific.com