[Edu-sig] Migrating to Projects - Was: Low Enrollments - programming as anti-intellectualism
peter at engcorp.com
Sat Nov 5 17:31:09 CET 2005
Laura Creighton wrote:
> (cc'd to Peter Hansen, because it is of interest and also mentions him - lac)
(Thanks Laura... I've temporarily subscribed to edu-sig in case my
comments/opinions can be of value to someone. If that concerns you,
don't worry, I won't stay long! <wink>)
> In a message of Fri, 04 Nov 2005 12:31:03 MST, Trent Oliphant writes:
>>This brings up an interesting discussion point, one in which I would be
>>extremely interested personally: How do you help students (or yourself)
>>move from writing scripts, functions, classes, modules etc. to writing
>>a larger project.
>>I know it may seem that I am talking about an IDE - but even those
>>assume that you already know how to do a project. So I get
>>I want them to work for me - but they just haven't yet.
>>I am a lone programmer - entirely self taught - and this is a daunting
>>task. I feel confident that I could handle programming most of the
>>individual components that would make up the project, but I don't even
>>know where to start and there appears to be nothing out there to help
>>me learn that.
> I think you are looking for the wrong sort of magic bullet. What you
> need to learn is how to think about large projects.
I could keep all of Laura's comments as they are, of course, right on
the mark. I'll comment directly on a couple of her points below and
then say one or two things that come to mind in response to Trent's
issues above. I suspect this could be a long response so I'll write
fast to make it easier on you. ;-)
> Whatever you get, make sure it is about Test Driven Design. There are
> other competing methodologies for designing large programs, which have
> a long history, of which the most famous is called Waterfall. All
> you need to know is to avoid them like the plague.
Other than to have a basic understanding of what those other
methodologies are, and more importantly why they fail (often, and
gravely), I'd agree that you can get pretty far with TDD alone in
building large projects.
I think the first thing to understand is that successful large projects
are just small projects that have grown. It is possible, more so in
theory than in practice, to "start big" by designing everything in
advance, documenting the heck out of it in requirements specs and design
docs and test plans, building the beast, testing it thoroughly, and
shipping it. Possible, but rarer than a white crow.
There are dozens of reasons why this is so, but I'll focus on the
positive side here instead: how do you grow a small project? You can
write one module, then another, and so on, but you'll just rightly say
"but I don't even know where to start." And assorted random modules
don't form a cohesive whole, and really this approach works only if you
already did "big design up front" and know where you're headed. Most of
the time, nobody really knows that, even though many people delude
themselves into believing that they do.
You could try writing one piece of functionality instead, even if it
involves code in a few different modules. Think of it as a "slice" of
the future application, maybe a bit of GUI, maybe a bit of the
algorithms, some persistence. You'll quickly get into trouble here too,
however, since when you start to evolve you'll have massive breakage,
awkward interfaces, and no idea where you stand after only a few changes.
Test-Driven Development will help with this, though I don't personally
find it sufficient. What it will do (as you try this "slice" approach)
is _lead_ you in the right direction as you code. You are writing the
tests _first_, before each chunk of code that will pass those tests.
You are building a body of tests which get re-run often, ensuring you
cannot write more than a few bad lines of code before an alarm goes off
telling you you're going off track. It forces you to implement less
awkward interfaces, and gives you a measuring stick to tell you where
What I find it doesn't do is lead you from at a "big picture" level.
What "slice" do you work on first? How do you prioritize the other
slices? How complete do you make each feature in the first pass? When
are you "done"?
These are areas where complete "methodologies" come in, and where
Extreme Programming has value for me. Without a "methodology" (one
could just call them project management strategies), you may have
excellent, working code, maybe even lots of it, but you'll look back on
it after six months and realize you've wasted the last four months on
features non-essential to your users, maybe things even you don't value.
Trent says, "I'm a lone programmer - entirely self taught - and this is
a daunting task. I feel confident that I could handle programming most
of the individual components that would make up the project, but I don't
even know where to start ...."
I feel exactly like that at the start of every project! (And, by the
way, I'm self-taught in programming too, so I remember something of what
you're feeling here.)
Really, even right now I'm responding to this email because I'm daunted
by the two projects which I should be working on instead, one of which
is really too large for one person, and the other of which, while I can
handle, has discouraged me after we made a false start (customer
specified wrong requirements, we wasted our time) and I haven't rebuilt
my "gumption" enough to try it a second time.
Interesting side point to note, perhaps: these are issues which
traditional "heavy-weight" processes like Waterfall pretty much ignore.
Human concerns, emotions? Irrelevant. This is _programming_.
Technology! Get your ego out of the way and use this tool correctly and
you'll get the job done in no time.
Agile processes do deal with soft issues, however. (See
http://www.agilealliance.org for a start, and make sure to read the
brief Manifesto before spending any more time on that site... this stuff
doesn't "fit" some people's brains.)
In this case, XP helps me figure out small, achievable, prioritized
steps. It basically points me (with relatively little effort on my
part) at where I should start, specifically, what precise "slice" is of
most value _right now_. It lets me define better how much of that slice
I need, and thus defines the tests which I should write when I get into
the actual coding (i.e. Test-Driven Development mode). It let's me get
going in spite of the daunting feeling (which I suspect never goes away).
I expect some folks here and certainly many elsewhere would disagree
with much of this. They feel that non-agile approaches work just fine,
and in their own cases they might be right. I can't argue about that,
and I don't. I do consistently come in to my new customers' sites and
hear the same kind of talk. "Look how we've got this nice design
document we wrote late last year. This is our master plan, our guide."
Universally these customers are _way_ off that plan, in most cases not
even remotely doing what it says they should do, except in a very
hand-waving kind of way.
They've talked themselves into believing that because they wrote a
document which has a table of contents that appears to cover the entire
scope of the system, they must be doing things right. I'm skeptical,
but have an open mind. Maybe this is going to be the first customer
I've had who really and truly can do things this way (though, if so, why
did they call me in? Hmmm...)
Then we get into the code, and start looking at the upcoming features,
and I'm flipping through the design document looking for the matching
text and finding empty sections with "TBD" (To Be Determined) written
just under the titles. A block diagram that has about six different
major components that don't actually exist in the hardware. A proposed
schedule that ended five months earlier even though the code I'm looking
at is barely prototype quality. Signs that only two people in the
company have a current copy, and one of those is missing the second half.
I could go on. Thankfully I won't, and don't need to, because so far
each of these customers is slowly and quietly letting go of their old
ways as we come in and, often within a couple of weeks, start giving
them a warm fuzzy feeling that their project, while late relative to
their original expectations, and struggling with quality issues, is in
the right hands. Why? Because we've identified some priorities, we've
started writing some tests, and we aren't making unrealistic promises
and more. (And their pain is fresh, so they're meek enough for a while
that we can push them around a little. ;-)
To build large systems, start small. Break the work into small tasks,
prioritize, work on them one at a time. Write automated tests for
everything. Aggressively limit complexity, including in the code, the
tool set, the technologies you are using, and the administrative
processes with which your bosses smother you. Use a revision control
system. Consider a wiki (for team work) and an issue tracker, but never
add a tool until you've struggled without it for a while and are certain
it will make your life easier. Measure your progress and always adapt;
go back and improve your approach the next time.
The following sites might be of interest, though unfortunately I do
_not_ maintain a current set of useful links in the area. I tend to do
that for my own purposes as I'm learning, but once I've adopted a
technique/tool/mindset I tend to stop reading about it and just use it.
(In this case, I've had this mindset for five years, which is the
longest time any one approach has held my attention and give me
benefits, and so far this one shows no signs of stopping. But I don't
believe it will always be my primary approach.)
* http://www.agilealliance.org/ (mentioned above)
* http://www.extremeprogramming.org/ (somewhat dated)
I hope some of these thoughts prove helpful...
More information about the Edu-sig