Design tips requested for OOP forum system (using pickle)

[Graham Ashton]

In article <eb8b2b6e.0111291359.64a1ee62 at posting.google.com>, "Simon
Willison" <cs1spw at bath.ac.uk> wrote:

> I'm something of a Python newbie but I'm keen to learn so I've decided
> to throw myself in at the deep end.

Good plan. The fastest way to learn, I reckon.

> Anyway - I've decided to code a flat threaded web forum system ... using
> pickled objects in flat files to store the data (rather than using a
> database or a delimited flat file format of some sort).

Here's an idea to think about; see if you can work out how to design a
fairly generic interface to your storage. Implement it first with pickle
(cPickle is much faster by the way) and then see if you can fiddle around
with it a bit so that it supports different storage systems (e.g. anydbm),
selectable at run time. Design patterns will be your friend here, and are
very interesting ways of taking your OOD knowledge beyond the standard
inheritance and delegation techniques.

Top pickle tip (cPickle isn't available everywhere):

  try:
      import cPickle
      pickle = cPickle
  except ImportError:
      import pickle

> User - an object representing a single forum user account Forum - an
> object representing a forum (e.g "General Chat" or "Site Suggestions")
> Thread - an object representing a thread Post - an object representing
> an actual post

Do you need a thread? I don't know. Would it be better to have threads
specified through relationships between posts, so each post would have an
attribute that would somehow point to children? I'm not sure, I'm just
thinking (briefly) aloud. How will your interface be driven and generated?
Let that guide you to begin with.

> In addition I'm keen on having Moderator and Admin classes which extend
> User and are used for forum accounts with extra ablities.

Fine, but that kind of simple inheritance isn't the most interesting kind
of problem for you to solve here. The structural aspects are more
complicated, and consequently interesting. By "structural" I mean how you
will need to organise the various elements of your software. How will the
user interface communicate with your storage system? Will you have a layer
of indirection on top of the storage? I'm not suggesting you should (I've
not thought it through), I'm just trying to illustrate what I'd be
thinking about if I was you.

> Post objects will be held in an array within a Thread object
> (representing the thread the posts were posted on). Thread objects will
> be held in an array within a Forum object.

Sounds reasonable, but.... How will you select the next 100 posts that
come after a certain date? Wouldn't you like to be able to say "I want a
sub set of this list that contains only these posts"? Looping over an
enormous array won't scale very well there. Is an array the best data
structure?

As you're doing it as a learning exercise, try playing around with
different implementations of your Forum class - don't fix it's internal
design this early. Decide on your initial interface for your Forum class
and then build things that talk to it. Then play around with the inner
workings of the Forum class to see how best to implement it.

> One thing I haven't quite figured out yet is how to relate posts to the
> user object for the user that posted that post. Should I look at storing
> some kind of user ID and then searching through my user objects for an
> object with that ID stored within it every time I display a post?

If you do go that way you will want a fast database engine. I wouldn't
though (go that way, I mean). I'd start by storing all data about a post
in the Post object and throw the User object concept in the bin for a bit.
Adding it later would be a useful exercise in refactoring if you decide
you really want it.

> I'm also not entirely sure how to do things like ordering posts witin
> threads by date - should I ensure they are in order within the array or
> sort the objects by their date property (which I assume will require me
> to write my own ordering algorithm, presumably not hard but something
> I've never had to do before).

Assume you'll want to order them in different ways in the future and pick
a data structure/storage system that lets you sort them efficiently. You
can sort objects based on the values of their attributes quite easily.

> My final problem (I think) is how to split pickled data up. Should I
> have an OverallForum object which contains ALL the objects (and data)
> within it and pickle/unpickle the whole thing for every page view
> (sounds inefficient) or pickle individual forums in their own files.

I've barely used pickle so can't help you much with the one big
file/little file speed trade off. I imagine little files would be a good
place to start though (less RAM required to load them, less time to seek
right through them).

> Any tips / pointers / warnings that I'm going about this project
> completely the wrong way would be more than welcome :)

I'm afraid I've not really thought about how I'd design it if I was you,
just done a quick brain dump. Have a look here though:

  http://ootips.org/

-- 
Graham