[Tutor] Conditional attribute access / key access

Knacktus knacktus at googlemail.com
Tue Aug 31 15:53:48 CEST 2010


> The sizes given are in bytes. So 200,000 instances of this class, plus
> the list to hold them, would take approximately 34 megabytes. An entry
> level PC these days has 1000 megabytes of memory. "Huge"? Not even
> close.

The items hold a lot of metadata, which I didn't provide in my example. 
Depending on the source up to 30 addional attributes per item, mainly 
strings. And I will have several sources.

>> So far
>> there're no restrictions about how to model the items. They can be
>> dicts, objects of a custom class (preferable with __slots__) or
>> namedTuple.
>>
>> Those items have references to each other using ids.
>
> That approach sounds slow and ponderous to me. Why don't you just give
> items direct references to each other, instead of indirect using ids?
>

Unfortunately I have to able to use a relational database later on. 
Currently I'm using a document database for developement. That's where 
the ids are coming from and you're right: They are a pain ... ;-)

>
> If you *need* indirection, say because you are keeping the data in a
> database and you want to only lazily load it when needed, rather than
> all at once, then the right approach is probably a proxy object:
>
> class PartProxy(object):
>      def __init__(self, database_id):
>          self._info = None
>          self.database_id = database_id
>      @property
>      def info(self):
>          if self._info is None:
>              self._info = get_from_database(self.database_id)
>          return self._info
>

That's it! Excactly what I was looking for! That eases the id-pain. Thanks!!

Jan



More information about the Tutor mailing list