[Tutor] Re: Directory operation: why so darn slow?

Emile van Sebille emile@fenx.com
Sun Nov 3 02:33:01 2002


Johannes Graumann:
> 1) the operation represented by the snippet below is
> ridiculisly slow (didn't use to be - before I had a several
> level deep dictionary) - is there anything I can do about
> that?

It's hard to know exactly, but I'd suspect you're possibly in and out of
the function contrast.data() alot in your loop, and seeing the impact of
that overhead.  See if the changes below help:

>
> repres_filtered = {}

conform = contrast.data()['conform']

> for dataset in contrast.data()['conform.keys():

for dataset in conform.keys():

>  seqcov_count = 0
>   for element in contrast.data()['conform'][dataset]['seqcov']:

  for element in conform[dataset]['seqcov']:

>    if re.match(r"\d+",element):
>     seqcov_count = seqcov_count + 1
>   if not seqcov_count < repres:
>    repres_filtered[dataset]=contrast.data()['conform'][dataset]

   repres_filtered[dataset]=conform[dataset]
>

In short, I don't know what contrast.data() has to do to return a
dictionary, but I would guess you can structure things so it only has to
do it once.

> 2) does the fact that the dictionary is created by the class

...recreated, perhaps?

> imply that I can not change it (delete it)? I would prefer
> to do something like
>
> for dataset in contrast.data()['conform'].keys():
>  seqcov_count = 0
>   for element in contrast.data()['conform'][dataset]['seqcov']:
>    if re.match(r"\d+",element):
>     seqcov_count = seqcov_count + 1
>   if not seqcov_count < repres:
>    del contrast.data()['conform'][dataset]
>

You'll probably find the answer to this one as well as you rework
things.  That dictionary probably wants to be an attribute of the class
instance, and not a creation of a class method.

HTH,



--

Emile van Sebille
emile@fenx.com

---------