[pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm

Leonardo Santagada santagada at gmail.com
Thu Feb 12 12:20:25 CET 2009

On Feb 12, 2009, at 6:14 AM, Alexandre Fayolle wrote:

> Le Wednesday 11 February 2009 19:49:59 Kelly F. Hickel, vous avez  
> écrit :
>> Ahh, yes, now I remember.  Psyco ONLY works with 32 bit python,  
>> whereas
>> I MUST use 64 bit because the memory footprint for cvs2svn with our  
>> repo
>> is just too large for 32 bit processes.
>> So, dead in the water again?
> Hmm, just a quick question: are you sure your problem is CPU bound  
> (as opposed
> to IO bound)? If it is not, you are looking at the wrong place to  
> solve your
> performance issue, and you should rather look for a faster disk,  
> faster
> network, etc. Also from what you say hereabove, maybe adding some  
> RAM could
> help.

I'm answering this as we already talked about it in some private  
emails. He already figured out the problem is cpu bond, one of the  
cpus goes to 100% and stays almost the whole time this way for 6.5  
days. His machine has 32 gb of RAM and the python conversion program  
uses good part of it that's why he can't run the thing on 32bit mode.  
His repo is only 6gb in size so I think maybe the problem is either a  
bad performing algo no cvs2svn or a bad cvs library (or whatever is  
used to access cvs).

If it is the cvs library there is nothing no one can do (even psyco or  
pypy would not help). If the problem is a wrong pure python algo it  
should probably be changed (if a better one exists of course).

Now why would a program reading a 6gb repo, even using a database file  
for intermediate data still uses tons of ram to do this conversion I  
don't know.

Leonardo Santagada
santagada at gmail.com

More information about the Pypy-dev mailing list