[pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm
Leonardo Santagada
santagada at gmail.com
Thu Feb 12 12:20:25 CET 2009
On Feb 12, 2009, at 6:14 AM, Alexandre Fayolle wrote:
> Le Wednesday 11 February 2009 19:49:59 Kelly F. Hickel, vous avez
> écrit :
>> Ahh, yes, now I remember. Psyco ONLY works with 32 bit python,
>> whereas
>> I MUST use 64 bit because the memory footprint for cvs2svn with our
>> repo
>> is just too large for 32 bit processes.
>>
>> So, dead in the water again?
>
> Hmm, just a quick question: are you sure your problem is CPU bound
> (as opposed
> to IO bound)? If it is not, you are looking at the wrong place to
> solve your
> performance issue, and you should rather look for a faster disk,
> faster
> network, etc. Also from what you say hereabove, maybe adding some
> RAM could
> help.
I'm answering this as we already talked about it in some private
emails. He already figured out the problem is cpu bond, one of the
cpus goes to 100% and stays almost the whole time this way for 6.5
days. His machine has 32 gb of RAM and the python conversion program
uses good part of it that's why he can't run the thing on 32bit mode.
His repo is only 6gb in size so I think maybe the problem is either a
bad performing algo no cvs2svn or a bad cvs library (or whatever is
used to access cvs).
If it is the cvs library there is nothing no one can do (even psyco or
pypy would not help). If the problem is a wrong pure python algo it
should probably be changed (if a better one exists of course).
Now why would a program reading a 6gb repo, even using a database file
for intermediate data still uses tons of ram to do this conversion I
don't know.
--
Leonardo Santagada
santagada at gmail.com
More information about the Pypy-dev
mailing list