Trouble with psyco

[Jp Calderone]

On Mon, 22 Nov 2004 02:07:46 -0800, Dick Moores <rdm at rcblue.com> wrote:
>psyco is acting a bit psycho for me.
> 
> Please see my spinForWeb.py at <http://www.rcblue.com/Python/spinForWeb.py>
> 
> When psyco is in use, entering an integer somewhere between 2000 and 2500 
> causes my computer to freeze. Not really freeze but the program doesn't 
> finish, and I have to quit with a ^Q.
> 
> psyco is really impressive, but I'm disappointed that I can't demonstrate 
> (to friends) counting with it to numbers above 2 billion.
> 
> If I remark out the "psyco.bind(spin)" line, there's no problem no matter 
> what integer I enter. Can someone explain what the problem with psyco is?
> 
> Windows XP, Python 2.3.4
> 
> Thanks,

  I added a print to your inner loop:

      while k < max:
          k += 1
          if not (k % 1000000L):
              print 'at', k

  Progress is regularly reported, but it is quite, quite slow.  So the problem is not that psyco is breaking your program.  Neither is it speeding anything up though, and in fact seems to be slowing it down over some sets of data.

  When you cross sys.maxint (a bit above 2 billion, usually), internally Python stops storing integers as platform-native data types and switches to a data type which can grow almost without bound.  Since this new data type isn't native to your architecture, every operation on it is vastly slower, but Python must switch because the platform-native data type cannot take on the values you are asking.

  Python works very hard to make each loop around its evaluator.  You're asking it to get around quite a few times, doing very little _actual_ work each time.  This is one of the worst cases for performance in Python.

  With the psyco version, the call to spin() is effectively made into a single operation, as far as the interpreter loop is concerned.  That's part of why it's so much faster - instead of one loop around the interpreter per "k += 1", it's just one loop around the interpreter.  Big difference.  But you've discovered one of the costs associated with this - Control-C generally sends SIGINT to the process: Python notices this and sets a flag which will _later_ cause KeyboardInterrupt to be raised.  In this case, with psyco enabled, "later" happens when spin() returns, because calling spin() was just one operation, as far as Python is concerned.

  Without psyco, "later" happens very frequently, almost constantly even, and so Control-C generally has an instant effect.

  Hope this helps,

  Jp