Mailman 3 August 2010 - pypy-dev

Building pypy on freebsd 64bit
by Shadrin Eugene 18 Nov '11

18 Nov '11

Hi! I tryed to build pypy on the freebsd, 64bit platform, but failed. First of all I downloaded a 32bit-Python to run pypy. And then: [user@server_name ~/pypy-trunk/pypy/translator/goal]$ ~/py32/bin/python translate.py -Ojit [translation:info] Translating target as defined by targetpypystandalone [platform:execute] gcc -m32 -c -O3 -pthread -fomit-frame-pointer /var/tmp/usession-trunk-22/gcctest.c -o /var/tmp/usession-trunk-22/gcctest.o [platform:WARNING] /var/tmp/usession-trunk-22/gcctest.c:28:2: warning: no newline at end of file [platform:execute] gcc -m32 /var/tmp/usession-trunk-22/gcctest.o -L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32 -pthread -o /var/tmp/usession-trunk-22/gcctest printf("sizeof short=%ld\n", (long)sizeof(short)); printf("sizeof unsigned short=%ld\n", (long)sizeof(unsigned short)); printf("sizeof int=%ld\n", (long)sizeof(int)); printf("sizeof unsigned int=%ld\n", (long)sizeof(unsigned int)); printf("sizeof long=%ld\n", (long)sizeof(long)); printf("sizeof unsigned long=%ld\n", (long)sizeof(unsigned long)); printf("sizeof signed char=%ld\n", (long)sizeof(signed char)); printf("sizeof unsigned char=%ld\n", (long)sizeof(unsigned char)); printf("sizeof long long=%ld\n", (long)sizeof(long long)); printf("sizeof unsigned long long=%ld\n", (long)sizeof(unsigned long long)); printf("sizeof size_t=%ld\n", (long)sizeof(size_t)); printf("sizeof time_t=%ld\n", (long)sizeof(time_t)); printf("sizeof wchar_t=%ld\n", (long)sizeof(wchar_t)); printf("sizeof mode_t=%ld\n", (long)sizeof(mode_t)); printf("sizeof pid_t=%ld\n", (long)sizeof(pid_t)); sizeof short=2 sizeof unsigned short=2 sizeof int=4 sizeof unsigned int=4 sizeof long=8 sizeof unsigned long=8 sizeof signed char=1 sizeof unsigned char=1 sizeof long long=8 sizeof unsigned long long=8 sizeof size_t=8 sizeof time_t=8 sizeof wchar_t=4 sizeof mode_t=2 sizeof pid_t=4 Traceback (most recent call last): File "translate.py", line 300, in <module> main() File "translate.py", line 205, in main targetspec_dic, translateconfig, config, args = parse_options_and_load_target() File "translate.py", line 153, in parse_options_and_load_target targetspec_dic = load_target(targetspec) File "translate.py", line 105, in load_target mod = __import__(specname) File "targetpypystandalone.py", line 5, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/__init__.py", line 1, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/objspace.py", line 3, in <module> from pypy.interpreter import pyframe, function, special File "/place/home/hitrobot/pypy-trunk/pypy/interpreter/pyframe.py", line 13, in <module> from pypy.rlib import jit, rstack File "/place/home/hitrobot/pypy-trunk/pypy/rlib/rstack.py", line 10, in <module> from pypy.rpython.lltypesystem import rffi, lltype File "/place/home/hitrobot/pypy-trunk/pypy/rpython/lltypesystem/rffi.py", line 824, in <module> sys.maxint, sizeof(lltype.Signed))) AssertionError: Mixed configuration of the word size of the machine: the underlying Python was compiled with maxint=2147483647, but the C compiler says that 'long' is 8 bytes I got this error, then I added "-m32" flag to gcc argument list (in the posix.py source file) , but error occured again :( Does anybody tryed to build pypy-c on the 64bit FreeBsd? Thank you for your answer! -- Shadrin Eugene

5 7

Loop invaraints
by Hakan Ardo 07 Sep '10

07 Sep '10

Hi, some time ago, there were some discussion about loop invaraints, but no conclusion. What do you think about the following approach: - Let optimize_loop mark the arguments in loop.inputargs as invariant if they appear at the same position in the jump instruction at the end before calling propagate_formward - Let the optimize_... methods emit operations that only uses invariant arguments to some preamble instead of emitting them to self.newoperations whenever that is safe. Also, the result of these operations should probably be marked as invariant. - Insert the created preamble at every point where the loop is called, right before the jump. - When compiling a bridge from a failing guard, run the the preamble through propagate_formward and discard the emitted operations, to inherit that part of the state of Optimizer. This should place the invariant instructions at the end of the entry bridge, which is a suitable place, right? At the end of a bridge from a failing guard that maintains the invariants the optimizer should remove the inserted preamble again, right? And at the end of a bridge that invalidates them, enough of the preamble will be kept to maintain correct behavior, right? -- Håkan Ardö

6 12

gpgpu and pypy
by Jorge Timón 01 Sep '10

01 Sep '10

Hi, I'm just curious about the feasibility of running python code in a gpu by extending pypy. I don't have the time (and probably the knowledge neither) to develop that pypy extension, but I just want to know if it's possible. I'm interested in languages like openCL and nvidia's CUDA because I think the future of supercomputing is going to be GPGPU. There's people working in bringing GPGPU to python: http://mathema.tician.de/software/pyopencl http://mathema.tician.de/software/pycuda Would it be possible to run python code in parallel without the need (for the developer) of actively parallelizing the code? I'm not talking about code of hard concurrency, but of code with intrinsic parallelism (let's say matrix multiplication). Would a JIT compilation be capable of detecting parallelism? Would it be interesting or that's a job we must leave to humans by now? What do you think? I don't know if I had explain myself because English is not my first language. Cheers, Jorge Timón

9 13

What's wrong with >>> open(’xxx’, ’w’).write(’stuff’) ?
by sakesun roykiatisak 21 Aug '10

21 Aug '10

Hi, I encountered this quite a few times when learning pypy from internet resources: the code like this >>> open(’xxx’, ’w’).write(’stuff’) This code is not working on pypy because it rely on CPython refcounting behaviour. I don't get it. Why ? I thought the code should be similar to storing the file object in temporary variable like this >>> f = open('xxx', 'w') >>> f.write('stuff') >>> del f Also, I've tried that with both Jython and IronPython and they all work fine. Why does this cause problem to pypy ? Do I have to avoid writing code like this in the future ?

7 9

JIT'ed function performance degrades
by Hart's Antler 20 Aug '10

20 Aug '10

I am starting to learn how to use the JIT, and i'm confused why my function gets slower over time, twice as slow after running for a few minutes. Using a virtualizable did speed up my code, but it still has the degrading performance problem. I have yesterdays SVN and using 64bit with boehm. I understand boehm is slower, but overall my JIT'ed function is many times slower than un-jitted, is this expected behavior from boehm? code is here: http://pastebin.com/9VGJHpNa

5 8

Re: [pypy-dev] What's wrong with >>> open(’xxx’, ’w’).write(’stuff’) ?
by sakesun roykiatisak 19 Aug '10

19 Aug '10

That's make sense. I've tried on both IronPython and Jython with: ipy -c "open(’xxx’, ’w’).write(’stuff’)" jython -c "open(’xxx’, ’w’).write(’stuff’)" When the interpreter terminate the file is closed. That's why it didn't cause any problem. Perhaps, I should always use "with" statement from now on. >>> with open('xxx', 'w') as f: f.write('stuff') Thanks On Thu, Aug 19, 2010 at 11:40 AM, Aaron DeVore <aaron.devore(a)gmail.com>wrote: > If I understand correctly, PyPy will garbage collect (and close) the > file object at an indeterminate time. That time could be as long as > until the program exits. Because CPython uses reference counting, it > closes the file immediately after the file object goes out of scope. > > Of course, I may be entirely wrong. > > -Aaron DeVore > > On Wed, Aug 18, 2010 at 9:25 PM, sakesun roykiatisak <sakesun(a)gmail.com> > wrote: > > Hi, > > I encountered this quite a few times when learning pypy from internet > > resources: > > the code like this > >>>> open(’xxx’, ’w’).write(’stuff’) > > This code is not working on pypy because it rely on CPython refcounting > > behaviour. > > I don't get it. Why ? I thought the code should be similar to storing > the > > file object in temporary variable like this > >>>> f = open('xxx', 'w') > >>>> f.write('stuff') > >>>> del f > > Also, I've tried that with both Jython and IronPython and they all work > > fine. > > Why does this cause problem to pypy ? Do I have to avoid writing code > like > > this in the future ? > > _______________________________________________ > > pypy-dev(a)codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > > >

3 5

PyPy speed center not updating any more
by Armin Rigo 16 Aug '10

16 Aug '10

Hi all, The PyPy speed center does not display any update more recent than July 29. The buildbot infrastructure correctly puts them into files codespeak.net:~buildmaster/bench_results/REV.json, but the web site at http://speed.pypy.org/ does not get updated. Help please! A bientot, Armin.

3 5

JIT Failure on lltype.Array access
by Dan Roberts 16 Aug '10

16 Aug '10

As best I can tell, the JIT cannot handle my code properly, it corrupts memory and returns 0.0 for float arrays. I don't know whether the true problem is in my code or the JIT, but I need to get this resolved quickly. I know the JIT and my code are interacting badly because py.py works fine (though slow) and translated pypy-c with jit and --jit threshold=9999999 both work fine. Here's what I've tried to resolve the issue: Removing my _immutable_fields_ hints. Hand implementing bh_{get,set}arrayitem_raw_{r,i,f} (though I don't know my implementation was right, I simply copied the gc version and removed the first offset (since raw arrays have no header right? Although I expect that the gc version would have simply gotten 0 for the header size... I tried it anyways) A few thoughts: descr.py alludes to a FloatArrayDescr which I never raw defined Could the asm backend be part of the problem? Rather than the code in llmodel.py? Unfortunately I'm ill equipped to resolve this issue, so any help is appreciated (I'm on my phone but I'll happily furnish exact errors upon request)

2 1

RPython function callback from C
by Hart's Antler 15 Aug '10

15 Aug '10

I have the PortAudio blocking API working, simple reading and writing to the sound card works. PortAudio also has an async API where samples are fed to a callback as they stream in. But i'm not sure how to define a RPython function that will be called as a callback from C, is this even possible? I see some references in the source of rffi that seems to suggest it is possible. Full source code is here http://pastebin.com/6YHbT7CU I'm passing the callback like this: def stream_callback( *args ): print 'stream callback' return 0 # 0=continue, 1=complete, 2=abort stream_callback_ptr = rffi.CCallback([], rffi.INT) OpenDefaultStream = rffi.llexternal( 'Pa_OpenDefaultStream', [ StreamRefPtr, # PaStream** rffi.INT, # numInputChannels rffi.INT, # numOutputChannels rffi.INT, # sampleFormat rffi.DOUBLE, # double sampleRate rffi.INT, # unsigned long framesPerBuffer #rffi.VOIDP, #PaStreamCallback *streamCallback stream_callback_ptr, rffi.VOIDP, #void *userData ], rffi.INT, # return compilation_info=eci, _callable=stream_callback ) entrypoint(): ... callback = lltype.nullptr( stream_callback_ptr.TO ) ok = OpenDefaultStream( streamptr, 2, 2, Int16, 22050.0, 512, callback, userdata )

2 1

Re: [pypy-dev] pypy-dev Digest, Vol 361, Issue 5
by Andrew Francis 15 Aug '10

15 Aug '10

Hi Kevin: Message: 4 Date: Thu, 12 Aug 2010 19:42:54 -0400 From: Kevin Ar18 <kevinar18(a)hotmail.com> Subject: Re: [pypy-dev] pre-emptive micro-threads utilizing shared memory message passing? To: <pypy-dev(a)codespeak.net> Message-ID: <SNT110-W30CE421CB01A3D07C92F8DAA970(a)phx.gbl> Content-Type: text/plain; charset="iso-8859-1" >I don't mind replying to the mailing list unless it annoys someone? Maybe >some people could be interested by this discussion. I am finding it a bit difficult to follow this thread. I am not sure who is saying what. Also I don't know if you are talking about an entirely new system or the stackless.py module. >In my case, I look at message passing from the perspective of the >tasklet. A tasklet can either be assigned a certain number of "in ports" >and a certain number of "out ports." In this case the "in ports" are the >.read() end of a queue or stream and the "out ports" are the .send() part >of a queue or stream. A part of the model that Stackless uses is that tasklets have channels. Channels have send() and receive() operations. >For the scheduler, I would need to control when a tasklet runs. >Currently, I am thinking that I would look at all the "in ports" that a >tasklet has and make sure each one has some data. Only then would the >tasklet be scheduled to run by the scheduler. The current scheduler already does this. However there are no in or out ports, just operations that can proceed. >Couldn't all those ports (channels) be read one at a time, then the >processing could be done? If you are using stackless.py - the tasklet will block if it encounters a channel with no target on the other side. I wrote a select() function that allows monitoring on multiple channels. >Good idea. If there's no data to read, the tasklet can yield. ... but I >need to know when the tasklet can be put back into the scheduler queue I don't want to toot my horn but I gave a talk that covers how rendez-vous semantics works at EuroPython: http://andrewfr.wordpress.com/2010/07/24/prototyping-gos-select-and-beyond/ Cheers, Andrew

3 4