Mailman 3 December 2010 - pypy-dev

Re: [pypy-dev] [pypy-svn] pypy commit 7b8aa74da1fb: Handle output on standard error when invoking Mercurial.
by Maciej Fijalkowski 15 Dec '10

15 Dec '10

This commit contains no tests whatsoever, it would be cooler if it did. On Tue, Dec 14, 2010 at 8:19 PM, Bitbucket <commits-noreply(a)bitbucket.org> wrote: > # HG changeset patch -- Bitbucket.org > # Project pypy > # URL http://bitbucket.org/pypy/pypy/overview > # User Dan Villiom Podlaski Christiansen <danchr(a)gmail.com> > # Date 1292349796 -3600 > # Node ID 7b8aa74da1fb06bd5af8b37e086538e4e7d5d0a1 > # Parent 1f81ed99bfbc17bbb3cc0920ac612218b4194f8c > Handle output on standard error when invoking Mercurial. > > Instead of just piped to the process standard error, we use a logger > to warn if 'hg' wrote anything on stderr. > > --- a/pypy/tool/version.py > +++ b/pypy/tool/version.py > @@ -13,24 +13,39 @@ def get_mercurial_info(): > hgexe = py.path.local.sysfind('hg') > > if hgexe and os.path.isdir(os.path.join(pypyroot, '.hg')): > + def maywarn(err): > + if not err: > + return > + > + from pypy.tool.ansi_print import ansi_log > + log = py.log.Producer("version") > + py.log.setconsumer("version", ansi_log) > + log.WARNING('Errors getting Mercurial information:' + err) > + > env = dict(os.environ) > # get Mercurial into scripting mode > env['HGPLAIN'] = '1' > # disable user configuration, extensions, etc. > env['HGRCPATH'] = os.devnull > > - p = Popen([str(hgexe), 'id', '-i', pypyroot], stdout=PIPE, env=env) > + p = Popen([str(hgexe), 'id', '-i', pypyroot], > + stdout=PIPE, stderr=PIPE, env=env) > hgid = p.stdout.read().strip() > + maywarn(p.stderr.read()) > > - p = Popen([str(hgexe), 'id', '-t', pypyroot], stdout=PIPE, env=env) > + p = Popen([str(hgexe), 'id', '-t', pypyroot], > + stdout=PIPE, stderr=PIPE, env=env) > hgtags = [t for t in p.stdout.read().strip().split() if t != 'tip'] > + maywarn(p.stderr.read()) > > if hgtags: > return 'PyPy', hgtags[0], hgid > else: > # use the branch instead > - p = Popen([str(hgexe), 'id', '-b', pypyroot], stdout=PIPE, env=env) > + p = Popen([str(hgexe), 'id', '-b', pypyroot], > + stdout=PIPE, stderr=PIPE, env=env) > hgbranch = p.stdout.read().strip() > + maywarn(p.stderr.read()) > > return 'PyPy', hgbranch, hgid > else: > _______________________________________________ > pypy-svn mailing list > pypy-svn(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn >

3 2

Re: [pypy-dev] Problem building C Extension
by Kevin Gill 14 Dec '10

14 Dec '10

On Tue, 14 Dec 2010 13:49 +0200, "Maciej Fijalkowski" <fijall(a)gmail.com> wrote: > Not to comment on issue, but it's actually pretty good that > zope.interface C speedups don't work. They won't be speedups in pypy > case even if they did work. > I accept that they may not be speedups. However, I cannot be certain whether C code in an egg represents C alternative to python functionality, or whether the functionality is implemented in C. Perhaps this information is in distutils (setup.py) - I haven't noticed it. > On Tue, Dec 14, 2010 at 12:13 PM, Kevin Gill <kevin(a)movieextras.ie> > wrote: > > Hi, > > > > I am new to PyPy. > > > > I built pypy using... > > > > $ python translate.py --opt=jit targetpypystandalone.py > > > > When I try to build eggs with C extensions it doesn't work. I am using pip in a virtualenv. The build > > uses the wrong compiler "cc" instead of "gcc" and does not use CFLAGS. > > > > For example: > > > > $ pip -v install zope.interface==3.6.1 > > > > shows this output > > > > cc -I/home/kevin/pypp/env/include -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o > > src/zope/interface/_zope_interface_coptimizations.c: In function ‘verifying_changed’: > > src/zope/interface/_zope_interface_coptimizations.c:1349: warning: assignment makes pointer from integer without a cast > > cc -shared build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o -o build/lib.linux-x86_64-2.5/zope/interface/_zope_interface_coptimizations.pypy-14.so > > /usr/bin/ld: build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC > > build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o: could not read symbols: Bad value > > collect2: ld returned 1 exit status > > > > I did some digging through the source. I found that the compiler configuration is normally loaded > > via distutils/sysconfig.py, but that module was effectively disabled in pypy. I cannot override > > using environment variables for example because the customisation function is as follows: > > > > def customize_compiler(options): > > """Dummy method to let some easy_install packages that have > > optional C speedup components. > > """ > > pass > > > > I see from the mailing list that people are compiling C extensions. What am I doing wrong? > > > > Thanks > > > > Kevin > > > > ------------------------------------------------------------ > > Version of pypy that I am using. > > > > $ svn info > > Path: . > > URL: http://codespeak.net/svn/pypy/trunk > > Repository Root: http://codespeak.net/svn > > Repository UUID: fd0d7bf2-dfb6-0310-8d31-b7ecfe96aada > > Revision: 79796 > > Node Kind: directory > > Schedule: normal > > Last Changed Author: afa > > Last Changed Rev: 79794 > > Last Changed Date: 2010-12-03 21:52:49 +0000 (Fri, 03 Dec 2010) > > > > > > > > > > > > _______________________________________________ > > pypy-dev(a)codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > > >

1 0

Problem building C Extension
by Kevin Gill 14 Dec '10

14 Dec '10

Hi, I am new to PyPy. I built pypy using... $ python translate.py --opt=jit targetpypystandalone.py When I try to build eggs with C extensions it doesn't work. I am using pip in a virtualenv. The build uses the wrong compiler "cc" instead of "gcc" and does not use CFLAGS. For example: $ pip -v install zope.interface==3.6.1 shows this output cc -I/home/kevin/pypp/env/include -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o src/zope/interface/_zope_interface_coptimizations.c: In function ‘verifying_changed’: src/zope/interface/_zope_interface_coptimizations.c:1349: warning: assignment makes pointer from integer without a cast cc -shared build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o -o build/lib.linux-x86_64-2.5/zope/interface/_zope_interface_coptimizations.pypy-14.so /usr/bin/ld: build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o: could not read symbols: Bad value collect2: ld returned 1 exit status I did some digging through the source. I found that the compiler configuration is normally loaded via distutils/sysconfig.py, but that module was effectively disabled in pypy. I cannot override using environment variables for example because the customisation function is as follows: def customize_compiler(options): """Dummy method to let some easy_install packages that have optional C speedup components. """ pass I see from the mailing list that people are compiling C extensions. What am I doing wrong? Thanks Kevin ------------------------------------------------------------ Version of pypy that I am using. $ svn info Path: . URL: http://codespeak.net/svn/pypy/trunk Repository Root: http://codespeak.net/svn Repository UUID: fd0d7bf2-dfb6-0310-8d31-b7ecfe96aada Revision: 79796 Node Kind: directory Schedule: normal Last Changed Author: afa Last Changed Rev: 79794 Last Changed Date: 2010-12-03 21:52:49 +0000 (Fri, 03 Dec 2010)

2 1

Migration to mercurial
by Antonio Cuni 14 Dec '10

14 Dec '10

Hi all, finally, it's happening :-). Thanks to Ronny's work, we are going to complete the migration to mercurial very soon. The pypy buildbot is already configured to pull from the bitbucket repository: http://bitbucket.org/pypy/pypy I have tried to run a test build, and it seems to work ok. We will see tomorrow morning how the nightly runs went. If everything goes right, tomorrow we will declare the hg repo as the "official" one: the svn repo will be made read-only (as soon as armin does it :-)), the mercurial one will be synchronized to include the latest svn commits, and from that point everyone should start to commit to mercurial. If you don't have write access to the bitbucket repo, please ask me or anyone else who is an admin. A note for people working on active branches: before you can work on those, you need to perform two simple manual steps, as detailed in this howto: http://codespeak.net/svn/pypy/extradoc/planning/hg-migration/active-branche… ciao, Anto

4 5

Новое сообщение
by Vovan 13 Dec '10

13 Dec '10

http://samec.org.ua/

1 0

setrecursionlimit deprecation?
by Dan Stromberg 12 Dec '10

12 Dec '10

BTW, what's going on with the deprecation of sys.setrecursionlimit? What should I use instead? /home/dstromberg/src/home-svn/backshift/trunk/treap.py:40: DeprecationWarning: setrecursionlimit deprecated sys.setrecursionlimit(min_heap_size)

7 9

Trunk failing to build?
by Dan Stromberg 12 Dec '10

12 Dec '10

Does today's trunk fail to build? I'm getting the following on what I believe to be an unmodified trunk: [Timer] Timings: [Timer] annotate --- 1178.9 s [Timer] rtype_lltype --- 691.9 s [Timer] pyjitpl_lltype --- 1287.3 s [Timer] backendopt_lltype --- 621.4 s [Timer] stackcheckinsertion_lltype --- 101.2 s [Timer] database_c --- 11.8 s [Timer] =========================================== [Timer] Total: --- 3892.4 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 290, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/translator/driver.py", line 812, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/translator/driver.py", line 288, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/translator/driver.py", line 508, in task_database_c [translation:ERROR] database = cbuilder.build_database() [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/translator/c/genc.py", line 157, in build_database [translation:ERROR] sandbox=self.config.translation.sandbox) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/translator/c/database.py", line 65, in __init__ [translation:ERROR] self.gctransformer = self.gcpolicy.transformerclass(translator) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/rpython/memory/gctransform/framework.py", line 494, in __init__ [translation:ERROR] annhelper.finish() # at this point, annotate all mix-level helpers [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/rpython/annlowlevel.py", line 240, in finish [translation:ERROR] self.finish_annotate() [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/rpython/annlowlevel.py", line 259, in finish_annotate [translation:ERROR] ann.complete_helpers(self.policy) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py", line 176, in complete_helpers [translation:ERROR] self.complete() [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py", line 250, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py", line 473, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py", line 533, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py", line 735, in consider_op [translation:ERROR] raise_nicer_exception(op, str(graph)) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py", line 732, in consider_op [translation:ERROR] resultcell = consider_meth(*argcells) [translation:ERROR] File "<3972-codegen /home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/annrpython.py:770>", line 3, in consider_op_simple_call [translation:ERROR] return arg.simple_call(*args) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/unaryop.py", line 174, in simple_call [translation:ERROR] return obj.call(getbookkeeper().build_args("simple_call", args_s)) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/unaryop.py", line 677, in call [translation:ERROR] return bookkeeper.pbc_call(pbc, args) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/bookkeeper.py", line 662, in pbc_call [translation:ERROR] results.append(desc.pycall(schedule, args, s_previous_result)) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/description.py", line 275, in pycall [translation:ERROR] inputcells = self.parse_arguments(args) [translation:ERROR] File "/home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/description.py", line 255, in parse_arguments [translation:ERROR] raise TypeError, "signature mismatch: %s" % e.getmsg(self.name) [translation:ERROR] TypeError': signature mismatch: get_L2cache_linux2() takes exactly 1 argument (0 given) [translation:ERROR] .. v1047 = simple_call((function get_L2cache_linux2)) [translation:ERROR] .. '(pypy.rpython.memory.gc.env:244)estimate_best_nursery_size' [translation:ERROR] Processing block: [translation:ERROR] block@3 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (pypy.rpython.memory.gc.env:244)estimate_best_nursery_size [translation:ERROR] containing the following operations: [translation:ERROR] v1047 = simple_call((function get_L2cache_linux2)) [translation:ERROR] v1048 = simple_call((function best_nursery_size_for_L2cache), v1047) [translation:ERROR] --end-- [translation] start debugger... > /home/dstromberg/src/pypy-src/pypy-for-trunk-builds/trunk/pypy/annotation/description.py(255)parse_arguments() -> raise TypeError, "signature mismatch: %s" % e.getmsg(self.name) (Pdb+)

3 3

gdbm and dbm merge - ctypes bug?
by Dan Stromberg 11 Dec '10

11 Dec '10

I'm finding that my gdbm module (based on ctypes) is infrequently (but it might be consistent) giving gibberish when run on pypy 1.4, but not when run on cpython 2.6 or cpython 3.1. More specifically, sometimes the keys come up looking like undefined memory. The values associated with those keys always seem OK. I believe I am comparing apples to apples - I'm using gdbm_ctypes with the cpython's too, even though they come with a module for some doing the same thing - for the sake of comparison. Is there a known problem with pypy's ctypes? The code is at http://stromberg.dnsalias.org/cgi-bin/viewvc.cgi/gdbm-ctypes/trunk/?root=svn The tests are at http://stromberg.dnsalias.org/cgi-bin/viewvc.cgi/backshift/trunk/?root=svn , and are a bit involved right now, but I could look into finding a minimal code snippet that shows the same problem.

2 2

os.major? os.minor?
by Dan Stromberg 11 Dec '10

11 Dec '10

What would be the best way to add os.major and os.minor to the os module? They appear to be C macros on some systems, so I'm guessing most FFI's aren't going to see them without a wrapper. On Linux, there's a C function that a macro is #ifdef'd to, but that seems less than amazingly portable.

3 8

Leysin Winter sprint: Jan 16-22
by Armin Rigo 10 Dec '10

10 Dec '10

===================================================================== PyPy Leysin Winter Sprint (16-22nd January 2011) ===================================================================== As announced on http://morepypy.blogspot.com/2010/12/leysin-winter-sprint.html, the next PyPy sprint will be in Leysin, Switzerland, for the seventh time. This is a fully public sprint: newcomers and topics other than those proposed below are welcome. ------------------------------ Goals and topics of the sprint ------------------------------ * Now that we have released 1.4, and plan to release 1.4.1 soon (possibly before the sprint), the sprint itself is going to be mainly working on fixing issues reported by various users. Of course this does not prevent people from showing up with a more precise interest in mind. If there are newcomers, we will gladly give introduction talks. * We will also work on polishing and merging the long-standing branches that are around, which could eventually lead to the next PyPy release. These branches are notably: - fast-forward (Python 2.7 support, by Benjamin, Amaury, and others) - jit-unroll-loops (improve JITting of smaller loops, by Hakan) - arm-backend (a JIT backend for ARM, by David) - jitypes2 (fast ctypes calls with the JIT, by Antonio). * And as usual, the main side goal is to have fun in winter sports :-) We can take a day off for ski. ----------- Exact times ----------- The work days should be 16-22 January 2011. People may arrive on the 15th already and/or leave on the 23rd. ----------------------- Location & Accomodation ----------------------- Leysin, Switzerland, "same place as before". Let me refresh your memory: both the sprint venue and the lodging will be in a very spacious pair of chalets built specifically for bed & breakfast: http://www.ermina.ch/. The place has a good ADSL Internet connexion with wireless installed. You can of course arrange your own lodging anywhere (as long as you are in Leysin, you cannot be more than a 15 minutes walk away from the sprint venue), but I definitely recommend lodging there too -- you won't find a better view anywhere else (though you probably won't get much worse ones easily, either :-) Please *confirm* that you are coming so that we can adjust the reservations as appropriate. The rate so far has been around 60 CHF a night all included in 2-person rooms, with breakfast. There are larger rooms too (less expensive) and maybe the possibility to get a single room if you really want to. Please register by svn: http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-winter-2011/people… or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint You need a Swiss-to-(insert country here) power adapter. There will be some Swiss-to-EU adapters around -- bring a EU-format power strip if you have one. ===================================================================== Armin Rigo

1 1