Mailman 3 June 2012 - pypy-dev

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

MinGW32 support PyPy with mscr90.dll
by bookaa 19 Sep '12

19 Sep '12

Windows system, after set cc=mingw32-gcc PyPy can not pass many tests: pytest.py pypy/translator/test/test_unsimplify.py pytest.py pypy/translator/c/test/test_extfunc.py::test_dictlike_environ_getitem ... I find this is because MinGW32 gcc compile default use msvcrt.dll, but we need msvcr90.dll, as Python 2.7 use msvcr90.dll. >>> import ctypes >>> ctypes.util.find_msvcrt() 'msvcr90.dll'And even python.exe dependence on msvcr90.dllref to http://trac.seqan.de/wiki/HowTo/SelectASpecificMsvcrtVersionWithMinGw this is how I make MinGW32 gcc use msvcr90.dll instead: save these to you MinGW directory as file D:\MinGW\lib\gcc\mingw32\4.6.2\specs *cpp: %{posix:-D_POSIX_SOURCE} %{mthreads:-D_MT} -D__MSVCRT_VERSION__=0x0900 *libgcc: %{mthreads:-lmingwthrd} -lmingw32 %{shared-libgcc:-lgcc_s} %{!shared-libgcc:-lgcc_eh} -lgcc -lmoldname90 -lmingwex -lmsvcr90 I suggest this instruction should be add to PyPy doc: MinGW32 support

2 1

Developer selection for Py3k
by Antonio Cuni 31 Aug '12

31 Aug '12

Hi all, as you probably know, the Py3k [1] proposal is getting funded thanks to our generous donors. During the first round, three of use were selected: me, Alex Gaynor and Benjamin Peterson. However, due to unforeseen unavailability of Alex and Benjamin, we are now looking for one more developer to help with the py3k work. If you are interested in getting paid to work on the Py3k proposal, please apply by replying to this email. To be applicable you need to be an experienced PyPy developer, preferably with a previous experience on the Python Interpreter part of PyPy. ciao, Anto

6 7

Help (again)
by Léonard de Haro 08 Aug '12

08 Aug '12

Hi everyone! First of all, thaks for your answers last time. Second of all, I need your help again. I'm coding an imperative version of my interpret and to do so, I coded continuations. I used a similat technique to the one used in the Prolog interpret you advised me to take a look at last time. I define a serie of class all inheriting from a simulated-abstract Contk class. Every class has a method apply which contains instruction for what's next to do. All these methods take the same arguments and (except in one case) return the same kind of tuple. The interpret function uses 4 variables that are updated: expr, env, cont, val. Thus, the result of .apply(_,_,_,_) is the tuple for update. class Contk(object): def __init__(self,*arg): raise NotImplementedError("For abstract class") def apply(self,expr,env,val): raise NotImplementedError("For abstract class") class Endk(Contk): def __init__(self,val): self.val = val def apply(self,expr, env, val): return self.val class Idk(Contk): def __init__(self): pass def apply(self, expr, env, val): return expr, env, Endk(val), val There is a class for each continuation needed by the interpret. The trouble is, I get this error: [translation:ERROR] AnnotatorError': annotation of 'union' degenerated to SomeObject() [translation:ERROR] Simple call of incompatible family: [translation:ERROR] (KeyError getting at the binding!) [translation:ERROR] [translation:ERROR] In <FunctionGraph of (RPinterpretImperative:27)Idk.apply at 0x1b31be8>: [translation:ERROR] Happened at file RPinterpretImperative.py line 28 [translation:ERROR] [translation:ERROR] ==> return expr, env, Endk(val), val [translation:ERROR] [translation:ERROR] Previous annotation: [translation:ERROR] (none) [translation:ERROR] .. v0 = simple_call((classobj Endk), val_0) [translation:ERROR] .. '(RPinterpretImperative:27)Idk.apply' [translation:ERROR] Processing block: [translation:ERROR] block@12 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (RPinterpretImperative:27)Idk.apply [translation:ERROR] containing the following operations: [translation:ERROR] v0 = simple_call((classobj Endk), val_0) [translation:ERROR] v1 = newtuple(expr_0, env_0, v0, val_0) [translation:ERROR] --end-- So at first I thought it was due to tuple that would not accept newly created objects, but after a few tests, it appears that not. Then I tought the annotator couldn't realize by himself that returning an Endk object is the same as returning a Contk, so I annotated the whole file with assertion to force him to realize that my variable cont should be a Contk not any other kind of subclass, but it didn't work either. So I don't know what to do now. I've even tried to implement apply outside of the classes, or to make a function for each parameters returned, but nothing worked. You can find the source here : https://github.com/zebign1/RPython-internship/tree/master/Interpreters/ifF1… (treeClass.py and parser.py as tools, the bugged file is RPinterpretImperative.py) Thanks again. Leonard

3 3

Hello! and Video Tutorials
by Paulo Köch 04 Jul '12

04 Jul '12

Hello! My name is Paulo Köch. I'm from Portugal and looking to contribute to PyPy. Right now, I'm still just digesting the available docs. I'll bet back to you as soon as I feel informed enough to start hacking. In my quest for documentation, I stumbled on the videos and their torrents. However, the tracker at http://codespeak.net:80/ always replies with something that my BT client doesn't understand. Going there with a browser, the site seems to be somewhat defunct, containing just links to other sites. So: * Where can I find these videos? * Why aren't they on youtube or something? Cheers! Paulo Köch

6 12

Translation with pypy picking up wrong libraries
by David Naylor 30 Jun '12

30 Jun '12

Hi, I'm experimenting with using non-default library directories by tweeking pypy.tool.lib_pypy however I came up with the following error when translating with pypy: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 274, in main [translation:ERROR] default_goal='compile') [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 810, in from_targetspec [translation:ERROR] spec = target(driver, args) [translation:ERROR] File "targetpypystandalone.py", line 228, in target [translation:ERROR] return self.get_entry_point(config) [translation:ERROR] File "targetpypystandalone.py", line 236, in get_entry_point [translation:ERROR] rebuild = import_from_lib_pypy('ctypes_config_cache/rebuild') [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/tool/lib_pypy.py", line 12, in import_from_lib_pypy [translation:ERROR] return modname.pyimport() [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/py/_path/local.py", line 531, in pyimport [translation:ERROR] mod = __import__(modname, None, None, ['__doc__']) [translation:ERROR] File "/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/rebuild.py", line 11, in <module> [translation:ERROR] execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py)) [translation:ERROR] IOError: [Errno 2] No such file or directory: '/usr/local/pypy-1.9/pypy/tool/autopath.py' This error does not happen when translating with python. Upon investigation I found that pypy has: # pypy -c 'import sys; import pypy.tool.lib_pypy; print "ctypes_config_cache" in sys.modules.keys()' True but python has: # python -c 'import sys; import pypy.tool.lib_pypy; print "ctypes_config_cache" in sys.modules.keys()' False also: # pypy -c 'import sys; import pypy.tool.lib_pypy; print sys.modules["ctypes_config_cache"]' <module 'ctypes_config_cache' from '/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/__init__.pyc'> note the above always happens regardless of pypy.tool.lib_pypy being modified or not. It appears to me that there are two bugs: 1) pypy ends up importing ctypes_config_cache when it shouldn't??? 2) py/_path/local.py doesn't ensure import from correct location And attached is a patch for fixing (2), which works around (1). Regards

3 7

5k mud runner & 2k family mud runner
by Mud - Obstacles - Beer & Music 29 Jun '12

29 Jun '12

v\:* {behavior:url(#default#VML);}o\:* {behavior:url(#default#VML);}w\:* {behavior:url(#default#VML);}.shape {behavior:url(#default#VML);} NormalEmail.dotm 1 0 2012-06-07T19:39:00Z 1 1038 5714 47 13 6739 12.00 0 21 false false false FR X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Tableau Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";}   More information: click here. Testing your endurance is only the beginning. Having fun with friends, family and mud is pretty awesome and the after PARTY promises to be ... well a party. Music, rockin out on a Saturday afternoon, Cold beer and other beverages, food for body and soul (that's the beer). Yup and your not gonna find grandmas music either. This is your party... OK...We know that Maple Ski Ridge's vertical will not leave you breathless. Elevation, yup we have some... But for what we lack in straight up- we will more than make up for in sheer craziness in "Features"; Mud..water.. ledges..tires(obligatory).. hills..slop and all other types of things for you to go over -thru -around -and under.   We don't think you will be disappointed. What should you bring???? Unless Great Grams parties, leave her at home. White T shirts that you no longer love are OK.  Better yet, your younger sisters/brothers clothes are a great idea... And a towel to dry off. Yup and if you need clothes to party in after... More info will be posted here as we go. Such as music line up, how many registrations and pictures of the event in the making.  Stay tuned....and sign up now.    WHEN:    Saturday July 28th 2012. Need we say rain or shine. MUD! 9:30 Streaching warm up 10:00 Briefing 10:15 5k mud start 11:00 2k family mud start 11:30 Streaching relaxation 12:00 Prices 12:30 Costumes awards 12:30 BBQ + Cold beer + Music (live band)       WHERE:     Maple Ski Ridge   2725 Mariaville Road   Schenectady NY 12306  HOW?    Registration as a participant: Click here.     HOW MUCH?  5K Mud Run -  $45.00 pp. First 100 get free Run the Ridge t-shirt, beverage and dog. 2K Family Mud Run $20.00 per adult and $10.00 per munchkin(10 and under).       Sponsor opportunities: For your company, for your products, for your team, advertise your brand (TV, Logos, Booth, Tent...). Or just because Run the Ridge is currently seeking sponsors to build the best and most interesting event. Click here, we will send you our "Run the Ridge sponsorship brochure". Register now as a sponsor: Click here.      Powered by  

1 0

Re: [pypy-dev] [pypy/pypy] rbigint.py improvements (pull request #72)
by Armin Rigo 27 Jun '12

27 Jun '12

Hi Stian, On Fri, Jun 22, 2012 at 3:59 AM, Stian Andreassen <pullrequests-noreply(a)bitbucket.org> wrote: > https://bitbucket.org/pypy/pypy/pull-request/72/rbigintpy-improvements > Title: rbigint.py improvements I suppose this needs discussion on pypy-dev. It is great, but the major issue is that it uses "__int128", which is an integer type that is only sometimes available --- and more often not: it is present on MSVC, but for GCC it requires 4.6 (and likely 64-bit machines, not 32-bit ones). There is no way we can just merge it into trunk: it would stop compiling on most of our buildbots, to start with. So the discussion is about what to do from there. Stian, I assume you'd have to add the proper conditional compilation and support the case of no int128, unless we can think of something better. A bientôt, Armin.

1 0

Placement Site - Offer
by Celia Connoley 26 Jun '12

26 Jun '12

pypy-dev(a)codespeak.net : Your site has a good look and feel but you're not getting as much web traffic as you could be getting. Can we show you how to change that? It won't cost you anything for us to review your online business and the results could spike your web revenue. Reply to us today and we will do a free assessment of your web presence. Don't forget to include the best way we can contact you. Sincerely, Celia Connoley

1 0

CLI translate error with pypy-1.9
by David Naylor 22 Jun '12

22 Jun '12

Hi, FYI, translating pypy for the CLI backend using mono-2.11.1 in a 32bit environment results in the following error: [Timer] Timings: [Timer] annotate --- 309.3 s [Timer] rtype_ootype --- 326.7 s [Timer] ========================================== [Timer] Total: --- 636.0 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 791, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 285, in _do [translation:ERROR] res = func() [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 383, in task_rtype_ootype [translation:ERROR] crash_on_first_typeerror=insist) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 211, in specialize [translation:ERROR] self.specialize_more_blocks() [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 254, in specialize_more_blocks [translation:ERROR] self.specialize_block(block) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 408, in specialize_block [translation:ERROR] self.gottypererror(e, block, hop.spaceop, newops) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 406, in specialize_block [translation:ERROR] self.translate_hl_to_ll(hop, varmapping) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 535, in translate_hl_to_ll [translation:ERROR] resultvar = hop.dispatch() [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 768, in dispatch [translation:ERROR] return translate_meth(self) [translation:ERROR] File "<18375-codegen /tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py:610>", line 4, in translate_op_simple_call [translation:ERROR] return r_arg1.rtype_simple_call(hop) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rbuiltin.py", line 126, in rtype_simple_call [translation:ERROR] return self._call(hop2) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rbuiltin.py", line 117, in _call [translation:ERROR] v_result = bltintyper(hop2, **kwds_i) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rlib/jit.py", line 802, in specialize_call [translation:ERROR] v_value = hop.inputarg(repr, arg=2) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 788, in inputarg [translation:ERROR] return self.llops.convertvar(v, r_binding, converted_to) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 933, in convertvar [translation:ERROR] (r_from, r_to)) [translation:ERROR] TyperError: don't know how to convert from <StringRepr String> to <StringRepr * GcStruct rpy_string { hash, chars }> [translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts [translation:ERROR] .. block@12 with 1 exits [translation:ERROR] .. v552 = simple_call((function _set_param), driver_0, name_0, value_0) I know the CLI backend is not really supported but I am sharing this in the event someone is interested. Regards, David

4 12