Mailman 3 February 2013 - pypy-dev

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 11 Mar '14

11 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

numpy fails trigonometry with complex numbers, what to do?
by Matti Picus 23 Dec '13

23 Dec '13

I am trying to complete complex numbers in numpypy. Progress is good, I picked up from previous work on the numpypy-complex2 branch. Complex numbers come with extensive tests, it seems all the corner cases are covered. In porting the tests to numpypy, I came across a problem: numpy returns different results than cmath. Some of the differences are due to the fact that numpy does not raise a ValueError for dividing by 0 or other silly input values, but other differences are inexplicable (note the sign of the imaginary part): >>> numpy.arccos(complex(0.,-0.)) (1.5707963267948966-0j) >>> cmath.acos(complex(0.,-0.)) (1.5707963267948966+0j) >>> or this one: >>> cmath.acos(complex(float('inf'),2.3)) -infj >>> numpy.arccos(complex(float('inf'),2.3)) (0.78539816339744828-inf*j) Should I ignore the inconsistencies, or fix the 700 out of 2300 test instance failures? What should pypy's numpypy do - be consistent with numpy or with cmath? cmath is easier and probably faster (no need to mangle results or input args), so I would prefer cmath to trying to understand the logic behind numpy. Matti 2280c2365844

4 4

Slow int code
by Roger Flores 03 Mar '13

03 Mar '13

Hi guys. I've been looking at two simple routines using jitviewer to figure out why they're so much slower than expected. I've also noticed that http://pypy.org/performance.html has the line "Bad examples include doing computations with large longs – which is performed by unoptimizable support code.". I'm worried that my 32 bit int code is falling into this, and I'm wondering what I can do to avoid it? Trivial code like if (self.low ^ self.high) & 0x80000000 == 0: is expanding into several dozen asm instructions. I'm suspecting that lines like self.low = (self.low << 1) & 0xffffffff with it's shift left are convincing the jit to consider the int to need 64 bits (large long?) instead of 32. Ideas? The asm is clearly operating on QWORDs and calling routines to do the bit arithmetic instead of single instructions. Is this what that line in performance.html is warning about? -Roger BTW Fijal's jitviewer is a *must see* for anyone interested in how pypy makes their code fast!

4 11

Early Bird Rates Expire TOMORROW
by Joe Benjamin 28 Feb '13

28 Feb '13

"Early Bird" registration savings and opportunity to submit your company EXPIRE This Tomorrow, Feb. 28th at midnight. A youngStartUp Ventures Conference The Life Sciences & Healthcare Venture Summit Where Innovation Meets Capital MARCH 20, 2013 | Digital Sandbox | New York City Friends, A quick reminder that "early bird" registration savings of 50% and the early submission deadline to present for The 2013 Life Sciences & Healthcare Venture Summit, EXPIRE Tomorrow, February 28th at midnight. Up to fifty emerging-growth entrepreneurial companies will be selected to present at this year's Life Sciences and Healthcare Venture Summit to be held on Wednesday, March 20th in New York City. This is your chance to apply to participate as a presenter before an audience of leading Venture Capitalists, private investors, investment bankers, corporate investors and strategic partners. To Register Now and take advantage of the 50% registration savings before they expire click here. To apply to present: click here. Whether you’re a startup seeking capital and exposure, or an investor seeking new deals, The 2013 Life Sciences and Healthcare venture summit is one event you don't want to miss! MORE THAN FORTY EARLY STAGE VCs ON INTERACTIVE PANELS: Discover what industries they favor, what they are looking for in their next investment, and what you must do to get them excited. Click Here to Register Now! "Early bird" Registration savings expire Tomorrow, February 28th Partial List of VCs and experts confirmed to speak (many more in attendance): 3M New Ventures 5AM Ventures Ascent Biomedical Ventures BD Ventures (Becton Dickinson) CHL Healthcare Partners Cross Atlantic Capital Partners Early Stage Partners Humana Ventures Launch Capital Longitude Capital Millennium Technology Value Partners Mitsui & Co. Global Investment MP Healthcare Venture MVM Life Science Partners New Leaf Venture Partners NGN Capital Novo Ventures Osage University Partners Oxford Bioscience Partners Psilos Qualcomm Ventures Romulus Capital Signet Healthcare Partners Skyline Ventures SR One Technology Partners TEXO Ventures Thomas, McNerney & Partner TPG Biotech Triathlon Medical Ventures SEEKING CAPITAL & EXPOSURE? Get Noticed > Get Funded > Grow Faster 50 companies seeking initial rounds of venture funding will be chosen to present their innovative investment opportunities at The 2013 Life Sciences & Healthcare Venture Summit. If you are looking for capital, don't miss this opportunity to present your company to a leading audience of more than 400 active Venture Capitalists, Angel Investors, Corporate VCs, senior management of emerging companies and potential strategic partners. Featured Company Benefits include: Access and exposure to influential investors Media Exposure Two Page Company Profile published in event guide distributed to all investors and attendees Complimentary passes for 3 company executives Investor Pitch and Presentation Coaching Presentation slot Introductions to leading investors and strategic partners Apply to Present: To be considered for one of the Top Innovator slots, please email iwant2present(a)youngstartup.com for an application. To RSVP by phone or for more information call 212-202-1002. For additional information on presenting opportunities, please visit: http://www.youngstartup.com/lifesciences2013/overview.php HIGH- LEVEL NETWORKING: An excellent opportunity to meet and network with active early stage VCs, angel investors, Corporate VCs, investment bankers, senior executives of emerging companies and strategic partners. FEES: "Early bird" registration savings of 50% EXPIRE Tomorrow, February 28th at midnight Entrepreneurs: Now only $395 Investors: Now only $495 Service Providers: Now only $695 REGISTER NOW and take advantage of the lowest rates available. Click Here to Register Now! We look forward to seeing you there. Sincerely, Joe Benjamin, Founder & CEO youngStartup Ventures PS: If you want to inquire about group rates contact Rivka at 718-447-0009 or rivka(a)youngstartup.com. Founding Sponsor Diamond Sponsor Platinum Sponsor Gold Sponsors Silver Sponsor Marketing Solutions Sponsor Industry Partners For more information about sponsorship contact joe (at) youngstartup.com Unsubscribe: To no longer receive youngStartup Ventures research reports or invitations click here - youngStartup Ventures, Inc. - 258 Crafton Ave. Staten Island, NY, 10314

1 0

continulet stacks on the heap and other schemes for io/threading
by Ronny Pfannschmidt 27 Feb '13

27 Feb '13

Hi, over the last few weeks few ideas have been brooding in the back of my head, in particular after seeing how rust creates its stacks and handles io. The basis is allocating all stacks as non-movable structures on the heap. This would remove the need to copy the c/rpython level stack for continulets and enable to move them between native threads which is essentially enabling a M:N threading scheme. On top of that i would like to introduce transformation similar to the sandbox that would defer all IO to a io loop in a separate thread. Additionally it should change the threading abstractions to use said continuation instead of os level threads the result in case of a success would be a python that defers all blocking operations to a io loop in a separate thread im currently investigating libuv, since it does well for async io and also has utilities to defer blocking calls to c code to a pool of native threads -- Ronny

3 2

cffi in stdlib
by Maciej Fijalkowski 27 Feb '13

27 Feb '13

Hello. I would like to discuss on the language summit a potential inclusion of cffi[1] into stdlib. This is a project Armin Rigo has been working for a while, with some input from other developers. It seems that the main reason why people would prefer ctypes over cffi these days is "because it's included in stdlib", which is not generally the reason I would like to hear. Our calls to not use C extensions and to use an FFI instead has seen very limited success with ctypes and quite a lot more since cffi got released. The API is fairly stable right now with minor changes going in and it'll definitely stablize until Python 3.4 release. Notable projects using it: * pypycore - gevent main loop ported to cffi * pgsql2cffi * sdl-cffi bindings * tls-cffi bindings * lxml-cffi port * cairo-cffi * pyzmq * a bunch of others So relatively a lot given that the project is not even a year old (it got 0.1 release in June). As per documentation, the advantages over ctypes: * The goal is to call C code from Python. You should be able to do so without learning a 3rd language: every alternative requires you to learn their own language (Cython, SWIG) or API (ctypes). So we tried to assume that you know Python and C and minimize the extra bits of API that you need to learn. * Keep all the Python-related logic in Python so that you don’t need to write much C code (unlike CPython native C extensions). * Work either at the level of the ABI (Application Binary Interface) or the API (Application Programming Interface). Usually, C libraries have a specified C API but often not an ABI (e.g. they may document a “struct” as having at least these fields, but maybe more). (ctypes works at the ABI level, whereas Cython and native C extensions work at the API level.) * We try to be complete. For now some C99 constructs are not supported, but all C89 should be, including macros (and including macro “abuses”, which you can manually wrap in saner-looking C functions). * We attempt to support both PyPy and CPython, with a reasonable path for other Python implementations like IronPython and Jython. * Note that this project is not about embedding executable C code in Python, unlike Weave. This is about calling existing C libraries from Python. so among other things, making a cffi extension gives you the same level of security as writing C (and unlike ctypes) and brings quite a bit more flexibility (API vs ABI issue) that let's you wrap arbitrary libraries, even those full of macros. Cheers, fijal .. [1]: http://cffi.readthedocs.org/en/release-0.5/

2 3

translating pypy benchmarks to C
by Ghitulete Razvan 25 Feb '13

25 Feb '13

Hi, I've been trying for some time now to translate python code into C. After playing around with the pypy interactive translator shell and worked just fine. But the I tried to translate the pypy benchmarks from speed.pypy.orginto C code, but I seem to be running into all kinds of trouble. So far I've tried with bm_ai.py which seems to fail because it uses closures(or so i'm told by the translator), and bm_threading.py seems to fail while processing threading.py. Is there something I'm doing wrong? P.S.: I'm simply running the translator.py script with -s option on slightly modified versions of the above mentioned files(adding an entry point). -- Sincerely, Razvan Ghitulete Universitatea Politehnica Bucuresti

8 17

Updates to hgignore for rpython split
by Justin Bogner 25 Feb '13

25 Feb '13

The .hgignore file got missed when we moved rpython/ out of pypy/. Can someone take a look at this patch and commit it if it looks good? Thanks.

1 0

how to check if jit is available in my build
by Davide Del Vento 23 Feb '13

23 Feb '13

Folks, I compiled pypy 1.9 and 2.0-beta1 from source, and the few small tests I ran were slower than expected. I am wondering if I did everything "right" and if there is a runtime check that would give me a definitive answer to the question "is jit available in this build"? Google seems to not know the answer. Thanks Davide

2 2

[rpython] What might prevent a function call from beeing invlined by the JIT?
by Костя Лопухин 22 Feb '13

22 Feb '13

In what cases does the jit decide not to inline a function call, but place "call_may_force" instead? The context is that I have a simple interpreter, like an expanded kermit, and I am testing how the jit helps - it perfectly unboxes wrapped objects in a loop, but does not inline function calls - I wonder what am I missing here. I found a place where PyPy interpreter makes a call and did not see anything special there.

2 3