Mailman 3 May 2016 - pypy-dev

Game state search dominated by copy.deepcopy
by Eli Stevens (Gmail) May 16, 2016

May 16, 2016

Hello, I'm in the process of working on a hobby project to have an AI searching through a game state space. I recently ran what I have so far on pypy (I had been doing initial work on cpython), and got two results that were unexpected: - The total execution time was basically identical between cpython and pypy - The runtime on both pythons was about 50% copy.deepcopy (called on the main game state object) The runtime of the script that I've been using is in the 30s to 2m range, depending on config details, and the implementation fits the following pretty well: """ The JIT is generally good at speeding up straight-forward Python code that spends a lot of time in the bytecode dispatch loop, i.e., running actual Python code – as opposed to running things that only are invoked by Python code. Good examples include numeric calculations or any kind of heavily object-oriented program. """ I have already pulled out constant game information (static info like unit stats, etc.) into an object that isn't copied, and a lot of the numerical data that is copied is stored in a numpy array so that I don't have hundreds of dicts, ints, etc. First, is there a good way to speed up object copying? I've tried pickling to a cache, and unpickling from there (so as to only pickle once), but that didn't make a significant difference. http://vmprof.com/#/905dfb71d28626bff6341a5848deae73 (deepcopy) http://vmprof.com/#/545f1243b345eb9e41d73a9043a85efd (pickle) Second, what's the best way to start figuring out why pypy isn't able to outperform cpython on my program? Thanks for any pointers, Eli

2 3

call for papers: Dynamic Language Symposium 2016, Nov 1, Amsterdam
by Carl Friedrich Bolz May 12, 2016

May 12, 2016

Call for papers ## 12th Dynamic Languages Symposium (DLS 2016) ### Co-located with SPLASH 2016 ### In association with ACM SIGPLAN ### November 1, 2016, Amsterdam <http://www.dynamic-languages-symposium.org/dls-16/index.html> The 12th Dynamic Languages Symposium (DLS) at SPLASH 2016 invites high quality papers reporting original research and experience related to the design, implementation, and applications of dynamic languages. Areas of interest include but are not limited to: * Innovative language features * Innovative implementation techniques * Innovative applications * Development environments and tools * Experience reports and case studies * Domain-oriented programming * Very late binding, dynamic composition, and run-time adaptation * Reflection and meta-programming * Software evolution * Language symbiosis and multi-paradigm languages * Dynamic optimization * JIT compilation * Soft/optional/gradual typing * Hardware support * Educational approaches and perspectives * Semantics of dynamic languages ### Submissions and proceedings Submissions must not have been published previously nor being under review at other events. Research papers should describe work that advances the current state of the art. Experience papers should be of broad interest and should describe insights gained from substantive practical applications. The program committee will evaluate each contributed paper based on its relevance, significance, clarity, and originality. Papers are to be submitted electronically at <https://dls16.hotcrp.com/> in PDF format. Submissions must be in the ACM format with 10-point fonts and should not exceed 12 pages. Please see full details in the following link: <http://conf.researchr.org/track/dls-2016/dls-2016-papers#Instructions-for-A…> DLS 2016 will run a two-phase reviewing process to help authors make their final papers the best that they can be. Accepted papers will be published in the ACM Digital Library and will be freely available for one month, starting two weeks before the event. ### Important dates * Submissions: Jun 10, 2016 (UTC, firm deadline) * First phase notification: Jul 22, 2016 * Revisions due: July 29, 2016 * Final notification: Aug 14, 2016 * Camera ready: Aug 26, 2016 * DLS: Nov 1, 2016 **AUTHORS TAKE NOTE:** The official publication date is the date the proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of your conference. The official publication date affects the deadline for any patent filings related to published work. ### Program chair Roberto Ierusalimschy, PUC-Rio, Brazil dls16(a)inf.puc-rio.br ### Program committee * Carl Friedrich Bolz, King’s College London, UK * Gilad Bracha, Google, USA * Marcus Denker, INRIA, France * Zachary DeVito, Stanford, USA * Jonathan Edwards, CDG Labs, USA * Matthew Flatt, University of Utah, USA * Elisa Gonzalez Boix, Vrije Universiteit Brussel, Belgium * Robert Hirschfeld, Hasso Plattner Institute Potsdam, Germany * Roberto Ierusalimschy, PUC-Rio, Brazil (chair) * Shriram Krishnamurthi, Brown University, USA * Benjamin Livshits, Microsoft Research, USA * Priya Nagpurkar, IBM Research, USA * Joe Gibbs Politz, Swarthmore College, USA * Chris Seaton, Oracle Labs, UK * Manuel Serrano, INRIA, France * Sam Tobin-Hochstadt, Indiana University, USA * Laurence Tratt, King’s College London, UK * Jan Vitek, Northeastern University, USA * Haichuan Wang, Huawei America Research Center, USA

1 0

Copying a slice of a list in RPython
by Omer Katz May 10, 2016

May 10, 2016

Hi guys, While implementing the BList strategy I found a need to be able to efficiently copy part of a list to another list (and sometimes to the same list). I'd rather avoid implementing something like https://github.com/DanielStutzbach/blist/blob/master/blist/_blist.c#L189 if it already exists in RPython. Someone on IRC mentioned that there is a function in the rgc module called ll_arraycopy but I'm unsure how to use it. I could implement it myself but I don't think it should live under objspace/listobject.py. Is there a utility module for objspace I should put those functions in? Thanks, Omer Katz.

2 1

PyPy & Django: recommended mySQL module?
by Yury V. Zaytsev May 9, 2016

May 9, 2016

Hi, I'm thinking of trying PyPy on a Python 2.7 application based on Django, and I'm wondering what mySQL module would be the recommended one these days to use for the ORM backend? I understand that the lastest version MySQL-Python should just work, although will be going through CPyExt, so I'm not sure how stable it's gonna be / what's the performance going to look like. It seems that there is also a pure Python connector called PyMySQL, which should work out of the box and not rely on CPyExt. Has anyone done any recent benchmarks and/or can give me a hint which of the two is currently the way to go? Many thanks! -- Sincerely yours, Yury V. Zaytsev

5 8

BList strategy
by Omer Katz May 8, 2016

May 8, 2016

https://github.com/DanielStutzbach/blist/ is an implementation of lists that is asymptotically faster. Quoting from them: "The blist uses a flexible, hybrid array/tree structure and only needs to move a small portion of items in memory, specifically using O(log n) operations. For small lists, the blist and the built-in list have virtually identical performance." I looked into PyPy's current list strategies and it seems we're not implementing the same algorithm as blist. I had a chance to use the package before and the implementation is sound. I'm wondering if PyPy could by a chance benefit from implementing the blist algorithm as a strategy. Possibly even switching it to be the default. What do you guys think? Omer Katz

2 4

Create own windows distribution - Borrowing from github repo and conda
by Sayth Renshaw May 8, 2016

May 8, 2016

Hi Wanting to know from your experience how hard it would be to create a python distribution for windows based on pypy ? For example if it contained the "essential" and sometimes hardest packages plus a few of the biggest names would this be feasible? For example as packages numpy, django, scikit, matplotlib or bokeh and spyder and ipython. For example in the anaconda distribution there is a github repo https://github.com/rvianello/conda-pypy which has created the option to replace the conda config and direct all packages to a different repo and use pypy as the default distribution. using something like that as a base and borrowing from the conda base would this be feasible, got to be good from an adoption stand point. Thoughts? Sayth

2 1

Fwd: Bounce action notification
by Armin Rigo May 8, 2016

May 8, 2016

Hi all, I wanted to communicate this to whoever added the pypy-dev list to "all-mail-archive.com": the subsciption has been disabled---excessive or fatal bounces. Armin

1 0

How can I retrieve target-level language's bytecode in functions of JitHookInterface?
by 黄若尘 May 5, 2016

May 5, 2016

Hi all, My name is Ruochen Huang, I’m developing an Erlang bytecode virtual machine called Pyrlang. Recently I’m trying to apply a new tracing JIT policy to Pyrlang, I found there are some traces that were too long and therefore aborted by JIT according to rpython’s JIT log. In order to find those ‘too long’ traces in my benchmark programs, I’m trying to implement the JitHookInterface in rpython, and rewrite the on_abort function there. However, until now I can only print out the rpython level bytecode instructions from the ‘operations’ arguments, but what I want are the target language (e.g., Erlang in my project) bytecode instructions. Could somebody tell me that is it possible to retrieve it by calling some rpython API? Best Regards, Ruochen Huang Tokyo Institute of Technology

2 1

PyPy 5.1.1 released
by Matti Picus May 3, 2016

May 3, 2016

I have released a bug-fix for 5.1, the blog post has details http://morepypy.blogspot.com/2016/05/pypy-511-bugfix-released.html

1 0

cpyext-auto-gil?
by Armin Rigo May 1, 2016

May 1, 2016

Hi all, hi Matti, If a CPython C extension module is written in a way that calls some PyXxx() function without the GIL (hoping that the code in this particular function is kept simple enough that it still works), then in the same situation it would loudly complain and crash on PyPy. This is a problem for numpy, which had to be worked around. Now I did a minimal change in the branch cpyext-auto-gil: instead of complaining, we acquire/release the GIL. Does it sound like a good idea? Could you check if it solves the numpy issue? A bientôt, Armin.

2 1