Mailman 3 April 2014 - pypy-dev

Disabling garbage collection
by Martin Koch 04 Apr '14

04 Apr '14

Hi List Is there a simple way of building a version of pypy where garbage collection of the old generation is disabled? I have a very large, fairly static graph in memory, and I would rather not traverse it all on major collections. I don't care too much if there are leaks, since my application runs on a host with lots of ram, can be restarted fairly often, and real-time performance is preferable to blocking because of garbage collection. I have tried just removing the body of incminimark.py/major_collection_step(), but this causes pypy to freeze up after a while. Thanks, /Martin Koch

2 1

Dynamic Languages Symposium: Call for Papers
by Edd Barrett 04 Apr '14

04 Apr '14

=========================================================================== Dynamic Languages Symposium 2014 October 21, 2014 Co-located with SPLASH 2014, Portland, OR, USA http://www.dynamic-languages-symposium.org/dls-14/ =========================================================================== The 10th Dynamic Languages Symposium (DLS) at SPLASH 2014 is the premier forum for researchers and practitioners to share knowledge and research on dynamic languages, their implementation, and applications. The influence of dynamic languages -- from Lisp to Smalltalk to Python to Javascript -- on real-world practice, and research, continues to grow. DLS 2014 invites high quality papers reporting original research, innovative contributions, or experience related to dynamic languages, their implementation, and applications. Accepted papers will be published in the ACM Digital Library, and freely available for 2 weeks before and after the event itself. Areas of interest include but are not limited to: * Innovative language features and implementation techniques * Development and platform support, tools * Interesting applications * Domain-oriented programming * Very late binding, dynamic composition, and run-time adaptation * Reflection and meta-programming * Software evolution * Language symbiosis and multi-paradigm languages * Dynamic optimization * Hardware support * Experience reports and case studies * Educational approaches and perspectives * Semantics of dynamic languages Submissions Submissions should not have been published previously nor be under review at other events. Research papers should describe work that advances the current state of the art. Experience papers should be of broad interest and should describe insights gained from substantive practical applications. The program committee will evaluate each contributed paper based on its relevance, significance, clarity, length, and originality. Papers are to be submitted electronically at http://www.easychair.org/conferences?conf=dls14 in PDF format. Submissions must be in the ACM format (see http://www.sigplan.org/authorInformation.htm) and not exceed 12 pages. Authors are reminded that brevity is a virtue. DLS 2014 will run a two-phase reviewing process to help authors make their final papers the best that they can be. After the first round of reviews, papers will be rejected, conditionally accepted, or unconditionally accepted. Conditionally accepted papers will be given a list of issues raised by reviewers. Authors will then submit a revised version of the paper with a cover letter explaining how they have / why they have not addressed these issues. The reviewers will then consider the cover letter and revised paper and recommend final acceptance / rejection. Important dates Submissions: June 8 2014 (FIRM DEADLINE) First phase notification: July 14 2014 Revisions due: August 4 2014 Final notification: August 11 2014 Camera ready: August 15 2014 DLS: October 21 2014 Programme chair Laurence Tratt, King's College London, UK e-mail: dls14(a)easychair.org Publicity chair Edd Barrett, King's College London, UK Programme committee Gilad Bracha, Google, US Jonathan Edwards, MIT, US Robert Hirschfeld, Hasso-Plattner-Institut Potsdam, DE Roberto Ierusalimschy, PUC-Rio, BR Sergio Maffeis, Imperial College London, UK Stefan Marr, INRIA, FR Oscar Nierstrasz, University of Bern, CH James Noble, Victoria University of Wellington, NZ Shriram Krishnamurthi, Brown University, US Chris Seaton, University of Manchester, UK Nikolai Tillmann, Microsoft Research, US Sam Tobin-Hochstadt, Indiana University, US Jan Vitek, Purdue University, US Christian Wimmer, Oracle Labs, US Peng Wu, IBM Research, US

1 0

Why CFFI is not useful - need direct ABI access 4 humans
by anatoly techtonik 03 Apr '14

03 Apr '14

I know what C in CFFI stands for C way of doing things, so I hope people won't try to defend that position and instead try to think about what if we have to re-engineer ABI access from scratch, for explicit and obvious debug binary interface. CFFI is not useful for Python programmers and here is why. The primary reason is that it requires you to know C. And knowing C requires you to know about OS architecture. And knowing about OS architecture requires you to know about ABI, which is: http://stackoverflow.com/a/3784697 This is how the compiler builds an application. It defines things (but is not limited too): How parameters are passed to functions (registers/stack). Who cleans parameters from the stack (caller/callee). Where the return value is placed for return. How exceptions propagate. The problematic part of it is that you need to think of OS ABI in terms of unusual C abstractions. Coming through several levels of them. Suppose you know OS ABI and you know that you need direct physical memory access to set bytes for a certain call in this way: 0024: 00 00 00 6C 33 33 74 00 How would you do this in Python? The most obvious way is with byte string - \x00\x00\x00\x6c\x33\x33\x74\x00 but that's not how you prepare the data for the call if, for example, 00 6C means anything to you. What is the Python way to convert 00 6C to convenient Python data structure and back and is it Pythonic (user friendly and intuitive)? import struct struct.unpack('wtf?', '\x00\x6C') If you try to lookup the magic string in struct docs: http://docs.python.org/2/library/struct.html#format-characters You'll notice that there is the mapping between possible combinations of these 2 bytes to some Python type is very mystic. First it requires you to choose either "short" or "unsigned short", but that's not enough for parsing binary data - you need to figure out the proper "endianness" and make up a magic string for it. This is just for two bytes. Imagine a definition for a binary protocol with variable message size and nested data structures. You won't be able to understand it by reading Python code. More than that - Python *by default* uses platform specific "endianness", it is uncertain (implicit) about it, so not only you should care about "endianness", but also be an expert to find out which is the correct metrics for you. Look at this: 0024: 00 00 00 6C 33 33 74 00 Where is "endianness", "alignment", "size" from this doc http://docs.python.org/2/library/struct.html#byte-order-size-and-alignment People need to *start* with this base and this concept and that's why it is harmful. CFFI proposes to provide a better interface to skip this complexity by getting back to roots and use C level. That's a pretty nice hack for C guys, I am sure it makes them completely happy, but for academic side of PyPy project, for Python interpreter and other projects build over RPython it is important to have a tool that allows to experiment with binary interfaces in convenient, readable and direct way, makes it easier for humans to understand (by reading Python code) how Python instructions are translated by JIT into binary pieces in computer memory, pieces that will be processed by operating system as a system function call on ABI level. But let's not digress, and get back to the point that struct module doesn't allow to work with structured data. In Python the only alternative standard way to define binary structure is ctypes. ctypes documentation is no better for binary guy: http://docs.python.org/2/library/ctypes.html#fundamental-data-types See how that binary guy suffered to map binary data to Python structures through ctypes: https://bitbucket.org/techtonik/discovery/src/eacd864e6542f14039c9b31eecf94… And I am saying that this is the best way available from standard library. It is pretty close to Django models, but for binary data. ctypes still is worse that struct in one thing - looking into docs, there are no size specifiers for any kind of C type, so no guarantee that 2 bytes are read as 4 bytes or worse. By looking at the ctypes code it is hard to figure out size of structure and when it may change. I can't hardly name ctypes mapping process as user friendly and resulting code as intuitive. Probably nobody could, and that's why CFFI was born. But CFFI took a different route - instead of trying to map C types to binary data (ABI level), it decided to go onto API level. While it exposes many better tool, it basically means you are dealing with C interface again - not with Pythonic interface for binary data. I am not saying that CFFI is bad - I am saying that it is good, but not enough, and that it can be fixed with cleanroom engineering approach for a broader scope of modern usage pattern for binary data than just calling OS API in C way. Why we need it? I frankly think that Stackless way of doing thing without C stack is the future, and the problem with not that not many people can see how it works, builds alternative system without classic C stack with (R)Python. Can CFFI help this? I doubt that. So, that am I proposing. Just an idea. Given the fact that I am mentally incapable of filling 100 sheet requirement to get funding under H2020, the fact that no existing commercial body could be interested to support the development as an open source project and the fact that hacking on it alone might become boring, giving this idea is the least I can do. Cleanroom engineering. http://en.wikipedia.org/wiki/Cleanroom_software_engineering "The focus of the Cleanroom process is on defect prevention, rather than defect removal." When we talk about Pythonic way of doing thing, how can we define "a defect"? Basically, we talking about user experience - the emotions that user experiences when he uses Python for the given task. What is the task at hand? For me - it is working with binary data in Python - not just parsing save games, but creating binary commands such as OS systems calls that are executed by certain CPU, GPU or whatever is on the receiver end of whatever communication interface is used. This is hardware independent and platform neutral way of doing things. So, the UX is the key, but properties of engineered product are not limited single task. The cleanroom approach allows to concentrate on the defect - when user experience will start to suffer because of the conflicts between tasks that users are trying to accomplish. For PyPy project I see the value in library for compositing of binary structures in that these operations can be pipelined and optimized at run-time in a highly effective fashion. I think that convenient binary tool is the missing brick in the basement of academic PyPy infrastructure to enable universal interoperability from (R)Python with other digital systems by providing a direct interface to the binary world. I think that 1973 year views on "high level" and "low level" systems are a little bit outdated now that we have Python, Ruby, Erlang and etc. Now C is just not a very good intermediary for "low level" access. But frankly, I do not think that with advent of networking, binary can be called a low level anymore. It is just another data format that can be as readable for humans as program structure written in Python. P.S. I have some design ideas how to make an attractive gameplay out of binary data by "coloring" regions and adding "multi-level context" to hex dumps. This falls out of scope of this issue, and requires more drawing that texting, but if somebody wants to help me with sharing the vision - I would not object. It will help to make binary world more accessible, especially for new people, who start coding with JavaScript and Python. -- anatoly t.

10 18