Mailman 3 January 2014 - pypy-dev

Validate SSL/TLS
by Donald Stufft 27 Jan '14

27 Jan '14

Hey there! So I’d like SSL/TLS in Python to be better and at the prompting of Alex I’m opening this thread to discuss the possibility of incorporating some of these changes into PyPy. Basically it boils down to is PyPy willing to: a) Have SSL hostname validation back ported to it b) Have that turned on by default ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

4 9

Building -pie binary
by Roberto De Ioris 27 Jan '14

27 Jan '14

Hi, while trying to simplify the usage of pypy in uWSGI, i started working with pie mode (gcc -pie option). Position independent executables can be dl-opened (at least) on Linux and OSX as normal shared libraries. As dealing with libpypy-c is basically messy, this could be a solution. I think distributing a -pie enabled binary package would push adoption of uWSGI+pypy solutions (currently the biggest deployment i have found rely on libpypy-c distributed by my company that is months old), and generally will simplify the embedding of pypy in other apps. I still have not tried to translate a pypy source tree with -pie flag, so maybe nothing will work (even if i do not see why). I will make an attempt in the next few hours Regards -- Roberto De Ioris http://unbit.it

2 1

Boost.Python and PyPy -- status?
by Toby St Clere Smithe 24 Jan '14

24 Jan '14

Hi there, I have written a (C)Python extension using Boost::Python, and would like to get it working on PyPy. I came across [1], but that's probably a bit out of date, and the link given there discussing the status then[2] doesn't work any more. Is there an up-to-date status page? Or, does anyone know more? I don't currently have much time to invest in solving many residual issues myself, but I should do come spring / summer. [1] https://mail.python.org/pipermail/pypy-dev/2010-April/005816.html [2] http://paste.pocoo.org/show/203663/ Regards, Toby Smithe

9 15

PyPy 2 unicode class
by Johan Råde 24 Jan '14

24 Jan '14

At the Leysin Sprint Armin outlined a new design of the PyPy 2 unicode class. He gave two versions of the design: A: unicode with a UTF-8 implementation and a UTF-32 interface. B: unicode with a UTF-8 implementation, a UTF-16 interface on Windows and a UTF-32 interface on UNIX-like systems. (Armin claims that this design can be implemented efficiently.) Armin asked me (a Windows developer) which design I prefer. My answer is A. Why? Short answer: Because surrogate pairs are a pain in the ... ------- Long answer -------- ** A well-designed interface should accurately model the problem domain. ** In this case the problem domain contains the following abstract objects: 1. Unicode code points (i.e. integers in [0, 0x110000)) 2. Unicode characters 3. Unicode strings The relations between these abstract objects are: * There is a 1-1 correspondence between Unicode code points and Unicode characters. * Unicode strings are abstract sequences of Unicode characters. With a UTF-32 interface there is a 1-1 correspondence between these abstract objects and the following concrete Python 2 objects: 1'. int objects n with 0 <= n < 0x110000 2'. unicode objects s with len(s) == 1 3'. unicode objects With a UTF-16 interface there is not a 1-1 correspondence between 2 and 2'. Then we are not modelling the problem domain any more, and we have to deal with nonsense such as: >>> s = u'\N{PHAISTOS DISC SIGN FLUTE}' >>> len(s) 2 Next, would such a change break any existing Python 2 code on Windows? Yes it will. For instance the following code for counting characters in a string: f = [0] * (1 << 16) for c in s: f[ord(c)] += 1 On the other hand, some broken code will be fixed by this change. There is a lot of Windows code that does not handle surrogate pairs correctly. For instance, in an earlier version of Notepad, you had to press Delete twice to delete a character with Unicode code point >= 0x10000. Such broken code might be fixed by this change. I believe that fixing broken code will be more common than breaking working code, so the overall effect on existing code will be beneficial. ---------------------- Finally, I very much dislike the term UTF-32 interface. You can, and should, explain that interface without mentioning encodings at all; just describe the problem domain and how the interface models the problem domain. Encodings should be an implementation detail, and should not leak through the interface at all. Python should make you think in terms of abstractions, not in terms of bytes. Python is not C. Let's call it the natural interface. (I hope this makes more sense than my ramblings on IRC last night.)

7 11

Questions on stdlib-2.7.5 failures
by Christian Hudon 24 Jan '14

24 Jan '14

Hello, I'm working again on the stdlib-2.7.5 branch of Pypy. I have a couple questions, but since nobody seems to be around on irc right now, I'm asking them here. :-) First, I kicked the buildbots yesterday, and there are a *lot* of errors due to tests not being able to import the MAXREPEAT constant from the _sre module. But it seems to me that it is defined in pypy/module/_sre/__init__.py... Am I missing something? Are the errors due to a build problem or something else? Second, I'm trying to reproduce some of the more interesting failures (test_multiprocessing, test_str), but when I try to run the tests on my laptop, they fail like this instead: platform darwin -- Python 2.7.5 -- pytest-2.2.4.dev2 -- /usr/bin/python collected 1 items test_str.py [snip] Traceback (application-level): File "/users/chrish/Code/pypy/pypy/tool/pytest/run-script/regrverbose.py", line 4 in <module> from test import test_support File "/users/chrish/Code/pypy/lib-python/2.7/test/test_support.py", line 21 in <module> import struct File "/users/chrish/Code/pypy/lib-python/2.7/struct.py", line 1 in <module> from _struct import * ImportError: No module named _struct FAILED I don't really see anything that could look like a module called "_struct" in the source code... Am I missing something here too? Thanks in advance, Christian

3 3

pypyjit/test_pypy_c
by Armin Rigo 23 Jan '14

23 Jan '14

Hi all, There are failures in pypyjit/test_pypy_c/test_string.py that shows that the JIT gets much worse code in some cases. This is probably related to the refactor-str-types branch merge (at least the dates match). I found and fixed one issue, which was that some code in stringmethods.py has a loop in the slow path, preventing the JIT from inlining the fast path. There are still failures though. I suppose I should mention again that merging big branches without checking all tests first is a Bad Thing to do. A bientôt, Armin.

2 2

PyPy Status Blog Updates?
by Jean-François Caron 23 Jan '14

23 Jan '14

Hi Devs, I just lurk on the mailing list so I know stuff is happening, but I've noticed that the PyPy Status Blog hasn't been updated in nearly two months, so others might not know what's happening with PyPy these days. Is there enough news to make a post? I'd be personally interested in knowing what's been happening with numpypy, PyPyROOT, and PyGame/PySDL, since those are the libs I use. Jean-François

3 2

STM minor collections
by Armin Rigo 22 Jan '14

22 Jan '14

Hi Remi, I thought about another possible way to handle minor collections in stm-c7. If we synchronize the threads to do minor collections, then we no longer need to force minor collections after each transaction. This is possible if we use a single coordinated nursery instead of one per thread. This is similar to Java GCs, which are multithreaded but still have only one nursery, out of which the threads allocate "blocks" at a time and then subdivide them in a thread-local way (here a "block" can be some small number of pages). After a transaction commits, the committed memory in the nursery can be seen by all threads, and we can unshare pages in order to modify objects. This is fine as long as we wait for all threads to reach a safe point when we do a minor collection --- which could then be done either using some general parallel GC technique, or something more subtle where each thread moves objects that are likely to be their own --- for example, objects that they allocated themselves. The main difference with Java GCs is what to do with pages that have been unshared. I think that reasoning about the total amount of syscalls needed, we can see that we may as well re-share the pages after a minor collection (because the alternative is to skip that page until the next major collection, but then it will have to be re-shared during the next major collection anyway). A bientôt, Armin.

1 0

[CFP] Dyla'14, Workshop on Dynamic Languages and Applications, at PLDI, Edinburgh
by Carl Friedrich Bolz 22 Jan '14

22 Jan '14

[CFP] Dyla'14, Workshop on Dynamic Languages and Applications, at PLDI, Edinburgh - What: 8th Workshop on Dynamic Languages and Applications. - Where: Co-located with PLDI'14, Edinburgh, UK - When: June 12th, Sponsored by ACM SIGPLAN Submission deadline: March 15th More info on the website: http://www.lifl.fr/dyla14/ Dyla is a place where developers and researchers can discuss new advances in the design, implementation and application of dynamically-typed languages. The expected audience of this workshop includes practitioners and researchers sharing the same interest in dynamically-typed languages. Lua, Python, Ruby, JavaScript and others are gaining a significant popularity both in industry and academia. Nevertheless, each community has the tendency to only look at what it produces. Broadening the scope of each community is the goal of the workshop. To achieve this goal Dyla's program and organization committees are composed of leading persons from many such languages. Topics -- - live programming - programming language extensions - programming environment extensions - domain-specific languages & tooling - executing environments - static & dynamic analyses - meta-object protocols - optional type-checking - reverse engineering - testing environments Organizing committee: -- - Damien Cassou, University of Lille 1, FR - Carl Friedrich Bolz, King's College London, GB - Johan Andersson, Burtcorp in Gothenburg, SE - Roberto Ierusalimschy, Catholic Univ. in Rio de Janeiro, BR - Tom Van Cutsem, Vrije Universiteit Brussel, BE Program committee: -- - Anne Etien, University Lille 1, France - David Schneider, Heinrich-Heine-Universität Düsseldorf, DE - Didier Verna, EPITA/LRDE, France - Edd Barrett, Department of Informatics, King's College London, GB - Joe Gibbs Politz, Brown University, USA - Peng Wu, IBM T.J. Watson Research Center, USA - Tim Felgentreff, Hasso-Plattner-Institut, DE - T. Stephen Strickland, University of Maryland, US - Yoshiki Oshima, Viewpoints Research Institute, USA - Zachary P. Beane, Portland, USA - the 5 workshop organizers Abstract: -- Java and C# have been a major influence in the adoption of object-oriented language characteristics: academic features like interfaces, garbage collection, and meta-programming became technologies generally accepted by the industry. However, with the adoption of these languages, their limitations became apparent, as testified by industry reactions: invokedynamic has been included in the latest Java virtual machine release; the dynamic language runtime (DLR) is gaining popularity; C# adopted dynamic as a valid static type. Researchers and practitioners struggle with static type systems, overly complex abstract grammars, simplistic concurrency mechanisms, limited reflection capabilities, and the absence of higher-order language constructs such as delegation, closures, and continuations. Dynamic languages such as Ruby, Python, JavaScript and Lua are a step forward in addressing these problems and are getting more and more popular. To make these languages mainstream, practitioners have to look back and pick mechanisms up in existing dynamic languages such as Lisp, Scheme, Smalltalk and Self. The goal of this workshop is to act as a forum where practitioners can discuss new advances in the design, implementation and application of dynamically-typed languages that, sometimes radically, diverge from the statically typed class-based mainstream. Another objective is to discuss new as well as older "forgotten" languages and features in this context. Format and Submission Information -- The workshop will have a demo-oriented style. The idea is to allow participants to demonstrate new and interesting features and discuss what they feel is relevant for the dynamic-language community. To participate in the workshop, you can either - submit (before March 15th 2014) an article (ACM Tighter Alternate style) describing your presentation and/or tool. Your article, which must include from 2 to 15 pages, will be carefully reviewed by the program committee. If accepted, your article will be presented during the workshop and be published to the ACM Digital Library (at your option) and the workshop's web site. Please submit to http://www.easychair.org/conferences/?conf=dyla14. - or give a 10-minute lightning demo of your work. A dedicated session will be allocated for this, provided there is ample time available. In this case, send us the title of your demo. A session on pair programming is also planned. People will then get a chance to share their technologies by interacting with other participants.

1 0

I like your profile
by Little Girlie 22 Jan '14

22 Jan '14

1 0