Mailman 3 February 2017 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

Pandas on PyPy
by Singh, Yashwardhan 28 Feb '17

28 Feb '17

Hi Everyone, I was hoping to use Pandas with PyPy to speed up applications, but ran into compatibility issues with PyPy. Has anyone tried to make PyPy works with Pandas, by any experiment/hacking method? If yes, would you share your findings with me? If not, could someone points me to where the obstacles are and any potential approach, or plan by this community? Any help is greatly appreciated. Yash

3 2

GIL
by Sebastian K 26 Feb '17

26 Feb '17

Hello everbody, can someone explain me why pypy needs the gil? In CPython the main reason is the reference counting but pypy doesnt use reference counting. Thanks in advance. Regards Sebastian

4 5

Leysin Winter Sprint 2017
by Robert Zaremba 24 Feb '17

24 Feb '17

Hi, I would like to join the Leysin sprint. Any additional steps I have to do? When do I need to confirm dates? I will be available 25/02 - 1/03, thought if there will be nothing on 25 then I prefer to come on 26/02 morning. About me: I'm a software specialist, programming in Python for 11 years, thought during my last 3 years I was doing mostly Go, and Python for some visualization. I would like to work on Python 3.5 support. Robert Zaremba

4 7

'@rpython/$(TARGET)' problem
by John Zhang 20 Feb '17

20 Feb '17

Hi all (Armin?), I have been troubled by the line rpython/translator/platform/drawin.py:35, where it adds ‘@rpath/$(TARGET)’ to the linker arguments for shared libraries. This assumes that the usage case will be from a Makefile, where $(TARGET) is defined. However, in ExternalCompilationInfo.compile_shared_library function, which I use to compile a few support C files into a shared library dependency, there is no Makefile involved, yet the flag is still involved in the final linking. This seemed to pose a problem for other shared library that depends on the compiled library, and dlopen was complaining that it cannot find library ‘@rpath/$(TARGET)’. I’m wondering if there is a good solution for this. A possible solution would be to add a default parameter for _args_for_shared, which could be altered if not compiled using Makefile. What do you think? The attached is a suggested solution. Regards, John Zhang ------------------------------------------------------ John Zhang Research Assistant Programming Languages, Design & Implementation Division Computer Systems Group ANU College of Engineering & Computer Science 108 North Rd The Australian National University Acton ACT 2601 john.zhang(a)anu.edu.au<mailto:john.zhang@anu.edu.au>

2 1

VMProf 0.4.0
by Richard Plangger 15 Feb '17

15 Feb '17

Hi there, I'm currently finishing up vmprof native profilng for CPython & PyPy. Here are some highlights (soon to be released in 0.4.0): * Native profiling (C stack) is included in the profile (using libunwind) for Linux & Mac * Windows 64bit support (no native profiling) * The platform that reads the profile can be different from the platform that generated it * vmprof.com updates to the flamegraph and general style changes * Documentation updates I'm happy to receive any feedback. PyPy support is nearly finished, but there is one issue: On my branch (called vmprof-native), libunwind is required to be available on the system. Which means that we need to install those on the buildbots (which is fine) and libunwind suddenly becomes a dependency when building PyPy on Linux and Mac. Is the latter ok? Or should libunwind be an optional dependency? Cheers, Richard

3 4

vtable: non-GC allocation unit used only within RPython programs
by Kunshan Wang 09 Feb '17

09 Feb '17

Hi all, I am working on porting RPython to the Mu micro virtual machine (http://microvm.org/). I have a question regarding the translation of the vtables of RPython objects. Background: Mu is a micro virtual machine. Its type system is similar to the level of the LL type system. Mu has a garbage collector and a stack unwinder built in, so LL-typed RPython programs can be straightforwardly translated to the Mu intermediate representation (IR) without injecting exception handling and GC. Mu has a garbage-collected heap and a permanent static space, both are traced by GC (i.e. the GC can identify all reference fields in them). Unlike RPython where only GcStruct, GcArray and their derivatives are GC-ed, any type can be allocated in the GC heap (Yes, you can allocate a single int64 in the heap if it makes sense to your language.). The GC maintains its own metadata (invisible to the Mu client) to perform GC for any types. But Mu is also minimal: Mu only allocates exactly what fields the client tells Mu to allocate, but not more. Particularly, Mu does not provide any RTTI to its client (think about the C language with reference types and GC). Therefore, OOP language implementers have to implement their own vtables and add vtable reference fields to object headers. Unlike RPython where the traced-ness of Ptr<T> is determined by whether T is GC-ed or not, Mu has distinct untraced pointers (uptr<T>) which are just raw addresses, object references (ref<T>) which refer to heap objects, and internal references (iref<T>) which is a more powerful (but harder to implement) reference type that can refer to a field of either a heap object or a static variable. Mu can build 'boot images'. A boot image is an executable image which contains a preserved heap in addition to executable code. It is similar to the executable program the RPython C backend generates, but preserved heap objects can still be garbage-collected as usual --- they are not C-level static variables. Problem: When translating the LL type Ptr<T> to the Mu counterpart, our current approach is: 1. If T is a GcStruct or a GcArray, translate it to ref<T>, 2. otherwise, translate it to uptr<T>. Ideally, object references are used inside Mu (all RPython programs are translated to Mu IR), and uptr<T> are only used to interact with external C programs --- uptr<T> are just addresses. Most RPython programs seem to follow this pattern: GcStructs are used within the RPython program, and non-GC Structs can be passed to external C programs by Ptr. But vtables appear to be a special case (see rpython/rtyper/rclass.py:160): 1. It is a Struct, but not GcStruct. So it is not allocated in the GC heap. 2. But it is only used internally for RPython. It is never exposed to native C programs. It is a valid approach to implement vtables as non-GC objects, because the number of RPython types is determined at compile time, and there are only as many vtables as GC-ed types. But I think our current translation strategy --- translating RPython Ptr<vtable> to Mu uptr<vtable> --- is problematic: this link only exists within Mu programs from GC objects to their respective vtables. It shouldn't be translated to uptr<T> which should only be used for the native interface. Furthermore, Mu restricts the T of uptr<T> to be untraced types. vtables contain many function references, which are a kind of traced reference in Mu. So referring to vtable by uptr<vtable> will not work unless we relax the restriction to allow accessing traced reference fields using uptr, which is undesirable because uptr is specifically designed to access native C data, not Mu objects. Currently, we allocate all vtables as static variables (still traced by the GC). So, an alternative to uptr<T> is iref<T> which is a traced reference type, and can still be used to refer to static variables. Now, we see there are three kinds of RPython-level Ptr<T> values: 1. Those pointing to GC objects. 2. Those pointing to non-GC objects which can be accessed by native programs (e.g. the byte buffers to be read by the `write` system call when printing) 3. Those pointing to non-GC internal objects within RPython (e.g. vtable) The problem is: there seems to be not enough static type information to distinguish between case 2 and case 3. If we could distinguish between 2 and 3, we could translate case 2 to uptr<T> and case 3 to iref<T>. But RPython seems to be using Ptr for these two purposes interchangeably. Therefore, the Mu backend will have to tell whether an RPython-level Ptr<T> field is ever exposed to native programs or not, which can only be decided at run time. Understandably, using the C backend, both vtables and C-accessible objects are represented as C-level global variables of struct types. Therefore it was not necessary to make such distinction, because both are accessed by C-level pointers after translation. So I would like to know: Is vtable the only non-GC Struct type that is used internally within an RPython program (instead of exposed to native C programs)? If it is, we can make a special case, and translate Ptr<OBJECT_VTABLE> to iref<OBJECT_VTABLE>. But the ideal way to implement vtables is translating them to GC objects. In this way, vtables can be managed just like any other GC objects. Since Mu do not automatically add RTTI to heap objects, Mu can still allocate vtables on the GC heap even though vtable is not an RttiStruct. Regards, Kunshan Wang Australian National University

2 1

Modifying Interpreter-level Object from Application
by Frank Wang 07 Feb '17

07 Feb '17

Hi, At the application level, I want to modify some interpreter-level attributes of an object. Right now, I have the interpreter level functions that allow me to modify the interpreter object. Is the easiest way to have an application access interpreter level attributes to use a Mixed Module with interpreter level definitions? I'm having a bit of trouble getting it to work, so I was wondering if there was a much easier way for an application to modify interpreter objects or call interpreter level functions that modify an object. Thanks, Frank

3 4

pypy-dev February 2017