Mailman 3 April 2006 - pypy-dev

From edu-sig, Guido on connecting squeak and python
by Laura Creighton April 21, 2006

April 21, 2006

------- Forwarded Message Return-Path: edu-sig-bounces(a)python.org Delivery-Date: Fri Apr 21 10:34:25 2006 Return-Path: <edu-sig-bounces(a)python.org> Message-ID: <ca471dc20604210129o1fd21363nfa71bb9a90940ee5(a)mail.gmail.com> Date: Fri, 21 Apr 2006 09:29:14 +0100 From: "Guido van Rossum" <guido(a)python.org> To: "Paul D. Fernhout" <pdfernhout(a)kurtz-fernhout.com> There's a lot to read in your post and the links you post -- more than I have time ofr right now. Let me try to prune some of the ideas. I'm not interested in switching to Jython for this purpose; nor am I interested in directly linking to code that's part of Squeak -- unless, perhaps, there's some low-level code that is independent of the rest of the Squeak environment while providing some functionality we need. I'm also not interested in making Python an entirely self-contained system such as Squeak is -- much of Python's strengths come from its capabilities as a glue language, seamlessly integrating with other software on many different platforms. But, after encouragement from Alan Kay, I *am* interested in producing a Squeak-like environment *on top* of Python. Alan suggested using a slightly different starting point than Squeak; modern graphics cards have a wealth of functionality that can be accessed directly. I'm no graphics expert, but I believe OpenGL and perhaps SVG could be the right basis to get started. The approach that seems to make the most sense to me (but I'm open for alternatives) is to start out by producing a solid low-level graphics package like this that can work across platforms (Linux, Windows and OSX preferably); once that is settled, we could build an application resembling Squeak's UI. There's probably more to it; but typing this email at a busy conference my thoughts are a bit distracted. - --Guido On 4/21/06, Paul D. Fernhout <pdfernhout(a)kurtz-fernhout.com> wrote: > I've long been interested in making Python development more Squeak > Smalltalk like. See for example a recent post of mine to the Jython user > mailing list with some code (would be also useful for CPython with a few > changes): > [jython-users] ReloaderWindow 0.2 (improvements to selective reloading) > http://sourceforge.net/mailarchive/message.php?msg_id=14482359 > > On the topic of an integrated 2D/3D crossplatform solution for Python > (like Squeak has), I'd like to point to my related comments on this list > from six (!) years back: > > [Edu-sig] Common Graphical Framework for Python Tutorials? > Fri, 04 Feb 2000 11:16:39 -0500 > http://mail.python.org/pipermail/edu-sig/2000-February/000032.html > > [Edu-sig] a modest proposal II > Fri, 04 Feb 2000 18:03:01 -0500 > http://mail.python.org/pipermail/edu-sig/2000-February/000063.html > > [Edu-sig] IDLE/TK limitations for learning environments > Fri, 04 Feb 2000 18:32:54 -0500 > http://mail.python.org/pipermail/edu-sig/2000-February/000065.html > > [Edu-sig] Not well supported on the Mac? > Sun, 28 May 2000 15:24:21 -0400 > http://mail.python.org/pipermail/edu-sig/2000-May/000495.html > > Glad to see some interest in such ideas is perking up here. :-) > > It's all quite doable with enough effort. Though I'd watch out for Squeak > licensing and some Squeak unfinished complexity management issues. I think > it might be better to just use the Squeak base cross-platform ideas or > base code (or perhaps base a new work on wxWidgets) and build a larger > common framework using Python technology and the Python license (and yet > also of interest to Squeakers, like by adding in support for a Smalltalk > parser). > > Alternatively, one could build on top of Jython -- see my post on this > list from last year on this topic: > > [Edu-sig] On Jython for education > Wed Oct 19 15:26:02 CEST 2005 > http://mail.python.org/pipermail/edu-sig/2005-October/005410.html > > I think the Jython-based approach might be easiest, though one then has to > wrestle with other Java community and licensing issues. [I personally > think the Squeak approach would be more stable and maintainable though, > just 2000 lines of core C to port per platform, with widgets built on > that, and a dynamic loading facility for other native code.] A > cross-platform system supporting both Python and Smalltalk (and perhaps > Java) on a JVM with a complete Smalltalk-like development environment > (including cross-image debugging and development) and with 3D plus some > sort of PythonCard/HyperCard framework out of the box, which had the > option of running as a browser plugin, would be really neat. Probably at > least few person months (for me :-) to get that going to the point where > it reached a critical mass and was something people wanted to use or build > on top of though. I've worked on bits and pieces of all these ideas in a > variety of contexts, but never had a chance to put them all together. > > --Paul Fernhout > > Andre Roberge wrote: > > And, if I may quote from Kirby's follow-up post > > http://controlroom.blogspot.com/2006/04/shuttleworth-summit-day-two.html > > --- > > [...] > > Loosely coupled tools, with a bottom-up, open source curriculum > > writing process, will leave the question of tools somewhat open-ended. > > The lesson plans will specify the software needed, with multiple paths > > possible. > > --- > > +1. I couldn't agree more :-) > > ============ > > [...] > > Momentum seems to be building for a stronger graphics engine, either > > 2D or 3D, with Python bindings, that'll run interactively from within > > a browser. The Squeak folks may be willing to contribute to this > > effort. Guido feels we'll need to recruit new talent for this, as the > > Python community is currently pretty maxed out on projects. Should > > such an engine be developed, turtle stuff would be incorporated > > therein. > > ======= > > > > I would love to see this happening and would definitely be willing to > > contribute to such an effort. Of course, I would use this to port > > rur-ple to the web (as a first step). Anybody else is as excited > > about this possibility as I am? > > > > André > > _______________________________________________ > Edu-sig mailing list > Edu-sig(a)python.org > http://mail.python.org/mailman/listinfo/edu-sig > - -- - --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Edu-sig mailing list Edu-sig(a)python.org http://mail.python.org/mailman/listinfo/edu-sig ------- End of Forwarded Message

1 0

pypy-sync thursday 5:30 GMT+2
by hpk＠trillke.net April 20, 2006

April 20, 2006

Hi there, the next pypy-sync meeting of active developers is THURSDAY, 20th April, 5:30 PM GMT+2 (30-45 minutes) at #pypy-sync on freenode with the following topics (plus the ones that get mailed ahead of the meeting ): - activity reports + blockers - summer of X, how do we go about it? there are three possibilities: - aim for Summer of PyPy through the EU (becomes unlikely) - aim for becoming an "Summer of Code" organisation - provide mentors and participate at Googles Summer-of-Code through the PSF and try to get a bit of external funding for having participants attend sprints? decision time :) - what needs to be done until Iceland (21st May) for 0.9? Samuele and Holger went through the issue tracker and identified 0.9 release issues in Leysin. But we are missing assignments ... could everyone have a look and take responsibility for resolving particular issues? https://codespeak.net/issue/pypy-dev/ (and hit "Show 0.9"). Of course it is great if you assign yourself already before the sync meeting so we can see what is missing. - tackling and assigning the major 0.9 tasks to be worked on until Iceland: - integrating/refining the new stackless CFG transformations instead of the genc-support - adding missing support (if any) for app level stackless module (e.g. yield_current_frame_to_caller ?) - implementing tasklet pickling - advancing rctypes to support interfacing with CPython runtime through a CPyObjSpace - tackling the actual extension module compiler tool (including testing improvements and writing a tutorial) - overhauling and updating translation documentation (also in preparation for EU reports pending in June) best, holger

3 2

Python Software Foundation seeks mentors and students for Google Summer of Code
by Neal Norwitz April 20, 2006

April 20, 2006

This spring and summer, Google will again provide stipends for students (18+, undergraduate thru PhD programs) to write new open-source code. The Python Software Foundation (PSF) http://www.python.org/psf/ will again act as a sponsoring organization in Google's Summer of Code, matching mentors and projects benefiting Python and Python users. Projects can include work on the core Python language, programmer utilities, libraries, packages, frameworks related to Python, or other Python implementations like Jython, PyPy, or IronPython. Please add your project ideas to the existing set at http://wiki.python.org/moin/SummerOfCode Mentoring instructions are also on this page. The deadline is soon, so please sign up as a mentor early. If you are a student considering a project, you should start deciding now. Feel free to ask questions on python-dev(a)python.org The main page for the Summer of Code is http://code.google.com/summerofcode.html At the bottom are links to StudentFAQ, MentorFAQ, and TermsOfService. The first two have the timeline. Note that student applications are due between May 1, 17:00 PST and May 8, 17:00 PST. People interested in mentoring a student though PSF are encouraged to contact me, Neal Norwitz at nnorwitz(a)gmail.com. People unknown to Guido or myself should find a couple of people known within the Python community who are willing to act as references. Feel free to contact me if you have any questions. I look forward to meeting many new mentors and students. Let's improve Python! Cheers, n

1 0

Statically type-def-ing Python
by Brian C. Lum April 19, 2006

April 19, 2006

Dear Pypy group, Let me start by thanking you all for being so helpful in the past while I was learning to work with Pypy. I am trying to do research on how to type-def variables in Python at compile time. I have been reading different papers on bounds checking and static detection with scripting languages (for security), but I have not been able to find a feasible solution with Python. I was wondering if anyone knew some good references. I honestly cannot think of any way to type-def Python, except using heuristics to make guesses at what a variable might possibly be. Thank you in advance for your help, Brian

2 1

funny slowdown
by Christian Tismer April 19, 2006

April 19, 2006

Hi friends, at some time over easter, my compilation time on Windows for extension modules increased remarkably, from a few seconds to a minute or more. Does anybody have an idea what might have caused this? thzanks - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

Summer of Code 2006
by Niklaus Haldimann April 18, 2006

April 18, 2006

Hi there Google is doing Summer of Code again this year: http://code.google.com/soc/ It would be possible to enter PyPy directly as a mentoring organization this time, instead of going through the PSF. Last year, student slots were given to mentoring organizations proportional to the number of applications. If there are enough applications for PyPy proper that might bring more students on board than taking slots from the PSF pool as last year (but maybe PyPy's influence in the PSF is big enough, and this doesn't really matter). What is the status of the Summer of PyPy idea, btw? Nik

7 6

Avoiding code duplication
by Antonio Cuni April 17, 2006

April 17, 2006

Hi all, in these days I'm completing the implementation of ootypesystem/rlist.py, but I've found that often I have to write code very similar to those in lltypesystem/rlist.py. Let's explain by an example; consider the ll_listindex function in both files: # lltypesystem def ll_listindex(lst, obj, eqfn): items = lst.ll_items() lng = lst.ll_length() j = 0 while j < lng: if eqfn is None: if items[j] == obj: return j else: if eqfn(items[j], obj): return j j += 1 raise ValueError # can't say 'list.index(x): x not in list' # ootypesystem def ll_listindex(lst, obj, eqfn): lng = lst.length() j = 0 while j < lng: if eqfn is None: if lst.getitem_nonneg(j) == obj: return j else: if eqfn(lst.getitem_nonneg(j), obj): return j j += 1 raise ValueError # can't say 'list.index(x): x not in list' As you can see they are quite similar, but I can't find an elegant way to merge them. There are two problems: 1) the names of some methods don't match (e.g., ll_length vs. length 2) the setitem/getitem interface is very different The first problem is easy to solve; it should be sufficient to rename the methods in ootype.List._GENERIC_METHODS. The same isn't true for the second problem; one possibility could be to pass some dummy placeholder as argument in the same style of dum_checkidx/dum_nocheck, but I guess the code will became a nightmare. Another solution could be to add an extra level of indirection but I guess this could bring to some efficiency penalty. Any idea? ciao Anto

3 5

type inference info derived from test suites for the tested?
by Bengt Richter April 17, 2006

April 17, 2006

Please excuse if this is OT, but I was wondering about the subject topic, i.e., using a test code suite (run with special-command-line-option?) as a kind of substitute for pragmas within a language e.g. defining a function, and winding up with some intermediate representation of the constrained function that can be used in the compilation and linking of an application that does not include the function test suite per se, yet makes use of the implicit info from the test code (per command line option that caused the compiler's info transfer to the test-info-constrained-function's intermediate representation). Maybe this could also be viewed as a kind of global optimization using combined test and application code, but factoring out and pre-caching some results of separated function+test_function inferences, and editing out test code from the final linking. I guess this is trying to use test code as a general source of auxiliary information using inference as a way to be independent of particular languages' ability to express explicit type declarations or contracts etc. TIA for any pointers/urls where I might read about this kind of thing, and whether it's a dead end or a trail being explored, either in pypy or elsewhere. Regards, Bengt Richter

2 1

Mixed modules for both PyPy and CPython
by Armin Rigo April 16, 2006

April 16, 2006

Hi all, This is a follow-up on last night's IRC discussion. It's come to a point where e-mail is better. The topic is "WP3", i.e. how to write in Python built-in modules that work for both CPython and PyPy ("built-in" == "extension" == "compileable" for the purpose of this mail). There are two levels of issues involved in this: * In which style the modules should be written in the first place? * Which style is easier to support in both CPython and PyPy given what we have now? In PyPy, we are using "mixed modules" with an interp-level and an app-level part, both optional, with explicit ways to interact between them. The interp-level part is what gets compiled to C. It contains code like 'space.add(w_1, w_2)' to perform an app-level addition. Christian is working on an alternative, which is a single-source approach where the annotator's failures to derive types are taken as a hint that the corresponding variables should contain wrapped objects. This is a convenient but hard-to-control extension of a historical hack, which works mostly because we needed it while we were still developing the annotator and the rtyper. I am clearly biased towards "solution 1", which is to reuse the mixed modules style that we have evolved over several years. Here is some implementation effort comparison between the two styles (1=mixed module, 2=single source). It is easy to develop and test in 2, as it runs on CPython with no further efforts. For 1, the module developer needs the whole of PyPy to test his complete module. We could ease this by writing a dummy interface-compatible gateway and object space, performing sanity-checks and giving useful error messages. Then the developer only needs to check his code against this. Annotation problems in the module are easier to spot early in the model 1; indeed, the fact that with 2 we cannot gracefully crash on SomeObjects, and moreover the need for many fragile-looking small extensions to the flow object space and the annotator, is what makes me most uneasy about 2. The mixed modules are designed for PyPy, so they work there already now. For the single-source approach to work on PyPy, however, there would be many interesting and delicate problems -- both for py.py and for the translated pypy-c. I'd rather avoid to think about it too much right now :-) For translation, for 2 we already have the basic machinery implemented as a leftover from PyPy "early ages". But I want to convince you that the basic support for 1 is extremely easy, or will soon be. We need a new object space to compile with the mixed module; but this "CPyObjSpace" is completely trivial. It could be based on rctypes, in which case it would look like this: class CPyObjSpace: def newint(self, intval): return ctypes.pydll.PyInt_FromLong(intval) def add(self, w_1, w_2): return ctypes.pydll.PyNumber_Add(w_1, w_2) ... Note that this even works on top of CPython! (Not out-of-the-box on Linux, however, where for obscure reasons CPython is by default not compiled as a separate .so file...). The gateway code can be written in the same style, by using ctypes.pydll.PyObject_Call() to call the app-level parts, and ctypes callbacks for the reverse direction. The calls like 'ctypes.pydll.PyInt_FromLong()' return an instance of 'ctypes.py_object', which naturally plays the role of a wrapped object for the CPyObjSpace. The more difficult bits of translation are e.g. support for creating at interp-level types that are exposed to app-level, in particular about the special methods __add__, __getitem__ etc. There is an example of that in pypy.module.Numeric.interp_numeric, where __getitem__ is added to the TypeDef of "array". This creates some difficulty that is common to approaches 1 and 2, and which I think Christian is also working on. In 1 we would probably make the TypeDef declaration turn, for CPython, into static PyTypeObject structures. The special methods can be recognized and put into the proper slots, with a bit of wrapping. The whole PyTypeObject structure with its pointers to functions could be expressed with ctypes too: # use pypy.rpython.rctypes.ctypes_platform to get at the layout PyTypeObject = ctypes_platform.getstruct("PyTypeObject *", ...) def TypeDef(name, **rawdict): mytype = PyTypeObject() mytype.tp_name = name if '__getitem__' in rawdict: # build a ctypes callback mp_subscript = callback_wrapper(rawdict['__dict__']) mytype.tp_as_mapping.mp_subscript = mp_subscript return mytype This gives us prebuilt PyTypeObject structures in ctypes, which get compiled into the C code by rctypes. At the same time, running on top of CPython is still possible; ctypes will let you issue the following kind of calls dynamically when running on CPython: myinstance = ctypes.pydll.PyObject_Call(mytype) The same line translated by rctypes becomes in the C file: PyObject* myinstance = PyObject_Call(mytype); Of course this latter part is dependent on rctypes being completed first. I understand and respect Christian's needs for something that works right now; nevertheless, at this point I think the mixed modules approach is the best medium-term solution. This is all open to discussion, of course. ...but do keep in mind that the people who completed the annotator as it is now are not likely to appreciate the addition of Yet More Ad-Hoc Heuristics all over the place :-/ A bientot, Armin

6 14

ext module testing modes
by hpk＠trillke.net April 15, 2006

April 15, 2006

Hi Armin, On Sat, Apr 15, 2006 at 13:02 +0200, arigo(a)codespeak.net wrote: > Author: arigo > Date: Sat Apr 15 13:02:14 2006 > New Revision: 25852 > > Added: > pypy/dist/pypy/rpython/rctypes/socketmodule/test_addr.py (contents, props changed) > Modified: > pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py > pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py > Log: > Very incomplete implementation of getaddrinfo(), with a test > (only works on on-line machines so far). The idea is that rctypes > should now support all ctypes constructions that were necessary. > I will start a regular mixed-module _socket based on this, but > first we need to figure out how to best test mixed-modules based > on ctypes -- ideally, they should be testable and compilable > without the rest of the PyPy interpreter... regarding py.test support: I think eventually we may have the following testing distinctions regarding ext modules: - test mixed module with std objspace (running on top of cpython) (current default) - test mixed module with cpy-objspace connected to CPython runtime via rctypes - test mixed module on top of pypy-c I guess the second testing mode could be specified by "--objspace=cpy" and for the third we may simply allow to specify "--appdirect" which would trigger application level tests to run directly on the executable instead through pypy interpreter indirection. (with "--exec=/path/to/executable" you can already point to pypy-c but PyPy does not support enough for py.test to run this way). makes sense? holger

2 1