Mailman 3 March 2007 - pypy-dev

translator.js.test invokes pygame (AKA, yet more annoyances from Scott)
by Scott Dial 17 Apr '07

17 Apr '07

Greetings, There is a bug somewhere in the translator.js.test suite that invokes pygame. I haven't tracked it down but I just thought I'd mention it because I'm sure someone else will know better than me. -Scott -- Scott Dial scott(a)scottdial.com scodial(a)cs.indiana.edu

3 10

Towards pypy-jvm
by Antonio Cuni 04 Apr '07

04 Apr '07

Hi Niko, hi all! I've read in the IRC logs that there has been a bit of discussion about what genjvm still lacks in order to translate pypy. Some weeks ago I also tried to translate pypy-jvm; it seems that the two most important missing features are r_dict and weakrefs. The good news is that with some hacks it's possible to get a pypy version that doesn't make use of r_dict or weakrefs: have a look at this IRC log: http://tismerysoft.de/pypy/irc-logs/pypy/%23pypy.log.20070307 The bad news is that even with those changes, jvm crashed because of a failed assertion, then I gave up. I've no clue what it's going wrong, but maybe it's not something terribly wrong to fix. I hope that this infos can help you in some way :-). ciao Anto

3 9

Paul Fernhout wants edit-and-continue in python
by Laura Creighton 31 Mar '07

31 Mar '07

How hard would it be to give it to him? Laura ------- Forwarded Message Return-Path: edu-sig-bounces(a)python.org Delivery-Date: Sat Mar 31 06:18:19 2007 From: "Paul D. Fernhout" <pdfernhout(a)kurtz-fernhout.com> User-Agent: Icedove 1.5.0.9 (X11/20061220) MIME-Version: 1.0 To: "edu-sig(a)python.org" <edu-sig(a)python.org> To clarify, on PataPata, at the moment I am using Jython to prototype a version of a Smalltalk-like system and syntax for the Java (JVM) platform. I am doing this in part to have a system which supports "edit and continue", and also in part just because it is fun. I think Java is a terrible language for a human to use because it is too verbose (among other reasons). Still, after ten years, the JVM is a not-that-terrible platform which has the advantage of hype and installed base and can supply about 1/2 C speeds in a portable way. So Java (or JVM byte codes) isn't that awful as a portable machine language -- and it involves less work for a single programmer to maintain a complex application across multiple platforms than for C. Personally, I like Swing, but then, I come from the world (VisualWorks) where most core Swing designers come from. :-) There is no question in my mind that Jython is a much saner way to write code for the JVM than Java in most situations. And Jython is probably a better choice for most JVM things than most of these JVM languages: http://www.robert-tolksdorf.de/vmlanguages.html (unless you already know one of the other languages or have a problem which has a neatly packaged solution in one of those systems). Still, for anything other than writing arbitrary portable fast code (1/2 C) or code that interfaces easily with Java libraries, Python by itself is probably a more reliable cross platform solution than JVM approaches in many situations. In any case, there is still a lot of Python in my own noodling around, at least for now. :-) And also, while I learned a lot about prototypes and their strengths and weaknesses when writing Python code for PataPata to emulate prototypes in Python, if anything, I developed a better respect for how much you could do quirky things in plain old Python when you wanted to or needed to, thus minimizing some of the need to use prototypes for everything. Still, there are always tempting new projects like "Factor" http://factorcode.org/ which uses FORTH ideals to do amazing things (like with OpenGL) in fairly small footprints and with minimal external dependencies and with fairly few core developer-years of work (though the developers are unusually bright. :-) From the web page: "Factor has an optimizing native compiler, automatic memory management with a generational garbage collector, a powerful collections library, and various advanced language features such as higher-order programming, continuations, and extensible syntax. Factor development is done in an interactive environment with a graphical user interface. There is a single-stepping debugger which can travel backwards in time, and extensive documentation. An easy to use C library interface allows one to call C libraries without writing any glue code in C. Bindings for OpenGL, FreeType, X11, Cocoa, and Windows APIs are provided." (I can hope AK's FONC http://vpri.org/mailman/listinfo/fonc http://www.viewpointsresearch.org/html/writings.htm in a few years will produce something even beyond that, if it succeeds). In any case, I see the JVM, Python, C, FONC, Squeak, and Factor as all possible substrates for running a higher level system on top of. A well designed self-reflective system really should be able to run on all of them with appropriate back ends and interface layers. It just requires a higher level of abstraction than, for example, how PataPata already runs on both top of Python/TK and Jython/Swing. And also a willingness to sacrifice some performance in some situations. :-) Right now, on a 3Ghz AMD64, this new VM in Jython is doing about 25000 message sends a second (and about 30K in Python). This is about 1/1000 the speed of C, but I think when the system reflects on itself and can generate pure Java, I am hoping to have it at Python-ish and Squeak-ish speeds of about 1/20 to 1/100 C overall for loops and calls and sends. But given how fast computers continue to get, I'm not too concerned about performance. Even if it remained at 1/1000 C speed, in ten years, that would mean it would run about the speed of C on today's computers - -- and there are plenty of interesting applications one can run on today's computers. I also have Jython GUI that lets me single step through hand-compiled code. (None of this has been checked in so SourceForge yet though.) From an educational point of view, looking at such a system and changing it, even if the system was never used for any other activity) could help Python-literate people understand more about message passing and VM construction and (abstractly) how a CPU works. I'm certainly learning something from it. :-) I certainly remain completely pro-Python for commercial development. And I remain pro-Python as the language of choice to teach most kids with at the appropriate age. It's a great language. And a great community. Python generally plays nice with other systems (including C and Java) and looks a lot like what most programmers already know, making it an easy "upsell" as a dynamic language compared to, say, Smalltalk or Lisp. http://dictionary.reference.com/browse/upsell And Python has really hit its stride now seeing all the great job postings for people knowledgeable in it to do very interesting work. If I was teaching a kid programming right now, I would recommend Python as the best all-around choice without hesitation, especially as a first language. (Although, for kids who wanted to be computer or electronic engineers, I might still suggest Forth as a best first language, so they learn good factoring habits early on, plus a basic familiarity with low level issues in an interactive environment. But even then, Python should come next, for its practicality. :-) Still, as much as I like Python, I think lack of "edit and continue" in Python is a big issue -- both for advanced users and beginners, especially given how Smalltalk, Ruby, C#, Visual Basic, and other languages have it. Having to rerun Python applications due to typos or to insert print statements is the biggest thorn in my paw when I use Python. As I have said, I think lack of "Edit and continue" probably reduces Python developer productivity to about 1/3 of what it could be (although you can get part of that back by hacking your application to use a reloader window like the one I posted on the Jython user list). Still, even with this glaring handicap relative to other dynamic and static languages like Ruby or VB or C#, Python still usually comes out on top IMHO over many other languages for most common programming tasks for various reasons based on its strengths. Which just shows what an awesome language Python is, still winning the race while putt-putting on two cylinders instead of vrooming by on the six cylinders everyone else has out of the box. Some kinda magic going on there. :-) I started to look into seeing how "edit and continue" could be addressed with Python, but, especially given Guido saying it was "impossible", and looking in the email archives of various lists to see it has been brought up multiple times on comp.lang.python and such and shot down, http://www.google.com/search?hl=en&q=python+%22edit+and+continue%22 it seems it might be easier to build something from scratch which supports it (on top of Java using those libraries) than to try (and fail) to improve Python. I also have always preferred Smalltalk keyword syntax over Python, as it is easily extensible (and easily readable (to me), so there are always forces tugging me towards creating alternatives, which people here will rightfully say stand a snowball's chance of seeing widespread adoption. The bottom line on "Edit and continue" is that making it work properly likely requires deep thinking and understanding about the entire Python runtime system (as well as getting pickup of your changes by various IDE developers), and Python as a system has moved so far along the development curve that coming from a standing start up to an understanding of Python's current internals necessarily beyond that of Guido (who says it is impossible :-) is likely a long hard effort. During which time Python will keep moving. It's not exactly a best first choice of Python internals hacking. Guido is obviously the best person to tackle it, I think; just need to figure out how to motivate him to care about it instead of other cool and important things. Now if I could just get hold of some "first herring" perhaps? :-) http://travel.independent.co.uk/europe/article548699.ece http://en.wikipedia.org/wiki/Herring Still, if a student in the "Summer of Code" (or others) got interested in the project, then I would revisit my personal support for it. I'm sure it's doable -- just a matter of how much of Python's (or Jython's) internals would need to be strewn about the floor before it worked (and an IDE's too, I guess) -- and then how many of those scattered pieces could be fit back together under the hood. The shortest path is modifying an existing IDE (PyDev? IDLE?) and an existing Python or Jython VM so it breaks on any exception before it is thrown (avoiding the likely most "impossible" part of restarting a thrown Python exception), and then providing ways in the IDE where you can specify which modules and exceptions are debugged and which are thrown, and then having an ability to selectively modify a function and reenter it with the same preserved entry state (sort of like a continuation). And of course, you need to then save the change back to the source file. We already have the reloading code; we would just copy over the one modified function. I have poked around some in Jython internals, so for me, adding "edit and continue" to Jython would be the easier possibility than for CPython (but of a more limited audience). If I recall correctly, Jython uses Java exceptions, which are not restartable, so that approach would likely always be be limited to breaking on where they are raised (unless the JVM itself changes). Still, if/when I get the Smalltalk-syntax support on this new base (not much time for it these days for various reasons), I don't see any reason a Python-syntax could not also be added to it (especially using one of the Python-in-Python compilers as the base). So, I fully expect the outcome of what I am doing could support a Python-ish syntax to get edit and continue. But that really is not what we all want, as opposed to the real thing -- Python 2.6 with "edit and continue support". :-) Perhaps someone with experience writing a PEP might want to work with me on at least defining the problem in a way it could be formally submitted? Of course now that I look even harder at people's comments on "edit and continue" I just came across this: http://haacked.com/archive/2006/02/08/OnReligiousWarsinSoftware.aspx listing "Edit and continue" as #5 on "Well Known Religious" wars. :-) Which completely surprises me how people could be against it (and yes, I have read some counter arguments, http://www.codinghorror.com/blog/archives/000507.html but never seen one from someone who had ever actually used the feature extensively, at least when well implemented in a system like Smalltalk, including ones that allow "coding in the debugger" as a development style: http://www.google.com/search?hl=en&q=%22coding+in+the+debugger%22 ) Still, for fixing typos alone, this "edit and continue" feature could save many people (especially beginners) a lot of time with Python IMHO. If Python had it, I'd probably stop thinking about Smalltalk so much. :-) - --Paul Fernhout Michael Tobis wrote: > This is all great stuff! Thanks to all who responded here or in email! > > However so far this all goes to only half of the questions I am trying > to address. > > I'd also like to consider the bad news. At least three important > projects that I know of have abandoned Python in favor of Java or > Squeak: > > 1) Alice > 2) Patapata > 3) Mark Guzdial's CS for artists course/textbook > > It is also plain to see that self-directed web-centric beginners are > more likely to gravitate to Javascript, Actionscript/Flash or PHP. > > What are the roots of these choices? Should we be concerned about > them? If so, is there anything we can and should realistically do > about them? _______________________________________________ Edu-sig mailing list Edu-sig(a)python.org http://mail.python.org/mailman/listinfo/edu-sig ------- End of Forwarded Message

2 1

congratulations on the 1.0 py-py release!
by Luke Kenneth Casson Leighton 30 Mar '07

30 Mar '07

dear pypy developers, congratulations on reaching the 1.0 release of py-py: the project you are working on is very significant and is also a major achievement that you should be proud of. [today, py-py: tomorrow, the rest of the world ... :) ] i was stunned and delighted to see the just-in-time compiler capabilities and also the mention of "virtual machines". i therefore wanted to make you aware of a project that you should consider absorbing and adapting: a Python "Microkernel" Operating System, called "cleese": http://www.jtauber.com/cleese the developers of cleese managed a proof-of-concept back in 2003 and would love to continue work on it. the concept of an entirely python-esque environment, including a development tool (e.g. http://leo.sf.net) right from the bare metal up just... absolutely tickles me pink :) and - i note on the documentation: the idea of using py-py for embedded systems. how much more "embedded" can you get if the entire OS is cut and out-the-window! good luck, l.

2 1

socket and select on windows
by Alexander Shafranov 30 Mar '07

30 Mar '07

Hi! It seems, that in 1.0 release, socket and and select modules aren't implemented for windows. Does someone work on implementation? I could give a try.. Alexander.

2 2

special methods
by Simon Burton 30 Mar '07

30 Mar '07

The attached script is an attempt to get __str__ to work in rpython. After annotation, it looks for str(someinstance), rewriting the block to call __str__. Then the modified blocks are fed back into the annotator. It seems to work. (And now i know a lot more about the internals of the translation process...) I had a think about over-riding consider_op_str. It would seem natural for this code to go there, but the annotator is not really set up to handle rewriting. Why do i not need to manually call the flow operation on the __str__ ? (see the flow_str function) It seems the annotator somehow must re-trigger the flowing operation when it finds that __str__ is being called. thanks, Simon.

4 17

pypy/dist => pypy/trunk, re
by Armin Rigo 29 Mar '07

29 Mar '07

Hi all, So, time to move the development of PyPy to http://codespeak.net/svn/pypy/trunk and keep http://codespeak.net/svn/pypy/dist for the latest known-stable copy of the trunk? Armin

3 3

Finding Carl Friedrich Bolz thesis
by Maurice Ling 28 Mar '07

28 Mar '07

Hi, Does anyone knows where to get Carl Friedrich Bolz's masters thesis? Thanks Cheers maurice

3 3

PyPy 1.0: JIT compilers for free (and more)
by Samuele Pedroni 28 Mar '07

28 Mar '07

========================================== PyPy 1.0: JIT compilers for free and more ========================================== Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in "RPython": - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online "play1" demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? ================================ Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target "C", we can can "mix in" different garbage collectors and threading models, including micro-threads aka "Stackless". The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights ============================== Here is a summary list of key features included in PyPy 1.0: - The Just-In-Time compiler generator, now capable of generating the first JIT compiler versions of our Python interpreter: http://codespeak.net/pypy/dist/pypy/doc/jit.html - More Python interpreter optimizations (a CALL_METHOD bytecode, a method cache, rope-based strings), now running benchmarks at around half of CPython's speed (without the JIT): http://codespeak.net/pypy/dist/pypy/doc/interpreter-optimizations.html - The Python interpreter can be translated to .NET and enables interactions with the CLR libraries: http://codespeak.net/pypy/dist/pypy/doc/cli-backend.html http://codespeak.net/pypy/dist/pypy/doc/clr-module.html - Aspect Oriented Programming facilities (based on mutating the Abstract Syntax Tree): http://codespeak.net/pypy/dist/pypy/doc/aspect_oriented_programming.html http://codespeak.net/pypy/extradoc/eu-report/D10.1_Aspect_Oriented_Programm… - The JavaScript backend has evolved to a point where it can be used to write AJAX web applications with it. This is still an experimental technique, though. For demo applications which also showcase various generated Python and PROLOG interpreters, see: http://play1.codespeak.net/ - Proxying object spaces and features of our Python interpreter: - Tainting: a 270-line proxy object space tracking and boxing sensitive information within an application. - Transparent proxies: allow the customization of both application and builtin objects from application level code. Now featuring an initial support module (tputil.py) for working with transparent proxies. For a detailed description and discussion of high level backends and Python interpreter features, please see our extensive "D12" report: http://codespeak.net/pypy/extradoc/eu-report/D12.1_H-L-Backends_and_Feature… Funding partners and organisations ===================================================== PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a 28 month European Union IST research grant for the period from December 2004 to March 2007. The full partners of that consortium are: Heinrich-Heine University (Germany), Open End (Sweden) merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany)

2 1

making the release branch for 1.0: release/1.0.x
by Samuele Pedroni 26 Mar '07

26 Mar '07

If possible avoid code changes on dist to avoid confusion, unless they are bug fixes that need to be ported to the branch. We should release at some point tomorrow. Thanks, Samuele Pedroni

1 0