Mailman 3 November 2007 - pypy-dev

Re: [pypy-dev] missing things for making PyPy "production" ready (for some value of production)
by Laura Creighton 15 Nov '07

15 Nov '07

In a message of Wed, 14 Nov 2007 17:40:54 GMT, Douglas McNeil writes: ></lurk> > >Arbitrary code is less interesting to me than JIT-powered fast numerical >code. Moreover, we numerics types have much lower standards of >"production ready" than the general public, and are willing to turn on >options with names like --make-dangerous-assumptions-about-code-for-speed >-do-not-use-this-flag-really-do-not-use-it-i-warned-you. > I have this uncontrollable urge to make these options for you. :-) thanks for brightening an already pretty bright day. Laura

1 0

Do we want to do this?
by Laura Creighton 15 Nov '07

15 Nov '07

http://www.fosdem.org/2008/call_for_devrooms "FOSDEM will take place on 23 and 24 February 2008, and we provide so-called Developer Rooms ("devrooms") as well as Stands (booths) to Free and Opensource projects. For every devroom we invite a project to organize talks, open discussions and/or hacking sessions. We offer an environment for developer collaboration, but we have only a limited number of rooms at our disposal. " Something for FOSS projects? All Stand or Developer room request must be sent in *before Monday 2007-11-26, 23:59 CET*. ----------------- Looks like it could be fun Laura

1 0

Pypy and app-level ctypes
by Maciek Fijalkowski 14 Nov '07

14 Nov '07

As a discussion regarding pypy being production-ready heats a bit, I've go a practical question: How useful would be to have app-level ctypes for pypy? Another question how less useful it would be if it won't provide python-api part (which would be as easy to provide as C-level bindings for CPython I think)? Also how much helps situation the fact that we have ctypes_platform thingie, which would help you to take away platform dependencies (ie will ask gcc for correct sizes of certain types, structures, #defs etc.) Cheers, fijal :.

2 2

Re: [pypy-dev] pypy on Tim Bray's blog
by Gary Robinson 14 Nov '07

14 Nov '07

> Well, I guess what we really need to get adoption in this area is a > reason for people to switch to PyPy's Python interpreter. Right now > doing this means you get a slower Python, with less available > extensions. Of course you get nice features like lazy objects, > Stackless, transparent proxies, etc. But, it's not clear to me that > these features are actually worth switching for anyone, after all there > is no massive adoption of Stackless Python either (which has some of the > benefits and lacks some of the disadvantages of PyPy). The thing that I've been waiting for is for pypy to live up to its promise of being significantly faster than cPython. If it was, and barring other major problems, I'd be using it in production right now. -- Gary Robinson CTO Emergent Music, LLC grobinson(a)goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net

1 0

Re: [pypy-dev] pypy on Tim Bray's blog
by Laura Creighton 14 Nov '07

14 Nov '07

In a message of Mon, 12 Nov 2007 19:53:33 +0100, Martijn Faassen writes: >Hey, > >Laura Creighton wrote: >> In a message of Sun, 11 Nov 2007 09:14:54 +0100, Martijn Faassen writes >: >>> Hi there, >>> >>> http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy >>> >>> It might be worth it for someone to comment on it, as the one comment >>> there is rather negative about PyPy (it'd really help to have PyPy >>> interpreters in production somewhere, as before you do, the criticism >in >>> the comment cannot be properly countered) >> >> Somebody not in the project can comment on it if they like. I'm >> not in the habit of responding to troll-bait. If that person >> sincerely thinks that we haven't produced much in 2 years then >> he either is completely unaware of what we have done, or he >> cannot understand it. > > > I'm fine with combatting ignorance, but not the with people in the > > 'my-sports-teams-is-better-than-your-sports-team' mode. > >It's a negative message in a widely read blog. Even if it's offensive >and wrong, it's also bad marketing for the project. By now any (minor) >damage it did is done though, I guess, as it'll disappear off people's >radar again. > >I of course don't agree that you didn't accomplish anything, but of >course on the other hand I continue on my mission to poke you guys >gently about seeing PyPy into production (my own work). > >Once PyPy is used in production there's an easy way to blow any such >criticism out of the water. If it never happens, you've accomplished >research goals, which are worthwhile by itself, but I'd prefer to see >something more pragmatic myself in addition. :) Us, too. But Samuele, Armin, Jacob and myself are now on a 2 week US road show speaking with major players in US companies every day of the week. We don't have time for baby-trolls even if we had the inclination. <snip> >I also live close to that world. It's an audience you want to reach, I >think, so it was interesting to read what came out of your meeting from >that side. > >I guess he'll believe you once you have a compliant JITed PyPy >interpreter running on the JVM in production, then. :) Nothing that dramatic. Just a Ruby that runs at the speed of CPython. Though, goodness knows, perhaps John Rose and Charles Nutter will tell him that we are for real, and that will be enough. It's hard to know the difference between a genuine revolution in the industry, and a lot of hot air. But the odds are on anything radical being the second. After all, if it is so hot, why hasn't somebody else already done it? There is no answer to that. The hard things that change the world were often done by people who weren't expected to amount to anything, and just kept on truckin' anyway. Sooner, rather than later, Tim Bray will have his answer, and nobody will blame him for being skeptical about our ability to deliver on what we promise, because we promise a lot, and something that has never been done before. Sometimes the caterpillar wants to become a larger, healthier caterpillar. Sometimes it's time to become a butterfly. >>> It's too bad the potential of a compliant, maintainable Python >>> implementation on the JVM doesn't seem to make it to his blog. I assum >e >>> you also pitched actual work on *Python* on, say, the JVM, to him as a > >>> possible candidate for funding. As opposed to any arbitrary language, >>> which basically lets Tim change the topic to a Ruby discussion. >> >> Tim is very Ruby focused. He's not interested in python, he said, >> because 'the ruby community is more vibrant'. Vibrant is measured >> in body-count. > >body-count? As in how many people are part of it (I imagine Python would >do pretty well compared to Python) or how many people die in horrible >flame wars? :) I think 'how many people attend Ruby developers conferences' and 'how many people use rails' but I am not 100% convinced of that. <snip> >It's glad to hear from your side, thanks! I didn't mean to criticize you >at all, I was just curious to explain the blog entry which presented >PyPy in a somewhat unusual light. > >> We go spend a whole day at Sun on Monday, focused on the JVM. >> John Rose, who is organising this, and who knows Samuele from >> a currently running Java expert committee, has invited some >> people, including some JRuby people to attend. John Rose is >> already _really excited about this_. And we left with him very impressed. Wew had a blast. I think he would liek to attend a Sprint. And he would like it if we made our next pypy focus -- getting the jit to work with the JVM. Wouldn't that be fun. >> There is nothing but good news here. Be happy for us. > >All right, I'm happy for you. :) Thank you, and we will keep your concerns under consideration. But please remember that the PyPy dev team needs to eat, pay rent, and the like. While developing a Ruby front end may not be the most fun thing we could do, it may be plenty more fun than the other things we would end up doing instead just to make the financial ends meet. Laura, happier than ever today. > >Regards, > >Martijn > >_______________________________________________ >pypy-dev(a)codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev

8 14

unicode error when building the JVM version of PyPy
by Martijn Faassen 13 Nov '07

13 Nov '07

Hi there, I just tried to build the JVM version of the trunk with the following command (which may be altogether wrong): python2.4 translate.py --text --batch --backend=jvm targetpypystandalone.py A while into the translation, I got the following error, which I'm dumping here in case it might be useful to someone: [jvm:graphs] Rendered 7600/7648 (approx. 99.37%) [Timer] Timings: [Timer] annotate - 384.4 s [Timer] rtype_ootype - 112.8 s [Timer] backendopt_ootype - 85.2 s [Timer] source_jvm - 111.1 s [Timer] ======================================== [Timer] Total: - 693.5 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 273, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 772, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/tool/taskengine.py", line 112, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 281, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 739, in task_source_jvm [translation:ERROR] self.jvmsource = self.gen.generate_source() [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/genjvm.py", line 260, in generate_source [translation:ERROR] GenOO.generate_source(self) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/genoo.py", line 68, in generate_source [translation:ERROR] self.db.gen_constants(self.ilasm) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/database.py", line 35, in gen_constants [translation:ERROR] self.constant_generator.gen_constants(ilasm) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/constant.py", line 108, in gen_constants [translation:ERROR] super(JVMConstantGenerator, self).gen_constants(ilasm) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 245, in gen_constants [translation:ERROR] self._initialize_data(gen, all_constants) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 268, in _initialize_data [translation:ERROR] if not const.initialize_data(self, gen): [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 542, in initialize_data [translation:ERROR] push_constant(self.db, FIELD_TYPE, value, gen) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 55, in push_constant [translation:ERROR] return constgen.push_primitive_constant(gen, TYPE, value) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 122, in push_primitive_constant [translation:ERROR] gen.push_primitive_constant(TYPE, value) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/generator.py", line 1103, in push_primitive_constant [translation:ERROR] self.load_string(str(value._str)) [translation:ERROR] UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 0: ordinal not in range(128) Might have something to do with the recent unicode work? Regards, Martijn

2 1

__file__ attribute of mixed modules
by Antonio Cuni 13 Nov '07

13 Nov '07

Hi all, I'm having troubles running pypy-cli on windows. One of the problem is due to the __file__ attribute of mixed modules; currently it's something like path/to/mixed/module/*.py. When site.py runs, it tries to compute the abspath of every module loaded; after a bit of indirections, abspath call the posix__getfullpath helper written in C#. The most obvious (and probably most correct) way to implement posix__getfullpath is to delegate to System.IO.Path.GetFullPath; here is where problems come, since the CLR implementation of GetFullPath complains if we pass it a name with an asterisk (Mono doesn't). This prevents pypy-cli to start. One possible solution would be to place a check inside the C# helper and not call GetFullPath in case there is an asterisk in the name (or maybe remove the asterisk, call GetFullPath and re-insert the asterisk). I think this solution is ugly and hackish. Another solution is to change the way we assign __file__ to mixed modules; e.g., we could use the name of the directory itself, or __init__.py instead of *.py. Or maybe some weird name like 'i_dont_really_exist.py', etc. What do you think? ciao Anto

2 1

Re: [pypy-dev] pypy on Tim Bray's blog
by Laura Creighton 12 Nov '07

12 Nov '07

In a message of Sun, 11 Nov 2007 09:14:54 +0100, Martijn Faassen writes: >Hi there, > >http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy > >It might be worth it for someone to comment on it, as the one comment >there is rather negative about PyPy (it'd really help to have PyPy >interpreters in production somewhere, as before you do, the criticism in >the comment cannot be properly countered) Somebody not in the project can comment on it if they like. I'm not in the habit of responding to troll-bait. If that person sincerely thinks that we haven't produced much in 2 years then he either is completely unaware of what we have done, or he cannot understand it. I'm fine with combatting ignorance, but not the with people in the 'my-sports-teams-is-better-than-your-sports-team' mode. > >I'm not sure how to interpret Tim Bray's post. Frankly, I was a bit >disappointed by what made it in there. It's interesting to analyze as it >gives some clue about how your pitch was received and understood. Tim can understand all the implications of new technology and we really impressed him. His problem was in believing that we had done what we had said, and now that we can do what we promised. And he is not techie -- in the sense of being a language designer who has read all the neat scientific papers of language design -- he lives close to the programming-not-deeply-informed-by-computer science world. > >It's too bad the potential of a compliant, maintainable Python >implementation on the JVM doesn't seem to make it to his blog. I assume >you also pitched actual work on *Python* on, say, the JVM, to him as a >possible candidate for funding. As opposed to any arbitrary language, >which basically lets Tim change the topic to a Ruby discussion. Tim is very Ruby focused. He's not interested in python, he said, because 'the ruby community is more vibrant'. Vibrant is measured in body-count. He's funding all the reimplimentations of Ruby, and know s the people, and likes the people. So he has a certain amount of loyalty to them, too. He really wants to make Rails faster. He says Django is "as good as rails" or something to that effect, but that doesn't change the fact that what he wants to do is make rails faster. >It's also too bad that the whole idea of instrumenting the translation >process to do neat stuff doesn't really make it to his blog post. The >tainting stuff is mentioned, but that's it. What about garbage >collectors and JITs? He saw all of these. When I asked about funding for tools to make it easier to, say, find out if Erlang-type concurrency was a good idea, he said he would think about it. His wide-finder experiment only showed that the Erlang string-processing libraries aren't as good as Python's, not if Erlang's concurrency model would be a good things to stick into languages. He liked the idea of experimentation, very much, but we had just blown him away with the architecture that he didn't want to think about that then. Plus he got an emergency trip to Japan put in his plate at short notice. Think of this as very positive. We go spend a whole day at Sun on Monday, focused on the JVM. John Rose, who is organising this, and who knows Samuele from a currently running Java expert committee, has invited some people, including some JRuby people to attend. John Rose is already _really excited about this_. What's the worst that can happen? We get to support our Python habit by building an interpreter for Ruby. Somebody was going to come along and do it sooner or later, i.e. as soon as PyPy is making everything go at speed, not just integer math. i.e. as soon as we stop spending 50% of our time in the garbage collector! There is nothing but good news here. Be happy for us. >Regards, > >Martijn > >_______________________________________________ >pypy-dev(a)codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev

2 1

pypy on Tim Bray's blog
by Martijn Faassen 11 Nov '07

11 Nov '07

Hi there, http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy It might be worth it for someone to comment on it, as the one comment there is rather negative about PyPy (it'd really help to have PyPy interpreters in production somewhere, as before you do, the criticism in the comment cannot be properly countered) I'm not sure how to interpret Tim Bray's post. Frankly, I was a bit disappointed by what made it in there. It's interesting to analyze as it gives some clue about how your pitch was received and understood. It's too bad the potential of a compliant, maintainable Python implementation on the JVM doesn't seem to make it to his blog. I assume you also pitched actual work on *Python* on, say, the JVM, to him as a possible candidate for funding. As opposed to any arbitrary language, which basically lets Tim change the topic to a Ruby discussion. It's also too bad that the whole idea of instrumenting the translation process to do neat stuff doesn't really make it to his blog post. The tainting stuff is mentioned, but that's it. What about garbage collectors and JITs? Regards, Martijn

2 1

Re: [pypy-dev] [pypy-svn] r48350 - pypy/dist/pypy/objspace/flow
by Carl Friedrich Bolz 08 Nov '07

08 Nov '07

tismer(a)codespeak.net wrote: > Author: tismer > Date: Wed Nov 7 05:18:57 2007 > New Revision: 48350 > > Modified: > pypy/dist/pypy/objspace/flow/specialcase.py > Log: > some over-due relaxing to __import__ annotation > > Modified: pypy/dist/pypy/objspace/flow/specialcase.py > ============================================================================== > --- pypy/dist/pypy/objspace/flow/specialcase.py (original) > +++ pypy/dist/pypy/objspace/flow/specialcase.py Wed Nov 7 05:18:57 2007 > @@ -6,7 +6,18 @@ > from pypy.tool.cache import Cache > > def sc_import(space, fn, args): > - w_name, w_glob, w_loc, w_frm = args.fixedunpack(4) > + args_w, kwds_w = args.unpack() > + assert kwds_w == {}, "should not call %r with keyword arguments" % (fn,) > + assert len(args_w) > 0 and len(args_w) <= 4, 'import needs 1 to 4 arguments' > + w_name = args_w[0] > + w_None = space.wrap(None) > + w_glob, w_loc, w_frm = w_None, w_None, w_None > + if len(args_w) > 1: > + w_glob = args_w[1] > + if len(args_w) > 2: > + w_loc = args_w[2] > + if len(args_w) > 3: > + w_frm = args_w[3] > if not isinstance(w_loc, Constant): > # import * in a function gives us the locals as Variable > # we always forbid it as a SyntaxError Hi Christian, could you please write a couple of tests for this? It's very hard to see what this code is doing and whether it is broken without tests for its functionality. Cheers, Carl Friedrich

4 5