Brett Cannon wrote:

On Thu, Mar 26, 2009 at 18:05, Terry Reedy wrote:

...

If one adds type annotations so that values can be unboxed, would not Cython, etc, do even better for speedup?

Nope as Unladen is planning to re-implement the eval loop, something Cython doesn't optimize without the need to suddenly start adding type annotations to code.

Sorry, but this is not correct. When you let Cython compile a standard Python module, it will not use Python's eval loop inside the module. It is only used by module external code. And you can use "pyximport" to let Cython (try to) also compile external modules on-the-import. This even works for a couple of stdlib modules by now. And you are not required to use type annotations to compile your code into a faster C module. Cython is often faster than CPython by 10-30% even for plain Python code. Plus, you can use function decorators for type annotations to keep your code 100% Python (2.4) compatible, and you can put a .pxd file with type annotations next to your .py module to provide the type annotations externally. This can get you a speedup of factors, without modifying your Python code at all. Stefan

On Thu, Mar 26, 2009 at 11:40 PM, Collin Winter <collinw@gmail.com> wrote:

In fact, right now I'm adding a last few tests before putting our cPickle patches up on the tracker for further review.

Put me in the nosy list when you do; and when I get some free time, I will give your patches a complete review. I've already taken a quick look at cPickle changes you did in Unladen and I think some (i.e., the custom memo table) are definitely worthy to be merged in the mainlines. Cheers, -- Alexandre

Collin Winter wrote:

On Thu, Mar 26, 2009 at 8:05 PM, Terry Reedy <tjreedy@udel.edu> wrote:

...

An ars technica articla just linked to in a python-list post

http://arstechnica.com/open-source/news/2009/03/google-launches-project-to-b...

calls the following project "Google launched" http://code.google.com/p/unladen-swallow/wiki/ProjectPlan

(Though the project page does not really claim that.)

Hi, I'm the tech lead for Unladen Swallow. Jeffrey Yasskin and Thomas Wouters are also working on this project.

Unladen Swallow is Google-sponsored, but not Google-owned. This is an open-source branch that we're working on, focused on performance, and we want to move all of our work upstream as quickly as possible. In fact, right now I'm adding a last few tests before putting our cPickle patches up on the tracker for further review.

Thank you for the answers and comments. Pickle speedups will be welcomed by many. It comes up on python-list from time to time.

...

I am sure some people here might find this interesting.

I'd love to have a faster CPython, but this note: "Will probably kill Python Windows support (for now)." would kill merger back into mainline (for now) without one opposing being 'conservative'.

To clarify, when I wrote 'conservative', I wasn't being disparaging. A

Sorry I was mislead. Perhaps you might want to rewrite to sound more like what you have written here -- something like "There is a risk we will be unable to justify the more radical changes that we propose."

resistance to change can certainly be a good thing, and something that I think is very healthy in these situations. We certainly have to prove ourselves, especially given some of the fairly radical things we're thinking of [1]. We believe we can justify these changes, but I *do* want to be forced to justify them publicly; I don't think python-dev would be doing its job if some of these things were merely accepted without discussion. [snip]

Terry

On Fri, Mar 27, 2009 at 11:50, Paul Moore <p.f.moore@gmail.com> wrote:

2009/3/27 Collin Winter <collinw@gmail.com>:

...

In particular, Windows support is one of those things we'll need to address on our end. LLVM's Windows support may be spotty, or there may be other Windows issues we inadvertently introduce. None of the three of us have Windows machines, nor do we particularly want to acquire them :), and Windows support isn't going to be a big priority. If we find that some of our patches have Windows issues, we will certainly fix those before proposing their inclusion in CPython.

On the assumption (sorry, I've done little more than read the press releases so far) that you're starting from the CPython base and incrementally patching things, you currently have strong Windows support. It would be a shame if that got gradually chipped away through neglect, until it became a big job to reinstate it.

If the Unladen Swallow team doesn't include any Windows developers, you're a bit stuck, I guess, but could you not at least have a Windows buildbot which keeps tabs on the current status? Then you might encourage interested Windows bystanders to check in occasionally and maybe offer fixes.

As things stand, the press releases give me the impression (as a Windows user without a lot of time to invest in contributing) that this project is irrelevant to me, and I should ignore it until you announce "proper" Windows support. By which time, it may have fallen completely off my radar. (On a smaller scale, this happened with virtualenv - I found to my surprise that it now supported Windows, and had for some time without me realising, because when it started it was Unix-only and I had not bothered to keep track of it). Maybe again it's something that could be clarified in the announcements.

It's not a matter of chipping away support. It's a matter of wishing to not write our own JIT, but rather leverage other people's work. That currently means LLVM, but LLVM has a weak Windows story at the moment. Of course, LLVM has little Windows support because it doesn't have any Windows users :-) The changes done so far are (mostly) orthogonal to Windows (the actual performance benefits may depend a little on the platform), but the future work will not be. On the other hand, getting the Windows story straightened out is mostly a matter of getting Windows support in LLVM, and not work specific to Unladen Swallow; hopefully, we can use some of the Windows knowledge on python-dev or elsewhere in the world for that. Notice how I said 'currently' and 'at the moment' and 'future work'. LLVM is also a work in progress, See, for instance, http://llvm.org/docs/GettingStartedVS.html . (And if we were to write our own JIT, we would have the same problem but worse: none of us would be able to write an effective one for Windows, if at all, and we would have a much smaller developer pool to work with. And it would take much longer in the first place.) -- Thomas Wouters <thomas@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Fri, Mar 27, 2009 at 5:50 AM, Paul Moore <p.f.moore@gmail.com> wrote:

2009/3/27 Collin Winter <collinw@gmail.com>:

...

In particular, Windows support is one of those things we'll need to address on our end. LLVM's Windows support may be spotty, or there may be other Windows issues we inadvertently introduce. None of the three of us have Windows machines, nor do we particularly want to acquire them :), and Windows support isn't going to be a big priority. If we find that some of our patches have Windows issues, we will certainly fix those before proposing their inclusion in CPython.

On the assumption (sorry, I've done little more than read the press releases so far) that you're starting from the CPython base and incrementally patching things, you currently have strong Windows support. It would be a shame if that got gradually chipped away through neglect, until it became a big job to reinstate it.

That's correct, we're starting with CPython 2.6.1.

If the Unladen Swallow team doesn't include any Windows developers, you're a bit stuck, I guess, but could you not at least have a Windows buildbot which keeps tabs on the current status? Then you might encourage interested Windows bystanders to check in occasionally and maybe offer fixes.

We're definitely going to set up buildslaves for Windows and other platforms (currently we're only running Linux buildslaves). We're trying to solicit 20% time help from Google Windows developers, but that experience is relatively rare compared to the vast sea of Linux-focused engineers (though that's true of the open-source community in general). Also, it may be that some of the components we're reusing don't support Windows, or perhaps worse, offer degraded performance on Windows. We believe we can fix these problems as they come up -- we certainly don't want Windows issues to prevent patches from going into mainline -- but it's still a risk that Windows issues may slow down our development or prevent us from doing something fancy down the road, and I wanted to be up front about that risk. I've updated our ProjectPlan in hopes of clarifying this. That section of the docs was copy/pasted off a slide, and was a bit too terse :) Collin

Nick Coghlan wrote:

Collin Winter wrote:

...

That would be a bikeshed discussion of such magnitude, you'd have to invent new colors to paint the thing.

Octarine. Definitely octarine :)

I'm not so sure of the color itself, but its name should definitely rhyme with "orange." --Scott David Daniels Scott.Daniels@Acm.Org