Mailman 3 June 2010 - Python-Dev

Re: [Python-Dev] bytes / unicode
by P.J. Eby June 28, 2010

June 28, 2010

At 12:43 PM 6/27/2010 +1000, Nick Coghlan wrote: >While full support for third party strings and >byte sequence implementations is an interesting idea, I think it's >overkill for the specific problem of making it easier to write >str/bytes agnostic functions for tasks like URL parsing. OTOH, to write your partial implementation is almost as complex - it still must take into account joining and formatting, and so by that point, you've just proposed a new protocol for coercion... so why not just make the coercion protocol explicit in the first place, rather than hardwiring a third type's worth of special cases? Remember, bytes and strings already have to detect mixed-type operations. If there was an API for that, then the hardcoded special cases would just be replaced, or supplemented with type slot checks and calls after the special cases. To put it another way, if you already have two types special-casing their interactions with each other, then rather than add a *third* type to that mix, maybe it's time to have a protocol instead, so that the types that care can do the special-casing themselves, and you generalize to N user types. (Btw, those who are saying that the resulting potential for N*N interaction makes the feature unworkable seem to be overlooking metaclasses and custom numeric types -- two Python features that in principle have the exact same problem, when you use them beyond a certain scope. At least with those features, though, you can generally mix your user-defined metaclasses or numeric types with the Python-supplied basic ones and call arbitrary Python functions on them, without as much heartbreak as you'll get with a from-scratch stringlike object.) All that having been said, a new protocol probably falls under the heading of the language moratorium, unless it can be considered "new methods on builtins"? (But that seems like a stretch even to me.) I just hate the idea that functions taking strings should have to be *rewritten* to be explicitly type-agnostic. It seems *so* un-Pythonic... like if all the bitmasking functions you'd ever written using 32-bit int constants had to be rewritten just because we added longs to the language, and you had to upcast them to be compatible or something. Sounds too much like C or Java or some other non-Python language, where dynamism and polymorphy are the special case, instead of the general rule.

6 7

Re: [Python-Dev] email package status in 3.X
by lutz＠rmi.net June 27, 2010

June 27, 2010

I wasn't calling Python 3 a turd. I was trying to show the strangeness of the logic behind your rationalization. And failing badly... (maybe I should have used "tar ball"?) What I'm suggesting is that extreme caution be exercised from this point forward with all things 3.X-related. Whether you wish to accept this or not, 3.X has a negative image to many. This suggestion specifically includes not abandoning current 3.X email package users as a case in point. Ripping the rug out from new 3.X users after they took the time to port seems like it may be just enough to tip the scales altogether. --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) > -----Original Message----- > From: Michael Foord <fuzzyman(a)voidspace.org.uk> > To: lutz(a)rmi.net > Subject: Re: [Python-Dev] email package status in 3.X > Date: Fri, 18 Jun 2010 18:27:46 +0100 > > On 18/06/2010 18:22, lutz(a)rmi.net wrote: > >> Python 3.0 was *declared* to be an experimental release, and by most > >> standards 3.1 (in terms of the core language and functionality) was a > >> solid release. > >> > >> Any reasonable expectation about Python 3 adoption predicted that it > >> would take years, and would include going through a phase of difficulty > >> and disappointment... > >> > > Declaring something to be a turd doesn't change the fact that > > it's a turd. > > Right - but *you're* the one calling it a turd, which is not a helpful > approach or likely to achieve *anything* useful. I still have no idea > what you are actually suggesting. > > > I have a feeling that most people outside this > > list would have much rather avoided the difficulty and > > disappointment altogether. > > > > Let's be honest here; 3.X was released to the community in part > > as an extended beta. > > Correction - 3.0 was an experimental release. That is not true of 3.1 > and future releases. > > All the best, > > Michael > > That's not a problem, unless you drop the > > word "beta". And if you're still not buying that, imagine the sort > > of response you'd get if you tried to sell software that billed > > itself as "experimental", and promised a phase of "disappointment". > > Why would you expect the Python world to react any differently? > > > > > >> Whilst I agree that there are plenty of issues to workon, and I don't > >> underestimate the difficulty of some of them, I think "half-baked" is > >> very much overblown. Whilst you have a lot to say about how much of a > >> problem this is I don't understand what you are suggesting be *done*? > >> > > I agree that 3.X isn't all bad, and I very much hope it succeeds. And > > no, I have no answers; I'm just reporting the perception from downwind. > > > > So here it is: The prevailing view is that 3.X developers hoisted things > > on users that they did not fully work through themselves. Unicode is > > prime among these: for all the talk here about how 2.X was broken in > > this regard, the implications of the 3.X string solution remain to be > > fully resolved in the 3.X standard library to this day. What is a > > common Python user to make of that? > > > > --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) > > > > > > > > > -- > http://www.ironpythoninaction.com/ > http://www.voidspace.org.uk/blog > > READ CAREFULLY. By accepting and reading this email you agree, on behalf of > your employer, to release me from all obligations and waivers arising from > any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, > clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and > acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with > your employer, its partners, licensors, agents and assigns, in perpetuity, > without prejudice to my ongoing rights and privileges. You further represent > that you have the authority to release me from any BOGUS AGREEMENTS on behalf > of your employer. > > >

6 7

Python Library Support in 3.x (Was: email package status in 3.X)
by Stephen Thorne June 27, 2010

June 27, 2010

Steve Holden Wrote: > We are also attempting to enable tax-deductible fund raising to increase > the likelihood of David's finding support. Perhaps we need to think > about a broader campaign to increase the quality of the python 3 > libraries. I find it very annoying that the #python IRC group still has > "Don't use Python 3" in it's topic. They adamantly refuse to remove it > until there is better library support, and they are the guys who see the > issues day in day out so it is hard to argue with them (and I don't > think an autocratic decision-making process would be appropriate). Yes, #python keeps the text "It's too early to use Python 3.x" in its topic. Library support is the only reason. -- Regards, Stephen Thorne Development Engineer

33 98

Re: [Python-Dev] bytes / unicode
by P.J. Eby June 27, 2010

June 27, 2010

At 03:53 PM 6/27/2010 +1000, Nick Coghlan wrote: >We could talk about this even longer, but the most effective way >forward is going to be a patch that improves the URL parsing >situation. Certainly, it's the only practical solution for the immediate problems in 3.2. I only mentioned that I "hate the idea" because I'd be more comfortable if it was explicitly declared to be a temporary hack to work around the absence of a string coercion protocol, due to the moratorium on language changes. But, since the moratorium *is* in effect, I'll try to make this my last post on string protocols for a while... and maybe wait until I've looked at the code (str/bytes C implementations) in more detail and can make a more concrete proposal for what the protocol would be and how it would work. (Not to mention closer to the end of the moratorium.) >There are a *very small* number of APIs where it is appropriate to >be polymorphic This is only true if you focus exclusively on bytes vs. unicode, rather than the general issue that it's currently impractical to pass *any* sort of user-defined string type through code that you don't directly control (stdlib or third-party). >The virtues of a separate poly_str type are that: >1. It can be simple and implemented in Python, dispatching to str or >bytes as appropriate (probably in the strings module) >2. No chance of impacting the performance of the core interpreter (as >builtins are not affected) Note that adding a string coercion protocol isn't going to change core performance for existing cases, since any place where the protocol would be invoked would be a code branch that either throws an error or *already* falls back to some other protocol (e.g. the buffer protocol). >3. Lower impact if it turns out to have been a bad idea How many protocols have been added that turned out to be bad ideas? The only ones that have been removed in 3.x, IIRC, are three-way compare, slice-specific operations, and __coerce__... and I'm going to miss __cmp__. ;-) However, IIUC, the reason these protocols were dropped isn't because they were "bad ideas". Rather, they're things that can be implemented in terms of a finer-grained protocol. i.e., if you want __cmp__ or __getslice__ or __coerce__, you can always implement them via a mixin that converts the newer fine-grained protocols into invocations of the older protocol. (As I plan to do for __cmp__ in the handful of places I use it.) At the moment, however, this isn't possible for multi-string operations outside of __add__/__radd__ and comparison -- the coercion rules are hard-wired and can't be overridden by user-defined types.

1 0

Re: [Python-Dev] bytes / unicode
by P.J. Eby June 27, 2010

June 27, 2010

At 04:49 PM 6/25/2010 +0900, Stephen J. Turnbull wrote: >P.J. Eby writes: > > > This doesn't have to be in the functions; it can be in the > > *types*. Mixed-type string operations have to do type checking and > > upcasting already, but if the protocol were open, you could make an > > encoded-bytes type that would handle the error checking. > >Don't you realize that "encoded-bytes" is equivalent to use of a very >limited profile of ISO 2022 coding extensions? Such as Emacs/MULE >internal encoding or TRON code? It has been tried. It does not work. > >I understand how types can do such checking; my point is that the >encoded-bytes type doesn't have enough information to do it in the >cases where you think it is better than converting to str. There are >*no useful operations* that can be done on two encoded-bytes with >different encodings unless you know the ultimate target codec. I do know the ultimate target codec -- that's the point. IOW, I want to be able to do to all my operations by passing target-encoded strings to polymorphic functions. Then, the moment something creeps in that won't go to the target codec, I'll be able to track down the hole in the legacy code that's letting bad data creep in. > The >only sensible way to define the concatenation of ('ascii', 'English') >with ('euc-jp','ÆüËÜ¸ì') is something like ('ascii', 'English', >'euc-jp','ÆüËÜ¸ì'), and *not* ('euc-jp','EnglishÆüËÜ¸ì'), because you >don't know that the ultimate target codec is 'euc-jp'-compatible. >Worse, you need to build in all the information about which codecs are >mutually compatible into the encoded-bytes type. For example, if the >ultimate target is known to be 'shift_jis', it's trivially compatible >with 'ascii' and 'euc-jp' requires a conversion, but latin-9 you can't >have. The interaction won't be with other encoded bytes, it'll be with other *unicode* strings. Ones coming from other code, and literals embedded in the stdlib. >No, the problem is not with the Unicode, it is with the code that >allows characters not encodable with the target codec. And which code that is, precisely, is the thing that may be very difficult to find, unless I can identify it at the first point it enters (and corrupts) my output data. When dealing with a large code base, this may be a nontrivial problem.

3 6

PyPy 1.3 released
by Maciej Fijalkowski June 26, 2010

June 26, 2010

======================= PyPy 1.3: Stabilization ======================= Hello. We're please to announce release of PyPy 1.3. This release has two major improvements. First of all, we stabilized the JIT compiler since 1.2 release, answered user issues, fixed bugs, and generally improved speed. We're also pleased to announce alpha support for loading CPython extension modules written in C. While the main purpose of this release is increased stability, this feature is in alpha stage and it is not yet suited for production environments. Highlights of this release ========================== * We introduced support for CPython extension modules written in C. As of now, this support is in alpha, and it's very unlikely unaltered C extensions will work out of the box, due to missing functions or refcounting details. The support is disable by default, so you have to do:: import cpyext before trying to import any .so file. Also, libraries are source-compatible and not binary-compatible. That means you need to recompile binaries, using for example:: python setup.py build Details may vary, depending on your build system. Make sure you include the above line at the beginning of setup.py or put it in your PYTHONSTARTUP. This is alpha feature. It'll likely segfault. You have been warned! * JIT bugfixes. A lot of bugs reported for the JIT have been fixed, and its stability greatly improved since 1.2 release. * Various small improvements have been added to the JIT code, as well as a great speedup of compiling time. Cheers, Maciej Fijalkowski, Armin Rigo, Alex Gaynor, Amaury Forgeot d'Arc and the PyPy team

2 1

"2 or 3" link on python.org
by Stephen Thorne June 25, 2010

June 25, 2010

Steve Holden Wrote: > Given the amount of interest this thread has generated I can't help > wondering why it isn't more prominent in python.org content. Is the > developer community completely disjoint with the web content editor > community? > > If there is such a disconnect we should think about remedying it: a > large "Python 2 or 3?" button could link to a reasoned discussion of the > pros and cons as evinced in this thread. That way people will end up > with the right version more often (and be writing Python 2 that will > more easily migrate to Python 3, if they cannot yet use 3). > > There seems to be a perception that the PSF can help fund developments, > and indeed Jesse Noller has made a small start with his sprint funding > proposal (which now has some funding behind it). I think if it is to do > so the Foundation will have to look for substantial new funding. I do > not currently understand where this funding would come from, and would > like to tap your developer creativity in helping to define how the > Foundation can effectively commit more developer time to Python. > > GSoC and GHOP are great examples, but there is plenty of room for all > sorts of initiatives that result in development opportunities. I'd like > to help. I am extremely keen for this to happen. Does anyone have ownership of this project? There was some discussion of it up-list but the discussion fizzled. -- Regards, Stephen Thorne Development Engineer Netbox Blue

7 13

docs - Copy
by Rich Healey June 25, 2010

June 25, 2010

http://docs.python.org/library/copy.html Just near the bottom it reads: """Shallow copies of dictionaries can be made using dict.copy(), and of lists by assigning a slice of the entire list, for example, copied_list = original_list[:].""" Surely this is a typo? To my understanding, copied_list = original_list[:] gives you a clean copy (slicing returns a new object....) Can this be updated? Or someone explain to me why it's correct? Cheers Example: >>> t = [1, 2, 3] >>> y = t >>> u = t[:] >>> y[1] = "rawr" >>> t [1, 'rawr', 3] >>> u [1, 2, 3] >>>

6 8

Schedule for Python 2.6.6
by Barry Warsaw June 25, 2010

June 25, 2010

Benjamin is still planning to release Python 2.7 final on 2010-07-03, so it's time for me to work out the release schedule for Python 2.6.6 - likely the last maintenance release for Python 2.6. Because summer schedules are crazy, and I want to leave two weeks between 2.6.6 rc1 and 2.6.6 final, my current schedule looks like: * Python 2.6.6 rc 1 on Monday 2010-08-02 * Python 2.6.6 final on Monday 2010-08-16 This should give folks plenty of time to relax after 2.7 final, and still be able to get those last minute fixes into the 2.6 tree. Let me know if these dates don't work for you. -Barry

3 7

Creating APIs that work as both decorators and context managers
by Michael Foord June 25, 2010

June 25, 2010

Hello all, I've put a recipe up on the Python cookbook for creating APIs that work as both decorators and context managers and wonder if it would be considered a useful addition to the functools module. http://code.activestate.com/recipes/577273-decorator-and-context-manager-fr… I wrote this after writing almost identical code the second time for "patch" in the mock module. (The patch decorator can be used as a decorator or as a context manager and I was writing a new variant.) Both py.test and django have similar code in places, so it is not an uncommon pattern. It is only 40 odd lines (ignore the ugly Python 2 & 3 compatibility hack), so I'm fine with it living on the cookbook - but it is at least slightly fiddly to write and has the added niceness of providing the optional exception handling semantics of __exit__ for decorators as well. Example use (really hope email doesn't swallow the whitespace - my apologies in advance if it does): from context import Context class mycontext(Context): def __init__(self, *args): """Normal initialiser""" def start(self): """ Called on entering the with block or starting the decorated function. If used in a with statement whatever this method returns will be the context manager. """ def finish(self, *exc): """ Called on exit. Arguments and return value of this method have the same meaning as the __exit__ method of a normal context manager. """ @mycontext('some', 'args') def function(): pass with mycontext('some', 'args') as something: pass I'm not entirely happy with the name of the class or the start and finish methods, so open to suggestions there. start and finish *could* be __enter__ and __exit__ - but that would make the class you implement *look* like a normal context manager and I thought it was better to distinguish them. Perhaps before and after? All the best, Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

2 2