Mailman 3 June 2010 - Python-Dev

Re: [Python-Dev] email package status in 3.X
by P.J. Eby 25 Jun '10

25 Jun '10

At 10:57 AM 6/20/2010 -0700, Guido van Rossum wrote: >The problem comes exactly where you find it: when *porting* existing >code that uses aforementioned ways to alleviate the pain, you find >that the hacks no longer work and a properly layered design is needed >that clearly distinguishes between which variables contain bytes and >which text. Actually, I would say that it's more that (in the network protocol case) we *have* bytes, some of which we would like to *treat* as text, yet do not wish to constantly convert back and forth to full-blown unicode -- especially since the protocols themselves designate ASCII or latin-1 at the transport layer (sometimes with odder encodings above, but these already have to be explicitly dealt with by existing code). While reading over this thread, I'm wondering whether at least my (WSGI-related) problems in this area would be solved by the availability of a type (say "bstr") that was simply a wrapper providing string-like behavior over an underlying bytes, byte array, or memoryview, that would produce objects of compatible type when combined with strings (by encoding them to match). Then, I could wrap bytes with it to pass them to string operations, and then feed them back into everything else. The bstr type ideally would be directly compatible with bytes I/O, or at least have a .bytes attribute that would be. It seems like that would reduce WSGI porting issues quite a bit, since it would mostly consist of throwing extra bstr() calls in where things are breaking, and maybe grabbing the .bytes attribute for I/O. This approach would still be explicit as to what types you're working with, but would not require O(n) *conversions* at every interaction boundary. It would be limited, of course, to single-byte encodings with all characters (0-255) valid. OTOH, maybe there should just be a bytestrings module with bytestrings.ascii and bytestrings.latin1, and between the two that should cover the network protocol needs quite well. Actually, if the Python 3 str() constructor could do O(1) conversion for the latin-1 case (i.e., just wrapped the underlying bytes), I would just put, "bstr = lambda x: str(x,'latin-1')" at the top of my programs and have roughly the same effect. This idea is still a bit half-baked, but a more baked version might be just the ticket for porting stuff that used str to work with bytes in 2.x, if only because writing, e.g.: newurl = bstr(urljoin(bstr(base), 'subdir')) seems so much saner than writing *this* everywhere: newurl = str(urljoin(str(base, 'latin-1'), 'subdir'), 'latin-1') It is perhaps a bit late to propose this idea, since ideally we would also want to use it in 2.x to aid porting. But I'm curious if any other people here experiencing byte/unicode woes in relation to network protocols would find this a solution to their chief frustration. (i.e., that the stdlib often insists now on strings, where effectively bytes were usable before, and thus one must do conversions both coming and going.)

23 85

Summary of Python tracker Issues
by Python tracker 25 Jun '10

25 Jun '10

ACTIVITY SUMMARY (2010-06-18 - 2010-06-25) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2795 open (+38) / 18104 closed (+14) / 20899 total (+52) Open issues with patches: 1130 Average duration of open issues: 712 days. Median duration of open issues: 503 days. Open Issues Breakdown open 2765 (+38) languishing 13 ( +0) pending 16 ( +0) Issues Created Or Reopened (55) _______________________________ os.path.normcase documentation/behaviour unclear on Mac OS X 2010-06-25 http://bugs.python.org/issue3485 reopened ezio.melotti patch uuid.uuid4() generates non-unique values on OSX 2010-06-21 http://bugs.python.org/issue8621 reopened skrah patch test_support.run_unittest cmdline options and arguments 2010-06-20 http://bugs.python.org/issue9028 reopened techtonik errors='replace' works in IDLE, fails at Windows command line. 2010-06-18 http://bugs.python.org/issue9029 created jvanpraag ctypes variable limits 2010-06-18 http://bugs.python.org/issue9030 created kumma distutils uses invalid "-Wstrict-prototypes" flag when compili 2010-06-18 http://bugs.python.org/issue9031 created matteo.vescovi xmlrpc: Transport.request() should also catch socket.error(EPI 2010-06-18 http://bugs.python.org/issue9032 created haypo patch cmd module tab misbehavior 2010-06-19 http://bugs.python.org/issue9033 created slcott datetime module should use int32_t for date/time components 2010-06-20 http://bugs.python.org/issue9034 created belopolsky os.path.ismount on windows doesn't support windows mount point 2010-06-20 http://bugs.python.org/issue9035 created Oren_Held Simplify Py_CHARMASK 2010-06-20 http://bugs.python.org/issue9036 created skrah patch, needs review Add explanation as to how to raise a custom exception in the e 2010-06-20 http://bugs.python.org/issue9037 created jonathan.underwood patch test_distutils failure 2010-06-20 http://bugs.python.org/issue9038 created pitrou IDLE and module Doc 2010-06-20 http://bugs.python.org/issue9039 created Yoda_Uchiha using MIMEApplication to attach a PDF raises a TypeError excep 2010-06-21 http://bugs.python.org/issue9040 created Enrico.Sartori raised exception is misleading 2010-06-21 http://bugs.python.org/issue9041 created kumma Gettext cache and classes 2010-06-21 http://bugs.python.org/issue9042 created v_peter patch 2to3 doesn't handle byte comparison well 2010-06-21 CLOSED http://bugs.python.org/issue9043 created vdupras [optparse] confusion over an option and its value without any 2010-06-21 http://bugs.python.org/issue9044 created kszawala 2.7rc1: 64-bit OSX installer is not built with 64-bit tkinter 2010-06-21 http://bugs.python.org/issue9045 created srid Python 2.7rc2 doesn't build on Mac OS X 10.4 2010-06-21 http://bugs.python.org/issue9046 created lemburg Python 2.7rc2 includes -isysroot twice on each gcc command lin 2010-06-21 http://bugs.python.org/issue9047 created lemburg no OS X buildbots in the stable list 2010-06-21 http://bugs.python.org/issue9048 created janssen buildbot UnboundLocalError in nested function 2010-06-21 CLOSED http://bugs.python.org/issue9049 created Andreas Hofmeister UnboundLocalError in nested function 2010-06-21 CLOSED http://bugs.python.org/issue9050 created Andreas Hofmeister Improve pickle format for aware datetime instances 2010-06-21 http://bugs.python.org/issue9051 created belopolsky 2.7rc2 fails test_urllib_localnet tests on OS X 2010-06-21 CLOSED http://bugs.python.org/issue9052 created janssen distutils compiles extensions so that Python.h cannot be found 2010-06-21 http://bugs.python.org/issue9053 created exarkun pyexpat configured with "--with-system-expat" is incompatible 2010-06-21 http://bugs.python.org/issue9054 created dmalcolm patch test_issue_8959_b fails when run from a service 2010-06-21 http://bugs.python.org/issue9055 created pmoore buildbot Adding additional level of bookmarks and section numbers in py 2010-06-22 http://bugs.python.org/issue9056 created pengyu.ut Distutils2 needs a home page 2010-06-22 http://bugs.python.org/issue9057 created dabrahams PyUnicodeDecodeError_Create asserts that various arguments are 2010-06-22 CLOSED http://bugs.python.org/issue9058 created dmalcolm patch Backwards compatibility 2010-06-23 CLOSED http://bugs.python.org/issue9059 created Raven Python/dup2.c doesn't compile on (at least) newlib 2010-06-23 http://bugs.python.org/issue9060 created torne patch cgi.escape Can Lead To XSS Vulnerabilities 2010-06-23 http://bugs.python.org/issue9061 created Craig.Younkins urllib.urlopen crashes when launched from a thread 2010-06-23 CLOSED http://bugs.python.org/issue9062 created olivier-berten TZ examples in datetime.rst are incorrect 2010-06-23 http://bugs.python.org/issue9063 created belopolsky pdb enhancement up/down traversals 2010-06-23 http://bugs.python.org/issue9064 created vandyswa patch tarfile: default root:root ownership is incorrect. 2010-06-23 http://bugs.python.org/issue9065 created jsbronder patch Standard type codes for array.array, same as struct 2010-06-24 http://bugs.python.org/issue9066 created cmcqueen1975 Use macros from pyctype.h 2010-06-24 http://bugs.python.org/issue9067 created skrah "from . import *" 2010-06-24 CLOSED http://bugs.python.org/issue9068 created bhy test_float failure on Solaris 2010-06-24 http://bugs.python.org/issue9069 created mark.dickinson Timestamps are rounded differently in py3k and trunk 2010-06-24 CLOSED http://bugs.python.org/issue9070 created belopolsky TarFile doesn't support member files with a leading "./" 2010-06-24 CLOSED http://bugs.python.org/issue9071 created free.ekanayaka Unloading modules - memleaks? 2010-06-24 CLOSED http://bugs.python.org/issue9072 created yappie Tkinter module missing from install on OS X 10.6.4 2010-06-24 http://bugs.python.org/issue9073 created RolandJ [includes patch] subprocess module closes standard file descri 2010-06-24 http://bugs.python.org/issue9074 created kr patch ssl module sets "debug" flag on SSL struct 2010-06-24 CLOSED http://bugs.python.org/issue9075 created pitrou Add C-API documentation for PyUnicode_AsDecodedObject/Unicode 2010-06-24 http://bugs.python.org/issue9076 created haypo patch argparse does not handle arguments correctly after -- 2010-06-24 CLOSED http://bugs.python.org/issue9077 created iElectric Fix C API documentation of unicode 2010-06-24 http://bugs.python.org/issue9078 created haypo patch Make gettimeofday available in time module 2010-06-25 http://bugs.python.org/issue9079 created belopolsky patch, needs review Provide list prepend method (even though it's not efficient) 2010-06-25 CLOSED http://bugs.python.org/issue9080 created andybuckley Issues Now Closed (43) ______________________ MultiMethods with type annotations in 3000 1035 days http://bugs.python.org/issue1004 benjamin.peterson patch subprocess.list2cmdline doesn't do pipe symbols 975 days http://bugs.python.org/issue1300 chops(a)demiurgestudios.com easy Popen.poll always returns None 816 days http://bugs.python.org/issue2475 tjreedy Python interpreter uses Unicode surrogate pairs only before th 713 days http://bugs.python.org/issue3297 haypo patch py3k shouldn't use -fno-strict-aliasing anymore 712 days http://bugs.python.org/issue3326 benjamin.peterson patch create a numbits() method for int and long types 699 days http://bugs.python.org/issue3439 mark.dickinson patch, needs review os.path.realpath() get the wrong result 554 days http://bugs.python.org/issue4654 r.david.murray Compiling python 2.5.2 under Wine on linux. 527 days http://bugs.python.org/issue4883 BreamoreBoy 3.0 sqlite doc: most examples refer to pysqlite2, use 2.x synt 516 days http://bugs.python.org/issue5005 tjreedy Implement a way to change the python process name 448 days http://bugs.python.org/issue5672 piro patch setup build with Platform SDK, finding vcvarsall.bat 407 days http://bugs.python.org/issue5969 georg.brandl Failing test_signal.py on Redhat 4.1.2-44 407 days http://bugs.python.org/issue5972 georg.brandl datetime.strptime doesn't support %z format ? 1 days http://bugs.python.org/issue6641 merwok patch webbrowser.get("firefox") does not work on Mac with installed 243 days http://bugs.python.org/issue7192 ronaldoussoren patch Backport 3.x nonlocal keyword to 2.7 117 days http://bugs.python.org/issue8018 mark.dickinson test_heapq interfering with test_import on py3k 65 days http://bugs.python.org/issue8440 tim.golden enumerate() test cases do not cover optional start argument 46 days http://bugs.python.org/issue8636 merwok patch _ssl.c uses PyWeakref_GetObject but doesn't incref result 45 days http://bugs.python.org/issue8682 pitrou patch Remove "w" format of PyParse_ParseTuple() 27 days http://bugs.python.org/issue8850 haypo patch msvc9compiler.py: find_vcvarsall() doesn't work with VS2008 on 23 days http://bugs.python.org/issue8854 lemburg patch, 64bit execfile does not work with UNC paths 21 days http://bugs.python.org/issue8869 tim.golden getargs.c: release the buffer on error 18 days http://bugs.python.org/issue8926 haypo patch PyArg_Parse*(): "z" should not accept bytes 16 days http://bugs.python.org/issue8949 haypo patch PyArg_Parse*(): factorize code of 's' and 'z' formats, and 'u' 16 days http://bugs.python.org/issue8951 haypo patch WINFUNCTYPE wrapped ctypes callbacks not functioning correctly 12 days http://bugs.python.org/issue8959 theller Year range in timetuple 5 days http://bugs.python.org/issue9005 belopolsky patch os.path.normcase(None) does not raise an error on linux and sh 8 days http://bugs.python.org/issue9018 ezio.melotti patch, easy 2to3 doesn't handle byte comparison well 0 days http://bugs.python.org/issue9043 merwok UnboundLocalError in nested function 1 days http://bugs.python.org/issue9049 mark.dickinson UnboundLocalError in nested function 0 days http://bugs.python.org/issue9050 merwok 2.7rc2 fails test_urllib_localnet tests on OS X 0 days http://bugs.python.org/issue9052 belopolsky PyUnicodeDecodeError_Create asserts that various arguments are 0 days http://bugs.python.org/issue9058 benjamin.peterson patch Backwards compatibility 0 days http://bugs.python.org/issue9059 ezio.melotti urllib.urlopen crashes when launched from a thread 0 days http://bugs.python.org/issue9062 orsenthil "from . import *" 0 days http://bugs.python.org/issue9068 brett.cannon Timestamps are rounded differently in py3k and trunk 0 days http://bugs.python.org/issue9070 belopolsky TarFile doesn't support member files with a leading "./" 1 days http://bugs.python.org/issue9071 free.ekanayaka Unloading modules - memleaks? 0 days http://bugs.python.org/issue9072 yappie ssl module sets "debug" flag on SSL struct 0 days http://bugs.python.org/issue9075 pitrou argparse does not handle arguments correctly after -- 1 days http://bugs.python.org/issue9077 iElectric Provide list prepend method (even though it's not efficient) 0 days http://bugs.python.org/issue9080 andybuckley webbrowser.open_new() opens in an existing browser window 2463 days http://bugs.python.org/issue812089 r.david.murray mbcs encoding ignores errors 2394 days http://bugs.python.org/issue850997 haypo patch Top Issues Most Discussed (10) ______________________________ 19 Non-uniformity in randrange for large arguments. 7 days open http://bugs.python.org/issue9025 19 2.7: eval hangs on AIX 8 days open http://bugs.python.org/issue9020 17 Python 2.7rc2 doesn't build on Mac OS X 10.4 4 days open http://bugs.python.org/issue9046 14 test_float failure on Solaris 1 days open http://bugs.python.org/issue9069 13 msvc9compiler.py: find_vcvarsall() doesn't work with VS2008 on 23 days closed http://bugs.python.org/issue8854 10 no OS X buildbots in the stable list 4 days open http://bugs.python.org/issue9048 10 os.path.normcase(None) does not raise an error on linux and sho 8 days closed http://bugs.python.org/issue9018 8 Provide list prepend method (even though it's not efficient) 0 days closed http://bugs.python.org/issue9080 8 Improve quality of Python/dtoa.c 9 days open http://bugs.python.org/issue9009 8 Add Mercurial support to patchcheck 10 days open http://bugs.python.org/issue8999

1 0

Re: [Python-Dev] bytes / unicode
by P.J. Eby 24 Jun '10

24 Jun '10

At 10:29 AM 6/21/2010 -0700, Guido van Rossum wrote: >Perhaps there are more situations where a polymorphic API would be >helpful. Such APIs are not always so easy to implement, because they >have to be careful with literals or other constants (and even more so >mutable state) used internally -- but it can be done, and there are >plenty of examples in the stdlib. What if we could use the time machine to make the APIs that *were* polymorphic, regain their previously-polymorphic status, without needing to actually *change* any of the code of those functions? That's what Barry's ebytes proposal would do, with appropriate coercion rules. Passing ebytes into such a function would yield back ebytes, even if the function used strings internally, as long as those strings could be encoded back to bytes using the ebytes' encoding. (Which would normally be the case, since stdlib constants are almost always ASCII, and the main use cases for ebytes would involve ascii-extended encodings.) >I'm stil unclear on exactly what bstr is supposed to be, but it sounds >a bit like one of the rejected proposals for having a single >(Unicode-capable) str type that is implemented using different width >encodings (Latin-1, UCS-2, UCS-4) underneath. Not quite - as modified by Barry's proposal (which I like better than mine) it'd be an object that just combines bytes with an attribute indicating the underlying encoding. When it interacts with strings, the strings are *encoded* to bytes, rather than upgrading the bytes to text. This is actually a big advantage for error-detection in any application where you're working with data that *must* be encodable in a specific encoding for output, as it allows you to catch errors much *earlier* than you would if you only did the encoding at your output boundary. Anyway, this would not be the normal bytes type or string type; it's "bytes with an encoding". It's also more general than Unicode, in the sense that it allows you to work with character sets that don't really *have* a proper Unicode mapping. One issue I remember from my "enterprise" days is some of the Asian-language developers at NTT/Verio explaining to me that unicode doesn't actually solve certain issues -- that there are use cases where you really *do* need "bytes plus encoding" in order to properly express something. Unfortunately, I never quite wrapped my head around the idea, I just remember it had something to do with the fact that Unicode has single character codes that mean different things in different languages, such that you were actually losing information by converting to unicode, or something like that. (Or maybe the characters were expressed differently in certain encodings according to what language they came from, so you couldn't roundtrip them through unicode without losing information. I think that's probably was what it was; maybe somebody here can chime in more on that point.) Anyway, a type like this would need to have at least a bit of support from the core language, because the str type would need to be able to handle at least the __contains__ and %/.format() coercion cases, since these functions don't have __r*__ equivalents that a user-implemented type could provide... and strings don't have anything like a '__coerce__' either. If sufficient hooks existed, then an ebytes could be implemented outside the stdlib, and still used within it.

4 3

os.getgroups() on MacOS X Was: red buildbots on 2.7
by Alexander Belopolsky 24 Jun '10

24 Jun '10

On Wed, Jun 23, 2010 at 2:08 AM, Ronald Oussoren <ronaldoussoren(a)mac.com> wrote: .. >> >>> * [Ronald's proposal] results in posix.getgroups not reflecting results of posix.setgroups >>> >> >> This effectively substitutes getgrouplist called on the current user >> for getgroups. In 3.x, I believe the correct action will be to >> provide direct access to getgrouplist which is while not POSIX (yet?), >> is widely available. > > I don't mind adding getgrouplist, but that issue is seperator from this one. BTW. Appearently getgrouplist is posix > (<http://refspecs.freestandards.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-gener…>), although this isn't a > requirement for being added to the posix module. > (The link you provided leads to "Linux Standard Base Core Specification," which is different from POSIX, but the distinction is not relevant for our discussion.) > > It is still my opinion that the second option is preferable for better compatibility with system tools, even if the patch > is more complicated and the library function we use can be considered to be broken. Let me try to formulate what the disagreement is. There are two different group lists that can be associated with a running process: 1) The list of current supplementary group IDs maintained by the system for each process and stored in per-process system tables; and 2) The list of the groups that include the uid under which the process is running as a member. The first list is returned by a system call getgroups and the second can be obtained using system database access functions as follows: pw = getpwuid(getuid()) getgrouplist(pw->pw_name, ..) The first list can be modified by privileged processes using setgroups system call, while the second changes when system databases change. The problem that _DARWIN_C_SOURCE introduces is that it replaces system getgroups with a database query effectively making the true process' list of supplementary group IDs inaccessible to programs. See source code at <http://www.opensource.apple.com/source/Libc/Libc-594.1.4/sys/getgroups.c>. The problem is complicated by the fact that OSX true getgroups call appears to truncate the list of groups to NGROUPS_MAX=16. Note, however that it is not clear whether the system call truncates the list or the underlying process tables are limited to 16 entries and additional groups are ignored when the process is created. In my view, getgroups and getgrouplist are two fundamentally different operations and both should be provided by the os module. Redefining os.getgroups to invoke getgrouplist instead of system getgroups on one particular platform to work around that platform's system call limitation is not right.

6 7

Use of cgi.escape can lead to XSS vulnerabilities
by Craig Younkins 24 Jun '10

24 Jun '10

Hello, The method in question: http://docs.python.org/library/cgi.html#cgi.escape http://svn.python.org/view/python/tags/r265/Lib/cgi.py?view=markup # at the bottom "Convert the characters '&', '<' and '>' in string s to HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. If the optional flag quote is true, the quotation mark character ('"') is also translated; this helps for inclusion in an HTML attribute value, as in <A HREF="...">. If the value to be quoted might include single- or double-quote characters, or both, consider using the quoteattr() function in the xml.sax.saxutils module instead." cgi.escape never escapes single quote characters, which can easily lead to a Cross-Site Scripting (XSS) vulnerability. This seems to be known by many, but a quick search reveals many are using cgi.escape for HTML attribute escaping. The intended use of this method is unclear to me. Up to and including the latest published version of Mako (0.3.3), this method was the HTML escaping method. Used in this manner, single-quoted attributes with user-supplied data are easily susceptible to cross-site scripting vulnerabilities. Proof of concept in Mako: >>> from mako.template import Template >>> print Template("<div class='${data}'>", default_filters=['h']).render(data="' onload='alert(1);' id='") <div class='' onload='alert(1);' id=''> I've emailed Michael Bayer, the creator of Mako, and this will be fixed in version 0.3.4. While the documentation says "if the value to be quoted might include single- or double-quote characters... [use the] xml.sax.saxutils module instead," it also implies that this method will make input safe for HTML. Because this method escapes 4 of the 5 key XML characters, it is reasonable to expect some will use it in the manner Mako did. I suggest rewording the documentation for the method making it more clear what it should and should not be used for. I would like to see the method changed to properly escape single-quotes, but if it is not changed, the documentation should explicitly say this method does not make input safe for inclusion in HTML. Shameless plug: http://www.PythonSecurity.org/<http://www.pythonsecurity.org/> Craig Younkins

3 3

swig/python and intel's threadedbuildginblocks
by tullarisc 23 Jun '10

23 Jun '10

Hi, I've compiled intel's OSS threadedbuidlingblocks library on OpenBSD and put everything in some swig interfaces. Here you go: http://tullarisc.xtreemhost.com/swig.ttb.tgz Love, tullarisc. -- View this message in context: http://old.nabble.com/swig-python-and-intel%27s-threadedbuildginblocks-tp28… Sent from the Python - python-dev mailing list archive at Nabble.com.

1 0

Re: [Python-Dev] email package status in 3.X
by lutz＠rmi.net 23 Jun '10

23 Jun '10

> Python 3.0 was *declared* to be an experimental release, and by most > standards 3.1 (in terms of the core language and functionality) was a > solid release. > > Any reasonable expectation about Python 3 adoption predicted that it > would take years, and would include going through a phase of difficulty > and disappointment... Declaring something to be a turd doesn't change the fact that it's a turd. I have a feeling that most people outside this list would have much rather avoided the difficulty and disappointment altogether. Let's be honest here; 3.X was released to the community in part as an extended beta. That's not a problem, unless you drop the word "beta". And if you're still not buying that, imagine the sort of response you'd get if you tried to sell software that billed itself as "experimental", and promised a phase of "disappointment". Why would you expect the Python world to react any differently? > Whilst I agree that there are plenty of issues to workon, and I don't > underestimate the difficulty of some of them, I think "half-baked" is > very much overblown. Whilst you have a lot to say about how much of a > problem this is I don't understand what you are suggesting be *done*? I agree that 3.X isn't all bad, and I very much hope it succeeds. And no, I have no answers; I'm just reporting the perception from downwind. So here it is: The prevailing view is that 3.X developers hoisted things on users that they did not fully work through themselves. Unicode is prime among these: for all the talk here about how 2.X was broken in this regard, the implications of the 3.X string solution remain to be fully resolved in the 3.X standard library to this day. What is a common Python user to make of that? --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz)

18 35

red buildbots on 2.7
by Bill Janssen 23 Jun '10

23 Jun '10

Considering that we've just released 2.7rc2, there are an awful lot of red buildbots for 2.7. In fact, I don't remember having seen a green buildbot for OS X and 2.7. Shouldn't these be fixed? On OS X Leopard, I'm seeing failures in test_py3kwarn, test_urllib2_localnet, test_uuid. On OS X Tiger, I'm seeing failures in test_pep277, test_py3kwarn, test_ttk_guionly, and test_urllib2_localnet. We don't have a buildbot running Snow Leopard, apparently. Bill

14 42

UserDict in 2.7
by Raymond Hettinger 23 Jun '10

23 Jun '10

There's an entry in whatsnew for 2.7 to the effect of "The UserDict class is now a new-style class". I had thought there was a conscious decision to not change any existing classes from old-style to new-style. IIRC, Martin had championed this idea and had rejected all of proposals to make existing classes inherit from object. Raymond

6 10

Re: [Python-Dev] email package status in 3.X
by P.J. Eby 23 Jun '10

23 Jun '10

At 10:20 PM 6/21/2010 +1000, Nick Coghlan wrote: >For the idea of avoiding excess copying of bytes through multiple >encoding/decoding calls... isn't that meant to be handled at an >architectural level (i.e. decode once on the way in, encode once on >the way out)? Optimising the single-byte codec case by minimising data >copying (possibly through creative use of PEP 3118) may be something >that we want to look at eventually, but it strikes me as something of >a premature optimisation at this point in time (i.e. the old adage >"first get it working, then get it working fast"). The issue is, I'd like to have an idempotent incantation that I can use to make the inputs and outputs to stdlib functions behave in a type-safe manner with respect to bytes, in cases where bytes are really what I want operated on. Note too that this is an argument for symmetry in wrapping the inputs and outputs, so that the code doesn't have to "know" what it's dealing with! After all, right now, if a stdlib function might return bytes or unicode depending on runtime conditions, I can't even hardcode an .encode() call -- it would fail if the return type is a bytes. This basically goes against the "tell, don't ask" pattern, and the Pythonically idempotent approach. That is, Python builtins normally return you back the same thing if it's already what you want - int(someInt)-> someInt, iter(someIter)->someIter, etc. Since this incantation may need to be used often, and in places that are not known to me in advance, I would like it to not impose new overhead in unexpected places. (i.e., the usual argument brought against making changes to the 'list' type that would change certain operations from O(1) to O(log something)). It's more about predictability, and having One *Obvious* Way To Do It, as opposed to "several ways, which you need to think carefully about and restructure your entire architecture around if necessary". One obvious way means I can focus on the mechanical effort of porting *first*, without having to think. So, the performance issue isn't really about performance *per se*, so much as about the "mental UI" of the language. You could just as easily lie and tell me that your bstr implementation is O(1), and I would probably be happy and never notice, because the issue was never really about performance as such, but about having to *think* about it. (i.e., breaking flow.) Really, the entire issue can presumably be dealt with by some series of incantations - it's just code after all. But having to sit and think about *every* situation where I'm dealing with bytes/unicode distinctions seems like a torture compared to being able to say, "okay, so when dealing with this sort of API and this sort of data, this is the One Obvious Way to do the conversions." It's One Obvious Way that I want, but some people seem to be arguing that the One Obvious Way is to Think Carefully About It Every Time -- and that seems to violate the "Obvious" part, IMO. ;-)

9 17