Mailman 3 February 2005 - Python-Dev

PEP 309
by Raymond Hettinger 28 Feb '05

28 Feb '05

Paul Moore wrote: > Personally, I'd rather see partial as it stands, with its current > limitations, included. The alternative seems to be a potentially long > discussion, petering out without conclusion, and the whole thing > missing Python 2.5. (I know that's a long way off, but this already > happened with 2.4...) -0 My preference is that it not go in as-is. It is better to teach how to write a closure than to introduce a new construct that has its own problems and doesn't provide a real improvement over what we have now. Despite my enthusiasm for functional programming and the ideas in PEP, I find the PFA implementation annoying. I had tried out the implementation that was being pushed for Py2.4 and found it wanting. Hopefully, it has improved since then. Here are a few thoughts based on trying to apply it to my existing code. * The PFA implementation proposed for Py2.4 ran slower than an equivalent closure. If the latest implementation offers better performance, then that may be a reason for having it around. * Having PFA only handle the first argument was a PITA with Python. In Haskell and ML, function signatures seems to have been designed with argument ordering better suited for left curries. In contrast, Python functions seem to have adverse argument ordering where the variables you want to freeze appear toward the right. This is subjective and may just reflect the way I was aspiring to use "partial" to freeze options and flags rather than the first argument of a binary operator. Still, I found closures to be more flexible in that they could handle any argument pattern and could freeze more than one variable or keyword at a time. * The instance method limitation never came up for me. However, it bites to have a tool working in a way that doesn't match your mental model. We have to document the limitations, keep them in mind while programming, and hope to remember them as possible causes if bugs ever arise. It would be great if these limitations could get ironed out. * Using the word "partial" instead of "lambda" traded one bit of unreadability for another. The "lambda" form was often better because it didn't abstract away its mechanism and because it supported more general expressions. I found that desk-checking code was harder because I had to mentally undo the abstraction to check that the argument signature was correct. * It is not clear that the proposed implementation achieves one of the principal benefits laid out in the PEP: "I agree that lambda is usually good enough, just not always. And I want the possibility of useful introspection and subclassing." If we get a better implementation, it would be nice if the PEP were updated with better examples. The TkInter example is weak because we often want to set multiple defaults at the same time (foreground, background, textsize, etc) and often those values are config options rather than hardwired constants. Raymond

7 16

Quick access to Python bug reports in Thunderbird
by "Martin v. Löwis" 27 Feb '05

27 Feb '05

I find that I frequently need to open a bug report whose bug number is in a python-dev email I read with Mozilla Thunderbird. I now tried to automate things a bit more, and found the excellent DictionarySearch plugin: http://dictionarysearch.mozdev.org/index.html To install, follow these steps: 1. Download the plugin (for Thunderbird, make sure to download to disk) http://dictionarysearch.mozdev.org/download.php/http://downloads.mozdev.org… 2. In Thunderbird, open Tools/Extensions/Install, select the file downloaded 3. Open Tools/Extensions again, double-click DictionarySearch. 4. In "Dictionary 1", fill in Text: Python Bug $ Access Key: y (as P will conflict with Print) URL: http://www.python.org/sf/$ To use the plugin, select a bug report number in a message, select the context menu, and type y (or click the Python Bug menu entry). Regards, Martin P.S. In case you wonder why this plugin is DictionarySearch: it is meant to lookup words in an email in a dictionary, e.g. Wikipedia.

3 4

textwrap.py wordsep_re
by Karl Chen 27 Feb '05

27 Feb '05

Hi, textwrap.fill() is awesome. Except when the string to wrap contains dates -- which I would like not to be filled. In general I think wordsep_re can be smarter about what it decides are hyphenated words. For example, this code: print textwrap.fill('aaaaaaaaaa 2005-02-21', 18) produces: aaaaaaaaaa 2005- 02-21 A slightly tweaked wordsep_re: textwrap.TextWrapper.wordsep_re =\ re.compile(r'(\s+|' # any whitespace r'[^\s\w]*\w+[a-zA-Z]-(?=[a-zA-Z]\w+)|' # hyphenated words r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash print textwrap.fill('aaaaaaaaaa 2005-02-21', 18) behaves better: aaaaaaaaaa 2005-02-21 What do you think about changing the default wordsep_re? -- Karl 2005-02-21 03:32

2 3

Yet another patch review batch + request
by Alan McIntyre 27 Feb '05

27 Feb '05

I've taken a look at the following patches and made comments on the associated tracker items: patch [977553] Speed up EnumKey call Looks like a good idea, but needs some cleanup and at least one test case. I think I know enough about the Windows registry functions to finish this one up if the original author doesn't have time. patch [1075147] Flush stdout/stderr if closed (fix for bug 1074011) Looking at current sysmodule.c shows that this change (plus some error checking) has already been made, and the referenced bug has already been closed. patch [1104111] setup.py --help and --help-commands altered. Simple, helpful changes; got one 1+ and no negative comments over on distutils-sig: http://mail.python.org/pipermail/distutils-sig/2005-January/004378.html patch [1107973] tarfile.ExFileObject iterators Doesn't break anything and makes ExFileObject more file-like, so I'm for it. Passes regression tests. patch [1110248] patch for gzip.GzipFile.flush() related to bug 1110242 Seems reasonable & is simple. Passes regression tests. ---------------------------------------------------------------- I would appreciate it if somebody could have a look at my patch: [1121142] ZipFile.open - read-only file-like obj for files in archive Another patch related to mine: patch [992750] zipfile and big zipped file This is the patch that I looked at before I submitted patch 1121142; if my patch is accepted (or rejected, I guess :-), this one should probably be closed since they implement the same idea. Thanks, Alan

1 0

Patch review & request
by Michiel Jan Laurens de Hoon 27 Feb '05

27 Feb '05

This is my last patch review in a series of five; next time I'll send them in a bunch of five as requested. I reviewed the following patches: [ 684500 ] extending readline functionality [ 723201 ] PyArg_ParseTuple problem with 'L' format [ 981773 ] crach link c++ extension by mingw [ 985713 ] bug skipping optional keyword arguments of type "w#" [ 1093585 ] sanity check for readline remove/replace I'd like to ask your attention for patch #1121234, which is a bug fix for a reference count problem in Tkinter. Nothing complicated, don't worry. Thanks! --Michiel. Patch [ 985713 ] bug skipping optional keyword arguments of type "w#" --------------------------------------------------------------------- The convertsimple function in Python/getargs.c converts arguments for PyArg_Parse, PyArg_ParseTuple etc. The function skipitem takes care of skipping optional keyword arguments. Some of the valid formats are included in convertsimple but missing in skipitem. The patch shows the example of the "w#" format. If the argument format is "s|w#H" with corresponding keywords "string", "buffer", "ushort", you can skip the "ushort" keyword but not the "buffer" keyword. So this works: >>> my_dummy_func("text", buffer=my_buffer) but this doesn't: >>> my_dummy_func("text", ushort=my_ushort) It results in the error message Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: argument 2 impossible<bad format char> The patch fixes this by adding the appropriate formats to the skipitem function. The patch was written for the "w#" format; other formats that are missing are listed in the patch description. It seems that the missing formats in the skipitem function are simply due to an oversight. Running the test suite revealed no problems with this patch. So I think it can be applied. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

1 0

pystone
by stelios xanthakis 27 Feb '05

27 Feb '05

Hi, It seems that in pystone, Proc1 the else branch is never reached. Is this OK ?? Stelios

1 0

UserString
by Raymond Hettinger 25 Feb '05

25 Feb '05

I noticed that UserString objects have methods that do not accept other UserString objects as arguments: >>> from UserString import UserString >>> UserString('slartibartfast').count(UserString('a')) Traceback (most recent call last): File "<pyshell#2>", line 1, in -toplevel- UserString('slartibartfast').count(UserString('a')) File "C:\PY24\lib\UserString.py", line 66, in count return self.data.count(sub, start, end) TypeError: expected a character buffer object >>> UserString('abc') in UserString('abcde') Traceback (most recent call last): File "<pyshell#1>", line 1, in -toplevel- UserString('abc') in UserString('abcde') File "C:\PY24\lib\UserString.py", line 35, in __contains__ return char in self.data TypeError: 'in <string>' requires string as left operand This sort of thing is easy to test for and easy to fix. The question is whether we care about updating this module anymore or is it a relic. Also, is the use case one that we care about. AFAICT, this has never come up before. Raymond

8 15

python-dev Summary for 2005-01-16 through 2005-01-31 [draft]
by Brett C. 25 Feb '05

25 Feb '05

A month late. Glad I don't get paid for this or I probably would have been fired by now. =) For this summary draft I am bringing back the header this one time in hopes of getting some proof-reading of it. I did a major restructuring with some accompanying rewrites to make it easier to navigate. Details are in the Summary Announcements section. And you can ignore XXX and any %s points since this is just a direct paste of the template I use for generating the Summaries. I am hoping to get this summary out either Saturday or Sunday since I am anxious to try out AMK's new RSS feed for the Summaries (see the Intro section or Summary Announcements for the URI). -------------------------------- python-dev Summary for %(start_ISO_date)s through %(end_ISO_date)s ++++++++++++++++++++++++++++++++++++++++++++++++++++ .. contents:: ===== Intro ===== This is a summary of traffic on the `python-dev mailing list`_ from %(start_traditional_date)s through %(end_traditional_date)s. It is intended to inform the wider Python community of on-going developments on the list on a semi-monthly basis. An archive_ of previous summaries is available online. An `RSS feed`_ of the titles of the summaries is available. You can also watch comp.lang.python or comp.lang.python.announce for new summaries (or through their email gateways of python-list or python-announce, respectively, as found at http://mail.python.org). This is the XXX summary written by Brett Cannon (XXX). To contact me, please send email to brett at python.org. Do *not* post to comp.lang.python if you wish to reach me. The `Python Software Foundation`_ is the non-profit organization that holds the intellectual property for Python. It also tries to forward the development and use of Python. If you find the python-dev Summary helpful please consider making a donation. You can make a donation at http://python.org/psf/donations.html . Every penny helps so even a small donation with a credit card, check, or by PayPal helps. If you are looking for a way to expand your knowledge of Python's development and inner-workings, consider writing the python-dev Summaries yourself! I am willing to hand over the reins to someone who is willing to do a comparable or better job of writing the Summaries. If you are interested, please email me at brett at python.org. -------------------- Commenting on Topics -------------------- To comment on anything mentioned here, just post to `comp.lang.python`_ (or email python-list(a)python.org which is a gateway to the newsgroup) with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join `python-dev`_! ------------------------- How to Read the Summaries ------------------------- The in-development version of the documentation for Python can be found at http://www.python.org/dev/doc/devel/ and should be used when looking up any documentation on new code; otherwise use the current documentation as found at http://docs.python.org/ . PEPs (Python Enhancement Proposals) are located at http://www.python.org/peps/ . To view files in the Python CVS online, go to http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . Reported bugs and suggested patches can be found at the SourceForge_ project page. Please note that this summary is written using reStructuredText_. Any unfamiliar punctuation is probably markup for reST_ (otherwise it is probably regular expression syntax or a typo =); you can safely ignore it. I do suggest learning reST, though; it's simple and is accepted for `PEP markup`_ and can be turned into many different formats like HTML and LaTeX. Unfortunately, even though reSt is standardized, the wonders of programs that like to reformat text do not allow me to guarantee you will be able to run the text version of this summary through Docutils_ as-is unless it is from the `original text file`_. .. _python-dev: http://www.python.org/dev/ .. _SourceForge: http://sourceforge.net/tracker/?group_id=5470 .. _python-dev mailing list: http://mail.python.org/mailman/listinfo/python-dev .. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python .. _PEP Markup: http://www.python.org/peps/pep-0012.html .. _Docutils: http://docutils.sf.net/ .. _reST: .. _reStructuredText: http://docutils.sf.net/rst.html .. _PSF: .. _Python Software Foundation: http://python.org/psf/ .. _last summary: http://www.python.org/dev/summary/%(html_prev_summary)s .. _original text file: http://www.python.org/dev/summary/%(ht_name)s .. _archive: http://www.python.org/dev/summary/ .. _RSS feed: http://www.python.org/dev/summary/channews.rdf ===================== Summary Announcements ===================== ----------------------------------------- School sure likes to destroy my free time ----------------------------------------- A month late, that much closer to having this hectic quarter being over. Sorry for being so delinquent with this summary but school has kept me busy and obviously the Real World has to take precedence over volunteer work. Now if I could only get paid for doing this... =) And if you hate the summaries being late, you could do it yourself. This is not meant to be a flippant comment! I am always willing to hand over development of the summaries to anyone who is willing to do a comparable job. If you are interested feel free to email me. I have now made this a permanent offer in the header in case someone comes along later and decides they want to do this. ---------------------- RSS feed now available ---------------------- Thanks entirely to one of my Summaries author predecessors, A.M. Kuchling, the python-dev Summaries are available as an `RSS feed`_. The feed contains the titles of every summary and so will be updated with the newest summaries as soon as they are posted online. ---------- New format ---------- I have done a thorough restructuring of the header and the Summary Announcements section. The purpose of this is to make finding information in the header much easier. It also keeps consistency by sectioning off everything as in the Summary section. The other reason is for the ``contents`` directive in reST_. This will provide a more thorough table of contents for the web version of the summary at the very top of the summaries. This will allow people to jump directly to the section of the summary they care about the most. Obviously this perk only exists in the HTML version of the Summaries. But don't feel left out, text readers! I am also playing with the amount of whitespace between sections. As of right now I am leaning towards a single whitespace between section headers and text and two newlines between the end of text and a new section. I think it reads more easily and seems a little less forced. Obviously it lengthens the overall summary, but I think it is worth it for the ease of reading. Then again I could be totally wrong about all of this and manage to alienate every person who reads the summaries regularly. =) ======= Summary ======= --------------------- Python 2.3.5 released --------------------- Consider how late this summary is I bet you already knew Python 2.3.5 was already out the door. =) With Python 2.4 out in the world this means there is a very high probability 2.3.6 will never exist and this marks the end of the 2.3 branch. Contributing threads: - `2.3.5 delayed til next week <>`__ - `2.3 BRANCH FREEZE imminent! <>`__ - `RELEASED Python 2.3.5, release candidate 1 <>`__ ------------------------------------------------------ Making magic type conversion methods act like __str__ ------------------------------------------------------ Walter Dörwald discovered that when you subclass 'unicode' and call unicode() on an instance of the subclass it will not call the implementation of __unicode__ of the subclass but instead will call unicode.__unicode__ . When in the same scenario with strings, though, str() calls the subclass' __str__ method. Turns out 'int' and 'float' act like 'unicode' while 'complex' acts like 'str'. So who is right? Docs say 'str' is wrong, but this is mainly an artifact of pre-2.2 inability to subclass types. Turns out 'str' is acting properly. `Patch #1109424`_ implements the proper semantics and will eventually go in for 2.5 (won't touch 2.4 since it is a semantic change). .. Patch# 1109424: http://www.python.org/sf/1109424 Contributing threads: - `__str__ vs. __unicode__ <>`__ --------------------------------------------- Speeding up function calls to C API functions --------------------------------------------- Neal Norwitz posted the patch found at http://www.python.org/sf/1107887 to help with function calls to C code. The idea is to expand the family of values used in PyMethodDef.ml_flags for argument types to include specifying the number of minimum and maximum number of arguments. This can provide a speedup by allowing the eval loop to unpack everything in the C stack and skip packing arguments in a tuple. But not everyone was sure it was worth the extra need to specify all of this for functions. Regardless of that and any other objections this would be more of a Python 3000 thing. Which also led to a quick shift in topic to how Python 3.0 will be developed. Guido said it would be piece-meal. Read http://joelonsoftware.com/articles/fog0000000069.html for why. Contributing threads: - `Speed up function calls <>`__ - `Moving towards Python 3.0 <>`__ ------------------------------------------------------------ How to handle memory allocation with the presence of threads ------------------------------------------------------------ Evan Jones has been working on a patch to allow the garbage collector free up memory of small objects. This led him to ask questions in terms of memory usage in the face of threading at the C level. While the GIL usually needs to be held for any operation that touches Python code, he was not sure if this held for the memory API. Tim Peters clarified all of this by pointing out the documentation in the C API manual about the GIL. In a nutshell the memory API is not exempt from needing to hold the GIL, so hold it. It was also pointed out there was a bunch of code to allow people to mix usage of PyMem_* functions with PyObject_* functions. That was purely done for backwards-compatibility back in the day. Mixing these two APIs for memory is very bad. Don't do it! Contributing threads: - `Improving the Python Memory Allocator <>`__ - `Python Interpreter Thread Safety? <>`__ -------------------------- Slicing iterators rejected -------------------------- Nick Coghlan proposed allowing iterators to be sliced liked other sequence types. That way something like ``enumerate("ABCD")[1:]`` would work. But Guido rejected it. With itertools.islice existence it does not provide new functionality. Plus "Iterators are for single sequential access" according to Guido, and thus should not be confused with sequences. Contributing threads: - `Allowing slicing of iterators <>`__ =============== Skipped Threads =============== - redux: fractional seconds in strptime - how to test behavior wrt an extension type? - Strange segfault in Python threads and linux kernel 2.6 - Unix line endings required for PyRun* breaking embedded Python - Zen of Python - PyCon: The Spam Continues ;-) - Patch review: [ 1094542 ] add Bunch type to collections module

1 0

__str__ vs. __unicode__
by Walter Dörwald 25 Feb '05

25 Feb '05

__str__ and __unicode__ seem to behave differently. A __str__ overwrite in a str subclass is used when calling str(), a __unicode__ overwrite in a unicode subclass is *not* used when calling unicode(): ------------------------------- class str2(str): def __str__(self): return "foo" x = str2("bar") print str(x) class unicode2(unicode): def __unicode__(self): return u"foo" x = unicode2(u"bar") print unicode(x) ------------------------------- This outputs: foo bar IMHO this should be fixed so that __unicode__() is used in the second case too. Bye, Walter Dörwald

7 18

Comment regarding PEP 328
by Josiah Carlson 25 Feb '05

25 Feb '05

In a recent discussion in a SF patch, I noticed that PEP 328* only seems to support relative imports within packages, while bare import statements use the entirety of sys.path, not solving the shadowing of standard library module names. I have certainly forgotten bits of discussion from last spring, but I would offer that Python could offer standard library shadowing protection through the use of an extended PEP 328 semantic. More specifically; after a 'from __future__ import absolute_import' statement, any import in the module performing "import foo" will only check for foo in the standard library, and the use of the leading period, "from . import foo", the will signify relative to the current path. ** The lack of a 'from __future__ import absolute_import' statement in a module will not change the import semantic of that module. This allows current code to continue to work, and for those who want to choose names which shadow the standard library modules, a way of dealing with their choices. Further, in the case of PEP 328, the package relative imports were to become the default in 2.6 (with deprecation in 2.5, availability in 2.4), but with the lack of an implementation, perhaps those numbers should be incremented. If the behavior I describe is desireable, it would subsume PEP 328, and perhaps should also become the default behavior at some point in time (perhaps in the same adjusted timeline as PEP 328). Alternatively, PEP 328 could be implemented as-is, and a second future import could be defined which offers this functionality, being permanently optional (or on a different timeline) via the future import. Essentially, it would ignore the empty path "" in sys.path when the functionality has been enabled via the proper future import in the current module. - Josiah * PEP 328 first describes the use of parenthesis in import statements so that long import listings do not require backslash-escaping of newlines. It then describes a semantic for not checking sys.path when performing an import, as well as allowing parent, cousin, uncle, etc., imports via additional leading periods. "from . import foo" for sibling imports, "from .. import foo" for parent imports, etc. ** I describe the semantic as being a per-module option, as this is the only backwards-compatible mechanism in the near future (Python 2.5). An import implementation would merely check the existance of the proper name binding -> object pair in the importer's global namespace. The standard library would need to be modified if or when current behavior is deprecated (this would extend the modules needing to be modified due to PEP 328).

4 8