Mailman 3 August 2002 - Python-Dev

mimetypes patch #554192
by Walter Dörwald Sept. 4, 2002

Sept. 4, 2002

Patch http://www.python.org/sf/554192 adds a function to mimetypes.py that returns all known extensions for a mimetype, e.g. >>> import mimetypes >>> mimetypes.guess_all_extensions("image/jpeg") ['.jpg', '.jpe', '.jpeg'] Martin v. Loewis and I were discussing whether it would make sense to make the helper method add_type (which is used for adding a mapping between one type and one extension) visible on the module level. Any comments? Bye, Walter Dörwald

3 6

PyString_DecodeEscape and PEP293
by Walter Dörwald Sept. 3, 2002

Sept. 3, 2002

A recent checkin added a function PyString_DecodeEscape() to stringobject.c. To make this function PEP293 compatible it would need access to unicode_decode_call_errorhandler which is defined static in unicodeobject.c. Does PyString_DecodeEscape() really need an errors argument? If yes, we could either move it to unicodeobject.c or make unicode_decode_call_errorhandler externally visible. Another problem that I noticed is that string-escape can't be used for encoding Unicode objects: >>> u"\u0100".encode("string-escape") Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: escape_encode() argument 1 must be str, not unicode Bye, Walter Dörwald

2 4

HAVE_CONFIG_H
by Guido van Rossum Sept. 3, 2002

Sept. 3, 2002

I see no references to HAVE_CONFIG_H in the source code (except one #undef in readline.c), yet we #define it on the command line. Is that still necessary? --Guido van Rossum (home page: http://www.python.org/~guido/)

6 16

To commit or not to commit
by Walter Dörwald Sept. 2, 2002

Sept. 2, 2002

I'm ready to commit the PEP293 implementation. I've added LaTeX documentation in libcodecs.tex and libexcs.tex. There are only a few minor open issues (reflecting exception attribute modifications in args, PyString_DecodeEscape), but I guess we'll fix/document those in time. Any objections against committing the patch? Bye, Walter Dörwald

5 9

Re: [Python-Dev] PEP 277 (unicode filenames): please review
by Matthias Urlichs Sept. 1, 2002

Sept. 1, 2002

Martin: > Indeed, that would be consistent. I deliberately want to leave this > out of PEP 277. On Unix, things are not that clear - as Jack points > out, readlink() and getcwd() also need consideration. > Linux and MacOSX use UTF-8 and should probably be treated as such, i.e. I want to open("äöü"), not open("äöü".encode("utf-8")). One interesting tidbit is that MacOSX requires Unicode filenames to be in NFD. I don't know whether anybody agreed on a standard normal form for Linux. > In this terrain, Windows has the cleaner API (they consider file names > as character strings, not as byte strings), so doing the right thing > is easier. > Byte strings are perfectly OK if they have a common encoding (meaning UTF-8, in some accepted normal form). Character strings are bad if their interpretation, or indeed their usability, changes with the presense of some random environment variable / registry entry / whatever. Under these constraints, calling it a character string vs. a byte string, and/or using it as such, is a matter of programmers' convenience. -- Matthias Urlichs

2 1

Re: [Python-Dev] tiny optimization in ceval mainloop
by Michael Hudson Sept. 1, 2002

Sept. 1, 2002

"Steve Holden" <sholden(a)holdenweb.com> writes: > > A bunch of 0.5% improvements add up. If there's not much cost in > > complexity, why not go for it? > > > > Yeah, right, we just need 200 of them and we're laughing. Computation in > infinitesimal time. Multiply up doesn't have the same ring to it, does it? Cheers, M. -- I don't have any special knowledge of all this. In fact, I made all the above up, in the hope that it corresponds to reality. -- Mark Carroll, ucam.chat

2 1

Re: RE: RE: [Python-Dev] The first trustworthy <wink> GBayes results
by Paul Graham Aug. 31, 2002

Aug. 31, 2002

I see, if you count the punctuation as part of the token, you end up with undersized-corpus effects. Esp if you are case-sensitive too. If I were you I'd map your input down into a narrower set of tokens, or you'll get too many errors. --pg --Tim Peters wrote: > [Paul Graham] > > Don't count words multiple times, and you'll probably > > get fewer false positives. That's the main reason I > > don't do it-- because it magnifies the effect of some > > random word like water happening to have a big spam > > probability. > > Yes, that makes sense, but I'm trained not to think <wink>. Experiment will > decide it (although I *expect* it's a good change, and counting multiple > occurrences was obviously a factor in several of the rare false positives). > If spam really is different, it should be different in several distinct > ways. > > > (Incidentally, why so high? In my db it's only 0.3930784.) --pg > > I expect it's because this tokenizer *only* split on whitespace. > Punctuation was left intact. So, e.g., on the Python discussion list stuff > like > > The new approach blows it out of the water: > and > This is very deep water; > and > Then you'll take to Python like a duck takes to water! > > are counted as "water:" and "water;" and "water!", not as "water". > > The spam corpus is chock full o' "water", though: > > + Porn sites advertising water sports. > + Assorted bottled water pitches. > + Assorted "oxygenated water" pitches. > + Claims of environmental friendliness explicated via stuff like > "no harmful chlorine to pollute the water or air!". > + Pitches for weight-loss gimmicks emphasizing that you'll really > loss fat, not just reduce water retention. > + Pitches for weight-loss gimmicks empphasizing that you'll reduce > water retention as well as lose fat. > + One repeated bizarre analogy for how a breast enlargement cream > works in the way "a sponge absorbs water". > + This revolutionary new flat garden hose will really cut your water > bills. > + Ditto this miracle new laundry tablet lets you use a fraction of > the water needed by old-fashioned detergents. > + Survivalist pitches often mention water in the same sentence as > air and medical care. > > I got tired then <wink>. >

2 2

tiny optimization in ceval mainloop
by Jeremy Hylton Aug. 31, 2002

Aug. 31, 2002

I noticed that one frequently executed line in the mainloop is testing whether either the ticker has dropped to 0 or if there are things_to_do. Would it be kosher to just drop the ticker to 0 whenever things_to_do is set to true? Then you'd only need to check the ticker each time through. Jeremy Index: ceval.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v retrieving revision 2.332 diff -c -c -r2.332 ceval.c *** ceval.c 23 Aug 2002 14:11:35 -0000 2.332 --- ceval.c 29 Aug 2002 23:19:18 -0000 *************** *** 378,383 **** --- 378,384 ---- int Py_AddPendingCall(int (*func)(void *), void *arg) { + PyThreadState *tstate; static int busy = 0; int i, j; /* XXX Begin critical section */ *************** *** 395,400 **** --- 396,404 ---- pendingcalls[i].func = func; pendingcalls[i].arg = arg; pendinglast = j; + + tstate = PyThreadState_GET(); + tstate->ticker = 0; things_to_do = 1; /* Signal main loop */ busy = 0; /* XXX End critical section */ *************** *** 669,675 **** async I/O handler); see Py_AddPendingCall() and Py_MakePendingCalls() above. */ ! if (things_to_do || --tstate->ticker < 0) { tstate->ticker = tstate->interp->checkinterval; if (things_to_do) { if (Py_MakePendingCalls() < 0) { --- 673,679 ---- async I/O handler); see Py_AddPendingCall() and Py_MakePendingCalls() above. */ ! if (--tstate->ticker < 0) { tstate->ticker = tstate->interp->checkinterval; if (things_to_do) { if (Py_MakePendingCalls() < 0) {

7 21

alternate reiter proposal
by John Williams Aug. 30, 2002

Aug. 30, 2002

Hi, this is my first post, so go easy on me! :-) I got this idea from the "cogen" discussion, seeing how the lack of a reliable re-iterability test makes it hard to write lazy multi-pass algorithms. Rather than (1) assuming iterators are re-iterable, (2) "forcing" iterators to be re-iterable by eagerly converting them to lists, or (3) trying to heuristically guess whether an iterator is re-iterable, why not combine the best of (1) and (2) while avoiding (3) entirely? This class will lazily convert an interator to a list on the first pass and then iterate over the saved list on all subsequent passes. class reiter(object): def __init__(self, iterable): self.iterator = iter(iterable) self.cache = [] def __iter__(self): if self.iterator is None: return iter(self.cache) else: return self def next(self): try: element = self.iterator.next() self.cache.append(element) return element except StopIteration: self.iterator = None raise __________________________________________________ Do You Yahoo!? Yahoo! Finance - Get real-time stock quotes http://finance.yahoo.com

2 2

Re: [Python-Dev] tiny optimization in ceval mainloop
by Michael Hudson Aug. 30, 2002

Aug. 30, 2002

Jeremy Hylton <jeremy(a)alum.mit.edu> writes: > The difference I saw with only the ticker check in ceval was only > about 1% for pystone. Python was always faster with the change, but > never by much. A bunch of 0.5% improvements add up. If there's not much cost in complexity, why not go for it? Cheers, M. -- 6. Symmetry is a complexity-reducing concept (co-routines include subroutines); seek it everywhere. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

4 3