Mailman 3 June 2002 - Python-Dev

Python strptime
by Brett Cannon 21 Jun '02

21 Jun '02

I have implemented strptime in pure Python (SF patch #474274) as a drop-in replacement for the time module's version, but there is the issue of the time module being a C extension. Any chance of getting a Python module stub for time (assuming this patch is good enough to be accepted)? There is also obviously the option of doing something like a time2, but is there enough other time-manipulating Python code out there to warrant another module? It could be used for housing naivetime and any other code that does not directly stem from some ANSI C function. -Brett C.

8 25

Weird problem with exceptions raised in extension module
by Greg Ewing 21 Jun '02

21 Jun '02

I'm getting strange behaviour when raising an exception in a extension module generated by Pyrex. The extension module does the equivalent of def foo(): raise TypeError("Test-Exception") If I invoke it with the following Python code: try: mymodule.foo() except IOError: print "blarg" the following happens: Traceback (most recent call last): File "<stdin>", line 3, in ? SystemError: 'finally' pops bad exception This only happens when the try-except catches something *other* than the exception being raised. If the exception being raised is caught, or no exception catching is done, the exception is handled properly. Also, it only happens when an *intance* is used as the exception object. If I do this instead: raise TypeError, "Test-Exception" the problem doesn't occur. The relevant piece of C code generated by Pyrex is as follows. Can anyone see if I'm doing anything wrong? (I'm aware that there's a missing Py_DECREF, but it shouldn't be causing this sort of thing.) The Python version I'm using is 2.2. __pyx_1 = __Pyx_GetName(__pyx_b, "TypeError"); if (!__pyx_1) goto __pyx_L1; __pyx_2 = PyString_FromString(__pyx_k1); if (!__pyx_2) goto __pyx_L1; __pyx_3 = PyTuple_New(1); if (!__pyx_3) goto __pyx_L1; PyTuple_SET_ITEM(__pyx_3, 0, __pyx_2); __pyx_2 = 0; __pyx_4 = PyObject_CallObject(__pyx_1, __pyx_3); if (!__pyx_4) goto __pyx_L1; Py_DECREF(__pyx_3); __pyx_3 = 0; PyErr_SetNone(__pyx_4); Py_DECREF(__pyx_4); __pyx_4 = 0; goto __pyx_L1; /*...*/ __pyx_L1:; Py_XDECREF(__pyx_1); Py_XDECREF(__pyx_2); Py_XDECREF(__pyx_3); Py_XDECREF(__pyx_4); __pyx_r = 0; __pyx_L0:; return __pyx_r; Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg(a)cosc.canterbury.ac.nz +--------------------------------------+

2 3

Vacation
by Guido van Rossum 21 Jun '02

21 Jun '02

I should mention that I'm going on vacation. I'm not going to run a vacation program -- I've had poor results with those in the past, and I expect to be sporadically checking email. But I don't expect to be responding to python-dev mail until I'm back. I'm leaving later today and plan to return Monday July 8, back at work the 9th. Of course, I'll see a bunch of you all at EuroPython! I'm looking forward to it -- it sounds like it's gonna be a great conference! --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

RE: [Python-Dev] PEP 292, Simpler String Substitutions
by Delaney, Timothy 21 Jun '02

21 Jun '02

> From: barry(a)zope.com [mailto:barry@zope.com] > > >>>>> "MS" == Martin Sjögren <martin(a)strakt.com> writes: > > MS> What's the advantage of using ${name} and ${country} instead? > > There's a lot of empirical evidence that %(name)s is quite error > prone. Perhaps an unadorned %(name) should default to %(name)s? Tim Delaney

10 13

unifying read method semantics
by Piers Lauder 21 Jun '02

21 Jun '02

A user of imaplib's IMAP4_SSL class has complained that the "read" and "write" methods don't behave correctly, sometimes omitting to handle all the requested data. This is a bug - I should have noticed this common misconception when installing the submitted sub-class into imaplib. However, this is a common enough gotcha for python programmers that I wondered if it is worthwhile fixing it once and for all. Ie: mandate that core python modules providing read/write methods guarantee that all the data is sent by write() (or exception), and all the requested data is read() (or exception). The last time this came up the socketmodule code got a "sendall" method. However, this doesn't exist in the ssl portion of socketmodule.c. And while I'm on the topic - please could we always support "readline" (or "makefile") methods in C modules? Surely the following code now necessary in imaplib must make CPU-time conscious programmers wince: def readline(self): """Read line from remote.""" line = "" while 1: char = self.sslobj.read(1) line += char if char == "\n": return line :-) Piers Lauder.

2 1

RE: [Patches] [ python-Patches-566100 ] Rationalize DL_IMPORT and DL_EXPORT
by Tim Peters 21 Jun '02

21 Jun '02

In case anyone wants prettier Python source code, check out Mark Hammond's patch. It deserves more consideration than 50 identical ways to spell string interpolation <wink>: http://www.python.org/sf/566100

1 0

h2py
by Gustavo Niemeyer 21 Jun '02

21 Jun '02

Hi everyone! I was thinking about working a little bit in the h2py tool. But first, I'd like to understand what's the current position of its utility. Should I worry about it, or this tool and its generated files, are something to be obsoleted soon? Thanks! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

2 2

Re: [Python-Dev] PEP 292, Simpler String Substitutions
by Fredrik Lundh 20 Jun '02

20 Jun '02

Barry wrote: > def birth(self, name): > country = self.countryOfOrigin['name'] > return '${name} was born in ${country}'.sub() now explain why the above is a vast improvement over: def birth(self, name): country = self.countryOfOrigin['name'] return join(name, ' was born in ', country) (for extra bonus, explain how sub() can be made to execute substantially faster than a join() function) </F>

18 42

FW: PEP 292, Simpler String Substitutions
by Damien Morton 20 Jun '02

20 Jun '02

Youre right. I only threw that out there as a talking point rather than a serious suggestion. I take it you agree with my assertion that putting the format string before the variable would be less error prone? (if it didnt destroy the current usage). Given that the $ notation is all-new, perhaps prefixing with the format string should be considered as in: "$4.2f{height}" In fact, if we are going to revisit format strings why not ditch the format character and keep the numeric specifier only. Determine the format character by the type of the variable. For x = "hello", "$4.2{x}" == "$4s{x}" -> "hell" For x = 3.7865, "$4.2{x}" == "$4.2f{x}" -> "3.78" > -----Original Message----- > From: pinard(a)titan.progiciels-bpi.ca > [mailto:pinard@titan.progiciels-bpi.ca] On Behalf Of François Pinard > Sent: Thursday, 20 June 2002 10:39 > To: Damien Morton > Subject: Re: PEP 292, Simpler String Substitutions > > > [Damien Morton] > > > Why not alter the notation to allow the format specifier to come > > before the name part. "%4.2f(height)" I think would be a whole lot > > less error prone, and would allow for the format specifier > to default > > to "s" where omitted. > > Hello, Damien. > > "%4.2f(height)" already has the meaning of "%4.2f", which is > complete in itself, and then "(height)", which is a constant > string -- you understand what I mean. Altering the notation > as you suggest would undoubtedly break many, many > applications, so we should guess it is not acceptable. > > -- > François Pinard http://www.iro.umontreal.ca/~pinard >

1 0

Re: [Python-Dev] PEP 292, Simpler String Substitutions
by Aahz 20 Jun '02

20 Jun '02

I'm about a pure 0 on this proposal in the aggregate I'm -1 on .sub() as the name; I'd rather it be called .interp() (this mainly due to confusing with the existing re.sub() and str.replace()) I'm +0 on putting this functionality in the text module instead of adding a string method I'm +0 on trying to find a solution that uses % instead of $ (There's Only One Way) I'm -1 on ${name}; there's no reason not to at least use $(name) for consistency -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ Project Vote Smart: http://www.vote-smart.org/

1 0

proposed patch
by David Abrahams 20 Jun '02

20 Jun '02

I suggested: "What about making a public interface to apply_slice() and assign_slice() which I can call in the future? Perhaps PyObject_Get/Set/DelSlice()?" If I submitted such a patch would it be likely to be accepted? I don't want to waste my time on this if it's a bad idea. -Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

1 0

RE: [Python-Dev] Re: PEP 292, Simpler String Substitutions
by Delaney, Timothy 20 Jun '02

20 Jun '02

> From: Patrick K. O'Brien [mailto:pobrien@orbtech.com] > > Definitely. And habit. Since I program mostly in Python I'm used to {} > meaning dictionary and I'm used to typing parens everywhere In that case you should be happy. ${} is using a dictionary as its source ... Actually, for consistency, it should probably be $[] to suggest accessing a dictionary element. But I won't go down that path ;) Tim Delaney

1 0

RE: [Python-Dev] PEP 292, Simpler String Substitutions
by Damien Morton 20 Jun '02

20 Jun '02

LOL. Youre right of course :) `...` is already in python though. Better to generalise and/or extend an already existing construct than to add a new one, I would assert. Youre also right about `...` being less visible than other pairs of delimiters. I think, though, that if modifying the % notation, that the requirement should be for delimiters that are not parentheses, and ones that wouldn't interfere with putting expressions between them. This would allow for eval()ing the content between the delimiters. > -----Original Message----- > From: guido(a)pcp02138704pcs.reston01.va.comcast.net > [mailto:guido@pcp02138704pcs.reston01.va.comcast.net] On > Behalf Of Guido van Rossum > Sent: Thursday, 20 June 2002 00:08 > To: Damien Morton > Subject: Re: [Python-Dev] PEP 292, Simpler String Substitutions > > > > I stand by my position though. Ive been programing for a long time, > > and I have rarely come across the $ notation. Mind you, I > don't work > > in unix very often, and I would hazard a guess that the $ > substitution > > comes mainly from unix. > > So does `...`. :-) > > --Guido van Rossum (home page: http://www.python.org/~guido/) >

2 1

Re: [Python-Dev] PEP 292, Simpler String Substitutions
by Paul Prescod 20 Jun '02

20 Jun '02

"Barry A. Warsaw" wrote: > >... The mapping argument is optional; if it is omitted then the > mapping is taken from the locals and globals of the context in > which the .sub() method is executed. For example: > > def birth(self, name): > country = self.countryOfOrigin['name'] > return '${name} was born in ${country}' > > birth('Guido') You forgot the "implicit .sub"() feature. >... > - We can simply allow the exception (likely a NameError or > KeyError) to propagate. Explicit! > - We can return the original substitution placeholder unchanged. Silently guess??? Overall it isn't bad...it's a little weird to have a method that depends on sys._getframe(1) (or as the say in Tcl-land "upvar"). It may set a bad precedent... Paul Prescod

4 6

RE: [Python-Dev] PEP 292, Simpler String Substitutions
by Delaney, Timothy 20 Jun '02

20 Jun '02

> From: Guido van Rossum [mailto:guido@python.org] > > > > > There's a lot of empirical evidence that %(name)s is quite error > > > > prone. > > > > > > Perhaps an unadorned %(name) should default to %(name)s? > > Ambiguous, hence even more error-prone. Fair enough. I couldn't off the top of my head think of an ambiguous case, but of course there's '%(thing)s'ly-yours' ... PyChecker2 should *definitely* include checking format strings IMO, irrespective of whether $ formatting gets in. But only as a warning (because of the above case). Tim Delaney

3 2

Re: PEP 292, Simpler String Substitutions
by Ka-Ping Yee 20 Jun '02

20 Jun '02

On Tue, 18 Jun 2002, Barry A. Warsaw wrote: > def birth(self, name): > country = self.countryOfOrigin['name'] > return '${name} was born in ${country}' > > birth('Guido') > > returns > > 'Guido was born in the Netherlands' I assume you in fact meant return '${name} was born in ${country}'.sub() for the third line above? > print s.sub({'name': 'Guido', > 'country': 'the Netherlands'}) Have you considered the possibility of accepting keyword arguments instead? They would be slightly more pleasant to write: print s.sub(name='Guido', country='the Netherlands') This is motivated because i imagine relative frequencies of use to be something like this: 1. sub() [most frequent] 2. sub(name=value, ...) [nearly as frequent] 3. sub(dictionary) [least frequent] If you decide to use keyword arguments, you can either allow both keyword arguments and a single dictionary argument, or you can just accept keyword arguments and people can pass in dictionaries using **. -- ?!ng

4 4

RE: [Python-Dev] PEP 292, Simpler String Substitutions
by Delaney, Timothy 20 Jun '02

20 Jun '02

> From: barry(a)zope.com [mailto:barry@zope.com] > > >>>>> "GvR" == Guido van Rossum <guido(a)python.org> writes: > > GvR> That's a matter of validating the template before accepting > GvR> it. > > True, which isn't hard to do. You can write a regexp to extract the > $names and then validate those. In fact, I think this is what newer > versions of xgettext do for Python code (albeit with the %(name)s > syntax). Of course, once you've validated a string, it's almost no extra work to do the interpolation in place (at least in source code size). Tim Delaney

1 0

PEP 8: Lists/tuples
by Aahz 19 Jun '02

19 Jun '02

I don't really want to open a can of worms here, but as I'm finishing up my OSCON slides, I remembered a conversation earlier where Guido said that tuples should be used for heterogeneous items (i.e. lightweight structs) while lists should be used for homogeneous items. Should this preference be enshrined in PEP 8? (I'm -1 myself, but I'd like to know what to tell my class.) -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ Project Vote Smart: http://www.vote-smart.org/

3 2

Re: making dbmmodule still broken
by Skip Montanaro 19 Jun '02

19 Jun '02

Guido> On my 2yo Mandrake 8.1 (?) system, when I do "make" in the latest Guido> CVS tree, I always get an error from building dbmmodule.c: Okay, in between ringing up credit card charges I took another look at building the dbm module. I can't cvs diff at the moment, but here's what I propose (and what changes I just made locally): * Remove the ndbm.h and gdbm/ndbm.h checks from configure.in and run autoconf. * Change the block of code in setup.py that checks for dbm libraries and include files to if platform not in ['cygwin']: if (self.compiler.find_library_file(lib_dirs, 'ndbm') and find_file("ndbm.h", inc_dirs, []) is not None): exts.append( Extension('dbm', ['dbmmodule.c'], define_macros=[('HAVE_NDBM_H',None)], libraries = ['ndbm'] ) ) elif (self.compiler.find_library_file(lib_dirs, 'gdbm') and find_file("gdbm/ndbm.h", inc_dirs, []) is not None): exts.append( Extension('dbm', ['dbmmodule.c'], define_macros=[('HAVE_GDBM_NDBM_H',None)], libraries = ['gdbm'] ) ) elif db_incs is not None: exts.append( Extension('dbm', ['dbmmodule.c'], library_dirs=[dblib_dir], include_dirs=db_incs, define_macros=[('HAVE_BERKDB_H',None), ('DB_DBM_HSEARCH',None)], libraries=dblibs)) This does two things. It removes the else clause which would never have worked (no corresponding include files would have been found). It also performs the two include file tests I removed from configure.in and defines the appropriate macros. Building after making these changes I get gdbm. If I mv gdbm/ndbm.h out of the way or comment out the first elif branch I get Berkeley DB. I don't have an ndbm library on my system, so I can't exercise the first branch. I think it would probably be a good idea to alert the person running make what library the module will be linked with. Anyone else agree? Skip

3 4

Re: [Python-Dev] extended slicing again
by Michael Hudson 19 Jun '02

19 Jun '02

Michael Hudson <mwh(a)python.net> writes: > Guido van Rossum <guido(a)python.org> writes: > > > IOW slice(a, b, None) should be considered equivalent to L[a:b] in all > > situations. > > OK. I'll do this soon. It's not as bad as I thought at first -- only > mutable sequences are affected, so it's only lists and arrays that > need to be tweaked. That was easy! Cheers, M. -- I have no disaster recovery plan for black holes, I'm afraid. Also please be aware that if it one looks imminent I will be out rioting and setting fire to McDonalds (always wanted to do that) and probably not reading email anyway. -- Dan Barlow

1 0

Tcl adept wanted for Stackless problem
by Christian Tismer 19 Jun '02

19 Jun '02

Dear Lists, there is still some problem with Tcl and Stackless hiding. Is there anybody around who knows the Tcl/Tk sources about as I know Python's? My big question is: When does Tcl use C stack entries as globals, which are passed as function arguments to interpreter calls? This is of special interest when Python callbacks are invoked. The problem is, that I use stack slicing, which moves parts of the C stack away at some times, and I need to know when I am forbidden to do that. It is solved to some extent, but not all. That's unfortunately not all. A friend is running the Tcl/Tk mainloop in one real thread, and stackless tasklets are running in another one (of course calling into Tcl). When tasklet switches are performed (again, this is moving stack contents around), there appear to be crashes, too. This kind of slicing should be allowed IMHO, since these are different contexts, which shouldn't interact at all. Are there any structures which are shared between threads that use Tcl/Tk? Something that may not disappear during some operations? Which part of the documentation / the Tcl/Tk source should I read to find this out wihout learning everything? This is really an urgent problem which is blocking me. If somebody has good knowledge, please show up! :-) thanks & cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

2 2

Re: [Python-Dev] PEP 292, Simpler String Substitutions
by Raymond Hettinger 19 Jun '02

19 Jun '02

From: "Barry A. Warsaw" <barry(a)zope.com> > A Simpler Proposal > > Here we propose the addition of a new string method, called .sub() > which performs substitution of mapping values into a string with > special substitution placeholders. These placeholders are > introduced with the $ character. The following rules for > $-placeholders apply: > > 1. $$ is an escape; it is replaced with a single $ Hmm, some strings (at least in the spam I receive) contain $$$$$$. How about ${$}? > 2. $identifier names a substitution placeholder matching a mapping > key of "identifier". "identifier" must be a Python identifier > as defined in [2]. The first non-identifier character after > the $ character terminates this placeholder specification. +1 > > 3. ${identifier} is equivalent to $identifier and for clarity, > this is the preferred form. It is required for when valid > identifier characters follow the placeholder but are not part of > the placeholder, e.g. "${noun}ification". +1 > Handling Missing Keys > > What should happen when one of the substitution keys is missing > from the mapping (or the locals/globals namespace if no argument > is given)? There are two possibilities: > > - We can simply allow the exception (likely a NameError or > KeyError) to propagate. > > - We can return the original substitution placeholder unchanged. And/Or, - Leave placeholder unchanged unless default argument supplied: mystr.sub(mydict, undefined='***') # Fill unknowns with stars And/Or, - Raise an exception only if specified: mystr.sub(mydict, undefined=NameError) And/Or - Return a count of the number of missed substitutions: nummisses = mystr.sub(mydict) > BDFL proto-pronouncement: It should always raise a NameError when > the key is missing. There may not be sufficient use case for soft > failures in the no-argument version. I had written a minature mail-merge program and learned that the NameError approach is a PITA. It makes sense if the mapping is defined inside the program; however, externally supplied mappings (like a mergelist) can be expected to have "holes" and launching exceptions makes it harder to recover than having a default behavior. The best existing Python comparison is the str.replace() method which does not bomb-out when the target string is not found. Raymond Hettinger

4 5

Re: [Python-Dev] PEP 292, Simpler String Substitutions
by Guido van Rossum 19 Jun '02

19 Jun '02

> I'm so behind on my email, that the anticipated flamefest will surely > die down before I get around to reading it. Yet still, here is a new > PEP. :) No, the flamefest won't start until you post to c.l.py. :) --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Re: [Python-Dev] Slicing
by Michael Hudson 19 Jun '02

19 Jun '02

"David Abrahams" <david.abrahams(a)rcn.com> writes: > I did a little experiment to see if I could use a uniform interface for > slicing (from C++): > > >>> range(10)[slice(3,5)] > Traceback (most recent call last): > File "<stdin>", line 1, in ? > TypeError: sequence index must be integer > >>> class Y(object): > ... def __getslice__(self, a, b): > ... print "getslice",a,b > ... > >>> y = Y() > >>> y[slice(3,5)] > Traceback (most recent call last): > File "<stdin>", line 1, in ? > TypeError: unsubscriptable object > >>> y[3:5] > getslice 3 5 > > This seems to indicate that I can't, in general, pass a slice object to > PyObject_GetItem in order to do slicing.** Correct? No. The time machine has got you here; update to CVS and try again. This comes down to the (slightly odd, IMHO) distinction between sequences and mappings, which doesn't really appear at the Python level. type_pointer->tp_as_sequence->sq_item takes a single int as a parameter type_pointer->tp_as_mapping->mp_subscr takes a PyObject*. Builtin sequences (as of last week) have mp_subscr methods that handle slices. I haven't checked, but would be amazed if PyObject_GetItem can't now be used with sliceobjects. (PS: I'm not sure I've got all the field names right here. They're close). > So I went looking around for alternatives to PyObject_GetItem. I found > PySequence_GetSlice, but that takes int parameters, and AFAIK there's no > rule saying you can't slice on strings, for example. > > Further experiments revealed: > > >>> y['hi':'there'] > Traceback (most recent call last): > File "<stdin>", line 1, in ? > TypeError: unsubscriptable object > >>> class X(object): > ... def __getitem__(self, x): > ... print 'getitem',x > ... > >>> X()['hi':'there'] > getitem slice('hi', 'there', None) > > So I /can/ slice on strings, but only through __getitem__(). And... > > >>> class Z(Y): > ... def __getitem__(self, x): > ... print 'getitem',x > ... > >>> Z()[3:5] > getslice 3 5 > >>> Z()['3':5] > getitem slice('3', 5, None) > > So Python is doing some dispatching internally based on the types of the This area is very messy. > slice elements, but: > > >>> class subint(int): pass > ... > >>> subint() > 0 > >>> Z[subint():5] > Traceback (most recent call last): > File "<stdin>", line 1, in ? > TypeError: unsubscriptable object This last one is easy: you're trying to subscript the class object! > So it's looking at the concrete type of the slice elements. I'm not > sure I actually understand how this one fails. > > I want to make a generalized getslice function in C which can operate on a > triple of arbitrary objects. Here's the python version I came up with: > > def getslice(x,start,finish): > if (type(start) is type(finish) is int > and hasattr(type(x), '__getslice__')): > return x.__getslice__(start, finish) > else: > return x.__getitem__(slice(start,finish)) > > Have I got the logic right here? You can't do this logic from Python, AFAIK. I think PyObject_GetItem is your best bet. Cheers, M. -- The only problem with Microsoft is they just have no taste. -- Steve Jobs, (From _Triumph of the Nerds_ PBS special) and quoted by Aahz Maruch on comp.lang.python

2 1

Negative long literals (was Re: Does Python need a '>>>' operator?)
by Tim Peters 19 Jun '02

19 Jun '02

I think Beni has a very nice idea here, especially for people who can't visualize 2's-complement (not mentioning Guido by name <wink>). -----Original Message----- From: Beni Cherniavksy <cben(a)tx.technion.ac.il> Sent: Monday, June 10, 2002 1:57 AM To: python-list(a)python.org Subject: Re: Does Python need a '>>>' operator? ... [quotes of old postings deleted] ... I just got another idea: use 0x1234 for 0-filled numbers and 1xABCD for 1-filled ones. That way you impose no restrictions on what follows the prefix and keep backward compatibility. 0xFFFFFFFF stays a 2^n-1 _positive_ number, as it should be. The look of 1x is weird at first but it is very logical... -- Beni Cherniavsky <cben(a)tx.technion.ac.il> -- http://mail.python.org/mailman/listinfo/python-list

4 12