Mailman 3 March 2009 - Python-Dev

Approaches to argument type-checking
by Tennessee Leeuwenburg March 12, 2009

March 12, 2009

Hi all, I am currently looking at issue 5236. This issue regards the exception raised when a bytes string is passed into time.strptime. In addition to the specific question I have regarding this issue, I wasn't sure if this was something for python-dev or for the issue comment. However, it does concern general Python coding approach, so just give me a pointer over whether this is better kept on the tracker or whether posting to the list was a good idea (I'm slowly learning!) EXAMPLE OF PROBLEM: >>> time.strptime(b'2009', "%Y") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/tjl/python3/lib/python3.1/_strptime.py", line 454, in _strptime_time return _strptime(data_string, format)[0] File "/home/tjl/python3/lib/python3.1/_strptime.py", line 322, in _strptime found = format_regex.match(data_string) TypeError: can't use a string pattern on a bytes-like object WHEREAS: >>> time.strftime(b"%Y") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: strftime() argument 1 must be str, not bytes What is occurring here is that the arguments to strftime are being type-checked up-front, whereas in strptime they are not. Further, srtptime is implemented in a python file, _strptime.py, whiel strftime is implemented in timemodule.c. It appears as though it is generally the case (or at least often the case) that C functions are making use of the vgetargs function which performs a goodly bit of type checking. However, the same does not seem to hold for the Python interface functions. >From the Python interpreter perspective, though, both are in the time module (time.strftime and time.strptime) so the inconsistency is a bit jarring. I can see that I could solve this a few ways: * Do a false-parse of the arguments using the same vgetargs1 method, but not do anything with the return value * Perform a type-check on the relevant argument, data_string, in Python and raise a more specific Exception * Write some kind of generic type-checking helper method which could be re-used Is there a general strategy used in Python development which I should be aware of? i.e. is it customary to type-check every argument of an interface method? Or is it customary not to perform type checking up-front and simply allow the exception to occur deeper in the code? Are there performance issues surrounding defensive programming? Regards, -Tennessee -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

2 1

Re: [Python-Dev] Ext4 data loss
by Hrvoje Niksic March 11, 2009

March 11, 2009

Christian Heimes wrote: > Guido van Rossum wrote: >> Let's not think too Unix-specific. If we add such an API it should do >> something on Windows too -- the app shouldn't have to test for the >> presence of the API. (And thus the API probably shouldn't be called >> fsync.) > > In my initial proposal one and a half hour earlier I suggested 'sync()' > as the name of the method and 'synced' as the name of the flag that > forces a fsync() call during the close operation. Maybe it would make more sense for "synced" to force fsync() on each flush, not only on close. I'm not sure how useful it is, but that's what "synced" would imply to me. Maybe it would be best to avoid having such a variable, and expose a close_sync() method instead?

3 2

Re: [Python-Dev] [Python-ideas] Adding a test discovery into Python
by Guilherme Polo March 11, 2009

March 11, 2009

On Wed, Mar 11, 2009 at 7:37 PM, Raymond Hettinger <python(a)rcn.com> wrote: > [Christian Heimes] >>>> >>>> I'm +1 for a simple (!) test discovery system. I'm emphasizing on simple >>>> because there are enough frameworks for elaborate unit testing. > > Test discovery is not the interesting part of the problem. Interesting or not, it is a problem that is asking for a solution, this kind of code is being duplicated in several places for no good reason. > > Axiom: The more work involved in writing tests, the fewer > tests that will get written. At some point you will have to run them too, I don't think you want to reimplement the discovery part yet another time. -- -- Guilherme H. Polo Goncalves

1 0

Can I modify string.Formatter._vformat?
by Eric Smith March 11, 2009

March 11, 2009

I'm implementing support for auto-numbering of str.format() fields (see http://bugs.python.org/issue5237). I'm reasonably sure that when I'm done modifying the C implementation I'll need to change the signatures of string.Formatter._vformat, str._formatter_parser, and/or str._formatter_field_name_split. (They need to support the state needed to track the auto-number field counter.) I've always considered these internal implementation details of str.format and string.Formatter. They begin with underscores and are not documented. Is there any problem with modifying these in 2.7 and 3.1? I assume not, but I want to make sure it doesn't give anyone heartburn. Eric.

3 2

Re: [Python-Dev] Ext4 data loss
by Hrvoje Niksic March 11, 2009

March 11, 2009

Joachim König wrote: > To me, the flaw seem to be in the close() call (of the operating > system). I'd expect the data to be > in a persistent state once the close() returns. I wouldn't, because that would mean that every cp -r would effectively do an fsync() for each individual file it copies, which would bog down in the case of copying many small files. Operating systems aggressively buffer file systems for good reason: performance of the common case. > Why has this ext4 problem not come up for other filesystems? It has come up for XFS many many times, for example https://launchpad.net/ubuntu/+bug/37435 ext3 was resillient to the problem because of its default allocation policy; now that ext4 has implemented the same optimization XFS had before, it shares the problems.

1 0

Deprecated __cmp__ and total ordering
by Mart Sõmermaa March 11, 2009

March 11, 2009

__cmp__ used to provide a convenient way to make all ordering operators work by defining a single method. For better or worse, it's gone in 3.0. To provide total ordering without __cmp__ one has to implement all of __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all but a few cases it suffices only to provide a "real" implementation for e.g. __lt__ and define all the other methods in terms of it as follows: class TotalOrderMixin(object): def __lt__(self, other): raise NotImplemented # override this def __gt__(self, other): return other < self def __le__(self, other): return not (other < self) def __ge__(self, other): return not (self < other) __eq__ and __ne__ are somewhat special, although it is possible to define them in terms of __lt__ def __eq__(self, other): return not (self == other) def __ne__(self, other): return self < other or other < self it may be inefficient. So, to avoid the situation where all the projects that match http://www.google.com/codesearch?q=__cmp__+lang%3Apython have to implement their own TotalOrderMixin, perhaps one could be provided in the stdlib? Or even better, shouldn't a class grow automagic __gt__, __le__, __ge__ if __lt__ is provided, and, in a similar vein, __ne__ if __eq__ is provided?

3 8

Re: [Python-Dev] Addition of further status options to tracker
by Brett Cannon March 11, 2009

March 11, 2009

[adding python-dev back on to the email] On Tue, Mar 10, 2009 at 15:51, Tennessee Leeuwenburg <tleeuwenburg(a)gmail.com > wrote: > >>> Pretty much. I've got two views. One is that I'd like to search for >>> issues that are up for grabs which I could take over, hack on, and generally >>> not get underfoot of core development activity. >>> >> >> OK, let's do what is necessary to flag issues like this so that people can >> do this. That's why I like the "Under Development" status. Could rename >> "open" to "available" or "unsolved" to more clearly mark those issues as up >> for grabs. >> > > Yep. I like that too. > > >> Typically we use nosy lists to get specific people's attention. That or >> the priority gets bumped if it turns out to be an important issue. Lastly, >> people can email python-dev directly asking for input if all other attempts >> to get attention have failed. >> > > Now that I am understanding the tracker system better, I think it's fine to > just add reviews to the tracker issue and that will be enough to grab > attention. There is always the option of emailing the list. > > >> >> In other words you want some way to flag an issue that just needs to be >> talked about but is not ready to be coded. So status would go "open/new" -> >> "chatting/needs help" -> "under dev" -> "closed" with "pending" fitting in >> when necessary. Sound about right? >> >> My worry with this is making sure the field gets updated. >> > > Sounds exactly right. I'm not so concerned about this field being updated. > If it doesn't, but someone is clearly working on it, then it's not really > holding anyone back. Tracker janitors (although I do prefer gardeners!) can > worry about whether the field is set correctly, and developers can just get > on with doing their work. > > I like: > "Open/New" > "Needs help / Chatting" > "Under development" > "Pending feedback" > "Closed" > > very much. > > As long as "Under Dev" and "Pending" are time-based to switch to "chatting" or "closed" respectively, I am fine with this. What do other people think? Too heavy-handed? Just right to help people get people involved? -Brett > <snip> ...I can understand that, but I would worry that if we flag >> something as under development it will simply be ignored by other people >> when they could actually help out (write the docs for someone, etc.). Or >> even worse that someone gets to a certain point with the development, walks >> away short of finishing the work (say doesn't get the docs finished) and >> everyone continues to ignore the issue because it is still flagged as under >> development. > > > >> If we can come up with a simple solution to this problem (perhaps have >> issues set to under development with no activity shift down a status level >> after a month) then maybe we will have something everyone can be happy with. > > > > Maybe we can just revert anything that is under development back to "needs > help" after a month of inactivity? > > Cheers, > -T >

1 0

Fwd: Addition of further status options to tracker
by Tennessee Leeuwenburg March 10, 2009

March 10, 2009

On Wed, Mar 11, 2009 at 9:43 AM, Brett Cannon <brett(a)python.org> wrote: > > > On Tue, Mar 10, 2009 at 15:33, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > >> Brett Cannon wrote: >> > If we can come up with a simple solution to this problem (perhaps have >> > issues set to under development with no activity shift down a status >> > level after a month) then maybe we will have something everyone can be >> > happy with. >> >> If an issue is assigned, then somebody has claimed it and is working on >> it (or it has at least been sent to a specific person for consideration). >> > > Right, but that is for core developers only. I think Tennessee is worrying > about non-core folks. > Absolutely, I don't have any issue with the way the most important issues are being worked on now, I just think that less-experienced developers could do a lot more to help out with simple tasks / early-stage tasks. If the rest of us can help ease the burden by getting issues properly sorted out before they go to core developers (writing unit tests, sorting out requirements clearly, documentation, patch suggestions etc) then they won't need to spend as much time on simple maintenance. Cheers, -T -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

1 0

Regexp 2.7
by Antoine Pitrou March 10, 2009

March 10, 2009

Hello, Matthew Barnett has been doing a lot of work on the regular expressions engine (it seems he hasn't finished yet) under http://bugs.python.org/issue2636. However, the patches are really huge and touch all of the sre internals. I wonder what the review process can be for such patches? Is there someone knowledgeable enough to be able to review them? Regards Antoine.

8 10

Addition of further status options to tracker
by Tennessee Leeuwenburg March 10, 2009

March 10, 2009

Hi all, I am beginning reviewing some more issues in the tracker. I think it would be useful to have the following status options (new status options marked with a '+'): Open: Means that the issue has been created and not further reviewed + Request Approved: Means that the issue is marked as worth further development by the community + Specification Approved: Means that the functionality to be developed is written down in some form, and agreed at least in general terms (preferably in fairly specific terms) + Under Development: Means that an implementation is currently under development Pending Feedback: Means that work is suspended pending feedback + Under Final Review: Means that a patch has been submitted and may be a flag that core developers can usefully review the work done without having to revisit the whole process Closed: Means that the issue is either suspended indefinitely or has been resolved (see resolution value) My reasoning for this is as follows: Example one: I am currently reviewing issue 2706 which covers datetime.timedelta operators. I have a reasonable amount of familiarity with time programming and think I have some valuable input on the functionality aspects, however I'm not a super-great C programmer. I'd like to help with coming up with the final feature specification, but then leave it to others to consider the merits of the C code patch. Being able to help get this marked as "request approved", then "specification approved" might help speed up a lot of the process of developing the patch and getting it accepted. It also saves code developers from having to develop an implementation while a functionality debate is still ongoing... I think that having these additional steps will help to avoid erroneous development of non-features, and also break up the review process into more manageable chunks of work. Of course I realise not everything needs such a delineated process, but by the same token it might assist in keeping some of the less visible/popular changes moving through the process... Comments welcome! Crocker's rules, but please keep it constructive... -T -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

6 11