Mailman 3 July 2008 - Python-ideas

"While" suggestion
by Stavros Korokithakis 22 Aug '08

22 Aug '08

Hello all, I have noticed that sometimes "while" loops produce "unpythonic" patterns, such as the following: data = my_file.read(1024) while data: do_something(data) data = my_file.read(1024) The assignment is repeated, which is less than optimal. Since we don't have a while statement that does the check in the end, would it not be better if the syntax of while could be amended to include something like this (in the spirit of the new "with" keyword)?: while my_file.read(1024) as data: do_something(data) This would terminate the loop when myfile.read() evaluated to False, and it is more pythonic than repeating onesself. I contacted GvR about this, and he replied that this syntax would have to be part of the expression than part of the while, which I agree would be less than ideal. However, I don't see why it would have to be part of the expression, since the "while" could easily assign the value of the expression to the variable and break if it evaluates to False. I would appreciate any thoughts on this, Stavros Korokithakis

28 75

fnmatch/glob with sub-expressions
by Erick Tryzelaar 21 Jul '08

21 Jul '08

I need to handle more advanced glob pattern matching than what glob/fnmatch provides. I need to support patterns like this: "a{b*c,d[ef]}" -> "a(b.*c|d[ef])$" "a{b{c,d},e}" -> "a(b(c|d)|e)$" This syntax is used in many shells, like bash and zsh, and other languages, like ruby. Does anyone else think this might be worthwhile in the standard library? It's a pretty simple change, but it could break code though, so we'd probably have to disable it by default.

2 1

Ideas towards GIL removal
by Greg Ewing 18 Jul '08

18 Jul '08

I've been thinking about some ideas for reducing the amount of refcount adjustment that needs to be done, with a view to making GIL removal easier. 1) Permanent objects In a typical Python program there are many objects that are created at the beginning and exist for the life of the program -- classes, functions, literals, etc. Refcounting these is a waste of effort, since they're never going to go away. So perhaps there could be a way of marking such objects as "permanent" or "immortal". Any refcount operation on a permanent object would be a no-op, so no locking would be needed. This would also have the benefit of eliminating any need to write to the object's memory at all when it's only being read. 2) Objects owned by a thread Python code creates and destroys temporary objects at a high rate -- stack frames, argument tuples, intermediate results, etc. If the code is executed by a thread, those objects are rarely if ever seen outside of that thread. It would be beneficial if refcount operations on such objects could be carried out by the thread that created them without locking. To achieve this, two extra fields could be added to the object header: an "owning thread id" and a "local reference count". (The existing refcount field will be called the "global reference count" in what follows.) An object created by a thread has its owning thread id set to that thread. When adjusting an object's refcount, if the current thread is the object's owning thread, the local refcount is updated without locking. If the object has no owning thread, or belongs to a different thread, the object is locked and the global refcount is updated. The object is considered garbage only when both refcounts drop to zero. Thus, after a decref, both refcounts would need to be checked to see if they are zero. When decrementing the local refcount and it reaches zero, the global refcount can be checked without locking, since a zero will never be written to it until it truly has zero non-local references remaining. I suspect that these two strategies together would eliminate a very large proportion of refcount-related activities requiring locking, perhaps to the point where those remaining are infrequent enough to make GIL removal practical. -- Greg

8 19

Re: [Python-ideas] Unittest PEP do's and don'ts (BDFL pronouncement)
by Guido van Rossum 17 Jul '08

17 Jul '08

Please move all discussions of unittest frameworks to python-ideas(a)python.org. It is an interesting topic -- so interesting, in fact, that exploring all the different ideas under discussion is overwhelming the primary purpose of python-dev, which at this point is to get the 2.6 and 3.0 releases into shape. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Re: [Python-ideas] Openine unicode files
by Terry Reedy 09 Jul '08

09 Jul '08

Noorhan Abbas wrote: > Hello, > I wonder if someone can advise me on how to open unicode utf-8 files > without using the codecs library . I am trying to use the codecs.open() > from within Google Appengine but it is not working. When posting about something 'not working', post at least a line of code and the resulting exception traceback, if there is one, or a printing of the erroneous value, if that instead is the problem.

1 0

print in the interactive interpreter? plus shell API idea
by Roman Susi 07 Jul '08

07 Jul '08

hi! I am sure that Python interactive interpreter is a good tool. Before it were simple to print something with print. Now extra ( and ) are needed and also extra caution. Is it possible to have some simplified "print" in the interactive interpreter? IMHO, Basic's "?" is quite a good candidate. Maybe, it should not even be Python syntax but Python interactive shell feature. cf: >>> print(functools.wraps.__doc__) and >>> ? functools.wraps.__doc__ Also, some discussions about http://bugs.python.org/issue449227 (can't recall where they were, maybe private) indicate that maybe some kind of Python shell API could be interesting to facilitate Python interpreter's interaction with shells and provide better, more dynamic, experience to the shells' users (e.g. readline-like features, inline helps and hints, etc). With the API it will be easier for "vendors" to come up with CLUI/GUI/whatever shells. Shells will be thus decoupled from the interpreter itself. (WSGI is pretty similar in intention to abstract web service interaction, maybe the one I suggest may be called Rich Interpreter Shell Gateway Interface (RISGI) or something like that). The idea is quite raw, maybe not even pre-PEPable yet, because I am not really shell developer. It also differs from the way Python gets programmatically embed into something because the interface is really meant ot be like Python interpreter's "telnet" or "X", for humans. There could be addons to the shell protocol use (debugger or profiler subshells, datastructure/class/module browser subshell/subprotocol, help subshells). Special case shells (using same protocol) may be used, eg, for mobile phone Python. To put these topics in two: "standard" Python shell may provide keystroke shortcuts for some useful cases (like, today's shell is having underscore to mean most recently calculated expression value). So, "?" will not be part of the syntax but a shell's shortcut. I think that such shell protocol is quite unique for the programming languages and may be real breakthrough in interactive shell field and/or even whole CLUI. -Roman

7 9

Re: [Python-ideas] "While" suggestion
by BJörn Lindqvist 04 Jul '08

04 Jul '08

On Thu, Jul 3, 2008 at 9:38 PM, Mike Meyer <mwm(a)mired.org> wrote: >> #while True idiom (current Python) >> while True: >> number = random.randint(1, 100) >> if number not in picked_numbers: >> break > > I don't believe that actually qualifies as "current python", any more > than the other current alternative: > > # duplicate setup (current Python) > number = random.randint(1, 100) > while number in picked_numbers: > number = random.randint(1, 100) > > does. > > Both bits of code smell. One has the duplicated line; one has a test > of a constant and a block exiting from the middle, all of which make > the code slightly less clear. Yup. Non-for loops just aren't very strong in Python. > I tend to write either one, depending on what the duplicated code > looks like. For this case, I actually prefer the latter. On the other > hand, I'd probably write this one using a *third* alternative idiom: > > # Picked initial value (current Python) > number = picked_numbers[0] > while number in picked_numbers: > number = random.randint(1, 100) I didn't say that the picked_numbers list had to be non-empty so you lose. :) -- mvh Björn

1 0

Re: [Python-ideas] "While" suggestion
by Stavros Korokithakis 03 Jul '08

03 Jul '08

Hmm, the regular expression use case (and any other use case where you need to act based on the output of a function) looks very interesting, but I can understand how there would be a few problems with it for things like "if myfunc() > 10 as output". Still, it is worth investigating. Stavros Mike Meyer wrote: > On Thu, 3 Jul 2008 11:46:51 -0300 "Facundo Batista" <facundobatista(a)gmail.com> wrote: > >> 2008/7/3 Stavros Korokithakis <stavros(a)korokithakis.net>: >> >>> while my_file.read(1024) as data: >>> do_something(data) >> Python explicitly disallows inline assignment ("while >> a=myfile.read():") because of the error propensity when confusing it >> with the comparation ("=="). But here we're gaining the same >> advantage, without that risk. >> >> So, taking into account that... >> >> a) We already have "as" as an statement. >> >> b) We already use "as" as an assignment [1] >> >> c) This will allow more concise and intuitive code >> >> ... I'm definitely +1 to this proposition. >> > > I kinda like it as well. In fact, treating "as" as an assignment > operator instead of making it part of the syntax of the while > statement solves my problem with the suggestion. > > The original suggestion only dealt with a *very* special case: a loop > where 1) you needed to initialize the data before the loop started; 2) > the initialization statement was use to refresh the data in the loop, > and 3) the value of the data as a bool determined whether you wanted > to continue. > > If #3 isn't true, a while+as statement leaves you back at the original > code duplication. as as inline assignment it lets you deal with it > without the repitition: > > while my_file.read(1024) as data < 1024: > do_something_with_1k(data) > do_something_with_lessthan_1k(data) > > It also provides a nice way to deal with the long-standing ugliness > related to wanting to do repeated regular expression tests: > > if myre.match(user_input) as match: > do_something_with_matching_data(match) > elif myre2.match(user_input) as match: > do_something_else(match) > > instead of: > > match = myre.match(user_input) > if match: > do_something_with_matching_data(match) > else: > match = myre2.match(user_input) > if match: > do_something_else(match) > > which doesn't yield to the usual solution of a dict of functions. > >> In any case, Stavros, this would need a PEP... > > I'm not convinced (it ought to interact cleanly with with, but....), > but it certainly looks promising! > > +1 to at least investigating it. > > <mike

1 0

Reducing colon uses to increase readability
by Andre Roberge 03 Jul '08

03 Jul '08

Hi everyone, In Python, the humble colon (:) has multiples uses: 1. as a signal to indentation increase, signaling a block of code, such as 1a) for function or class definitions 1b) for while/for/if/elif/else blocks 1c) for try/except/finally blocks In these cases, the majority opinion (to which I subscribe) is that using a colon increases readability. I am NOT suggesting to removing the colon in those instances. However, the colon has also some other uses. 2. in slices [a:b:c] 3. in dict assignments {a:b} 4. in lambda assignments (lambda x: x+1) I would argue that, in these last three examples, there might be better choices. (some of these choices have been inspired by reading http://www.resolversystems.com/documentation/index.php/Differences_Between_…) I don't expect the following suggestions to immediately convince everyone (or anyone!) ... but, at least they will be on record. Slices: --------- I would argue that, the usual slicing notation would be more readable if it were as follows: [a -> b; c] Thus [1:10:2] would become [1 -> 10; 2] [1:10] would become [1 -> 10] The "shorter" combinations would not gain in terms of readability; they would be as follows: [ :10 : 2] would become [10; 2] [ :10] would become [10;] [:: -1] would become [; -1] [:] would become [;] If such a change were to be made, an second slicing notation, *with a different meaning*, could be introduced: [a => b; c] This would be an inclusive range, i.e. [a => b] is equivalent to [a -> b+1] dict assignments ------------------------ Here again, I would argue that using "->" instead of ":" would make the code more readable - at least for beginners. numbers = {'one' -> 1, 'two' -> 2} instead of numbers = {'one': 1, 'two': 2} lambda assignments --------------------------- Once again, same choice. lambda x -> x+1 is, I think, more readable than lambda x: x+1 (but perhaps the last two [dicts and lambda] largely depends on the font choice...) ====== Other considerations: If "->" were to be adopted for dict or lambda assignments, then the "naturalness" of their choice for slices would be reduced. An alternative might be inspired from the mathematical notation [a, ..., b; c] I realize that this is "much" longer than [a: b: c]. Final comment: I have seen other alternatives for simple slices suggested in the past such as [a..b] and [a...b] which would be the equivalent of [a->b] and [a=>b]; however, the extra "." might sometimes be difficult to read, whereas the difference between "->" and "=>" is much easier to see. Cheers, André

9 12

Python-ideas July 2008