Mailman 3 October 2008 - Python-ideas

Re: [Python-ideas] Idea: Lazy ... statement
by Jim Jewett 15 Oct '08

15 Oct '08

On Mon, Oct 13, 2008 at 4:12 AM, Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > So I think it would be good to have a dedicated syntax > for lazy imports, so the top-level foo package can say > something like > from foo.thing lazily import Thing > from foo.stuff lazily import Stuff Since this can be (awkwardly) done now, I don't think the problem is big enough for another keyword. On the other hand, a "lazy" keyword that worked in other contexts might have some merit. That would solve at least one major objection to setdefault-style functions. def setdefault(self, k, lazy d=None): ... or even lazy def yikes(msg): """Something went very wrong; report for debugging""" import logging ...logging config setup... ... Note that I'm not yet ready to volunteer on fleshing out the details, let alone how to implement it efficiently. (Would it require the moral equivalent of quasiquoting?) -jJ

4 6

Re: [Python-ideas] Automatic total ordering
by Terry Jones 15 Oct '08

15 Oct '08

Oops..... didn't see your footnote URL. Sorry! Terry

1 0

Re: [Python-ideas] Automatic total ordering
by Terry Jones 15 Oct '08

15 Oct '08

>>>>> "George" == George Sakkis <george.sakkis(a)gmail.com> writes: George> Now that 3.x fixes the arbitrary object comparison wart and drops George> (?) __cmp__, it seems it's a good time to do something with the George> missing rich comparators gotcha, e.g. given a class that defines George> __eq__ and __lt__, automatically provide the rest missing George> comparisons. Note http://www.voidspace.org.uk/python/weblog/arch_d7_2008_10_04.shtml#e1018 Regards, Terry

1 0

VS2005 project improvement
by Victor Martin Ulloa 15 Oct '08

15 Oct '08

I'm trying to comppile python 2.5 as a static library using VS2005 SP1. I realized you only have Debug, PG and Release configurations. Of course, like you provide the sources, I could modify it manually... but I think will be much better to provide the Debug_Static and Release_Static configurations for noob user like me :P Also, you are using this code in the pyconfig.h: /* For an MSVC DLL, we can nominate the .lib files used by extensions */ #ifdef MS_COREDLL # ifndef Py_BUILD_CORE /* not building the core - must be an ext */ # if defined(_MSC_VER) /* So MSVC users need not specify the .lib file in their Makefile (other compilers are generally taken care of by distutils.) */ # ifdef _DEBUG # pragma comment(lib,"python25_d.lib") # else # pragma comment(lib,"python25.lib") # endif /* _DEBUG */ # endif /* _MSC_VER */ # endif /* Py_BUILD_CORE */ #endif /* MS_COREDLL */ This does not allow the user to rename the output library ( for example, to pytoncore_static_debug.lib ). It would be very desireable to allow the user to change the python library output name... and use these names as defaults: python25_md.lib -> python 2.5, multithread debug C CRT python25_mdd.libl -> python 2.5, multithread debug DLL C CRT python25_static_debug.lib -> python 2.5, multithread debug static library C CRT python25_static.lib -> python 2.5, multithread static library C CRT On the other hand, I see the python 3.0rc1 solution has been saved using VS2008. I think that's bad, because VS2005 users won't be able to open the solution. Ideally, you should provide a python solution for each Visual Studio: PCBuild_VC6 PCBuild_VC2002 PCBuild_VC2003 PCBuild_VC2005 PCBuild_VC2008 or provide just the VC6 solution that can be easily converted by all the modern Visual Studio versions. thanks.

3 2

For-loop variable scope: simultaneous possession and ingestion of cake
by Greg Ewing 15 Oct '08

15 Oct '08

There's been another discussion on c.l.py about the problem of lst = [] for i in range(10): lst.append(lambda: i) for f in lst: print f() printing 9 ten times instead of 0 to 9. The usual response is to do lst.append(lambda i=i: i) but this is not a very satisfying solution. For one thing, it's still abusing default arguments, something that lexical scoping was supposed to have removed the need for, and it won't work in some situations, such as if the function needs to take a variable number of arguments. Also, most other languages which have lexical scoping and first-class functions don't seem to suffer from problems like this. To someone familiar with one of those languages (e.g. Scheme, Haskell) it looks as if there's something broken about the way scoping of nested functions works in Python. However, it's not lambda that's broken, it's the for loop. In Scheme, for example, the way you normally write the equivalent of a Python for-loop results in a new scope being created for each value of the loop variable. Previous proposals to make for-loop variables local to the loop have stumbled on the problem of existing code that relies on the loop variable keeping its value after exiting the loop, and it seems that this is regarded as a desirable feature. So I'd like to propose something that would satisfy both requirements: 0. There is no change if the loop variable is not referenced by a nested function defined in the loop body. The vast majority of loop code will therefore be completely unaffected. 1. If the loop variable is referenced by such a nested function, a new local scope is effectively created to hold each successive value of the loop variable. 2. Upon exiting the loop, the final value of the loop variable is copied into the surrounding scope, for use by code outside the loop body. Rules 0 and 1 would also apply to list comprehensions and generator expressions. There is a very simple and efficient way to implement this in current CPython: If the loop variable is referenced by a nested function, it will be in a cell. Instead of rebinding the existing cell, each time around the loop a new cell is created, replacing the previous cell. Immediately before exiting the loop, one more new cell is created and the final value of the loop variable copied into it. Implementations other than CPython that aren't using cells may need to do something more traditional, such as compiling the loop body as a separate function. I think this arrangement would allow almost all existing code to continue working, and new code to be written that takes advantage of final values of loop variables. There would be a few obscure situations where the results would be different, for example if a nested function modifies the loop variable and expects the result to be reflected in the value seen from outside the loop. But I can't imagine such cases being anything other than extremely rare. The benefit would be that almost all code involving loops and nested functions would behave intuitively, Python would free itself from any remaining perception of having broken scope rules, and we would finally be able to consign the default-argument hack to the garbage collector of history. -- Greg

14 51

Re: [Python-ideas] Where/how to propose an addition to a standard module?
by Joe Strout 14 Oct '08

14 Oct '08

On Oct 13, 2008, at 8:46 AM, pruebauno(a)latinmail.com wrote: > Whenever I needed such functionality I used the re module. The benefit > is that it uses unix style regular expression syntax and an egrep/awk/ > perl/ruby user can understand it. You should show a few examples where > your proposal looks better than just using RE. Well, I suppose if you're already used to RE, then maybe it's not obvious that to an RE newbie, this: regex = re.compile("The (?P<object>.*?) in (?P<location>.*?) falls mainly in the (?P<subloc>.*?).") d = regex.match(text).groupdict() is far harder to read and type correctly than this: templ = Template("This $object in $location falls mainly in the $subloc") d = templ.match(text) Any other example would show the same simplification. Of course, if you're the sort of person who uses RE, you probably don't use Template.substitute either, since you probably like and are comfortable with the string % operator. But Template.substitute was introduced to make it easier to handle the common, simple substitution operations, and I believe adding a Template.match method would do the same thing for common, simple matching operations. Here's a more fleshed-out proposal, with rationale and references -- see if this makes it any clearer why I think this would be a fine addition to the Template class. Abstract Introduces a new function on the string.Template [1] class, match(), to perform the approximate inverse of the existing substitute() function. That is, it attempts to match an input string against a template, and if successful, returns a dictionary providing the matched text for each template field. Rationale PEP 292 [2] added a simplified string substitution feature, allowing users to easily substitute text for named fields in a template string. The inverse operation is also useful: given a template and an input string, one wishes to find the text in the input string matching the fields in the template. However, Python currently has no easy way to do it. While this named matching operation can be accomplished using RegEx, the constructions required are somewhat complex and error prone. It can also be done using third-party modules such as pyparse, but again the setup requires more code and is not obvious to programmers inexperienced with that module. In addition, the Template class already has all the data needed to perform this operation, so it is a natural fit to simply add a new method on this class to perform a match, in addition to the existing method to perform a substitution. Proposal Proposed is the addition of one new function, on the existing Template class, as follows: def match(text, greedy=false) 'match' is a new function which accepts one required parameter, an input string; and one optional parameter, 'greedy', which determines whether matches should be done in a greedy manner, equivalent to regex pattern '(.*)'; or in a non-greedy manner, equivalent to '(.*?)'. If the input string can be matched to the template pattern (respecting the 'greedy' flag), then match returns a dictionary, where each field in the pattern maps to the corresponding part of the input string. If the input string cannot be matched to the template pattern, then match returns None. Examples: >>> from string import Template >>> s = Template('$name was born in ${country}') >>> print s.match('Guido was born in the Netherlands') {'name':'Guido', 'country':'the Netherlands'} >>> print s.match('Spam was born as a canned ham') None Note that when the match is successful, the resulting dictionary could be passed through Template.substitute to reconstitute the original input string. Conversely, any string created by Template.substitute could be matched by Template.match (though in unusual cases, the resulting dictionary might not exactly match the original, e.g. if the string could be matched in multiple ways). Thus, .match and .substitute are inverse operations. References [1] Template Strings http://www.python.org/doc/2.5.2/lib/node40.html [2] PEP 292: Simpler String Substitutions http://www.python.org/dev/peps/pep-0292/

4 5

Using Git to Manage Python Installation(s)
by Jeet Sukumaran 13 Oct '08

13 Oct '08

I've recently started using Git to manage the Python installations on my system (2.4, 2.5, 2.6 etc.),, and I have found that it has revolutionized my development ecosystem. Full versioning + documentation of all changes to each of the frameworks (including reversion in case something breaks), multiple parallel variants/package combinations of each installation (including a clean vanilla build) with seamless in-situ switching among them, easy instantiation and disposal of sandbox/experimental branches, etc. etc. I've written more about it here: http://jeetworks.org/node/24 It probably lies outside the purview of the development of Python itself (as opposed to Python development), but I thought I share this with you because I find it so useful! Furthermore, as I mention at the end of the above-referenced post, at the moment the big downside to this approach is the extra book-keeping burden that falls on the user. So much of this can be automated, though, with a distutils or setuptools post-install hook ... so maybe that might be something to consider?

2 1

Re: [Python-ideas] if-syntax for regular for-loops
by Vitor Bosshard 12 Oct '08

12 Oct '08

> On Fri, Oct 3, 2008 at 12:33 PM, Andreas Nilsson wrote: > > Thanks for the pointer! > > I don't buy the argument that newlines automagically improves readability > > though. You also get increased nesting suggesting something interesting is > > happening where it isn't and that hurts readability. > > And as Vitor said, all other constructions of the form 'for i in items' can > > have if-conditions attached to them, it's really not that far-fetched to > > assume that the loop behaves the same way. Consistency good, surprises bad. > > Yeah, I know what you mean, and I kind of liked the idea of adding the > if statement to the for loop (for consistency, if nothing else), but > it's been discussed before, and plenty of people have made the same > argument. Probably not worth it. Besides consistency I think the one major benefit to for..if loops is that often you don't just save a line, but also an indentation level (whenever you use the if clause solely as a filter), which actually increases readability, specially when whatever you do within the loop is relatively long, with its own indentations. The syntax just feels natural. For example: for i in somelist if i.pending: <do stuff> I really don't see any disadvantage here. Vitor PS: moving the discussion from python-dev to python-ideas. ____________________________________________________________________________________ ¡Todo sobre Amor y Sexo! La guía completa para tu vida en Mujer de Hoy. http://mujerdehoy.telemundo.yahoo.com/

3 4

Re: [Python-ideas] For-loop variable scope: simultaneous possession and ingestion of cake
by Jan Kanis 10 Oct '08

10 Oct '08

On 10/10/2008, Bruce Leban <bruce(a)leapyear.org> wrote: > > > On Thu, Oct 9, 2008 at 3:06 AM, Jan Kanis <jan.kanis(a)phil.uu.nl> wrote: > > > > On 08/10/2008, Bruce Leban <bruce(a)leapyear.org> wrote: > > > I don't think this is better. Not that I'm proposing we add macros to > the > > > language but if we did then > > > > > > macro find(a, x, i): > > > for i in range(len(x)): > > > if x[i]: > > > return lambda: x[i] > > > > > > would operate differently for find(a, x, i) and find(a, x, i[0]). I > think > > > that's a bad idea. > > > OK, I screwed up that example. > > macro find(a, x, i): > for i in range(len(x)): > if x[i]: > z.append(lambda: x[i]) Yes, that one would behave differently... so IF macros were added (which they won't), and IF someone would find it usefull to pass in the index variable through the macro, instead of just using a regular local variable and not exposing it (for which I don't see any use), this macro would have to be SLIGHTLY REWRITTEN. (And as there are no macros yet, there's no backward compatibility problem). New version: macro find(a, x, i): for j in range(len(x)): i = j # but why would you want that? if x[j]: z.append(lambda: x[j]) Jan

1 0

Py3k invalid unicode idea
by Dillon Collins 10 Oct '08

10 Oct '08

I don't know an awful lot about unicode, so rather than clog up the already lengthy threads on the 3k list, I figured I'd just toss this idea out over here. As I understand it, there is a fairly major issue regarding malformed unicode data being passed to python, particularly on startup and for filenames. This has lead to much discussion and ultimately the decision (?) to mirror a variety of OS functions to work with both bytes and unicode. Obviously this puts us on a slope of questionable friction to reverting back to 2.x where unicode wasn't "core". My thought is this: When passed invalid unicode, keep it invalid. This is largely similar to the UTF-8b ideas that were being tossed around, but a tad different. The idea would be to maintain invalid byte sequences by use of the private use area in the unicode spec, but be explicit about this conversion to the program. In particular, I'm suggesting the addition of the following (I'll use "surrogate" to refer to the invalid bytes in a unicode string): 1) Encoding 'raw'. Force all bytes to be converted to surrogate values. Decoding to raw converts the bytes back, and gives an error on valid unicode characters(!). This would enable applications to effectively interface with the system using bytes (by setting default encoding or the like), but not require any API changes to actually support the bytes type. 2) Error handler 'force' (or whatever). For decoding, when an invalid byte is encountered, replace with a surrogate. For encoding, write the invalid byte. 2a) Decoding invalid unicode or encoding a string with surrogates raises a UnicodeError (unless handler 'force' is specified or encoding is 'raw'). 3) string method 'valid'. 'valid()' would return False if the string contains at least one surrogate and True otherwise. This would allow programs to check if the string is correct, and handle it not. This would be of particular value when reading boot information like sys.argv as that would use the 'force' error handler in order to prevent boot failure. How the invalid bytes would be stored internally is certainly a matter of hot debate on the 3k list. As I mentioned before, I am not intimately familiar with unicode, so I don't have much to suggest. If I had to implement it myself now, I'd probably use a piece of the private use area as an escape (much like '\\' does). Finally, there seems to be much concern about internal invalid unicode wreaking havoc when tossed to external programs/libraries. I have to say that I don't really see what the problem is, because whenever python writes unicode, oughtn't it be buffered by "encode"? In that case you'd either get an error or would be explicitly allowing invalid strings (via 'raw' or 'force'). And besides, if python has to deal with bad unicode, these libraries should have to too ;). Even more finally, let me apologize in advance if I missed something on another list or this is otherwise too redundant.

3 5