Mailman 3 October 2008 - Python-ideas

Recognise __init__.so
by Greg Ewing Oct. 31, 2008

Oct. 31, 2008

We've just been having a discussion on the Cython list about implementing the __init__ file of a package as an extension module. It turns out that just putting an __init__.so file in a directory isn't enough to make Python recognise it as a package. However, if there is both an __init__.py and an __init__.so, it's recognised as a package, and the __init__.so gets loaded. So it's possible, but only by exploiting some undocumented and probably accidental behaviour. So how about officially … [View More]

2 1

Idea: Lazy import statement
by Greg Ewing Oct. 23, 2008

Oct. 23, 2008

Problem: You have a package containing a large number of classes, of which only a few are typically used by any given application. If you put each class into its own submodule, then client code is required to use a lot of tedious 'from foo.thingy import Thingy' statements to import the classes it wants to use. This also makes all the submodule names part of the API and makes it hard to rearrange the packaging without breaking code. If you try to flatten the namespace by importing all the … [View More]

7 8

Is this the right forum for a proposed new stdlib function?
by Joe Strout Oct. 22, 2008

Oct. 22, 2008

I'd like to float the idea of an addition of a new function to the Template class (in the string module). I'm a bit of a newbie around here, though, and uncertain of the proper procedure. Is this the right mailing list for that, or should I be using python-list instead? Thanks, - Joe

7 10

Optimistic Concurrency
by skip＠pobox.com Oct. 21, 2008

Oct. 21, 2008

Is optimistic concurrency http://en.wikipedia.org/wiki/Optimistic_concurrency_control a possible option for removing the GIL in Python? Skip

7 22

Python's core needs to be more lightweight
by Victor Martin Ulloa Oct. 20, 2008

Oct. 20, 2008

I noticed a minimalistic pytoncore compiled with VS2005 ( eliminating COMDAT and un-used references, optimized for size ) passes the 2Mb of .LIB ( statically linked ). I think that's a lot considering that LUA uses less than 100Kb. For small game consoles and devices very small RAM and storage would be very good to reduce this size. thx __________________________________________________ Correo Yahoo! Espacio para todos tus mensajes, antivirus y antispam ¡gratis! Regístrate ya - http://correo.yahoo.es

2 1

Re: [Python-ideas] VS2005 project improvement
by Victor Martin Ulloa Oct. 19, 2008

Oct. 19, 2008

I see this code in the python3.0rc1 makebuild_info project for VS2005: int main(int argc, char*argv[]) { char command[500] = "cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL "; int do_unlink, result; if (argc != 2) { fprintf(stderr, "make_buildinfo $(ConfigurationName)\n"); return EXIT_FAILURE; } if (strcmp(argv[1], "Release") == 0) { strcat_s(command, CMD_SIZE, "-MD "); } else if (strcmp(argv[1], "Debug") == 0) { strcat_s(… [View More]

1 0

User forums
by Victor Martin Ulloa Oct. 18, 2008

Oct. 18, 2008

Why are we using a 90's mailing list? Why not a modern phpBB web forum? __________________________________________________ Correo Yahoo! Espacio para todos tus mensajes, antivirus y antispam ¡gratis! Regístrate ya - http://correo.yahoo.es

6 5

Re: [Python-ideas] VS2005 project improvement
by Victor Martin Ulloa Oct. 17, 2008

Oct. 17, 2008

>I see project files for VS6, VC7.1 and VS8 in the directory >http://svn.python.org/projects/python/trunk/PC/ :) But the Solution files(.SLN) in the PC folder are just for VS2008. __________________________________________________ Correo Yahoo! Espacio para todos tus mensajes, antivirus y antispam ¡gratis! Regístrate ya - http://correo.yahoo.es

2 1

Re: [Python-ideas] VS2005 project improvement
by Victor Martin Ulloa Oct. 17, 2008

Oct. 17, 2008

And btw, to build static libraries is a nightmare :P You must add Debug_Static and Relese_Static configurations too. __________________________________________________ Correo Yahoo! Espacio para todos tus mensajes, antivirus y antispam ¡gratis! Regístrate ya - http://correo.yahoo.es

1 0

Automatic total ordering
by George Sakkis Oct. 17, 2008

Oct. 17, 2008

Now that 3.x fixes the arbitrary object comparison wart and drops (?) __cmp__, it seems it's a good time to do something with the missing rich comparators gotcha, e.g. given a class that defines __eq__ and __lt__, automatically provide the rest missing comparisons. Yes, it can be done with a custom metaclass or (in 2.6+) with a class decorator [1] but (a) 99% of the time that's what one expects so "explicit is better than implicit" doesn't count and (b) a bulitin implementation might well be … [View More]

11 24

Re: [Python-ideas] Idea: Lazy ... statement
by Jim Jewett Oct. 16, 2008

Oct. 16, 2008

On Mon, Oct 13, 2008 at 4:12 AM, Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > So I think it would be good to have a dedicated syntax > for lazy imports, so the top-level foo package can say > something like > from foo.thing lazily import Thing > from foo.stuff lazily import Stuff Since this can be (awkwardly) done now, I don't think the problem is big enough for another keyword. On the other hand, a "lazy" keyword that worked in other contexts might have some … [View More]

4 6

Re: [Python-ideas] Automatic total ordering
by Terry Jones Oct. 15, 2008

Oct. 15, 2008

Oops..... didn't see your footnote URL. Sorry! Terry

1 0

Re: [Python-ideas] Automatic total ordering
by Terry Jones Oct. 15, 2008

Oct. 15, 2008

>>>>> "George" == George Sakkis <george.sakkis(a)gmail.com> writes: George> Now that 3.x fixes the arbitrary object comparison wart and drops George> (?) __cmp__, it seems it's a good time to do something with the George> missing rich comparators gotcha, e.g. given a class that defines George> __eq__ and __lt__, automatically provide the rest missing George> comparisons. Note http://www.voidspace.org.uk/python/weblog/arch_d7_2008_10_04.shtml#e1018 Regards, Terry

1 0

VS2005 project improvement
by Victor Martin Ulloa Oct. 15, 2008

Oct. 15, 2008

I'm trying to comppile python 2.5 as a static library using VS2005 SP1. I realized you only have Debug, PG and Release configurations. Of course, like you provide the sources, I could modify it manually... but I think will be much better to provide the Debug_Static and Release_Static configurations for noob user like me :P Also, you are using this code in the pyconfig.h: /* For an MSVC DLL, we can nominate the .lib files used by extensions */ #ifdef MS_COREDLL # ifndef Py_BUILD_CORE /* not … [View More]

3 2

For-loop variable scope: simultaneous possession and ingestion of cake
by Greg Ewing Oct. 15, 2008

Oct. 15, 2008

There's been another discussion on c.l.py about the problem of lst = [] for i in range(10): lst.append(lambda: i) for f in lst: print f() printing 9 ten times instead of 0 to 9. The usual response is to do lst.append(lambda i=i: i) but this is not a very satisfying solution. For one thing, it's still abusing default arguments, something that lexical scoping was supposed to have removed the need for, and it won't work in some situations, such as if the function needs … [View More]to take a variable number of arguments. Also, most other languages which have lexical scoping and first-class functions don't seem to suffer from problems like this. To someone familiar with one of those languages (e.g. Scheme, Haskell) it looks as if there's something broken about the way scoping of nested functions works in Python. However, it's not lambda that's broken, it's the for loop. In Scheme, for example, the way you normally write the equivalent of a Python for-loop results in a new scope being created for each value of the loop variable. Previous proposals to make for-loop variables local to the loop have stumbled on the problem of existing code that relies on the loop variable keeping its value after exiting the loop, and it seems that this is regarded as a desirable feature. So I'd like to propose something that would satisfy both requirements: 0. There is no change if the loop variable is not referenced by a nested function defined in the loop body. The vast majority of loop code will therefore be completely unaffected. 1. If the loop variable is referenced by such a nested function, a new local scope is effectively created to hold each successive value of the loop variable. 2. Upon exiting the loop, the final value of the loop variable is copied into the surrounding scope, for use by code outside the loop body. Rules 0 and 1 would also apply to list comprehensions and generator expressions. There is a very simple and efficient way to implement this in current CPython: If the loop variable is referenced by a nested function, it will be in a cell. Instead of rebinding the existing cell, each time around the loop a new cell is created, replacing the previous cell. Immediately before exiting the loop, one more new cell is created and the final value of the loop variable copied into it. Implementations other than CPython that aren't using cells may need to do something more traditional, such as compiling the loop body as a separate function. I think this arrangement would allow almost all existing code to continue working, and new code to be written that takes advantage of final values of loop variables. There would be a few obscure situations where the results would be different, for example if a nested function modifies the loop variable and expects the result to be reflected in the value seen from outside the loop. But I can't imagine such cases being anything other than extremely rare. The benefit would be that almost all code involving loops and nested functions would behave intuitively, Python would free itself from any remaining perception of having broken scope rules, and we would finally be able to consign the default-argument hack to the garbage collector of history. -- Greg [View Less]

14 51

Re: [Python-ideas] Where/how to propose an addition to a standard module?
by Joe Strout Oct. 14, 2008

Oct. 14, 2008

On Oct 13, 2008, at 8:46 AM, pruebauno(a)latinmail.com wrote: > Whenever I needed such functionality I used the re module. The benefit > is that it uses unix style regular expression syntax and an egrep/awk/ > perl/ruby user can understand it. You should show a few examples where > your proposal looks better than just using RE. Well, I suppose if you're already used to RE, then maybe it's not obvious that to an RE newbie, this: regex = re.compile("The (?P<object>.*?) in … [View More](?P<location>.*?) falls mainly in the (?P<subloc>.*?).") d = regex.match(text).groupdict() is far harder to read and type correctly than this: templ = Template("This $object in $location falls mainly in the $subloc") d = templ.match(text) Any other example would show the same simplification. Of course, if you're the sort of person who uses RE, you probably don't use Template.substitute either, since you probably like and are comfortable with the string % operator. But Template.substitute was introduced to make it easier to handle the common, simple substitution operations, and I believe adding a Template.match method would do the same thing for common, simple matching operations. Here's a more fleshed-out proposal, with rationale and references -- see if this makes it any clearer why I think this would be a fine addition to the Template class. Abstract Introduces a new function on the string.Template [1] class, match(), to perform the approximate inverse of the existing substitute() function. That is, it attempts to match an input string against a template, and if successful, returns a dictionary providing the matched text for each template field. Rationale PEP 292 [2] added a simplified string substitution feature, allowing users to easily substitute text for named fields in a template string. The inverse operation is also useful: given a template and an input string, one wishes to find the text in the input string matching the fields in the template. However, Python currently has no easy way to do it. While this named matching operation can be accomplished using RegEx, the constructions required are somewhat complex and error prone. It can also be done using third-party modules such as pyparse, but again the setup requires more code and is not obvious to programmers inexperienced with that module. In addition, the Template class already has all the data needed to perform this operation, so it is a natural fit to simply add a new method on this class to perform a match, in addition to the existing method to perform a substitution. Proposal Proposed is the addition of one new function, on the existing Template class, as follows: def match(text, greedy=false) 'match' is a new function which accepts one required parameter, an input string; and one optional parameter, 'greedy', which determines whether matches should be done in a greedy manner, equivalent to regex pattern '(.*)'; or in a non-greedy manner, equivalent to '(.*?)'. If the input string can be matched to the template pattern (respecting the 'greedy' flag), then match returns a dictionary, where each field in the pattern maps to the corresponding part of the input string. If the input string cannot be matched to the template pattern, then match returns None. Examples: >>> from string import Template >>> s = Template('$name was born in ${country}') >>> print s.match('Guido was born in the Netherlands') {'name':'Guido', 'country':'the Netherlands'} >>> print s.match('Spam was born as a canned ham') None Note that when the match is successful, the resulting dictionary could be passed through Template.substitute to reconstitute the original input string. Conversely, any string created by Template.substitute could be matched by Template.match (though in unusual cases, the resulting dictionary might not exactly match the original, e.g. if the string could be matched in multiple ways). Thus, .match and .substitute are inverse operations. References [1] Template Strings http://www.python.org/doc/2.5.2/lib/node40.html [2] PEP 292: Simpler String Substitutions http://www.python.org/dev/peps/pep-0292/ [View Less]

4 5

Using Git to Manage Python Installation(s)
by Jeet Sukumaran Oct. 13, 2008

Oct. 13, 2008

I've recently started using Git to manage the Python installations on my system (2.4, 2.5, 2.6 etc.),, and I have found that it has revolutionized my development ecosystem. Full versioning + documentation of all changes to each of the frameworks (including reversion in case something breaks), multiple parallel variants/package combinations of each installation (including a clean vanilla build) with seamless in-situ switching among them, easy instantiation and disposal of sandbox/experimental … [View More]

2 1

Re: [Python-ideas] if-syntax for regular for-loops
by Vitor Bosshard Oct. 13, 2008

Oct. 13, 2008

> On Fri, Oct 3, 2008 at 12:33 PM, Andreas Nilsson wrote: > > Thanks for the pointer! > > I don't buy the argument that newlines automagically improves readability > > though. You also get increased nesting suggesting something interesting is > > happening where it isn't and that hurts readability. > > And as Vitor said, all other constructions of the form 'for i in items' can > > have if-conditions attached to them, it's really not that far-fetched to … [View More]

3 4

Re: [Python-ideas] For-loop variable scope: simultaneous possession and ingestion of cake
by Jan Kanis Oct. 10, 2008

Oct. 10, 2008

On 10/10/2008, Bruce Leban <bruce(a)leapyear.org> wrote: > > > On Thu, Oct 9, 2008 at 3:06 AM, Jan Kanis <jan.kanis(a)phil.uu.nl> wrote: > > > > On 08/10/2008, Bruce Leban <bruce(a)leapyear.org> wrote: > > > I don't think this is better. Not that I'm proposing we add macros to > the > > > language but if we did then > > > > > > macro find(a, x, i): > > > for i in range(len(x)): > > > … [View More]

1 0

Py3k invalid unicode idea
by Dillon Collins Oct. 10, 2008

Oct. 10, 2008

I don't know an awful lot about unicode, so rather than clog up the already lengthy threads on the 3k list, I figured I'd just toss this idea out over here. As I understand it, there is a fairly major issue regarding malformed unicode data being passed to python, particularly on startup and for filenames. This has lead to much discussion and ultimately the decision (?) to mirror a variety of OS functions to work with both bytes and unicode. Obviously this puts us on a slope of … [View More]questionable friction to reverting back to 2.x where unicode wasn't "core". My thought is this: When passed invalid unicode, keep it invalid. This is largely similar to the UTF-8b ideas that were being tossed around, but a tad different. The idea would be to maintain invalid byte sequences by use of the private use area in the unicode spec, but be explicit about this conversion to the program. In particular, I'm suggesting the addition of the following (I'll use "surrogate" to refer to the invalid bytes in a unicode string): 1) Encoding 'raw'. Force all bytes to be converted to surrogate values. Decoding to raw converts the bytes back, and gives an error on valid unicode characters(!). This would enable applications to effectively interface with the system using bytes (by setting default encoding or the like), but not require any API changes to actually support the bytes type. 2) Error handler 'force' (or whatever). For decoding, when an invalid byte is encountered, replace with a surrogate. For encoding, write the invalid byte. 2a) Decoding invalid unicode or encoding a string with surrogates raises a UnicodeError (unless handler 'force' is specified or encoding is 'raw'). 3) string method 'valid'. 'valid()' would return False if the string contains at least one surrogate and True otherwise. This would allow programs to check if the string is correct, and handle it not. This would be of particular value when reading boot information like sys.argv as that would use the 'force' error handler in order to prevent boot failure. How the invalid bytes would be stored internally is certainly a matter of hot debate on the 3k list. As I mentioned before, I am not intimately familiar with unicode, so I don't have much to suggest. If I had to implement it myself now, I'd probably use a piece of the private use area as an escape (much like '\\' does). Finally, there seems to be much concern about internal invalid unicode wreaking havoc when tossed to external programs/libraries. I have to say that I don't really see what the problem is, because whenever python writes unicode, oughtn't it be buffered by "encode"? In that case you'd either get an error or would be explicitly allowing invalid strings (via 'raw' or 'force'). And besides, if python has to deal with bad unicode, these libraries should have to too ;). Even more finally, let me apologize in advance if I missed something on another list or this is otherwise too redundant. [View Less]

3 5

For-loop variable scope: simultaneous possession and ingestion of cake
by Carl Johnson Oct. 9, 2008

Oct. 9, 2008

Why does the evil default args hack work? Because it immediately evaluates the argument and stores the result into the lambda object's default values list. So, what we need is a keyword that means "immediately evaluate the argument and store the result, instead of looking up the name again later." Since I can't think of a good name for this, I will use a terrible name for it, "immanentize." >>> i = 1 >>> def f(): ... return immanentize i ... >>> i = … [View More]2 >>> f() 1 >>> lst = [] >>> for i in range(10): ... lst.append(lambda: immanentize i) ... >>> [f() for f in _] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> def side_effect_maker(): ... print("blah") ... return datetime.datetime.now() ... >>> def f(): ... creation_time = immanentize side_effect_maker() ... return datetime.datetime.now() - creation_time ... "blah" >>> f() #Assume it takes you 5 seconds to type "f()" and press enter datetime.timedelta(0, 5, 0) Etc. Behind the scenes, this would look like syntatic sugar for something like creating a new variable name, evaluating the expression at initial compile time, setting the variable name to be the result of the evaluation, and replacing the immanentize expression with the variable name. Like this: >>> lst = [] >>> for i in range(10): ... random_variable_name_0..9 = i ... # ie. on the first loop around random_variable_name_0 = 0, on the second loop random_variable_name_1 = 1, etc. ... lst.append(lambda: random_variable_name_0..9) ... # ie. the first loop is evaluated as lst.append(lambda(random_variable_name_0)), etc. ... >>> [f() for f in _] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> def side_effect_maker(): ... print("blah") ... return datetime.datetime.now() ... >>> random_variable_name_number_0 = side_effect_maker() "blah" >>> def f(): ... creation_time = random_variable_name_number_0 ... return datetime.datetime.now() - creation_time I think this proposal is better than using "local:" or "scope:" because it doesn't create nested blocks when you just want to freeze out one particular value. One interesting side effect of having an immanentize keyword is that in Python 4000, we could (if we wanted to) get rid of the supposed "wart" of having x=[] as a default arg leading to unexpected results for Python newbies. Just make it so that to get the current behavior you type >>> def f(x=immanentize []): ... x.append(1) ... return x ... >>> f() [1] >>> f() [1, 1] Whereas, without immanentize we can do what newbies expect and evaluate the defaults afresh each time. >>> def f(x=[]): ... x.append(1) ... return x ... >>> f() [1] >>> f() [1] Obviously, "immanentize" is a terrible name, and only just barely a word of English. Perhaps we could call it an "anonymous variable" instead? -- Carl [View Less]

8 16

if-syntax for regular for-loops
by Andreas Nilsson Oct. 3, 2008

Oct. 3, 2008

Hi. I'm reposting this here after erroneously posting it on python-dev. I use list comprehensions and generator expressions a lot and lately I've found myself writing a lot of code like this: for i in items if i.some_field == some_value: i.do_something() Naturally it won't work but it seems like a pretty straight-forward extension to allow compressing simple loops to fit on one line. The alternative, in my eyes, suggests there's something more happening than a simple include-test … [View More]

2 1