Mailman 3 May 2009 - Python-ideas

Re: [Python-ideas] 80 character line width vs. something wider
by Raymond Hettinger May 22, 2009

May 22, 2009

> I consider that a critical clue that I should consider refaceoring. > Not just the one long line, but the entire structure that resulted in > that deep nesting. After you get the class and method def in, each > level should indicate a possible control flow branch. So deeply > nested implies lots of branches implies a complicated structure. > Which means it's time to consider refactoring to simplify it. Okay, I'm convinced. 80 is EXACTLY the right number. 79 … [View More]

6 10

Re: [Python-ideas] 80 character line width vs. something wider
by Aaron Rubin May 22, 2009

May 22, 2009

Yes, I used your information. Feel free to add more information gleaned from it to the list of facts. If your numbers weren't accurate, then it would be nice to get an accurate number. Perhaps Greg Ewing might already have something useful for us to use. I started to write a script to analyze the standard library and quickly realized I should be using a lexical analyzer to weed out comments and (funny enough) wrapped lines themselves from the analysis. I took your numbers as a rough fact, … [View More]

1 0

Default arguments in Python - the return
by Pascal Chambon May 20, 2009

May 20, 2009

Hello, I'm surely not original in any way there, but I'd like to put back on the table the matter of "default argument values". Or, more precisely, the "one shot" handling of default values, which makes that the same mutable objects, given once as default arguments, come back again and again at each function call. They thus become some kinds of "static variables", which get polluted by the previous calls, whereas many-many-many python users still believe that they get a fresh new value … [View More]at each function call. I think I understand how default arguments are currently implemented (and so, "why" - technically - it does behave this way), but I'm still unsure of "why" - semantically - this must be so. I've browsed lots of google entries on that subject, but as far as I'm concerned, I've found nothing in favor current semantic. I've rather found dozens, hundreds of posts of people complaining that they got biten by this gotcha, many of them finishing with a "Never put mutable values in default arguments, unless you're very very sure of what you're doing !". And no one seemed to enjoy the possibilities of getting "potentially static variables" this way. Static variables are imo a rather bad idea, since they create "stateful functions", that make debugging and maintenance more difficult ; but when such static variable are, furthermore, potentially non-static (i.e when the corresponding function argument is supplied), I guess they become totally useless and dangerous - a perfect way to get hard-to-debug behaviours. On the other hand, when people write "def func(mylist=[]):", they basically DO want a fresh new list at each call, be it given by the caller or the default argument system. So it's really a pity to need tricks like >/ def f(a, L=None): />/ if L is None: />/ L = [] /to get what we want (and if None was also a possible value ? what other value should we put as a placeholder for "I'd like None or a fresh new list but I can't say it directly ?"). So I'd like to know : are there other "purely intellectual" arguments for/against the current semantic of default arguments (I might have missed some discussion on this subject, feel free to point them ? Currently, this default argument handling looks, like a huge gotcha for newcomers, and, I feel, like an embarrassing wart to most pythonistas. Couldn't it be worth finding a new way of doing it ? Maybe there are strong arguments against a change at that level ; for example, performance issues (I'm not good in those matters). But I need to ensure. So here are my rough ideas on what we might do - if after having the suggestions from expert people, it looks like it's worth writting a PEP, I'll be willing to particpateon it. Basically, I'd change the python system so that, when a default argument expression is encountered, instead of being executed, it's wrapped in some kind of zero-argument lambda expression, which gets pushed in the "func_defaults" attribute of the function. And then, each time a default argument is required in a function call, this lambda expression gets evaluated and gives the expected value. I guess this will mean some overhead during function call, so this might become another issue. It's also a non retrocompatible change, so I assume we'd have to use a "from __future__ import XXX" until Python4000. But I think the change is worth the try, because it's a trap which waits for all the python beginners. So, if this matters hasn't already been marked somewhere as a no-go, I eagerly await the feedback of users and core developpers on the subject. :) By the way, I'm becoming slightly allergical to C-like languages (too much hassle for too little gain, compared to high level dynamic languages), but if that proposition goes ahead, and no one wants to handle the implementation details, I'll put the hands in the engine ^^ Regards, Pascal [View Less]

25 123

Add a "archive_tree" in the tarfile module
by Tarek Ziadé May 18, 2009

May 18, 2009

Hello, It happened to me several time : re-write a small function to archive a directory, to mimic "tar -xzvf archive.tgz directory" What about adding in tarfile this small function: def archive_tree(archive_name, src_dir [[, add_empty_dir], callable]) -> creates an archive that contains the files in directory, by recursively walking into it. if add_empty_dir is set to False (default: True), empty directory founded will not be added if … [View More]

4 3

Re: [Python-ideas] func attribute & default arg
by Nick Coghlan May 17, 2009

May 17, 2009

spir wrote: > Le Sat, 16 May 2009 17:53:09 +1000, Nick Coghlan <ncoghlan(a)gmail.com> > s'exprima ainsi: > >> Since option 1 will work for any arbitrary object, while option 2 >> is limited to objects which support copy.copy (or copy.deepcopy, >> depending on the copying semantics chosen), the "late evaluation of >> default arguments" (option 1) approach is the more promising angle >> of attack by far. > > Which objects do not support copy.… [View More]

1 0

func attribute & default arg
by spir May 16, 2009

May 16, 2009

Hello, I think there is some confusion on the topic of static variables in relation to default arguments and function attributes. Or, maybe, my point of view is different from the one of the majority on python-ideas. (a rather long post) To sum up, I think the core of the issue is not about static/dymamic evaluation, or compile time/run time; rather it lies in the fact that python internally does not separate a default *value* from the corresponding local *variable*. What may be, for the … [View More]programmer, the meaning and purpose of a default argument? I think there is nothing misleading in the fact that it is evaluated at definition time. Instead, this is precisely the way we see them, otherwise there would be no gotcha and we would not discuss. This semantics is properly denoted, I guess, by a default beeing part of the function definition's headline: def f(arg, default=value): ....... Once set, a default becomes a kind of constant attribute of the function. So that the gotcha happens precisely when this expectation is deceived because the local variable called 'default' may change back the previously defined value. To address this issue, having a default value recomputed on every call is not only overkill, but also semantically aside the point. What we need is only *caching* a default value while the function def is first evaluated. A possible implementation (probably requiring few changes) may be to add a kind of __defaults__ dict attribute to functions. This would be filled at definition time and used at call time. Even if probably not needed, for python provides better ways, having it as an available attribute also lets programmers tweak a func's guts if they like it -- e.g. for runtime updatable defaults: func.__defaults__["default_arg"] = newDefaultValue() With this semantics, we are free to use and update local variables safely, without worrying of changing the default value. Here is a short and probably non-exhaustive review of common uses of defaults. -1- real default arg which value is immutable No issue. -2- real default arg which value is mutable but not updated in func body No issue. With the above proposal, we do not need to worry anyway; and we can change the code later safely to case -3-. I find this important. -3- real default arg which value is mutable and updated in func body Present implementation: gotcha. Proposal: default argument and local var point to distinct values. No more issue. Possibly use a __defaults__ dict attribute. -4- default used as static var A common use of mutable defaults is a workaround for python seemingly not having static variables. Actually, python programmers have them in the form of function attributes. Since this feature seems +/- unknown (undocumented?), some programmers write for instance: def f(arg, l=[]): <update l> while the following better expresses the semantics, imho, because l is not a real argument: def f(arg): <update f.l> f.l = [] Generators can also be used in cases where the purpose of the func is (mainly) to return successive values. def f(arg): l = [] while True: <update l> yield something -5- func factory In the case of a factory, default arguments are used as function definition *parameters*. Similarly to the previous case, they are rather function attributes, with the difference that this time they are constant ones. Anyway, the proposal does not change the semantics here because each generated function has its own parameter. Still, a common idiom using a dummy default, such as: def paramPower(exponent): def power(number, exponent=exponent): return number ** exponent return power would be better expressed, without any trick: def paramPower(exponent): def power(number): return number ** power.exponent power.exponent = exponent return power -6- variable defaults If we want a runtime evaluation of a default argument, we usually express it in the function body and use a sentinel. This would not change, and anyway it seems to make the intent clearer than in the case where the default would be silently recomputed (read: "explicit better than implicit"): from random import random # provides runtime default number_count = 3 # may also be variable UNDEF = object() def f(i, numbers=UNDEF): # case numbers not provided: must be evaluated at runtime if numbers is UNDEF: numbers = [int(random()*10) for i in range(number_count)] # change pointed number numbers[i] = int(random()*10) print numbers numbers = [1,2,3] f(0, numbers) ; f(2, numbers) ; f(0) ; f(2) ==> [8, 2, 3] [8, 2, 2] [5, 3, 5] [8, 4, 1] Using the proposal of a default attribute, an alternative would be to re-compute the value in calling code -- only when needed: def f(i, numbers=[random() for i in range(number_count)]): # change pointed number numbers[i] = random() print numbers ....... if cond: f.__defaults__['numbers'] = [random() for i in range(number_count)] f(i) A transition phase could be planned toward cached defaults, possibly with proper warnings: "Warning: use of a mutable default value updated in the function body. From python 3.x, default values will be cached so as not to change between calls. If your intention is instead to have a kind of static variable, rather set an attribute on the function." Now, why are func attributes rarely used; while, as shown above, they provide a clear and handy way to express several semantic patterns -- and defaults are (mis)used instead? I wonder whether we could make them more obvious using a dedicated syntax. In the thread about default arguments, a proposed syntax, to have defaults dynamically reevaluated at each call (which I find *wrong*), was to reuse the keyword "yield". The meaning of this idiom is rather for me the creation of some data at definition time. As a consequence, I found it a rather good possibility to denote the creation of a func attribute. : def f(arglist) yield attr=val: for more attributes, reuse import-like syntax: def f(arglist) yield attr1=val1, attr2=val2...: Having them defined outside the parens properly differenciate arguments from attributes (and the content of parens is already complicated enough with defaults and '*' and '**' thingies). Also, the "yield keyword not only suggests the creation of data, but also is used in contexts (generators) where state is kept between function calls. For instance, the following code using a func with a custom attribute: startup_numbers = [4, 5] def f(number, sorting=False): f.numbers.append(number) if sorting: f.numbers.sort() print f.numbers f.numbers = startup_numbers could be expressed with a func def header as follows: def f(number, sorting=False) yield numbers=startup_numbers: without the need to separately set the attribute on the func. Sure, we do not need that at all. Moreover, when the func happens to be a method instance, then the attribute better fits on the instance object. I'm just looking for a way to let programmers be (more) aware of func attributes. To have cached default solving the common gotcha, we need python programmers to use defaults only as defaults, and use other solutions for semantics such as static vars. So that the transition be easier. Denis ------ la vita e estrany [View Less]

5 5

Re: [Python-ideas] Default arguments in Python - the return - running out of ideas but...
by Pascal Chambon May 15, 2009

May 15, 2009

Greg Falcon a écrit : > Notice that > > def foo(): > def myfunc(a, b, c = (yield [])): > pass > > already has a meaning in Python. Under your proposal, > > Aowh... I didn't even think this was legal, but indeed this works... so it's obviously dead for "yield". > Anyhow, since you've decided you want add a new kind of default > argument spelled differently than the standard one, I have to ask what > this buys you beyond > > def foo(): &… [View More]

9 12

Off topic but easy question
by Tennessee Leeuwenburg May 15, 2009

May 15, 2009

Hi all, I'm trying to find a web link describing the voting system of +1, +0, -0, -1 to show some people at work, but my Google-fu is weak today. Can someone point me in the right direction? Thanks, -T -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

2 2

Add a "sitedirs" list in the site module
by Tarek Ziadé May 14, 2009

May 14, 2009

Hello Right now there's no way to differenciate in sys.path the directories that where added through site.addsitedirs calls, (like when python starts up and initializes site-packages directories) Anyone can change sys.path, and add elements in them. So there's no way to answer to this simple question for sure : "what are the site-packages directories ?" What about keeping track of all directories added through site.addsitedirs calls, with a simple list called "sitedirs" This list would … [View More]

2 1

Re: [Python-ideas] Default arguments in Python - the return - running out of ideas but...
by Bruce Leban May 14, 2009

May 14, 2009

Sure you can do it with a decorator. Although you might want one default to have this behavior and one not to. And putting the * right next to the lambda makes this a bit more explicit to my eye. --- Bruce Georg Brandl <g.brandl(a)gmx.net> wrote: >Bruce Leban schrieb: >> Here's what I'd like: >> >> def myfunc(a, b, c = *lambda: expression): >> stuff >> >> The use of the lambda keyword here makes the scope of any variables in >> the … [View More]

1 0