Mailman 3 April 2011 - Python-ideas

Re: [Python-ideas] [Python-Dev] python and super
by Carl Meyer 16 Apr '11

16 Apr '11

(Moving this thread to python-ideas, as it seems better suited for here). On 04/15/2011 11:02 AM, Michael Foord wrote: > Well yes, but it is also a bug in the copy of unittest2 embedded in > django - so whilst it can be fixed in unittest2 (simply deleting the > setUp and tearDown methods which do nothing but override > unittest.TestCase.setUp and tearDown) it *also* needs to be fixed in > django. Yup. > This particular issue does illustrate the problem well though - the > methods in unittest2 don't call up to their parent class (which is fine > because those methods are empty), but in not calling up also they > prevent sibling methods being called in a multiple inheritance situation. > > So for those who have been saying that not wanting to call up to parents > is a valid use case, yes I quite agree. But you have to be aware that > because of the semantics of super, not calling up to your parents > basically prevents those methods being used in the presence of multiple > inheritance. Right. I agree that the current behavior of super in that regard is surprising, but I don't see how the "go ahead and call sibling methods anyway" fix is plausible. What happens to the return value of that call? There's nowhere for it to go - super() would have to somehow implicitly integrate two different return values, which is all kinds of bad magic. Consider these classes: class UncooperativeBase: def method(self): # look Ma, no super() call! return ["UncooperativeBase"] class OtherBase: def method(self): return ["OtherBase"] + super().method() class Child(UncooperativeBase): def method(self): return ["Child"] + super().method() class GrandChild(Child, OtherBase): def method(self): return ["GrandChild"] + super().method() Currently, OtherBase.method() is never called, because UncooperativeBase.method() breaks the super() chain. Any proposal to have super() ensure that OtherBase.method() is called needs to explain what exactly happens to its return value. This would normally be handled explicitly by UncooperativeBase, since it would call super() and do something with the return value. But with the explicit chain broken, you end up with two different chains, and at some point their return values would need integrating. Carl

3 3

Copy-on-write when forking a python process
by jac 16 Apr '11

16 Apr '11

Hi all, Sorry for cross posting, but I think that this group may actually be more appropriate for this discussion. Previous thread is at: http://groups.google.com/group/comp.lang.python/browse_thread/thread/1df510… I am wondering if anything can be done about the COW (copy-on-write) problem when forking a python process. I have found several discussions of this problem, but I have seen no proposed solutions or workarounds. My understanding of the problem is that an object's reference count is stored in the "ob_refcnt" field of the PyObject structure itself. When a process forks, its memory is initially not copied. However, if any references to an object are made or destroyed in the child process, the page in which the objects "ob_refcnt" field is located in will be copied. My first thought was the obvious one: make the ob_refcnt field a pointer into an array of all object refcounts stored elsewhere. However, I do not think that there would be a way of doing this without adding a lot of complexity. So my current thinking is that it should be possible to disable refcounting for an object. This could be done by adding a field to PyObject named "ob_optout". If ob_optout is true then py_INCREF and py_DECREF will have no effect on the object: from refcount import optin, optout class Foo: pass mylist = [Foo() for _ in range(10)] optout(mylist) # Sets ob_optout to true for element in mylist: optout(element) # Sets ob_optout to true Fork_and_block_while_doing_stuff(mylist) optin(mylist) # Sets ob_optout to false for element in mylist: optin(element) # Sets ob_optout to false I realize that using shared memory is a possible solution for many of the situations one would wish to use the above solution, but I think that there are enough situations where one wishes to use the os's cow mechanism and is prohibited from doing so to warrant a fix. Has anyone else looked into the COW problem? Are there workarounds and/or other plans to fix it? Does the solution I am proposing sound reasonable, or does it seem like overkill? Does anyone see any (technical) problems with it? Thanks, --jac

9 19

PEP-3150
by Carl M. Johnson 15 Apr '11

15 Apr '11

Can we start a new thread just for discussion of PEP-3150? I personally don't like any of the proposals to add assignment to list comprehensions/generator expressions and I think that no matter how it's spelt the idea is a misfeature since there's no need to cram more info into a one-liner list comp/gen ex, but I do find the given-clause very interesting and potentially useful. In terms of positives for the feature, assignment in a list comp/gen ex has the potential to save a few characters and maybe be more readable (although I haven't seen any proposal that strikes me as more readable yet), but PEP-3150 has the four different benefits listed previously by Nick, most notably that it creates a temporary namespace (a honking great idea!). So, it strikes me that the two issues are different enough that it might be useful to separate the discussions out.

3 2

Re: [Python-ideas] Assignments in list/generator expressions
by Laura Creighton 15 Apr '11

15 Apr '11

In a message of Sun, 10 Apr 2011 20:18:35 EDT, Eugene Toder writes: <snip> >This particular feature is not the top on my wish list either. I just >don't see a good explanation for why it's missing. It's a lesser-known >feature in Haskell (no mention in Wikipedia! :-), but it is handy at >times. As mentioned above, people suggest it for Python from time to >time. <snip> The problem, as I see it is that you see it as ``missing''. You have some internal model of computer languages and not having it disagrees with your notion of 'completeness' or 'symmetry' or 'consistency' or something along those lines. But the reason I was drawn to python in the first place was that it was an elegant language, which in practice meant that it didn't have a lot of cruft I found in other languages. After all, if I had wanted to use those other languages, I know where to find them. And I actually enjoy using Haskell, on those very rare occasions when I have a problem that I find suited to the language. But turning Python more Haskell-like has no appeal to me. I need a set of very concrete use cases of 'I am trying to do this' and either 'I cannot' or 'I am doing it this way and oh, how slow/ugly/error prone/ it is!' before I am interested in a proposal. Change for the sake of completeness has no value to me. I may be blessed in that I live in Gothenburg, Sweden, where Chalmers University is one of the great centres of Haskell-admiration. So I have many friends who love the language so much they try to do everything in it. Debugging a failing webserver written in Haskell has been painful. There is real value in using whitespace and indentation to separate logical ideas, and jamming everything into a comprehension completely destroys this. The aesthetic gain in symmetry comes a very, very, distant second to me in this race. Laura

15 43

Re: [Python-ideas] PEP-3150
by Carl M. Johnson 15 Apr '11

15 Apr '11

On Wed, Apr 13, 2011 at 3:48 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Thu, Apr 14, 2011 at 8:33 AM, Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: >> assigned to the __doc__ attribute of whatever object the >> RHS expression returns. > > Ah, true, that would make a lot of sense. It also generalises nicely > to other cases like "return" and "yield" as well. I'll see if I can > thrash out something sensible along those lines. So, in the case of def decorator(f): return decorated given: "Docstring A..." def decorated(*args, **kwargs): "Docstring B..." do something... it would be docstring A that gets put on the decorated function, not docstring B? I guess I can kind of see the reasoning there, but it seems a little weird. Also, this case (and the general issue of callbacks, thunks, etc.) seems to be crying out for a special convenience syntax to save a level indenting. But would that just add needlessly to the complexity of the language? -- Carl

5 9

Assignments in list/generator expressions
by Mathias Panzenböck 12 Apr '11

12 Apr '11

I often have things like this: >>> ys = [f(x) for x in xs if f(x)] Obviously there is a redundant function call. You could rephrase that code to the following in order to avoid it: >>> ys = [y for y in (f(x) for x in xs) if y] But I think this is cumbersome and the extra generator overhead is unnecessary. So I propose this syntax: >>> ys = [f(x) as y for x in xs if y] It could be transformed to this: >>> ys = [] >>> for x in xs: >>> y = f(x) >>> if y: >>> ys.append(y) It's 6:18am here so I might not think straight and there is something fundamentally wrong with this. So please flame away. -panzi

13 24

Re: [Python-ideas] Assignments in list/generator expressions
by Laura Creighton 12 Apr '11

12 Apr '11

In a message of Sun, 10 Apr 2011 09:31:53 EDT, Eugene Toder writes: >> It isn't strangely missing at all, it's just hard: >> http://www.python.org/dev/peps/pep-3150/ > >Local definition in list comprehension is significantly simpler that >'given'. Semantically, 'with name = expr' can be treated as a more >readable form of 'for name in [expr]'. Implementation can use a simple >assignment. > >Eugene We'll have to disagree then. I think 'for name in [expr]' is a whole lot more readable than 'with name = expr'. Indeed, my recent experience in code reading had shown, I thought, that a good number of people are using the with statement whenever they possibly can, to the detriment of readability. Comments like this make me wonder if the problem is that those people actually find those constructs more readable. If we largely cannot agree on what makes for more readable code, then I think we have a different problem than the one I thought we had (people showing off their knowledge of how to use every new feature). Laura

6 12

Re: [Python-ideas] Assignments in list/generator expressions
by Laura Creighton 11 Apr '11

11 Apr '11

In a message of Sun, 10 Apr 2011 14:31:38 EDT, Eugene Toder writes: <snip> >It's well known that minor syntax details cause a lot of disagreement >(http://www.haskell.org/haskellwiki/Wadlers_Law), so that's not very >surprising. However, are you sure you're not confusing python's with >statement and 'with' word in my email, which is simply a placeholder >for appropriate keyword to mirror Haskell's 'let'? Or was this just an >occasion to complain about with statement abuse, unrelated to list >comprehension discussion? <snip> >Eugene I actually had something else in mind. One problem that you get whenever you extend a language 'i.e. make it more expressive' is that it now becomes possible for people to write old things they want to do in new ways. And, of course, this is what you want for the case of <whatever you want the more expressiveness for>, but it is difficult to may a syntax change that doesn't admit a whole lot of other possibilities as well. The argument I have heard around here is 'good programmers don't write unreadable code, as a matter of personal virtue'. This ignores the fact that I have to read a lot of code written by either not-so-good or not-so-virtuous programmers, but I am coming to the conclusion that the problem may be deeper than that. I now believe that some of the hard-to-read stuff I am reading is written by people who actually find this way of writing easier to read and understand than the sort I prefer. Thus I now think that 'expressiveness' is a limited human good, not an absolute one, (like health). You should want you language to be in the golden middle between 'not epressive enough' and 'too expressive'. If your language is sufficiently limited, then idioms will show up, and become widely adopted precisely because they are one of only a few ways (ideally the only way) to do it. This already pleases the people who find this way to do things readable, and those who would naturally pick a different way of doing things are stuck with it, and possibly over time and with familiarity will come to find this way readable as well. But with too much expressibility there comes a danger we will fragment into different sub-communities, those who think that syntax X is more readable than syntax Y and vice versa. There is no reason to believe that these linguistic communities will ever converge. This is the hidden cost of allowing more expressiveness in your language. Laura

3 2

AST Transformation Hooks for Domain Specific Languages
by Nick Coghlan 10 Apr '11

10 Apr '11

(Oops, let's try that again with the correct destination address this time...) A few odds and ends from recent discussions finally clicked into something potentially interesting earlier this evening. Or possibly just something insane. I'm not quite decided on that point as yet (but leaning towards the latter). Anyway, without further ado, I present: AST Transformation Hooks for Domain Specific Languages ====================================================== Consider: # In some other module ast.register_dsl("dsl.sql", dsl.sql.TransformAST) # In a module using that DSL import dsl.sql def lookup_address(name : dsl.sql.char, dob : dsl.sql.date) from dsl.sql: select address from people where name = {name} and dob = {dob} Suppose that the standard AST for the latter looked something like: DSL(syntax="dsl.sql", name='lookup_address', args=arguments( args=[arg(arg='name', annotation=<Normal AST for "dsl.sql.char">), arg(arg='dob', annotation=<Normal AST for "dsl.sql.date">)], vararg=None, varargannotation=None, kwonlyargs=[], kwarg=None, kwargannotation=None, defaults=[], kw_defaults=[]), body=[Expr(value=Str(s='select address\nfrom people\nwhere name = {name} and dob = {dob}'))], decorator_list=[], returns=None) (For those not familiar with the AST, the above is actually just the existing Function node with a "syntax" attribute added) At *compile* time (note, *not* function definition time), the registered AST transformation hook would be invoked and would replace that DSL node with "standard" AST nodes. For example, depending on the design of the DSL and its support code, the above example might be equivalent to: @dsl.sql.escape_and_validate_args def lookup_address(name: dsl.sql.char, dob: dsl.sql.date): args = dict(name=name, dob=dob) query = "select address\nfrom people\nwhere name = {name} and dob = {dob}" return dsl.sql.cursor(query, args) As a simpler example, consider something like: def f() from all_nonlocal: x += 1 y -= 2 That was translated at compile time into: def f(): nonlocal x, y x += 1 y -= 2 My first pass at a rough protocol for the AST transformers suggests they would only need two methods: get_cookie() - Magic cookie to add to PYC files containing instances of the DSL (allows recompilation to be forced if the DSL is updated) transform_AST(node) - a DSL() node is passed in, expected to return an AST containing no DSL nodes (SyntaxError if one is found) Attempts to use an unregistered DSL would trigger SyntaxError So there you are, that's the crazy idea. The stoning of the heretic may now commence :) Where this idea came from was the various discussions about "make statement" style constructs and a conversation I had with Eric Snow at Pycon about function definition time really being *too late* to do anything particularly interesting that couldn't already be handled better in other ways. Some tricks Dave Malcolm had done to support Python level manipulation of the AST during compilation also played a big part, as did Eugene Toder's efforts to add an AST optimisation step to the compilation process. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 16

Developer wish list
by geremy condra 10 Apr '11

10 Apr '11

Earlier this year there was some discussion[0] about putting up a page on the wiki where developers could list the feature proposals they most wanted and most hated for the benefit of those posting to python-ideas. It's taken me a while to get around to it, but I've put up a skeleton for the page at [1] and would love it if some of you guys would take a look, let me know what you like/don't like, and maybe even post a few of your pet projects or pet peeves. Thanks for your time and effort, Geremy Condra [0]: http://mail.python.org/pipermail/python-ideas/2011-March/009230.html [1]: http://wiki.python.org/moin/wishlist

1 0