[Python-ideas] Namespace creation with syntax short form

[Andrew Barnert]

On Feb 13, 2015, at 5:20, Peter Otten <__peter__ at web.de> wrote:

> Andrew Barnert wrote:
> 
>> On Feb 13, 2015, at 4:05, Peter Otten
>> <__peter__ at web.de> wrote:
>> 
>>> Steven D'Aprano wrote:
>>> 
>>>> On Fri, Feb 13, 2015 at 11:19:58AM +0100, Morten Z wrote:
>>>>> Namespaces are widely used in Python, but it appear that creating a
>>>>> namespace
>>>>> has no short form, like [] for list or {} for dict, but requires the
>>>>> lengthy types.SimpleNamespace, with prior import of types.
>>>> 
>>>> from types import SimpleNamespace as NS
>>>> ns = NS(answer=42)
>>>> 
>>>> Python did without a SimpleNamespace type for 20+ years. I don't think
>>>> it is important enough to deserve dedicated syntax, especially yet
>>>> another overloading of parentheses. Not everything needs to be syntax.
>>> 
>>> If there were syntactic support for on-the-fly namespace "packing" and
>>> "unpacking" that could have a significant impact on coding style, similar
>>> to that of named arguments.
>>> 
>>> Functions often start out returning a tuple and are later modified to
>>> return a NamedTuple. For backward compatibility reasons you are either
>>> stuck with the original data or you need a workaround like os.stat_result
>>> which has more attributes than items.
>> 
>> There's really nothing terrible about that workaround, except that it's
>> significantly harder to implement in Python (with namedtuple) than in C
>> (with structseq).
>> 
>> More importantly, I don't see how named unpacking helps that. If your
>> function originally returns a 3-tuple, and people write code that expects
>> a 3-tuple, how do you change it to return a 3-tuple that's also a
>> 4-namespace under your proposal?
>> 
>> Meanwhile:
>> 
>>> "Named (un)packing" could avoid that.
>>> 
>>>>>> def f():
>>> ...    return a:1, b:2, c:3
>>>>>> f()
>>> (a:1, b:2, c:3)
>> 
>> So what type is that? Is there a single type for all "named packing
>> tuples" (like SimpleNamespace), or a separate type for each one (like
>> namedtuple), or some kind of magic that makes issubclass(a, b) true if b
>> has the same names as a plus optionally more names at the end, or ...?
> 
> I'm an adept of duck typing, so 
> 
> a:1, b:2, c:3
> 
> has to create an object with three attributes "a", "b", and "c" and
> 
> :a, :b, :c = x
> 
> works on any x with three attributes "a", "b", and "c".

That doesn't answer either of my questions. Again:

How does this help when you started with a 3-tuple and realize you need a 4th value, ala stat? Since this is your motivating example, if the proposal doesn't help that example, what's the point?

What type is this? Just saying "I'm an adept of duck typing" doesn't answer the question. Python objects have types, and this is important.

> Implementationwise
> 
>> some kind of magic that makes issubclass(a, b) true if b
>> has the same names as a plus optionally more names at the end, or ...?
> 
> would be OK, but as there is no order of the names there is also no "end".

OK, that makes this seem even _less_ useful for the stat case.

The reasons to use namedtuple/structseq in the first place for stat are (a) so existing code written to the original tuple interface can continue to treat the value as a tuple while newer code can access the new attributes that don't fit; and/or (b) because the order has some intrinsic/historical/whatever meaning so it sometimes makes sense to extract it by position. You're proposing something as an improvement for that case, but it fails both ways.

> In general this should start as restrictive as possible, e. g. 
> issubclass(type(a), type(b)) == a is b wouldn't bother me as far as 
> usability is concerned.

OK, so a new type for each namespace object?

>> How does this interact with existing unpacking? Can I extract the values
>> as a tuple, e.g., "x, *y = f()" to get x=1 and y=(2, 3)? Or the key-value
>> pairs with "**kw = f()"? Or maybe ::kw to get them as a namespace instead
>> of a dict? Can I mix them, with "x, *y, z:c" to get 1, (2,), and 3 (and
>> likewise with **kw or ::ns)?
> 
> I'd rather not mix positional and and named (un)packing.
> 
>> Does this only work with magic namespaces, or with anything with
>> attributes? For example, if I have a Point class, "does x:x = pt" do the
>> same thing as "x = pt.x", or does that only work if I don't use a Point
>> class and instead pass around (x:3, y:4) with no type information?

This is a really important question, and you've skipped it.

I think you're tying unpacking and packing together too closely in your head--like people who expect that the **kw object you pass as an argument to a function call must end up as the **kw parameter value, or that parameters with default values have anything to do with keyword arguments. In your case, assignment targets and expressions are partially parallel syntax and similar semantics, but the distinction is important.

For example, "a, b = 2, 3" does not create a tuple "(a, b)" to assign to. And you can of course do "a[0], b.c = x" and expect that to work. So, you have to consider how your unpacking fits into that. Look at the grammar for assignment targets, and work out where the colons will go. And then, what are the semantics? If you don't know what they should be, that's why it's important to be able to answer questions like "can I use namespace unpacking for any object with the right attributes, or only for magic namespace objects?" If the answer is the former, the semantics can be dead simple: namespace unpacking uses getattr. If not, you need to invent some new mechanism, and it needs to make sense. For that matter, that question is important in its own right. With the former answer, namespace unpacking is perfectly useful on its own--e.g., to extract a couple of attributes out of a stat result. So you can argue for two related but separate proposals on their own merits, giving you a wider range of use cases both to argue for each proposal itself better and to think through the details.

And likewise, the thing in the return statement is a kind of expression--in particular, a display expression for some type. That means it has a type and a value that you can ask questions about.

I know you said "I regret that I let Andrew lure me into a discussion of those details so soon. The point of my original answer was that if an acceptable syntax can be 
found real code can profit from it." But your entire proposal is about syntactic sugar, and if you don't actually have an acceptable syntax and aren't willing to answer the questions to work toward one, you don't have a proposal.

>> Since the parens are clearly optional, does that mean that "a:3" is a
>> single-named namespace object, or do you need the comma as with tuples? (I
>> think the former may make parsing ambiguous, but I haven't thought it
>> through...)
> 
> No comma, please, if at all possible.

As pointed out, that immediately makes a:1, b:2 ambiguous as a single 2-namespace or a 2-tuple of 1-namespaces. How do you resolve that? And again, I'm pretty sure that's not the only place this would be ambiguous. (It may be worth looking at how YAML deals with its pretty similar ambiguities, even though it obviously has a different grammar from Python expressions.)

>>>>>> :c, :b, :a = f() # order doesn't matter
>>>>>> :a, :c = f()     # we don't care about b
>>>>>> a                # by default bound name matches attribute name
>> 
>> Why do you need, or want, that last comment? With similar features, you
>> have to be explicit; e.g., if you want to pass a local variable named a to
>> a keyword parameter named a, you have to write a=a, not just =a. How is
>> this case different?
> 
> I expect the bound name to match that of the attribute in ninety-nine 
> percent or so. Other than that -- no difference.

First, I don't that expectation is true. Look at iterable unpacking. It's certainly _common_ that you return x, y from a function and the result gets assigned to locals named x and y, but it's nowhere near 99%. This is where having real use cases--and being able to answer questions about, e.g., whether you can unpack any object so--would be handy. I'm pretty sure that if I wanted to get the times out of a stat(infile), I'd want the locals to be named something like inatime, inmtime, inctime, not st_atime, st_mtime, and st_ctime.

(Sorry for harping on stat, but since it's the one real-life use case you've come up with that isn't a toy example, it's the most obvious thing to look at. If you think there's something special about stat that makes it not relevant in general, please explain what it is and give a better use case.)

>>> 1
>>>>>> x:a, y:c = f()   # we want a different name
>>>>>> x, y
>>> (1, 3)
> 
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/