On 19 April 2016 at 11:06, Chris Angelico wrote:

That said, though: I don't often feel the yearning for quick namedtuples. A function that today returns a namedtuple of 'x' and 'y' can't in the future grow a 'z' without breaking any callers that unpack "x, y = func()", so extensible return objects have to forego unpacking (eg using a dict or SimpleNamespace).

Possibly the docs for namedtuple should refer the user to SimpleNamespace as an alternative if "being a tuple subclass" isn't an important requirement. Or possibly SimpleNamespace should be in the collections module (alongside namedtuple) rather than in the types module? The issue may simply be that SimpleNamespace isn't as discoverable as it should be? Paul

On 19 April 2016 at 11:46, Chris Angelico wrote:

...

The issue may simply be that SimpleNamespace isn't as discoverable as it should be?

That's entirely possible. For the situations where order is insignificant and unpacking is unnecessary, a SimpleNamespace would be perfect, if people knew about it. Probably not worth moving the actual class, but a quick little docs link might help.

http://bugs.python.org/issue26805

On 19 April 2016 at 15:09, Joseph Martinot-Lagarde wrote:

...

That's entirely possible. For the situations where order is insignificant and unpacking is unnecessary, a SimpleNamespace would be perfect, if people knew about it. Probably not worth moving the actual class, but a quick little docs link might help.

How big a problem would it be to actually move the class to collections ? For example at the top of the documentation there is a table with all the available container classes, having SimpleNamespace would be far more discoverable than a paragraph in namedtuple.

Not incredibly hard, in theory. However... Move the definition: SimpleNamespace = type(sys.implementation) to collections (but note that this isn't *really* the definition, the actual definition is in C, in the sys module, and is unnamed - to that extent types, which is for exposing names for types that otherwise aren't given built in names, is actually the right place for this definition!) Update the documentation. Update the tests (test/test_types.py) - this probably involves *moving* those tests to test_collections.py. Add a backward compatibility name in types. Add some tests for that backward compatibility name. Work out what to do about pickle compatibility:

...

...

from pickle import dumps from types import SimpleNamespace dumps(SimpleNamespace(a=1)) b'\x80\x03ctypes\nSimpleNamespace\nq\x00)Rq\x01}q\x02X\x01\x00\x00\x00aq\x03K\x01sb.'

Note that the type name (types.SimpleNamespace) is embedded in the pickle. How do we avoid breaking pickles? That's probably far from everything - this was 5 minutes' quick investigation, and I'm not very experienced at doing this type of refactoring. I'm pretty sure I've missed some things.

The class could still be importable from types.SimpleNamespace for backward compatibility.

Where it's defined matters in the pickle case - so it's not quite that simple.

Sorry for my noob questions...

Not at all. It's precisely because it "seems simple" that the feedback people get seems negative at times - I hope the above gives you a better idea of what might be involved in such a seemingly simple change. So thanks for asking, and giving me the opportunity to clarify :-) In summary: It's not likely to be worth the effort, even though it looks simple at first glance. Paul

I'm repeatedly surprised that SimpleNamespace isn't in collections.

+1 ! But I still think that my proposal has some value outside of SimpleNamespace. :)

On Tue, Apr 19, 2016 at 01:47:09PM +0100, Paul Moore wrote:

On 19 April 2016 at 13:34, Joseph Martinot-Lagarde wrote:

...

But I still think that my proposal has some value outside of SimpleNamespace. :)

The biggest issue is likely to be that the added value doesn't justify the cost of new syntax, and a language change.

I think there are more problems with the proposal than just "not useful enough". See my previous post.

On the other hand, functions preserving the order of keyword arguments (which would allow you to write a cleaner factory function) is probably a change with wider value, as well as being less intrusive (it's a semantic change, but there's no change in syntax) and so may be more likely to get through.

It's a pretty big semantic change though. As I point out in my previous post, at the moment we can be sure that changing the order of keyword arguments does not change anything: spam(a=1, b=2) and spam(b=2, a=1) will always have the same semantics. Making keyword arguments ordered will break that assumption, and even if it doesn't break any existing code, it will make it harder to reason about future code. No more can you trust that the order of keyword arguments will have no effect on the result of calling a function. I don't see the current behaviour of keyword arguments as a limitation, I see it as a feature. I don't need to care about the order of keyword arguments, only their names. -- Steve

On Tue, Apr 19, 2016 at 10:06:28PM +1000, Steven D'Aprano wrote:

On Tue, Apr 19, 2016 at 11:25:07AM +0100, Paul Moore wrote:

...

Possibly the docs for namedtuple should refer the user to SimpleNamespace as an alternative if "being a tuple subclass" isn't an important requirement. Or possibly SimpleNamespace should be in the collections module (alongside namedtuple) rather than in the types module?

+1

I'm repeatedly surprised that SimpleNamespace isn't in collections.

I've thought about this some more, and I've decided that I don't think it should be in collections. It's not a collection. Strangely, "collection" is not in the glossary: https://docs.python.org/dev/glossary.html (neither is "container") and there doesn't appear to be an ABC for what makes a collection, but the docs seem to suggest that "collections" are another word for "containers": Quote: 8.3. collections — Container datatypes Source code: Lib/collections/__init__.py This module implements specialized container datatypes providing alternatives to Python’s general purpose built-in containers, dict, list, set, and tuple. https://docs.python.org/dev/library/collections.html which mostly matches the old (and I mean really old, going back to Python 1.5 days) informal usage of "collection" as "a string, list, tuple or dict", that is, a sequence or mapping. (These days, I'd add set to the list as well.) SimpleNamespace is neither a sequence nor a mapping. We do have a "Container" ABC: https://docs.python.org/dev/library/collections.abc.html#collections.abc.Con... and SimpleNamespace doesn't match that ABC either. So I think SimpleNamespace should stay where it is. I guess I'll just have to learn that it's a type, not a collection :-) -- Steve

On Tue, Apr 19, 2016 at 08:47:13PM +0000, Joseph Martinot-Lagarde wrote:

...

SimpleNamespace is neither a sequence nor a mapping.

I don't know the exact definition of a mapping, but to me SimpleNamespace is like a dict with restricted keys (valid attributes only) and a nicer syntax. `my_dict["key"]` is roughly equivalent to `my_namespace.key`.

That is (I believe) how Javascript and PHP treat it, but they're not exactly the best languages to emulate, or at least not blindly. Attributes and keys represent different concepts. Attributes represent an integral part of the object: dog.tail me.head car.engine book.cover while keys represent arbitrary (or nearly so) data associated with some data collection: books['The Lord Of The Rings'] kings['Henry VIII'] prisoner[239410] colours['AliceBlue'] They don't just use different syntaxes, they have different purposes, and while it is tempting to (mis)use the shorter attribute syntax for key lookups: colours.AliceBlue it is risky to conflate the two. Suppose you have a book called "update" (presumably a book of experimental poetry by somebody who dislikes uppercase letters): books.update Either the key shadows the update method, or the method shadows the book, or you get an error. All of these scenarios are bad. I'd consider giving Python an alternate key-lookup syntax purely as syntactic sugar: colours~AliceBlue # sugar for colours['AliceBlue'] books~update # books['update'] before I would consider adding a standard library class that conflates attribute- and key-lookup. If people want to do this in their own code (and I do see the attraction, even if I think it is a bad idea), so be it, but the standard library shouldn't encourage it. -- Steve

On 04/19/2016 08:00 PM, Chris Angelico wrote:

On Wed, Apr 20, 2016 at 12:48 PM, Steven D'Aprano wrote:

...

Attributes and keys represent different concepts. Attributes represent an integral part of the object: [...] it is risky to conflate the two.

Enumerations blur the line. You could define 'colours' thus:

class colours(enum.IntEnum): AliceBlue = 0xf0f8ff

However, enumerations have a very specific purpose -- they aren't a generic collection you can modify and update at will (at least, not without a *lot* of effort). -- ~Ethan~

On 19 April 2016 at 15:26, Random832 wrote:

On Tue, Apr 19, 2016, at 06:25, Paul Moore wrote:

...

Possibly the docs for namedtuple should refer the user to SimpleNamespace as an alternative if "being a tuple subclass" isn't an important requirement.

Just my two cents - for the use cases that I would find this useful for (I usually end up just using tuple), a hashable "FrozenSimpleNamespace" would be nice. Or even one that has immutable key fields that are used as the basis for the hash but that you can hang other values off of (SemiFrozen?)

Having done some digging just now... SimpleNamespace is basically just exposing for end users, the type that is used to build the sys.implementation object. The definition is *literally* nothing more than SimpleNamespace = type(sys.implementation) Variations such as you suggest may indeed be useful, but the existing implementation was essentially a free benefit of "we use this one ourselves in the core". Going beyond that, any additional types probably need to justify themselves a bit better - and at that point the usual "publish on PyPI as a standalone module and see how useful they turn out to be" suggestion probably applies. Paul

On 19 April 2016 at 15:51, Joseph Martinot-Lagarde wrote:

...

Going beyond that, any additional types probably need to justify themselves a bit better - and at that point the usual "publish on PyPI as a standalone module and see how useful they turn out to be" suggestion probably applies.

The core of my proposal depends on modifications in the langage itself, it needs either adding a new syntax, either keeping the order of keywords arguments. Right now I don't see how it could work as a standalone module, but I'll think about it.

I was referring to randon832's suggested variants on SimpleNamespace. As you say, your proposal is basically a language change and you're completely right to assume that needs to be implemented in the core. Sorry if I wasn't clear enough. Paul

On 04/19/2016 08:15 AM, Koos Zevenhoven wrote:

My main objection to putting SimpleNamespace in collections is that we might want something even better in collections, perhaps ComplexNamespace?

A ComplexNamespace is a class. :) -- ~Ethan~

Chris Angelico writes:

On Tue, Apr 19, 2016 at 7:49 PM, Joseph Martinot-Lagarde wrote:

...

I'd be happy to make a factory function for my personal use, but the order of kwargs is not respected. To have an elegant syntax, it has to be a construct of the language.

There have been proposals to have kwargs retain some order (either by having it actually be an OrderedDict, or by changing the native dict type to retain order under fairly restricted circumstances). With that, you could craft the factory function easily.

I saw some proposals but forgot about them, thanks for the reminder. I'd prefer my proposal for two reasons: - the syntax is nicer that a factory function - it wouldn't be possible to add keyword arguments to __getitem__ without breaking compatibility Those issues are relatively minor, i'd be happy with a factory function

That said, though: I don't often feel the yearning for quick namedtuples. A function that today returns a namedtuple of 'x' and 'y' can't in the future grow a 'z' without breaking any callers that unpack "x, y = func()", so extensible return objects have to forego unpacking (eg using a dict or SimpleNamespace). My intention wasn't to allow extensible return objects, just easier access to the return values.

There *is* a use-case for this, but it's fairly narrow - you have to have enough names that you don't want to just unpack them everywhere, yet still have an order, AND the set of names has to be constant. It's true that python already has some commodities in place. When I first heard about namedtuples I was pretty excited, but when I tried in practice they are not that easy to use, with lots of duplications (the name, and possibly the field names) and a global declaration.

For all my use cases (not only function returns) I would completely replace namedtuple by anonymouns namedtuple.

Chris Angelico writes:

On Tue, Apr 19, 2016 at 9:31 PM, Joseph Martinot-Lagarde wrote:

...

Chris Angelico writes:

...

On Tue, Apr 19, 2016 at 7:49 PM, Joseph Martinot-Lagarde wrote:

...

I'd be happy to make a factory function for my personal use, but the

order

...

...

...

of kwargs is not respected. To have an elegant syntax, it has to be a construct of the language.

There have been proposals to have kwargs retain some order (either by having it actually be an OrderedDict, or by changing the native dict type to retain order under fairly restricted circumstances). With that, you could craft the factory function easily.

I saw some proposals but forgot about them, thanks for the reminder.

I'd prefer my proposal for two reasons: - the syntax is nicer that a factory function - it wouldn't be possible to add keyword arguments to __getitem__ without breaking compatibility

Those issues are relatively minor, i'd be happy with a factory function

Agreed, the syntax is a lot nicer - for this specific situation. Don't forget, though, that every piece of new syntax carries with it the not insignificant cost of complexity; and some things are better represented by function calls than syntax (cf 'print'). With "from types import SimpleNamespace as NS" at the top of your code, you can use the keyword-arguments form of the constructor to be almost the same as your proposal, without any new syntax. When a subsequent maintainer looks at your code, s/he can quickly look up at the imports to see what's going on, instead of having to learn another piece of syntax.

I agree on the principle, but to me this syntax looks less complex to me compared to the actual namedtuple where you have to use a factory to create a class to be able to use a namedtuple. And if you use a namedtuple as a return value, the definition of the namedtuple class will typically be relatively far away from the actual use of it. SimpleNamespace is nice but would break compatibility for return values.

I have to agree. Fortunately it isn't hard to create a namedtuple factory: As you said before, it would be if the order of the keyword arguments was conserved.

def NS(fields, *values): return namedtuple('anonymous', fields.split())(*values)

That cuts down the duplication some, but it's far from perfect.

It cuts down the duplication at the cost of readability. There is also the Matlab way which looks a bit better: from collections import namedtuple def NS(*args): fields = args[::2] values = args[1::2] return namedtuple('anonymous', fields)(*values) print(NS("x", 12, "y", 15)) But readability counts, and it's very easy to introduce bugs as adding or removing arguments can impact the whole chain. Initially coming from Matlab, the greatest interest I found in python was named arguments for functions! :)

A general warning about this (x=12, y=16) idea: It's not so different from using dicts as cheap structs, constructing objects on the fly using either {'x': 12, 'y': 16} or dict(x=12, y=16). The problem with all these this is that if you use this idiom a lot, you're invariably going to run into cases where a field is missing, and you're spending a lot of time tracking down where the object was created incorrectly. Using a namedtuple (or any other construct that asks you to pre-declare the type used) the incorrect construction will cause a failure right at the point where it's being incorrectly constructed, making it much simpler to diagnose. In fully typed languages like Haskell or Java this wouldn't be a problem, but in languages like Python or Perl it is a real concern. -- --Guido van Rossum (python.org/~guido)

A general warning about this (x=12, y=16) idea: It's not so different from using dicts as cheap structs, constructing objects on the fly using either {'x': 12, 'y': 16} or dict(x=12, y=16). The problem with all these this is

Guido van Rossum writes: that if you use this idiom a lot, you're invariably going to run into cases where a field is missing, and you're spending a lot of time tracking down where the object was created incorrectly. Using a namedtuple (or any other construct that asks you to pre-declare the type used) the incorrect construction will cause a failure right at the point where it's being incorrectly constructed, making it much simpler to diagnose. I think that it depends a lot on what these are used for. What I primarily though about was to be able to easily return a namadtuple as a function output. In this case the namedtuple (anonymous or not) is created in a single place, so having to declare the namedtuple before using it has a limited interest, and has the drawbacks I presented before (code duplication, declaration far away from the use). What you're talking about is more like a container which would be used all around an application, where you need to ensure the corecntess of the struct everywhere. In this case a standard nametuple is a better fit. I feel like it's similar to the separation you talked about before, script vs application. My view is more script-like, yours is more application-like. Joseph

On Tue, Apr 19, 2016 at 9:58 AM, João Bernardo wrote:

On Tue, Apr 19, 2016 at 12:06 PM, Guido van Rossum wrote:

...

Every call to namedtuple() creates a new class, which is a very expensive operation. On my machine the simplest namedtuple call taks around 350 usec.

Isn't that because the code for namedtuple is using exec instead a more pythonic approach of metaclassing to make it "faster" but taking longer to actually create the class? http://bugs.python.org/issue3974

That's part of it, but no matter how you do it, it's still creating a new class object, you can't get around that. And class objects are just expensive. Any idiom that ends up creating a new class object for each instance that's created is a big anti-pattern.

Maybe instead of changing namedtuple (which seems to be a taboo), there could be a new anonymous type. Something built-in, with its own syntax.

That's essentially what the (x=12, y=16) proposal is about, IIUC -- it would just be a single new type, so (x=12, y=16).__class__ would be the same class object as (a='', b=3.14). But I have serious reservations about that idiom too. -- --Guido van Rossum (python.org/~guido)

On 19.04.2016 19:35, Guido van Rossum wrote:

...

That's essentially what the (x=12, y=16) proposal is about, IIUC -- it would just be a single new type, so (x=12, y=16).__class__ would be the same class object as (a='', b=3.14).

The proposal reminds me of JavaScript's "object".

So, "(x=12, y=16).__class__ == type(object())"?

No, it would have its own class but it would be the same for all of these object, independently of the fields. This is different from namedtuples who need the creation of a subclass for each set of fields.

On Tue, Apr 19, 2016 at 4:06 AM, Chris Angelico wrote:

There have been proposals to have kwargs retain some order (either by having it actually be an OrderedDict, or by changing the native dict type to retain order under fairly restricted circumstances). With that, you could craft the factory function easily.

It may be worth dusting off one of those proposals and seeing if it can move forward.

FYI, PEP 468: "Preserving the order of **kwargs in a function." [1] The PEP exists for exactly the sort of use case being discussed here. In my case it was an alternate enum proposal I was making back when enums were under discussion. Regardless, I'd still like to see the PEP land. It's just a matter of finding time. I'll see if I can get some consensus leading up to and at PyCon next month. The actual implementation for PEP 468 is almost trivial. Now that we have a C-implementation of OrderedDict, it's mostly just a matter of addressing performance concerns [2], particularly those expressed by Guido, or falling back to one of the alternatives discussed in the PEP. Note that a related concept is represented with my work to make the class definition namespace ordered by default. [3] It would allow class decorators to have access to the order in which the class's attributes were defined. -eric [1] https://www.python.org/dev/peps/pep-0468/ [2] https://www.python.org/dev/peps/pep-0468/#performance [3] http://bugs.python.org/issue24254

Steven D'Aprano writes:

py> from collections import namedtuple py> my_tuple = namedtuple("_", "x y z")(1, 2, 3) py> print(my_tuple) _(x=1, y=2, z=3)

This looks nice with a short number of short attribute names, but in a real case separating the values from their corresponding field hurts readability. Compare: my_tuple = namedtuple("_", "site, gisement, range_nominal, range_bista, is_monostatic")(12.45, 45.0, 3000.0, 2998.5357, True) my_tuple = (site=12.45, gisement=45.0, range_nominal=3000.0, range_bista=2998.5357, is_monostatic=True) As a side note, if the empty string would be allowed for the class name the printed value would look better. The fact that it's not possible right now looks liek an implementation "detail".

...

Another hting I don't like about namedtuples is the duplication of the name. TYpical declarations look like `Point = namedtuple('Point', ['x', 'y'])`, where `Point` is repeated two times. I'll go one step further and say that the name is useless most of the time, so let's just get rid of it.

The name is not useless. It is very useful for string representations, debugging, introspection, and generally having a clue what the object represents. How else do you know what kind of object you are dealing with?

In my opinion, avoiding having to repeat the name twice is a "nice to have", not a "must have".

Of course he name is not completely useless, but you can often know what a namedtuple represents in your application or your script from the field names only (or the variable name). SimpleNamespace live very well without a name, for example.

...

my_point = (x=12, y=16) # (x=12, y=16) my_point[0] # 12

How do you know that the 0th item is field "x"?

Keyword arguments are not ordered.

In my proposal these are not keywords arguments, they are a syntax construct to build a anonymous namedtuple. That's why I can't just create a factory function. The order is determiend by the construction, and can be found by introspection (simply printing the object will do). I suppose that as a developper it's a shift from how keywords behave now, but as a user a namedtuple is still a tuple, and it keeps the ordering.

Consider that since they're tuples, we should be able to provide the items as positional arguments to the construct, just like regular namedtuples:

pt = (x=12, y=16) type(pt)(1, 2)

They inherits tuples, that doesn't mean that they behave in all cases like tuples. In that case I guess that this form would not be allowed. I don't know how bad this looks for a python dev though!

Obviously we don't want every instance to belong to a distinct class, so we need some sort of cache. SimpleNamespace solves this problem by making all instances belong to the same class. That's another difference between namedtuple and what you seem to want.

Yes, that's another difference, I don't want to replicate namedtuples exactly (otherwise I'd just use a namedtuple). I don't think that you need a cache, more like a frozendict per instance which binds the fields to the corresponding index. Maybe it's what you call cache ?

From your comments I get that a problem is that the only way to create an anonymous namedtuple would be via the syntax construct, it's not possible to use the class constructor directly. I would argue that it's not a problem becaus if you need to do that you can just use a standard namedtuple, that's exactly what it's here for. Anonymous namedtuple are for quick, easy, and readable use. Maybe that means that the added syntax complexity is not worth it.