a simple namespace type
Below I've included a pure Python implementation of a type that I wish was a builtin. I know others have considered similar classes in the past without any resulting change to Python, but I'd like to consider it afresh[1][2]. class SimpleNamespace: """A simple attribute-based namespace.""" def __init__(self, **kwargs): self.__dict__.update(kwargs) # or self.__dict__ = kwargs def __repr__(self): keys = sorted(k for k in self.__dict__ if not k.startswith('_')) content = ("{}={!r}".format(k, self.__dict__[k]) for k, v in keys) return "{}({})".format(type(self).__name__, ", ".join(content)) This is the sort of class that people implement all the time. There's even a similar one in the argparse module, which inspired the second class below[3]. If the builtin object type were dict-based rather than slot based then this sort of namespace type would be mostly superfluous. However, I also understand how that would add an unnecessary resource burden on _all_ objects. So why not a new type? Nick Coghlan had this objection recently to a similar proposal[4]: Please, no. No new just-like-a-namedtuple-except-you-can't-iterate-over-it type, and definitely not one exposed in the collections module. We've been over this before: collections.namedtuple *is* the standard library's answer for structured records. TOOWTDI, and the way we have already chosen includes iterability as one of its expected properties. As you can see he's referring to "structured records", but I expect that his objections could be extended somewhat to this proposal. I see where he's coming from and agree relative to structured records. However, I also think that a simple namespace type would be a benefit to different use cases, namely where you want a simple dynamic namespace. Making a simple namespace class is trivial and likely just about everyone has written one: "class Namespace: pass" or even "type('Namespace', (), {})". Obviously the type in this proposal has more meat, but that's certainly not necessary. So why a new type? The main reason is that as a builtin type the simple namespace type could be used in builtin modules[5][6][7]. Thoughts? -eric [1] http://mail.python.org/pipermail/python-dev/2012-May/119387.html [2] http://mail.python.org/pipermail/python-dev/2012-May/119393.html [3] http://hg.python.org/cpython/file/dff6c506c2f1/Lib/argparse.py#l1177 [4] http://mail.python.org/pipermail/python-dev/2012-May/119412.html [5] http://mail.python.org/pipermail/python-dev/2012-May/119395.html [6] http://mail.python.org/pipermail/python-dev/2012-May/119399.html [7] http://mail.python.org/pipermail/python-dev/2012-May/119402.html -------------------------- class Namespace(SimpleNamespace): def __dir__(self): return sorted(k for k in self.__dict__ if not k.startswith('_')) def __eq__(self, other): return self.__dict__ == other.__dict__ def __ne__(self, other): return self.__dict__ != other.__dict__ def __contains__(self, name): return name in self.__dict__
On Tue, May 22, 2012 at 7:26 PM, Eric Snow <ericsnowcurrently@gmail.com>wrote:
I've implemented this a few times as well. I called it "AttributeDict" or "Record". I think adding an __iter__ method would be beneficial. E.g. class SimpleNamespace : def __init__(self, **kwargs): self.__dict__.update(kwargs) # or self.__dict__ = kwargs self.__iter__ = lambda: iter(kwargs.keys()) Why do we need this imo: * sometimes x.something feels better than x['something'] * to ease duck-typing, making mocks, etc. * Named tuple feels clunky for certain dynamic cases (why do I need to create the type for a one-off?) I wonder if SimpleNameSpace should allow __getitem__ as well... Yuval
On Thu, May 24, 2012 at 5:59 AM, Yuval Greenfield <ubershmekel@gmail.com> wrote:
I'd like to limit the syntactic overlap with dict as much as possible. Effectively this is just a simple but distinct facade around dict to give a namespace with attribute access. I suppose part of the question is how much of the Mapping interface would belong instead to a hypothetical Namespace interface. (I'm definitely _not_ proposing such an unnecessary extra level of abstraction). Regardless, if you want to do dict things then you can get the underlying dict using vars(ns) or ns.__dict__ on your instance. Alternately you can subclass the SimpleNamespace type to get all the extra goodies you want, as I showed with the Namespace class at the bottom of my first message.
Yup.
I wonder if SimpleNameSpace should allow __getitem__ as well...
Same thing: just use vars(ns) or a subclass of SimpleNamespace. -eric
On Thu, May 24, 2012 at 10:17 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
I tend to call it "Struct(ure)" -- I guess I like C better than Pascal. :-)
+1
Possibly there is a (weird?) parallel with namedtuple. The end result is somewhat similar: you get to use attribute names instead of the accessor syntax (x[y]) of the underlying type. But the "feel" of the type is different, and inherits more of the underlying type (namedtuple is immutable and has a fixed set of keys, whereas the type proposed here is mutable and allows arbitrary keys as long as they look like Python names).
-- --Guido van Rossum (python.org/~guido)
On Thu, May 24, 2012 at 12:14 PM, Guido van Rossum <guido@python.org> wrote:
Yeah, the feel is definitely different. I've been thinking about this because of the code for sys.implementation. Using a structseq would probably been the simplest approach there, but a named tuple doesn't feel right. In contrast, a SimpleNamespace would fit much better. As far as this goes generally, the pattern of a simple, dynamic attribute-based namespace has been implemented a zillion times (and it's easy to do). This is because people find a simple dynamic namespace really handy and they want the attribute-access interface rather than a mapping. In contrast, a namedtuple is, as Nick said, "the standard library's answer for structured records". It's an immutable (attribute-based) namespace implementing the Sequence interface. It's a tuple and directly reflects the underlying concept of tuples in Python by giving the values names. SimpleNamespace (and the like) isn't a structured record. It's only job is to be an attribute-based namespace with as simple an interface as possible. So why isn't a type like SimpleNamespace in the stdlib? Because it's trivial to implement. There's a certain trivial-ness threshold a function/type must pass before it gets canonized, and rightly so. Anyway, while many would use something like SimpleNamespace out the the standard library, my impetus was having it as a builtin type so I could use it for sys.implementation. :) FWIW, I have an implementation (pure Python + c extension) of SimpleNamespace on PyPI: http://pypi.python.org/pypi/simple_namespace -eric
On Sat, May 26, 2012 at 2:53 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
There is no good name for such a type. "Namespace" is a bad name, because the term "namespace" is already a general term that describes a lot of things in Python (and outside it) and shouldn't share a name with a specific thing, this type. That this specific type would also be within the more general namespace-concept only makes that worse. So, what do you call it? Also, is this here because you don't like typing the square brackets and quotes? If so, does it only save you three characters and is that worth the increase to the language size? A final complaint against: would the existence of this fragment python-learners education to the point that they would defer learning and practicing to use classes properly? Sorry to complain, but someone needs to in python-ideas! ;-) Calvin
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On Sat, May 26, 2012 at 3:02 PM, Calvin Spealman <ironfroggy@gmail.com> wrote:
Yeah, I've seen it called at least 10 different things. I'm certainly open to whatever works best. I've called it "namespace" because it is one of the two kinds of namespace in Python: mapping ([]-access) and object (dotted-access). The builtin dict fills the one role and the builtin object type almost fills the other. I guess "dotted_namespace" or "attribute_namespace" would work if "namespace" is too confusing.
This is definitely the stick against which to measure! It boils down to this: for me dotted-access communicates a different, more stable sort of namespace than does []-access (a la dicts). Certainly it is less typing, but that isn't really a draw for me. Dotted access is a little easier to read, which is nice but not the big deal for me. No, the big deal is the conceptual difference inherent to access via string vs. access via identifier. Though Python does not currently have a basic, dynamic, attribute-based namespace type, it's trivial to make one: "class Namespace: pass" or "type('Namespace', (), {})". While this has been done countless times, it's so simple that no one has felt like it belonged in the language. And I think that's fine, though it wouldn't hurt to have something a little more than that (see my original message). So if it's so easy, why bother adding it? Well, "class Namespace: pass" is not so simple to do using the C API. That's about it. (I *do* think people would be glad to have a basic attribute-based namespace type in the langauge.
This is an excellent point. I suppose it depends on who was teaching, and how a new simple "namespace" type were exposed and documented. It certainly is not a replacement for classes, which have much more machinery surrounding state/methods/class-ness. If it made it harder to learn Python then it would definitely have to bring *a lot* to the table.
Sorry to complain, but someone needs to in python-ideas! ;-)
Hey, I was more worried about the crickets I was hearing. :) -eric
On Sat, May 26, 2012 at 7:33 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
This is probably the best case I've heard for such a type. Intent expression is important!
The best names I was able to get crowdsourced from #python this morning are: - record - flexobject - attrobject - attrdict - nameddict - namedobject and the absolute worst name: - Object -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Calvin Spealman schrieb am Sun, 27. May 2012, um 09:42:26 -0400:
Since the proposed type is basically an `object` allowing attributes, another option would be `attrobject`. Adding an `__iter__()` method, as proposed earlier in this thread, seems unnecessary; you can simply iterate over `vars(x)` for an `attrobject` instance `x`. Cheers, Sven
On 2012-05-27 19:08, Sven Marnach wrote:
Is this whole class really necessary? As said before, this type is implemented numerous times: * empty class (included in the Python Tutorial) [1] * argparse.Namespace [2] * multiprocessing.managers.Namespace [3] * bunch (PyPI) that inherits from dict, instead of wrapping __dict__ [4] * many more... Each of them has a different semantics. Each is suited for a slightly different use case and they are so easy to implement. So you can customize to your liking - fields can or can't begin with "_", the later __repr__ comment or the color of the shed. Still, it seems they do not have a "killer feature" like namedtuple's efficiency. Noticeable is how much they resemble a dict. Some let you iterate over the keys, test for equality and even all of the builtin dict methods (bunch). If you already use vars() for iteration, you might want a dict. Funny that except for the easy "class Namespace: pass", the rest fail repr for recursive/self-referential objects:
The next snippet use the fact that dict's __repr__ knows how to handle recursion to solve the RuntimeError problem: def __repr__(self): return "{}({!r})".format(self.__class__.__name__, self.__dict__) TB [1] http://docs.python.org/dev/tutorial/classes.html#odds-and-ends [2] http://hg.python.org/cpython/file/c1eab1ef9c0b/Lib/argparse.py#l1177 [3] http://hg.python.org/cpython/file/c1eab1ef9c0b/Lib/multiprocessing/managers.... [4] http://pypi.python.org/pypi/bunch
On Sun, May 27, 2012 at 1:58 PM, T.B. <bauertomer@gmail.com> wrote:
All of the re-implementations of essentially the same thing is exactly why a standard version is constantly suggested. That said, it is so simple that it easily has many variants, because it is only the base of the different ideas all these things implement.
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On 27 May 2012 20:35, Eric V. Smith <eric@trueblade.com> wrote:
What about an `__iter__()` method that works like `dict.items()`? Then you can do a round trip with ns = attrobject(**d) and d = dict(ns) allowing you to quickly convert between attribute-based and item-based access in either direction.
Slightly easier bar to reach: could the various incarnations be improved by using a new varobject type as a base class (e.g. I know I often use namedtuple as a base class rather than instantiating them directly, although I do the latter, too). There's also a potentially less controversial alternative: just add an easy spelling for "type(name, (), {})" to the C API. -- Sent from my phone, thus the relative brevity :) On May 28, 2012 5:54 AM, "Eric V. Smith" <eric@trueblade.com> wrote:
On Sun, May 27, 2012 at 2:09 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Good point. I do the same.
There's also a potentially less controversial alternative: just add an easy spelling for "type(name, (), {})" to the C API.
I really like this. There's a lot of boilerplate to create just a simple type like this in the C API. I'll see what I can come up with. :) As a namespace, it would be good to have a nice repr, but that's not a show stopper. -eric
On Mon, May 28, 2012 at 10:34 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
Eric Snow dixit (2012-05-28, 10:34):
Using classes as 'attribute containers' is suboptimal (which means that in performance-critical parts of code you would have to implement a namespace-like type anyway -- if you wanted to have attr-based syntax, of course). There should be one obvious way to do it. Now there is no one. Cheers. *j
On Tue, May 22, 2012 at 7:26 PM, Eric Snow <ericsnowcurrently@gmail.com>wrote:
I've implemented this a few times as well. I called it "AttributeDict" or "Record". I think adding an __iter__ method would be beneficial. E.g. class SimpleNamespace : def __init__(self, **kwargs): self.__dict__.update(kwargs) # or self.__dict__ = kwargs self.__iter__ = lambda: iter(kwargs.keys()) Why do we need this imo: * sometimes x.something feels better than x['something'] * to ease duck-typing, making mocks, etc. * Named tuple feels clunky for certain dynamic cases (why do I need to create the type for a one-off?) I wonder if SimpleNameSpace should allow __getitem__ as well... Yuval
On Thu, May 24, 2012 at 5:59 AM, Yuval Greenfield <ubershmekel@gmail.com> wrote:
I'd like to limit the syntactic overlap with dict as much as possible. Effectively this is just a simple but distinct facade around dict to give a namespace with attribute access. I suppose part of the question is how much of the Mapping interface would belong instead to a hypothetical Namespace interface. (I'm definitely _not_ proposing such an unnecessary extra level of abstraction). Regardless, if you want to do dict things then you can get the underlying dict using vars(ns) or ns.__dict__ on your instance. Alternately you can subclass the SimpleNamespace type to get all the extra goodies you want, as I showed with the Namespace class at the bottom of my first message.
Yup.
I wonder if SimpleNameSpace should allow __getitem__ as well...
Same thing: just use vars(ns) or a subclass of SimpleNamespace. -eric
On Thu, May 24, 2012 at 10:17 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
I tend to call it "Struct(ure)" -- I guess I like C better than Pascal. :-)
+1
Possibly there is a (weird?) parallel with namedtuple. The end result is somewhat similar: you get to use attribute names instead of the accessor syntax (x[y]) of the underlying type. But the "feel" of the type is different, and inherits more of the underlying type (namedtuple is immutable and has a fixed set of keys, whereas the type proposed here is mutable and allows arbitrary keys as long as they look like Python names).
-- --Guido van Rossum (python.org/~guido)
On Thu, May 24, 2012 at 12:14 PM, Guido van Rossum <guido@python.org> wrote:
Yeah, the feel is definitely different. I've been thinking about this because of the code for sys.implementation. Using a structseq would probably been the simplest approach there, but a named tuple doesn't feel right. In contrast, a SimpleNamespace would fit much better. As far as this goes generally, the pattern of a simple, dynamic attribute-based namespace has been implemented a zillion times (and it's easy to do). This is because people find a simple dynamic namespace really handy and they want the attribute-access interface rather than a mapping. In contrast, a namedtuple is, as Nick said, "the standard library's answer for structured records". It's an immutable (attribute-based) namespace implementing the Sequence interface. It's a tuple and directly reflects the underlying concept of tuples in Python by giving the values names. SimpleNamespace (and the like) isn't a structured record. It's only job is to be an attribute-based namespace with as simple an interface as possible. So why isn't a type like SimpleNamespace in the stdlib? Because it's trivial to implement. There's a certain trivial-ness threshold a function/type must pass before it gets canonized, and rightly so. Anyway, while many would use something like SimpleNamespace out the the standard library, my impetus was having it as a builtin type so I could use it for sys.implementation. :) FWIW, I have an implementation (pure Python + c extension) of SimpleNamespace on PyPI: http://pypi.python.org/pypi/simple_namespace -eric
On Sat, May 26, 2012 at 2:53 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
There is no good name for such a type. "Namespace" is a bad name, because the term "namespace" is already a general term that describes a lot of things in Python (and outside it) and shouldn't share a name with a specific thing, this type. That this specific type would also be within the more general namespace-concept only makes that worse. So, what do you call it? Also, is this here because you don't like typing the square brackets and quotes? If so, does it only save you three characters and is that worth the increase to the language size? A final complaint against: would the existence of this fragment python-learners education to the point that they would defer learning and practicing to use classes properly? Sorry to complain, but someone needs to in python-ideas! ;-) Calvin
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On Sat, May 26, 2012 at 3:02 PM, Calvin Spealman <ironfroggy@gmail.com> wrote:
Yeah, I've seen it called at least 10 different things. I'm certainly open to whatever works best. I've called it "namespace" because it is one of the two kinds of namespace in Python: mapping ([]-access) and object (dotted-access). The builtin dict fills the one role and the builtin object type almost fills the other. I guess "dotted_namespace" or "attribute_namespace" would work if "namespace" is too confusing.
This is definitely the stick against which to measure! It boils down to this: for me dotted-access communicates a different, more stable sort of namespace than does []-access (a la dicts). Certainly it is less typing, but that isn't really a draw for me. Dotted access is a little easier to read, which is nice but not the big deal for me. No, the big deal is the conceptual difference inherent to access via string vs. access via identifier. Though Python does not currently have a basic, dynamic, attribute-based namespace type, it's trivial to make one: "class Namespace: pass" or "type('Namespace', (), {})". While this has been done countless times, it's so simple that no one has felt like it belonged in the language. And I think that's fine, though it wouldn't hurt to have something a little more than that (see my original message). So if it's so easy, why bother adding it? Well, "class Namespace: pass" is not so simple to do using the C API. That's about it. (I *do* think people would be glad to have a basic attribute-based namespace type in the langauge.
This is an excellent point. I suppose it depends on who was teaching, and how a new simple "namespace" type were exposed and documented. It certainly is not a replacement for classes, which have much more machinery surrounding state/methods/class-ness. If it made it harder to learn Python then it would definitely have to bring *a lot* to the table.
Sorry to complain, but someone needs to in python-ideas! ;-)
Hey, I was more worried about the crickets I was hearing. :) -eric
On Sat, May 26, 2012 at 7:33 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
This is probably the best case I've heard for such a type. Intent expression is important!
The best names I was able to get crowdsourced from #python this morning are: - record - flexobject - attrobject - attrdict - nameddict - namedobject and the absolute worst name: - Object -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Calvin Spealman schrieb am Sun, 27. May 2012, um 09:42:26 -0400:
Since the proposed type is basically an `object` allowing attributes, another option would be `attrobject`. Adding an `__iter__()` method, as proposed earlier in this thread, seems unnecessary; you can simply iterate over `vars(x)` for an `attrobject` instance `x`. Cheers, Sven
On 2012-05-27 19:08, Sven Marnach wrote:
Is this whole class really necessary? As said before, this type is implemented numerous times: * empty class (included in the Python Tutorial) [1] * argparse.Namespace [2] * multiprocessing.managers.Namespace [3] * bunch (PyPI) that inherits from dict, instead of wrapping __dict__ [4] * many more... Each of them has a different semantics. Each is suited for a slightly different use case and they are so easy to implement. So you can customize to your liking - fields can or can't begin with "_", the later __repr__ comment or the color of the shed. Still, it seems they do not have a "killer feature" like namedtuple's efficiency. Noticeable is how much they resemble a dict. Some let you iterate over the keys, test for equality and even all of the builtin dict methods (bunch). If you already use vars() for iteration, you might want a dict. Funny that except for the easy "class Namespace: pass", the rest fail repr for recursive/self-referential objects:
The next snippet use the fact that dict's __repr__ knows how to handle recursion to solve the RuntimeError problem: def __repr__(self): return "{}({!r})".format(self.__class__.__name__, self.__dict__) TB [1] http://docs.python.org/dev/tutorial/classes.html#odds-and-ends [2] http://hg.python.org/cpython/file/c1eab1ef9c0b/Lib/argparse.py#l1177 [3] http://hg.python.org/cpython/file/c1eab1ef9c0b/Lib/multiprocessing/managers.... [4] http://pypi.python.org/pypi/bunch
On Sun, May 27, 2012 at 1:58 PM, T.B. <bauertomer@gmail.com> wrote:
All of the re-implementations of essentially the same thing is exactly why a standard version is constantly suggested. That said, it is so simple that it easily has many variants, because it is only the base of the different ideas all these things implement.
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On 27 May 2012 20:35, Eric V. Smith <eric@trueblade.com> wrote:
What about an `__iter__()` method that works like `dict.items()`? Then you can do a round trip with ns = attrobject(**d) and d = dict(ns) allowing you to quickly convert between attribute-based and item-based access in either direction.
Slightly easier bar to reach: could the various incarnations be improved by using a new varobject type as a base class (e.g. I know I often use namedtuple as a base class rather than instantiating them directly, although I do the latter, too). There's also a potentially less controversial alternative: just add an easy spelling for "type(name, (), {})" to the C API. -- Sent from my phone, thus the relative brevity :) On May 28, 2012 5:54 AM, "Eric V. Smith" <eric@trueblade.com> wrote:
On Sun, May 27, 2012 at 2:09 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Good point. I do the same.
There's also a potentially less controversial alternative: just add an easy spelling for "type(name, (), {})" to the C API.
I really like this. There's a lot of boilerplate to create just a simple type like this in the C API. I'll see what I can come up with. :) As a namespace, it would be good to have a nice repr, but that's not a show stopper. -eric
On Mon, May 28, 2012 at 10:34 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
Eric Snow dixit (2012-05-28, 10:34):
Using classes as 'attribute containers' is suboptimal (which means that in performance-critical parts of code you would have to implement a namespace-like type anyway -- if you wanted to have attr-based syntax, of course). There should be one obvious way to do it. Now there is no one. Cheers. *j
participants (10)
-
Calvin Spealman -
Eric Snow -
Eric V. Smith -
Guido van Rossum -
Jan Kaliszewski -
Nick Coghlan -
Oscar Benjamin -
Sven Marnach -
T.B. -
Yuval Greenfield