Hi all! So this is a proposal for a new soft language keyword: namespace I started writing this up a few hours ago and then realized as it was starting to get away from me that there was no way this was going to be even remotely readable in email format, so I created a repo for it instead and will just link to it here. The content of the post is the README.md, which github will render for you at the following link: https://github.com/matthewgdv/namespace I'll give the TLDR form here, but I would ask that before you reply please read the full thing first, since I don't think a few bullet-points give the necessary context. You might end up bringing up points I've already addressed. It will also be very hard to see the potential benefits without seeing some actual code examples. TLDR: - In a single sentence: this proposal aims to add syntactic sugar for setting and accessing module/class/local attributes with dots in their name - the syntax for the namespace keyword is similar to the simplest form of a class definition statement (one that implicitly inherits from object), so: namespace some_name: ... # code goes here - any name bound within the namespace block is bound in exactly the same way it would be bound if the namespace block were not there, except that the namespace's name and a dot are prepended to the key when being inserted into the module/class/locals dict. - a namespace block leaves behind an object that serves to process attribute lookups on it by prepending its name plus a dot to the lookup and then delegating it to whatever object it is in scope of (module/class/locals) - This would allow for small performance wins by replacing the use of class declarations that is currently common in python for namespacing, as well as making the writer's intent explicit - Crucially, it allows for namespacing the content of classes, by grouping together related methods. This improves code clarity and is useful for library authors who design their libraries with IDE autocompletion in mind. This cannot currently be done by nesting classes. I live in the UK so I'm going to bed now (after working on this for like the last 6 hours). I'll be alive again in maybe 8 hours or so and will be able to reply to any posts here then. Cheers everyone :)
Nope! I know it's hard to convey tone through text, so I just wanna assure you I'm not being snarky or whatever. With that said, I'd ask that you please read through the full proposal before making assumptions like 'this is exactly the same as types.SimpleNamespace'. I know it's a bit long (sorry for that!), but that's the only way we can be on the same page for this discussion. There are several code examples in there which do things that could not be done by using types.SimpleNamespace. Also, types.SimpleNamespace doesn't offer the same declarative syntax (where assigning to names within its block gets transformed into attributes). Its interface is mostly dynamic, whereas the way a 'namespace' block would be used would be declarative. The closest analogue to the proposed 'namespace' block, would not be types.SimpleNamespace, but rather, a 'class' definition that makes use of the syntactic sugar for declaring class attributes when assigning to names within the class block. It might well be that the consensus is that this suggestion is *too similar* to that and *doesn't offer enough* to warrant adding a new keyword. That's fine. I wrote this up knowing that it's a pretty high bar to add any new keyword to an existing language and there's a good chance this doesn't go anywhere. I still just wanted to get a conversation going to get a feel for the community's appetite for something like this. But yeah, probably the biggest difference between a namespace block and a class block would be that a namespace block within a class block still allows any functions that are defined within it to become bound methods for that class, which would not be the case using a nested class block. Also, attributes are not actually stored within the namespace, but rather within whatever scope the namespace block ultimately lives. This is very different to how nested classes or types.SimpleNamespace works. I'm happy to answer any questions, just please read the full proposal first :) On Sun, May 2, 2021 at 1:25 AM David Mertz <mertz@gnosis.cx> wrote:
I read the entire long description, but admittedly skimmed some. Certainly 90%+ can be done, in almost the same way, with `class Mine(Simple namespace)`, but I could have missed some corner. Perhaps highlight that. But I am certain I remain -1, in any event. On Sun, May 2, 2021, 5:16 AM Matt del Valle <matthewgdv@gmail.com> wrote:
On Sun, 2 May 2021 at 00:57, Matt del Valle <matthewgdv@gmail.com> wrote:
Sometimes it’s useful to write classes that are similar to: class proxy: …. def __getattr__(self, name): return wrap(getattr(self.object, name)) How would this pattern interact with objects that use namespaces? I’m guessing the interpreter wouldn’t know to call getattr with ‘ns.attr’ rather than just ‘ns’? Steve
This shouldn't be a problem. For instance: class Wrapped: namespace foo: bar = True facade = proxy(Wrapped()) facade.foo.bar # True Basically, when you try to look up 'foo' on the proxy object, the '__getattr__' returns the namespace object, which then forwards on the second attribute lookup ('bar') to the 'Wrapped' class. It happens in two steps rather than one. So what actually gets looked up is vars(wrapped_instance)['foo.bar'], because the 'foo' object that is left behind by the namespace block forwards the attribute lookup on to the Wrapped class (while prepending itself to the dict lookup). Does that make sense? On Sun, May 2, 2021 at 10:30 AM Stestagg <stestagg@gmail.com> wrote:
A further point to Steve's example is that a proxy object like the one he described would probably have to change its implementation ever so slightly in order to work as expected in a multi-step lookup involving namespaces (so that the `wrap` function is only applied to the final value at the end). Probably something like this: from types import NamespaceType # similar to FunctionType class proxy: …. def __getattr__(self, name): if isinstance(value := getattr(self.object, name), NamespaceType): return type(self)(value) # create another proxy object for the next lookup return wrap(value) On Sun, May 2, 2021 at 10:45 AM Matt del Valle <matthewgdv@gmail.com> wrote:
OK, tell me what this does. namespace A: namespace B: foo = bar On Sat, May 1, 2021 at 7:55 PM Matt del Valle <matthewgdv@gmail.com> wrote:
-- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman@redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
So for simplicity let's assume that the content of 'bar' is 3, and let's assume that the module is __main__ Then:
vars(sys.modules[__name__])["A.B.foo"] 3
And it can be accessed in the usual way like:
A.B.foo 3
Likewise for the namespace objects that are created during the namespace block:
vars(sys.modules[__name__])["A"] <namespace object <A> of <module '__main__' (built-in)>>
A.B <namespace object <A.B> of <module '__main__' (built-in)>>
The key point here is that if you dynamically iterate over the contents of the module you will get all the objects that were set within any namespace blocks. Unlike nesting class blocks and using class attributes, you won't have to recursively descend into the namespaces. The namespace objects are left behind as a convenience to enable classic attribute access, but they don't 'own' the attributes that were set within their scope. The (in this case) module does. The only reference the namespace has is (in this case) to the module. Note that namespaces within modules aren't the most interesting case. Namespaces within class blocks and for class instance attributes are much more likely to be where this feature would offer big wins in code clarity. For a class block: class Example: namespace foo: def bar(self): ... # implementation goes here Then:
vars(Example)["foo.bar"] <function __main__.Example.foo.bar(self)>
Which should generally be accessed like:
Example.foo.bar <function __main__.Example.foo.bar(self)>
Likewise, the namespace object:
Example.foo <namespace object <foo> of <class '__main__. Example'>>
And from an instance the function becomes a method as normal:
There are code examples in here that hopefully give a better feel for the benefits of grouping methods of classes into namespaces: https://github.com/matthewgdv/namespace On Mon, May 3, 2021 at 1:25 PM Calvin Spealman <cspealma@redhat.com> wrote:
On Mon, 3 May 2021 at 19:24, David Mertz <mertz@gnosis.cx> wrote:
So yes... as I thought, SimpleNamespace does EVERYTHING described by the proposal, just without needing more keywords:
Except that the code and description of the proposal explicitly outline behaviours that SimpleNamespace does not provide (and aren’t trivially possible to add) It would be useful to discuss the relative merits/lack thereof of the differences (for the record, I am ambivalent about the benefits of the proposal being enough to warrant a language change)
On Mon, May 3, 2021 at 6:37 PM Stestagg <stestagg@gmail.com> wrote:
I've looked and been unable to find an example of that. Can you show one? -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
The first example in the doc lays out the difference: Assignments within the namespace block have this special behaviour whereby the assigned-to name is changed to be: ‘<namespace name>.<assignment name>’ And the assignment is made in the ‘parent scope’ of the namespace. I.e. (again, as described in the doc): class A: Namespace B: C = 1 Results in: A.__dict__ == {‘B.C’: 1, ‘B’: <namespace proxy>} Note the ‘B.C’ entry Now for me, the only place where is starts to be interesting is with methods within namespaces, where the ‘self’ binding is made against to top-level class, and not against the namespace. This does make for some nicer nested API definitions (a-la pandas DataFrame.str.xxx methods) where an intermediate name os used just to group and organise methods. On Mon, 3 May 2021 at 19:40, David Mertz <mertz@gnosis.cx> wrote:
On Mon, May 3, 2021 at 6:49 PM Stestagg <stestagg@gmail.com> wrote:
OK, that is a difference. I don't really understand why it's desirable, but I see the difference. I can make that happen with a metaclass that would allow: class A(MagicNamespace): class B: C = 1 A.__dict__ == {'B.C': 1, 'B': <class 'module.B'>} So that wouldn't be identical in the repr(), but is there something else important here? In general, I find it anathema to add syntax for something that is easily handled by existing inheritance mechanisms. If I want to, I can even make every class defined within the body of a MagicNamespace magically become a MagicNamespace too, if that's a thing someone wants. Actually, I might not even need a metaclass, a MagicNamespace.__new__() method might be adequate. I haven't fully thought it out. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
I've read the proposal, and this thread. Questions: 1. It seems that I could get close to what you're aiming for by just using underscores in names, and grouping them together in the class definition. Is there any advantage to declaring methods in a namespace beyond indentation and the dot notation? 2. If __dict__ contains "B.C" and "B", then presumably the interpreter would need to try combinations against the outer __dict__ as well as B. Is the namespace proxy you've mentioned intended to prevent further lookup in the "B" attribute? 3. Can namespaces be nested? If so, will their attributed they always resolve to flat set of attributes in the encapsulating class? 4. What would you expect getattr("A.B", "C") to yield? Paul On Mon, 2021-05-03 at 19:49 +0100, Stestagg wrote:
@Paul Bryan, I'll try to address your questions one-by-one 1. It seems that I could get close to what you're aiming for by just using
What you're suggesting (using underscores for namespacing) is something I've seen done many times (and done myself plenty of times) and it's precisely one of the main things that I think could be better under this proposal. Reusing a prefix like that across methods has always felt a bit hacky to me and like it doesn't follow the DRY principle very well. If you wanted to rename the prefix at a later point you would have to go through every single method that uses it, instead of just doing a single refactor/rename on the name at the top of the namespace block. But you've pretty much perfectly identified the benefits here, I'll just elaborate on them a bit. - the indentation visually separates blocks of conceptually-grouped attributes/methods in the actual code (a gain in clarity when code is read) - the dot notation you use to invoke such methods improves the experience for library consumers by giving a small amount conceptually-linked autocompletions at each namespace step within a class with a large API, rather getting a huge flat list. Furthermore, the proposed namespaces have other selling-points outside of the use-case of grouping methods within a class. While the benefit of namespaces within modules is more dubious because you can use class blocks for namespacing (and people often do), can see a few ways that the namespace proposal is better: - you can put functions inside a namespace block, which would become methods if you had put them in a class block - you don't have the same (in some cases extremely unintuitive) scoping/variable binding rules that you do within a class block (see the link in my doc). It's all just module scope. - it mode clearly indicates intent (you don't want a whole new class, just a new namespace) When using the namespaces within a method (using self): - It allows you to namespace out your instance attributes without needing to create intermediate objects (an improvement to the memory footprint, and less do-nothing classes to clutter up your codebase) - While the above point of space complexity will not alway be relevant I think the more salient point is that creating intermediate objects for namespacing is often cognitively more effort than it's worth. And humans are lazy creatures by nature. So I feel like having an easy and intuitive way of doing it would have a positive effect on people's usage patterns. It's one of those things where you likely wouldn't appreciate the benefits until you'd actually gotten to play around with it a bit in the wild. For example, you could rewrite this: class Legs: def __init__(self, left, right): self.left, self.right = left, right class Biped: def __init__(self): self.legs = Legs(left=LeftLeg(), right=RightLeg()) As this: class Biped: def __init__(self): namespace self.legs: left, right = LeftLeg(), RightLeg() And sure, the benefit for a single instance of this is small. But across a large codebase it adds up. It completely takes away the tradeoff between having neatly namespaced code where it makes sense to do so and writing a lot of needless intermediate classes. SimpleNamespace does not help you here as much as you would think because it cannot be understood by static code analysis tools when invoked like this: class Biped: def __init__(self): self.legs = SimpleNamespace(left=LeftLeg(), right=RightLeg()) So it is a terrible idea to use it in this way to write any kind of library code. You could invoke it declaratively: class Biped: def __init__(self): class Legs(SimpleNamespace): left, right = LeftLeg(), RightLeg() self.legs = Legs Not only is this an unintuitive usage pattern and creates lots of unnecessary classes (one for each new instance of Biped), but it significantly reduces code clarity (imagine having lots of these inside a single method). By contrast, a namespace block seems to me to increase code clarity, though I will grant that this is subjective. 2. If __dict__ contains "B.C" and "B", then presumably the interpreter
The namespace proxy must know its fully-qualified name all the way up to its parent scope (this is the bit that would require some magic in the python implementation), so it only needs to forward on a single attribute lookup to its parent scope. It does not need to perform several intermediate lookups on all of its parent namespaces. So in the case of: namespace A: namespace B: C = True
A.B <namespace object <A.B> of <module '__main__' (built-in)>>
Note that namespace B 'knows' that its name is 'A.B', not just 'B' The implementation of this namespace proxy object might look something like this, only implemented in C to hopefully be more performant than this: class NamespaceProxy: ... def __getattr__(self, name): return getattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}") def __setattr__(self, name, value): setattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}", value) The names of the dunders are hypothetical and not something I'm concerned about at this stage. Traversing all the way through A.B.C does involve 2 intermediate lookups (looking up 'A.B' on the parent scope from namespace A, then looking up 'A.B.C' on the parent scope from namespace A.B). But once you have a reference to a deeply nested namespace, looking up any value on it is only a single lookup step. 3. Can namespaces be nested? If so, will their attributed they always
resolve to flat set of attributes in the encapsulating class?
Yes, namespaces can be nested arbitrarily, and they will always set their attributes in the nearest real scope (module/class/locals). There's an example of this early on in the doc: namespace constants: NAMESPACED_CONSTANT = True namespace inner: ANOTHER_CONSTANT = "hi" Which is like: vars(sys.modules[__name__])["constants.NAMESPACED_CONSTANT"] = Truevars(sys.modules[__name__])["constants.inner.ANOTHER_CONSTANT"] = "hi" 4. What would you expect getattr(A.B, "C") to yield?
It would be no different than:
A.B.C
It happens in two steps: 1) A.B - this looks up vars(<module '__main__' (built-in)>)['A.B'], which is a namespace object 2) getattr(<namespace object <A.B> of <module '__main__' (built-in)>>, "C") - this looks up "C" on namespace "A.B", which forwards this lookup on to the module as: vars(<module '__main__' (built-in)>)['A.B.C'] The namespace proxies created in a namespace block do nothing more than forward on any attribute access/attribute setting operations performed on them to their parent scope, so you can do any attribute operations to them that you could do to a 'real' intermediate object. I hope I've answered everything with reasonable clarity. I'm kind of exhausted from writing this up, but I'll finally just quickly mention in response to David Mertz that your proposal to use a metaclass for this wouldn't really serve to provide virtually any of the benefits I listed out in my answer to Paul (part 1, in particular). it wouldn't work for modules/locals, and it can't be used at the instance level with `self`, only the class level. It would also restrict usage of other metaclasses. What it would end up doing (saving class attributes of nested classes to its own __dict__ with a fully-qualified name) is an implementation detail of my proposal rather than one of the primary benefits. From where I'm standing it feels like you just took an immediate initial dislike to the proposal and didn't even really give it a chance by actually thinking through the pros/cons before giving your -1. It's fine if you think it's not useful enough to be added. Maybe that's true. But it would be more helpful to maybe ask questions like other people are doing and be sure you understand the rationale/implications of the proposal before making your mind up. Three negative responses in you still didn't understand a really simple implementation point that's written very early on in the toy example of the doc, which differentiates it from types.SimpleNamespace. Then when Steve pointed that out to you, you immediately dug your heels in again without really thinking your 'it can be done with a metaclass' counterpoint all the way through. At least give yourself the chance to change your mind! Even if you end up settling on your initial intuition, at least that way you'll know that you've explored all the options and done your best to land on the most informed opinion you can. Anyways, I'm sorry if I totally misrepresented the situation. It's hard sometimes to tell over a medium like this what page someone else is on. I just figured I'd tell you how it felt to me. I hope I wasn't totally off-base. On Mon, May 3, 2021 at 8:16 PM Paul Bryan <pbryan@anode.ca> wrote:
Response inline. On Mon, 2021-05-03 at 23:30 +0100, Matt del Valle wrote:
Sure, it's a bit hacky, but virtually every editor can do it in a few keystrokes, assuming no naming collision that requires you to cherry- pick values to be substituted.
So, is it a prerequisite that whatever object in which I'm trying to establish a namespace must support getattr/setattr? Also, it occurs to me that if I can declare a namespace in any object, I might be tempted to (or might inadvertently) monkey patch external objects with it. Any thoughts on guarding against that, or is this "we're adults here" case?
A problem I sense here is the fact that the interpreter would always need to attempt to resolve "A.B.C" as getattr(getattr(A, "B"), "C") and getattr(A, "B.C"). Since the proxy would be there to serve up the namespace's attributes, why not just let it and do away with "B.C" in A.__dict__? What will the performance costs be of attempting to get an attribute in two calls instead of one?
So, if in a nested scenario, A.B.C.D, I'm trying to understand the combination of getattr calls to resolve D. Would it just still be two attempts, getattr(A, "B.C.D") and getattr(getattr(getattr(A, "B"), "C"), "D")? If it were to become a Cartesian product of calls, there'll likely be a problem. 🤔️
On 04/05/2021 14:39, Paul Bryan wrote:
I don't pretend to fully understand the proposal and how it would be implemented, but IMO adding an overhead (not to mention more complicated semantics) to *every* chained attribute lookup is enough to kill the proposal, given that it seems to have relatively slight benefits. Best wishes Rob Cliffe
It's a good thing that isn't the case then :p Sorry, couldn't resist that quip. But more seriously, this would not be an issue. Normal attribute lookups would remain wholly unchanged. Because attribute lookups on namespace proxies *are* normal attribute lookups. That's the entire reason I've proposed it the way I have (well, that and the fact that it allows methods within namespaces to still bind to the parent class basically 'for free' without requiring any magical behaviour). Because aside from the magic behaviour of the namespace block (the way it binds names to its parent scope) it does not require *any* changes to the current attribute lookup semantics. In fact, it is built entirely on them. The only *possible* difference would be that perhaps namespace proxies could be special-cased if any room for optimization could be found at the implementation stage (purely hypothetical) to attempt to make them more performant than the rough python pseudocode sketch of their behaviour I posted yesterday in response to Paul. So fear not, no current semantics would be harmed in the making of this film keyword. So, is it a prerequisite that whatever object in which I'm trying to
establish a namespace must support getattr/setattr?
I'd imagine so, yes. At the very least, any namespaceable (new word!) object would have to have some sort of __dict__ or equivalent (the built-in `vars` would have to work on them), because otherwise I don't see how they could store attributes. Also, it occurs to me that if I can declare a namespace in any object, I
Nothing stops you from monkey-patching objects right now. Declaring namespaces on arbitrary live objects you've obtained from who-knows-where would have all the same benefits to code conciseness and clarity as declaring them on `self` within a method, versus conventional monkey-patching. You could, for instance, group all the attributes you're monkey-patching in under a single namespace with a descriptive name (namespace custom? namespace extension? namespace monkey???) to keep track of what belongs to the original object and what doesn't, more easily. Monkey-patching is often considered an anti-pattern, but sometimes there's a library that does 99% of what you need but without that last 1% it's worthless for your use-case. Rather than forking and rewriting the entire library yourself, sometimes monkey-patching is a better solution. Practicality over purity, right? So yes, this definitely falls under the "consenting adults" category. A problem I sense here is the fact that the interpreter would always need
Well, the performance would be, at worst, identical to a set of chained attribute lookups currently, which also have to be done in multiple steps, one at a time. We want to keep to existing python semantics as much as possible so that the change is minimally invasive (As Rob Cliffe pointed out, this is a proposal for syntactic sugar, so it probably doesn't warrant changes to the semantics of something.so fundamental to the language as attribute lookups). Buy hey, if there's some room for optimization under-the-hood that would allow nested namespace lookups to be done in a single step in certain situations that would be undetectable to an end-user of the language, then that would be fine too, assuming it didn't add too much burden of maintenance. I somewhat doubt it would be possible to do that though, because the namespace proxies would be python objects like any other. You can pass references to them around. So in the case of: namespace A: namespace B: C = "foo" You could just grab a reference to B:
ref = A.B
and then later on look up C on the reference:
ref.C # this looks up vars(sys.modules[__name__])['A.B.C'] 'foo'
The attribute lookup behavior behaves exactly like you would expect any similar set of chained attribute lookups to behave. No different than: class A: class B: C = "foo" So, if in a nested scenario, A.B.C.D, I'm trying to understand the
The only way you could ever resolve D in a single attribute lookup like getattr(A, "B.C.D") is if you literally typed that statement out verbatim:
getattr(A, "B.C.D")
That would actually work, because `namespace A` would prepend its name to the attribute lookup and forward it on to its parent scope, serving up (sys.modules[__name__])['A.B.C.D'], which is where the key under which D is stored in the module globals. But if you just look it up using normal attribute access syntax, like you would 99% of the time:
A.B.C.D
Then it would be done in 3 separate lookups, one for each dot, exactly the way you described: getattr(getattr(getattr(A, "B"), "C"), "D") Looking B up on `namespace A` gets you `namespace A.B`, looking up C on `namespace A.B` gets you `namespace A.B.C`, and looking up D on `namespace A.B.C` finally gets you a reference to whatever D contains. This isn't a cartesian product though. It's just one attribute lookup per level of reference nesting, just the same as any other form of attribute access. As a side-note to everyone who has asked questions and participated in the discussion so far, thank you for at least entertaining my proposal. I'm under no delusions about the incredibly high bar that adding a new keyword to the language represents. But I still think discussions like this are important! Even if only one in every hundred such ideas actually has potential, if we don't brainstorm them up and discuss them as a community we'd never end up finding the ones that are valuable enough to seriously consider taking forward to a PEP. I've found that on this list there are a few people who fancy themselves gatekeepers and will dismiss virtually any new idea that gets posted here out-of-hand in a way that is often pretty unkind, and without really offering up much of a rationale. Maybe it's just a fundamental difference of mindset, but to me, the idea that python is 'finished' and should remain largely untouched forever is kind of horrifying. Programming languages that don't develop and evolve just become obsolete. It seems to me that with literally every recent new enhancement to the language that I've been incredibly excited about in the last few years (f-strings, assignment expressions, pattern matching) there's been a huge number of people loudly complaining that 'we don't really need this' and it just baffles me. Of course we don't *need* syntactic sugar. We could all write code like this:
(3).__add__((4).__mul__(2)) 11
rather than:
3 + 4*2
and evens = [] for num in range(1, 11): if not num % 2: evens.append(num) rather than: [num for num in range(1, 11) if not num % 2] But I think it's hard to argue that a language without any syntactic sugar is better, or for that matter is something anyone would actually enjoy using. So yeah, to make this point explicit. *This is a proposal for syntactic sugar*. It doesn't really add anything you can't do currently, although many of the things that would be idiomatic and trivial with `namespace` would be horrible antipatterns without it (and equivalent implementations available currently are much more verbose and less performant). but it should hopefully allow for more organized and DRYer code, and more easy usage of namespacing in situations where it makes sense and would be otherwise non-trivial to add them in (there are several examples of this in this thread and in the doc). Arguing that this is pointless because it doesn't add completely new functionality misses the mark in the same way that it's not particularly helpful to argue that listcomps are pointless because for-loops exist. When they were first introduced many people hated them, and yet they (and other types of comprehensions) are one of the most beloved features of the language nowadays. I can respect that many people will feel that the upsides of this proposal don't justify the growth in complexity of the language. I obviously disagree (or else I wouldn't have written all this up!) and am using this thread to try to convince you that it is. But even if I'm unsuccessful maybe this will put the idea on someone else's radar who might come up with a better and more compelling suggestion down the road. Maybe something like how people wanted switch-cases and they were judged to be not sufficiently worth it, but then led to the match statement later on (basically switch-cases on steroids that also do your laundry and walk your dog). So cheers, everyone. On Tue, May 4, 2021 at 6:53 PM Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:
On Wed, May 5, 2021 at 11:34 AM Matt del Valle <matthewgdv@gmail.com> wrote:
So if A is capable of looking up "B.C.D", but it's also capable of looking up "B" and following the chain... what happens when you reassign something? What does this line of code do?
A.B.C.D = "spam"
Does it change what getattr(A, "B.C.D") returns?
I've found that on this list there are a few people who fancy themselves gatekeepers and will dismiss virtually any new idea that gets posted here out-of-hand in a way that is often pretty unkind, and without really offering up much of a rationale. Maybe it's just a fundamental difference of mindset, but to me, the idea that python is 'finished' and should remain largely untouched forever is kind of horrifying.
That's not why ideas get dismissed out of hand. Onus is not on the status quo to prove itself; onus is on the proposal to show its value. If you want your idea to be taken seriously, you have to demonstrate that it would give some sort of real improvement.
That's not the same, though. Binary operators are NOT syntactic sugar for method calls - they do a number of checks that allow for reflected methods and such.
There are some things that are syntactic sugar for gigantic blobs (even worse than this one - check out what "yield from iter" actually does), and there, the benefit is much easier to demonstrate. The expressiveness of a simple and clear line of code IS the benefit.
So yeah, to make this point explicit. This is a proposal for syntactic sugar. It doesn't really add anything you can't do currently, although many of the things that would be idiomatic and trivial with `namespace` would be horrible antipatterns without it (and equivalent implementations available currently are much more verbose and less performant). but it should hopefully allow for more organized and DRYer code, and more easy usage of namespacing in situations where it makes sense and would be otherwise non-trivial to add them in (there are several examples of this in this thread and in the doc).
Arguing that this is pointless because it doesn't add completely new functionality misses the mark in the same way that it's not particularly helpful to argue that listcomps are pointless because for-loops exist. When they were first introduced many people hated them, and yet they (and other types of comprehensions) are one of the most beloved features of the language nowadays.
Since Python is Turing-complete, there's nothing that will ever truly be "new functionality" in that sense. That's not the point. The point is whether it allows programmer intent to be expressed in a better way - and, specifically, whether the benefits in expressiveness are worth the costs.
I can respect that many people will feel that the upsides of this proposal don't justify the growth in complexity of the language.
Precisely. :)
Maybe something like how people wanted switch-cases and they were judged to be not sufficiently worth it, but then led to the match statement later on (basically switch-cases on steroids that also do your laundry and walk your dog).
That's actually a great example, since they've been requested so many times over the years. Why did the proposals keep getting knocked back? Because they didn't really offer any notable advantage. The match statement has some *huge* advantages, but it also has a lot of costs (it's complex, does a lot of different things, etc, etc), and the pushback has been quite significant. So I'll give you the same advice that many people have been given on this list: Get concrete examples. Go through the Python standard library and pull out a number of situations that would be able to be rewritten using this namespace syntax, and show how they'd be better for it. (Or some other large codebase, but the standard library is usually the best place to start.) Toy examples like "A.B.C.D" are all well and good for explaining what the syntax *does*, but they're not very good at explaining why the syntax *has value*. ChrisA
My comments follow, interleaved with Matt's. On Mon, May 03, 2021 at 11:30:51PM +0100, Matt del Valle wrote:
Indeed, that is something I often miss: a way to conceptually group named functions, classes and variables which is lighter weight than separating them into a new file. But you don't need a new keyword for that. A new keyword would be nice, but grouping alone may not be sufficient to justify a keyword.
We don't need a new keyword for people to separate names with dots. Although I agree with your position regarding nested APIs, *to a point*, I should mention that, for what it is worth, it goes against the Zen: Flat is better than nested. [...]
- you can put functions inside a namespace block, which would become methods if you had put them in a class block
This is a feature which I have *really* missed.
On the other hand, I don't think I like this. What I would expect is that namespaces ought to be a separate scope. To give an example: def spam(): return "spam spam spam!" def eggs(): return spam() namespace Shop: def spam(): return "There's not much call for spam here." def eggs(): return spam() print(eggs()) # should print "spam spam spam!" print(Shop.eggs()) # should print "There's not much call for spam here." If we have a namespace concept, it should actually be a namespace, not an weird compiler directive to bind names in the surrounding global scope.
- it mode clearly indicates intent (you don't want a whole new class, just a new namespace)
Indeed.
I'm not entirely sure what this means.
Oh, I hope that's not what you consider a good use-case! For starters, the "before" with two classes seems to be a total misuse of classes. `Legs` is a do-nothing class, and `self.legs` seems to be adding an unnecessary level of indirection that has no functional or conceptual benefit. I hope that the purpose of "namespace" is not to encourage people to write bad code like the above more easily.
Surely that's just a limitation of the *specific* tools. There is no reason why they couldn't be upgraded to understand SimpleNamespace. [...]
If I have understood you, that means that things will break when you do: Z = A del A Z.B.C # NameError name 'A.B' is not defined Objects should not rely on their parents keeping the name they were originally defined under. [...]
That's no different from the situation today: obj = spam.eggs.cheese.aardvark.hovercraft obj.eels # only one lookup needed
Can I just say that referencing `vars(sys.modules[__name__])` *really* works against the clarity of your examples? Are there situations where that couldn't be written as globals()["constants.NAMESPACED_CONSTANT"] instead? And remind me, what's `Truevars`? -- Steve
A is capable of looking up B.C.D in one step if you use `getattr` because the way A forwards on attribute lookups is roughly equivalent to (copy-pasting this from an earlier post): class NamespaceProxy: ... def __getattr__(self, name): return getattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}") def __setattr__(self, name, value): setattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}", value) So it would concatenate its own name ("A") with a dot with the name being looked up ("B.C.D") and then serve up globals()['A.B.C.D'], which is the key under which the value of D is stored. It can also follow the chain because:
globals()['A'] <namespace object <A> of <module '__main__' (built-in)>>
globals()['A.B'] <namespace object <A.B> of <module '__main__' (built-in)>>
globals()['A.B.C'] <namespace object <A.B.C> of <module '__main__' (built-in)>>
These also exist in the module globals(), (they were set there with those names in the `namespace` statement). A chained lookup just involves looking these namespace objects up one-by-one in the globals().
A.B.C.D = "spam"
Does it change what getattr(A, "B.C.D") returns?
Yes, it would, because it is no different to just explicitly setting: globals()[''A.B.C.D'] = "spam" That's not why ideas get dismissed out of hand. Onus is not on the
I get this, I do. And you're not wrong. I guess what I'm trying to say is that there's a lot to be said for being kind. I've seen several instances of people coming here with a new idea and sort-of cringed sympathetically at the kind of response they've gotten. I can just imagine someone relatively new to python getting really puppy-dog excited because they think they've got something worthwhile to contribute (without understanding why it may not work) and jumping through a few hoops to get onto this list only to be practically yelled out of the room by cranky old developers. What a horribly demoralizing experience that must be! There are some people who genuinely act like it is a personal attack on them when someone else comes along to discuss their shiny new idea on a list intended for... discussing potential new language ideas. I don't think that reflects on our community very well to newcomers. It costs us nothing to be kind, even if you *do* have to say something another person might not like to hear. There are ways to soften it and be informational rather than rude. Especially in a medium like this where tone doesn't carry and it's easy to misconstrue something as harsher than the person meant it. Even little things like your smiley face earlier: Precisely. :)
Go a shockingly long way towards conveying that we're on the same page having a discussion to try and improve python together, rather than an ego-fuelled shouting match. That's not the same, though. Binary operators are NOT syntactic sugar
for method calls - they do a number of checks that allow for reflected methods and such.
Fair enough, I just used the first example that came to mind, which in this case was wrong. My point was more about syntactic sugar in general.
I'm guessing this was a typo and you meant to type: print(spam()) # should print "spam spam spam!" print(Shop.spam()) # should print "There's not much call for spam here." Because if you did, then this is precisely how it *would* work under this proposal. :) In the module globals() you would have the first version of `spam` under the key 'spam', and the second one under 'Shop.spam', which is the object that the attribute lookup `Shop.spam` would return If we have a namespace concept, it should actually be a namespace, not
an weird compiler directive to bind names in the surrounding global scope.
This is an implementation detail that the end-user would basically not need to worry about except when iterating dynamically over the globals()/class __dict__/object __dict__. But that's the entire point. `namespace` is intended to build namespaces within some scope, not to create whole new scopes. If you wanted a new scope you could create a new class/object/module.
See I disagree on this point. I agree that the 'before' code is not at all ideal. But it is what you would have to do currently if you wanted to namespace out your Leg objects with statically analyzable names (instead of using a SimpleNamespace, or a dict, or similar). Sure, in this toy example the benefit is minor. There's only two attributes in `Biped` and they are both legs. You could get away with writing them as `self.left_leg` and `self.right_leg`. But I would argue that this violates DRY principles somewhat (just like with the example with pseudo-namespaced methods using underscores that Paul mentioned earlier) and feels hacky. What if the project requirements later on changed and you needed to change the '_leg' suffix to '_ambulatory_appendage'? For two attributes it's not so bad, but what if you've got an `Octoped` class with eight distinct and uniquely identifiable legs? At some point in the process of refactoring 8 separate attributes you would probably realize that your code is repeating itself, which is one of the very first lessons programming students are taught it should never do. I would argue that this is better: class Octoped: def __init__(self): self.foo = "bar" namespace self.leg: namespace left: anterior = LeftLeg() posterior = LeftLeg() ... # other anatomical terms namespace right: anterior = RightLeg() ... # same as above And now you have neatly namespaced legs, accessible like this: octoped = Octoped() octoped.leg.left.posterior octoped.leg.right.anterior octoped.foo Creating lots of unnecessary classes *just for namespacing*, as we currently have to resort to, isn't ideal. What you're calling 'an unnecessary level of indirection', quickly becomes a big boost to clarity (both in the written code and when using said code in an IDE) in more complex situations. Ask yourself why library authors namespace out the API of their library across multiple modules/packages. It's the same reason. Surely that's just a limitation of the *specific* tools. There is no
reason why they couldn't be upgraded to understand SimpleNamespace.
See but I would argue that the reason static analysis tools don't give autocompletions for SimpleNamespace is fundamentally the same reason they don't give them for dicts. Basically, those are mappings that are *intended* to change over the course of their lifetime and have keys added/popped/altered (for example, in argparse). So attempting to statically analyze them would be not very useful at best, and misleading at worst. This is different to attributes, which are considered to be more 'permanent' (for lack of a better word). When using an attribute you haven't assigned to elsewhere most linters will give you an 'undefined reference' warning, for instance. Depending on your strictness settings they might also warn you if you try to assign to an attribute that you haven't declared within the constructor. I don't think it would be appropriate to strictly statically check SimpleNamespace any more than it would be to try to do it with dictionary keys. Different tools for different jobs. If I have understood you, that means that things will break when you do:
This wouldn't happen. It would work as expected. You would still have a reference to `namespace A` as normal, only bound to the name `Z` rather than `A`. I don't see anything in the semantics of `del` that would cause this to behave in an unexpected way. I guess you might be confused that something being set on Z later on, like:
Z.foo = "bar"
is saved under `globals()['A.foo']` rather than `globals()[Z.foo]`, but if you follow the namespace object referenced by `Z` back to its source you would see that it's because it was originally declared as `namespace A`. Basically, a namespace acquires its name during the namespace block (taking into account any other namespace blocks it is nested under) and then doesn't change when you pass it around as a reference. That's no different from the situation today:
obj = spam.eggs.cheese.aardvark.hovercraft obj.eels # only one lookup needed
Yes, exactly. I've been trying to explain that attribute lookup for namespaces uses *the same* semantics as normal attribute lookup (it just looks things up on its parent scope rather than 'owning' the attributes itself). It doesn't require any special-cased semantics. Can I just say that referencing `vars(sys.modules[__name__])` *really*
You're right. I forgot that globals() is literally just a reference to the module '__dict__', not some magical accessor object for module attributes. I'll use `globals` from now on, starting with this post. Thank you :) And remind me, what's `Truevars`?
I think when quoting me your editor must've moved the `True` to the following line, the original version is: vars(sys.modules[__name__])["constants.NAMESPACED_CONSTANT"] = Truevars(sys.modules[__name__])["constants.inner.ANOTHER_CONSTANT"] = "hi" Although, as you've pointed out, it is better written as: globals()["constants.NAMESPACED_CONSTANT"] = Trueglobals()["constants.inner.ANOTHER_CONSTANT"] = "hi" On Wed, May 5, 2021 at 7:11 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Wed, May 5, 2021 at 8:33 PM Matt del Valle <matthewgdv@gmail.com> wrote:
Creating lots of unnecessary classes just for namespacing, as we currently have to resort to, isn't ideal.
What you're calling 'an unnecessary level of indirection', quickly becomes a big boost to clarity (both in the written code and when using said code in an IDE) in more complex situations. Ask yourself why library authors namespace out the API of their library across multiple modules/packages. It's the same reason.
I absolutely agree that creating lots of classes just for namespacing is unideal. Where we disagree is where the problem is. You think the problem is in the "creating lots of classes". I think the problem is in "needing that much namespacing" :) That's another reason to go looking for *real* examples. Every example given in this thread can be dismissed for being too simple, while simultaneously being defended as "just an example". Three levels of namespacing just to find one leg out of eight is a complete waste of effort. So.... find us some better examples to discuss, examples that really support your proposal. These ones, as mentioned, are great for explaining/discussing what it does, but not for explaining why it needs to be done. ChrisA
On Wed, 5 May 2021 at 11:33, Matt del Valle <matthewgdv@gmail.com> wrote:
I'm not the OP, but I read their question precisely as it was written. The global eggs() returns the value from calling spam() and should use the *global* spam. The eggs in namespace Shop calls spam and returns its value, and I'd expect that call to resolve to Shop.spam, using the namespace eggs is defined in. If that's not how you imagine namespaces working, I think they are going to be quite non-intuitive for at least a certain set of users (including me...) Paul
Whoops. I totally misread that! My brain unhelpfully supplied string quotes in the return statement of the `eggs` methods, rather than parsing them as function calls. ...for some reason I could not explain to you. Maybe I've grown too dependent on syntax highlighting! I see what Steven meant now. Ignore what I typed earlier, then. To be honest, I do completely agree. I think the most useful and least surprising implementation would be to give the namespace block its own temporary scope (with outer names still available), and then only at the end of the block just before the scope is destroyed do any names bound to it get set on the parent scope. So: def spam(): return "spam spam spam!" def eggs(): return spam() namespace Shop: old_spam = spam # No NameError, the `spam` from module scope is still accessible here if we want it! def spam(): return "There's not much call for spam here." new_spam = spam # this now refers to the function defined immediately above def eggs(): return spam() # this refers to the same object as `new_spam`, not the `spam` from globals() # at this point globals()['Shop.spam'] would still raise a NameError. It doesn't exist yet! namespace Shelf: this_is_shop_spam = spam # However, we can access it from inside another nested namespace as a simple name. The scoping rules on nested classes would not allow this! # at this point globals()['Shop.spam'] exists! Sorry Steven! I totally misunderstood your example. It was a really good point. Thanks :) On Wed, May 5, 2021 at 12:06 PM Paul Moore <p.f.moore@gmail.com> wrote:
Nope! I know it's hard to convey tone through text, so I just wanna assure you I'm not being snarky or whatever. With that said, I'd ask that you please read through the full proposal before making assumptions like 'this is exactly the same as types.SimpleNamespace'. I know it's a bit long (sorry for that!), but that's the only way we can be on the same page for this discussion. There are several code examples in there which do things that could not be done by using types.SimpleNamespace. Also, types.SimpleNamespace doesn't offer the same declarative syntax (where assigning to names within its block gets transformed into attributes). Its interface is mostly dynamic, whereas the way a 'namespace' block would be used would be declarative. The closest analogue to the proposed 'namespace' block, would not be types.SimpleNamespace, but rather, a 'class' definition that makes use of the syntactic sugar for declaring class attributes when assigning to names within the class block. It might well be that the consensus is that this suggestion is *too similar* to that and *doesn't offer enough* to warrant adding a new keyword. That's fine. I wrote this up knowing that it's a pretty high bar to add any new keyword to an existing language and there's a good chance this doesn't go anywhere. I still just wanted to get a conversation going to get a feel for the community's appetite for something like this. But yeah, probably the biggest difference between a namespace block and a class block would be that a namespace block within a class block still allows any functions that are defined within it to become bound methods for that class, which would not be the case using a nested class block. Also, attributes are not actually stored within the namespace, but rather within whatever scope the namespace block ultimately lives. This is very different to how nested classes or types.SimpleNamespace works. I'm happy to answer any questions, just please read the full proposal first :) On Sun, May 2, 2021 at 1:25 AM David Mertz <mertz@gnosis.cx> wrote:
I read the entire long description, but admittedly skimmed some. Certainly 90%+ can be done, in almost the same way, with `class Mine(Simple namespace)`, but I could have missed some corner. Perhaps highlight that. But I am certain I remain -1, in any event. On Sun, May 2, 2021, 5:16 AM Matt del Valle <matthewgdv@gmail.com> wrote:
On Sun, 2 May 2021 at 00:57, Matt del Valle <matthewgdv@gmail.com> wrote:
Sometimes it’s useful to write classes that are similar to: class proxy: …. def __getattr__(self, name): return wrap(getattr(self.object, name)) How would this pattern interact with objects that use namespaces? I’m guessing the interpreter wouldn’t know to call getattr with ‘ns.attr’ rather than just ‘ns’? Steve
This shouldn't be a problem. For instance: class Wrapped: namespace foo: bar = True facade = proxy(Wrapped()) facade.foo.bar # True Basically, when you try to look up 'foo' on the proxy object, the '__getattr__' returns the namespace object, which then forwards on the second attribute lookup ('bar') to the 'Wrapped' class. It happens in two steps rather than one. So what actually gets looked up is vars(wrapped_instance)['foo.bar'], because the 'foo' object that is left behind by the namespace block forwards the attribute lookup on to the Wrapped class (while prepending itself to the dict lookup). Does that make sense? On Sun, May 2, 2021 at 10:30 AM Stestagg <stestagg@gmail.com> wrote:
A further point to Steve's example is that a proxy object like the one he described would probably have to change its implementation ever so slightly in order to work as expected in a multi-step lookup involving namespaces (so that the `wrap` function is only applied to the final value at the end). Probably something like this: from types import NamespaceType # similar to FunctionType class proxy: …. def __getattr__(self, name): if isinstance(value := getattr(self.object, name), NamespaceType): return type(self)(value) # create another proxy object for the next lookup return wrap(value) On Sun, May 2, 2021 at 10:45 AM Matt del Valle <matthewgdv@gmail.com> wrote:
OK, tell me what this does. namespace A: namespace B: foo = bar On Sat, May 1, 2021 at 7:55 PM Matt del Valle <matthewgdv@gmail.com> wrote:
-- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman@redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
So for simplicity let's assume that the content of 'bar' is 3, and let's assume that the module is __main__ Then:
vars(sys.modules[__name__])["A.B.foo"] 3
And it can be accessed in the usual way like:
A.B.foo 3
Likewise for the namespace objects that are created during the namespace block:
vars(sys.modules[__name__])["A"] <namespace object <A> of <module '__main__' (built-in)>>
A.B <namespace object <A.B> of <module '__main__' (built-in)>>
The key point here is that if you dynamically iterate over the contents of the module you will get all the objects that were set within any namespace blocks. Unlike nesting class blocks and using class attributes, you won't have to recursively descend into the namespaces. The namespace objects are left behind as a convenience to enable classic attribute access, but they don't 'own' the attributes that were set within their scope. The (in this case) module does. The only reference the namespace has is (in this case) to the module. Note that namespaces within modules aren't the most interesting case. Namespaces within class blocks and for class instance attributes are much more likely to be where this feature would offer big wins in code clarity. For a class block: class Example: namespace foo: def bar(self): ... # implementation goes here Then:
vars(Example)["foo.bar"] <function __main__.Example.foo.bar(self)>
Which should generally be accessed like:
Example.foo.bar <function __main__.Example.foo.bar(self)>
Likewise, the namespace object:
Example.foo <namespace object <foo> of <class '__main__. Example'>>
And from an instance the function becomes a method as normal:
There are code examples in here that hopefully give a better feel for the benefits of grouping methods of classes into namespaces: https://github.com/matthewgdv/namespace On Mon, May 3, 2021 at 1:25 PM Calvin Spealman <cspealma@redhat.com> wrote:
On Mon, 3 May 2021 at 19:24, David Mertz <mertz@gnosis.cx> wrote:
So yes... as I thought, SimpleNamespace does EVERYTHING described by the proposal, just without needing more keywords:
Except that the code and description of the proposal explicitly outline behaviours that SimpleNamespace does not provide (and aren’t trivially possible to add) It would be useful to discuss the relative merits/lack thereof of the differences (for the record, I am ambivalent about the benefits of the proposal being enough to warrant a language change)
On Mon, May 3, 2021 at 6:37 PM Stestagg <stestagg@gmail.com> wrote:
I've looked and been unable to find an example of that. Can you show one? -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
The first example in the doc lays out the difference: Assignments within the namespace block have this special behaviour whereby the assigned-to name is changed to be: ‘<namespace name>.<assignment name>’ And the assignment is made in the ‘parent scope’ of the namespace. I.e. (again, as described in the doc): class A: Namespace B: C = 1 Results in: A.__dict__ == {‘B.C’: 1, ‘B’: <namespace proxy>} Note the ‘B.C’ entry Now for me, the only place where is starts to be interesting is with methods within namespaces, where the ‘self’ binding is made against to top-level class, and not against the namespace. This does make for some nicer nested API definitions (a-la pandas DataFrame.str.xxx methods) where an intermediate name os used just to group and organise methods. On Mon, 3 May 2021 at 19:40, David Mertz <mertz@gnosis.cx> wrote:
On Mon, May 3, 2021 at 6:49 PM Stestagg <stestagg@gmail.com> wrote:
OK, that is a difference. I don't really understand why it's desirable, but I see the difference. I can make that happen with a metaclass that would allow: class A(MagicNamespace): class B: C = 1 A.__dict__ == {'B.C': 1, 'B': <class 'module.B'>} So that wouldn't be identical in the repr(), but is there something else important here? In general, I find it anathema to add syntax for something that is easily handled by existing inheritance mechanisms. If I want to, I can even make every class defined within the body of a MagicNamespace magically become a MagicNamespace too, if that's a thing someone wants. Actually, I might not even need a metaclass, a MagicNamespace.__new__() method might be adequate. I haven't fully thought it out. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
I've read the proposal, and this thread. Questions: 1. It seems that I could get close to what you're aiming for by just using underscores in names, and grouping them together in the class definition. Is there any advantage to declaring methods in a namespace beyond indentation and the dot notation? 2. If __dict__ contains "B.C" and "B", then presumably the interpreter would need to try combinations against the outer __dict__ as well as B. Is the namespace proxy you've mentioned intended to prevent further lookup in the "B" attribute? 3. Can namespaces be nested? If so, will their attributed they always resolve to flat set of attributes in the encapsulating class? 4. What would you expect getattr("A.B", "C") to yield? Paul On Mon, 2021-05-03 at 19:49 +0100, Stestagg wrote:
@Paul Bryan, I'll try to address your questions one-by-one 1. It seems that I could get close to what you're aiming for by just using
What you're suggesting (using underscores for namespacing) is something I've seen done many times (and done myself plenty of times) and it's precisely one of the main things that I think could be better under this proposal. Reusing a prefix like that across methods has always felt a bit hacky to me and like it doesn't follow the DRY principle very well. If you wanted to rename the prefix at a later point you would have to go through every single method that uses it, instead of just doing a single refactor/rename on the name at the top of the namespace block. But you've pretty much perfectly identified the benefits here, I'll just elaborate on them a bit. - the indentation visually separates blocks of conceptually-grouped attributes/methods in the actual code (a gain in clarity when code is read) - the dot notation you use to invoke such methods improves the experience for library consumers by giving a small amount conceptually-linked autocompletions at each namespace step within a class with a large API, rather getting a huge flat list. Furthermore, the proposed namespaces have other selling-points outside of the use-case of grouping methods within a class. While the benefit of namespaces within modules is more dubious because you can use class blocks for namespacing (and people often do), can see a few ways that the namespace proposal is better: - you can put functions inside a namespace block, which would become methods if you had put them in a class block - you don't have the same (in some cases extremely unintuitive) scoping/variable binding rules that you do within a class block (see the link in my doc). It's all just module scope. - it mode clearly indicates intent (you don't want a whole new class, just a new namespace) When using the namespaces within a method (using self): - It allows you to namespace out your instance attributes without needing to create intermediate objects (an improvement to the memory footprint, and less do-nothing classes to clutter up your codebase) - While the above point of space complexity will not alway be relevant I think the more salient point is that creating intermediate objects for namespacing is often cognitively more effort than it's worth. And humans are lazy creatures by nature. So I feel like having an easy and intuitive way of doing it would have a positive effect on people's usage patterns. It's one of those things where you likely wouldn't appreciate the benefits until you'd actually gotten to play around with it a bit in the wild. For example, you could rewrite this: class Legs: def __init__(self, left, right): self.left, self.right = left, right class Biped: def __init__(self): self.legs = Legs(left=LeftLeg(), right=RightLeg()) As this: class Biped: def __init__(self): namespace self.legs: left, right = LeftLeg(), RightLeg() And sure, the benefit for a single instance of this is small. But across a large codebase it adds up. It completely takes away the tradeoff between having neatly namespaced code where it makes sense to do so and writing a lot of needless intermediate classes. SimpleNamespace does not help you here as much as you would think because it cannot be understood by static code analysis tools when invoked like this: class Biped: def __init__(self): self.legs = SimpleNamespace(left=LeftLeg(), right=RightLeg()) So it is a terrible idea to use it in this way to write any kind of library code. You could invoke it declaratively: class Biped: def __init__(self): class Legs(SimpleNamespace): left, right = LeftLeg(), RightLeg() self.legs = Legs Not only is this an unintuitive usage pattern and creates lots of unnecessary classes (one for each new instance of Biped), but it significantly reduces code clarity (imagine having lots of these inside a single method). By contrast, a namespace block seems to me to increase code clarity, though I will grant that this is subjective. 2. If __dict__ contains "B.C" and "B", then presumably the interpreter
The namespace proxy must know its fully-qualified name all the way up to its parent scope (this is the bit that would require some magic in the python implementation), so it only needs to forward on a single attribute lookup to its parent scope. It does not need to perform several intermediate lookups on all of its parent namespaces. So in the case of: namespace A: namespace B: C = True
A.B <namespace object <A.B> of <module '__main__' (built-in)>>
Note that namespace B 'knows' that its name is 'A.B', not just 'B' The implementation of this namespace proxy object might look something like this, only implemented in C to hopefully be more performant than this: class NamespaceProxy: ... def __getattr__(self, name): return getattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}") def __setattr__(self, name, value): setattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}", value) The names of the dunders are hypothetical and not something I'm concerned about at this stage. Traversing all the way through A.B.C does involve 2 intermediate lookups (looking up 'A.B' on the parent scope from namespace A, then looking up 'A.B.C' on the parent scope from namespace A.B). But once you have a reference to a deeply nested namespace, looking up any value on it is only a single lookup step. 3. Can namespaces be nested? If so, will their attributed they always
resolve to flat set of attributes in the encapsulating class?
Yes, namespaces can be nested arbitrarily, and they will always set their attributes in the nearest real scope (module/class/locals). There's an example of this early on in the doc: namespace constants: NAMESPACED_CONSTANT = True namespace inner: ANOTHER_CONSTANT = "hi" Which is like: vars(sys.modules[__name__])["constants.NAMESPACED_CONSTANT"] = Truevars(sys.modules[__name__])["constants.inner.ANOTHER_CONSTANT"] = "hi" 4. What would you expect getattr(A.B, "C") to yield?
It would be no different than:
A.B.C
It happens in two steps: 1) A.B - this looks up vars(<module '__main__' (built-in)>)['A.B'], which is a namespace object 2) getattr(<namespace object <A.B> of <module '__main__' (built-in)>>, "C") - this looks up "C" on namespace "A.B", which forwards this lookup on to the module as: vars(<module '__main__' (built-in)>)['A.B.C'] The namespace proxies created in a namespace block do nothing more than forward on any attribute access/attribute setting operations performed on them to their parent scope, so you can do any attribute operations to them that you could do to a 'real' intermediate object. I hope I've answered everything with reasonable clarity. I'm kind of exhausted from writing this up, but I'll finally just quickly mention in response to David Mertz that your proposal to use a metaclass for this wouldn't really serve to provide virtually any of the benefits I listed out in my answer to Paul (part 1, in particular). it wouldn't work for modules/locals, and it can't be used at the instance level with `self`, only the class level. It would also restrict usage of other metaclasses. What it would end up doing (saving class attributes of nested classes to its own __dict__ with a fully-qualified name) is an implementation detail of my proposal rather than one of the primary benefits. From where I'm standing it feels like you just took an immediate initial dislike to the proposal and didn't even really give it a chance by actually thinking through the pros/cons before giving your -1. It's fine if you think it's not useful enough to be added. Maybe that's true. But it would be more helpful to maybe ask questions like other people are doing and be sure you understand the rationale/implications of the proposal before making your mind up. Three negative responses in you still didn't understand a really simple implementation point that's written very early on in the toy example of the doc, which differentiates it from types.SimpleNamespace. Then when Steve pointed that out to you, you immediately dug your heels in again without really thinking your 'it can be done with a metaclass' counterpoint all the way through. At least give yourself the chance to change your mind! Even if you end up settling on your initial intuition, at least that way you'll know that you've explored all the options and done your best to land on the most informed opinion you can. Anyways, I'm sorry if I totally misrepresented the situation. It's hard sometimes to tell over a medium like this what page someone else is on. I just figured I'd tell you how it felt to me. I hope I wasn't totally off-base. On Mon, May 3, 2021 at 8:16 PM Paul Bryan <pbryan@anode.ca> wrote:
Response inline. On Mon, 2021-05-03 at 23:30 +0100, Matt del Valle wrote:
Sure, it's a bit hacky, but virtually every editor can do it in a few keystrokes, assuming no naming collision that requires you to cherry- pick values to be substituted.
So, is it a prerequisite that whatever object in which I'm trying to establish a namespace must support getattr/setattr? Also, it occurs to me that if I can declare a namespace in any object, I might be tempted to (or might inadvertently) monkey patch external objects with it. Any thoughts on guarding against that, or is this "we're adults here" case?
A problem I sense here is the fact that the interpreter would always need to attempt to resolve "A.B.C" as getattr(getattr(A, "B"), "C") and getattr(A, "B.C"). Since the proxy would be there to serve up the namespace's attributes, why not just let it and do away with "B.C" in A.__dict__? What will the performance costs be of attempting to get an attribute in two calls instead of one?
So, if in a nested scenario, A.B.C.D, I'm trying to understand the combination of getattr calls to resolve D. Would it just still be two attempts, getattr(A, "B.C.D") and getattr(getattr(getattr(A, "B"), "C"), "D")? If it were to become a Cartesian product of calls, there'll likely be a problem. 🤔️
On 04/05/2021 14:39, Paul Bryan wrote:
I don't pretend to fully understand the proposal and how it would be implemented, but IMO adding an overhead (not to mention more complicated semantics) to *every* chained attribute lookup is enough to kill the proposal, given that it seems to have relatively slight benefits. Best wishes Rob Cliffe
It's a good thing that isn't the case then :p Sorry, couldn't resist that quip. But more seriously, this would not be an issue. Normal attribute lookups would remain wholly unchanged. Because attribute lookups on namespace proxies *are* normal attribute lookups. That's the entire reason I've proposed it the way I have (well, that and the fact that it allows methods within namespaces to still bind to the parent class basically 'for free' without requiring any magical behaviour). Because aside from the magic behaviour of the namespace block (the way it binds names to its parent scope) it does not require *any* changes to the current attribute lookup semantics. In fact, it is built entirely on them. The only *possible* difference would be that perhaps namespace proxies could be special-cased if any room for optimization could be found at the implementation stage (purely hypothetical) to attempt to make them more performant than the rough python pseudocode sketch of their behaviour I posted yesterday in response to Paul. So fear not, no current semantics would be harmed in the making of this film keyword. So, is it a prerequisite that whatever object in which I'm trying to
establish a namespace must support getattr/setattr?
I'd imagine so, yes. At the very least, any namespaceable (new word!) object would have to have some sort of __dict__ or equivalent (the built-in `vars` would have to work on them), because otherwise I don't see how they could store attributes. Also, it occurs to me that if I can declare a namespace in any object, I
Nothing stops you from monkey-patching objects right now. Declaring namespaces on arbitrary live objects you've obtained from who-knows-where would have all the same benefits to code conciseness and clarity as declaring them on `self` within a method, versus conventional monkey-patching. You could, for instance, group all the attributes you're monkey-patching in under a single namespace with a descriptive name (namespace custom? namespace extension? namespace monkey???) to keep track of what belongs to the original object and what doesn't, more easily. Monkey-patching is often considered an anti-pattern, but sometimes there's a library that does 99% of what you need but without that last 1% it's worthless for your use-case. Rather than forking and rewriting the entire library yourself, sometimes monkey-patching is a better solution. Practicality over purity, right? So yes, this definitely falls under the "consenting adults" category. A problem I sense here is the fact that the interpreter would always need
Well, the performance would be, at worst, identical to a set of chained attribute lookups currently, which also have to be done in multiple steps, one at a time. We want to keep to existing python semantics as much as possible so that the change is minimally invasive (As Rob Cliffe pointed out, this is a proposal for syntactic sugar, so it probably doesn't warrant changes to the semantics of something.so fundamental to the language as attribute lookups). Buy hey, if there's some room for optimization under-the-hood that would allow nested namespace lookups to be done in a single step in certain situations that would be undetectable to an end-user of the language, then that would be fine too, assuming it didn't add too much burden of maintenance. I somewhat doubt it would be possible to do that though, because the namespace proxies would be python objects like any other. You can pass references to them around. So in the case of: namespace A: namespace B: C = "foo" You could just grab a reference to B:
ref = A.B
and then later on look up C on the reference:
ref.C # this looks up vars(sys.modules[__name__])['A.B.C'] 'foo'
The attribute lookup behavior behaves exactly like you would expect any similar set of chained attribute lookups to behave. No different than: class A: class B: C = "foo" So, if in a nested scenario, A.B.C.D, I'm trying to understand the
The only way you could ever resolve D in a single attribute lookup like getattr(A, "B.C.D") is if you literally typed that statement out verbatim:
getattr(A, "B.C.D")
That would actually work, because `namespace A` would prepend its name to the attribute lookup and forward it on to its parent scope, serving up (sys.modules[__name__])['A.B.C.D'], which is where the key under which D is stored in the module globals. But if you just look it up using normal attribute access syntax, like you would 99% of the time:
A.B.C.D
Then it would be done in 3 separate lookups, one for each dot, exactly the way you described: getattr(getattr(getattr(A, "B"), "C"), "D") Looking B up on `namespace A` gets you `namespace A.B`, looking up C on `namespace A.B` gets you `namespace A.B.C`, and looking up D on `namespace A.B.C` finally gets you a reference to whatever D contains. This isn't a cartesian product though. It's just one attribute lookup per level of reference nesting, just the same as any other form of attribute access. As a side-note to everyone who has asked questions and participated in the discussion so far, thank you for at least entertaining my proposal. I'm under no delusions about the incredibly high bar that adding a new keyword to the language represents. But I still think discussions like this are important! Even if only one in every hundred such ideas actually has potential, if we don't brainstorm them up and discuss them as a community we'd never end up finding the ones that are valuable enough to seriously consider taking forward to a PEP. I've found that on this list there are a few people who fancy themselves gatekeepers and will dismiss virtually any new idea that gets posted here out-of-hand in a way that is often pretty unkind, and without really offering up much of a rationale. Maybe it's just a fundamental difference of mindset, but to me, the idea that python is 'finished' and should remain largely untouched forever is kind of horrifying. Programming languages that don't develop and evolve just become obsolete. It seems to me that with literally every recent new enhancement to the language that I've been incredibly excited about in the last few years (f-strings, assignment expressions, pattern matching) there's been a huge number of people loudly complaining that 'we don't really need this' and it just baffles me. Of course we don't *need* syntactic sugar. We could all write code like this:
(3).__add__((4).__mul__(2)) 11
rather than:
3 + 4*2
and evens = [] for num in range(1, 11): if not num % 2: evens.append(num) rather than: [num for num in range(1, 11) if not num % 2] But I think it's hard to argue that a language without any syntactic sugar is better, or for that matter is something anyone would actually enjoy using. So yeah, to make this point explicit. *This is a proposal for syntactic sugar*. It doesn't really add anything you can't do currently, although many of the things that would be idiomatic and trivial with `namespace` would be horrible antipatterns without it (and equivalent implementations available currently are much more verbose and less performant). but it should hopefully allow for more organized and DRYer code, and more easy usage of namespacing in situations where it makes sense and would be otherwise non-trivial to add them in (there are several examples of this in this thread and in the doc). Arguing that this is pointless because it doesn't add completely new functionality misses the mark in the same way that it's not particularly helpful to argue that listcomps are pointless because for-loops exist. When they were first introduced many people hated them, and yet they (and other types of comprehensions) are one of the most beloved features of the language nowadays. I can respect that many people will feel that the upsides of this proposal don't justify the growth in complexity of the language. I obviously disagree (or else I wouldn't have written all this up!) and am using this thread to try to convince you that it is. But even if I'm unsuccessful maybe this will put the idea on someone else's radar who might come up with a better and more compelling suggestion down the road. Maybe something like how people wanted switch-cases and they were judged to be not sufficiently worth it, but then led to the match statement later on (basically switch-cases on steroids that also do your laundry and walk your dog). So cheers, everyone. On Tue, May 4, 2021 at 6:53 PM Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:
On Wed, May 5, 2021 at 11:34 AM Matt del Valle <matthewgdv@gmail.com> wrote:
So if A is capable of looking up "B.C.D", but it's also capable of looking up "B" and following the chain... what happens when you reassign something? What does this line of code do?
A.B.C.D = "spam"
Does it change what getattr(A, "B.C.D") returns?
I've found that on this list there are a few people who fancy themselves gatekeepers and will dismiss virtually any new idea that gets posted here out-of-hand in a way that is often pretty unkind, and without really offering up much of a rationale. Maybe it's just a fundamental difference of mindset, but to me, the idea that python is 'finished' and should remain largely untouched forever is kind of horrifying.
That's not why ideas get dismissed out of hand. Onus is not on the status quo to prove itself; onus is on the proposal to show its value. If you want your idea to be taken seriously, you have to demonstrate that it would give some sort of real improvement.
That's not the same, though. Binary operators are NOT syntactic sugar for method calls - they do a number of checks that allow for reflected methods and such.
There are some things that are syntactic sugar for gigantic blobs (even worse than this one - check out what "yield from iter" actually does), and there, the benefit is much easier to demonstrate. The expressiveness of a simple and clear line of code IS the benefit.
So yeah, to make this point explicit. This is a proposal for syntactic sugar. It doesn't really add anything you can't do currently, although many of the things that would be idiomatic and trivial with `namespace` would be horrible antipatterns without it (and equivalent implementations available currently are much more verbose and less performant). but it should hopefully allow for more organized and DRYer code, and more easy usage of namespacing in situations where it makes sense and would be otherwise non-trivial to add them in (there are several examples of this in this thread and in the doc).
Arguing that this is pointless because it doesn't add completely new functionality misses the mark in the same way that it's not particularly helpful to argue that listcomps are pointless because for-loops exist. When they were first introduced many people hated them, and yet they (and other types of comprehensions) are one of the most beloved features of the language nowadays.
Since Python is Turing-complete, there's nothing that will ever truly be "new functionality" in that sense. That's not the point. The point is whether it allows programmer intent to be expressed in a better way - and, specifically, whether the benefits in expressiveness are worth the costs.
I can respect that many people will feel that the upsides of this proposal don't justify the growth in complexity of the language.
Precisely. :)
Maybe something like how people wanted switch-cases and they were judged to be not sufficiently worth it, but then led to the match statement later on (basically switch-cases on steroids that also do your laundry and walk your dog).
That's actually a great example, since they've been requested so many times over the years. Why did the proposals keep getting knocked back? Because they didn't really offer any notable advantage. The match statement has some *huge* advantages, but it also has a lot of costs (it's complex, does a lot of different things, etc, etc), and the pushback has been quite significant. So I'll give you the same advice that many people have been given on this list: Get concrete examples. Go through the Python standard library and pull out a number of situations that would be able to be rewritten using this namespace syntax, and show how they'd be better for it. (Or some other large codebase, but the standard library is usually the best place to start.) Toy examples like "A.B.C.D" are all well and good for explaining what the syntax *does*, but they're not very good at explaining why the syntax *has value*. ChrisA
My comments follow, interleaved with Matt's. On Mon, May 03, 2021 at 11:30:51PM +0100, Matt del Valle wrote:
Indeed, that is something I often miss: a way to conceptually group named functions, classes and variables which is lighter weight than separating them into a new file. But you don't need a new keyword for that. A new keyword would be nice, but grouping alone may not be sufficient to justify a keyword.
We don't need a new keyword for people to separate names with dots. Although I agree with your position regarding nested APIs, *to a point*, I should mention that, for what it is worth, it goes against the Zen: Flat is better than nested. [...]
- you can put functions inside a namespace block, which would become methods if you had put them in a class block
This is a feature which I have *really* missed.
On the other hand, I don't think I like this. What I would expect is that namespaces ought to be a separate scope. To give an example: def spam(): return "spam spam spam!" def eggs(): return spam() namespace Shop: def spam(): return "There's not much call for spam here." def eggs(): return spam() print(eggs()) # should print "spam spam spam!" print(Shop.eggs()) # should print "There's not much call for spam here." If we have a namespace concept, it should actually be a namespace, not an weird compiler directive to bind names in the surrounding global scope.
- it mode clearly indicates intent (you don't want a whole new class, just a new namespace)
Indeed.
I'm not entirely sure what this means.
Oh, I hope that's not what you consider a good use-case! For starters, the "before" with two classes seems to be a total misuse of classes. `Legs` is a do-nothing class, and `self.legs` seems to be adding an unnecessary level of indirection that has no functional or conceptual benefit. I hope that the purpose of "namespace" is not to encourage people to write bad code like the above more easily.
Surely that's just a limitation of the *specific* tools. There is no reason why they couldn't be upgraded to understand SimpleNamespace. [...]
If I have understood you, that means that things will break when you do: Z = A del A Z.B.C # NameError name 'A.B' is not defined Objects should not rely on their parents keeping the name they were originally defined under. [...]
That's no different from the situation today: obj = spam.eggs.cheese.aardvark.hovercraft obj.eels # only one lookup needed
Can I just say that referencing `vars(sys.modules[__name__])` *really* works against the clarity of your examples? Are there situations where that couldn't be written as globals()["constants.NAMESPACED_CONSTANT"] instead? And remind me, what's `Truevars`? -- Steve
A is capable of looking up B.C.D in one step if you use `getattr` because the way A forwards on attribute lookups is roughly equivalent to (copy-pasting this from an earlier post): class NamespaceProxy: ... def __getattr__(self, name): return getattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}") def __setattr__(self, name, value): setattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}", value) So it would concatenate its own name ("A") with a dot with the name being looked up ("B.C.D") and then serve up globals()['A.B.C.D'], which is the key under which the value of D is stored. It can also follow the chain because:
globals()['A'] <namespace object <A> of <module '__main__' (built-in)>>
globals()['A.B'] <namespace object <A.B> of <module '__main__' (built-in)>>
globals()['A.B.C'] <namespace object <A.B.C> of <module '__main__' (built-in)>>
These also exist in the module globals(), (they were set there with those names in the `namespace` statement). A chained lookup just involves looking these namespace objects up one-by-one in the globals().
A.B.C.D = "spam"
Does it change what getattr(A, "B.C.D") returns?
Yes, it would, because it is no different to just explicitly setting: globals()[''A.B.C.D'] = "spam" That's not why ideas get dismissed out of hand. Onus is not on the
I get this, I do. And you're not wrong. I guess what I'm trying to say is that there's a lot to be said for being kind. I've seen several instances of people coming here with a new idea and sort-of cringed sympathetically at the kind of response they've gotten. I can just imagine someone relatively new to python getting really puppy-dog excited because they think they've got something worthwhile to contribute (without understanding why it may not work) and jumping through a few hoops to get onto this list only to be practically yelled out of the room by cranky old developers. What a horribly demoralizing experience that must be! There are some people who genuinely act like it is a personal attack on them when someone else comes along to discuss their shiny new idea on a list intended for... discussing potential new language ideas. I don't think that reflects on our community very well to newcomers. It costs us nothing to be kind, even if you *do* have to say something another person might not like to hear. There are ways to soften it and be informational rather than rude. Especially in a medium like this where tone doesn't carry and it's easy to misconstrue something as harsher than the person meant it. Even little things like your smiley face earlier: Precisely. :)
Go a shockingly long way towards conveying that we're on the same page having a discussion to try and improve python together, rather than an ego-fuelled shouting match. That's not the same, though. Binary operators are NOT syntactic sugar
for method calls - they do a number of checks that allow for reflected methods and such.
Fair enough, I just used the first example that came to mind, which in this case was wrong. My point was more about syntactic sugar in general.
I'm guessing this was a typo and you meant to type: print(spam()) # should print "spam spam spam!" print(Shop.spam()) # should print "There's not much call for spam here." Because if you did, then this is precisely how it *would* work under this proposal. :) In the module globals() you would have the first version of `spam` under the key 'spam', and the second one under 'Shop.spam', which is the object that the attribute lookup `Shop.spam` would return If we have a namespace concept, it should actually be a namespace, not
an weird compiler directive to bind names in the surrounding global scope.
This is an implementation detail that the end-user would basically not need to worry about except when iterating dynamically over the globals()/class __dict__/object __dict__. But that's the entire point. `namespace` is intended to build namespaces within some scope, not to create whole new scopes. If you wanted a new scope you could create a new class/object/module.
See I disagree on this point. I agree that the 'before' code is not at all ideal. But it is what you would have to do currently if you wanted to namespace out your Leg objects with statically analyzable names (instead of using a SimpleNamespace, or a dict, or similar). Sure, in this toy example the benefit is minor. There's only two attributes in `Biped` and they are both legs. You could get away with writing them as `self.left_leg` and `self.right_leg`. But I would argue that this violates DRY principles somewhat (just like with the example with pseudo-namespaced methods using underscores that Paul mentioned earlier) and feels hacky. What if the project requirements later on changed and you needed to change the '_leg' suffix to '_ambulatory_appendage'? For two attributes it's not so bad, but what if you've got an `Octoped` class with eight distinct and uniquely identifiable legs? At some point in the process of refactoring 8 separate attributes you would probably realize that your code is repeating itself, which is one of the very first lessons programming students are taught it should never do. I would argue that this is better: class Octoped: def __init__(self): self.foo = "bar" namespace self.leg: namespace left: anterior = LeftLeg() posterior = LeftLeg() ... # other anatomical terms namespace right: anterior = RightLeg() ... # same as above And now you have neatly namespaced legs, accessible like this: octoped = Octoped() octoped.leg.left.posterior octoped.leg.right.anterior octoped.foo Creating lots of unnecessary classes *just for namespacing*, as we currently have to resort to, isn't ideal. What you're calling 'an unnecessary level of indirection', quickly becomes a big boost to clarity (both in the written code and when using said code in an IDE) in more complex situations. Ask yourself why library authors namespace out the API of their library across multiple modules/packages. It's the same reason. Surely that's just a limitation of the *specific* tools. There is no
reason why they couldn't be upgraded to understand SimpleNamespace.
See but I would argue that the reason static analysis tools don't give autocompletions for SimpleNamespace is fundamentally the same reason they don't give them for dicts. Basically, those are mappings that are *intended* to change over the course of their lifetime and have keys added/popped/altered (for example, in argparse). So attempting to statically analyze them would be not very useful at best, and misleading at worst. This is different to attributes, which are considered to be more 'permanent' (for lack of a better word). When using an attribute you haven't assigned to elsewhere most linters will give you an 'undefined reference' warning, for instance. Depending on your strictness settings they might also warn you if you try to assign to an attribute that you haven't declared within the constructor. I don't think it would be appropriate to strictly statically check SimpleNamespace any more than it would be to try to do it with dictionary keys. Different tools for different jobs. If I have understood you, that means that things will break when you do:
This wouldn't happen. It would work as expected. You would still have a reference to `namespace A` as normal, only bound to the name `Z` rather than `A`. I don't see anything in the semantics of `del` that would cause this to behave in an unexpected way. I guess you might be confused that something being set on Z later on, like:
Z.foo = "bar"
is saved under `globals()['A.foo']` rather than `globals()[Z.foo]`, but if you follow the namespace object referenced by `Z` back to its source you would see that it's because it was originally declared as `namespace A`. Basically, a namespace acquires its name during the namespace block (taking into account any other namespace blocks it is nested under) and then doesn't change when you pass it around as a reference. That's no different from the situation today:
obj = spam.eggs.cheese.aardvark.hovercraft obj.eels # only one lookup needed
Yes, exactly. I've been trying to explain that attribute lookup for namespaces uses *the same* semantics as normal attribute lookup (it just looks things up on its parent scope rather than 'owning' the attributes itself). It doesn't require any special-cased semantics. Can I just say that referencing `vars(sys.modules[__name__])` *really*
You're right. I forgot that globals() is literally just a reference to the module '__dict__', not some magical accessor object for module attributes. I'll use `globals` from now on, starting with this post. Thank you :) And remind me, what's `Truevars`?
I think when quoting me your editor must've moved the `True` to the following line, the original version is: vars(sys.modules[__name__])["constants.NAMESPACED_CONSTANT"] = Truevars(sys.modules[__name__])["constants.inner.ANOTHER_CONSTANT"] = "hi" Although, as you've pointed out, it is better written as: globals()["constants.NAMESPACED_CONSTANT"] = Trueglobals()["constants.inner.ANOTHER_CONSTANT"] = "hi" On Wed, May 5, 2021 at 7:11 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Wed, May 5, 2021 at 8:33 PM Matt del Valle <matthewgdv@gmail.com> wrote:
Creating lots of unnecessary classes just for namespacing, as we currently have to resort to, isn't ideal.
What you're calling 'an unnecessary level of indirection', quickly becomes a big boost to clarity (both in the written code and when using said code in an IDE) in more complex situations. Ask yourself why library authors namespace out the API of their library across multiple modules/packages. It's the same reason.
I absolutely agree that creating lots of classes just for namespacing is unideal. Where we disagree is where the problem is. You think the problem is in the "creating lots of classes". I think the problem is in "needing that much namespacing" :) That's another reason to go looking for *real* examples. Every example given in this thread can be dismissed for being too simple, while simultaneously being defended as "just an example". Three levels of namespacing just to find one leg out of eight is a complete waste of effort. So.... find us some better examples to discuss, examples that really support your proposal. These ones, as mentioned, are great for explaining/discussing what it does, but not for explaining why it needs to be done. ChrisA
On Wed, 5 May 2021 at 11:33, Matt del Valle <matthewgdv@gmail.com> wrote:
I'm not the OP, but I read their question precisely as it was written. The global eggs() returns the value from calling spam() and should use the *global* spam. The eggs in namespace Shop calls spam and returns its value, and I'd expect that call to resolve to Shop.spam, using the namespace eggs is defined in. If that's not how you imagine namespaces working, I think they are going to be quite non-intuitive for at least a certain set of users (including me...) Paul
Whoops. I totally misread that! My brain unhelpfully supplied string quotes in the return statement of the `eggs` methods, rather than parsing them as function calls. ...for some reason I could not explain to you. Maybe I've grown too dependent on syntax highlighting! I see what Steven meant now. Ignore what I typed earlier, then. To be honest, I do completely agree. I think the most useful and least surprising implementation would be to give the namespace block its own temporary scope (with outer names still available), and then only at the end of the block just before the scope is destroyed do any names bound to it get set on the parent scope. So: def spam(): return "spam spam spam!" def eggs(): return spam() namespace Shop: old_spam = spam # No NameError, the `spam` from module scope is still accessible here if we want it! def spam(): return "There's not much call for spam here." new_spam = spam # this now refers to the function defined immediately above def eggs(): return spam() # this refers to the same object as `new_spam`, not the `spam` from globals() # at this point globals()['Shop.spam'] would still raise a NameError. It doesn't exist yet! namespace Shelf: this_is_shop_spam = spam # However, we can access it from inside another nested namespace as a simple name. The scoping rules on nested classes would not allow this! # at this point globals()['Shop.spam'] exists! Sorry Steven! I totally misunderstood your example. It was a really good point. Thanks :) On Wed, May 5, 2021 at 12:06 PM Paul Moore <p.f.moore@gmail.com> wrote:
participants (10)
-
Calvin Spealman -
Chris Angelico -
David Mertz -
Matt del Valle -
Paul Bryan -
Paul Moore -
Rob Cliffe -
Stestagg -
Steven D'Aprano -
Wes Turner