That said, though, I'm not sure that this was even what you were asking about :)

It would be significantly more helpful if the OP had an example for the specific case where they wanted contextually defined variables to take priority over the classdict, and explained _why_ this behavior might be desirable:

...

There is at least one case in which I would like to do the reverse, and have the classdict be secondary to (masked by) any variables defined in the context surrounding the execution of the class body.

...

I'm not sure what the best way to deal with this would be, but my first thought is to maybe have a property that can be set on the metaclass type such as `metacls.relictant_classdict = True`.

Describing the specific use case in detail and providing an example would also help with determining the practical value of adding the described property. I'm not certain that a property would be the best way to do it (a new optional parameter for the __prepare__ attribute might be an alternative). But before that, we have to determine whether or not the feature would have enough practical value to justify implementing in any capacity. This is of course assuming that what the OP wants doesn't already exist in some other form (if Chris's example isn't what the OP is looking for) or that it's even reasonably possible to implement in the first place. On Sat, Oct 19, 2019 at 11:39 PM Chris Angelico wrote:

On Sun, Oct 20, 2019 at 1:58 PM Steve Jorgensen wrote:

...

When using a custom classdict to implement a DSL or use in the body of a

class definition, from what I can tell by experiment, the classdict takes priority, and the surrounding context is only referenced if trying to get an item from the classdict raises `KeyError`.

...

There is at least one case in which I would like to do the reverse, and

have the classdict be secondary to (masked by) any variables defined in the context surrounding the execution of the class body.

...

I have been able to at least unmask builtins by having the classdict

object first try to get a result from `__builtins__` and then fall back to itself. Also, in the actual class body, declaring a variable as `global` seems to be a workaround for global variables, but that has to be done explicitly in the body of any subclass that needs it. Also, that does not handle non-globals in its surrounding context.

...

I'm not sure what the best way to deal with this would be, but my first

thought is to maybe have a property that can be set on the metaclass type such as `metacls.relictant_classdict = True`.

...

Not entirely sure I follow here; are you talking about unadorned names during the execution of the class body itself, or the way attribute lookup is done on the resulting object? If you want to be able to mess with attribute lookup even when the attribute does exist, __getattribute__ is your friend:

https://docs.python.org/3/reference/datamodel.html#object.__getattribute__

For changing the behaviour of class body execution, it's a bit harder. Do you actually need the class members to be available as unadorned names, but such that the surrounding context takes precedence? That could be awkward, since there's no way (as far as I know) to get a "hook" whenever NameError would have been raised. But you can return any dict-like object from the metaclass's __prepare__ method, and if its __getitem__ and __setitem__ are disconnected, you could get the effect you want. Here's one possibility that gets kinda close:

class BlackBox(dict): def __getitem__(self, item): if item == "v": return self raise KeyError __getattr__ = dict.__getitem__

class Meta(type): @classmethod def __prepare__(meta, cls, bases): return BlackBox()

var = "global" class Demo(metaclass=Meta): var = "local" print("var =", var) print("v.var =", v.var)

Accessing local names can be done with the "v." prefix, global names are directly accessible, but missing globals won't be looked up locally.

That said, though, I'm not sure that this was even what you were asking about :)

ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/64VR43... Code of Conduct: http://python.org/psf/codeofconduct/

We try to do the same thing in various libraries. We've settled on using existing python and end up with syntax like: class MyForm(Form): field = Field() or in your case class Colors(TokenContainer): red = Token() green = Token() blue = Token() (this is using tri.token). The discussion on creating namespaces seem to match this type of usage more than what you've suggested here.

On 20 Oct 2019, at 07:57, Steve Jorgensen wrote:

Several requests for a specific use case. This is something I have been working on just to practice getting a deeper understanding of Python metaprogramming, so whether this is something that someone should be trying to do is, of course, a matter of opinion.

The idea is to use a class as a singleton collection of choices where every choice appears both as an attribute of the class/singleton and as a value/label pair when treating it as a sequence.

Usage example:

class Colors(Choices): with choices(): RED GREEN BLUE

Colors.RED -> 'Red' Colors.GREEN -> 'Green' Colors.BLUE -> 'Blue'

tuple(Colors) -> (('RED', 'Red'), ('GREEN', 'Green'), ('BLUE', 'Blue'))

This can be implemented using a mapping object (classdict) returned from the metaclass' `__prepare__` class method. The classdict is checked prior to checking the surrounding context and takes priority unless its `__getitem__` method raises `KeyError`.

The problem in this case is that the classdict generates results dynamically, and it has no way of knowing whether a variable reference would have been resolved in the surrounding context or not. Therefore, it will ALWAYS take precedence, even masking builtins like `print`, `range`, etc. This is surprising behavior and generally undesirable.

As I mentioned already, it is possible to work around the problem for builtins by having the classdict object specifically test for those, but that doesn't solve the problem of masking other variables from the surrounding context. Masking those is also surprising behavior.

Steve Jorgensen wrote:

...

When using a custom classdict to implement a DSL or use in the body of a class definition, from what I can tell by experiment, the classdict takes priority, and the surrounding context is only referenced if trying to get an item from the classdict raises KeyError. There is at least one case in which I would like to do the reverse, and have the classdict be secondary to (masked by) any variables defined in the context surrounding the execution of the class body. I have been able to at least unmask builtins by having the classdict object first try to get a result from __builtins__ and then fall back to itself. Also, in the actual class body, declaring a variable as global seems to be a workaround for global variables, but that has to be done explicitly in the body of any subclass that needs it. Also, that does not handle non-globals in its surrounding context. I'm not sure what the best way to deal with this would be, but my first thought is to maybe have a property that can be set on the metaclass type such as `metacls.reluctant_classdict = True` [typo corrected from original post].

---

Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HATKS2... Code of Conduct: http://python.org/psf/codeofconduct/

On 20 Oct 2019, at 12:41, Steve Jorgensen wrote:

Anders Hovmöller wrote:

...

We try to do the same thing in various libraries. We've settled on using existing python and end up with syntax like: class MyForm(Form): field = Field() or in your case class Colors(TokenContainer): red = Token() green = Token() blue = Token() (this is using tri.token).

This seems like a good approach and is similar to the strategy Enum uses.

Site note: When we saw enums landing we hoped we could ditch tri.token but unfortunately enums have a bunch of limitations and restrictions that make them quite unusable for our more general use case :(

Oops. Obviously, the above is pseudo-code because I left `('~OTHER~', 'Something else')` the first assert, and I have `(Food, Food)` instead of `(Food,) * 4` for the second assert. Possibly some other mistakes as well.

On Oct 19, 2019, at 22:57, Steve Jorgensen wrote:

The idea is to use a class as a singleton collection of choices where every choice appears both as an attribute of the class/singleton and as a value/label pair when treating it as a sequence.

What you’re describing is just enum.Enum, minus half its features, but a sequence instead of a mapping. I’m not sure why you’d ever want this, but if you did, you could just subclass EnumMeta and add/override the sequence methods. At any rate, almost all of that seems irrelevant to your issue. Making it act like both a singleton with members and like a sequence is already trivial to do today. What you’re looking for is a way to auto-assign attributes on lookup during class definition. Which is an idea that was actually brought up for enums, and specifically rejected in PEP 435, but there’s a proof of concept implementation of it anyway if you want it. It’s also worth looking at the implementation of the various alternative enum implementations on PyPI (both the ones that pre-existed 3.4, and the ones that build on it).

On Oct 20, 2019, at 03:36, Steve Jorgensen wrote:

Andrew Barnert wrote:

...

What you’re describing is just enum.Enum, minus half its features, but a sequence instead of a mapping. I’m not sure why you’d ever want this,

Actually, it's a bit different. This is for building a collection of value/label pairs where the values are as likely to be strings as numbers, and its sequence behavior is as a list of value/label pairs (e.g. for use as the choices for a Django field).

But the values of an Enum can be strings. And there are multiple PyPI projects that build Django choices on top of Enum, which show that the only thing you need is to override __getitem__ to replace its mapping behavior with sequence behavior. I believe this was also one of the examples of what you can build for yourself on top of EnumMeta in Nick Coghlan’s blog post explaining the design of enum. Maybe the additional functionality in Enum gets in your way for some reason I don’t understand, that doesn’t affect other people who want Django choices. But otherwise, I don’t see how it really is any different. And again, none of this different-from-Enum behavior is relevant to your problem anyway. The fact that you think it is, and think it can’t be done with Enum, makes it harder to understand where the actual problem is. Maybe you’re not just trying to do the same magic that all the autoenum/simplenum/etc. packages on PyPI and the PoC code from the PEP and the section in Nick’s blog post all do (in a couple different ways), or maybe it’s insufficiently magic for your purposes, but without knowing how and why it’s different or insufficient it’s impossible to know whether there’s an actual limitation to fix in Python, or just a problem in your design or implementation that could be easily fixed to work in existing Python.

Andrew Barnert wrote:

On Oct 20, 2019, at 03:36, Steve Jorgensen stevej@stevej.name wrote:

And there are multiple PyPI projects that build Django choices on top of Enum, which show that the only thing you need is to override __getitem__ to replace its mapping behavior with sequence behavior. I believe this was also one of the examples of what you can build for yourself on top of EnumMeta in Nick Coghlan’s blog post explaining the design of enum. Maybe the additional functionality in Enum gets in your way for some reason I don’t understand, that doesn’t affect other people who want Django choices. But otherwise, I don’t see how it really is any different. And again, none of this different-from-Enum behavior is relevant to your problem anyway. The fact that you think it is, and think it can’t be done with Enum, makes it harder to understand where the actual problem is. Maybe you’re not just trying to do the same magic that all the autoenum/simplenum/etc. packages on PyPI and the PoC code from the PEP and the section in Nick’s blog post all do (in a couple different ways), or maybe it’s insufficiently magic for your purposes, but without knowing how and why it’s different or insufficient it’s impossible to know whether there’s an actual limitation to fix in Python, or just a problem in your design or implementation that could be easily fixed to work in existing Python.

So… You're exactly right. I just had a very limited understanding of what enum was about and had not dug into it deeply enough to find out how flexible it really is. Here's an implementation that I came up with in a short hacking session using enum. Tests: from choices import ChoiceEnum, Label as L def test_creates_auto_valued_auto_labeled_choices(): class Food(ChoiceEnum): APPLE = () ICED_TEA = () assert tuple(tuple(fc) for fc in Food) == ( ('APPLE', 'Apple'), ('ICED_TEA', 'Iced Tea'), ) def test_creates_auto_labeled_choices_with_given_values(): class Food(ChoiceEnum): APPLE = 1 ICED_TEA = 2 assert tuple(tuple(fc) for fc in Food) == ( (1, 'Apple'), (2, 'Iced Tea'), ) def test_creates_choices_with_given_values_and_labels(): class Food(ChoiceEnum): CHICKEN_MCNUGGETS = (1, 'Chicken McNuggets') CHICKEN_PROVENCAL = (2, 'Chicken Provençal') assert tuple(tuple(fc) for fc in Food) == ( (1, 'Chicken McNuggets'), (2, 'Chicken Provençal'), ) def test_creates_auto_valued_choices_with_given_values(): class Food(ChoiceEnum): CHX_MN = L('Chicken McNuggets') CHX_PV = L('Chicken Provençal') assert tuple(tuple(fc) for fc in Food) == ( ('CHX_MN', 'Chicken McNuggets'), ('CHX_PV', 'Chicken Provençal'), ) Implementation: from enum import Enum class ChoiceEnum(Enum): def __init__(self, src=None, label=None): super().__init__() if isinstance(src, Label): value = None label = str(src) else: value = src self._value_ = self.name if value is None else value self.label = label or _label_from_name(self.name) def __getitem__(self, idx): if idx == 0: return self.value elif idx == 1: return self.label else: raise IndexError('Index value must be 0 or 1') def __len__(self): return 2 class Label(str): pass def _label_from_name(name): return name.replace('_', ' ').title()

Ethan Furman wrote:

...

Experimenting is good! However, you'll want to either build your own metaclass and/or prepared dict, or do some work on your __new__/__init__ methods for building enum members. Currently, you are reassigning _value_ in __init__, which leaves some internal structures not matching the Enum. --> class Food(ChoiceEnum): ... APPLE = () ... ICED_TEA = () ... --> Food['APPLE'] --> Food.APPLE --> Food('APPLE') Traceback (most recent call last): ... ValueError: 'APPLE' is not a valid Food

Thanks for that info. Per the example in https://docs.python.org/3/library/enum.html#when-to-use-new-vs-init, it looks like I can properly set the `_value_` property in the `__new__` method of the `Enum` subclass without needing to subclass `EnumMeta`. Am I understanding that correctly?

On 10/19/2019 11:25 PM, Andrew Barnert via Python-ideas wrote:

On Oct 19, 2019, at 22:57, Steve Jorgensen wrote:

...

The idea is to use a class as a singleton collection of choices where every choice appears both as an attribute of the class/singleton and as a value/label pair when treating it as a sequence.

What you’re looking for is a way to auto-assign attributes on lookup during class definition. Which is an idea that was actually brought up for enums, and specifically rejected in PEP 435, but there’s a proof of concept implementation of it anyway if you want it.

It is actually implemented in the `aenum`* library. The code in question is in _EnumDict, and my workaround for the problem was another variable (IIRC, _ignore_) that listed any names that were outside the Enum's scope. -- ~Ethan~ * disclosure: I'm the one to hold responsible for the stdlib enum, enum34, and aenum.

Steven D'Aprano wrote:

I've read the entire thread, and I don't understand what builtins has to do with your problem.

Just that, as a workaround, I can have the classdict object raise KeyError for anything that's a member of builtins so that the builtins are used instead of being masked.

But regardless, you shouldn't touch __builtins__, it is a private implementation detail of CPython. The right way is: import builtins # Python 3 import __builtin__ as builtins # Python 2

and then use builtins.

Thanks for setting me straight on this. Much appreciated.

On 20/11/2019 08:25, Stephen J. Turnbull wrote:

Rob Cliffe via Python-ideas writes:

...

I (and no doubt others) am less likely to engage with a thread if I have to spend more time than necessary trying to understand it.

You write as if that's a bad thing.;-) I consider that I often make the largest contribution to this list by reading more posts and stifling the impulse to reply to any of them.:-) YMMV, of course.

You used YMMV with malice aforethought, didn't you? :-) -- Rhodri James *-* Kynesim Ltd