[Python-Dev] PEP 487: Simpler customization of class creation

[Nikita Nemkin]

On Mon, Jun 20, 2016 at 9:48 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jun 16, 2016 at 3:24 PM, Nikita Nemkin <nikita at nemkin.ru> wrote:
>>
>> I didin't know that PyPy has actually implemented packed ordered dicts!
>>
>> https://morepypy.blogspot.ru/2015/01/faster-more-memory-efficient-and-more.html
>> https://mail.python.org/pipermail/python-dev/2012-December/123028.html
>>
>> This old idea by Raymond Hettinger is vastly superior to
>> __definition_order__ duct tape (now that PyPy has validated it).
>> It also gives kwarg order for free, which is important in many
>> metaprogramming scenarios.
>> Not to mention memory usage reduction and dict operations speedup...
>
>
> That idea is only vastly superior if we want to force all other Python
> implementations to also have an order-preserving dict with the same
> semantics and API.

Right. Ordered by default is a very serious implementation constraint.
It's only superior in a sense that it completely subsumes/obsoletes
PEP 520.

> I'd like to hear more about your metaprogramming scenarios -- often such
> things end up being code the author is ashamed of. Perhaps they should stay
> in the shadows? Or could we do something to make it so you won't have to be
> ashamed of it?

What I meant is embedding declarative domain-specific languages
in Python. Examples of such languages include SQL table
definitions, binary data definitions (in-memory C structs or
wire protocol), GUI definitions (look up enaml for an interesting
example), etc. etc. DSLs are a well defined field and the point
of embedding into Python is to implement in Python and to
empower DSL with Python constructs for generation and logic.

Basic blocks for a declarative language are lists and "objects" -
groups of ordered, named fields.

Representing lists is easy and elegant, commas make a tuple
and [] makes a list.

It's when trying to represent "objects" the issues arise.
Literal dicts are "ugly" (for DSL purposes) and unordered.
Lists of 2-tuples are even uglier. Py3 gave us __prepare__ for
ordered class bodies, and this became a first valid option.
For example, SQL table:

    class MyTable(SqlTable):
        field1 = Type1(options...)
        field2 = Type2()

Unfortunately, class declarations don't look good when nested,
and nesting is a common thing.

    class MainWindow:
        caption = "Window"
        class HSplit:
            label1 = Label(...)
            text1 = Text(...)

You get the idea.
Another option for expressing "objects" are function calls with kwargs:

    packet = Struct(type=uint8,
                    length=uint32,
                    body=Array(uint8, 'type'))

Looks reasonably clean, but more often than not requires kwargs
to be ordered. THIS is the scenario I was talking about.

Function attributes also have a role, but being
attached to function definitions, their scope is somewhat limited.

Of course, all of the above is largely theoretical, for two basic
reasons:
1) Python syntax/runtime is too rigid for a declarative DSL.
   (Specifically, _embedded_ DSL. The syntax alone can be re-used
   with ast.parse, but it's a different scenario.)
2) DSLs in general are grossly unpythonic, hiding loads of magic
   and unfamiliar semantics behind what looks like a normal Python.
   It's not something to be ashamed of, but the benefit
   rarely justifies the (maintenance) cost.

To be clear: I'm NOT advocating for ordered kwargs. Embedding
DSLs into Python is generally a bad idea.

PS. __prepare__ enables many DSL tricks. In fact, it's difficult to
imagine a use case that's not related to some attempt at DSL.
Keyword-only args also help: ordered part of the definition can go
into *args, while attributes/options are kw-only args.