[Python-Dev] The Return Of Argument Clinic
larry at hastings.org
Tue Aug 6 01:53:39 CEST 2013
On 08/05/2013 02:55 AM, Nick Coghlan wrote:
> On 5 August 2013 18:48, Larry Hastings<larry at hastings.org> wrote:
>> Question 0: How should we integrate Clinic into the build process?
> Isn't solving the bootstrapping problem the reason for checking in the
> clinic-generated output? If there's no Python available, we build what
> we have (without the clinic step), then we build it again *with* the
> clinic step.
It solves the bootstrapping problem, but that's not the only problem
Clinic presents to the development workflow.
If you modify some Clinic DSL in a C file in the CPython tree, then run
"make", should the Makefile re-run Clinic over that file? If you say
"no", then there's no problem. If you say "yes", then we have the
problem I described.
>> Question 1: Which C function nomenclature?
> Consider this from the client side, and I believe it answers itself:
> other code in the module will be expected the existing signature, so
> that signature needs to stay with the existing name, while the new C
> implementation function gets the new name.
One vote for "os_stat_impl". Bringing the sum total of votes up to 1! ;-)
>> Question 2: Emit code for modules and classes?
>> There are some complications to this, one of which I'll
>> discuss next. But I put it to you, gentle reader: how
>> much boilerplate should Argument Clinic undertake to
>> generate, and how much more class and module metadata
>> should be wired in to it?
> I strongly recommend deferring this. Incremental development is good,
> and getting this bootstrapped at all is going to be challenging enough
> without trying to do everything at once.
I basically agree. But you glossed over an important part of that
question, "how much more class and module metadata should be wired in
Originally Clinic didn't ask for full class and module information, you
just specified the full dotted path and that was that. But that's
ambiguous; Clinic wouldn't be able to infer what was a module vs what
was a class. And in the future, if/when it generates module and class
boilerplate, obviously it'll need to know the distinction. I figure,
specifying the classes and modules doesn't add a lot of additional cost,
but it'll very likely save us a lot of time in the long run, so I made
it a requirement. (WAGNI!)
Anyway, I guess what I was really kind of trying to get at here was:
a) are there any other obvious bits of metadata Clinic should require
right now for functions,
b) what other metadata might Clinic take in the future--not because I
want to add it, but just so we can figure out the next question,
c) to what degree can we future-proof Clinic 1.0 so extension authors
can more easily straddle versions.
Thinking about it more with a fresh perspective, maybe all we need is a
Clinic version number directive. This would declare the minimum Clinic
version--which would really just track the Python version it shipped
with--that you may use to process this file. Like so:
As long as the code Clinic generates is backwards compatible for Python
3.4, I think this will has it covered. We may at times force developers
to use fresher versions of Python to process Clinic stuff, but I don't
think that's a big deal.
>> Question 4: Return converters returning success/failure?
>> Can we live with PyErr_Occurred() here?
> Armin's suggestion of a valid return value (say, -1) that indicates
> "error may have occurred" sounds good to me.
Yes indeed, and thanks Armin for pointing it out. This works perfectly
in Clinic, as each individual return converter controls the code
generated for cleanup afterwards. So it can be a completely local
policy per-return-converter what the magic value is. Heck, you could
have multiple int converters with different magic return values (not
that that seems like a good idea).
>> Question 5: Keep too-magical class decorator Converter.wrap?
>> I'd like to keep it in, and anoint it as the preferred way
>> of declaring Converter subclasses. Anybody else have a strong
>> opinion on this either way?
> Can't you get the same effect without the magic by having a separate
> "custom_init" method that the main __init__ method promises to call
> with the extra keyword args after finishing the other parts of the
> initialization? Them a custom converter would just look like:
> class path_t_converter(Converter):
> def custom_init(self, *, allow_fd=False, nullable=False):
I can get the same effect without reusing the name __init__, but I
wouldn't say I can do it "without the magic". The whole point of the
decorator is magic.
Let's say I go with your proposal. What happens if someone makes a
Converter, and wraps it with Converter.wrap, and defines their own
__init__? It would never get called. Silently, by default, which is
worse--though I could explicitly detect such an __init__ and throw an
exception I guess. Still, now we have a class where you can't use the
name __init__, you have to use this funny other name, for arbitrary
My metaphor for why I prefer my approach is the set of "os" module
functions that allow "specifying a file descriptor":
Taking the example of os.chdir(), yes, it would have been more correct
to require specifying the file descriptor as a separate argument, like
But this would have meant that when using "fd" the first parameter would
always be None. And the first parameter and the "fd" do the same thing,
just with different types. So while normally we eschew polymorphic
parameters (and with good reason) in this case I think practicality beat
And I think the same holds here. Since class instance initialization
functions in Python are called __init__, and this is a class instance
initialization function, I think it should be called __init__. By
decorating with Converter.wrap, you're signing a contract that says "yes
it's a fake __init__, that's what I want". A real __init__ in this
class would never be called, which is the whole point, so we might as
well reuse the name for our slightly-fake __init__.
Let me put it this way: Which is more surprising to the person
unfamiliar with the code? That this __init__ doesn't get all the
parameters, and the base class __init__ is getting called
automatically? Or that this funny function "custom_init" is what gets
called, and this class is not allowed to have a function called __init__?
In case I didn't make it clear: the actual call site for constructing
these objects is buried deep in clinic.py. Users don't create them
directly, or at least I don't know why they'd ever need to do so.
Instead, they're created inside Clinic preprocessor blocks in your
source files, where they look like tis:
parameter_name: Converter(argument=value, argument2=value) = default
Using the funny magic of Converter.wrap makes this and the
implementation look a great deal more alike. So I remain a fan of
Converter.wrap and calling the initialization function __init__.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev