Announcing PEP 436: The Argument Clinic DSL
Following up on a conversation on python-dev from last December: http://mail.python.org/pipermail/python-dev/2012-December/122920.html I'm pleased to announced PEP 436, proposing Argument Clinic for adoption into the CPython source tree. http://www.python.org/dev/peps/pep-0436/ Argument Clinic itself hasn't changed much in the intervening three months; I did add a prototype extension mechanism for adding user types at runtime, and I allow writing the generated code to a separate file. The PEP is adopted out of the "clinic.txt" included with the prototype, updated and with a new rationale. For what it's worth, the bug tracker issue is here: http://bugs.python.org/issue16612 I'm guessing python-dev is the right place for the ten-thousand-foot view topics: the merits of the specific proposed DSL syntax, the possible runtime extension API, and the approach as a whole. So for now let's just use the bug tracker issue for code reviews and the like. The prototype implementation is checked in here: https://bitbucket.org/larry/python-clinic/ as well as being posted to the above issue on the tracker in patch form. I look forward to your comments, //arry/
Larry Hastings
http://bugs.python.org/issue16612
I'm guessing python-dev is the right place for the ten-thousand-foot view topics: the merits of the specific proposed DSL syntax, the possible runtime extension API, and the approach as a whole. So for now let's just use the bug tracker issue for code reviews and the like.
For anyone who isn't following the issue: A PEP proposing a different DSL will be forthcoming either this or next weekend. Stefan Krah
On Tue, Feb 26, 2013 at 12:30 AM, Stefan Krah
Larry Hastings
wrote: http://bugs.python.org/issue16612
I'm guessing python-dev is the right place for the ten-thousand-foot view topics: the merits of the specific proposed DSL syntax, the possible
extension API, and the approach as a whole. So for now let's just use
runtime the bug
tracker issue for code reviews and the like.
For anyone who isn't following the issue: A PEP proposing a different DSL will be forthcoming either this or next weekend.
If the two proposals share at least the motivation, would it not be more constructive to just have them listed as alternatives in a single PEP? This could serve as a useful single source of information for the new proposed features. Eli
Eli Bendersky
For anyone who isn't following the issue: A PEP proposing a different DSL will be forthcoming either this or next weekend.
If the two proposals share at least the motivation, would it not be more constructive to just have them listed as alternatives in a single PEP? This could serve as a useful single source of information for the new proposed features.
I find PEPs that propose multiple alternatives hard to read. My PEP won't be short: It will contain a token specifications, a yacc grammar and an attempt to specify the semantics of the DSL. Stefan Krah
On Wed, Feb 27, 2013 at 12:40 AM, Stefan Krah
Eli Bendersky
wrote: For anyone who isn't following the issue: A PEP proposing a different DSL will be forthcoming either this or next weekend.
If the two proposals share at least the motivation, would it not be more constructive to just have them listed as alternatives in a single PEP? This could serve as a useful single source of information for the new proposed features.
I find PEPs that propose multiple alternatives hard to read. My PEP won't be short: It will contain a token specifications, a yacc grammar and an attempt to specify the semantics of the DSL.
More importantly, the competing PEPs have different champions, so you and Larry need to be free to make the best case you can. Cheers, Nick. P.S. I was thinking the last time we had truly competing PEPs was way back when the with statement was designed, but it turns out the competing namespace package designs were also separate PEPs. PEP 315 was more an after the fact here's-a-better-way-to-implement-this redesign of PEP 309, and PEP 3150 vs 403 is just me having an ongoing discussion with myself (and occasionally others) over a feature I doubt would ever make it past Guido even if I do eventually decide to propose one of them. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Feb 25, 2013 at 4:11 PM, Larry Hastings
Following up on a conversation on python-dev from last December:
http://mail.python.org/pipermail/python-dev/2012-December/122920.html
I'm pleased to announced PEP 436, proposing Argument Clinic for adoption into the CPython source tree.
You really need to provide some realistic usage examples in the PEP. It's hard to get a feel for the DSL just from looking at the spec. Eli
On 2/25/2013 7:11 PM, Larry Hastings wrote:
Following up on a conversation on python-dev from last December:
http://mail.python.org/pipermail/python-dev/2012-December/122920.html
I'm pleased to announced PEP 436, proposing Argument Clinic for adoption into the CPython source tree.
The PEP gives an internal, developer-focused rationale. I think there is also an external, user-focused rationale. As much as possible (with obvious caveats about type introspection), I think it should be transparent to users (other than speed) whether a function is implemented in Python or C. People who love keyword calling currently have to learn which module by module and function by function. Tranparency will become even more important as more modules are dual coded, and as users move code between implementations. I presume some of the complexities of PyArg_ParseTupleAndKeywords arise from pursuing this goal. And if I understand correctly, this PEP should reduce some of the complexities. The 'future goal' of C function signature info will further aid transparency. This user rationale suggests that positional-only functions should be discouraged. It also suggests another future goal: a clinic backend that would instead output a skeleton python file with def header lines and docstrings, leaving the bodies to be filled in. One could also wish for the opposite: a python file processor that would convert def header lines and docstrings to a skeleton C file with clinic specifications.
Do we need to support tuple unpacking?
Well, Python 3 dropped that for python functions, so it is not needed needed for conversions to or from python 3 files. I suspect the same rationale applies to C as well as Python functions: the unpacking can be done explicitly in the body, as needed. -- Terry Jan Reedy
On 02/26/2013 08:11 AM, Terry Reedy wrote:
The PEP gives an internal, developer-focused rationale. I think there is also an external, user-focused rationale. As much as possible (with obvious caveats about type introspection), I think it should be transparent to users (other than speed) whether a function is implemented in Python or C. People who love keyword calling currently have to learn which module by module and function by function. Tranparency will become even more important as more modules are dual coded, and as users move code between implementations. I presume some of the complexities of PyArg_ParseTupleAndKeywords arise from pursuing this goal. And if I understand correctly, this PEP should reduce some of the complexities. The 'future goal' of C function signature info will further aid transparency.
This user rationale suggests that positional-only functions should be discouraged.
I think positional-only functions should be discouraged, but we don't really need Clinic to do that. I suppose we don't need Clinic to support signatures for built-in functions either, though Clinic would be a much bigger help in that department. So, here's my interpretation of the above suggestion: one user-focused rationale for Clinic is that it'll make it easier for developers to write built-ins that behave identically to functions written in Python, and this benefits users. Do you agree with that?
It also suggests another future goal: a clinic backend that would instead output a skeleton python file with def header lines and docstrings, leaving the bodies to be filled in.
Well, right now Clinic has no visibility into variables, just functions. So such a skeleton python file would be a bit incomplete. That's mildly interesting to consider, though--telling Clinic about module variables, that is. Another possible use of Clinic is to do conformance testing for alternative implementations, that is, having PyPy (for example) make sure that they got their reimplemented standard library API correct. //arry/
On 2/26/2013 1:47 PM, Larry Hastings wrote:
On 02/26/2013 08:11 AM, Terry Reedy wrote:
The PEP gives an internal, developer-focused rationale. I think there is also an external, user-focused rationale. As much as possible (with obvious caveats about type introspection), I think it should be transparent to users (other than speed) whether a function is implemented in Python or C. People who love keyword calling currently have to learn which module by module and function by function. Tranparency will become even more important as more modules are dual coded, and as users move code between implementations. I presume some of the complexities of PyArg_ParseTupleAndKeywords arise from pursuing this goal. And if I understand correctly, this PEP should reduce some of the complexities. The 'future goal' of C function signature info will further aid transparency.
This user rationale suggests that positional-only functions should be discouraged.
I think positional-only functions should be discouraged, but we don't
If I were writing something like Clinic, I would be tempted to not have that option. But I was actually thinking about something in the positional-only writeup that mentioned the possibility of adding something to the positional-only option.
really need Clinic to do that. I suppose we don't need Clinic to support signatures for built-in functions either, though Clinic would be a much bigger help in that department.
So, here's my interpretation of the above suggestion: one user-focused rationale for Clinic is that it'll make it easier for developers to write built-ins that behave identically to functions written in Python, and this benefits users. Do you agree with that?
yes, especially with signatures.
It also suggests another future goal: a clinic backend that would instead output a skeleton python file with def header lines and docstrings, leaving the bodies to be filled in.
Well, right now Clinic has no visibility into variables, just functions. So such a skeleton python file would be a bit incomplete.
Module variables are option few enough to copy by hand. Methods are more of a problem. As I understand it, C module files are structured something like the following, which is 'unusual' for a python file. def meth1_impl(... def meth2_impl(... class C: meth1 = meth1_impl meth2 = meth2_impl So producing an idiomatic skeleton would require more that an alternate backend. Rather function entries should be stored and not output until the structure (class) definition is encountered. Two passes would be a lot easier than one.
That's mildly interesting to consider, though--telling Clinic about module variables, that is. Another possible use of Clinic is to do conformance testing for alternative implementations, that is, having PyPy (for example) make sure that they got their reimplemented standard library API correct.
//arry/
-- Terry Jan Reedy
On 02/26/2013 06:30 PM, Terry Reedy wrote:
On 2/26/2013 1:47 PM, Larry Hastings wrote:
I think positional-only functions should be discouraged, but we don't
If I were writing something like Clinic, I would be tempted to not have that option. But I was actually thinking about something in the positional-only writeup that mentioned the possibility of adding something to the positional-only option.
Clinic needs to be usable for every builtin in order to be a credible solution. And the positional-only approach to parsing is sufficiently different from the positional-and-keywords approach that I couldn't sweep the conceptual difference under the rug--I had to explicitly support weird stuff like optional positional arguments on the /left/. So I really don't have a choice. All we can do is say "please don't use this for new code".
As I understand it, C module files are structured something like the following, which is 'unusual' for a python file.
def meth1_impl(...
def meth2_impl(...
class C: meth1 = meth1_impl meth2 = meth2_impl
At the moment Clinic is agnostic about where in Python the callable lives. The "module name" in the DSL is really just used in documentation and to construct the name of the static functions. So if you specify a class name as your "module name" it'll work fine and look correct. However, maybe we need to think about this a little more when it comes time to add Signature metadata. //arry/
participants (5)
-
Eli Bendersky
-
Larry Hastings
-
Nick Coghlan
-
Stefan Krah
-
Terry Reedy