Add ability to disable the autogeneration of the function signature in a ufunc docstring.
Hi all, In a recent scipy pull request (https://github.com/scipy/scipy/pull/459), I ran into the problem of ufuncs automatically generating a signature in the docstring using arguments such as 'x' or 'x1, x2'. scipy.special has a lot of ufuncs, and for most of them, there are much more descriptive or conventional argument names than 'x'. For now, we will include a nicer signature in the added docstring, and grudgingly put up with the one generated by the ufunc. In the long term, it would be nice to be able to disable the automatic generation of the signature. I submitted a pull request to numpy to allow that: https://github.com/numpy/numpy/pull/3149 Comments on the pull request would be appreciated. Thanks, Warren
On Fri, Mar 15, 2013 at 6:47 PM, Warren Weckesser <warren.weckesser@gmail.com> wrote:
Hi all,
In a recent scipy pull request (https://github.com/scipy/scipy/pull/459), I ran into the problem of ufuncs automatically generating a signature in the docstring using arguments such as 'x' or 'x1, x2'. scipy.special has a lot of ufuncs, and for most of them, there are much more descriptive or conventional argument names than 'x'. For now, we will include a nicer signature in the added docstring, and grudgingly put up with the one generated by the ufunc. In the long term, it would be nice to be able to disable the automatic generation of the signature. I submitted a pull request to numpy to allow that: https://github.com/numpy/numpy/pull/3149
Comments on the pull request would be appreciated.
The functionality seems obviously useful, but adding a magic public attribute to all ufuncs seems like a somewhat clumsy way to expose it? Esp. since ufuncs are always created through the C API, including docstring specification, but this can only be set at the Python level? Maybe it's the best option but it seems worth taking a few minutes to consider alternatives. Brainstorming: - If the first line of the docstring starts with "<funcname>(" and ends with ")", then that's a signature and we skip adding one (I think sphinx does something like this?) Kinda magic and implicit, but highly backwards compatible. - Declare that henceforth, the signature generation will be disabled by default, and go through and add a special marker like "__SIGNATURE__" to all the existing ufunc docstrings, which gets replaced (if present) by the automagically generated signature. - Give ufunc arguments actual names in general, that work for things like kwargs, and then use those in the automagically generated signature. This is the most work, but it would mean that people don't have to remember to update their non-magic signatures whenever numpy adds a new feature like out= or where=, and would make the docstrings actually accurate, which right now they aren't: In [7]: np.add.__doc__.split("\n")[0] Out[7]: 'add(x1, x2[, out])' In [8]: np.add(x1=1, x2=2) ValueError: invalid number of arguments - Allow some special syntax to describe the argument names in the docstring: "__ARGNAMES__: a b\n" -> "add(a, b[, out])" - Something else... -n
15.03.2013 22:39, Nathaniel Smith kirjoitti: [clip]
- Something else...
How about: scrap the automatic signatures altogether, and directly use the docstring provided to the ufunc creation function? I suspect ufuncs are not very widely used in 3rd party code, as it requires somewhat tricky messing with the C API. The backwards compatibility issue is also just a documentation issue, so nothing drastic. -- Pauli Virtanen
On Fri, Mar 15, 2013 at 9:19 PM, Pauli Virtanen <pav@iki.fi> wrote:
15.03.2013 22:39, Nathaniel Smith kirjoitti: [clip]
- Something else...
How about: scrap the automatic signatures altogether, and directly use the docstring provided to the ufunc creation function?
I suspect ufuncs are not very widely used in 3rd party code, as it requires somewhat tricky messing with the C API. The backwards compatibility issue is also just a documentation issue, so nothing drastic.
True enough. I guess a question is how much it bothers us that there are tons of ufunc arguments that are just not mentioned in the interpreter docstrings: http://docs.scipy.org/doc/numpy/reference/ufuncs.html#optional-keyword-argum... Obviously not a huge amount of we'd have altered the auto-generation already to include them :-) But IMHO it would be kind of nice if ?np.add mentioned the existence of things like where= and dtype=... and if we decide that docstrings ought to mention such things, then it's going to be a right hassle updating them all by hand every time some new ufunc feature is added. -n
On Fri, Mar 15, 2013 at 4:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
In a recent scipy pull request (https://github.com/scipy/scipy/pull/459), I ran into the problem of ufuncs automatically generating a signature in
On Fri, Mar 15, 2013 at 6:47 PM, Warren Weckesser <warren.weckesser@gmail.com> wrote: the
docstring using arguments such as 'x' or 'x1, x2'. scipy.special has a lot of ufuncs, and for most of them, there are much more descriptive or conventional argument names than 'x'. For now, we will include a nicer signature in the added docstring, and grudgingly put up with the one generated by the ufunc. In the long term, it would be nice to be able to disable the automatic generation of the signature. I submitted a pull request to numpy to allow that: https://github.com/numpy/numpy/pull/3149
Comments on the pull request would be appreciated.
The functionality seems obviously useful, but adding a magic public attribute to all ufuncs seems like a somewhat clumsy way to expose it? Esp. since ufuncs are always created through the C API, including docstring specification, but this can only be set at the Python level? Maybe it's the best option but it seems worth taking a few minutes to consider alternatives.
Agreed; exposing the flag as part of the public Python ufunc API is unnecessary, since this is something that would rarely, if ever, be changed during the life of the ufunc.
Brainstorming:
- If the first line of the docstring starts with "<funcname>(" and ends with ")", then that's a signature and we skip adding one (I think sphinx does something like this?) Kinda magic and implicit, but highly backwards compatible.
- Declare that henceforth, the signature generation will be disabled by default, and go through and add a special marker like "__SIGNATURE__" to all the existing ufunc docstrings, which gets replaced (if present) by the automagically generated signature.
- Give ufunc arguments actual names in general, that work for things like kwargs, and then use those in the automagically generated signature. This is the most work, but it would mean that people don't have to remember to update their non-magic signatures whenever numpy adds a new feature like out= or where=, and would make the docstrings actually accurate, which right now they aren't:
I'm leaning towards this option. I don't know if there would still be a need to disable the automatic generation of the docstring if it was good enough. In [7]: np.add.__doc__.split("\n")[0]
Out[7]: 'add(x1, x2[, out])'
In [8]: np.add(x1=1, x2=2) ValueError: invalid number of arguments
- Allow some special syntax to describe the argument names in the docstring: "__ARGNAMES__: a b\n" -> "add(a, b[, out])"
- Something else...
-n _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 20 Mar 2013 17:11, "Warren Weckesser" <warren.weckesser@gmail.com> wrote:
On Fri, Mar 15, 2013 at 4:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Fri, Mar 15, 2013 at 6:47 PM, Warren Weckesser <warren.weckesser@gmail.com> wrote:
Hi all,
In a recent scipy pull request (https://github.com/scipy/scipy/pull/459),
ran into the problem of ufuncs automatically generating a signature in
I the
docstring using arguments such as 'x' or 'x1, x2'. scipy.special has a lot of ufuncs, and for most of them, there are much more descriptive or conventional argument names than 'x'. For now, we will include a nicer signature in the added docstring, and grudgingly put up with the one generated by the ufunc. In the long term, it would be nice to be able to disable the automatic generation of the signature. I submitted a pull request to numpy to allow that: https://github.com/numpy/numpy/pull/3149
Comments on the pull request would be appreciated.
The functionality seems obviously useful, but adding a magic public attribute to all ufuncs seems like a somewhat clumsy way to expose it? Esp. since ufuncs are always created through the C API, including docstring specification, but this can only be set at the Python level? Maybe it's the best option but it seems worth taking a few minutes to consider alternatives.
Agreed; exposing the flag as part of the public Python ufunc API is unnecessary, since this is something that would rarely, if ever, be changed during the life of the ufunc.
Brainstorming:
- If the first line of the docstring starts with "<funcname>(" and ends with ")", then that's a signature and we skip adding one (I think sphinx does something like this?) Kinda magic and implicit, but highly backwards compatible.
- Declare that henceforth, the signature generation will be disabled by default, and go through and add a special marker like "__SIGNATURE__" to all the existing ufunc docstrings, which gets replaced (if present) by the automagically generated signature.
- Give ufunc arguments actual names in general, that work for things like kwargs, and then use those in the automagically generated signature. This is the most work, but it would mean that people don't have to remember to update their non-magic signatures whenever numpy adds a new feature like out= or where=, and would make the docstrings actually accurate, which right now they aren't:
I'm leaning towards this option. I don't know if there would still be a need to disable the automatic generation of the docstring if it was good enough.
Certainly it would be nice for ufunc argument handling to better match python argument handling! Just needs someone willing to do the work... *cough* ;-) -n
In [7]: np.add.__doc__.split("\n")[0] Out[7]: 'add(x1, x2[, out])'
In [8]: np.add(x1=1, x2=2) ValueError: invalid number of arguments
- Allow some special syntax to describe the argument names in the docstring: "__ARGNAMES__: a b\n" -> "add(a, b[, out])"
- Something else...
-n _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (3)
-
Nathaniel Smith -
Pauli Virtanen -
Warren Weckesser