[Python-Dev] The docstring hack for signature information has to go

Larry Hastings larry at hastings.org
Tue Feb 4 10:12:58 CET 2014

On 02/03/2014 08:19 PM, Guido van Rossum wrote:
> But why do you even need a flag? Reading issue 20075 where the 
> complaint started, it really feels that the change was an overreaction 
> to a very minimal problem.

I'll cop to that.  I'm pretty anxious about trying to "get it right".  
My worry was (and is) that this hiding-the-signature-in-the-docstring 
approach is a cheap hack, and it will have unexpected and undesirable 
side-effects that will in retrospect seem obvious.  This is FUD I 
admit.  But it seems to me if we did it "the right way", with a 
"PyMethodDefEx", we'd be able to do a lot better job predicting the 

> A few docstrings appear truncated. Big deal. We can rewrite the ones 
> that are reported as broken (either by adjusting the docstring to not 
> match the patter or by adjusting it to match the pattern better, 
> depending on the case). Tons of docstrings contain incorrect info, we 
> just fix them when we notice the issue, we don't declare the language 
> broken.

I don't think #20075 touches on it, but my biggest concern was 
third-party modules.  If you maintain a Python module, you very well 
might compile for 3.4 only to find that the first line of your 
docstrings have mysteriously vanished.  You'd have to be very current on 
changes in Python 3.4 to know what was going on.  It seemed like an 
overly efficient way of pissing off external module maintainers.

(Why would they vanish?  The mechanical separator for __doc__ vs 
__text_signature__ would accept them, but unless they're 100% compatible 
Python ast.parse will reject them.  So they'd get stripped from your 
docstring, but you wouldn't get a valid signature in return.)

I'd feel much better with an explicit flag--explicit is better than 
implicit, after all.  But here's a reminder, to make it easier for you 
to say "no".  That would mean adding an explicit flag to all the objects 
which support a signature hidden in the docstring:

  * PyTypeObject (has tp_flags, we've used 18 of 32 bits by my count)
  * PyMethodDef (has ml_flags, 7 of 32 bits in use)
  * PyMethodDescrObject (reuses PyMethodDef)
  * PyWrapperDescrObject (has d_base->flags, 1 of 32 bits in use)
  * wrapperobject (reuses PyWrapperDescrObject)

Argument Clinic would write the PyMethodDefs, so we'd get those for 
free.  The last three all originate from a PyMethodDef, so when we 
copied out the docstring pointer we could also propagate the flag. But 
we'd have to add the flag by hand to the PyTypeObjects.

If you won't let me have a flag, can I at least have a more-clever 
marker?  How about this:

    <name-of-function>(...)\n \n

Yes, the last four characters are right-parenthesis, newline, space, and 
newline.  Benefits:

  * The odds of finding *that* in the wild seem remote.
  * If this got displayed as help in 3.3 the user would never notice the

For the record, here are things that may be in the signature that aren't 
legal Python syntax and therefore might be surprising:

  * "self" parameters (and "module" and "type") are prefixed with '$'.
  * Positional-only parameters will soon be delimited by '/', just as
    keyword-only parameters are currently delimited by '*'. (Hasn't
    happened yet.  Needs to happen for 3.4, in order for
    inspect.Signature to be accurate.)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140204/9f014563/attachment.html>

More information about the Python-Dev mailing list