Adding a (small) feature to 3.4 for Argument Clinic: inspect.Signature supporting simple named constants for default values
Serhiy Storchaka ran into a ticklish problem with Argument Clinic and inspect.Signature information for builtins.
Consider pattern_match() in Modules/_sre.c. This implements the match method on a pattern object; in other words, re.compile().match(). The third parameter, endpos, defaults to PY_SSIZE_T_MAX in C. What should inspect.Signature() report as the default value for endpos? And how should it get that value?
Before you answer, consider how inspect.Signature works for builtins.
Argument Clinic hides a signature for the function as the first line of
the docstring; the initialization of the code object strips that off and
puts it in a separate member called __text_signature__.
inspect.Signature pulls that out string and passes it in to ast.parse,
then walks the tree it gets back, pulls out the arguments and their
default values and goes on from there.
We can't turn PY_SSIZE_T_MAX into an integer at Argument Clinic preprocessing time, because this could be done on a completely different architecture than the computer where Python is running. We can't stuff it in at compile time because the macro could devolve into an arbitrary expression (with | or + or something) so while that might work here that's not a general solution. We can't do it at runtime becuase the docstring is a static string.
The best solution seems to be: allow simple symbolic constants as default values. For example, for endpos we'd use sys.maxsize. The code in inspect.Signature would have to support getting an Attribute node, and look up the first field ("sys" in this case) in sys.modules. You could then specify PY_SSIZE_T_MAX as the default for generated C code, and life would be a dream.
I've posted a prototype patch on the tracker:
http://bugs.python.org/issue20144
It need tests and such but I think the basic technology is fine.
The thing is, I feel like this is borderline between bug fix and new feature. But without adding this, we would make a lot of the Argument Clinic conversions pretty messy. So I want to check it in. I just don't want to piss everybody off in the process.
Can you guys live with this?
/arry
p.s.
For what it's worth, the documentation for match() dodges this problem by outright lying. It claims that the prototype for the function is:
match(string[, pos[, endpos]])
which is a lie. pattern_match() parses its arguments by calling PyArg_ParseTupleAndKeywords() with a format string of "O|nn". Which means, for example, you could call:
match("abc", endpos=5)
The documentation suggests this is invalid but it works fine. So my feeling is, this is a legitimate problem, and those who came before me swept it under the rug.
On 06.01.2014 22:34, Larry Hastings wrote:
p.s.
For what it's worth, the documentation for match() dodges this problem by outright lying. It claims that the prototype for the function is:
match(string[, pos[, endpos]])
which is a lie. pattern_match() parses its arguments by calling PyArg_ParseTupleAndKeywords() with a format string of "O|nn". Which means, for example, you could call:
match("abc", endpos=5)
The documentation suggests this is invalid but it works fine. So my feeling is, this is a legitimate problem, and those who came before me swept it under the rug.
Looks like a documentation bug to me. Several functions were changed to be keyword aware some years ago and the docs you quoted still list the old positional format.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Jan 06 2014)
Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 01/06/2014 02:10 PM, M.-A. Lemburg wrote:
Looks like a documentation bug to me. Several functions were changed to be keyword aware some years ago and the docs you quoted still list the old positional format.
Your intuition was spot on. Revision 0a97f5dde3a7, by the effbot, Tue Oct 03 2000.
Tsk tsk, effbot,
//arry/
FWIW, this would probably be better on python-dev, but maybe you wanted to cut down on potential noise and churn?
On Jan 06, 2014, at 01:34 PM, Larry Hastings wrote:
The thing is, I feel like this is borderline between bug fix and new feature. But without adding this, we would make a lot of the Argument Clinic conversions pretty messy. So I want to check it in. I just don't want to piss everybody off in the process.
Can you guys live with this?
I sure can. To me, AC is "the new feature" and as we continue to adopt it in Python 3.4, we'll find more corner cases like this one. Better to do what we can to improve the situation now rather than have to wait until 3.5.
-Barry
On 01/06/2014 02:21 PM, Barry Warsaw wrote:
On Jan 06, 2014, at 01:34 PM, Larry Hastings wrote:
The thing is, I feel like this is borderline between bug fix and new feature. But without adding this, we would make a lot of the Argument Clinic conversions pretty messy. So I want to check it in. I just don't want to piss everybody off in the process.
Can you guys live with this?
I sure can. To me, AC is "the new feature" and as we continue to adopt it in Python 3.4, we'll find more corner cases like this one. Better to do what we can to improve the situation now rather than have to wait until 3.5.
Agreed.
-- ~Ethan~
On 7 Jan 2014 07:34, "Ethan Furman" ethan@stoneleaf.us wrote:
On 01/06/2014 02:21 PM, Barry Warsaw wrote:
On Jan 06, 2014, at 01:34 PM, Larry Hastings wrote:
The thing is, I feel like this is borderline between bug fix and new
feature.
But without adding this, we would make a lot of the Argument Clinic conversions pretty messy. So I want to check it in. I just don't want to piss everybody off in the process.
Can you guys live with this?
I sure can. To me, AC is "the new feature" and as we continue to adopt it in Python 3.4, we'll find more corner cases like this one. Better to do what we can to improve the situation now rather than have to wait until 3.5.
Agreed.
Same from me. I see it as similar to ensurepip and the import changes - as we've been fully incorporating those, we noticed some edge cases and API issues that we want to tweak before the first release candidate.
Cheers, Nick.
-- ~Ethan~
python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers
On 1/6/2014 5:21 PM, Barry Warsaw wrote:
FWIW, this would probably be better on python-dev, but maybe you wanted to cut down on potential noise and churn?
Better here, I think.
On Jan 06, 2014, at 01:34 PM, Larry Hastings wrote:
The thing is, I feel like this is borderline between bug fix and new feature. But without adding this, we would make a lot of the Argument Clinic conversions pretty messy. So I want to check it in. I just don't want to piss everybody off in the process.
Can you guys live with this?
I sure can. To me, AC is "the new feature" and as we continue to adopt it in Python 3.4, we'll find more corner cases like this one. Better to do what we can to improve the situation now rather than have to wait until 3.5.
I agree with Barry. To my understanding, the reason for the internal 'feature freeze' with beta 0, with betas yet to come, is so that we can test new API's and *possibly adjust the details*, based on experience, before the final ..0 release and public feature freeze. All new features are provisional until then (we could rip out a new feature), which is why betas are not for production use, even if they are otherwise the best release available for some purposes. The fix versus feature question, for changes to the new feature, comes into play again after ..0, when considering ..1, etc.
Terry
participants (6)
-
Barry Warsaw
-
Ethan Furman
-
Larry Hastings
-
M.-A. Lemburg
-
Nick Coghlan
-
Terry Reedy