Hi The present behaviour is: >>> len() TypeError: len() takes exactly one argument (0 given) >>> len(1, 2) TypeError: len() takes exactly one argument (2 given) This is nothing special about built-ins. The same goes for: >>> def f(): pass >>> f(1) TypeError: f() takes 0 positional arguments but 1 was given I find this behaviour perfectly satisfactory. The OP's problem can be resolved by looking at the exception, whose type is TypeError. For this to be completely reliable, certain aspects the exception message need to be stable. It may be worth stating that this will be done, at least for certain exception. By the way, using the exception instance will produce code that works with earlier versions of Python. -- Jonathan

On Wed, Mar 04, 2020 at 10:12:34AM +0000, Jonathan Fine wrote:

I find this behaviour perfectly satisfactory. The OP's problem can be resolved by looking at the exception, whose type is TypeError.

For this to be completely reliable, certain aspects the exception message need to be stable. It may be worth stating that this will be done, at least for certain exception.

Error messages are not part of the function API and are not guaranteed to be a constant part of the language. We shouldn't expect every single Python interpreter (CPython, IronPython, Jython, Nuitka, MicroPython, PyPy, Stackless etc) to use exactly the same error messages. It shouldn't be a *feature change* if an error message changes. We shouldn't have to use a deprecation period or a `__future__` import before we can improve error messages, fix spelling or grammatical errors, or add extra information to the error message. Error messages might be localised into the native language of the user. Parsing error messages is a fragile and error-prone technique that should be avoided if at all possible. -- Steven

Steven D'Aprano wrote: Another alternative is to testing for key phrases in the exception

message:

if "too few arguments" in err.args[0]: ...

but that's fragile and only works until the message gets changed to say "not enough arguments". (The error message is not part of the API, so it can change without warning.)

We are talking here about an exception message that is generated by core Python. So there would be some warning of change, namely the review process for core Python. I agree that there is a problem to be solved here. Perhaps it is easier and better for core Python to decide to freeze certain messages, than to add a new exception type. -- Jonathan

On Wed, Mar 4, 2020 at 9:58 PM Steven D'Aprano <steve@pearwood.info> wrote:

The important thing is to be able to programmatically distinguish the case where arguments don't match the parameters from the case where an argument is the wrong type.

Or where you're trying to call something that isn't callable. There are several distinct "phases" to calling a function, and TypeError could come up from any of them:

...

...

"spam"() <stdin>:1: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma? Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'str' object is not callable len(1, 2) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: len() takes exactly one argument (2 given) len(1) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'int' has no len()

Being able to easily distinguish "that isn't callable" from "the parameters don't line up" from "the function, during execution, raised TypeError" would be useful. So, I agree that the bikeshed is of value here. As to colour, I'd be (weakly) inclined toward: 1) ParameterError and maybe UncallableError as well 2) Type flags with class constants eg ParameterError.TOO_MANY Even if TOO_MANY is technically just the integer 2 (as opposed to an IntEnum), encouraging people to use that will make it a lot easier to read. C code can use a #define for maximum performance, and then the cost of looking up the attribute is paid only if the exception is caught AND checked for that status - minimal overhead on creation. ChrisA

On Mar 4, 2020, at 03:00, Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Mar 04, 2020 at 01:08:35AM -0800, Andrew Barnert wrote:

...

I liked everything up to using magic numbers like this.

I'm not wedded to the idea of error number. I chose them because I expect there will only be a few of them (three?) and the majority of the time you won't bother to check the number.

OK, but what’s the downside of giving you a named constant instead of making you use the magic number 2? Making CPython 400 bytes bigger? Forcing the help for ParameterError to display the kinds of error the same way all named constants are displayed when you had a better way to display them?

...

But this is just arguing about the colour of the bikeshed. Do you agree with me that the bikeshed itself is useful?

I’m +0 on this (if there’s an implementation that can cover all of the common cases). If people are switching on the error message string in real code (and I think I’ve done that once, for Python 2.7/3.3 code where I auto-generated wrappers around appscript proxies), there ought to be another way, and having a new error subclass is the obvious pythonic better way. The only question is whether it comes up often enough to be worth fixing.

...

The other question: what exactly do you propose to change? I don’t think you can do this on the caller side (the ceval bytecode handler and the PyCall functions).

I don't know enough about the Python internals to give a definitive answer, but I'm assuming/hoping that there is a single, or at most a few, places in the interpreter that matches up arguments to formal parameters, and if there's a discrepency it currently raises TypeError. PyCall sounds promising :-)

I’m pretty sure there isn’t a single place, and PyCall is not useful—but I think there may be a small handful of places that’s good enough. IIRC, it works like this: The PyCall functions and the bytecode handler eventually call the same code, but that code just prepares the arguments in a generic way that the callee can match them: as a tuple of positional args and a dict of keyword args. It doesn’t even know anything about the callee. The parsing of that tuple and dict is up to the callee. Which can be anything. But usually, it’s either: * a function (or type or thing with __call__) written in Python, in which case the PyFunction object handles matching args to params and creating exceptions * a C function that just passes the tuple and dict to one of a small set of PyArg parse functions, which do the matching and exceptions * a C function with auto-generated argclinic code, which I think is the same as above, but if not argclinic is just one more place to change A C function could just manually parse the tuple and dict, but at that point it’s like a Python function that asks for *args, **kw and parses those, and I think you’re right that we don’t need to worry about those. Especially since in most cases they’re just proxies or bridges to some other function that will parse the args the normal way and they’ll just pass that exception up. So, I think those 2 or 3 places (that may turn out to be like 6 places in the CPython source) may be sufficient. But you’d have to check that, not just go by my memory. :) Also, it’s probably worth checking some very popular special cases: ctypes (with argtypes), cffi, a couple of proxy callables like partial and MethodType, Cython, the code generator from NumPy, a few static and dynamic bridge libraries like boost::python and PyObjC, … I think most of them will already just work, except maybe for some dynamic bridges (e.g., if PyObjC is just converting the tuple and dict to an NSInvocation, calling that via ObjC, and then parsing the ObjC error to generate the Python one…), which I think would be perfectly acceptable. But it’s worth knowing whether there will be such cases.

...

That still leaves functions that parse *args and **kw and raise TypeError manually, whether in Python or in C, but I suppose you can say that in that case it really isn’t a parameter error (the signature really is *args, **kw so everything matches).

Indeed.

I don't have any expectations about those functions. If the maintainer of the function eventually changes whatever error they are currently raising to ParameterError, that will be grand, but I don't expect every third-party function that implements some custom variety of parameter handling to support this. If they do, great, if they don't, we're no worse off than the status quo.

...

What about inspect.Signature.bind?

Do you have a specific concern?

Just whether you’re proposing to change it as part of the proposal. I think you do want to, and I think it’ll be easy, but I don’t think either answer would be a deal breaker. More generally, I’m just trying to make sure things like inspect.signature, and partial and so on, have been considered before anyone makes a decision.

On Wed, Mar 04, 2020 at 12:19:25PM +0200, Alex Hall wrote:

Can you explain where `except ParameterError` would produce better code than the best current alternative?

Serhiy came up with a good use-case that I hadn't considered: functions which take arbitrary callback functions as argument, e.g. sorted() with a key function. Both of these calls raise TypeError, but for different reasons: # right number of arguments passed to the key function, # but the wrong argument type sorted([3, 5, 1], key=len) # wrong number of arguments passed to the key function sorted([3, 5, 1], key=isinstance) It might be useful to be able to distinguish the two cases. My earlier use-case still stands: feature detection where a function has changed its parameter list. More on this below.

It seems to me that using this is like stating "this code might be wrong" and would generally produce bad code.

It seems to me that your comment above could be equally said about *any* use of exceptions: "this code might be wrong, so we'll stick it in a try...except block". Only that's not how most of us actually use exceptions.

For example if I wanted to use math.gcd on multiple values but I need to support Python versions that only allow two arguments, then I'll just apply it to two arguments at a time, maybe with functools.reduce.

I've done that myself. Not specifically with gcd, but with other functions, which I don't remember off the top of my head. gcd is a particularly simple example because it is so easy to wrap. Excluding function header and docstring, the code is just six lines: if len(args) == 0: return 1 elif len(args) == 1: return abs(args[0]) else: return reduce(_gcd, args) Not all cases will be that simple, or they may involve a serious performance cost. In general, the official version is going to be faster and better tested than the wrapper version. Why shouldn't I use the fast, well-tested standard version if it is available? I just need a reliable way to detect the needed feature. This is not specifically about gcd. It could be about functions of arbitrary complexity: try: # Test whether the aardvark function supports a # `hovercraft` parameter. result = aardvark( spam, eggs, cheese, hovercraft=1 ) except ParameterError: # No hovercraft parameter allowed, create a wrapper. ... TypeError is less satisfactory because I cannot easily distinguish the two cases: 1. The 'hovercraft' parameter is not available; 2. The 'hovercraft' parameter is available, but I have made some other error in the function call which results in a TypeError.

And in general if I want to distinguish between versions I'd much rather just check the version and thus assert "this code is correct for this version".

Feature detection is more reliable. What if the caller of my library has back-ported the new, advanced version of aardvark() to their Python? Instead of using the faster, better tested official backport, my code will use my wrapper. What if the caller is using my library with another interpreter, say PyPy or Jython, which doesn't support the feature I need? With feature detection, my code will simply fall back to the wrapper. With version checking, it will wrongly assert the feature is available, and then crash later. As far as I can tell, feature detection has more or less completely overshadowed version checks in the browser and Javascript space. In my opinion, checking the version should only be used when it is too hard to detect the feature.

If you catch ParameterError, how do you know that it came directly from the line you caught it from and not deeper within?

Does it matter? If the call aardvark(*args, hovercraft=1) leaks a ParameterError from deep inside its internals, then even if the feature is technically available according to a version check, it is too buggy to use and I need to use my wrapper.

If you want to check that you're passing parameters properly, I suggest writing code like this:

import inspect

signature = inspect.signature(func)

And what if the function I'm checking doesn't support signature metadata? https://docs.python.org/3/library/inspect.html#inspect.signature I can wrap the call to signature() in try...except, but what's my fallback if I can't check the signature? py> inspect.signature(math.gcd) Traceback (most recent call last): [...] TypeError: 'feature_version' is an invalid keyword argument for compile() -- Steven

It seems to me that this thread is a bit too focused. I know I’ve found that often I wish I knew better exactly what caused a particular Exception— there are many that can be raised for various reasons, and you may need to know which one in order to handle it properly. At this point, parsing the error msg is often the only option. Though since you can tack on arbitrary data to an Exception, some have ways of knowing. So maybe a more formal and standardized way to to specify more detailed information is in order. But it shouldn’t be just about TypeError. On the other hand, subclassing exceptions works pretty well, so maybe that’s fine. It worked pretty well for OSError, for example. As for TypeError and calling functions. I believe the OP wanted to distinguish between calling the function the wrong way: e.g. the wrong [number of, name of, order of] arguments, and calling the function with the wrong type of a given parameter. And Python being dynamically typed, calling a function with the wrong Type is really an arbitrary somthing-went-wrong-inside-the-function, not an error with calling per se. And I think this is a useful distinction. It's actually a common problem, when you do: try: func(...) Except SomeError: do_something you never know where in the entire call stack SomeError came from. In the case at hand, a TypeError may have come from how you are calling func, or it may have come from somewhere inside func(), or indeed, anywhere else deep in that hierachy. So I think it would be very useful to have an Exception that is about the "how the function was called" part. Of course, it could still have come from another call inside the top-l;evel function, but I think it sill makes a useful distinction. -CHB On Wed, Mar 4, 2020 at 7:11 AM Alex Hall <alex.mojaki@gmail.com> wrote:

...

Both of these calls raise TypeError, but for different reasons: # right number of arguments passed to the key function, # but the wrong argument type sorted([3, 5, 1], key=len)

# wrong number of arguments passed to the key function sorted([3, 5, 1], key=isinstance)

It might be useful to be able to distinguish the two cases.

I'm hearing a lot of hypotheticals and nothing close to what feels like a real use case. Both those cases are completely wrong and I don't know why I'd need to distinguish the reason, or even catch TypeError at all.

...

...

It seems to me that using this is like stating "this code might be wrong" and would generally produce bad code. It seems to me that your comment above could be equally said about any use of exceptions: "this code might be wrong, so we'll stick it in a try...except block". Only that's not how most of us actually use exceptions.

I don't understand, you seem to be contradicting yourself between "any use of exceptions" and "that's not how most of us actually use exceptions". The examples you're giving explicitly look like "this code might be wrong". The examples of exceptions I'm looking at in my own code are more like "the inputs might be wrong" or "something might go wrong beyond my control".

...

try: # Test whether the aardvark function supports a # `hovercraft` parameter. result = aardvark( spam, eggs, cheese, hovercraft=1 ) except ParameterError: # No hovercraft parameter allowed, create a wrapper. ...

This is a weird hypothetical. Please provide a real example. Code that you have actually written vs what you wish you could have written. I have certainly never wanted to do something like this. And if I was in such a situation that's still not the way I'd want to solve it.

...

What if the caller of my library has back-ported the new, advanced version of aardvark() to their Python? Instead of using the faster, better tested official backport, my code will use my wrapper.

I don't really understand how this would work, can you elaborate?

...

...

If you catch ParameterError, how do you know that it came directly from the line you caught it from and not deeper within? Does it matter? If the call aardvark(*args, hovercraft=1) leaks a ParameterError from deep inside its internals, then even if the feature is technically available according to a version check, it is too buggy to use and I need to use my wrapper.

But maybe the function `spam` that you passed to aardvark didn't take the right number of parameters?

To make it more concrete, imagine code like this from a time when newer versions of `sort` accepted `key` but older versions could only accept `cmp` (I'm not sure if this is actually what happened, it's not important).

try: lst.sort(key=foo_key) except ParameterError: lst.sort(cmp=foo_cmp)

Does `except ParameterError:` mean that `sort` doesn't accept `key`, or does it mean that `foo_key` is wrong?

...

py> inspect.signature(math.gcd) Traceback (most recent call last): [...] TypeError: 'feature_version' is an invalid keyword argument for compile()

That's interesting, what version of Python gives you that? It works for me. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WEPACE... Code of Conduct: http://python.org/psf/codeofconduct/