The `assert' statment was created the same as in previous languages like C/C++: a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly. Examples are array bounds and object state integrity constraints. Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Furthermore, validation checks are an extremily common use case. I have been using assert in my in-house code for input validation for a long time, the only thing preventing me from doing the same in public code is the fact that it only raises AssertionError's while library code is expected to raise TypeError or ValueError on invalid input. The last drop that incited me to do this proposition was https://stackoverflow.com/questions/2142202/what-are-acceptable-use-cases-fo... where none of the _seven_ answer authors (beside me) managed to make a _single realistic use case_ for `assert' as a debug-only check. --- So, I'm hereby proposing to: * make `assert' stay in optimized mode * change its syntax to raise other types of exceptions E.g.: "assert condition, type, value" or "assert condition, type, exception_constructor_argument(s)" to maintain backward compatibility. -- Regards, Ivan
On Tue, Nov 28, 2017 at 1:12 PM, Ivan Pozdeev via Python-ideas <python-ideas@python.org> wrote:
The `assert' statment was created the same as in previous languages like C/C++: a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly. Examples are array bounds and object state integrity constraints.
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Furthermore, validation checks are an extremily common use case.
I have been using assert in my in-house code for input validation for a long time, the only thing preventing me from doing the same in public code is the fact that it only raises AssertionError's while library code is expected to raise TypeError or ValueError on invalid input.
Actually, Python does have a way of disabling assertions (the -O flag), so they should be treated the same way they are in C. Assertions should not be used as shorthands for "if cond: raise Exc" in the general case. ChrisA
Actually, Python does have a way of disabling assertions (the -O flag), so they should be treated the same way they are in C. Assertions should not be used as shorthands for "if cond: raise Exc" in the general case. I'm claiming, and provided evidence, that there are no use cases for
On 28.11.2017 5:19, Chris Angelico wrote: this in Python, so no-one (of any significance) will suffer when the disabling is cut out. In any case, we will probably do it with a __future__ statement for a transitional period. -- Regards, Ivan
On Mon, Nov 27, 2017 at 9:28 PM, Ivan Pozdeev via Python-ideas < python-ideas@python.org> wrote:
On 28.11.2017 5:19, Chris Angelico wrote:
Actually, Python does have a way of disabling assertions (the -O
flag), so they should be treated the same way they are in C. Assertions should not be used as shorthands for "if cond: raise Exc" in the general case.
I'm claiming, and provided evidence, that there are no use cases for this in Python, so no-one (of any significance) will suffer when the disabling is cut out. In any case, we will probably do it with a __future__ statement for a transitional period.
I think it would be interesting to investigate how assert statements are used in the wild. I can think of three kinds of uses: 1) Nonredundant checking: The assertion aims to be a concise way to raise exceptions under certain conditions that might arise even presuming that the code is internally correct. (For example, to check that the interface with another piece of code is as expected, or to validate user input.) This seems to be the use case the OP is targeting. It is probably not the use envisioned by the designers of the assert statement. But perhaps assert is frequently used in this way by people who don't know about the optimization flag or assume their program won't be run with optimization. 2) Redundant checking: The assertion acts as a test of code correctness. If the code is correct, it should have no effect; and presuming the code is well-tested, it can thus be ignored with optimization turned on. But it is useful as a sanity check of a tricky algorithm, and can help a reader to understand the implementation. Is it often the case that optimizing these assertions away brings significant performance savings? 3) Temporary debugging: I often write something like assert False,(x,y,z) as a quick way to force the program to terminate and print some values. Unlike print(x,y,z), this has the advantage of being blatantly obvious as debugging code that does not belong in the final version. It seems unlikely that one would purposefully turn on optimization in conjunction with temporary assertions, so this use case may be irrelevant to the proposal. Best, Nathan
Nathan Schneider wrote:
I think it would be interesting to investigate how assert statements are used in the wild. I can think of three kinds of uses:
assert statements are also a form of executable documentation. It says something like "I think the internal state of my code must be this way, otherwise I don't really understand what I wrote". Now, maybe you could argue that such use should be enabled unconditionally, but I disagree since usually your understanding of the state of your code is correct, so the additional checks are unnecessary, and people *do* use -O and -OO in practice. And these days it's even more practical to do so, given the `opt-*` level of pyc tagging: % python3 -c "import foo" % python3 -O -c "import foo" % python3 -OO -c "import foo" % ls foo/__pycache__/ __init__.cpython-36.opt-1.pyc __init__.cpython-36.pyc __init__.cpython-36.opt-2.pyc I also wonder how this would interact with pytest's um, 'hijacking' of the assert statement. Cheers, -Barry
On 11/29/2017 08:31 PM, Barry Warsaw wrote:
Nathan Schneider wrote:
I think it would be interesting to investigate how assert statements are used in the wild. I can think of three kinds of uses:
assert statements are also a form of executable documentation. It says something like "I think the internal state of my code must be this way, otherwise I don't really understand what I wrote".
Now, maybe you could argue that such use should be enabled unconditionally, but I disagree since usually your understanding of the state of your code is correct, so the additional checks are unnecessary, and people *do* use -O and -OO in practice. And these days it's even more practical to do so, given the `opt-*` level of pyc tagging:
% python3 -c "import foo" % python3 -O -c "import foo" % python3 -OO -c "import foo" % ls foo/__pycache__/ __init__.cpython-36.opt-1.pyc __init__.cpython-36.pyc __init__.cpython-36.opt-2.pyc
I also wonder how this would interact with pytest's um, 'hijacking' of the assert statement.
Pytest compiles from source, and uses its own cache tags which look like ".cpython-36-PYTEST.pyc". https://github.com/pytest-dev/pytest/blob/master/_pytest/assertion/rewrite.p...
(for some reason google groups has accepted the message but the mailing list rejected it. Re-posting it, sorry for the inconvenience) Le mardi 28 novembre 2017 04:22:13 UTC+1, Nathan Schneider a écrit :
I think it would be interesting to investigate how assert statements are used in the wild. I can think of three kinds of uses:
1) Nonredundant checking: 2) Redundant checking: 3) Temporary debugging:
Hello there I am very much interested by this topic, as I spent a couple months trying to come up with a solution with the obvious constraint of not changing the language. My goal was type checking and value validation for applications wishing to get these features (so obviously, with the possibly to not disable it at runtime even if the rest of the application is optimized). For type checking I discovered PEP484 and many type-checkers (enforce, pytypes...) out there, but for value validation I found nothing really satisfying to me except many sources saying not to use assert. I personally align with the idea already mentioned in this thread that assert is not a tool for "end-user value validation", but a tool for other kind of purposes - that are perfectly valid but are just not the same that "end-user value validation" (if the name is not perfect, feel free to propose another). What I think we're missing is “get consistent and customizable validation outcome, whatever the inner validation means”. Typically ‘assert isfinite(x)‘ is today a good example of what is wrong: 1. it can be disabled globally by end-users even if the lib developer does not want it, 2. if x is None the exception is different from the exception you get if x is not finite, 3. you can not customize the exception type for easy error codes internationalization, you can only customize the message. My proposal would be to rather define another statement for example 'validate <expression> <exception, exception_type or exception_message>' with specific constraints on the type of exception to ensure consistency (for example only subclasses of a ValidationError defined in the stdlib) You can find my attempt to do that in the valid8 project https://smarie.github.io/python-valid8, with the 'assert_valid(...)' function. With the current language limitations I could not define something as simple as 'validate <expression> <exception, exception_type or exception_message>', but I created a mini lambda library to at least keep some level of simplicity for the <expression>. The result is for example : class InvalidSurface(ValidationError): help_msg = 'Surface should be a positive number' assert_valid('surface', surf, x > 0, error_type=InvalidSurface) Oh, by the way, in valid8 I have the notion of variable name everywhere (again, because thats interesting to the application) but I'm not sure whether it would make sense to keep it in a 'validate' statement. Let me know what you think Kind regards Sylvain
16.01.18 11:38, smarie пише:
You can find my attempt to do that in the valid8 project https://smarie.github.io/python-valid8 <https://smarie.github.io/python-valid8>, with the 'assert_valid(...)' function. With the current language limitations I could not define something as simple as 'validate <expression> <exception, exception_type or exception_message>', but I created a mini lambda library to at least keep some level of simplicity for the <expression>. The result is for example :
class InvalidSurface(ValidationError): help_msg = 'Surface should be a positive number'
assert_valid('surface', surf, x > 0, error_type=InvalidSurface)
What is the advantage over the simple if not x > 0: raise InvalidSurface('surface') ?
On Tue, Jan 16, 2018 at 01:38:25AM -0800, smarie wrote:
Typically ‘assert isfinite(x)‘ is today a good example of what is wrong:
1. it can be disabled globally by end-users even if the lib developer does not want it,
That's not a bug, or even a problem ("wrong"), it is the very purpose of assert. Assertions are intended to allow the end user to disable the checks. If you, the developer, don't want a check to be disabled, then you shouldn't call it an assertion and use assert. I sometimes wish that Python included a richer set of assertions rather than just a single `assert` keyword. Something like Eiffel's concept of pre-conditions, post-conditions and invariants, where each can be enabled or disabled independently. But even with Python's sparse assertion API, the reason for assert is for debugging checks which can optionally be disabled. Checks which should never be disabled don't belong as assertions.
2. if x is None the exception is different from the exception you get if x is not finite
This is a little untidy, but I don't see why this should be considered "wrong". That's just the way Python operates: however you write a check, it is going to rely on the check succeeding, and if it raises an exception, you will see that exception. That's a good thing, not "wrong" -- if my assertion isfinite(x) fails because x is None, I'd much rather see a TypeError than AssertionError: x is not finite
3. you can not customize the exception type for easy error codes internationalization, you can only customize the message.
assert is not intended to be used to present user-friendly error messages to the end-user. The end-user should never see an AssertionError, and if they do, that's a bug, not a recoverable error or an avoidable environmental/input error. I have come to the conclusion that a good way to think about assertions is that they are often "checked comments". Comments like: # when we get here, x is a finite number quickly get out of sync with code. To quote Michael Foord: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. " -- Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 Instead, turn it into an assertion, if you can: assert isfinite(x) which is both inline developer documentation ("x is a finite number") and a debugging aid (it will fail during development is x is not a finite number). And, if the cost of those assertions becomes excessive, the end-user can disable them.
My proposal would be to rather define another statement for example
'validate <expression> <exception, exception_type or exception_message>'
The problem with a statement called "validate" is that it will break a huge number of programs that already include functions and methods using that name. But apart from the use of a keyword, we already have a way to do almost exactly what you want: if not expression: raise ValidationError(message) after defining some appropriate ValidationError class. And it is only a few key presses longer than the proposed: validate expression, ValidationError, message -- Steve
Le mardi 16 janvier 2018 11:24:34 UTC+1, Steven D'Aprano a écrit :
That's not a bug, or even a problem ("wrong"), it is the very purpose of assert. Assertions are intended to allow the end user to disable the checks.
If you, the developer, don't want a check to be disabled, then you shouldn't call it an assertion and use assert.
That is exactly what I'm saying. It seems that we both agree that applicative value validation is different from asserts, and that assert should not be used for applicative value validation. For this reason, I do not suggest to go in the direction the OP is mentioning but rather to explicitly separate the 2 concepts by creating a new statement for value validation.
The problem with a statement called "validate" is that it will break a huge number of programs that already include functions and methods using that name.
You definitely make a point here. But that would be the case for absolutely *any* language evolution as soon as the proposed statements are plain old english words. Should it be a show-stopper ? I dont think so.
But apart from the use of a keyword, we already have a way to do almost exactly what you want:
if not expression: raise ValidationError(message)
after defining some appropriate ValidationError class. And it is only a few key presses longer than the proposed:
validate expression, ValidationError, message
This is precisely what is not good in my opinion: here you do not separate <validation means> from <validation intent>. Of course if <validation means> is just a "x > 0" statement, it works, but now what if you rely on a 3d-party provided validation function (or even yours) such as e.g. "is_foo_compliant" ? if not is_foo_compliant(x): raise ValidationError(message) What if this third part method raises an exception instead of returning False in some cases ? try: if not is_foo_compliant(x): raise ValidationError(message) except: raise MyValidationError(message) What if you want to compose this third party function with *another* one that returns False but not exceptions ? Say, with an OR ? (one or the other should work). Yields: try: if not is_foo_compliant(x): raise ValidationError(message) except: if not is_bar_compliant(x): raise MyValidationError(message) It starts to be quite ugly, messy... while the applicative intent is clear and could be expressed directly as: validate is_foo_compliant(x) or is_bar_compliant(x) ValidationError(message) The goal is really to let developers express their applicative intent =(what should be checked and what is the outcome if anything goes wrong), and give them confidence that the statement will always fail the same way, whatever the failure modes /behaviour of the checkers used in the statement. This is what valid8 <https://smarie.github.io/python-valid8/> proposes today with assert_valid, but obviously again having it built-in in the language would be much more concise. Note that the class and function decorators (@validate_arg, @validate_field...) would remain useful anyway. -- Sylvain
Le mardi 16 janvier 2018 16:37:29 UTC+1, smarie a écrit :
validate is_foo_compliant(x) or is_bar_compliant(x) ValidationError(message)
This was a typo in this case since we use the base ValidationError it even would simplify to validate is_foo_compliant(x) or is_bar_compliant(x), message
On Tue, Jan 16, 2018 at 07:37:29AM -0800, smarie wrote: [...]
The problem with a statement called "validate" is that it will break a huge number of programs that already include functions and methods using that name.
You definitely make a point here. But that would be the case for absolutely *any* language evolution as soon as the proposed statements are plain old english words. Should it be a show-stopper ? I dont think so.
It is not a show-stopper, but it is a very, very larger barrier to adding new keywords. If there is a solution to the problem that doesn't require a new keyword, that is almost always preferred over breaking people's code when they upgrade.
But apart from the use of a keyword, we already have a way to do almost exactly what you want:
if not expression: raise ValidationError(message)
after defining some appropriate ValidationError class. And it is only a few key presses longer than the proposed:
validate expression, ValidationError, message
This is precisely what is not good in my opinion: here you do not separate <validation means> from <validation intent>. Of course if <validation means> is just a "x > 0" statement, it works, but now what if you rely on a 3d-party provided validation function (or even yours) such as e.g. "is_foo_compliant" ?
There's no need to invent a third-party validation function. It might be my own validation function, or it might be a simple statement like: if variable is None: ... which can fail with NameError if "variable" is not defined. Or "x > 0" can fail if x is not a number. Regardless of what the validation check does, there are two ways it can not pass: - the check can fail; - or the check can raise an exception. The second generally means that the check code itself is buggy or incomplete, which is why unittest reports these categories separately. That is a good thing, not a problem to be fixed. For example: # if x < 0: raise ValueError('x must not be negative') validate x >= 0, ValueError, 'x must not be negative' Somehow my code passes a string to this as x. Wouldn't you, the developer, want to know that there is a code path that somehow results in x being a string? I know I would. Maybe that will become obvious later on, but it is best to determine errors as close to their source as we can. With your proposed validate keyword, the interpreter lies to me: it says that the check x >= 0 *fails* rather than raises, which implies that x is a negative number. Now I waste my time trying to debug how x could possibly be a negative number when the symptom is actually very different (x is a string). Hiding the exception is normally a bad thing, but if I really want to do that, I can write a helper function: def is_larger_or_equal(x, y): try: return x >= y except: return False If I find myself writing lots of such helper functions, that's probably a hint that I am hiding too much information. Bare excepts have been called the most diabolic Python anti-pattern: https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ so hiding exceptions *by default* (as your proposed validate statement would do) is probably not a great idea. The bottom line is, if my check raises an exception instead of passing or failing, I want to know that it raised. I don't want the error to be hidden as a failed check.
if not is_foo_compliant(x): raise ValidationError(message)
What if this third part method raises an exception instead of returning False in some cases ?
Great! I would hope it did raise an exception if it were passed something that it wasn't expecting and can't deal with. There may be some cases where I want a validation function to ignore all errors, but if so, I will handle them individually with a wrapper function, which let's me decide how to handle individual errors: def my_foo_compliant(x): try: return is_foo_compliant(x) except SpamError, EggsError: return True except CheeseError: return False except: raise But I can count the number of times I've done that in practice on the fingers of one hand. [...]
The goal is really to let developers express their applicative intent =(what should be checked and what is the outcome if anything goes wrong), and give them confidence that the statement will always fail the same way, whatever the failure modes /behaviour of the checkers used in the statement.
I don't agree that this is a useful goal for the Python interpreter to support as a keyword or built-in function. If you want to create your own library to do this, I wish you good luck, but I would not use it and I honestly think that it is a trap: something that seems to be convenient and useful but actually makes maintaining code harder by hiding unexpected, unhandled cases as if they were expected failures. -- Steve
On 28.11.2017 03:28, Ivan Pozdeev via Python-ideas wrote:
On 28.11.2017 5:19, Chris Angelico wrote:
Actually, Python does have a way of disabling assertions (the -O flag), so they should be treated the same way they are in C. Assertions should not be used as shorthands for "if cond: raise Exc" in the general case. I'm claiming, and provided evidence, that there are no use cases for this in Python, so no-one (of any significance) will suffer when the disabling is cut out.
... except those who have relied on this behavior for around two decades to make their code run faster in production environments and those who run on memory constrained systems using -OO to remove doc-strings as well. asserts are meant as debug tool and runtime way to document application internal expectations of code following them. Once all tests pass they are no longer needed. If you find that you still need them, you should recode the asserts as proper if statements. As example use case, asserts testing user provided data are not a good idea, since you cannot test all possible user inputs. For those cases, an if statement is the right way to implement your checks. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 28 2017)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Assertions should not be used as shorthands for "if cond: raise Exc" in the general case.
I'm just a lurker and usually I agree with why the suggested features shouldn't be implemented, but I actually might chime in to pitch this one a bit more -- and I think it can be done nicely without breaking backward compatibility. As a scientist, I like assert statements for two reasons: 1) you can disable them to speed up the code, and 2) it's how we think as scientists. Consider these two pieces of code, and which one you'd prefer to read: if not condition: raise ValueError assert condition: raise ValueError As a scientist, I prefer the second one because I naturally read it as: "condition is true, therefore ..." and I can predict the next step of the algorithm naturally by filling in the "..." in my mind. It makes the assumptions of the code a bit more explicit than the code `if not condition:`, which I must think about and to translate to "the condition must be true" before I can continue reading. That, in addition to being able to disable the assert statements, makes me like and use them a reasonable amount. However, every time I do use them, I always think "crap, when this breaks I'm going to have to come back here and read my code/comments because the error message isn't very helpful", and that makes me not want to write assert statements. So I like writing them because while I'm writing/reading the code, they make sense, but I don't like reading their error message output because it isn't useful to me as a user/developer. I realize this is very minor, but I actually really like it, and I think the below syntax would be pretty nice and backwards compatible: assert condition: raise ValueError Jason
On 11/27/17 9:12 PM, Ivan Pozdeev via Python-ideas wrote:
The `assert' statment was created the same as in previous languages like C/C++: a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly. Examples are array bounds and object state integrity constraints.
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Furthermore, validation checks are an extremily common use case.
I have been using assert in my in-house code for input validation for a long time, the only thing preventing me from doing the same in public code is the fact that it only raises AssertionError's while library code is expected to raise TypeError or ValueError on invalid input.
The last drop that incited me to do this proposition was https://stackoverflow.com/questions/2142202/what-are-acceptable-use-cases-fo... where none of the _seven_ answer authors (beside me) managed to make a _single realistic use case_ for `assert' as a debug-only check.
---
So, I'm hereby proposing to:
* make `assert' stay in optimized mode * change its syntax to raise other types of exceptions
E.g.: "assert condition, type, value" or "assert condition, type, exception_constructor_argument(s)" to maintain backward compatibility.
You are proposing: assert condition, type, value Why not just use Python as it is now, with this:? if not condition: raise type(value) I don't see a reason to change the assert statement. You can do what you need without the change. --Ned.
On 28.11.2017 6:34, Ned Batchelder wrote:
You are proposing:
assert condition, type, value
Not specifically this, that's just an example. Actually, the way I'm using them, assert condition, "error message", type would probably be the most expressive way.
Why not just use Python as it is now, with this:?
It's the most expressive way the language provides to write that logic. With Python's design focus on promoting expressive, readable and intuitive syntax, that's enough of a reason.
if not condition: raise type(value)
I don't see a reason to change the assert statement. You can do what you need without the change.
I can do anything in any Turing-complete language without any changes to the language. That's no reason to never change anything, is it. The rationale basically is: * As it was intended, the statement has no practical use -- basically a rudiment, due to disappear eventually * It can instead be reused as syntax sugar to cover a very common use case -- Regards, Ivan
On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas wrote:
Actually, the way I'm using them,
assert condition, "error message", type
would probably be the most expressive way.
I disagree that is expressive -- I call it *misleading*. I see something which looks like an assertion (that is, a checked comment, a contract, a check on an internal piece of logic etc) but it is actually being used as a test.
I can do anything in any Turing-complete language without any changes to the language. That's no reason to never change anything, is it.
"We can change this" is not a reason to change this. There needs to be a *good* reason to change, and you have given no good reasons for this change.
The rationale basically is: * As it was intended, the statement has no practical use -- basically a rudiment, due to disappear eventually
Nonsense. I make extensive use of assert as a way of checking assertions, and I will fight tooth and nail against any proposal to either remove it or to misuse it for public input tests instead of assertions.
* It can instead be reused as syntax sugar to cover a very common use case
There is no need for such syntactic sugar. It would be harmful to use assert for something which is not an assertion. -- Steve
On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas wrote:
Actually, the way I'm using them,
assert condition, "error message", type
would probably be the most expressive way. I disagree that is expressive -- I call it *misleading*. I see something which looks like an assertion (that is, a checked comment, a contract, a check on an internal piece of logic etc) but it is actually being used as a test.
I can do anything in any Turing-complete language without any changes to the language. That's no reason to never change anything, is it. "We can change this" is not a reason to change this. There needs to be a *good* reason to change, and you have given no good reasons for this change.
The rationale basically is: * As it was intended, the statement has no practical use -- basically a rudiment, due to disappear eventually Nonsense. I make extensive use of assert as a way of checking assertions, and I will fight tooth and nail against any proposal to either remove it or to misuse it for public input tests instead of assertions. I invite you to show me a single use case for those "assertions" because after ~20 years of experience in coding (that included fairly large
On 28.11.2017 8:59, Steven D'Aprano wrote: projects), I've yet to see one. Any, every check that you make at debug time either * belongs in production as well (all the more because it's harder to diagnose there), or * belongs in a test -- something coded independently from the program (if your code as a whole cannot be trusted, how any specific part of it can?), or * isn't needed at all because a fault will inevitably surface somewhere down the line (as some exception or an incorrect result that a test will catch). Finally, I've got much experience using existing code outside its original use cases, where the original author's assumptions may no longer hold but the specific logic can be gauded to produce the desired result. Coding these assumptions in would undermine that goal. So, I see "debug assertions" as either intentionally compromizing correctness for performance (a direct opposite of Python's design principles), or as an inferiour, faulty, half-measure rudiment from times when CI wasn't a thing (thus not something that should be taught and promoted as a best practice any longer).
* It can instead be reused as syntax sugar to cover a very common use case There is no need for such syntactic sugar. It would be harmful to use assert for something which is not an assertion.
-- Regards, Ivan
Just a note : in typechecked code (such as mypy's source code) assert is used to guide the checker: assert isinstance(x, CallableType) return x.args # checker knows it's valid So the assert becomes a kind of type annotation. The runtime check helps during tests, but is not that important - failure will be caught relatively soon. And I believe that the ability to remove the check at runtime is important, since isinstance calls have non-negligible impact on performance in mypy. (but other contributors here can correct me on this). Elazar בתאריך יום ג׳, 28 בנוב׳ 2017, 09:12, מאת Ivan Pozdeev via Python-ideas < python-ideas@python.org>:
On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas wrote:
Actually, the way I'm using them,
assert condition, "error message", type
would probably be the most expressive way. I disagree that is expressive -- I call it *misleading*. I see something which looks like an assertion (that is, a checked comment, a contract, a check on an internal piece of logic etc) but it is actually being used as a test.
I can do anything in any Turing-complete language without any changes to the language. That's no reason to never change anything, is it. "We can change this" is not a reason to change this. There needs to be a *good* reason to change, and you have given no good reasons for this change.
The rationale basically is: * As it was intended, the statement has no practical use -- basically a rudiment, due to disappear eventually Nonsense. I make extensive use of assert as a way of checking assertions, and I will fight tooth and nail against any proposal to either remove it or to misuse it for public input tests instead of assertions. I invite you to show me a single use case for those "assertions" because after ~20 years of experience in coding (that included fairly large
On 28.11.2017 8:59, Steven D'Aprano wrote: projects), I've yet to see one.
Any, every check that you make at debug time either * belongs in production as well (all the more because it's harder to diagnose there), or * belongs in a test -- something coded independently from the program (if your code as a whole cannot be trusted, how any specific part of it can?), or * isn't needed at all because a fault will inevitably surface somewhere down the line (as some exception or an incorrect result that a test will catch).
Finally, I've got much experience using existing code outside its original use cases, where the original author's assumptions may no longer hold but the specific logic can be gauded to produce the desired result. Coding these assumptions in would undermine that goal.
So, I see "debug assertions" as either intentionally compromizing correctness for performance (a direct opposite of Python's design principles), or as an inferiour, faulty, half-measure rudiment from times when CI wasn't a thing (thus not something that should be taught and promoted as a best practice any longer).
* It can instead be reused as syntax sugar to cover a very common use
case
There is no need for such syntactic sugar. It would be harmful to use assert for something which is not an assertion.
-- Regards, Ivan
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 28.11.2017 10:22, Elazar wrote:
Just a note : in typechecked code (such as mypy's source code) assert is used to guide the checker:
assert isinstance(x, CallableType) return x.args # checker knows it's valid
So the assert becomes a kind of type annotation. The runtime check helps during tests, but is not that important - failure will be caught relatively soon. And I believe that the ability to remove the check at runtime is important, since isinstance calls have non-negligible impact on performance in mypy. (but other contributors here can correct me on this).
This results in two different interfaces. In normal mode, it enforces types while in -O, accepts anything that passes the duck test.
Elazar
בתאריך יום ג׳, 28 בנוב׳ 2017, 09:12, מאת Ivan Pozdeev via Python-ideas <python-ideas@python.org <mailto:python-ideas@python.org>>:
On 28.11.2017 8:59, Steven D'Aprano wrote: > On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas wrote: > >> Actually, the way I'm using them, >> >> assert condition, "error message", type >> >> would probably be the most expressive way. > I disagree that is expressive -- I call it *misleading*. I see something > which looks like an assertion (that is, a checked comment, a contract, a > check on an internal piece of logic etc) but it is actually being used > as a test. > > >> I can do anything in any Turing-complete language without any changes to >> the language. That's no reason to never change anything, is it. > "We can change this" is not a reason to change this. There needs to be a > *good* reason to change, and you have given no good reasons for this > change. > > >> The rationale basically is: >> * As it was intended, the statement has no practical use -- basically a >> rudiment, due to disappear eventually > Nonsense. I make extensive use of assert as a way of checking > assertions, and I will fight tooth and nail against any proposal to > either remove it or to misuse it for public input tests instead of > assertions. I invite you to show me a single use case for those "assertions" because after ~20 years of experience in coding (that included fairly large projects), I've yet to see one.
Any, every check that you make at debug time either * belongs in production as well (all the more because it's harder to diagnose there), or * belongs in a test -- something coded independently from the program (if your code as a whole cannot be trusted, how any specific part of it can?), or * isn't needed at all because a fault will inevitably surface somewhere down the line (as some exception or an incorrect result that a test will catch).
Finally, I've got much experience using existing code outside its original use cases, where the original author's assumptions may no longer hold but the specific logic can be gauded to produce the desired result. Coding these assumptions in would undermine that goal.
So, I see "debug assertions" as either intentionally compromizing correctness for performance (a direct opposite of Python's design principles), or as an inferiour, faulty, half-measure rudiment from times when CI wasn't a thing (thus not something that should be taught and promoted as a best practice any longer). > >> * It can instead be reused as syntax sugar to cover a very common use case > There is no need for such syntactic sugar. It would be harmful > to use assert for something which is not an assertion. > > >
-- Regards, Ivan
_______________________________________________ Python-ideas mailing list Python-ideas@python.org <mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Regards, Ivan
On Tue, Nov 28, 2017 at 10:11:46AM +0300, Ivan Pozdeev via Python-ideas wrote:
I invite you to show me a single use case for those "assertions" because after ~20 years of experience in coding (that included fairly large projects), I've yet to see one.
I already mentioned not one but multiple use-cases for assertions: - checked comments (assertions are documentation, not just code) - checks of internal algorithm logic - design-by-contract style pre- and post-conditions - checking program invariants - defensive programming against conditions that "cannot happen". John Regehr wrote an excellent post on this from the perspective of a systems programmer: https://blog.regehr.org/archives/1091 (he even links to a post by our own Ned Batchelder) but many of his use-cases for assert applies just as well to Python as C. Assertions also work great with fuzzers: http://www.squarefree.com/2014/02/03/fuzzers-love-assertions/ I've also written on assertions before: https://import-that.dreamwidth.org/676.html Assertions can also be used as a kind of "Continuous Testing" that operates in debug builds (or in Python in __debug__ mode): every time you run the code, it tests its own internal state by running the asserts. As Regehr puts it: "When we write an assertion, we are teaching a program to diagnose bugs in itself." If the code you have worked with needed none of these things, assertions might not be useful to you. But that doesn't mean there are no use-cases for assertions. Not everyone is lucky to work with code that is so self-documenting that checked comments are redundant, or code so obviously correct that there is no point to checking its internal state. In your opening post, you gave a use-case for assertions: "a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly." Now you say there are none. Would you like to revise one of those statements?
Any, every check that you make at debug time either * belongs in production as well (all the more because it's harder to diagnose there), or
I'm not going to say that claim is entirely wrong. For example, Microsoft's research project Midori used two kinds of assertions: Debug.Assert only runs under debugging; Release.Assert always runs and it also promised that all contracts will either be checked at runtime, or the compiler can prove that the contract is always satisfied and so can skip the check. Otherwise they cannot be disabled. http://joeduffyblog.com/2016/02/07/the-error-model/ But Midori was written in a custom systems language based on C#, and the lessons from it probably don't apply directly to a rapid application development language / scripting language like Python. In any case, Midori's approach was rather unusual even compared to other systems languages. More conventionally, Eiffel (for example) allows the developer to enable or disable contract checking. By default, pre-conditions are checked in release builds and post-conditions and invariants are skipped, but each one can be enabled or disabled individually. We have the choice to choose faster code or more extensive error checking, depending on which we value more. I think that's an excellent tradeoff to have. Python currently only has a very coarse switch that can only turn assertions on or off, but even that coarse switch is better than nothing.
* belongs in a test -- something coded independently from the program (if your code as a whole cannot be trusted, how any specific part of it can?), or
I love tests. I write unit tests and doc tests all the time. (Well, except when I'm being lazy, when I only write doc tests.) But tests cannot replace assertions. There are at least two problems: - external tests don't have access to the function locals; - and even if you can write a test for something, its in the wrong place to be useful as a replacement of an assertion. John Regehr discusses this exact issue (see link above) and writes that "there is a strong synergy between assertions and unit tests". If you do them right, they complement each other. And I would argue that assertions are a kind of test integrated in the code. Neither assertions nor unit tests can find all bugs. The wise programmer uses both. In an earlier post, I gave an actual assertion from one of my functions. Here it is again: assert 0 <= r < abs(y) Its not obvious from that line in isolation, but both r and y are local variables of a function that calculates a mathematical result (hence the short and undescriptive names). How can I possibly write a test to check this? They are *local* to the function, so I have no access to them from outside of the test! (Actually, in this *specific case*, r is part of the return value and could be extracted by a test function. But y isn't.) In general, assert is great for tests that occur inside the body of a function, checking assertions about the function internals. You cannot replace that with an external test. Here's an assertion from another function: c = collections.Counter(symbols) assert c The Counter c is an internal implementation detail of this one function. It would be silly to expose it (how?) so I can write a test to check it. Even if I could write such a test, that would be the wrong place for the check. I want the check there inside the function, not in a completely different .py file. The assertion is as much for me, the reader of the code, as for the interpreter. After reading that assertion, I can read the rest of the function knowing that it is safe to assume that c will always have at least one key. It is a checked comment: `assert c` is better than: # c will not be empty because the assertion is checked at runtime unless I disable assert checking, while comments are "lies in code" that rapidly become obsolete or inaccurate.
* isn't needed at all because a fault will inevitably surface somewhere down the line (as some exception or an incorrect result that a test will catch).
You don't know that an error will be raised. The program may simply do the wrong thing and silently return garbage that you have no way of knowing is garbage. Nor do you know that a test will catch the problem. Most real world code does not have even close to 100% test coverage -- which is why people are always discovering new bugs. But even if there is an obvious failure later on, or a failing test, it is better to catch errors sooner rather than later. The longer it takes to discover the error, the harder it is to debug. Assertions can reduce the distance between where the bug occurs and where you notice it.
Finally, I've got much experience using existing code outside its original use cases, where the original author's assumptions may no longer hold but the specific logic can be gauded to produce the desired result. Coding these assumptions in would undermine that goal.
Of course assertions can be misused. The same applies to code that defeats duck-typing with excessive isinstance() checks, or people who make everything private and all classes final (in languages that support that). John Regehr also discusses some poor ways to misuse assertions. This is not a good argument for changing assert.
So, I see "debug assertions" as either intentionally compromizing correctness for performance (a direct opposite of Python's design principles),
You have that backwards: assertions compromise performance for correctness. I'm very aware that every time I write an assert, that's a runtime check that compromises performance. But I do so when I believe that the gain in correctness, or the advantage in finding bugs closer to their origin, or even their value as documentation, outweighs the performance cost.
or as an inferiour, faulty, half-measure rudiment from times when CI wasn't a thing (thus not something that should be taught and promoted as a best practice any longer).
If you think that Continuous Integration is a suitable replacement for assertions, then I think you have misunderstood either CI or assertions or both. That's like saying that now that we have CI, we don't need to check the return code on C functions, or catch exceptions. CI and assertions are complementary, not in opposition. CI is a development practice to ensure that the master is always in a working state. That's great. But how do you know the master is working? You need *tests*, and assertions complement tests. Unit tests are not a replacement for assertions. You wouldn't say "unit tests are obsolete now that we have integration tests", or "we don't need fuzzers, we have regression tests". Why would you say that you don't need assertions just because you are using CI? -- Steve
On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
The `assert' statment was created the same as in previous languages like C/C++: a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly.
That's not the sole use for assertions. Assertions are useful as checked comments, as "this can't happen" checks, as checks on your algorithm logic, as a poor man's form of design by contract, and more.
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode".
Apart from -O which disables assertions. But in any case, the best use of assertions is not checking things which the interpreter is going to do anyway, but checking things which the interpreter can not and does not check automatically: your program logic. There is no way that the Python interpreter is going to do this check automatically, unless I write the assertion: assert 0 <= r < abs(y) That is copied straight out of one of my functions. The point is, if you are only using assertions to check for things which the interpreter already does, you're not making good use of assertions.
Furthermore, validation checks are an extremily common use case. I have been using assert in my in-house code for input validation for a long time,
I maintain that using assert for input validation for public functions (even if "public" in this case remains in-house) is certainly an abuse of assert, and STRONGLY oppose any change that supports that bad habit. Its okay to use asserts to validate input to *private* functions and methods (although even that is a slight code smell) but not public functions.
So, I'm hereby proposing to:
* make `assert' stay in optimized mode * change its syntax to raise other types of exceptions
Very strong -1 on this. We already have syntax for raising other sorts of exceptions: raise. if condition: raise Exception(message) is the right way to spell assert not condition, Exception, message The two require virtually the same amount of typing, so you don't even save any effort by using assert with an alternate exception type. Leave assert for assertions. It is a *good* thing that assertions are visually distinct from input error checking. As things stand now, assert is a clear signal to the reader that this is an internal check which may be disabled safely. -- Steve
On 28 November 2017 at 15:41, Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode".
Apart from -O which disables assertions. But in any case, the best use of assertions is not checking things which the interpreter is going to do anyway, but checking things which the interpreter can not and does not check automatically: your program logic. There is no way that the Python interpreter is going to do this check automatically, unless I write the assertion:
assert 0 <= r < abs(y)
That is copied straight out of one of my functions.
I'll make the same observation I usually do each time one of these threads comes up: * I'm opposed to making assert substantially different from the way it works now * I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic The reasons I prefer this approach: - assert is a statement *solely* so that the compiler can optimise it out. If it's not optional, it doesn't need to be a statement any more - if the existing assert statements are left alone, there are no performance or compatibility concerns for folks that rely on the current behaviour - if it's a function, it doesn't need to be called "assert", it can use the more imperative term "ensure" (meaning "ensure this condition is true before continuing") - if it's a function, it can easily be emulated on old versions via compatibility libraries - "ensure() is required, assert is optional" is a better answer to complaints about assertions being optional than suggesting "if cond: raise AssertionError(msg)" as a reasonable alternative to "assert cond, msg" - if it's a function, we get access to all the regular function machinery, so we're not restricted to positional-only arguments the way the assert statement is My initial proposed behaviour for the function: def ensure(cond, msg=None, exc_type=RuntimeError): """Raise an exception if the given condition is not true""" if not cond: if msg is None: frame = sys._getframe(1) line = frame.f_lineno modname = frame.f_globals.get("__name__", "<unknown module>") msg = f"Condition not met on line {line:d} in {modname!r}" raise exc_type(msg) Cheers, Nick. P.S. No, I'm not offering to write that PEP myself, I'd just be in favour of the idea if someone else were to write it :) -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 28 November 2017 at 13:36, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 28 November 2017 at 15:41, Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode".
Apart from -O which disables assertions. But in any case, the best use of assertions is not checking things which the interpreter is going to do anyway, but checking things which the interpreter can not and does not check automatically: your program logic. There is no way that the Python interpreter is going to do this check automatically, unless I write the assertion:
assert 0 <= r < abs(y)
That is copied straight out of one of my functions.
I'll make the same observation I usually do each time one of these threads comes up:
* I'm opposed to making assert substantially different from the way it works now * I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic
The reasons I prefer this approach:
- assert is a statement *solely* so that the compiler can optimise it out. If it's not optional, it doesn't need to be a statement any more - if the existing assert statements are left alone, there are no performance or compatibility concerns for folks that rely on the current behaviour - if it's a function, it doesn't need to be called "assert", it can use the more imperative term "ensure" (meaning "ensure this condition is true before continuing") - if it's a function, it can easily be emulated on old versions via compatibility libraries - "ensure() is required, assert is optional" is a better answer to complaints about assertions being optional than suggesting "if cond: raise AssertionError(msg)" as a reasonable alternative to "assert cond, msg" - if it's a function, we get access to all the regular function machinery, so we're not restricted to positional-only arguments the way the assert statement is
My initial proposed behaviour for the function:
def ensure(cond, msg=None, exc_type=RuntimeError): """Raise an exception if the given condition is not true""" if not cond: if msg is None: frame = sys._getframe(1) line = frame.f_lineno modname = frame.f_globals.get("__name__", "<unknown module>") msg = f"Condition not met on line {line:d} in {modname!r}" raise exc_type(msg)
Cheers, Nick.
P.S. No, I'm not offering to write that PEP myself, I'd just be in favour of the idea if someone else were to write it :)
+1 on everything Nick said. Paul
On 2017-11-28 11:36 AM, Nick Coghlan wrote:
On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Apart from -O which disables assertions. But in any case, the best use of assertions is not checking things which the interpreter is going to do anyway, but checking things which the interpreter can not and does not check automatically: your program logic. There is no way that the Python interpreter is going to do this check automatically, unless I write the assertion:
assert 0 <= r < abs(y)
That is copied straight out of one of my functions. I'll make the same observation I usually do each time one of these
On 28 November 2017 at 15:41, Steven D'Aprano <steve@pearwood.info> wrote: threads comes up:
* I'm opposed to making assert substantially different from the way it works now * I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic
The reasons I prefer this approach:
- assert is a statement *solely* so that the compiler can optimise it out. If it's not optional, it doesn't need to be a statement any more - if the existing assert statements are left alone, there are no performance or compatibility concerns for folks that rely on the current behaviour - if it's a function, it doesn't need to be called "assert", it can use the more imperative term "ensure" (meaning "ensure this condition is true before continuing") - if it's a function, it can easily be emulated on old versions via compatibility libraries - "ensure() is required, assert is optional" is a better answer to complaints about assertions being optional than suggesting "if cond: raise AssertionError(msg)" as a reasonable alternative to "assert cond, msg" - if it's a function, we get access to all the regular function machinery, so we're not restricted to positional-only arguments the way the assert statement is
My initial proposed behaviour for the function:
def ensure(cond, msg=None, exc_type=RuntimeError): """Raise an exception if the given condition is not true""" if not cond: if msg is None: frame = sys._getframe(1) line = frame.f_lineno modname = frame.f_globals.get("__name__", "<unknown module>") msg = f"Condition not met on line {line:d} in {modname!r}" raise exc_type(msg)
Cheers, Nick.
P.S. No, I'm not offering to write that PEP myself, I'd just be in favour of the idea if someone else were to write it :)
I'd prefer (cond, exc_type=RuntimeError, msg=None). For example: ensure(x != 0, ZeroDivisionError) ensure(x > 0, IndexError, "list index out of range") ensure(x != FAIL, msg="failing") I'm not sure how you'd neatly handle no-args constructors and multiple-arguments constructors, tho.
On 28.11.2017 16:36, Nick Coghlan wrote:
On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Apart from -O which disables assertions. But in any case, the best use of assertions is not checking things which the interpreter is going to do anyway, but checking things which the interpreter can not and does not check automatically: your program logic. There is no way that the Python interpreter is going to do this check automatically, unless I write the assertion:
assert 0 <= r < abs(y)
That is copied straight out of one of my functions. I'll make the same observation I usually do each time one of these
On 28 November 2017 at 15:41, Steven D'Aprano <steve@pearwood.info> wrote: threads comes up:
* I'm opposed to making assert substantially different from the way it works now * I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic
The reasons I prefer this approach:
- assert is a statement *solely* so that the compiler can optimise it out. If it's not optional, it doesn't need to be a statement any more Another benefit of a statement vs function is only evaluating the error-related arguments when there's an error - if the existing assert statements are left alone, there are no performance or compatibility concerns for folks that rely on the current behaviour - if it's a function, it doesn't need to be called "assert", it can use the more imperative term "ensure" (meaning "ensure this condition is true before continuing") - if it's a function, it can easily be emulated on old versions via compatibility libraries - "ensure() is required, assert is optional" is a better answer to complaints about assertions being optional than suggesting "if cond: raise AssertionError(msg)" as a reasonable alternative to "assert cond, msg" - if it's a function, we get access to all the regular function machinery, so we're not restricted to positional-only arguments the way the assert statement is
My initial proposed behaviour for the function:
def ensure(cond, msg=None, exc_type=RuntimeError): """Raise an exception if the given condition is not true""" if not cond: if msg is None: frame = sys._getframe(1) line = frame.f_lineno modname = frame.f_globals.get("__name__", "<unknown module>") msg = f"Condition not met on line {line:d} in {modname!r}" raise exc_type(msg)
Cheers, Nick.
P.S. No, I'm not offering to write that PEP myself, I'd just be in favour of the idea if someone else were to write it :)
-- Regards, Ivan
On Tue, Nov 28, 2017 at 6:08 PM Ivan Pozdeev via Python-ideas < python-ideas@python.org> wrote:
On 28.11.2017 16:36, Nick Coghlan wrote:
On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Apart from -O which disables assertions. But in any case, the best use of assertions is not checking things which the interpreter is going to do anyway, but checking things which the interpreter can not and does not check automatically: your program logic. There is no way that the Python interpreter is going to do this check automatically, unless I write the assertion:
assert 0 <= r < abs(y)
That is copied straight out of one of my functions. I'll make the same observation I usually do each time one of these
On 28 November 2017 at 15:41, Steven D'Aprano <steve@pearwood.info> wrote: threads comes up:
* I'm opposed to making assert substantially different from the way it works now * I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic
The reasons I prefer this approach:
- assert is a statement *solely* so that the compiler can optimise it out. If it's not optional, it doesn't need to be a statement any more Another benefit of a statement vs function is only evaluating the error-related arguments when there's an error
I'm not sure what the use case is, but it could be implemented easily as ensure(not hasattr(e, "exception")) or raise e.exception ... had "raise" been an expression, an idea repeatedly rejected here. It's still implementable with a "throw()" function. Elazar
- if the existing assert statements are left alone, there are no performance or compatibility concerns for folks that rely on the current behaviour - if it's a function, it doesn't need to be called "assert", it can use the more imperative term "ensure" (meaning "ensure this condition is true before continuing") - if it's a function, it can easily be emulated on old versions via compatibility libraries - "ensure() is required, assert is optional" is a better answer to complaints about assertions being optional than suggesting "if cond: raise AssertionError(msg)" as a reasonable alternative to "assert cond, msg" - if it's a function, we get access to all the regular function machinery, so we're not restricted to positional-only arguments the way the assert statement is
My initial proposed behaviour for the function:
def ensure(cond, msg=None, exc_type=RuntimeError): """Raise an exception if the given condition is not true""" if not cond: if msg is None: frame = sys._getframe(1) line = frame.f_lineno modname = frame.f_globals.get("__name__", "<unknown module>") msg = f"Condition not met on line {line:d} in {modname!r}" raise exc_type(msg)
Cheers, Nick.
P.S. No, I'm not offering to write that PEP myself, I'd just be in favour of the idea if someone else were to write it :)
-- Regards, Ivan
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 11/28/2017 08:03 AM, Ivan Pozdeev via Python-ideas wrote:
On 28.11.2017 16:36, Nick Coghlan wrote:
it doesn't need to be a statement any more
Another benefit of a statement vs function is only evaluating the error-related arguments when there's an error
The bulk of any processing in assert (or ensure()) should be the actual check -- if that fails, only state information should be included in the exception as anything more complicated runs the risk of also being wrong as the code is now in a failed state. In other words, the "error-related arguments" will often be extremely cheap compared to the test itself. -- ~Ethan~
On 28.11.2017 20:23, Ethan Furman wrote
On 11/28/2017 08:03 AM, Ivan Pozdeev via Python-ideas wrote:
On 28.11.2017 16:36, Nick Coghlan wrote:
it doesn't need to be a statement any more
Another benefit of a statement vs function is only evaluating the error-related arguments when there's an error
The bulk of any processing in assert (or ensure()) should be the actual check -- if that fails, only state information should be included in the exception as anything more complicated runs the risk of also being wrong as the code is now in a failed state. In other words, the "error-related arguments" will often be extremely cheap compared to the test itself.
My experience is the contrary. The check is usually trivial -- a type check or comparison. While a useful error message contains problem details, so it incorporates string formatting from a variable or two or expressions thereof, like "expected <type>, got <type(var)>".
-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Regards, Ivan
On 2017-11-28 04:27 PM, Ivan Pozdeev via Python-ideas wrote:
On 28.11.2017 20:23, Ethan Furman wrote
On 11/28/2017 08:03 AM, Ivan Pozdeev via Python-ideas wrote:
On 28.11.2017 16:36, Nick Coghlan wrote:
it doesn't need to be a statement any more
Another benefit of a statement vs function is only evaluating the error-related arguments when there's an error
The bulk of any processing in assert (or ensure()) should be the actual check -- if that fails, only state information should be included in the exception as anything more complicated runs the risk of also being wrong as the code is now in a failed state. In other words, the "error-related arguments" will often be extremely cheap compared to the test itself.
My experience is the contrary. The check is usually trivial -- a type check or comparison. While a useful error message contains problem details, so it incorporates string formatting from a variable or two or expressions thereof, like "expected <type>, got <type(var)>".
ensure(check, lambda: raise TypeError("expected <type>, got <type(var)>"))?
-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 29 November 2017 at 02:03, Ivan Pozdeev via Python-ideas <python-ideas@python.org> wrote:
On 28.11.2017 16:36, Nick Coghlan wrote:
I'll make the same observation I usually do each time one of these threads comes up:
* I'm opposed to making assert substantially different from the way it works now * I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic
The reasons I prefer this approach:
- assert is a statement *solely* so that the compiler can optimise it out. If it's not optional, it doesn't need to be a statement any more
Another benefit of a statement vs function is only evaluating the error-related arguments when there's an error
If you're worried about that level of micro-optimisation, it's straightforward enough to write your own wrapper function that accepts the components needed to build a suitably formatted message. There's no need to make the builtin more complicated to cope with it (that smooth transition from the builtin behaviour to customised behaviour is one of the other advantages of using a plain function). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
28.11.17 15:36, Nick Coghlan пише:
* I'm opposed to making assert substantially different from the way it works now
If sometimes we will change assert I would prefer to make it more similar to assert in pytest. Output values of the final and intermediate expressions. The hardest problem -- the repr can be very long and expensive, we need an alternate protocol for producing a shortened representation.
* I'm in favour of adding a new "ensure()" builtin that encapsulates the check-and-raise logic
Then may be add a new "die()" buildin? ;-) 0 <= r < abs(y) or die()
Ivan Pozdeev via Python-ideas writes:
The `assert' statment was created the same as in previous languages like C/C++: a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly. Examples are array bounds and object state integrity constraints.
I use assert in Python as I use it in C: as a statement that something should NEVER happen, algorithmically. I do occasionally have expensive assertions in inner loops, and do use -O to disable them when I assess that speed is important enough to offset the increased risk. So, -1. Oh, yeah: I have had programs stop because I got the algorithm wrong, and because I implemented it incorrectly. I understand the point made elsewhere about scientists' thinking about "assert", but as a scientist myself, I disagree. It is very useful to me to distinguish between validating data and validating algorithms. "assert" is how we do the latter.
On Tue, Nov 28, 2017 at 05:54:12PM +0900, Stephen J. Turnbull wrote:
I understand the point made elsewhere about scientists' thinking about "assert", but as a scientist myself, I disagree. It is very useful to me to distinguish between validating data and validating algorithms. "assert" is how we do the latter.
I'm not sure which point about scientists you are referring to -- I don't seem to have that email. Is that in this thread? -- Steve
On 11/28/2017 03:30 AM, Steven D'Aprano wrote:
On Tue, Nov 28, 2017 at 05:54:12PM +0900, Stephen J. Turnbull wrote:
I understand the point made elsewhere about scientists' thinking about "assert", but as a scientist myself, I disagree. It is very useful to me to distinguish between validating data and validating algorithms. "assert" is how we do the latter.
I'm not sure which point about scientists you are referring to -- I don't seem to have that email. Is that in this thread?
Privately forwarded. (Yes, it is in this thread.) -- ~Ethan~
participants (17)
-
Barry Warsaw -
Chris Angelico -
Elazar -
Ethan Furman -
Ivan Pozdeev -
Jason Maldonis -
M.-A. Lemburg -
Nathan Schneider -
Ned Batchelder -
Nick Coghlan -
Paul Moore -
Petr Viktorin -
Serhiy Storchaka -
smarie -
Soni L. -
Stephen J. Turnbull -
Steven D'Aprano