Grr, Google Groups gateway messes up reply-to. Apologies to anyone who gets a double-post, please can posters ensure that reply-to is set to the list, and *not* to the Google Groups gateway? Thanks. Paul On 16 January 2018 at 15:54, Paul Moore
On 16 January 2018 at 15:37, smarie
wrote: If you, the developer, don't want a check to be disabled, then you shouldn't call it an assertion and use assert.
That is exactly what I'm saying. It seems that we both agree that applicative value validation is different from asserts, and that assert should not be used for applicative value validation. For this reason, I do not suggest to go in the direction the OP is mentioning but rather to explicitly separate the 2 concepts by creating a new statement for value validation.
The problem with a statement called "validate" is that it will break a huge number of programs that already include functions and methods using that name.
You definitely make a point here. But that would be the case for absolutely *any* language evolution as soon as the proposed statements are plain old english words. Should it be a show-stopper ? I dont think so.
Why does this need to be a statement at all? Unlike assert, it's always executed, so it can be defined as a simple function:
def validate(test, message): if not test: raise ValidartionError(message)
But apart from the use of a keyword, we already have a way to do almost exactly what you want:
if not expression: raise ValidationError(message)
after defining some appropriate ValidationError class. And it is only a few key presses longer than the proposed:
validate expression, ValidationError, message
This is precisely what is not good in my opinion: here you do not separate <validation means> from <validation intent>. Of course if <validation means> is just a "x > 0" statement, it works, but now what if you rely on a 3d-party provided validation function (or even yours) such as e.g. "is_foo_compliant" ?
if not is_foo_compliant(x): raise ValidationError(message)
What if this third part method raises an exception instead of returning False in some cases ?
try: if not is_foo_compliant(x): raise ValidationError(message) except: raise MyValidationError(message)
What if you want to compose this third party function with *another* one that returns False but not exceptions ? Say, with an OR ? (one or the other should work). Yields:
try: if not is_foo_compliant(x): raise ValidationError(message) except: if not is_bar_compliant(x): raise MyValidationError(message)
It starts to be quite ugly, messy... while the applicative intent is clear and could be expressed directly as:
validate is_foo_compliant(x) or is_bar_compliant(x) ValidationError(message)
I don't see how a validate statement avoids having to deal with all of the complexity you mention here. And it's *far* easier to handle this as a standalone function - if you find a new requirement like the ones you suggest above, you simply modify the function (and release an updated version of your package, if you choose to release your code on PyPI) and you're done. With a new statement, you'd need to raise a Python feature request, wait for at least the next Python release to see the modification, and *still* have to support people on older versions of Python with the unfixed version.
Also, a validate() function will wor on older versions of Python, all the way back to Python 2.7 if you want.
Paul
Thanks Paul Le mardi 16 janvier 2018 16:56:57 UTC+1, Paul Moore a écrit :
Why does this need to be a statement at all? Unlike assert, it's always executed, so it can be defined as a simple function
Very good point. Actually that's what I already provide in valid8 with the assert_valid
function. See all examples here
https://smarie.github.io/python-valid8/#usage-examples-inline.
Let's consider this example where users want to define on-the-fly one of
the validation functions, and combine it with another with a 'or':
assert_valid('surface', surf, or_(lambda x: (x >= 0) & (x < 10000), is_foo_compliant), help_msg="surface should be 0= I don't see how a validate statement avoids having to deal with all of the complexity you mention here. It would obviously need to be quite smart :) but it is possible, since I
can do it with assert_valid today.
I fixed the reply-to this time, looks like you're still getting messed
up by Google Groups.
On 16 January 2018 at 16:25, smarie
Let's consider this example where users want to define on-the-fly one of the validation functions, and combine it with another with a 'or':
assert_valid('surface', surf, or_(lambda x: (x >= 0) & (x < 10000), is_foo_compliant), help_msg="surface should be 0=
How ugly for something so simple ! I tried to make it slightly more compact by developping a mini lambda syntax but it obviously makes it slower.
Why do you do this? What's the requirement for delaying evaluation of
the condition? A validate statement in Python wouldn't be any better
able to do that, so it'd be just as ugly with a statement. There's no
reason I can see why I'd ever need delayed evaluation, so what's wrong
with just
assert_valid(0 <= surf < 10000 and is_foo_compliant(surf),
help_msg="surface should be 0= There are three reasons why having a 'validate' statement would improve
this: * no more parenthesis: more elegant and readable
* inline use of python (1): no more use of lambda or mini_lambda, no
performance overhead
* inline use of python (2): composition would not require custom function
composition operators such as 'or_' (above) or mini-lambda composition
anymore, it could be built-in in any language element used after <validate> resulting in validate (surf >= 0) & (surf < 10000) or is_foo_compliant(surf),
"surface should be 0= So how will that work any differently than the function version I gave
above? In terms of the language syntax, it would just be
validate EXPR, EXPR
and the first EXPR will be evaluated as a boolean, and if it's false
an exception will be raised with the second expression as the message.
There's no delayed evaluation involved, so a function would work
exactly the same. (I removed the variable name alias 'surface' since I don't know if it should
remain or not) Elegant, isn't it ? No more so than my function version, but yes far more so than yours...
Paul
(trying with direct reply this time)
Why do you do this? What's the requirement for delaying evaluation of the condition?
Thanks for challenging my poorly chosen examples :)
The primary requirement is about *catching* unwanted/uncontrolled/heterogenous exceptions happening in the underlying functions that are combined together to provide the validation means, so as to provide a uniform/consistent outcome however diverse the underlying functions are (they can return booleans or raise exceptions, or both).
In your proposal, if 'is_foo_compliant' raises an exception, it will not be caught by 'assert_valid', therefore the ValidationError will not be raised. So this is not what I want as an application developer.
--
Sylvain
-----Message d'origine-----
De : Paul Moore [mailto:p.f.moore@gmail.com]
Envoyé : mardi 16 janvier 2018 18:01
À : Sylvain MARIE
Let's consider this example where users want to define on-the-fly one of the validation functions, and combine it with another with a 'or':
assert_valid('surface', surf, or_(lambda x: (x >= 0) & (x < 10000), is_foo_compliant), help_msg="surface should be 0=
How ugly for something so simple ! I tried to make it slightly more compact by developping a mini lambda syntax but it obviously makes it slower.
Why do you do this? What's the requirement for delaying evaluation of the condition? A validate statement in Python wouldn't be any better able to do that, so it'd be just as ugly with a statement. There's no reason I can see why I'd ever need delayed evaluation, so what's wrong with just
assert_valid(0 <= surf < 10000 and is_foo_compliant(surf), help_msg="surface should be 0= There are three reasons why having a 'validate' statement would
improve
this: * no more parenthesis: more elegant and readable
* inline use of python (1): no more use of lambda or mini_lambda, no
performance overhead
* inline use of python (2): composition would not require custom
function composition operators such as 'or_' (above) or mini-lambda
composition anymore, it could be built-in in any language element used
after <validate> resulting in validate (surf >= 0) & (surf < 10000) or
is_foo_compliant(surf), "surface should be 0= So how will that work any differently than the function version I gave above? In terms of the language syntax, it would just be
validate EXPR, EXPR
and the first EXPR will be evaluated as a boolean, and if it's false an exception will be raised with the second expression as the message.
There's no delayed evaluation involved, so a function would work exactly the same. (I removed the variable name alias 'surface' since I don't know if it
should remain or not) Elegant, isn't it ? No more so than my function version, but yes far more so than yours...
Paul
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________
On 16 January 2018 at 17:36, Sylvain MARIE
(trying with direct reply this time)
Why do you do this? What's the requirement for delaying evaluation of the condition?
Thanks for challenging my poorly chosen examples :)
The primary requirement is about *catching* unwanted/uncontrolled/heterogenous exceptions happening in the underlying functions that are combined together to provide the validation means, so as to provide a uniform/consistent outcome however diverse the underlying functions are (they can return booleans or raise exceptions, or both).
In your proposal, if 'is_foo_compliant' raises an exception, it will not be caught by 'assert_valid', therefore the ValidationError will not be raised. So this is not what I want as an application developer.
Ah, OK. But nothing in your proposal for a new statement suggests you
wanted that, and assert doesn't work like that, so I hadn't realised
that's what you were after.
You could of course simply do:
def assert_valid(expr, help_msg):
# Catch exceptions in expr() as you see fit
if not expr():
raise ValidationError(help_msg)
assert_valid(lambda: 0 <= surf < 10000 and is_foo_compliant(surf),
help_msg="surface should be 0=
Perhaps the OP can look into Python macro libraries to get the wanted syntax? https://github.com/lihaoyi/macropy On Tue, Jan 16, 2018 at 2:35 PM, Paul Moore
(trying with direct reply this time)
Why do you do this? What's the requirement for delaying evaluation of
On 16 January 2018 at 17:36, Sylvain MARIE
wrote: the condition? Thanks for challenging my poorly chosen examples :)
The primary requirement is about *catching* unwanted/uncontrolled/heterogenous
exceptions happening in the underlying functions that are combined together to provide the validation means, so as to provide a uniform/consistent outcome however diverse the underlying functions are (they can return booleans or raise exceptions, or both).
In your proposal, if 'is_foo_compliant' raises an exception, it will not
be caught by 'assert_valid', therefore the ValidationError will not be raised. So this is not what I want as an application developer.
Ah, OK. But nothing in your proposal for a new statement suggests you wanted that, and assert doesn't work like that, so I hadn't realised that's what you were after.
You could of course simply do:
def assert_valid(expr, help_msg): # Catch exceptions in expr() as you see fit if not expr(): raise ValidationError(help_msg)
assert_valid(lambda: 0 <= surf < 10000 and is_foo_compliant(surf), help_msg="surface should be 0=
No need for a whole expression language :-)
Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Juancarlo *Añez*
I think having a means for such validations separate from assertions would
be helpful.
However, I agree with Steven that 'validate' would be a bad keyword choice.
Besides breaking compatibility with programs that use 'validate', it would
break
wsgiref.validate
https://docs.python.org/3.6/library/wsgiref.html#module-wsgiref.validate
in the standard library.
ᐧ
On Tue, Jan 16, 2018 at 2:22 PM, Juancarlo Añez
Perhaps the OP can look into Python macro libraries to get the wanted syntax?
https://github.com/lihaoyi/macropy
On Tue, Jan 16, 2018 at 2:35 PM, Paul Moore
wrote: (trying with direct reply this time)
Why do you do this? What's the requirement for delaying evaluation of
On 16 January 2018 at 17:36, Sylvain MARIE
wrote: the condition? Thanks for challenging my poorly chosen examples :)
The primary requirement is about *catching*
unwanted/uncontrolled/heterogenous exceptions happening in the underlying functions that are combined together to provide the validation means, so as to provide a uniform/consistent outcome however diverse the underlying functions are (they can return booleans or raise exceptions, or both).
In your proposal, if 'is_foo_compliant' raises an exception, it will
not be caught by 'assert_valid', therefore the ValidationError will not be raised. So this is not what I want as an application developer.
Ah, OK. But nothing in your proposal for a new statement suggests you wanted that, and assert doesn't work like that, so I hadn't realised that's what you were after.
You could of course simply do:
def assert_valid(expr, help_msg): # Catch exceptions in expr() as you see fit if not expr(): raise ValidationError(help_msg)
assert_valid(lambda: 0 <= surf < 10000 and is_foo_compliant(surf), help_msg="surface should be 0=
No need for a whole expression language :-)
Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Juancarlo *Añez*
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Jan 17, 2018 at 12:19:51PM -0500, Nikolas Vanderhoof wrote:
I think having a means for such validations separate from assertions would be helpful.
What semantics would this "validate" statement have, and how would it be different from what we can write now? if not condition: raise SomeException(message) validate condition, SomeException, message # or some other name Unless it does something better than a simple "if ... raise", there's not much point in adding a keyword just to save a few keystrokes. To justify a keyword, it needs to do something special that a built-in function can't do, like delayed evaluation (without wrapping the expression in a function). -- Steve
On 18 January 2018 at 07:46, Steven D'Aprano
To justify a keyword, it needs to do something special that a built-in function can't do, like delayed evaluation (without wrapping the expression in a function).
My reaction to these threads for a while has been "We should just add a function for unconditional assertions in expression form", and I finally got around to posting that to the issue tracker rather than leaving it solely in mailing list posts: https://bugs.python.org/issue32590 The gist of the idea is to add a new ensure() builtin along the lines of: class ValidationError(AssertionError): pass _MISSING = object() def ensure(condition, msg=_MISSING, exc_type=ValidationError): if not condition: if msg is _MISSING: msg = condition raise exc_type(msg) There's no need to involve the compiler if you're never going to optimise the code out, and code-rewriters like the one in pytest can be taught to recognise "ensure(condition)" as being comparable to an assert statement. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 17/01/2018 17:19, Nikolas Vanderhoof wrote:
I think having a means for such validations separate from assertions would be helpful. However, I agree with Steven that 'validate' would be a bad keyword choice. Besides breaking compatibility with programs that use 'validate', it would break wsgiref.validate https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.python.org%2F3.6%2Flibrary%2Fwsgiref.html%23module-wsgiref.validate&data=02%7C01%7C%7Caa26210dce644b49a87508d55dcebb23%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636518064831624779&sdata=Q9x3Boch1FnCgVnd4SNmR35b0g%2BPHszNJYHrDol2%2BKk%3D&reserved=0 in the standard library.
ᐧ
To me it looks like this discussion has basically split into two separate use cases: 1. Using assert in a way that it will not (ever) get turned off. 2. The specific case of ensuring that a variable/parameter is an instance of a specific type. and I would like to suggest two separate possibly syntaxes that might make sense. 1. For asserts that should not be disabled we could have an always qualifier optionally added to assert, either as "assert condition exception always" or "assert always condition exception", that disables the optimisation for that specific exception. This would make it clearer that the developer needs this specific check always. Alternatively, we could consider a scoped flag, say keep_asserts, that sets the same. 2. For the specific, and to me more desirable, use case of ensuring type compliance how about an ensure keyword, (or possibly function), with a syntax of "ensure var type" or "ensure(var, type)" which goes a little further by attempting to convert the type of var to type and only if var cannot be converted raises a type exception. This second syntax could, of course, be implemented as a library function rather than a change to python itself. Either option could have an optional exception to raise, with the default being a type error. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.
On Thu, Jan 18, 2018 at 4:21 PM, Steve Barnes
1. For asserts that should not be disabled we could have an always qualifier optionally added to assert, either as "assert condition exception always" or "assert always condition exception", that disables the optimisation for that specific exception. This would make it clearer that the developer needs this specific check always. Alternatively, we could consider a scoped flag, say keep_asserts, that sets the same.
But if they're never to be compiled out, why do they need special syntax? assert always x >= 0, "x must be positive" can become if x < 0: raise ValueError("x must be positive") I haven't yet seen any justification for syntax here. The nearest I've seen is that this "ensure" action is more like: try: cond = x >= 0 except BaseException: raise AssertionError("x must be positive") else: if not cond: raise AssertionError("x must be positive") Which, IMO, is a bad idea, and I'm not sure anyone was actually advocating it anyway. ChrisA
On Thu, Jan 18, 2018 at 05:22:06PM +1100, Chris Angelico wrote:
I haven't yet seen any justification for syntax here. The nearest I've seen is that this "ensure" action is more like:
try: cond = x >= 0 except BaseException: raise AssertionError("x must be positive") else: if not cond: raise AssertionError("x must be positive")
Which, IMO, is a bad idea, and I'm not sure anyone was actually advocating it anyway.
My understanding is that Sylvain was advocating for that. -- Steve
On 01/17/2018 10:59 PM, Steven D'Aprano wrote:
On Thu, Jan 18, 2018 at 05:22:06PM +1100, Chris Angelico wrote:
I haven't yet seen any justification for syntax here. The nearest I've seen is that this "ensure" action is more like:
try: cond = x >= 0 except BaseException: raise AssertionError("x must be positive") else: if not cond: raise AssertionError("x must be positive")
Which, IMO, is a bad idea, and I'm not sure anyone was actually advocating it anyway.
My understanding is that Sylvain was advocating for that.
Agreed. Which, as has been pointed out, is an incredibly bad idea. -- ~Ethan~
I sometimes wish that Python included a richer set of assertions rather than just a single `assert` keyword. Something like Eiffel's concept of pre-conditions, post-conditions and invariants, where each can be enabled or disabled independently.
Has something like this been proposed for Python before?
This seems to align more with the intended use of assert that's been
pointed out in this thread.
In what case though would one want to disable some but not all of these
pre, post, or invariant assertions?
On Thu, Jan 18, 2018 at 2:12 AM, Ethan Furman
On 01/17/2018 10:59 PM, Steven D'Aprano wrote:
On Thu, Jan 18, 2018 at 05:22:06PM +1100, Chris Angelico wrote:
I haven't yet seen any justification for syntax here. The nearest I've
seen is that this "ensure" action is more like:
try: cond = x >= 0 except BaseException: raise AssertionError("x must be positive") else: if not cond: raise AssertionError("x must be positive")
Which, IMO, is a bad idea, and I'm not sure anyone was actually advocating it anyway.
My understanding is that Sylvain was advocating for that.
Agreed. Which, as has been pointed out, is an incredibly bad idea.
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
ᐧ
On Thu, Jan 18, 2018 at 7:51 PM, Nikolas Vanderhoof < nikolasrvanderhoof@gmail.com> wrote:
I sometimes wish that Python included a richer set of assertions rather
than just a single `assert` keyword. Something like Eiffel's concept of pre-conditions, post-conditions and invariants, where each can be enabled or disabled independently.
Has something like this been proposed for Python before? This seems to align more with the intended use of assert that's been pointed out in this thread. In what case though would one want to disable some but not all of these pre, post, or invariant assertions?
Oh, many times, starting in the late '90s IIRC (Paul Dubois was a big fan). The problems are twofold: (a) it would require a lot of new keywords or ugly syntax; and (b) there would have to be a way to enable each form separately *per module or package*. Eiffel solves that (AFAIC) through separate compilation -- e.g. a stable version of a library might disable invariants and post-conditions but keep pre-conditions, since those could be violated by less mature application code; or a mature application could disable all checks and link with optimized library binaries that also have disabled all checks. I'm sure other scenarios are also viable. But that solution isn't available in Python, where command line flags apply to *all* modules being imported. (Note: even if you have a solution for (b), getting past (a) isn't so easy. So don't get nerd-sniped by the solution for (b) alone.) -- --Guido van Rossum (python.org/~guido)
Thank you for your explanation!
ᐧ
On Thu, Jan 18, 2018 at 11:00 PM, Guido van Rossum
On Thu, Jan 18, 2018 at 7:51 PM, Nikolas Vanderhoof < nikolasrvanderhoof@gmail.com> wrote:
I sometimes wish that Python included a richer set of assertions rather
than just a single `assert` keyword. Something like Eiffel's concept of pre-conditions, post-conditions and invariants, where each can be enabled or disabled independently.
Has something like this been proposed for Python before? This seems to align more with the intended use of assert that's been pointed out in this thread. In what case though would one want to disable some but not all of these pre, post, or invariant assertions?
Oh, many times, starting in the late '90s IIRC (Paul Dubois was a big fan).
The problems are twofold: (a) it would require a lot of new keywords or ugly syntax; and (b) there would have to be a way to enable each form separately *per module or package*. Eiffel solves that (AFAIC) through separate compilation -- e.g. a stable version of a library might disable invariants and post-conditions but keep pre-conditions, since those could be violated by less mature application code; or a mature application could disable all checks and link with optimized library binaries that also have disabled all checks. I'm sure other scenarios are also viable.
But that solution isn't available in Python, where command line flags apply to *all* modules being imported.
(Note: even if you have a solution for (b), getting past (a) isn't so easy. So don't get nerd-sniped by the solution for (b) alone.)
-- --Guido van Rossum (python.org/~guido)
On Thu, Jan 18, 2018 at 08:00:01PM -0800, Guido van Rossum wrote:
On Thu, Jan 18, 2018 at 7:51 PM, Nikolas Vanderhoof < nikolasrvanderhoof@gmail.com> wrote:
I sometimes wish that Python included a richer set of assertions rather
than just a single `assert` keyword. Something like Eiffel's concept of pre-conditions, post-conditions and invariants, where each can be enabled or disabled independently.
Has something like this been proposed for Python before? This seems to align more with the intended use of assert that's been pointed out in this thread. In what case though would one want to disable some but not all of these pre, post, or invariant assertions?
Oh, many times, starting in the late '90s IIRC (Paul Dubois was a big fan).
The problems are twofold: (a) it would require a lot of new keywords or ugly syntax; and (b) there would have to be a way to enable each form separately *per module or package*. Eiffel solves that (AFAIC) through separate compilation -- e.g. a stable version of a library might disable invariants and post-conditions but keep pre-conditions, since those could be violated by less mature application code; or a mature application could disable all checks and link with optimized library binaries that also have disabled all checks. I'm sure other scenarios are also viable.
But that solution isn't available in Python, where command line flags apply to *all* modules being imported.
(Note: even if you have a solution for (b), getting past (a) isn't so easy. So don't get nerd-sniped by the solution for (b) alone.)
Indeed. I fear that Python's design will never be a good match for Eiffel's Design By Contract. Nevertheless I still have hope that there could be things we can learn from it. After all, DBC is as much a state of mind as it is syntax. Here's a blast from the past: https://www.python.org/doc/essays/metaclasses/ http://legacy.python.org/doc/essays/metaclasses/Eiffel.py This was my first introduction to the idea of software contracts! -- Steve
I haven't yet seen any justification for syntax here. The nearest I've seen is that this "ensure" action is more like:
try: cond = x >= 0 except BaseException: raise AssertionError("x must be positive") else: if not cond: raise AssertionError("x must be positive")
Which, IMO, is a bad idea, and I'm not sure anyone was actually advocating it anyway.
ChrisA
Indeed, I was the one advocating for it :) Based on all the feedback I received from this discussion, I realized that my implementation was completely flawed by the fact that I had done the class and functions decorators first, and wanted to apply the same pattern to the inline validator, resulting in this assert_valid with overkill delayed evaluation. Resulting in me saying that the only way out would be a new python language element. I tried my best to update valid8 and reached a new stable point with version 3.0.0, providing 2 main utilities for inline validation: - the simple but not so powerful `quick_valid` function - the more verbose (2 lines) but much more generic `wrap_valid` context manager (that's the best I can do today !) The more capable but delayed-evaluation based `assert_valid` is not recommended anymore, or just a tool to replicate what is done in the function and class validation decorators. Like the decorators, it adds the ability to blend two styles of base functions (boolean testers and failure raisers) with boolean operators seamlessly. But the complexity is not worth it for inline validation (it seems to be worth it for decorators). See https://smarie.github.io/python-valid8 for the new updated documentation. I also updated the problem description page at https://smarie.github.io/python-valid8/why_validation/ so as to keep a reference of the problem description and "wishlist" (whether it is implemented by this library or by new language elements in the future). Do not hesitate to contribute or send me your edits (off-list). I would love to get feedback from anyone concerning this library, whether you consider it's useless or "interesting but...". We should probably take this offline though, so as not to pollute the initial thread. Thanks again, a great weekend to all (end of the day here in france ;) ) Kind regards Sylvain
participants (11)
-
Chris Angelico
-
Ethan Furman
-
Guido van Rossum
-
Juancarlo Añez
-
Nick Coghlan
-
Nikolas Vanderhoof
-
Paul Moore
-
smarie
-
Steve Barnes
-
Steven D'Aprano
-
Sylvain MARIE