[Python-ideas] Runtime assertion with no overhead when not active

[Eloi Gaudry]

On Sun, 2018-05-06 at 01:30 +1000, Steven D'Aprano wrote:
> On Sat, May 05, 2018 at 08:04:45AM +0000, Eloi Gaudry wrote:
> 
> > Hi folks,
> > I intend to add a runtime assertion feature in python.
> 
> I'm very interested in this idea, but I'm afraid your draft PEP
> isn't 
> very clear about this feature. Comments below.
Thanks, I'll try to clarify this here.

> > Rationale
> > There is no runtime assert relying on python grammar available.
> > For 
> > diagnostics and/or debugging reasons, it would be valuable to add
> > such 
> > a feature.
> 
> Why not this?
> 
>     if DEBUG_ENABLED:
>         run_check(expression)
> 
> where run_check is an ordinary function. If DEBUG_ENABLED is false,
> the 
> only runtime overhead is the name lookup and bool test, the
> expression 
> is not evaluated.
for the sake of clarity across any Python application/extension for
instance, and because having run_check defined as a statement seems
actually the good thing to do.
> 
> What features does your suggestion offer over this simple technique?
Code readiness I think (kiss) : simpler and shorter.
The simple technique would still be used for other purpose (assert that
are context based for instance).


> 
> [...]
> > A brief example why avoiding evaluating the expression is needed
> > to 
> > avoid any overhead in case the runtime assert should be ignored.
> > ::
> > runtime_assert( 999 in { i:None for i in range( 10000000 ) } )
> 
> You say that you have been using this feature in production for two 
> years. You should give real-life examples, not fake examples like
> the 
> above. That example would be better written as:
> 
>     runtime_assert(True)
> 
> since no part of the test actually depends on either the enormous
> dict 
> or range objects that you build only to throw away.
Real life examples would be more application-based I think.

In my case, I used this for switching off some heaving computing
expression evaluation at runtime. Those expression would have been
responsible for sorting vectors and communicating them through MPI
using some method from our framework, in order to finally return a
boolean.


> > Usage
> > ::
> > 
> > runtime_assert( expr )
> > 
> > #would result in 
> > if expr and runtime_assert_active:
> >     print RuntimeAssertionError()
> 
> Surely the test should be the other way around?
> 
>     if runtime_assert_active and expr:
>         print(RuntimeAssertionError())
otherwise the expression is evaluated regardless of whether the
runtime 
> assertions are active or not.
You are absolutely right :)

> Please use Python 3 syntax for the PEP, as Python 2 is in
> maintenance 
> mode and will absolutely not get any new language features.
Sure

> 
> Some more observations:
> 
> I like the idea of more control over runtime checking, but your PEP 
> should justify why this is a good idea and why "just write more unit 
> tests" isn't the solution.
My main reasons is the one I gave in the context description of the
PEP. Assert remains an efficient way for controlling your application
behavior when running in production (as opposed to a debug build).

Writing more unit tests is always a good practice, but :
- it does not necessarily cover all possible paths that are runtime
dependent (an application relies on various app-parameters and
architecture-parameters for instance)
- it does not offer a solution for your extension/application consumers
(that might be using your application in a very specific set of
paths/set of decision/etc.). 

> What happens if the caller has defined their own function or
> variable 
> called runtime_assert?
making runtime_assert a statement (as assert is already) would forbid
such a definition, thus it would break backward compatibility (in those
specific cases) for sure.


> Can the caller customise the assertion error message?
At this stage, no, but I don't know if this would be something
difficult to add.

> If you are changing the grammar, why not make runtime_assert a
> statement?
I'm not really changing the grammar, just making 'runtime_assert' a
statement as 'assert' already is (and IMHO could have been named as
debug_assert). There is no significant difference between both assert
in term of implementation.

> Can the caller set an application-wide global across all modules, or
> do 
> they have to set
> 
>     runtime_assert_active = True
> 
> in every module they want to use runtime_assert?
I would vote in favor of having two different ways to activate or not
the runtime assertion evaluation:
- the first one would consist of controlling a global variable at the
C-level (the current recipe in cPython) and would be used by python
extension. This is the solution I am using personnaly.
- the second would use a set method available in pure python (not
implemented, to be done at C-level too I think) to easily (de)activate
all the  runtime asserts.