Re: [Python-ideas] Defer Statement

I agree that the stated use cases are better handled with ExitStack. One area where `defer` might be useful is in lazy-evaluating global constants. For example in a genomics library used at my work, one module involves compiling a large number of regular expressions, and setting them as global contants in the module, like: FOO1_re = re.compile(r'...') FOO_TO_BAR_re = {foo: complicated_computation_of_regex(foo) for foo in LONG_LIST_OF_THINGS} ... This utility module is imported in a lot of places in the codebase, which meant that importing almost anything from our codebase involved precompiling all these regular expressions, which took around 500ms to to run the anything (the test runner, manually testing code in the shell, etc.) It would be ideal to only do these computations if/when they are needed. This is a more general issue than this specific example, e.g. for libraries which parse large data sources like pycountry (see my PR https://bitbucket.org/flyingcircus/pycountry/pull-requests/10/improve-module... for one possible ugly solution using proxy objects; the author instead went with the simpler, less general solution of manually deciding when the data is needed). See also django.utils.functional.lazy https://docs.djangoproject.com/en/1.11/_modules/django/utils/functional/, which is used extensively in the framework. A statement like: `defer FOO = lengthy_computation_of_foo()` which deferred the lengthy computation until it is used for something would be useful to allow easily fixing these issues without writing ugly hacks like proxy objects or refactoring code into high-overhead cached properties or the like. Alternatively, if there are less ugly or bespoke ways to handle this kind of issue, I'd be interested in hearing them. Best, Lucas On Sat, Jun 3, 2017 at 11:38 PM, Nick Coghlan wrote:

On 3 June 2017 at 22:24, Nick Coghlan wrote:

...

So while I'm definitely sympathetic to the use case (otherwise ExitStack wouldn't have a callback() method), "this would be useful" isn't a sufficient argument in this particular case - what's needed is a justification that this pattern of resource management is common enough to justify giving functions an optional implicit ExitStack instance and assigning a dedicated keyword for adding entries to it.

It occurred to me that I should elaborate a bit further here, and point out explicitly that one of the main benefits of ExitStack (and, indeed, the main reason it exists) is that it allows resource lifecycle management to be deterministic, *without* necessarily tying it to function calls or with statements.

The behave BDD test framework, for example, defines hooks that run before and after each feature and scenario, as well as before and after the entire test run. I use those to set up "scenario_cleanup", "_feature_cleanup" and "_global_cleanup" stacks as part of the testing context: https://github.com/leapp-to/prototype/blob/master/ integration-tests/features/environment.py#L49

If a test step implementor allocates a resource that needs to be cleaned up, they register it with "context.scenario_cleanup", and then the "after scenario" hook takes care of closing the ExitStack instance and cleaning everything up appropriately.

For me, that kind of situation is when I'm most inclined to reach for ExitStack, whereas when the cleanup needs align with the function call stack, I'm more likely to reach for contexlib.contextmanager or an explicit try/finally.

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/