I agree that the stated use cases are better handled with ExitStack.  One area where `defer` might be useful is in lazy-evaluating global constants. For example in a genomics library used at my work, one module involves compiling a large number of regular expressions, and setting them as global contants in the module, like:

FOO1_re = re.compile(r'...')
FOO_TO_BAR_re = {foo: complicated_computation_of_regex(foo) for foo in LONG_LIST_OF_THINGS}
...

This utility module is imported in a lot of places in the codebase, which meant that importing almost anything from our codebase involved precompiling all these regular expressions, which took around 500ms to to run the anything (the test runner, manually testing code in the shell, etc.) It would be ideal to only do these computations if/when they are needed. This is a more general issue than this specific example, e.g. for libraries which parse large data sources like pycountry (see my PR for one possible ugly solution using proxy objects; the author instead went with the simpler, less general solution of manually deciding when the data is needed). See also django.utils.functional.lazy, which is used extensively in the framework.

A statement like: `defer FOO = lengthy_computation_of_foo()` which deferred the lengthy computation until it is used for something would be useful to allow easily fixing these issues without writing ugly hacks like proxy objects or refactoring code into high-overhead cached properties or the like.

Alternatively, if there are less ugly or bespoke ways to handle this kind of issue, I'd be interested in hearing them.

Best,
Lucas



On Sat, Jun 3, 2017 at 11:38 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 3 June 2017 at 22:24, Nick Coghlan <ncoghlan@gmail.com> wrote:
> So while I'm definitely sympathetic to the use case (otherwise
> ExitStack wouldn't have a callback() method), "this would be useful"
> isn't a sufficient argument in this particular case - what's needed is
> a justification that this pattern of resource management is common
> enough to justify giving functions an optional implicit ExitStack
> instance and assigning a dedicated keyword for adding entries to it.

It occurred to me that I should elaborate a bit further here, and
point out explicitly that one of the main benefits of ExitStack (and,
indeed, the main reason it exists) is that it allows resource
lifecycle management to be deterministic, *without* necessarily tying
it to function calls or with statements.

The behave BDD test framework, for example, defines hooks that run
before and after each feature and scenario, as well as before and
after the entire test run. I use those to set up "scenario_cleanup",
"_feature_cleanup" and "_global_cleanup" stacks as part of the testing
context: https://github.com/leapp-to/prototype/blob/master/integration-tests/features/environment.py#L49

If a test step implementor allocates a resource that needs to be
cleaned up, they register it with "context.scenario_cleanup", and then
the "after scenario" hook takes care of closing the ExitStack instance
and cleaning everything up appropriately.

For me, that kind of situation is when I'm most inclined to reach for
ExitStack, whereas when the cleanup needs align with the function call
stack, I'm more likely to reach for contexlib.contextmanager or an
explicit try/finally.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/