[Python-Dev] Re: Can we limit the effects of module execution to sys.modules? (was Fix import errors to have data)

[Jim Fulton]

Tim Peters wrote:
> [Tim, wants to keep insane modules out of sys.modules]
> 
> [Jim Fulton]
> 
...

>>Do you think it's practical to limit the effects of module import to
>>sys.modules, even by convention?
> 
> 
> I'm sure you didn't intend that to be *so* extreme -- like surely a
> module is allowed to initialize its own module-level variables.   If I
> read "effects" as "effects visible outside the module", then that's
> what you said <wink>.

Yup. :)


>>If it is possible to limit the effects of import (even by convention),
>>then I think it would be practical to roll-back changes to sys.modules.
>>If it's not practical to limit the effects of  module import then I think
>>the problem is effectively unsolveable, short of making Python transactional.
> 
> 
> There we don't agree -- I think it's already practical, based on that
> virtually no Python application *intends* to catch errors from imports
> other than ImportError, so that almost all "real bugs" in module
> initialization are intended to stop execution.  In turn, in the cases
> where ImportErrors are intentionally caught now, they generally occur
> in "import blocks" near the starts of all modules in the failing
> import chain, and so none of the modules involved have yet *done* any
> non-trivial initialization -- they're all still trying to import the
> stuff they need to *start* doing the meat of their initialization. 

Except in cases where imports are places later in a module to make
circular imports work.  It would be nice to disallow circular imports,
although I don't know if that's possible.  (Some time, when I have time,
I'd like to see if we can get rid of them in Zope 3.  I'd like to have a
tool to report circular imports, to keep them from creeping in, which
they do so easily.)

Having said that, you make a good point.  If all modules did their imports
before making any changes outside the modules, then it would be pretty safe
to clean up the imports.

> If
> some modules happen to import successfully along the way, fine, they
> should stay in sys.modules, and then importing them again later won't
> run their initialization code again. 

This only works if all of the modules do all of their imports
before doing other work.

If there are circular imports, you could have:

   A defines class C
   A imports B
   B imports C from A
   A imports D and gets an import error

If we clean up A and D, then B has a class from a module that doesn't exist.

Hm. Suppose we have:

   A imports B
   B imports A
   A imports D and gets an import error

We clean up A and D. What about the A that was imported into B?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org