[Import-SIG] My objections to implicit package directories

[Guido van Rossum]

On Thu, Mar 15, 2012 at 6:50 PM, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On Thu, Mar 15, 2012 at 1:50 PM, Guido van Rossum <guido at python.org> wrote:
[...]
>> It would also be odd to find that
>>
>>  import foo
>>  import foo.bar
>>
>> would fail, whereas
>>
>>  import foo.bar
>>  import foo
>>
>
> I would prefer neither `import foo` succeed, in fact, if foo is only
> an implied namespace package. The import machinery could treat it the
> same, up to the point that it would bind the name, where it would
> check if it was such an implied package and raise an appropriate
> exception. Maybe some ImportError subclass? So, this would not be in
> an error in the loading portion of the import, but in adding it to the
> namespace where the import statement was executed.

I'm sure someone can also construct a counterexample where this will
break existing code -- something that inspects a package hierarchy
might not expect that foo.bar is importable but yet foo is not. (And
if it is not, does that mean that it isn't in sys.modules? That
violates a whole 'nother set of expectations.)

>> would succeed, because as a side effect of "import foo.bar", a module
>> object for foo is created and inserted as sys.modules['foo'].
>>
>> I really don't want to have to maintain and explain such complexity --
>> nor the complexity that follows from attempts to "fix" this objection.
>>
>> Finally, in your example, why on earth would unittest/mock/ exist as
>> an empty directory???
>
> It was only meant as an example where the try/except for import
> fallbacks is natural and wise. It seemed your previous statements were
> calling all such things a wart.

Heh. I still find try / import / except / other import a warty
pattern. I agree it happens. And I agree it can be broken if an empty
directory creates a dummy for the first attempted import. But then,
there are plenty of other ways to break that (e.g. an unrelated .py
file). To debug this you just print the __file__ of the suspected
module, and then you'll know. And I find the modifications to the
import rules proposed to avoid the wart, wartier.

(PS. We agreed that the __file__ of a namespace package would be equal
to the first directory name plus a trailing os.sep -- os.path.split()
correctly breaks this into the directory name and an empty string, and
os.path.dirname() matches that behavior.)

-- 
--Guido van Rossum (python.org/~guido)