[Import-SIG] My objections to implicit package directories

[Guido van Rossum]

On Wed, Mar 14, 2012 at 8:33 PM, PJ Eby <pje at telecommunity.com> wrote:
> On Mon, Mar 12, 2012 at 11:49 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> So the only backwards incompatibility is that "import foo" may succeed
>> where it previously failed if there is a directory foo/ somewhere on
>> sys.path but no foo.py and no foo/__init__.py anywhere. I don't think
>> this is a big deal.
>
>
> Actually, it *is* a big deal.  An early draft of PEP 402 was like the
> newly-proposed approach, and it was shot down by an *actual code sample from
> a real package* that somebody posted to show the problem.  The motivating
> example is actually presented in the PEP: trying to "import json" with a
> json/ directory present -- and the "json" package *not* present -- and then
> trying to use its contents.  The PEP works around this by taking the
> namespace concept to its logical extreme and saying you can only import the
> *contents* of the namespace, because the namespace itself doesn't exist.
>  (i.e., it's virtual.)
>
> tl;dr version: the killer problem with allowing "import foo" to succeed is
> that code which does try/except ImportError to test for a package's presence
> will get false positives.  That's the motivating reason for only allowing
> sub-namespace package imports to succeed: it prevents the immediate breakage
> of real code, the moment the feature is introduced.  :-(

Well, too bad. It's too much of a wart. Such try/except clauses make
me sad anyway -- and we're trying to get rid of them by e.g.
encouraging the pattern where heapq.py tries to import _heapq rather
than having user code try first _heapq and then heapq. (And the idiom
used in heapq.py doesn't mind if _heapq exists as an empty package.)

-- 
--Guido van Rossum (python.org/~guido)