[Python-Dev] Dropping __init__.py requirement for subpackages

[M.-A. Lemburg]

Guido van Rossum wrote:
> On 4/26/06, Barry Warsaw <barry at python.org> wrote:
>> On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote:
>>
>>> So I have a very simple proposal: keep the __init__.py requirement for
>>> top-level pacakages, but drop it for subpackages. This should be a
>>> small change. I'm hesitant to propose *anything* new for Python 2.5,
>>> so I'm proposing it for 2.6; if Neal and Anthony think this would be
>>> okay to add to 2.5, they can do so.

I'm not really sure what this would buy us.

Newbies would still forget the __init__.py in top-level packages
(not all newbies work for Google).

Oldies would have trouble recognizing a directory as being a Python
package, rather than just a collection of modules - you wouldn't
go hunting up the path to find the top-level __init__.py file
which identifies the directory as being a sub-package of some
top-level package (not all oldies work for Google either where
you only have a single top-level package).

-1. It doesn't appear to make things easier and breaks symmetry.

>> But if it's there, then nothing changes, right?  IOW, if we want to
>> expose names in the subpackage's namespace, we can still do it in the
>> subpackage's __init__.py.  It's just that otherwise empty subpackage
>> __init__.py files won't be required.
> 
> Correct. This is an important clarification.
> 
>> What would the following print?
>>
>> import toplevel.sub1.sub2
>> print toplevel.sub1.sub2.__file__
>>
>> If it's "<path>/sub1/sub2" then that kind of breaks a common idiom of
>> using os.path.dirname() on a module's __file__ to find co-located
>> resources.  Or at least, you have to know whether to dirname its
>> __file__ or not (which might not be too bad, since you'd probably know
>> how that package dir is organized anyway).
> 
> Oh, cool gray area. I propose that if there's no __init__.py it prints
> '<path>/sub1/sun2/' i.e. with a trailing slash; that causes dirname to
> just strip the '/'. (It would be a backslash on Windows of course).
>
>> I dunno.  Occasionally it trips me up too, but it's such an obvious and
>> easy fix that it's never bothered me enough to care.
> 
> But you're not a newbie. for a newbie who's just learned about
> packages, is familiar with Java, and missed one line in the docs, it's
> an easy mistake to make and a tough one to debug.

Why not make the ImportError's text a little smarter instead,
e.g. let the import mechanism check for this particular gotcha ?

This would solve the newbie problem without any changes
to the import scheme.

>> I can't think of
>> an example, but I suppose it's still possible that lifting this
>> requirement could cause some in-package located data directories to be
>> mistakenly importable.  I'd be somewhat more worried about frameworks
>> that dynamically import things having to be more cautious about
>> cleansing their __import__() arguments now.
> 
> But (assuming 2.6 and absolute import) what would be the danger of
> importing such a package? Presumably it contains no *.py or *.so files
> so there's no code there; but even so you'd have to go out of your way
> to import it (since if the directory exists it can't also be a
> subpackage or submodule name that's in actual use).

Wasn't there agreement on postponing the absolute imports
to Py3k due to the common use-case of turning e.g. 3rd party
top-level packages into sub-packages of an application ?

Without absolute imports your proposed scheme is going to
break, since can easily mask top-level packages or modules.

>> I'd be -1 but the remote possibility of you being burned at the stake by
>> your fellow Googlers makes me -0 :).
> 
> I'm not sure I understand what your worry is.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 26 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::