Project organization and import

Bruno Desthuilliers bdesth.quelquechose at free.quelquepart.fr
Mon Mar 5 23:18:14 CET 2007


Martin Unsal a écrit :
> On Mar 5, 12:45 am, "bruno.desthuilli... at gmail.com"
> <bruno.desthuilli... at gmail.com> wrote:
> 
>>Remember that you can put code in
>>the __init__.py of a package, and that this code can import sub-
>>packages/modules namespaces, making the package internal organisation
>>transparent to user code
> 
> 
> Sure, but that doesn't solve the problem.
> 
> Say you have a package "widgets" with classes ScrollBar, Form, etc.
> You want the end user to "import widgets" and then invoke
> "widgets.ScrollBar()". As far as I know there are only two ways to do
> this, both seriously flawed: 1) Put all your code in one module
> widgets.py, 2) use "from scrollbar import *" in widgets/__init__.py,
> which is semi-deprecated 

"deprecated" ? Didn't see any mention of this so far. But it's bad form, 
since it makes hard to know where some symbol comes from.

# widgets.__init
from scrollbar import Scrollbar, SomeOtherStuff, some_function, SOME_CONST

> and breaks reload().

> 
>>Also remember that you can "import as", ie:
>>
>>import some_package.some_subpackage.some_module as some_module
> 
> 
> Sure but that doesn't eliminate the unfortunate interaction between
> Python class organization and filesystem heirarchy.

*class* organization ? It's not Java here. Nothing forces you to use 
classes.

> For example, say
> you want to organize the widgets package as follows:
> 
> widgets/scrollbar/*.py
> widgets/form/*.py
> widgets/common/util.py
> 
> Other than messing around with PYTHONPATH, which is horrible, I don't
> see how to import util.py from the widget code.

Some of us still manage to do so without messing with PYTHONPATH.

> 
>>Bad form IMHO. Packages and module names should be all_lower,
>>classnames CamelCased.
> 
> 
> You're still stuck doing foo.Foo() everywhere in your client code,

from foo import Foo

But:
> which is ugly 

It's not ugly, it's informative. At least you know where Foo comes from.

 > and wastes space,

My. Three letters and a dot...

> or using "from foo import *" which is
> broken.

cf above.

> 
>>>but I
>>>believe this is categorically the wrong thing to do in large projects.
>>
>>Oh yes ? Why ?
> 
> 
> For myriad reasons, just one of them being the one I stated -- smaller
> files with one functional unit each

Oh. So you're proposing that each and any single function goes in a 
separate file ?

> are more amenable to source code
> management with multiple developers.

This is not my experience.

> We could discuss this till we're blue in the face but it's beside the
> point. For any given project, architecture, and workflow, the
> developers are going to have a preference for how to organize the code
> structurally into files, directories, packages, etc. The language
> itself should not place constraints on them. The mere fact that it is
> supposedly "Pythonic" to put more functionality in one file indicates
> to me that the Python package system is obstructing some of its users
> who have perfectly good reasons to organize their code differently.

It has never been an issue for me so far.

> 
>>>you're going to want files to contain the smallest
>>>practical functional blocks. I feel pretty confident saying that "put
>>>more stuff in one file" is the wrong answer, even if it is the
>>>Pythonic answer.
>>
>>Is this actually based on working experience ? It seems that there are
>>enough not-trivial Python projects around to prove that it works just
>>fine.
> 
> 
> Yes. I've worked extensively on several projects in several languages
> with multi-million lines of code

I meant, based on working experience *with Python* ? I've still not seen 
a "multi-million" KLOC project in Python - unless of course you include 
all the stdlib and the interpreter itself, and even then I doubt we get 
so far.

> and they invariably have coding
> styles that recommend one functional unit (such as a class), or at
> most a few closely related functional units per file.

Which is what I see in most Python packages I've seen so far. But we may 
not have the same definition for "a few" and "closely related" ?

> In Python, most of the large projects I've looked at use "from foo
> import *" liberally.

I've seen few projects using this. And I wouldn't like having to 
maintain such a project.

> I guess my question boils down to this. Is "from foo import *" really
> deprecated or not?

This syntax is only supposed to be a handy shortcut for quick testing 
and exploration in an interactive session. Using it in production code 
is considered bad form.

> If everyone has to use "from foo import *"

I never did in 7 years.

> despite
> the problems it causes, how do they work around those problems (such
> as reloading)?

Do you often have a need for "reloading" in production code ???

Martin, I'm not saying Python is perfect, but it really feels like 
you're worrying about things that are not problems.



More information about the Python-list mailing list