[py-dev] getting syspath handling right (was Re: making py.test ignore an __init__.py)

[lahwran]

Apparently there is a way to remove the __init__.py and have django
function, so rather than making pytest bend over backwards even farther,
I'm going to remove the __init__.py. thank heavens for that.

On Mon, Dec 17, 2012 at 2:18 AM, holger krekel <holger at merlinux.eu> wrote:

> Hi lahwran, all,
>
> On Sun, Dec 16, 2012 at 23:34 -0700, lahwran wrote:
> > Hi, I've got a bit of a problem related to how pytest determines the
> fully
> > qualified name for a module. I have a django 1.3 layout project which has
> > an __init__.py at its root, due to oddities in how django functions. so I
> > have something like this layout:
> >
> > ~/project_venv/ # containing the virtualenv for the project
> > ~/project_venv/project/ # the directory I cd to when I work
> > ~/project_venv/project/.git
> > ~/project_venv/project/__init__.py # because django's stuff doesn't work
> > without it
> > ~/project_venv/project/applications/
> > ~/project_venv/project/applications/__init__.py
> > ~/project_venv/project/applications/projectapp/
> > ~/project_venv/project/applications/projectapp/__init__.py
> > ~/project_venv/project/applications/projectapp/some_file.py
> > ~/project_venv/project/applications/projectapp/some_other_file.py
> > ~/project_venv/project/applications/projectapp/tests/
> > ~/project_venv/project/applications/projectapp/tests/__init__.py
> > ~/project_venv/project/applications/projectapp/tests/test_some_file.py
> >
> ~/project_venv/project/applications/projectapp/tests/test_some_other_file.py
> >
> > in test_some_file.py, I have something like:
> >
> >     from applications.projectapp.some_file import Herp, Derp, doop
> >
> > and in test_some_other_file.py, I have something similar:
> >
> >     from applications.projectapp.some_other_file import Herk, Derk, foo
> >     from applications.projectapp.tests.test_some_file import
> > SomeTestUtilityThingy
> >
> > however, because project/ has an __init__.py, when pytest does its
> > collection, rather than importing
> > applications.projectapp.tests.test_some_file, it imports,
> > project.applications.projectapp.tests.test_some_file! worse, when
> > test_some_other_file.py imports test_some_file, it creates a *duplicate*
> -
> > the import creates an applications.projectapp.tests.test_some_file when
> > project.applications.projectapp.tests.test_some_file already existed.
>
> evil.  If it would be some arbitrary project promoting this strange
> __init__.py file practice i'd be inclined to say "fix it".  If a project
> like Django really promotes this, then i guess we have to deal with it :/
>
> > so, my question is: how can I tell pytest to please just pretend that
> > __init__.py doesn't exist? right now I'm using conftest.py to shove
> > dirname(__file__) onto sys.path, but that was written before I thought to
> > check if there was an __init__.py in the root of the project.
>
> It's tricky business to get sys-path manipulations workable in all
> the different kind of real life situations.  I think nose just adds
> the dir of the test module to sys.path and imports it, and if another
> test module with the same basename exists in a different directories
> unloads the module.  Or maybe it even always performs reloading, not sure.
> In nose2 this mechanism is to be dropped, anyway.
>
> In python3 unittest discovery requires a "toplevel" directory setting.
> If you don't set it from the command line the current working dir is
> used.  Unless i am missing something  that makes importing between
> test modules rather fragile.
>
> pytest always tries to import under a fully qualified name by walking the
> directories up that contain an __init__.py.  This avoids the reloading
> business (which i think is not a good idea, nose2/Jason seem to agree)
> and, unlike unittest, it gives test modules a reliable cross-importing
> behaviour.  Given the above problem, I think we should refine the pytest
> algorithm to also take existing sys.path settings into account and stop
> going further up if we find one.  In your situation it would stop at
> "projects/" even though it contains an __init__.py file which would
> otherwise lead it to go further up.
>
> With python3.3 we need to also allow no __init__.py files at all (the
> new namespace import stuff) and just check directly for a sys.path that
> fits.
>
> Sounds like a plan?  Note that i was also considering inifile-settings
> or new hooks to influence the behaviour.  But this is not easy to get
> right in all situations.  At least i didn't manage.  Eventually i came
> up with the above refinement.  It's anyway better if things work by
> default and you don't have to read lengthy explanations like this
> mail here :)
>
> best,
> holger
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20121217/661f9424/attachment.html>