Hi Kevin,

Sorry for not being able to respond to this earlier. I have time only in the evening.

On Dec 3, 2015, at 3:57 AM, KP <patter001@gmail.com> wrote:

Yes, the https://github.com/pypa/pip/issues/3 definitely sounds like my issue. Seems there is some concern over the nspkg.pth there as well. It seems taht the nspkg.pth is a fine idea to replace the install of the __init__.py, but that it just doesn’t work to fully extend the locations in which a namespace can reside.

I now understand what the issue is and can help perhaps with some advice below.

Either way, there a few other workarounds posted there as well, I will check them out to see if any of them are more palatable than the one I posted here.

You need to make foo.bar-1.0.0-py2.7-nspkg.pth look like this:

import sys, types, os, pkgutil; p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('foo',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('foo',types.ModuleType('foo')); mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p); mp[:] = m and pkgutil.extend_path(mp, 'foo') or mp

Notice the difference from the default in use of pkgutil.extend_path() that helps with editable packages.

How to make this happen automatically is explained below.

Thanks Lele for sending that link!

-Kevin

I also want to thank Lele for pointing out the possible cause of your issues.

Now, I’d like to separate two things here:

1. Defining namespaces correctly — that should be done as we talked before in the standard way.

2. Issue with the tooling — in this case pip is the tool that causes issues.

Let’s talk about it more. These are the reasons I didn’t recognize Kevin’s issue. :-)

- The namespaces work great when defined correctly (#1 above) with zc.buildout and other tools.

  There are good open-source build systems out there that take care of namespaces properly.

- When one makes a good custom build system, it can fix the pip deficiencies too.

  For example, the build system specifically used in a company, team, …

So, my advice is to:

1. Make a custom distutils Distribution extension, for example, MyDistribution (or some better name)

    (a) make custom install_egg_info class that overrides either:

          -  two templates for pth files (for newer versions of setuptools) or

          - install_namespaces method (for older versions of setuptools)

    (b) override get_command_class method in the MyDistribution class with your own install_egg_info

2. Build this package and install it.

3. Use distclass=MyDistribution in your foo package setup.py (import it from your package).

Using this approach everything will work for every of your packages automatically.

I attached your original code from foo_test.zip and added to it a minimal mydist package + a sample __init__.py file that I think you should use in your namespace packages (if you want).

Kevin, you can try this by doing the following:

- create a virtual environment

- source bin/activate

- go to mydist package and run “python setup.py sdist”

- pip install /path/to/mydist-1.0.0.tar.gz

- go to your foo_bar package and run “python setup.py sdist”

- pip install /path/to/foo.bar-1.0.0.tar.gz

- pip install -e /path/to/foo_more

- start the python interpreter in your virtual environment

- confirm that it works

>>> import foo.bar
foo.bar imported
>>> import foo.more
foo.more imported

I know that this is not a fix for pip, but it is a generic solution that I hope helps you.

Perhaps I should make a little better example (better names, better __init__.py, …) and post it as a little project on GitHub that people can refer to.

All the best,

Zvezdan

On Thu, Dec 3, 2015 at 3:19 AM, Lele Gaifax <lele@metapensiero.it> wrote:

Zvezdan Petkovic <zvezdanpetkovic@gmail.com> writes:

> Hi KP,
>
>> On Dec 2, 2015, at 7:00 PM, KP <patter001@gmail.com> wrote:
>>
>> >It looks like you are trying to find a workaround for the problem that perhaps is not a problem at all if you use the standard approach properly.
>>
>> I'm definitely _trying_ to use a standard approach...That is why I am here posting. Put simply this seems like a valid use case:
>>
>> >pip install foo.bar
>> >pip install -e svn+http://<path-to-foo.more>
>>
>> Even if both tools have the namespace_package foo, the "foo.more" will not properly import.
>>
>> How is this going against standard approaches?
>
> I don’t know.
> Without seeing the source code for these packages (which Marius already asked for) everything is hypothetical.
> All I know is that I’m using namespaces successfully for over a decade now and never had the need to work around them.
>
> Show us the code.  We may be able to help better if we can try and reproduce.
> Otherwise, it’s hard.

It seems that KP case is either https://github.com/pypa/pip/issues/3160 or
https://github.com/pypa/pip/issues/3, isn't it? Both come with sample code
that demonstrate the problem.

ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig