On Dec 2, 2015, at 7:00 PM, KP <patter001@gmail.com> wrote:

>It looks like you are trying to find a workaround for the problem that perhaps is not a problem at all if you use the standard approach properly.

I'm definitely _trying_ to use a standard approach...That is why I am here posting. Put simply this seems like a valid use case:

>pip install foo.bar 

>pip install -e svn+http://<path-to-foo.more>

Even if both tools have the namespace_package foo, the "foo.more" will not properly import.

How is this going against standard approaches?

On Wed, Dec 2, 2015 at 9:54 PM, Zvezdan Petkovic <zvezdanpetkovic@gmail.com> wrote:

Hi KP,

Maybe I didn’t follow the thread from the beginning, but your use case is not clear to me. 

On Dec 1, 2015, at 1:31 PM, KP <patter001@gmail.com> wrote:

(sorry for the stupid previous early send)

Just to recap:

1. if you don’t put namespace_packages in the setup.py, then it will uninstall the shared __init__.py when you uninstall any of the packages

Right. You need to declare the namespace in setup.py.

2. If you put namespace_packages, then there is a pth file created for the shared directory (site-packages/foo) and no foo/__init__.py is created (even if it is in your package)

Correct. That is exactly the behavior that’s expected. There can be many packages in the namespace (e.g., zope.* or zope.app.*) and they all share the namespace, but none of them owns that __init__.py. That’s why it’s not installed there and is replaced with a *.pth file.

#2 - breaks things like : doing a source checkout that participates in this namespace_package…If you do this then only the lib/site-packages/foo/<modules> are importable

Again, it’s not clear what are you referring to?

Are you doing a source checkout over your installed packages / virtual environment?

I sure hope not.

You can use pip —editable for adding your source checkouts to virtual environment or installed packages.

If you use build tools (such as zc.buildout) and install your packages, they will end up over several egg / wheel directories “parallel” to each other, but the proper namespace declaration will ensure that they are all importable. You cannot even do the source checkout over such an installation.

It looks like you are trying to find a workaround for the problem that perhaps is not a problem at all if you use the standard approach properly.

Solution appears to be:

create a standalone “foo" package that has ONLY the shared __init__.py, and do NOT set "namespace_packages" .

No, that is not the solution. That’s a sure way to break the namespace in the egg/wheel based installations.

This seems to associate the __init__.py with the “foo” tool, so that only when you uninstall the "foo" tool does the __init__.py get uninstalled.

Why do you worry so much about uninstalling __init__.py.

If it’s not there (and it isn’t with properly declared namespaces) why does it matter?

It also has the shared __init__.py in lib/site-packages, so it seems that enables the other namespace packages that are source checkouts or added to path via other methods.

What other methods?

The key here is to make decision how do you want to install your packages?

Pick one and use it instead fighting the tooling.

Hopefully this works long term, and maybe is useful to someone else out there….

I doubt it,

It’s an incorrect advice.

Zvezdan

Thanks,

On Tue, Dec 1, 2015 at 4:29 PM, KP <patter001@gmail.com> wrote:

Just to recap:

1. if you don't put namespace_packages in the setup.py, then it will uninstall the shared __init__.py when you uninstall any of the packages

2. If you put namespace_packages, then there is a pth file created for the shared directory (site-packages/foo) and no foo/__init__.py is created (even if it is in your package)

#2 - breaks things like : doing a source checkout that participates in this namespace_package...If you do this then only the lib/site-packages/foo/<modules> are importable

Solution appears to be:

On Tue, Dec 1, 2015 at 7:17 AM, KP <patter001@gmail.com> wrote:

yes, both of those statements are true.

However, with the namespace_packages = ['foo'], the lib\site-packages\foo\__init__.py does not get installed (even though it is in the source tree). Instead there's just a dir with "foo/bar/__init__.py" and "foo/blah/__init__.py". I will try to look in the "wheel" side of things next I guess. Perhaps pip is doing something since it seems to install even source distributables by first converting to a wheel.

On Tue, Dec 1, 2015 at 1:38 AM, Marius Gedminas <marius@gedmin.as> wrote:

On Mon, Nov 30, 2015 at 06:59:31PM -0500, KP wrote:
> I'm not sure where the issue is, but when I specify a namespace_package in
> the setup.py file, I can indeed have multiple packages with the same base
> (foo.bar, foo.blah, etc...). The files all install in to the same
> directory. It drops  the foo/__init__.py that would be doing the
> extend_path, and instead adds a ".pth" file that is a bit over my head.
>
> The problem is that it does not seem to traverse the entire sys.path to
> find multiple foo packages.

Does every foo.x package specify namespace_packages=['foo']?

Do they all ship an identical foo/__init__.py with

    import pkg_resources
    pkg_resources.declare_namespace(__name__)

?

AFAIU you need both things in every package, if you want to use
namespace packages.

> If I do not specify namespace_packages and instead just use the
> pkgutil.extend_path, then this seems to allow the packages to be in
> multiple places in the sys.path.
>
> Is there something additional for the namespace_package that i need to
> specify in order for all of the sys.path to be checked?
>
> I'm using 18.5 setuptools....but I am not sure if this somehow ties in to
> wheel/pip, since I'm using that for the actual install.

Marius Gedminas
--
Give a man a computer program and you give him a headache, but teach him to
program computers and you give him the power to create headaches for others for
the rest of his life...
        -- R. B. Forest

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig