[Distutils] namespace_package

Wed Dec 2 22:00:52 EST 2015

>It looks like you are trying to find a workaround for the problem that
perhaps is not a problem at all if you use the standard approach properly.

I'm definitely _trying_ to use a standard approach...That is why I am here
posting. Put simply this seems like a valid use case:

>pip install foo.bar
>pip install -e svn+http://<path-to-foo.more>

Even if both tools have the namespace_package foo, the "foo.more" will not
properly import.

How is this going against standard approaches?

On Wed, Dec 2, 2015 at 9:54 PM, Zvezdan Petkovic <zvezdanpetkovic at gmail.com>
wrote:

> Hi KP,
>
> Maybe I didn’t follow the thread from the beginning, but your use case is
> not clear to me.
>
> On Dec 1, 2015, at 1:31 PM, KP <patter001 at gmail.com> wrote:
>
> (sorry for the stupid previous early send)
>
> Just to recap:
>
> 1. if you don’t put namespace_packages in the setup.py, then it will
> uninstall the shared __init__.py when you uninstall any of the packages
>
>
> Right. You need to declare the namespace in setup.py.
>
> 2. If you put namespace_packages, then there is a pth file created for
> the shared directory (site-packages/foo) and no foo/__init__.py is created
> (even if it is in your package)
>
>
> Correct. That is exactly the behavior that’s expected. There can be many
> packages in the namespace (e.g., zope.* or zope.app.*) and they all share
> the namespace, but none of them owns that __init__.py. That’s why it’s not
> installed there and is replaced with a *.pth file.
>
> #2 - breaks things like : doing a source checkout that participates in
> this namespace_package…If you do this then only the
> lib/site-packages/foo/<modules> are importable
>
>
> Again, it’s not clear what are you referring to?
> Are you doing a source checkout over your installed packages / virtual
> environment?
> I sure hope not.
> You can use pip —editable for adding your source checkouts to virtual
> environment or installed packages.
>
> If you use build tools (such as zc.buildout) and install your packages,
> they will end up over several egg / wheel directories “parallel” to each
> other, but the proper namespace declaration will ensure that they are all
> importable. You cannot even do the source checkout over such an
> installation.
>
> It looks like you are trying to find a workaround for the problem that
> perhaps is not a problem at all if you use the standard approach properly.
>
> Solution appears to be:
>
> create a standalone “foo" package that has ONLY the shared __init__.py,
> and do NOT set "namespace_packages" .
>
>
> No, that is not the solution. That’s a sure way to break the namespace in
> the egg/wheel based installations.
>
> This seems to associate the __init__.py with the “foo” tool, so that only
> when you uninstall the "foo" tool does the __init__.py get uninstalled.
>
>
> Why do you worry so much about uninstalling __init__.py.
> If it’s not there (and it isn’t with properly declared namespaces) why
> does it matter?
>
> It also has the shared __init__.py in lib/site-packages, so it seems that
> enables the other namespace packages that are source checkouts or added to
> path via other methods.
>
>
> What other methods?
> The key here is to make decision how do you want to install your packages?
> Pick one and use it instead fighting the tooling.
>
>
> Hopefully this works long term, and maybe is useful to someone else out
> there….
>
>
> I doubt it,
> It’s an incorrect advice.
>
> Zvezdan
>
>
> Thanks,
>
>
>
>
> On Tue, Dec 1, 2015 at 4:29 PM, KP <patter001 at gmail.com> wrote:
>
>> Just to recap:
>>
>> 1. if you don't put namespace_packages in the setup.py, then it will
>> uninstall the shared __init__.py when you uninstall any of the packages
>> 2. If you put namespace_packages, then there is a pth file created for
>> the shared directory (site-packages/foo) and no foo/__init__.py is created
>> (even if it is in your package)
>> #2 - breaks things like : doing a source checkout that participates in
>> this namespace_package...If you do this then only the
>> lib/site-packages/foo/<modules> are importable
>>
>> Solution appears to be:
>>
>>
>> On Tue, Dec 1, 2015 at 7:17 AM, KP <patter001 at gmail.com> wrote:
>>
>>> yes, both of those statements are true.
>>>
>>> However, with the namespace_packages = ['foo'], the
>>> lib\site-packages\foo\__init__.py does not get installed (even though it is
>>> in the source tree). Instead there's just a dir with "foo/bar/__init__.py"
>>> and "foo/blah/__init__.py". I will try to look in the "wheel" side of
>>> things next I guess. Perhaps pip is doing something since it seems to
>>> install even source distributables by first converting to a wheel.
>>>
>>>
>>> On Tue, Dec 1, 2015 at 1:38 AM, Marius Gedminas <marius at gedmin.as>
>>> wrote:
>>>
>>>> On Mon, Nov 30, 2015 at 06:59:31PM -0500, KP wrote:
>>>> > I'm not sure where the issue is, but when I specify a
>>>> namespace_package in
>>>> > the setup.py file, I can indeed have multiple packages with the same
>>>> base
>>>> > (foo.bar, foo.blah, etc...). The files all install in to the same
>>>> > directory. It drops  the foo/__init__.py that would be doing the
>>>> > extend_path, and instead adds a ".pth" file that is a bit over my
>>>> head.
>>>> >
>>>> > The problem is that it does not seem to traverse the entire sys.path
>>>> to
>>>> > find multiple foo packages.
>>>>
>>>> Does every foo.x package specify namespace_packages=['foo']?
>>>>
>>>> Do they all ship an identical foo/__init__.py with
>>>>
>>>>     import pkg_resources
>>>>     pkg_resources.declare_namespace(__name__)
>>>>
>>>> ?
>>>>
>>>> AFAIU you need both things in every package, if you want to use
>>>> namespace packages.
>>>>
>>>> > If I do not specify namespace_packages and instead just use the
>>>> > pkgutil.extend_path, then this seems to allow the packages to be in
>>>> > multiple places in the sys.path.
>>>> >
>>>> > Is there something additional for the namespace_package that i need to
>>>> > specify in order for all of the sys.path to be checked?
>>>> >
>>>> > I'm using 18.5 setuptools....but I am not sure if this somehow ties
>>>> in to
>>>> > wheel/pip, since I'm using that for the actual install.
>>>>
>>>> Marius Gedminas
>>>> --
>>>> Give a man a computer program and you give him a headache, but teach
>>>> him to
>>>> program computers and you give him the power to create headaches for
>>>> others for
>>>> the rest of his life...
>>>>         -- R. B. Forest
>>>>
>>>> _______________________________________________
>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>>
>>>>
>>>
>>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20151202/a121c70c/attachment.html>