[Distutils] namespace_package

[KP]

I apologize for bringing up this old thread again, but I was working on
another namespace package, and going through this posted BKM again. First
off I noticed I did not give a good Thank You for posting the fix, my
apologies there.

I noticed the original issue (https://github.com/pypa/pip/issues/3)  was
still open, and was wondering, what is the downfall for the new .pth file
that was posted. Any reason that the proposed fix could not be permanently
added to setuptools?



On Fri, Dec 4, 2015 at 12:31 AM, Zvezdan Petkovic <zvezdanpetkovic at gmail.com
> wrote:

> Hi Kevin,
>
> Sorry for not being able to respond to this earlier. I have time only in
> the evening.
>
>
> On Dec 3, 2015, at 3:57 AM, KP <patter001 at gmail.com> wrote:
>
> Yes, the https://github.com/pypa/pip/issues/3 definitely sounds like my
> issue. Seems there is some concern over the nspkg.pth there as well. It
> seems taht the nspkg.pth is a fine idea to replace the install of the
> __init__.py, but that it just doesn’t work to fully extend the locations in
> which a namespace can reside.
>
>
>
> I now understand what the issue is and can help perhaps with some advice
> below.
>
>
> Either way, there a few other workarounds posted there as well, I will
> check them out to see if any of them are more palatable than the one I
> posted here.
>
>
>
> You need to make foo.bar-1.0.0-py2.7-nspkg.pth look like this:
>
> import sys, types, os, pkgutil; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
> *('foo',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not
> ie and sys.modules.setdefault('foo',types.ModuleType('foo')); mp = (m
> or []) and m.__dict__.setdefault('__path__',[]); (p not in mp)
> and mp.append(p); mp[:] = m and pkgutil.extend_path(mp, 'foo') or mp
>
> Notice the difference from the default in use of pkgutil.extend_path()
> that helps with editable packages.
>
> How to make this happen automatically is explained below.
>
>
> Thanks Lele for sending that link!
>
> -Kevin
>
>
>
> I also want to thank Lele for pointing out the possible cause of your
> issues.
>
> Now, I’d like to separate two things here:
>
> 1. Defining namespaces correctly — that should be done as we talked before
> in the standard way.
>
> 2. Issue with the tooling — in this case pip is the tool that causes
> issues.
>
> Let’s talk about it more. These are the reasons I didn’t recognize Kevin’s
> issue. :-)
>
> - The namespaces work great when defined correctly (#1 above) with
> zc.buildout and other tools.
>   There are good open-source build systems out there that take care of
> namespaces properly.
>
> - When one makes a good custom build system, it can fix the pip
> deficiencies too.
>   For example, the build system specifically used in a company, team, …
>
> So, my advice is to:
>
> 1. Make a custom distutils Distribution extension, for example,
> MyDistribution (or some better name)
>
>     (a) make custom install_egg_info class that overrides either:
>
>           -  two templates for pth files (for newer versions of
> setuptools) or
>           - install_namespaces method (for older versions of setuptools)
>
>     (b) override get_command_class method in the MyDistribution class with
> your own install_egg_info
>
> 2. Build this package and install it.
>
> 3. Use distclass=MyDistribution in your foo package setup.py (import it
> from your package).
>
> Using this approach everything will work for every of your packages
> automatically.
>
> I attached your original code from foo_test.zip and added to it a minimal
> mydist package + a sample __init__.py file that I think you should use in
> your namespace packages (if you want).
>
> Kevin, you can try this by doing the following:
>
> - create a virtual environment
> - source bin/activate
> - go to mydist package and run “python setup.py sdist”
> - pip install /path/to/mydist-1.0.0.tar.gz
> - go to your foo_bar package and run “python setup.py sdist”
> - pip install /path/to/foo.bar-1.0.0.tar.gz
> - pip install -e /path/to/foo_more
> - start the python interpreter in your virtual environment
> - confirm that it works
>
> >>> import foo.bar
> foo.bar imported
> >>> import foo.more
> foo.more imported
>
> I know that this is not a fix for pip, but it is a generic solution that I
> hope helps you.
>
> Perhaps I should make a little better example (better names, better
> __init__.py, …) and post it as a little project on GitHub that people can
> refer to.
>
> All the best,
>
> Zvezdan
>
>
>
>
>
>
> On Thu, Dec 3, 2015 at 3:19 AM, Lele Gaifax <lele at metapensiero.it> wrote:
>
>> Zvezdan Petkovic <zvezdanpetkovic at gmail.com> writes:
>>
>> > Hi KP,
>> >
>> >> On Dec 2, 2015, at 7:00 PM, KP <patter001 at gmail.com> wrote:
>> >>
>> >> >It looks like you are trying to find a workaround for the problem
>> that perhaps is not a problem at all if you use the standard approach
>> properly.
>> >>
>> >> I'm definitely _trying_ to use a standard approach...That is why I am
>> here posting. Put simply this seems like a valid use case:
>> >>
>> >> >pip install foo.bar
>> >> >pip install -e svn+http://<path-to-foo.more>
>> >>
>> >> Even if both tools have the namespace_package foo, the "foo.more" will
>> not properly import.
>> >>
>> >> How is this going against standard approaches?
>> >
>> > I don’t know.
>> > Without seeing the source code for these packages (which Marius already
>> asked for) everything is hypothetical.
>> > All I know is that I’m using namespaces successfully for over a decade
>> now and never had the need to work around them.
>> >
>> > Show us the code.  We may be able to help better if we can try and
>> reproduce.
>> > Otherwise, it’s hard.
>>
>> It seems that KP case is either https://github.com/pypa/pip/issues/3160
>> or
>> https://github.com/pypa/pip/issues/3, isn't it? Both come with sample
>> code
>> that demonstrate the problem.
>>
>> ciao, lele.
>> --
>> nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
>> real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
>> lele at metapensiero.it  |                 -- Fortunato Depero, 1929.
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160911/a23c2925/attachment-0001.html>