[Pythonmac-SIG] py2app and nested packages

[Chris Barker]

On Tue, Apr 24, 2012 at 4:29 AM, Ronald Oussoren

>> example: the "pubsub" package is delivered with wxPython, so it is
>> commonly imported thusly:
>>
>> from wx.lib.pubsub import Publisher
>
> wx.lib.pubsub is a bad example, it manipulates __path__ which causes problems with py2app because it cannot detect this.  I added a recipe to py2app's repository a while ago that should fix this particular problem.


I thought I had the latest -- I'll go check that out.

Also, there is a bit of trick if you force it to import it's dyanamic
module, but if you've solved that one, great!


> That's a bug, the code that implements the packages options assumes that all mentioned packages are toplevel packages.  It should be fairly easy to fix that, although this requires some path manipulation trickery because the subpackage will be in a different location that the parent: the parent is in site-packages.zip while the child is not.

right -- do you have an idea how to solve that? I don't!

> The option "packages" currently always stores the included package outside of site-packages.zip to ensure that data files can be loaded in the old pre-pkgresources way (that is, by opening paths relative to somepackage.__file__).

which is useful, but another thought -- maybe there could be two options:

"packages" and "full_packages" or something -- so one would simple add
the package to the modulegraph, to catch dynamic imports, and the
other would do what is done now -- include the whole darn thing.

One other issue with the "whole darn thing" option, is that you get
the *.py, *.pyc, Na *.pyo files -- so it's maybe three times as big as
it could be -- I wonder if it's worth cleaning that up.

>> Is there a way to "turn off" a recipe (other than deleting it from the install)?

> No, and I'd prefer to fix recipes instead of having a way to disable them.

I understand that, and generally agree -- but it may be that there is
no one recipe that works for everyone. For example, Scipy is weird
enough that probably the only way to be sure it'll work in all cases
is to include the whole darn thing -- but in my case at least, I'm
using a couple functions in scipy.special, and it works fine if that
is all I include.

Maybe if there is a way to pass options to recipes?

> The (scipy) recipe should get a way to specify which dynamic imports are present, simular to the hardcoded knowledge about the stdlib in modulegraph. That way py2app can include only the bits that are really necessary.

If you give me a pointer - maybe to the similar modulegraph code,
perhaps I could work on that.

>> IIRC the matplotlib recipe includes the entire package because of the data files that are in the package, including only the code doesn't work.

I've poked into this more -- it turn out that there is special-case
code in matplotlib itself for py2exe that looks for the data-files in
a different place when sys.frozen -- I'm adding some code for py2app
as well -- if I get that working, I can submit a patch to MPL and the
recipe.

Is there a way to specify data files in a recipe?

>>- Is there a way to apply excludes after the recipe? As far as I can
>> tell, if the recipe includes it, excludes doesn't remove it.

> That's correct, and is a bug: user specified actions should override > automatic actions, including the recipes.

OK -- I suspect the issue is there is no way to exclude stuff that is
going to go outside of the zip file -- the full packages. It looks
like what it does is remove it from modulegraph only.

Thanks for your work (and ideas) on this.

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov