[Chicago] import question

[Kumar McMillan]

On Jan 23, 2008 10:07 AM, Massimo Di Pierro <mdipierro at cs.depaul.edu> wrote:
> if I import it it works. But in web2py apps, ideally the individual
> applications should carry packages with themselves because they are
> not supposed to have dependencies. It is a long story.
>
> installing is what I do not want to do.

if you are trying to package 3rd party deps with an app, it is my
humble opinion that you should alter sys.path instead of turning them
into submodules.  That is, mimic "installation."  This will make
future maintenance of updating the dependency *much* easier.  I would
propose a directory structure that mimics site-packages instead of a
myapp.contrib approach.  I think django can mostly get away with a
contrib submodule because these are modules that were written to live
specifically in django.contrib.  What you are doing is different, you
are distributing copies of modules that were written to be used as a
global python modules.  Here is a possible structure :


frozen/
    openid/
        __init__.py
    simplejson/
        __init__.py
myapp/
    __init__.py
    something
        __init__.py

the tricky part is altering sys.path at the right time but if you
already have a standalone app then there should be one point of entry
into that app already.  Call it run_my_app.py and have it look
something like:

#!/auto/generated/path/to/python

import sys
sys.path = [
    '/auto/generated/path/to/frozen'
] + sys.path
from myapp import main
main()

... and an analogous test_my_app.py script as well.

Then, anytime your app does "import simplejson" or "import openid" it
will get the *frozen* version that you want to package with your app,
not a global one from site-packages.  You ever want to update
simplejson?  No problem!  Just replace the one in frozen.  This would
be a more maintainable use of 3rd party python modules.  Of course,
the contrib submodule approach would work if *every* single 3rd party
package supported relative imports.  This will only happen when it is
an acceptable time to drop 2.4 support, which is a long way off.
Maybe our children will live through that era :)

BTW ... I think bundling dependencies is a smart idea and I wish it
were a bit easier to do in Python.  Imagine a production server where
you have 10 apps deployed and you want to upgrade a global package;
this is a regression test nightmare that no matter how talented your
QA team is will be very very hard to get right.  At my company we have
a tool that uses setuptools to make a local package of eggs per
application but it's still a hack of sorts and is prone to breakage
when setuptools changes.

Of course, the industry standard solution to this (Amazon does this I
believe) is to use vmware to create one single "OS" per application,
for ultimate isolation.  If you have that luxury then this is the way
to go.

Kumar