[Distutils] Packager/Installer for Jython Deps?

[Alexander Dorsk]

Hi Packaging Folks,

I'm looking for some tips on developing a packaging/install system for
Jython distributions.

Ideally I would like a create a system that would manage not only python
dependencies, but also java dependencies.

Perhaps a quick example will help to illustrate:

= Begin Example A =

Say I'm working on a project, and I want to install a package,
'my_jython_h2_stuff' that has these dependencies:
- the sqlalchemy python package
- the H2 Database .jar

In my dream world I would do something like this:
$ jython virtualenv.py jython_env;
$ source jython_env/bin/activate;
$ jython_env/bin/some_jython_package_installer install my_jython_h2_stuff

The end result of the above commands would be:
- The pure python components of my_jython_h2_stuff are in
jython_env/Lib/*/site_packages/my_jython_h2_stuff
- sqlalchemy is in jython_env/Lib/*/site_packages/sqalchemy
- The H2.jar file is in some place where jython can find it, e.g.
jython_env/javalibs/h2.jar

Then I could do something like this:

$ jython -c "import my_jython_h2_stuff"

Furthermore, the 'my_jython_h2_stuff' dependencies could be described in a
distutils2-style setup.cfg w/ sections like:
---
[requires-dist]
sqlalchemy

#A. Define single jars, through some uri/vcs scheme
[requires-jars]
h2.jar
http://h2.org/download/h2.jar

#B. Define individual jars, via maven-style IDs
[requires-jars-maven]
org.h2:h2.jdbc.jar:1.14

#C. Define requirements in a pom.xml file
# Defines path to .pom file, relative to setup.cfg file.
[requires-pom]
pom.xml
---

At this point, since we're still in my dream world, Jennifer Connelly would
show up in a boat with umbrella drinks and a fresh rhubarb pie. But I
digress.

= End Example A =

Hopefully that example helps illustrate what I mean.

Currently it's somewhat tricky to manage both .jar and python dependencies
for Jython projects, due to Jython's mixed heritage. You might say that
Jython is the Obama of programming; a hybrid that grapples with its dual
heritage and yet draws upon it for strength. It's complicated sometimes.

However, with recent developments such as the distutils2 style of keeping
metadata outside setup.py, and tools like sunng87's jip <
https://github.com/sunng87/jip >, I think there may be ways to address the
difficulty of managing dependencies for jython distributions.

One way might be to create an installer, or an extension to an existing
installer that handles special jython-specific metadata, and this leads me
to some questions:

Q1: Is there an existing installer that could handle special cases of
metadata?

Q2: If 'no' to Q1, would it be better to (A) create a new installer from
scratch, or (B) is there a way to extend an existing installer, such as pip?

Q3: If (B) for Q2, is there documentation on how how to do this? I didn't
see anything the pip documents about extending PIP to handle special
metadata.

I'd love to hear any pointers.

There is possibly great value to be gained if a Jython distribution
installer could be developed. For example, it could be possible to easily
develop and distribute python apps via self-contained stand-alone .jars .
Imagine something like this:

1. create a jython_installer requirements file (just like a pip
requirements file)
2. $ jython create_jython_virtualenv -r requirements.txt my_app_virtualenv
3. (do custom app development in the virtual env, and define an application
entry point)
4. jython virtualenv_to_jar my_app_virtualenv > my_standalone_app.jar
5. mail my_standalone_app.jar to some windows user.
6. user double-clicks my_standalone_app.jar, and- *it just runs*, no
installers or anything.

It would also be possible to include jython ports in the same source
bundles as the python originals. For example, a numpy port that uses java
libs instead of c extensions. The .py source would check for the platform
and import jython or cython modules appropriately, and requisite .jars
would be described in the setup.cfg file. For normal cython installs via
pip or the such, the .jar metadata would be ignored. For jython installs
via a jython_installer, the .jars would be fetched.

And the naming possibilities for such a packager are endless. "Samuel L.
Jackson's 'Snakes-in-a-jar', or perhaps "PeanutCan", like the old
Snake-in-a-can prank... < http://www.youtube.com/watch?v=GADUPKy4jos >.

Anyway, if anyone has any tips on the above questions, I'm would love to
hear 'em.

-Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20121202/3c8b31b9/attachment.html>