distutils (packaging) newbie questions please....

Tue Oct 7 15:45:21 EDT 2003

Alex

Your email motivated me to buy your book.  I think Nutshell
was a VERY good purchase.  Why? Lots of other docs for newbies
don't go into too much detail.  Your book is great for the completeness.
Intermediate users need that.... especially if they don't like reading
python.org docs online.

CS

Alex Martelli <aleax at aleax.it> wrote in message news:<SSzeb.148956$hE5.5054153 at news1.tin.it>...
> Christian Seberino wrote:
> 
> > I really want to learn distutils to professionally package some open
> > source software.
> > 
> > I had some distutils questions after reading python.org distutil docs
> > on packaging code
> > if you don't mind.  I would really appreciate some help.....
> > 
> > 
> > 0. I'm confused about use of the word "package" in setup.py.
> >     packages = ["<package1>", "<package2>", ...]
> >     Does package here just mean "a new directory of source code to
> > consider"???
> 
> No, it means a Python package.  The concept of "package" (a module
> that may contain other modules, and corresponds to a directory with
> an __init__.py file) exists in Python quite independently from
> distutils -- the distutils just SUPPORT the concept, if the code
> you want to package-up uses Python packages.
> 
> 
> > 1. What exactly goes into the __init__.py files? I know what to put in
> > setup.py
> >     and I know __init__.py is used for packages but I don't know what
> > to put IN this file.
> 
> Again, not a distutils-related question.  A Python package's __init__.py
> has the module-body of the package (remember, any package IS a module).
> If you need no Python statements whatsoever in that body, __init__.py
> may be empty (it must still exist, or else Python will refuse to
> consider that directory as a package).  Normally you do want, at least,
> to assign the appropriate list to __all__ .
> 
> In p. 124 of "Python in a Nutshell", i.e. the section named "Packages",
> I summarize the issue as follows:
> """
> A package named P resides in a subdirectory, also called P, of some
> directory in sys.path.  The module body of P is in the file P/__init__.py.
> You must have a file named P/__init__.py, even if it's empty (representing
> an empty module body), in order to indicate to Python that directory P
> is indeed a package.
> """
> 
> Do you find this unclear, or was it hard for you to find -- or didn't
> you even considering checking out the Nutshell book before setting
> yourself the task to "professionally package" some Python software?
> Feedback on this (private or public as you wish) may help me do a
> better job next time.  [Note that if the issue is with the purchase
> price of the Nutshell book, there IS a perfectly legal way to read it
> for free: the "Safari" online book service of O'Reilly offers the
> first two weeks free, so you can join it, and read Python in a Nutshell,
> which is among the over 1000 books on Safari, paying nothing as long
> as you cancel before two weeks are up].
> 
> 
> > 2. About the data_files option for setup.py....
> > 
> >       data_files = [ "<installation directory>", [ "<file1>",
> > "<file2>", ...] ]
> > 
> >    Is there a way to AVOID having to type EVERY single data file?? Can
> > I specify just the directory or use wild cards??
> 
> You can form the list of file names in any way you wish, including,
> for example, using the glob module from Python's standard library.
> setup.py is a Python script, and any Python construct is available
> to you as the author of that script.
> 
> For example, suppose we wanted to package up (for installation under
> directory foodir) all the files in our directory named datadir EXCEPT
> those whose file-extension is '.not' .  That's easy...:
> 
> """
> from distutils.core import setup
> 
> import glob
> 
> thefiles = []
> for afile in glob.glob('datadir/*'):
>     if afile.endswith('.not'): continue
>     thefiles.append(afile)
> 
> print "Packaging up", thefiles
> 
> setup(name='justdata', data_files = [ [ 'foodir', thefiles ] ] )
> """
> 
> Here's an example of how this works:
> 
> [alex at lancelot tryp]$ ls datadir
> aa.yes  bb.not  cc.yes
> [alex at lancelot tryp]$ python setup.py bdist
> Packaging up ['datadir/aa.yes', 'datadir/cc.yes']
> running bdist
> running bdist_dumb
> running build
> installing to build/bdist.linux-i686/dumb
> running install
> running install_data
> creating build/bdist.linux-i686/dumb
> creating build/bdist.linux-i686/dumb/usr
> creating build/bdist.linux-i686/dumb/usr/local
> creating build/bdist.linux-i686/dumb/usr/local/foodir
> copying datadir/aa.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
> copying datadir/cc.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
> tar -cf /x/tryp/dist/justdata-0.0.0.linux-i686.tar .
> gzip -f9 /x/tryp/dist/justdata-0.0.0.linux-i686.tar
> removing 'build/bdist.linux-i686/dumb' (and everything under it)
> [alex at lancelot tryp]$ tar tvzf dist/justdata-0.0.0.linux-i686.tar.gz
> drwxrwxr-x alex/alex         0 2003-10-01 14:25:40 ./
> drwxrwxr-x alex/alex         0 2003-10-01 14:25:40 ./usr/
> drwxrwxr-x alex/alex         0 2003-10-01 14:25:40 ./usr/local/
> drwxrwxr-x alex/alex         0 2003-10-01 14:25:40 ./usr/local/foodir/
> -rw-rw-r-- alex/alex         0 2003-10-01 14:13:48 ./usr/local/foodir/aa.yes
> -rw-rw-r-- alex/alex         0 2003-10-01 14:13:55 ./usr/local/foodir/cc.yes
> [alex at lancelot tryp]$
> 
> 
> > 3.  python setup.py sdist   will make a package of SOURCE code and
> >     using "bdist" will put BINARIES in package.  What does this
> > mean???
> >     Will bdist packages unpack to pyc (bytecode) files and sdist will
> > unpack
> >     py (noncompiled) files????  What if I wanted py and pyc files
> > installed
> >     from the package?? (i.e. source and compiled versions)
> 
> Hmmm, if the issue was so unclear (and important) to you, was it
> SO hard to try things out...?  Say that setup.py is:
> 
> from distutils.core import setup
> 
> setup(name='onemodule', py_modules=['hello'])
> 
> and hello.py in the same directory is:
> 
> def greet(name='World'):
>     return 'Hello, %s!' % name
> 
> Now, running python setup.py bdist gives the output:
> 
> [alex at lancelot tryp]$ python setup.py bdist
> running bdist
> running bdist_dumb
> running build
> running build_py
> creating build/lib
> copying hello.py -> build/lib
> installing to build/bdist.linux-i686/dumb
> running install
> running install_lib
> creating build/bdist.linux-i686/dumb
> creating build/bdist.linux-i686/dumb/usr
> creating build/bdist.linux-i686/dumb/usr/local
> creating build/bdist.linux-i686/dumb/usr/local/lib
> creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3
> creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
> copying build/lib/hello.py ->
> build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
> byte-compiling
> build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages/hello.py
> to hello.pyc
> tar -cf /x/tryp/dist/onemodule-0.0.0.linux-i686.tar .
> gzip -f9 /x/tryp/dist/onemodule-0.0.0.linux-i686.tar
> removing 'build/bdist.linux-i686/dumb' (and everything under it)
> 
> Notice that hello.py is copied into the directory to be packaged, as
> well as hello,pyc.  And indeed we can double check what's in the tarball:
> 
> [alex at lancelot tryp]$ tar tvzf dist/onemodule-0.0.0.linux-i686.tar.gz
> drwxrwxr-x alex/alex         0 2003-10-01 14:30:41 ./
> drwxrwxr-x alex/alex         0 2003-10-01 14:30:41 ./usr/
> drwxrwxr-x alex/alex         0 2003-10-01 14:30:41 ./usr/local/
> drwxrwxr-x alex/alex         0 2003-10-01 14:30:41 ./usr/local/lib/
> drwxrwxr-x alex/alex         0 2003-10-01 14:30:41
> ./usr/local/lib/python2.3/
> drwxrwxr-x alex/alex         0 2003-10-01 14:30:42
> ./usr/local/lib/python2.3/site-packages/
> -rw-rw-r-- alex/alex        56 2003-10-01 14:30:10
> ./usr/local/lib/python2.3/site-packages/hello.py
> -rw-rw-r-- alex/alex       335 2003-10-01 14:30:42
> ./usr/local/lib/python2.3/site-packages/hello.pyc
> [alex at lancelot tryp]$
> 
> See?  Both the .py AND the .pyc, as expected.
> 
> Should you want to REMOVE the .py files, for example, that can
> easily be achieved by operating directly on the tarball or zipfile,
> after distutils is finished (customizing distutils for such
> purposes, on the other hand, is harder).
> 
> 
> Alex