| Author: | Todd Greenwood |
| Title: | SimpleExampleEgg |
| Keywords: | python, egg, setuptools, distutils, tutorial, doctests, unit tests, deploy, install, example |
| Version: | 0.1 |
| License: | GPL v. 2 |
| Date: | Dec 2005 |
This aims to be a super simple example of python, setuptools, docutils, unit-tests, and eggs.
ContentsHopefully, this will clarify, in my own mind, at least, how to make super simple apps that have all sorts of cool extras such as working unit tests, launch scripts, deploy and install flawlessly...etc. Ideally, I'll be able to write apps that 'just work'. So my end users can just point some tool (easy_install) at a file (an egg) and voila, a working app.
Here's what I'd like to have happen for this exercise:
user clicks a link
app downloads and installs
user runs the app's internal test suite
Along the way, I'll play around with:
deploying the app locally, and to pipy
user may recieve the app via email, a web link, file copy, etc.
doctests and unit tests
Note, this documentation was written using Mikolaj Machowski's Vim reStructured Text plugin (vst). It is the coolest thing since the 'intraweb'. Look at the source 'readme.rest' and the generated 'readme.html'. The code listings and sample code output were generated as part of the build, very little 'copy and paste' here.
dir listing:
apple.py docs __init__.py orange.py simpletests.py
apple.py:
import sys def doConsole(): print make_pie('CONSOLE') def make_pie(who): """ >>> import apple as a >>> a.make_pie('Todd') 'Todd likes pie!!!' """ return '%s likes pie!!!'%who # def _test(): import doctest return doctest.testmod() # if __name__ == "__main__": _test() if len(sys.argv)>1: print make_pie(sys.argv[1])
Let's run this tiny app to see what we get:
$ python apple.py Mr.Guido
Mr.Guido likes pie!!!
setup.py:
from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages # setup(name = "SimpleExampleEgg", version = "0.1", description = "test", author = "Todd Greenwood", author_email = "t.greenwoodgeer@gmail.com", entry_points = {'console_scripts': [ 'make_apple_pie = fruit.apple:doConsole' ]}, packages = find_packages(exclude=['ez_setup'] ), package_data = {'':['docs/*.html', 'docs/*.rest','docs/*.vim']}, test_suite = 'fruit.simpletests.getTestSuite', license = "GNU Lesser General Public License", classifiers = [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)", "Programming Language :: Python", "Topic :: Utilities", ], zip_safe=True, )
simpletests.py:
import apple import unittest import doctest # def getTestSuite(): suite = unittest.TestSuite() for mod in apple,: suite.addTest(doctest.DocTestSuite(mod)) return suite # runner = unittest.TextTestRunner() runner.run(getTestSuite())
Let's run the test suite:
$ python simpletests.py
. ---------------------------------------------------------------------- Ran 1 test in 0.007s OK
Now, let's run the test suite from setuptools:
$ python setup.py test
running test running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt writing entry points to ./SimpleExampleEgg.egg-info/entry_points.txt running build_ext . ---------------------------------------------------------------------- Ran 1 test in 0.008s OK Doctest: fruit.apple.make_pie ... ok ---------------------------------------------------------------------- Ran 1 test in 0.001s OK
Next, let's make sure the build system works. But first, I should mention that this documentation is generated via a vim pluggin.
So, while from a command line you would execute:
$ python setup bdist_eggBecause this doc lives in ./fruit/docs/readme.rest, I will use the pbu utility to walk the directory tree and launch setup.py. If you would like to build the docs, here is the process:
install pbu:
$ easy_install buildutilsbuild the egg using the 'pbu' utility:
$ cd ./fruit/docs $ pbu dbist_egg
So that's exactly what the docs do. So, to see all this in action, build the docs :
$ vim readme.rest -s runvim.vim BTW - runvim.vim is just a quickie script to execute :Vsti within vim, write, save, and exit vim.
Back to the build system, let's do a build :
running bdist_egg running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt writing entry points to ./SimpleExampleEgg.egg-info/entry_points.txt installing library code to build/bdist.linux-i686/egg running install_lib warning: install_lib: 'build/lib' does not exist -- no Python modules to install creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/entry_points.txt -> build/bdist.linux-i686/egg/EGG-INFO creating 'dist/SimpleExampleEgg-0.1-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it)
And lastly, let's check to make sure that we have what we expect in the output:
``unzip -l /home/tgreenwo/active/SimpleExampleEgg/dist/SimpleExampleEgg-0.1-py2.4.egg'``
Archive: /home/tgreenwo/active/SimpleExampleEgg/dist/SimpleExampleEgg-0.1-py2.4.egg Length Date Time Name -------- ---- ---- ---- 290 12-14-05 12:11 fruit/orange.py 330 12-14-05 15:33 fruit/apple.py 240 12-14-05 12:26 fruit/simpletests.py 0 12-14-05 12:13 fruit/__init__.py 556 12-14-05 16:24 fruit/orange.pyc 754 12-14-05 16:24 fruit/apple.pyc 537 12-14-05 16:24 fruit/simpletests.pyc 124 12-14-05 16:24 fruit/__init__.pyc 18896 12-14-05 14:54 fruit/docs/readme.html 8768 12-14-05 16:02 fruit/docs/readme.rest 141 12-14-05 12:36 fruit/docs/vstrc.vim 14 12-14-05 14:32 fruit/docs/runvim.vim 489 12-14-05 16:24 EGG-INFO/PKG-INFO 6 12-14-05 16:24 EGG-INFO/top_level.txt 58 12-14-05 16:24 EGG-INFO/entry_points.txt 0 12-14-05 16:24 EGG-INFO/zip-safe -------- ------- 31203 16 files
Some things I should mention....
apply the svn 'trick' to get the ez_setup file
get a freebie ftp account to test uploading to
create a pipy account so that you can upload there, too
Per the Peak website, install ez_setup.py as an svn external thingy. Here are the steps:
$ cd SimpleExampleEgg/ $ svn propedit svn:externals . # enter this in the editor: ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup # slurp down the latest ez_setup file $ svn update # you should see the ez_setup dir now..., if you don't, try $ svn propget svn:externals ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup
It would be nice to be able to create a project that just housed the ez_setup project, and then just reference that project in all my other projects. However, apparently svn does not support chained external dependencies. So each project will either have to have a copy of ez_setup.py, or this external reference.
this part is easy:
$ python setup.py register
now just follow the instructions..:
running register We need to know who you are, so please choose either: 1. use your existing login, 2. register as a new user, 3. have the server generate a new password for you (and email it to you), or 4. quit Your selection [default 1]:
you'll need your username and password the first time you deploy. subsequent deploys will store your log in info somewhere in your home dir
Now, we want to do your basic 'app' stuff:
script generation
build
deploy
test
The idea here is that if I have some cool command line app, then how do I access it when it's all tucked inside an egg? The generated wrapper scripts do just that.
add entry points to setup.py:
setup(
...
entry_points = {'console_scripts': [
'make_apple_pie = fruit.apple:make_pie'
]},
note that you cannot pass command line vars to the method:
tgreenwo@luxor~/active/SimpleExampleEgg$ make_apple_pie
Traceback (most recent call last):
File "/usr/bin/make_apple_pie", line 7, in ?
sys.exit(
TypeError: make_pie() takes exactly 1 argument (0 given)
tgreenwo@luxor~/active/SimpleExampleEgg$ python make_apple_pie
python: can't open file 'make_apple_pie': [Errno 2] No such file or
directory
tgreenwo@luxor~/active/SimpleExampleEgg$ which make_apple_pie
/usr/bin/make_apple_pie
tgreenwo@luxor~/active/SimpleExampleEgg$ cat /usr/bin/make_apple_pie
#!/home/tgreenwo/bin/python
# EASY-INSTALL-ENTRY-SCRIPT:
# 'SimpleExampleEgg==0.1','console_scripts','make_apple_pie'
__requires__ = 'SimpleExampleEgg==0.1'
import sys
from pkg_resources import load_entry_point
sys.exit(
load_entry_point('SimpleExampleEgg==0.1', 'console_scripts',
'make_apple_pie')()
)
add a method to apple.py w/o params:
#apple.py
def doConsole(): print make_pie('CONSOLE')
#setup.py
entry_points = {'console_scripts': [
'make_apple_pie = fruit.apple:doConsole'
]},
install (as root) again
easy_install dist/SimpleExampleEgg-0.1-py2.4.egg
and run again:
tgreenwo@luxor~/active/SimpleExampleEgg$ /usr/bin/make_apple_pie CONSOLE likes pie!!!
We did this already, but, we'll do it again here.
Build the app into an egg:
$ python setup bdist_egg
running bdist_egg running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt writing entry points to ./SimpleExampleEgg.egg-info/entry_points.txt installing library code to build/bdist.linux-i686/egg running install_lib warning: install_lib: 'build/lib' does not exist -- no Python modules to install creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/entry_points.txt -> build/bdist.linux-i686/egg/EGG-INFO creating 'dist/SimpleExampleEgg-0.1-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it)
And lastly, let's check to make sure that we have what we expect:
$ unzip -l /home/tgreenwo/active/SimpleExampleEgg/dist/SimpleExampleEgg-0.1-py2.4.egg
Archive: /home/tgreenwo/active/SimpleExampleEgg/dist/SimpleExampleEgg-0.1-py2.4.egg Length Date Time Name -------- ---- ---- ---- 290 12-14-05 12:11 fruit/orange.py 330 12-14-05 15:33 fruit/apple.py 240 12-14-05 12:26 fruit/simpletests.py 0 12-14-05 12:13 fruit/__init__.py 556 12-14-05 16:24 fruit/orange.pyc 754 12-14-05 16:24 fruit/apple.pyc 537 12-14-05 16:24 fruit/simpletests.pyc 124 12-14-05 16:24 fruit/__init__.pyc 18896 12-14-05 14:54 fruit/docs/readme.html 8768 12-14-05 16:02 fruit/docs/readme.rest 141 12-14-05 12:36 fruit/docs/vstrc.vim 14 12-14-05 14:32 fruit/docs/runvim.vim 489 12-14-05 16:24 EGG-INFO/PKG-INFO 6 12-14-05 16:24 EGG-INFO/top_level.txt 58 12-14-05 16:24 EGG-INFO/entry_points.txt 0 12-14-05 16:24 EGG-INFO/zip-safe -------- ------- 31203 16 files
create a deployment directory to house the egg links:
$ cd working $ mkdir deploy
add the 'deploy' directory to your pythonpath:
export PYTHONPATH=/home/tgreenwo/working/twisted:/home/tgreenwo/working/django:/home/tgreenwo/working/deploy
install the egg locally in dev mode:
$ python setup.py develop
running develop running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt writing entry points to ./SimpleExampleEgg.egg-info/entry_points.txt running build_ext Creating /usr/lib/python2.4/site-packages/SimpleExampleEgg.egg-link (link to .) error: /usr/lib/python2.4/site-packages/SimpleExampleEgg.egg-link: Permission denied
this time, we specify the deploy directory:
$ python setup.py develop -d /home/tgreenwo/working/deploy
running develop running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt writing entry points to ./SimpleExampleEgg.egg-info/entry_points.txt running build_ext Creating /home/tgreenwo/data/working/deploy/SimpleExampleEgg.egg-link (link to .) Installing make_apple_pie script to /home/tgreenwo/working/deploy Installed /home/tgreenwo/active/SimpleExampleEgg/fruit/docs Because this distribution was installed --multi-version or --install-dir, before you can import modules from this package in an application, you will need to 'import pkg_resources' and then use a 'require()' call similar to one of these examples, in order to select the desired version: pkg_resources.require("SimpleExampleEgg") # latest installed version pkg_resources.require("SimpleExampleEgg==0.1") # this exact version pkg_resources.require("SimpleExampleEgg>=0.1") # this version or higher Processing dependencies for SimpleExampleEgg==0.1
build and deploy in one step:
$ setup.py register sdist bdist_egg upload
I had problems, so I enabled the following:
$ python setup.py register sdist bdist_egg upload --show-response
Once you noodle thru all the errors for your log in, password, and whether all the meta information like url and topic are set, then the egg will upload.
General steps are:
su or sudo
easy_install [egg name] or
easy_install [pipy project name]
run unit tests
Simple. One step:
root$ easy_install SimpleExampleEgg Searching for SimpleExampleEgg Reading http://www.python.org/pypi/SimpleExampleEgg/ Best match: SimpleExampleEgg 0.1 Downloading http://cheeseshop.python.org/packages/2.4/S/SimpleExampleEgg/SimpleExampleEgg-0.1-py2.4.egg#md5=d53e38ea496e03d30b632dc08265128b Processing SimpleExampleEgg-0.1-py2.4.egg Moving SimpleExampleEgg-0.1-py2.4.egg to /usr/lib/python2.4/site-packages Adding SimpleExampleEgg 0.1 to easy-install.pth file Installing make_apple_pie script to /usr/bin Installed /usr/lib/python2.4/site-packages/SimpleExampleEgg-0.1-py2.4.egg Processing dependencies for SimpleExampleEgg
done
this should run the unit tests for the installed app:
$ python -c "from unittest import main; main(None)" fruit.simpletests.getTestSuite
let's try that:
python -c 'from unittest import main; main(None)' fruit.simpletests.getTestSuite
. ---------------------------------------------------------------------- Ran 1 test in 0.007s OK . ---------------------------------------------------------------------- Ran 1 test in 0.001s OK
vim + vst rocks
eggs are cool