-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm new to distutils/and the egg system. I have not researched the archives as much as I should, so please excuse me if this is familiar territory... Q: Is it possible for dependency checks to work for non pipy deployed eggs? - A.egg - B.egg (setup_requires and install_requires A.egg) - email A and B eggs to friend - tell friend to : easy_install B - but B.egg does not install A...in fact, in my test case, it fails b/c it cannot find A in pipy - in this example, what happens if friend does not have net connection on target machine? how can they install from the eggs alone and get the dep checking to work? Q: How does my friend run the unittests on the egg that they have just installed? As I am developing, I frequently run the unittests and after every bdist_egg build, I also run the unit tests that way, too. How can the consumer do the same? Q: One of my apps is a command line app, say foo.py is a command line app that takes arg1, arg2,... How can the user run this app when it is encased in an egg? Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up? Thanks in advance. - -Todd -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDl9Hfz6uXX4lQc/URAnY4AJ47DFHL/cpEltmqoUyGqzhsK5FjRACgt01v akc3Wh3IOXVv2jiS8TszhuE= =Rv/x -----END PGP SIGNATURE-----
At 10:25 PM 12/7/2005 -0800, tgreenwood wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'm new to distutils/and the egg system. I have not researched the archives as much as I should, so please excuse me if this is familiar territory...
Q: Is it possible for dependency checks to work for non pipy deployed eggs? - A.egg - B.egg (setup_requires and install_requires A.egg) - email A and B eggs to friend - tell friend to : easy_install B - but B.egg does not install A...in fact, in my test case, it fails b/c it cannot find A in pipy
This will work if the eggs are both in the current directory: easy_install -f. B -f (aka --find-links) specifies URLs or directories that will be scanned *before* going to PyPI to meet dependencies.
- in this example, what happens if friend does not have net connection on target machine? how can they install from the eggs alone and get the dep checking to work?
No net connection is necessary if all dependencies can be found using -f directories.
Q: How does my friend run the unittests on the egg that they have just installed? As I am developing, I frequently run the unittests and after every bdist_egg build, I also run the unit tests that way, too. How can the consumer do the same?
Well, if it's a unit test suite that's contained within the egg, then they could use the unittest utility just as they normally would.
Q: One of my apps is a command line app, say foo.py is a command line app that takes arg1, arg2,... How can the user run this app when it is encased in an egg?
EasyInstall generates wrapper scripts during installation that ensure the needed eggs are on sys.path and run the scripts. This works for both standard distutils scripts (specified using setup(scripts=[...])) and for entry point scripts (see the setuptools doc). The --script-dir argument to easy_install controls where scripts are installed; it defaults to the same as the installation directory if an installation directory is manually specified; otherwise it defaults to wherever the distutils are configured to install scripts.
Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up?
Using -f with the URL of a web page with download links to your eggs or source distributions. See http://turbogears.org/download/ for one example. In the simplest case, a simple Apache directory listing will do the trick; see http://peak.telecommunity.com/snapshots/ for an example of that approach. Simply having someone use a URL like that with -f ensures that any download links on the page will be given precedence over PyPI listings, if it's possible to meet all requirements that way. Only if a dependency can't be satisfied by links on a -f page, will PyPI be consulted.
On 8 Dec 2005 at 11:54, Phillip J. Eby wrote:
Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up?
Using -f with the URL of a web page with download links to your eggs or source distributions. See http://turbogears.org/download/ for one
Awsome, I need the same thing too. What if the url passed to -f requires authentication? -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com AOL-IM or SKYPE: BKClements
At 01:27 PM 12/8/2005 -0500, Brad Clements wrote:
On 8 Dec 2005 at 11:54, Phillip J. Eby wrote:
Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up?
Using -f with the URL of a web page with download links to your eggs or source distributions. See http://turbogears.org/download/ for one
Awsome, I need the same thing too.
What if the url passed to -f requires authentication?
Then it won't work. :) Urllib doesn't provide login prompts by default. I imagine the feature could be added, though, and I wouldn't be averse to including support, someone wants to implement it.
Great, thanks Phillip. Now that you mention the part about the scripts generation, I remember reading that in the docs. Thanks for clearing things up. Regarding the unit tests...let me add a bit more to the question. Typically, as I'm developing a module, I embed doctests in the module. This lets me maintain a known good state throughout my dev cycle. Then, as I start to wrap things up, I create a standard unit test to wrap the doctests. Finally, I reference a method in the unit test that gives back the test suite (code sample attached). I reference this test suite in the setup.py for the given egg. So the question is, given that my friend/client is receiving eggs, how can they perform the equivalent of '$python setup.py test'? Currently, I have suggested that they unzip the egg and run the unittests dirctly from the unzipped files...something like this: - unzip foo.egg - cd foo - python src/foo/unittest.py I'm attaching rest and html formatted copies of my blog tutorial entry on using setup tools...feel free to comment on that as well. small.py:: #example method with doctest compatible docstring def removeme(): """ >>> import small as m >>> m.removeme() 'remove me' """ return 'remove me' def _test(): import doctest return doctest.testmod() if __name__ == "__main__": _test() small_unittest.py:: import small import unittest import doctest def getTestSuite(): suite = unittest.TestSuite() for mod in small,: suite.addTest(doctest.DocTestSuite(mod)) return suite runner = unittest.TextTestRunner() runner.run(getTestSuite()) setup.py:: from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages setup(name = "small", version = "0.1", description = "test", author = "Todd Greenwood", author_email = "t.greenwoodgeer@gmail.com", url = "http://www.angelfire.com/planet/tango", packages = find_packages('src'), package_dir = {'':'src'}, package_data = {'small':['test/*.txt']}, test_suite = 'small.small_unittest.getTestSuite', license = "GNU Lesser General Public License", classifiers = [ "Development Status :: 1 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)", "Programming Language :: Python", "Topic :: Database :: Front-Ends", ] ) -Todd Phillip J. Eby wrote:
At 10:25 PM 12/7/2005 -0800, tgreenwood wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'm new to distutils/and the egg system. I have not researched the archives as much as I should, so please excuse me if this is familiar territory...
Q: Is it possible for dependency checks to work for non pipy deployed eggs? - A.egg - B.egg (setup_requires and install_requires A.egg) - email A and B eggs to friend - tell friend to : easy_install B - but B.egg does not install A...in fact, in my test case, it fails b/c it cannot find A in pipy
This will work if the eggs are both in the current directory:
easy_install -f. B
-f (aka --find-links) specifies URLs or directories that will be scanned *before* going to PyPI to meet dependencies.
- in this example, what happens if friend does not have net connection on target machine? how can they install from the eggs alone and get the dep checking to work?
No net connection is necessary if all dependencies can be found using -f directories.
Q: How does my friend run the unittests on the egg that they have just installed? As I am developing, I frequently run the unittests and after every bdist_egg build, I also run the unit tests that way, too. How can the consumer do the same?
Well, if it's a unit test suite that's contained within the egg, then they could use the unittest utility just as they normally would.
Q: One of my apps is a command line app, say foo.py is a command line app that takes arg1, arg2,... How can the user run this app when it is encased in an egg?
EasyInstall generates wrapper scripts during installation that ensure the needed eggs are on sys.path and run the scripts. This works for both standard distutils scripts (specified using setup(scripts=[...])) and for entry point scripts (see the setuptools doc).
The --script-dir argument to easy_install controls where scripts are installed; it defaults to the same as the installation directory if an installation directory is manually specified; otherwise it defaults to wherever the distutils are configured to install scripts.
Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up?
Using -f with the URL of a web page with download links to your eggs or source distributions. See http://turbogears.org/download/ for one example. In the simplest case, a simple Apache directory listing will do the trick; see http://peak.telecommunity.com/snapshots/ for an example of that approach. Simply having someone use a URL like that with -f ensures that any download links on the page will be given precedence over PyPI listings, if it's possible to meet all requirements that way.
Only if a dependency can't be satisfied by links on a -f page, will PyPI be consulted.
===================== Small Tutorial ===================== :Author: Todd Greenwood :Title: Small Tutorial -- How to use DocTest, Unittest, and Python Packages :Keywords: :Version: 1.0 :License: GPL v. 2 :Date: Dec 01, 2005 I was struggling with creating a simple python test case that uses doctest, then unittest, then python *eggs*. Along the way I found out some things with respect to python packages. This is meant as a beginner example (and to help keep things fresh in my mind). I am writing this using Vim7 and Mikolaj Machowskihe's vst.vim_ plugin. .. contents:: :depth: 3 Introduction ------------ The tutorial is composed of a very simple test class, small.py, that is modified in several small ways. Basically, there are two examples here: * Simple Case : No Nested Packages * Complex Case : Nested Packages Simple Case ----------- Here is what the src tree for the simple example looks like: dir listing:: small/src/small/test/test.txt small/src/small/small2.py small/src/small/small.py small/src/small/small_unittest.pyc small/src/small/__init__.py small/src/small/small_unittest.py small/src/small_unittest.py small/setup_mult.py small/ez_setup.py small/setup.py Modules +++++++ We have two test classes, small and small2.py. They look like this: small.py:: #example method with doctest compatible docstring def removeme(): """ >>> import small as m >>> m.removeme() 'remove me' """ return 'remove me' def _test(): import doctest return doctest.testmod() if __name__ == "__main__": _test() small2.py is almost identical, with the only difference in the method: small2.py:: def saveme(): """ >>> import small2 as m >>> m.saveme() 'save me' """ return 'save me' Doctests ++++++++ These two test modules show a simple use of the doctest_ module. These have to have some glue to the standard python unittests in order for the python egg *test* functionality to work. So, we'll add a python class to grok the modules and spit out unittest stuff. Unittests +++++++++ small_unittest.py:: import small, small2 import unittest import doctest def getTestSuite(): suite = unittest.TestSuite() for mod in small,small2,: suite.addTest(doctest.DocTestSuite(mod)) return suite runner = unittest.TextTestRunner() runner.run(getTestSuite()) One of the things I *thought* would work would be to just import small, and the statement ``for mod in small,:`` would find small.small and small.small2. Not the case. So I explicitly state the modules to test as ``for mod in small,small2,:``. Also note that we define a ``getTestSuite()`` method. This is handy for setting up the python setuptools tests (shown below). Run this test from ``small/src/small/`` and you get: ``small/src/small$ python small_unittest.py``:: .. ---------------------------------------------------------------------- Ran 2 tests in 0.004s OK Now cd to ``small/src`` and check out the unit test there: small_unittest.py:: from small import small as s1 from small import small2 as s2 import unittest import doctest def getTestSuite(): suite = unittest.TestSuite() for mod in s1,s2,: suite.addTest(doctest.DocTestSuite(mod)) return suite runner = unittest.TextTestRunner() runner.run(getTestSuite()) The thing to note here is how small.small and small.small2 are referenced:: from small import small as s1 from small import small2 as s2 For some reason, this gave me plenty of problems. Hence this tutorial. Setuptools ++++++++++ Now cd to ``small`` and check out the python egg stuff: setup.py:: from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages setup(name = "small", version = "0.1", description = "test", author = "Todd Greenwood", author_email = "t.greenwoodgeer@gmail.com", url = "http://www.angelfire.com/planet/tango", packages = find_packages('src'), package_dir = {'':'src'}, package_data = {'small':['test/*.txt']}, test_suite = 'small.small_unittest.getTestSuite', license = "GNU Lesser General Public License", classifiers = [ "Development Status :: 1 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)", "Programming Language :: Python", "Topic :: Database :: Front-Ends", ] ) Referencing Modules ******************* The thing to notice here is this line:: test_suite = 'small.small_unittest.getTestSuite', This shows how to reference the modules, and how to get the test suite from the unittest. Also note the package_data line...that's how you include other files in the package. See peak_ for more. Check out the peak_ website for more info on setuptools and the egg utilities. For now, we'll do two things, run the tests, and then create an egg distribution. Running Tests ************* Run the tests: ``small$ python setup.py test``:: running test running egg_info writing src/small.egg-info/PKG-INFO writing top-level names to src/small.egg-info/top_level.txt running build_ext .. ---------------------------------------------------------------------- Ran 2 tests in 0.005s OK Doctest: small.small.removeme ... ok Doctest: small.small2.saveme ... ok ---------------------------------------------------------------------- Ran 2 tests in 0.005s OK Building a Distribution (egg) ***************************** Create a distribution with ``python setup.py bdist_egg``: ``small$ python setup.py bdist_egg``:: running bdist_egg running egg_info writing src/small.egg-info/PKG-INFO writing top-level names to src/small.egg-info/top_level.txt installing library code to build/bdist.linux-i686/egg running install_lib running build_py copying src/small/small2.py -> build/lib/small copying src/small/small.py -> build/lib/small copying src/small/__init__.py -> build/lib/small copying src/small/small_unittest.py -> build/lib/small copying src/small/test/test.txt -> build/lib/small/test creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/.svn creating build/bdist.linux-i686/egg/.svn/text-base creating build/bdist.linux-i686/egg/.svn/prop-base creating build/bdist.linux-i686/egg/.svn/props creating build/bdist.linux-i686/egg/.svn/wcprops creating build/bdist.linux-i686/egg/.svn/tmp creating build/bdist.linux-i686/egg/.svn/tmp/text-base creating build/bdist.linux-i686/egg/.svn/tmp/prop-base creating build/bdist.linux-i686/egg/.svn/tmp/props creating build/bdist.linux-i686/egg/.svn/tmp/wcprops copying build/lib/.svn/entries -> build/bdist.linux-i686/egg/.svn copying build/lib/.svn/empty-file -> build/bdist.linux-i686/egg/.svn copying build/lib/.svn/README.txt -> build/bdist.linux-i686/egg/.svn copying build/lib/.svn/format -> build/bdist.linux-i686/egg/.svn creating build/bdist.linux-i686/egg/small creating build/bdist.linux-i686/egg/small/.svn creating build/bdist.linux-i686/egg/small/.svn/text-base copying build/lib/small/.svn/text-base/small.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/text-base copying build/lib/small/.svn/text-base/__init__.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/text-base copying build/lib/small/.svn/text-base/small_unittest.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/text-base creating build/bdist.linux-i686/egg/small/.svn/prop-base copying build/lib/small/.svn/prop-base/small.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/prop-base copying build/lib/small/.svn/prop-base/__init__.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/prop-base copying build/lib/small/.svn/prop-base/small_unittest.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/prop-base creating build/bdist.linux-i686/egg/small/.svn/props copying build/lib/small/.svn/props/small.py.svn-work -> build/bdist.linux-i686/egg/small/.svn/props copying build/lib/small/.svn/props/__init__.py.svn-work -> build/bdist.linux-i686/egg/small/.svn/props copying build/lib/small/.svn/props/small_unittest.py.svn-work -> build/bdist.linux-i686/egg/small/.svn/props creating build/bdist.linux-i686/egg/small/.svn/wcprops creating build/bdist.linux-i686/egg/small/.svn/tmp creating build/bdist.linux-i686/egg/small/.svn/tmp/text-base creating build/bdist.linux-i686/egg/small/.svn/tmp/prop-base creating build/bdist.linux-i686/egg/small/.svn/tmp/props creating build/bdist.linux-i686/egg/small/.svn/tmp/wcprops copying build/lib/small/.svn/entries -> build/bdist.linux-i686/egg/small/.svn copying build/lib/small/.svn/empty-file -> build/bdist.linux-i686/egg/small/.svn copying build/lib/small/.svn/README.txt -> build/bdist.linux-i686/egg/small/.svn copying build/lib/small/.svn/format -> build/bdist.linux-i686/egg/small/.svn creating build/bdist.linux-i686/egg/small/test creating build/bdist.linux-i686/egg/small/test/.svn creating build/bdist.linux-i686/egg/small/test/.svn/text-base copying build/lib/small/test/.svn/text-base/test.txt.svn-base -> build/bdist.linux-i686/egg/small/test/.svn/text-base creating build/bdist.linux-i686/egg/small/test/.svn/prop-base copying build/lib/small/test/.svn/prop-base/test.txt.svn-base -> build/bdist.linux-i686/egg/small/test/.svn/prop-base creating build/bdist.linux-i686/egg/small/test/.svn/props copying build/lib/small/test/.svn/props/test.txt.svn-work -> build/bdist.linux-i686/egg/small/test/.svn/props creating build/bdist.linux-i686/egg/small/test/.svn/wcprops creating build/bdist.linux-i686/egg/small/test/.svn/tmp creating build/bdist.linux-i686/egg/small/test/.svn/tmp/text-base creating build/bdist.linux-i686/egg/small/test/.svn/tmp/prop-base creating build/bdist.linux-i686/egg/small/test/.svn/tmp/props creating build/bdist.linux-i686/egg/small/test/.svn/tmp/wcprops copying build/lib/small/test/.svn/entries -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/.svn/empty-file -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/.svn/README.txt -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/.svn/format -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/test.txt -> build/bdist.linux-i686/egg/small/test copying build/lib/small/__init__.py -> build/bdist.linux-i686/egg/small copying build/lib/small/small.py -> build/bdist.linux-i686/egg/small copying build/lib/small/small2.py -> build/bdist.linux-i686/egg/small copying build/lib/small/small_unittest.py -> build/bdist.linux-i686/egg/small byte-compiling build/bdist.linux-i686/egg/small/__init__.py to __init__.pyc byte-compiling build/bdist.linux-i686/egg/small/small.py to small.pyc byte-compiling build/bdist.linux-i686/egg/small/small2.py to small2.pyc byte-compiling build/bdist.linux-i686/egg/small/small_unittest.py to small_unittest.pyc creating build/bdist.linux-i686/egg/EGG-INFO copying src/small.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO copying src/small.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating 'dist/small-0.1-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it) Complex Case ------------ The complex case splits out the test modules into a nested package hierarchy like this: Package Hierarchy +++++++++++++++++ package hierarchy:: small/src/ small/src/size/ small/src/size/small small/src/size/large The dir listing below should make this clear: dir listing:: small/mult_tree_src/size/small/test/test.txt small/mult_tree_src/size/small/small.pyc small/mult_tree_src/size/small/small.py small/mult_tree_src/size/small/small_unittest.pyc small/mult_tree_src/size/small/__init__.py small/mult_tree_src/size/small/__init__.pyc small/mult_tree_src/size/small/small_unittest.py small/mult_tree_src/size/large small/mult_tree_src/size/large/large_unittest.py small/mult_tree_src/size/large/large.py small/mult_tree_src/size/large/large.pyc small/mult_tree_src/size/large/__init__.py small/mult_tree_src/size/large/__init__.pyc small/mult_tree_src/size/__init__.py small/mult_tree_src/size/__init__.pyc small/mult_tree_src/small-mult-test.egg-info small/mult_tree_src/small-mult-test.egg-info/top_level.txt small/mult_tree_src/small-mult-test.egg-info/PKG-INFO small/mult_tree_src/size_unittest.pyc small/mult_tree_src/size_unittest.py The Example Apps ++++++++++++++++ small/mult_tree_src/size/small/small.py:: #example method with doctest compatible docstring def smallme(): """ >>> import small as m >>> m.smallme() 'small me' """ return 'small me' def _test(): import doctest return doctest.testmod() if __name__ == "__main__": _test() The only difference between small.py and large.py is the name of the method. small/mult_tree_src/size/large/large.py:: def largeme(): """ >>> import large as l >>> l.largeme() 'large me' """ return 'large me' Unit Tests ++++++++++ Again, the only difference between the unit tests is the import and the name of the module adedd to the test suite. small/mult_tree_src/size/small/small_unittest.py:: import small import unittest import doctest def getTestSuite(): suite = unittest.TestSuite() for mod in small,: suite.addTest(doctest.DocTestSuite(mod)) return suite runner = unittest.TextTestRunner() runner.run(getTestSuite()) small/mult_tree_src/size/large/large_unittest.py:: import large import unittest import doctest def getTestSuite(): suite = unittest.TestSuite() for mod in large,: suite.addTest(doctest.DocTestSuite(mod)) return suite runner = unittest.TextTestRunner() runner.run(getTestSuite()) The unittests for size are: small/mult_tree_src/size_unittest.py:: from size.small import small from size.large import large import unittest import doctest def getTestSuite(): suite = unittest.TestSuite() for mod in small,large,: suite.addTest(doctest.DocTestSuite(mod)) return suite runner = unittest.TextTestRunner() runner.run(getTestSuite()) So, again, the thing to look at is how the import works (and again, the thing that threw me for awhile): note:: from size.small import small from size.large import large SetupTools ++++++++++ small/setup_mult.py:: from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages setup(name = "small_mult_test", version = "0.1", description = "test", author = "Todd Greenwood", author_email = "t.greenwoodgeer@gmail.com", url = "http://www.angelfire.com/planet/tango", packages = find_packages('mult_tree_src'), package_dir = {'':'mult_tree_src'}, package_data = {'':['test/*.txt']}, test_suite = 'size_unittest.getTestSuite', license = "GNU Lesser General Public License", classifiers = [ "Development Status :: 1 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)", "Programming Language :: Python", "Topic :: Database :: Front-Ends", ] ) Things to note: * test_suite : again, we reference the fx that gives us the test suite to run, in this case, it is:: def getTestSuite(): suite = unittest.TestSuite() for mod in small,large,: suite.addTest(doctest.DocTestSuite(mod)) return suite * package data : i'm not doing much with that now, but some tests in other packages that I'm writing place test data there. * url : need to see if that works for downloading dependencies (next tutorial, not this one) Running Tests ************* Run the tests for size : ``small$ python setup_mult.py test``:: running test running egg_info writing mult_tree_src/small-mult-test.egg-info/PKG-INFO writing top-level names to mult_tree_src/small-mult-test.egg-info/top_level.txt running build_ext .. ---------------------------------------------------------------------- Ran 2 tests in 0.009s OK Doctest: size.small.small.smallme ... ok Doctest: size.large.large.largeme ... ok ---------------------------------------------------------------------- Ran 2 tests in 0.002s OK Building Distributions (eggs) ***************************** Build the egg dist for size: ``small$ python setup_mult.py bdist_egg``:: running bdist_egg running egg_info writing mult_tree_src/small-mult-test.egg-info/PKG-INFO writing top-level names to mult_tree_src/small-mult-test.egg-info/top_level.txt installing library code to build/bdist.linux-i686/egg running install_lib running build_py creating build/lib/size copying mult_tree_src/size/__init__.py -> build/lib/size creating build/lib/size/small copying mult_tree_src/size/small/small.py -> build/lib/size/small copying mult_tree_src/size/small/__init__.py -> build/lib/size/small copying mult_tree_src/size/small/small_unittest.py -> build/lib/size/small creating build/lib/size/large copying mult_tree_src/size/large/large_unittest.py -> build/lib/size/large copying mult_tree_src/size/large/large.py -> build/lib/size/large copying mult_tree_src/size/large/__init__.py -> build/lib/size/large creating build/lib/size/small/test copying mult_tree_src/size/small/test/test.txt -> build/lib/size/small/test creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/.svn creating build/bdist.linux-i686/egg/.svn/text-base creating build/bdist.linux-i686/egg/.svn/prop-base creating build/bdist.linux-i686/egg/.svn/props creating build/bdist.linux-i686/egg/.svn/wcprops creating build/bdist.linux-i686/egg/.svn/tmp creating build/bdist.linux-i686/egg/.svn/tmp/text-base creating build/bdist.linux-i686/egg/.svn/tmp/prop-base creating build/bdist.linux-i686/egg/.svn/tmp/props creating build/bdist.linux-i686/egg/.svn/tmp/wcprops copying build/lib/.svn/entries -> build/bdist.linux-i686/egg/.svn copying build/lib/.svn/empty-file -> build/bdist.linux-i686/egg/.svn copying build/lib/.svn/README.txt -> build/bdist.linux-i686/egg/.svn copying build/lib/.svn/format -> build/bdist.linux-i686/egg/.svn creating build/bdist.linux-i686/egg/small creating build/bdist.linux-i686/egg/small/.svn creating build/bdist.linux-i686/egg/small/.svn/text-base copying build/lib/small/.svn/text-base/small.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/text-base copying build/lib/small/.svn/text-base/__init__.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/text-base copying build/lib/small/.svn/text-base/small_unittest.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/text-base creating build/bdist.linux-i686/egg/small/.svn/prop-base copying build/lib/small/.svn/prop-base/small.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/prop-base copying build/lib/small/.svn/prop-base/__init__.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/prop-base copying build/lib/small/.svn/prop-base/small_unittest.py.svn-base -> build/bdist.linux-i686/egg/small/.svn/prop-base creating build/bdist.linux-i686/egg/small/.svn/props copying build/lib/small/.svn/props/small.py.svn-work -> build/bdist.linux-i686/egg/small/.svn/props copying build/lib/small/.svn/props/__init__.py.svn-work -> build/bdist.linux-i686/egg/small/.svn/props copying build/lib/small/.svn/props/small_unittest.py.svn-work -> build/bdist.linux-i686/egg/small/.svn/props creating build/bdist.linux-i686/egg/small/.svn/wcprops creating build/bdist.linux-i686/egg/small/.svn/tmp creating build/bdist.linux-i686/egg/small/.svn/tmp/text-base creating build/bdist.linux-i686/egg/small/.svn/tmp/prop-base creating build/bdist.linux-i686/egg/small/.svn/tmp/props creating build/bdist.linux-i686/egg/small/.svn/tmp/wcprops copying build/lib/small/.svn/entries -> build/bdist.linux-i686/egg/small/.svn copying build/lib/small/.svn/empty-file -> build/bdist.linux-i686/egg/small/.svn copying build/lib/small/.svn/README.txt -> build/bdist.linux-i686/egg/small/.svn copying build/lib/small/.svn/format -> build/bdist.linux-i686/egg/small/.svn creating build/bdist.linux-i686/egg/small/test creating build/bdist.linux-i686/egg/small/test/.svn creating build/bdist.linux-i686/egg/small/test/.svn/text-base copying build/lib/small/test/.svn/text-base/test.txt.svn-base -> build/bdist.linux-i686/egg/small/test/.svn/text-base creating build/bdist.linux-i686/egg/small/test/.svn/prop-base copying build/lib/small/test/.svn/prop-base/test.txt.svn-base -> build/bdist.linux-i686/egg/small/test/.svn/prop-base creating build/bdist.linux-i686/egg/small/test/.svn/props copying build/lib/small/test/.svn/props/test.txt.svn-work -> build/bdist.linux-i686/egg/small/test/.svn/props creating build/bdist.linux-i686/egg/small/test/.svn/wcprops creating build/bdist.linux-i686/egg/small/test/.svn/tmp creating build/bdist.linux-i686/egg/small/test/.svn/tmp/text-base creating build/bdist.linux-i686/egg/small/test/.svn/tmp/prop-base creating build/bdist.linux-i686/egg/small/test/.svn/tmp/props creating build/bdist.linux-i686/egg/small/test/.svn/tmp/wcprops copying build/lib/small/test/.svn/entries -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/.svn/empty-file -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/.svn/README.txt -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/.svn/format -> build/bdist.linux-i686/egg/small/test/.svn copying build/lib/small/test/test.txt -> build/bdist.linux-i686/egg/small/test copying build/lib/small/__init__.py -> build/bdist.linux-i686/egg/small copying build/lib/small/small.py -> build/bdist.linux-i686/egg/small copying build/lib/small/small2.py -> build/bdist.linux-i686/egg/small copying build/lib/small/small_unittest.py -> build/bdist.linux-i686/egg/small creating build/bdist.linux-i686/egg/size copying build/lib/size/__init__.py -> build/bdist.linux-i686/egg/size creating build/bdist.linux-i686/egg/size/small copying build/lib/size/small/small.py -> build/bdist.linux-i686/egg/size/small copying build/lib/size/small/__init__.py -> build/bdist.linux-i686/egg/size/small copying build/lib/size/small/small_unittest.py -> build/bdist.linux-i686/egg/size/small creating build/bdist.linux-i686/egg/size/small/test copying build/lib/size/small/test/test.txt -> build/bdist.linux-i686/egg/size/small/test creating build/bdist.linux-i686/egg/size/large copying build/lib/size/large/large_unittest.py -> build/bdist.linux-i686/egg/size/large copying build/lib/size/large/large.py -> build/bdist.linux-i686/egg/size/large copying build/lib/size/large/__init__.py -> build/bdist.linux-i686/egg/size/large byte-compiling build/bdist.linux-i686/egg/small/__init__.py to __init__.pyc byte-compiling build/bdist.linux-i686/egg/small/small.py to small.pyc byte-compiling build/bdist.linux-i686/egg/small/small2.py to small2.pyc byte-compiling build/bdist.linux-i686/egg/small/small_unittest.py to small_unittest.pyc byte-compiling build/bdist.linux-i686/egg/size/__init__.py to __init__.pyc byte-compiling build/bdist.linux-i686/egg/size/small/small.py to small.pyc byte-compiling build/bdist.linux-i686/egg/size/small/__init__.py to __init__.pyc byte-compiling build/bdist.linux-i686/egg/size/small/small_unittest.py to small_unittest.pyc byte-compiling build/bdist.linux-i686/egg/size/large/large_unittest.py to large_unittest.pyc byte-compiling build/bdist.linux-i686/egg/size/large/large.py to large.pyc byte-compiling build/bdist.linux-i686/egg/size/large/__init__.py to __init__.pyc creating build/bdist.linux-i686/egg/EGG-INFO copying mult_tree_src/small-mult-test.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO copying mult_tree_src/small-mult-test.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating 'dist/small_mult_test-0.1-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it) Ok, that's it for now. -Todd .. Links .. _doctest: http://docs.python.org/lib/module-doctest.html .. _peak: http://peak.telecommunity.com/DevCenter/PythonEggs
At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote: small_unittest.py::
import small import unittest import doctest
def getTestSuite(): suite = unittest.TestSuite() for mod in small,: suite.addTest(doctest.DocTestSuite(mod)) return suite
runner = unittest.TextTestRunner() runner.run(getTestSuite())
If 'small.small_unittest' is the actual name of the module, then doing this: python -c "from unittest import main; main(None)" small.small_unittest.getTestSuite will run the test, even if it's inside an egg. No unzipping is necessary. If the egg wasn't installed with a .pth file (i.e. --multi-version or to a non-standard location), you'll need to make sure it's on PYTHONPATH, along with its dependencies.
Here is what the src tree for the simple example looks like:
dir listing::
small/src/small/test/test.txt small/src/small/small2.py small/src/small/small.py small/src/small/small_unittest.pyc small/src/small/__init__.py small/src/small/small_unittest.py small/src/small_unittest.py
This looks like a duplicate. I'm also not sure why you have a small2, or why you bother having a separate test subdirectory. Indeed, for this "small" of an example, a src/ directory is probably overkill. For clarity, I'd also suggest calling the topmost directory something like SmallExample to help distinguish your project name (SmallExample) from your package name (small).
small/setup_mult.py
What's this file for?
small/ez_setup.py
If you're using Subversion, btw, you can use the svn:externals trick so that you don't have to manually maintain this file; see the setuptools manual under: http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-...
The thing to note here is how small.small and small.small2 are referenced::
from small import small as s1 from small import small2 as s2
For some reason, this gave me plenty of problems.
It's generally not a good idea to name a module the same as a package, and definitely not the same as a class within the module, or else it gets unclear in code which one you're talking about. In a language like Java, there's no such ambiguity because syntactically you can't refer to a class where a package should be or vice versa, but in Python there are only objects, so you should not name them all the same thing or you (and/or the interpreter) will be confused. :) Similarly, I suspect that your example has way too many files perhaps because you have a Java background? It really isn't necessary to split Python projects up into so many small pieces; it just makes more work for you and doesn't get you any benefit until the files get too large to conveniently work with.
Complex Case ------------
The complex case splits out the test modules into a nested package hierarchy like this: ... Package Hierarchy +++++++++++++++++
package hierarchy::
small/src/ small/src/size/ small/src/size/small small/src/size/large
Ouch. I thought the previous example was the complex one. ;) Seriously, your simple example is way more complex than it needs to be. This bigger one makes my head hurt, so I'm not going to try to comment on it further, except to suggest that its top-level directory should be named "small-mult-test" since that's your project name. Also, I'm guessing you mean your package hiearchy is size, size.small, and size.large. The small/src stuff isn't part of the package hierarchy, just the directory hierarchy. By the way, the only reason to have a 'src' directory under your main project directory is if you have a bunch of stuff that's *not* source code there. Such as documentation, documentation directories, test directories, examples, etc. Your projects here have none of those things, only a setup script, so there's no reason to have a 'src' subdirectory; you could just put the 'small' or 'size' package directly under the project directory, and in fact this is the most common layout for published Python packages. The 'src' or 'lib' subdirectory approach is mainly used by large projects with a lot of other things in their project directory, and by people who got used to that format by working on large projects. :)
from size.small import small from size.large import large
I said I wasn't going to comment further, but this is technically a repetition of my earlier comment: please don't name packages and modules the same thing, you will confuse yourself and everyone else who will never be sure if 'small' means 'size.small' or 'size.small.small' or worse, 'size.small.small.small'. Eeek!
* url : need to see if that works for downloading dependencies (next tutorial, not this one)
Yes, it does, *if* you register your package with PyPI. The URL given will be scanned for download links when and if easy_install looks for your package on PyPI.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip - This is great feedback. I wrote this tutorial for myself because, in the course of writing a more involved application, I realized that I did not really understand how to use the egg/setuptools functionality. So, the example is definitely contrived. So, if I may summarize your guidelines: 1. Add complexity as needed ~ a) keep source in the root dir unless there is a need to differentiate ~ disparate types of files, such as source/docs/tests/etc ~ b) use the svn copy of ez_setup.py rather than checking this in to the repository 2. Clear use of names ~ a) use different names for packages and modules ~ b) again, nest only as necessary And, above all else, never admit to having a java background ;o). Thanks again. I'll re-factor as suggested and throw this at you one more time. - -Todd Phillip J. Eby wrote: | At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote: | small_unittest.py:: | |> import small |> import unittest |> import doctest |> |> def getTestSuite(): |> suite = unittest.TestSuite() |> for mod in small,: |> suite.addTest(doctest.DocTestSuite(mod)) |> return suite |> |> runner = unittest.TextTestRunner() |> runner.run(getTestSuite()) | | If 'small.small_unittest' is the actual name of the module, then doing this: | | python -c "from unittest import main; main(None)" | small.small_unittest.getTestSuite | | will run the test, even if it's inside an egg. No unzipping is | necessary. If the egg wasn't installed with a .pth file (i.e. | --multi-version or to a non-standard location), you'll need to make sure | it's on PYTHONPATH, along with its dependencies. | |> Here is what the src tree for the simple example looks like: |> |> dir listing:: |> |> small/src/small/test/test.txt |> small/src/small/small2.py |> small/src/small/small.py |> small/src/small/small_unittest.pyc |> small/src/small/__init__.py |> small/src/small/small_unittest.py |> small/src/small_unittest.py | | This looks like a duplicate. I'm also not sure why you have a small2, or | why you bother having a separate test subdirectory. Indeed, for this | "small" of an example, a src/ directory is probably overkill. For clarity, | I'd also suggest calling the topmost directory something like SmallExample | to help distinguish your project name (SmallExample) from your package name | (small). | |> small/setup_mult.py | | What's this file for? | | |> small/ez_setup.py | | If you're using Subversion, btw, you can use the svn:externals trick so | that you don't have to manually maintain this file; see the setuptools | manual under: | | http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-... | | | |> The thing to note here is how small.small and small.small2 are referenced:: |> |> from small import small as s1 |> from small import small2 as s2 |> |> For some reason, this gave me plenty of problems. | | It's generally not a good idea to name a module the same as a package, and | definitely not the same as a class within the module, or else it gets | unclear in code which one you're talking about. In a language like Java, | there's no such ambiguity because syntactically you can't refer to a class | where a package should be or vice versa, but in Python there are only | objects, so you should not name them all the same thing or you (and/or the | interpreter) will be confused. :) | | Similarly, I suspect that your example has way too many files perhaps | because you have a Java background? It really isn't necessary to split | Python projects up into so many small pieces; it just makes more work for | you and doesn't get you any benefit until the files get too large to | conveniently work with. | | |> Complex Case |> ------------ |> |> The complex case splits out the test modules into a nested package |> hierarchy like this: | ... |> Package Hierarchy |> +++++++++++++++++ |> |> package hierarchy:: |> |> small/src/ |> small/src/size/ |> small/src/size/small |> small/src/size/large | | Ouch. I thought the previous example was the complex one. ;) Seriously, | your simple example is way more complex than it needs to be. This bigger | one makes my head hurt, so I'm not going to try to comment on it further, | except to suggest that its top-level directory should be named | "small-mult-test" since that's your project name. Also, I'm guessing you | mean your package hiearchy is size, size.small, and size.large. The | small/src stuff isn't part of the package hierarchy, just the directory | hierarchy. | | By the way, the only reason to have a 'src' directory under your main | project directory is if you have a bunch of stuff that's *not* source code | there. Such as documentation, documentation directories, test directories, | examples, etc. Your projects here have none of those things, only a setup | script, so there's no reason to have a 'src' subdirectory; you could just | put the 'small' or 'size' package directly under the project directory, and | in fact this is the most common layout for published Python packages. The | 'src' or 'lib' subdirectory approach is mainly used by large projects with | a lot of other things in their project directory, and by people who got | used to that format by working on large projects. :) | | |> from size.small import small |> from size.large import large | | I said I wasn't going to comment further, but this is technically a | repetition of my earlier comment: please don't name packages and modules | the same thing, you will confuse yourself and everyone else who will never | be sure if 'small' means 'size.small' or 'size.small.small' or worse, | 'size.small.small.small'. Eeek! | | |> * url : need to see if that works for downloading dependencies (next |> tutorial, not this one) | | Yes, it does, *if* you register your package with PyPI. The URL given will | be scanned for download links when and if easy_install looks for your | package on PyPI. | | -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDmRzIz6uXX4lQc/URAsaWAKCXNcWjf7o59m2MgM2B8Nmx1CfLewCgpqOW wA2wfi08C4RPW151NngP7BU= =N5e6 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Perhaps the docs could flesh out a bit more on the svn/ez_setup trick. As an example of a full set of instructions: - - mkdir testApp - - touch testApp/foo.py - - svn import testApp file:///....path to my repos - - cd to some working area - - svn co file:///path-to-repos/testApp testApp - - cd testApp - - svn propedit svn:externals . - - enter this in to the editor, save and exit >>> ~ ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup - - modify setup.py like so: """ from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages setup(name = "testApp", ~ version = "0.1", ~ description = "test", ~ packages = find_packages(exclude=['ez_setup'], ''), ~ package_dir = {'':''}, ~ classifiers = [ ~ "Development Status :: 1 - Alpha", ~ "Intended Audience :: Developers", ~ "License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)", ~ "Programming Language :: Python", ~ "Topic :: Database :: Front-Ends", ~ ] ~ ) """ - - check to see your mods to the externals : svn propget svn:externals - - get the ez_setup files : svn update - - do a dir listing, you should see: ~ ./ez_setup/ ~ ./ez_setup/__init__.py ~ ./ez_setup/README.txt - - Todd Todd Greenwood-Geer wrote: | Phillip - | | This is great feedback. I wrote this tutorial for myself because, in the | course of writing a more involved application, I realized that I did not | really understand how to use the egg/setuptools functionality. So, the | example is definitely contrived. | | So, if I may summarize your guidelines: | | 1. Add complexity as needed | ~ a) keep source in the root dir unless there is a need to differentiate | ~ disparate types of files, such as source/docs/tests/etc | ~ b) use the svn copy of ez_setup.py rather than checking this in to the | repository | | 2. Clear use of names | ~ a) use different names for packages and modules | ~ b) again, nest only as necessary | | And, above all else, never admit to having a java background ;o). | | Thanks again. I'll re-factor as suggested and throw this at you one more | time. | | -Todd | | Phillip J. Eby wrote: | | At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote: | | small_unittest.py:: | | | |> import small | |> import unittest | |> import doctest | |> | |> def getTestSuite(): | |> suite = unittest.TestSuite() | |> for mod in small,: | |> suite.addTest(doctest.DocTestSuite(mod)) | |> return suite | |> | |> runner = unittest.TextTestRunner() | |> runner.run(getTestSuite()) | | | | If 'small.small_unittest' is the actual name of the module, then doing | this: | | | | python -c "from unittest import main; main(None)" | | small.small_unittest.getTestSuite | | | | will run the test, even if it's inside an egg. No unzipping is | | necessary. If the egg wasn't installed with a .pth file (i.e. | | --multi-version or to a non-standard location), you'll need to make sure | | it's on PYTHONPATH, along with its dependencies. | | | |> Here is what the src tree for the simple example looks like: | |> | |> dir listing:: | |> | |> small/src/small/test/test.txt | |> small/src/small/small2.py | |> small/src/small/small.py | |> small/src/small/small_unittest.pyc | |> small/src/small/__init__.py | |> small/src/small/small_unittest.py | |> small/src/small_unittest.py | | | | This looks like a duplicate. I'm also not sure why you have a small2, or | | why you bother having a separate test subdirectory. Indeed, for this | | "small" of an example, a src/ directory is probably overkill. For | clarity, | | I'd also suggest calling the topmost directory something like | SmallExample | | to help distinguish your project name (SmallExample) from your package | name | | (small). | | | |> small/setup_mult.py | | | | | What's this file for? | | | | | |> small/ez_setup.py | | | | If you're using Subversion, btw, you can use the svn:externals trick so | | that you don't have to manually maintain this file; see the setuptools | | manual under: | | | | | http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-... | | | | | | | |> The thing to note here is how small.small and small.small2 are | referenced:: | |> | |> from small import small as s1 | |> from small import small2 as s2 | |> | |> For some reason, this gave me plenty of problems. | | | | It's generally not a good idea to name a module the same as a package, | and | | definitely not the same as a class within the module, or else it gets | | unclear in code which one you're talking about. In a language like Java, | | there's no such ambiguity because syntactically you can't refer to a | class | | where a package should be or vice versa, but in Python there are only | | objects, so you should not name them all the same thing or you (and/or | the | | interpreter) will be confused. :) | | | | Similarly, I suspect that your example has way too many files perhaps | | because you have a Java background? It really isn't necessary to split | | Python projects up into so many small pieces; it just makes more work for | | you and doesn't get you any benefit until the files get too large to | | conveniently work with. | | | | | |> Complex Case | |> ------------ | |> | |> The complex case splits out the test modules into a nested package | |> hierarchy like this: | | ... | |> Package Hierarchy | |> +++++++++++++++++ | |> | |> package hierarchy:: | |> | |> small/src/ | |> small/src/size/ | |> small/src/size/small | |> small/src/size/large | | | | Ouch. I thought the previous example was the complex one. ;) | Seriously, | | your simple example is way more complex than it needs to be. This bigger | | one makes my head hurt, so I'm not going to try to comment on it further, | | except to suggest that its top-level directory should be named | | "small-mult-test" since that's your project name. Also, I'm guessing you | | mean your package hiearchy is size, size.small, and size.large. The | | small/src stuff isn't part of the package hierarchy, just the directory | | hierarchy. | | | | By the way, the only reason to have a 'src' directory under your main | | project directory is if you have a bunch of stuff that's *not* source | code | | there. Such as documentation, documentation directories, test | directories, | | examples, etc. Your projects here have none of those things, only a | setup | | script, so there's no reason to have a 'src' subdirectory; you could just | | put the 'small' or 'size' package directly under the project | directory, and | | in fact this is the most common layout for published Python packages. | The | | 'src' or 'lib' subdirectory approach is mainly used by large projects | with | | a lot of other things in their project directory, and by people who got | | used to that format by working on large projects. :) | | | | | |> from size.small import small | |> from size.large import large | | | | I said I wasn't going to comment further, but this is technically a | | repetition of my earlier comment: please don't name packages and modules | | the same thing, you will confuse yourself and everyone else who will | never | | be sure if 'small' means 'size.small' or 'size.small.small' or worse, | | 'size.small.small.small'. Eeek! | | | | | |> * url : need to see if that works for downloading dependencies (next | |> tutorial, not this one) | | | | Yes, it does, *if* you register your package with PyPI. The URL given | will | | be scanned for download links when and if easy_install looks for your | | package on PyPI. | | | | -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDmSc4z6uXX4lQc/URAoVlAKCzwzH1wr3G9x7nzPOFeWdn97oU/QCgjrqO q9gT1aNL+ZagzWeoxqnVQIk= =L/oc -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 PROBLEM: I would like to be able to execute the egg build from several directories, like so: $ cd active/SimpleExampleEgg $ python setup.py bdist_egg then, at other times, i'd like to change to a package directory and execute the builds from there: $ cd active/SimpleExampleEgg/fruit/docs $ python ../../setup.py bdist_egg this places the output in active/SimpleExampleEgg/fruit/docs (build, dist), which is fine by me. REASON: I'm executing builds as part of my documentation process using vim and a restructured text pluggin to vim. Everything seems to hang together a bit better if I execute the vim/rest code from the directory it is in, [project]/fruit/docs, in this case. ALTERNATIVE: I suppose I could stay in the project root directory. This means I'd have to include my documentation via resource_filename(...), and is not the end of the world. BUG: Perhaps this is exposing an interesting bug. I would expect the egg creation to either be identical for these two cases, or fail altogether for the second case. REPRO: greenwo@luxor~/active/SimpleExampleEgg$ python setup.py bdist_egg running bdist_egg running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt installing library code to build/bdist.linux-i686/egg running install_lib running build_py creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/fruit copying build/lib/fruit/orange.py -> build/bdist.linux-i686/egg/fruit copying build/lib/fruit/apple.py -> build/bdist.linux-i686/egg/fruit copying build/lib/fruit/__init__.py -> build/bdist.linux-i686/egg/fruit copying build/lib/fruit/simpletests.py -> build/bdist.linux-i686/egg/fruit creating build/bdist.linux-i686/egg/fruit/docs copying build/lib/fruit/docs/traditional.css -> build/bdist.linux-i686/egg/fruit/docs copying build/lib/fruit/docs/readme.rest -> build/bdist.linux-i686/egg/fruit/docs copying build/lib/fruit/docs/readme.html -> build/bdist.linux-i686/egg/fruit/docs copying build/lib/fruit/docs/default.css -> build/bdist.linux-i686/egg/fruit/docs copying build/lib/fruit/docs/oldstyle.css -> build/bdist.linux-i686/egg/fruit/docs copying build/lib/fruit/docs/vstrc.vim -> build/bdist.linux-i686/egg/fruit/docs byte-compiling build/bdist.linux-i686/egg/fruit/orange.py to orange.pyc byte-compiling build/bdist.linux-i686/egg/fruit/apple.py to apple.pyc byte-compiling build/bdist.linux-i686/egg/fruit/__init__.py to __init__.pyc byte-compiling build/bdist.linux-i686/egg/fruit/simpletests.py to simpletests.pyc creating build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO creating 'dist/SimpleExampleEgg-0.1-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it) tgreenwo@luxor~/active/SimpleExampleEgg$ unzip -l dist/SimpleExampleEgg-0.1-py2.4.egg Archive: dist/SimpleExampleEgg-0.1-py2.4.egg ~ Length Date Time Name ~ -------- ---- ---- ---- ~ 290 12-14-05 12:11 fruit/orange.py ~ 286 12-14-05 12:11 fruit/apple.py ~ 0 12-14-05 12:13 fruit/__init__.py ~ 240 12-14-05 12:26 fruit/simpletests.py ~ 556 12-14-05 12:47 fruit/orange.pyc ~ 625 12-14-05 12:47 fruit/apple.pyc ~ 124 12-14-05 12:47 fruit/__init__.pyc ~ 537 12-14-05 12:47 fruit/simpletests.pyc ~ 14863 12-14-05 12:36 fruit/docs/traditional.css ~ 5079 12-14-05 12:43 fruit/docs/readme.rest ~ 12958 12-14-05 12:43 fruit/docs/readme.html ~ 6464 12-14-05 12:43 fruit/docs/default.css ~ 14140 12-14-05 12:36 fruit/docs/oldstyle.css ~ 141 12-14-05 12:36 fruit/docs/vstrc.vim ~ 532 12-14-05 12:47 EGG-INFO/PKG-INFO ~ 6 12-14-05 12:47 EGG-INFO/top_level.txt ~ 0 12-14-05 12:47 EGG-INFO/zip-safe ~ -------- ------- ~ 56841 17 files tgreenwo@luxor~/active/SimpleExampleEgg$ cd fruit/docs/ tgreenwo@luxor~/active/SimpleExampleEgg/fruit/docs$ python ../../setup.py bdist_egg running bdist_egg running egg_info writing ./SimpleExampleEgg.egg-info/PKG-INFO writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt installing library code to build/bdist.linux-i686/egg running install_lib warning: install_lib: 'build/lib' does not exist -- no Python modules to install creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO copying ./SimpleExampleEgg.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO creating 'dist/SimpleExampleEgg-0.1-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it) tgreenwo@luxor~/active/SimpleExampleEgg/fruit/docs$ unzip -l dist/SimpleExampleEgg-0.1-py2.4.egg Archive: dist/SimpleExampleEgg-0.1-py2.4.egg ~ Length Date Time Name ~ -------- ---- ---- ---- ~ 532 12-14-05 12:48 EGG-INFO/PKG-INFO ~ 1 12-14-05 12:48 EGG-INFO/top_level.txt ~ 0 12-14-05 12:48 EGG-INFO/zip-safe ~ -------- ------- ~ 533 3 files tgreenwo@luxor~/active/SimpleExampleEgg/fruit/docs$ - -Todd -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDoIepz6uXX4lQc/URAmBeAJ9MNkw8rvTpZedpH7seUY8kjA6QKgCfe+sK D/zaLhfsLcju0dOUMDOVZyo= =jtO/ -----END PGP SIGNATURE-----
At 12:59 PM 12/14/2005 -0800, Todd Greenwood-Geer wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
PROBLEM: I would like to be able to execute the egg build from several directories, like so:
$ cd active/SimpleExampleEgg $ python setup.py bdist_egg
then, at other times, i'd like to change to a package directory and execute the builds from there:
$ cd active/SimpleExampleEgg/fruit/docs $ python ../../setup.py bdist_egg
Stop right there. The distutils do *not* support this, and neither does setuptools. 'setup.py' scripts *must* be run from the directory where they are located, so I didn't bother reading the rest of your message. If you want to run setup.py this way, you should 'easy_install buildutils' and then use its "pbu" command, e.g.: pbu bdist_egg The "pbu" command automatically searches parent directories for a setup.py, then changes to the directory where the setup.py was found, and runs it for you. This is the only safe way to do it. (When "pbu" exits, you will however still be in the directory you were in before.)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Great. The pbu utility is perfect. Thanks, yet again. Phillip J. Eby wrote: | At 12:59 PM 12/14/2005 -0800, Todd Greenwood-Geer wrote: |> -----BEGIN PGP SIGNED MESSAGE----- |> Hash: SHA1 |> |> PROBLEM: |> I would like to be able to execute the egg build from several |> directories, like so: |> |> $ cd active/SimpleExampleEgg |> $ python setup.py bdist_egg |> |> then, at other times, i'd like to change to a package directory and |> execute the builds from there: |> |> $ cd active/SimpleExampleEgg/fruit/docs |> $ python ../../setup.py bdist_egg | | Stop right there. The distutils do *not* support this, and neither does | setuptools. 'setup.py' scripts *must* be run from the directory where they | are located, so I didn't bother reading the rest of your message. | | If you want to run setup.py this way, you should 'easy_install buildutils' | and then use its "pbu" command, e.g.: | | pbu bdist_egg | | The "pbu" command automatically searches parent directories for a setup.py, | then changes to the directory where the setup.py was found, and runs it for | you. This is the only safe way to do it. | | (When "pbu" exits, you will however still be in the directory you were in | before.) | | -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDoJnhz6uXX4lQc/URAj1NAKDG6yXdR0w6Y3dqjIaKmZ9InjRN+QCeLe5s L2/6Bs3qn/rTjAPs6RrHn4k= =Z29e -----END PGP SIGNATURE-----
I've taken to heart your suggestions. The result is what I hope is a very simple example / tutorial that demonstrates: * the file structure of a project destined to be an egg * how to use doctests and unit tests with setuptools / eggs * how to build and deploy an egg * how to register with pypi * how to (and not to) generate wrapper scripts * how to write documentation using restructured text Instructions: $ easy_install SimpleExampleEgg That's it. You have now installed v 0.2. The tutorial docs are the egg: /usr/lib/python2.4/site-packages/SimpleExampleEgg-0.2-py2.4.egg:fruit/docs/readme.html Could I get a quick review of this? I'm attaching the readme.html. -Todd Phillip J. Eby wrote:
At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote: small_unittest.py::
import small import unittest import doctest
def getTestSuite(): suite = unittest.TestSuite() for mod in small,: suite.addTest(doctest.DocTestSuite(mod)) return suite
runner = unittest.TextTestRunner() runner.run(getTestSuite())
If 'small.small_unittest' is the actual name of the module, then doing this:
python -c "from unittest import main; main(None)" small.small_unittest.getTestSuite
will run the test, even if it's inside an egg. No unzipping is necessary. If the egg wasn't installed with a .pth file (i.e. --multi-version or to a non-standard location), you'll need to make sure it's on PYTHONPATH, along with its dependencies.
Here is what the src tree for the simple example looks like:
dir listing::
small/src/small/test/test.txt small/src/small/small2.py small/src/small/small.py small/src/small/small_unittest.pyc small/src/small/__init__.py small/src/small/small_unittest.py small/src/small_unittest.py
This looks like a duplicate. I'm also not sure why you have a small2, or why you bother having a separate test subdirectory. Indeed, for this "small" of an example, a src/ directory is probably overkill. For clarity, I'd also suggest calling the topmost directory something like SmallExample to help distinguish your project name (SmallExample) from your package name (small).
small/setup_mult.py
What's this file for?
small/ez_setup.py
If you're using Subversion, btw, you can use the svn:externals trick so that you don't have to manually maintain this file; see the setuptools manual under:
http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-...
The thing to note here is how small.small and small.small2 are referenced::
from small import small as s1 from small import small2 as s2
For some reason, this gave me plenty of problems.
It's generally not a good idea to name a module the same as a package, and definitely not the same as a class within the module, or else it gets unclear in code which one you're talking about. In a language like Java, there's no such ambiguity because syntactically you can't refer to a class where a package should be or vice versa, but in Python there are only objects, so you should not name them all the same thing or you (and/or the interpreter) will be confused. :)
Similarly, I suspect that your example has way too many files perhaps because you have a Java background? It really isn't necessary to split Python projects up into so many small pieces; it just makes more work for you and doesn't get you any benefit until the files get too large to conveniently work with.
Complex Case ------------
The complex case splits out the test modules into a nested package hierarchy like this: ... Package Hierarchy +++++++++++++++++
package hierarchy::
small/src/ small/src/size/ small/src/size/small small/src/size/large
Ouch. I thought the previous example was the complex one. ;) Seriously, your simple example is way more complex than it needs to be. This bigger one makes my head hurt, so I'm not going to try to comment on it further, except to suggest that its top-level directory should be named "small-mult-test" since that's your project name. Also, I'm guessing you mean your package hiearchy is size, size.small, and size.large. The small/src stuff isn't part of the package hierarchy, just the directory hierarchy.
By the way, the only reason to have a 'src' directory under your main project directory is if you have a bunch of stuff that's *not* source code there. Such as documentation, documentation directories, test directories, examples, etc. Your projects here have none of those things, only a setup script, so there's no reason to have a 'src' subdirectory; you could just put the 'small' or 'size' package directly under the project directory, and in fact this is the most common layout for published Python packages. The 'src' or 'lib' subdirectory approach is mainly used by large projects with a lot of other things in their project directory, and by people who got used to that format by working on large projects. :)
from size.small import small from size.large import large
I said I wasn't going to comment further, but this is technically a repetition of my earlier comment: please don't name packages and modules the same thing, you will confuse yourself and everyone else who will never be sure if 'small' means 'size.small' or 'size.small.small' or worse, 'size.small.small.small'. Eeek!
* url : need to see if that works for downloading dependencies (next tutorial, not this one)
Yes, it does, *if* you register your package with PyPI. The URL given will be scanned for download links when and if easy_install looks for your package on PyPI.
On Thu, 8 Dec 2005, Todd Greenwood-Geer wrote: [...]
the setup.py for the given egg. So the question is, given that my friend/client is receiving eggs, how can they perform the equivalent of '$python setup.py test'? Currently, I have suggested that they unzip the egg and run the unittests dirctly from the unzipped files...something like this:
- unzip foo.egg - cd foo - python src/foo/unittest.py [...]
1. To avoid confusion, rename your module unittest so as not to have the same name as Python stdlib module unittest (don't call it 'test' either, for the same reason -- module test contains *Python's* unittests; personally I find it unfortunate that standard Python comes with a module named 'test', but there it is). Let's say you call it 'tests'. 2. Make module tests importable from your own package, so one can do: from mypkg import tests 3. Use setuptools to provide a script that imports and runs module tests direct from the egg. John
participants (5)
-
Brad Clements -
John J Lee -
Phillip J. Eby -
tgreenwood -
Todd Greenwood-Geer