Mailman 3 February 2010 - Distutils-SIG

ImportError running 'test' on namespace package - other package in namespace not found
by Jason R. Coombs 17 Apr '20

17 Apr '20

I've got two projects: mynamespace.myprojectA and mynamespace.myprojectB myprojectB depends on myprojectA. I'm using setuptools 0.6c8 to manage both projects. Both projects are registered using 'setup develop'. Both projects are accessible from an interactive interpreter: PS C:\Users\me\projects> python Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import mynamespace.myprojectA >>> import mynamespace.myprojectB >>> from mynamespace.myprojectA import mymoduleZ However, when I run 'setup test' in myprojectB, the tests fail with File ".mymoduleZ.py", line NNN, in [some context] from mynamespace.myprojectA.mymoduleZ import MyClassC ImportError: No module named myprojectA.mymoduleZ In setup.py, the test_suite is nose.collector. I searched and couldn't find anyone else with this problem. Is this a supported configuration? Is there something I can do to make tests work with interdependent projects with the same root namespace? If there's not something obvious I should be doing differently, I'm happy to put together a minimal test case that reproduces the problem. Any suggestions are appreciated. Sincerely, Jason R. Coombs

3 6

Upgrading dependencies
by Charlie Moad 29 Apr '18

29 Apr '18

Is it currently possible to upgrade dependencies as well when upgrading a packages? If not, this would be a really nice feature to add to easy_install. Maybe a call like: easy_install --upgrade --upgrade-deps Package Obviously it makes sense to leave this off by default. Thanks, Charlie

2 1

Changes to --find-links and --upgrade in SVN
by Phillip J. Eby 29 Apr '18

29 Apr '18

Elvelind Grandin reported a problem with the "develop" command that turned out to be a flaw in its --find-links support. Specifically, it wasn't ever processing the links. :) In the process of fixing it, I wound up cleaning up an annoying (to me, at least) quirk of the previous workings of --find-links. It used to be that find-links would always be processed first, no matter what, even if you were doing a completely local operation. This would've been especially annoying if it carried over into "develop", so I made some changes. Now, if an item passed to --find-links is local (a filename or file: URL), or a direct link to an egg or other distribution, it is indexed immediately. Remote URLs are now only retrieved if a dependency can't be resolved locally, or if you use the -U or --upgrade options (this goes for "develop" too). Note that this is a behavior change for easy_install, which was effectively treating --find-links as though you'd specified --upgrade in certain cases. So, if you're used to getting upgrades downloaded as a result of using --find-links, please note that this will no longer happen. EasyInstall will now *only* go online if a dependency can't be resolved locally, if -U or --upgrade is used, or if you provided suitable direct URLs via an argument or --find-links, or via a link in a local .html file.

2 1

Problems with svn in windows
by Sharky On PTNet 29 Apr '18

29 Apr '18

Hi, I´m trying to do "easy_install setuptools==dev" in a windows box but have an error with svn command. """... 'svn' is not recognized as an internal or external command, ...""" I have installed TortoiseSVN but it seems that don't have command line, is all with the Explorer. Can you tell me what client I may use in windows? Thanks for all, Helio Pereira Sharky @ PTNet

3 2

local development scenarios
by tgreenwood 29 Apr '18

29 Apr '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm new to distutils/and the egg system. I have not researched the archives as much as I should, so please excuse me if this is familiar territory... Q: Is it possible for dependency checks to work for non pipy deployed eggs? - A.egg - B.egg (setup_requires and install_requires A.egg) - email A and B eggs to friend - tell friend to : easy_install B - but B.egg does not install A...in fact, in my test case, it fails b/c it cannot find A in pipy - in this example, what happens if friend does not have net connection on target machine? how can they install from the eggs alone and get the dep checking to work? Q: How does my friend run the unittests on the egg that they have just installed? As I am developing, I frequently run the unittests and after every bdist_egg build, I also run the unit tests that way, too. How can the consumer do the same? Q: One of my apps is a command line app, say foo.py is a command line app that takes arg1, arg2,... How can the user run this app when it is encased in an egg? Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up? Thanks in advance. - -Todd -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDl9Hfz6uXX4lQc/URAnY4AJ47DFHL/cpEltmqoUyGqzhsK5FjRACgt01v akc3Wh3IOXVv2jiS8TszhuE= =Rv/x -----END PGP SIGNATURE-----

5 12

Choice of dependencies?
by John J Lee 29 Apr '18

29 Apr '18

Is it possible for a package to depend on one of several packages, with the user having the option to pick? For example, my package P might depend on package A, plus either package B or package C. I know I could (ab)use extras_require, but in my case this is a real dependency rather than an optional extra. The other option I have is to just pick one of the dependencies (B or C in my example above), and thereby force people to install it even though it's not strictly required. Neither option seems particularly attractive. John

3 6

setuptools doc patch
by John J Lee 29 Apr '18

29 Apr '18

The comma removed by the patch below implies that users working from a Subversion checkout should not use EasyInstall to periodically fetch the latest version. That's not what you intended to say, I assume? http://peak.telecommunity.com/DevCenter/setuptools You should also inform your users of the need to run this command, if they -are working from a Subversion checkout, rather than using EasyInstall to +are working from a Subversion checkout rather than using EasyInstall to periodically fetch the latest version. Hmm, probably much better than the patch above: -You should also inform your users of the need to run this command, if they -are working from a Subversion checkout, rather than using EasyInstall to -periodically fetch the latest version. +If your users are working from a Subversion checkout rather than using +EasyInstall, you should also inform them of the need to run this command +to periodically fetch the latest version. John

2 2

Re: [Distutils] distutils install probs on suse 9.0
by Mats Wichmann 29 Apr '18

29 Apr '18

At 05:43 AM 5/18/2004, David Handy wrote: >On Tue, 18 May 2004, Floris van der Tak wrote: > >> my attempt to install the distutils on suse 9.0 (which provides python >> 2.3) fails: > >Why are you installing distutils on a system that already has Python 2.3 >installed? Python 2.3 normally comes with distutils as part of the Python >standard library. > >Did suse distribute a version of Python that had distutils removed? It should come from python-devel

2 1

Sample setup.py for Numeric Python
by Greg Ward 29 Apr '18

29 Apr '18

Hi again -- [cc'd to Paul Dubois: you said you weren't following the distutils sig anymore, but this directly concerns NumPy and I'd like to get your input!] here's that sample setup.py for NumPy. See below for discussion (and questions!). ------------------------------------------------------------------------ #!/usr/bin/env python # Setup script example for building the Numeric extension to Python. # This does sucessfully compile all the .dlls. Nothing happens # with the .py files currently. # Move this file to the Numerical directory of the LLNL numpy # distribution and run as: # python numpysetup.py --verbose build_ext # # created 1999/08 Perry Stoll __rcsid__ = "$Id: numpysetup.py,v 1.1 1999/09/12 20:42:48 gward Exp $" from distutils.core import setup setup (name = "numerical", version = "0.01", description = "Numerical Extension to Python", url = "http://www.python.org/sigs/matrix-sig/", ext_modules = [ ( '_numpy', { 'sources' : [ 'Src/_numpymodule.c', 'Src/arrayobject.c', 'Src/ufuncobject.c' ], 'include_dirs' : ['./Include'], 'def_file' : 'Src/numpy.def' } ), ( 'multiarray', { 'sources' : ['Src/multiarraymodule.c'], 'include_dirs' : ['./Include'], 'def_file': 'Src/multiarray.def' } ), ( 'umath', { 'sources': ['Src/umathmodule.c'], 'include_dirs' : ['./Include'], 'def_file' : 'Src/umath.def' } ), ( 'fftpack', { 'sources': ['Src/fftpackmodule.c', 'Src/fftpack.c'], 'include_dirs' : ['./Include'], 'def_file' : 'Src/fftpack.def' } ), ( 'lapack_lite', { 'sources' : [ 'Src/lapack_litemodule.c', 'Src/dlapack_lite.c', 'Src/zlapack_lite.c', 'Src/blas_lite.c', 'Src/f2c_lite.c' ], 'include_dirs' : ['./Include'], 'def_file' : 'Src/lapack_lite.def' } ), ( 'ranlib', { 'sources': ['Src/ranlibmodule.c', 'Src/ranlib.c', 'Src/com.c', 'Src/linpack.c', ], 'include_dirs' : ['./Include'], 'def_file' : 'Src/ranlib.def' } ), ] ) ------------------------------------------------------------------------ First, what d'you think? Too clunky and verbose? Too much information for each extension? I kind of think so, but I'm not sure how to reduce it elegantly. Right now, the internal data structures needed to compile a module are pretty obviously exposed: is this a good thing? Or should there be some more compact form for setup.py that will be expanded later into the full glory we see above? I've already made one small step towards reducing the amount of cruft by factoring 'include_dirs' out and supplying it directly as a parameter to 'setup()'. (But that needs code not in the CVS archive yet, so I've left the sample setup.py the same for now.) The next thing I'd like to do is get that damn "def_file" out of there. To support it in MSVCCompiler, there's already an ugly hack that unnecessarily affects both the UnixCCompiler and CCompiler classes, and I want to get rid of that. (I refer to passing the 'build_info' dictionary into the compiler classes, if you're familiar with the code -- that dictionary is part of the Distutils extension-building system, and should not propagate into the more general compiler classes.) But I don't want to give these weird "def file" things standing on the order of source files, object files, libraries, etc., because they seem to me to be a bizarre artifact of one particular compiler, rather than something present in a wide range of C/C++ compilers. Based on the NumPy model, it seems like there's a not-too-kludgy way to handle this problem. Namely: if building extension "foo": if file "foo.def" found in same directory as "foo.c" add "/def:foo.def" to MSVC command line this will of course require some platform-specific code in the build_ext command class, but I figured that was coming eventually, so why put it off? ;-) To make this hack work with NumPy, one change would be necessary: rename Src/numpy.def to Src/_numpy.def to match Src/_numpy.c, which implements the _numpy module. Would this be too much to ask of NumPy? (Paul?) What about other module distributions that support MSVC++ and thus ship with "def" files? Could they be made to accomodate this scheme? Thanks for your feedback -- Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

4 5

Can now compile simple extensions
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I found the time to hack in the ability to compile extension modules to the Distutils. Mainly, this meant adding a 'build_ext' command which uses a CCompiler instance for all its dirty work. I also had to add a few methods to CCompiler (and, of course, UnixCCompiler) to make this work. And I added a new module, 'spawn', which takes care of running sub-programs more efficiently and robustly (no shell involved) than os.system. That's needed, obviously, so we can run the compiler! If you're in the mood for grubbing over raw source code, then get the latest from CVS or download a current snapshot. See http://www.python.org/sigs/distutils-sig/implementation.html for a link to the code snapshot. I'm still waiting for more subclasses of CCompiler to appear. At the very least, we're going to need MSVCCompiler to build extensions on Windows. Any takers? Also, someone who knows the Mac, and how to run compilers programmatically there, will have to figure out how to write a Mac-specific concrete CCompiler subclass. The spawn module also needs a bit of work to be portable. I suspect that _win32_spawn() (the intended analog to my _posix_spawn()) will be easy to implement, if it even needs to go in a separate function at all. It looks from the Python Library documentation for 1.5.2 that the os.spawnv() function is all we need, but it's a bit hard to figure out just what's needed. Windows wizards, please take a look at the 'spawn()' function and see if you can make it work on Windows. As for actually compiling extensions: well, if you can figure out the build_ext command, go ahead and give it a whirl. It's a bit cryptic right now, since there's no documentation and no example setup.py. (I have a working example at home, but it's not available online.) If you feel up to it, though, see if you can read the code and figure out what's going on. I'm just hoping *I'll* be able to figure out what's going on when I get back from the O'Reilly conference next week... ;-) Enjoy -- Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

[buildout] branches languishing? (site-packages and distutils scripts)
by Chris Withers 02 Apr '12

02 Apr '12

Hi All, I'm wondering what the state of play is with the following branches: reinout-scripts gary-4-include-site-packages What more needs to happen for these to get merged to trunk and a release made? cheers, Chris

10 28

distribute 0.6.10 and convert_2to3_doctests
by Barry Warsaw 23 Apr '10

23 Apr '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I have a Python package called 'munepy' <https://launchpad.net/munepy> which provides yet another flavor of enums. I'm working on the code for various reasons and I thought I'd take the opportunity to learn how to support both Python 2 and 3 from the same code base. I've updated the code so that it supports Python 2.6 at a minimum, and made it 'python -3' clean. I've switched it from using setuptools to using distribute. I was looking at this page: http://packages.python.org/distribute/python3.html and it seems very cool that distribute can help make my life easier by allowing me to support both Python 2 and 3 from the same code base. I set use_2to3=True in my setup.py and indeed % python3 setup.py test (On Ubuntu 9.10) runs 2to3 over my .py files. However, my doctests are in separate .txt files and I cannot seem to get the right incantation for convert_2to3_doctests. In the root of my source directory, my doctest lives at munepy/docs/README.txt, so I put this in my setup.py: ... use_2to3 = True, convert_2to3_doctests = [ 'munepy/docs/README.txt', ], ... but I never see that the README.txt is ever 'fixed'. Indeed, the test fails because of a syntax error when the doctest tries to print something using Python 2 syntax. I'm clearly not using this setup argument correctly, but I can't tell where I'm going wrong. How do you use convert_2to3_doctests? TIA, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iQIcBAEBCAAGBQJLRRn+AAoJEBJutWOnSwa/U78P/06neAOc4cbeaKty7OX1AvUo 9QKwNrmH5U/spbvvThPKCl+XXPDJdD4K40zCSaq98vGXdjxIgBapCvQi3/lUBtKM M5CBVJvsbJnaeAZUjUS2EZ+CLXkMAmWA/8znmo3xsGJ+8+F77Wb9iTwobQpQqmWJ z43BLaiT62fzWqC4vsBjG6JMF0VZZZ51IRrzD48lasYItHa4cUQa/brC8jwGUhE4 060zo+cof5UvbMRNP0OU69p0M5PztOtszLKHaOhD8EsAF8r1PWtOT2oGpij8jLst GBp82J+JOlgWiEFSmVLaSOl/0D0193ut+PtHLCLDOJ9sVXgIKv4RvWMip1/0W1bB btneAIUJ/CFxCJuBcadkGtwS9geuaM+ma2rlYOpwTBncBEZx9XVPOxnEuCyUSrSo 03QXZ6xxOKf1Xepg3AjDD2KSxrAZHoG4aLVGdxqFyAebpC9hiKYRrLswe5mg/ycj eo5lpKjGplL5Md0O9ZgJmqk0UH4CVZy7Imrv8BMQcMCZLkbs42FnMXtXVWEVQ9qm iA3T54Yi5dOZB56JjnkftyS15in8W5OiV1X78WtblQ7/f5oR1UXq72mZU3k+3nrX kxWfzsHAo1v34+WHUagmYZqLYtm/zFT+zHwoyzzMcWpJSTH8WIOud9IuSkg5gA3S B2iMITFAZnXKzB644gIP =GqOL -----END PGP SIGNATURE-----

7 20

Re: [Distutils] RFC: Barely minimal CI + tests + packaging in Python WAS: Guidelines for where to put tests & how to package them
by P.J. Eby 11 Mar '10

11 Mar '10

At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote: >PS: BTW, how could I trigger easy_install(ation) at a given point >while implementing a distutils command, and let the command perform >further actions if deps are installed correctly ? Setuptools' "Distribution" object has a method for fetching dependencies. See setuptools' "test" command for an example. (This doesn't install the dependencies globally, just drops eggs into the build directory. But they're there and available to be reused for installation in a later phase, under normal circumstances.)

4 12

Fwd: The fate of Distutils in Python 2.7
by Tarek Ziadé 01 Mar '10

01 Mar '10

FIY (I didn't cross-post to make it easier for following the threads) ---------- Forwarded message ---------- From: Tarek Ziadé <ziade.tarek(a)gmail.com> Date: Fri, Feb 26, 2010 at 10:44 PM Subject: The fate of Distutils in Python 2.7 To: Python-Dev <python-dev(a)python.org> Hello, This is a follow-up of the Pycon summit + sprints on packaging. This is what we have planned to do: 1. refactor distutils in a new standalone version called distutils2 [this is done already and we are actively working in the code] 2. completely revert distutils in Lib/ and Doc/ so the code + doc is the same than the current 2.6.x branch 3. leave the new sysconfig module, that is used by the Makefile and the site module The rest of the work will happen in distutils2 and we will try to release a version asap for Python 2.x and 3.x (2.4 to 3.2), and the goal is to put it back in the stdlib in Python 3.3 Distutils in Python will be feature-frozen and I will only do bug fixes there. All feature requests will be redirected to Distutils2. I think the easiest way to manage this for me and for the feedback of the community is to add in bugs.python.org a "Distutils2" component, so I can start to reorganize the issues in there and reassign new issues to Distutils2 when it applies. Regards Tarek -- Tarek Ziadé | http://ziade.org -- Tarek Ziadé | http://ziade.org

3 6

Re: [Distutils] layout and setup.py for packaging documentation
by P.J. Eby 01 Mar '10

01 Mar '10

At 10:03 PM 2/28/2010 +0100, Jean Daniel wrote: >Can this be simpler? Yes. Don't install docs with your package. People who want them installed locally can just download your source install or use easy_install -e. Also, if your module is popular enough that people make Linux system packages for it, they will make sure the docs get put in a blessed install location. Python doesn't currently have a blessed install location for documentation, though perhaps it *should* have one in distutils2.

5 8

layout and setup.py for packaging documentation
by Jean Daniel 28 Feb '10

28 Feb '10

Hello, I am looking for the simplest way to package the html files related to a pure python module. At this point, I would like to just embed them in the source tarball generated by distribute and leave it to a debian or fedora package to install them. I would like a source tarball laid out as the following: wordish-1.0.2/ setup.py wordish.py docs/ index.html command-ref.html So far, I tended to use a single file python module because there was no need for more and I wanted to keep it simple: wordish/ setup.py wordish.py test_wordish.py The setup.py is minimal: setup( py_modules = [ 'wordish' ], ... ) But If I want to include docs, I must use the package_data command which expects a package name. So I think that to embed documentation in the source tarball, I should change the way I lay out the source repository to something like : wordish/ setup.py wordish wordish.py __init__.py docs/ index.html command-ref.html test_wordish.py The corresponding setup.py would be setup( packages = [ 'wordish' ], package_data = {'wordish':['docs/*']}, ... I am not sure this is the right way to do it, just to include documentations in the tarball: - the package_data requires a package name, so I turn my pure python module into a package just for this reason - package_data requires the data directory to be inside the package directory even though the html files are generated from doc sources located elsewhere, - there is an extra level of directory to get to the Python sources module, it is my_virtualenv/wordish/wordish/wordish.py which does not add much. Can this be simpler? Thanks for your help,

1 0

Re: [Distutils] [TIP] Guidelines for where to put tests & how to package them
by P.J. Eby 26 Feb '10

26 Feb '10

At 02:27 PM 2/26/2010 -0500, Olemis Lang wrote: >test_suite is ( supported | introduced | defined ) by >setuptools.comand.test.test class > > > Is it required to be a > > callable > >yep No. test_suite is to be an object recognized by the test_loader as convertible to a unittest TestSuite. For example, a module, TestCase subclass, etc. You are probably thinking of the test_runner option, not test_suite. >test command performs test discovery (as a last recourse AFAICR, /me >cc-ing to P.J. Eby because probably he has further updates ;o) using a >loader: setuptools.comand.test.xxx Setuptools' test loader will scan modules and packages, but you still have to tell it where your top-level test package or module is. (The test_suite.) >BTW, I am strongly -1 for using the discovery protocol introduced in >2.7 (AFAICR), specially because it collides with test_suite and many >other arguments ... so it would be very nice when defining packaging >conventions, layout ... not to enforce the use of that protocol, so >that if another discovery is used then it could still be compatible >with the more abstract good practices. (all this said from my selfish >position, I confess) I'm not sure what you're talking about here; I'm not much in the loop for what's going on with distutils updates, and even more so for unittest updates.

1 0

Re: [Distutils] RFC: Barely minimal CI + tests + packaging in Python WAS: Guidelines for where to put tests & how to package them
by P.J. Eby 26 Feb '10

26 Feb '10

At 11:19 AM 2/26/2010 -0500, Olemis Lang wrote: >On Fri, Feb 26, 2010 at 10:52 AM, P.J. Eby <pje(a)telecommunity.com> wrote: > > At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote: > >> > >> Well not as happily as I thought in first place. The fact is that > >> `setuptools` test command creates the runner by calling the class > >> specified in `test_runner` arg but supplies no parameters. In this > >> case I'd like to specify `xmlrunner:XMLTestRunner` (is that the > >> correct syntax ?) to obtain a JUnit XML report, but I need to specify > >> the path where those reports will be outputted, so that the CI tool > >> (Hudson in this case) can publish'em and generate the nice graphs we > >> all love . Considering the current implementation, I still need to > >> override `run_tests` method as a whole, and repeat 99% of what is > >> implemented in there, in order to get the exact runner I need > >> > >> :( > >> > >> My suggestion (hint, $0.02 ... whatever ;o) is > >> > >> 1. use a factory method (e.g `create_runner` ) returning an > >> instance of test runner. This method could be overridden by > >> subclasses if further options or whatever needs to be considered > >> to instanciate the runner > > > > I'm confused. Why don't you just set test_runner to point to your factory > > function, then? That's what nose does. (Well, in its case it's the > > test_loader, but same basic idea.) > > > >Well in theory it is possible (and did not think about that in the >first place, thnx ;o) but : > > 1. How could I know about extra command line > params (without parsing them once again of course ;o) ? > 2. In the case of the build script, I'm writing a setup script rather than > a framework (e.g. nose). > 2.1 The XML runner is not always used, but only when running at > the CI buildenv (I supposed I could just use > `-r xmlrunner:XMLTestRunner` but I need an extra arg) > How could I specify output path in cmdline ? An environment variable would solve these problems, at least if you have a decent shell. ;-) > 2.2 So I suppose I should write a function to do that but > - Under tests pkg? ... (Unit | functional | ...) tests >don't need it > - In setup.py itself ? ... What should I supply in to -r arg ? If you are running setup.py from the command line (as opposed to programmatically via easy_install), then it's __main__:whatever. (It won't work if setup.py isn't the "true" __main__, though, e.g., if the script is being execfile'd.) > How could I specify output path in cmdline ? > 3. In the case of `dutest` well ... that could help (depending on > the answer to 1 ;o) > 3.1 How could I reuse features already stored in `Distribution` > (especially values set after finalize_xxx ) ? You can't, unless you... > 4. (Hypothetical so far, so I'm prepared to receive tomatoes ;o) > What if I want to reuse (subclass) `test` command ? Which is perfectly allowable. You'll just have to duplicate that one method. Of course, if you write a suitable patch to refactor it to call another method, I'll happily put it in the trunk, since you're working from trunk to get the test_runner feature anyway. ("Suitable" = tested by you, doesn't break or remove existing functionality, & other "I'll know it if I see it" criteria ;-) )

1 0

Re: [Distutils] RFC: Barely minimal CI + tests + packaging in Python WAS: Guidelines for where to put tests & how to package them
by P.J. Eby 26 Feb '10

26 Feb '10

At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote: >Well not as happily as I thought in first place. The fact is that >`setuptools` test command creates the runner by calling the class >specified in `test_runner` arg but supplies no parameters. In this >case I'd like to specify `xmlrunner:XMLTestRunner` (is that the >correct syntax ?) to obtain a JUnit XML report, but I need to specify >the path where those reports will be outputted, so that the CI tool >(Hudson in this case) can publish'em and generate the nice graphs we >all love . Considering the current implementation, I still need to >override `run_tests` method as a whole, and repeat 99% of what is >implemented in there, in order to get the exact runner I need > >:( > >My suggestion (hint, $0.02 ... whatever ;o) is > > 1. use a factory method (e.g `create_runner` ) returning an > instance of test runner. This method could be overridden by > subclasses if further options or whatever needs to be considered > to instanciate the runner I'm confused. Why don't you just set test_runner to point to your factory function, then? That's what nose does. (Well, in its case it's the test_loader, but same basic idea.)

2 1

Re: [Distutils] RFC: Barely minimal CI + tests + packaging in Python WAS: Guidelines for where to put tests & how to package them
by P.J. Eby 26 Feb '10

26 Feb '10

At 12:10 PM 2/25/2010 -0500, Olemis Lang wrote: >Thnx for your reply ! > >On Thu, Feb 25, 2010 at 12:00 PM, P.J. Eby <pje(a)telecommunity.com> wrote: > > At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote: > >> > >> PS: BTW, how could I trigger easy_install(ation) at a given point > >> while implementing a distutils command, and let the command perform > >> further actions if deps are installed correctly ? > > > > Setuptools' "Distribution" object has a method for fetching dependencies. > > See setuptools' "test" command for an example. (This doesn't install the > > dependencies globally, just drops eggs into the build directory. But > > they're there and available to be reused for installation in a later phase, > > under normal circumstances.) > > > >So this means that setuptools `test` command already retrieves dependencies ? Yes - it also retrieves any testing-specific dependencies (per the "tests_require" setup() argument), so that if you use another testing library like py.test or nose, it can be used instead. >If so, then I shouldn't care about that, because I'd only need to >override `run_tests` method in order to do what I want to do (i.e. use >another test runner ;o) In that case, you may want to simply use the setuptools trunk version (setuptools==dev), which supports a test_runner setup() argument to the test command. ;-)

2 2

zc.buildout - order of extends processing
by Martin Aspeli 26 Feb '10

26 Feb '10

Hi, In my quest to make re-usable buildout components, I've started using a pattern like this: In a server-specific buildout file (e.g. prod-web-master.cfg) I have e.g.: [buildout] extends = buildout.d/base.cfg buildout.d/zope.cfg buildout.d/relstorage.cfg buildout.d/haproxy.cfg [ports] haproxy = 8100 [eggs] main = my.package ... Each file in buildout.cfg adds additional functionality, usually by defining new parts and doing e.g. parts += haproxy-build haproxy-config [haproxy-build] recipe = zc.recipe.cmmi ... Sometimes, they also extend some shared variables, e.g. [eggs] main += RelStorage With this type of setup, the order in which configuration is loaded and processed becomes important, in particular for += and -= type operations and straightforward overrides. For example, I may have a default ${ports:haproxy} defined in haproxy.cfg, but overridden in prod-web-slave.cfg because the slave node wants it to run on a different port, say. I can't find much documentation about how this is supposed to work. I've had situations where I've been surprised by the order in which things are processed, in particular if I extend one file that in turn extends another. So, a few questions - Is the order well-defined? e.g. is it always doing a depth-first or breath-first inclusion? Is the load order the same as the execution order? - Do settings in the top-level config (the one run with bin/buildout -c <file>) always override extended ones? - More generally, if A extends B and B extends C, does configuration in B always take precedence over configuration in C with the same part/option? Does a += in B always extend the options set in C? - If I have multiple files all extending a common base, e.g. A extends B and C, and both B and C extend a common D, when does the common base (D) get processed? Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

1 0

how to use mr.developer?
by }--o 24 Feb '10

24 Feb '10

hi, I'm just experimenting with distutils, and would like to move my django apps under it. One of my apps uses django-sugar that should be checked out from github. bin/buildout check out my django-sugar repo under src/, and creates an egg-link under develop-eggs, but it's not added to sys.path for the bin/* binaries. Is there a simple way to do it? I've seen the extra_paths variable, but then I still have to write each ${buildout:auto-checkout} line again. this is my buildout.cfg file: [buildout] parts = python django-1.1 develop = . eggs = django-contacts django-l10n django-registration eggs-directory = /home/nagyv/.buildout/eggs find-links = ${buildout:eggs-directory} download-cache = /home/nagyv/.buildout/dlcache extensions = mr.developer sources = sources auto-checkout = uni-form sugar [python] recipe = zc.recipe.egg interpreter = python eggs = ${buildout:eggs} [django-1.1] recipe = djangorecipe version = 1.1.1 project = contacts projectegg = contacts settings = testsettings test = contacts testrunner = test-1.1 eggs = ${buildout:eggs} [sources] uni-form = git git://github.com/nagyv/django-uni-form.git sugar = git git://github.com/nagyv/django-sugar.git thanks for your help! Viktor

2 1

how to use mr.developer?
by }--o 24 Feb '10

24 Feb '10

hi, I'm just experimenting with distutils, and would like to move my django apps under it. One of my apps uses django-sugar that should be checked out from github. bin/buildout check out my django-sugar repo under src/, and creates an egg-link under develop-eggs, but it's not added to sys.path for the bin/* binaries. Is there a simple way to do it? I've seen the extra_paths variable, but then I still have to write each ${buildout:auto-checkout} line again. this is my buildout.cfg file: [buildout] parts = python django-1.1 develop = . eggs = django-contacts django-l10n django-registration eggs-directory = /home/nagyv/.buildout/eggs find-links = ${buildout:eggs-directory} download-cache = /home/nagyv/.buildout/dlcache extensions = mr.developer sources = sources auto-checkout = uni-form sugar [python] recipe = zc.recipe.egg interpreter = python eggs = ${buildout:eggs} [django-1.1] recipe = djangorecipe version = 1.1.1 project = contacts projectegg = contacts settings = testsettings test = contacts testrunner = test-1.1 eggs = ${buildout:eggs} [sources] uni-form = git git://github.com/nagyv/django-uni-form.git sugar = git git://github.com/nagyv/django-sugar.git thanks for your help! Viktor

2 1

setuptools and additional scripts handling
by Manlio Perillo 24 Feb '10

24 Feb '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi. I'm starting to use setuptools for my projects and I have a few questions. In my web applications, usually I have some scripts that needs to be installed as cron scripts. I would like to install them using setuptools, since it is important that the correct python interpreter is used (I use virtualenv), and setuptools does this automatically.. Is it possible to declare a custom directory where some of the scripts needs to be installed? Thanks Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkt6nGQACgkQscQJ24LbaUT8QACgg2tkF6vJWWTiBoFnWtr6tb1X C7sAnjzu/l/ytRMD44STkTiiVJ/uIxmy =hSnP -----END PGP SIGNATURE-----

4 9

Re: [Distutils] from distutils.core import DistutilsOptionError
by P.J. Eby 23 Feb '10

23 Feb '10

At 06:05 PM 2/23/2010 -0500, Tarek Ziadé wrote: >Or do you mean that you consider the exception classes located in >Distutils to be a common need >for people that write setup.py scripts ? A setup script may want to explicitly throw (or catch) distutils errors, and having only one place to import these things from makes distutils easier to use. I don't consider it a *common* need, but I do consider it part of distutils' "core API", in the sense that if your code is either invoking distutils or being invoked by it, you may need to throw or catch those errors. In general, everything that's a *distutils-defined* symbol (not types, os, etc.) in distutils.core, I thought was intended to be imported from there, as they are all things that could be considered part of the "core API". (And apparently, at least Thomas thought the same thing.) (If I'd been writing distutils, I'd have named the module distutils.api rather than distutils.core, and explicitly created an __all__ list for it, but it's pretty much the same thing.)

2 1

Re: [Distutils] from distutils.core import DistutilsOptionError
by P.J. Eby 23 Feb '10

23 Feb '10

At 05:32 PM 2/23/2010 -0500, Tarek Ziadé wrote: >2/ the work done in 2.7 so far was following the usual deprecation >process for relocated >APIs or removed APIs, but nothing for misplaced import statements. We >did discuss this in the past and said it was fine this way. Btw, imports from distutils.core *are* API; its docstring describes it as "The only module that needs to be imported to use the Distutils". It's supposed to be a one-stop shop for importing common things needed by setup scripts, including the errors, Extension, Command, Distribution, etc. So, the unused imports in distutils.core are not "misplaced import statements", nor is there a good reason (AFAICT) for changing the API to require the average setup script author to remember which module to import things from. There should be only one obvious way to do it, and right now, the one obvious way to import things in a setup script is from distutils.core.

2 1

from distutils.core import DistutilsOptionError
by Thomas Heller 23 Feb '10

23 Feb '10

One of my setup scripts contains: from distutils.core import DistutilsOptionError which doesn't work anymore with Python trunk. Bug in Python, or in my setup script? -- Thanks, Thomas

3 3

[issue103] develop command needs a way to install extra requirements
by Tres Seaver 22 Feb '10

22 Feb '10

New submission from Tres Seaver <tseaver(a)palladion.com>: When running 'python setup.py develop', it would be useful to install one (or more) of the 'extra' requirements for the package-under-development. The attached patch allows installing one at a time: I don't have the options fu to make it a repeatable option. ---------- assignedto: pje files: setuptools-develop_w_extra.patch messages: 506 nosy: pje, tseaver priority: feature status: unread title: develop command needs a way to install extra requirements Added file: http://bugs.python.org/setuptools/file67/setuptools-develop_w_extra.patch _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue103> _______________________________________________

1 0

Re: [Distutils] setuptools-0.6c11.win32-py2.6.exe
by P.J. Eby 22 Feb '10

22 Feb '10

At 04:13 AM 2/21/2010 +0100, Daniel Niccoli wrote: >Hello Phillip, > >if you install the x86-64 version of python, the windows installer >"setuptools-0.6c11.win32-py2.6.exe" does not find any Python >installation. I had to export the registry key >"HKEY_LOCAL_MACHINE\SOFTWARE\Python", change it to >"HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python" and import it >afterwards to be able to install the software. The installer should >be changed to avoid trouble for other people. This is a *Python* installer issue; Please see: http://bugs.python.org/issue6792 Until the issue in Python itself is fixed, the only things I can do are document the problem and/or stop distributing an .exe installer.

2 1

Re: [Distutils] using distribute instead of distutils
by P.J. Eby 22 Feb '10

22 Feb '10

At 05:20 PM 2/22/2010 +0100, Jean Daniel wrote: >Hello, I am writing a pure python module called wordish, which is >similar to doctest in that it test snippets of code, parsed from an >rst article and produce a report. wordish is available on Pypi in >version 1.0 and the documentation is available at >jdb.github.com/wordish. So far, I only distributed one file, >*wordish.py* and the file was used like the following:: Â Â Â >python -m wordish my_article.rst I think I will turn to 'distribute' >for the future release so that console entry points can be created >automatically and installed in the shell path. I would like wordish >to be used in this way:: Â Â Â wordish my_article.rst I have read >the documentation for distutils, distribute but still, but the >overlap that the tools, and the problems that each tool solve. Can >my setup.py support the installation of the following files, or >should they be included in debian/fedora packages instead? 1. few >examples files? 2. static html documentation files? 3. man pages for >the console scripts? The distutils doesn't really support installing such things well; you can certainly include them in your *source* distribution, though. >Also, I upload a source distribution (sdist) to the Python Package >Index 4. should I use an *egg* instead? No, it's not necessary. Eggs are mainly useful as a drop-in-and-use format for application plugins (such as extensions to Chandler, Trac, and other plugin-enabled application environments), and distributing precompiled C extensions on certain platforms. easy_install and pip can both operate just fine on source distributions, and if your code is only pure Python, you don't need to distribute binaries.

2 1

using distribute instead of distutils
by Jean Daniel 22 Feb '10

22 Feb '10

Hello, I am writing a pure python module called wordish, which is similar to doctest in that it test snippets of code, parsed from an rst article and produce a report. wordish is available on Pypi in version 1.0 and the documentation is available at jdb.github.com/wordish. So far, I only distributed one file, *wordish.py* and the file was used like the following:: python -m wordish my_article.rst I think I will turn to 'distribute' for the future release so that console entry points can be created automatically and installed in the shell path. I would like wordish to be used in this way:: wordish my_article.rst I have read the documentation for distutils, distribute but still, but the overlap that the tools, and the problems that each tool solve. Can my setup.py support the installation of the following files, or should they be included in debian/fedora packages instead? 1. few examples files? 2. static html documentation files? 3. man pages for the console scripts? Also, I upload a source distribution (sdist) to the Python Package Index 4. should I use an *egg* instead? what does it change for me or the users? Thanks for your help,

1 0

[issue102] setuptools-0.6c11 regression in package_index.PackageIndex.fetch_distribution
by Tres Seaver 22 Feb '10

22 Feb '10

New submission from Tres Seaver <tseaver(a)palladion.com>: package_index.PackageIndex.fetch_distribution changed between 0.6c9 and 0.6c11, adding some logging, and returning a clone of the found distribution. However, the new code neglects to check that the dist is not None before cloning it. The attached patch fixes this bug. ---------- assignedto: pje files: setuptools-0.6c11-find_distribution_miss.patch messages: 504 nosy: pje, tseaver priority: bug status: unread title: setuptools-0.6c11 regression in package_index.PackageIndex.fetch_distribution Added file: http://bugs.python.org/setuptools/file66/setuptools-0.6c11-find_distributio… _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue102> _______________________________________________

1 0

setuptools-0.6c11.win32-py2.6.exe
by Daniel Niccoli 20 Feb '10

20 Feb '10

Hello Phillip, if you install the x86-64 version of python, the windows installer "* setuptools-0.6c11.win32-py2.6.exe"* does not find any Python installation. I had to export the registry key "HKEY_LOCAL_MACHINE\SOFTWARE\Python", change it to "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python" and import it afterwards to be able to install the software. The installer should be changed to avoid trouble for other people. Kind regards Daniel

1 0

upload size limit on pypi
by kiorky 19 Feb '10

19 Feb '10

Hello, i am again packaging some webapp as a paster template. Now, it's turn to hudson to join minitage.paste.extras. The distribution will be around 50MB. What is the current distribution limit, and if lower, can we again increase the size ? -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF

2 3

Re: [Distutils] include special filetypes and subdirectories
by P.J. Eby 18 Feb '10

18 Feb '10

At 02:11 AM 2/19/2010 +0100, Assem Chelli wrote: >i have eclipse with cvs and pydev plugin's ,how can i do it,or its >automatically If your files are under CVS control (in which case you will have a bunch of directories named "CVS" in your project), then all you need is to add the include_package_data=True in your setup(). >2010/2/19, P.J. Eby <pje(a)telecommunity.com>: > > At 01:34 AM 2/19/2010 +0100, Assem Chelli wrote: > >>Hi > >> > >> > >> > >>how to change setup.py to include special filetypes and subdirectories > >>recursively in python eggs > > > > > > Option 1: Put them under revision control using CVS, Subversion, or > > any revision control system that has a setuptools plugin (in which > > case, you'll need the plugin installed in your development > > environment). Then, add 'include_package_data=True' in your setup() call. > > > > Option 2: Use the package_data setup() argument (see the distutils > > docs), and explicitly list the files and directories to > > include. This is not recursive, but will also work with plain distutils. > > > > Option 3: Use a MANIFEST.in file (see the distutils docs) and specify > > the directories to include, and also place > > 'include_package_data=True' in your setup() call. > > > > Of these, option 1 is generally the "best" option since it doesn't > > require you to maintain information in two or more places; the files > > are included as a side effect of your ordinary development practices, > > as long as you're using a revision control system. > > > >

1 0

include special filetypes and subdirectories
by Assem Chelli 18 Feb '10

18 Feb '10

Hi how to change setup.py to include special filetypes and subdirectories recursively in python eggs

2 1

Re: [Distutils] include special filetypes and subdirectories
by P.J. Eby 18 Feb '10

18 Feb '10

At 01:34 AM 2/19/2010 +0100, Assem Chelli wrote: >Hi > > > >how to change setup.py to include special filetypes and subdirectories >recursively in python eggs Option 1: Put them under revision control using CVS, Subversion, or any revision control system that has a setuptools plugin (in which case, you'll need the plugin installed in your development environment). Then, add 'include_package_data=True' in your setup() call. Option 2: Use the package_data setup() argument (see the distutils docs), and explicitly list the files and directories to include. This is not recursive, but will also work with plain distutils. Option 3: Use a MANIFEST.in file (see the distutils docs) and specify the directories to include, and also place 'include_package_data=True' in your setup() call. Of these, option 1 is generally the "best" option since it doesn't require you to maintain information in two or more places; the files are included as a side effect of your ordinary development practices, as long as you're using a revision control system.

1 0

dependencies
by Assem Chelli 18 Feb '10

18 Feb '10

Hi, how can i add the dependencies of my project in setup.py ,

1 0

Changing bdist_rpm locations to conform to other rpm packaging
by Stanley A. Klein 15 Feb '10

15 Feb '10

I'm trying to build rpms for some Python packages located in an svn repository. I would prefer to use the same locations for builds and results as for non-Python rpm packages. I can set --rpm-base=$HOME/rpmbuild and some of the process happens in subdirectories of rpmbuild, but in the end, the source rpms are moved from rpmbuild/SRPMS to a dist directory under the package home, the binary rpms are moved from rpmbuild/RPMS to the same directory, and the tar.gz are similarly moved. I would prefer to have everything created as part of the build remain in the rpmbuild directory. How can I prevent files from being stored back in the svn repository as a result of the package build? Stan Klein

1 0

Buildout: How to include installed packages in sys.path?
by Gustavo Narea 15 Feb '10

15 Feb '10

Hello. I've written a Buildout recipe which relies on a function defined in a package whose name is set in the recipe's options. That package must have already been installed. According to the documentation, I should use zc.buildout.easy_install:install() like this: install(["mydistribution"], None) But if I use None, I get an exception because Buildout tries to use .startswith() with None. Then, if instead of None, I use buildout['buildout']['directory'], it tries to install the distribution pulling all its dependencies (ignoring the fact I set a download cache in ~/.buildout/default.cfg). I just want to make a package available in sys.path. How should I do that from a Buildout recipe? Note I really just want to add it sys.path, not get the actual function/module, because it's going to be used by a function called from my recipe, not by my recipe. The following code illustrates how the recipe will look like: ==== class MyRecipe(object): def __init__(self, buildout, name, options): self.the_package = options['package'] def install(self): add_to_sys_path(self.the_package) do_something_useful(self.the_package) return () ==== I'm using zc.buildout 1.4.3. Thanks in advance. -- Gustavo Narea. Software Developer. 2degrees, Ltd. <http://www.2degreesnetwork.com/>.

2 4

Attaching changelog
by Michał Klich 14 Feb '10

14 Feb '10

Hi, I am trying to attach changelog to spec file using --changelog= as a parameter. My changelog test file has lines starting with (*) but i keep getting error: %changelog entries must start with * when building rpm. Is it a right way to add changelog to spec file? Thank you for your help -- Michał Klich klich.michal(a)gmail.com michal(a)michalklich.com http://www.michalklich.com

1 0

setup.py#package_data - howto include entire folder when installing
by Aljoša Mohorović 11 Feb '10

11 Feb '10

i've created MANIFEST.in and setup.py and when i call "python setup.py sdist" archive is created normally and archive contains all files. when i call "python setup.py install" data files are not installed and i have no idea why. i've been searching for a solution but every combination i've tried failed. i usually have only html templates so something like: package_data = {'mypkg': ['templates/*.html',]} works as expected but now i have a folder with different types of files and nested folder structure. my question is: - how can i define in setup.py to include folder (let's call it "data") and everything in data so that when "python setup.py install" is called that folder is copied with python code? something like "package_data={'mypkg': ['data/*']},"? Aljosa Mohorovic

2 2

Distributing __only__ *.pyc
by Torsten Mohr 11 Feb '10

11 Feb '10

Hi, not sure if this is possible, i'd like to distribute a pure python module as .pyc because there is some code in there that idon't want to spread yet. I plan to release the code as open source later, but it is not possible for now. I know that something like disassembling is possible but for i don't expect that somebody will take that effort. Is it possible to distribute a module.pyc using distutils? How would that setup script look like? Thanks for any hints, Torsten.

5 5

Distributing (complicated) C extensions
by James Porter 10 Feb '10

10 Feb '10

Before I get started with the meat of this message (it's pretty long - for me, anyway), I should give a brief overview. I'm developing Python bindings for a set of interfaces called ITAPS, which contains a few independent libraries. If you want to look at my setup.py file (it's pretty scary), you can download the source at <http://pypi.python.org/pypi/PyTAPS/0.9.1>. Basically, I have three somewhat-related questions, which I wrote a big pile of words about below. The short version is: * Are multiple distributions inherently better/saner than one distribution with a handful of optional features? * What's the best way to detect whether libraries exist to turn said features on/off? * How do I install header files when using setuptools (or easy_install)? Ok, now for the long version: First, this distribution is fairly complicated, with a handful of setuptools Features controlling the various libraries I want to work with. I've waffled on whether this should be one distribution or several, but the former is a little problematic because the "core" module doesn't actually correlate to an independent thing on the C side. It really just piggybacks on each of the libraries. So basically there's no reason for the "core" module to exist in isolation, and it would actually be impossible to compile unless one of the "real" libraries was present. Is rolling this all together a terrible idea? Or a good one? Second, having a single distribution with multiple features that can be enabled/disabled obviously requires a bit of work for configuration. Right now, it tries to detect if each library is installed, and if not, it disables that feature automatically. I mostly do this to make easy_install "just work" even if you don't have everything. I should explain how I do configuration now: each library has a Makefile fragment that defines useful variables, e.g. $MYLIB_INCLUDEDIRS, that I read in and add into the appropriate places in the Extension. Each of my Features has a global --feature-path option that lets you point to the Makefile fragment. If you don't specify that, it steps through some environment variables and tries to find it. If it still can't find it, it'll assume you specified all the stuff you need as options to build_ext. Then I try to compile a simple file with the necessary header to make sure I got the paths right. All of this work is done in an overrided version of build_ext (grabbing the Makefile fragment happens during finalize_options and the compilation test happens during run). This seems pretty hacky to me, especially since there's a config command in distutils that does some of what I want. However, I'm not sure how to actually *use* this. I assume I override the command, but what would it look like from a user's perspective? How do I connect it up to the build process? Finally, I'd like to be able to install header files from my distribution. Looking through the distutils docs, there's an install_headers command that gets run as a part of install, and I can just specify headers=['foo.h'] as an option to setup. However, this doesn't seem to work with setuptools/distribute; it looks like they hijack the install command and, instead of actually running it directly, they manually run install_lib, install_data, etc. This is made worse by the fact that, even if I stuck with vanilla distutils, easy_install appears to inject its version of Distribution into my setup.py anyway. (There's a way to work around that, but it's really messy. Think yanking the distutils.core Distribution class - "_Distribution" in the code - out of setuptools.dist. Barf.) If you got all the way here, thanks for reading! Hopefully someone with a little more experience than me in this can give me a few pointers. - Jim

1 0

PEP 376 - RECORD file / Data files + pip feedback ?
by Tarek Ziadé 09 Feb '10

09 Feb '10

Hi, The part that still requires some work is how to handle files prefixes in the RECORD file. Last time we said that we could benefit of having a PREFIXES file. Then Wolojda started to work on a much more complete solution to track data file locations : http://mail.python.org/pipermail/distutils-sig/2009-November/014424.html But IIRC he's not able to work on this at this time. (CCing him in case..) What I suggest is that we leave this new work for a future enhanced version of PEP 376, and try to finish a version that we would want to have in Python 2.7 (that's coming very soon), where RECORD contains what we want to make it possible to remove installed files + provide the query API. As far as I am concerned, the RECORD thingie needs more live examples on various platforms (that's what I am working on now), But since Pip has now a uninstaller for a few months, the question is: what do you Pip guys think about this RECORD file ? (cc'ing Ian as well - I know the other involved in that are listening here :)) Regards Tarek -- Tarek Ziadé | http://ziade.org

8 17

Building a platform-independent dist?
by skip＠pobox.com 09 Feb '10

09 Feb '10

It's been several months since I created a distribution using the bdist command. Now I can't seem to build a simple gztar distribution. My MANIFEST.in looks like this: include README RELEASE-NOTES LICENSE MANIFEST include lockfile.py setup.py ACKS recursive-include doc *.rst conf.py Makefile MANIFEST is: ACKS LICENSE MANIFEST README RELEASE-NOTES lockfile.py setup.py doc/glossary.rst doc/index.rst doc/lockfile.rst and I'm executing python setup.py bdist --formats=gztar (where python is 2.7a2+). The output is a .tar.gz file but it includes ".macosx-10.4-i386" in the filename and the tar file itself has more structure and fewer files than I think it should have: % tar tfz dist/lockfile-0.8.macosx-10.4-i386.tar.gz ./ ./Users/ ./Users/skip/ ./Users/skip/local/ ./Users/skip/local/lib/ ./Users/skip/local/lib/python2.7/ ./Users/skip/local/lib/python2.7/site-packages/ ./Users/skip/local/lib/python2.7/site-packages/lockfile-0.8-py2.7.egg-info ./Users/skip/local/lib/python2.7/site-packages/lockfile.py ./Users/skip/local/lib/python2.7/site-packages/lockfile.pyc setup.py is simple: from distutils.core import setup setup(name='lockfile', ... bunch of kwd args elided ... ) Where are the missing files? How do I create a platform-independent distribution? -- Skip Montanaro - skip(a)pobox.com - http://www.smontanaro.net/

3 5

Fwd: PEP 345 and PEP 386
by Tarek Ziadé 09 Feb '10

09 Feb '10

FYI, PEP 345 and 386 are now accepted. Thanks to all the people that helped in building them ! ---------- Forwarded message ---------- From: Guido van Rossum <guido(a)python.org> Date: Wed, Feb 10, 2010 at 12:29 AM Subject: Re: PEP 345 and PEP 386 To: Tarek Ziadé <ziade.tarek(a)gmail.com> Cc: Python-Dev <python-dev(a)python.org> On Thu, Feb 4, 2010 at 2:55 PM, Tarek Ziadé <ziade.tarek(a)gmail.com> wrote: > On Thu, Feb 4, 2010 at 8:20 PM, Guido van Rossum <guido(a)python.org> wrote: > [..] >> I have one comment on PEP 345: Why is author-email mandatory? I'm sure >> there are plenty of cases where either the author doesn't want their >> email address published, or their last know email address is no longer >> valid. (Tarek responded off-line that it isn't all that mandatory; I >> propose to say so in the PEP.) > > Yes, I propose to remove the mandatory flag from that field. Since this is done I now approve both PEP 345 and PEP 386 (which is not to say that small editorial changes to the text couldn't be made). -- --Guido van Rossum (python.org/~guido) -- Tarek Ziadé | http://ziade.org

1 0

stdeb vs dependencies?
by K. Richard Pixley 08 Feb '10

08 Feb '10

What is the expected work flow around package dependencies between python packages and debs in stdeb? I was expecting it to declare any deb dependencies found, then translate any python dependencies into the names they would have if those pypi packages were translated using stdeb. What I'm seeing with stdeb-0.5.1 is that no dependencies are declared at all, but instead, I'm seeing unresolved macro references to ${python:Depends}. --rich

2 6

Re: [Distutils] Distutils roadmap of refactoring for 2.7
by P.J. Eby 08 Feb '10

08 Feb '10

At 06:29 PM 1/23/2010 +0100, Tarek Ziadé wrote: >2010/1/23 P.J. Eby <pje(a)telecommunity.com>: > > At 01:00 PM 1/23/2010 +0100, Tarek Ziadé wrote: > >> > >> 3 - dir_util, archive_util and file_util are going to be removed in > >> favor of calls to shutil. > > > > By removed, do you simply mean that distutils will stop using them, but the > > modules will still be there? (i.e., deprecated and phased out, rather than > > simply dropped from existence altogether.) > >I'll use the same strategy than sysconfig : > >- if the API is just moved to another place and works exactly the same way > (e.g. like what's planned for make_archive), it will be dropped, and the > documentation will refer to the new place. Is this the standard procedure for relocation of stdlib APIs across 2.x releases? I was under the impression that the standard is to do such things across two release cycles with a deprecation.

7 8

Re: [Distutils] PEP 376 -- new round
by P.J. Eby 05 Feb '10

05 Feb '10

At 01:55 AM 2/5/2010 +0100, Tarek Ziadé wrote: >I think it's still useful, because it points the root packages that >can be removed >safely without breaking the system -- even if leaving orphaned >packages behind. > >Any opinion ? I think it's a good idea to have a way to tell what packages were *not* installed to satisfy dependencies. Merely plotting the dependency graph doesn't tell you this, because you could have a non-root orphan - i.e., something that was installed to fill a dependency, but the depender(s) have now vanished. I guess what I'm saying is, a thing that is not needed by anything else could either be an orphan (due to other uninstalls) *or* a root (manually chosen for install), and there is no way to tell them apart just by following the graph.

5 7

Re: [Distutils] PEP 376 -- new round
by Laura Creighton 05 Feb '10

05 Feb '10

You need to think about dependencies more, I think. Having worked on building such a thing, the hellish problem comes when there are more than one way to support a package .. i.e. this is my new testing plugin, it works with either py.test or nose. Laura

2 1

PEP 386 - suggest_normalized version
by Tarek Ziadé 05 Feb '10

05 Feb '10

Hi, Find attached the latest state of PEP 386 suggest_normalized() over PyPI. Summary: Total Packages : 9058 Already Match : 8055.0 (88.93%) Have Suggestion : 792.0 (8.74%) No Suggestion : 211.0 (2.33%) -> the attached file contains the details on the suggestion made, and the versions that couldn't be treated I am sure it still can be improved but I think it is already quite good. Regards Tarek -- Tarek Ziadé | http://ziade.org

1 0

PEP 376 -- new round
by Tarek Ziadé 04 Feb '10

04 Feb '10

Hello, As mentioned on python-dev, I would like to do a new round of work on PEP 376 before Pycon, We had a short off-list conversation with Guido on PEP 376, I'd like to share here so we can discuss the point, it's about the REQUESTED file: Guido's question: I have a question about REQUESTED. It seems that the absence of this file means that the package is installed as a dependency of another package, so that it can be removed if that other package is removed . But since there is no record of the requesting package, it would seem problematic to find out when a dependent package can be removed, since there could possibly be multiple packages depending on it. Maybe it would be better to keep track of a list of packages that depend on a given package, adding a special "root" or "explicit" line for packages that were explicitly requested by the user? My answer: Distutils doesn't provide a feature where dependencies gets installed or removed. It works at the package level only, and provides tools to install or uninstall one package at a time. So we didn't want to provide a record of a full dependency graph through this file because we have considered that this work is up to a full-featured install/uninstall tool that wants to install and remove dependencies. This tool can calculate this graph when working, as it does when installing dependencies, since everything is available in the metadata once we have PEP 345 -- with the "requires-dist" field. IOW: 1/ an orphaned package is a package that doesn't have a REQUESTED file, *and* that is not used by another package anymore. So this can only be known by a uninstaller tool that deals with dependencies. 2/ a requested package is a package that has the REQUESTED file Now for Distutils, the REQUESTED file is useful to mark the packages that were explicitely wanted. For example to develop against them. vs things that where installed as a dependency. Of course the caveat is that a full-featured uninstall tool might be slow to compute the full dependency graph when trying to remove dependencies, because it will have to take a look at all other REQUESTED packages. Installers/uninstallers that are built on the top of Distutils could probably create their own dependency records inside each .egg-info placeholders, to speed up the work. But I am not sure it's such a big deal to look at all metadata files when performing an uninstallation. It's ok for this operation to be kind of slow I guess. And I am also not sure how this could work if the user uses different kind of installers. This REQUESTED file is at the boundary of what we want to provide in Distutils. Going further seemed dangerous at this point because as we said last year at the summit, we want to make distutils a toolkit that provides some standards and a common ground for installers out there, rather than a full-featured package manager. IOW: we don't want to tell the tools out there how they should deal with the dependencies installation / uninstallation process, but give them tools to deal efficiently with one package at a time. I realize this is not clear enough in the PEP. Let me know if what I've explained makes sense to you. If so I'll work on adding details on these points. If not, it means we need more work on this. So Guido wonders if adding this file is really necessary since higher-level installers/uninstallers will have to build the graph anyways. I think it's still useful, because it points the root packages that can be removed safely without breaking the system -- even if leaving orphaned packages behind. Any opinion ? Regards, Tarek -- Tarek Ziadé | http://ziade.org

2 1

modules in different folders but under same namespace
by Riccardo-Maria BIANCHI 04 Feb '10

04 Feb '10

Hi, I have a package structured like this: package/__init__.py src/ __init__.py mod1.py share/__init__.py mod2.py Now I can import them as: package.src.mod1 package.share.mod2 How can I use Distutils to be able to import both of them under the same "package" namespace as: import package.mod1 import package.mod2 ? Thanks a lot in advance for your kind help! Best regards, Ric.

5 10

nailed-down versions by having custom index
by Reinout van Rees 03 Feb '10

03 Feb '10

Hi all, I'd like to do a bit of experimenting with a custom index with only the right packages in just the right versions. Some possible advantages I'm hoping to see for my usecase: - Isolation from one of various servers being down (pypi, sourceforge). - Guaranteed versions even when not using buildout. (Stuff needs "python setup.py install" in the system python, apparently, for arcgis/arcview on windows to be able to use it). Personally, I'd just let everyone use buildout, but I think I won't be able to get that working with that "install it in site-packages" requirement. (Ideas welcome, of course). Question: how to maintain such a custom index? iirc chrism has a script to build an apache-served folder structure from a bunch of eggs/tarballs, so just having the tarballs is enough. Probably just putting those tarballs in one location is enough anyway. But how to get the right collection of tarballs in there? Is that a by-hand approach? Can you automate it? Question 2: I thought you could only have one index. So installing something extra from pypi seems out. How do people handle that? I'm totally used to buildout and its [versions] list and buildout.dumppickedversions and KGS lists, so I need to reset my thinking on this a little. I need some input :-) Reinout -- Reinout van Rees - reinout(a)vanrees.org - http://reinout.vanrees.org Programmer/advisor at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets"

6 8