Mailman 3 March 2010 - Distutils-SIG

RECORD format in PEP 376
by Tarek Ziadé 30 Mar '10

30 Mar '10

Hello, I'd like to make the RECORD file dead simple for PEP 376, and remove all the relocatable work we've started in it, since it'll go in a second file. This would be done in a second phase, with the work done at Pycon. So what I am proposing to keep in the PEP is this: =========================== The `RECORD` file is a CSV file, composed of records, one line per installed file. The ``csv`` module is used to read the file, with these options: - field delimiter : `,` - quoting char : `"`. - line terminator : ``os.linesep`` (so ``\r\n`` or ``\n``) Each record is composed of three elements. - the file's full **path**, as defined by the target system/ - the **MD5** hash of the file, encoded in hex. Notice that `pyc` and `pyo` generated files don't have any hash because they are automatically produced from `py` files. So checking the hash of the corresponding `py` file is enough to decide if the file and its associated `pyc` or `pyo` files have changed. - the file's size in bytes The ``csv`` module is used to generate this file, so the field separator is ",". Any "," characters found within a field is escaped automatically by ``csv``. When the file is read, the `U` option is used so the universal newline support (see PEP 278 [#pep278]_) is activated, avoiding any trouble reading a file produced on a platform that uses a different new line terminator. =========================== I think this is getting close to what Pip has today. Opinions ? Regards, Tarek -- Tarek Ziadé | http://ziade.org

2 3

executable option in buildout
by Martin Raspaud 29 Mar '10

29 Mar '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I just wrote a small recipe for buildout that installs an egg from its develop version and tarballs it. So far, so good, I got it to work fine with a few example packages. Now I got a problem when trying to use the "executable" option in the buildout.cfg file: it seems to have no effect. The recipe runs with the default python from the system (2.5) not the one I supply in the executable option (2.4) I attach my buildout.cfg and the recipe. Am I doing something wrong ? Thanks, Martin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJLsYuLAAoJEBdvyODiyJI4pJAH/R6SRFprxS6R1maJ7blyEhtM leJaoYdTwPpFTqLiEUFz/GvWwr6Up1yxyIinxl7SY8kw7wKd80244MHJVke2Vjf3 nim2IxyCC/E6+j4AE+vQkYx/G5nTJSFHWUd4I1k8OR6iuYuEF2WLWI9m0TBgPe6d JaHzI+LO8FTduDp4JqPMeAg4KLI8pykhSOb4ikkBbbQXp0P99w8wHfbowHongLX2 HRECWofuzjUmv7swIOQ/+/MRlbzd2QwBVZFeXrbABGxx1GQCNpOVqjvVTAUkaBf+ 79N55/LBbtVnvGH7wf6cLvZSHawDxF9XQ5OcXi1nHZD5jQ9F0l8koKq8cIJXSgc= =v3Ew -----END PGP SIGNATURE----- [buildout] executable=/local_disk/usr/dists/python-2.4/bin/python develop = smhi_deploy /local_disk/usr/src/mpop /local_disk/usr/src/pyresample parts = deploy [deploy] recipe = smhi_deploy path = torc eggs = /local_disk/usr/src/mpop /local_disk/usr/src/pyresample import logging, os, zc.buildout, pkg_resources import sys from distutils.sysconfig import get_python_version import shutil class SmhiDeploy: def __init__(self, buildout, name, options): self.name, self.options = name, options options['path'] = os.path.join( buildout['buildout']['directory'], options['path'], ) if not os.path.isdir(os.path.dirname(options['path'])): logging.getLogger(self.name).error( 'Cannot create %s. %s is not a directory.', options['path'], os.path.dirname(options['path'])) raise zc.buildout.UserError('Invalid Path') d_eggs = buildout['buildout'].get('develop') d_eggs = [os.path.abspath(e.strip()) for e in d_eggs.split('\n')] options['eggs'] = '\n'.join([e for e in d_eggs if e in options['eggs']]) options['executable'] = buildout['buildout']['executable'] self.executable = options['executable'] print options def install(self): old_exe = sys.executable print sys.executable sys.executable = self.executable print sys.executable path = self.options['path'] logging.getLogger(self.name).info( 'Creating directory %s', os.path.basename(path)) if not os.path.exists(path): os.mkdir(path) d_eggs = self.options['eggs'] d_eggs = [os.path.abspath(e.strip()) for e in d_eggs.split('\n')] old_dir = os.getcwd() os.chdir(path) for degs in d_eggs: dir, proj = os.path.split(degs) version = pkg_resources.get_distribution(proj).version name = pkg_resources.get_distribution(proj).project_name dist = pkg_resources.Distribution( None, None, name, version, get_python_version(), pkg_resources.get_platform() ) os.chdir(degs) install_dir = os.path.join(path, name.upper()) if not os.path.exists(install_dir): os.mkdir(install_dir) old_path = sys.path[:] sys.path.insert(0, install_dir) _run_setup("easy_install", "--install-dir=" + install_dir, "--always-unzip", "--site-dirs=" + install_dir, "--script-dir=" + os.path.join(install_dir, str(dist.egg_name() + ".egg"), "bin"), ".") sys.path[:] = old_path egg_dir = os.path.join(install_dir, str(dist.egg_name() + ".egg")) true_egg_dir = os.path.join(install_dir, str(dist.egg_name())) os.rename(egg_dir, true_egg_dir) os.chdir(path) import tarfile tar = tarfile.open(os.path.join(path, str(dist.egg_name()) + ".tar.gz"), "w:gz") tar.add(name.upper()) tar.close() shutil.rmtree(name.upper()) os.chdir(old_dir) sys.executable = old_exe return path update=install def _run_setup(*args): old_args = sys.argv sys.argv = ["setup.py"] + list(args) old_path = sys.path[:] sys.path.insert(0, os.getcwd()) __import__('setup') del sys.modules['setup'] sys.argv = old_args sys.path[:] = old_path

1 0

FAQ (was: Re: Bootstrap script for package management tool in Python 2.7)
by anatoly techtonik 29 Mar '10

29 Mar '10

On Mon, Mar 29, 2010 at 5:18 PM, Tarek Ziadé <ziade.tarek(a)gmail.com> wrote: > [..] >>> http://guide.python-distribute.org >> >> I can see any FAQ. To me the FAQ is something that could be posted to >> distutils ML once a month to reflect current state of packaging. It >> should also carry version number. So anybody can comment on the FAQ, >> ask another question or ask to make a change. > > The guide was built by people in the community, and contains a > "current state of packaging" section. > We are trying to keep it accurate with what we are doing. > > You have a mailing list where you can ask question, and you are > welcome to contribute changes. > > Adding a "FAQ" containing links to sections in the guide, or a > summarized answered is a good idea though ! If do not post a link to exact chapter in the FAQ as a full response to user's question at least once a week then it is not a FAQ. -- anatoly t.

7 8

zc.buildout: bootstrap.py script
by René Fleschenberg 29 Mar '10

29 Mar '10

Hi, The code in bootstrap.py doesn't use the if __name__ == "__main__" idiom. This can lead to funny behaviour if it is imported for some reason (I stumbled upon this in conjunction with nose). Is there a reason for not checking for __name__ == "__main__", or should this be changed? -- René Fleschenberg Am Stadtgarten 28, 45276 Essen, Germany Phone: +49 1577 170 7363 E-Mail: rene(a)fleschenberg.net

1 0

Re: [Distutils] [Python-Dev] At least one package management tool for 2.7
by Tarek Ziadé 29 Mar '10

29 Mar '10

On Wed, Mar 24, 2010 at 10:59 AM, anatoly techtonik <techtonik(a)gmail.com> wrote: > I wonder if there are many people here who don't use some kind of > "easy_install" package for package management in their Python / > virtualenv installations? I propose to include at least one such > package that is capable to auto-update itself in Python 2.7 > > C:\~env\Python27>python.exe -m easy_install > C:\~env\Python27\python.exe: No module named easy_install > > C:\~env\Python27>python.exe -m pip > C:\~env\Python27\python.exe: No module named pip > > > It bugs me when I have to troubleshoot things on yet another machine > that doesn't have some kind of `setuptools` installed. Or when I have > to test some bug in my package on different Python version with a > clean install and need some dependencies. We are working on distutils2 right now to improve the situation, and Ian has proposed to work on the possible inclusion of virtualenv in the stldib as well. I'll talk for distutils2 : The plan is to provide a distutils2 standalone version that can be installed from 2.4 to 3.x, and that will provide a basic installer/uninstaller via -m. Distutils2 is planned to be reintegrated in the stdlib in Python 3.3, and my goal is to release it when Python 2.7 final is released. The open question is: do we want to include a full installer that takes care of installing / removing dependencies as well ? I think not. Pip already provides this feature on the top of distutils (and distutils2 later I guess) and is not hard to install on the top of Python. But the "auto-update" story seems interesting, can you expand on this ? Tarek -- Tarek Ziadé | http://ziade.org

11 34

Distutils2@GSOC
by Tarek Ziadé 28 Mar '10

28 Mar '10

Hello, I am proposing a Disutils2 topic for GSOC. I have started to list possible tasks at : http://wiki.python.org/moin/SummerOfCode/Distutils2 If you are thinking about some possible tasks, please let me know ! Regards Tarek -- Tarek Ziadé | http://ziade.org

1 0

Help with Disutils / MinGW
by James Reynolds 28 Mar '10

28 Mar '10

Hello All, I'm trying to write my first extension module, and I am getting the following error in my command prompt and I was hoping you all could help me. I have taken the following steps already: 1. My path is set for mingw/bin as well as python31. 2. There is a file in my disutils folder called disutils.cfg that says [build] compiler = mingw32 3. The instructions in the 3.1 documentation state the following: "These instructions only apply if you’re using a version of Python prior to 2.4.1 with a MinGW prior to 3.0.0 (with binutils-2.13.90-20030111- 1. http://docs.python.org/py3k/install/index.html 2. I am using Python 3.1 and the latest MinGW. 4. I tested gcc/mingw by doing C:\python31>gcc -shared pdv.c -o pdv.dll and the test was successful (or at least I was not given any errors while doing the compile). The pdv is a simple math formula. I am able to use the 64 bit version of this dll in ctypes, but not the 32 version. 5. I searched on the internet and the closest thing I can find is the following: http://bugs.python.org/issue4709 6. I suspected I might be having an issue with 64 bit compatibility, but I can't get the mingw64 bit compiler to run: "error: don't know how to compile C/C++ code on platform 'nt' with 'x86_64-w64-mingw32-gcc' compiler. I am running Windows 7 64 bit version, I have installed the Python 3.1.2 64 bit interpreter. I have the MinGW32 and the one referenced above as well. Below you will find the following One, the error report two,my setup.py file three, the file I am trying to turn into a python extension module by running the following two commands: python setup.py build python setup.py install #1 Microsoft Windows [Version 6.1.7600] Copyright (c) 2009 Microsoft Corporation. All rights reserved. > c:\Python31\Lib\finance>python setup.py build running build running build_ext building 'finance' extension creating build creating build\temp.win-amd64-3.1 creating build\temp.win-amd64-3.1\Release C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python31\include > -IC:\Pytho n31\PC -c finance.c -o build\temp.win-amd64-3.1\Release\finance.o finance.c: In function `PyInit_finance': finance.c:31: warning: implicit declaration of function `Py_Module_Create' finance.c:31: warning: return makes pointer from integer without a cast writing build\temp.win-amd64-3.1\Release\finance.def creating build\lib.win-amd64-3.1 C:\MinGW\bin\gcc.exe -mno-cygwin -shared -s > build\temp.win-amd64-3.1\Release\fin ance.o build\temp.win-amd64-3.1\Release\finance.def -LC:\Python31\libs > -LC:\Pyth on31\PCbuild\amd64 -lpython31 -lmsvcr90 -o > build\lib.win-amd64-3.1\finance.pyd build\temp.win-amd64-3.1\Release\finance.o:finance.c:(.text+0x2b): undefined > ref erence to `_imp__PyArg_ParseTuple' build\temp.win-amd64-3.1\Release\finance.o:finance.c:(.text+0x5c): undefined > ref erence to `_imp__Py_BuildValue' build\temp.win-amd64-3.1\Release\finance.o:finance.c:(.text+0x74): undefined > ref erence to `Py_Module_Create' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 > c:\Python31\Lib\finance> #2 from distutils.core import setup, Extension > setup(name = "finance", version = "1.0", ext_modules = [Extension("finance", ["finance.c"])]) #3 #include <Python.h> #include <math.h> > static PyObject * pdv(PyObject *self, PyObject *args) { double value, rate, timex, denom, pdvx; if (!PyArg_ParseTuple(args, "ddd", &value, &rate, &timex)) return NULL; denom = (double) pow ((1 + rate), (timex)); pdvx = value / denom; return Py_BuildValue("d", pdvx); } PyMethodDef pdvMethods[] = { {"pdv", pdv, METH_VARARGS, "Returns the Present Discounted Value given > of a single future value"}, {NULL, NULL, 0, NULL} }; > static struct PyModuleDef financemodule = { PyModuleDef_HEAD_INIT, "finance", /* name of module */ NULL, /* module documentation, may be NULL */ -1, /* size of per-interpreter state of the module, or -1 if the module keeps state in global variables. */ pdvMethods }; > PyMODINIT_FUNC PyInit_finance(void) { return Py_Module_Create(&financemodule); }

2 1

Buildout: Version conflicts with system-wide libraries
by Gustavo Narea 27 Mar '10

27 Mar '10

Hello, On my Ubuntu desktop, httplib2 v0.4.0 is installed system-wide because it's required by many things. However, I need to use v0.6.0 of that package in my Buildout-powered project. But when I run Buildout, I get this error: ===== VersionConflict: (httplib2 0.4.0 (/usr/lib/pymodules/python2.5), Requirement.parse('httplib2==0.6.0')) ===== I've hit this problem before and what I had done is to remove the system-wide install, which I cannot do this time. There has to be a way for Buildout to ignore the global install, but I could not find the answer after reading the documentation. Can you please help me? Cheers, -- Gustavo Narea. Software Developer. 2degrees, Ltd. <http://www.2degreesnetwork.com/>.

9 14

Re: [Distutils] Integrating the packaging guide
by John Posner 26 Mar '10

26 Mar '10

On 3/21/2010 11:52 AM, John Posner wrote: > >> Thanks for joining and welcome to the team. And Don't Panic ;-) Not panicking, exactly ... I've looked over the entire Hitchhiker's Guide. After a very nice Quick Start and pretty good initial sections in "Introduction to Packaging", things start to get more stream-of-consciousness-y and (to me) confusing. This seems to reflect the real world -- the "Current State of Packaging" diagram shows that this area of Python functionality is in a state of flux. Here's a bunch of questions -- not necessarily orthogonal to each other -- concerning the Guide. I'm posting to the entire SIG, to see if there's consensus out there. 1. What is the intended relationship between the Hitchhiker's Guide, a short document, and these much lengthier documents: For *distutils* -- Distributing Python Modules (http://docs.python.org/distutils/index.html) Installing Python Modules (http://docs.python.org/install/index.html) For *setuptools* -- Building and Distributing Packages with setuptools (http://peak.telecommunity.com/DevCenter/setuptools) For *distribute* -- Welcome to Distribute’s documentation! (http://packages.python.org/distribute/) 2. In the many areas where disutils/setuptools/distribute provide overlapping functionality, should the Hitchhiker's Guide promote *the one best way* to accomplish a task, or should it compare and contrast the various ways? 3. Should the Hitchhiker's Guide be an overview document, with cross-refs to large sections of the disutils/setuptools/distribute documents? Or should it be a detailed document, with (many more) cross-refs to detail sections of the disutils/setuptools/distribute documents? Thanks for helping me get up to speed ... -John

2 1

Re: [Distutils] log.py->logging ; trac for distutils2
by P.J. Eby 26 Mar '10

26 Mar '10

At 12:48 PM 3/26/2010 +0530, Zubin Mithra wrote: >I had found that this was a pending task by looking through the >code-base; and the task was quite trivial. Did you take into consideration the effect of the -v and -q flags, both at the global and specific-command levels? If not, backporting it to the main distutils might be a problem. (Also, ISTR that the logging module by default sends things to stderr that distutils currently sends to stdout; that might be a backwards compatibility issue as well.)

2 1