At 02:38 PM 11/27/2006 +0100, Jan Matejek wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Phillip J. Eby napsal(a):

...

Just a suggestion, but one issue that I think needs addressing is the FHS language that leads some Linux distros to believe that they should change Python's normal installation layout (sometimes in bizarre ways) (...) Other vendors apparently also patch Python in various ways to support their FHS-based theories of how Python should install files.

+1 on that. There should be a clear (and clearly presented) idea of how Python is supposed to be laid out in the distribution-provided /usr hierarchy. And it would be nice if this idea complied to FHS.

It would also be nice if somebody finally admitted the existence of /usr/lib64 and made Python aware of it ;e)

Actually, I meant that (among other things) it should be clarified that it's alright to e.g. put .pyc and data files inside Python library directories, and NOT okay to split them up.

Actually, I meant that (among other things) it should be clarified that it's alright to e.g. put .pyc and data files inside Python library directories, and NOT okay to split them up.

Phillip, Just to be clear: I understand you are not in favour of re-packaging data from python projects (projects in the distutils sense), separately and I strongly agree with this view. Are you opposed to developers choosing to *not* bundle data as python package data ? How much, if any, of the setuptools / distutils conventions do you think could sensibly peculate up to the LSB ? There are a couple of cases in ubuntu/debian (as of 6.10 edgy) that I think are worth considering: python2.4 profile (pstats) etc, was removed due to licensing issues rather than FHS. Should not be an issue for python2.5 but what, in general, can a vendor do except break python if their licensing policy cant accommodate all of pythons batteries ? python2.4 distutils is excluded by default. This totally blows in my view but I appreciate this one is a minefield of vendor packaging politics. It has to be legitimate for Python / setuptools too provide packaging infrastructure and conventions that are viable on more than linux. Is it unreasonable for a particular vendor to decide that, on their platform, the will disable Python's packaging conventions ? Is there any way to keep the peace on this one ? Cheers, Robin On 27/11/06, Phillip J. Eby wrote:

At 02:38 PM 11/27/2006 +0100, Jan Matejek wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Phillip J. Eby napsal(a):

...

Just a suggestion, but one issue that I think needs addressing is the FHS language that leads some Linux distros to believe that they should change Python's normal installation layout (sometimes in bizarre ways) (...) Other vendors apparently also patch Python in various ways to support their FHS-based theories of how Python should install files.

+1 on that. There should be a clear (and clearly presented) idea of how Python is supposed to be laid out in the distribution-provided /usr hierarchy. And it would be nice if this idea complied to FHS.

It would also be nice if somebody finally admitted the existence of /usr/lib64 and made Python aware of it ;e)

Actually, I meant that (among other things) it should be clarified that it's alright to e.g. put .pyc and data files inside Python library directories, and NOT okay to split them up.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/robinbryce%40gmail.com

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Nov 28, 2006, at 8:53 AM, Anthony Baxter wrote:

(The only other packaging thing like this that I'm aware of is python-minimal in Ubuntu. This is done for installation purposes and wacky dependency issues that occur when a fair chunk of the O/S is actually written in Python. It's worth noting that the entirety of the Python stdlib is a required package, so it doesn't cause issues.)

There's a related issue that may or may not be in scope for this thread. For distros like Gentoo or Ubuntu that rely heavily on their own system Python for the OS to work properly, I'm quite loathe to install Cheeseshop packages into the system site-packages. I've had Gentoo break occasionally when I did this for example (though I don't remember the details now), so I always end up installing my own /usr/ local/bin/python and installing my 3rd party packages into there. Even though site-packages is last on sys.path, installing 3rd party packages can still break the OS if the system itself installs incompatible versions of such packages into its site-packages. Mailman's philosophy is to install the 3rd party packages it requires into its own 'pythonlib' directory that gets put first on sys.path. It does this for several reasons: I want to be able to override stdlib packages such as email with newer versions, I don't want to have to mess around at all with the system's site-packages, and I don't want updates to the system Python to break my application. I question whether a distro built on Python can even afford to allow 3rd party packages to be installed in their system's site-packages. Maybe Python needs to extend its system-centric view of site-packages with an application-centric and/or user-centric view of extensions? The only reason I can think of for Mailman /not/ using its own pythonlib is to save on disk space, and really, who cares about that any more? I submit that most applications of any size will have way more application data than duplicated Python libraries. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRWxVQ3EjvBPtnXfVAQIuMAQAkciyaHCwLnkN+8GwbhUro+vJuna+JObP AZaNzPKYABITqu5fKPl3aEvQz+9pNUvjM2c/q5p1m/9n34ZBURfgpHa3yk7QcbW0 sud8utdW6wMHMuWVw/1lQNaZ2GeJz9E4CgO93btfgiMLFIrcnBxr6uw5NqTrMwOc 4iIupbjYfUg= =Nxff -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Nov 28, 2006, at 2:41 PM, Mike Orr wrote:

On 11/28/06, Barry Warsaw wrote:

...

For distros like Gentoo or Ubuntu that rely heavily on their own system Python for the OS to work properly, I'm quite loathe to install Cheeseshop packages into the system site-packages. I've had Gentoo break occasionally when I did this for example (though I don't remember the details now), so I always end up installing my own /usr/ local/bin/python and installing my 3rd party packages into there. Even though site-packages is last on sys.path, installing 3rd party packages can still break the OS if the system itself installs incompatible versions of such packages into its site-packages.

One wishes distro vendors would install a separate copy of Python for their internal OS stuff so that broken-library or version issues wouldn't affect the system. That would be worth putting into the standard.

Agreed. But that would just eliminate one potential source of "application" conflict (defining the OS itself as just another application). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRWyYEnEjvBPtnXfVAQJCKQP7BXVOYUIvbEBgFK7nWHieBqRGXzohhKNZ SN5qV4P6uZGnCtjp1Z4W8U82X8TH+X3Ovx02mS+GN+nrlyF7AVhDr/mSLXI90Kan 1dqOhAIz5rBeT03/k0SpAPSiBhonl4zF4ZmezGaz3lif2CjsH6PT9153Mv7wXb1N ut2QIhXnejA= =jhbd -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Nov 28, 2006, at 4:05 PM, Guido van Rossum wrote:

On 11/28/06, Barry Warsaw wrote:

...

There's a related issue that may or may not be in scope for this thread. For distros like Gentoo or Ubuntu that rely heavily on their own system Python for the OS to work properly, I'm quite loathe to install Cheeseshop packages into the system site-packages.

I wonder if would help if we were to add a vendor-packages directory where distros can put their own selection of 3rd party stuff they depend on, to be searched before site-packages, and a command-line switch that ignores site-package but still searches vendor-package. (-S would almost do it but probably suppresses too much.)

I keep thinking I'd like to treat the OS as just another application, so that there's nothing special about it and the same infrastructure could be used for other applications with lots of entry level scripts. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRWzKAHEjvBPtnXfVAQK9AAQAsJS2Ag9yBO+dLGiZdJlaWAj64zWcd9oi zqaE95/y53iXBvMBynglROApDEdOsnv/1/XSx1+2gZVIkuFvHLplbqZWVCsZ56r+ nAcTzFXsM2zPBSECKWuSfxBUILKalRdaIXKOUjgd0iZTrCbt3EeTmZlxMTKq9sGU 1Scr8sHSpIE= =rNjl -----END PGP SIGNATURE-----

Guido van Rossum schrieb:

I wonder if would help if we were to add a vendor-packages directory where distros can put their own selection of 3rd party stuff they depend on, to be searched before site-packages, and a command-line switch that ignores site-package but still searches vendor-package. (-S would almost do it but probably suppresses too much.)

Patch #1298835 implements such a vendor-packages directory. I have reopened the patch to reconsider it. I take your message as a +1 for that feature. Regards, Martin

Hi Anthony, On Wed, Nov 29, 2006 at 12:53:14AM +1100, Anthony Baxter wrote:

...

python2.4 distutils is excluded by default.

I still have no idea why this was one - I was also one of the people who jumped up and down asking Debian/Ubuntu to fix this idiotic decision.

I could not agree more. Nowadays, whenever I get an account on a new Linux machine, the first thing I have to do is reinstall Python correctly in my home dir because the system Python lacks distutils. Wasteful. (There are some applications and libraries that use distutils at run-time to compile things, and I'm using such applications and libraries on a daily basis.) Armin

Jan Claeys wrote:

Op woensdag 29-11-2006 om 12:23 uur [tijdzone +0100], schreef Armin Rigo:

...

I could not agree more. Nowadays, whenever I get an account on a new Linux machine, the first thing I have to do is reinstall Python correctly in my home dir because the system Python lacks distutils. Wasteful. (There are some applications and libraries that use distutils at run-time to compile things, and I'm using such applications and libraries on a daily basis.)

I think you should blame the sysadmins, and kick them to install python properly for use by a developer, because every distro I know provides distutils... ;-)

I think the point is that some distros (Debian is the one that springs to mind most readily, but I'm not a distro archivist) require a separate install for distutils even though it's been a part of the standard *Python* distro since 2.3 (2.2?) So, it isn't that you can't get distutils, it's that you have to take an extra step over and above installing Python. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden

Jan Claeys wrote: [...]

Probably the Debian maintainers could have named packages differently to make things less confusing for newbies (e.g. by having the 'pythonX.Y' packages being meta-packages that depend on all binary packages built from the upstream source package), but that doesn't mean splitting "python" (or other projects) up in several packages is wrong. E.g. when installing on an flash drive, people are probably quite happy to leave the 20 MiB of Python documentation out...

Right, who cares about newbies, they're only the future of the language, after all. I take your point that some flexibility is advantageous once you get past the newbie stage, but I think that here we are talking about trying to avoid mis-steps that will potentially put people off making that transition.

Maybe python.org can include several logical "divisions" in the python.org distribution and make it easy for OS distro packagers to make separate packages if they want to, as most of them are quite happy to have less work to do, provided the upstream "divisions" do more or less what they want. ;-) (Oh, and such a division should IMHO also include a "minimal python" for embedded/low-resource hardware use, where things like distutils, GUI toolkits, a colelction of 20 XML libraries and documentation are most likely not needed.)

If only there were some guarantee that the distros would respect any project partitioning imposed by python-deb we might stand a chance of resolving these issues. By and large they do tend to go their own way, though. I suppose the only alternative is prominently-posted materials on python.org about "Python on Debian", "Python on Ubuntu", ... and various addition to the FAQs. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden

Jan Claeys schrieb:

Like I said, it's possible to split Python without making things complicated for newbies.

You may have that said, but I don't believe its truth. For example, most distributions won't include Tkinter in the "standard" Python installation: Tkinter depends on _tkinter depends on Tk depends on X11 client libraries. Since distributors want to make X11 client libraries optional, they exclude Tkinter. So people wonder why they can't run Tkinter applications (search comp.lang.python for proof that people wonder about precisely that). I don't think the current packaging tools can solve this newbie problem. It might be solvable if installation of X11 libraries would imply installation of Tcl, Tk, and Tkinter: people running X (i.e. most desktop users) would see Tkinter installed, yet it would be possible to omit Tkinter. Regards, Martin

Fredrik Lundh schrieb:

maybe we could just ask distributors to prepare a page that describes what portions of the standard distribution they do include, and in what packages they've put the various components, and link to those from the library reference and/or the wiki or FAQ? is there perhaps some machine-readable format that could be used for this?

Debian has its machine-readable (in some sense) package list at packages.debian.org. Through several links, you get to http://packages.qa.debian.org/p/python2.4.html which gives you the list of binary packages produced from the source package: idle-python2.4 python2.4 python2.4-dbg python2.4-dev python2.4-doc python2.4-examples python2.4-minimal Interestingly enough, _tkinter.so isn't included in any of these; instead, you have to look at http://packages.qa.debian.org/p/python-stdlib-extensions.html Depending on what precisely "this" is you want to use it for, there are other lists as well. Regards, Martin

On Fri, Dec 01, 2006 at 12:42:42AM +0100, Jan Claeys wrote:

Op donderdag 30-11-2006 om 21:48 uur [tijdzone +0000], schreef Steve Holden:

...

I think the point is that some distros (Debian is the one that springs to mind most readily, but I'm not a distro archivist) require a separate install for distutils even though it's been a part of the standard *Python* distro since 2.3 (2.2?)

So, it isn't that you can't get distutils, it's that you have to take an extra step over and above installing Python.

No, it just means that several parts of the python.org source package are spread over several binary packages, just like happens with hundreds or thousands of other packages, and any Debian (or Ubuntu or other distro doing this) administrator worth his or her money should be aware of that, and be able to find those packages.

In both the current Debian and Ubuntu releases, the "python2.4" binary package includes distutils. See for yourself at http://packages.debian.org/cgi-bin/search_contents.pl?searchmode=filelist&word=python2.4&version=stable&arch=i386&page=1&number=all if you like. So I'm not sure what the fuss is about.

Maybe python.org can include several logical "divisions" in the python.org distribution and make it easy for OS distro packagers to make separate packages if they want to, as most of them are quite happy to have less work to do, provided the upstream "divisions" do more or less what they want. ;-) (Oh, and such a division should IMHO also include a "minimal python" for embedded/low-resource hardware use, where things like distutils, GUI toolkits, a colelction of 20 XML libraries and documentation are most likely not needed.)

There's already a "python2.4-minimal" package in Debian/Ubuntu that would probably be a good starting point for an embedded distribution that cares about space more than providing a complete Python. -Andrew.

Armin Rigo a écrit :

Now I only have to hope that 2.4.4 makes its way out of 'unstable' soon. As far as I can tell sysadmins installing the current 'testing' would still be getting a Python 2.4.3, not modern enough to cope with the arithmetic overflow issues introduced by the cutting-edge GCC of 'testing'.

well, the usual way to express your "hope" would be to file a bug on the python2.4 package, to make sure the maintainers are aware of the problem. However, it looks like 2.4.4 made it to testing today, so it might not be necessary. Cheers, BC

Armin Rigo schrieb:

Ah, distutils are distributed in the base package now, but not the 'config' subdirectory of a standard library's normal installation, which makes distutils a bit useless.

I should go and file a bug, I guess. The reason why I did not do it so far, is that the fact that no one did tends to show that they really think distributing 'config' is not the "right" thing to do. A reasoning along the lines of: "this subdir contains include files, so it should go in the python-dev package". I'm not too interested in fighting this kind of preconception.

It really depends on what you are trying to do with distutils. If you want to package or install a distutils module, you need much more than distutils: you may also need a compiler, you may need the complete set of Python header files, and you may need header files for the libraries your package depends on. So if you use distutils to install a package, it's IMO reasonable to require that the -dev package is installed. People use distutils for other purposes today as well, and these purposes might be supported now. Regards, Martin

Robin Bryce schrieb:

python2.4 profile (pstats) etc, was removed due to licensing issues rather than FHS. Should not be an issue for python2.5 but what, in general, can a vendor do except break python if their licensing policy cant accommodate all of pythons batteries ?

If some vendor has a valid concern about the licensing of a certain piece of Python, they should bring that up while the LSB is being defined.

python2.4 distutils is excluded by default. This totally blows in my view but I appreciate this one is a minefield of vendor packaging politics. It has to be legitimate for Python / setuptools too provide packaging infrastructure and conventions that are viable on more than linux.

Again, that a decision for the LSB standard to make. If LSB defines that distutils is part of LSB (notice the *If*: this is all theoretical; the LSB doesn't yet define anything for Python), then each vendor can still chose to include distutils or not, if they don't, they wouldn't comply to that version of LSB. So it is *always* their choice what standard to follow. OTOH, certain customers demand LSB conformance, so a vendor that choses not to follow LSB may lose customers. I personally agree that "Linux standards" should specify a standard layout for a Python installation, and that it should be the one that "make install" generates (perhaps after "make install" is adjusted). Whether or not it is the *LSB* that needs to specify that, I don't know, because the LSB does not specify a file system layout. Instead, it incorporates the FHS - which might be the right place to define the layout of a Python installation. For the LSB, it's more import that "import httplib" gives you something working, no matter where httplib.py comes from (or whether it comes from httplib.py at all). Regards, Martin

Robin Bryce schrieb:

Yes, especially with the regard to the level you pitch for LSB. I would go as far as to say that if this "contract in spirit" is broken by vendor repackaging they should: * Call the binaries something else because it is NOT python any more. * Setup the installation layout so that it does NOT conflict or overlap with the standard layout. * Call the whole package something else.

I think that would be counter-productive. If applied in a strict sense, you couldn't call it Python anymore if it isn't in /usr/local. I see no point to that. It shouldn't be called Python anymore if it doesn't implement the Python language specification. No vendor is modifying it in such a way that print "Hello" stops working.

Is it a bad idea to suggest that: Python grows a vendor_variant attribute somewhere in the standard lib; That its content is completely dictated by a new ./configure argument which is the empty string by default; And, request that it is left empty by re-packagers if the installation is 'reasonably standard' ?

I'm not sure in what applications that would be useful.

I would strongly prefer _not_ write code that is conditional on such an attribute. However if there was a clear way for a vendor to communicate "This is not a standard python runtime" to the python run time, early failure (in the application) with informative error messages becomes much more viable.

Again: none of the vendors modifies Python in a way that what you get is "not a standard Python runtime". They *all* are "standard Python runtimes". Regards, Martin

Jan Matejek schrieb:

+1 on that. There should be a clear (and clearly presented) idea of how Python is supposed to be laid out in the distribution-provided /usr hierarchy. And it would be nice if this idea complied to FHS.

The LSB refers to the FHS, so it is clear that LSB support for Python will have follow the LHS. Specifically, LSB 3.1 includes FHS 2.3 as a normative reference.

It would also be nice if somebody finally admitted the existence of /usr/lib64 and made Python aware of it ;e)

I don't think this is really relevant for Python. The FHS specifies that 64-bit libraries must be in /lib64 on AMD64-Linux. It is silent on where to put Python source files and .pyc files, and, indeed, putting them into /usr/lib/pythonX.Y seems to be FHS-conforming: # /usr/lib includes object files, libraries, and internal binaries that # are not intended to be executed directly by users or shell scripts. In any case, changing Python is certainly out of the scope of the LSB committee: they might put requirements on Python installations, but it's not their job to "fix" Python. Regards, Martin