Mailman 3 April 2008 - Python-Dev

Re: [Python-Dev] [Distutils] how to easily consume just the parts of eggs that are good for you
by Phillip J. Eby 09 Apr '08

09 Apr '08

At 11:48 PM 4/9/2008 +0100, Paul Moore wrote: >On 09/04/2008, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > It would be, if .eggs were a packaging format, rather than a binary > > distribution/runtime format. > > > > Remember "eggs are to Python as jars are to Java" -- a Java .jar > > doesn't contain documentation either, unless it's needed at > > runtime. Same for configuration files. > >And yet, Java doesn't have an equivalent of easy_install for jar >files, to my knowledge. Actually, OSGi and Eclipse plugins and "feature sites" come quite close, and setuptools rips off many of its features from them. OSGi is basically a standard for additional .jar metadata to encompass dependencies and other info.

1 0

windows (was: how to easily consume just the parts of eggs that are good for you)
by Jim Jewett 09 Apr '08

09 Apr '08

> Are the Linux users happy with having a Python > package manager that ignores RPM/apt? Why > should Windows users be any happier? Because, as you noted, the add/remove programs application is severely limited. > I've read one too many "Windows is so broken > that people who use it obviously don't care about > doing things right" postings this week I'm honestly not sure that such fine-grained control is the right user interface, particularly for a non-shared system. But even if it were, Windows doesn't really have it, and it isn't so valuable that a solution which works only for python could do much better than the existing 3rd-party setup tools. As a windows user, I don't want python packages showing up in the add/remove programs list, because it won't help me, and will make the few times I do use that tool even more awkward. That said, I agree that if python does package management, offering windows users the choice of using that application is probably a good idea. The catch is that "package managers" seem to offer far more fine-grained power (even without dependency resolution) than windows. Duplicating this would add lots of complexity just for windows -- and still might not be all that useful. I'm already used to looking for an uninstall.exe in the directory of anything I can actually uninstall, and accepting that most things just don't go away cleanly. As a programmer, this feels wrong, but ... it is probably a good tradeoff for the time I don't want to spend maintaining things. If I really wanted a fancy tool that took care of dependencies and alternate versions, I would be willing to run something python-specific, or to treat each package as a subcomponent that I managed through "Change an existing program" applied to python. But realistically, I don't see such a tool being used often enough to justify inclusion in the core. -jJ

1 0

Re: [Python-Dev] [Distutils] how to easily consume just the parts of eggs that are good for you
by Phillip J. Eby 09 Apr '08

09 Apr '08

At 10:30 AM 4/9/2008 -0700, zooko wrote: >PEP 262 sounds like a non-starter to me because > >1. It appears to be backwards-incompatible with setuptools/ >easy_install/eggs, thus losing all the recently gained cooperation >that I mentioned in the previous paragraph, and No. It provides a forward-compatibility path for the distutils, so that easy_install doesn't have to install things in .egg format in the future. There's no compatibility breakage at all. >2. It defines a new database file, No, it defines *files*. One file per installed distribution, containing (among other things) an installation manifest. >where I would prefer either: > a. Doing away with database files entirely and relying on the >filesystem alone to hold that information, or ...which is what PEP 262 *does*. Unfortunately, PEP 262's title is bad for marketing, as you've effectively pointed out. It would be better titled something "package installation manifests" or "package contents files", or something of that sort. > b. Continuing to use the current ".pth" database file format, >possibly improved by having native support for .pth files in the >Python import machinery. These mechanisms are orthogonal to this issue. >3. Because of #2, it triggers programmers to exclaim "They are >planning to reinvent apt!", thus making it unlikely that the new >proposal will recapture the cooperation that setuptools has already >(slowly) gained. Yeah, we need a new name. Everybody is going off of "database of installed packages" and thinking "apt", because they aren't paying any closer attention. However, given that we are discussing this on Python-Dev and distutils-sig, I do think it's reasonable to expect (if perhaps not reasonable to receive) that people discussing the PEP have *read* the freaking PEP first, prior to trashing it or offering alternatives. And it's not like I'm personally offended or anything -- I didn't even write the PEP in question. But what's the point of having PEPs if people read nothing but their titles? We could just delete everything but PEP 0. :) >Perhaps PEP 262 and my proposal are not actually alternatives, but >are complementary. As I've already pointed out, your proposal does not address multiple installed versions of a package, and I see no sane way to modify it to do so. >What I want is for the already implemented, tested, and deployed >code- re-use features of setuptools/easy_install to be more widely >accepted. This is best and most easily achieved by fixing the two >most frequent objections to setuptools/easy_install: 1. That you >can't conveniently install into an arbitrary directory, and 2. that >it subverts the meaning of your PYTHONPATH. As I've already stated, the only way for these problems to be fixed is for easy_install to not install files in .egg form -- which also solves the general objection to using .eggs in the first place. And the only way to do that, is to have a way to keep track of what files are installed. Rather than have easy_install come up with its own way of doing that, I would prefer to share a standard with the distutils. Hence, the PEP discussion. For earlier versions of Python, it will still be possible to install and uninstall with setuptools using this approach. You just won't be able to uninstall pure distutils-based packages, unless you installed them using easy_install. Meanwhile, it has occurred to me that the easiest way of handling compatibility is not to require that other packaging tools mark their files for non-removability, but simply not allow easy_install to remove or overwrite anything that *isn't* claimed by a manifest. In that way, easy_install would be immediately usable in the new mode, without any updates to Python or to system packaging tools.

1 0

Re: [Python-Dev] Python Leopard DLL Hell
by Brian Cole 09 Apr '08

09 Apr '08

I have learned that this is a specific behavior of OS X. I have submitted a formal bug report to Apple about the problem. It appears that this is documented by Apple as acceptable: http://developer.apple.com/documentation/DeveloperTools/Reference/MachORefe… Whereas, linux will respect the fact you gave it a specific shared library: http://linux.die.net/man/3/dlopen If I am provided a workaround by apple I will post a python patch. A little scary that someone can circumvent my application by just setting an environment variable. -Brian Cole On Tue, Apr 8, 2008 at 7:52 PM, Michael Torrie <torriem(a)gmail.com> wrote: > Brian Cole wrote: > > That appears to be working correctly at first glance. The argument to > > dlopen is the correct shared library. Unfortunately, either python or > > OS X is lying to me here. If I inspect the python process with OS X's > > Activity Monitor and look at the "Open Files and Ports" tab, it shows > > that the _foo.so shared library is actually the one located inside > > $DYLD_LIBRARY_PATH. > > > > So this problem may not be python's, but I place it here as a first > > shot (maybe I'm using the imp module incorrectly). > > Sounds like you're going to need to learn how to use dtrace. Then you > can more closely monitor exactly what python and the loader are doing. > dtrace is very complicated (borrowed from Solaris) but extremely > powerful. Worth learning anyway, but sounds like it's probably the > debugging tool you need. > > Another thing you can do is check through the python source code and see > how the os x-specific code is handling this type of situation. > > -- > http://mail.python.org/mailman/listinfo/python-list >

1 0

fixing tests on windows
by Neal Norwitz 09 Apr '08

09 Apr '08

The Windows buildbots are still failing because some tests keep files opened. This causes subsequent tests which use the same file to fail. Here is a recent run which had a failure early on: http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1209/st… I'm assuming the first failure (test_bufio) was due to an open file. (Can't tell, no error msg.) That means the problem was in that test or an earlier test. The only earlier tests are: test_import test_sys test_descr test_xdrlib test_urllibnet test_binhex test_strptime test_importhooks test_copy test_hmac test_genericpath test_complex test_timeout test_quopri test_marshal test_zipfile test_mutants test_csv test_ucn test_hash test_wsgiref test_mmap test_ftplib test_pickletools test_codecmaps_cn Of those tests, only some of them reference TESTFN which is the probable file: test_import 22 test_descr 3 test_urllibnet 2 test_binhex 2 test_genericpath 44 test_complex 3 test_marshal 33 test_zipfile 136 test_mutants 7 test_mmap 32 I inspected test_mmap which had been modified somewhat recently and tried to fix a few things. I'm not sure I helped and there are probably more problems lurking. This issue may not be noticed in the default test run. It definitely occurs when the tests are run in a random order as they are on the buildbots. Try running: ./python.exe ./Lib/test/regrtest.py -r That should help provoke more errors. As always, patches are greatly appreciated. Cheers, n

12 33

py3k: print function treats sep=None and end=None in an unintuitive way
by Alessandro Guido 09 Apr '08

09 Apr '08

Can anybody please point me why print('a', 'b', sep=None, end=None) should produce "a b\n" instead of "ab"? I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some ml threads but did not find a good reason justifying such a strange behaviour. Thanks. -Alessandro Guido

4 8

Changing all network-oriented tests to facilitate (virtually unlimited) parallel test execution [Issue#: 2550]
by Trent Nelson 09 Apr '08

09 Apr '08

I've forwarding my most recent update to issue 2550 here such that the proposed patch (and in general, the approach to network-oriented test cases) can be vetted by a wider audience: http://bugs.python.org/file9980/trunk.2550-2.patch This patch works towards fixing a large proportion of the tests that were written in such a way that often leads to buildbot errors when port conflicts arise. With this patch applied, I can run many instances of the test suite in parallel and none of the network-oriented tests fail (which they do currently if you try and do this). Plenty of discussion (mostly me replying to my own comments) on the subject at: http://bugs.python.org/issue2550. Anyone have any issues with this new approach? I'm particularily interested in whether or not people disagree with the assumption I've drawn regarding never using SO_REUSEADDR in tests for TCP/IP sockets (see below). Trent. ________________________________________ From: python-bugs-list-bounces+tnelson=onresolve.com(a)python.org [python-bugs-list-bounces+tnelson=onresolve.com(a)python.org] On Behalf Of Trent Nelson [report(a)bugs.python.org] Sent: 08 April 2008 12:49 To: python-bugs-list(a)python.org Subject: [issue2550] SO_REUSEADDR doesn't have the same semantics on Windows as on Unix Trent Nelson <tnelson(a)onresolve.com> added the comment: Invested quite a few cycles on this issue last night. The more time I spent on it, the more I became convinced that every single test working with sockets should be changed in one fell swoop in order to facilitate (virtually unlimited) parallel test execution without fear of port conflicts. I've attached a second patch, trunk.2550-2.patch, which is my progress so far on doing just this. The main changes can be expressed by the following two points: a) do whatever it takes in network-oriented tests to ensure unique ports are obtained (relying on the bind_port() and find_unused_port() methods exposed by test_support) b) never, ever, ever call SO_REUSEADDR on a socket from a test; because we're putting so much effort into obtaining a unique port, this should never be necessary -- in the rare cases that our attempts to obtain a unique port fail, then we absolutely should fail with EADDRINUSE, as the ability to obtain a unique port for the duration of a client/server test is an invariant that we *must* be able to depend upon. If the invariant is broken, fail immediately (don't mask the problem with SO_REUSEADDR). With this patch applied, I can spawn a handful of Python processes and run the entire test suite (without -r, ensuring all tests are run in the same order, which should encourage port conflicts (if there were any)) without any errors*. Doing that now is completely and utterly impossible. [*] Well, almost without error. All the I/O related tests that try and open @test fail. I believe there's still outstanding work to do with this patch with regards to how the intracacies of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE should be handled in the rest of the stdlib. I'm still thinking about the best approach for this. However, the patch as it currently stands is still quite substantial so I wanted to get it out sooner rather than later for review. (I'll forward this to python-dev@ to try and encourage more eyes from people with far more network-fu than I.) Added file: http://bugs.python.org/file9980/trunk.2550-2.patch __________________________________ Tracker <report(a)bugs.python.org> <http://bugs.python.org/issue2550> __________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/tnelson%40onresolve…

2 2

[OT] Wingware IDE key for sprinters at PyCon
by Trent Nelson 08 Apr '08

08 Apr '08

Anyone happen to have the key handy that Wingware were giving out to sprinters at PyCon? For the life of me, I can't find what I did with that piece of paper. If someone could forward me it off list, that'd be great. Trent.

1 1

New project : Spyke python-to-C compiler
by James Coughlan 08 Apr '08

08 Apr '08

Hello, Your project sounds very interesting! I would be happy to send you some test code -- perhaps you could give me some more details about what you're looking for. (E.g. *small,* self-contained python/numpy functions.) Also, are you aware of Shed Skin? "Shed Skin is an experimental Python-to-C++ compiler. It can convert pure, but implicitly statically typed Python programs into optimized C++ code." http://shed-skin.blogspot.com/ Best, James -- ------------------------------------------------------- James Coughlan, Ph.D., Scientist The Smith-Kettlewell Eye Research Institute Email: coughlan(a)ski.org URL: http://www.ski.org/Rehab/Coughlan_lab/ Phone: 415-345-2146 Fax: 415-345-8455 -------------------------------------------------------

1 0

py3k: print function treats sep=None and end=None in an unintuitive way
by Alessandro Guido 08 Apr '08

08 Apr '08

Can anybody please point me why print('a', 'b', sep=None, end=None) should produce "a b\n" instead of "ab"? I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some ml threads but did not find a good reason justifying such a strange behaviour. Thanks. -Alessandro Guido

1 0