Mailman 3 March 2008 - Python-Dev

Introducing the ``make check`` command
by Brett Cannon March 18, 2008

March 18, 2008

After having one too many commits be rejected by having trailing whitespace, I wrote a script that runs reindent.py over all .py files that are to be checked in. But since there are other things that one should be aware of when creating a patch I let people know if they edited anything in the Doc directory, Misc/ACKS, and Misc/NEWS. There are a couple of things I would like to see the command gain. One is to detect if any files (outside of Doc) have changed since the last run of regrtest. That would require writing out a file, though, so one can at least stat the file to get the modification time. Do people have any issues with the idea? I would also like to more or less codify whether a patch means someone needs to be added to Misc/ACKS or not. I should be able to run ``svn diff`` to get the output and see if enough lines have changed. Could then write it out to a common file so that one does not need to run the command again if the patch is needed. Lastly, I would like to have it strip trailing whitespace in C files. The only problem is that I don't know if removing trailing whitespace could possibly cause a problem or not. Anyone know? -Brett

2 2

map, filter, zip in future_builtins
by David Wolever March 18, 2008

March 18, 2008

I'm working on #2171 -- putting map, filter, zip in 2.6's future_builtins. It has been suggested that it would be simplest to just return itertools.(imap, izip, ifilter), which is what py3k/Python/ bltinmodule.c, revision 61356 did. The advantage of this is that it's really easy and the behaviour seems to be identical. The disadvantage is that the two aren't identical: >>> type(map(lambda x: x, [1, 2, 3])) # Python 3 <type 'map'> >>> type(map(lambda x: x, [1, 2, 3])) == map True >>> type(map(lambda x: x, [1, 2, 3])) # Python 2.6, with the patch <type 'itertools.imap'> >>> type(map(lambda x: x, [1, 2, 3])) == map False Recommendations?

2 2

adding json to stdlib (was: Re: PEP 361: Python 2.6/3.0 release schedule)
by Neal Norwitz March 18, 2008

March 18, 2008

On Tue, Mar 18, 2008 at 11:17 AM, John Millikin <jmillikin(a)gmail.com> wrote: > > Possible features for 2.6 > > New modules in the standard library: > > - JSON implementation > > > Have there been any plans made for which one? All of the No. This was something I added as a nice to have for 2.6. > implementations I'm aware of (cjson, simplejson, jsonlib, demjson, > python-json) have serious issues that in my opinion should be fixed > before inclusion in the standard library[1]. I am the author of > jsonlib, and am willing to write patches for whichever implementation > becomes the standard, but it would be very nice to know what to focus > on. I'm not familiar enough with any of the libraries to comment. If it's premature to add a particular implementation, that's fine, we can wait. > Apologies if this has been discussed already, but there was no link to > a PEP in 361 and a search of python-dev and c.l.p returned no relevant > results. I don't believe it has been discussed before. I've changed the subject and would like to discuss this now. It would be great if you could pull together and get the community behind a single solution that is robust enough to include in the stdlib. If that an be finished for 2.6, great. If it waits for 2.7, that would still be fine. n > > [1] > * cjson and python-json have almost complete absense of Unicode support. > * simplejson is slow and writes incorrect unicode escapes. > * demjson is far too forgiving when parsing and accepts all sorts of > invalid input. > * jsonlib is not PEP 8-compliant and has terrible error handling. > * python-json is abandoned, slow, and lacks Unicode support. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com >

2 1

Introducing test coverage stats
by Jerry Seutter March 18, 2008

March 18, 2008

I have added a bugtracker issue that adds unit test coverage statistics. Issue 2403: http://bugs.python.org/issue2403 Directions are on the issue page. Titus: Brent suggested I bug you to review this. Test, complain, flame. Feedback welcome. Jerry Seutter

1 1

Re: [Python-Dev] PEP 365 (Adding the pkg_resources module)
by Phillip J. Eby March 18, 2008

March 18, 2008

At 12:59 PM 3/17/2008 -0500, Guido van Rossum wrote: >On Mon, Mar 17, 2008 at 12:45 PM, Phillip J. Eby ><pje(a)telecommunity.com> wrote: > > At 12:17 PM 3/17/2008 -0500, Guido van Rossum wrote: > > >There will be no egg support in the standard library. > > > > Are there any qualifications on that statement, or is this in the > > same category as "from __future__ import braces"? > >IIUC eggs are a method of package management that includes support for >dependencies, multiple versions, and C extensions in zip files, as >well as conventions for naming these and for encoding metadata (e.g. >how to find out the version or the dependencies). This whole set of >conventions is IMO too much to include into the stdlib ATM -- if only >because it has proved controversial in the past. Maybe a few years >from now it will be no longer controversial and then my objections >will disappear. So, does this mean that the bootstrap tool must not use eggs? That seems a little bit odd, in that setuptools will at least need its .egg-info directory to get installed, and all of the people who'll be using this initially will be using it precisely in order to have support for eggs... So, it might be simpler all around to just clear up the "controversy". To the best of my recollection, only MAL and MvL have ever objected on Python-Dev to the idea of supporting eggs. Note: I'm specifically segregating "egg support" from the topic of including setuptools or easy_install in the stdlib directly. There are many legitimate reservations and open questions about the latter, including availability of volunteer support, choice of defaults, whether to replace distutils with setuptools, etc. etc. I recognize and respect the validity of those issues, which is precisely why I withdrew setuptools from inclusion in Python 2.5. However, regarding support for eggs, my understanding is that there were only two objections to eggs -- even at the time of the 2.5 setuptools discussions. And even though MvL objects to the idea of eggs in *principle*, I didn't read his recent posts as objecting to having the bootstrap tool download and install eggs in *practice*. (Although I hope he will clarify that stance one way or the other.) That leaves MAL, whose objections to PEP 365 centered on the API (he said he was "+1 on the concepts being added to the stdlib, -1 on adding the module in its current state"). Among other concerns, he wanted pkg_resources to be split into pkgutil and a new "egglib" module. I don't have a problem with this in principle, if there were a pkg_resources module that reconstituted the merged API. (But there are some practical problems with that approach, such as trying to split namespace package support between two theoretically-unrelated modules.) I would guess, however, that MAL's issues with the pkg_resources API would not apply to a bootstrap module whose sole purpose was to download eggs and put them on sys.path. Or, perhaps he would object *more*, I don't know. We could certainly ask him, though. :) So, was there anyone else you were counting towards "controversy"? The only other person I recall objecting to setuptools in any way on Python-Dev was effbot, and IIUC his objections were practical/administrative re: supporting easy_install and setuptools, not to the idea of .egg support in general. In summary, I think the controversy on Python-Dev regarding .egg support has actually been over for some time now.

4 5

Re: [Python-Dev] PEP 365 (Adding the pkg_resources module)
by Phillip J. Eby March 18, 2008

March 18, 2008

At 12:31 AM 3/18/2008 -0500, Guido van Rossum wrote: >I am hoping that someone will create a simpler bootstrap module that >is able to download a file of pure Python code and install it, perhaps >by running its setup.py, assuming that it only depends on distutils >(or other things previously installed). I will welcome such a module >into the stdlib. I'm not sure a PEP is even needed, though interested >parties are certainly welcome to write a PEP specifying the behavior >first. With 2.6 and 3.0 slated for release in September, there should >be enough time to get this done before then. Unfortunately, as I've already tried to explain, "download a file ... and install it" is not a sufficiently well-specified requirement to implement a robust tool. Even if it is not to support arbitrary existing distutils sources, there still needs to be a way to document precisely what the tool does and does not support installing, so that users can produce correct files for it to consume, register them properly with PyPI, etc. And as I said before (perhaps not very well) the distutils documentation has already proven to be inadequate as such a specification, both for users to create these files -- and even more important -- for programs to consume them. (For example, distutils source distribution tarball filenames are not unambiguously machine-parseable.) That's why I think some specific "format" (i.e. conventions) have to be defined for this to work, even if it's merely a set of documented restrictions on a distutils-based layout, file naming conventions, versioning, etc. In other words, you can't have your cake and eat it, too. If there's to be a bootstrap tool, you must bless *some* set of packaging conventions, including file naming, version parsing, and so on. Can we use setuptools' version parsing scheme to identify the "latest stable version", for example? What about setuptools' filename component canonicalization and escaping rules? Frankly, I don't care what the conventions are, only that they be unambiguously defined and reasonably implementable for producers and consumers alike. I just want there to be *some* sort of robust, documented, standard installation bootstrap vector in the stdlib, so that setuptools, zc.buildout, and virtualenv don't have to maintain their own (or depend on setuptools maintaining its own). But not only have you rejected the *only* existing robust and well-documented conventions for automated processing of Python libraries, you say you "have no time for this part of the thread" when I ask what conventions you want to bless *instead*. So I'm at a bit of a loss for what we're supposed to do now.

2 1

Re: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns
by Jeff Rush March 18, 2008

March 18, 2008

Marius Gedminas wrote: > On Mon, Mar 17, 2008 at 08:37:30PM -0400, Phillip J. Eby wrote: >> At 05:10 PM 3/17/2008 -0500, Jeff Rush wrote: >>> People also want a greater variety of file_finders to be included with >>> setuptools. Instead of just CVS and SVN, they want it to comprehend >>> Mercurial, Bazaar, Git and so forth. >> Did you point them to the Cheeseshop? There are plugins already >> available for all the systems you mentioned, plus Darcs and >> Monotone. If you mean "included" as in "bundled", this doesn't make >> a whole lot of sense to me. They knew there were plugins out there, of various quality and availability but wanted them bundled. ;-) It's a pain to track them down. Perhaps if the RPM format were broken out from setuptools, as the inclusion of some formats leads them to believe the set is just incomplete, not intentionally sparse. >> I'd think that if you're using >> setuptools as a developer (the only reason you need the file finders, >> since source distributions include a prebuilt manifest), you'd not >> have a problem saying "easy_install setuptools-git" or adding a >> "setup_requires='setuptools-git'" line to your setup.py. (Although >> the latter would only be needed for *development*, not deployment.) > > setup_requires looks like a solution, but it requires extra attention > from the developers who write the setup.py. Writing a setup.py is > already quite complicated -- I usually end up copying an existing one > and modifying it. As a compromise, of making new formats easily available but not bundled, and not requiring special action within setup.py, setuptools could treat --formats=dpkg as an implicit setup_requires= and pull it from PyPI. And the --list-formats option could query PyPI for the possibilities, just as --list-classifiers does today. If would require a few standards in keywording/classifying those format eggs but we already need those standards for other projects, such as locating recipes for buildout and plugins for trac. -Jeff

1 0

Change in priority fields
by Brett Cannon March 18, 2008

March 18, 2008

Barry, Neal, and myself had a conversation and changed the priority fields in the tracker. You can click on 'priority' to see an explanation, but the new fields are: - release blocker - critical - high - normal - low So "release blocker" blocks a release. "Critical" could very easily block a release, but not the current one. "High" issues should be addressed, but won't block anything. "Normal" is normal. And "low" is for spelling errors and such. -Brett

5 8

PEP 361: Python 2.6/3.0 release schedule
by Barry Warsaw March 18, 2008

March 18, 2008

Greetings from Pycon 2008! Neal Norwitz and I have worked out the schedule for Python 2.6 and 3.0, which will be released in lockstep. We will be following a monthly release schedule, with releases to occur on the first Wednesday of the month. We'll move to a 2 week schedule for the release candidates. Executive summary: Python 2.6 and 3.0 finals are planned for September 3, 2008. PEP 361 contains all the gory details; from the PEP: Feb 29 2008: Python 2.6a1 and 3.0a3 are released Apr 02 2008: Python 2.6a2 and 3.0a4 planned May 07 2008: Python 2.6a3 and 3.0a5 planned Jun 04 2008: Python 2.6b1 and 3.0b1 planned Jul 02 2008: Python 2.6b2 and 3.0b2 planned Aug 06 2008: Python 2.6rc1 and 3.0rc1 planned Aug 20 2008: Python 2.6rc2 and 3.0rc2 planned Sep 03 2008: Python 2.6 and 3.0 final http://www.python.org/dev/peps/pep-0361/ This schedule gives everybody plenty of advanced notice, so you should be able to get your code in on time. Please be very careful about not breaking the branches. Neal and I will be cracking the whip of public shame when your commit turns the buildbots red. Embarrassing Pycon pictures of you will be posted if such broken revisions cause us to slip a release, and remember, we know how to use GIMP. On behalf of everyone, here's to an awesome release! -Barry Python 2.6/3.0 release manager

6 6

test_support.have_unicode
by Virgil Dupras March 18, 2008

March 18, 2008

The test_support unit has this have_unicode. Do we need the Python's test unit to be *that* backward compatible? Is there still an implementation of Python that doesn't support unicode? If there is, should the test suite care? As a side question. Considering that I'm not sure whether have_unicode is relevant or not, is it more appropriate to create a ticket for it or to ask python-dev? Virgil Dupras

5 5