Mailman 3 March 2009 - Python-Dev

Re: [Python-Dev] "setuptools has divided the Python community"
by P.J. Eby 25 Mar '09

25 Mar '09

At 10:11 AM 3/25/2009 -0500, Olemis Lang wrote: >... but Trac plugins *do require* egg files ... (AFAIK after reading >Trac docs and implementation of plugin upload from /admin/plugins, egg >cache for plugins ... and so on ...) and this is what I was talking >about ... apps (e.g. Trac) depending *today* on setuptools (e.g. egg >files ...) Trac uses entry points to access plugins. Those plugins do not require .egg files, unless Trac specifically wrote code of their own that requires .egg files. If you install a plugin from source using pip or a system package manager, no .egg file is involved -- you'll have an .egg-info directory instead, alongside the installed code. The pkg_resources API for accessing entry points works with either .egg files or .egg-info directories.

3 5

More GSoC - make sure you list your projects!
by C. Titus Brown 25 Mar '09

25 Mar '09

Hi all, just a quick note -- anyone who has an idea for a core Python Google Summer of Code project, has not had that project panned on this list ;), and is willing to act as a CONTACT for students who want to apply (not necessarily a mentor!) should make sure that those ideas are posted on the wiki, at http://wiki.python.org/moin/SummerOfCode/2009 Without a contact and a link on the wiki page, students won't have any idea that the project exists... I will endeavor to find mentors for as many of the core Python projects as possible, once we have a clearer sense of how many applicants we have and how many applications are good. cheers, --titus -- C. Titus Brown, ctb(a)msu.edu

1 0

In case I seem to disappear...
by Guido van Rossum 25 Mar '09

25 Mar '09

I've received several emails recently indicating that someone is trying to get me unsubscribed from python-dev. I'm sure they're just pranksters, but just in case they succeed, if I seem to be absent for a really long time, let me know. After all mailman passwords are not secure. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Re: [Python-Dev] "setuptools has divided the Python community"
by P.J. Eby 25 Mar '09

25 Mar '09

At 08:32 AM 3/25/2009 -0500, Olemis Lang wrote: >Sometimes it also happens that, once one such build/packaging systems >is adopted, it is difficult to switch to using another one, since apps >(... and plugins systems ...) are frequently hard-coupled to the pkg >sys «exotic features» and support (... what comes to my mind right now >is Trac plugins which are a little bit dependant on setuptools entry >points and so on ...). Please note that entry points are not coupled to easy_install. They have a documented file format and API that's *distributed* with setuptools, but is not dependent on it and does not require .egg files, either. There's nothing stopping an independent implementation in the stdlib, ala PEP 376's metadata API, which is designed to be backward compatible with setuptools. It could be extended to support entry points as well.

3 2

Re: [Python-Dev] "setuptools has divided the Python community"
by P.J. Eby 25 Mar '09

25 Mar '09

At 07:40 AM 3/25/2009 -0700, Guido van Rossum wrote: >Well, sorry, but this complex layered interdependent architecture is >one of the *causes* of confusion -- apart from you nobody knows what >is what exactly, I'll pick a minor nit here... buildout, pip, and a wide variety of other tools and frameworks out there wouldn't exist if I were really the only person who knew "what's what". And I believe most of those people will be at the summit. That having been said, the setuptools documentation definitely sucks for the casual reader or curious observer, as opposed to those who have serious itches to scratch in the area of dependencies or plugins. Combine that with the practical-but-impure choices I made in easy_install to get *something* working, and you have a recipe for the current situation. pkg_resources, for example, is only bundled with setuptools because it couldn't go in the stdlib when it was written. easy_install, OTOH, is bundled with setuptools because *setuptools* isn't in the stdlib! (And of course, both use pkg_resources.) So ironically, setuptools is bundled in the way that it is, precisely *because* there's no support for dependencies in the stdlib... and nicely illustrates why smaller libraries (and less bundling) is a *good* thing.

1 0

"setuptools has divided the Python community"
by Olemis Lang 25 Mar '09

25 Mar '09

On Wed, Mar 25, 2009 at 8:36 AM, Tarek Ziadé <ziade.tarek(a)gmail.com> wrote: > On Wed, Mar 25, 2009 at 1:25 PM, Antoine Pitrou <solipsis(a)pitrou.net> wrote: >> Paul Moore <p.f.moore <at> gmail.com> writes: >>> >>> 3. Setuptools, unfortunately, has divided the Python distribution >>> community quite badly. >> >> Wait a little bit, and it's gonna be even worse, now that buildout and pip seem >> to become popular. For example, the TurboGears people are considering switching >> from setuptools to pip... >> >> Combined with >> the current trend that everything must be exploded into lots of interdependent >> but separately packaged libraries (the paste/pylons syndrome... try pulling >> something like TurboGears2 and see how many third-party packages it installs), I >> fear this is going to generate a very painful user/developer experience :-( >> > > I think we are in a point in Python development where we need to clearly define > how dependencies work. +1 > And this has to be defined in Python (in Distutils) > for the sake of standardization. > Well, I mentionned I did it using apt + insinstall support ... but I'd love to have all these things in Python ... with or without relying on OS-specific infrastructure for managing pkgs, deps and so on ... It is possible to have two approaches : - a thin layer on top of OS-specific (dpkg + apt, rpm + yum, wininstall, mac os ... ) pkg managers so that distutils be able to use them directly whille still keeping a single model from the end-user perspective ... At least there exists some kind of python-debian pkg ... {{{ $ apt-cache show python-debian Package: python-debian Priority: optional Section: universe/devel Installed-Size: 268 Maintainer: Ubuntu MOTU Developers <ubuntu-motu(a)lists.ubuntu.com> Original-Maintainer: Debian python-debian Maintainers <pkg-python-debian-maint(a)lists.alioth.debian.org> Architecture: all Version: 0.1.9 [...] Depends: python (>= 2.4), python-support (>= 0.7.1) Recommends: python-apt [...] Size: 43662 [...] Description: python modules to work with Debian-related data formats This package provides python modules that abstract many formats of Debian related files. Currently handled are: * Debtags information (debian_bundle.debtags module) * debian/changelog (debian_bundle.changelog module) * Packages files, pdiffs (debian_bundle.debian_support module) * Control files of single or multple RFC822-style paragraphs, e.g debian/control, .changes, .dsc, Packages, Sources, Release, etc. (debian_bundle.deb822 module) * Raw .deb and .ar files, with (read-only) access to contained files and meta-information [...] }}} Since there are a lot such systems (at least for Linux ...) ... this could also mean that it could be very complex to handle all this diversity ... However there are a few which are highly influential ... but this is certainly a concern ... - A fully fledged implementation from scratch so that distutils be in charge, all apps be consistent with Py-std ... and users have potentially two pkg systems (.. the built-in and the Py-specific ;) ; and I say potentially since it's quite sudden so far and perhaps there is a way to smoothly integrate both systems (which are potentially many of them ;) > I think the Language Summit tomorrow is a good place to discuss about > these problems, Hope it be available (recorded ?) some day so as to hear some opinions ... ;) > and to make sure pip, setuptools and zc.buildout rely on the same > standards and pieces. > Oh yes ! +1 > PEP 376 is my first attempt to make it happen, and my goal is to see another > pre-PEP coming out of thea language summit, adressing the dependencies problem. > ;) > I can't hear that setuptools has divided the Python community. Said like this ... some might think that setuptools is responsible for someone else's choices and actions ... and I dont think so ... > It has provided > solutions to real problems we had in web development. It's unperfect, > and it has to be > fixed and integrated into Python. But it should not be done outside Python imho. > I mostly agree with all this, but I would like to know something ... in case a single pkg management sys be ready for Py (... I mean integrated into Py ...) and so on ... Why will we need distutils comands like : bdist_msi bdist_wininst bdist_rpm <... more «non-standard» candidates like py2exe, stdeb, ...> ? Will all this still be necessary and maintained ? As long as installing /uninstalling them be handled correctly in built-in as well as Py systems (considering the duplicate «package repositories» ...) ... I think this could be fine ... but details are details ... ;) > If you had worked with Zope 5 years ago, you would understand what > setuptools and > zc.buildout brought to these communities. And today's experience is a > whole less painful trust me. > Yes ... setuptools has undoubtedly its own value ... :o) > But I agree that the sizes of the packages are too small now, and it has gone > to far. Installing a web app like Plone is scary (+100 packages) > > But this is the responsability of Zope, TG, etc to distribute their packages in > bigger pieces I guess. > I do think so ... The same happens with Trac plugins. In case of browsing the archive of trac-users mailing list anyone can realize that there are many related plugins people'd like to install at once to get the whole functionality. Nowadays in some cases they have to install ... *a loooot* and then configure ... *a loooot* in order to get the whole thing working out ... and there are tiny but useful plugins ... Besides further contributions seldom end-up with potential interoperability issues and related features might not be tested as a whole ... And things that should be in core (e.g. AccountManager plugin ... since it seems there is no way to log out if it's not installed ... ;) are not still there (... took some time until trac-devs decided to incorporate it during PyCon'09 sprint ... ;) But this unification is definitely up to the plugin & framew devs ... IMO -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article:

1 0

tracker status options
by Tennessee Leeuwenburg 25 Mar '09

25 Mar '09

Hi all, I'm continuing to (slowly) work through issues. I have been looking particularly at a lot of the open issues regarding strftime. It would be great to put in some of those extra status options that were discussed recently... "Open/New" "Needs help / Chatting" "Under development" "Pending feedback" "Closed" For everyone's reference, after some debate, the above list of status options was where the conversation finished. So the two options "Needs help / chatting" and "Under development" would need to be added. Issues that are "Under development" should revert to "Needs help / chatting" after a month of inactivity. Would it be possible to get these put in? Maybe I'm one of a small number of people who are nibbling at the bottom end of the bugs, but I find it somewhat frustrating not to be able to classify the issues that I find into "needs help / chatting" vs "under development" to help make more sense of the search results. Cheers, -T

7 13

tracker status options
by R. David Murray 25 Mar '09

25 Mar '09

On Tue Mar 24 02:06:29 CET 2009, Tennessee Leeuwenburg <tleeuwenburg at gmail.com> wrote: > Perhaps some developers have a well-established workflow and interpret these > flag in a particular, consistent fashion, however part of the purpose of the > issue tracker is to allow a diverse group to work on development as a group. > On that basis, I feel that more documentation, and clearer terminology, is > required in order to support the bug resolution process. > > I have identified some gaps in the workflow process which impact me > personally in classifying issues. These issues will not impact on all > developers; clearly they will be entirely superfluous, perhaps even > confusing, for some individuals. However the impact still remains for some. > There appears to be a general agreement that ability to distinguish between > issues on the following bases would be useful: > * Whether the nature of the requirement is still under debate or whether > it is well-understood So, having triaged a few issues, here are my thoughts. The current workflow is roughly: o test needed o patch needed o patch review o commit review One can look at these and see what needs to be done "next". I think that in practice the above list actually expands something like this: o consensus needed o test needed o patch needed o patch needs work o patch review o commit review The first of these additional items is equivalent to your bullet item above. I would propose that the issue, regardless of whether or not it is a bug fix or a feature request, goes into 'consensus needed' whenever there is significant debate as to the validity of the bug or feature request. It moves to an appropriate later stage when agreement has been reached (which may be by fiat of a developer or the BDFL....so the triage people would need to know who can "pronounce"). The second addition is not as clearly useful. 'patch needs work' could be the result of a developer review, or of any other review. That is, at that stage we are trying to reach consensus that the patch is the correct solution to the problem or feature request. An issue could bounce back and forth between 'patch review' and 'patch needs work' multiple times (and it would probably be best if the patch submitter could request 'patch review'). But one could argue that the issue could just as easily be bounced back and forth between 'patch review' and 'patch needed', since one would need to read the comment stream to figure out what was actually going on anyway. I think it would be a useful addition, since it would enable someone to search for 'patch needed' in order to look for issues to pick up, whereas 'patch needs work' would indicate someone was working on it. (Of course, that someone could also stop working on it...but if one wished to find such issues, one could simply sort 'patch needs work' issues by last activity date.) Hmm. Or perhaps it should be "patch needs consensus", given the issue I'm looking at where I want to set it to this stage :) > * Whether the issue is "up for grabs" by any developer or whether > responsibility lies with an individual or group already Isn't that covered by 'assigned to'? > * Whether the issue is ready for more serious consideration by more > experienced or core developers Hmm. Theoretically that's covered by 'patch review'. Given that 'commit review' is (currently?) reserved for issues being considered for addition to a release candidate or final release, perhaps we need an additional stage 'core review' that would come after 'patch review'. Then triage could promote issues from 'patch review' to 'core review' if triage thinks it is ready for review by someone with commit privileges. This of course assumes that people other than core developers are actually doing patch review. I certainly intend to, but how many of us are there really going to be? -- R. David Murray http://www.bitdance.com

5 8

Re: [Python-Dev] speeding up PyObject_GetItem
by Hrvoje Niksic 25 Mar '09

25 Mar '09

Nick Coghlan wrote: >> Many of the routines in abstract.c check their parameters for NULL, as a >> sanity check, and throw a SystemError if NULL is detected. I'm tempted >> to suggest making these checks only when Py_DEBUG is defined, but I >> suspect if you wanted it that way, you would have changed it already. ;) >> >> Assuming you really want the NULL checks in production Python, there are >> 5 checks for NULL even though there are only two parameters. Seems like >> overkill? > > The main problem is that many of these methods are not only used > internally, but are *also* part of the public C API made available to > extension modules. We want misuse of the latter to trigger exceptions, > not segfault the interpreter. Agreed, and more importantly, I have yet to be convinced that those NULL checks introduce a measurable slowdown. Daniel, have you tried measuring the performance difference with only the NULL checks removed?

7 13

Re: [Python-Dev] GSoC 2009: Roundup / Python Tracker enhancements
by Daniel (ajax) Diniz 25 Mar '09

25 Mar '09

Aahz wrote: > Is this for GSoC? If yes, please make sure to include that tag in the > Subject: line to make it easier to track. Oops, makes a lot of sense :) Daniel

1 0