setuptools development cycle
I'm a little bit worried about setuptools development cycle. We seem to be stalled at a 0.6 pre-release that is quite stable and widely used in production. The next feature release, 0.7, seems to anticipate a major refactoring and major new features. I'm a bit worried about this both from a fear of potential disruption, and a need for incremental feature development.
I propose that the grand re-factoring of setuptools planned for 0.7 be moved off the trunk and to a 2.x release. 0.6 should be marked final. Is it on the table to remove features from setuptools? If not, then I think it should setuptools should get a 1.0 release. (I do still think that setuptools, while wonderful, is too complex, so I'd love to see it simplified.) I'd then like to see incremental feature development in the 1.x line.
I'd be interested in hearing Phillip's vision for setuptools' future.
Jim
-- Jim Fulton Zope Corporation
At 10:26 AM 10/3/2007 -0400, Jim Fulton wrote:
I'm a little bit worried about setuptools development cycle. We seem to be stalled at a 0.6 pre-release that is quite stable and widely used in production. The next feature release, 0.7, seems to anticipate a major refactoring and major new features. I'm a bit worried about this both from a fear of potential disruption, and a need for incremental feature development.
I propose that the grand re-factoring of setuptools planned for 0.7 be moved off the trunk and to a 2.x release. 0.6 should be marked final.
This is coming soon, but I want to finish some ez_setup improvements first. There will be a "known issues" list to cover things like the incomplete/broken FTP support; I've given up on trying to keep hacking it into 0.6.
Is it on the table to remove features from setuptools?
Yes - and most are *already* removed from the 0.7 trunk. That was the main reason for the branch.
Other (relatively minor) refactorings expected in 0.7 are to make URL handlers for easy_install pluggable, and to add build-time plugin options (so that people can implement custom build steps).
The biggest feature additions in 0.7 would be support for uninstallation, package inspection, virtual environment management, and ability to easy_install eggs using "classic/development" (i.e., .egg-info) installation layout.
Other new features planned for 0.7 include "or-ed" requirements, build/install of shared libraries, and "standard extras" (a way to specify to easy_install a set of extras that should be installed if they are defined by the targets).
None of this is what I'd consider "grand" refactoring; they're all either new features or minor refactorings to make things pluggable. The most complex of them is probably the URL handler one, and only because the existing URL handling in setuptools.package_index is so hosed.
Now, if you include pkg_resources in, I would rather like to refactor the way the global objects (working set and resource manager) are handled, but I don't know how practical or even possible it is. There are other things in pkg_resources that could use refactoring, but I'm not sure I dare.
On Oct 3, 2007, at 1:06 PM, Phillip J. Eby wrote:
At 10:26 AM 10/3/2007 -0400, Jim Fulton wrote:
I'm a little bit worried about setuptools development cycle. We seem to be stalled at a 0.6 pre-release that is quite stable and widely used in production. The next feature release, 0.7, seems to anticipate a major refactoring and major new features. I'm a bit worried about this both from a fear of potential disruption, and a need for incremental feature development.
I propose that the grand re-factoring of setuptools planned for 0.7 be moved off the trunk and to a 2.x release. 0.6 should be marked final.
This is coming soon, but I want to finish some ez_setup improvements first. There will be a "known issues" list to cover things like the incomplete/broken FTP support; I've given up on trying to keep hacking it into 0.6.
I wonder what "this" you are referring to.
Is it on the table to remove features from setuptools?
Yes - and most are *already* removed from the 0.7 trunk. That was the main reason for the branch.
I'm confused on a number of counts. Is the 0.7 "trunk" the same as the setuptools trunk? Is there also a branch for 0.7?
Is there a list of removed features somewhere? Do we get any input?
Other (relatively minor) refactorings expected in 0.7 are to make URL handlers for easy_install pluggable, and to add build-time plugin options (so that people can implement custom build steps).
The biggest feature additions in 0.7 would be support for uninstallation, package inspection, virtual environment management, and ability to easy_install eggs using "classic/ development" (i.e., .egg-info) installation layout.
Other new features planned for 0.7 include "or-ed" requirements, build/install of shared libraries, and "standard extras" (a way to specify to easy_install a set of extras that should be installed if they are defined by the targets).
None of this is what I'd consider "grand" refactoring; they're all either new features or minor refactorings to make things pluggable. The most complex of them is probably the URL handler one, and only because the existing URL handling in setuptools.package_index is so hosed.
Now, if you include pkg_resources in, I would rather like to refactor the way the global objects (working set and resource manager) are handled, but I don't know how practical or even possible it is. There are other things in pkg_resources that could use refactoring, but I'm not sure I dare.
"Grand" is simply the impression I had. The list of features above is extensive though. Some of the items seem fairly big, or maybe I just can't tell what they are. Do you plan to try to get all of them into the next feature release? Or do you plan them a list of ideas for future releases.
Jim
-- Jim Fulton Zope Corporation
At 01:12 PM 10/3/2007 -0400, Jim Fulton wrote:
On Oct 3, 2007, at 1:06 PM, Phillip J. Eby wrote:
At 10:26 AM 10/3/2007 -0400, Jim Fulton wrote:
0.6 should be marked final.
This is coming soon, but I want to finish some ez_setup improvements first. There will be a "known issues" list to cover things like the incomplete/broken FTP support; I've given up on trying to keep hacking it into 0.6.
I wonder what "this" you are referring to.
"0.6 should be marked final."
Is it on the table to remove features from setuptools?
Yes - and most are *already* removed from the 0.7 trunk. That was the main reason for the branch.
I'm confused on a number of counts. Is the 0.7 "trunk" the same as the setuptools trunk? Is there also a branch for 0.7?
Is there a list of removed features somewhere? Do we get any input?
Yes, no, no, and maybe, respectively. :)
"Grand" is simply the impression I had. The list of features above is extensive though. Some of the items seem fairly big, or maybe I just can't tell what they are. Do you plan to try to get all of them into the next feature release?
They'll probably all appear over the course of a long string of alphas. I don't have a detailed implementation plan yet.
The current status of setuptools is basically this: my initial creation was to scratch various distutils-related itches. After OSAF became my client, my distutils-related itches got a lot bigger. :) OSAF funding got it to where it is today, but for all practical purposes both mine and OSAF's itches are fairly-well scratched, so I don't have much call to work on it these days -- I spend more time *supporting* it than working on it. And OSAF's goals for the next year or so will likely not justify much in the way of focused development.
So, unless I end up with other client(s) who have setuptools-related needs, I'm unlikely to have much time available for setuptools development in the next year. I could probably knock out everything but the biggest 0.7 features in a month, if that was all I had to work on. But the odds of that month being one of the next 12 on the calendar seems slim right now.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Phillip J. Eby wrote:
Is it on the table to remove features from setuptools?
Yes - and most are *already* removed from the 0.7 trunk. That was the main reason for the branch.
I'm confused on a number of counts. Is the 0.7 "trunk" the same as the setuptools trunk? Is there also a branch for 0.7?
Is there a list of removed features somewhere? Do we get any input?
Yes, no, no, and maybe, respectively. :)
I've just done a comparison of the changelogs for the 0.6 branch and the trunk, since the 0.6 branch was committed (nearly 18 months ago, 2006-04-14). I'm attaching the analysis, but the summary would be that I don't see any work unique to the trunk, beyond perhaps some prep work for the goal of including setuptools in the standard library for Python 2.5. Basically, all real feature changes and bug fixes have been backported to the 0.6 branch.
At this point, would we be better off moving aside the current trunk, and making a new trunk from the 0.6 branch after getting 0.6 finally released? We would then "cherry pick" any remaining important changes from the old trunk, while moving forward on top of a codebase which is actually getting used.
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com
Comparing setuptools-0.6 branch and setuptools-trunk
SVN URLs
- 0.6 branch:
http://svn.python.org/projects/sandbox/branches/setuptools-0.6
- trunk:
http://svn.python.org/projects/sandbox/trunk/setuptools
Notes
- 0.6 branch was created from the trunk in r45397, 2006-04-14, with the following comment:
Creating a branch for the "stable" 0.6 development line of setuptools, so that it can continue forward to 0.6b1 and onward, while using the trunk for 0.7 development and Python 2.5 snapshots thereof.
- (r51932,51935,51937) backport a bunch of 2.5 compatibility work (r51935 says "all known")
Changes on trunk
- (r58257) Fix cross-platform line-end problem with SOURCES.txt: distutils doesn't expect manifest files to be shipped across platforms. (backported to 0.6 branch, r58529)
- (r57964) Remove all references to the Cheese Shop, and switch to using the new PyPI "/simple" REST API. (backported to 0.6 branch, r57965)
- (r57945) Fix import problems with system-installed nested namespace packages, due to the parent package not having the child package as an attribute. (backported to 0.6 branch, r57946)
- (r57943) Fix precedence issue between .egg and .egg-info packages when both are installed in site-packages. (backported to 0.6 branch, r57944)
- (r57370) Fix downloading from ftp: URLs (which don't have a content-type). (backported to 0.6 branch, r56371)
- (r56320) Fix a problem with the findall() fix. :( (backported to 0.6 branch, r56321)
- (r56276) Fix distutils.filelist.findall() crashing on broken symlinks. Fix egg_info failures on new, uncommitted SVN directories. (backported to 0.6 branch, r56277)
- (r55711) Misc. fixes:
* Don't treat directories with a '.' in the name as packages
* Don't include ez_setup in find_packages()
* HTML-decode URLs scraped from web pages (d'oh!)
* Fix cache dir defaults on Windows when multiple env vars used
* Doc updates
(backported to 0.6 branch, r54712)
- (r54966) es Setting svn:eol properties everywhere where they weren't set yet.
- (r54242) Changed setuptools.package_index.PackageIndex.open_url to include the url in the exception. (forward port from 0.6 branch, r54241)
- (r54071) Fix problem activating dependencies for tests (backported to 0.6 branch, r54072)
- (r53895) Fix "test" command possibly failing if an older version of the project being tested is installed on sys.path ahead of the test source directory. (backported to 0.6 branch, r53896)
- (r53876) Add --local-snapshots-ok flag, to allow building eggs from projects installed using "setup.py develop". (backported to 0.6 branch, r53877)
- (r53870) Get rid of 'sets' module usage under Python 2.4+, so that no warnings are issued by Python 2.6. (backported to 0.6 branch, r53871)
- (r53868) Indicate when dependency processing is finished, so that you can tell which dependencies go with what. (backported to 0.6 branch, r53869)
- (r53834) Respect possible entry point override of 'easy_install' command. (backported to 0.6 branch, r53835)
- (r53813) Fix error if script contains null byte. (backported to 0.6 branch, r53814)
- (r53792,53794) Support .egg-link paths being relative (backported to 0.6 branch, r53794)
- (r53790) Add --egg-path option to force .egg-link files to use relative paths (allowing them to be shared across platforms on a networked drive). (backported to 0.6 branch, r53791)
- (r53673) Ensure extracted libraries have correct permissions on Posix systems (e.g. Cygwin, which is where I found the problem.) (backported to 0.6 branch, r53674)
- (r53645) Match Python 2.5 pydoc
- (r53614) Fix script language detection so that scripts built on Windows are correctly identified as still being Python code! (backported to 0.6 branch, r53615)
- (r53611) Fixed mangling line endings when an old-style source script came from Windows. (backported to 0.6 branch, r53612)
- (r53547) Fix #! parsing problems w/whitespace in quoted strings or at the end of the #! line. (backported to 0.6 branch, r53548)
- (r53541) If a page can't be spidered, warn and ignore rather than aborting. (backported to 0.6 branch, r53542)
- (r53539) Fix "bashism" in eggsecutable header. (backported to 0.6 branch, r53540)
- (r53538) Fix missing dev links in new README (forward ported from 0.6 branch, r53495)
- (r53318) Workaround for Python 2.5 distutils bug: when bdist_wininst files are uploaded, they are marked as suitable for "any" Python version, even if an explicit --target-version was specified. (backported to 0.6 branch, r53319)
- (r53315) Fix not making Cygwin .dll's executable when installing in non-zip-safe mode. (backported to 0.6 branch, r53317)
- (r53314) Fix typo in 'bdist_rpm' command (backported to 0.6 branch, r53317)
- (r53312) Remove unused import (Forward ported from 0.6 branch, r53311)
- (r53309) Don't double-upload .exe files under Python 2.5. :(
- (r53307) Formatting fixes for README (backported to 0.6 branch in r53308)
- (r53268) Fix not generating correct .pth for parent namespace packages when installing --single-version-externally-managed. (backported to 0.6 branch in r53271)
- (r53202) Add Basic Auth support for http URLs with embedded credentials. If an authenticated page contains links to the same protocol and host, those links should inherit the same credentials. (backported to 0.6 branch in r53204)
- (r53198) Fix os.open() sandboxing code that refused anything but read-only access. (backported to 0.6 branch in r53199)
- (r53195) Switch default package-index to cheeseshop.python.org, and make setuptools' home page its cheeseshop page. (backported to 0.6 branch in r53196)
- (r53191,53193) Overhaul Windows script wrappers to support bdist_wininst better. (backported to 0.6 branch in r53194)
- (r53185) Partial support for cross-platform generation of bdist_wininst .exe's. (backported to 0.6 branch in r53186)
- (r53183) Work around a distutils bdist_wininst bug: if the user has specified an --install-lib via a configuration file, the path is hardcoded into the .exe file in place of PLATLIB/ or PURELIB/. (backported to 0.6 branch in r53184)
- (r53181) Fix import problem for bdist_rpm w/Python<2.5 (backported to 0.6 branch in r53182)
- (r53177) Removed all special support for Sourceforge mirrors, since SF finally replaced their crazy mirror system with something sane. (backported to 0.6 branch in r53178)
- (r53175) New installation instructions and credits (backported to 0.6 branch in r53176)
- (r52886) Fix problem w/' ' in sys.executable on Windows. (backported to 0.6 branch in r52887)
- (r52437) Fixed not allowing os.open() of paths outside the sandbox, even if they are opened read-only (e.g. /dev/urandom). (backported to 0.6 branch in r52438)
- (r52046) Use cross-platform relative paths in .pth if target is anywhere inside the .pth file's directory. (backported to 0.6 branch in r52047)
- (r52023) Fix problem generating "eggsecutable" header if dist/ dir doesn't exst yet. (backported to 0.6 branch in r52024)
- (r52012) Allow explicit selection of Sourceforge mirror(s) with --sf-mirror, and further refine download/retry algorithm. (backported to 0.6 branch in r52013)
- (r52008) Should've used distutils.log.warn instead of warnings.warn (backported to 0.6 branch in r52009)
- (r52006) Handle empty revision numbers in SVN 1.4 "entries" format (backported to 0.6 branch in r52007)
- (r52003) Fix SF download problems when server returns HTML instead of a 404. Use sf-mirrors.telecommunity.com as a fallback. (backported to 0.6 branch in r52004)
- (r52001) Fix "dev" versions being considered newer than release candidates. :( (backported to 0.6 branch in r52002)
- (r51968) Add support for "eggsecutable" headers: a /bin/sh script that is prepended to an .egg file to allow it to be run as a script on Unix-ish platforms. (backported to 0.6 branch in r51969)
- (r51967) Fix broken imports. (Forward-port from 0.6 branch, r51966)
- (r51963) Fix bdist_wininst files not being uploaded by "upload" (backported to 0.6 branch in r51964, r51965)
- (r51960) Support uploading bdist_rpm files on older Python versions (backported to 0.6 branch in r51961)
- (r51955) Fix easy_install not recognizing win32.exe files that include a custom bitmap. (backported to 0.6 branch in r51957)
- (r51931) Retain 2.3/2.4 compatibility as well...
- (r51930) Python 2.5 compatibility fix
- (r51899) Support svn 1.4 working copy format (backported to 0.6 branch in r51900)
- (r51790) Prevent deprecation warnings coming from generated scripts when sys.executable contains non-ASCII characters. (backported to 0.6 branch in r51791)
- (r51783) Make "setup.py develop" of a setuptools-using project install setuptools, if needed, instead of only downloading the egg. (backported to 0.6 branch in r51784)
- (r51781) Don't check installation directory writability and site/.pth setup when using --editable. (backported to 0.6 branch in r51782)
- (r51777) Support setuptools .egg being in current directory when bootstrapping on an offline machine. Output what version/location is conflicting when a newer version of setuptools is requested. (backported to 0.6 branch in r51778)
- (r51485) Added quoting of script arguments and extended the quoting logic to handle embedded quotes.
Added support for passing a single argument on the shebang line to pass things like -O and -i.
Fixed bug in handling trailing whitespace in Python command. (backported to 0.6 branch in r51487)
- (r51243) Fix wrong Mac OS X installation paths. (backported to 0.6 branch in r51244)
- (r50921) Sync pkgutil from Python trunk
- (r50755) Fix a problem with eggs specified directly on PYTHONPATH on case-insensitive filesystems possibly not showing up in the default working set. (backported to 0.6 branch in r50756)
- (r50734) Update MD5 matching for current PyPI code. :( (backported to 0.6 branch in r50735)
- (r50732) Identify the setuptools version as part of the User-Agent string when spidering pages or downloading files. (backported to 0.6 branch in r50733)
- (r50702) Make sdist-from-sdist builds keep the exact same version number, even if --tag-date was used to build the original sdist. (backported to 0.6 branch in r50703)
- (r50701) Edit an sdist's setup.cfg to include any egg_info options that were used to build it. (backported to 0.6 branch in r50703)
- (r50699) Restored support for extra_path when using backward compatibility mode. (backported to 0.6 branch in r50700)
- (r50658) Fix three problems reported by Bob Ippolito and Jim Fulton
* Fixed ``AttributeError`` when trying to download a ``setup_requires`` dependency when a distribution lacks a ``dependency_links`` setting.
* Made ``zip-safe`` and ``not-zip-safe`` flag files contain a single byte, so as to play better with packaging tools that complain about zero-length files.
* Made ``setup.py develop`` respect the ``--no-deps`` option, which it previously was ignoring.
(backported to 0.6 branch in r50659)
- (r50592) Fix doubled --tag-build option. (backported to 0.6 branch in r50593)
- (r50591) Fix doc typo (forward port from 0.6 branch in r50589)
- (r50580) Don't warn about possible misspelling if we know that a project with that name does in fact exist. (backported to 0.6 branch in r50581)
- (r50562) Crosslink "Creating your own package index" to the new docs.
- (r50561) Add documentation for package index "API" (layout/content rules) (backported to 0.6 branch in r50563)
- (r50556) Reduce screenscraping required for a package index, adding new 'rel="download"' and 'rel="homepage"' handling (not yet supported by PyPI). (backported to 0.6 branch in r50557)
- (r50554) Allow use of file:// URLs for --index-url. (backported to 0.6 branch in r50555)
- (r50550) Fix not recognizing HTML 404 pages from package indexes. (backported to 0.6 branch in r50552)
- (r50545) Ensure that sys.path_importer_cache is updated when an existing zipfile or directory is deleted. (backported to 0.6 branch in r50546)
- (r50543) Don't warn about missing README(.txt) unless creating an sdist (backported to 0.6 branch in r50544)
- (r50538) Tweak docs for subversion release stuff to make better use of new options.
- (r50536) Update aliases for easier release using new options
- (r50535) Add --no-date and --no-svn-revision options to make creating release snapshots easier. (backported to 0.6 branch in r50539)
- (r50533) Fix "register" command not necessarily reflecting build tags. (backported to 0.6 branch in r50534)
- (r50531) Include more detailed version ranges spec, and make Requirement.specs a public/documented attribute. (backported to 0.6 branch in r50532)
- (r50529) Fix broken error message for socket error during upload. (backported to 0.6 branch in r50530)
- (r47050) Fix ftp:// directory listing URLs from causing a crash when used in the URL or Download URL slot on PyPI. (backported to 0.6 branch in r47051)
- (r46985) Implement detection of non-Python scripts, as described in http://mail.python.org/pipermail/distutils-sig/2006-June/006359.html (backported to 0.6 branch in r46986)
- (r46790) Allow .py files found by the include_package_data option to be automatically included. Remove duplicate data file matches if both include_package_data and package_data are used to refer to the same files. (backported to 0.6 branch in r46791)
- (r46788) Fix mysterious errors during initial setuptools install, caused by ez_setup trying to run easy_install twice (backported to 0.6 branch in r46789)
- (r46781) Fix sometimes not detecting local packages installed outside of "site" directories. (backported to 0.6 branch in r46782)
- (r46725) Fix local --find-links eggs not being copied except with --always-copy. (backported to 0.6 branch in r46726)
- (r46721) Fix bdist_egg not including files in .egg-info subdirectories. (backported to 0.6 branch in r46722)
- (r46713) Clarify the limitations of get_provider(packagename) (backported to 0.6 branch in r46714)
- (r46711) Fix a duplicate path insertion bug on case-insensitive filesystem. (backported to 0.6 branch in r46712)
- (r46563) Fix a problem w/relative path generation if you install an egg whose name begins with 'import'. (backported to 0.6 branch in r46564)
- (r46389) Don't make things warnings that aren't; update info text for --multi-version. (backported to 0.6 branch in r46390)
- (r46140) Don't install or update a ``site.py`` patch when installing to a ``PYTHONPATH`` directory with ``--multi-version``, unless an ``easy-install.pth`` file is already in use there. (backported to 0.6 branch in r46143)
- (r45983) Better ambiguity management: accept #egg name/version even if processing what appears to be a correctly-named distutils file, and ignore .egg files with no '-' (backported to 0.6 branch in r45984)
- (r45789,45790) Check for -m only in 2.4; remove out-of-date zipfile analysis docs (backported to 0.6 branch in r45791)
- (r45764) Fix entry point parsing when a standalone module name has whitespace between it and the extras. (backported to 0.6 branch in r45765)
- (r45719,45720) Added "internals" documentation
- (r45694,45695) Strip 'module' from the end of compiled extension modules when computing the name of a .py loader/wrapper. (backported to 0.6 branch in r45696)
- (r45628) Recognize 'U' as a valid read-only mode for open() (backported to 0.6 branch in r45629)
- (r45626) Ignore bdist_dumb distributions when looking for downloads. (backported to 0.6 branch in r45627)
- (r45555) Support file:// links to directories in --find-links, so that easy_install can build packages from local source checkouts. (backported to 0.6 branch in r45556)
- (45536) Split ``get_platform()`` into ``get_supported_platform()`` and ``get_build_platform()`` to work around a Mac versioning problem (backported to 0.6 branch in r45538)
- (r45515,45517,45519,45521,45554) Python 2.5 compatibility changes
- (445514) Back/sideport a bunch of 2.5 modules (pkutil, pydoc, doctest) for use in 2.3 / 2.4.
- (r45415) Trap absolute paths passed as 'package_dirs' (backported to 0.6 branch in r45416)
- (r45405) Work toward inclusion in Python 2.5
- (r45403) Don't eagerly import namespace packages.
At 05:31 PM 10/4/2007 -0400, Tres Seaver wrote:
At this point, would we be better off moving aside the current trunk, and making a new trunk from the 0.6 branch after getting 0.6 finally released?
No way am I going to do the 2.5 work over again, so at least this particular portion of "we" would not be better off. :)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Phillip J. Eby wrote:
At 05:31 PM 10/4/2007 -0400, Tres Seaver wrote:
At this point, would we be better off moving aside the current trunk, and making a new trunk from the 0.6 branch after getting 0.6 finally released?
No way am I going to do the 2.5 work over again, so at least this particular portion of "we" would not be better off. :)
I was really trying to get an enumeration of the stuff done on that branch which was not already backported. For instance, there are a half dozen or so 0.6 branch checkins which talk about backporting 2.5 compatibility fixes: are there others not ported?
The real source of my worry is that anything which is on the trunk but *not* on the 0.6 branch is potentially bitrotting, as nobody (AFAIK) is actually using the trunk at this point. Given that your itch is currently scratched (I assume by 0.6?), I don't know who is able to help maintain the trunk at all.
Here are the changelog messages from the trunk which I could not identify as being ported (back or forward) to the 0.6 branch:
- (r45403) Don't eagerly import namespace packages.
- (r45405) Work toward inclusion in Python 2.5
- (445514) Back/sideport a bunch of 2.5 modules (pkutil, pydoc, doctest) for use in 2.3 / 2.4.
- (r45515,45517,45519,45521,45554) Python 2.5 compatibility changes
- (r45719,45720) Added "internals" documentation
- (r50538) Tweak docs for subversion release stuff to make better use of new options.
- (r50536) Update aliases for easier release using new options
- (r50562) Crosslink "Creating your own package index" to the new docs.
- (r50921) Sync pkgutil from Python trunk
- (r51931) Retain 2.3/2.4 compatibility as well...
- (r51930) Python 2.5 compatibility fix
- (r53309) Don't double-upload .exe files under Python 2.5. :(
- (r53645) Match Python 2.5 pydoc
- (r54966) es Setting svn:eol properties everywhere where they weren't set yet.
I can't figure out from checkin comments (and a few diffs) exactly how much of this stuff is "feature removal" (which your earlier message said had been mostly done already), versus fixes to make it run under Python 2.5 (doesn't the 0.6 branch work fine under 2.5?), versus cleanups to make the Python stdlib folks happy (or whatver).
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com
At 03:48 PM 10/5/2007 -0400, Tres Seaver wrote:
Phillip J. Eby wrote:
At 05:31 PM 10/4/2007 -0400, Tres Seaver wrote:
At this point, would we be better off moving aside the current trunk, and making a new trunk from the 0.6 branch after getting 0.6 finally released?
No way am I going to do the 2.5 work over again, so at least this particular portion of "we" would not be better off. :)
I was really trying to get an enumeration of the stuff done on that branch which was not already backported. For instance, there are a half dozen or so 0.6 branch checkins which talk about backporting 2.5 compatibility fixes: are there others not ported?
The 2.5 work I'm referring to was about consolidation, code sharing with the stdlib, and re-organization more than it was about compatibility fixes (which of course were backported).
The real source of my worry is that anything which is on the trunk but *not* on the 0.6 branch is potentially bitrotting, as nobody (AFAIK) is actually using the trunk at this point.
I use it almost exclusively, and that's where all the development is done -- which is why 99% of the changes appear on the trunk first, then get ported to 0.6. I also know there are a few people who work off the trunk, although of course it's not nearly as widespread.
I can't figure out from checkin comments (and a few diffs) exactly how much of this stuff is "feature removal" (which your earlier message said had been mostly done already),
There may be some things I've forgotten to remove as yet, but anything that the docs (or code) issues warnings about changing, is either already changed in the trunk, or should be.
Hi,
I think easy_install should try to have a look at what Maven2 did. I _know_ this is never a good idea to give some java examples in a python list, but maven2 has some very interesting concepts, most of them being absolutely language agnostic. (note: Maven2, not Maven1). In fact, I think Maven2 could be implemented in any language (except that almost all examples and plugins are about java... too bad :-( ). May be there is something to learn from their experience.
First, there is the POM. This is the equivalent of the setup.py, where we put all the description of the project. We find the authors names, the code location, the tests location, the artifacts we can create, the dependencies, etc. What is really interesting is that this description is really _only_ a description. There is absolutely no "action" (script). The POM is really conceptually close to setup.py, without any action in it. It contains the structure of the project.
Then come the plugins. I think they are the equivalent of setuptools extensions. And really, what makes Maven so nice is really the set of plugins. Nothing else (IMHO). A few examples that are really nice: Having a fully available Eclipse project ready, with all dependencies downloaded in the local cache and included in the classpath is just one command away ("mvn eclipse"). Ok, it means a few other things, such as an artifact repository, but it's nice anyway. Testing is "mvn test", building is "mvn build", etc. (Yes I know, that's similar to some setuptools commands)
So basically, I really think that setup.py should benefit from this experience:
- Turn setup.py into a real description-only file (no commands), or invent another file
- Have a local repository of eggs, _not_ included in the python path by default
- Get rid of the site-package hell: find a nice way to start a python with a predefined set of eggs (yes, the equivalent of naming all the eggs in the python path, but in a more simple way, such as "python --python_path_description=myconfig", myconfig being an alias to a set of eggs)
- Manage the dependencies by downloading the eggs in the local repository (fully independent from python itself)
- Extract everything from the core setuptools which could be in a plugin (everything which does not read the description of the project and calls the plugins)
- Create a set of plugins to deal with most common use cases, such as packaging eggs, compiling libraries, handling special types of files etc...
- And create a nice wiki tutorial to explain how to create the best plugins you have ever dreamed of
- Let others write the plugins for you
Actually, I believe the plugin orientation is already chosen ? ( http://mail.python.org/pipermail/distutils-sig/2007-January/007113.html )
And I know that finding time is the most difficult part of it...
My 2 cents.
2007/10/3, Jim Fulton jim@zope.com:
On Oct 3, 2007, at 1:06 PM, Phillip J. Eby wrote:
At 10:26 AM 10/3/2007 -0400, Jim Fulton wrote:
I'm a little bit worried about setuptools development cycle. We seem to be stalled at a 0.6 pre-release that is quite stable and widely used in production. The next feature release, 0.7, seems to anticipate a major refactoring and major new features. I'm a bit worried about this both from a fear of potential disruption, and a need for incremental feature development.
I propose that the grand re-factoring of setuptools planned for 0.7 be moved off the trunk and to a 2.x release. 0.6 should be marked final
This is coming soon, but I want to finish some ez_setup improvements first. There will be a "known issues" list to cover things like the incomplete/broken FTP support; I've given up on trying to keep hacking it into 0.6.
I wonder what "this" you are referring to.
Is it on the table to remove features from setuptools?
Yes - and most are *already* removed from the 0.7 trunk. That was the main reason for the branch.
I'm confused on a number of counts. Is the 0.7 "trunk" the same as the setuptools trunk? Is there also a branch for 0.7?
Is there a list of removed features somewhere? Do we get any input?
Other (relatively minor) refactorings expected in 0.7 are to make URL handlers for easy_install pluggable, and to add build-time plugin options (so that people can implement custom build steps).
The biggest feature additions in 0.7 would be support for uninstallation, package inspection, virtual environment management, and ability to easy_install eggs using "classic/ development" (i.e., .egg-info) installation layout.
Other new features planned for 0.7 include "or-ed" requirements, build/install of shared libraries, and "standard extras" (a way to specify to easy_install a set of extras that should be installed if they are defined by the targets).
None of this is what I'd consider "grand" refactoring; they're all either new features or minor refactorings to make things pluggable. The most complex of them is probably the URL handler one, and only because the existing URL handling in setuptools.package_index is so hosed.
Now, if you include pkg_resources in, I would rather like to refactor the way the global objects (working set and resource manager) are handled, but I don't know how practical or even possible it is. There are other things in pkg_resources that could use refactoring, but I'm not sure I dare.
"Grand" is simply the impression I had. The list of features above is extensive though. Some of the items seem fairly big, or maybe I just can't tell what they are. Do you plan to try to get all of them into the next feature release? Or do you plan them a list of ideas for future releases.
Jim
-- Jim Fulton Zope Corporation
Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Laurent Ploix wrote:
Hi,
I think easy_install should try to have a look at what Maven2 did. I _know_ this is never a good idea to give some java examples in a python list, but maven2 has some very interesting concepts, most of them being absolutely language agnostic. (note: Maven2, not Maven1). In fact, I think Maven2 could be implemented in any language (except that almost all examples and plugins are about java... too bad :-( ). May be there is something to learn from their experience.
First, there is the POM. This is the equivalent of the setup.py, where we put all the description of the project. We find the authors names, the code location, the tests location, the artifacts we can create, the dependencies, etc. What is really interesting is that this description is really _only_ a description. There is absolutely no "action" (script). The POM is really conceptually close to setup.py, without any action in it. It contains the structure of the project.
Then come the plugins. I think they are the equivalent of setuptools extensions. And really, what makes Maven so nice is really the set of plugins. Nothing else (IMHO). A few examples that are really nice: Having a fully available Eclipse project ready, with all dependencies downloaded in the local cache and included in the classpath is just one command away ("mvn eclipse"). Ok, it means a few other things, such as an artifact repository, but it's nice anyway. Testing is "mvn test", building is "mvn build", etc. (Yes I know, that's similar to some setuptools commands)
So basically, I really think that setup.py should benefit from this experience:
- Turn setup.py into a real description-only file (no commands), or
invent another file
For backward compatibility this doesn't really work. Potentially you could have a setup.py:
""" from mavon_style_lib import setup
setup(__file__) """
And have that function read the description-only file.
I personally would prefer, say, PKG-INFO to be the canonical source of information instead of setup.py. But then, I also find it handy sometimes to generate long_description from a segment of a text file in my documentation. That's easy currently, and would be much harder with a description-only file.
- Have a local repository of eggs, _not_ included in the python path by
default
If you easy_install -m then they won't be on the python path. But it's all very fuzzy; I don't think many people make good use of -m in deployments. I know I don't.
- Get rid of the site-package hell: find a nice way to start a python
with a predefined set of eggs (yes, the equivalent of naming all the eggs in the python path, but in a more simple way, such as "python --python_path_description=myconfig", myconfig being an alias to a set of eggs)
This is what zc.buildout and virtualenv do, more or less. zc.buildout includes building the initial set of eggs; in virtualenv I'll be suggesting poach-eggs (https://svn.openplans.org/svn/PoachEggs/trunk) to install a set of packages.
- Manage the dependencies by downloading the eggs in the local
repository (fully independent from python itself)
In buildutils I've added a bundle command, which kind of does this -- collecting all the eggs for a particular package. You do this with "python setup.py bundle". Buildutils is available with "hg clone https://www.knowledgetap.com/hg/buildutils/" (sorry it's not fully released at the moment).
- Extract everything from the core setuptools which could be in a plugin
(everything which does not read the description of the project and calls the plugins)
Setuptools already self-provides lots of its functionality as plugins. I don't see any need to extract these into separate packages.
- Create a set of plugins to deal with most common use cases, such as
packaging eggs, compiling libraries, handling special types of files etc...
- And create a nice wiki tutorial to explain how to create the best
plugins you have ever dreamed of
This needs some work; I'd like to make a paster create template in buildutils for creating setuptools' extensions. It would be pretty small. It's not terribly hard to write a setuptools' extension now, though you do have to deal with the somewhat crufty interface of distutils commands. It's not really so bad, though.
participants (5)
-
Ian Bicking -
Jim Fulton -
Laurent Ploix -
Phillip J. Eby -
Tres Seaver