Mailman 3 March 2008 - Python-Dev

2.6 and 3.0 tasks
by Guido van Rossum March 22, 2008

March 22, 2008

Moving this to a new subject to keep the discussion of tasks and the discussion of task tracking tools separate. On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes <lists(a)cheimes.de> wrote: > I did a quick brainstorming with me, myself and I. I came up with a list > of (IMHO) important tasks. > > * Stabilize the C API of Python 3.0. I like to rename several prefixes > to reduce the confusing: PyBytes -> PyByteArray, +1 (also +1 to backporting this to 2.6) > … [View More]PyString -> PyBytes ... -1. This will make merging code from 2.6 harder, and causes more work for porting C extensions. > * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday > and he said he is trying to get it done during the PyCon sprint. Maybe > somebody can assist him? Does he need assistance? > When the new buffer protocol is available in 2.6 we can start back > porting bytesarray and the new IO framework. Are these really so closely tied that they have to wait before they can be backported? > * Replace Windows API calls with wide versions to support unicode for > file names, environment etc. +1. This should be broken into separate tasks for each API. > * Get the stdlib reorg done +1. But again, I think this is many small tasks. > and add the fixers to 2to3 +1. I think quite a few changes have not had a fixer added. Again, I think we should maintain a specific list of needed fixers; fixers can easily be developed independently. > * Speed up 2to3. I suggested a GSoC task to improve and speed up 2to3 > but it may be too late when we plan to ship out 3.0 in August. While I know that some people are expecting to use a development model that invokes 2to3 very frequently, I think this is at best a nice-to-have. (I also don't see how it could be done, but maybe I'm blind for the obvious, as the original author.) I have some other tasks to add: - getargs.c (Georg posted about this; his post actually inspired my post.) - Tweak structseq to be more like namedtuple. Maybe they could have a common ABC. I also think we should get rid of the concept of "hidden" fields (that are accessible by name but not through the tuple API). - Devise a way to handle str instances pickled in 2.6 and unpickled in 3.0, and also bytes instances pickled in 3.0 and unpickled in 2.6. - Someone should go over https://spreadsheets.google.com/ccc?key=pCKY4oaXnT81FrGo3ShGHGg and extract more tasks. -- --Guido van Rossum (home page: http://www.python.org/~guido/) [View Less]

9 22

2.6 and 3.0 project management
by Guido van Rossum March 22, 2008

March 22, 2008

Python 3.0 and 2.6 are coming along really nice. I am optimistic that we can make the projected August date for the final releases of 2.6 and 3.0. As you may remember, Barry (the new release manager for both) suggested that we synchronize releases of 2.6 and 3.0. Not only could this potentially save the release manager and his assistants some time, doing the final releases together sends a clear signal to the community that both versions will receive equal support. However, looking at the … [View More]

12 26

Proposal: Slightly Changed Semantics for from-Import
by Armin Ronacher March 22, 2008

March 22, 2008

Hi all, I propose a small change on the "from"-import behavior. Currently there are two ways to import a module name foo from bar. Namely "import foo.bar as bar" and "from foo import bar". The main problem with the second is that it will not work in every situation. Modules are registered with their names in sys.modules before the import but become and attribute of their parent after. This leads to the surprising behavior that "from foo.bar import baz" works but "from foo import bar" not … [View More]

2 1

Re: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond
by Phillip J. Eby March 22, 2008

March 22, 2008

At 02:14 PM 3/22/2008 +0000, Paul Moore wrote: >For the system Python, I need: >- a single way to list what's installed (including version) >- a single way to uninstall items as needed >- a way (or more than one) to install 3rd party software *which ties >into the above* Right, and the PEP effort is devoted to having one way to store the information required, to tie these things together. If there is a standard way to store that info on your system, then it doesn't matter … [View More]

1 0

Testing documentation snippets
by Georg Brandl March 22, 2008

March 22, 2008

Hi, the newest version of Sphinx supports testing doctest (and other) snippets in the documentation. Since we have many examples in the docs that may get out of date, I think this is a valuable thing to have. I've started making the doctests runnable with Sphinx in three documents; the functional howto, the RE and the Decimal docs. If you want to know how to run the doctests, it should be as easy as $ cd Doc $ make doctest PYTHON=../python on UNIX. The PYTHON=../python is needed to really … [View More]

1 0

Python source code on Bazaar vcs
by Matthieu Brucher March 22, 2008

March 22, 2008

Sorry for the double post, Jeroen :| ---------- Forwarded message ---------- From: Matthieu Brucher <matthieu.brucher(a)gmail.com> Date: 21 mars 2008 10:03 Subject: Re: [Python-Dev] Python source code on Bazaar vcs To: Jeroen Ruigrok van der Werven <asmodai(a)in-nomine.org> 2008/3/21, Jeroen Ruigrok van der Werven <asmodai(a)in-nomine.org>: > > (To provide counterweight.) > > > -On [20080320 20:44], Barry Warsaw (barry(a)python.org) wrote: > >We have … [View More]

2 1

Primer on distributed revision control?
by skip＠pobox.com March 22, 2008

March 22, 2008

With all these distributed revision control systems now available (bzr, hg, darcs, svk, many more), I find I need an introduction to the concepts and advantages of repository distribution. It seems to me that it has the potential for leading to anarchy, though I can see how some things would be improved (working offline, maintaining local patches). It's not obvious how I push changes back upstream. Can someone point me to some useful content (web pages or books) which will help me wrap my … [View More]

6 5

httplib &c. timeouts and global state
by John J Lee March 22, 2008

March 22, 2008

http://python.org/sf/2451 """ The new timeout support in 2.6 makes use of new function socket.create_connection(). socket.create_connection() provides no way to disable timeouts, other than by relying on socket.getdefaulttimeout() returning None. This is unfortunate, because it was the purpose of the new timeout support to allow control of timeouts without reliance on global state. setdefaultsocket.create_connection() should always call sock.settimeout() with the timeout passed to … [View More]create_connection(), unless a special non-None value is passed indicating that the global default is to be used. Specific modules may then use that special non-None value where required, to preserve backwards-compatibility. """ Facundo doesn't seem to agree. If I understand him correctly, he objects to the use of a special value other than None as an argument value. I think avoiding this minor complication is secondary to avoiding gratuitously imposing global state on users where it's not needed. Much existing code uses .setdefaulttimeout(), because legacy code does not expose the socket timeout. Of course, not all of that code can be changed immediately (otherwise, why did we wait so long before Facundo did his valuable work on this?). Of course, very often, code that calls socket.setdefaulttimeout(some_non_none_value) wants a default timeout for everything. But that isn't *always* the case, and standard library modules that expose the timeout feature should not make that assumption. For example, an application might only want to set a timeout for some non-essential network operation. As another example, useful but poorly-written library code might call .setdefaulttimeout(). Also, ISTM there's a big difference between nobody having contributed the time to implement the feature on the one hand, and deliberately imposing the global socket timeout state on users on the other! Facundo indicates in the tracker discussion for 2451 that a decision was made here about this, but doesn't recall exactly which post, and directed me here. What I found in the archive is this thread (sorry for the non-python.org URL): http://www.gossamer-threads.com/lists/python/dev/555039?do=post_view_thread… In that discussion, this issue is raised, but I don't see any resolution that answers the objection made in issue 2451. Anyway, apologies in advance if there was a decision here that takes account of the above objection. I do want to thank Facundo for adding the timeout support to modules such as httplib. But I also want to loudly complain about this detail :-) John [View Less]

1 0

Adventures with x64, VS7 and VS8 on Windows
by Mark Hammond March 22, 2008

March 22, 2008

Hi all, I hope the cross-post is appropriate. I've started playing with getting the pywin32 extensions building under the AMD64 architecture. I started building with Visual Studio 8 (it was what I had handy) and I struck a few issues relating to the compiler version that I thought worth sharing. * In trying to build x64 from a 32bit VS7 (ie, cross-compiling via the PCBuild directory), the python.exe project fails with: pythoncore fatal error LNK1112: module machine type 'X86' … [View More]conflicts with target machine type 'AMD64' is this a known issue, or am I doing something wrong? * The PCBuild8 project files appear to work without modification (I only tried native compilation here though, not a cross-compile) - however, unlike the PCBuild directory, they place all binaries in a 'PCBuild8/x64' directory. While this means that its possible to build for multiple architectures from the same source tree, it makes life harder for tools like 'distutils' - eg, distutils already knows about the 'PCBuild' directory, but it knows nothing about either PCBuild8 or PCBuild8/x64. A number of other build processes also know to look inside a PCBuild directory (eg, Mozilla), so instead of formalizing PCBuild8, I think we should merge PCBuild8 into PCBuild. This could mean PCBuild/vs7 and PCBuild/vs8 directories with the "project" files, but binaries would still be generated in the 'PCBuild' (or PCBuild/x64) directory. This would mean the same tree isn't capable of hosting 2 builds from different VS compilers, but I think that is reasonable (if it's a problem, just have multiple source directories). I understand that PCBuild8 is not "official", but in the assumption that future versions of Python will use a compiler later than VS7, it makes sense to me to clean this up now - what are others opinions on this? * Re the x64 directory used by the PCBuild8 process. IMO, it makes sense to generate x64 binaries to their own directory - my expectation is that cross-compiling between platforms is a reasonable use-case, and we should support multiple achitectures for the same compiler version. This would mean formalizing the x64 directory in both 'PCBuild' and distutils, and leaving other external build processes to update as they support x64 builds. Does this make sense? Would this fatally break other scripts used for packaging (eg, the MSI framework)? * Wide characters in VS8: PC/pyconfig.h defines PY_UNICODE_TYPE as 'unsigned short', which corresponds with both 'WCHAR' and 'wchar' in previous compiler versions. VS8 defines this as wchar_t, which I'm struggling to find a formal definition for beyond being 2 bytes. My problem is that code which assumes a 'Py_UNICODE *' could be used in place of a 'WCHAR *' now fails. I believe the intent on Windows has always been "PyUNICODE == 'native unicode'" - should PC/pyconfig.h reflect this (ie, should pyconfig.h grow a version specific definition of PyUNICODE as wchar_t)? * Finally, as something from left-field which may well take 12 months or more to pull off - but would there be any interest to moving the Windows build process to a cygwin environment based on the existing autoconf scripts? I know a couple of projects are doing this successfully, including Mozilla, so it has precendent. It does impose a greater burden on people trying to build on Windows, but I'd suggest that in recent times, many people who are likely to want to build Python on Windows are already likely to have a cygwin environment. Simpler mingw builds and nuking MSVC specific build stuff are among the advantages this would bring. It is not worth adding this as "yet another windows build option" - so IMO it is only worth progressing with if it became the "blessed" build process for windows - if there is support for this, I'll work on it as the opportunity presents itself... I'm (obviously) only suggesting we do this on the trunk and am happy to make all agreed changes - but I welcome all suggestions or critisisms of this endeavour... Cheers, Mark [View Less]

12 57

Re: [Python-Dev] PEP 365 (Adding the pkg_resources module)
by Phillip J. Eby March 22, 2008

March 22, 2008

At 03:43 PM 3/18/2008 -0500, Guido van Rossum wrote: >Only very few people would care about writing a setup >script that works with this bootstrap module; basically only package >manager implementers. That's true today, sure, but as soon as it is widely available, others are sure to want to use it too. I just want a bright-line distinction between what is and isn't bootstrappable, rather than a murky region of "maybe, if you're not doing anything too complicated". >There … [View More]seems to be a misunderstanding about what I am proposing we do >instead. The boostrap installer should only be powerful enough to >allow it to be used to install a real package manager like setuptools. Which is why PEP 365 proposed only downloading an archive to a cache directory, and optionally running something from it. It explicitly disavows "installation" of anything, since the downloaded archive wouldn't have been added to sys.path except for the duration of the bootstrap process, and no scripts were to be installed. (Indeed, apart from the methods it would have used to locate the archive on PyPI, and to determine what to run from inside it, there was nothing particularly egg-specific about the proposed bootstrapping process.) So, to fully egg-neutralize the bootstrapping approach, we need only know how to locate an appropriate archive, and how to determine what to run from it. For the latter, we could use the already-in-2.6 convention of running __main__ from a zipfile or directory. (Too bad distutils source distributions have an extra directory name embedded in them, so one can't just execute them directly. Otherwise, we could've just let people drop in a __main__.py next to setup.py. OTOH, maybe it would be enough to use setuptools' algorithm for finding setup.py to locate __main__.py, and I'm fairly sure *that* can be briefly expressed in the PEP.) The other open question is a naming convention and version detection, so that the bootstrap tool can identify which of the files listed on PyPI is suitable for its use. (Both with regard to the version selection, and file type.) However, if PyPI were to grow support for designating the appropriate files and/or versions in some other way, we wouldn't need a naming convention as such. Without one or the other, the bootstrap tool would have to grow a version parsing scheme of some type, and play guessing games with file extensions. (Which is one reason I limited PEP 365's scope to downloading eggs actually *uploaded* to PyPI, rather than arbitrary packages *linked* from PyPI.) So, if I had to propose something right now, I would be inclined to propose: * using setuptools' version parsing semantics for interpretation of alpha/beta/dev/etc. releases * having a bdist_bootstrap format that's essentially a bdist_dumb .zip file with the internal path prefixes stripped off, making it an importable .zip with a different file extension. (Or maybe just .pyboot.zip?) The filename convention would use setuptools' canonicalization and escaping of names and version numbers, to allow unambiguous machine parsing of the filename. A __main__ module would have to be present for the archive to be run, as opposed to just being downloaded to a temporary directory. * calling the bootstrap module 'bootstrap', as in 'python -m bootstrap projectname optionalversion'. The module would expose an API to allow it to be used programmatically as well as the command line, so that bootstrapped packages can use the bootstrap process to locate dependencies if they so desire. (Today's package management tools, at least, are all based on setuptools, so if it's not present they'll need to download that before beginning their own bootstrapping process.) Apart from keeping the PEP self-contained and short, is there anything in this that you think you would object to? (You may reserve the right, of course, to later not like something in the details of setuptools' version/filename rules, after I've put them into the PEP, or really, anything else. I'm just asking if there's anything that's obviously offensive at this point, before I spend time on a new PEP.) [View Less]

7 9