Mailman 3 May 2008 - Python-Dev

Re: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
by M.-A. Lemburg 27 May '08

27 May '08

On 2008-05-27 12:56, Thomas Heller wrote: > M.-A. Lemburg schrieb: >> On 2008-05-26 23:34, Christian Heimes wrote: >>> M.-A. Lemburg schrieb: >>>> Isn't that an awefuly confusing approach ? >>>> >>>> Wouldn't it be better to keep PyString APIs and definitions in >>>> stringobject.c|h and only add a new bytesobject.h header file >>>> that #defines the PyBytes APIs in terms of PyString APIs ? That >>>> maintains backwards compatibility and allows Python internals to >>>> use the new API names. >>>> >>>> With your approach, you've basically backported the confusing >>>> notion in Py3k that str() maps PyUnicode, only that in Py2 str() >>>> will now map to PyBytes. >>> The last time I brought up the topic, I had a lengthy discussion >>> with Guido. At first I wanted to rename the API in Python 3.0 only. >>> Guido argued that it's going to cause too much merge conflicts. He >>> then suggested the approach I implemented today. >> That's the same argument that came up in the module renaming >> discussion. >> >> I have a feeling that we should be looking for better merge tools, >> rather than implement code changes that cause more trouble than do >> good, just because our existing tools aren't smart enough. > > There are applications out there that dynamically import the python dll > and link to the exported functions by name; they will all break. The exported APIs still use the old names. Just the source code versions of the APIs change to the new names and they now live in different files as well. > I believe in the past we have been more carefull with changes like these. > Even when python api functions were turned into cpp macros, we provided > exported functions for them; for a few examples see the function definitions > near line 1778 in file Python/pythonrun.c . IMO, we should keep using that strategy for Python 2.x. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 27 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 40 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611

3 2

itertools additions: one(), single_valued()
by Andreas Klöckner 27 May '08

27 May '08

Hi all, I find the following two operations functions useful and general enough that I would like to propose them for addition to itertools: 8< ------------------------------------------------------------------------- def single_valued(iterable): it = iter(iterable) try: first_item = it.next() except StopIteration: raise ValueError, "empty iterable passed to 'single_valued()'" for other_item in it: if other_item != first_item: raise ValueError, "non-single-valued iterable'" return first_item 8< ------------------------------------------------------------------------- This first one embodies the assertion that all values of the iterable are identical. If they are, that value is returned. Otherwise, an exception is thrown. Maybe this should be rephrased such that the assertion part is evaluated via an actual "assert", thereby turning it off in optimized mode. Example use case: You get a list of lists, and expect each to be the same length. Typical treatment: list_len = len(lists[0]) New treatment: list_len = single_valued(len(l) for l in lists) Added benefits: The assumption is verified and clearly visible from the source. "lists" may now be an iterable. 8< ------------------------------------------------------------------------- def one(iterable): it = iter(iterable) try: v = it.next() except StopIteration: raise ValueError, "empty iterable passed to 'one()'" try: v2 = it.next() raise ValueError, "iterable with more than one entry passed to 'one()'" except StopIteration: return v 8< ------------------------------------------------------------------------- This one is a blatant rip-off from SQLAlchemy. It basically allows most searches to be written using generator expressions: what_i_am_looking_for = one(item for item in items if predicate(item)) This also encodes and checks the assumption that the sought item is unique within the list of candidates. Again, the assertion part could be turned off in optimized mode. Opinions? Andreas

4 5

API bug in new socket timeout interfaces
by John J Lee 26 May '08

26 May '08

I'm hoping to draw somebody's attention to this before it's too late. This affects multiple stdlib modules: http://bugs.python.org/issue2451 """ The new timeout support in 2.6 makes use of new function socket.create_connection(). socket.create_connection() provides no way to disable timeouts, other than by relying on socket.getdefaulttimeout() returning None. This is unfortunate, because it was the purpose of the new timeout support to allow control of timeouts without reliance on global state. """ The author of the patch that introduced this problem (Facundo Batista) has admitted that the behaviour he implemented does not match the discussion about the feature that took place on python-dev, but has not found time to review my patch in the last two months. I fear this broken API will make it into 2.6 (and 3.0?). More could be done here re other issues introduced by the patch (I've listed those that I noticed on issue 2451 -- including at least one other API issue), but the patch attached to issue2451 at least resolves the most clear-cut problem. Thanks John

2 1

Slice as a copy... by design?
by Facundo Batista 26 May '08

26 May '08

Hi! A thread in PyAr raised the question that, considering that strings are immutable, why a slice of a string is a copy and not a reference to a part of that string. I couldn't answer why, so I'm asking here...Is it because the reference counting will be complicated? Is it because it'd be inefficient in other way? It's something else? Or is something that could be done... but is not done yet? Thank you very much! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

15 21

VS8 include dirs grow without bound
by Jim Kleckner 26 May '08

26 May '08

I tracked down a testing failure for Cython tests on the Windows platform to be the result of how the Visual C environment variables are being detected. In the function, query_vcvarsall, the .bat file: "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat" is being run with the arguments: x86 & set This in turn calls the platform-specific file vcvars32.bat (or other). In this file, the PATH, INCLUDE, and LIBPATH environment variables are simply being appended/prepended with their values. initialize() then just overrides the os.environ values with these monotonically growing variables. It seems that duplicate names should be filtered out from these paths to prevent overflow. The attached patch seems to fix this issue, if a bit brute force. Please review as a candidate. ===== @set PATH=%DevEnvDir%;%VCINSTALLDIR%\BIN;%VSINSTALLDIR%\Common7\Tools;%VSINSTALLDIR%\Common7\Tools\bin;%FrameworkDir%\%Framework35Version%;%FrameworkDir%\%Framework35Version%\Microsoft .NET Framework 3.5 (Pre-Release Version);%FrameworkDir%\%FrameworkVersion%;%VCINSTALLDIR%\VCPackages;%PATH% @set INCLUDE=%VCINSTALLDIR%\ATLMFC\INCLUDE;%VCINSTALLDIR%\INCLUDE;%INCLUDE% @set LIB=%VCINSTALLDIR%\ATLMFC\LIB;%VCINSTALLDIR%\LIB;%LIB% @set LIBPATH=%FrameworkDir%\%Framework35Version%;%FrameworkDir%\%FrameworkVersion%;%VCINSTALLDIR%\ATLMFC\LIB;%VCINSTALLDIR%\LIB;%LIBPATH% ===== --- msvc9compiler.py.orig 2008-05-26 12:29:18.529288200 -0700 +++ msvc9compiler.py 2008-05-26 12:34:11.403841800 -0700 @@ -193,6 +193,17 @@ reduced_paths.append(np) return reduced_paths +def removeDuplicates(variable): + """Remove duplicate values of an environment variable. + """ + oldList = variable.split(os.pathsep) + newList = [] + for i in oldList: + if i not in newList: + newList.append(i) + newVariable = os.pathsep.join(newList) + return newVariable + def find_vcvarsall(version): """Find the vcvarsall.bat file @@ -257,7 +268,7 @@ if key in interesting: if value.endswith(os.pathsep): value = value[:-1] - result[key] = value + result[key] = removeDuplicates(value) if len(result) != len(interesting): raise ValueError(str(list(result.keys())))

2 2

zipimport._zip_directory_cache interface
by Maciej Fijalkowski 26 May '08

26 May '08

_zip_directory_cache is a cache for modules imported by zipimporter. It's a normal dictionary exposing gory details of zipimport implementation. It wouldn't be a bad thing on it's own, but pkgutil.py uses this knowledge, while _zip_directory_cache has no single test nor documentation (besides info in docstring). It's a bit of a pain to actually implement this in other python implementations (especially that cache can be internal and exposed differently). I think it should be either specified better (and stripped from _) or zipimport should expose some nicer interface to it's cache. What do you think? Cheers, fijal

2 1

Re: [Python-Dev] Slice as a copy... by design?
by Maciej Fijalkowski 26 May '08

26 May '08

> Hi! > > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. > > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? > > Thank you very much! We implemented this feature in pypy. It's turned of by default, because nobody could come out with a real world example benefiting from it. Regarding worst-case scenario, we only create this special object if slice length is not shorter than some percentage of source string (I think 75%, can be tuned). You still need to pay for a dispatch (even if small price) wherever you use string, to check if this is special object or not. Cheers, fijal

1 0

Please svnmerge your changes
by Christian Heimes 25 May '08

25 May '08

Hello fellow developers! I've been busy with personal work in the past weeks. At present I'm still moving into my new apartment. It has been a real challenge to install an IKEA kitchen in a house built before WW2 all by myself. On the one hand it's fun but on the other hand it costs me most of my free time at night. At least this building has a shelter in its cellar so I'm mostly protected in the case of an air strike. *g* In order to get all code merged before the first betas I need your help. Please everybody grab a couple of your checkins and merge them yourself. You can find the list of required merges at http://rafb.net/p/cghbTk63.html Christian

3 3

Mercurial mirrors of trunk and py3k
by Antoine Pitrou 25 May '08

25 May '08

Hello, For those of you who have been using my Mercurial mirrors of the Python SVN repository, I had to move them to another site as the original site has been down for a few days for unknown reasons. Here are the new URLs: http://hg.pitrou.net/public/cpython/trunk/ http://hg.pitrou.net/public/py3k/py3k/ If you want those to become the default pulling location, you just have to edit the .hg/hgrc for your local clone(s) and fill in the appropriate URL. Regards Antoine.

1 0

Re: [Python-Dev] Stabilizing the C API of 2.6 and 3.0
by Christian Heimes 25 May '08

25 May '08

Stefan Behnel schrieb: > will that be included by Python.h by default? Only in Python 2.6 Christian

1 0