Patch #849407 proposes to change the meaning of the
urllib reporthook so that it takes the amount of the
data read instead of the block size as its second
argument.
While this is a behavior change (and even for
explicitly-documented behavior), I still propose
to apply the change:
- in many cases, the number of bytes read will
equal to the block size, so no change should
occur
- the signature (number of parameters) does not
change, so applications shouldn't crash because
of that change
- applications that do use the parameter to
estimate total download time now get a better
chance to estimate since they learn about
short reads.
What do you think?
Regards,
Martin
Hi,
this might seem a bit late, and perhaps I was just blind,
but I miss something like a summary how the Python
summer of code projects went, and what the status of the ones
that were meant to improve the standard library, e.g. the
C decimal implementation, is.
cheers,
Georg
I was looking around for an interface to POSIX capabilities from Python
under Linux. I couldn't find anything that did the job, so I wrote the
attached PosixCapabilities module. It has a number of shortcomings:
* it is written using ctypes to interface directly to libcap;
* it assumes the sizes/types of various POSIX defined types;
* it only gets/sets process capabilities;
* it can test/set/clear capability flags.
Despite the downsides, I think it would be good to get the package out
there. If anyone wishes to adopt it, update it, rewrite it and/or put
it into the distribution, then feel free.
Regards,
Matt
--
Matt Kern
http://www.undue.org/
Here's the summary for the first half of October. As always, comments
and corrections are greatly appreciated.
=============
Announcements
=============
-----------------------------
QOTF: Quotes of the Fortnight
-----------------------------
Martin v. Löwis on a small change to Python that wouldn't affect many
applications:
I'm pretty sure someone will notice, though; someone always notices.
Contributing thread:
- `Caching float(0.0)
<http://mail.python.org/pipermail/python-dev/2006-October/069175.html>`__
Steve Holden reminds us that patch submissions are dramatically
preferred to verbose thread discussions:
This thread has disappeared down a rat-hole, never to re-emerge
with anything of significant benefit to users. C'mon, guys, implement
a patch or leave it alone :-)
Contributing thread:
- `Caching float(0.0)
<http://mail.python.org/pipermail/python-dev/2006-October/069190.html>`__
=========
Summaries
=========
--------------
Caching floats
--------------
Nick Craig-Wood discovered that he could save 7MB in his application
by adding the following simple code::
if age == 0.0:
age = 0.0
A large number of his calculations were producing the value 0.0, which
meant that many copies of 0.0 were being stored. Since all 0.0
literals refer to the same object, the code above was removing all the
duplicate copies of 0.0.
Skip Montanaro and played around a bit with floatobject.c, and found
that Python's test suite allocated a large number of small integral
floats (though only a couple hundred were generally allocated at the
same time). Kristján V. Jónsson played around with caching for float
values between -10.0 and 10.0 with the EVE server and got a 25%
savings in allocations.
There was some concern that for systems with both +0.0 and -0.0, the
cache might cause problems, since determining which zero you have
seemed difficult. However, James Y Knight showed how to do this fairly
easily in C with a double/uint64_t union. Eventually, people agreed
that it should be fine to just cache +0.0.
Kristján V. Jónsson and Josiah Carlson proposed patches, but nothing
was posted to SourceForge.
Contributing threads:
- `Caching float(0.0)
<http://mail.python.org/pipermail/python-dev/2006-September/069049.html>`__
- `Caching float(0.0)
<http://mail.python.org/pipermail/python-dev/2006-October/069092.html>`__
--------------------------------------------
Buffer overrun in repr() and Python releases
--------------------------------------------
The implications of PSF-2006-001_, a buffer overrun problem in repr(),
were considered for the various Python releases. The bug had been
fixed before Python 2.5 was released, and had been applied to the
Python 2.4 branch shortly before Python 2.4.4 was released. The
security advisory provided patches for both Python 2.3 and 2.4, but to
make sure that full source releases were available for all major
versions of Python still in use, it looked like there would be a
source-only 2.3.6 release (source-only because neither Mac nor Windows
builds were affected).
.. _PSF-2006-001: http://www.python.org/news/security/PSF-2006-001/
Contributing threads:
- `Security Advisory for unicode repr() bug?
<http://mail.python.org/pipermail/python-dev/2006-October/069247.html>`__
- `2.3.6 for the unicode buffer overrun
<http://mail.python.org/pipermail/python-dev/2006-October/069327.html>`__
---------------------------
Build system for python.org
---------------------------
Anthony, Barry Warsaw, Georg Brandl and others indicated that the
current website build system was making releases and other updates
more difficult than they should be. Most people didn't have enough
cycles to spare for this, but Michael Foord said he could help with a
transition to rest2web_ if that was desirable. Fredrik Lundh also
suggested a few options, including his older proposal to `use
Django`_. No definite plans were made though.
.. _rest2web: http://www.voidspace.org.uk/python/rest2web/
.. _use Django: http://effbot.org/zone/pydotorg-cache.htm
Contributing thread:
- `2.3.6 for the unicode buffer overrun
<http://mail.python.org/pipermail/python-dev/2006-October/069327.html>`__
--------------------
String concatenation
--------------------
Larry Hastings posted a `patch for string concatenation`_ that delays
the creation of a new string until someone asks for the string's
value. As a result, the following code would be about as fast as the
``''.join(strings)`` idiom::
result = ''
for s in strings:
result += s
To achieve this, he had to change ``PyStringObject.ob_sval`` from a
``char[1]`` array, to a ``char *``. Reaction was mixed -- some people
really disliked using ``join()``, while others didn't see the need for
such a change.
.. _patch for string concatenation: http://bugs.python.org/1569040
Contributing thread:
- `PATCH submitted: Speed up + for string concatenation, now as fast
as "".join(x) idiom
<http://mail.python.org/pipermail/python-dev/2006-October/069224.html>`__
----------------------------
PEP 315: Enhanced While Loop
----------------------------
Hans Polak revived the discussion about `PEP 315`_, which proposes a
do-while loop for Python that would allow the current code::
while True:
<setup code>
if not <condition>:
break
<loop body>
to be written instead as::
do:
<setup code>
while <condition>:
<loop body>
Hans was hoping to simplify the situation where there is no ``<loop
body>`` following the ``<condition>`` test and a number of syntax
suggestions were proposed to this end. In the end, Guido indicated
that none of the suggestions were acceptable, and Raymond Hettinger
offered to withdraw the PEP.
.. _PEP 315: http://www.python.org/dev/peps/pep-0315/
Contributing threads:
- `PEP 351 - do while
<http://mail.python.org/pipermail/python-dev/2006-September/069088.html>`__
- `PEP 351 - do while
<http://mail.python.org/pipermail/python-dev/2006-October/069100.html>`__
- `PEP 315 - do while
<http://mail.python.org/pipermail/python-dev/2006-October/069205.html>`__
------------------------------
PEP 355: path objects rejected
------------------------------
Luis P Caamano asked about the status of `PEP 355`_, which aimed to
introduce an object-oriented reorganization of Python's path-related
functions. Guido indicated that the current "amalgam of unrelated
functionality" was unacceptable and pronounced it dead. Nick Coghlan
elaborated the "amalgam" point, explaining that `PEP 355`_ lumped
together all the following:
- string manipulation operations
- abstract path manipulation operations
- read-only traversal of a concrete filesystem
- addition and removal of files/directories/links within a concrete filesystem
Jason Orendorff pointed out some other problems with the PEP:
- the motivation was weak
- the API had too many methods
- it didn't fix all the perceived problems with the existing APIs
- it would have introduced a Second Way To Do It without being clearly
better than the current way
There were some rumors of a new PEP based on Twisted's filepath_
module, but nothing concrete at the time of this summary.
.. _PEP 355: http://www.python.org/dev/peps/pep-0355/
.. _filepath: http://twistedmatrix.com/trac/browser/trunk/twisted/python/filepath.py
Contributing threads:
- `PEP 355 status
<http://mail.python.org/pipermail/python-dev/2006-September/069059.html>`__
- `PEP 355 status
<http://mail.python.org/pipermail/python-dev/2006-October/069093.html>`__
----------------------------------
Processes and threading module API
----------------------------------
Richard Oudkerk proposed a module that would make processes usable
with an API like that of the threading module. People seemed unsure as
to whether it would be better to have a threading-style or
XML-RPC-style API. A few other relevant modules were identified,
including PyXMLRPC_ and POSH_. No clear winner emerged.
.. _PyXMLRPC: http://sourceforge.net/projects/py-xmlrpc/
.. _POSH: http://poshmodule.sourceforge.net/
Contributing thread:
- `Cloning threading.py using proccesses
<http://mail.python.org/pipermail/python-dev/2006-October/069297.html>`__
------------------------------------
Python 3.0: registering methods in C
------------------------------------
After a brief exchange about which ``tp_flags`` implied which others,
there was some discussion on how to simplify ``tp_flags`` for Python
3000. Raymond Hettinger suggested that the NULL or non-NULL status of
a slot should be enough to indicate its presence. Martin v. Löwis
pointed out that this would require recompiling extension modules for
every release, since if a new slot is added, extension modules from
earlier releases wouldn't even *have* the slot. Fredrik Lundh
suggested a `dynamic registration method`_ instead, which would look
something like::
static PyTypeObject *NoddyType;
NoddyType = PyType_Setup("noddy.Noddy", sizeof(Noddy));
PyType_Register(NoddyType, PY_TP_DEALLOC, Noddy_dealloc);
PyType_Register(NoddyType, PY_TP_DOC, "Noddy objects");
...
PyType_Register(NoddyType, PY_TP_NEW, Noddy_new);
if (PyType_Ready(&NoddyType) < 0)
return;
People thought this looked like a good idea, and Fredrik Lundh planned
to look into it seriously for Python 3000.
.. _dynamic registration method: http://effbot.org/zone/idea-register-type.htm
Contributing thread:
- `2.4.4: backport classobject.c HAVE_WEAKREFS?
<http://mail.python.org/pipermail/python-dev/2006-October/069235.html>`__
-----------------------------------------
Tracker Recommendations: JIRA and Roundup
-----------------------------------------
The PSF Infrastructure Committee announced their recommendations for
trackers to replace SourceForge. Both JIRA and Roundup were definite
improvements over SourceForge, though the Infrastructure Committee was
leaning towards JIRA since Atlassian had offered to host it for them.
Roundup was still under consideration if 6-10 admins could volunteer
to maintain the installation.
(More updates on this in the next summary.)
Contributing thread:
- `PSF Infrastructure Committee's recommendation for a new issue
tracker <http://mail.python.org/pipermail/python-dev/2006-October/069139.html>`__
-----------------------------
PEP 302: import hooks phase 2
-----------------------------
Brett Cannon announced that he'd be working on a C implementation of
phase 2 of `PEP 302`_. Phillip J. Eby pointed out that phase 2 could
not be implemented in a backwards-compatible way, and so the code
should be targeted at the p3yk branch. He also suggested that
rewriting the import mechanisms in Python was probably going to be
easier than trying to do it in C, particularly since some of the
pieces were already available in the pkgutil module. Neal Norwitz
strongly agreed, pointing out that string and list manipulation, which
is necessary in a variety of places in the import mechanisms, is much
easier in Python than in C. Brett promised a Python implementation as
part of his research work.
.. _PEP 302: http://www.python.org/dev/peps/pep-0302/
Contributing thread:
- `Created branch for PEP 302 phase 2 work (in C)
<http://mail.python.org/pipermail/python-dev/2006-October/069133.html>`__
------------------------------------------
Web crawlers and development documentation
------------------------------------------
Fredrik Lundh noticed that Google was sometimes finding the
`development documentation`_ instead of the `current release
documentation`_. A.M. Kuchling added a ``robots.txt`` to keep crawlers
out of the development area.
.. _development documentation: http://docs.python.org/dev/
.. _current release documentation: http://docs.python.org/
Contributing thread:
- `what's really new in python 2.5 ?
<http://mail.python.org/pipermail/python-dev/2006-October/069158.html>`__
-----------------------------------------
Buildbots, compile errors and batch files
-----------------------------------------
Tim Peters noticed that bsddb was getting compile errors on Windows
but the buildbots were not reporting anything. Because some additional
commands were added after the call to ``devenv.com`` in the
``build.bat`` script, the error status was not getting propagated
appropriately. After Tim and Martin v. Löwis figured out how to repair
this, the buildbots were again able to report compile errors.
Contributing thread:
- `2.4 vs Windows vs bsddb
<http://mail.python.org/pipermail/python-dev/2006-October/069283.html>`__
---------------------------------
Python 2.5 and Visual Studio 2005
---------------------------------
Kristján V. Jónsson showed that using Visual Studio 2005 instead of
Visual Studio 2003 gave a 7% gain in speed, and a 10% gain when
performance guided optimization (PGO) was enabled. While the
"official" compiler can't get changed at a point release, everyone
agreed that making the PCBuild8 directory work out of the box and
adding an appropriate buildslave was a good idea. Kristján promised to
look into setting up a buildslave.
Contributing thread:
- `Python 2.5 performance
<http://mail.python.org/pipermail/python-dev/2006-October/069338.html>`__
----------------------------------
Distributing debug build of Python
----------------------------------
David Abrahams asked if python.org would be willing to post links to
the ActiveState debug builds of Python to make it easier for
Boost.Python_ users to obtain a debug build. People seemed to think
that Boost.Python_ users should be able to create a debug build of
Python themselves if necessary.
.. _Boost.Python: http://www.boost.org/libs/python/doc/index.html
Contributing thread:
- `Plea to distribute debugging lib
<http://mail.python.org/pipermail/python-dev/2006-October/069325.html>`__
-----------------------------------------
Unmarshalling/Unpickling multiple objects
-----------------------------------------
Tim Lesher proposed adding a generator to marshal and pickle so that
instead of::
while True:
try:
obj = marshal.load(fobj) # or pickle.load(fobj)
except EOFError:
break
... do something with obj ...
you could write something like::
for obj in marshal.loaditer(fobj): # or pickle.loaditer(fobj)
... do something with obj ...
when you wanted to load multiple objects in sequence from the same
file. Both Perforce and Mailman store objects in a way that would
benefit from such a function, so it seemed like such an API might be
reasonable. No patch had been submitted at the time of this summary.
Contributing thread:
- `Iterating over marshal/pickle
<http://mail.python.org/pipermail/python-dev/2006-October/069277.html>`__
-------------------------------
spawnvp and spawnvpe on Windows
-------------------------------
Alexey Borzenkov asked why spawnvp and spawnvpe weren't available in
Python on Windows even though they were implemented in the CRT. He got
the usual answer, that no one had submitted an appropriate patch, but
that such a patch would be a reasonable addition for Python 2.6.
Fredrik Lundh pointed out that the subprocess module was probably a
better choice than spawnvp and spawnvpe anyway.
Contributing thread:
- `Why spawnvp not implemented on Windows?
<http://mail.python.org/pipermail/python-dev/2006-October/069342.html>`__
==================
Previous Summaries
==================
- `difficulty of implementing phase 2 of PEP 302 in Python source
<http://mail.python.org/pipermail/python-dev/2006-October/069116.html>`__
- `Python Doc problems
<http://mail.python.org/pipermail/python-dev/2006-October/069252.html>`__
- `Signals, threads, blocking C functions
<http://mail.python.org/pipermail/python-dev/2006-October/069334.html>`__
===============
Skipped Threads
===============
- `Removing __del__
<http://mail.python.org/pipermail/python-dev/2006-September/068875.html>`__
- `Tix not included in 2.5 for Windows
<http://mail.python.org/pipermail/python-dev/2006-October/069095.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2006-October/069098.html>`__
- `HAVE_UINTPTR_T test in configure.in
<http://mail.python.org/pipermail/python-dev/2006-October/069103.html>`__
- `OT: How many other people got this spam?
<http://mail.python.org/pipermail/python-dev/2006-October/069117.html>`__
- `2.4.4 fixes <http://mail.python.org/pipermail/python-dev/2006-October/069160.html>`__
- `2.4.4 fix: Socketmodule Ctl-C patch
<http://mail.python.org/pipermail/python-dev/2006-October/069189.html>`__
- `[Python-checkins] r51862 -
python/branches/release25-maint/Tools/msi/msi.py
<http://mail.python.org/pipermail/python-dev/2006-October/069201.html>`__
- `Fwd: [ python-Feature Requests-1567948 ] poplib.py list interface
<http://mail.python.org/pipermail/python-dev/2006-October/069223.html>`__
- `Can't check in on release25-maint branch
<http://mail.python.org/pipermail/python-dev/2006-October/069256.html>`__
- `if __debug__: except Exception, e: pdb.set_trace()
<http://mail.python.org/pipermail/python-dev/2006-October/069271.html>`__
- `2.5, 64 bit <http://mail.python.org/pipermail/python-dev/2006-October/069275.html>`__
- `BUG (urllib2) Authentication request header is broken on long
usernames and passwords
<http://mail.python.org/pipermail/python-dev/2006-October/069284.html>`__
- `[Python-3000] Sky pie: a "var" keyword
<http://mail.python.org/pipermail/python-dev/2006-October/069292.html>`__
- `Proprietary code in python?
<http://mail.python.org/pipermail/python-dev/2006-October/069294.html>`__
- `DRAFT: python-dev summary for 2006-08-16 to 2006-08-31
<http://mail.python.org/pipermail/python-dev/2006-October/069299.html>`__
- `BRANCH FREEZE, release24-maint for 2.4.4c1. 00:00UTC, 11 October
2006 <http://mail.python.org/pipermail/python-dev/2006-October/069302.html>`__
- `2.4 vs Windows vs bsddb [correction]
<http://mail.python.org/pipermail/python-dev/2006-October/069310.html>`__
- `RELEASED Python 2.4.4, release candidate 1
<http://mail.python.org/pipermail/python-dev/2006-October/069326.html>`__
- `ConfigParser: whitespace leading comment lines
<http://mail.python.org/pipermail/python-dev/2006-October/069337.html>`__
- `Exceptions and slicing
<http://mail.python.org/pipermail/python-dev/2006-October/069363.html>`__
- `Proposal: No more standard library additions
<http://mail.python.org/pipermail/python-dev/2006-October/069398.html>`__
- `[py3k] Re: Proposal: No more standard library additions
<http://mail.python.org/pipermail/python-dev/2006-October/069399.html>`__
- `Modulefinder
<http://mail.python.org/pipermail/python-dev/2006-October/069412.html>`__
- `VC6 support on release25-maint
<http://mail.python.org/pipermail/python-dev/2006-October/069426.html>`__
- `os.utime on directories: bug fix or new feature?
<http://mail.python.org/pipermail/python-dev/2006-October/069431.html>`__
- `Problem building module against Mac Python 2.4 and Python 2.5
<http://mail.python.org/pipermail/python-dev/2006-October/069434.html>`__
Brett:
> As for the adding of logging to the stdlib modules ... we need the mentors
> to step forward and say something about that.
The logging additions are not ready for stdlib inclusion at this time.
Some modules are closer than others, but whether it makes sense to add
them piecemeal is a different question.
-jJ
Discussion of what we want in terms of the schema for the new issue tracker
has begun. If you wish to give feedback on what you would like each issue
to have in terms of data then please file an issue in the meta tracker at
http://psf.upfronthosting.co.za/roundup/meta/ . You can see the current
test tracker at http://psf.upfronthosting.co.za/roundup/tracker/ . And the
tracker-discuss mailing list is at
http://mail.python.org/mailman/listinfo/tracker-discuss (although you can
bypass the list and use the meta tracker to your ideas relating to the
schema).
If you do participate through the meta tracker please sign up for an account
so that it is not anonymous. I really hope that Anthony and Neal can
participate so that we can make sure the tracker does what they need to make
their lives easier during a release. And obviously everyone who still works
with bugs and patches should participate as well. We can change the schema
even after we launch to the new tracker, but it would be nice to minimize
the amount of feature churn once the tracker is up and going.
-Brett
I have a fix to my bug report 1593035 regarding
python-2.5 not working with readline on ia64-Linux.
This bug was found while trying to port SAGE
(http://modular.math.washington.edu/sage/) to ia64-Linux.
The problem is caused by the line of Modules/readline.c
in flex_complete()
return completion_matches(text, *on_completion);
In readline-5.2, completion_matches() is defined in
compat.c as
char ** completion_matches(const char *,rl_compentry_func_t *);
But in Modules/readline.c completion_matches() by default is
assumed to return an int, and on_completion() is defined as
char *.
To fix the problem, both the function itself and the
second argument need to be cast to the correct types
in Modules/readline.c/flex_complete()
return (char **) completion_matches(text, (rl_compentry_func_t *)*on_completio
n);
and completion_matches needs to be defined as an external
function. I added the following else clause to the ifdef
at the top of Modules/readline.c/flex_complete()
#ifdef HAVE_RL_COMPLETION_MATCHES
#define completion_matches(x, y) \
rl_completion_matches((x), ((rl_compentry_func_t *)(y)))
#else
extern char ** completion_matches(const char *,rl_compentry_func_t *);
#endif
Kate Minola
University of Maryland, College Park
Wow. Did you catch this news?
http://www-tech.mit.edu/V125/N65/coursevi.html
The first four weeks of C1 will be a lot like the first
four weeks of 6.001, Abelson said. The difference is
that programming will be done in Python and not Scheme.
-- ?!ng
I'd like to share an observation on portability of extension
modules to Python 2.5: python-ldap would crash on Solaris, see
http://groups.google.com/group/comp.lang.python/msg/a678a969c90f21ab?dmode=…
It turns out that this was caused by a mismatch in malloc
"families" (PyMem_Del vs. PyObject_Del):
http://sourceforge.net/tracker/index.php?func=detail&aid=1575329&group_id=2…
So if Python 2.5 crashes in malloc/free, it's probably a good
guess that some extension module failed use correct APIs.
There is probably not much we can do about this: it's already
mentioned in "Porting to 2.5" of whatsnew25. It would be good
if people were aware of this issue (and the other changes to
the C API); thus I hope that this message/thread makes it to
the python-dev summary :-)
Regards,
Martin
