I wrote an epoll implementation which can be used as a drop-in replacement
for parts of the select module (assuming the program is using only poll).
The code can currently be used by doing:
import epoll as select
It was released under the Python license on sourceforge:
Is there any interest in incorporating this into the standard python
I am a recent subscriber to this list. A couple of months ago
I downloaded the Python source out of curiosity, then decided
to see if I could scratch a couple of small itches.
One of them was syntax errors reported on one line but actually
resulting from unbalanced brackets on the previous line. (The
folks on this list may not make such silly mistakes any more,
but several postings on comp.lang.python suggest that I am not
entirely alone in finding it an annoyance.) I have modified
the parser and error reporting code to provide more explicit
information in such cases, and am now looking for some guidance
as to whether it is worth submitting a patch.
With my change a syntax error on a continuation line looks like
File "test.py", line 42 (statement started on line 41)
y = 0
SyntaxError: invalid syntax
The patch adds a new attribute to the SytaxError exception
object (the line number on which the statement started). I
don't have the experience to judge whether this would be
considered a compatibility problem. An alternative that does
not require modifying the exception would be to change the
message text without identifying the specific line number:
File "test.py", line 42
y = 0
SyntaxError: invalid syntax in continuation of prior line
However I think the continuation indication properly belongs
with the location information rather than with the message.
(The loss of the starting line number itself is no big deal;
it will usually be one less than the error line number.)
[... a huge number of reference leaks reported ...]
FYI, I "reduced" the relatively simple test_bisect's leaks to this
libreftest = """
No actual doctests here.
from sys import gettotalrefcount as trc
for i in range(10):
doctest.master = None
if __name__ == "__main__":
Running that here in a debug build:
So it leaks 6 references per iteration, and merely invoking
doctest.testmod() is all it takes to provoke it. Comment out the:
line in test_bisect.py, and test_bisect stops leaking too (which isn't
a trivial stmt: its doctests are a small part of test_bisect --
_most_ of it isn't leaking).
What happens next isn't obvious to me -- nobody has touched doctest.py
in over 2 weeks, so that's not the _source_ of the problem. Alas, I
won't have computer access for the next 13 hours or so, and have to
leave this here for now.
FWIW, the biggest change that went in since that last "normal" refleak
run is the new exception reworking, so that has to be a top suspect.
I'm playing with dicts that mangle the hash they receive before using it
for hashing. The goal was to detect obscure dict order dependencies in
my own programs, but I couldn't resist and ran the Python test suite
with various mangling schemes. As expected -- what is not tested is
broken -- I found and fixed tons of small dependencies in the tests
themselves, plus one in base64.py.
Now I'm stumbling upon this test for urllib2:
>>> mgr = urllib2.HTTPPasswordMgr()
>>> add = mgr.add_password
>>> add("Some Realm", "http://example.com/", "joe", "password")
>>> add("Some Realm", "http://example.com/ni", "ni", "ni")
Currently, we use the highest-level path where more than one
>>> mgr.find_user_password("Some Realm", "http://example.com/ni")
Returning the outermost path is a bit strange, if you ask me, but I am
no expert here. Stranger is the fact that the actual implement actually
returns, not the outermost path at all -- there is no code to do that --
but a random pick, the first match in dictionary order. The comment in
the test is just misleading. I believe that urllib2 should be fixed to
always return the *innermost* path, but I need confirmation about
in the last time, I've found myself reimplementing a generator that provides a
sliding-window-view over a sequence, and I think this function is of a greater
usefullness, so that it might be included in itertools.
Basically, what the generator does it return all m consecutive elements from a
sequence [i0, i1, ... in]. It then returns [i0, i1, i2], [i1, i2, i3], ...
[in-2, in-1, in] (assuming that m = 3).
In code, it looks like this:
>>> list(iwindow(range(0,5), 3))
[[0, 1, 2], [1, 2, 3], [2, 3, 4]]
This list can be generated by using izip(a, a[1:], ..., a[n:]), but typing all
the sequence arguments gets tedious. If a is not a sequence but a generator,
tee(...) and islice has to be used.
It might be possible that the windows should be padded, so that the sequence of
windows starts with [pad, pad, ..., i0] and ends with [in, pad, pad, ...]
>>> list(iwindow(range(0,5), 3, pad=True))
[[None, None, 0], [None, 0, 1], [0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, None],
[4, None, None]]
Additionally, the value used for padding can be specified. This makes the
argument list of this function rather long, but the last two arguments are
iwindow(iterable, window_size=3, pad = False, padding_value = None)
Some open question remain:
- should iwindow return lists or tuples?
- what happens if the length of the iterable is smaller than the window size,
and no padding is specified? Is this an error? Should the generator return no
value at all or one window that is too small?
I've attached a Python implementation of this function. If the function is
deemed to be actually useful, I'd be happy to brush up my C and provide a C
implementation along with docs and tests.
PS: Please CC me, as I'm not subscribed to the list
Torsten Marek <shlomme(a)gmx.net>
ID: A244C858 -- FP: 1902 0002 5DFC 856B F146 894C 7CC5 451E A244 C858
Raymond Hettinger wrote:
> Fredrik Lundh wrote:
>>does anyone remember? given what we're currently working on,
>>implementing it would take roughly no time at all. do people still
>>think it's a good idea ?
>></F> <!-- note: angle brackets, forward slash -->
> Yes. It went to my todo list and is awaiting some free raymond-cycles
> to finish it up. I've been task saturated of late but would like to get
> this a number of other patches complete for Py2.5.
> Also, Crutcher had long ago posted a patch for exec/eval to accept
> generic mapping arguments. If someone like Tim or /F has the time,
> feel free to pick it up.
This reminds me I am tasked with trying to find out what the interface
to timeit.py is supposed to look like. Raymond, your name has been
mentioned as someone who took part int he discussions. Google hasn't
given me a lot to go on. Anyone?
[Follow-ups to python-dev would be best, I suspect].
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Love me, love my blog http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden
First off, good work to everyone involved. You did a tremendous job.
I just hope to hell you're done, because I can't keep up! :-)
It would help me enormously if someone could summarize the status and
everything that went on. These are the things that would help me the
* What are the speed diffs before/after the sprint
* What was modified (summary)
* What is left to do
* Which branches are still planning to remain active
* Lessons learned, how we can improve for the next time
* Suggestions for further areas to look into improving
It looks like there were a lot of additions to the string test suite,
that's great. I'm not sure if the other areas touched got similar
boosts to their tests. It would be good to upgrade all tests to
verify corner cases of the implementation. These tests should also be
documented that they are to test the implementation rather than the
language spec. We don't want to write obscure tests that can't pass
in other impls like Jython, IronPython, or PyPy.
I will turn my amd64 box back on tomorrow and will also run Python
through valgrind and pychecker when I get a chance. There are a
couple of new Coverity complaints that need to be addressed.
several string methods accepts either strings or objects that support
the buffer api, and ends up raising a "expected a character buffer
object" error if you pass in something else. this isn't exactly helpful
for non-experts -- the term "character buffer object" doesn't appear in
any python tutorials I've seen.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: expected a character buffer object
I think should be fixed, but I cannot make up my mind on what's the best
way to fix it:
1. map all errors from PyObject_AsCharBuffer to "expected string
or compatible object", or some such.
2. modify PyObject_AsCharBuffer so it returns different negative
error codes depending on whether the object doesn't support the
protocol at all, or if something went wrong when using it.
3. to avoid changing the PyObject_AsCharBuffer return value, add a
PyObject_AsCharBufferEx that does the above
4. do something entirely different
Some time ago a warning was introduced for directories on sys.path
that don't contain an __init__.py but have the same name as a package/
module that is being imported.
Is it intentional that this triggers for toplevel imports? These
warnings are triggered in the build process for PyObjC, which is in
itself pretty harmless but very annoying.
We have a source-deps directory that contains several external
packages that are used during the build. Those external packages are
included through the svn:external mechanism and are in directories
named after those packages, with a .pth file to include them on
sys.path. In most cases the package name is the same as the python
package, which is triggering the warning.
Our package structure:
subprocess.pth # includes subproces on sys.path
py2app.pth # includes py2app on sys.path
... # the regular sources, documentation and stuff