> I had an idea this morning for a simple extension to Python's iterator
> protocol that would allow the user to force an iterator to raise
> StopIteration on the next call to next(). My thought was to add a new
> method to iterators called stop().
There's no need to change the iterator protocol for your example use
case; you could just define a simple iterator-wrapper:
stopped = False
def __init__(self, iter):
self.iter = iter()
raise StopIteration('iterator stopped.')
self.stopped = True
And then just replace:
> generator = some_generator_function()
generator = InterruptableIterator(some_generator_function())
You can convert your repository to that branch with
svn switch svn+ssh://firstname.lastname@example.org/python/branches/ast-objects
in the toplevel directory. In particular, this features
The status is currently this:
- asdl_c generates a type hierarchy: "Sum" productions have one type
per constructor, inheriting from a type for the sum; plain products
only have a type for the product.
- attributes are in the base type, accessible through o->_base.attr;
projections of the product types are accessible directly through
- all projections must be non-NULL. Sequences are represented through
potentially empty lists; optional types are potentially represented
through Py_None. bool is gone; use Py_True/Py_False. The only
primitive type remaining is int (which only occurs in lineno)
- the types currently have only a constructor, a dealloc function,
and an _Check macro.
- Naming is this: for cross-object-file visible symbols (functions
and global variables), a Py_ prefix is used. Otherwise, I use
the type name or constructor name directly. There is a #define
for the Py_<type>_New function, so you can also write <type>(params).
Parameter order for the types is: projections first, then
- For compatibility with the current code, the Sum base types
also have the _kind enumeration (although that appears not
to get initialized right now).
For asdl_c, I see the following things as TODOs:
- add support for traversing the types from C, through tp_members
(read-only). Optionally add support for pickling.
- add support for garbage collection. I don't expect this to be
necessary right now, but will be if the API is exposed, and it
doesn't cost much.
The bigger chunk of necessary changes is in using these, starting
Feel free to commit any changes to that branch that you consider
helpful. To avoid duplicated work, posting a note here might
On numerous occasions, requests have been made for the ability to easily add
user-defined data to logging events. For example, a multi-threaded server
application may want to output specific information to a particular server
thread (e.g. the identity of the client, specific protocol options for the
client connection, etc.)
This is currently possible, but you have to subclass the Logger class and
override its makeRecord method to put custom attributes in the LogRecord.
These can then be output using a customised format string containing e.g.
"%(foo)s %(bar)d". The approach is usable but requires more work than
I'd like to propose a simpler way of achieving the same result, which
requires use of an additional optional keyword argument in logging calls.
The signature of the (internal) Logger._log method would change from
def _log(self, level, msg, args, exc_info=None)
def _log(self, level, msg, args, exc_info=None, extra_info=None)
The extra_info argument will be passed to Logger.makeRecord, whose signature
will change from
def makeRecord(self, name, level, fn, lno, msg, args, exc_info):
def makeRecord(self, name, level, fn, lno, msg, args, exc_info,
makeRecord will, after doing what it does now, use the extra_info argument
If type(extra_info) != types.DictType, it will be ignored.
Otherwise, any entries in extra_info whose keys are not already in the
LogRecord's __dict__ will be added to the LogRecord's __dict__.
Can anyone see any problems with this approach? If not, I propose to post
the approach on python-list and then if there are no strong objections,
check it in to the trunk. (Since it could break existing code, I'm assuming
(please correct me if I'm wrong) that it shouldn't go into the
Of course, if anyone can suggest a better way of doing it, I'm all ears :-)
> I couldn't think of a good reason why it should be possible to overwrite these
> values with values from a user-supplied dictionary, other than to spoof log
> entries in some way. The intention is to stop a user accidentally overwriting
> one of the above attributes.
This makes sense, but is it worth the time to check on each logging call?
Transferring part of the discussion of Thomas Lee's PEP 341 patch to
python-dev. . .
Neal Norwitz wrote in the SF patch tracker:
> Thomas, I hope you will write up this experience in coding
> this patch. IMO it clearly demonstrates a problem with the
> new AST code that needs to be addressed. ie, Memory
> management is not possible to get right. I've got a 700+
> line patch to ast.c to correct many more memory issues
> (hopefully that won't cause conflicts with this patch). I
> would like to hear ideas of how the AST code can be improved
> to make it much easier to not leak memory and be safe at the
> same time.
As Neal pointed out, it's tricky to write code for the AST parser and compiler
without accidentally letting memory leak when the parser or compiler runs into
a problem and has to bail out on whatever it was doing. Thomas's patch got to
v5 (based on Neal's review comments) with memory leaks still in it, my review
got rid of some of them, and we think Neal's last review of v6 of the patch
got rid of the last of them.
I am particularly concerned about the returns hidden inside macros in the AST
compiler's symbol table generation and bytecode generation steps. At the
moment, every function in compile.c which allocates code blocks (or anything
else for that matter) and then calls one of the VISIT_* macros is a memory
leak waiting to happen.
Something I've seen used successfully (and used myself) to deal with similar
resource-management problems in C code is to use a switch statement, rather
than getting goto-happy.
Specifically, the body of the entire function is written inside a switch
statement, with 'break' then used as the equivalent of "raise Exception". For
/* Real function body goes here */
/* Error cleanup code goes here */
It avoids the potential for labelling problems that arises when goto's are
used for resource cleanup. It's a far cry from real exception handling, but
it's the best solution I've seen within the limits of C.
A particular benefit comes when macros which may abort function execution are
used inside the function - if those macros are rewritten to use break instead
of return, then the function gets a chance to clean up after an error.
P.S. Getting rid of the flow control macros entirely is another option, of
course, but it would make compile.c and symtable.c a LOT harder to follow.
Raymond Chen's articles notwithstanding, a preprocessor-based mini-language
does make sense in some situations, and I think this is one of them.
Particularly since the flow control macros are private to the relevant
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
Let's have a Python bug day this Sunday. One goal might be
to assess bugs and patches, and make a list of ones we can work on at
the Python core sprint at PyCon
Meeting on IRC: #python-dev on irc.freenode.net
Date: Sunday, December 4th
Time: roughly 9AM to 3PM Eastern (2PM to 8PM UTC).
People on the US West Coast may want to show up from 9AM to
3PM Pacific time (12PM to 6PM Eastern), because it'll be more
Unless there are any objections, I'd like to create a space in the
sandbox for the standalone email package miscellany. This currently
lives in the mimelib project's hidden CVS on SF, but that seems pretty
Basically I'm just going to add the test script, setup.py, generated
html docs and a few additional unit tests, along with svn:external refs
to pull in Lib/email from the appropriate Python svn tree. This way,
I'll be able to create standalone email packages from the sandbox (which
I need to do because I plan on fixing a few outstanding email bugs).
on my machine, "make test" hangs at test_colorsys.
Careful investigation shows that when the bytecode is freshly generated
by "make all" (precisely in test___all__) the .pyc file is different from what a
direct call to "regrtest.py test_colorsys" produces.
Curiously, a call to "regrtest.py test___all__" instead of "make test" produces
the correct bytecode.
I can only suspect some AST bug here.
Mail address is perfectly valid!
I've not had much luck in searching for a discussion on this in the
Python-Dev archives, so bear with me.
I had an idea this morning for a simple extension to Python's iterator
protocol that would allow the user to force an iterator to raise
StopIteration on the next call to next(). My thought was to add a new
method to iterators called stop().
In my situation it would be useful as a control-flow mechanism, but I
imagine there are many other use cases for it:
generator = some_generator_function()
for x in generator:
... deeply ...
... nested ...
... control-flow ...
# Terminates the for-loop, but
# finishes the current iteration
... more stuff ...
I'm curious if anything like this has been proposed in the past. If so,
could someone kindly point me to any relevant mailing list threads?