It is our pleasure to announce the release of version 1.6.1 of the SimPy
discrete event simulation package. Download from
http://simpy.sourceforge.net/ .
What is SimPy?
==============
SimPy (= Simulation in Python) is an object-oriented, process-based
discrete-event simulation language based on standard Python. It is released
under the GNU Lesser GPL (LGPL). SimPy provides the modeler with components
of a simulation model including processes, for active components like
customers, messages, and vehicles, and resources, for passive components
that form limited capacity congestion points like servers, checkout
counters, and tunnels. It also provides monitor variables to aid in
gathering statistics. Random variates are provided by the standard Python
random module.
SimPy is in use at many universities, research institutes and in industry.
What is new?
============
- Addition of Tally data collection class as an alternative
to Monitor. It is intended for collecting very large data sets (50000
observations or more)
more efficiently in storage space and time than Monitor.
- Addition of function setHistogram to class Monitor for initializing
histograms.
- Addition of function allEventTimes (returns event times of all scheduled
events).
- Change of Resource to work with Tally (new Resource
API is backwards-compatible with 1.6).
- Revised function allEventNotices() for debugging/teaching purposes. It
returns
a pretty-printed string with event times and names of process instances.
Nature/purpose of this release?
===============================
SimPy 1.6.1 is a minor sub-release of version 1.6. It does not add any new
simulation constructs,
but just provides support to efficient simulation data collection of large
numbers of observations.
Enjoy, and don't forget to give us you feedback!
Klaus Müller
Tony Vignaux
======================================
Open Source Developers' Conference,
26-28 February, 2006, Netanya, Israel
======================================
Announcement and Call for Papers
http://www.osdc.org.il/call_for_paper.html
About
------
The Open Source Developers' Conferences (OSDCs) are grassroots symposia
originating from Australia. The conference tries to bring together the
users of various OS technologies such as Perl, Python, Ruby, Tcl, PHP,
MySQL, PostgreSQL, Subversion and more.
The Israeli OSDC is organized by the Israeli Perl Mongers and the
Israeli Python user group.
Our aim is to bring together practitioners, scholars, students,
programmers, researchers and managers from different areas to discuss
their views on various aspects of the Open Source technologies, to share
knowledge and to have fun while doing so.
Structure
----------
The conference will last for 3 days with 3 parallel tracks of
presentations. Presentations will be given in either Hebrew or English,
depending on the speaker's preference (the language will be listed in
advance).
Call for Papers
----------------
Recommended Topics for Python Related Presentations:
* Python based web & content-management technology (e.g. Zope)
* Major or Interesting packages (e.g. Twisted)
* Language development and new implementations (e.g. PyPy)
* Interfacing other languages (e.g. boost-python)
The examples given above are suitable for long presentations or
tutorials, but we also accept 40 and 20 minutes presentations, and 5
minute lightning talks.
Any other topics which are interesting and relevant to Open Source
Developers are also welcome.
For submitting proposals: http://www.osdc.org.il/proposal.html
For updates and more details:
http://www.osdc.org.il/index.html
[The HTML version of this Summary will be available at
http://www.python.org/dev/summary/2005-10-16_2005-10-31.html]
=============
Announcements
=============
--------------
AST for Python
--------------
As of October 21st, Python's compiler now uses a real Abstract Syntax Tree
(AST)! This should make experimenting with new syntax much easier, as well
as allowing some optimizations that were difficult with the previous
Concrete Syntax Tree (CST). While there is no Python interface to the AST
yet, one is intended for the not-so-distant future.
Thanks again to all who contributed, most notably: Armin Rigo, Brett Cannon,
Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Neil Schemenauer,
Nick Coghlan and Tim Peters.
Contributing threads:
- `AST branch merge status <
http://mail.python.org/pipermail/python-dev/2005-October/057347.html>`__
- `AST branch update <
http://mail.python.org/pipermail/python-dev/2005-October/057387.html>`__
- `AST branch is in? <
http://mail.python.org/pipermail/python-dev/2005-October/057483.html>`__
- `Questionable AST wibbles <
http://mail.python.org/pipermail/python-dev/2005-October/057489.html>`__
- `[Jython-dev] Re: AST branch is in? <
http://mail.python.org/pipermail/python-dev/2005-October/057642.html>`__
[SJB]
--------------------
Python on Subversion
--------------------
As of October 27th, Python is now on Subversion! The new repository is
http://svn.python.org/projects/. Check the `Developers FAQ`_ for information
on how to get yourself setup with Subversion. Thanks again to Martin v.
Löwis for making this possible!
.. _Developers FAQ: http://www.python.org/dev/devfaq.html#subversion-svn
Contributing threads:
- `Migrating to subversion <
http://mail.python.org/pipermail/python-dev/2005-October/057424.html>`__
- `Freezing the CVS on Oct 26 for SVN switchover <
http://mail.python.org/pipermail/python-dev/2005-October/057537.html>`__
- `CVS is read-only <
http://mail.python.org/pipermail/python-dev/2005-October/057679.html>`__
- `Conversion to Subversion is complete <
http://mail.python.org/pipermail/python-dev/2005-October/057690.html>`__
[SJB]
---------------
Faster decoding
---------------
M.-A. Lemburg checked in Walter Dörwald's patches that improve decoding
speeds by using a character map. These should make decoding into mac-roman
or iso8859-1 nearly as fast as decoding into utf-8. Thanks again guys!
Contributing threads:
- `Unicode charmap decoders slow <
http://mail.python.org/pipermail/python-dev/2005-October/057341.html>`__
- `New codecs checked in <
http://mail.python.org/pipermail/python-dev/2005-October/057505.html>`__
- `KOI8_U (New codecs checked in) <
http://mail.python.org/pipermail/python-dev/2005-October/057576.html>`__
[SJB]
=========
Summaries
=========
---------------------
Strings in Python 3.0
---------------------
Guido proposed that in Python 3.0, all character strings would be unicode,
possibly with multiple internal representations. Some of the issues:
- Multiple implementations could make the C API difficult. If utf-8, utf-16
and utf-32 are all possible, what types should the C API pass around?
- Windows expects utf-16, so using any other encoding will mean that calls
to Windows will have to convert to and from utf-16. However, even in current
Python, all strings passed to Windows system calls have to undergo 8 bit to
utf-16 conversion.
- Surrogates (two code units encoding one code point) can slow indexing down
because the number of bytes per character isn't constant. Note that even
though utf-32 doesn't need surrogates, they may still be used (and must be
interpreted correctly) in utf-32 data. Also, in utf-32, "graphemes" (which
correspond better to the traditional concept of a "character" than code
points do) may still be composed of multiple code points, e.g. "é" (e with a
accent) can be written as "e" + "'".
This last issue was particularly vexing -- Guido thinks "it's a bad idea to
offer an indexing operation that isn't O(1)". A number of proposals were put
forward, including:
- Adding a flag to strings to indicate whether or not they have any
surrogates in them. This makes indexing O(1) when no surrogates are in a
string, but O(N) otherwise.
- Using a B-tree instead of an array for storage. This would make all
indexing O(log N).
- Discouraging using the indexing operations by providing an alternate API
for strings. This would require creating iterator-like objects that keep
track of position in the unicode object. Coming up with an API that's as
usable as the slicing API seemed difficult though.
Contributing thread:
- `Divorcing str and unicode (no more implicit conversions). <
http://mail.python.org/pipermail/python-dev/2005-October/057362.html>`__
[SJB]
-------------------
Unicode identifiers
-------------------
Martin v. Löwis suggested lifting the restriction that identifiers be ASCII.
There was some concern about confusability, with the contention that
confusions like "O" (uppercase O) for "0" (zero) and "1" (one) for "l"
(lowercase L) would only multiply if larger character sets were allowed.
Guido seemed less concerned about this problem than about about how easy it
would be to share code across languages. Neil Hodgson pointed out that even
though a transliteration into English exists for Japanese, the coders he
knew preferred to use relatively meaningless names, and Oren Tirosh
indicated that Israeli programmers often preferred transliterations for
local business terminology. In either case, with or without unicode
identifiers the code would already be hard to share. In the end, people
seemed mostly in favor of the idea, though there was some suggestion that it
should wait until Python 3.0.
Contributing threads:
- `Divorcing str and unicode (no more implicit conversions). <
http://mail.python.org/pipermail/python-dev/2005-October/057362.html>`__
- `i18n identifiers (was: Divorcing str and unicode (no more implicit
conversions). <
http://mail.python.org/pipermail/python-dev/2005-October/057812.html>`__
- `i18n identifiers <
http://mail.python.org/pipermail/python-dev/2005-October/057813.html>`__
[SJB]
-----------------
Property variants
-----------------
People still seem not quite pleased with properties, both in the syntax, and
in how they interact with inheritance. Guido proposed changing the
property() builtin to accept strings for fget, fset and fdel in addition to
functions (as it currently does). If strings were passed, the property()
object would have late-binding behavior, that is, the function to call
wouldn't be looked-up until the attribute was accessed. Properties whose
fget, fset and fdel functions can be overridden in subclasses might then
look like::
class C(object):
foo = property('getFoo', 'setFoo', None, 'the foo property')
def getFoo(self):
return self._foo
def setFoo(self, foo):
self._foo = foo
There were mixed reactions to this proposal. People liked getting the
expected behavior in subclasses, but it does violate DRY (Don't Repeat
Yourself). I posted an `alternative solution`_ using metaclasses that would
allow you to write properties like::
class C(object):
class foo(Property):
"""The foo property"""
def get(self):
return self._foo
def set(self, foo):
self._foo = foo
which operates correctly with subclasses and follows DRY, but introduces a
confusion about the referrent of "self". There were also a few suggestions
of introducing a new syntax for properties (see `Generalizing the class
declaration syntax`_) which would have produced things like::
class C(object):
Property foo():
"""The foo property"""
def get(self):
return self._foo
def set(self, foo):
self._foo = foo
At the moment at least, it looks like we'll be sticking with the status quo.
.. _alternative solution:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/442418
Contributing threads:
- `Definining properties - a use case for class decorators? <
http://mail.python.org/pipermail/python-dev/2005-October/057350.html>`__
- `Defining properties - a use case for class decorators? <
http://mail.python.org/pipermail/python-dev/2005-October/057407.html>`__
- `properties and block statement <
http://mail.python.org/pipermail/python-dev/2005-October/057419.html>`__
- `Property syntax for Py3k (properties and block statement) <
http://mail.python.org/pipermail/python-dev/2005-October/057427.html>`__
[SJB]
-------------------
PEP 343 resolutions
-------------------
After Guido accepted the idea of adding a __with__() method to the context
protocol, `PEP 343`_ was reverted to "Proposed" until the remaining details
could be ironed out. The end results were:
- The slot name "__context__" will be used instead of "__with__".
- The builtin name "context" is currently offlimits due to its ambiguity.
- Generator-iterators do NOT have a native context.
- The builtin function "contextmanager" will convert a generator-function
into a context manager.
- The "__context__" slot will NOT be special cased. If it defines a
generator, the __context__() function should be decorated with
@contextmanager.
- When the result of a __context__() call returns an object that lacks an
__enter__() or __exit__() method, an AttributeError will be raised.
- Only locks, files and decimal.Context objects will gain __context__()
methods in Python 2.5.
Guido seemed to agree with all of these, but has not yet pronounced on the
revised `PEP 343`_.
.. _PEP 343: http://www.python.org/peps/pep-0343.html
Contributing threads:
- `PEP 343 updated <
http://mail.python.org/pipermail/python-dev/2005-October/057349.html>`__
- `Proposed resolutions for open PEP 343 issues <
http://mail.python.org/pipermail/python-dev/2005-October/057516.html>`__
- `PEP 343 - multiple context managers in one statement <
http://mail.python.org/pipermail/python-dev/2005-October/057637.html>`__
- `PEP 343 updated with outcome of recent discussions <
http://mail.python.org/pipermail/python-dev/2005-October/057769.html>`__
[SJB]
---------------
Freeze protocol
---------------
Barry Warsaw propsed `PEP 351`_, which suggests a freeze() builtin which
would call the __freeze__() method on an object if that object was not
hashable. This would allow dicts to automatically make frozen copies of
mutable objects when they were used as dict keys. It could reduce the need
for "x" and "frozenx" builtin pairs, since the frozen versions could be
automatically derived when needed. Raymond Hettinger indicated some problems
with the proposal:
- sets.Set supported something similar, but found that it was not really
helpful in practice.
- Freezing a list into a tuple is not appropriate since they do not have all
the same methods.
- Errors can arise when the mutable object gets out of sync with its frozen
copy.
- Manually freezing things when necessary is relatively simple.
Noam Raphael proposed a copy-on-change mechanism which would essentially
give frozen copies of an object a reference to that object. When the object
is about to be modified, a copy would be made, and all frozen copies would
be pointed at this. Thus an object that was mutable but never changed could
have lightweight frozen copies, while an object that did change would have
to pay the usual copying costs. Noam and Josiah Carlson then had a rather
heated debate about how feasible such a copy-on-change mechanism would be
for Python.
.. _PEP 351: http://www.python.org/peps/pep-0351.html
Contributing thread:
- `PEP 351, the freeze protocol <
http://mail.python.org/pipermail/python-dev/2005-October/057543.html>`__
[SJB]
----------------------------------
Required superclass for Exceptions
----------------------------------
Guido and Brett Cannon introduced `PEP 352`_ which proposes that all
Exceptions be required to derive from a new exception class, BaseException.
The chidren of BaseException would be KeyboardInterrupt, SystemExit and
Exception (which would contain the remainder of the current hierarchy). The
goal here is to make the following code do the right thing::
try:
...
except Exception:
...
Currently, this code fails to catch string exceptions and other exceptions
that do not derive from Exception, and it (probably) inappropriately catches
KeyboardInterrupt and SystemExit which are supposed to indicate that Python
is shutting down. The current plan is to introduce BaseException and have
KeyboardInterrupt and SystemExit multiply inherit from Exception and
BaseException. The PEP lists the roadplan for deprecating the various other
types of exceptions.
The PEP also attempts to standardize on the arguments to Exception objects,
so that by Python 3.0, all Exceptions will support a single argument which
will be stored as their "message" attribute.
Guido was ready to accept it on October 31st, but it has not been marked as
Accepted yet.
.. _PEP 352: http://www.python.org/peps/pep-0352.html
Contributing threads:
- `PEP 352: Required Superclass for Exceptions <
http://mail.python.org/pipermail/python-dev/2005-October/057736.html>`__
- `PEP 352 Transition Plan <
http://mail.python.org/pipermail/python-dev/2005-October/057750.html>`__
[SJB]
-----------------------------------------
Generalizing the class declaration syntax
-----------------------------------------
Michele Simionato suggested a generalization of the class declaration
syntax, so that::
<callable> <name> <tuple>:
<definitions>
would be translated into::
<name> = <callable>("<name>", <tuple>, <dict-of-definitions>)
Where <dict-of-definitions> is simply the namespace that results from
executing <definitions>. This would actually remove the need for the class
keyword, as classes could be declared as::
type <classname> <bases>:
<definitions>
There were a few requests for a PEP, but nothing has been made available
yet.
Contributing thread:
- `Definining properties - a use case for class decorators? <
http://mail.python.org/pipermail/python-dev/2005-October/057435.html>`__
[SJB]
--------------------
Task-local variables
--------------------
Phillip J. Eby introduced a pre-PEP proposing a mechanism similar to
thread-local variables, to help co-routine schedulers to swap state between
tasks. Essentially, the scheduler would be required to take a snapshot of a
coroutine's variables before a swap, and restore that snapshot when the
coroutine is swapped back. Guido asked people to hold off on more PEP
343-related proposals until with-blocks have been out in the wild for at
least a release or two.
Contributing thread:
- `Pre-PEP: Task-local variables <
http://mail.python.org/pipermail/python-dev/2005-October/057464.html>`__
[SJB]
-----------------------------------------
Attribute-style access for all namespaces
-----------------------------------------
Eyal Lotem proposed replacing the globals() and locals() dicts with "module"
and "frame" objects that would have attribute-style access instead of
__getitem__-style access. Josiah Carlson noted that the first is already
available by doing ``module = __import__(__name__)``, and suggested that
monkeying around with function locals is never a good idea, so adding
additional support for doing so is not useful.
Contributing threads:
- `Early PEP draft (For Python 3000?) <
http://mail.python.org/pipermail/python-dev/2005-October/057251.html>`__
[SJB]
---------------------------------
Yielding all items of an iterator
---------------------------------
Gustavo J. A. M. Carneiro was looking for a nicer way of indicating that all
items of an iterable should be yielded. Currently, you probably want to use
a for-loop to express this, e.g.::
for step in animate(win, xrange(10)): # slide down
yield step
Andrew Koenig suggested that the syntax::
yield from <x>
be equivalent to::
for i in x:
yield i
People seemed uncertain as to whether or not there were enough use cases to
merit the additional syntax.
Contributing thread:
- `Coroutines, generators, function calling <
http://mail.python.org/pipermail/python-dev/2005-October/057405.html>`__
[SJB]
-----------------------------------------
Getting an AST without the Python runtime
-----------------------------------------
Thanks to the merging of the AST branch, Evan Jones was able to fully
divorce the Python parse from the Python runtime so that you can get AST
objects without having to have Python running. He made the divorced AST
parser available on `his site`_.
.. _his site: http://evanjones.ca/software/pyparser.html
Contributing thread:
- `Parser and Runtime: Divorced! <
http://mail.python.org/pipermail/python-dev/2005-October/057684.html>`__
[SJB]
===============
Skipped Threads
===============
- `Pythonic concurrency - offtopic <
http://mail.python.org/pipermail/python-dev/2005-October/057294.html>`__
- `Sourceforge CVS access <
http://mail.python.org/pipermail/python-dev/2005-October/057342.html>`__
- `Weekly Python Patch/Bug Summary <
http://mail.python.org/pipermail/python-dev/2005-October/057343.html>`__
- `Guido v. Python, Round 1 <
http://mail.python.org/pipermail/python-dev/2005-October/057366.html>`__
- `Autoloading? (Making Queue.Queue easier to use) <
http://mail.python.org/pipermail/python-dev/2005-October/057368.html>`__
- `problem with genexp <
http://mail.python.org/pipermail/python-dev/2005-October/057370.html>`__
- `PEP 3000 and exec <
http://mail.python.org/pipermail/python-dev/2005-October/057380.html>`__
- `Pythonic concurrency - offtopic <
http://mail.python.org/pipermail/python-dev/2005-October/057442.html>`__
- `enumerate with a start index <
http://mail.python.org/pipermail/python-dev/2005-October/057459.html>`__
- `list splicing <
http://mail.python.org/pipermail/python-dev/2005-October/057479.html>`__
- `bool(iter([])) changed between 2.3 and 2.4 <
http://mail.python.org/pipermail/python-dev/2005-October/057481.html>`__
- `A solution to the evils of static typing and interfaces? <
http://mail.python.org/pipermail/python-dev/2005-October/057485.html>`__
- `PEP 267 -- is the semantics change OK? <
http://mail.python.org/pipermail/python-dev/2005-October/057506.html>`__
- `DRAFT: python-dev Summary for 2005-09-01 through 2005-09-16 <
http://mail.python.org/pipermail/python-dev/2005-October/057508.html>`__
- `int(string) (was: DRAFT: python-dev Summary for 2005-09-01 through
2005-09-16) <
http://mail.python.org/pipermail/python-dev/2005-October/057510.html>`__
- `LXR site for Python CVS <
http://mail.python.org/pipermail/python-dev/2005-October/057511.html>`__
- `int(string) <
http://mail.python.org/pipermail/python-dev/2005-October/057512.html>`__
- `Comparing date+time w/ just time <
http://mail.python.org/pipermail/python-dev/2005-October/057514.html>`__
- `AST reverts PEP 342 implementation and IDLE starts working again <
http://mail.python.org/pipermail/python-dev/2005-October/057528.html>`__
- `cross compiling python for embedded systems <
http://mail.python.org/pipermail/python-dev/2005-October/057534.html>`__
- `Inconsistent Use of Buffer Interface in stringobject.c <
http://mail.python.org/pipermail/python-dev/2005-October/057589.html>`__
- `Reminder: PyCon 2006 submissions due in a week <
http://mail.python.org/pipermail/python-dev/2005-October/057618.html>`__
- `MinGW and libpython24.a <
http://mail.python.org/pipermail/python-dev/2005-October/057624.html>`__
- `make testall hanging on HEAD? <
http://mail.python.org/pipermail/python-dev/2005-October/057662.html>`__
- `"? operator in python" <
http://mail.python.org/pipermail/python-dev/2005-October/057673.html>`__
- `[Docs] MinGW and libpython24.a <
http://mail.python.org/pipermail/python-dev/2005-October/057693.html>`__
- `Help with inotify <
http://mail.python.org/pipermail/python-dev/2005-October/057705.html>`__
- `[Python-checkins] commit of r41352 - in python/trunk: . Lib Lib/distutils
Lib/distutils/command Lib/encodings <
http://mail.python.org/pipermail/python-dev/2005-October/057780.html>`__
- `svn:ignore <
http://mail.python.org/pipermail/python-dev/2005-October/057783.html>`__
- `svn checksum error <
http://mail.python.org/pipermail/python-dev/2005-October/057790.html>`__
- `svn:ignore (Was: [Python-checkins] commit of r41352 - in python/trunk: .
Lib Lib/distutils Lib/distutils/command Lib/encodings) <
http://mail.python.org/pipermail/python-dev/2005-October/057793.html>`__
- `StreamHandler eating exceptions <
http://mail.python.org/pipermail/python-dev/2005-October/057798.html>`__
- `a different kind of reduce... <
http://mail.python.org/pipermail/python-dev/2005-October/057814.html>`__
========
Epilogue
========
This is a summary of traffic on the `python-dev mailing list`_ from
October 16, 2005 through October 31, 2005.
It is intended to inform the wider Python community of on-going
developments on the list on a semi-monthly basis. An archive_ of
previous summaries is available online.
An `RSS feed`_ of the titles of the summaries is available.
You can also watch comp.lang.python or comp.lang.python.announce for
new summaries (or through their email gateways of python-list or
python-announce, respectively, as found at http://mail.python.org).
This is the 6th summary written by the python-dev summary dyad of
Steve Bethard and Tony Meyer (more thanks!).
To contact us, please send email:
- Steve Bethard (steven.bethard at gmail.com <http://gmail.com>)
- Tony Meyer (tony.meyer at gmail.com <http://gmail.com>)
Do *not* post to comp.lang.python if you wish to reach us.
The `Python Software Foundation`_ is the non-profit organization that
holds the intellectual property for Python. It also tries to advance
the development and use of Python. If you find the python-dev Summary
helpful please consider making a donation. You can make a donation at
http://python.org/psf/donations.html . Every penny helps so even a
small donation with a credit card, check, or by PayPal helps.
--------------------
Commenting on Topics
--------------------
To comment on anything mentioned here, just post to
`comp.lang.python`_ (or email python-list(a)python.org which is a
gateway to the newsgroup) with a subject line mentioning what you are
discussing. All python-dev members are interested in seeing ideas
discussed by the community, so don't hesitate to take a stance on
something. And if all of this really interests you then get involved
and join `python-dev`_!
-------------------------
How to Read the Summaries
-------------------------
The in-development version of the documentation for Python can be
found at http://www.python.org/dev/doc/devel/ and should be used when
looking up any documentation for new code; otherwise use the current
documentation as found at http://docs.python.org/ . PEPs (Python
Enhancement Proposals) are located at http://www.python.org/peps/ .
To view files in the Python CVS online, go to
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . Reported
bugs and suggested patches can be found at the SourceForge_ project
page.
Please note that this summary is written using reStructuredText_.
Any unfamiliar punctuation is probably markup for reST_ (otherwise it
is probably regular expression syntax or a typo =); you can safely
ignore it. We do suggest learning reST, though; it's simple and is
accepted for `PEP markup`_ and can be turned into many different
formats like HTML and LaTeX. Unfortunately, even though reST is
standardized, the wonders of programs that like to reformat text do
not allow us to guarantee you will be able to run the text version of
this summary through Docutils_ as-is unless it is from the
`original text file`_.
.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list:
http://mail.python.org/mailman/listinfo/python-dev
.. _c.l.py:
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _PEP Markup: http://www.python.org/peps/pep-0012.html
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html
.. _PSF:
.. _Python Software Foundation: http://python.org/psf/
.. _last summary:
http://www.python.org/dev/summary/2005-09-16_2005-09-30.html
.. _original text file:
http://www.python.org/dev/summary/2005-10-16_2005-10-31.ht
.. _archive: http://www.python.org/dev/summary/
.. _RSS feed: http://www.python.org/dev/summary/channews.rdf
Title: python-dev Summary for 2005-10-01 through 2005-10-15
Content-type: text/x-rst
Encoding: utf-8
python-dev Summary for 2005-10-01 through 2005-10-15
++++++++++++++++++++++++++++++++++++++++++++++++++++
.. contents::
[The HTML version of this Summary is available at
http://www.python.org/dev/summary/2005-10-01_2005-10-15.html]
=============
Announcements
=============
----------------------------
QOTF: Quote of the Fortnight
----------------------------
>From Phillip J. Eby:
So, if threads are "easy" in Python compared to other
langauges, it's *because of* the GIL, not in spite of it.
Contributing thread:
- `Pythonic concurrency <
http://mail.python.org/pipermail/python-dev/2005-October/057062.html>`__
[SJB]
----------------------------------------
GCC/G++ Issues on Linux: Patch available
----------------------------------------
Christoph Ludwig provided the previously `promised patch`_ to address some
of the issues in compiling Python with GCC/G++ on Linux. The patch_ keeps
ELF systems like x86 / Linux from having any dependencies on the C++
runtime, and allows systems that require main() to be a C++ function to be
configured appropriately.
.. _promised patch:
http://www.python.org/dev/summary/2005-07-01_2005-07-15.html#gcc-g-issues-o…
.. _patch: http://python.org/sf/1324762
Contributing thread:
- `[C++-sig] GCC version compatibility <
http://mail.python.org/pipermail/python-dev/2005-October/057230.html>`__
[SJB]
=========
Summaries
=========
---------------------
Concurrency in Python
---------------------
Michael Sparks spent a bit of time descibing the current state and future
goals of the Kamaelia_ project. Mainly, Kamaelia aims to make concurrency as
simple and easy to use as possible. A scheduler manages a set of generators
that communicate with each other through Queues. The long term goals include
being able to farm the various generators off into thread or processes as
needed, so that whether your concurrency model is cooperative, threaded or
process-based, your code can basically look the same.
There was also continued discussion about how "easy" threads are. Shane
Hathaway made the point that it's actually locking that's "insanely
difficult", and approaches that simplify how much you need to think about
locking can keep threading relatively easy -- this was one of the strong
points of ZODB. A fairly large camp also got behind the claim that threads
are easy if you're limited to only message passing. There were also a few
comments about how Python makes threading easier, e.g. through the GIL (see
`QOTF: Quote of the Fortnight`_) and through threading.threads's
encapsulation of thread-local resources as instance attributes.
.. _Kamaelia: http://kamaelia.sourceforge.ne
Contributing threads:
- `Pythonic concurrency - cooperative MT <
http://mail.python.org/pipermail/python-dev/2005-October/056898.html>`__
- `Pythonic concurrency <
http://mail.python.org/pipermail/python-dev/2005-October/057023.html>`__
[SJB]
-------------------------------------
Organization of modules for threading
-------------------------------------
A few people took issue with the current organization of the threading
modules into Queue, thread and threading. Guido views Queue as an
application of threading, so putting it in the threading module is
inappropriate (though with a deeper package structure, it should definitely
be a sibling). Nick Coghlan suggested that Queue should be in a threadtools
module (in parallel with itertools), while Skip proposed a hierarchy of
modules with thread and lock being in the lowest level one, and Thread and
Queue being in the highest level. Aahz suggested (and Guido approved)
deprecating the thread module and renaming it to _thread at least in Python
3.0. It seems the deprecation may happen sooner though.
Contributing threads:
- `Making Queue.Queue easier to use <
http://mail.python.org/pipermail/python-dev/2005-October/057184.html>`__
- `Autoloading? (Making Queue.Queue easier to use) <
http://mail.python.org/pipermail/python-dev/2005-October/057216.html>`__
- `threadtools (was Re: Autoloading? (Making Queue.Queue easier to use)) <
http://mail.python.org/pipermail/python-dev/2005-October/057262.html>`__
- `Threading and synchronization primitives <
http://mail.python.org/pipermail/python-dev/2005-October/057269.html>`__
[SJB]
-------------------------
Speed of Unicode decoding
-------------------------
Tony Nelson found that decoding with a codec like mac-roman or iso8859-1 can
take around ten times as long as decoding with utf-8. Walter Dörwald
provided a patch_ that implements the mapping using a unicode string of
length 256 where undefined characters are mapped to u"\ufffd". This dropped
the decode time for mac-roman to nearly the speed of the utf-8 decoding.
Hye-Shik Chang showed off a fastmap decoder with comparable performance. In
the end, Walter's patch was accepted.
.. patch: http://www.python.org/sf/1313939
Contributing thread:
- `Unicode charmap decoders slow <
http://mail.python.org/pipermail/python-dev/2005-October/056958.html>`__
[SJB]
------------------
Updates to PEP 343
------------------
Jason Orendorff proposed replacing the __enter__() and __exit__() methods on
context managers with a simple __with__() method instead. While Guido was
unconvinced that __enter__() and __exit__() should be dropped, he was
convinced that context managers should have a __with__() method in parallel
with the __iter__() method for iterators. There was some talk of
special-casing the @contextmanager decorator on the __with__() method, but
no conclusion.
Contributing threads:
- `Proposed changes to PEP 343 <
http://mail.python.org/pipermail/python-dev/2005-October/057040.html>`__
- `PEP 343 and __with__ <
http://mail.python.org/pipermail/python-dev/2005-October/056931.html>`__
[SJB]
----------------------
str and unicode issues
----------------------
Martin Blais wanted to completely disable the implicit conversions between
unicode and str, so that you would always be forced to call either .encode()
or .decode() to convert between one and the other. This is already available
through adding ``sys.setdefaultencoding('undefined')`` to your
sitecustomize.py file, but the suggestion started another long discussion
over unicode issues. Antoine Pitrou suggested that a good rule of thumb is
to convert to unicode everything that is semantically textual, and to only
use str for what is to be semantically treated as a string of bytes. Fredrik
Lundh argued against this for efficiency reasons -- pure ASCII text would
consume more space as a unicode object.
There were suggestions that in Python 3.0, opening files in text mode will
require an encoding and produce string objects, while opening files in
binary mode will produce bytes objects. The bytes() type will be a mutable
array of bytes, which can be converted to a string object by specifying an
encoding.
Contributing threads:
- `Divorcing str and unicode (no more implicit conversions). <
http://mail.python.org/pipermail/python-dev/2005-October/056916.html>`__
- `unifying str and unicode <
http://mail.python.org/pipermail/python-dev/2005-October/056934.html>`__
- `bytes type <
http://mail.python.org/pipermail/python-dev/2005-October/056945.html>`__
[SJB]
----------------------------------------------------------------------
Allowing \*args syntax in tuple unpacking and before keyword arguments
----------------------------------------------------------------------
Gustavo Niemeyer propsed the oft-seen request for allowing the \*args syntax
in tuple unpacking, e.g.::
for first, second, *rest in iterator:
Guido requested a PEP, saying that he wasn't convinced that there was much
of a gain over the already valid::
for item in iterator:
(first, second), rest = item[2:], item[:2]
Greg Ewing and others didn't like Guido's suggestion as it violates DRY
(Don't Repeat Yourself). Others also chimed in with some examples in support
of the proposal, but no one has yet put together a PEP.
In a related matter, Guido indicated that he wants to be able to write
keyword-only arguments after a \*args, so that you could, for example,
write::
f(a, b, *args, foo=1, bar=2, **kwds)
People seemed almost unanimously in support of this proposal, but, to quote
Nick Coghlan, it has still "never bugged anyone enough for them to actaully
get around to fixing it".
Contributing thread:
- `Extending tuple unpacking <
http://mail.python.org/pipermail/python-dev/2005-October/057056.html>`__
[SJB]
----------
AST Branch
----------
Guido gave the AST branch a three week ultimatum: either the branch should
be merged into MAIN within the next three weeks, or the branch should be
abandoned entirely. This jump-started work on the branch, and the team was
hoping to merge the changes the weekend of October 15th.
Contributing threads:
- `Python 2.5a1, ast-branch and PEP 342 and 343 <
http://mail.python.org/pipermail/python-dev/2005-September/056449.html>`__
- `Python 2.5 and ast-branch <
http://mail.python.org/pipermail/python-dev/2005-October/056986.html>`__
- `AST branch update <
http://mail.python.org/pipermail/python-dev/2005-October/057281.html>`__
[SJB]
-----------------------------------
Allowing "return obj" in generators
-----------------------------------
Piet Delport suggested having ``return obj`` in generators be translated
into ``raise StopIteration(obj)``. The return value of a generator function
would thus be available as the first arg in the StopIteration exception.
Guido asked for some examples to give the idea a better motivation, and felt
uncomfortable with the return value being silently ignored in for-loops. The
idea was postponed until at least one release after a PEP 342 implementation
enters Python, so that people can have some more experience with coroutines.
Contributing threads:
- `Proposal for 2.5: Returning values from PEP 342 enhanced generators <
http://mail.python.org/pipermail/python-dev/2005-October/056957.html>`__
- `PEP 342 suggestion: start(), __call__() and unwind_call() methods <
http://mail.python.org/pipermail/python-dev/2005-October/057042.html>`__
- `New PEP 342 suggestion: result() and allow "return with
arguments" in generators (was Re: PEP 342 suggestion: start(),
__call__() and unwind_call() methods) <
http://mail.python.org/pipermail/python-dev/2005-October/057116.html>`__
[SJB]
-----------------------------
API for the line-number table
-----------------------------
Greg Ewing suggested trying to simplify the line-number table (lnotab) by
simply matching each byte-code index with a file and line number. Phillip J.
Eby pointed out that this would make the stdlib take up an extra megabyte,
suggesting two tables instead, one matching bytecodes to line numbers, and
one matching the first line-number of a chunk with its file. Michael Hudson
suggested that what we really want is an API for accessing the lnotab, so
that the implementation that is chosen is less important. The conversation
trailed off without a resolution.
Contributing thread:
- `Simplify lnotab? (AST branch update) <
http://mail.python.org/pipermail/python-dev/2005-October/057285.html>`__
[SJB]
------------------------------
Current directory and sys.path
------------------------------
A question about the status of `the CurrentVersion registry entry`_ led to a
discussion about the different behaviors of sys.path across platforms.
Apparently, on Windows, sys.path includes the current directory and the
directory of the script being executed, while on Linux, it only includes the
directory of the script.
.. _the CurrentVersion registry entry:
http://www.python.org/windows/python/registry.html
Contributing thread:
- `PythonCore\CurrentVersion <
http://mail.python.org/pipermail/python-dev/2005-October/057095.html>`__
[SJB]
----------------------------------
Changing the __class__ of builtins
----------------------------------
As of Python 2.3, you can no longer change the __class__ of any builtin.
Phillip J. Eby suggested that these rules might be overly strict; modules
and other mutable objects could probably reasonably have their __class__s
changed. No one seemed really opposed to the idea, but no one offered up a
patch to make the change either.
Contributing thread:
- `Assignment to __class__ of module? (Autoloading? (Making
Queue.Queueeasier to use)) <
http://mail.python.org/pipermail/python-dev/2005-October/057253.html>`__
[SJB]
------------------------------------------
exec function specification for Python 3.0
------------------------------------------
In Python 3.0, exec is slated to become a function (instead of a statement).
Currently, the presence of an exec statement in a function can cause some
subtle changes since Python has to worry about exec modifying function
locals. Guido suggested that the exec() function could require a namespace,
basically dumping the exec-in-local-namespace altogether. People seemed
generally in favor of the proposal, though no official specification was
established.
Contributing thread:
- `PEP 3000 and exec <
http://mail.python.org/pipermail/python-dev/2005-October/057135.html>`__
[SJB]
------------------------------------
Adding opcodes to speed up self.attr
------------------------------------
Phillip J. Eby experimented with adding LOAD_SELF and SELF_ATTR opcodes to
improve the speed of object-oriented programming. This gained about a 5%
improvement in pystone, which isn't organized in a very OO manner. People
seemed uncertain as to whether paying the cost of adding two opcodes to gain
a 5% speedup was worth it. No decision had been made at the time of this
summary.
Contributing thread:
- `LOAD_SELF and SELF_ATTR opcodes <
http://mail.python.org/pipermail/python-dev/2005-October/057321.html>`__
[SJB]
--------------------------------------
Dropping support for --disable-unicode
--------------------------------------
Reinhold Birkenfeld tried unsuccessfully to make the test-suite pass with
--disable-unicode set. M.-A. Lemburg suggested that the feature should be
ripped out entirely, to simplify the code. Martin v. Löwis suggested
deprecating it to give people a chance to object. The plan is now to add a
note to the configure switch that the feature will be removed in Python 2.6.
Contributing threads:
- `Tests and unicode <
http://mail.python.org/pipermail/python-dev/2005-October/056897.html>`__
- `--disable-unicode (Tests and unicode) <
http://mail.python.org/pipermail/python-dev/2005-October/056920.html>`__
[SJB]
-----------------------------------------
Bug in __getitem__ inheritance at C level
-----------------------------------------
Travis Oliphant discovered that the addition of the mp_item and sq_item
descriptors and the resolution of any comptetion for __getitem__ calls is
done *before* the inheritance of any slots takes place. This means that if
you create a type in C that supports the sequence protocol, and tries to
inherit the mapping protocol from a parent C type which does not support the
sequence protocol, __getitem__ will point to the parent type's __getitem__
instead of the child type's __getitem__. This seemed like more of a bug than
a feature, so the behavior may be changed in future Pythons.
Contributing thread:
- `Why does __getitem__ slot of builtin call sequence methods first? <
http://mail.python.org/pipermail/python-dev/2005-October/056901.html>`__
[SJB]
================
Deferred Threads
================
- `Early PEP draft (For Python 3000?) <
http://mail.python.org/pipermail/python-dev/2005-October/057251.html>`__
- `Pythonic concurrency - offtopic <
http://mail.python.org/pipermail/python-dev/2005-October/057294.html>`__
===============
Skipped Threads
===============
- `PEP 350: Codetags <
http://mail.python.org/pipermail/python-dev/2005-October/056894.html>`__
- `Active Objects in Python <
http://mail.python.org/pipermail/python-dev/2005-October/056896.html>`__
- `IDLE development <
http://mail.python.org/pipermail/python-dev/2005-October/056907.html>`__
- `Help needed with MSI permissions <
http://mail.python.org/pipermail/python-dev/2005-October/056908.html>`__
- `C API doc fix <
http://mail.python.org/pipermail/python-dev/2005-October/056910.html>`__
- `Static builds on Windows (continued) <
http://mail.python.org/pipermail/python-dev/2005-October/056976.html>`__
- `Removing the block stack (was Re: PEP 343 and __with__) <
http://mail.python.org/pipermail/python-dev/2005-October/057001.html>`__
- `Removing the block stack <
http://mail.python.org/pipermail/python-dev/2005-October/057008.html>`__
- `Lexical analysis and NEWLINE tokens <
http://mail.python.org/pipermail/python-dev/2005-October/057014.html>`__
- `PyObject_Init documentation <
http://mail.python.org/pipermail/python-dev/2005-October/057039.html>`__
- `Sourceforge CVS access <
http://mail.python.org/pipermail/python-dev/2005-October/057051.html>`__
- `__doc__ behavior in class definitions <
http://mail.python.org/pipermail/python-dev/2005-October/057066.html>`__
- `Sandboxed Threads in Python <
http://mail.python.org/pipermail/python-dev/2005-October/057082.html>`__
- `Weekly Python Patch/Bug Summary <
http://mail.python.org/pipermail/python-dev/2005-October/057092.html>`__
- `test_cmd_line failure on Kubuntu 5.10 with GCC 4.0 <
http://mail.python.org/pipermail/python-dev/2005-October/057094.html>`__
- `defaultproperty (was: Re: RFC: readproperty) <
http://mail.python.org/pipermail/python-dev/2005-October/057120.html>`__
- `async IO and helper threads <
http://mail.python.org/pipermail/python-dev/2005-October/057121.html>`__
- `defaultproperty <
http://mail.python.org/pipermail/python-dev/2005-October/057129.html>`__
- `Fwd: defaultproperty <
http://mail.python.org/pipermail/python-dev/2005-October/057131.html>`__
- `C.E.R. Thoughts <
http://mail.python.org/pipermail/python-dev/2005-October/057137.html>`__
- `problem with genexp <
http://mail.python.org/pipermail/python-dev/2005-October/057175.html>`__
- `Python-Dev Digest, Vol 27, Issue 44 <
http://mail.python.org/pipermail/python-dev/2005-October/057207.html>`__
- `Europeans attention please! <
http://mail.python.org/pipermail/python-dev/2005-October/057233.html>`__
========
Epilogue
========
This is a summary of traffic on the `python-dev mailing list`_ from
October 01, 2005 through October 15, 2005.
It is intended to inform the wider Python community of on-going
developments on the list on a semi-monthly basis. An archive_ of
previous summaries is available online.
An `RSS feed`_ of the titles of the summaries is available.
You can also watch comp.lang.python or comp.lang.python.announce for
new summaries (or through their email gateways of python-list or
python-announce, respectively, as found at http://mail.python.org).
This is the 5th summary written by the python-dev summary taskforce of
Steve Bethard and Tony Meyer (thanks Steve!).
To contact us, please send email:
- Steve Bethard (steven.bethard at gmail.com <http://gmail.com>)
- Tony Meyer (tony.meyer at gmail.com <http://gmail.com>)
Do *not* post to comp.lang.python if you wish to reach us.
The `Python Software Foundation`_ is the non-profit organization that
holds the intellectual property for Python. It also tries to advance
the development and use of Python. If you find the python-dev Summary
helpful please consider making a donation. You can make a donation at
http://python.org/psf/donations.html . Every penny helps so even a
small donation with a credit card, check, or by PayPal helps.
--------------------
Commenting on Topics
--------------------
To comment on anything mentioned here, just post to
`comp.lang.python`_ (or email python-list(a)python.org which is a
gateway to the newsgroup) with a subject line mentioning what you are
discussing. All python-dev members are interested in seeing ideas
discussed by the community, so don't hesitate to take a stance on
something. And if all of this really interests you then get involved
and join `python-dev`_!
-------------------------
How to Read the Summaries
-------------------------
The in-development version of the documentation for Python can be
found at http://www.python.org/dev/doc/devel/ and should be used when
looking up any documentation for new code; otherwise use the current
documentation as found at http://docs.python.org/ . PEPs (Python
Enhancement Proposals) are located at http://www.python.org/peps/ .
To view files in the Python CVS online, go to
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . Reported
bugs and suggested patches can be found at the SourceForge_ project
page.
Please note that this summary is written using reStructuredText_.
Any unfamiliar punctuation is probably markup for reST_ (otherwise it
is probably regular expression syntax or a typo =); you can safely
ignore it. We do suggest learning reST, though; it's simple and is
accepted for `PEP markup`_ and can be turned into many different
formats like HTML and LaTeX. Unfortunately, even though reST is
standardized, the wonders of programs that like to reformat text do
not allow us to guarantee you will be able to run the text version of
this summary through Docutils_ as-is unless it is from the
`original text file`_.
.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list:
http://mail.python.org/mailman/listinfo/python-dev
.. _c.l.py:
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _PEP Markup: http://www.python.org/peps/pep-0012.html
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html
.. _PSF:
.. _Python Software Foundation: http://python.org/psf/
.. _last summary:
http://www.python.org/dev/summary/2005-09-01_2005-09-15.html
.. _original text file:
http://www.python.org/dev/summary/2005-10-01_2005-10-15.ht
.. _archive: http://www.python.org/dev/summary/
.. _RSS feed: http://www.python.org/dev/summary/channews.rdf
[The HTML version of this Summary will be available at
http://www.python.org/dev/summary/2005-09-16_2005-09-30.html]
=============
Announcements
=============
-----------------------------
QOTF: Quotes of the fortnight
-----------------------------
We have two quotes this week, one each from the two biggest threads of this
fortnight: concurrency and conditional expressions. The first quote, from
Donovan Barda, puts Python's approach to threading into perspective:
The reality is threads were invented as a low overhead way of easily
implementing concurrent applications... ON A SINGLE PROCESSOR.
Taking into account threading's limitations and objectives, Python's
GIL is the best way to support threads. When hardware (seriously)
moves to multiple processors, other concurrency models will start to
shine.
Our second QOTF, by yours truly (hey, who could refuse a nomination from
Guido?), is a not-so-subtle reminder to leave syntax decisions to Guido:
Please no more syntax proposals! ... We need to leave the syntax to
Guido. We've already proved that ... we can't as a community agree
on a syntax. That's what we have a BDFL for. =)
Contributing threads:
- `GIL, Python 3, and MP vs. UP <
http://mail.python.org/pipermail/python-dev/2005-September/056609.html>`__
- `Adding a conditional expression in Py3.0 <
http://mail.python.org/pipermail/python-dev/2005-September/056617.html>`__
[SJB]
-------------------
Compressed MSI file
-------------------
Martin v. Löwis discovered that a little more than a `MiB`_ could be saved
in the size of Python installer by using LZX:21 instead of the standard
MSZIP when compressing the CAB file. After confirmation from several testers
that the new format worked, the change (for Python 2.4.2 and beyond) was
made.
.. _MiB: http://en.wikipedia.org/wiki/Mibibyte
Contributing thread:
- `Compressing MSI files: 2.4.2 candidate? <
http://mail.python.org/pipermail/python-dev/2005-September/056694.html>`__
[TAM]
=========
Summaries
=========
-----------------------
Conditional expressions
-----------------------
Raymond Hettinger proposed that the ``and`` and ``or`` operators be modified
in Python 3.0 to produce only booleans instead of producing objects,
motivating this proposal in part by the common (mis-)use of ``<cond> and
<true-expr> or <false-expr>`` to emulate a conditional expression. In
response, Guido suggested that that the conditional expression discussion of
`PEP 308`_ be reopened. This time around, people seemed almost unanimously
in support of adding a conditional expression, though as before they
disagreed on syntax. Fortunately, this time Guido cut the discussion short
and pronounced a new syntax: ``<true-expr> if <cond> else <false-expr>``.
Although it has not been implemented yet, the plan is for it to appear in
Python 2.5.
.. _PEP 308: http://www.python.org/peps/pep-0308.html
Contributing threads:
- `"and" and "or" operators in Py3.0 <
http://mail.python.org/pipermail/python-dev/2005-September/056510.html>`__
- `Adding a conditional expression in Py3.0 <
http://mail.python.org/pipermail/python-dev/2005-September/056546.html>`__
- `Conditional Expression Resolution <
http://mail.python.org/pipermail/python-dev/2005-September/056846.html>`__
[SJB]
---------------------
Concurrency in Python
---------------------
Once again, the subject of removing the global interpreter lock (GIL) came
up. Sokolov Yura suggested that the GIL be replaced with a system where
there are thread-local GILs that cooperate to share writing; Martin v. Löwis
suggested that he try to implement his ideas, and predicted that he would
find that doing so would be a lot of work, would require changes to all
extension modules (likely to introduce new bugs, particularly race
conditions), and possibly decrease performance. This kicked off several long
threads about multi-processor coding.
A long time ago (circa Python 1.4), Greg Stein experimented with free
threading, which did yield around a 1.6 times speedup on a dual-processor
machine. To avoid the overhead of multi-processor locking on a uniprocessor
machine, a separate binary could be distributed. Some of the code apparently
did make it into Python 1.5, but the issue died off because no-one provided
working code, or a strategy for what to do with existing extension modules.
Guido pointed out that it is not clear at this time how multiple processors
will be used as they become the norm. With the threaded programming model (
e.g. in Java) there are problems with concurrent modification errors
(without locking) or deadlocks and livelocks (with locking). Guido's hunch
(and mine, FWIW) is that instead of writing massively parallel applications,
we will continue to write single-threaded applications that are tied
together at the process level rather than at the thread level. He also
pointed out that it's likely that most problems get little benefit out of
multiple processors.
Guido threw down the gauntlet: rather than the endless discussion about this
topic, someone should come up with a GIL-free Python (not necessarily
CPython) and demonstrate its worth. Phillip J. Eby reminded everyone that
Jython, IronPython, and PyPy exist, and that someone could, for example,
create a multiprocessor-friendly backend for PyPy.
Guido also pointed out that fast threading benefits from fast context
switches, which benefits from small register sets, and that the current
trend in chips is towards larger register sets. In addition, multiple
processors with shared memory don't scale all that well (multiple processors
with explicit interprocess communication (IPC) channels scale much better).
These all favour multi-processing over multi-threading. Donovan Baarda went
so far as to say (a QOTF, as above), that Python's GIL is the best way to
support threads, which are for single-processor use, and that when
multiple-processor platforms have matured more other concurrency models will
likewise mature. OTOH, Bob Ippolito pointed out that (in many operating
systems) there isn't a lot of difference between threads and processes, and
that threads can typically still use IPC. Bob argued that the biggest
argument for threading is that lots of existing C/C++ code uses threads.
Simon Percivall argued that the problem is that Python offers ("out of the
box") some support for multi-threaded programming, but little for
multi-process programming beyond the basics (e.g. data sharing,
communication, control over running processes, dealing out tasks to be
handled). Simon suggested that the best way to stop people complaining about
the GIL is to provide solid, standardized support for multi-process
programming. The idea of a "multiprocess" module gained a reasonable amount
of support.
Phillip J. Eby outlined an idea he is considering PEPifying, in which one
could switch all context variables (such as the Decimal context and the
sys.* variables) simulaneously and instantaneously when changing execution
contexts (like switching between coroutines). He has a prototype
implementation of the basic idea, which is less than 200 lines of Python and
very fast. However, he pointed out that it's not completely PEP-ready at
this point, and he needs to continue considering various parts of the
concept.
Bruce Eckel joined the thread, and suggested that low-level threads people
are only now catching up to objects, but as far as concurrency goes their
brains still think in terms of threads, so they naturally apply thread
concepts to objects. He believes that pthread-style thinking is two steps
backwards: you effectively throw open the innards of the object that you
just spent time decoupling from the rest of your system, and the coupling is
not predictable.
Bruce and Guido had discussed offlist "active objects": defining a class as
"active" would install a worker thread and concurrent queue in each object
of that class, automatically turn method calls into tasks and enqueue them,
and prevent any other interaction other than enqueued messages. Guido felt
that if multiple active objects could co-exist in the same process, but be
prevented (by the language implementation) from sharing data except via
channels, and dynamic reallocation of active objects across multiple CPUs
were possible, then this might be a solution. He pointed out that an
implementation would really be needed to prove this.
Phillip and Martin pointed out that preventing any other interaction other
than enqueued messages is the difficult part; each active object would, for
example, have to have its own sys.modules. Phillip felt that such a solution
(which Bruce posed as "a" solution, not "the" solution) wouldn't help with
GIL removal, but would help with effective use of multiprocessor machines on
platforms where fork() is available, if the API works across processes as
well as threads.
Bruce then restarted the discussion, putting forth eight criteria that he
felt would be necessary for the "pythonic" solution to concurrency. Items on
the list were discussed further, with some disagreement about what was
possible. The concurrency discussion continues next month...
Contributing threads:
- `Variant of removing GIL. <
http://mail.python.org/pipermail/python-dev/2005-September/056423.html>`__
- `GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) <
http://mail.python.org/pipermail/python-dev/2005-September/056458.html>`__
- `GIL, Python 3, and MP vs. UP <
http://mail.python.org/pipermail/python-dev/2005-September/056498.html>`__
- `Active Objects in Python <
http://mail.python.org/pipermail/python-dev/2005-September/056752.html>`__
- `Pythonic concurrency <
http://mail.python.org/pipermail/python-dev/2005-September/056801.html>`__
- `Pythonic concurrency - cooperative MT <
http://mail.python.org/pipermail/python-dev/2005-September/056860.html>`__
[TAM]
-----------------------------------
Removing nested function parameters
-----------------------------------
Brett Cannon proposed removing support for nested function parameters so
that instead of being able to write::
def f((x, y)):
print x, y
you'd have to write something like::
def f(arg):
x, y = arg
print x, y
Brett (with help from Guido) motivated this removal (for Python 3.0) by a
few factors:
(1) The feature has low visibility: "For every user who is fond of them
there are probably ten who have never even heard of it." - Guido
(2) The feature can be difficult to read for some people.
(3) The feature doesn't add any power to the language; the above functions
emit essentially the same byte-code.
(4) The feature makes function parameter introspection difficult because
tuple unpacking information is not stored in the function object.
In general, people were undecided on this proposal. While a number of people
said they used the feature and would miss it, many of them also said that
their code wouldn't suffer that much if the feature was removed. No decision
had been made at the time of the summary.
Contributing thread:
- `removing nested tuple function parameters <
http://mail.python.org/pipermail/python-dev/2005-September/056459.html>`__
[SJB]
-----------------------------------------
Evaluating iterators in a boolean context
-----------------------------------------
In Python 2.4 some builtin iterators gained __len__ methods when the number
of remaining items could be made available. This broke some of Guido's code
that tested iterators for their boolean value (to distinguish them from
None). Raymond Hettinger (who supplied the original patch) argued that
`testing for None`_ using boolean tests was in general a bad idea, and that
knowing the length of an iterator, when possible, had a number of use cases
and allowed for some performance gains. However, Guido felt strongly that
iterators should not supply __len__ methods, as this would lead to some
people writing code expecting this method, which would then break when it
received an iterator which could not determine its own length. The feature
will be rolled back in Python 2.5, and Raymond will likely move the __len__
methods to private methods in order to maintain the performance gains.
.. _testing for None:
http://www.python.org/peps/pep-0290.html#testing-for-none
Contributing threads:
- `bool(iter([])) changed between 2.3 and 2.4 <
http://mail.python.org/pipermail/python-dev/2005-September/056576.html>`__
- `bool(container) [was bool(iter([])) changed between 2.3 and 2.4] <
http://mail.python.org/pipermail/python-dev/2005-September/056879.html>`__
[SJB]
--------------------------------------------------
Properties that only call the getter function once
--------------------------------------------------
Jim Fulton proposed adding a new builtin for a property-like descriptor that
would only call the getter method once, so that something like::
class Spam(object):
@readproperty
def eggs(self):
... expensive computation of eggs
self.eggs = result
return result
would only do the eggs computation once. Currently, you can't do this with a
property() because the ``self.eggs = result`` statement tries to call the
property's ``fset`` method instead of replacing the property with the result
of the eggs() call. A few other people commented that they'd needed similar
functionality at times, and Guido seemed moderately interested in the idea,
but there was no final resolution.
Contributing thread:
- `RFC: readproperty <
http://mail.python.org/pipermail/python-dev/2005-September/056769.html>`__
[SJB]
--------
Codetags
--------
Micah Elliott submitted his `Codetags PEP 350`_ (after revisions following
the comp.lang.python discussion) to python-dev for comment. A common feeling
was that this (particularly synonyms) was over-engineering; Guido pointed
out that he only uses XXX, and this is certainly the most common (although
not only) example in the Python source itself. Some suggestions were made,
many of which Micah integrated into the PEP.
The suggestion was made that an implementation should precede approval of
the PEP. Micah indicated that he would continue development on the tools,
and that he encourages anyone interested in using a standard set of
codetages to give these a try.
.. _Codetags PEP 350: http://python.org/peps/pep-0350.html
- `PEP 350: Codetags <
http://mail.python.org/pipermail/python-dev/2005-September/056744.html>`__
[TAM]
----------------------------
Improving set implementation
----------------------------
Raymond Hettinger suggested a "small, but interesting, C project" to
determine whether the setobject.c implementation would be improved by
recoding the set_lookkey() function to optimize key insertion order using
Brent's variation of Algorithm D (c.f. Knuth vol. III, section 6.4, p525).
It has the potential to boost performance for uniquification applications
with duplicate keys being identified more quickly, and possibly also more
frequent retirement of dummy entires during insertion operations.
Andrew Durdin pointed out that Brent's variation depends on the next probe
position for a key being derivated from the key and its current position,
which is incompatible with the current perturbation system; Raymond replaced
perturbation with a secondary hash with linear probing. Antoine Pitrou did
some `experimenting with this`_, resulting in a -5% to 2% speedup with
various benchmarks.
Raymond has also been experimenting with a simpler approach: whenever there
are more than three probes, always swap the new key into the first position
and then unconditionally re-insert the swapped-out key. He reported that,
most of the time, this gives an improvement, and it doesn't require changing
the perturbation logic. This simpler approach is cheap to implement, but the
benefits are also smaller, with it improving only the worse collisions.
.. _experimenting with this: http://pitrou.net/python/sets
- `C coding experiment <
http://mail.python.org/pipermail/python-dev/2005-September/055965.html>`__
[TAM]
--------------
Relative paths
--------------
Nathan Bullock suggested a ''relpath(path_a, path_b)'' addition to
os.paththat returns a relative path from path_a to path_b. Trent Mick
pointed out
that there are a `couple of`_ `recipes for this`_, as well as `Jason
Orendorff's Path module`_. Several people supported this idea, and hopefully
either Nathan or one of the recipe authors will submit a patch with this
functionality.
.. _couple of:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302594
.. _recipes for this:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/208993
.. _Jason Orendorff's Path module:
http://www.jorendorff.com/articles/python/path/
Contributing threads:
- `os.path.diff(path1, path2) <
http://mail.python.org/pipermail/python-dev/2005-September/056391.html>`__
- `os.path.diff(path1, path2) (and a first post) <
http://mail.python.org/pipermail/python-dev/2005-September/056703.html>`__
[TAM]
----------------------------------
Adding a vendor-packages directory
----------------------------------
Rich Burridge followed up a `comp.lang.python thread`_ about a
"vendor-packages" directory for Python by submitting a `patch`_ and asking
for comments about the proposal on python-dev. General consensus was that
the proposal needed a better rationale, explaining why this improved on
simply adding a .pth file to the site-packages directory.
Rich explained that the rationale is that Python files supplied by the
vendor (Sun, Apple, RedHat, Microsoft) with their operating system software
should go in a separate base directory to differentiate them from Python
files installed specifically at the site. However, Bob Ippolito pointed out
that, as of OS X 10.4 ("Tiger") Apple already does this via a .pth file ("
Extras.pth"), which points to
''/System/Library/Frameworks/Python.framework/Versions/2.3/Extras/lib/python''
and includes wxPython by default.
Bob also pointed out that such a "vendor-packages.pth" should look like
''import site; site.addsitedir('/usr/lib/python2.4/vendor-packages')'' so
that packages like Numeric, PIL, and PyObjC, which take advantage of .pth
files themselves, work when installed to the vendor-packages location.
Phillip J. Eby pointed out that it would be good to have a document for
"Python Distributors" that explained these kind of things, and suggested
that perhaps a volunteer or two could be found within the distutils-SIG to
do this.
.. _comp.lang.python thread:
http://mail.python.org/pipermail/python-list/2005-September/300029.html
.. _patch:
http://sourceforge.net/tracker/index.php?func=detail&aid=1298835&group_id=5…
Contributing thread:
- `vendor-packages directory <
http://mail.python.org/pipermail/python-dev/2005-September/056682.html>`__
[TAM]
=======================
Version numbers on OS X
=======================
Guido asked if platform.system_alias() could be improved on OS X by mapping
uname()'s ''Darwin x.y'' to ''OS X 10.(x-4).y''. Bob Ippolito and others
pointed out that this was not a good idea, because uname() only reports on
the kernel version number and not the Cocoa API, which is really what OS X
10.x.y refers to. He pointed out that the correct way to do it using a
public API is to used gestalt, which is what platform.mac_ver() does.
On further inspection, it was discovered that parsing the
/System/Library/CoreServices/SystemVersion.plist property list is also a
supported API, and would not rely on access to the Carbon API set. Bob and
Wilfredo Sánchez Vega provided sample code that would parse this plist;
Marc-Andre Lemburg suggested that a patch be written for system_alias() that
would use this method (if possible) for Mac OS.
Contributing thread:
- `Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py <
http://mail.python.org/pipermail/python-dev/2005-September/056651.html>`__
[TAM]
================
Deferred Threads
================
- `Python 2.5a1, ast-branch and PEP 342 and 343 <
http://mail.python.org/pipermail/python-dev/2005-September/056449.html>`__
===============
Skipped Threads
===============
- `Visibility scope for "for/while/if" statements <
http://mail.python.org/pipermail/python-dev/2005-September/056669.html>`__
- `inplace operators and __setitem__ <
http://mail.python.org/pipermail/python-dev/2005-September/056766.html>`__
- `Repository for python developers <
http://mail.python.org/pipermail/python-dev/2005-September/056717.html>`__
- `For/while/if statements/comprehension/generator expressions unification <
http://mail.python.org/pipermail/python-dev/2005-September/056508.html>`__
- `list splicing <
http://mail.python.org/pipermail/python-dev/2005-September/056472.html>`__
- `Compatibility between Python 2.3.x and Python 2.4.x <
http://mail.python.org/pipermail/python-dev/2005-September/056437.html>`__
- `python optimization <
http://mail.python.org/pipermail/python-dev/2005-September/056441.html>`__
- `test__locale on Mac OS X <
http://mail.python.org/pipermail/python-dev/2005-September/056463.html>`__
- `possible memory leak on windows (valgrind report) <
http://mail.python.org/pipermail/python-dev/2005-September/056478.html>`__
- `Mixins. <
http://mail.python.org/pipermail/python-dev/2005-September/056481.html>`__
- `2.4.2c1 fails test_unicode on HP-UX ia64 <
http://mail.python.org/pipermail/python-dev/2005-September/056551.html>`__
- `2.4.2c1: test_macfs failing on Tiger (Mac OS X 10.4.2) <
http://mail.python.org/pipermail/python-dev/2005-September/056558.html>`__
- `test_ossaudiodev hangs <
http://mail.python.org/pipermail/python-dev/2005-September/056559.html>`__
- `unintentional and unsafe use of realpath() <
http://mail.python.org/pipermail/python-dev/2005-September/056616.html>`__
- `Alternative name for str.partition() <
http://mail.python.org/pipermail/python-dev/2005-September/056630.html>`__
- `Weekly Python Patch/Bug Summary <
http://mail.python.org/pipermail/python-dev/2005-September/056713.html>`__
- `Possible bug in urllib.urljoin <
http://mail.python.org/pipermail/python-dev/2005-September/056736.html>`__
- `Trasvesal thought on syntax features <
http://mail.python.org/pipermail/python-dev/2005-September/056741.html>`__
- `Fixing pty.spawn() <
http://mail.python.org/pipermail/python-dev/2005-September/056750.html>`__
- `64-bit bytecode compatibility (was Re: [PEAK] ez_setup on 64-bit linux
problem) <
http://mail.python.org/pipermail/python-dev/2005-September/056811.html>`__
- `C API doc fix <
http://mail.python.org/pipermail/python-dev/2005-September/056827.html>`__
- `David Mertz on CA state e-voting panel <
http://mail.python.org/pipermail/python-dev/2005-September/056840.html>`__
- `[PATCH][BUG] Segmentation Fault in xml.dom.minidom.parse <
http://mail.python.org/pipermail/python-dev/2005-September/056844.html>`__
- `linecache problem <
http://mail.python.org/pipermail/python-dev/2005-September/056856.html>`__
========
Epilogue
========
This is a summary of traffic on the `python-dev mailing list`_ from
September 16, 2005 through September 30, 2005.
It is intended to inform the wider Python community of on-going
developments on the list on a semi-monthly basis. An archive_ of
previous summaries is available online.
An `RSS feed`_ of the titles of the summaries is available.
You can also watch comp.lang.python or comp.lang.python.announce for
new summaries (or through their email gateways of python-list or
python-announce, respectively, as found at http://mail.python.org).
This is the 4th summary written by the python-dev summary duo of
Steve Bethard and Tony Meyer (I feel like the White Rabbit in
Wonderland...).
To contact us, please send email:
- Steve Bethard (steven.bethard at gmail.com <http://gmail.com>)
- Tony Meyer (tony.meyer at gmail.com <http://gmail.com>)
Do *not* post to comp.lang.python if you wish to reach us.
The `Python Software Foundation`_ is the non-profit organization that
holds the intellectual property for Python. It also tries to advance
the development and use of Python. If you find the python-dev Summary
helpful please consider making a donation. You can make a donation at
http://python.org/psf/donations.html . Every penny helps so even a
small donation with a credit card, check, or by PayPal helps.
--------------------
Commenting on Topics
--------------------
To comment on anything mentioned here, just post to
`comp.lang.python`_ (or email python-list(a)python.org which is a
gateway to the newsgroup) with a subject line mentioning what you are
discussing. All python-dev members are interested in seeing ideas
discussed by the community, so don't hesitate to take a stance on
something. And if all of this really interests you then get involved
and join `python-dev`_!
-------------------------
How to Read the Summaries
-------------------------
The in-development version of the documentation for Python can be
found at http://www.python.org/dev/doc/devel/ and should be used when
looking up any documentation for new code; otherwise use the current
documentation as found at http://docs.python.org/ . PEPs (Python
Enhancement Proposals) are located at http://www.python.org/peps/ .
To view files in the Python CVS online, go to
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . Reported
bugs and suggested patches can be found at the SourceForge_ project
page.
Please note that this summary is written using reStructuredText_.
Any unfamiliar punctuation is probably markup for reST_ (otherwise it
is probably regular expression syntax or a typo =); you can safely
ignore it. We do suggest learning reST, though; it's simple and is
accepted for `PEP markup`_ and can be turned into many different
formats like HTML and LaTeX. Unfortunately, even though reST is
standardized, the wonders of programs that like to reformat text do
not allow us to guarantee you will be able to run the text version of
this summary through Docutils_ as-is unless it is from the
`original text file`_.
.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list:
http://mail.python.org/mailman/listinfo/python-dev
.. _c.l.py:
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _PEP Markup: http://www.python.org/peps/pep-0012.html
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html
.. _PSF:
.. _Python Software Foundation: http://python.org/psf/
.. _last summary:
http://www.python.org/dev/summary/2005-08-01_2005-08-15.html
.. _original text file:
http://www.python.org/dev/summary/2005-09-16_2005-09-30.ht
.. _archive: http://www.python.org/dev/summary/
.. _RSS feed: http://www.python.org/dev/summary/channews.rdf
We will again be holding PyGTA at the Linux Caffe @ 7pm this Tuesday.
If no one else stands up to suggest a topic I'm thinking I'll
discuss/show the OpenGL-ctypes work I've been doing on and off for the
past few months. I'll try to get a laptop on which to present it so it
can produce a few pretty graphics while we discuss how it's written and
why I've gone with a foreign-function interface to replace the
SWIG-wrapped one.
More details can be found here:
http://web.engcorp.com/pygta/wiki/NextMeeting
Have fun,
Mike
--
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.comhttp://blog.vrplumber.com
I'm pleased to announce FormEncode 0.4.
What Changed?
-------------
Lots of cleanups and clarifications. Also a module to integrate with
SQLObject. Read all about the changes:
http://formencode.org/news.html
What is it?
-----------
FormEncode is a package for form validation and conversion. It also
includes modules for parsing, filling, and extracting metadata from HTML
forms. It features robust conversion both of incoming and outgoing
data, attention paid to helpful error messages, and a wide variety of
pre-build validators. It also supports composition of validators, and
validating structured data, including nested and repeating form elements.
FormEncode is being used in several projects, including Subway,
TurboGears, and SQLObject.
Where is it?
------------
Website and docs:
http://formencode.org
Download:
http://cheeseshop.python.org/pypi/FormEncode
--
Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org
PyCon is now looking for sponsors to help fund the conference; please
see <http://us.pycon.org/TX2006/HowToSponsor> for more information.
Sponsors help ensure that PyCon remains a low-cost conference.
Several levels of sponsorship are available to match your company's
budget.
Sponsoring is also a good targeted way to reach a large set of Python
users. If you want to advertise something -- a product, a book, a job
opening -- to the PyCon community, it costs only $200 to include your
marketing material in the tote bag given to all conference attendees.
For more details, see <http://us.pycon.org/TX2006/HowToSponsor>.
A.M. Kuchling
Chair, PyCon 2006
amk(a)amk.ca
=========================
Python Pepernoten meeting
=========================
On December 9th the PUN (Python Usergroup Netherlands) will hold it's
"Pepernoten" meeting in De Waag in Amsterdam. This is a free and freely
accessible meeting.
Location and time
-----------------
The meeting will take place in `De Waag`_ in Amsterdam. De Waag is
located in the centre of Amsterdam on the `Nieuwmarkt 4`_ (about 15
minutes walk from Central Station or by Metro lines 51, 53 and 54,
station Nieuwmarkt). We'll start at 10am and the program will probably
last until about 4pm. Afterwards there is time for informal socialising.
Programme
---------
Below is the provisional program. If you'd like to give a talk please
let us know. All talks will be in Dutch.
* Jack Jansen - Bgen en C++ (like SWIG but good ;-)
* Stani Michiels - SPE (open source IDE)
* Dick Kniep - Lindix (large project with various connections to for
example OpenOffice)
* Martijn Faassen - lxml (super-fast xml library) (provisional)
* Jeroen Vloothuis - Zope 3 technology outside of Zope
* Lightning talks
Lightning talks
---------------
The idea of the Lightning talks is to give people the opportunity to
share their findings. If you have something to show off but don't have
enough material to fill a full presentation this is your call.
We have several registered lightning talks already; the exact programme
will be announced later.
Registration and costs
----------------------
Registration is free, although food and drink are to be paid for by
yourself. We do ask you to register yourself online (in Dutch) at:
http://nl.python.org/aanmelden_pepernoten/
.. _De Waag: http://www.waag.org/waagsite/
.. _Nieuwmarkt 4: http://tinyurl.com/d73r7