So we were all rather quiet in the last half of September. The whole summary
fits on two sheets of 8.5x11 (normally it is over 10 and I have hit over 20
when I was summarizing *everything*).
Going to send this out no earlier than Friday night so send in corrections by then.
----------------------------------
=====================
Summary Announcements
=====================
Wow. This must have been the easiest summary I have ever done. Why can't they
all be like this? I didn't even skip that much!
=========
Summaries
=========
------------------------------------------
Assume nothing when mutability is possible
------------------------------------------
Tim Peters discovered a new way to create an infinite list thanks to generator
expressions. But what really came out of this whole discussion came about when
someone else came up with an example that exposed a bug in list.extend().
The first thing was that "you can't assume anything about a mutable object
after potentially calling back into Python." Basically you can't assume the
state of any mutable object was not changed if you execute Python code from C.
While it might seem handy to store state while in a loop for instance, you
can't count on things not change by the time you get control back so you just
have to do it the hard way and get state all over again.
Second was that you need to be careful when dealing with iterators. If you
mutate an iterator while iterating you don't have a guarantee it won't explode
in your face. Unless you explicitly support it, document it, and take care to
protect against it then just don't assume you can mutate an iterator while
using it.
Contributing threads:
- `A cute new way to get an infinite loop <>`__
- `More data points <>`__
----------------------------
The less licenses the better
----------------------------
The idea of copying some code from OpenSSH_ for better pty handling was
proposed. This was frowned upon since that becomes one more legal issue to
keep track of. Minimizing the licenses that Python must keep track of and make
sure to comply with, no matter how friendly, is a good thing.
.. _OpenSSH: http://www.openssh.com/
Contributing threads:
- `using openssh's pty code <>`__
------------------------------------------------------------------------
Trying to deal with the exception hierarchy and a backwards-friendly way
-------------------------------------------------------------------------
Nick Coghlan came up with the idea of having a tuple that contained all of the
exceptions you normally would not want to catch in a blanket 'except'
statement; KeyboardInterrupt, MemoryError, SystemExit, etc.). This tuple was
proposed to live in sys.special_exceptions with the intended usage of::
try:
pass # stuff...
except sys.special_exceptions:
raise # exceptions that you would not want to catch should keep propogating
up the call chain
except:
pass # if you reach here the exception should not be a *huge* deal
Obviously the best solution is to just clean up the exception hierarchy, but
that breaks backwards-compatibility. But this idea seemed to lose steam.
Contributing threads:
- `Proposing a sys.special_exceptions tuple <>`__
===============
Skipped Threads
===============
- Decimal, copyright and license
- Planning to drop gzip compression for future releases.
- built on beer?
- Noam's open regex requests
- Socket/Asyncore bug needs attention
- open('/dev/null').read() -> MemoryError
- Finding the module from PyTypeObject?
- Odd compile errors for bad genexps
- Running a module as a script
Hi,
Again forgive me if the answer is somewhere, but I could not find it.
I am looking for a way to build and make an installer out of Python on a
Win x86_64 platform which is not supported yet by the Python team.
My plan is to compile Python for this platform (several developers on the
list told me it should be straightforward), and make an installer out of
the Python binaries in order to be able to distribute the Python binaries
internally.
I downloaded the evaluation copy of Wise Installer System 9, but it
complains that the installer file (which I assume is PCBuild\Python20.wse)
is corrupted.
I need to establish exactly the version of the installer to use, if we need
to acquire a license for the Wise installer.
- Could someone give me some pointer about the exact version of the Wise
Installer to use in order to be able to replicate the same installer that
the Python team is distributing on Windows?
- The ISS files (Inno) do not seem up to date. Is anybody having a ISS
version of the installer which works for 2.3.4?
Thanks,
alex.
> Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
>
> Python 2.3.x candidate.
why bother?
the unicode object you're copying to holds size+1 characters, so all the
code does is copying an extra NULL character... completely harmless.
</F>
Ray Hettinger's fix 1.29 to PC/pyconfig.h:
Revision 1.29 - (view) (download) (annotate) - [select for diffs]
CVS Tags: r24a2, r24a3
Changes since 1.28: +4 -2 lines
Restore compilation on MSVC++ 6.0
===================================
/* Atleast VC 7.1 has them. If some compiler does not provide them,
#ifdef appropriately .*/
#define HAVE_UINTPTR_T 1
#define HAVE_INTPTR_T 1
===================================
/* VC 7.1 has them and VC 6.0 does not. VC 6.0 has a version number of
1200.
If some compiler does not provide them, modify the #if appropriately. */
#if _MSC_VER != 1200
#define HAVE_UINTPTR_T 1
#define HAVE_INTPTR_T 1
#endif
===================================
Tests the wrong thing. His test presumes every non VC 7.1 PC compiler
has those types defined. For non-MSC compilers, _MSC_VER is undefined,
and the expression
#if _MSC_VER != 1200
becomes:
#if 0 != 1200
(Not what was intended, I presume). Even:
#if _MSC_VER >= 1200
would be preferable (better failure, will presumably work with VC 7.2),
but I believe the correct test should be:
#if HAVE_STD_INT
as I inexpertly explained in patch 1020042.
This affects any C compilers on Windows that don't have the uintptr_t
and intptr_t types (those without the newer include file stdint.h)
except VC 6.0. Such compilers cannot compile any file that contains
#include "Python.h"
I think this needs to be fixed before 2.4 goes out.
--
-- Scott David Daniels
Scott.Daniels(a)Acm.Org
Hi Pythonistas and interested developers,
PyPy, the python-in-python implementation, is steadily moving
on. The next coding sprint will take place in Vilnius,
Lithunia, from
15th to 23rd of November, 2004
and is organized by the nice Programmers of Vilnius (POV)
company. See http://codespeak.net/pypy/index.cgi?doc for more
in-depth information about PyPy.
Again, we will be heading towards a first generated C version
of our already pretty compliant Python interpreter and types
implementation. Last time, before the EuroPython 2004
conference, we actually had a similar goal (a PyPy/C-version) but
discovered we had to largely refactor the basic model for
attribute accesses. We are now closely mirroring the marvelous
"descriptor"-mechanism of CPython.
If you are interested to participate in our fun and somewhat
mind-altering python sprint event then please subscribe at
http://codespeak.net/moin/pypy/moin.cgi/VilniusSprintAttendants
and look around for more information. You'll find that most of
the core PyPy developers are already determined to come. There
are also many areas that need attention so that we should
have tasks suited for different levels of expertise.
At http://codespeak.net/moin/pypy/moin.cgi/VilniusSprintTasks
you'll find our sprint planning task list which will probably
grow in the next weeks.
Note that our EU funding efforts are at the final stage now.
In the next weeks, quite likely before the Vilnius sprint, we
_hope_ to get a "go!" from the european commission. One side
effect would be that coders - probably restricted to european
citizens - may generally apply for getting travel and
accomodation costs refunded for PyPy sprints. This would
reduce the barrier of entry to the question if you like to
spend your time with a pypy sprint. However, we probably need
some time to work out the details after when we get more
information from the EU.
If you have any questions don't hesitate to contact
pypy-sprint(a)codespeak.net or one of us personally.
cheers & a bientot,
Holger Krekel, Armin Rigo
Since the spring of 2003, I have been developing Crux, which is a computer
program for phylogenetic inferencing (bioinformatics) research. In March
of 2004, I switched Crux to using Python from having used a different
embeddable interpreter. For the most part, I have been very happy with
Python, but Python's garbage collector has been a major source of
frustration.
Below, I describe my trials and tribulations with Python's GC. I also
offer some suggestions for changes to Python; if any of the proposed
changes receive favorable feedback, I am willing to help develop patches to
Python. Naturally, if I am somehow abusing Python, and there are better
ways to do things, I'd be happy to hear how to improve Crux.
The important aspect of Crux is that it deals with trees. These trees are
unrooted (there is no up or down), and multifurcating (nodes can have an
arbitrary number of neighboring nodes). Thus, the trees are
self-referential, and without the cyclic GC capabilities of Python, there
would be little hope of making these trees integrate well with Python.
Following is a diagram that illustrates the linkage between various objects
for a simple tree. Crux exposes all of the components of the trees as
Python objects. All lines in the diagram represent bi-directional
references (except for the T-->N line). Every object refers to the tree
object; those lines are left out in order to reduce clutter.
T: Tree N N
N: Node \ /
E: Edge R R
R: Ring \ /
E E
\ /
R---------R
/ \ / \
/ \ / \
| \ / |
| \ / |
| T--->N |
| | |
\ | /
\ | /
\----R----/
|
E
|
R
|
N
At the C (not Python object) level, the R-E-R construct is actually a set
of structures that are allocated/deallocated as a single unit. Edges are
*always* connected to two rings, so there's no point in allocating these
separately.
Also, lone ring objects actually are rings with one member; they refer to
themselves (prev and next pointers).
That should be enough information to understand the problems I encountered.
1) I originally had lone rings refer to themselves (ob_refcnt started out
at 3; 2 self-references and one reference held by the associated edge).
This didn't work. It appears that the cyclic GC does not actually
calculate the number of live references to objects (references that are
reachable by traversing all objects accessible from the root set);
instead it assumes that if tp_clear() doesn't drop the reference count
to a number that equals the number of references from live objects,
there must still be references from live objects. Unfortunately,
visit()ing self does not work, so there is no way to convince Python
that all references are from unreachable objects.
Working around this in Crux requires a lot of extra reference counting
complexity, because there are three different cases for reference
counts, depending on how many members there are in a ring (1, 2, or 3+
members).
2) This issue is really just a different manifestation of issue (1).
At the C (not Python object) level, each node only actually stores a
pointer to a single member of the associated ring. Given a single ring
member, it is possible to traverse the ring and reach all other ring
members. As mentioned in issue (1), the cyclic GC expects tp_traverse()
to call visit() once for each reference held. It is not enough for a
node to visit() one ring member; it must visit() all ring members, in
order for the GC to count how many references are from unreachable
objects, versus reachable from the root set.
In summary, issues (1) and (2) are due to how the cyclic GC does the
"marking" phase of mark/sweep GC. My expectation of mark/sweep GC is
that it should be sufficient to assure that all objects reachable from
the root set are visit()ed at least once; it should not be important how
many times each unreachable object is visit()ed.
I don't have a deep enough understanding of the Python interpreter to
give a detailed suggestion for improvement. I suspect that determining
the root set is not an easy operation; if this is the case, then I think
we're stuck with the current design. If determining the root set *is*
easy (or even possible), then I would suggest using a standard
mark/sweep collector, where unreachable objects are scheduled for
destruction. tp_traverse(), tp_clear(), and tp_dealloc() would retain
the same structure; the only difference would be the logic that
determines which objects can be destroyed.
3) A strange thing can happen when tp_clear() is called. Suppose that an
edge object is being cleared, and it drops its references to the
associated rings. If ob_refcnt of one of the rings drops to 0 as a
result, Python will tp_dealloc() the ring *right* *now*, without ever
calling tp_clear() on the ring. That means that I have to keep track of
whether tp_clear() has been called on objects, if it is at all important
that tp_clear() be called, so that I can manually do so in tp_dealloc(),
if necessary. It is in my opinion reasonable to have cleanup code in
tp_clear(), with the assumption that it will be called precisely once,
but Python makes me do extra work to make sure that this happens.
This should be pretty easy to change. A single bit per object is needed
to keep track of whether tp_clear() has been called. I think this only
needs to be done for object types that support cyclic GC.
4) There is no way to control the order in which objects are
tp_dealloc()ed. This is a problem for the R-E-R construct, since at a
low level, these objects are always linked together. What I would like
to do is defer tp_dealloc() on the edge until after both rings have been
deleted. Instead, I am forced to use a reference-counted deletion
function.
Not calling self->ob_type->tp_free() on the edge in tp_dealloc() until
later is not a reasonable option, because this defers deletion of the
edge until a later round of garbage collection.
This could be addressed in the Python interpreter by paying heed to the
return value of tp_dealloc(). If the return value is non-zero, move the
object to the end of the list of objects to be destroyed, so that
destruction is tried later. This allows the module to impose its own
destruction ordering.
I look forward to feedback.
Thank you,
Jason Evans
> +\begin{seealso}
> +\seepep{324}{subprocess - New process module}{Written and implemented by Peter Astrand, with
> assistance from Fredrik Lundh and others.}
> +\end{seealso}
I don't know how to add ISO-8859-1 characters to a Latex document,
but in HTML notation, Peter's last name should be Åstrand.
cheers /F
Sorry if I'm asking something that has been answered before, but I could
not find a clear answer anywhere.
Is there a project page for Windows 64bits? Is it already on CVS?
My company tries to assess if we can plan on using python on Windows 64 bits.
We want to do it natively (i.e. no 32b emulation), because we have some
python extensions we want to use in the process.
Thanks!
alex.
