Mailman 3 October 2004 - Python-Dev

Reducing core dependencies for Python on OS X / distutils nit
by Bob Ippolito Oct. 16, 2004

Oct. 16, 2004

Remove CoreServices / CoreFoundation dependencies in core http://python.org/sf/1035255 It came to my attention a couple weeks ago that Python unnecessarily links to a few frameworks on OS X, so I generated a patch that removed these dependencies. I would hope that this is uncontroversial. Presumably Jack has been really busy lately so I understand that this patch didn't make it into b1, but I would like to get it rolled into CVS by *somebody* after CVS is reopened so the final of Python … [View More]

2 2

Wither docs for the subprocess module?
by Raymond Hettinger Oct. 16, 2004

Oct. 16, 2004

Is anyone working on documenting the module for the library reference? Raymond

2 2

python-dev Summary for 2004-09-16 through 2004-09-30 [draft]
by Brett C Oct. 15, 2004

Oct. 15, 2004

So we were all rather quiet in the last half of September. The whole summary fits on two sheets of 8.5x11 (normally it is over 10 and I have hit over 20 when I was summarizing *everything*). Going to send this out no earlier than Friday night so send in corrections by then. ---------------------------------- ===================== Summary Announcements ===================== Wow. This must have been the easiest summary I have ever done. Why can't they all be like this? I didn't even … [View More]skip that much! ========= Summaries ========= ------------------------------------------ Assume nothing when mutability is possible ------------------------------------------ Tim Peters discovered a new way to create an infinite list thanks to generator expressions. But what really came out of this whole discussion came about when someone else came up with an example that exposed a bug in list.extend(). The first thing was that "you can't assume anything about a mutable object after potentially calling back into Python." Basically you can't assume the state of any mutable object was not changed if you execute Python code from C. While it might seem handy to store state while in a loop for instance, you can't count on things not change by the time you get control back so you just have to do it the hard way and get state all over again. Second was that you need to be careful when dealing with iterators. If you mutate an iterator while iterating you don't have a guarantee it won't explode in your face. Unless you explicitly support it, document it, and take care to protect against it then just don't assume you can mutate an iterator while using it. Contributing threads: - `A cute new way to get an infinite loop <>`__ - `More data points <>`__ ---------------------------- The less licenses the better ---------------------------- The idea of copying some code from OpenSSH_ for better pty handling was proposed. This was frowned upon since that becomes one more legal issue to keep track of. Minimizing the licenses that Python must keep track of and make sure to comply with, no matter how friendly, is a good thing. .. _OpenSSH: http://www.openssh.com/ Contributing threads: - `using openssh's pty code <>`__ ------------------------------------------------------------------------ Trying to deal with the exception hierarchy and a backwards-friendly way ------------------------------------------------------------------------- Nick Coghlan came up with the idea of having a tuple that contained all of the exceptions you normally would not want to catch in a blanket 'except' statement; KeyboardInterrupt, MemoryError, SystemExit, etc.). This tuple was proposed to live in sys.special_exceptions with the intended usage of:: try: pass # stuff... except sys.special_exceptions: raise # exceptions that you would not want to catch should keep propogating up the call chain except: pass # if you reach here the exception should not be a *huge* deal Obviously the best solution is to just clean up the exception hierarchy, but that breaks backwards-compatibility. But this idea seemed to lose steam. Contributing threads: - `Proposing a sys.special_exceptions tuple <>`__ =============== Skipped Threads =============== - Decimal, copyright and license - Planning to drop gzip compression for future releases. - built on beer? - Noam's open regex requests - Socket/Asyncore bug needs attention - open('/dev/null').read() -> MemoryError - Finding the module from PyTypeObject? - Odd compile errors for bad genexps - Running a module as a script [View Less]

13 21

About the Wise installer on Windows
by Alexandre Parenteau Oct. 15, 2004

Oct. 15, 2004

Hi, Again forgive me if the answer is somewhere, but I could not find it. I am looking for a way to build and make an installer out of Python on a Win x86_64 platform which is not supported yet by the Python team. My plan is to compile Python for this platform (several developers on the list told me it should be straightforward), and make an installer out of the Python binaries in order to be able to distribute the Python binaries internally. I downloaded the evaluation copy of Wise … [View More]

6 10

Re: python/dist/src/Objects unicodeobject.c, 2.228, 2.229
by Fredrik Lundh Oct. 15, 2004

Oct. 15, 2004

> Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar. > > Python 2.3.x candidate. why bother? the unicode object you're copying to holds size+1 characters, so all the code does is copying an extra NULL character... completely harmless. </F>

2 2

Patch wanted
by Scott David Daniels Oct. 15, 2004

Oct. 15, 2004

Ray Hettinger's fix 1.29 to PC/pyconfig.h: Revision 1.29 - (view) (download) (annotate) - [select for diffs] CVS Tags: r24a2, r24a3 Changes since 1.28: +4 -2 lines Restore compilation on MSVC++ 6.0 =================================== /* Atleast VC 7.1 has them. If some compiler does not provide them, #ifdef appropriately .*/ #define HAVE_UINTPTR_T 1 #define HAVE_INTPTR_T 1 =================================== /* VC 7.1 has them and VC 6.0 does not. VC 6.0 has a version number of … [View More]

2 3

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser Oct. 14, 2004

Oct. 14, 2004

Patch / Bug Summary ___________________ Patches : 240 open ( -1) / 2655 closed (+15) / 2895 total (+14) Bugs : 766 open ( +0) / 4514 closed (+22) / 5280 total (+22) RFE : 155 open ( +1) / 131 closed ( +0) / 286 total ( +1) New / Reopened Patches ______________________ True/False instead of 1/0 in libstdtypes.tex (2004-10-06) CLOSED http://python.org/sf/1041364 opened by Gerrit Holl bbetter document popenX 'cmd' argument (2004-10-07) CLOSED http://python.org/sf/1042705 … [View More]

1 0

PyPy Vilnius Sprint 15-23 nov 2004
by hpk＠trillke.net Oct. 14, 2004

Oct. 14, 2004

Hi Pythonistas and interested developers, PyPy, the python-in-python implementation, is steadily moving on. The next coding sprint will take place in Vilnius, Lithunia, from 15th to 23rd of November, 2004 and is organized by the nice Programmers of Vilnius (POV) company. See http://codespeak.net/pypy/index.cgi?doc for more in-depth information about PyPy. Again, we will be heading towards a first generated C version of our already pretty compliant Python interpreter and types … [View More]

1 0

Cyclic GC issues
by Jason Evans Oct. 14, 2004

Oct. 14, 2004

Since the spring of 2003, I have been developing Crux, which is a computer program for phylogenetic inferencing (bioinformatics) research. In March of 2004, I switched Crux to using Python from having used a different embeddable interpreter. For the most part, I have been very happy with Python, but Python's garbage collector has been a major source of frustration. Below, I describe my trials and tribulations with Python's GC. I also offer some suggestions for changes to Python; if any of … [View More]the proposed changes receive favorable feedback, I am willing to help develop patches to Python. Naturally, if I am somehow abusing Python, and there are better ways to do things, I'd be happy to hear how to improve Crux. The important aspect of Crux is that it deals with trees. These trees are unrooted (there is no up or down), and multifurcating (nodes can have an arbitrary number of neighboring nodes). Thus, the trees are self-referential, and without the cyclic GC capabilities of Python, there would be little hope of making these trees integrate well with Python. Following is a diagram that illustrates the linkage between various objects for a simple tree. Crux exposes all of the components of the trees as Python objects. All lines in the diagram represent bi-directional references (except for the T-->N line). Every object refers to the tree object; those lines are left out in order to reduce clutter. T: Tree N N N: Node \ / E: Edge R R R: Ring \ / E E \ / R---------R / \ / \ / \ / \ | \ / | | \ / | | T--->N | | | | \ | / \ | / \----R----/ | E | R | N At the C (not Python object) level, the R-E-R construct is actually a set of structures that are allocated/deallocated as a single unit. Edges are *always* connected to two rings, so there's no point in allocating these separately. Also, lone ring objects actually are rings with one member; they refer to themselves (prev and next pointers). That should be enough information to understand the problems I encountered. 1) I originally had lone rings refer to themselves (ob_refcnt started out at 3; 2 self-references and one reference held by the associated edge). This didn't work. It appears that the cyclic GC does not actually calculate the number of live references to objects (references that are reachable by traversing all objects accessible from the root set); instead it assumes that if tp_clear() doesn't drop the reference count to a number that equals the number of references from live objects, there must still be references from live objects. Unfortunately, visit()ing self does not work, so there is no way to convince Python that all references are from unreachable objects. Working around this in Crux requires a lot of extra reference counting complexity, because there are three different cases for reference counts, depending on how many members there are in a ring (1, 2, or 3+ members). 2) This issue is really just a different manifestation of issue (1). At the C (not Python object) level, each node only actually stores a pointer to a single member of the associated ring. Given a single ring member, it is possible to traverse the ring and reach all other ring members. As mentioned in issue (1), the cyclic GC expects tp_traverse() to call visit() once for each reference held. It is not enough for a node to visit() one ring member; it must visit() all ring members, in order for the GC to count how many references are from unreachable objects, versus reachable from the root set. In summary, issues (1) and (2) are due to how the cyclic GC does the "marking" phase of mark/sweep GC. My expectation of mark/sweep GC is that it should be sufficient to assure that all objects reachable from the root set are visit()ed at least once; it should not be important how many times each unreachable object is visit()ed. I don't have a deep enough understanding of the Python interpreter to give a detailed suggestion for improvement. I suspect that determining the root set is not an easy operation; if this is the case, then I think we're stuck with the current design. If determining the root set *is* easy (or even possible), then I would suggest using a standard mark/sweep collector, where unreachable objects are scheduled for destruction. tp_traverse(), tp_clear(), and tp_dealloc() would retain the same structure; the only difference would be the logic that determines which objects can be destroyed. 3) A strange thing can happen when tp_clear() is called. Suppose that an edge object is being cleared, and it drops its references to the associated rings. If ob_refcnt of one of the rings drops to 0 as a result, Python will tp_dealloc() the ring *right* *now*, without ever calling tp_clear() on the ring. That means that I have to keep track of whether tp_clear() has been called on objects, if it is at all important that tp_clear() be called, so that I can manually do so in tp_dealloc(), if necessary. It is in my opinion reasonable to have cleanup code in tp_clear(), with the assumption that it will be called precisely once, but Python makes me do extra work to make sure that this happens. This should be pretty easy to change. A single bit per object is needed to keep track of whether tp_clear() has been called. I think this only needs to be done for object types that support cyclic GC. 4) There is no way to control the order in which objects are tp_dealloc()ed. This is a problem for the R-E-R construct, since at a low level, these objects are always linked together. What I would like to do is defer tp_dealloc() on the edge until after both rings have been deleted. Instead, I am forced to use a reference-counted deletion function. Not calling self->ob_type->tp_free() on the edge in tp_dealloc() until later is not a reasonable option, because this defers deletion of the edge until a later round of garbage collection. This could be addressed in the Python interpreter by paying heed to the return value of tp_dealloc(). If the return value is non-zero, move the object to the end of the list of objects to be destroyed, so that destruction is tried later. This allows the module to impose its own destruction ordering. I look forward to feedback. Thank you, Jason Evans [View Less]

7 12

[Python-Dev] Re: python/dist/src/Doc/whatsnew whatsnew24.tex, 1.108, 1.109
by Fredrik Lundh Oct. 14, 2004

Oct. 14, 2004

> +\begin{seealso} > +\seepep{324}{subprocess - New process module}{Written and implemented by Peter Astrand, with > assistance from Fredrik Lundh and others.} > +\end{seealso} I don't know how to add ISO-8859-1 characters to a Latex document, but in HTML notation, Peter's last name should be Åstrand. cheers /F

4 3