Mailman 3 August 2002 - Python-Dev

Re: [Python-Dev] seeing off SET_LINENO
by Michael Hudson Aug. 1, 2002

Aug. 1, 2002

"Mark Hammond" <mhammond(a)skippinet.com.au> writes: > > "Mark Hammond" <mhammond(a)skippinet.com.au> writes: > > > > IMO, the Python debugger "interface" should include function entry. > > > > There goes the time machine: it does. I just think everyone ignores > > 'call' messages because they're a bit redundant today (because of the > > matter under discussion). > > Yes, I should have said "continue to include function entry". > > I understood that a patch under discussion may have *removed* this facility > from the debugger. Nononononononono. No. No. Currently a trace function can be called for four reasons: 'call', 'line', 'return' and 'raise'. 'call' is called high up in eval_frame, on entry to the code object (I suspect it is also called on resumption of generators, but also suspect that this is accidental). 'return' is called when the main loop finished with why == WHY_RETURN or WHY_YIELD, 'raise' ditto but why == WHY_EXCEPTION. None of these are affected by my patch. At the moment 'line' is called by the SET_LINENO opcode. My patch changes it to be called when the co_lnotab indicates execution has moved onto a different line. The reason this changes behaviour is that currently a SET_LINENO opcode is emitted for the def line of every function (I guess this is to cope with def functions_like_this(): return 1 ). After my patch there are no SET_LINENO opcodes, so execution is never on the def line[*], so no 'line' trace event is generated for the def line, so a debugger that only listens to the 'line' events and ignores the 'call' events will not stop on that line. If my patch goes in, I'll probably change pdb to catch 'call' events, and nag authors of other debuggers that they should do the same. It is possible to generate an extra 'line' trace event to mimic the old behaviour, but it's gross. > While I agree it is redundant and most debuggers will choose to > ignore it, I believe removing it from the low level debugger hooks > would be a mistake. Now I've spent some minutes explaining myself, you can explain to me where you got the idea that I was even considering doing so from! Cheers, M. [*] For a typical function which has no code on the def line. -- 34. The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

3 2

What to do about the Wiki?
by Guido van Rossum Aug. 1, 2002

Aug. 1, 2002

I don't know what to do about the Moinmoin Wiki on python.org. Lots of useful information was recently moved to the Wiki, like the editors list and Andrew Kuchling's bookstore. But the Wiki brought the website down twice this weekend, by growing without bounds. To prevent this from happening again, we've disabled the Wiki, but that's not a solution. Juergen Hermann, Moinmoin's author, said he fixed a few things, but also said that Moinmoin is essentially vulnerable to "recursive wget" (e.g. someone trying to suck up the entire Wiki by following links). Apparently this is what brought the site down this weekend -- if I understand correctly, an in-memory log was growing too fast. There are a lot of links in the Wiki, e.g. for each Wiki page there's the page itself, the edit form, the history, various other actions, etc. I believe that Juergen has fixed the log-growing problem. Should we enable the Wiki again and hope for the best? --Guido van Rossum (home page: http://www.python.org/~guido/)

6 10

Re: [Python-checkins] python/dist/src/Misc python.man,1.24,1.25
by Martin v. Löwis Aug. 1, 2002

Aug. 1, 2002

Jack Jansen <Jack.Jansen(a)oratrix.com> writes: > > ! Force stdin, stdout and stderr to be totally unbuffered. Note that > > ! there is internal buffering in xreadlines(), readlines() and file-object > > ! iterators ("for line in sys.stdin") which is not influenced by this > > ! option. To work around this, you will want to use "sys.stdin.readline()" > > ! inside a "while 1:" loop. > > For readlines() I think this is the right thing to do, but > xreadlines() and file iterators could actually "do the right thing" > and revert to a slower scheme if the underlying stream is unbuffered? > Or is this overkill? I'm not sure. The patch describes the current state; if anybody improves that, they should change the man page, too. Regards, Martin

1 0

PEP 298 - the Fixed Buffer Interface
by Thomas Heller Aug. 1, 2002

Aug. 1, 2002

Here is PEP 298 - the Fixed Buffer Interface, posted to get feedback from the Python community. Enjoy! Thomas PS: I'll going to a 2 weeks vacation at the end of this week, so don't hold your breath on replies from me if you post after, let's say, thursday. ----- PEP: 298 Title: The Fixed Buffer Interface Version: $Revision: 1.3 $ Last-Modified: $Date: 2002/07/30 16:52:53 $ Author: Thomas Heller <theller(a)python.net> Status: Draft Type: Standards Track Created: 26-Jul-2002 Python-Version: 2.3 Post-History: Abstract This PEP proposes an extension to the buffer interface called the 'fixed buffer interface'. The fixed buffer interface fixes the flaws of the 'old' buffer interface as defined in Python versions up to and including 2.2, see [1]: The lifetime of the retrieved pointer is clearly defined. The buffer size is returned as a 'size_t' data type, which allows access to large buffers on platforms where sizeof(int) != sizeof(void *). Specification The fixed buffer interface exposes new functions which return the size and the pointer to the internal memory block of any python object which chooses to implement this interface. The size and pointer returned must be valid as long as the object is alive (has a positive reference count). So, only objects which never reallocate or resize the memory block are allowed to implement this interface. The fixed buffer interface omits the memory segment model which is present in the old buffer interface - only a single memory block can be exposed. Implementation Define a new flag in Include/object.h: /* PyBufferProcs contains bf_getfixedreadbuffer and bf_getfixedwritebuffer */ #define Py_TPFLAGS_HAVE_GETFIXEDBUFFER (1L<<15) This flag would be included in Py_TPFLAGS_DEFAULT: #define Py_TPFLAGS_DEFAULT ( \ .... Py_TPFLAGS_HAVE_GETFIXEDBUFFER | \ .... 0) Extend the PyBufferProcs structure by new fields in Include/object.h: typedef size_t (*getfixedreadbufferproc)(PyObject *, void **); typedef size_t (*getfixedwritebufferproc)(PyObject *, void **); typedef struct { getreadbufferproc bf_getreadbuffer; getwritebufferproc bf_getwritebuffer; getsegcountproc bf_getsegcount; getcharbufferproc bf_getcharbuffer; /* fixed buffer interface functions */ getfixedreadbufferproc bf_getfixedreadbufferproc; getfixedwritebufferproc bf_getfixedwritebufferproc; } PyBufferProcs; The new fields are present if the Py_TPFLAGS_HAVE_GETFIXEDBUFFER flag is set in the object's type. The Py_TPFLAGS_HAVE_GETFIXEDBUFFER flag implies the Py_TPFLAGS_HAVE_GETCHARBUFFER flag. The getfixedreadbufferproc and getfixedwritebufferproc functions return the size in bytes of the memory block on success, and fill in the passed void * pointer on success. If these functions fail - either because an error occurs or no memory block is exposed - they must set the void * pointer to NULL and raise an exception. The return value is undefined in these cases and should not be used. Usually the getfixedwritebufferproc and getfixedreadbufferproc functions aren't called directly, they are called through convenience functions declared in Include/abstract.h: int PyObject_AsFixedReadBuffer(PyObject *obj, void **buffer, size_t *buffer_len); int PyObject_AsFixedWriteBuffer(PyObject *obj, void **buffer, size_t *buffer_len); These functions return 0 on success, set buffer to the memory location and buffer_len to the length of the memory block in bytes. On failure, or if the fixed buffer interface is not implemented by obj, they return -1 and set an exception. Backward Compatibility The size of the PyBufferProcs structure changes if this proposal is implemented, but the type's tp_flags slot can be used to determine if the additional fields are present. Reference Implementation Will be uploaded to the SourceForge patch manager by the author. Additional Notes/Comments Python strings, Unicode strings, mmap objects, and maybe other types would expose the fixed buffer interface, but the array type would *not*, because its memory block may be reallocated during its lifetime. Community Feedback Greg Ewing doubts the fixed buffer interface is needed at all, he thinks the normal buffer interface could be used if the pointer is (re)fetched each time it's used. This seems to be dangerous, because even innocent looking calls to the Python API like Py_DECREF() may trigger execution of arbitrary Python code. Neil Hodgson wants to expose pointers to memory blocks with limited lifetime: do some kind of lock operation on the object, retrieve the pointer, use it, and unlock the object again. While the author sees the need for this, it cannot be addressed by this proposal. Beeing required to call a function after not using the pointer received by the getfixedbufferprocs any more seems too error prone. Credits Scott Gilbert came up with the name 'fixed buffer interface'. References [1] The buffer interface http://mail.python.org/pipermail/python-dev/2000-October/009974.html [2] The Buffer Problem http://www.python.org/peps/pep-0296.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

5 6