Mailman 3 August 1999 - Python-Dev

Re: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/api api.tex
by Jack Jansen 05 Aug '99

05 Aug '99

> All righty. I'll send some doc on this stuff. Somebody else did the > initial buffer interface, but it seems that it has fallen to me now :-) I think I did, but I gladly bequeath it to you. (Hmm, that's the first time I typed "bequeath", I think). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

1 0

Re: [Python-checkins] CVS: python/dist/src/Doc/api api.tex
by Greg Stein 04 Aug '99

04 Aug '99

Fred L. Drake wrote: > > Update of /projects/cvsroot/python/dist/src/Doc/api > In directory weyr:/home/fdrake/projects/python/Doc/api > > Modified Files: > api.tex > Log Message: > > Started documentation on buffer objects & types. Very preliminary. > > Greg Stein: Please help with this; it's your baby! > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://www.python.org/mailman/listinfo/python-checkins All righty. I'll send some doc on this stuff. Somebody else did the initial buffer interface, but it seems that it has fallen to me now :-) Please give me a little while to get to this, though. I'm in and out of town for the next four weeks. <SubtleAnnouncement> I'm in the process of moving into a new house in Palo Alto, CA, and I'm travelling back and forth until Anni and I move for real in September. </SubtleAnnouncement> I should be able to get to this by the weekend, or possibly in a couple weeks. Cheers, -g -- Greg Stein, http://www.lyra.org/

2 1

Pickling w/ low overhead
by David Ascher 03 Aug '99

03 Aug '99

An issue which has dogged the NumPy project is that there is (to my knowledge) no way to pickle very large arrays without creating strings which contain all of the data. This can be a problem given that NumPy arrays tend to be very large -- often several megabytes, sometimes much bigger. This slows things down, sometimes a lot, depending on the platform. It seems that it should be possible to do something more efficient. Two alternatives come to mind: -- define a new pickling protocol which passes a file-like object to the instance and have the instance write itself to that file, being as efficient or inefficient as it cares to. This protocol is used only if the instance/type defines the appropriate slot. Alternatively, enrich the semantics of the getstate interaction, so that an object can return partial data and tell the pickling mechanism to come back for more. -- make pickling of objects which support the buffer interface use that inteface's notion of segments and use that 'chunk' size to do something more efficient if not necessarily most efficient. (oh, and make NumPy arrays support the buffer interface =). This is simple for NumPy arrays since we want to pickle "everything", but may not be what other buffer-supporting objects want. Thoughts? Alternatives? --david

3 5

Re: [Python-Dev] Buffer interface in abstract.c?
by Jack Jansen 03 Aug '99

03 Aug '99

> > Why not pass the index to the As*Buffer routines as well and make getsegcount > > available too? > > Simply because multiple segments hasn't been seen. All objects > supporting the buffer interface have a single segment. Hmm. And I went out of my way to include this stupid multi-buffer stuff because the NumPy folks said they couldn't live without it (and one of the reasons for the buffer stuff was to allow NumPy arrays, which may be discontiguous, to be written efficiently). Can someone confirm that the Numeric stuff indeed doesn't use this? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

2 1

Buffer interface in abstract.c?
by Mark Hammond 03 Aug '99

03 Aug '99

Hi all, Im trying to slowly wean myself over to the buffer interfaces. My exploration so far indicates that, for most cases, simply replacing "PyString_FromStringAndSize" with "PyBuffer_FromMemory" handles the vast majority of cases, and is preferred when the data contains arbitary bytes. PyArg_ParseTuple("s#", ...) still works correctly as we would hope. However, performing this explicitly is a pain. Looking at getargs.c, the code to achieve this is a little too convoluted to cut-and-paste each time. Therefore, I would like to propose these functions to be added to abstract.c: int PyObject_GetBufferSize(); void *PyObject_GetReadWriteBuffer(); /* or "char *"? */ const void *PyObject_GetReadOnlyBuffer(); Although equivalent functions exist for the buffer object, I can't see the equivalent abstract implementations - ie, that work with any object supporting the protocol. Im willing to provide a patch if there is agreement a) the general idea is good, and b) my specific spelling of the idea is OK (less likely - PyBuffer_* seems better, but loses any implication of being abstract?). Thoughts? Mark.

5 5