Mailman 3 August 1999 - Python-Dev

Some more constants for the socket module
by M.-A. Lemburg Aug. 9, 1999

Aug. 9, 1999

Following the recent discussion on c.l.p about socket options, I found that the socket module does not define all constants defined in the (Linux) socket header file. Below is a patch that adds a few more (note that the SOL_* constants should be used for the setsockopt() level, not the IPPROTO_* constants). --- socketmodule.c~ Sat Aug 7 17:56:05 1999 +++ socketmodule.c Sat Aug 7 18:10:07 1999 @@ -2005,14 +2005,48 @@ initsocket() PySocketSock_Type.tp_doc = sockettype_doc; Py_INCREF(&PySocketSock_Type); if (PyDict_SetItemString(d, "SocketType", (PyObject *)&PySocketSock_Type) != 0) return; + + /* Address families (we only support AF_INET and AF_UNIX) */ +#ifdef AF_UNSPEC + insint(moddict, "AF_UNSPEC", AF_UNSPEC); +#endif insint(d, "AF_INET", AF_INET); #ifdef AF_UNIX insint(d, "AF_UNIX", AF_UNIX); #endif /* AF_UNIX */ +#ifdef AF_AX25 + insint(moddict, "AF_AX25", AF_AX25); /* Amateur Radio AX.25 */ +#endif +#ifdef AF_IPX + insint(moddict, "AF_IPX", AF_IPX); /* Novell IPX */ +#endif +#ifdef AF_APPLETALK + insint(moddict, "AF_APPLETALK", AF_APPLETALK); /* Appletalk DDP */ +#endif +#ifdef AF_NETROM + insint(moddict, "AF_NETROM", AF_NETROM); /* Amateur radio NetROM */ +#endif +#ifdef AF_BRIDGE + insint(moddict, "AF_BRIDGE", AF_BRIDGE); /* Multiprotocol bridge */ +#endif +#ifdef AF_AAL5 + insint(moddict, "AF_AAL5", AF_AAL5); /* Reserved for Werner's ATM */ +#endif +#ifdef AF_X25 + insint(moddict, "AF_X25", AF_X25); /* Reserved for X.25 project */ +#endif +#ifdef AF_INET6 + insint(moddict, "AF_INET6", AF_INET6); /* IP version 6 */ +#endif +#ifdef AF_ROSE + insint(moddict, "AF_ROSE", AF_ROSE); /* Amateur Radio X.25 PLP */ +#endif + + /* Socket types */ insint(d, "SOCK_STREAM", SOCK_STREAM); insint(d, "SOCK_DGRAM", SOCK_DGRAM); #ifndef __BEOS__ /* We have incomplete socket support. */ insint(d, "SOCK_RAW", SOCK_RAW); @@ -2048,11 +2082,10 @@ initsocket() insint(d, "SO_OOBINLINE", SO_OOBINLINE); #endif #ifdef SO_REUSEPORT insint(d, "SO_REUSEPORT", SO_REUSEPORT); #endif - #ifdef SO_SNDBUF insint(d, "SO_SNDBUF", SO_SNDBUF); #endif #ifdef SO_RCVBUF insint(d, "SO_RCVBUF", SO_RCVBUF); @@ -2111,14 +2144,43 @@ initsocket() #ifdef MSG_ETAG insint(d, "MSG_ETAG", MSG_ETAG); #endif /* Protocol level and numbers, usable for [gs]etsockopt */ -/* Sigh -- some systems (e.g. Linux) use enums for these. */ #ifdef SOL_SOCKET insint(d, "SOL_SOCKET", SOL_SOCKET); #endif +#ifdef SOL_IP + insint(moddict, "SOL_IP", SOL_IP); +#else + insint(moddict, "SOL_IP", 0); +#endif +#ifdef SOL_IPX + insint(moddict, "SOL_IPX", SOL_IPX); +#endif +#ifdef SOL_AX25 + insint(moddict, "SOL_AX25", SOL_AX25); +#endif +#ifdef SOL_ATALK + insint(moddict, "SOL_ATALK", SOL_ATALK); +#endif +#ifdef SOL_NETROM + insint(moddict, "SOL_NETROM", SOL_NETROM); +#endif +#ifdef SOL_ROSE + insint(moddict, "SOL_ROSE", SOL_ROSE); +#endif +#ifdef SOL_TCP + insint(moddict, "SOL_TCP", SOL_TCP); +#else + insint(moddict, "SOL_TCP", 6); +#endif +#ifdef SOL_UDP + insint(moddict, "SOL_UDP", SOL_UDP); +#else + insint(moddict, "SOL_UDP", 17); +#endif #ifdef IPPROTO_IP insint(d, "IPPROTO_IP", IPPROTO_IP); #else insint(d, "IPPROTO_IP", 0); #endif @@ -2266,10 +2328,32 @@ initsocket() #ifdef IP_ADD_MEMBERSHIP insint(d, "IP_ADD_MEMBERSHIP", IP_ADD_MEMBERSHIP); #endif #ifdef IP_DROP_MEMBERSHIP insint(d, "IP_DROP_MEMBERSHIP", IP_DROP_MEMBERSHIP); +#endif +#ifdef IP_DEFAULT_MULTICAST_TTL + insint(moddict, "IP_DEFAULT_MULTICAST_TTL", IP_DEFAULT_MULTICAST_TTL); +#endif +#ifdef IP_DEFAULT_MULTICAST_LOOP + insint(moddict, "IP_DEFAULT_MULTICAST_LOOP", IP_DEFAULT_MULTICAST_LOOP); +#endif +#ifdef IP_MAX_MEMBERSHIPS + insint(moddict, "IP_MAX_MEMBERSHIPS", IP_MAX_MEMBERSHIPS); +#endif + + /* TCP options */ +#ifdef TCP_NODELAY + insint(moddict, "TCP_NODELAY", TCP_NODELAY); +#endif +#ifdef TCP_MAXSEG + insint(moddict, "TCP_MAXSEG", TCP_MAXSEG); +#endif + + /* IPX options */ +#ifdef IPX_TYPE + insint(moddict, "IPX_TYPE", IPX_TYPE); #endif /* Initialize gethostbyname lock */ #ifdef USE_GETHOSTBYNAME_LOCK gethostbyname_lock = PyThread_allocate_lock(); -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 146 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

2 2

Re: [Python-Dev] marshal (was:Buffer interface in abstract.c? )
by Jack Jansen Aug. 9, 1999

Aug. 9, 1999

> Aside: wouldn't it make sense to move from marshal to pickle and > depreciate marshal altogether ? cPickle is quite fast and much more > flexible than marshal, plus it already provides mechanisms for > registering new types. This is probably the best idea so far. Just remove the buffer-workaround in marshall, keep it functioning for the things it is used for now (like pyc files) and refer people to (c)Pickle for new development. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

1 0

Re: [Python-Dev] marshal (was:Buffer interface in abstract.c? )
by Jack Jansen Aug. 9, 1999

Aug. 9, 1999

Recently, Greg Stein <gstein(a)lyra.org> said: > I would suggest backing out the marshalling of buffer-interface objects > and adding a mechanism for arbitrary type objects to marshal themselves. > Without the second part, arrays and Unicode objects aren't marshallable > at all (seems bad). This sounds like the right approach. It would require 2 slots in the tp_ structure and a little extra glue for the typecodes (currently marshall knows all the 1-letter typecodes for all objecttypes it can handle, but types marshalling their own objects would require a centralized registry of object types. For the time being it would probably suffice to have the mapping of type<->letter be hardcoded in marshal.h, but eventually you probably want a more extensible scheme, where Joe R. Extension-Writer could add a marshaller to his objects and know it won't collide with someone else's. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

2 1

mmapfile module
by Andrew M. Kuchling Aug. 6, 1999

Aug. 6, 1999

A while back the suggestion was made that the mmapfile module be added to the core distribution, and there was a guardedly positive reaction. Should I go ahead and do that? No one reported any problems when I asked for bug reports, but that was probably because no one tried it; putting it in the core would cause more people to try it. I suppose this leads to a more important question: at what point should we start checking 1.6-only things into the CVS tree? For example, once the current alphas of the re module are up to it (they're not yet), when should they be checked in? -- A.M. Kuchling http://starship.python.net/crew/amk/ Kids! Bringing about Armageddon can be dangerous. Do not attempt it in your home. -- Terry Pratchett & Neil Gaiman, _Good Omens_

5 4

Re: [Python-Dev] Buffer interface in abstract.c?
by Jack Jansen Aug. 5, 1999

Aug. 5, 1999

Why not pass the index to the As*Buffer routines as well and make getsegcount available too? Then you could code things like for(i=0; i<PyObject_GetBufferCount(obj); i++) { if ( PyObject_AsCharBuffer(obj, &buf, &count, i) < 0 ) return -1; write(fp, buf, count); } -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

5 4

Re: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/api api.tex
by Jack Jansen Aug. 5, 1999

Aug. 5, 1999

> All righty. I'll send some doc on this stuff. Somebody else did the > initial buffer interface, but it seems that it has fallen to me now :-) I think I did, but I gladly bequeath it to you. (Hmm, that's the first time I typed "bequeath", I think). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

1 0

Re: [Python-checkins] CVS: python/dist/src/Doc/api api.tex
by Greg Stein Aug. 4, 1999

Aug. 4, 1999

Fred L. Drake wrote: > > Update of /projects/cvsroot/python/dist/src/Doc/api > In directory weyr:/home/fdrake/projects/python/Doc/api > > Modified Files: > api.tex > Log Message: > > Started documentation on buffer objects & types. Very preliminary. > > Greg Stein: Please help with this; it's your baby! > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://www.python.org/mailman/listinfo/python-checkins All righty. I'll send some doc on this stuff. Somebody else did the initial buffer interface, but it seems that it has fallen to me now :-) Please give me a little while to get to this, though. I'm in and out of town for the next four weeks. <SubtleAnnouncement> I'm in the process of moving into a new house in Palo Alto, CA, and I'm travelling back and forth until Anni and I move for real in September. </SubtleAnnouncement> I should be able to get to this by the weekend, or possibly in a couple weeks. Cheers, -g -- Greg Stein, http://www.lyra.org/

2 1

Pickling w/ low overhead
by David Ascher Aug. 3, 1999

Aug. 3, 1999

An issue which has dogged the NumPy project is that there is (to my knowledge) no way to pickle very large arrays without creating strings which contain all of the data. This can be a problem given that NumPy arrays tend to be very large -- often several megabytes, sometimes much bigger. This slows things down, sometimes a lot, depending on the platform. It seems that it should be possible to do something more efficient. Two alternatives come to mind: -- define a new pickling protocol which passes a file-like object to the instance and have the instance write itself to that file, being as efficient or inefficient as it cares to. This protocol is used only if the instance/type defines the appropriate slot. Alternatively, enrich the semantics of the getstate interaction, so that an object can return partial data and tell the pickling mechanism to come back for more. -- make pickling of objects which support the buffer interface use that inteface's notion of segments and use that 'chunk' size to do something more efficient if not necessarily most efficient. (oh, and make NumPy arrays support the buffer interface =). This is simple for NumPy arrays since we want to pickle "everything", but may not be what other buffer-supporting objects want. Thoughts? Alternatives? --david

3 5

Re: [Python-Dev] Buffer interface in abstract.c?
by Jack Jansen Aug. 3, 1999

Aug. 3, 1999

> > Why not pass the index to the As*Buffer routines as well and make getsegcount > > available too? > > Simply because multiple segments hasn't been seen. All objects > supporting the buffer interface have a single segment. Hmm. And I went out of my way to include this stupid multi-buffer stuff because the NumPy folks said they couldn't live without it (and one of the reasons for the buffer stuff was to allow NumPy arrays, which may be discontiguous, to be written efficiently). Can someone confirm that the Numeric stuff indeed doesn't use this? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

2 1

Buffer interface in abstract.c?
by Mark Hammond Aug. 3, 1999

Aug. 3, 1999

Hi all, Im trying to slowly wean myself over to the buffer interfaces. My exploration so far indicates that, for most cases, simply replacing "PyString_FromStringAndSize" with "PyBuffer_FromMemory" handles the vast majority of cases, and is preferred when the data contains arbitary bytes. PyArg_ParseTuple("s#", ...) still works correctly as we would hope. However, performing this explicitly is a pain. Looking at getargs.c, the code to achieve this is a little too convoluted to cut-and-paste each time. Therefore, I would like to propose these functions to be added to abstract.c: int PyObject_GetBufferSize(); void *PyObject_GetReadWriteBuffer(); /* or "char *"? */ const void *PyObject_GetReadOnlyBuffer(); Although equivalent functions exist for the buffer object, I can't see the equivalent abstract implementations - ie, that work with any object supporting the protocol. Im willing to provide a patch if there is agreement a) the general idea is good, and b) my specific spelling of the idea is OK (less likely - PyBuffer_* seems better, but loses any implication of being abstract?). Thoughts? Mark.

5 5