Mailman 3 May 2006 - Python-checkins

June 5, 2006

Author: kristjan.jonsson Date: Sat May 27 17:41:31 2006 New Revision: 46472 Added: python/trunk/PCbuild8/ python/trunk/PCbuild8/Uninstal.wse python/trunk/PCbuild8/_bsddb.vcproj python/trunk/PCbuild8/_ctypes.vcproj python/trunk/PCbuild8/_ctypes_test.vcproj python/trunk/PCbuild8/_elementtree.vcproj python/trunk/PCbuild8/_msi.vcproj python/trunk/PCbuild8/_socket.vcproj python/trunk/PCbuild8/_sqlite3.vcproj python/trunk/PCbuild8/_ssl.mak python/trunk/PCbuild8/_ssl.… [View More]

3 2

r46144 - in python/trunk: Lib/struct.py Modules/_struct.c Modules/structmodule.c
by bob.ippolito June 3, 2006

June 3, 2006

Author: bob.ippolito Date: Tue May 23 21:12:41 2006 New Revision: 46144 Added: python/trunk/Lib/struct.py (contents, props changed) python/trunk/Modules/_struct.c (contents, props changed) Removed: python/trunk/Modules/structmodule.c Log: patch #1493701: performance enhancements for struct module Added: python/trunk/Lib/struct.py ============================================================================== --- (empty file) +++ python/trunk/Lib/struct.py Tue May 23 21:12:41 2006 @… [View More]@ -0,0 +1,76 @@ +""" +Functions to convert between Python values and C structs. +Python strings are used to hold the data representing the C struct +and also as format strings to describe the layout of data in the C struct. + +The optional first format char indicates byte order, size and alignment: + @: native order, size & alignment (default) + =: native order, std. size & alignment + <: little-endian, std. size & alignment + >: big-endian, std. size & alignment + !: same as > + +The remaining chars indicate types of args and must match exactly; +these can be preceded by a decimal repeat count: + x: pad byte (no data); c:char; b:signed byte; B:unsigned byte; + h:short; H:unsigned short; i:int; I:unsigned int; + l:long; L:unsigned long; f:float; d:double. +Special cases (preceding decimal count indicates length): + s:string (array of char); p: pascal string (with count byte). +Special case (only available in native format): + P:an integer type that is wide enough to hold a pointer. +Special case (not in native mode unless 'long long' in platform C): + q:long long; Q:unsigned long long +Whitespace between formats is ignored. + +The variable struct.error is an exception raised on errors. +""" +__version__ = '0.1' + +from _struct import Struct, error + +_MAXCACHE = 100 +_cache = {} + +def _compile(fmt): + # Internal: compile struct pattern + if len(_cache) >= _MAXCACHE: + _cache.clear() + s = Struct(fmt) + _cache[fmt] = s + return s + +def calcsize(fmt): + """ + Return size of C struct described by format string fmt. + See struct.__doc__ for more on format strings. + """ + try: + o = _cache[fmt] + except KeyError: + o = _compile(fmt) + return o.size + +def pack(fmt, *args): + """ + Return string containing values v1, v2, ... packed according to fmt. + See struct.__doc__ for more on format strings. + """ + try: + o = _cache[fmt] + except KeyError: + o = _compile(fmt) + return o.pack(*args) + +def unpack(fmt, s): + """ + Unpack the string, containing packed C structure data, according + to fmt. Requires len(string)==calcsize(fmt). + See struct.__doc__ for more on format strings. + """ + try: + o = _cache[fmt] + except KeyError: + o = _compile(fmt) + return o.unpack(s) + Added: python/trunk/Modules/_struct.c ============================================================================== --- (empty file) +++ python/trunk/Modules/_struct.c Tue May 23 21:12:41 2006 @@ -0,0 +1,1355 @@ +/* struct module -- pack values into and (out of) strings */ + +/* New version supporting byte order, alignment and size options, + character strings, and unsigned numbers */ + +#include "Python.h" +#include "structseq.h" +#include "structmember.h" +#include <ctype.h> + + +/* compatibility macros */ +#if (PY_VERSION_HEX < 0x02050000) +typedef int Py_ssize_t; +#endif + + + +/* The translation function for each format character is table driven */ + +typedef struct _formatdef { + char format; + int size; + int alignment; + PyObject* (*unpack)(const char *, + const struct _formatdef *); + int (*pack)(char *, PyObject *, + const struct _formatdef *); +} formatdef; + +typedef struct _formatcode { + const struct _formatdef *fmtdef; + int offset; + int repeat; +} formatcode; + +/* Struct object interface */ + +typedef struct { + PyObject_HEAD + int s_size; + int s_len; + formatcode *s_codes; + PyObject *s_format; + PyObject *weakreflist; /* List of weak references */ +} PyStructObject; + +PyAPI_DATA(PyTypeObject) PyStruct_Type; + +#define PyStruct_Check(op) PyObject_TypeCheck(op, &PyStruct_Type) +#define PyStruct_CheckExact(op) ((op)->ob_type == &PyStruct_Type) + + +/* Exception */ + +static PyObject *StructError; + + +/* Define various structs to figure out the alignments of types */ + + +typedef struct { char c; short x; } st_short; +typedef struct { char c; int x; } st_int; +typedef struct { char c; long x; } st_long; +typedef struct { char c; float x; } st_float; +typedef struct { char c; double x; } st_double; +typedef struct { char c; void *x; } st_void_p; + +#define SHORT_ALIGN (sizeof(st_short) - sizeof(short)) +#define INT_ALIGN (sizeof(st_int) - sizeof(int)) +#define LONG_ALIGN (sizeof(st_long) - sizeof(long)) +#define FLOAT_ALIGN (sizeof(st_float) - sizeof(float)) +#define DOUBLE_ALIGN (sizeof(st_double) - sizeof(double)) +#define VOID_P_ALIGN (sizeof(st_void_p) - sizeof(void *)) + +/* We can't support q and Q in native mode unless the compiler does; + in std mode, they're 8 bytes on all platforms. */ +#ifdef HAVE_LONG_LONG +typedef struct { char c; PY_LONG_LONG x; } s_long_long; +#define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(PY_LONG_LONG)) +#endif + +#define STRINGIFY(x) #x + +#ifdef __powerc +#pragma options align=reset +#endif + +/* Helper to get a PyLongObject by hook or by crook. Caller should decref. */ + +static PyObject * +get_pylong(PyObject *v) +{ + PyNumberMethods *m; + + assert(v != NULL); + if (PyInt_Check(v)) + return PyLong_FromLong(PyInt_AS_LONG(v)); + if (PyLong_Check(v)) { + Py_INCREF(v); + return v; + } + m = v->ob_type->tp_as_number; + if (m != NULL && m->nb_long != NULL) { + v = m->nb_long(v); + if (v == NULL) + return NULL; + if (PyLong_Check(v)) + return v; + Py_DECREF(v); + } + PyErr_SetString(StructError, + "cannot convert argument to long"); + return NULL; +} + +/* Helper routine to get a Python integer and raise the appropriate error + if it isn't one */ + +static int +get_long(PyObject *v, long *p) +{ + long x = PyInt_AsLong(v); + if (x == -1 && PyErr_Occurred()) { + if (PyErr_ExceptionMatches(PyExc_TypeError)) + PyErr_SetString(StructError, + "required argument is not an integer"); + return -1; + } + *p = x; + return 0; +} + + +/* Same, but handling unsigned long */ + +static int +get_ulong(PyObject *v, unsigned long *p) +{ + if (PyLong_Check(v)) { + unsigned long x = PyLong_AsUnsignedLong(v); + if (x == (unsigned long)(-1) && PyErr_Occurred()) + return -1; + *p = x; + return 0; + } + else { + return get_long(v, (long *)p); + } +} + +#ifdef HAVE_LONG_LONG + +/* Same, but handling native long long. */ + +static int +get_longlong(PyObject *v, PY_LONG_LONG *p) +{ + PY_LONG_LONG x; + + v = get_pylong(v); + if (v == NULL) + return -1; + assert(PyLong_Check(v)); + x = PyLong_AsLongLong(v); + Py_DECREF(v); + if (x == (PY_LONG_LONG)-1 && PyErr_Occurred()) + return -1; + *p = x; + return 0; +} + +/* Same, but handling native unsigned long long. */ + +static int +get_ulonglong(PyObject *v, unsigned PY_LONG_LONG *p) +{ + unsigned PY_LONG_LONG x; + + v = get_pylong(v); + if (v == NULL) + return -1; + assert(PyLong_Check(v)); + x = PyLong_AsUnsignedLongLong(v); + Py_DECREF(v); + if (x == (unsigned PY_LONG_LONG)-1 && PyErr_Occurred()) + return -1; + *p = x; + return 0; +} + +#endif + +/* Floating point helpers */ + +static PyObject * +unpack_float(const char *p, /* start of 4-byte string */ + int le) /* true for little-endian, false for big-endian */ +{ + double x; + + x = _PyFloat_Unpack4((unsigned char *)p, le); + if (x == -1.0 && PyErr_Occurred()) + return NULL; + return PyFloat_FromDouble(x); +} + +static PyObject * +unpack_double(const char *p, /* start of 8-byte string */ + int le) /* true for little-endian, false for big-endian */ +{ + double x; + + x = _PyFloat_Unpack8((unsigned char *)p, le); + if (x == -1.0 && PyErr_Occurred()) + return NULL; + return PyFloat_FromDouble(x); +} + + +/* A large number of small routines follow, with names of the form + + [bln][up]_TYPE + + [bln] distiguishes among big-endian, little-endian and native. + [pu] distiguishes between pack (to struct) and unpack (from struct). + TYPE is one of char, byte, ubyte, etc. +*/ + +/* Native mode routines. ****************************************************/ +/* NOTE: + In all n[up]_<type> routines handling types larger than 1 byte, there is + *no* guarantee that the p pointer is properly aligned for each type, + therefore memcpy is called. An intermediate variable is used to + compensate for big-endian architectures. + Normally both the intermediate variable and the memcpy call will be + skipped by C optimisation in little-endian architectures (gcc >= 2.91 + does this). */ + +static PyObject * +nu_char(const char *p, const formatdef *f) +{ + return PyString_FromStringAndSize(p, 1); +} + +static PyObject * +nu_byte(const char *p, const formatdef *f) +{ + return PyInt_FromLong((long) *(signed char *)p); +} + +static PyObject * +nu_ubyte(const char *p, const formatdef *f) +{ + return PyInt_FromLong((long) *(unsigned char *)p); +} + +static PyObject * +nu_short(const char *p, const formatdef *f) +{ + short x; + memcpy((char *)&x, p, sizeof x); + return PyInt_FromLong((long)x); +} + +static PyObject * +nu_ushort(const char *p, const formatdef *f) +{ + unsigned short x; + memcpy((char *)&x, p, sizeof x); + return PyInt_FromLong((long)x); +} + +static PyObject * +nu_int(const char *p, const formatdef *f) +{ + int x; + memcpy((char *)&x, p, sizeof x); + return PyInt_FromLong((long)x); +} + +static PyObject * +nu_uint(const char *p, const formatdef *f) +{ + unsigned int x; + memcpy((char *)&x, p, sizeof x); + return PyLong_FromUnsignedLong((unsigned long)x); +} + +static PyObject * +nu_long(const char *p, const formatdef *f) +{ + long x; + memcpy((char *)&x, p, sizeof x); + return PyInt_FromLong(x); +} + +static PyObject * +nu_ulong(const char *p, const formatdef *f) +{ + unsigned long x; + memcpy((char *)&x, p, sizeof x); + return PyLong_FromUnsignedLong(x); +} + +/* Native mode doesn't support q or Q unless the platform C supports + long long (or, on Windows, __int64). */ + +#ifdef HAVE_LONG_LONG + +static PyObject * +nu_longlong(const char *p, const formatdef *f) +{ + PY_LONG_LONG x; + memcpy((char *)&x, p, sizeof x); + return PyLong_FromLongLong(x); +} + +static PyObject * +nu_ulonglong(const char *p, const formatdef *f) +{ + unsigned PY_LONG_LONG x; + memcpy((char *)&x, p, sizeof x); + return PyLong_FromUnsignedLongLong(x); +} + +#endif + +static PyObject * +nu_float(const char *p, const formatdef *f) +{ + float x; + memcpy((char *)&x, p, sizeof x); + return PyFloat_FromDouble((double)x); +} + +static PyObject * +nu_double(const char *p, const formatdef *f) +{ + double x; + memcpy((char *)&x, p, sizeof x); + return PyFloat_FromDouble(x); +} + +static PyObject * +nu_void_p(const char *p, const formatdef *f) +{ + void *x; + memcpy((char *)&x, p, sizeof x); + return PyLong_FromVoidPtr(x); +} + +static int +np_byte(char *p, PyObject *v, const formatdef *f) +{ + long x; + if (get_long(v, &x) < 0) + return -1; + if (x < -128 || x > 127){ + PyErr_SetString(StructError, + "byte format requires -128<=number<=127"); + return -1; + } + *p = (char)x; + return 0; +} + +static int +np_ubyte(char *p, PyObject *v, const formatdef *f) +{ + long x; + if (get_long(v, &x) < 0) + return -1; + if (x < 0 || x > 255){ + PyErr_SetString(StructError, + "ubyte format requires 0<=number<=255"); + return -1; + } + *p = (char)x; + return 0; +} + +static int +np_char(char *p, PyObject *v, const formatdef *f) +{ + if (!PyString_Check(v) || PyString_Size(v) != 1) { + PyErr_SetString(StructError, + "char format require string of length 1"); + return -1; + } + *p = *PyString_AsString(v); + return 0; +} + +static int +np_short(char *p, PyObject *v, const formatdef *f) +{ + long x; + short y; + if (get_long(v, &x) < 0) + return -1; + if (x < SHRT_MIN || x > SHRT_MAX){ + PyErr_SetString(StructError, + "short format requires " STRINGIFY(SHRT_MIN) + "<=number<=" STRINGIFY(SHRT_MAX)); + return -1; + } + y = (short)x; + memcpy(p, (char *)&y, sizeof y); + return 0; +} + +static int +np_ushort(char *p, PyObject *v, const formatdef *f) +{ + long x; + unsigned short y; + if (get_long(v, &x) < 0) + return -1; + if (x < 0 || x > USHRT_MAX){ + PyErr_SetString(StructError, + "short format requires 0<=number<=" STRINGIFY(USHRT_MAX)); + return -1; + } + y = (unsigned short)x; + memcpy(p, (char *)&y, sizeof y); + return 0; +} + +static int +np_int(char *p, PyObject *v, const formatdef *f) +{ + long x; + int y; + if (get_long(v, &x) < 0) + return -1; + y = (int)x; + memcpy(p, (char *)&y, sizeof y); + return 0; +} + +static int +np_uint(char *p, PyObject *v, const formatdef *f) +{ + unsigned long x; + unsigned int y; + if (get_ulong(v, &x) < 0) + return -1; + y = (unsigned int)x; + memcpy(p, (char *)&y, sizeof y); + return 0; +} + +static int +np_long(char *p, PyObject *v, const formatdef *f) +{ + long x; + if (get_long(v, &x) < 0) + return -1; + memcpy(p, (char *)&x, sizeof x); + return 0; +} + +static int +np_ulong(char *p, PyObject *v, const formatdef *f) +{ + unsigned long x; + if (get_ulong(v, &x) < 0) + return -1; + memcpy(p, (char *)&x, sizeof x); + return 0; +} + +#ifdef HAVE_LONG_LONG + +static int +np_longlong(char *p, PyObject *v, const formatdef *f) +{ + PY_LONG_LONG x; + if (get_longlong(v, &x) < 0) + return -1; + memcpy(p, (char *)&x, sizeof x); + return 0; +} + +static int +np_ulonglong(char *p, PyObject *v, const formatdef *f) +{ + unsigned PY_LONG_LONG x; + if (get_ulonglong(v, &x) < 0) + return -1; + memcpy(p, (char *)&x, sizeof x); + return 0; +} +#endif + +static int +np_float(char *p, PyObject *v, const formatdef *f) +{ + float x = (float)PyFloat_AsDouble(v); + if (x == -1 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + memcpy(p, (char *)&x, sizeof x); + return 0; +} + +static int +np_double(char *p, PyObject *v, const formatdef *f) +{ + double x = PyFloat_AsDouble(v); + if (x == -1 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + memcpy(p, (char *)&x, sizeof(double)); + return 0; +} + +static int +np_void_p(char *p, PyObject *v, const formatdef *f) +{ + void *x; + + v = get_pylong(v); + if (v == NULL) + return -1; + assert(PyLong_Check(v)); + x = PyLong_AsVoidPtr(v); + Py_DECREF(v); + if (x == NULL && PyErr_Occurred()) + return -1; + memcpy(p, (char *)&x, sizeof x); + return 0; +} + +static formatdef native_table[] = { + {'x', sizeof(char), 0, NULL}, + {'b', sizeof(char), 0, nu_byte, np_byte}, + {'B', sizeof(char), 0, nu_ubyte, np_ubyte}, + {'c', sizeof(char), 0, nu_char, np_char}, + {'s', sizeof(char), 0, NULL}, + {'p', sizeof(char), 0, NULL}, + {'h', sizeof(short), SHORT_ALIGN, nu_short, np_short}, + {'H', sizeof(short), SHORT_ALIGN, nu_ushort, np_ushort}, + {'i', sizeof(int), INT_ALIGN, nu_int, np_int}, + {'I', sizeof(int), INT_ALIGN, nu_uint, np_uint}, + {'l', sizeof(long), LONG_ALIGN, nu_long, np_long}, + {'L', sizeof(long), LONG_ALIGN, nu_ulong, np_ulong}, + {'f', sizeof(float), FLOAT_ALIGN, nu_float, np_float}, + {'d', sizeof(double), DOUBLE_ALIGN, nu_double, np_double}, + {'P', sizeof(void *), VOID_P_ALIGN, nu_void_p, np_void_p}, +#ifdef HAVE_LONG_LONG + {'q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_longlong, np_longlong}, + {'Q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, +#endif + {0} +}; + +/* Big-endian routines. *****************************************************/ + +static PyObject * +bu_int(const char *p, const formatdef *f) +{ + long x = 0; + int i = f->size; + do { + x = (x<<8) | (*p++ & 0xFF); + } while (--i > 0); + /* Extend the sign bit. */ + if (SIZEOF_LONG > f->size) + x |= -(x & (1L << (8*f->size - 1))); + return PyInt_FromLong(x); +} + +static PyObject * +bu_uint(const char *p, const formatdef *f) +{ + unsigned long x = 0; + int i = f->size; + do { + x = (x<<8) | (*p++ & 0xFF); + } while (--i > 0); + if (f->size >= 4) + return PyLong_FromUnsignedLong(x); + else + return PyInt_FromLong((long)x); +} + +static PyObject * +bu_longlong(const char *p, const formatdef *f) +{ + return _PyLong_FromByteArray((const unsigned char *)p, + 8, + 0, /* little-endian */ + 1 /* signed */); +} + +static PyObject * +bu_ulonglong(const char *p, const formatdef *f) +{ + return _PyLong_FromByteArray((const unsigned char *)p, + 8, + 0, /* little-endian */ + 0 /* signed */); +} + +static PyObject * +bu_float(const char *p, const formatdef *f) +{ + return unpack_float(p, 0); +} + +static PyObject * +bu_double(const char *p, const formatdef *f) +{ + return unpack_double(p, 0); +} + +static int +bp_int(char *p, PyObject *v, const formatdef *f) +{ + long x; + int i; + if (get_long(v, &x) < 0) + return -1; + i = f->size; + do { + p[--i] = (char)x; + x >>= 8; + } while (i > 0); + return 0; +} + +static int +bp_uint(char *p, PyObject *v, const formatdef *f) +{ + unsigned long x; + int i; + if (get_ulong(v, &x) < 0) + return -1; + i = f->size; + do { + p[--i] = (char)x; + x >>= 8; + } while (i > 0); + return 0; +} + +static int +bp_longlong(char *p, PyObject *v, const formatdef *f) +{ + int res; + v = get_pylong(v); + if (v == NULL) + return -1; + res = _PyLong_AsByteArray((PyLongObject *)v, + (unsigned char *)p, + 8, + 0, /* little_endian */ + 1 /* signed */); + Py_DECREF(v); + return res; +} + +static int +bp_ulonglong(char *p, PyObject *v, const formatdef *f) +{ + int res; + v = get_pylong(v); + if (v == NULL) + return -1; + res = _PyLong_AsByteArray((PyLongObject *)v, + (unsigned char *)p, + 8, + 0, /* little_endian */ + 0 /* signed */); + Py_DECREF(v); + return res; +} + +static int +bp_float(char *p, PyObject *v, const formatdef *f) +{ + double x = PyFloat_AsDouble(v); + if (x == -1 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + return _PyFloat_Pack4(x, (unsigned char *)p, 0); +} + +static int +bp_double(char *p, PyObject *v, const formatdef *f) +{ + double x = PyFloat_AsDouble(v); + if (x == -1 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + return _PyFloat_Pack8(x, (unsigned char *)p, 0); +} + +static formatdef bigendian_table[] = { + {'x', 1, 0, NULL}, + {'b', 1, 0, bu_int, bp_int}, + {'B', 1, 0, bu_uint, bp_int}, + {'c', 1, 0, nu_char, np_char}, + {'s', 1, 0, NULL}, + {'p', 1, 0, NULL}, + {'h', 2, 0, bu_int, bp_int}, + {'H', 2, 0, bu_uint, bp_uint}, + {'i', 4, 0, bu_int, bp_int}, + {'I', 4, 0, bu_uint, bp_uint}, + {'l', 4, 0, bu_int, bp_int}, + {'L', 4, 0, bu_uint, bp_uint}, + {'q', 8, 0, bu_longlong, bp_longlong}, + {'Q', 8, 0, bu_ulonglong, bp_ulonglong}, + {'f', 4, 0, bu_float, bp_float}, + {'d', 8, 0, bu_double, bp_double}, + {0} +}; + +/* Little-endian routines. *****************************************************/ + +static PyObject * +lu_int(const char *p, const formatdef *f) +{ + long x = 0; + int i = f->size; + do { + x = (x<<8) | (p[--i] & 0xFF); + } while (i > 0); + /* Extend the sign bit. */ + if (SIZEOF_LONG > f->size) + x |= -(x & (1L << (8*f->size - 1))); + return PyInt_FromLong(x); +} + +static PyObject * +lu_uint(const char *p, const formatdef *f) +{ + unsigned long x = 0; + int i = f->size; + do { + x = (x<<8) | (p[--i] & 0xFF); + } while (i > 0); + if (f->size >= 4) + return PyLong_FromUnsignedLong(x); + else + return PyInt_FromLong((long)x); +} + +static PyObject * +lu_longlong(const char *p, const formatdef *f) +{ + return _PyLong_FromByteArray((const unsigned char *)p, + 8, + 1, /* little-endian */ + 1 /* signed */); +} + +static PyObject * +lu_ulonglong(const char *p, const formatdef *f) +{ + return _PyLong_FromByteArray((const unsigned char *)p, + 8, + 1, /* little-endian */ + 0 /* signed */); +} + +static PyObject * +lu_float(const char *p, const formatdef *f) +{ + return unpack_float(p, 1); +} + +static PyObject * +lu_double(const char *p, const formatdef *f) +{ + return unpack_double(p, 1); +} + +static int +lp_int(char *p, PyObject *v, const formatdef *f) +{ + long x; + int i; + if (get_long(v, &x) < 0) + return -1; + i = f->size; + do { + *p++ = (char)x; + x >>= 8; + } while (--i > 0); + return 0; +} + +static int +lp_uint(char *p, PyObject *v, const formatdef *f) +{ + unsigned long x; + int i; + if (get_ulong(v, &x) < 0) + return -1; + i = f->size; + do { + *p++ = (char)x; + x >>= 8; + } while (--i > 0); + return 0; +} + +static int +lp_longlong(char *p, PyObject *v, const formatdef *f) +{ + int res; + v = get_pylong(v); + if (v == NULL) + return -1; + res = _PyLong_AsByteArray((PyLongObject*)v, + (unsigned char *)p, + 8, + 1, /* little_endian */ + 1 /* signed */); + Py_DECREF(v); + return res; +} + +static int +lp_ulonglong(char *p, PyObject *v, const formatdef *f) +{ + int res; + v = get_pylong(v); + if (v == NULL) + return -1; + res = _PyLong_AsByteArray((PyLongObject*)v, + (unsigned char *)p, + 8, + 1, /* little_endian */ + 0 /* signed */); + Py_DECREF(v); + return res; +} + +static int +lp_float(char *p, PyObject *v, const formatdef *f) +{ + double x = PyFloat_AsDouble(v); + if (x == -1 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + return _PyFloat_Pack4(x, (unsigned char *)p, 1); +} + +static int +lp_double(char *p, PyObject *v, const formatdef *f) +{ + double x = PyFloat_AsDouble(v); + if (x == -1 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + return _PyFloat_Pack8(x, (unsigned char *)p, 1); +} + +static formatdef lilendian_table[] = { + {'x', 1, 0, NULL}, + {'b', 1, 0, lu_int, lp_int}, + {'B', 1, 0, lu_uint, lp_int}, + {'c', 1, 0, nu_char, np_char}, + {'s', 1, 0, NULL}, + {'p', 1, 0, NULL}, + {'h', 2, 0, lu_int, lp_int}, + {'H', 2, 0, lu_uint, lp_uint}, + {'i', 4, 0, lu_int, lp_int}, + {'I', 4, 0, lu_uint, lp_uint}, + {'l', 4, 0, lu_int, lp_int}, + {'L', 4, 0, lu_uint, lp_uint}, + {'q', 8, 0, lu_longlong, lp_longlong}, + {'Q', 8, 0, lu_ulonglong, lp_ulonglong}, + {'f', 4, 0, lu_float, lp_float}, + {'d', 8, 0, lu_double, lp_double}, + {0} +}; + + +static const formatdef * +whichtable(char **pfmt) +{ + const char *fmt = (*pfmt)++; /* May be backed out of later */ + switch (*fmt) { + case '<': + return lilendian_table; + case '>': + case '!': /* Network byte order is big-endian */ + return bigendian_table; + case '=': { /* Host byte order -- different from native in aligment! */ + int n = 1; + char *p = (char *) &n; + if (*p == 1) + return lilendian_table; + else + return bigendian_table; + } + default: + --*pfmt; /* Back out of pointer increment */ + /* Fall through */ + case '@': + return native_table; + } +} + + +/* Get the table entry for a format code */ + +static const formatdef * +getentry(int c, const formatdef *f) +{ + for (; f->format != '\0'; f++) { + if (f->format == c) { + return f; + } + } + PyErr_SetString(StructError, "bad char in struct format"); + return NULL; +} + + +/* Align a size according to a format code */ + +static int +align(int size, int c, const formatdef *e) +{ + if (e->format == c) { + if (e->alignment) { + size = ((size + e->alignment - 1) + / e->alignment) + * e->alignment; + } + } + return size; +} + + +/* calculate the size of a format string */ + +static int +prepare_s(PyStructObject *self) +{ + const formatdef *f; + const formatdef *e; + formatcode *codes; + + const char *s; + const char *fmt; + char c; + int size, len, numcodes, num, itemsize, x; + + fmt = PyString_AS_STRING(self->s_format); + + f = whichtable((char **)&fmt); + + s = fmt; + size = 0; + len = 0; + numcodes = 0; + while ((c = *s++) != '\0') { + if (isspace(Py_CHARMASK(c))) + continue; + if ('0' <= c && c <= '9') { + num = c - '0'; + while ('0' <= (c = *s++) && c <= '9') { + x = num*10 + (c - '0'); + if (x/10 != num) { + PyErr_SetString( + StructError, + "overflow in item count"); + return -1; + } + num = x; + } + if (c == '\0') + break; + } + else + num = 1; + + e = getentry(c, f); + if (e == NULL) + return -1; + + switch (c) { + case 's': /* fall through */ + case 'p': len++; break; + case 'x': break; + default: len += num; break; + } + if (c != 'x') numcodes++; + + itemsize = e->size; + size = align(size, c, e); + x = num * itemsize; + size += x; + if (x/itemsize != num || size < 0) { + PyErr_SetString(StructError, + "total struct size too long"); + return -1; + } + } + + self->s_size = size; + self->s_len = len; + codes = PyMem_MALLOC((numcodes + 1) * sizeof(formatcode)); + if (codes == NULL) { + PyErr_NoMemory(); + return -1; + } + self->s_codes = codes; + + s = fmt; + size = 0; + while ((c = *s++) != '\0') { + if (isspace(Py_CHARMASK(c))) + continue; + if ('0' <= c && c <= '9') { + num = c - '0'; + while ('0' <= (c = *s++) && c <= '9') + num = num*10 + (c - '0'); + if (c == '\0') + break; + } + else + num = 1; + + e = getentry(c, f); + + size = align(size, c, e); + if (c != 'x') { + codes->offset = size; + codes->repeat = num; + codes->fmtdef = e; + codes++; + } + size += num * e->size; + } + codes->fmtdef = NULL; + codes->offset = -1; + codes->repeat = -1; + + return 0; +} + +static PyObject * +s_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +{ + PyObject *self; + static PyObject *not_yet_string; + + assert(type != NULL && type->tp_alloc != NULL); + + self = type->tp_alloc(type, 0); + if (self != NULL) { + PyStructObject *s = (PyStructObject*)self; + Py_INCREF(Py_None); + s->s_format = Py_None; + s->s_codes = NULL; + s->s_size = -1; + s->s_len = -1; + } + return self; +} + +static int +s_init(PyObject *self, PyObject *args, PyObject *kwds) +{ + PyStructObject *soself = (PyStructObject *)self; + PyObject *o_format = NULL; + int ret = 0; + static char *kwlist[] = {"format", 0}; + + assert(PyStruct_Check(self)); + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "S:Struct", kwlist, + &o_format)) + return -1; + + Py_INCREF(o_format); + Py_XDECREF(soself->s_format); + soself->s_format = o_format; + + ret = prepare_s(soself); + return ret; +} + +static void +s_dealloc(PyStructObject *s) +{ + int sts = 0; + if (s->weakreflist != NULL) + PyObject_ClearWeakRefs((PyObject *)s); + if (s->s_codes != NULL) { + PyMem_FREE(s->s_codes); + } + Py_XDECREF(s->s_format); + s->ob_type->tp_free((PyObject *)s); +} + +PyDoc_STRVAR(s_unpack__doc__, +"unpack(str) -> (v1, v2, ...)\n\ +\n\ +Return tuple containing values unpacked according to this Struct's format.\n\ +Requires len(str) == self.size. See struct.__doc__ for more on format\n\ +strings."); + +static PyObject * +s_unpack(PyObject *self, PyObject *inputstr) +{ + PyStructObject *soself; + PyObject *result; + char *restart; + formatcode *code; + Py_ssize_t i; + + soself = (PyStructObject *)self; + assert(PyStruct_Check(self)); + assert(soself->s_codes != NULL); + if (inputstr == NULL || !PyString_Check(inputstr) || + PyString_GET_SIZE(inputstr) != soself->s_size) { + PyErr_Format(StructError, + "unpack requires a string argument of length %d", soself->s_size); + return NULL; + } + result = PyTuple_New(soself->s_len); + if (result == NULL) + return NULL; + + + restart = PyString_AS_STRING(inputstr); + i = 0; + for (code = soself->s_codes; code->fmtdef != NULL; code++) { + Py_ssize_t n; + PyObject *v; + const formatdef *e = code->fmtdef; + const char *res = restart + code->offset; + if (e->format == 's') { + v = PyString_FromStringAndSize(res, code->repeat); + if (v == NULL) + goto fail; + PyTuple_SET_ITEM(result, i++, v); + } else if (e->format == 'p') { + n = *(unsigned char*)res; + if (n >= code->repeat) + n = code->repeat - 1; + v = PyString_FromStringAndSize(res + 1, n); + if (v == NULL) + goto fail; + PyTuple_SET_ITEM(result, i++, v); + } else { + for (n = 0; n < code->repeat; n++) { + v = e->unpack(res, e); + if (v == NULL) + goto fail; + PyTuple_SET_ITEM(result, i++, v); + res += e->size; + } + } + } + + return result; +fail: + Py_DECREF(result); + return NULL; +}; + + +PyDoc_STRVAR(s_pack__doc__, +"pack(v1, v2, ...) -> string\n\ +\n\ +Return a string containing values v1, v2, ... packed according to this\n\ +Struct's format. See struct.__doc__ for more on format strings."); + +static PyObject * +s_pack(PyObject *self, PyObject *args) +{ + PyStructObject *soself; + PyObject *result; + char *restart; + formatcode *code; + Py_ssize_t i; + + soself = (PyStructObject *)self; + assert(PyStruct_Check(self)); + assert(soself->s_codes != NULL); + if (args == NULL || !PyTuple_Check(args) || + PyTuple_GET_SIZE(args) != soself->s_len) + { + PyErr_Format(StructError, + "pack requires exactly %d arguments", soself->s_len); + return NULL; + } + + result = PyString_FromStringAndSize((char *)NULL, soself->s_size); + if (result == NULL) + return NULL; + + restart = PyString_AS_STRING(result); + memset(restart, '\0', soself->s_size); + i = 0; + for (code = soself->s_codes; code->fmtdef != NULL; code++) { + Py_ssize_t n; + PyObject *v; + const formatdef *e = code->fmtdef; + char *res = restart + code->offset; + if (e->format == 's') { + v = PyTuple_GET_ITEM(args, i++); + if (!PyString_Check(v)) { + PyErr_SetString(StructError, + "argument for 's' must be a string"); + goto fail; + } + n = PyString_GET_SIZE(v); + if (n > code->repeat) + n = code->repeat; + if (n > 0) + memcpy(res, PyString_AS_STRING(v), n); + } else if (e->format == 'p') { + v = PyTuple_GET_ITEM(args, i++); + if (!PyString_Check(v)) { + PyErr_SetString(StructError, + "argument for 'p' must be a string"); + goto fail; + } + n = PyString_GET_SIZE(v); + if (n > (code->repeat - 1)) + n = code->repeat - 1; + if (n > 0) + memcpy(res + 1, PyString_AS_STRING(v), n); + if (n > 255) + n = 255; + *res = Py_SAFE_DOWNCAST(n, Py_ssize_t, unsigned char); + } else { + for (n = 0; n < code->repeat; n++) { + v = PyTuple_GET_ITEM(args, i++); + if (e->pack(res, v, e) < 0) + goto fail; + res += e->size; + } + } + } + + return result; + +fail: + Py_DECREF(result); + return NULL; + +} + + +/* List of functions */ + +static struct PyMethodDef s_methods[] = { + {"pack", s_pack, METH_VARARGS, s_pack__doc__}, + {"unpack", s_unpack, METH_O, s_unpack__doc__}, + {NULL, NULL} /* sentinel */ +}; + +PyDoc_STRVAR(s__doc__, "Compiled struct object"); + +#define OFF(x) offsetof(PyStructObject, x) + +static PyMemberDef s_memberlist[] = { + {"format", T_OBJECT, OFF(s_format), RO, + "struct format string"}, + {"size", T_INT, OFF(s_size), RO, + "struct size in bytes"}, + {"_len", T_INT, OFF(s_len), RO, + "number of items expected in tuple"}, + {NULL} /* Sentinel */ +}; + + +static +PyTypeObject PyStructType = { + PyObject_HEAD_INIT(&PyType_Type) + 0, + "Struct", + sizeof(PyStructObject), + 0, + (destructor)s_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + PyObject_GenericSetAttr, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ + s__doc__, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + offsetof(PyStructObject, weakreflist), /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + s_methods, /* tp_methods */ + s_memberlist, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + s_init, /* tp_init */ + PyType_GenericAlloc, /* tp_alloc */ + s_new, /* tp_new */ + PyObject_Del, /* tp_free */ +}; + +/* Module initialization */ + +PyMODINIT_FUNC +init_struct(void) +{ + PyObject *m = Py_InitModule("_struct", NULL); + if (m == NULL) + return; + + /* Add some symbolic constants to the module */ + if (StructError == NULL) { + StructError = PyErr_NewException("struct.error", NULL, NULL); + if (StructError == NULL) + return; + } + Py_INCREF(StructError); + PyModule_AddObject(m, "error", StructError); + Py_INCREF((PyObject*)&PyStructType); + PyModule_AddObject(m, "Struct", (PyObject*)&PyStructType); +} Deleted: /python/trunk/Modules/structmodule.c ============================================================================== [View Less]

3 2

r46505 - python/trunk/Tools/pybench/systimes.py
by marc-andre.lemburg June 2, 2006

June 2, 2006

Author: marc-andre.lemburg Date: Sun May 28 19:46:58 2006 New Revision: 46505 Added: python/trunk/Tools/pybench/systimes.py (contents, props changed) Log: Initial version of systimes - a module to provide platform dependent performance measurements. The module is currently just a proof-of-concept implementation, but will integrated into pybench once it is stable enough. License: pybench license. Author: Marc-Andre Lemburg. Added: python/trunk/Tools/pybench/systimes.py ===============… [View More]=============================================================== --- (empty file) +++ python/trunk/Tools/pybench/systimes.py Sun May 28 19:46:58 2006 @@ -0,0 +1,197 @@ +#!/usr/bin/env python + +""" systimes() user and system timer implementations for use by + pybench. + + This module implements various different strategies for measuring + performance timings. It tries to choose the best available method + based on the platforma and available tools. + + On Windows, it is recommended to have the Mark Hammond win32 + package installed. Alternatively, the Thomas Heller ctypes + packages can also be used. + + On Unix systems, the standard resource module provides the highest + resolution timings. Unfortunately, it is not available on all Unix + platforms. + + If no supported timing methods based on process time can be found, + the module reverts to the highest resolution wall-time timer + instead. The system time part will then always be 0.0. + + The module exports one public API: + + def systimes(): + + Return the current timer values for measuring user and system + time as tuple of seconds (user_time, system_time). + + Copyright (c) 2006, Marc-Andre Lemburg (mal(a)egenix.com). See the + documentation for further information on copyrights, or contact + the author. All Rights Reserved. + +""" +import time, sys, struct + +# +# Note: Please keep this module compatible to Python 1.5.2. +# +# TODOs: +# +# * Add ctypes wrapper for new clock_gettime() real-time POSIX APIs; +# these will then provide nano-second resolution where available. +# +# * Add a function that returns the resolution of systimes() +# values, ie. systimesres(). +# + +### Choose an implementation + +SYSTIMES_IMPLEMENTATION = None +USE_CTYPES_GETPROCESSTIMES = 'cytpes GetProcessTimes() wrapper' +USE_WIN32PROCESS_GETPROCESSTIMES = 'win32process.GetProcessTimes()' +USE_RESOURCE_GETRUSAGE = 'resource.getrusage()' +USE_PROCESS_TIME_CLOCK = 'time.clock() (process time)' +USE_WALL_TIME_CLOCK = 'time.clock() (wall-time)' +USE_WALL_TIME_TIME = 'time.time() (wall-time)' + +if sys.platform[:3] == 'win': + # Windows platform + try: + import win32process + except ImportError: + try: + import ctypes + except ImportError: + # Use the wall-time implementation time.clock(), since this + # is the highest resolution clock available on Windows + SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_CLOCK + else: + SYSTIMES_IMPLEMENTATION = USE_CTYPES_GETPROCESSTIMES + else: + SYSTIMES_IMPLEMENTATION = USE_WIN32PROCESS_GETPROCESSTIMES +else: + # All other platforms + try: + import resource + except ImportError: + pass + else: + SYSTIMES_IMPLEMENTATION = USE_RESOURCE_GETRUSAGE + +# Fall-back solution +if SYSTIMES_IMPLEMENTATION is None: + # Check whether we can use time.clock() as approximation + # for systimes() + start = time.clock() + time.sleep(0.1) + stop = time.clock() + if stop - start < 0.001: + # Looks like time.clock() is usable (and measures process + # time) + SYSTIMES_IMPLEMENTATION = USE_PROCESS_TIME_CLOCK + else: + # Use wall-time implementation time.time() since this provides + # the highest resolution clock on most systems + SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_TIME + +### Implementations + +def getrusage_systimes(): + return resource.getrusage(resource.RUSAGE_SELF)[:2] + +def process_time_clock_systimes(): + return (time.clock(), 0.0) + +def wall_time_clock_systimes(): + return (time.clock(), 0.0) + +def wall_time_time_systimes(): + return (time.time(), 0.0) + +# Number of clock ticks per second for the values returned +# by GetProcessTimes() on Windows. +# +# Note: Ticks returned by GetProcessTimes() are micro-seconds on +# Windows XP (though the docs say 100ns intervals) +WIN32_PROCESS_TIMES_TICKS_PER_SECOND = 10e6 + +def win32process_getprocesstimes_systimes(): + d = win32process.GetProcessTimes(win32process.GetCurrentProcess()) + # Note: I'm not sure whether KernelTime on Windows is the same as + # system time on Unix - I've yet to see a non-zero value for + # KernelTime on Windows. + return (d['UserTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, + d['KernelTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) + +def ctypes_getprocesstimes_systimes(): + creationtime = ctypes.c_ulonglong() + exittime = ctypes.c_ulonglong() + kerneltime = ctypes.c_ulonglong() + usertime = ctypes.c_ulonglong() + rc = ctypes.windll.kernel32.GetProcessTimes( + ctypes.windll.kernel32.GetCurrentProcess(), + ctypes.byref(creationtime), + ctypes.byref(exittime), + ctypes.byref(kerneltime), + ctypes.byref(usertime)) + if not rc: + raise TypeError('GetProcessTimes() returned an error') + return (usertime.value / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, + kerneltime.value / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) + +# Select the default for the systimes() function + +if SYSTIMES_IMPLEMENTATION is USE_RESOURCE_GETRUSAGE: + systimes = getrusage_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_PROCESS_TIME_CLOCK: + systimes = process_time_clock_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_CLOCK: + systimes = wall_time_clock_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_TIME: + systimes = wall_time_time_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_WIN32PROCESS_GETPROCESSTIMES: + systimes = win32process_getprocesstimes_systimes + +elif SYSTIMES_IMPLEMENTATION is USE_CTYPES_GETPROCESSTIMES: + systimes = ctypes_getprocesstimes_systimes + +else: + raise TypeError('no suitable systimes() implementation found') + +### Testing + +def some_workload(): + x = 0L + for i in xrange(10000000L): + x = x + 1L + +def test_workload(): + print 'Testing systimes() under load conditions' + t0 = systimes() + some_workload() + t1 = systimes() + print 'before:', t0 + print 'after:', t1 + print 'differences:', (t1[0] - t0[0], t1[1] - t0[1]) + print + +def test_idle(): + print 'Testing systimes() under idle conditions' + t0 = systimes() + time.sleep(1) + t1 = systimes() + print 'before:', t0 + print 'after:', t1 + print 'differences:', (t1[0] - t0[0], t1[1] - t0[1]) + print + +if __name__ == '__main__': + print 'Using %s as timer' % SYSTIMES_IMPLEMENTATION + print + test_workload() + test_idle() [View Less]

4 7

Re: [Python-checkins] r46472 - python/trunk/PCbuild8 python/trunk/PCbuild8/Uninstal.wse python/trunk/PCbuild8/_bsddb.vcproj python/trunk/PCbuild8/_ctypes.vcproj python/trunk/PCbuild8/_ctypes_test.vcproj python/trunk/PCbuild8/_elementtree.vcproj pytho
by Kristján V. Jónsson June 2, 2006

June 2, 2006

Hi there. Well, I was sort of hoping that it would "maintain itself" in an open source manner, but if it needs an official maintainter, we can do that. Particularly, we are interested in input from the community on how better to set up the projects in VS to perform the profile-guilded optimization step with minimal user input. Our current approach relies on a file called pythoncore_pgo_link.txt which needs to be manually updated when the project changes significantly. Martin, you have been … [View More]

5 5

r46300 - in python/trunk: Lib/socket.py Lib/test/test_socket.py Lib/test/test_struct.py Modules/_struct.c Modules/arraymodule.c Modules/socketmodule.c
by martin.blais June 1, 2006

June 1, 2006

Author: martin.blais Date: Fri May 26 14:03:27 2006 New Revision: 46300 Modified: python/trunk/Lib/socket.py python/trunk/Lib/test/test_socket.py python/trunk/Lib/test/test_struct.py python/trunk/Modules/_struct.c python/trunk/Modules/arraymodule.c python/trunk/Modules/socketmodule.c Log: Support for buffer protocol for socket and struct. * Added socket.recv_buf() and socket.recvfrom_buf() methods, that use the buffer protocol (send and sendto already did). * Added struct.… [View More]pack_to(), that is the corresponding buffer compatible method to unpack_from(). * Fixed minor typos in arraymodule. Modified: python/trunk/Lib/socket.py ============================================================================== --- python/trunk/Lib/socket.py (original) +++ python/trunk/Lib/socket.py Fri May 26 14:03:27 2006 @@ -140,7 +140,9 @@ __doc__ = _realsocket.__doc__ - __slots__ = ["_sock", "send", "recv", "sendto", "recvfrom", + __slots__ = ["_sock", + "recv", "recv_buf", "recvfrom_buf", + "send", "sendto", "recvfrom", "__weakref__"] def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _sock=None): @@ -149,8 +151,10 @@ self._sock = _sock self.send = self._sock.send self.recv = self._sock.recv + self.recv_buf = self._sock.recv_buf self.sendto = self._sock.sendto self.recvfrom = self._sock.recvfrom + self.recvfrom_buf = self._sock.recvfrom_buf def close(self): self._sock = _closedsocket() Modified: python/trunk/Lib/test/test_socket.py ============================================================================== --- python/trunk/Lib/test/test_socket.py (original) +++ python/trunk/Lib/test/test_socket.py Fri May 26 14:03:27 2006 @@ -9,6 +9,7 @@ import thread, threading import Queue import sys +import array from weakref import proxy PORT = 50007 @@ -852,8 +853,38 @@ self.assertRaises(socket.error, s.bind, address) +class BufferIOTest(SocketConnectedTest): + """ + Test the buffer versions of socket.recv() and socket.send(). + """ + def __init__(self, methodName='runTest'): + SocketConnectedTest.__init__(self, methodName=methodName) + + def testRecvBuf(self): + buf = array.array('c', ' '*1024) + nbytes = self.cli_conn.recv_buf(buf) + self.assertEqual(nbytes, len(MSG)) + msg = buf.tostring()[:len(MSG)] + self.assertEqual(msg, MSG) + + def _testRecvBuf(self): + buf = buffer(MSG) + self.serv_conn.send(buf) + + def testRecvFromBuf(self): + buf = array.array('c', ' '*1024) + nbytes, addr = self.cli_conn.recvfrom_buf(buf) + self.assertEqual(nbytes, len(MSG)) + msg = buf.tostring()[:len(MSG)] + self.assertEqual(msg, MSG) + + def _testRecvFromBuf(self): + buf = buffer(MSG) + self.serv_conn.send(buf) + def test_main(): - tests = [GeneralModuleTests, BasicTCPTest, TCPTimeoutTest, TestExceptions] + tests = [GeneralModuleTests, BasicTCPTest, TCPTimeoutTest, TestExceptions, + BufferIOTest] if sys.platform != 'mac': tests.extend([ BasicUDPTest, UDPTimeoutTest ]) Modified: python/trunk/Lib/test/test_struct.py ============================================================================== --- python/trunk/Lib/test/test_struct.py (original) +++ python/trunk/Lib/test/test_struct.py Fri May 26 14:03:27 2006 @@ -1,5 +1,8 @@ from test.test_support import TestFailed, verbose, verify +import test.test_support import struct +import array +import unittest import sys ISBIGENDIAN = sys.byteorder == "big" @@ -438,31 +441,6 @@ test_705836() -def test_unpack_from(): - test_string = 'abcd01234' - fmt = '4s' - s = struct.Struct(fmt) - for cls in (str, buffer): - data = cls(test_string) - assert s.unpack_from(data) == ('abcd',) - assert s.unpack_from(data, 2) == ('cd01',) - assert s.unpack_from(data, 4) == ('0123',) - for i in xrange(6): - assert s.unpack_from(data, i) == (data[i:i+4],) - for i in xrange(6, len(test_string) + 1): - simple_err(s.unpack_from, data, i) - for cls in (str, buffer): - data = cls(test_string) - assert struct.unpack_from(fmt, data) == ('abcd',) - assert struct.unpack_from(fmt, data, 2) == ('cd01',) - assert struct.unpack_from(fmt, data, 4) == ('0123',) - for i in xrange(6): - assert struct.unpack_from(fmt, data, i) == (data[i:i+4],) - for i in xrange(6, len(test_string) + 1): - simple_err(struct.unpack_from, fmt, data, i) - -test_unpack_from() - def test_1229380(): for endian in ('', '>', '<'): for cls in (int, long): @@ -478,3 +456,60 @@ if 0: # TODO: bug #1229380 test_1229380() + +class PackBufferTestCase(unittest.TestCase): + """ + Test the packing methods that work on buffers. + """ + + def test_unpack_from( self ): + test_string = 'abcd01234' + fmt = '4s' + s = struct.Struct(fmt) + for cls in (str, buffer): + data = cls(test_string) + self.assertEquals(s.unpack_from(data), ('abcd',)) + self.assertEquals(s.unpack_from(data, 2), ('cd01',)) + self.assertEquals(s.unpack_from(data, 4), ('0123',)) + for i in xrange(6): + self.assertEquals(s.unpack_from(data, i), (data[i:i+4],)) + for i in xrange(6, len(test_string) + 1): + simple_err(s.unpack_from, data, i) + for cls in (str, buffer): + data = cls(test_string) + self.assertEquals(struct.unpack_from(fmt, data), ('abcd',)) + self.assertEquals(struct.unpack_from(fmt, data, 2), ('cd01',)) + self.assertEquals(struct.unpack_from(fmt, data, 4), ('0123',)) + for i in xrange(6): + self.assertEquals(struct.unpack_from(fmt, data, i), + (data[i:i+4],)) + for i in xrange(6, len(test_string) + 1): + simple_err(struct.unpack_from, fmt, data, i) + + def test_pack_to( self ): + test_string = 'Reykjavik rocks, eow!' + writable_buf = array.array('c', ' '*100) + fmt = '21s' + s = struct.Struct(fmt) + + # Test without offset + s.pack_to(writable_buf, 0, test_string) + from_buf = writable_buf.tostring()[:len(test_string)] + self.assertEquals(from_buf, test_string) + + # Test with offset. + s.pack_to(writable_buf, 10, test_string) + from_buf = writable_buf.tostring()[:len(test_string)+10] + self.assertEquals(from_buf, (test_string[:10] + test_string)) + + # Go beyond boundaries. + small_buf = array.array('c', ' '*10) + self.assertRaises(struct.error, s.pack_to, small_buf, 0, test_string) + self.assertRaises(struct.error, s.pack_to, small_buf, 2, test_string) + +def test_main(): + test.test_support.run_unittest(PackBufferTestCase) + +if __name__ == "__main__": + test_main() + Modified: python/trunk/Modules/_struct.c ============================================================================== --- python/trunk/Modules/_struct.c (original) +++ python/trunk/Modules/_struct.c Fri May 26 14:03:27 2006 @@ -15,6 +15,7 @@ typedef int Py_ssize_t; #endif + /* PY_USE_INT_WHEN_POSSIBLE is an experimental flag that changes the struct API to return int instead of long when possible. This is often a significant performance improvement. */ @@ -24,7 +25,6 @@ /* The translation function for each format character is table driven */ - typedef struct _formatdef { char format; int size; @@ -1315,50 +1315,36 @@ return s_unpack_internal(soself, buffer + offset); } -PyDoc_STRVAR(s_pack__doc__, -"pack(v1, v2, ...) -> string\n\ -\n\ -Return a string containing values v1, v2, ... packed according to this\n\ -Struct's format. See struct.__doc__ for more on format strings."); -static PyObject * -s_pack(PyObject *self, PyObject *args) +/* + * Guts of the pack function. + * + * Takes a struct object, a tuple of arguments, and offset in that tuple of + * argument for where to start processing the arguments for packing, and a + * character buffer for writing the packed string. The caller must insure + * that the buffer may contain the required length for packing the arguments. + * 0 is returned on success, 1 is returned if there is an error. + * + */ +static int +s_pack_internal(PyStructObject *soself, PyObject *args, int offset, char* buf) { - PyStructObject *soself; - PyObject *result; - char *restart; formatcode *code; Py_ssize_t i; - - soself = (PyStructObject *)self; - assert(PyStruct_Check(self)); - assert(soself->s_codes != NULL); - if (args == NULL || !PyTuple_Check(args) || - PyTuple_GET_SIZE(args) != soself->s_len) - { - PyErr_Format(StructError, - "pack requires exactly %d arguments", soself->s_len); - return NULL; - } - - result = PyString_FromStringAndSize((char *)NULL, soself->s_size); - if (result == NULL) - return NULL; - - restart = PyString_AS_STRING(result); - memset(restart, '\0', soself->s_size); - i = 0; + + memset(buf, '\0', soself->s_size); + i = offset; for (code = soself->s_codes; code->fmtdef != NULL; code++) { Py_ssize_t n; PyObject *v; const formatdef *e = code->fmtdef; - char *res = restart + code->offset; + char *res = buf + code->offset; if (e->format == 's') { v = PyTuple_GET_ITEM(args, i++); if (!PyString_Check(v)) { PyErr_SetString(StructError, "argument for 's' must be a string"); - goto fail; + return -1; } n = PyString_GET_SIZE(v); if (n > code->size) @@ -1370,7 +1356,7 @@ if (!PyString_Check(v)) { PyErr_SetString(StructError, "argument for 'p' must be a string"); - goto fail; + return -1; } n = PyString_GET_SIZE(v); if (n > (code->size - 1)) @@ -1383,16 +1369,109 @@ } else { v = PyTuple_GET_ITEM(args, i++); if (e->pack(res, v, e) < 0) - goto fail; + return -1; } } + /* Success */ + return 0; +} + + +PyDoc_STRVAR(s_pack__doc__, +"pack(v1, v2, ...) -> string\n\ +\n\ +Return a string containing values v1, v2, ... packed according to this\n\ +Struct's format. See struct.__doc__ for more on format strings."); + +static PyObject * +s_pack(PyObject *self, PyObject *args) +{ + PyStructObject *soself; + PyObject *result; + + /* Validate arguments. */ + soself = (PyStructObject *)self; + assert(PyStruct_Check(self)); + assert(soself->s_codes != NULL); + if (args == NULL || !PyTuple_Check(args) || + PyTuple_GET_SIZE(args) != soself->s_len) + { + PyErr_Format(StructError, + "pack requires exactly %d arguments", soself->s_len); + return NULL; + } + + /* Allocate a new string */ + result = PyString_FromStringAndSize((char *)NULL, soself->s_size); + if (result == NULL) + return NULL; + + /* Call the guts */ + if ( s_pack_internal(soself, args, 0, PyString_AS_STRING(result)) != 0 ) { + Py_DECREF(result); + return NULL; + } + return result; +} -fail: - Py_DECREF(result); - return NULL; +PyDoc_STRVAR(s_pack_to__doc__, +"pack_to(buffer, offset, v1, v2, ...)\n\ +\n\ +Pack the values v2, v2, ... according to this Struct's format, write \n\ +the packed bytes into the given buffer at the given offset. Note that \n\ +the offset is not an optional argument. See struct.__doc__ for \n\ +more on format strings."); + +static PyObject * +s_pack_to(PyObject *self, PyObject *args) +{ + PyStructObject *soself; + char *buffer; + Py_ssize_t buffer_len, offset; + + /* Validate arguments. +1 is for the first arg as buffer. */ + soself = (PyStructObject *)self; + assert(PyStruct_Check(self)); + assert(soself->s_codes != NULL); + if (args == NULL || !PyTuple_Check(args) || + PyTuple_GET_SIZE(args) != (soself->s_len + 2)) + { + PyErr_Format(StructError, + "pack_to requires exactly %d arguments", + (soself->s_len + 2)); + return NULL; + } + + /* Extract a writable memory buffer from the first argument */ + if ( PyObject_AsWriteBuffer(PyTuple_GET_ITEM(args, 0), + (void**)&buffer, &buffer_len) == -1 ) { + return NULL; + } + assert( buffer_len >= 0 ); + + /* Extract the offset from the first argument */ + offset = PyInt_AsLong(PyTuple_GET_ITEM(args, 1)); + + /* Support negative offsets. */ + if (offset < 0) + offset += buffer_len; + + /* Check boundaries */ + if (offset < 0 || (buffer_len - offset) < soself->s_size) { + PyErr_Format(StructError, + "pack_to requires a buffer of at least %d bytes", + soself->s_size); + return NULL; + } + /* Call the guts */ + if ( s_pack_internal(soself, args, 2, buffer + offset) != 0 ) { + return NULL; + } + + return Py_None; } @@ -1400,6 +1479,7 @@ static struct PyMethodDef s_methods[] = { {"pack", (PyCFunction)s_pack, METH_VARARGS, s_pack__doc__}, + {"pack_to", (PyCFunction)s_pack_to, METH_VARARGS, s_pack_to__doc__}, {"unpack", (PyCFunction)s_unpack, METH_O, s_unpack__doc__}, {"unpack_from", (PyCFunction)s_unpack_from, METH_KEYWORDS, s_unpack_from__doc__}, {NULL, NULL} /* sentinel */ Modified: python/trunk/Modules/arraymodule.c ============================================================================== --- python/trunk/Modules/arraymodule.c (original) +++ python/trunk/Modules/arraymodule.c Fri May 26 14:03:27 2006 @@ -1975,9 +1975,9 @@ 0, /* tp_setattr */ 0, /* tp_compare */ (reprfunc)array_repr, /* tp_repr */ - 0, /* tp_as _number*/ - &array_as_sequence, /* tp_as _sequence*/ - &array_as_mapping, /* tp_as _mapping*/ + 0, /* tp_as_number*/ + &array_as_sequence, /* tp_as_sequence*/ + &array_as_mapping, /* tp_as_mapping*/ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ Modified: python/trunk/Modules/socketmodule.c ============================================================================== --- python/trunk/Modules/socketmodule.c (original) +++ python/trunk/Modules/socketmodule.c Fri May 26 14:03:27 2006 @@ -104,7 +104,10 @@ listen(n) -- start listening for incoming connections\n\ makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ recv(buflen[, flags]) -- receive data\n\ -recvfrom(buflen[, flags]) -- receive data and sender's address\n\ +recv_buf(buffer[, nbytes[, flags]]) -- receive data (into a buffer)\n\ +recvfrom(buflen[, flags]) -- receive data and sender\'s address\n\ +recvfrom_buf(buffer[, nbytes, [, flags])\n\ + -- receive data and sender\'s address (into a buffer)\n\ sendall(data[, flags]) -- send all data\n\ send(data[, flags]) -- send data, may not send all of it\n\ sendto(data[, flags], addr) -- send data to a given address\n\ @@ -205,7 +208,7 @@ functions are declared correctly if compiling with MIPSPro 7.x in ANSI C mode (default) */ -/* XXX Using _SGIAPI is the wrong thing, +/* XXX Using _SGIAPI is the wrong thing, but I don't know what the right thing is. */ #undef _SGIAPI /* to avoid warning */ #define _SGIAPI 1 @@ -223,8 +226,8 @@ #include <netdb.h> #endif -/* Irix 6.5 fails to define this variable at all. This is needed - for both GCC and SGI's compiler. I'd say that the SGI headers +/* Irix 6.5 fails to define this variable at all. This is needed + for both GCC and SGI's compiler. I'd say that the SGI headers are just busted. Same thing for Solaris. */ #if (defined(__sgi) || defined(sun)) && !defined(INET_ADDRSTRLEN) #define INET_ADDRSTRLEN 16 @@ -1194,10 +1197,10 @@ args->ob_type->tp_name); return 0; } - if (!PyArg_ParseTuple(args, "eti:getsockaddrarg", + if (!PyArg_ParseTuple(args, "eti:getsockaddrarg", "idna", &host, &port)) return 0; - result = setipaddr(host, (struct sockaddr *)addr, + result = setipaddr(host, (struct sockaddr *)addr, sizeof(*addr), AF_INET); PyMem_Free(host); if (result < 0) @@ -1225,12 +1228,12 @@ args->ob_type->tp_name); return 0; } - if (!PyArg_ParseTuple(args, "eti|ii", + if (!PyArg_ParseTuple(args, "eti|ii", "idna", &host, &port, &flowinfo, &scope_id)) { return 0; } - result = setipaddr(host, (struct sockaddr *)addr, + result = setipaddr(host, (struct sockaddr *)addr, sizeof(*addr), AF_INET6); PyMem_Free(host); if (result < 0) @@ -1839,7 +1842,7 @@ int res_size = sizeof res; /* It must be in the exception set */ assert(FD_ISSET(s->sock_fd, &fds_exc)); - if (0 == getsockopt(s->sock_fd, SOL_SOCKET, SO_ERROR, + if (0 == getsockopt(s->sock_fd, SOL_SOCKET, SO_ERROR, (char *)&res, &res_size)) /* getsockopt also clears WSAGetLastError, so reset it back. */ @@ -2135,95 +2138,126 @@ #endif /* NO_DUP */ - -/* s.recv(nbytes [,flags]) method */ - -static PyObject * -sock_recv(PySocketSockObject *s, PyObject *args) +/* + * This is the guts of the recv() and recv_buf() methods, which reads into a + * char buffer. If you have any inc/def ref to do to the objects that contain + * the buffer, do it in the caller. This function returns the number of bytes + * succesfully read. If there was an error, it returns -1. Note that it is + * also possible that we return a number of bytes smaller than the request + * bytes. + */ +static int +sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) { - int len, n = 0, flags = 0, timeout; - PyObject *buf; + int timeout, outlen = 0; #ifdef __VMS - int read_length; + int remaining, nread; char *read_buf; #endif - if (!PyArg_ParseTuple(args, "i|i:recv", &len, &flags)) - return NULL; - - if (len < 0) { - PyErr_SetString(PyExc_ValueError, - "negative buffersize in recv"); - return NULL; + if (!IS_SELECTABLE(s)) { + select_error(); + return -1; } - buf = PyString_FromStringAndSize((char *) 0, len); - if (buf == NULL) - return NULL; - - if (!IS_SELECTABLE(s)) - return select_error(); - #ifndef __VMS Py_BEGIN_ALLOW_THREADS timeout = internal_select(s, 0); if (!timeout) - n = recv(s->sock_fd, PyString_AS_STRING(buf), len, flags); + outlen = recv(s->sock_fd, cbuf, len, flags); Py_END_ALLOW_THREADS if (timeout) { - Py_DECREF(buf); PyErr_SetString(socket_timeout, "timed out"); - return NULL; + return -1; } - if (n < 0) { - Py_DECREF(buf); - return s->errorhandler(); + if (outlen < 0) { + /* Note: the call to errorhandler() ALWAYS indirectly returned + NULL, so ignore its return value */ + s->errorhandler(); + return -1; } - if (n != len) - _PyString_Resize(&buf, n); #else - read_buf = PyString_AsString(buf); - read_length = len; - while (read_length != 0) { + read_buf = cbuf; + remaining = len; + while (remaining != 0) { unsigned int segment; - segment = read_length /SEGMENT_SIZE; + segment = remaining /SEGMENT_SIZE; if (segment != 0) { segment = SEGMENT_SIZE; } else { - segment = read_length; + segment = remaining; } Py_BEGIN_ALLOW_THREADS timeout = internal_select(s, 0); if (!timeout) - n = recv(s->sock_fd, read_buf, segment, flags); + nread = recv(s->sock_fd, read_buf, segment, flags); Py_END_ALLOW_THREADS if (timeout) { - Py_DECREF(buf); PyErr_SetString(socket_timeout, "timed out"); - return NULL; + return -1; } - if (n < 0) { - Py_DECREF(buf); - return s->errorhandler(); + if (nread < 0) { + s->errorhandler(); + return -1; } - if (n != read_length) { - read_buf += n; + if (nread != remaining) { + read_buf += nread; break; } - read_length -= segment; + remaining -= segment; read_buf += segment; } - if (_PyString_Resize(&buf, (read_buf - PyString_AsString(buf))) < 0) - { - return NULL; - } + outlen = read_buf - cbuf; #endif /* !__VMS */ + + return outlen; +} + + +/* s.recv(nbytes [,flags]) method */ + +static PyObject * +sock_recv(PySocketSockObject *s, PyObject *args) +{ + int recvlen, flags = 0, outlen; + PyObject *buf; + + if (!PyArg_ParseTuple(args, "i|i:recv", &recvlen, &flags)) + return NULL; + + if (recvlen < 0) { + PyErr_SetString(PyExc_ValueError, + "negative buffersize in recv"); + return NULL; + } + + /* Allocate a new string. */ + buf = PyString_FromStringAndSize((char *) 0, recvlen); + if (buf == NULL) + return NULL; + + /* Call the guts */ + outlen = sock_recv_guts(s, PyString_AsString(buf), recvlen, flags); + if (outlen < 0) { + /* An error occured, release the string and return an + error. */ + Py_DECREF(buf); + return NULL; + } + if (outlen != recvlen) { + /* We did not read as many bytes as we anticipated, resize the + string if possible and be succesful. */ + if (_PyString_Resize(&buf, outlen) < 0) + /* Oopsy, not so succesful after all. */ + return NULL; + } + return buf; } @@ -2236,29 +2270,90 @@ the remote end is closed and all data is read, return the empty string."); -/* s.recvfrom(nbytes [,flags]) method */ +/* s.recv_buf(buffer, [nbytes [,flags]]) method */ -static PyObject * -sock_recvfrom(PySocketSockObject *s, PyObject *args) +static PyObject* +sock_recv_buf(PySocketSockObject *s, PyObject *args, PyObject *kwds) { - sock_addr_t addrbuf; - PyObject *buf = NULL; - PyObject *addr = NULL; - PyObject *ret = NULL; - int len, n = 0, flags = 0, timeout; - socklen_t addrlen; + static char *kwlist[] = {"buffer", "nbytes", "flags", 0}; - if (!PyArg_ParseTuple(args, "i|i:recvfrom", &len, &flags)) + int recvlen = 0, flags = 0, readlen; + char *buf; + int buflen; + + /* Get the buffer's memory */ + if (!PyArg_ParseTupleAndKeywords(args, kwds, "s#|ii:recv", kwlist, + &buf, &buflen, &recvlen, &flags)) return NULL; + assert(buf != 0 && buflen > 0); - if (!getsockaddrlen(s, &addrlen)) + if (recvlen < 0) { + PyErr_SetString(PyExc_ValueError, + "negative buffersize in recv"); return NULL; - buf = PyString_FromStringAndSize((char *) 0, len); - if (buf == NULL) + } + if (recvlen == 0) { + /* If nbytes was not specified, use the buffer's length */ + recvlen = buflen; + } + + /* Check if the buffer is large enough */ + if (buflen < recvlen) { + PyErr_SetString(PyExc_ValueError, + "buffer too small for requested bytes"); return NULL; + } - if (!IS_SELECTABLE(s)) - return select_error(); + /* Call the guts */ + readlen = sock_recv_guts(s, buf, recvlen, flags); + if (readlen < 0) { + /* Return an error. */ + return NULL; + } + + /* Return the number of bytes read. Note that we do not do anything + special here in the case that readlen < recvlen. */ + return PyInt_FromLong(readlen); +} + +PyDoc_STRVAR(recv_buf_doc, +"recv_buf(buffer, [nbytes[, flags]]) -> nbytes_read\n\ +\n\ +A version of recv() that stores its data into a buffer rather than creating \n\ +a new string. Receive up to buffersize bytes from the socket. If buffersize \n\ +is not specified (or 0), receive up to the size available in the given buffer.\n\ +\n\ +See recv() for documentation about the flags."); + + +/* + * This is the guts of the recv() and recv_buf() methods, which reads into a + * char buffer. If you have any inc/def ref to do to the objects that contain + * the buffer, do it in the caller. This function returns the number of bytes + * succesfully read. If there was an error, it returns -1. Note that it is + * also possible that we return a number of bytes smaller than the request + * bytes. + * + * 'addr' is a return value for the address object. Note that you must decref + * it yourself. + */ +static int +sock_recvfrom_guts(PySocketSockObject *s, char* cbuf, int len, int flags, + PyObject** addr) +{ + sock_addr_t addrbuf; + int n = 0, timeout; + socklen_t addrlen; + + *addr = NULL; + + if (!getsockaddrlen(s, &addrlen)) + return -1; + + if (!IS_SELECTABLE(s)) { + select_error(); + return -1; + } Py_BEGIN_ALLOW_THREADS memset(&addrbuf, 0, addrlen); @@ -2266,41 +2361,71 @@ if (!timeout) { #ifndef MS_WINDOWS #if defined(PYOS_OS2) && !defined(PYCC_GCC) - n = recvfrom(s->sock_fd, PyString_AS_STRING(buf), len, flags, + n = recvfrom(s->sock_fd, cbuf, len, flags, (struct sockaddr *) &addrbuf, &addrlen); #else - n = recvfrom(s->sock_fd, PyString_AS_STRING(buf), len, flags, + n = recvfrom(s->sock_fd, cbuf, len, flags, (void *) &addrbuf, &addrlen); #endif #else - n = recvfrom(s->sock_fd, PyString_AS_STRING(buf), len, flags, + n = recvfrom(s->sock_fd, cbuf, len, flags, (struct sockaddr *) &addrbuf, &addrlen); #endif } Py_END_ALLOW_THREADS if (timeout) { - Py_DECREF(buf); PyErr_SetString(socket_timeout, "timed out"); - return NULL; + return -1; } if (n < 0) { - Py_DECREF(buf); - return s->errorhandler(); + s->errorhandler(); + return -1; } - if (n != len && _PyString_Resize(&buf, n) < 0) + if (!(*addr = makesockaddr(s->sock_fd, (struct sockaddr *) &addrbuf, + addrlen, s->sock_proto))) + return -1; + + return n; +} + +/* s.recvfrom(nbytes [,flags]) method */ + +static PyObject * +sock_recvfrom(PySocketSockObject *s, PyObject *args) +{ + PyObject *buf = NULL; + PyObject *addr = NULL; + PyObject *ret = NULL; + int recvlen, outlen, flags = 0; + + if (!PyArg_ParseTuple(args, "i|i:recvfrom", &recvlen, &flags)) + return NULL; + + buf = PyString_FromStringAndSize((char *) 0, recvlen); + if (buf == NULL) return NULL; - if (!(addr = makesockaddr(s->sock_fd, (struct sockaddr *) &addrbuf, - addrlen, s->sock_proto))) + outlen = sock_recvfrom_guts(s, PyString_AS_STRING(buf), + recvlen, flags, &addr); + if (outlen < 0) { goto finally; + } + + if (outlen != recvlen) { + /* We did not read as many bytes as we anticipated, resize the + string if possible and be succesful. */ + if (_PyString_Resize(&buf, outlen) < 0) + /* Oopsy, not so succesful after all. */ + goto finally; + } ret = PyTuple_Pack(2, buf, addr); finally: - Py_XDECREF(addr); Py_XDECREF(buf); + Py_XDECREF(addr); return ret; } @@ -2309,6 +2434,57 @@ \n\ Like recv(buffersize, flags) but also return the sender's address info."); + +/* s.recvfrom_buf(buffer[, nbytes [,flags]]) method */ + +static PyObject * +sock_recvfrom_buf(PySocketSockObject *s, PyObject *args, PyObject* kwds) +{ + static char *kwlist[] = {"buffer", "nbytes", "flags", 0}; + + int recvlen = 0, flags = 0, readlen; + char *buf; + int buflen; + + PyObject *addr = NULL; + PyObject *ret = NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "s#|ii:recvfrom", kwlist, + &buf, &buflen, &recvlen, &flags)) + return NULL; + assert(buf != 0 && buflen > 0); + + if (recvlen < 0) { + PyErr_SetString(PyExc_ValueError, + "negative buffersize in recv"); + return NULL; + } + if (recvlen == 0) { + /* If nbytes was not specified, use the buffer's length */ + recvlen = buflen; + } + + readlen = sock_recvfrom_guts(s, buf, recvlen, flags, &addr); + if (readlen < 0) { + /* Return an error */ + goto finally; + } + + /* Return the number of bytes read and the address. Note that we do + not do anything special here in the case that readlen < recvlen. */ + ret = PyTuple_Pack(2, PyInt_FromLong(readlen), addr); + +finally: + Py_XDECREF(addr); + return ret; +} + +PyDoc_STRVAR(recvfrom_buf_doc, +"recvfrom_buf(buffer[, nbytes[, flags]]) -> (nbytes, address info)\n\ +\n\ +Like recv_buf(buffer[, nbytes[, flags]]) but also return the sender's address info."); + + /* s.send(data [,flags]) method */ static PyObject * @@ -2503,59 +2679,63 @@ /* List of methods for socket objects */ static PyMethodDef sock_methods[] = { - {"accept", (PyCFunction)sock_accept, METH_NOARGS, - accept_doc}, - {"bind", (PyCFunction)sock_bind, METH_O, - bind_doc}, - {"close", (PyCFunction)sock_close, METH_NOARGS, - close_doc}, - {"connect", (PyCFunction)sock_connect, METH_O, - connect_doc}, - {"connect_ex", (PyCFunction)sock_connect_ex, METH_O, - connect_ex_doc}, + {"accept", (PyCFunction)sock_accept, METH_NOARGS, + accept_doc}, + {"bind", (PyCFunction)sock_bind, METH_O, + bind_doc}, + {"close", (PyCFunction)sock_close, METH_NOARGS, + close_doc}, + {"connect", (PyCFunction)sock_connect, METH_O, + connect_doc}, + {"connect_ex", (PyCFunction)sock_connect_ex, METH_O, + connect_ex_doc}, #ifndef NO_DUP - {"dup", (PyCFunction)sock_dup, METH_NOARGS, - dup_doc}, + {"dup", (PyCFunction)sock_dup, METH_NOARGS, + dup_doc}, #endif - {"fileno", (PyCFunction)sock_fileno, METH_NOARGS, - fileno_doc}, + {"fileno", (PyCFunction)sock_fileno, METH_NOARGS, + fileno_doc}, #ifdef HAVE_GETPEERNAME - {"getpeername", (PyCFunction)sock_getpeername, - METH_NOARGS, getpeername_doc}, + {"getpeername", (PyCFunction)sock_getpeername, + METH_NOARGS, getpeername_doc}, #endif - {"getsockname", (PyCFunction)sock_getsockname, - METH_NOARGS, getsockname_doc}, - {"getsockopt", (PyCFunction)sock_getsockopt, METH_VARARGS, - getsockopt_doc}, - {"listen", (PyCFunction)sock_listen, METH_O, - listen_doc}, + {"getsockname", (PyCFunction)sock_getsockname, + METH_NOARGS, getsockname_doc}, + {"getsockopt", (PyCFunction)sock_getsockopt, METH_VARARGS, + getsockopt_doc}, + {"listen", (PyCFunction)sock_listen, METH_O, + listen_doc}, #ifndef NO_DUP - {"makefile", (PyCFunction)sock_makefile, METH_VARARGS, - makefile_doc}, + {"makefile", (PyCFunction)sock_makefile, METH_VARARGS, + makefile_doc}, #endif - {"recv", (PyCFunction)sock_recv, METH_VARARGS, - recv_doc}, - {"recvfrom", (PyCFunction)sock_recvfrom, METH_VARARGS, - recvfrom_doc}, - {"send", (PyCFunction)sock_send, METH_VARARGS, - send_doc}, - {"sendall", (PyCFunction)sock_sendall, METH_VARARGS, - sendall_doc}, - {"sendto", (PyCFunction)sock_sendto, METH_VARARGS, - sendto_doc}, - {"setblocking", (PyCFunction)sock_setblocking, METH_O, - setblocking_doc}, - {"settimeout", (PyCFunction)sock_settimeout, METH_O, - settimeout_doc}, - {"gettimeout", (PyCFunction)sock_gettimeout, METH_NOARGS, - gettimeout_doc}, - {"setsockopt", (PyCFunction)sock_setsockopt, METH_VARARGS, - setsockopt_doc}, - {"shutdown", (PyCFunction)sock_shutdown, METH_O, - shutdown_doc}, + {"recv", (PyCFunction)sock_recv, METH_VARARGS, + recv_doc}, + {"recv_buf", (PyCFunction)sock_recv_buf, METH_VARARGS | METH_KEYWORDS, + recv_buf_doc}, + {"recvfrom", (PyCFunction)sock_recvfrom, METH_VARARGS, + recvfrom_doc}, + {"recvfrom_buf", (PyCFunction)sock_recvfrom_buf, METH_VARARGS | METH_KEYWORDS, + recvfrom_buf_doc}, + {"send", (PyCFunction)sock_send, METH_VARARGS, + send_doc}, + {"sendall", (PyCFunction)sock_sendall, METH_VARARGS, + sendall_doc}, + {"sendto", (PyCFunction)sock_sendto, METH_VARARGS, + sendto_doc}, + {"setblocking", (PyCFunction)sock_setblocking, METH_O, + setblocking_doc}, + {"settimeout", (PyCFunction)sock_settimeout, METH_O, + settimeout_doc}, + {"gettimeout", (PyCFunction)sock_gettimeout, METH_NOARGS, + gettimeout_doc}, + {"setsockopt", (PyCFunction)sock_setsockopt, METH_VARARGS, + setsockopt_doc}, + {"shutdown", (PyCFunction)sock_shutdown, METH_O, + shutdown_doc}, #ifdef RISCOS - {"sleeptaskw", (PyCFunction)sock_sleeptaskw, METH_O, - sleeptaskw_doc}, + {"sleeptaskw", (PyCFunction)sock_sleeptaskw, METH_O, + sleeptaskw_doc}, #endif {NULL, NULL} /* sentinel */ }; @@ -3401,7 +3581,7 @@ if (strcmp(ip_addr, "255.255.255.255") == 0) { packed_addr = 0xFFFFFFFF; } else { - + packed_addr = inet_addr(ip_addr); if (packed_addr == INADDR_NONE) { /* invalid address */ @@ -3476,7 +3656,7 @@ "can't use AF_INET6, IPv6 is disabled"); return NULL; } -#endif +#endif retval = inet_pton(af, ip, packed); if (retval < 0) { @@ -3499,7 +3679,7 @@ return NULL; } } - + PyDoc_STRVAR(inet_ntop_doc, "inet_ntop(af, packed_ip) -> string formatted IP address\n\ \n\ @@ -3517,7 +3697,7 @@ #else char ip[INET_ADDRSTRLEN + 1]; #endif - + /* Guarantee NUL-termination for PyString_FromString() below */ memset((void *) &ip[0], '\0', sizeof(ip)); @@ -3595,7 +3775,7 @@ } else if (PyString_Check(hobj)) { hptr = PyString_AsString(hobj); } else { - PyErr_SetString(PyExc_TypeError, + PyErr_SetString(PyExc_TypeError, "getaddrinfo() argument 1 must be string or None"); return NULL; } [View Less]

7 9

Re: [Python-checkins] r46520 - in python/trunk: Doc/Makefile.deps Doc/lib/lib.tex Doc/lib/libfunctional.tex Doc/lib/libfunctools.tex Lib/test/test_functional.py Lib/test/test_functools.py Misc/NEWS Modules/_functoolsmodule.c Modules/functionalmodule.
by Tim Peters May 31, 2006

May 31, 2006

[Nick Coghlan] > ... (we should probably do something about that misleading ImportError -> > TestSkipped -> green buildbot behaviour. . . ) I looked at that briefly a few weeks back and gave up. Seemed the sanest thing was to entirely stop treating ImportError as "test skipped", and rewrite tests that legimately _may_ get skipped to catch expected ImportErrors and change them to TestSkipped themselves. A bit of framework might help; e.g., a test that expects to get skipped due to … [View More]

2 3

r46583 - sandbox/trunk/structbench/structformats.txt
by bob.ippolito May 31, 2006

May 31, 2006

Author: bob.ippolito Date: Wed May 31 20:19:56 2006 New Revision: 46583 Added: sandbox/trunk/structbench/structformats.txt (contents, props changed) Log: text file listing the sources for the formats that I compiled the benchmark from Added: sandbox/trunk/structbench/structformats.txt ============================================================================== --- (empty file) +++ sandbox/trunk/structbench/structformats.txt Wed May 31 20:19:56 2006 @@ -0,0 +1,75 @@ +# pickletools +<… [View More]

1 0

buildbot warnings in hppa Ubuntu dapper trunk
by buildbot＠python.org May 31, 2006

May 31, 2006

The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk… Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in alpha Debian trunk
by buildbot＠python.org May 31, 2006

May 31, 2006

The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/255 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,kristjan.jonsson,tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

r46582 - python/trunk/Modules/_struct.c
by tim.peters May 31, 2006

May 31, 2006

Author: tim.peters Date: Wed May 31 17:34:37 2006 New Revision: 46582 Modified: python/trunk/Modules/_struct.c Log: Repaired error in new comment. Modified: python/trunk/Modules/_struct.c ============================================================================== --- python/trunk/Modules/_struct.c (original) +++ python/trunk/Modules/_struct.c Wed May 31 17:34:37 2006 @@ -311,10 +311,10 @@ /* ulargest is the largest unsigned value with f->size bytes. * Note that the simpler: *… [View More]

1 0