Mailman 3 September 2011 - Python-checkins

cpython (3.2): From RFC 3629 5- and 6-bytes UTF-8 sequences are invalid, so remove them from
by ezio.melotti 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/106c57d52b47 changeset: 72160:106c57d52b47 branch: 3.2 parent: 72158:65bd22be72dc user: Ezio Melotti <ezio.melotti(a)gmail.com> date: Thu Sep 01 08:11:28 2011 +0300 summary: From RFC 3629 5- and 6-bytes UTF-8 sequences are invalid, so remove them from the doc. files: Doc/library/codecs.rst | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -839,7 +839,7 @@ characters: UTF-8. UTF-8 is an 8-bit encoding, which means there are no issues with byte order in UTF-8. Each byte in a UTF-8 byte sequence consists of two parts: Marker bits (the most significant bits) and payload bits. The marker bits -are a sequence of zero to six 1 bits followed by a 0 bit. Unicode characters are +are a sequence of zero to four ``1`` bits followed by a ``0`` bit. Unicode characters are encoded like this (with x being payload bits, which when concatenated give the Unicode character): @@ -852,12 +852,7 @@ +-----------------------------------+----------------------------------------------+ | ``U-00000800`` ... ``U-0000FFFF`` | 1110xxxx 10xxxxxx 10xxxxxx | +-----------------------------------+----------------------------------------------+ -| ``U-00010000`` ... ``U-001FFFFF`` | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | -+-----------------------------------+----------------------------------------------+ -| ``U-00200000`` ... ``U-03FFFFFF`` | 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx | -+-----------------------------------+----------------------------------------------+ -| ``U-04000000`` ... ``U-7FFFFFFF`` | 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx | -| | 10xxxxxx | +| ``U-00010000`` ... ``U-0010FFFF`` | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | +-----------------------------------+----------------------------------------------+ The least significant bit of the Unicode character is the rightmost x bit. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Remove duplicate text in os documentation.
by ross.lagerwall 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/65bd22be72dc changeset: 72158:65bd22be72dc branch: 3.2 parent: 72154:c8b6d4417274 user: Ross Lagerwall <rosslagerwall(a)gmail.com> date: Thu Sep 01 06:58:52 2011 +0200 summary: Remove duplicate text in os documentation. files: Doc/library/os.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -29,11 +29,6 @@ objects, and result in an object of the same type, if a path or file name is returned. -.. note:: - - If not separately noted, all functions that claim "Availability: Unix" are - supported on Mac OS X, which builds on a Unix core. - * An "Availability: Unix" note means that this function is commonly found on Unix systems. It does not make any claims about its existence on a specific operating system. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> default): Merge with 3.2.
by ross.lagerwall 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/e5c2d623c20b changeset: 72159:e5c2d623c20b parent: 72157:74671651540d parent: 72158:65bd22be72dc user: Ross Lagerwall <rosslagerwall(a)gmail.com> date: Thu Sep 01 07:02:31 2011 +0200 summary: Merge with 3.2. files: Doc/library/os.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -29,11 +29,6 @@ objects, and result in an object of the same type, if a path or file name is returned. -.. note:: - - If not separately noted, all functions that claim "Availability: Unix" are - supported on Mac OS X, which builds on a Unix core. - * An "Availability: Unix" note means that this function is commonly found on Unix systems. It does not make any claims about its existence on a specific operating system. -- Repository URL: http://hg.python.org/cpython

1 0

Daily reference leaks (0968acf0e6db): sum=0
by solipsis＠pitrou.net 01 Sep '11

01 Sep '11

results for 0968acf0e6db on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogjZSCdh', '-x']

1 0

cpython: news note on xattrs
by benjamin.peterson 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/74671651540d changeset: 72157:74671651540d user: Benjamin Peterson <benjamin(a)python.org> date: Wed Aug 31 22:21:44 2011 -0400 summary: news note on xattrs files: Misc/NEWS | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -268,6 +268,8 @@ Library ------- +- Issue #12720: Expose low-level Linux extended file attribute functions in os. + - Issue #10946: The distutils commands bdist_dumb, bdist_wininst and bdist_msi now respect a --skip-build option given to bdist. The packaging commands were fixed too. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> default): merge 3.2 (null)
by benjamin.peterson 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/cfccc755c457 changeset: 72156:cfccc755c457 parent: 72155:4eb0b1819bda parent: 72154:c8b6d4417274 user: Benjamin Peterson <benjamin(a)python.org> date: Wed Aug 31 22:16:34 2011 -0400 summary: merge 3.2 (null) files: -- Repository URL: http://hg.python.org/cpython

1 0

cpython: expose linux extended file system attributes (closes #12720)
by benjamin.peterson 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/4eb0b1819bda changeset: 72155:4eb0b1819bda parent: 72152:0968acf0e6db user: Benjamin Peterson <benjamin(a)python.org> date: Wed Aug 31 22:15:17 2011 -0400 summary: expose linux extended file system attributes (closes #12720) files: Doc/library/os.rst | 148 +++++++++++ Lib/test/test_os.py | 94 +++++++ Modules/posixmodule.c | 402 ++++++++++++++++++++++++++++++ configure | 2 +- configure.in | 2 +- pyconfig.h.in | 3 + 6 files changed, 649 insertions(+), 2 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -751,6 +751,26 @@ This function is not available on MacOS. +.. function:: fgetxattr(fd, attr) + + This works exactly like :func:`getxattr` but operates on a file descriptor, + *fd*, instead of a path. + + Availability: Linux + + .. versionadded:: 3.3 + + +.. function:: flistxattr(fd) + + This is exactly like :func:`listxattr` but operates on a file descriptor, + *fd*, instead of a path. + + Availability: Linux + + .. versionadded:: 3.3 + + .. function:: fdlistdir(fd) Like :func:`listdir`, but uses a file descriptor instead and always returns @@ -836,6 +856,27 @@ Availability: Unix. +.. function:: fremovexattr(fd, attr) + + This works exactly like :func:`removexattr` but operates on a file + descriptor, *fd*, instead of a path. + + Availability: Linux + + .. versionadded:: 3.3 + + +.. function:: fsetxattr(fd, attr, value, flags=0) + + This works exactly like :func:`setxattr` but on a file descriptor, *fd*, + instead of a path. + + + Availability: Linux + + .. versionadded:: 3.3 + + .. function:: futimesat(dirfd, path, (atime, mtime)) futimesat(dirfd, path, None) @@ -1536,6 +1577,17 @@ Availability: Unix. +.. function:: getxattr(path, attr) + + Return the value of the extended filesystem attribute *attr* for + *path*. *attr* can be bytes or str. If it is str, it is encoded with the + filesystem encoding. + + Availability: Linux + + .. versionadded:: 3.3 + + .. function:: lchflags(path, flags) Set the flags of *path* to the numeric *flags*, like :func:`chflags`, but do not @@ -1561,6 +1613,15 @@ Availability: Unix. +.. function:: lgetxattr(path, attr) + + This works exactly like :func:`getxattr` but doesn't follow symlinks. + + Availability: Linux + + .. versionadded:: 3.3 + + .. function:: link(source, link_name) Create a hard link pointing to *source* named *link_name*. @@ -1585,6 +1646,44 @@ .. versionchanged:: 3.2 The *path* parameter became optional. + +.. function:: listxattr(path) + + Return a list of the extended filesystem attributes on *path*. Attributes are + returned as string decoded with the filesystem encoding. + + Availability: Linux + + .. versionadded:: 3.3 + + +.. function:: llistxattr(path) + + This works exactly like :func:`listxattr` but doesn't follow symlinks. + + Availability: Linux + + .. versionadded:: 3.3 + + +.. function:: lremoveattr(path, attr) + + This works exactly like :func:`removeattr` but doesn't follow symlinks. + + Availability: Linux + + .. versionadded:: 3.3 + + +.. function:: lsetxattr(path, attr, value, flags=0) + + This works exactly like :func:`setxattr` but doesn't follow symlinks. + + Availability: Linux + + .. versionadded:: 3.3 + + .. function:: lstat(path) Perform the equivalent of an :c:func:`lstat` system call on the given path. @@ -1758,6 +1857,17 @@ successfully removed. +.. function:: removexattr(path, attr) + + Removes the extended filesystem attribute *attr* from *path*. *attr* should + be bytes or str. If it is a string, it is encoded with the filesystem + encoding. + + Availability: Linux + + .. versionadded:: 3.3 + + .. function:: rename(src, dst) Rename the file or directory *src* to *dst*. If *dst* is a directory, @@ -1794,6 +1904,44 @@ Availability: Unix, Windows. +.. data:: XATTR_SIZE_MAX + + The maximum size the value of an extended attribute can be. Currently, this + is 64 kilobytes on Linux. + + +.. data:: XATTR_CREATE + + This is a possible value for the flags argument in :func:`setxattr`. It + indicates the operation must create an attribute. + + +.. data:: XATTR_REPLACE + + This is a possible value for the flags argument in :func:`setxattr`. It + indicates the operation must replace an existing attribute. + + +.. function:: setxattr(path, attr, value, flags=0) + + Set the extended filesystem attribute *attr* on *path* to *value*. *attr* + must be a bytes or str with no embedded NULs. If it is str, it is encoded + with the filesystem encoding. *flags* may be :data:`XATTR_REPLACE` or + :data:`XATTR_CREATE`. If :data:`XATTR_REPLACE` is given and the attribute + does not exist, ``EEXISTS`` will be raised. If :data:`XATTR_CREATE` is given + and the attribute already exists, the attribute will not be created and + ``ENODATA`` will be raised. + + Availability: Linux + + .. note:: + + A bug in Linux kernel versions less than 2.6.39 caused the flags argument + to be ignored on some filesystems. + + .. versionadded:: 3.3 + + .. function:: stat(path) Perform the equivalent of a :c:func:`stat` system call on the given path. diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -14,6 +14,8 @@ from test import support import contextlib import mmap +import platform +import re import uuid import asyncore import asynchat @@ -1506,6 +1508,97 @@ raise +def supports_extended_attributes(): + if not hasattr(os, "setxattr"): + return False + try: + with open(support.TESTFN, "wb") as fp: + try: + os.fsetxattr(fp.fileno(), b"user.test", b"") + except OSError as e: + if e.errno != errno.ENOTSUP: + raise + return False + finally: + support.unlink(support.TESTFN) + # Kernels < 2.6.39 don't respect setxattr flags. + kernel_version = platform.release() + m = re.match("2.6.(\d{1,2})", kernel_version) + return m is None or int(m.group(1)) >= 39 + + +(a)unittest.skipUnless(supports_extended_attributes(), + "no non-broken extended attribute support") +class ExtendedAttributeTests(unittest.TestCase): + + def tearDown(self): + support.unlink(support.TESTFN) + + def _check_xattrs_str(self, s, getxattr, setxattr, removexattr, listxattr): + fn = support.TESTFN + open(fn, "wb").close() + with self.assertRaises(OSError) as cm: + getxattr(fn, s("user.test")) + self.assertEqual(cm.exception.errno, errno.ENODATA) + self.assertEqual(listxattr(fn), []) + setxattr(fn, s("user.test"), b"") + self.assertEqual(listxattr(fn), ["user.test"]) + self.assertEqual(getxattr(fn, b"user.test"), b"") + setxattr(fn, s("user.test"), b"hello", os.XATTR_REPLACE) + self.assertEqual(getxattr(fn, b"user.test"), b"hello") + with self.assertRaises(OSError) as cm: + setxattr(fn, s("user.test"), b"bye", os.XATTR_CREATE) + self.assertEqual(cm.exception.errno, errno.EEXIST) + with self.assertRaises(OSError) as cm: + setxattr(fn, s("user.test2"), b"bye", os.XATTR_REPLACE) + self.assertEqual(cm.exception.errno, errno.ENODATA) + setxattr(fn, s("user.test2"), b"foo", os.XATTR_CREATE) + self.assertEqual(sorted(listxattr(fn)), ["user.test", "user.test2"]) + removexattr(fn, s("user.test")) + with self.assertRaises(OSError) as cm: + getxattr(fn, s("user.test")) + self.assertEqual(cm.exception.errno, errno.ENODATA) + self.assertEqual(listxattr(fn), ["user.test2"]) + self.assertEqual(getxattr(fn, s("user.test2")), b"foo") + setxattr(fn, s("user.test"), b"a"*1024) + self.assertEqual(getxattr(fn, s("user.test")), b"a"*1024) + removexattr(fn, s("user.test")) + many = sorted("user.test{}".format(i) for i in range(100)) + for thing in many: + setxattr(fn, thing, b"x") + self.assertEqual(sorted(listxattr(fn)), many) + + def _check_xattrs(self, *args): + def make_bytes(s): + return bytes(s, "ascii") + self._check_xattrs_str(str, *args) + support.unlink(support.TESTFN) + self._check_xattrs_str(make_bytes, *args) + + def test_simple(self): + self._check_xattrs(os.getxattr, os.setxattr, os.removexattr, + os.listxattr) + + def test_lpath(self): + self._check_xattrs(os.lgetxattr, os.lsetxattr, os.lremovexattr, + os.llistxattr) + + def test_fds(self): + def getxattr(path, *args): + with open(path, "rb") as fp: + return os.fgetxattr(fp.fileno(), *args) + def setxattr(path, *args): + with open(path, "wb") as fp: + os.fsetxattr(fp.fileno(), *args) + def removexattr(path, *args): + with open(path, "wb") as fp: + os.fremovexattr(fp.fileno(), *args) + def listxattr(path, *args): + with open(path, "rb") as fp: + return os.flistxattr(fp.fileno(), *args) + self._check_xattrs(getxattr, setxattr, removexattr, listxattr) + + @support.reap_threads def test_main(): support.run_unittest( @@ -1529,6 +1622,7 @@ LinkTests, TestSendfile, ProgramPriorityTests, + ExtendedAttributeTests, ) if __name__ == "__main__": diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -107,6 +107,10 @@ #include <sched.h> #endif +#ifdef HAVE_ATTR_XATTR_H +#include <attr/xattr.h> +#endif + #if defined(__FreeBSD__) || defined(__DragonFly__) || defined(__APPLE__) #ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> @@ -9950,6 +9954,384 @@ } #endif +#ifdef HAVE_ATTR_XATTR_H + +static int +try_getxattr(const char *path, const char *name, + ssize_t (*get)(const char *, const char *, void *, size_t), + Py_ssize_t buf_size, PyObject **res) +{ + PyObject *value; + Py_ssize_t len; + + assert(buf_size <= XATTR_SIZE_MAX); + value = PyBytes_FromStringAndSize(NULL, buf_size); + if (!value) + return 0; + Py_BEGIN_ALLOW_THREADS; + len = get(path, name, PyBytes_AS_STRING(value), buf_size); + Py_END_ALLOW_THREADS; + if (len < 0) { + Py_DECREF(value); + if (errno == ERANGE) { + value = NULL; + } + else { + posix_error(); + return 0; + } + } + else if (len != buf_size) { + /* Can only shrink. */ + _PyBytes_Resize(&value, len); + } + *res = value; + return 1; +} + +static PyObject * +getxattr_common(const char *path, PyObject *name_obj, + ssize_t (*get)(const char *, const char *, void *, size_t)) +{ + PyObject *value; + const char *name = PyBytes_AS_STRING(name_obj); + + /* Try a small value first. */ + if (!try_getxattr(path, name, get, 128, &value)) + return NULL; + if (value) + return value; + /* Now the maximum possible one. */ + if (!try_getxattr(path, name, get, XATTR_SIZE_MAX, &value)) + return NULL; + assert(value); + return value; +} + +PyDoc_STRVAR(posix_getxattr__doc__, +"getxattr(path, attr) -> value\n\n\ +Return the value of extended attribute *name* on *path*."); + +static PyObject * +posix_getxattr(PyObject *self, PyObject *args) +{ + PyObject *path, *res, *name; + + if (!PyArg_ParseTuple(args, "O&O&:getxattr", PyUnicode_FSConverter, &path, + PyUnicode_FSConverter, &name)) + return NULL; + res = getxattr_common(PyBytes_AS_STRING(path), name, getxattr); + Py_DECREF(path); + Py_DECREF(name); + return res; +} + +PyDoc_STRVAR(posix_lgetxattr__doc__, +"lgetxattr(path, attr) -> value\n\n\ +Like getxattr but don't follow symlinks."); + +static PyObject * +posix_lgetxattr(PyObject *self, PyObject *args) +{ + PyObject *path, *res, *name; + + if (!PyArg_ParseTuple(args, "O&O&:lgetxattr", PyUnicode_FSConverter, &path, + PyUnicode_FSConverter, &name)) + return NULL; + res = getxattr_common(PyBytes_AS_STRING(path), name, lgetxattr); + Py_DECREF(path); + Py_DECREF(name); + return res; +} + +static ssize_t +wrap_fgetxattr(const char *path, const char *name, void *value, size_t size) +{ + /* Hack to share code. */ + return fgetxattr((int)(Py_uintptr_t)path, name, value, size); +} + +PyDoc_STRVAR(posix_fgetxattr__doc__, +"fgetxattr(fd, attr) -> value\n\n\ +Like getxattr but operate on a fd instead of a path."); + +static PyObject * +posix_fgetxattr(PyObject *self, PyObject *args) +{ + PyObject *res, *name; + int fd; + + if (!PyArg_ParseTuple(args, "iO&:fgetxattr", &fd, PyUnicode_FSConverter, &name)) + return NULL; + res = getxattr_common((const char *)(Py_uintptr_t)fd, name, wrap_fgetxattr); + Py_DECREF(name); + return res; +} + +PyDoc_STRVAR(posix_setxattr__doc__, +"setxattr(path, attr, value, flags=0)\n\n\ +Set extended attribute *attr* on *path* to *value*."); + +static PyObject * +posix_setxattr(PyObject *self, PyObject *args) +{ + PyObject *path, *name; + Py_buffer data; + int flags = 0, err; + + if (!PyArg_ParseTuple(args, "O&O&y*|i:setxattr", PyUnicode_FSConverter, + &path, PyUnicode_FSConverter, &name, &data, &flags)) + return NULL; + Py_BEGIN_ALLOW_THREADS; + err = setxattr(PyBytes_AS_STRING(path), PyBytes_AS_STRING(name), + data.buf, data.len, flags); + Py_END_ALLOW_THREADS; + Py_DECREF(path); + Py_DECREF(name); + PyBuffer_Release(&data); + if (err) + return posix_error(); + Py_RETURN_NONE; +} + +PyDoc_STRVAR(posix_lsetxattr__doc__, +"lsetxattr(path, attr, value, flags=0)\n\n\ +Like setxattr but don't follow symlinks."); + +static PyObject * +posix_lsetxattr(PyObject *self, PyObject *args) +{ + PyObject *path, *name; + Py_buffer data; + int flags = 0, err; + + if (!PyArg_ParseTuple(args, "O&O&y*|i:lsetxattr", PyUnicode_FSConverter, + &path, PyUnicode_FSConverter, &name, &data, &flags)) + return NULL; + Py_BEGIN_ALLOW_THREADS; + err = lsetxattr(PyBytes_AS_STRING(path), PyBytes_AS_STRING(name), + data.buf, data.len, flags); + Py_END_ALLOW_THREADS; + Py_DECREF(path); + Py_DECREF(name); + PyBuffer_Release(&data); + if (err) + return posix_error(); + Py_RETURN_NONE; +} + +PyDoc_STRVAR(posix_fsetxattr__doc__, +"fsetxattr(fd, attr, value, flags=0)\n\n\ +Like setxattr but operates on *fd* instead of a path."); + +static PyObject * +posix_fsetxattr(PyObject *self, PyObject *args) +{ + Py_buffer data; + const char *name; + int fd, flags = 0, err; + + if (!PyArg_ParseTuple(args, "iO&y*|i:fsetxattr", &fd, PyUnicode_FSConverter, + &name, &data, &flags)) + return NULL; + Py_BEGIN_ALLOW_THREADS; + err = fsetxattr(fd, PyBytes_AS_STRING(name), data.buf, data.len, flags); + Py_END_ALLOW_THREADS; + Py_DECREF(name); + PyBuffer_Release(&data); + if (err) + return posix_error(); + Py_RETURN_NONE; +} + +PyDoc_STRVAR(posix_removexattr__doc__, +"removexattr(path, attr)\n\n\ +Remove extended attribute *attr* on *path*."); + +static PyObject * +posix_removexattr(PyObject *self, PyObject *args) +{ + PyObject *path, *name; + int err; + + if (!PyArg_ParseTuple(args, "O&O&:removexattr", PyUnicode_FSConverter, &path, + PyUnicode_FSConverter, &name)) + return NULL; + Py_BEGIN_ALLOW_THREADS; + err = removexattr(PyBytes_AS_STRING(path), PyBytes_AS_STRING(name)); + Py_END_ALLOW_THREADS; + Py_DECREF(path); + Py_DECREF(name); + if (err) + return posix_error(); + Py_RETURN_NONE; +} + +PyDoc_STRVAR(posix_lremovexattr__doc__, +"lremovexattr(path, attr)\n\n\ +Like removexattr but don't follow symlinks."); + +static PyObject * +posix_lremovexattr(PyObject *self, PyObject *args) +{ + PyObject *path, *name; + int err; + + if (!PyArg_ParseTuple(args, "O&O&:lremovexattr", PyUnicode_FSConverter, &path, + PyUnicode_FSConverter, &name)) + return NULL; + Py_BEGIN_ALLOW_THREADS; + err = lremovexattr(PyBytes_AS_STRING(path), PyBytes_AS_STRING(name)); + Py_END_ALLOW_THREADS; + Py_DECREF(path); + Py_DECREF(name); + if (err) + return posix_error(); + Py_RETURN_NONE; +} + +PyDoc_STRVAR(posix_fremovexattr__doc__, +"fremovexattr(fd, attr)\n\n\ +Like removexattr but operates on a file descriptor."); + +static PyObject * +posix_fremovexattr(PyObject *self, PyObject *args) +{ + PyObject *name; + int fd, err; + + if (!PyArg_ParseTuple(args, "iO&:fremovexattr", &fd, + PyUnicode_FSConverter, &name)) + return NULL; + Py_BEGIN_ALLOW_THREADS; + err = fremovexattr(fd, PyBytes_AS_STRING(name)); + Py_END_ALLOW_THREADS; + Py_DECREF(name); + if (err) + return posix_error(); + Py_RETURN_NONE; +} + +static Py_ssize_t +try_listxattr(const char *path, ssize_t (*list)(const char *, char *, size_t), + Py_ssize_t buf_size, char **buf) +{ + Py_ssize_t len; + + *buf = PyMem_MALLOC(buf_size); + if (!*buf) { + PyErr_NoMemory(); + return -1; + } + Py_BEGIN_ALLOW_THREADS; + len = list(path, *buf, buf_size); + Py_END_ALLOW_THREADS; + if (len < 0) { + PyMem_FREE(*buf); + if (errno != ERANGE) + posix_error(); + return -1; + } + return len; +} + +static PyObject * +listxattr_common(const char *path, ssize_t (*list)(const char *, char *, size_t)) +{ + PyObject *res, *attr; + Py_ssize_t len, err, start, i; + char *buf; + + len = try_listxattr(path, list, 256, &buf); + if (len < 0) { + if (PyErr_Occurred()) + return NULL; + len = try_listxattr(path, list, XATTR_LIST_MAX, &buf); + if (len < 0) + return NULL; + } + res = PyList_New(0); + if (!res) { + PyMem_FREE(buf); + return NULL; + } + for (start = i = 0; i < len; i++) { + if (!buf[i]) { + attr = PyUnicode_DecodeFSDefaultAndSize(&buf[start], i - start); + if (!attr) { + Py_DECREF(res); + PyMem_FREE(buf); + return NULL; + } + err = PyList_Append(res, attr); + Py_DECREF(attr); + if (err) { + Py_DECREF(res); + PyMem_FREE(buf); + return NULL; + } + start = i + 1; + } + } + PyMem_FREE(buf); + return res; +} + +PyDoc_STRVAR(posix_listxattr__doc__, +"listxattr(path)\n\n\ +Return a list of extended attributes on *path*."); + +static PyObject * +posix_listxattr(PyObject *self, PyObject *args) +{ + PyObject *path, *res; + + if (!PyArg_ParseTuple(args, "O&:listxattr", PyUnicode_FSConverter, &path)) + return NULL; + res = listxattr_common(PyBytes_AS_STRING(path), listxattr); + Py_DECREF(path); + return res; +} + +PyDoc_STRVAR(posix_llistxattr__doc__, +"llistxattr(path)\n\n\ +Like listxattr but don't follow symlinks.."); + +static PyObject * +posix_llistxattr(PyObject *self, PyObject *args) +{ + PyObject *path, *res; + + if (!PyArg_ParseTuple(args, "O&:llistxattr", PyUnicode_FSConverter, &path)) + return NULL; + res = listxattr_common(PyBytes_AS_STRING(path), llistxattr); + Py_DECREF(path); + return res; +} + +static ssize_t +wrap_flistxattr(const char *path, char *buf, size_t len) +{ + /* Hack to share code. */ + return flistxattr((int)(Py_uintptr_t)path, buf, len); +} + +PyDoc_STRVAR(posix_flistxattr__doc__, +"flistxattr(path)\n\n\ +Like flistxattr but operates on a file descriptor."); + +static PyObject * +posix_flistxattr(PyObject *self, PyObject *args) +{ + long fd; + + if (!PyArg_ParseTuple(args, "i:flistxattr", &fd)) + return NULL; + return listxattr_common((const char *)(Py_uintptr_t)fd, wrap_flistxattr); +} + +#endif /* HAVE_ATTR_XATTR_H */ + static PyMethodDef posix_methods[] = { {"access", posix_access, METH_VARARGS, posix_access__doc__}, #ifdef HAVE_TTYNAME @@ -10399,6 +10781,20 @@ #ifdef HAVE_MKFIFOAT {"mkfifoat", posix_mkfifoat, METH_VARARGS, posix_mkfifoat__doc__}, #endif +#ifdef HAVE_ATTR_XATTR_H + {"setxattr", posix_setxattr, METH_VARARGS, posix_setxattr__doc__}, + {"lsetxattr", posix_lsetxattr, METH_VARARGS, posix_lsetxattr__doc__}, + {"fsetxattr", posix_fsetxattr, METH_VARARGS, posix_fsetxattr__doc__}, + {"getxattr", posix_getxattr, METH_VARARGS, posix_getxattr__doc__}, + {"lgetxattr", posix_lgetxattr, METH_VARARGS, posix_lgetxattr__doc__}, + {"fgetxattr", posix_fgetxattr, METH_VARARGS, posix_fgetxattr__doc__}, + {"removexattr", posix_removexattr, METH_VARARGS, posix_removexattr__doc__}, + {"lremovexattr", posix_lremovexattr, METH_VARARGS, posix_lremovexattr__doc__}, + {"fremovexattr", posix_fremovexattr, METH_VARARGS, posix_fremovexattr__doc__}, + {"listxattr", posix_listxattr, METH_VARARGS, posix_listxattr__doc__}, + {"llistxattr", posix_llistxattr, METH_VARARGS, posix_llistxattr__doc__}, + {"flistxattr", posix_flistxattr, METH_VARARGS, posix_flistxattr__doc__}, +#endif {NULL, NULL} /* Sentinel */ }; @@ -10848,6 +11244,12 @@ #endif #endif +#ifdef HAVE_ATTR_XATTR_H + if (ins(d, "XATTR_CREATE", (long)XATTR_CREATE)) return -1; + if (ins(d, "XATTR_REPLACE", (long)XATTR_REPLACE)) return -1; + if (ins(d, "XATTR_SIZE_MAX", (long)XATTR_SIZE_MAX)) return -1; +#endif + #if defined(PYOS_OS2) if (insertvalues(d)) return -1; #endif diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6090,7 +6090,7 @@ fi -for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +for ac_header in asm/types.h attr/xattr.h conio.h curses.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ ieeefp.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ sched.h shadow.h signal.h stdint.h stropts.h termios.h \ diff --git a/configure.in b/configure.in --- a/configure.in +++ b/configure.in @@ -1299,7 +1299,7 @@ # checks for header files AC_HEADER_STDC -AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +AC_CHECK_HEADERS(asm/types.h attr/xattr.h conio.h curses.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ ieeefp.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ sched.h shadow.h signal.h stdint.h stropts.h termios.h \ diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -64,6 +64,9 @@ /* Define if GCC supports __attribute__((format(PyArg_ParseTuple, 2, 3))) */ #undef HAVE_ATTRIBUTE_FORMAT_PARSETUPLE +/* Define to 1 if you have the <attr/xattr.h> header file. */ +#undef HAVE_ATTR_XATTR_H + /* Define to 1 if you have the `bind_textdomain_codeset' function. */ #undef HAVE_BIND_TEXTDOMAIN_CODESET -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): accept bytes for the AST 'string' type
by benjamin.peterson 01 Sep '11

01 Sep '11

http://hg.python.org/cpython/rev/c8b6d4417274 changeset: 72154:c8b6d4417274 branch: 3.2 parent: 72151:1e01543c3d0a user: Benjamin Peterson <benjamin(a)python.org> date: Wed Aug 31 22:13:03 2011 -0400 summary: accept bytes for the AST 'string' type This is a temporary kludge and all is well in 3.3. files: Misc/NEWS | 3 +++ Parser/asdl_c.py | 2 +- Python/Python-ast.c | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Accept bytes for the AST string type. This is temporary until a proper fix in + 3.3. + - Issue #9200: The str.is* methods now work with strings that contain non-BMP characters even in narrow Unicode builds. diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -805,7 +805,7 @@ static int obj2ast_string(PyObject* obj, PyObject** out, PyArena* arena) { - if (!PyUnicode_CheckExact(obj)) { + if (!PyUnicode_CheckExact(obj) && !PyBytes_CheckExact(obj)) { PyErr_SetString(PyExc_TypeError, "AST string must be of type str"); return 1; } diff --git a/Python/Python-ast.c b/Python/Python-ast.c --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -611,7 +611,7 @@ static int obj2ast_string(PyObject* obj, PyObject** out, PyArena* arena) { - if (!PyUnicode_CheckExact(obj)) { + if (!PyUnicode_CheckExact(obj) && !PyBytes_CheckExact(obj)) { PyErr_SetString(PyExc_TypeError, "AST string must be of type str"); return 1; } -- Repository URL: http://hg.python.org/cpython

1 0