Mailman 3 December 2011 - Python-checkins

Re: [Python-checkins] cpython: pyexpat uses the new Unicode API
by Amaury Forgeot d'Arc July 29, 2012

July 29, 2012

> changeset: 72548:a1be34457ccf > user: Victor Stinner <victor.stinner at haypocalc.com> > date: Sat Oct 01 01:05:40 2011 +0200 > summary: > pyexat uses the new Unicode API > > files: > Modules/pyexpat.c | 12 +++++++----- > 1 files changed, 7 insertions(+), 5 deletions(-) > > > diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c > --- a/Modules/pyexpat.c > +++ b/Modules/pyexpat.c > @@ -1234,11 +1234,13 @@ > static PyObject * > xmlparse_getattro(xmlparseobject *self, PyObject *nameobj) > { > - const Py_UNICODE *name; > + Py_UCS4 first_char; > int handlernum = -1; > > if (!PyUnicode_Check(nameobj)) > goto generic; > + if (PyUnicode_READY(nameobj)) > + return NULL; Why is this PyUnicode_READY necessary? Can tp_getattro pass unfinished unicode objects? I hope we don't have to update all extension modules? -- Amaury Forgeot d'Arc

2 1

cpython (merge 3.2 -> default): merge with 3.2
by sandro.tosi Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/0ddb0bb63550 changeset: 74211:0ddb0bb63550 parent: 74208:43b10c2d2553 parent: 74209:ed4f7d9e0e96 user: Sandro Tosi <sandro.tosi(a)gmail.com> date: Sat Dec 31 19:03:30 2011 +0100 summary: merge with 3.2 files: Doc/howto/functional.rst | 6 +++--- Doc/howto/webservers.rst | 10 +++++----- Doc/library/atexit.rst | 2 -- Doc/using/unix.rst | 4 ++-- 4 files changed, 10 insertions(+), 12 deletions(-) diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1151,9 +1151,9 @@ Mertz also wrote a 3-part series of articles on functional programming for IBM's DeveloperWorks site; see -`part 1 <http://www-128.ibm.com/developerworks/library/l-prog.html>`__, -`part 2 <http://www-128.ibm.com/developerworks/library/l-prog2.html>`__, and -`part 3 <http://www-128.ibm.com/developerworks/linux/library/l-prog3.html>`__, +`part 1 <http://www.ibm.com/developerworks/linux/library/l-prog/index.html>`__, +`part 2 <http://www.ibm.com/developerworks/linux/library/l-prog2/index.html>`__, and +`part 3 <http://www.ibm.com/developerworks/linux/library/l-prog3/index.html>`__, Python documentation diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst --- a/Doc/howto/webservers.rst +++ b/Doc/howto/webservers.rst @@ -274,7 +274,7 @@ Each web server requires a specific module. * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid - <http://fastcgi.coremail.cn/>`_. ``mod_fastcgi`` is the original one, but it + <http://httpd.apache.org/mod_fcgid/>`_. ``mod_fastcgi`` is the original one, but it has some licensing issues, which is why it is sometimes considered non-free. ``mod_fcgid`` is a smaller, compatible alternative. One of these modules needs to be loaded by Apache. @@ -365,7 +365,7 @@ A really great WSGI feature is middleware. Middleware is a layer around your program which can add various functionality to it. There is quite a bit of -`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already +`middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already available. For example, instead of writing your own session management (HTTP is a stateless protocol, so to associate multiple HTTP requests with a single user your application must create and manage such state via a session), you can @@ -396,9 +396,9 @@ .. seealso:: - A good overview of WSGI-related code can be found in the `WSGI wiki - <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers - <http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application + A good overview of WSGI-related code can be found in the `WSGI homepage + <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers + <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application supporting WSGI. You might be interested in some WSGI-supporting modules already contained in diff --git a/Doc/library/atexit.rst b/Doc/library/atexit.rst --- a/Doc/library/atexit.rst +++ b/Doc/library/atexit.rst @@ -6,9 +6,6 @@ .. moduleauthor:: Skip Montanaro <skip(a)pobox.com> .. sectionauthor:: Skip Montanaro <skip(a)pobox.com> -**Source code:** :source:`Lib/atexit.py` - --------------- The :mod:`atexit` module defines functions to register and unregister cleanup functions. Functions thus registered are automatically executed upon normal diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst --- a/Doc/using/unix.rst +++ b/Doc/using/unix.rst @@ -26,11 +26,11 @@ .. seealso:: - http://www.linux.com/articles/60383 + http://www.debian.org/doc/manuals/maint-guide/first.en.html for Debian users http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html for OpenSuse users - http://docs.fedoraproject.org/drafts/rpm-guide-en/ch-creating-rpms.html + http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM… for Fedora users http://www.slackbook.org/html/package-management-making-packages.html for Slackware users -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): fix some broken URLs
by sandro.tosi Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/e9f517f8ee87 changeset: 74210:e9f517f8ee87 branch: 2.7 parent: 74203:fb5ad61a2b35 user: Sandro Tosi <sandro.tosi(a)gmail.com> date: Sat Dec 31 18:13:59 2011 +0100 summary: fix some broken URLs files: Doc/bugs.rst | 6 ++++-- Doc/howto/webservers.rst | 10 +++++----- Doc/library/carbon.rst | 4 ++-- Doc/library/easydialogs.rst | 2 +- Doc/using/unix.rst | 4 ++-- 5 files changed, 14 insertions(+), 12 deletions(-) diff --git a/Doc/bugs.rst b/Doc/bugs.rst --- a/Doc/bugs.rst +++ b/Doc/bugs.rst @@ -57,12 +57,14 @@ Each bug report will be assigned to a developer who will determine what needs to be done to correct the problem. You will receive an update each time action is -taken on the bug. See http://www.python.org/dev/workflow/ for a detailed -description of the issue workflow. +taken on the bug. .. seealso:: + `Python Developer's Guide <http://docs.python.org/devguide/>`_ + Detailed description of the issue workflow and developers tools. + `How to Report Bugs Effectively <http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>`_ Article which goes into some detail about how to create a useful bug report. This describes what kind of information is useful and why it is useful. diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst --- a/Doc/howto/webservers.rst +++ b/Doc/howto/webservers.rst @@ -274,7 +274,7 @@ Each web server requires a specific module. * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid - <http://fastcgi.coremail.cn/>`_. ``mod_fastcgi`` is the original one, but it + <http://httpd.apache.org/mod_fcgid/>`_. ``mod_fastcgi`` is the original one, but it has some licensing issues, which is why it is sometimes considered non-free. ``mod_fcgid`` is a smaller, compatible alternative. One of these modules needs to be loaded by Apache. @@ -364,7 +364,7 @@ A really great WSGI feature is middleware. Middleware is a layer around your program which can add various functionality to it. There is quite a bit of -`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already +`middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already available. For example, instead of writing your own session management (HTTP is a stateless protocol, so to associate multiple HTTP requests with a single user your application must create and manage such state via a session), you can @@ -395,9 +395,9 @@ .. seealso:: - A good overview of WSGI-related code can be found in the `WSGI wiki - <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers - <http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application + A good overview of WSGI-related code can be found in the `WSGI homepage + <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers + <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application supporting WSGI. You might be interested in some WSGI-supporting modules already contained in diff --git a/Doc/library/carbon.rst b/Doc/library/carbon.rst --- a/Doc/library/carbon.rst +++ b/Doc/library/carbon.rst @@ -14,7 +14,7 @@ in Python (input and output buffers, especially). All methods and functions have a :attr:`__doc__` string describing their arguments and return values, and for additional description you are referred to `Inside Macintosh -<http://developer.apple.com/documentation/macos8/mac8.html>`_ or similar works. +<http://developer.apple.com/legacy/mac/library/#documentation/macos8/mac8.ht…>`_ or similar works. These modules all live in a package called :mod:`Carbon`. Despite that name they are not all part of the Carbon framework: CF is really in the CoreFoundation @@ -515,7 +515,7 @@ .. seealso:: - `Scrap Manager <http://developer.apple.com/documentation/mac/MoreToolbox/MoreToolbox-109.ht…>`_ + `Scrap Manager <http://developer.apple.com/legacy/mac/library/documentation/mac/MoreToolbox…>`_ Apple's documentation for the Scrap Manager gives a lot of useful information about using the Scrap Manager in applications. diff --git a/Doc/library/easydialogs.rst b/Doc/library/easydialogs.rst --- a/Doc/library/easydialogs.rst +++ b/Doc/library/easydialogs.rst @@ -143,7 +143,7 @@ .. seealso:: - `Navigation Services Reference <http://developer.apple.com/documentation/Carbon/Reference/Navigation_Servic…>`_ + `Navigation Services Reference <http://developer.apple.com/legacy/mac/library/#documentation/Carbon/Concept…>`_ Programmer's reference documentation for the Navigation Services, a part of the Carbon framework. diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst --- a/Doc/using/unix.rst +++ b/Doc/using/unix.rst @@ -26,11 +26,11 @@ .. seealso:: - http://www.linux.com/articles/60383 + http://www.debian.org/doc/manuals/maint-guide/first.en.html for Debian users http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html for OpenSuse users - http://docs.fedoraproject.org/drafts/rpm-guide-en/ch-creating-rpms.html + http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM… for Fedora users http://www.slackbook.org/html/package-management-making-packages.html for Slackware users -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): fix some broken URLs
by sandro.tosi Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/ed4f7d9e0e96 changeset: 74209:ed4f7d9e0e96 branch: 3.2 parent: 74206:2c3a769089d5 user: Sandro Tosi <sandro.tosi(a)gmail.com> date: Sat Dec 31 18:46:50 2011 +0100 summary: fix some broken URLs files: Doc/howto/functional.rst | 6 +++--- Doc/howto/webservers.rst | 10 +++++----- Doc/library/atexit.rst | 2 -- Doc/using/unix.rst | 4 ++-- 4 files changed, 10 insertions(+), 12 deletions(-) diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1151,9 +1151,9 @@ Mertz also wrote a 3-part series of articles on functional programming for IBM's DeveloperWorks site; see -`part 1 <http://www-128.ibm.com/developerworks/library/l-prog.html>`__, -`part 2 <http://www-128.ibm.com/developerworks/library/l-prog2.html>`__, and -`part 3 <http://www-128.ibm.com/developerworks/linux/library/l-prog3.html>`__, +`part 1 <http://www.ibm.com/developerworks/linux/library/l-prog/index.html>`__, +`part 2 <http://www.ibm.com/developerworks/linux/library/l-prog2/index.html>`__, and +`part 3 <http://www.ibm.com/developerworks/linux/library/l-prog3/index.html>`__, Python documentation diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst --- a/Doc/howto/webservers.rst +++ b/Doc/howto/webservers.rst @@ -274,7 +274,7 @@ Each web server requires a specific module. * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid - <http://fastcgi.coremail.cn/>`_. ``mod_fastcgi`` is the original one, but it + <http://httpd.apache.org/mod_fcgid/>`_. ``mod_fastcgi`` is the original one, but it has some licensing issues, which is why it is sometimes considered non-free. ``mod_fcgid`` is a smaller, compatible alternative. One of these modules needs to be loaded by Apache. @@ -365,7 +365,7 @@ A really great WSGI feature is middleware. Middleware is a layer around your program which can add various functionality to it. There is quite a bit of -`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already +`middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already available. For example, instead of writing your own session management (HTTP is a stateless protocol, so to associate multiple HTTP requests with a single user your application must create and manage such state via a session), you can @@ -396,9 +396,9 @@ .. seealso:: - A good overview of WSGI-related code can be found in the `WSGI wiki - <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers - <http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application + A good overview of WSGI-related code can be found in the `WSGI homepage + <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers + <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application supporting WSGI. You might be interested in some WSGI-supporting modules already contained in diff --git a/Doc/library/atexit.rst b/Doc/library/atexit.rst --- a/Doc/library/atexit.rst +++ b/Doc/library/atexit.rst @@ -6,9 +6,6 @@ .. moduleauthor:: Skip Montanaro <skip(a)pobox.com> .. sectionauthor:: Skip Montanaro <skip(a)pobox.com> -**Source code:** :source:`Lib/atexit.py` - --------------- The :mod:`atexit` module defines functions to register and unregister cleanup functions. Functions thus registered are automatically executed upon normal diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst --- a/Doc/using/unix.rst +++ b/Doc/using/unix.rst @@ -26,11 +26,11 @@ .. seealso:: - http://www.linux.com/articles/60383 + http://www.debian.org/doc/manuals/maint-guide/first.en.html for Debian users http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html for OpenSuse users - http://docs.fedoraproject.org/drafts/rpm-guide-en/ch-creating-rpms.html + http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM… for Fedora users http://www.slackbook.org/html/package-management-making-packages.html for Slackware users -- Repository URL: http://hg.python.org/cpython

1 0

cpython: fix Exemples/Examples typo; thanks to Ori Livneh from docs@
by sandro.tosi Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/43b10c2d2553 changeset: 74208:43b10c2d2553 user: Sandro Tosi <sandro.tosi(a)gmail.com> date: Sat Dec 31 12:59:08 2011 +0100 summary: fix Exemples/Examples typo; thanks to Ori Livneh from docs@ files: Doc/library/packaging.pypi.simple.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/packaging.pypi.simple.rst b/Doc/library/packaging.pypi.simple.rst --- a/Doc/library/packaging.pypi.simple.rst +++ b/Doc/library/packaging.pypi.simple.rst @@ -89,7 +89,7 @@ -Usage Exemples +Usage Examples --------------- To help you understand how using the `Crawler` class, here are some basic -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> default): Some formatting & grammar fixes for the multiprocessing doc
by eli.bendersky Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/73481bd9ca54 changeset: 74207:73481bd9ca54 parent: 74205:dbd3013c2685 parent: 74206:2c3a769089d5 user: Eli Bendersky <eliben(a)gmail.com> date: Sat Dec 31 07:22:27 2011 +0200 summary: Some formatting & grammar fixes for the multiprocessing doc files: Doc/library/multiprocessing.rst | 28 ++++++++++---------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -491,7 +491,7 @@ If you use :class:`JoinableQueue` then you **must** call :meth:`JoinableQueue.task_done` for each task removed from the queue or else the -semaphore used to count the number of unfinished tasks may eventually overflow +semaphore used to count the number of unfinished tasks may eventually overflow, raising an exception. Note that one can also create a shared queue by using a manager object -- see @@ -509,7 +509,7 @@ If a process is killed using :meth:`Process.terminate` or :func:`os.kill` while it is trying to use a :class:`Queue`, then the data in the queue is - likely to become corrupted. This may cause any other processes to get an + likely to become corrupted. This may cause any other process to get an exception when it tries to use the queue later on. .. warning:: @@ -711,7 +711,7 @@ (By default :data:`sys.executable` is used). Embedders will probably need to do some thing like :: - setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe')) + set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe')) before they can create child processes. (Windows only) @@ -730,7 +730,7 @@ Connection objects allow the sending and receiving of picklable objects or strings. They can be thought of as message oriented connected sockets. -Connection objects usually created using :func:`Pipe` -- see also +Connection objects are usually created using :func:`Pipe` -- see also :ref:`multiprocessing-listeners-clients`. .. class:: Connection @@ -751,7 +751,7 @@ .. method:: fileno() - Returns the file descriptor or handle used by the connection. + Return the file descriptor or handle used by the connection. .. method:: close() @@ -775,7 +775,7 @@ If *offset* is given then data is read from that position in *buffer*. If *size* is given then that many bytes will be read from buffer. Very large buffers (approximately 32 MB+, though it depends on the OS) may raise a - ValueError exception + :exc:`ValueError` exception .. method:: recv_bytes([maxlength]) @@ -1353,7 +1353,7 @@ >>>>>>>>>>>>>>>>>>> To create one's own manager, one creates a subclass of :class:`BaseManager` and -use the :meth:`~BaseManager.register` classmethod to register new types or +uses the :meth:`~BaseManager.register` classmethod to register new types or callables with the manager class. For example:: from multiprocessing.managers import BaseManager @@ -1604,9 +1604,9 @@ .. method:: apply(func[, args[, kwds]]) Call *func* with arguments *args* and keyword arguments *kwds*. It blocks - till the result is ready. Given this blocks, :meth:`apply_async` is better - suited for performing work in parallel. Additionally, the passed in - function is only executed in one of the workers of the pool. + until the result is ready. Given this blocks, :meth:`apply_async` is + better suited for performing work in parallel. Additionally, *func* + is only executed in one of the workers of the pool. .. method:: apply_async(func[, args[, kwds[, callback[, error_callback]]]]) @@ -1627,7 +1627,7 @@ .. method:: map(func, iterable[, chunksize]) A parallel equivalent of the :func:`map` built-in function (it supports only - one *iterable* argument though). It blocks till the result is ready. + one *iterable* argument though). It blocks until the result is ready. This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these @@ -2102,7 +2102,7 @@ On Windows many types from :mod:`multiprocessing` need to be picklable so that child processes can use them. However, one should generally avoid sending shared objects to other processes using pipes or queues. Instead - you should arrange the program so that a process which need access to a + you should arrange the program so that a process which needs access to a shared resource created elsewhere can inherit it from an ancestor process. Avoid terminating processes @@ -2181,7 +2181,7 @@ for i in range(10): Process(target=f, args=(lock,)).start() -Beware replacing sys.stdin with a "file like object" +Beware of replacing :data:`sys.stdin` with a "file like object" :mod:`multiprocessing` originally unconditionally called:: @@ -2299,7 +2299,7 @@ An example showing how to use queues to feed tasks to a collection of worker -process and collect the results: +processes and collect the results: .. literalinclude:: ../includes/mp_workers.py -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Some formatting & grammar fixes for the multiprocessing doc
by eli.bendersky Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/2c3a769089d5 changeset: 74206:2c3a769089d5 branch: 3.2 parent: 74204:ce349e1f923e user: Eli Bendersky <eliben(a)gmail.com> date: Sat Dec 31 07:20:26 2011 +0200 summary: Some formatting & grammar fixes for the multiprocessing doc files: Doc/library/multiprocessing.rst | 28 ++++++++++---------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -472,7 +472,7 @@ If you use :class:`JoinableQueue` then you **must** call :meth:`JoinableQueue.task_done` for each task removed from the queue or else the -semaphore used to count the number of unfinished tasks may eventually overflow +semaphore used to count the number of unfinished tasks may eventually overflow, raising an exception. Note that one can also create a shared queue by using a manager object -- see @@ -490,7 +490,7 @@ If a process is killed using :meth:`Process.terminate` or :func:`os.kill` while it is trying to use a :class:`Queue`, then the data in the queue is - likely to become corrupted. This may cause any other processes to get an + likely to become corrupted. This may cause any other process to get an exception when it tries to use the queue later on. .. warning:: @@ -692,7 +692,7 @@ (By default :data:`sys.executable` is used). Embedders will probably need to do some thing like :: - setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe')) + set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe')) before they can create child processes. (Windows only) @@ -711,7 +711,7 @@ Connection objects allow the sending and receiving of picklable objects or strings. They can be thought of as message oriented connected sockets. -Connection objects usually created using :func:`Pipe` -- see also +Connection objects are usually created using :func:`Pipe` -- see also :ref:`multiprocessing-listeners-clients`. .. class:: Connection @@ -732,7 +732,7 @@ .. method:: fileno() - Returns the file descriptor or handle used by the connection. + Return the file descriptor or handle used by the connection. .. method:: close() @@ -756,7 +756,7 @@ If *offset* is given then data is read from that position in *buffer*. If *size* is given then that many bytes will be read from buffer. Very large buffers (approximately 32 MB+, though it depends on the OS) may raise a - ValueError exception + :exc:`ValueError` exception .. method:: recv_bytes([maxlength]) @@ -1329,7 +1329,7 @@ >>>>>>>>>>>>>>>>>>> To create one's own manager, one creates a subclass of :class:`BaseManager` and -use the :meth:`~BaseManager.register` classmethod to register new types or +uses the :meth:`~BaseManager.register` classmethod to register new types or callables with the manager class. For example:: from multiprocessing.managers import BaseManager @@ -1580,9 +1580,9 @@ .. method:: apply(func[, args[, kwds]]) Call *func* with arguments *args* and keyword arguments *kwds*. It blocks - till the result is ready. Given this blocks, :meth:`apply_async` is better - suited for performing work in parallel. Additionally, the passed in - function is only executed in one of the workers of the pool. + until the result is ready. Given this blocks, :meth:`apply_async` is + better suited for performing work in parallel. Additionally, *func* + is only executed in one of the workers of the pool. .. method:: apply_async(func[, args[, kwds[, callback[, error_callback]]]]) @@ -1603,7 +1603,7 @@ .. method:: map(func, iterable[, chunksize]) A parallel equivalent of the :func:`map` built-in function (it supports only - one *iterable* argument though). It blocks till the result is ready. + one *iterable* argument though). It blocks until the result is ready. This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these @@ -2060,7 +2060,7 @@ On Windows many types from :mod:`multiprocessing` need to be picklable so that child processes can use them. However, one should generally avoid sending shared objects to other processes using pipes or queues. Instead - you should arrange the program so that a process which need access to a + you should arrange the program so that a process which needs access to a shared resource created elsewhere can inherit it from an ancestor process. Avoid terminating processes @@ -2139,7 +2139,7 @@ for i in range(10): Process(target=f, args=(lock,)).start() -Beware replacing sys.stdin with a "file like object" +Beware of replacing :data:`sys.stdin` with a "file like object" :mod:`multiprocessing` originally unconditionally called:: @@ -2257,7 +2257,7 @@ An example showing how to use queues to feed tasks to a collection of worker -process and collect the results: +processes and collect the results: .. literalinclude:: ../includes/mp_workers.py -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): indicate msg is an argument
by benjamin.peterson Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/ce349e1f923e changeset: 74204:ce349e1f923e branch: 3.2 parent: 74201:cd30304713ae user: Benjamin Peterson <benjamin(a)python.org> date: Fri Dec 30 23:08:09 2011 -0600 summary: indicate msg is an argument files: Doc/library/smtplib.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -281,10 +281,10 @@ envelope used by the transport agents. ``sendmail`` does not modify the message headers in any way. - msg may be a string containing characters in the ASCII range, or a byte + *msg* may be a string containing characters in the ASCII range, or a byte string. A string is encoded to bytes using the ascii codec, and lone ``\r`` - and ``\n`` characters are converted to ``\r\n`` characters. A byte string - is not modified. + and ``\n`` characters are converted to ``\r\n`` characters. A byte string is + not modified. If there has been no previous ``EHLO`` or ``HELO`` command this session, this method tries ESMTP ``EHLO`` first. If the server does ESMTP, message size and -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> default): merge 3.2
by benjamin.peterson Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/dbd3013c2685 changeset: 74205:dbd3013c2685 parent: 74202:656fa057aa28 parent: 74204:ce349e1f923e user: Benjamin Peterson <benjamin(a)python.org> date: Fri Dec 30 23:09:20 2011 -0600 summary: merge 3.2 files: Doc/library/smtplib.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -323,10 +323,10 @@ envelope used by the transport agents. ``sendmail`` does not modify the message headers in any way. - msg may be a string containing characters in the ASCII range, or a byte + *msg* may be a string containing characters in the ASCII range, or a byte string. A string is encoded to bytes using the ascii codec, and lone ``\r`` - and ``\n`` characters are converted to ``\r\n`` characters. A byte string - is not modified. + and ``\n`` characters are converted to ``\r\n`` characters. A byte string is + not modified. If there has been no previous ``EHLO`` or ``HELO`` command this session, this method tries ESMTP ``EHLO`` first. If the server does ESMTP, message size and -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Some formatting & grammar fixes for the multiprocessing doc
by eli.bendersky Dec. 31, 2011

Dec. 31, 2011

http://hg.python.org/cpython/rev/fb5ad61a2b35 changeset: 74203:fb5ad61a2b35 branch: 2.7 parent: 74200:5adbc87f1028 user: Eli Bendersky <eliben(a)gmail.com> date: Sat Dec 31 07:05:12 2011 +0200 summary: Some formatting & grammar fixes for the multiprocessing doc files: Doc/library/multiprocessing.rst | 34 ++++++++++---------- 1 files changed, 17 insertions(+), 17 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -282,7 +282,7 @@ if __name__ == '__main__': pool = Pool(processes=4) # start 4 worker processes - result = pool.apply_async(f, [10]) # evaluate "f(10)" asynchronously + result = pool.apply_async(f, [10]) # evaluate "f(10)" asynchronously print result.get(timeout=1) # prints "100" unless your computer is *very* slow print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]" @@ -472,7 +472,7 @@ If you use :class:`JoinableQueue` then you **must** call :meth:`JoinableQueue.task_done` for each task removed from the queue or else the -semaphore used to count the number of unfinished tasks may eventually overflow +semaphore used to count the number of unfinished tasks may eventually overflow, raising an exception. Note that one can also create a shared queue by using a manager object -- see @@ -490,7 +490,7 @@ If a process is killed using :meth:`Process.terminate` or :func:`os.kill` while it is trying to use a :class:`Queue`, then the data in the queue is - likely to become corrupted. This may cause any other processes to get an + likely to become corrupted. This may cause any other process to get an exception when it tries to use the queue later on. .. warning:: @@ -692,7 +692,7 @@ (By default :data:`sys.executable` is used). Embedders will probably need to do some thing like :: - setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe')) + set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe')) before they can create child processes. (Windows only) @@ -711,7 +711,7 @@ Connection objects allow the sending and receiving of picklable objects or strings. They can be thought of as message oriented connected sockets. -Connection objects usually created using :func:`Pipe` -- see also +Connection objects are usually created using :func:`Pipe` -- see also :ref:`multiprocessing-listeners-clients`. .. class:: Connection @@ -722,7 +722,7 @@ using :meth:`recv`. The object must be picklable. Very large pickles (approximately 32 MB+, - though it depends on the OS) may raise a ValueError exception. + though it depends on the OS) may raise a :exc:`ValueError` exception. .. method:: recv() @@ -732,7 +732,7 @@ .. method:: fileno() - Returns the file descriptor or handle used by the connection. + Return the file descriptor or handle used by the connection. .. method:: close() @@ -756,7 +756,7 @@ If *offset* is given then data is read from that position in *buffer*. If *size* is given then that many bytes will be read from buffer. Very large buffers (approximately 32 MB+, though it depends on the OS) may raise a - ValueError exception + :exc:`ValueError` exception .. method:: recv_bytes([maxlength]) @@ -1329,7 +1329,7 @@ >>>>>>>>>>>>>>>>>>> To create one's own manager, one creates a subclass of :class:`BaseManager` and -use the :meth:`~BaseManager.register` classmethod to register new types or +uses the :meth:`~BaseManager.register` classmethod to register new types or callables with the manager class. For example:: from multiprocessing.managers import BaseManager @@ -1579,10 +1579,10 @@ .. method:: apply(func[, args[, kwds]]) - Equivalent of the :func:`apply` built-in function. It blocks till the - result is ready. Given this blocks, :meth:`apply_async` is better suited - for performing work in parallel. Additionally, the passed - in function is only executed in one of the workers of the pool. + Equivalent of the :func:`apply` built-in function. It blocks until the + result is ready, so :meth:`apply_async` is better suited for performing + work in parallel. Additionally, *func* is only executed in one of the + workers of the pool. .. method:: apply_async(func[, args[, kwds[, callback]]]) @@ -1596,7 +1596,7 @@ .. method:: map(func, iterable[, chunksize]) A parallel equivalent of the :func:`map` built-in function (it supports only - one *iterable* argument though). It blocks till the result is ready. + one *iterable* argument though). It blocks until the result is ready. This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these @@ -2046,7 +2046,7 @@ On Windows many types from :mod:`multiprocessing` need to be picklable so that child processes can use them. However, one should generally avoid sending shared objects to other processes using pipes or queues. Instead - you should arrange the program so that a process which need access to a + you should arrange the program so that a process which needs access to a shared resource created elsewhere can inherit it from an ancestor process. Avoid terminating processes @@ -2125,7 +2125,7 @@ for i in range(10): Process(target=f, args=(lock,)).start() -Beware replacing sys.stdin with a "file like object" +Beware of replacing :data:`sys.stdin` with a "file like object" :mod:`multiprocessing` originally unconditionally called:: @@ -2243,7 +2243,7 @@ An example showing how to use queues to feed tasks to a collection of worker -process and collect the results: +processes and collect the results: .. literalinclude:: ../includes/mp_workers.py -- Repository URL: http://hg.python.org/cpython

1 0

Daily reference leaks (656fa057aa28): sum=0
by solipsis＠pitrou.net Dec. 31, 2011

Dec. 31, 2011

results for 656fa057aa28 on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloguYoz1d', '-x']

1 0

cpython (merge 3.2 -> default): Fix no-op tests in importlib.
by antoine.pitrou Dec. 30, 2011

Dec. 30, 2011

http://hg.python.org/cpython/rev/656fa057aa28 changeset: 74202:656fa057aa28 parent: 74199:f2b9f1d1dd6e parent: 74201:cd30304713ae user: Antoine Pitrou <solipsis(a)pitrou.net> date: Fri Dec 30 21:26:08 2011 +0100 summary: Fix no-op tests in importlib. files: Lib/importlib/test/source/test_file_loader.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/importlib/test/source/test_file_loader.py b/Lib/importlib/test/source/test_file_loader.py --- a/Lib/importlib/test/source/test_file_loader.py +++ b/Lib/importlib/test/source/test_file_loader.py @@ -256,6 +256,8 @@ with open(bytecode_path, 'rb') as file: self.assertGreater(len(file.read()), 8) + self._test_magic_only(test) + @source_util.writes_bytecode_files def test_bad_magic(self): # When the magic number is different, the bytecode should be @@ -276,6 +278,8 @@ with open(bc_path, 'rb') as file: self.assertGreater(len(file.read()), 8) + self._test_partial_timestamp(test) + @source_util.writes_bytecode_files def test_no_marshal(self): # When there is only the magic number and timestamp, raise EOFError. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Fix no-op tests in importlib.
by antoine.pitrou Dec. 30, 2011

Dec. 30, 2011

http://hg.python.org/cpython/rev/cd30304713ae changeset: 74201:cd30304713ae branch: 3.2 parent: 74198:d6f3e0b3c6fa user: Antoine Pitrou <solipsis(a)pitrou.net> date: Fri Dec 30 21:25:15 2011 +0100 summary: Fix no-op tests in importlib. files: Lib/importlib/test/source/test_file_loader.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/importlib/test/source/test_file_loader.py b/Lib/importlib/test/source/test_file_loader.py --- a/Lib/importlib/test/source/test_file_loader.py +++ b/Lib/importlib/test/source/test_file_loader.py @@ -256,6 +256,8 @@ with open(bytecode_path, 'rb') as file: self.assertGreater(len(file.read()), 8) + self._test_magic_only(test) + @source_util.writes_bytecode_files def test_bad_magic(self): # When the magic number is different, the bytecode should be @@ -276,6 +278,8 @@ with open(bc_path, 'rb') as file: self.assertGreater(len(file.read()), 8) + self._test_partial_timestamp(test) + @source_util.writes_bytecode_files def test_no_marshal(self): # When there is only the magic number and timestamp, raise EOFError. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): use booleans
by benjamin.peterson Dec. 30, 2011

Dec. 30, 2011

http://hg.python.org/cpython/rev/5adbc87f1028 changeset: 74200:5adbc87f1028 branch: 2.7 parent: 74191:9b681e0c04ed user: Benjamin Peterson <benjamin(a)python.org> date: Fri Dec 30 13:47:25 2011 -0600 summary: use booleans files: Doc/library/logging.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -67,7 +67,7 @@ If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers. - The constructor sets this attribute to 1. + The constructor sets this attribute to ``True``. .. method:: Logger.setLevel(lvl) -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> default): merge 3.2
by benjamin.peterson Dec. 30, 2011

Dec. 30, 2011

http://hg.python.org/cpython/rev/f2b9f1d1dd6e changeset: 74199:f2b9f1d1dd6e parent: 74197:7010fa9bd190 parent: 74198:d6f3e0b3c6fa user: Benjamin Peterson <benjamin(a)python.org> date: Fri Dec 30 13:47:49 2011 -0600 summary: merge 3.2 files: Doc/library/logging.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -65,7 +65,7 @@ If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers. - The constructor sets this attribute to 1. + The constructor sets this attribute to ``True``. .. method:: Logger.setLevel(lvl) -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): use booleans
by benjamin.peterson Dec. 30, 2011

Dec. 30, 2011

http://hg.python.org/cpython/rev/d6f3e0b3c6fa changeset: 74198:d6f3e0b3c6fa branch: 3.2 parent: 74184:4964712a7269 user: Benjamin Peterson <benjamin(a)python.org> date: Fri Dec 30 13:47:25 2011 -0600 summary: use booleans files: Doc/library/logging.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -65,7 +65,7 @@ If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers. - The constructor sets this attribute to 1. + The constructor sets this attribute to ``True``. .. method:: Logger.setLevel(lvl) -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #12715: Add an optional symlinks argument to shutil functions (copyfile,
by antoine.pitrou Dec. 30, 2011

Dec. 30, 2011

http://hg.python.org/cpython/rev/cf57ef65bcd0 changeset: 74194:cf57ef65bcd0 user: Antoine Pitrou <solipsis(a)pitrou.net> date: Thu Dec 29 18:54:15 2011 +0100 summary: Issue #12715: Add an optional symlinks argument to shutil functions (copyfile, copymode, copystat, copy, copy2). When that parameter is true, symlinks aren't dereferenced and the operation instead acts on the symlink itself (or creates one, if relevant). Patch by Hynek Schlawack. files: Doc/library/shutil.rst | 46 ++++- Lib/shutil.py | 101 +++++++++--- Lib/test/test_shutil.py | 219 ++++++++++++++++++++++++++++ Misc/NEWS | 5 + 4 files changed, 333 insertions(+), 38 deletions(-) diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -45,7 +45,7 @@ be copied. -.. function:: copyfile(src, dst) +.. function:: copyfile(src, dst[, symlinks=False]) Copy the contents (no metadata) of the file named *src* to a file named *dst*. *dst* must be the complete target file name; look at :func:`copy` for a copy that @@ -56,37 +56,56 @@ such as character or block devices and pipes cannot be copied with this function. *src* and *dst* are path names given as strings. + If *symlinks* is true and *src* is a symbolic link, a new symbolic link will + be created instead of copying the file *src* points to. + .. versionchanged:: 3.3 :exc:`IOError` used to be raised instead of :exc:`OSError`. + Added *symlinks* argument. -.. function:: copymode(src, dst) +.. function:: copymode(src, dst[, symlinks=False]) Copy the permission bits from *src* to *dst*. The file contents, owner, and - group are unaffected. *src* and *dst* are path names given as strings. + group are unaffected. *src* and *dst* are path names given as strings. If + *symlinks* is true, *src* a symbolic link and the operating system supports + modes for symbolic links (for example BSD-based ones), the mode of the link + will be copied. + .. versionchanged:: 3.3 + Added *symlinks* argument. -.. function:: copystat(src, dst) +.. function:: copystat(src, dst[, symlinks=False]) Copy the permission bits, last access time, last modification time, and flags from *src* to *dst*. The file contents, owner, and group are unaffected. *src* - and *dst* are path names given as strings. + and *dst* are path names given as strings. If *src* and *dst* are both + symbolic links and *symlinks* true, the stats of the link will be copied as + far as the platform allows. + .. versionchanged:: 3.3 + Added *symlinks* argument. -.. function:: copy(src, dst) +.. function:: copy(src, dst[, symlinks=False])) Copy the file *src* to the file or directory *dst*. If *dst* is a directory, a file with the same basename as *src* is created (or overwritten) in the directory specified. Permission bits are copied. *src* and *dst* are path - names given as strings. + names given as strings. If *symlinks* is true, symbolic links won't be + followed but recreated instead -- this resembles GNU's :program:`cp -P`. + .. versionchanged:: 3.3 + Added *symlinks* argument. -.. function:: copy2(src, dst) +.. function:: copy2(src, dst[, symlinks=False]) Similar to :func:`copy`, but metadata is copied as well -- in fact, this is just :func:`copy` followed by :func:`copystat`. This is similar to the - Unix command :program:`cp -p`. + Unix command :program:`cp -p`. If *symlinks* is true, symbolic links won't + be followed but recreated instead -- this resembles GNU's :program:`cp -P`. + .. versionchanged:: 3.3 + Added *symlinks* argument. .. function:: ignore_patterns(\*patterns) @@ -104,9 +123,9 @@ :func:`copy2`. If *symlinks* is true, symbolic links in the source tree are represented as - symbolic links in the new tree, but the metadata of the original links is NOT - copied; if false or omitted, the contents and metadata of the linked files - are copied to the new tree. + symbolic links in the new tree and the metadata of the original links will + be copied as far as the platform allows; if false or omitted, the contents + and metadata of the linked files are copied to the new tree. When *symlinks* is false, if the file pointed by the symlink doesn't exist, a exception will be added in the list of errors raised in @@ -140,6 +159,9 @@ Added the *ignore_dangling_symlinks* argument to silent dangling symlinks errors when *symlinks* is false. + .. versionchanged:: 3.3 + Copy metadata when *symlinks* is false. + .. function:: rmtree(path, ignore_errors=False, onerror=None) diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -82,8 +82,13 @@ return (os.path.normcase(os.path.abspath(src)) == os.path.normcase(os.path.abspath(dst))) -def copyfile(src, dst): - """Copy data from src to dst""" +def copyfile(src, dst, symlinks=False): + """Copy data from src to dst. + + If optional flag `symlinks` is set and `src` is a symbolic link, a new + symlink will be created instead of copying the file it points to. + + """ if _samefile(src, dst): raise Error("`%s` and `%s` are the same file" % (src, dst)) @@ -98,54 +103,94 @@ if stat.S_ISFIFO(st.st_mode): raise SpecialFileError("`%s` is a named pipe" % fn) - with open(src, 'rb') as fsrc: - with open(dst, 'wb') as fdst: - copyfileobj(fsrc, fdst) + if symlinks and os.path.islink(src): + os.symlink(os.readlink(src), dst) + else: + with open(src, 'rb') as fsrc: + with open(dst, 'wb') as fdst: + copyfileobj(fsrc, fdst) -def copymode(src, dst): - """Copy mode bits from src to dst""" - if hasattr(os, 'chmod'): - st = os.stat(src) - mode = stat.S_IMODE(st.st_mode) - os.chmod(dst, mode) +def copymode(src, dst, symlinks=False): + """Copy mode bits from src to dst. -def copystat(src, dst): - """Copy all stat info (mode bits, atime, mtime, flags) from src to dst""" - st = os.stat(src) + If the optional flag `symlinks` is set, symlinks aren't followed if and + only if both `src` and `dst` are symlinks. If `lchmod` isn't available (eg. + Linux), in these cases, this method does nothing. + + """ + if symlinks and os.path.islink(src) and os.path.islink(dst): + if hasattr(os, 'lchmod'): + stat_func, chmod_func = os.lstat, os.lchmod + else: + return + elif hasattr(os, 'chmod'): + stat_func, chmod_func = os.stat, os.chmod + else: + return + + st = stat_func(src) + chmod_func(dst, stat.S_IMODE(st.st_mode)) + +def copystat(src, dst, symlinks=False): + """Copy all stat info (mode bits, atime, mtime, flags) from src to dst. + + If the optional flag `symlinks` is set, symlinks aren't followed if and + only if both `src` and `dst` are symlinks. + + """ + def _nop(*args): + pass + + if symlinks and os.path.islink(src) and os.path.islink(dst): + stat_func = os.lstat + utime_func = os.lutimes if hasattr(os, 'lutimes') else _nop + chmod_func = os.lchmod if hasattr(os, 'lchmod') else _nop + chflags_func = os.lchflags if hasattr(os, 'lchflags') else _nop + else: + stat_func = os.stat + utime_func = os.utime if hasattr(os, 'utime') else _nop + chmod_func = os.chmod if hasattr(os, 'chmod') else _nop + chflags_func = os.chflags if hasattr(os, 'chflags') else _nop + + st = stat_func(src) mode = stat.S_IMODE(st.st_mode) - if hasattr(os, 'utime'): - os.utime(dst, (st.st_atime, st.st_mtime)) - if hasattr(os, 'chmod'): - os.chmod(dst, mode) - if hasattr(os, 'chflags') and hasattr(st, 'st_flags'): + utime_func(dst, (st.st_atime, st.st_mtime)) + chmod_func(dst, mode) + if hasattr(st, 'st_flags'): try: - os.chflags(dst, st.st_flags) + chflags_func(dst, st.st_flags) except OSError as why: if (not hasattr(errno, 'EOPNOTSUPP') or why.errno != errno.EOPNOTSUPP): raise -def copy(src, dst): +def copy(src, dst, symlinks=False): """Copy data and mode bits ("cp src dst"). The destination may be a directory. + If the optional flag `symlinks` is set, symlinks won't be followed. This + resembles GNU's "cp -P src dst". + """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) - copyfile(src, dst) - copymode(src, dst) + copyfile(src, dst, symlinks=symlinks) + copymode(src, dst, symlinks=symlinks) -def copy2(src, dst): +def copy2(src, dst, symlinks=False): """Copy data and all stat info ("cp -p src dst"). The destination may be a directory. + If the optional flag `symlinks` is set, symlinks won't be followed. This + resembles GNU's "cp -P src dst". + """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) - copyfile(src, dst) - copystat(src, dst) + copyfile(src, dst, symlinks=symlinks) + copystat(src, dst, symlinks=symlinks) def ignore_patterns(*patterns): """Function that can be used as copytree() ignore parameter. @@ -212,7 +257,11 @@ if os.path.islink(srcname): linkto = os.readlink(srcname) if symlinks: + # We can't just leave it to `copy_function` because legacy + # code with a custom `copy_function` may rely on copytree + # doing the right thing. os.symlink(linkto, dstname) + copystat(srcname, dstname, symlinks=symlinks) else: # ignore dangling symlink if the flag is on if not os.path.exists(linkto) and ignore_dangling_symlinks: diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -164,6 +164,197 @@ self.assertTrue(issubclass(exc[0], OSError)) self.errorState = 2 + @unittest.skipUnless(hasattr(os, 'chmod'), 'requires os.chmod') + @support.skip_unless_symlink + def test_copymode_follow_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'quux') + write_file(src, 'foo') + write_file(dst, 'foo') + os.symlink(src, src_link) + os.symlink(dst, dst_link) + os.chmod(src, stat.S_IRWXU|stat.S_IRWXG) + # file to file + os.chmod(dst, stat.S_IRWXO) + self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + shutil.copymode(src, dst) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # follow src link + os.chmod(dst, stat.S_IRWXO) + shutil.copymode(src_link, dst) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # follow dst link + os.chmod(dst, stat.S_IRWXO) + shutil.copymode(src, dst_link) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # follow both links + os.chmod(dst, stat.S_IRWXO) + shutil.copymode(src_link, dst) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + + @unittest.skipUnless(hasattr(os, 'lchmod'), 'requires os.lchmod') + @support.skip_unless_symlink + def test_copymode_symlink_to_symlink(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'quux') + write_file(src, 'foo') + write_file(dst, 'foo') + os.symlink(src, src_link) + os.symlink(dst, dst_link) + os.chmod(src, stat.S_IRWXU|stat.S_IRWXG) + os.chmod(dst, stat.S_IRWXU) + os.lchmod(src_link, stat.S_IRWXO|stat.S_IRWXG) + # link to link + os.lchmod(dst_link, stat.S_IRWXO) + shutil.copymode(src_link, dst_link, symlinks=True) + self.assertEqual(os.lstat(src_link).st_mode, + os.lstat(dst_link).st_mode) + self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # src link - use chmod + os.lchmod(dst_link, stat.S_IRWXO) + shutil.copymode(src_link, dst, symlinks=True) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # dst link - use chmod + os.lchmod(dst_link, stat.S_IRWXO) + shutil.copymode(src, dst_link, symlinks=True) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + + @unittest.skipIf(hasattr(os, 'lchmod'), 'requires os.lchmod to be missing') + @support.skip_unless_symlink + def test_copymode_symlink_to_symlink_wo_lchmod(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'quux') + write_file(src, 'foo') + write_file(dst, 'foo') + os.symlink(src, src_link) + os.symlink(dst, dst_link) + shutil.copymode(src_link, dst_link, symlinks=True) # silent fail + + @support.skip_unless_symlink + def test_copystat_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'qux') + write_file(src, 'foo') + src_stat = os.stat(src) + os.utime(src, (src_stat.st_atime, + src_stat.st_mtime - 42.0)) # ensure different mtimes + write_file(dst, 'bar') + self.assertNotEqual(os.stat(src).st_mtime, os.stat(dst).st_mtime) + os.symlink(src, src_link) + os.symlink(dst, dst_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXO) + if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'): + os.lchflags(src_link, stat.UF_NODUMP) + src_link_stat = os.lstat(src_link) + # follow + if hasattr(os, 'lchmod'): + shutil.copystat(src_link, dst_link, symlinks=False) + self.assertNotEqual(src_link_stat.st_mode, os.stat(dst).st_mode) + # don't follow + shutil.copystat(src_link, dst_link, symlinks=True) + dst_link_stat = os.lstat(dst_link) + if hasattr(os, 'lutimes'): + for attr in 'st_atime', 'st_mtime': + # The modification times may be truncated in the new file. + self.assertLessEqual(getattr(src_link_stat, attr), + getattr(dst_link_stat, attr) + 1) + if hasattr(os, 'lchmod'): + self.assertEqual(src_link_stat.st_mode, dst_link_stat.st_mode) + if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'): + self.assertEqual(src_link_stat.st_flags, dst_link_stat.st_flags) + # tell to follow but dst is not a link + shutil.copystat(src_link, dst, symlinks=True) + self.assertTrue(abs(os.stat(src).st_mtime - os.stat(dst).st_mtime) < + 00000.1) + + @support.skip_unless_symlink + def test_copy_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + write_file(src, 'foo') + os.symlink(src, src_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO) + # don't follow + shutil.copy(src_link, dst, symlinks=False) + self.assertFalse(os.path.islink(dst)) + self.assertEqual(read_file(src), read_file(dst)) + os.remove(dst) + # follow + shutil.copy(src_link, dst, symlinks=True) + self.assertTrue(os.path.islink(dst)) + self.assertEqual(os.readlink(dst), os.readlink(src_link)) + if hasattr(os, 'lchmod'): + self.assertEqual(os.lstat(src_link).st_mode, + os.lstat(dst).st_mode) + + @support.skip_unless_symlink + def test_copy2_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + write_file(src, 'foo') + os.symlink(src, src_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO) + if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'): + os.lchflags(src_link, stat.UF_NODUMP) + src_stat = os.stat(src) + src_link_stat = os.lstat(src_link) + # follow + shutil.copy2(src_link, dst, symlinks=False) + self.assertFalse(os.path.islink(dst)) + self.assertEqual(read_file(src), read_file(dst)) + os.remove(dst) + # don't follow + shutil.copy2(src_link, dst, symlinks=True) + self.assertTrue(os.path.islink(dst)) + self.assertEqual(os.readlink(dst), os.readlink(src_link)) + dst_stat = os.lstat(dst) + if hasattr(os, 'lutimes'): + for attr in 'st_atime', 'st_mtime': + # The modification times may be truncated in the new file. + self.assertLessEqual(getattr(src_link_stat, attr), + getattr(dst_stat, attr) + 1) + if hasattr(os, 'lchmod'): + self.assertEqual(src_link_stat.st_mode, dst_stat.st_mode) + self.assertNotEqual(src_stat.st_mode, dst_stat.st_mode) + if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'): + self.assertEqual(src_link_stat.st_flags, dst_stat.st_flags) + + @support.skip_unless_symlink + def test_copyfile_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'src') + dst = os.path.join(tmp_dir, 'dst') + dst_link = os.path.join(tmp_dir, 'dst_link') + link = os.path.join(tmp_dir, 'link') + write_file(src, 'foo') + os.symlink(src, link) + # don't follow + shutil.copyfile(link, dst_link, symlinks=True) + self.assertTrue(os.path.islink(dst_link)) + self.assertEqual(os.readlink(link), os.readlink(dst_link)) + # follow + shutil.copyfile(link, dst) + self.assertFalse(os.path.islink(dst)) + def test_rmtree_dont_delete_file(self): # When called on a file instead of a directory, don't delete it. handle, path = tempfile.mkstemp() @@ -190,6 +381,34 @@ actual = read_file((dst_dir, 'test_dir', 'test.txt')) self.assertEqual(actual, '456') + @support.skip_unless_symlink + def test_copytree_symlinks(self): + tmp_dir = self.mkdtemp() + src_dir = os.path.join(tmp_dir, 'src') + dst_dir = os.path.join(tmp_dir, 'dst') + sub_dir = os.path.join(src_dir, 'sub') + os.mkdir(src_dir) + os.mkdir(sub_dir) + write_file((src_dir, 'file.txt'), 'foo') + src_link = os.path.join(sub_dir, 'link') + dst_link = os.path.join(dst_dir, 'sub/link') + os.symlink(os.path.join(src_dir, 'file.txt'), + src_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO) + if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'): + os.lchflags(src_link, stat.UF_NODUMP) + src_stat = os.lstat(src_link) + shutil.copytree(src_dir, dst_dir, symlinks=True) + self.assertTrue(os.path.islink(os.path.join(dst_dir, 'sub', 'link'))) + self.assertEqual(os.readlink(os.path.join(dst_dir, 'sub', 'link')), + os.path.join(src_dir, 'file.txt')) + dst_stat = os.lstat(dst_link) + if hasattr(os, 'lchmod'): + self.assertEqual(dst_stat.st_mode, src_stat.st_mode) + if hasattr(os, 'lchflags'): + self.assertEqual(dst_stat.st_flags, src_stat.st_flags) + def test_copytree_with_exclude(self): # creating data join = os.path.join diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -422,6 +422,11 @@ Library ------- +- Issue #12715: Add an optional symlinks argument to shutil functions + (copyfile, copymode, copystat, copy, copy2). When that parameter is + true, symlinks aren't dereferenced and the operation instead acts on the + symlink itself (or creates one, if relevant). Patch by Hynek Schlawack. + - Add a flags parameter to select.epoll. - Issue #12798: Updated the mimetypes documentation. -- Repository URL: http://hg.python.org/cpython

2 1

Daily reference leaks (7010fa9bd190): sum=0
by solipsis＠pitrou.net Dec. 30, 2011

Dec. 30, 2011

results for 7010fa9bd190 on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogS_Ohzs', '-x']

1 0

cpython: evidently some buildbots don't have python 3 installed
by benjamin.peterson Dec. 29, 2011

Dec. 29, 2011

http://hg.python.org/cpython/rev/7010fa9bd190 changeset: 74197:7010fa9bd190 user: Benjamin Peterson <benjamin(a)python.org> date: Thu Dec 29 12:07:21 2011 -0600 summary: evidently some buildbots don't have python 3 installed files: Parser/asdl_c.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -1,4 +1,4 @@ -#! /usr/bin/env python3 +#! /usr/bin/env python """Generate C code from an ASDL description.""" # TO DO -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge default -> default): merge heads
by benjamin.peterson Dec. 29, 2011

Dec. 29, 2011

http://hg.python.org/cpython/rev/1477d7de4473 changeset: 74196:1477d7de4473 parent: 74195:ac100a4e18b8 parent: 74194:cf57ef65bcd0 user: Benjamin Peterson <benjamin(a)python.org> date: Thu Dec 29 12:04:28 2011 -0600 summary: merge heads files: Doc/library/shutil.rst | 46 ++++- Lib/shutil.py | 101 +++++++++--- Lib/test/test_shutil.py | 219 ++++++++++++++++++++++++++++ Misc/NEWS | 5 + 4 files changed, 333 insertions(+), 38 deletions(-) diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -45,7 +45,7 @@ be copied. -.. function:: copyfile(src, dst) +.. function:: copyfile(src, dst[, symlinks=False]) Copy the contents (no metadata) of the file named *src* to a file named *dst*. *dst* must be the complete target file name; look at :func:`copy` for a copy that @@ -56,37 +56,56 @@ such as character or block devices and pipes cannot be copied with this function. *src* and *dst* are path names given as strings. + If *symlinks* is true and *src* is a symbolic link, a new symbolic link will + be created instead of copying the file *src* points to. + .. versionchanged:: 3.3 :exc:`IOError` used to be raised instead of :exc:`OSError`. + Added *symlinks* argument. -.. function:: copymode(src, dst) +.. function:: copymode(src, dst[, symlinks=False]) Copy the permission bits from *src* to *dst*. The file contents, owner, and - group are unaffected. *src* and *dst* are path names given as strings. + group are unaffected. *src* and *dst* are path names given as strings. If + *symlinks* is true, *src* a symbolic link and the operating system supports + modes for symbolic links (for example BSD-based ones), the mode of the link + will be copied. + .. versionchanged:: 3.3 + Added *symlinks* argument. -.. function:: copystat(src, dst) +.. function:: copystat(src, dst[, symlinks=False]) Copy the permission bits, last access time, last modification time, and flags from *src* to *dst*. The file contents, owner, and group are unaffected. *src* - and *dst* are path names given as strings. + and *dst* are path names given as strings. If *src* and *dst* are both + symbolic links and *symlinks* true, the stats of the link will be copied as + far as the platform allows. + .. versionchanged:: 3.3 + Added *symlinks* argument. -.. function:: copy(src, dst) +.. function:: copy(src, dst[, symlinks=False])) Copy the file *src* to the file or directory *dst*. If *dst* is a directory, a file with the same basename as *src* is created (or overwritten) in the directory specified. Permission bits are copied. *src* and *dst* are path - names given as strings. + names given as strings. If *symlinks* is true, symbolic links won't be + followed but recreated instead -- this resembles GNU's :program:`cp -P`. + .. versionchanged:: 3.3 + Added *symlinks* argument. -.. function:: copy2(src, dst) +.. function:: copy2(src, dst[, symlinks=False]) Similar to :func:`copy`, but metadata is copied as well -- in fact, this is just :func:`copy` followed by :func:`copystat`. This is similar to the - Unix command :program:`cp -p`. + Unix command :program:`cp -p`. If *symlinks* is true, symbolic links won't + be followed but recreated instead -- this resembles GNU's :program:`cp -P`. + .. versionchanged:: 3.3 + Added *symlinks* argument. .. function:: ignore_patterns(\*patterns) @@ -104,9 +123,9 @@ :func:`copy2`. If *symlinks* is true, symbolic links in the source tree are represented as - symbolic links in the new tree, but the metadata of the original links is NOT - copied; if false or omitted, the contents and metadata of the linked files - are copied to the new tree. + symbolic links in the new tree and the metadata of the original links will + be copied as far as the platform allows; if false or omitted, the contents + and metadata of the linked files are copied to the new tree. When *symlinks* is false, if the file pointed by the symlink doesn't exist, a exception will be added in the list of errors raised in @@ -140,6 +159,9 @@ Added the *ignore_dangling_symlinks* argument to silent dangling symlinks errors when *symlinks* is false. + .. versionchanged:: 3.3 + Copy metadata when *symlinks* is false. + .. function:: rmtree(path, ignore_errors=False, onerror=None) diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -82,8 +82,13 @@ return (os.path.normcase(os.path.abspath(src)) == os.path.normcase(os.path.abspath(dst))) -def copyfile(src, dst): - """Copy data from src to dst""" +def copyfile(src, dst, symlinks=False): + """Copy data from src to dst. + + If optional flag `symlinks` is set and `src` is a symbolic link, a new + symlink will be created instead of copying the file it points to. + + """ if _samefile(src, dst): raise Error("`%s` and `%s` are the same file" % (src, dst)) @@ -98,54 +103,94 @@ if stat.S_ISFIFO(st.st_mode): raise SpecialFileError("`%s` is a named pipe" % fn) - with open(src, 'rb') as fsrc: - with open(dst, 'wb') as fdst: - copyfileobj(fsrc, fdst) + if symlinks and os.path.islink(src): + os.symlink(os.readlink(src), dst) + else: + with open(src, 'rb') as fsrc: + with open(dst, 'wb') as fdst: + copyfileobj(fsrc, fdst) -def copymode(src, dst): - """Copy mode bits from src to dst""" - if hasattr(os, 'chmod'): - st = os.stat(src) - mode = stat.S_IMODE(st.st_mode) - os.chmod(dst, mode) +def copymode(src, dst, symlinks=False): + """Copy mode bits from src to dst. -def copystat(src, dst): - """Copy all stat info (mode bits, atime, mtime, flags) from src to dst""" - st = os.stat(src) + If the optional flag `symlinks` is set, symlinks aren't followed if and + only if both `src` and `dst` are symlinks. If `lchmod` isn't available (eg. + Linux), in these cases, this method does nothing. + + """ + if symlinks and os.path.islink(src) and os.path.islink(dst): + if hasattr(os, 'lchmod'): + stat_func, chmod_func = os.lstat, os.lchmod + else: + return + elif hasattr(os, 'chmod'): + stat_func, chmod_func = os.stat, os.chmod + else: + return + + st = stat_func(src) + chmod_func(dst, stat.S_IMODE(st.st_mode)) + +def copystat(src, dst, symlinks=False): + """Copy all stat info (mode bits, atime, mtime, flags) from src to dst. + + If the optional flag `symlinks` is set, symlinks aren't followed if and + only if both `src` and `dst` are symlinks. + + """ + def _nop(*args): + pass + + if symlinks and os.path.islink(src) and os.path.islink(dst): + stat_func = os.lstat + utime_func = os.lutimes if hasattr(os, 'lutimes') else _nop + chmod_func = os.lchmod if hasattr(os, 'lchmod') else _nop + chflags_func = os.lchflags if hasattr(os, 'lchflags') else _nop + else: + stat_func = os.stat + utime_func = os.utime if hasattr(os, 'utime') else _nop + chmod_func = os.chmod if hasattr(os, 'chmod') else _nop + chflags_func = os.chflags if hasattr(os, 'chflags') else _nop + + st = stat_func(src) mode = stat.S_IMODE(st.st_mode) - if hasattr(os, 'utime'): - os.utime(dst, (st.st_atime, st.st_mtime)) - if hasattr(os, 'chmod'): - os.chmod(dst, mode) - if hasattr(os, 'chflags') and hasattr(st, 'st_flags'): + utime_func(dst, (st.st_atime, st.st_mtime)) + chmod_func(dst, mode) + if hasattr(st, 'st_flags'): try: - os.chflags(dst, st.st_flags) + chflags_func(dst, st.st_flags) except OSError as why: if (not hasattr(errno, 'EOPNOTSUPP') or why.errno != errno.EOPNOTSUPP): raise -def copy(src, dst): +def copy(src, dst, symlinks=False): """Copy data and mode bits ("cp src dst"). The destination may be a directory. + If the optional flag `symlinks` is set, symlinks won't be followed. This + resembles GNU's "cp -P src dst". + """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) - copyfile(src, dst) - copymode(src, dst) + copyfile(src, dst, symlinks=symlinks) + copymode(src, dst, symlinks=symlinks) -def copy2(src, dst): +def copy2(src, dst, symlinks=False): """Copy data and all stat info ("cp -p src dst"). The destination may be a directory. + If the optional flag `symlinks` is set, symlinks won't be followed. This + resembles GNU's "cp -P src dst". + """ if os.path.isdir(dst): dst = os.path.join(dst, os.path.basename(src)) - copyfile(src, dst) - copystat(src, dst) + copyfile(src, dst, symlinks=symlinks) + copystat(src, dst, symlinks=symlinks) def ignore_patterns(*patterns): """Function that can be used as copytree() ignore parameter. @@ -212,7 +257,11 @@ if os.path.islink(srcname): linkto = os.readlink(srcname) if symlinks: + # We can't just leave it to `copy_function` because legacy + # code with a custom `copy_function` may rely on copytree + # doing the right thing. os.symlink(linkto, dstname) + copystat(srcname, dstname, symlinks=symlinks) else: # ignore dangling symlink if the flag is on if not os.path.exists(linkto) and ignore_dangling_symlinks: diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -164,6 +164,197 @@ self.assertTrue(issubclass(exc[0], OSError)) self.errorState = 2 + @unittest.skipUnless(hasattr(os, 'chmod'), 'requires os.chmod') + @support.skip_unless_symlink + def test_copymode_follow_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'quux') + write_file(src, 'foo') + write_file(dst, 'foo') + os.symlink(src, src_link) + os.symlink(dst, dst_link) + os.chmod(src, stat.S_IRWXU|stat.S_IRWXG) + # file to file + os.chmod(dst, stat.S_IRWXO) + self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + shutil.copymode(src, dst) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # follow src link + os.chmod(dst, stat.S_IRWXO) + shutil.copymode(src_link, dst) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # follow dst link + os.chmod(dst, stat.S_IRWXO) + shutil.copymode(src, dst_link) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # follow both links + os.chmod(dst, stat.S_IRWXO) + shutil.copymode(src_link, dst) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + + @unittest.skipUnless(hasattr(os, 'lchmod'), 'requires os.lchmod') + @support.skip_unless_symlink + def test_copymode_symlink_to_symlink(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'quux') + write_file(src, 'foo') + write_file(dst, 'foo') + os.symlink(src, src_link) + os.symlink(dst, dst_link) + os.chmod(src, stat.S_IRWXU|stat.S_IRWXG) + os.chmod(dst, stat.S_IRWXU) + os.lchmod(src_link, stat.S_IRWXO|stat.S_IRWXG) + # link to link + os.lchmod(dst_link, stat.S_IRWXO) + shutil.copymode(src_link, dst_link, symlinks=True) + self.assertEqual(os.lstat(src_link).st_mode, + os.lstat(dst_link).st_mode) + self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # src link - use chmod + os.lchmod(dst_link, stat.S_IRWXO) + shutil.copymode(src_link, dst, symlinks=True) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + # dst link - use chmod + os.lchmod(dst_link, stat.S_IRWXO) + shutil.copymode(src, dst_link, symlinks=True) + self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode) + + @unittest.skipIf(hasattr(os, 'lchmod'), 'requires os.lchmod to be missing') + @support.skip_unless_symlink + def test_copymode_symlink_to_symlink_wo_lchmod(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'quux') + write_file(src, 'foo') + write_file(dst, 'foo') + os.symlink(src, src_link) + os.symlink(dst, dst_link) + shutil.copymode(src_link, dst_link, symlinks=True) # silent fail + + @support.skip_unless_symlink + def test_copystat_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + dst_link = os.path.join(tmp_dir, 'qux') + write_file(src, 'foo') + src_stat = os.stat(src) + os.utime(src, (src_stat.st_atime, + src_stat.st_mtime - 42.0)) # ensure different mtimes + write_file(dst, 'bar') + self.assertNotEqual(os.stat(src).st_mtime, os.stat(dst).st_mtime) + os.symlink(src, src_link) + os.symlink(dst, dst_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXO) + if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'): + os.lchflags(src_link, stat.UF_NODUMP) + src_link_stat = os.lstat(src_link) + # follow + if hasattr(os, 'lchmod'): + shutil.copystat(src_link, dst_link, symlinks=False) + self.assertNotEqual(src_link_stat.st_mode, os.stat(dst).st_mode) + # don't follow + shutil.copystat(src_link, dst_link, symlinks=True) + dst_link_stat = os.lstat(dst_link) + if hasattr(os, 'lutimes'): + for attr in 'st_atime', 'st_mtime': + # The modification times may be truncated in the new file. + self.assertLessEqual(getattr(src_link_stat, attr), + getattr(dst_link_stat, attr) + 1) + if hasattr(os, 'lchmod'): + self.assertEqual(src_link_stat.st_mode, dst_link_stat.st_mode) + if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'): + self.assertEqual(src_link_stat.st_flags, dst_link_stat.st_flags) + # tell to follow but dst is not a link + shutil.copystat(src_link, dst, symlinks=True) + self.assertTrue(abs(os.stat(src).st_mtime - os.stat(dst).st_mtime) < + 00000.1) + + @support.skip_unless_symlink + def test_copy_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + write_file(src, 'foo') + os.symlink(src, src_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO) + # don't follow + shutil.copy(src_link, dst, symlinks=False) + self.assertFalse(os.path.islink(dst)) + self.assertEqual(read_file(src), read_file(dst)) + os.remove(dst) + # follow + shutil.copy(src_link, dst, symlinks=True) + self.assertTrue(os.path.islink(dst)) + self.assertEqual(os.readlink(dst), os.readlink(src_link)) + if hasattr(os, 'lchmod'): + self.assertEqual(os.lstat(src_link).st_mode, + os.lstat(dst).st_mode) + + @support.skip_unless_symlink + def test_copy2_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'foo') + dst = os.path.join(tmp_dir, 'bar') + src_link = os.path.join(tmp_dir, 'baz') + write_file(src, 'foo') + os.symlink(src, src_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO) + if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'): + os.lchflags(src_link, stat.UF_NODUMP) + src_stat = os.stat(src) + src_link_stat = os.lstat(src_link) + # follow + shutil.copy2(src_link, dst, symlinks=False) + self.assertFalse(os.path.islink(dst)) + self.assertEqual(read_file(src), read_file(dst)) + os.remove(dst) + # don't follow + shutil.copy2(src_link, dst, symlinks=True) + self.assertTrue(os.path.islink(dst)) + self.assertEqual(os.readlink(dst), os.readlink(src_link)) + dst_stat = os.lstat(dst) + if hasattr(os, 'lutimes'): + for attr in 'st_atime', 'st_mtime': + # The modification times may be truncated in the new file. + self.assertLessEqual(getattr(src_link_stat, attr), + getattr(dst_stat, attr) + 1) + if hasattr(os, 'lchmod'): + self.assertEqual(src_link_stat.st_mode, dst_stat.st_mode) + self.assertNotEqual(src_stat.st_mode, dst_stat.st_mode) + if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'): + self.assertEqual(src_link_stat.st_flags, dst_stat.st_flags) + + @support.skip_unless_symlink + def test_copyfile_symlinks(self): + tmp_dir = self.mkdtemp() + src = os.path.join(tmp_dir, 'src') + dst = os.path.join(tmp_dir, 'dst') + dst_link = os.path.join(tmp_dir, 'dst_link') + link = os.path.join(tmp_dir, 'link') + write_file(src, 'foo') + os.symlink(src, link) + # don't follow + shutil.copyfile(link, dst_link, symlinks=True) + self.assertTrue(os.path.islink(dst_link)) + self.assertEqual(os.readlink(link), os.readlink(dst_link)) + # follow + shutil.copyfile(link, dst) + self.assertFalse(os.path.islink(dst)) + def test_rmtree_dont_delete_file(self): # When called on a file instead of a directory, don't delete it. handle, path = tempfile.mkstemp() @@ -190,6 +381,34 @@ actual = read_file((dst_dir, 'test_dir', 'test.txt')) self.assertEqual(actual, '456') + @support.skip_unless_symlink + def test_copytree_symlinks(self): + tmp_dir = self.mkdtemp() + src_dir = os.path.join(tmp_dir, 'src') + dst_dir = os.path.join(tmp_dir, 'dst') + sub_dir = os.path.join(src_dir, 'sub') + os.mkdir(src_dir) + os.mkdir(sub_dir) + write_file((src_dir, 'file.txt'), 'foo') + src_link = os.path.join(sub_dir, 'link') + dst_link = os.path.join(dst_dir, 'sub/link') + os.symlink(os.path.join(src_dir, 'file.txt'), + src_link) + if hasattr(os, 'lchmod'): + os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO) + if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'): + os.lchflags(src_link, stat.UF_NODUMP) + src_stat = os.lstat(src_link) + shutil.copytree(src_dir, dst_dir, symlinks=True) + self.assertTrue(os.path.islink(os.path.join(dst_dir, 'sub', 'link'))) + self.assertEqual(os.readlink(os.path.join(dst_dir, 'sub', 'link')), + os.path.join(src_dir, 'file.txt')) + dst_stat = os.lstat(dst_link) + if hasattr(os, 'lchmod'): + self.assertEqual(dst_stat.st_mode, src_stat.st_mode) + if hasattr(os, 'lchflags'): + self.assertEqual(dst_stat.st_flags, src_stat.st_flags) + def test_copytree_with_exclude(self): # creating data join = os.path.join diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -422,6 +422,11 @@ Library ------- +- Issue #12715: Add an optional symlinks argument to shutil functions + (copyfile, copymode, copystat, copy, copy2). When that parameter is + true, symlinks aren't dereferenced and the operation instead acts on the + symlink itself (or creates one, if relevant). Patch by Hynek Schlawack. + - Add a flags parameter to select.epoll. - Issue #12798: Updated the mimetypes documentation. -- Repository URL: http://hg.python.org/cpython

1 0

cpython: run AST toolchain on python3
by benjamin.peterson Dec. 29, 2011

Dec. 29, 2011

http://hg.python.org/cpython/rev/ac100a4e18b8 changeset: 74195:ac100a4e18b8 parent: 74193:e3ba7e1dd406 user: Benjamin Peterson <benjamin(a)python.org> date: Thu Dec 29 12:03:55 2011 -0600 summary: run AST toolchain on python3 files: Parser/asdl_c.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -1,4 +1,4 @@ -#! /usr/bin/env python +#! /usr/bin/env python3 """Generate C code from an ASDL description.""" # TO DO -- Repository URL: http://hg.python.org/cpython

1 0

Daily reference leaks (e3ba7e1dd406): sum=0
by solipsis＠pitrou.net Dec. 29, 2011

Dec. 29, 2011

results for e3ba7e1dd406 on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog0TeEK8', '-x']

1 0

cpython (merge default -> default): merge heads
by benjamin.peterson Dec. 28, 2011

Dec. 28, 2011

http://hg.python.org/cpython/rev/e3ba7e1dd406 changeset: 74193:e3ba7e1dd406 parent: 74192:0558a03b1bec parent: 74190:a7744f778646 user: Benjamin Peterson <benjamin(a)python.org> date: Wed Dec 28 12:01:51 2011 -0600 summary: merge heads files: Lib/distutils/tests/test_archive_util.py | 17 ++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py --- a/Lib/distutils/tests/test_archive_util.py +++ b/Lib/distutils/tests/test_archive_util.py @@ -2,6 +2,7 @@ """Tests for distutils.archive_util.""" import unittest import os +import sys import tarfile from os.path import splitdrive import warnings @@ -26,6 +27,18 @@ except ImportError: ZLIB_SUPPORT = False +def can_fs_encode(filename): + """ + Return True if the filename can be saved in the file system. + """ + if os.path.supports_unicode_filenames: + return True + try: + filename.encode(sys.getfilesystemencoding()) + except UnicodeEncodeError: + return False + return True + class ArchiveUtilTestCase(support.TempdirManager, support.LoggingSilencer, @@ -36,6 +49,8 @@ self._make_tarball('archive') @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run') + @unittest.skipUnless(can_fs_encode('årchiv'), + 'File system cannot handle this filename') def test_make_tarball_latin1(self): """ Mirror test_make_tarball, except filename contains latin characters. @@ -43,6 +58,8 @@ self._make_tarball('årchiv') # note this isn't a real word @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run') + @unittest.skipUnless(can_fs_encode('のアーカイブ'), + 'File system cannot handle this filename') def test_make_tarball_extended(self): """ Mirror test_make_tarball, except filename contains extended -- Repository URL: http://hg.python.org/cpython

1 0

cpython: fix weird indentation
by benjamin.peterson Dec. 28, 2011

Dec. 28, 2011

http://hg.python.org/cpython/rev/0558a03b1bec changeset: 74192:0558a03b1bec parent: 74189:456b02030100 user: Benjamin Peterson <benjamin(a)python.org> date: Wed Dec 28 12:01:31 2011 -0600 summary: fix weird indentation files: Objects/abstract.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -1984,7 +1984,7 @@ int PyMapping_Check(PyObject *o) { - return o && o->ob_type->tp_as_mapping && + return o && o->ob_type->tp_as_mapping && o->ob_type->tp_as_mapping->mp_subscript; } -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Limit test scope to those platforms that can save the target filenames.
by jason.coombs Dec. 28, 2011

Dec. 28, 2011

http://hg.python.org/cpython/rev/9b681e0c04ed changeset: 74191:9b681e0c04ed branch: 2.7 parent: 74185:e71e4bd45c89 user: Jason R. Coombs <jaraco(a)jaraco.com> date: Wed Dec 28 11:42:22 2011 -0500 summary: Limit test scope to those platforms that can save the target filenames. Reference #11638. files: Lib/distutils/tests/test_archive_util.py | 17 ++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py --- a/Lib/distutils/tests/test_archive_util.py +++ b/Lib/distutils/tests/test_archive_util.py @@ -4,6 +4,7 @@ import unittest import os +import sys import tarfile from os.path import splitdrive import warnings @@ -34,6 +35,18 @@ except ImportError: zlib = None +def can_fs_encode(filename): + """ + Return True if the filename can be saved in the file system. + """ + if os.path.supports_unicode_filenames: + return True + try: + filename.encode(sys.getfilesystemencoding()) + except UnicodeEncodeError: + return False + return True + class ArchiveUtilTestCase(support.TempdirManager, support.LoggingSilencer, @@ -289,6 +302,8 @@ self._make_tarball(u'archive') @unittest.skipUnless(zlib, "requires zlib") + @unittest.skipUnless(can_fs_encode(u'årchiv'), + 'File system cannot handle this filename') def test_make_tarball_unicode_latin1(self): """ Mirror test_make_tarball, except filename is unicode and contains @@ -297,6 +312,8 @@ self._make_tarball(u'årchiv') # note this isn't a real word @unittest.skipUnless(zlib, "requires zlib") + @unittest.skipUnless(can_fs_encode(u'のアーカイブ'), + 'File system cannot handle this filename') def test_make_tarball_unicode_extended(self): """ Mirror test_make_tarball, except filename is unicode and contains -- Repository URL: http://hg.python.org/cpython

1 0