
The recent discussion about lexical vs. dynamic scoping reminded me of the following issue, that plagues e.g. the new contextlib.redirect_stdout context manager: the standard file-like objects sys.std* are global instead of thread-local, making their manipulation unsafe in multithreaded programs (this issue is related to lexical vs dynamic scoping because in a language with (optional) dynamic scoping, these objects can be made dynamically scoped, thus solving the issue). Of course, changing sys.std* to being thread-local would be backwards incompatible, but perhaps some new objects, e.g. sys.thread_local_std*, could be added to sys, with print() and related functions using sys.thread_local_std* if it is set and sys.std* otherwise. Antony

On Thu, Feb 6, 2014 at 2:37 PM, Antony Lee <antony.lee@berkeley.edu> wrote:
How much of a problem is this in practice? What's your specific use case? I can't say that's I've ever had such a need, but then I haven't used contextlib.redirect_stdout() in a multi-threaded context. At that point I'd probably opt for using Python's logging system rather than sys.std*.
Regardless of the validity of the proposal, coming up with an API to support thread-local sys.std* should be doable in a backward-compatible way, as you imply. It's just a question of how much complexity we can justify in the underlying implementation and in changes to sys.std* (if any). FYI, for 3.5 I plan on working on two things related to your proposal: * restructuring the sys module (or something like it); * adding an API for managing thread-local contexts (a la decimal). That's not to say that either will find enough support to actually land, but I will be working on them. :) -eric

On Mon, Mar 31, 2014 at 7:22 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
* adding an API for managing thread-local contexts (a la decimal).
Numpy might be another consumer of such an API, for similar reasons to decimal: http://docs.scipy.org/doc/numpy/reference/generated/numpy.errstate.html#nump... An interesting constraint from numpy's point of view is that we want to be able to read the TLS context very quickly (e.g., no allocs) from C code. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org

My specific use case came from embedding the IPython QtConsole ( http://ipython.org/ipython-doc/stable/interactive/qtconsole.html) into a Qt app. Specifically, when a user types "print('foo')" into the QtConsole, I expect 'foo' to be printed in the QtConsole; however when "print('foo')" is called from the application code, I expect 'foo' to be printed in __stdout__ (the original stdout, i.e. the terminal). An in fact, IPython achieves this through temporary stdout redirection while exec'ing user code ( https://github.com/ipython/ipython/blob/master/IPython/kernel/inprocess/ipke... ). Obviously, this breaks down if application code tries to call print while redirection is taking place; more subtly, if user code (that is, typed by the user at the QtConsole) starts a new thread, then it would be nice if that new thread could inherit the current settings (that is, stdout being redirected to print to the QtConsole), which is not the case right now (delayed prints go to the terminal as the global stdout gets restored to __stdout__, not to the QtConsole). Perhaps a bit specific, but that was the original motivation for this request. Antony 2014-03-31 11:22 GMT-07:00 Eric Snow <ericsnowcurrently@gmail.com>:

On Thu, Feb 6, 2014 at 2:37 PM, Antony Lee <antony.lee@berkeley.edu> wrote:
How much of a problem is this in practice? What's your specific use case? I can't say that's I've ever had such a need, but then I haven't used contextlib.redirect_stdout() in a multi-threaded context. At that point I'd probably opt for using Python's logging system rather than sys.std*.
Regardless of the validity of the proposal, coming up with an API to support thread-local sys.std* should be doable in a backward-compatible way, as you imply. It's just a question of how much complexity we can justify in the underlying implementation and in changes to sys.std* (if any). FYI, for 3.5 I plan on working on two things related to your proposal: * restructuring the sys module (or something like it); * adding an API for managing thread-local contexts (a la decimal). That's not to say that either will find enough support to actually land, but I will be working on them. :) -eric

On Mon, Mar 31, 2014 at 7:22 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
* adding an API for managing thread-local contexts (a la decimal).
Numpy might be another consumer of such an API, for similar reasons to decimal: http://docs.scipy.org/doc/numpy/reference/generated/numpy.errstate.html#nump... An interesting constraint from numpy's point of view is that we want to be able to read the TLS context very quickly (e.g., no allocs) from C code. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org

My specific use case came from embedding the IPython QtConsole ( http://ipython.org/ipython-doc/stable/interactive/qtconsole.html) into a Qt app. Specifically, when a user types "print('foo')" into the QtConsole, I expect 'foo' to be printed in the QtConsole; however when "print('foo')" is called from the application code, I expect 'foo' to be printed in __stdout__ (the original stdout, i.e. the terminal). An in fact, IPython achieves this through temporary stdout redirection while exec'ing user code ( https://github.com/ipython/ipython/blob/master/IPython/kernel/inprocess/ipke... ). Obviously, this breaks down if application code tries to call print while redirection is taking place; more subtly, if user code (that is, typed by the user at the QtConsole) starts a new thread, then it would be nice if that new thread could inherit the current settings (that is, stdout being redirected to print to the QtConsole), which is not the case right now (delayed prints go to the terminal as the global stdout gets restored to __stdout__, not to the QtConsole). Perhaps a bit specific, but that was the original motivation for this request. Antony 2014-03-31 11:22 GMT-07:00 Eric Snow <ericsnowcurrently@gmail.com>:
participants (3)
-
Antony Lee
-
Eric Snow
-
Nathaniel Smith