For logging could we look at implementing posix aio on platforms that support it?  I would be willing to help code that.

Justin

On Jul 12, 2012 8:51 AM, <exarkun@twistedmatrix.com> wrote:
On 11:32 am, itamar@itamarst.org wrote:
>On 07/12/2012 05:17 AM, Tristan Seligmann wrote:
>>3) log observers should expect to be inovked in non-reactor threads,
>>making this a Mantissa bug.
>In theory this is the case - the logging howto says "The observer needs
>to be thread safe if you anticipate using threads in your program." The
>API docs don't mention threads, though..
>
>In practice however, probably every custom observer gets this wrong, so
>we really need some infrastructure to make this easier or to fix it.
>Possibly we should have some way to indicate an observer is thread-
>safe,
>assume by default they are not, and if they are not wrap them in a
>reactor.callFromThread wrapper.

The logging system is already a bottleneck in some applications.  Doing
even more work to try to make it properly thread-safe will probably slow
it down even more.

I think it was a mistake to try to make it possible to use
twisted.python.log free-threaded.  I think we should think about getting
rid of this claimed feature.  Perhaps it could be replaced with a more
explicit, probably more restricted, logging feature for non-reactor
threads.

Unless someone can magically fix the threading issues without hurting
performance.  Then, great, go for it.
>I would also like some infrastructure to make the file log observer run
>in its own thread, but that's almost the opposite use case :)

Sure, but that's probably based on top of an asynchronous file I/O API
that's not explicitly thread-based (but probably is thread-based in
practice, because what other kind of asynchronous file I/O even exists).

Also, to explicitly answer Tristan's question:

    It's a bug in the Mantissa log observer.

Jean-Paul

_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python