[Twisted-Python] Twisted on Jython
Hi, we've been asked if Autobahn runs under Jython. Since there is an upcoming Jython version that supports 2.7 Python API, I thought I'd give it a try. To my astonishment, I was able to get Autobahn working with minor efforts ... Tested: Twisted 12.3 under Jython from sources (together with zope.interface 4.05 and setuptools from sources) after applying the following patches. Stock AutobahnPython 0.6 from source. 1) Missing __builtin__.buffer The following is a workaround due to http://bugs.jython.org/issue1521 Include # workaround for Jython, see: http://bugs.jython.org/issue1521 import __builtin__ if not hasattr(__builtin__, 'buffer'): def _buffer(object, offset = None, size = None): if offset is None: offset = 0 if size is None: size = len(object) return object[offset:offset+size] __builtin__.buffer = _buffer at the beginning of "twisted/python/compat.py". 2) Weird platform ident Jython identifies itself as "Java" .. I was testing on Windows. twisted/python/runtime.py Line 36 #'java': 'java', 'java': 'win32', This is necessary on Windows since otherwise Posix API is assumed. Of course above is just a quick hack. C:\Users\oberstet>jython Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52) [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0 Type "help", "copyright", "credits" or "license" for more information.
import sys, os sys.platform 'java1.7.0' os.name 'java'
IMHO, os.name should return the real underlying OS on Jython, not "java" .. what do you think? === Where should those issue be adressed? Should Twisted include workarounds or should I lobby at Jython to "do the right" thing? Cheers, Tobias Mit freundlichen Grüßen, Tobias Oberstein Geschäftsführer +49(0)9131 940 3574 +49(0)176 23 752 055 tobias.oberstein@tavendo.de<mailto:tobias.oberstein@tavendo.de> Skype: toberstein<skype:toberstein?call> Twitter: @oberstet<https://twitter.com/oberstet> Xing: Tobias Oberstein<https://www.xing.com/profile/Tobias_Oberstein> [cid:image003.png@01CE1BF7.08C826D0] Tavendo GmbH · Am Weihersbach 13 · 91074 Herzogenaurach · Deutschland Office / Postanschrift : Waldstrasse 18 · 91054 Erlangen · Deutschland +49(+)700 GO TAVENDO · +49(0)700 46 828 36 36 · www.tavendo.de<http://www.tavendo.de/> Amtsgericht Fürth · HRB 10365 · USt-Id.Nr DE246462916 · Geschäftsführer Tobias Oberstein
it seems that a way to identify the actual OS on which Jython is running can be determined using Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52) [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0 Type "help", "copyright", "credits" or "license" for more information.
import java.lang.System java.lang.System.getProperty('os.name') u'Windows 7'
Mit freundlichen Grüßen, Tobias Oberstein Geschäftsführer +49(0)9131 940 3574 +49(0)176 23 752 055 tobias.oberstein@tavendo.de<mailto:tobias.oberstein@tavendo.de> Skype: toberstein<skype:toberstein?call> Twitter: @oberstet<https://twitter.com/oberstet> Xing: Tobias Oberstein<https://www.xing.com/profile/Tobias_Oberstein> [cid:image001.png@01CE1C06.BD7E36F0] Tavendo GmbH · Am Weihersbach 13 · 91074 Herzogenaurach · Deutschland Office / Postanschrift : Waldstrasse 18 · 91054 Erlangen · Deutschland +49(+)700 GO TAVENDO · +49(0)700 46 828 36 36 · www.tavendo.de<http://www.tavendo.de/> Amtsgericht Fürth · HRB 10365 · USt-Id.Nr DE246462916 · Geschäftsführer Tobias Oberstein Von: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] Im Auftrag von Tobias Oberstein Gesendet: Freitag, 8. März 2013 12:18 An: Twisted general discussion (twisted-python@twistedmatrix.com) Betreff: [Twisted-Python] Twisted on Jython Hi, we've been asked if Autobahn runs under Jython. Since there is an upcoming Jython version that supports 2.7 Python API, I thought I'd give it a try. To my astonishment, I was able to get Autobahn working with minor efforts ... Tested: Twisted 12.3 under Jython from sources (together with zope.interface 4.05 and setuptools from sources) after applying the following patches. Stock AutobahnPython 0.6 from source. 1) Missing __builtin__.buffer The following is a workaround due to http://bugs.jython.org/issue1521 Include # workaround for Jython, see: http://bugs.jython.org/issue1521 import __builtin__ if not hasattr(__builtin__, 'buffer'): def _buffer(object, offset = None, size = None): if offset is None: offset = 0 if size is None: size = len(object) return object[offset:offset+size] __builtin__.buffer = _buffer at the beginning of "twisted/python/compat.py". 2) Weird platform ident Jython identifies itself as "Java" .. I was testing on Windows. twisted/python/runtime.py Line 36 #'java': 'java', 'java': 'win32', This is necessary on Windows since otherwise Posix API is assumed. Of course above is just a quick hack. C:\Users\oberstet>jython Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52) [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0 Type "help", "copyright", "credits" or "license" for more information.
import sys, os sys.platform 'java1.7.0' os.name 'java'
IMHO, os.name should return the real underlying OS on Jython, not "java" .. what do you think? === Where should those issue be adressed? Should Twisted include workarounds or should I lobby at Jython to "do the right" thing? Cheers, Tobias Mit freundlichen Grüßen, Tobias Oberstein Geschäftsführer +49(0)9131 940 3574 +49(0)176 23 752 055 tobias.oberstein@tavendo.de<mailto:tobias.oberstein@tavendo.de> Skype: toberstein<skype:toberstein?call> Twitter: @oberstet<https://twitter.com/oberstet> Xing: Tobias Oberstein<https://www.xing.com/profile/Tobias_Oberstein> [cid:image002.png@01CE1C06.BD7E36F0] Tavendo GmbH · Am Weihersbach 13 · 91074 Herzogenaurach · Deutschland Office / Postanschrift : Waldstrasse 18 · 91054 Erlangen · Deutschland +49(+)700 GO TAVENDO · +49(0)700 46 828 36 36 · www.tavendo.de<http://www.tavendo.de/> Amtsgericht Fürth · HRB 10365 · USt-Id.Nr DE246462916 · Geschäftsführer Tobias Oberstein
On 11:18 am, tobias.oberstein@tavendo.de wrote:
Hi,
we've been asked if Autobahn runs under Jython. Since there is an upcoming Jython version that supports 2.7 Python API, I thought I'd give it a try.
To my astonishment, I was able to get Autobahn working with minor efforts ...
That's very cool. Can you file tickets in the issue tracker and attach patches there? The mailing list is where diffs go to be forgotten. Here's a start: http://twistedmatrix.com/trac/ticket/3413 Jean-Paul
Tested: Twisted 12.3 under Jython from sources (together with zope.interface 4.05 and setuptools from sources) after applying the following patches. Stock AutobahnPython 0.6 from source.
1) Missing __builtin__.buffer
The following is a workaround due to http://bugs.jython.org/issue1521
Include
# workaround for Jython, see: http://bugs.jython.org/issue1521 import __builtin__ if not hasattr(__builtin__, 'buffer'): def _buffer(object, offset = None, size = None): if offset is None: offset = 0 if size is None: size = len(object) return object[offset:offset+size] __builtin__.buffer = _buffer
at the beginning of "twisted/python/compat.py".
2) Weird platform ident
Jython identifies itself as "Java" .. I was testing on Windows.
twisted/python/runtime.py
Line 36
#'java': 'java', 'java': 'win32',
This is necessary on Windows since otherwise Posix API is assumed.
Of course above is just a quick hack.
C:\Users\oberstet>jython Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52) [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0 Type "help", "copyright", "credits" or "license" for more information.
import sys, os sys.platform 'java1.7.0' os.name 'java'
IMHO, os.name should return the real underlying OS on Jython, not "java" .. what do you think?
===
Where should those issue be adressed? Should Twisted include workarounds or should I lobby at Jython to "do the right" thing?
Cheers, Tobias
Mit freundlichen Grüßen,
Tobias Oberstein Geschäftsführer
+49(0)9131 940 3574 +49(0)176 23 752 055 tobias.oberstein@tavendo.de<mailto:tobias.oberstein@tavendo.de> Skype: toberstein<skype:toberstein?call> Twitter: @oberstet<https://twitter.com/oberstet> Xing: Tobias Oberstein<https://www.xing.com/profile/Tobias_Oberstein>
[cid:image003.png@01CE1BF7.08C826D0]
Tavendo GmbH · Am Weihersbach 13 · 91074 Herzogenaurach · Deutschland Office / Postanschrift : Waldstrasse 18 · 91054 Erlangen · Deutschland +49(+)700 GO TAVENDO · +49(0)700 46 828 36 36 · www.tavendo.de<http://www.tavendo.de/> Amtsgericht Fürth · HRB 10365 · USt-Id.Nr DE246462916 · Geschäftsführer Tobias Oberstein
To my astonishment, I was able to get Autobahn working with minor efforts ...
That's very cool. Can you file tickets in the issue tracker and attach patches there? The mailing list is where diffs go to be forgotten.
Here's a start: http://twistedmatrix.com/trac/ticket/3413
Hi Jean-Paul, done: http://twistedmatrix.com/trac/ticket/3413#comment:21 - Tobias
On Mar 8, 2013, at 11:47 AM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
To my astonishment, I was able to get Autobahn working with minor efforts ...
That's very cool. Can you file tickets in the issue tracker and attach patches there? The mailing list is where diffs go to be forgotten.
Here's a start: http://twistedmatrix.com/trac/ticket/3413
Hi Jean-Paul,
Tobias, Note that the primary missing criterion from this ticket is the presence of a buildbot running Jython. Any chance you'd want to contribute another bot for this purpose? Thanks! -glyph
Note that the primary missing criterion from this ticket is the presence of a buildbot running Jython. Any chance you'd want to contribute another bot for this purpose?
Hi Glyph, Ok. Makes sense. That would likely bring up some issues with Jython/Twisted then, which we could address .. Apart from that, my guess is that another important area might be reactor support .. as far as I can see Jython as some Posix wrappers which makes it run on Windows only with the less-than-stellar Windows-Select stuff, not IOCP or such. Same might apply to BSDs and Linux (don't know if they have wrappers for kqeue/epoll). At least from my totally unscientific performance measurements with Autobahn/Twisted, Jython/SELECT is slower than CPython/IOCP, not to speak of PyPy on stuff like kqueue/epoll .. - Tobias
On Mar 11, 2013, at 10:30 AM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
Note that the primary missing criterion from this ticket is the presence of a buildbot running Jython. Any chance you'd want to contribute another bot for this purpose?
Hi Glyph,
Ok. Makes sense. That would likely bring up some issues with Jython/Twisted then, which we could address ..
Apart from that, my guess is that another important area might be reactor support .. as far as I can see Jython as some Posix wrappers which makes it run on Windows only with the less-than-stellar Windows-Select stuff, not IOCP or such. Same might apply to BSDs and Linux (don't know if they have wrappers for kqeue/epoll).
At least from my totally unscientific performance measurements with Autobahn/Twisted, Jython/SELECT is slower than CPython/IOCP, not to speak of PyPy on stuff like kqueue/epoll ..
This is to be expected; Jython generally isn't very fast. If you want speed, use PyPy; it's going to be considerably faster than everything else. If you want to make a Jython-specific reactor for performance or platform-feature reasons, that would be a separate task, well after the basic select reactor and all of our support code works on Jython :). -glyph
On 07:47 pm, tobias.oberstein@tavendo.de wrote:
To my astonishment, I was able to get Autobahn working with minor efforts ...
That's very cool. Can you file tickets in the issue tracker and attach patches there? The mailing list is where diffs go to be forgotten.
Here's a start: http://twistedmatrix.com/trac/ticket/3413
Hi Jean-Paul,
Thanks! Now feel free to harass people however you like in order to get them to pay attention to that issue. :) I suspect one early task will be to get a Jython builder set up on http://buildbot.twistedmatrix.com/ again. Jean-Paul
[snip]
Thanks! Now feel free to harass people however you like in order to get them to pay attention to that issue. :)
I suspect one early task will be to get a Jython builder set up on http://buildbot.twistedmatrix.com/ again.
I can volunteer to run a Jython builder. A couple of weeks ago (2012-12-15) I sent an email to buildbot@twistedmatrix.com to ask for needed slaves. I have not got an answer yet. -- Adi Roiban
Adi Roiban <adi@roiban.ro> writes:
I suspect one early task will be to get a Jython builder set up on http://buildbot.twistedmatrix.com/ again.
I can volunteer to run a Jython builder.
Sure: https://twistedmatrix.com/trac/wiki/ContinuousIntegration/TestSlaveConfigura...
A couple of weeks ago (2012-12-15) I sent an email to buildbot@twistedmatrix.com to ask for needed slaves. I have not got an answer yet.
The slaves that we have are more driven by what people want to support, than by some central authority. There wouldn't be much point in having a jython builder, if there weren't people interested in working on jython support, for example. We don't have a list of slaves that we are looking for, currently. https://twistedmatrix.com/trac/wiki/NeededBuildSlaves Tom
On 06:58 pm, tom.prince@ualberta.net wrote:
Adi Roiban <adi@roiban.ro> writes:
I suspect one early task will be to get a Jython builder set up on http://buildbot.twistedmatrix.com/ again.
I can volunteer to run a Jython builder.
Sure: https://twistedmatrix.com/trac/wiki/ContinuousIntegration/TestSlaveConfigura...
A couple of weeks ago (2012-12-15) I sent an email to buildbot@twistedmatrix.com to ask for needed slaves. I have not got an answer yet.
The slaves that we have are more driven by what people want to support, than by some central authority. There wouldn't be much point in having a jython builder, if there weren't people interested in working on jython support, for example.
That said, adding redundant slaves for existing configurations will result in builds being completed more quickly, right? And can offer more resilience against offline slaves. Faster, more reliable builds can lead to less wasted time during development. So if this is true, and there are offers of hardware where we can run such slaves, I think this is worth pursuing. Jean-Paul
On 9 March 2013 21:53, <exarkun@twistedmatrix.com> wrote:
On 06:58 pm, tom.prince@ualberta.net wrote:
Adi Roiban <adi@roiban.ro> writes:
I suspect one early task will be to get a Jython builder set up on http://buildbot.twistedmatrix.com/ again.
I can volunteer to run a Jython builder.
Sure: https://twistedmatrix.com/trac/wiki/ContinuousIntegration/TestSlaveConfigura...
A couple of weeks ago (2012-12-15) I sent an email to buildbot@twistedmatrix.com to ask for needed slaves. I have not got an answer yet.
The slaves that we have are more driven by what people want to support, than by some central authority. There wouldn't be much point in having a jython builder, if there weren't people interested in working on jython support, for example.
That said, adding redundant slaves for existing configurations will result in builds being completed more quickly, right? And can offer more resilience against offline slaves. Faster, more reliable builds can lead to less wasted time during development. So if this is true, and there are offers of hardware where we can run such slaves, I think this is worth pursuing.
OK. I can provide X86 virtual machines for Twisted project. I have no idea how busy is the buildbot queue for Twisted. Can you please suggest what kind of machines will help the most? Thanks! -- Adi Roiban
[snip]
Thanks! Now feel free to harass people however you like in order to get them to pay attention to that issue. :)
;)
I suspect one early task will be to get a Jython builder set up on http://buildbot.twistedmatrix.com/ again.
Makes sense.
I can volunteer to run a Jython builder.
Thats great! I am interested in getting Twisted supported on Jython. If you could take on that one .. would be nice. I am running a FreeBSD buildbot and I still want to add a PyPy bot also .. it's just I need to find a little time ..
A couple of weeks ago (2012-12-15) I sent an email to buildbot@twistedmatrix.com to ask for needed slaves. I have not got an answer yet.
-- Adi Roiban
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (5)
-
Adi Roiban -
exarkun＠twistedmatrix.com -
Glyph -
Tobias Oberstein -
Tom Prince