Athena & browser 2-connection limit

Hi there,
Been thinking about this. It _would_ be cool if you could have more than one LivePage per browser session, like in popups, or even frames.
What do you think: Is there any obvious & fundamental reason why the ReliableMessageDelivery stuff couldn't be modified, so that one instance of it could be shared between several PageWidgets?
Harald

Hello Harald
Try the following before you proceed:
Run the nevow examples, go to calculator with IE7 or greater, then hit F5 for reload in rapid sequence. You will hit the point where the browser sits there seemingly deactivated with a waiting cursor. Wait for the time it takes for the socket to come out of its TIME_WAIT state and see the browser window come alive again.
You can provoke this behavior also by providing a sequence of LivePages, going from one LivePage to the other.
The Athena implementation which was rewritten at SVN version 13786 to implement a better disconnect logic collides in my experience with the limited connections IE's developer decided to use. At any particular time the browser accepts only two open sockets and if the user is fast enough to close and open a series of independent windows such that the tearing down of one connection provokes a 'half open' socket in TIME_WAIT state, IE7 & IE8 wait for the that socket to close completely. This problem is amplified by higher latency systems such when running IP traffic over satellite or mobile networks.
This is in my opinion a fundamental flaw in the current Athena implementation and leads to problems no end, because users WILL simply NOT behave and will do the impossible. User feedback in our case was so, that I was forced to live with the 'old' Athena stuff.
I have a site up which consists of almost only LivePages doing dialogs and fancy stuff, but it's using nevow/athena at SVN version 13786 and it took me an arm and a leg to bring it up to the point where it was running smoothly, now serving a constant load of more than 100 users 24/7. (see www.samschtigjass.ch, goto 'Einzelspieler-Version' for a guest game)
If you or anybody else sees a solution to that IE/TIME_WAIT problem I would be more than happy to invest time into furthering Athenas evolution, I like the concepts underneath and if they're coupled with a decent JScript library the 'real app in the browser' becomes a reality.
Werner
Harald Blåtand wrote:
Hi there,
Been thinking about this. It _would_ be cool if you could have more than one LivePage per browser session, like in popups, or even frames.
What do you think: Is there any obvious & fundamental reason why the ReliableMessageDelivery stuff couldn't be modified, so that one instance of it could be shared between several PageWidgets?
Harald
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

* Werner Thie wthie@thiengineering.ch [2008-09-16 09:08:28 +0200]:
If you or anybody else sees a solution to that IE/TIME_WAIT problem I would be more than happy to invest time into furthering Athenas evolution, I like the concepts underneath and if they're coupled with a decent JScript library the 'real app in the browser' becomes a reality.
FWIW, it's not the TIME_WAIT sockets that are the problem, but the CONNECTED sockets; it looks the code in Athena that tries to hook up an onbeforeunload handler is apparently doing something IE doesn't like, so the handler isn't actually hooked up. As a result, the code that would cause the transport connection(s) to be torn down never executes, and they only go away once the timeout (5 minutes by default, I believe) has expired.

Hi Tristan, how are you?
On Tue, Sep 16, 2008 at 12:22 PM, Tristan Seligmann <mithrandi@mithrandi.net
wrote:
- Werner Thie wthie@thiengineering.ch [2008-09-16 09:08:28 +0200]:
If you or anybody else sees a solution to that IE/TIME_WAIT problem I would be more than happy to invest time into furthering Athenas evolution, I like the concepts underneath and if they're coupled with a decent JScript library the 'real app in the browser' becomes a reality.
FWIW, it's not the TIME_WAIT sockets that are the problem, but the CONNECTED sockets; it looks the code in Athena that tries to hook up an onbeforeunload handler is apparently doing something IE doesn't like, so the handler isn't actually hooked up. As a result, the code that would cause the transport connection(s) to be torn down never executes, and they only go away once the timeout (5 minutes by default, I believe) has expired. --
Er.. Do you remember we talked about tearing down IE's connections some time ago? I did something to Divmod.Base.js that makes this work (so we have no problems with this anymore); think I even posted it. Had a look at the upcoming Nevow 0.9.32 SVN last week though, and couldn't find it. Should I have sent it elsewhere; maybe created a ticket? Are ordinary citizens allowed to create tickets in Divmod's trac?
Would be very interested in your thoughts on the original question in this thread - how do you feel about that??
Cheers, Harald

* Harald Blåtand kozneb@gmail.com [2008-09-16 12:50:48 +0200]:
Er.. Do you remember we talked about tearing down IE's connections some time ago? I did something to Divmod.Base.js that makes this work (so we have no problems with this anymore); think I even posted it. Had a look at the upcoming Nevow 0.9.32 SVN last week though, and couldn't find it. Should I have sent it elsewhere; maybe created a ticket? Are ordinary citizens allowed to create tickets in Divmod's trac?
Yes, although I believe you will need to create an account. A ticket would definitely be a good way to make sure it doesn't get lost.
Would be very interested in your thoughts on the original question in this thread - how do you feel about that??
It's an interesting idea, but is it actually possible to share it like that? I'm not very familiar with JS security protocols in the browser, so I'm not sure if it would run afoul of those restrictions or not. It would also mean introducing a separate layer to multiplex between all active livepages, since they would be sharing the same transports. Ideally, browsers would just stop being so restrictive about these things ;)

There are some post revolving around onbeforeunload/IE7 and a quick debugging session shows that no code is ever executed for onbeforeunload with IE.
Posts suggest that returning values from a func assigned to onbeforeunload are a nogo and that nulling the func is also leading to problems.
http://bustedmug.blogspot.com/2007/01/onbeforeunload-ie7-assigning-event.htm...
MS has comfirmed that the onbeforeunload event handler has an issue: http://support.microsoft.com/kb/946214/en-us
I would be more than happy to move up to the current implementation of nevow if there is a fix for this problem.python
I brought this issue up on the list last year when testing with IE7 beta but nobody was able to help ending in a few posts to divmod-dev on 11/14/2007
Thanks for your help, Werner
Tristan Seligmann wrote:
- Werner Thie wthie@thiengineering.ch [2008-09-16 09:08:28 +0200]:
If you or anybody else sees a solution to that IE/TIME_WAIT problem I would be more than happy to invest time into furthering Athenas evolution, I like the concepts underneath and if they're coupled with a decent JScript library the 'real app in the browser' becomes a reality.
FWIW, it's not the TIME_WAIT sockets that are the problem, but the CONNECTED sockets; it looks the code in Athena that tries to hook up an onbeforeunload handler is apparently doing something IE doesn't like, so the handler isn't actually hooked up. As a result, the code that would cause the transport connection(s) to be torn down never executes, and they only go away once the timeout (5 minutes by default, I believe) has expired.
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

On Tue, Sep 16, 2008 at 4:39 PM, Werner Thie wthie@thiengineering.chwrote:
There are some post revolving around onbeforeunload/IE7 and a quick debugging session shows that no code is ever executed for onbeforeunload with IE.
I came up with the following some months ago. Since then, we're not seeing
the problem on IE6 or 7. Would be interested in hearing whether it works for you. (Would have entered it as a Divmod ticket, but can't seem to find a way to create one, nor to create an account...)
Harald
------------------------------------------------------------------------------------------------------------------------ Here's what I've changed in the 0.9.31 JS code:
------------------ Nevow/Athena/__init__.js (2 additions) -------------------------- Nevow.Athena.Widget._initialize = function() { Divmod.debug("widget", "Instantiating live widgets"); Nevow.Athena.Widget._pageLoaded = true; // With the next line, sendCloseMessage will get called in IE. // Harald Divmod.Base.addUnLoadEvent(Nevow.Athena.page.deliveryChannel); // Harald Nevow.Athena.Widget._instantiateWidgets(); Divmod.debug("widget", "Finished instantiating live widgets"); }; Nevow.Athena.bootstrap = function (pageClassName, clientID) { var self = this; var pageClass = Divmod.namedAny(pageClassName); self.page = pageClass(clientID, Nevow.Athena._createMessageDelivery); Nevow.Athena.page = self.page; // Harald self.page.bindEvents(window); .....
------------------ Divmod/Base.js (1 added function at the end) -------------------------- Divmod.Base.addUnLoadEvent = function(channel) { /*** Harald's desperate try to get IE page refresh to work... ***/ window.attachEvent("onunload", function (e) {channel.sendCloseMessage();}); }; Divmod.Base.jsonRegistry = Divmod.Base.AdapterRegistry();
I'm not used to this coding style, so it's a bit of trial & error. Anyway, before, sendCloseMessage would _not_ be called; now it is, and I can refresh ad lib on IE 6.and 7.

On 04:30 pm, kozneb@gmail.com wrote:
On Tue, Sep 16, 2008 at 4:39 PM, Werner Thie wthie@thiengineering.chwrote:
There are some post revolving around onbeforeunload/IE7 and a quick debugging session shows that no code is ever executed for onbeforeunload with IE.
I came up with the following some months ago. Since then, we're not seeing
the problem on IE6 or 7. Would be interested in hearing whether it works for you. (Would have entered it as a Divmod ticket, but can't seem to find a way to create one, nor to create an account...)
Please feel free to enter a ticket, and to attach a patch.
We disabled registration due to some problems with spam and forgot to turn it back on. It's on for now, but I'd recommend registering soon because we are likely to turn it off again if we start having spam problems again :).
When you do file a ticket, please file it with a description of the *problem*, not a description of your changes. i.e. "Reloading an athena page in IE 6 and 7 causes a socket leak", not, "Make sendCloseMessage work in IE". It would be helpful if you were as detailed as possible about the problem.
It would also be very helpful if you could write some tests which demonstrate the problem; since the main development team of athena does very little testing in IE, a test object which thoroughly documents and fakes each version of IE's behavior with respect to these hooks would be especially helpful.

On Tue, Sep 16, 2008 at 9:46 PM, glyph@divmod.com wrote:
Please feel free to enter a ticket, and to attach a patch.
We disabled registration due to some problems with spam and forgot to turn it back on. It's on for now, but I'd recommend registering soon because we are likely to turn it off again if we start having spam problems again :).
Thanks glyph; we're almost there :-)
ErrorThe password file could not be updated. Trac requires read and write access to both the password file and its parent directory.

On 08:28 pm, kozneb@gmail.com wrote:
On Tue, Sep 16, 2008 at 9:46 PM, glyph@divmod.com wrote:
We disabled registration due to some problems with spam and forgot to turn it back on. It's on for now, but I'd recommend registering soon because we are likely to turn it off again if we start having spam problems again :).
Thanks glyph; we're almost there :-)
Error (...)
OK, it should *actually* work now :).

May I pick up this particular problem and its proposed solution up again:
In
Nevow.Athena.bootstrap = function (pageClassName, clientID) { var self = this; var pageClass = Divmod.namedAny(pageClassName); self.page = pageClass(clientID, Nevow.Athena._createMessageDelivery); self.page.bindEvents(window);
Divmod.Runtime.theRuntime.addLoadEvent(function transportStartup() { Divmod.debug("transport", "starting up"); self.page.deliveryChannel.start(); Divmod.debug("transport", "started up");
Divmod.debug("widget", "Instantiating live widgets"); Nevow.Athena.Widget._pageLoaded = true; Nevow.Athena.Widget._instantiateWidgets(); Divmod.debug("widget", "Finished instantiating live widgets"); window.document.body.onbeforeunload = window.onbeforeunload; }); };
window.onbeforeunload gets assigned a callstack with bindEvents(window). Odd though that the debugger shows the property still as null in IE when stopping after bindEvents. The post on bustedmug http://bustedmug.blogspot.com/2007/01/onbeforeunload-ie7-assigning-event.htm... and some debugging revealed that at least under IE7 window.onbeforeunload cannot be assigned to until window.document.body exists.
Adding the really odd assigment
window.document.body.onbeforeunload = window.onbeforeunload;
in the function transportStartup sets up everything perfectly for the IE browsers, does not disturb the other browsers tested, triggers the onbeforeunload reliably and refresh ad lib is possible with all tested browsers given down below, the lockups on the IE family are gone.
Tested calculator example with Firefox 2.0.0.14 Firefox 3.03 Chrome Build 2200 Safari 3.1.2 Opera 9.24 IE6.0.2800 IE7.0.5730 IE8.0.6001 (Beta)
How shall I proceed? Was there a ticket opened by Harald Blåtand?
Thanks, Werner
Harald Blåtand wrote:
On Tue, Sep 16, 2008 at 4:39 PM, Werner Thie wthie@thiengineering.ch wrote:
There are some post revolving around onbeforeunload/IE7 and a quick debugging session shows that no code is ever executed for onbeforeunload with IE.
I came up with the following some months ago. Since then, we're not seeing the problem on IE6 or 7. Would be interested in hearing whether it works for you. (Would have entered it as a Divmod ticket, but can't seem to find a way to create one, nor to create an account...)
Harald
Here's what I've changed in the 0.9.31 JS code:
------------------ Nevow/Athena/__init__.js (2 additions)
Nevow.Athena.Widget._initialize = function() { Divmod.debug("widget", "Instantiating live widgets"); Nevow.Athena.Widget._pageLoaded = true; // With the next line, sendCloseMessage will get called in IE. // Harald Divmod.Base.addUnLoadEvent(Nevow.Athena.page.deliveryChannel); // Harald Nevow.Athena.Widget._instantiateWidgets(); Divmod.debug("widget", "Finished instantiating live widgets"); }; Nevow.Athena.bootstrap = function (pageClassName, clientID) { var self = this; var pageClass = Divmod.namedAny(pageClassName); self.page = pageClass(clientID, Nevow.Athena._createMessageDelivery); Nevow.Athena.page = self.page; // Harald self.page.bindEvents(window); .....
------------------ Divmod/Base.js (1 added function at the end)
Divmod.Base.addUnLoadEvent = function(channel) { /*** Harald's desperate try to get IE page refresh to work... ***/ window.attachEvent("onunload", function (e) {channel.sendCloseMessage();}); }; Divmod.Base.jsonRegistry = Divmod.Base.AdapterRegistry();
I'm not used to this coding style, so it's a bit of trial & error. Anyway, before, sendCloseMessage would _not_ be called; now it is, and I can refresh ad lib on IE 6.and 7.
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

On Tue, Oct 7, 2008 at 7:18 PM, Werner Thie wthie@thiengineering.ch wrote:
May I pick up this particular problem and its proposed solution up again:
Was there a ticket opened by Harald Blåtand?
Yes: http://divmod.org/trac/ticket/2731 .
(The Subject of this thread has become a little misleading.)
Harald

Filed my proposed fix with attached unified diff as response to Haralds ticket.
For those interested the fix also works when used at application code level, just copy the line
window.document.body.onbeforeunload = window.onbeforeunload;
as last line into one of the widgets __init__ function.
Werner
Harald Blåtand wrote:
On Tue, Oct 7, 2008 at 7:18 PM, Werner Thie wthie@thiengineering.ch wrote:
May I pick up this particular problem and its proposed solution up again: Was there a ticket opened by Harald Blåtand?
Yes: http://divmod.org/trac/ticket/2731 .
(The Subject of this thread has become a little misleading.)
Harald
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

What is the current status of this fix please?
I have to find a fix for our athena app locking up in IE but I'm having difficulty following the comments in #2731 - am I right in thinking that the problem is fixed on the ie-onbeforeunload-2731-3 branch but hasn't yet been applied to the trunk?
I'm running version 0.9.31 and while I'd like to upgrade eventually to get Safari support, I really need a fix for IE7 without changing too much.
Paul.
On 8 Oct 2008, at 08:21, Werner Thie wrote:
Filed my proposed fix with attached unified diff as response to Haralds ticket.
For those interested the fix also works when used at application code level, just copy the line
window.document.body.onbeforeunload = window.onbeforeunload;
as last line into one of the widgets __init__ function.
Werner
Harald Blåtand wrote:
On Tue, Oct 7, 2008 at 7:18 PM, Werner Thie wthie@thiengineering.ch wrote: May I pick up this particular problem and its proposed solution up again: Was there a ticket opened by Harald Blåtand? Yes: http://divmod.org/trac/ticket/2731 . (The Subject of this thread has become a little misleading.) Harald
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

* Paul Thomas spongelavapaul@googlemail.com [2008-12-17 10:18:29 +0000]:
What is the current status of this fix please?
I have to find a fix for our athena app locking up in IE but I'm having difficulty following the comments in #2731 - am I right in thinking that the problem is fixed on the ie-onbeforeunload-2731-3 branch but hasn't yet been applied to the trunk?
That's correct; the last few issues with this branch are relatively minor, so with any luck it'll be merged in the near future.

On 17 Dec 2008, at 11:29, Tristan Seligmann wrote:
- Paul Thomas spongelavapaul@googlemail.com [2008-12-17 10:18:29
+0000]:
What is the current status of this fix please?
I have to find a fix for our athena app locking up in IE but I'm having difficulty following the comments in #2731 - am I right in thinking that the problem is fixed on the ie-onbeforeunload-2731-3 branch but hasn't yet been applied to the trunk?
That's correct; the last few issues with this branch are relatively minor, so with any luck it'll be merged in the near future.
That's good to hear. I'm running the branch with our app right now and it's looking generally good. IE seems to behave and the whole experience is 'snappier'. Our app makes fairly heavy use of Athena with multiple widgets.
I've hit one problem with a FlattenError but I suspect it's more to do with moving from such an old version rather than anything to do with the branch changes. I'll give it a good thrashing and let you know if anything bad crops up.
Paul.

On 07:08 am, wthie@thiengineering.ch wrote:
Hello Harald
Try the following before you proceed:
Run the nevow examples, go to calculator with IE7 or greater, then hit F5 for reload in rapid sequence. You will hit the point where the browser sits there seemingly deactivated with a waiting cursor. Wait for the time it takes for the socket to come out of its TIME_WAIT state and see the browser window come alive again.
You can provoke this behavior also by providing a sequence of LivePages, going from one LivePage to the other.
The Athena implementation which was rewritten at SVN version 13786 to
This doesn't look like a rewrite of Athena:
http://divmod.org/trac/changeset/13786
I think you meant SVN version 14017?
http://divmod.org/trac/changeset/14017
Or maybe something else?
implement a better disconnect logic collides in my experience with the limited connections IE's developer decided to use. At any particular time the browser accepts only two open sockets (...)
This is in my opinion a fundamental flaw in the current Athena implementation and leads to problems no end, because users WILL simply NOT behave and will do the impossible. User feedback in our case was so, that I was forced to live with the 'old' Athena stuff.
I don't think the flaw is in any way "fundamental". The whole design of Athena is based around the idea that all the pieces are pluggable and can be substituted. For example, ReliableMessageDelivery is a separate class specifically so that it can be modified in isolation and we can change this strategy without touching the rest of athena.
If you or anybody else sees a solution to that IE/TIME_WAIT problem I would be more than happy to invest time into furthering Athenas evolution, I like the concepts underneath and if they're coupled with a decent JScript library the 'real app in the browser' becomes a reality.
Several things come to mind.
One is that you could simply submit a patch to ReliableMessageDelivery to make it better. There are a number of obvious flaws with the current implementation and it would be nice if it could be fixed. This is not a very specific idea because browsers are awful, and I don't know of any collection of pure-Javascript gross kludges to work around their awfulness better than the current collection of gross kludges.
Another is that we could abuse a browser extension that supports real sockets - say, Flash - to provide a better "default" experience, but fall back to the current crummy behavior for the few users that don't have it. A long time ago we did this, and there was a flash movie called 'flashconduit' included with Nevow's predecessor (Woven). At the time this was impossible to build and test either in an automated way or on an open-source platform, so we abandoned that strategy.
Recently I've been playing with Flex, which is (among other things) a toolkit for building Flash movies out of actual source code rather than the atrocity that is a .FLA file. It's also amenable to automated builds and testing, which FLA files had impeded in the past.
Java is another option, and although the typically long startup time for applets wouldn't make it ideal, it might be a reasonable backup option. I hear .NET has something called "no touch deployment", and Firefox has a socket API for extensions which an extension could selectively expose to the web page context.
Anyway, all of these options are really just different ways that the RDM might be adjusted; all of them could be implemented and the best option selected automatically based on the user's browser. Given that you seem to have some specific requirements based on specific user experiences, I hope that you'll contribute patches that we can discuss and test in a variety of different browser situations.

On Sep 15, 2008, at 2:23 PM, Harald Blåtand wrote:
Hi there,
Been thinking about this. It _would_ be cool if you could have more than one LivePage per browser session, like in popups, or even frames.
What do you think: Is there any obvious & fundamental reason why the ReliableMessageDelivery stuff couldn't be modified, so that one instance of it could be shared between several PageWidgets?
I'm not sure if I understand exactly what you're trying to do, but I have implemented a multi-window Athena app by sharing the LiveElement among the windows.
The drawback is that if the window holding your rendered LiveElement goes away, your child windows will lose their connection to the server.
In my case, I just have one LiveElement that handles all the communication between client and server. This is part of the main window, which also has a function like this:
function getConnector(){ return Nevow.Athena.Widget.get($('athenaid:1-client- connector')); }
At this point I can callRemote() on the returned object instance.
This works for me because of the simplicity of my application; with only the one LiveElement it seems that the Athena ID never changes, so this has been working really well for popping up editors and complex dialogs and so forth.
Also, I don't think I've ever tested this very thoroughly on IE, so YMMV.
-phil

On Sep 16, 2008, at 10:46 AM, Phil Christensen wrote:
On Sep 15, 2008, at 2:23 PM, Harald Blåtand wrote:
Hi there,
Been thinking about this. It _would_ be cool if you could have more than one LivePage per browser session, like in popups, or even frames.
What do you think: Is there any obvious & fundamental reason why the ReliableMessageDelivery stuff couldn't be modified, so that one instance of it could be shared between several PageWidgets?
I'm not sure if I understand exactly what you're trying to do, but I have implemented a multi-window Athena app by sharing the LiveElement among the windows.
The drawback is that if the window holding your rendered LiveElement goes away, your child windows will lose their connection to the server.
In my case, I just have one LiveElement that handles all the communication between client and server. This is part of the main window, which also has a function like this:
function getConnector(){ return Nevow.Athena.Widget.get($('athenaid:1-client- connector')); }
Just wanted to mention that $ is just an alias for document.getElementById, it's not prototype or jquery or anything like that.
At this point I can callRemote() on the returned object instance.
This works for me because of the simplicity of my application; with only the one LiveElement it seems that the Athena ID never changes, so this has been working really well for popping up editors and complex dialogs and so forth.
Also, I don't think I've ever tested this very thoroughly on IE, so YMMV.

Been thinking about this. It _would_ be cool if you could have more than one LivePage per browser session, like in popups, or even frames.
What do you think: Is there any obvious & fundamental reason why the ReliableMessageDelivery stuff couldn't be modified, so that one instance of it could be shared between several PageWidgets?
Do you think 'multiple browser windows'?
If so, how would you cope with the different implementations the browsers have concerning their information sharing between windows, tabs?
IE, Chrome Safari, Opera handle session information differently from Firefox, which shares a lot of information between windows and tabs. How would you distinguish tabs and windows?
Session and LivePages I have successfully used combined although one has to be careful with the toremember dict used in sessions and in general how state information is stored, because its very easy to run into non collectable garbage problems.
I am currently trying to combine ext2.2 with athena, which is in my opinion one of the possible ways to go for RIAs. I am also very interested in AFLAX on the client side but have very limited time to do experiments.
Werner
participants (6)
-
glyph@divmod.com
-
Harald Blåtand
-
Paul Thomas
-
Phil Christensen
-
Tristan Seligmann
-
Werner Thie