getSession, componentized but not async?
I'm curious about request.getSession Per the docs: https://twistedmatrix.com/documents/current/web/howto/using-twistedweb.html It seems like it's tricky to use correctly. My code needs to: - define an Interface - define a class that implements my Interface - call registerAdapter, passing in my class, server.Session and my Interface - get a Session instance - brace myself - create an instance of my Interface class, passing in the Session instance - update that Interface instance.... it will persisted, but only in-memory By default, that data doesn't go to database/memcached/whatever, so it's only accessible in-process On first blush, that seems like a lot of legwork. I'm not clear on what utility's provided by all this versus say maintaining a dict as a class-attribute on Site or something. It's also (for me) counter-intuitive to be creating an instance of an _Interface_ and poking data into it. Also I was a bit surprised that getSession doesn't return a deferred, since it seems like it'd be common to want to persist session data in an external store so that multiple twisted-web processes can access it in a clustered/load-balanced setup. How do other folks go about that? I hacked something together a while ago to run session data into Redis, but what I ended up with required so much surgery on twisted web's classes that I figured I must be doing it wrong. I think Site, SessionFactory and Request were all customised. I was thinking about this again in the context of Cory's "Implement server-side HTTP/2 server push" ticket: https://twistedmatrix.com/trac/ticket/8485 In this context, I'd like to have access to my session data in multiple Resource objects without _necessarily_ having to round-trip to an external store each time to get/put the same data. In the case of http 1.1 requests, I guess there's no way around that round-trip, so it might be optimal if my Resource objects could be oblivious to the underlying protocol version and Session get/put mechanism. So it'd be great if the default Session mechanism could take care of me there, and I could just have my cake and eat it. Another wrinkle that surprised me when I was hacking on this was that there didn't seem to be a way to uniquely identify a request instance, so within the session code it was impossible to tell if two calls to getSession were coming from different points in the callback chain responding to a single Request, or if the second belonged to a different Request entirely. So my confusion is probably apparent at this stage :) I'm guessing others have been here before me. What approaches have you taken to storing your sessions? Are there good open source projects that I should look to for best practice? Thanks! DJM
Creating the session follows the normal Twisted modus operandi-- create a factory that returns the actual thing you want and attach the session to your site. I wrote an authenticating web proxy[1] that does just that. If you skim down to the `sessionFactory()` function, you can see it is not overly complicated. The docs on storing objects in a session[2] are a bit complex. They are basically taking the approach where the idea is when you can `request.getSession()`, you can pass in an interface and get back an object that corresponds to that interface. This can actually be useful in larger systems where you are making use of heavily componentized code, but in some cases it does seem like overkill. Since the main thing that the stock `Session` buys you is expiration, you can use the `Session.notifyOnExpire()` method to update other data structures. My authenticating proxy took that approach for keeping track of authenticated users (see the "txcasproxy.py" file around the `_expired` function). As for persisting session data-- I guess the idea is that storing or retrieving the session object doesn't need to be async because it is just an object in memory corresponding to a cookie in the request. However, writing to or reading from the session could definitely require async methods that talk to a back end store. This could potentially be one area where implementing a component with an interface like `IRedisSessionStorage` might be useful (get session synchonously from request passing in IRedisStorage, then call `storeThingInSession()` which returns a deferred). Thanks, Carl [1] https://github.com/cwaldbieser/txcasproxy/blob/master/txcasproxy/service.py [2] https://twistedmatrix.com/documents/current/web/howto/web-in-60/session-stor... On Tue, Jul 5, 2016 at 12:03 PM, Donal McMullan <donal.mcmullan@gmail.com> wrote:
I'm curious about request.getSession
Per the docs: https://twistedmatrix.com/documents/current/web/howto/using-twistedweb.html
It seems like it's tricky to use correctly. My code needs to: - define an Interface - define a class that implements my Interface - call registerAdapter, passing in my class, server.Session and my Interface - get a Session instance - brace myself - create an instance of my Interface class, passing in the Session instance - update that Interface instance.... it will persisted, but only in-memory
By default, that data doesn't go to database/memcached/whatever, so it's only accessible in-process
On first blush, that seems like a lot of legwork. I'm not clear on what utility's provided by all this versus say maintaining a dict as a class-attribute on Site or something. It's also (for me) counter-intuitive to be creating an instance of an _Interface_ and poking data into it.
Also I was a bit surprised that getSession doesn't return a deferred, since it seems like it'd be common to want to persist session data in an external store so that multiple twisted-web processes can access it in a clustered/load-balanced setup. How do other folks go about that?
I hacked something together a while ago to run session data into Redis, but what I ended up with required so much surgery on twisted web's classes that I figured I must be doing it wrong. I think Site, SessionFactory and Request were all customised.
I was thinking about this again in the context of Cory's "Implement server-side HTTP/2 server push" ticket: https://twistedmatrix.com/trac/ticket/8485
In this context, I'd like to have access to my session data in multiple Resource objects without _necessarily_ having to round-trip to an external store each time to get/put the same data. In the case of http 1.1 requests, I guess there's no way around that round-trip, so it might be optimal if my Resource objects could be oblivious to the underlying protocol version and Session get/put mechanism.
So it'd be great if the default Session mechanism could take care of me there, and I could just have my cake and eat it.
Another wrinkle that surprised me when I was hacking on this was that there didn't seem to be a way to uniquely identify a request instance, so within the session code it was impossible to tell if two calls to getSession were coming from different points in the callback chain responding to a single Request, or if the second belonged to a different Request entirely.
So my confusion is probably apparent at this stage :)
I'm guessing others have been here before me. What approaches have you taken to storing your sessions? Are there good open source projects that I should look to for best practice?
Thanks!
DJM
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
This can actually be useful in larger systems where you are making use of heavily componentized code, but in some cases it does seem like overkill.
That's close to the heart of the question. In larger systems, what is useful about the componentized session objects in particular (as opposed to components/adapters/interfaces in general)? It's a very particular API, so I guess there's a reason for that.
As for persisting session data-- I guess the idea is that storing or retrieving the session object doesn't need to be async because it is just an object in memory corresponding to a cookie in the request.
But doesn't that mean it's impossible to restart the process without destroying user session data? That doesn't seem ok. Isn't that a problem for your cas proxy? Thanks Carl DJM
On Jul 7, 2016, at 7:50 AM, Donal McMullan <donal.mcmullan@gmail.com> wrote:
This can actually be useful in larger systems where you are making use of heavily componentized code, but in some cases it does seem like overkill. That's close to the heart of the question. In larger systems, what is useful about the componentized session objects in particular (as opposed to components/adapters/interfaces in general)? It's a very particular API, so I guess there's a reason for that.
The issue is that larger systems may be integrating several components, each of which may have its own requirements of session data. Each stipulates those requirements in terms of an interface, and then the session is a mapping of {interface_describing_required_behavior: application_specific_behavior}. The reason this works better in "larger" systems is that if you have only a single codebase with only a single set of requirements on its session, you'll only ever need a single key in that mapping, and a single interface.
As for persisting session data-- I guess the idea is that storing or retrieving the session object doesn't need to be async because it is just an object in memory corresponding to a cookie in the request.
But doesn't that mean it's impossible to restart the process without destroying user session data? That doesn't seem ok. Isn't that a problem for your cas proxy?
No; the idea is that you have an object in the session with Deferred-returning methods to retrieve data from your data store. What the session store is doing synchronously is simply converting the cookie data into a token that can be used to access a back-end; the actual data lives in the back-end. -glyph
No; the idea is that you have an object in the session with Deferred-returning methods to retrieve data from your data store.
Ahhhh - Carl was trying to explain that. Sorry guys. I get it finally. The issue is that larger systems may be integrating several components,
each of which may have its own requirements of session data.
Hmm. Thanks glyph DJM On 7 July 2016 at 23:24, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Jul 7, 2016, at 7:50 AM, Donal McMullan <donal.mcmullan@gmail.com> wrote:
This can actually be useful in larger systems where you are making use of
heavily componentized code, but in some cases it does seem like overkill.
That's close to the heart of the question. In larger systems, what is useful about the componentized session objects in particular (as opposed to components/adapters/interfaces in general)? It's a very particular API, so I guess there's a reason for that.
The issue is that larger systems may be integrating several components, each of which may have its own requirements of session data. Each stipulates those requirements in terms of an interface, and then the session is a mapping of {interface_describing_required_behavior: application_specific_behavior}.
The reason this works better in "larger" systems is that if you have only a single codebase with only a single set of requirements on its session, you'll only ever need a single key in that mapping, and a single interface.
As for persisting session data-- I guess the idea is that storing or
retrieving the session object doesn't need to be async because it is just an object in memory corresponding to a cookie in the request.
But doesn't that mean it's impossible to restart the process without destroying user session data? That doesn't seem ok. Isn't that a problem for your cas proxy?
No; the idea is that you have an object in the session with Deferred-returning methods to retrieve data from your data store. What the session store is doing synchronously is simply converting the cookie data into a token that can be used to access a back-end; the actual data lives in the back-end.
-glyph
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
On Jul 7, 2016, at 7:54 PM, Donal McMullan <donal.mcmullan@gmail.com> wrote:
No; the idea is that you have an object in the session with Deferred-returning methods to retrieve data from your data store.
Ahhhh - Carl was trying to explain that. Sorry guys. I get it finally.
The issue is that larger systems may be integrating several components, each of which may have its own requirements of session data.
Hmm.
Thanks glyph
Happy to help - thanks for using Twisted :). -glyph
On Jul 7, 2016, at 5:14 AM, Carl Waldbieser <cwaldbieser@gmail.com> wrote:
As for persisting session data-- I guess the idea is that storing or retrieving the session object doesn't need to be async because it is just an object in memory corresponding to a cookie in the request. However, writing to or reading from the session could definitely require async methods that talk to a back end store. This could potentially be one area where implementing a component with an interface like `IRedisSessionStorage` might be useful (get session synchonously from request passing in IRedisStorage, then call `storeThingInSession()` which returns a deferred).
This is an extremely subtle point so I am very happy that it seems to have somehow gotten across :). Thanks for writing up this answer; it's close to exactly what I would have written.
participants (3)
-
Carl Waldbieser -
Donal McMullan -
Glyph Lefkowitz