[Twisted-Python] WSGI Question
I deploy code. Then I modify the code on my file system. The code does not automatically change on the server. It does this on the django dev server. How can I make this happen? I start the server as follows twistd -ny server.py # Django and static file server: root_resource = get_root_resource() root_resource.putChild("static", static.File("static")) http_factory = server.Site(root_resource, logPath="http.log") internet.TCPServer(STATIC_PORT, http_factory, interface=INTERFACE).setServiceParent(serviceCollection)
On Jan 20, 2010, at 9:58 AM, adamjamesdrew same wrote:
I deploy code. Then I modify the code on my file system. The code does not automatically change on the server. It does this on the django dev server. How can I make this happen?
This is a Django-specific feature that doesn't exist in Twisted. It wouldn't be too hard to write, but it's the kind of feature that tends to be useful more to web developers than authors of other kinds of applications. As it is, automatic code reloading is tricky to achieve in Python, since reload(modulename) only reloads modulename in the current scope, and must be reloaded everywhere that module is used. It looks to me like Django's dev server gets around this by forking a child process before loading any external modules, and killing that child any time a reload is needed. Then the child is re-spawned, and since the parent has yet to load any external modules, any attempt to do so in the child causes the modules to be loaded for the first time (for that child). Pretty clever, IMO... -phil
On Jan 20, 2010, at 10:37 AM, Phil Christensen wrote:
There is/was a Twisted ticket about this and a quick sketch of how it might be done for Twisted by using the Django module itself by one of the Twisted devs. I gave it a brief shot, but couldn't get it working in the time I had to spend on it. Sure would be handy, though... S
On Wed, Jan 20, 2010 at 9:58 AM, adamjamesdrew same <theiklabs@gmail.com> wrote:
The twisted modules are only loaded once because it asynchronous, so the twisted server is persistent in memory. You only load your twisted app once, whereas in django the modules are reloaded on every page refresh on the dev server. There are some nifty live reloading things you can do. One of them is to just reload the module (i think its a builtin method called reload) but only new objects will use the reloaded module. There is a module in twisted.python somewhere that does some live reloading magic to try and replace all the references with the newly loaded module. It's actually fairly complicated and if your in a development environment it is probably easier to reload twisted than add all the reloading stuff to your classes. I haven't used the twisted reloading stuff though. In my case I tore down some of the objects that were created with the classes in the module, reloaded the associated modules, then rebuilt those objects. And if you're doing that to your whole project, it would be about the same as just restarting the server. Someone from the twisted project can probably give you better information about reloading changes.
On Jan 20, 2010, at 10:40 AM, Landreville wrote:
No, this is not the case. Twisted's function in this regard is because there's no autoreload functionality, not because of anything to do with its asynchronous development style. The Django dev server *does not* reload on every page request, it only does so when the files in question have been modified. The Django dev server is also persistent in memory, and uses forking to isolate reloadable code into its own process. When the code fails, or needs to be reloaded, the child is killed and respawned. -phil
On Wed, Jan 20, 2010 at 6:58 AM, adamjamesdrew same <theiklabs@gmail.com> wrote:
I really want to getting reloading going. I already have a ticket here: http://github.com/clemesha/hotdot/issues#issue/2 that describes this issue, with a couple of notes. Please add to this ticket / post any patches if you getting anywhere with this. thanks! -Alex [1] http://github.com/clemesha/hotdot
-- Alex Clemesha clemesha.org
On Jan 20, 2010, at 3:23 PM, Alex Clemesha wrote:
I don't think it gets much better than Django's approach. Basically, the server code loads everything it needs to bootstrap, and then forks. All the request handling and subsequent code execution takes place in the child process, so that when modules are loaded, they are only loaded into the child's memory space. Then, before any request is processed, the child checks the originating files for all of the modules in sys.modules, and sees if any of them have a new modification date. If so, the child process is killed, which is detected by the parent, which respawns the child. The respawned child has the state of the parent process (which hasn't loaded any external modules), so modules get freshly imported. t.p.rebuild() is a big improvement over the stock reload() function, but in addition to requiring modules to be marked with the "ALLOW_TWISTED_REBUILD" flag, it suffers from the same issue that it's only able to reload modules that are in the current scope. -phil
On Jan 20, 2010, at 9:58 AM, adamjamesdrew same wrote:
I deploy code. Then I modify the code on my file system. The code does not automatically change on the server. It does this on the django dev server. How can I make this happen?
This is a Django-specific feature that doesn't exist in Twisted. It wouldn't be too hard to write, but it's the kind of feature that tends to be useful more to web developers than authors of other kinds of applications. As it is, automatic code reloading is tricky to achieve in Python, since reload(modulename) only reloads modulename in the current scope, and must be reloaded everywhere that module is used. It looks to me like Django's dev server gets around this by forking a child process before loading any external modules, and killing that child any time a reload is needed. Then the child is re-spawned, and since the parent has yet to load any external modules, any attempt to do so in the child causes the modules to be loaded for the first time (for that child). Pretty clever, IMO... -phil
On Jan 20, 2010, at 10:37 AM, Phil Christensen wrote:
There is/was a Twisted ticket about this and a quick sketch of how it might be done for Twisted by using the Django module itself by one of the Twisted devs. I gave it a brief shot, but couldn't get it working in the time I had to spend on it. Sure would be handy, though... S
On Wed, Jan 20, 2010 at 9:58 AM, adamjamesdrew same <theiklabs@gmail.com> wrote:
The twisted modules are only loaded once because it asynchronous, so the twisted server is persistent in memory. You only load your twisted app once, whereas in django the modules are reloaded on every page refresh on the dev server. There are some nifty live reloading things you can do. One of them is to just reload the module (i think its a builtin method called reload) but only new objects will use the reloaded module. There is a module in twisted.python somewhere that does some live reloading magic to try and replace all the references with the newly loaded module. It's actually fairly complicated and if your in a development environment it is probably easier to reload twisted than add all the reloading stuff to your classes. I haven't used the twisted reloading stuff though. In my case I tore down some of the objects that were created with the classes in the module, reloaded the associated modules, then rebuilt those objects. And if you're doing that to your whole project, it would be about the same as just restarting the server. Someone from the twisted project can probably give you better information about reloading changes.
On Jan 20, 2010, at 10:40 AM, Landreville wrote:
No, this is not the case. Twisted's function in this regard is because there's no autoreload functionality, not because of anything to do with its asynchronous development style. The Django dev server *does not* reload on every page request, it only does so when the files in question have been modified. The Django dev server is also persistent in memory, and uses forking to isolate reloadable code into its own process. When the code fails, or needs to be reloaded, the child is killed and respawned. -phil
On Wed, Jan 20, 2010 at 6:58 AM, adamjamesdrew same <theiklabs@gmail.com> wrote:
I really want to getting reloading going. I already have a ticket here: http://github.com/clemesha/hotdot/issues#issue/2 that describes this issue, with a couple of notes. Please add to this ticket / post any patches if you getting anywhere with this. thanks! -Alex [1] http://github.com/clemesha/hotdot
-- Alex Clemesha clemesha.org
On Jan 20, 2010, at 3:23 PM, Alex Clemesha wrote:
I don't think it gets much better than Django's approach. Basically, the server code loads everything it needs to bootstrap, and then forks. All the request handling and subsequent code execution takes place in the child process, so that when modules are loaded, they are only loaded into the child's memory space. Then, before any request is processed, the child checks the originating files for all of the modules in sys.modules, and sees if any of them have a new modification date. If so, the child process is killed, which is detected by the parent, which respawns the child. The respawned child has the state of the parent process (which hasn't loaded any external modules), so modules get freshly imported. t.p.rebuild() is a big improvement over the stock reload() function, but in addition to requiring modules to be marked with the "ALLOW_TWISTED_REBUILD" flag, it suffers from the same issue that it's only able to reload modules that are in the current scope. -phil
participants (5)
-
adamjamesdrew same -
Alex Clemesha -
Landreville -
Phil Christensen -
ssteinerX＠gmail.com