[Twisted-Python] How to configure setup.py for a twistd TAP app
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
I want to deploy my Twisted app as a Twisted Application Plugin for twistd as per http://twistedmatrix.com/projects/core/documentation/howto/tap.html I want to distribute my app as a tarball, so the user would untar it and run "python setup.py install". Then they would run "twistd myapp" to start it. How do I configure setup.py so that twisted/plugins/myapp.py is created? Where do I create the twisted/plugins directory? (I see how this works if I run twistd from the directory containing twisted/plugins, or if I set PYTHONPATH, but I don't expect my users to have to do this). Cheers, Chris
![](https://secure.gravatar.com/avatar/3a7e70f3ef2ad1539da42afc85c8d09d.jpg?s=120&d=mm&r=g)
On Dec 19, 2007 8:15 AM, Chris Miles <miles.chris@gmail.com> wrote:
How do I configure setup.py so that twisted/plugins/myapp.py is created?
Good question; I still haven't figured it out. I know the Divmod guys use the plugin system extensively, maybe one of them can say something on the subject?
Where do I create the twisted/plugins directory?
You should probably expect it to be already there. It needs to go into the main twisted/plugins directory, i.e. the one that "import twisted.plugins; print twisted.plugins.__file__" gives you. I get the feeling that this should either be really easy or really hard to do with distutils. If you manage to figure it out, I'd like to add the information to the Plugin howto. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/
![](https://secure.gravatar.com/avatar/7c39d1649d343d42a882de4d3d34b86b.jpg?s=120&d=mm&r=g)
See the twisted.python.dist module to understand installation process for the twisted subprojects. For real example check any plugin of the Twisted and make setup.py file in your own project directory. Be careful, in the twisted subprojects setup modules is in the topfiles directory. It necessary for the twisted's release script, you don't need it if you plugin isn't under twisted maintainence. Chris Miles wrote:
-- """ Best regards, Alexander Burtsev, Web Development Department, TV Channel 'Sport' """
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Thu, 20 Dec 2007 00:15:59 +1100, Chris Miles <miles.chris@gmail.com> wrote:
Since your app depends on Twisted, the user should already have a directory named "twisted/plugins" in their install directory (probably their system- wide site-packages, but not necessary). Twisted itself creates and installs this directory. Your installer needs to do just two things: put myapp.py into that directory and regenerate the plugin cache. One way in which to accomplish the first thing is like this: Pass a list like ["twisted.plugins"] for the packages keyword argument to distutils setup() Pass a dict like {'twisted': ['plugins/myapp.py']} for the package_data keyword argument to distutils setup() Include a line like "graft twisted" in your MANIFEST.in http://twistedmatrix.com/projects/core/documentation/howto/plugin.html#auto3 gives an example of how to do the second thing. Jean-Paul
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
On 20/12/2007, at 1:21 AM, Jean-Paul Calderone wrote:
Thanks Jean-Paul, that worked great, with both distutils.core.setup and twisted.python.dist.setup. When I used the latter I didn't need to regenerate the plugin cache, twistd picked up the new command immediately after install. Is this the advantage of twisted.python.dist.setup ? The technique also (mostly) worked using setuptools.setup. twistd picks up the new command, but spits out a UserWarning due to the appearance of the duplicate twisted package within the egg directory. /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site- packages/zope.interface-3.3.0.1-py2.4-macosx-10.3-fat.egg/zope/ __init__.py:3: UserWarning: Module twisted was already imported from / Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site- packages/twisted/__init__.pyc, but /Library/Frameworks/ Python.framework/Versions/2.4/lib/python2.4/site-packages/ myproject-0.0.0-py2.4.egg is being added to sys.path import pkg_resources Usage: twistd [options] ... I'd like to use setuptools, if possible, due to my project's dependencies on other setuptools-managed packages, but I can live without it if it's not fully compatible with Twisted application plugins. Cheers, Chris Miles
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Thu, 20 Dec 2007 18:23:44 +1100, Chris Miles <miles.chris@gmail.com> wrote:
Not quite. twisted.python.dist is an internal module, not part of Twisted's public API. Notice this line in its docstring: API Stability: Unstable. Don't use it outside of Twisted.
I'm not really sure what the above warning signifies, whether it indicates a real problem, or what would be necessary to fix it if so. I know that setuptools does a number of things which aren't compatible with distutils and the assumptions of Twisted, though. Jean-Paul
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
On 21/12/2007, at 1:30 AM, Jean-Paul Calderone wrote:
Understood. I'm happy with the solution using distutils.core.setup and forcing a plugin cache refresh. btw, the "Plugin Caching" section on http://twistedmatrix.com/projects/core/documentation/howto/plugin.html#auto3 has an error. The sample code uses "getPlugin" but it should be getPlugins. i.e. I needed to use (with Twisted 2.5.0): {{{ from twisted.plugin import IPlugin, getPlugins list(getPlugins(IPlugin)) }}} Cheers, Chris Miles
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
On 21/12/2007, at 5:43 PM, Chris Miles wrote:
I have documented the method I used to package my Twisted app using distutils. I used the Twisted finger tutorial as an example. http://chrismiles.livejournal.com/23399.html Hope this helps others. Cheers, Chris Miles
![](https://secure.gravatar.com/avatar/3a7e70f3ef2ad1539da42afc85c8d09d.jpg?s=120&d=mm&r=g)
On Dec 19, 2007 8:15 AM, Chris Miles <miles.chris@gmail.com> wrote:
How do I configure setup.py so that twisted/plugins/myapp.py is created?
Good question; I still haven't figured it out. I know the Divmod guys use the plugin system extensively, maybe one of them can say something on the subject?
Where do I create the twisted/plugins directory?
You should probably expect it to be already there. It needs to go into the main twisted/plugins directory, i.e. the one that "import twisted.plugins; print twisted.plugins.__file__" gives you. I get the feeling that this should either be really easy or really hard to do with distutils. If you manage to figure it out, I'd like to add the information to the Plugin howto. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/
![](https://secure.gravatar.com/avatar/7c39d1649d343d42a882de4d3d34b86b.jpg?s=120&d=mm&r=g)
See the twisted.python.dist module to understand installation process for the twisted subprojects. For real example check any plugin of the Twisted and make setup.py file in your own project directory. Be careful, in the twisted subprojects setup modules is in the topfiles directory. It necessary for the twisted's release script, you don't need it if you plugin isn't under twisted maintainence. Chris Miles wrote:
-- """ Best regards, Alexander Burtsev, Web Development Department, TV Channel 'Sport' """
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Thu, 20 Dec 2007 00:15:59 +1100, Chris Miles <miles.chris@gmail.com> wrote:
Since your app depends on Twisted, the user should already have a directory named "twisted/plugins" in their install directory (probably their system- wide site-packages, but not necessary). Twisted itself creates and installs this directory. Your installer needs to do just two things: put myapp.py into that directory and regenerate the plugin cache. One way in which to accomplish the first thing is like this: Pass a list like ["twisted.plugins"] for the packages keyword argument to distutils setup() Pass a dict like {'twisted': ['plugins/myapp.py']} for the package_data keyword argument to distutils setup() Include a line like "graft twisted" in your MANIFEST.in http://twistedmatrix.com/projects/core/documentation/howto/plugin.html#auto3 gives an example of how to do the second thing. Jean-Paul
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
On 20/12/2007, at 1:21 AM, Jean-Paul Calderone wrote:
Thanks Jean-Paul, that worked great, with both distutils.core.setup and twisted.python.dist.setup. When I used the latter I didn't need to regenerate the plugin cache, twistd picked up the new command immediately after install. Is this the advantage of twisted.python.dist.setup ? The technique also (mostly) worked using setuptools.setup. twistd picks up the new command, but spits out a UserWarning due to the appearance of the duplicate twisted package within the egg directory. /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site- packages/zope.interface-3.3.0.1-py2.4-macosx-10.3-fat.egg/zope/ __init__.py:3: UserWarning: Module twisted was already imported from / Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site- packages/twisted/__init__.pyc, but /Library/Frameworks/ Python.framework/Versions/2.4/lib/python2.4/site-packages/ myproject-0.0.0-py2.4.egg is being added to sys.path import pkg_resources Usage: twistd [options] ... I'd like to use setuptools, if possible, due to my project's dependencies on other setuptools-managed packages, but I can live without it if it's not fully compatible with Twisted application plugins. Cheers, Chris Miles
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Thu, 20 Dec 2007 18:23:44 +1100, Chris Miles <miles.chris@gmail.com> wrote:
Not quite. twisted.python.dist is an internal module, not part of Twisted's public API. Notice this line in its docstring: API Stability: Unstable. Don't use it outside of Twisted.
I'm not really sure what the above warning signifies, whether it indicates a real problem, or what would be necessary to fix it if so. I know that setuptools does a number of things which aren't compatible with distutils and the assumptions of Twisted, though. Jean-Paul
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
On 21/12/2007, at 1:30 AM, Jean-Paul Calderone wrote:
Understood. I'm happy with the solution using distutils.core.setup and forcing a plugin cache refresh. btw, the "Plugin Caching" section on http://twistedmatrix.com/projects/core/documentation/howto/plugin.html#auto3 has an error. The sample code uses "getPlugin" but it should be getPlugins. i.e. I needed to use (with Twisted 2.5.0): {{{ from twisted.plugin import IPlugin, getPlugins list(getPlugins(IPlugin)) }}} Cheers, Chris Miles
![](https://secure.gravatar.com/avatar/3e9b0b47a98cdc664bcf58fa3ff5a758.jpg?s=120&d=mm&r=g)
On 21/12/2007, at 5:43 PM, Chris Miles wrote:
I have documented the method I used to package my Twisted app using distutils. I used the Twisted finger tutorial as an example. http://chrismiles.livejournal.com/23399.html Hope this helps others. Cheers, Chris Miles
participants (4)
-
Burus Gmail
-
Chris Miles
-
Christopher Armstrong
-
Jean-Paul Calderone