Hi,
i using the Perspective Broker and i have problems with the speed of importing the modules. It takes 15 - 17 seconds for importing the Reactor and the PBServerFactory (or other pb imports). If i import the reactor or the pb imports alone it takes in avg. 2-3 seconds.
def ImportIt(): from twisted.internet import reactor from twisted.spread.pb import PBServerFactory
import timeit t = timeit.Timer(setup='from __main__ import ImportIt', stmt='ImportIt()') print t.timeit()
How can i improve this?
I running win7 32bit SP1 on a 2,93 Ghz Core 2 Duo with python 2.7.1 and twisted 11.0
cheers
Marcel
On Wed, 2011-11-02 at 10:34 +0100, Marcel Gädigk wrote:
Hi,
i using the Perspective Broker and i have problems with the speed of importing the modules. It takes 15 - 17 seconds for importing the Reactor and the PBServerFactory (or other pb imports). If i import the reactor or the pb imports alone it takes in avg. 2-3 seconds.
The example script you gave takes me 3.3 seconds to run, on a similar machine on Linux. So switching operating systems may be one solution :) If not:
0. Is Twisted on a network filesystem? That may be the problem.
1. Make sure you have .pyc in your installed Twisted directories. If you don't, you import all of Twisted with a user that has permissions to create them in those folders.
2. Maybe try a packaging Twisted in an .egg file.
On 12:36 pm, itamar@itamarst.org wrote:
- Maybe try a packaging Twisted in an .egg file.
This isn't a great idea, since Twisted doesn't support being installed as a zipfile (which is what I guess you meant, since a .egg directory wouldn't be any different from the standard installation in terms of import performance).
Jean-Paul
On Wed, 2011-11-02 at 13:01 +0000, exarkun@twistedmatrix.com wrote:
This isn't a great idea, since Twisted doesn't support being installed as a zipfile (which is what I guess you meant, since a .egg directory wouldn't be any different from the standard installation in terms of import performance).
Oh right, plugins don't work or something?
I deleted all *.pyc files from the twisted Folder. After the *.pyc generation python imports pb and reactor in around 5 seconds.
Thx for your help.
On Nov 2, 2011, at 9:10 AM, Itamar Turner-Trauring wrote:
On Wed, 2011-11-02 at 13:01 +0000, exarkun@twistedmatrix.com wrote:
This isn't a great idea, since Twisted doesn't support being installed as a zipfile (which is what I guess you meant, since a .egg directory wouldn't be any different from the standard installation in terms of import performance).
Oh right, plugins don't work or something?
Actually, you can pre-generate a dropin.cache for your zip file, and the plugin system will work fine out of a zip file, since about 5 years ago: http://twistedmatrix.com/trac/changeset/19305.
Basically, portions of Twisted can work just fine out of a zip file, and it might be adequate for many applications. But (A) we don't have a buildbot anywhere that tests things in a zipfile configuration, and (B) there are many, many modules within Twisted, especially within tests, which expect that they can look at __file__ and assume it's a path on the filesystem. So big chunks of it definitely won't work.
I'd love to fix this, but it is just _super_ low priority for me right now; I kinda hope that someone else will show up with another embedded system running Twisted and do some work in this area, since the last time I genuinely cared about zip files was when I was putting Twisted on a tiny flash card to install in buses :).
-glyph
On Wed, Nov 2, 2011 at 5:51 PM, Glyph Lefkowitz glyph@twistedmatrix.comwrote:
I'd love to fix this
-glyph
Is there a ticket? Maybe there should be...
Kevin Horn
On Wed, 02 Nov 2011 10:34:27 +0100 Marcel Gädigk spam@herox.net wrote:
i using the Perspective Broker and i have problems with the speed of importing the modules. It takes 15 - 17 seconds for importing the Reactor and the PBServerFactory (or other pb imports). If i import the reactor or the pb imports alone it takes in avg. 2-3 seconds.
def ImportIt(): from twisted.internet import reactor from twisted.spread.pb import PBServerFactory
import timeit t = timeit.Timer(setup='from __main__ import ImportIt', stmt='ImportIt()') print t.timeit()
Er, have read the timeit docs? The timeit() method will execute the given code *one million times* by default:
timeit(self, number=1000000) unbound timeit.Timer method Time 'number' executions of the main statement.
To be precise, this executes the setup statement once, and then returns the time it takes to execute the main statement a number of times, as a float measured in seconds. The argument is the number of times through the loop, defaulting to one million. The main statement, the setup statement and the timer function to be used are passed to the constructor.
You might want to rewrite your script without timeit and simply use the "time" command. Or call `t.timeit(1)` instead.
Antoine.
participants (6)
-
Antoine Pitrou -
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
Itamar Turner-Trauring -
Kevin Horn -
Marcel Gädigk