[Twisted-Python] Some beginner questions about "twisted.names.client" and ".tac" environment
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Twisted 2.1, twisted.named 0.2, here. I'm taking my first steps with Twisted (documentation -inexistence- nightmare :-), and my first project will be a bulk mailer as the backend of my mailing list system. The application would take the message and the subscriber list and a) resolve the MX for the domains and b) connect to the MX's and send the message, trying to minimice traffic sending a single envelope for several recipients sharing the domain or the MX's. I'm doing currently the DNS stuff. The result are promising, resolving about 200 domains per second in a 1.4GHz P4, so my biggest mailing list (about 31500 unique domains, múltiple subscribers per domain) is "resolved" in less than three minutes. Nice so far. The demo code (2Kbytes) is the following (if I'm violating the rules posting this code, please tell me): ===== # File "dns.tac" from twisted.application import service application = service.Application("DNS test") import time t=time.time() class resolucion(object) : def __init__(self,dominio) : from twisted.names import client d = client.lookupMailExchange(dominio,timeout=(60,)) d.addCallbacks(self._cbMailExchange, self._ebMailExchange) self.dominio=dominio def _cbMailExchange(self,results): # Callback for MX query global aun_pendientes aun_pendientes-=1 if not aun_pendientes : print "OK",time.time()-t return from twisted.internet import reactor reactor.stop() return if not len(pendientes) : return resolucion(pendientes.pop()) from twisted.names.dns import QUERY_TYPES for i in results[0] : n=i.payload.name tipo=QUERY_TYPES[i.payload.TYPE] if tipo=="MX" : return p=i.payload.preference print n,p, for j in results[2] : if n==j.name : print j.payload.dottedQuad(),"(%d)" %j.ttl break else : print "???" elif tipo=="CNAME" : redirigidos.append((self.dominio,i.payload.name)) def _ebMailExchange(self,failure): # Error callback for MX query global aun_pendientes aun_pendientes-=1 if not aun_pendientes : print "ERROR",time.time()-t return from twisted.internet import reactor reactor.stop() return if not len(pendientes) : return resolucion(pendientes.pop()) print "XXX",self.dominio print 'Lookup failed:' failure.printTraceback() pendientes=[] redirigidos=[] f=open("domain_list") for i in f : pendientes.append(i) aun_pendientes=len(pendientes) concurrencia=1000 for i in pendientes[:concurrencia] : resolucion(i) from twisted.names import client client.theResolver.resolvers[-1].dynServers=[('127.0.0.1', 53)] # client.theResolver.resolvers=[client.theResolver.resolvers[-1]] pendientes=pendientes[concurrencia:] ===== I launch the code as "twistd -ny dns.tac". The demo does 1000 resolutions in parallel. If you experiment with the code, reduce the value. Questions: 1. I get a warning: "[Uninitialized] /usr/local/lib/python2.4/site-packages/twisted/names/dns.py:1227: exceptions.DeprecationWarning: Deferred.setTimeout is deprecated. Look for timeout support specific to the API you are using instead." I'm using, the native "twisted.names" timeout API, as far as I know... 2. By default "twisted.names.client" uses the "/etc/resolv.conf" file to know which nameservers to use. I, nevertheless, want to use a particular nameserver, so: 2.1. I couldn't to find an appropiate API. I had to do a "hack", reading the "twisted.names" core to know implementation details: "client.theResolver.resolvers[-1].dynServers=[('127.0.0.1', 53)]" 2.2. The previous "hack" is only effective for future "twisted.names.client" instances. The previous ones use the "/etc/resolv.conf" entries. Putting the "hack" code before any instance creation doesn't work. 2.3. While reading the framework code, I saw that "client" uses a resolver chain: host, cache, network. But the cache is initially clear (of course) and NEVER ever gets populated, so we are not using it but checking missing entries eats CPU: 155 seconds for the unchanged code, 125 seconds if I drop the host and cache resolvers. A caching client would be very nice, if the client is long running (my original idea). 2.4. The resolution failure code is only called if the resolution timeouts. But if the domain doesn't exists, the code called is the "success" one, with a "nil" answer. So we can't diferenciate between inexistant domains and inexistant RRs. 3. How can I stop this ".tac"?. If I do "reactor.stop()", I get an infinite error, repeated forever: ===== [twisted.names.dns.DNSDatagramProtocol (UDP)] Traceback (most recent call last): File "/usr/local/lib/python2.4/site-packages/twisted/python/log.py", line 43, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/local/lib/python2.4/site-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/lib/python2.4/site-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/local/lib/python2.4/site-packages/twisted/internet/selectreactor.py", line 139, in _doReadOrWrite why = getattr(selectable, method)() --- <exception caught here> --- File "/usr/local/lib/python2.4/site-packages/twisted/internet/udp.py", line 113, in doRead data, addr = self.socket.recvfrom(self.maxPacketSize) exceptions.AttributeError: 'Port' object has no attribute 'socket' ===== I must kill -9 the "twistd" process. Thank you for your time and attention. Help greatly appreciated :-) - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/_/_/_/ PGP Key Available at KeyServ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBQ6SNsplgi5GaxT1NAQLMQwP/czYFLQ6+olTCvM0jdmMlaBgwxHsHdvxT /2mhWqtyhIf1Kdh6FioFQq13xqCfZxFIkwuUwTlG+ZmkSYK1iWZEmaS0CGa5YmuA d7miIFfL9Tfa3OLyV1nvqdCR3YtzH/ws9UuJ2DGnACRI++Of6gBVwGlhFDa7S57o wZcsYWAS6Sk= =/Pks -----END PGP SIGNATURE-----
On Sat, 17 Dec 2005 23:14:10 +0100, Jesus Cea <jcea@argo.es> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Twisted 2.1, twisted.named 0.2, here.
I'm taking my first steps with Twisted (documentation -inexistence- nightmare :-), and my first project will be a bulk mailer as the backend of my mailing list system.
The application would take the message and the subscriber list and a) resolve the MX for the domains and b) connect to the MX's and send the message, trying to minimice traffic sending a single envelope for several recipients sharing the domain or the MX's.
I'm doing currently the DNS stuff. The result are promising, resolving about 200 domains per second in a 1.4GHz P4, so my biggest mailing list (about 31500 unique domains, múltiple subscribers per domain) is "resolved" in less than three minutes.
Nice so far. The demo code (2Kbytes) is the following (if I'm violating the rules posting this code, please tell me):
===== # File "dns.tac"
from twisted.application import service
application = service.Application("DNS test")
You probably want to move most of your program out if "dns.tac" and into an importable Python module. Code defined inside .tac files lives in a weird world where some surprising rules apply. It's best to keep the .tac file as short as possible. Generally, you just want to create an Application and give it some children, importing from modules the definitions of all classes and functions needed to set this up.
import time t=time.time()
class resolucion(object) : def __init__(self,dominio) : from twisted.names import client d = client.lookupMailExchange(dominio,timeout=(60,))
Passing (60,) as the timeout might not be the best idea. This will cause the DNS client to send one request and then wait 60 seconds for a response. If either the request or the response is dropped (as often happens with UDP traffic), you will never get a result, and you will have to wait 60 seconds to discover this fact. If you don't want retransmission, a value of (15,) or so is probably better. However, I suspect you really do want retransmissions. The default timeout is also 60 seconds total, but performs several retransmissions during the interim.
d.addCallbacks(self._cbMailExchange, self._ebMailExchange) self.dominio=dominio
def _cbMailExchange(self,results): # Callback for MX query global aun_pendientes aun_pendientes-=1 if not aun_pendientes : print "OK",time.time()-t return from twisted.internet import reactor reactor.stop() return if not len(pendientes) : return
resolucion(pendientes.pop()) from twisted.names.dns import QUERY_TYPES for i in results[0] : n=i.payload.name tipo=QUERY_TYPES[i.payload.TYPE] if tipo=="MX" :
You can just use dns.MX here, instead of looking up "MX" in QUERY_TYPES.
return p=i.payload.preference print n,p, for j in results[2] : if n==j.name : print j.payload.dottedQuad(),"(%d)" %j.ttl break else : print "???" elif tipo=="CNAME" : redirigidos.append((self.dominio,i.payload.name))
def _ebMailExchange(self,failure): # Error callback for MX query global aun_pendientes aun_pendientes-=1 if not aun_pendientes : print "ERROR",time.time()-t return from twisted.internet import reactor reactor.stop() return if not len(pendientes) : return
resolucion(pendientes.pop()) print "XXX",self.dominio print 'Lookup failed:' failure.printTraceback()
pendientes=[] redirigidos=[]
f=open("domain_list") for i in f : pendientes.append(i)
aun_pendientes=len(pendientes)
concurrencia=1000
for i in pendientes[:concurrencia] : resolucion(i)
from twisted.names import client client.theResolver.resolvers[-1].dynServers=[('127.0.0.1', 53)] # client.theResolver.resolvers=[client.theResolver.resolvers[-1]]
To customize the server used by the resolver, you may want to create your own resolver instance, rather than relying on the defaults guessed by the resolver automatically created in the client module.
pendientes=pendientes[concurrencia:]
=====
I launch the code as "twistd -ny dns.tac".
The demo does 1000 resolutions in parallel. If you experiment with the code, reduce the value.
Questions:
1. I get a warning: "[Uninitialized] /usr/local/lib/python2.4/site-packages/twisted/names/dns.py:1227: exceptions.DeprecationWarning: Deferred.setTimeout is deprecated. Look for timeout support specific to the API you are using instead."
I'm using, the native "twisted.names" timeout API, as far as I know...
This is a problem internal to twisted.names. Your code isn't doing anything wrong to cause it. Hopefully this will be fixed by the next release.
2. By default "twisted.names.client" uses the "/etc/resolv.conf" file to know which nameservers to use. I, nevertheless, want to use a particular nameserver, so:
2.1. I couldn't to find an appropiate API. I had to do a "hack", reading the "twisted.names" core to know implementation details: "client.theResolver.resolvers[-1].dynServers=[('127.0.0.1', 53)]"
2.2. The previous "hack" is only effective for future "twisted.names.client" instances. The previous ones use the "/etc/resolv.conf" entries. Putting the "hack" code before any instance creation doesn't work.
2.3. While reading the framework code, I saw that "client" uses a resolver chain: host, cache, network. But the cache is initially clear (of course) and NEVER ever gets populated, so we are not using it but checking missing entries eats CPU: 155 seconds for the unchanged code, 125 seconds if I drop the host and cache resolvers.
A caching client would be very nice, if the client is long running (my original idea).
All three of these can be addressed by constructing your own resolver: from twisted.names import client myResolver = client.Resolver(servers=[('127.0.0.1', 53)]) This gives you a resolver which uses only localhost, doesn't involve any nasty hacks, and doesn't have an /etc/hosts resolver or a caching resolver to slow things down.
2.4. The resolution failure code is only called if the resolution timeouts. But if the domain doesn't exists, the code called is the "success" one, with a "nil" answer. So we can't diferenciate between inexistant domains and inexistant RRs.
Hmm. The non-existence of the domain is hidden by the very last step in performing the lookup. The Resolver class has a method, filterAnswers, which is used to turn a DNS response into the three-tuple of lists which all the lookup* methods return. You may want to subclass Resolver and override filterAnswers to behave differently when the `message' argument it is given has an `rCode' attribute equal to twisted.names.dns.ENAME, which indicates the name requested does not exist.
3. How can I stop this ".tac"?. If I do "reactor.stop()", I get an infinite error, repeated forever:
reactor.stop() is the correct way to end the program. If you still have this problem after you have split the program into multiple files, please post again. Jean-Paul
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jean-Paul Calderone wrote:
You probably want to move most of your program out if "dns.tac" and into an importable Python module. Code defined inside .tac files lives in a weird world where some surprising rules apply.
Any documentation about that?
It's best to keep the .tac file as short as possible. Generally, you just want to create an Application and give it some children, importing from modules the definitions of all classes and functions needed to set this up.
I reduced the "dns3.tac" to: ===== from twisted.application import service application = service.Application("DNS test") import sys sys.path=["."]+sys.path import dns3 ===== All logic is in "dns3.tac". Basically the same code posted in my previous message. I have the very same problem with "reactor.stop()": repeated exceptions, only stoppable using "kill -9". I don't get the "give it some children" point. Could you post some sample code?.
Passing (60,) as the timeout might not be the best idea. This will cause the DNS client to send one request and then wait 60 seconds for a response. If either the request or the response is dropped (as often happens with UDP traffic), you will never get a result, and you will have to wait 60 seconds to discover this fact.
I'm resolving several thousand of domains, so one minute more or less is not an issue. I don't want to retransmit since I'm using 127.0.0.1, so losing request (in the same machine) should be rare (udp backlog overflow, basically). Some domains takes a long time to resolve. So if I use an small value I load the server and get "Unexpected message (XXXXX) received from ('127.0.0.1', 53)", caused because the DNS server gets a late answers and my code already give up.
If you don't want retransmission, a value of (15,) or so is probably better.
Then I get a lot of "unexpected DNS responses". Seems a fair number of domains takes more than 15 seconds to fulfill the request :-p. In any case this number is not really important. I'm more concerned about the Twisted specific issues :p
You can just use dns.MX here, instead of looking up "MX" in QUERY_TYPES.
Good point. Changes done.
To customize the server used by the resolver, you may want to create your own resolver instance, rather than relying on the defaults guessed by the resolver automatically created in the client module.
Also a good point. Done and working fine. How can I easily use the cache resolver?. My problem is updating the cache when I get a response thru the network. In a long running daemon, caching DNS when I'm serving several hundreds of email for day woul dbe a big win. Maybe with my own overloaded cache class, but seems an obvious addition to standard twisted.names. :-? Maybe next release :-)
Hmm. The non-existence of the domain is hidden by the very last step in performing the lookup. The Resolver class has a method, filterAnswers, which is used to turn a DNS response into the three-tuple of lists which all the lookup* methods return. You may want to subclass Resolver and override filterAnswers to behave differently when the `message' argument it is given has an `rCode' attribute equal to twisted.names.dns.ENAME, which indicates the name requested does not exist.
That seems doable but an ugly hack :-). Perhaps a future "twisted.named" release could include a flag to easily differenciate between missing RR and nonexistent domain. Any hope?.
3. How can I stop this ".tac"?. If I do "reactor.stop()", I get an infinite error, repeated forever:
reactor.stop() is the correct way to end the program. If you still have this problem after you have split the program into multiple files, please post again.
Program splitted. Same problem :-/ The dode is basically the same that in my previous email. Moved 99% of code to "dns3.py". The dns3.tac" is trivial: ===== from twisted.application import service application = service.Application("DNS test") import sys sys.path=["."]+sys.path import dns3 ===== Help appreciated. Thanks for you time and efford. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/_/_/_/ PGP Key Available at KeyServ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBQ6SnzJlgi5GaxT1NAQLFgwP/U01pixJpJ39iwA8jZHOgfBJCOoOzI+h2 uhWh3bWHcecXddhutTCCwJb9rw1VFLBwTCyzoSXx2Qn6c1qLgKkSIt/My9RDD0js BPhJ2AV+8zQyc7ySAdYM3pTy4HNjw5JviDsymHs3KWw3AMDR/JhN46bXBgqARZtM IIe1xBs/HZc= =rXOt -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jesus Cea wrote:
All logic is in "dns3.tac". Basically the same code posted in my previous message.
Sorry. Logic in "dns3.py", of course. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/_/_/_/ PGP Key Available at KeyServ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBQ6So8Jlgi5GaxT1NAQIAVAP/fKxYtCRfIYA2n6Y9cyV3WPOsKfnaZqYu Nsq+0e0YwokLVFWB/33nAcnRZ2muW52Mmf4FHp/sFunzQ5gmwcF7+bSQVrN6+AYt g7do010nvONSOzJYLX9DJqwDD3r7DObbhvkAa4+0XLBEb7D394ErnhM50eIwwYNZ pQuFEZ5j264= =1N+i -----END PGP SIGNATURE-----
On Sun, 18 Dec 2005 01:05:32 +0100, Jesus Cea <jcea@argo.es> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jean-Paul Calderone wrote:
You probably want to move most of your program out if "dns.tac" and into an importable Python module. Code defined inside .tac files lives in a weird world where some surprising rules apply.
Any documentation about that?
Hmmm, not that I know of. The problems are all essentially consequences of the fact that .tac files are run using the built-in execfile() function (or equivalent), rather than being loaded as modules.
It's best to keep the .tac file as short as possible. Generally, you just want to create an Application and give it some children, importing from modules the definitions of all classes and functions needed to set this up.
I reduced the "dns3.tac" to:
=====
from twisted.application import service
application = service.Application("DNS test")
import sys sys.path=["."]+sys.path import dns3
=====
All logic is in "dns3.tac". Basically the same code posted in my previous message.
I have the very same problem with "reactor.stop()": repeated exceptions, only stoppable using "kill -9".
I don't get the "give it some children" point. Could you post some sample code?.
You want to tie your application logic to a Service subclass. To start with, I'd try something like this, in dns3.py: from twisted.application import service class DomainResolver(service.Service): def startService(self): # Copied the top-level code from the original dns.tac pendientes = [] redirigidos = [] f = open("domain_list") for i in f : pendientes.append(i) aun_pendientes = len(pendientes) concurrencia = 1000 for i in pendientes[:concurrencia]: resolucion(i) pendientes=pendientes[concurrencia:] Then, in dns3.tac after application is defined, from dns import DomainResolver DomainResolver().setServiceParent(application) This will delay the execution of your startup code until twistd is totally ready and the reactor is fully initialized.
Passing (60,) as the timeout might not be the best idea. This will cause the DNS client to send one request and then wait 60 seconds for a response. If either the request or the response is dropped (as often happens with UDP traffic), you will never get a result, and you will have to wait 60 seconds to discover this fact.
I'm resolving several thousand of domains, so one minute more or less is not an issue. I don't want to retransmit since I'm using 127.0.0.1, so losing request (in the same machine) should be rare (udp backlog overflow, basically).
Some domains takes a long time to resolve. So if I use an small value I load the server and get "Unexpected message (XXXXX) received from ('127.0.0.1', 53)", caused because the DNS server gets a late answers and my code already give up.
Okay, it sounds like you know what you're doing here :)
[snip]
To customize the server used by the resolver, you may want to create your own resolver instance, rather than relying on the defaults guessed by the resolver automatically created in the client module.
Also a good point. Done and working fine.
How can I easily use the cache resolver?. My problem is updating the cache when I get a response thru the network. In a long running daemon, caching DNS when I'm serving several hundreds of email for day woul dbe a big win.
Maybe with my own overloaded cache class, but seems an obvious addition to standard twisted.names. :-? Maybe next release :-)
I think the current caching resolver was a step in the wrong direction. A cache should probably *wrap* another resolver, not whatever weird thing it is doing now. If you write such a thing, it'd be great if you could submit it for inclusion :)
Hmm. The non-existence of the domain is hidden by the very last step in performing the lookup. The Resolver class has a method, filterAnswers, which is used to turn a DNS response into the three-tuple of lists which all the lookup* methods return. You may want to subclass Resolver and override filterAnswers to behave differently when the `message' argument it is given has an `rCode' attribute equal to twisted.names.dns.ENAME, which indicates the name requested does not exist.
That seems doable but an ugly hack :-). Perhaps a future "twisted.named" release could include a flag to easily differenciate between missing RR and nonexistent domain. Any hope?.
I think flags like this are ugly hacks as well. I completely agree that subclass/override is not a great way to get this functionality, but I'd like to think of a cleaner way to offer the new feature while still maintaining backwards compatibility.
3. How can I stop this ".tac"?. If I do "reactor.stop()", I get an infinite error, repeated forever:
reactor.stop() is the correct way to end the program. If you still have this problem after you have split the program into multiple files, please post again.
Program splitted. Same problem :-/
The dode is basically the same that in my previous email. Moved 99% of code to "dns3.py". The dns3.tac" is trivial:
Try the service class I used above. If you still see the exception, it may indicate a bug in Twisted's UDP support. If this case, could you attach the new version of the whole program? I'll take a closer look and see if I can nail down the exact cause. Jean-Paul
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jean-Paul Calderone wrote:
You want to tie your application logic to a Service subclass. To start with, I'd try something like this, in dns3.py:
The very same error :-(. My code for "dns3.tac": ===== from twisted.application import service application = service.Application("DNS test") import sys sys.path=["."]+sys.path from dns3 import DomainResolver DomainResolver().setServiceParent(application) ===== My code for "dns3.py": ===== class resolucion(object) : def __init__(self,dominio) : global myResolver d = myResolver.lookupMailExchange(dominio,timeout=(1,)) d.addCallbacks(self._cbMailExchange, self._ebMailExchange) self.dominio=dominio def _cbMailExchange(self,results): # Callback for MX query global aun_pendientes aun_pendientes-=1 if not aun_pendientes : import time print "OK",time.time()-t #return from twisted.internet import reactor reactor.stop() return if not len(pendientes) : return resolucion(pendientes.pop()) from twisted.names import dns for i in results[0] : n=i.payload.name tipo=i.payload.TYPE if tipo==dns.MX : return p=i.payload.preference print n,p, for j in results[2] : if n==j.name : print j.payload.dottedQuad(),"(%d)" %j.ttl break else : print "???" elif tipo==dns.CNAME : redirigidos.append((self.dominio,i.payload.name)) def _ebMailExchange(self,failure): # Error callback for MX query global aun_pendientes aun_pendientes-=1 if not aun_pendientes : import time print "ERROR",time.time()-t #return from twisted.internet import reactor reactor.stop() return if not len(pendientes) : return resolucion(pendientes.pop()) print "XXX",self.dominio print 'Lookup failed:' failure.printTraceback() from twisted.application import service class DomainResolver(service.Service): def startService(self) : global pendientes,redirigidos,aun_pendientes,t,myResolver from twisted.names import client myResolver=client.Resolver(servers=[('127.0.0.1', 53)]) import time t=time.time() pendientes=[] redirigidos=[] f=open("z1") for i in f : pendientes.append(i.split()[0]) f.close() aun_pendientes=len(pendientes) concurrencia=1 for i in pendientes[:concurrencia] : resolucion(i) pendientes=pendientes[concurrencia:] =====
This will delay the execution of your startup code until twistd is totally ready and the reactor is fully initialized.
Done, but not working :-( [... DNS timeouts ...]
Okay, it sounds like you know what you're doing here :)
I like to think so };-)
I think the current caching resolver was a step in the wrong direction. A cache should probably *wrap* another resolver, not whatever weird thing it is doing now.
I agree. Perhaps a subclass of the network DNS resolver could be fine. So we could have a "client" class and a "cachingClient" one. In any case, current code simply doesn't cache anything, but eats CPU cycles.
If you write such a thing, it'd be great if you could submit it for inclusion :)
I'm not familiar enough with Twisted internal, yet, to try. Nevertheless, I would suggest an optional parameter in the caching resolver constructor, pointing to a callable object to resolve misses. Sort of. Code changes would be minimal... [... about differenciatiing DNS misses from RR misses...]
I think flags like this are ugly hacks as well. I completely agree that subclass/override is not a great way to get this functionality, but I'd like to think of a cleaner way to offer the new feature while still maintaining backwards compatibility.
Keep me informed :-p
Try the service class I used above. If you still see the exception, it may indicate a bug in Twisted's UDP support. If this case, could you attach the new version of the whole program? I'll take a closer look and see if I can nail down the exact cause.
Not working. Same exception. Code posted. Thanks a lot for your help, Jean Paul. Greatly appreciated. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/_/_/_/ PGP Key Available at KeyServ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBQ6V9Nplgi5GaxT1NAQKAOwP9H251Jjvd/kNPJf7R+S9faHN9YFcSBluL YgBleuECDUG5WvJV2O+ZJwkHr05+qQ4qGzD13ydWnxdNCG1UMgWmIJ+TNfu2hOKN L+rQmNKIUKuPFTOHA+tACYqJaE9rkkcJumR7FON81kNJdEShs52aNUlnc1Q6A9IB iFyfaa9jcVI= =V82G -----END PGP SIGNATURE-----
participants (2)
-
Jean-Paul Calderone -
Jesus Cea