Collecting snmp data - threads, processes, twisted?

[Mike C. Fletcher]

Antal Rutz wrote:
...
> I'd like to collect snmp data from varoius network devices parallel.
>   
...
> Could you give me some advice how can I make my script really parallel?
>
> My options maybe:
> 1. pySNMP (as it's full in python)
> 2. subprocess (I'd like to make (find) a solution as threadpool.py)
>     (running yapsnmp, pySNMP or even net-snmp?...)
> 3. TwistedSNMP (seems too weird for me.. cannot imagine a simple 
> solution for my problem)
>   
This kind of parallel mass-query operation is TwistedSNMP's primary 
focus (it was created for a commercial product that scans thousands of 
agents).  If you really want all of the queries to run in parallel (i.e. 
send out all the queries within milliseconds of each other):

    agents = [ ... ]
    dl = []
    for agent in agents:
        dl.append( agent.get( (oid,oid,oid) ) )
    return defer.DeferredList( dl )

add a callback to the DeferredList which processes the 
(success,result_or_failure) list you get back.  If you want to process 
individual results/failures before the whole set returns, add the 
callback for that to the agent.get() result.

If you have thousands of devices you'll likely want to batch the queries 
into ~ 200 simultaneous queries and/or use multiple protocols (gives you 
more OS-level buffering).

Have fun,
Mike

________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com