[python-ldap] ldap.OPT_DESC, async ops and paged search controls

[Mark R Bannister]

On 27/01/2015 21:26, Michael Ströder wrote:
> Mark R Bannister wrote:
>> As you can see, I may have many LDAP objects in different states,
> There is no internal state of a LDAPObject instance. And I wonder why you open
> a separate LDAP connection for each worker thread. I don't think you need that
> since you can send several LDAP requests on one connection and dispatch the
> incoming results my msg_id to the queued request (see result type of
> LDAPObject.result2() and higher).
>
> 1. Just queue the incoming requests,
> 2. send the accompanying LDAP request on the single connection,
> 3. attach msg_id tp queued request and
> 4. have *one* result thread repeatedly calling LDAPObject.result2() and
> 5. finally dispatch the results by msg_id to the queued incoming request.
>
> With this approach you wouldn't even need so many worker threads and you're
> not overloading the server with so many connections.
>
> It's not trivial of course because you also have to take care of the result
> type to determine wheter all results for a single msg_id were received. But
> that's what you have to take care of anyway when calling
> LDAPObject.result(msgid=RES_ANY) like in your examples.
>
> Also decent caching is mandatory.
>
> YMMV though.
>
> Ciao, Michael.
>
So if I have just one thread reading results and dispatching results, I 
have a bottleneck don't I?  If that thread is busy reading a large 
result set for one request, and there is also a large amount of data 
waiting for another ten separate requests, there will be a delay in 
processing the others.  Whereas if I have ten separate workers reading 
their own data, I am spreading the load and improving the performance.  
It is a more scalable solution with performance that is directly 
proportional to the number of requests, not exponential.

That said, there may be some mileage in considering having a single LDAP 
object per worker, which behaves much as you describe, but the scope is 
limited to that worker's queue.

Best regards,
Mark.