[python-ldap] ldap.OPT_DESC, async ops and paged search controls

[Rich Megginson]

On 01/20/2015 04:16 PM, Mark R Bannister wrote:
> Hi,
>
> I've been using the new ldap.OPT_DESC feature introduced in 
> python-ldap 2.4.17 and have a question concerning the use of it with 
> asynchronous search operations and paged search controls.
>
> I have a daemon (http://dbis.sf.net) that launches async searches with 
> paged search controls (page size 1000), and then uses the 
> ldap.OPT_DESC feature to get the file descriptor back.  It then adds 
> the fd to a select() call, with the intention to be woken up when 
> there is something to do.  When it wakes up it calls ldap.result().
>
> So far this makes sense, right?
>
> Now, the behaviour I'm seeing is not necessarily what I would have 
> expected, so I want to check with others to see if this is correct or 
> not.  I have an example search operation that returns about 80,000 
> entries.  With a page size of 1000, would you not expect to go round 
> the loop and call select() 80 times?  Well that's not what I'm 
> getting.  It's looping round and calling select() 80,000 times, once 
> per entry.  Each time select() wakes me up, I call ldap.result() which 
> for the first 999 iterations returns (None, None).  Then, the 1,000th 
> time, it returns 1,000 entries.  I then switch to the next page and go 
> round the loop again, and the same thing happens again.
>
> Thinking that there was something wrong with my code, I ran strace to 
> confirm that when I was calling ldap.result() and got (None, None) 
> returned, there was some data transferred over the file descriptor.
>
> Does this seem right to you and is there anyway to optimise this? All 
> 80,000 entries are taking about 15 seconds to read into Python using 
> the python-ldap module compared with 5 seconds for native C.

What LDAP server are you using?  It may be something specific to your 
server.

>
> Thanks,
> Mark.
>
> _______________________________________________
> python-ldap mailing list
> python-ldap at python.org
> https://mail.python.org/mailman/listinfo/python-ldap