[python-ldap] syncrepl fix for pyasn1 >= 0.3

Sun Nov 19 20:39:59 EST 2017

Hi Ilya and Michael,

I was able to do testing using the latest versions of your code, and it seems that there is still some issue.

* I used the pyasn1 from Git commit 1729162c76963233a702cddf4dadb43c3208f4f7 (`pyasn1.__version__` returns 0.4.1).
* I used the latest python-ldap that I could get from CVS (`ldap.__version__` returns 2.5.1).  I have CVS version 1.2 of the file Lib/ldap/syncrepl.py.

The first test was to do a full syncrepl refresh of a directory.  This is what would happen if you are setting up a client from nothing.  The test completed successfully!

The second test was to do another syncrepl refresh, but with my existing data as a starting point.  That means the LDAP Server is having a cookie sent as part of the syncrepl search.

The first reply back from the server triggers this trace:

Traceback (most recent call last):
  File "/Users/akkornel/Library/Python/2.7/bin/syncrepl-client", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/Users/akkornel/git/syncrepl/syncrepl-client", line 178, in <module>
    loop_result = client.poll()
  File "/Users/akkornel/git/syncrepl/syncrepl_client/__init__.py", line 702, in poll
    all=1, timeout=3)
  File "/Users/akkornel/Library/Python/2.7/lib/python/site-packages/python_ldap-2.5.2-py2.7-macosx-10.12-x86_64.egg/ldap/syncrepl.py", line 448, in syncrepl_poll
    sim = SyncInfoMessage(resp)
  File "/Users/akkornel/Library/Python/2.7/lib/python/site-packages/python_ldap-2.5.2-py2.7-macosx-10.12-x86_64.egg/ldap/syncrepl.py", line 320, in __init__
    if comp is not None and comp.hasValue():
AttributeError: 'SyncIdSet' object has no attribute 'hasValue'

Looking at the LDAP trace, I see a Syncrepl Sync Info Message (from https://tools.ietf.org/html/rfc4533#section-2.5) is coming in.  It looks like the LDAP server decided to send choice [3], the syncIdSet.  But, that is an ASN.1 SEQUENCE, which I guess does not have the hasValue method.

My third test was to delete my local data store, and to start a persistent Syncrepl search.  Having no data store forced a complete refresh.  Unfortunately, the refresh failed at the very end.  In fact, I got a traceback in the same place as before, but with a different object.

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/akkornel/git/syncrepl/syncrepl_client/__init__.py", line 801, in run
    poll_result = self.poll()
  File "/Users/akkornel/git/syncrepl/syncrepl_client/__init__.py", line 702, in poll
    all=1, timeout=3)
  File "/Users/akkornel/git/python-ldap/Lib/ldap/syncrepl.py", line 448, in syncrepl_poll
    sim = SyncInfoMessage(resp)
  File "/Users/akkornel/git/python-ldap/Lib/ldap/syncrepl.py", line 320, in __init__
    if comp is not None and comp.hasValue():
AttributeError: 'RefreshDelete' object has no attribute 'hasValue'

So, this time it was a Syncrepl Sync Info Message containing a refreshDelete sequence.

Just out of curiosity, I tried going back to pyasn1 version 0.2.3 (which I know worked with older versions of pyldap), and I got this exception:

Traceback (most recent call last):
  File "/Users/akkornel/Library/Python/2.7/bin/syncrepl-client", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/Users/akkornel/git/syncrepl/syncrepl-client", line 178, in <module>
    loop_result = client.poll()
  File "/Users/akkornel/git/syncrepl/syncrepl_client/__init__.py", line 702, in poll
    all=1, timeout=3)
  File "/Users/akkornel/git/python-ldap/Lib/ldap/syncrepl.py", line 407, in syncrepl_poll
    all=0,
  File "/Users/akkornel/git/python-ldap/Lib/ldap/ldapobject.py", line 521, in result4
    decoded_resp_ctrls = DecodeControlTuples(resp_ctrls,resp_ctrl_classes)
  File "/Users/akkornel/git/python-ldap/Lib/ldap/controls/__init__.py", line 150, in DecodeControlTuples
    control.decodeControlValue(encodedControlValue)
  File "/Users/akkornel/git/python-ldap/Lib/ldap/syncrepl.py", line 193, in decodeControlValue
    if cookie.hasValue():
AttributeError: 'NoneType' object has no attribute 'hasValue' 

python-ldap 2.4.45 and and pyasn1 0.2.3 work.

If you would like me to try anything else, please let me know!

    Message: 1
    Date: Sun, 19 Nov 2017 16:57:27 +0100
    From: Ilya Etingof <ilya at glas.net>
    To: Michael Str?der <michael at stroeder.com>
    Cc: Christian Heimes <christian at python.org>, python-ldap at python.org
    Subject: Re: [python-ldap] syncrepl fix for pyasn1 >= 0.3
    Message-ID: <A63629A0-9EF5-486C-AA29-ABC413472A67 at glas.net>
    Content-Type: text/plain; charset=utf-8

    Hi Michael,

    I think pyasn1 0.4.1 is mostly ready to be released. I have been trying to run relevant Python-LDAP tests with pyasn1 master [1] to make sure there are no regressions. Tests pass but I am not convinced they actually run through the pyasn1 code at all.

    So I wonder if you have a proper test bench handy to give it a quick try?  Or teach me how to run them locally.

    Thanks!  ;-)

    1. https://github.com/etingof/pyasn1

    > On 13 Nov 2017, at 12:42, Michael Str?der <michael at stroeder.com> wrote:
    > 
    > Ilya Etingof wrote:
    >> Here is my first take on the `default` feature:
    >> 
    >>     https://github.com/etingof/pyasn1/pull/100
    > 
    > Looks good to me.
    > 
    >> I appreciate your suggestions and feedback!
    > 
    > And I'm really glad you're working on it so quickly.
    > 
    > Do you have plans to release 0.4.1 soon?
    > 
    > Ciao, Michael.
    > 

    ------------------------------

    Subject: Digest Footer

    _______________________________________________
    python-ldap mailing list
    python-ldap at python.org
    https://mail.python.org/mailman/listinfo/python-ldap

    ------------------------------

    End of python-ldap Digest, Vol 79, Issue 14
    *******************************************