On Fri, Aug 08, 2003 at 07:12:00PM -0400, Edmund Dengler wrote:

Digging a bit more into the DB side, and I wonder a bit about the API for adbapi. From what I can tell, the runQuery() will return a Deferred for the whole result set (which implies the result set is in memory). Is there a method or style for running a query in which each row will be passed back through a Deferred one at a time? (Basically, each cycle through reactor.run() would be returning a single row).

What I am looking for is how to deal with a large result set without requiring the whole set be in memory. Should I be looking elsewhere within Twisted to do this?

Here's what I did with Ldaptor (note how the callback and the returned Deferred play together): def search(self, filterText=None, filterObject=None, attributes=(), scope=pureldap.LDAP_SCOPE_wholeSubtree, derefAliases=pureldap.LDAP_DEREF_neverDerefAliases, sizeLimit=0, timeLimit=0, typesOnly=0, callback=None): """ Perform an LDAP search with this object as the base. @param filterText: LDAP search filter as a string. @param filterObject: LDAP search filter as LDAPFilter. Note if both filterText and filterObject are given, they are combined with AND. If neither is given, the search is made with a filter that matches everything. @param attributes: List of attributes to retrieve for the result objects. An empty list and means all. @param scope: Whether to recurse into subtrees. @param derefAliases: Whether to deref LDAP aliases. TODO write better documentation. @param sizeLimit: At most how many entries to return. 0 means unlimited. @param timeLimit: At most how long to use for processing the search request. 0 means unlimited. @param typesOnly: Whether to return attribute types only, or also values. @param callback: Callback function to call for each resulting LDAPEntry. None means gather the results into a list and give that to the Deferred returned from here. @return: A Deferred that will complete when the search is done. The Deferred gives None if callback was given and a list of the search results if callback is not given or is None. """ -- :(){ :|:&};:

On Fri, Aug 08, 2003 at 07:12:00PM -0400, Edmund Dengler wrote:

Digging a bit more into the DB side, and I wonder a bit about the API for adbapi. From what I can tell, the runQuery() will return a Deferred for the whole result set (which implies the result set is in memory). Is there a method or style for running a query in which each row will be passed back through a Deferred one at a time? (Basically, each cycle through reactor.run() would be returning a single row).

What I am looking for is how to deal with a large result set without requiring the whole set be in memory. Should I be looking elsewhere within Twisted to do this?

Here's what I did with Ldaptor (note how the callback and the returned Deferred play together): def search(self, filterText=None, filterObject=None, attributes=(), scope=pureldap.LDAP_SCOPE_wholeSubtree, derefAliases=pureldap.LDAP_DEREF_neverDerefAliases, sizeLimit=0, timeLimit=0, typesOnly=0, callback=None): """ Perform an LDAP search with this object as the base. @param filterText: LDAP search filter as a string. @param filterObject: LDAP search filter as LDAPFilter. Note if both filterText and filterObject are given, they are combined with AND. If neither is given, the search is made with a filter that matches everything. @param attributes: List of attributes to retrieve for the result objects. An empty list and means all. @param scope: Whether to recurse into subtrees. @param derefAliases: Whether to deref LDAP aliases. TODO write better documentation. @param sizeLimit: At most how many entries to return. 0 means unlimited. @param timeLimit: At most how long to use for processing the search request. 0 means unlimited. @param typesOnly: Whether to return attribute types only, or also values. @param callback: Callback function to call for each resulting LDAPEntry. None means gather the results into a list and give that to the Deferred returned from here. @return: A Deferred that will complete when the search is done. The Deferred gives None if callback was given and a list of the search results if callback is not given or is None. """ -- :(){ :|:&};: