[Python-3000] Nonlinearity in dbm.ndbm?

[Josiah Carlson]

On Sat, Sep 6, 2008 at 8:58 PM, Josiah Carlson <josiah.carlson at gmail.com> wrote:
> On Sat, Sep 6, 2008 at 8:37 PM,  <skip at pobox.com> wrote:
>>
>>    Josiah> The version I just posted to the tracker reads/writes about 30k
>>    Josiah> entries/second.  You may want to look at the differences (looks
>>    Josiah> to be due to your lack of a primary key/index).
>>
>>    me> Thanks.  The real speedup was to avoid using cursors.
>>
>> Let me take another stab at this.  My __setitem__ looks like this:
>>
>>    def __setitem__(self, key, val):
>>        c = self._conn.cursor()
>>        c.execute("replace into dict"
>>                  " (key, value) values (?, ?)", (key, val))
>>        self._conn.commit()
>>
>> This works (tests pass), but is slow (23-25 msec per loop).  If I change it
>> to this:
>>
>>    def __setitem__(self, key, val):
>>        self._conn.execute("replace into dict"
>>                           " (key, value) values (?, ?)", (key, val))
>>
>> which is essentially your __setitem__ without the type checks on the key and
>> value, it runs much faster (about 300 usec per loop), but the unit tests
>> fail.  This also works:
>>
>>    def __setitem__(self, key, val):
>>        self._conn.execute("replace into dict"
>>                           " (key, value) values (?, ?)", (key, val))
>>        self._conn.commit()
>>
>> I think you need the commits and have to suffer with the speed penalty.
>
> I guess I need to look at your unittests, because in my testing,
> reading/writing with a single instance works great, but if you want
> changes to be seen by other instances (in other threads or processes),
> you need to .commit() changes.  I'm thinking that that's a reasonable
> expectation; I never expected bsddbs to be able to share their data
> with other processes until I did a .sync(), but maybe I never expected
> much from my dbm-like interfaces?

I took sandbox/trunk/dbm_sqlite/Lib/test/test_dbm_sqlite.py, changed
some of the imports to be used with 2.6, got rid of the 'b' prefix on
bytes objects, and my implementation passes in 2.5 (I had to add
support for buffer, double-close, and the 'c' flag).

Maybe there's something funky going on with Python 3.0's sqlite3?

 - Josiah