[pypy-dev] Sqlite tests broken for pypyp 3.6

Jean-Daniel jeandaniel.browne at gmail.com
Mon Apr 2 16:49:29 EDT 2018


Hello team,

Here is Arianna and I's understanding of the problem we see on the buildbot
[1].

1. The tests are failing because CPython's sqlite3 module changed between
3.5 and 3.6 and Pypy's sqlite3 module needs to be modified to reflect that.

2. CPython's s sqlite3 now "no longer implicitly commits an open
transaction" [2] and we ported that commit in Pypy

3. Now, the "CheckTypeMapUsage" test [3] started to break, but that is
because that test relies on CPython's deterministic garbage collector:

4. The test creates a cursor that does a SELECT on a table. The SELECT cursor
does not go through all the data, before a DROP is executed on the table.
The sqlite3 library keeps track of the unfinalized SELECT and usually
rejects such a DROP with a 'database locked' error message. This is to make
sure the SELECT does silently get an incomplete view.

5. In CPython, the test does pass, because the SELECT cursor is not
assigned to a variable, and is deleted right away by the GC, which
finalizes the cursor, which unlocks the database.

6. We checked that, when the test is modified so that the cursor gets
assigned to a variable, then CPython's test receives the 'database locked'
error when the DROP is attempted. This confirms the point above.

7. In Pypy, the test used to pass in 3.5 because of a workaround
implemented in Pypy's sqlite3 module [4] that relies on "open transactions
being automatically committed", behavior which got removed in [2].


Our understanding is that the workaround breaks a guard from Sqlite, also
it introduces a significant skew between CPython and Pypy.


Here are the steps we suggest:
a. fix the CheckTypeMapUsage test that relies on the GC behavior in CPython
b. update the tests in Pypy
c. remove the workaround in Pypy
d. port the commit in [2] from CPython to Pypy.

What do you think about these steps?

Thanks a lot for your warm welcome at the Pypy Sprint, very enjoyable :)


[1]:
http://buildbot.pypy.org/summary/longrepr?testname=unmodified&builder=pypy
-c-app-level-linux-x86-64&build=3945&mod=lib-python.3.test.test_sqlite

[2]: https://github.com/python/cpython/commit/ab994ed8b97e1b0dac151ec827c857
f5e7277565

[3]:
https://github.com/python/cpython/blob/55966f3a0d5f1bf823bd22ce1abbde267b06552f/Lib/sqlite3/test/regression.py#L120

[4]:
https://bitbucket.org/pypy/pypy/src/045483a22d662e8f5305ef5c8b144623a2ff8c57/lib_pypy/_sqlite3.py?at=default&fileviewer=file-view-default#_sqlite3.py-850
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20180402/1dfccc06/attachment-0001.html>


More information about the pypy-dev mailing list