i have started investigating MM3 for my new, remote server and branched from lp:mailman, lp:postorius, and lp:hyperkitty.
I started in the MM3 branch directory and followed instructions in src/mailman/docs/START.rst.
I got the virtualenv ready to go and, in the local mailman branch directory, I executed:
$ node2 -v
All tests chug along nicely until:
/usr/local/src/0-mailman3/src/mailman/rest/docs/membership.rst ...
and it hangs longer than I think it should. After a <ctl-C> the last few trace-back lines are:
File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", line 374, in raw_execute self._run_execution(raw_cursor, args, params, statement) File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", line 388, in _run_execution self._check_disconnect(raw_cursor.execute, *args) File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", line 454, in _check_disconnect return function(*args, **kwargs) sqlite3.OperationalError: database is locked
Note I am running the MM3 installation, via ssh, on a remote host running Debian 7, 32-bit. (Note also postfix is running.)
I have no idea what to do next, and help or ideas would be greatly appreciated.
Best regards,
-Tom
Tom Browder writes:
We really appreciate your efforts to test the betas of Mailman 3. But please do be aware that although there are sites already successfully using Mailman 3 in production, the development team doesn't recommend use of any of the components (core, Postorius, HyperKitty) in production yet.
I have no idea what to do next,
Nothing. :-) I've already Cc'd (and set Reply-To to) mailman-developers, which is a more appropriate place for this report. (Many Mailman-Users are not interested in MM3 yet, while Mailman-Developers are by definition, as MM2 is basically end-of-life. Also, some relevant developers may read mailman-developers more frequently than they read mailman-users.)
Actually, I do have a couple of ideas. First, you should always report the whole error trace (if you think that's ugly in an email, attach it as a file). In this particular case, I suspect that the problem is in the test before the one that caused the Exception, which failed leaving the database locked. It would be very helpful to identify that test, which would probably be the *first* frame in the trace.
Second, you should look in the server's logs to see if there were any errors that might have caused the incomplete transaction.
and help or ideas would be greatly appreciated.
If you have no idea, then reporting to the developers is the best you can do. Use "Mailman Developers" mailman-developers@python.org or report via Launchpad.
Pretty clearly what's happened is that some previous test locked the database (probably anything that accesses the database does so at least long enough to read the whole record), and either (1) that test failed to unlock the database, (2) the test framework failed to unlock the database, or (3) the tests were improperly sequenced in some way and the database didn't get unlocked. It's quite possible that this failure could never be replicated in actual use, as tests often mock up some component that would normally ensure that any pending transaction gets discarded and the database unlocked.
Unfortunately I don't have an up-to-date test installation (it's on my list for early next week), and looking at the test file doesn't tell me anything. Perhaps Barry has an idea for a fix, or a workaround. And there's probably a way to skip that test, but I don't know nose very well.
Steve
Original report follows:
i have started investigating MM3 for my new, remote server and branched from lp:mailman, lp:postorius, and lp:hyperkitty.
I started in the MM3 branch directory and followed instructions in src/mailman/docs/START.rst.
I got the virtualenv ready to go and, in the local mailman branch directory, I executed:
$ node2 -v
All tests chug along nicely until:
/usr/local/src/0-mailman3/src/mailman/rest/docs/membership.rst ...
and it hangs longer than I think it should. After a <ctl-C> the last few trace-back lines are:
File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", line 374, in raw_execute self._run_execution(raw_cursor, args, params, statement) File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", line 388, in _run_execution self._check_disconnect(raw_cursor.execute, *args) File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", line 454, in _check_disconnect return function(*args, **kwargs) sqlite3.OperationalError: database is locked
Note I am running the MM3 installation, via ssh, on a remote host running Debian 7, 32-bit. (Note also postfix is running.)
On Wed, Feb 26, 2014 at 12:45 AM, Stephen J. Turnbull stephen@xemacs.org wrote:
Tom Browder writes: ... Actually, I do have a couple of ideas. First, you should always report the whole error trace (if you think that's ugly in an email, ...
Thanks, Steve, for the good advice.
-Tom
On Feb 26, 2014, at 02:45 PM, Stephen J. Turnbull wrote:
I have no idea what to do next,
This is clearly a bug, although I think it's relatively recent, so it might be worth seeing if earlier revisions avoid the problem. Yes, I can reproduce it.
The interesting thing is that the test is in rest/docs/membership.rst so these are multiprocess related bugs. Typically when this happens (and it will only be with the default SQLite database, as observed by others), it's a bug in the test, not necessarily a bug in the core.
Tests which involve multiple processes, as the REST tests do (i.e. the foreground testing process and a background runner process) have to be careful to release the database lock when they expect background processes to access the database. Releasing the lock means calling .commit() or .abort() at the appropriate time. The thing to keep in mind is that with Storm, even doing an operation that results in a database query opens a transaction and thus acquires the lock.
In the context of membership.rst, what this probably means is that somewhere in the doctest there's a database query with a missing explicit .commit() or .abort() before the background REST runner process executes. Tracing through the doctest to find out exactly where it hangs usually helps isolate where the missing commit/abort should go.
Of course, it's possible that there's a missing commit/abort in the core, but I rather doubt it, since that's pretty well tied into the REST runner's HTTP transaction machinery, and other REST tests don't exhibit the hang.
Tom, if you're up for debugging it, that would be great. If not, no worries. The test suite hangs for me, so I'll find some time this weekend to take a look.
-Barry
Guys, please take care to delete the mailman-users address if you're using reply-to-all on this thread. Progress on MM3 is undoubtedly of interest to MM-users, but the gory details of broken tests probably is not....
Reply-to-munging-still-considered-harmful-ly y'rs
P.S. Maybe I should revive the "optimal default reply algorithm" proto-RFC?
participants (3)
-
Barry Warsaw -
Stephen J. Turnbull -
Tom Browder