[ python-Bugs-1581906 ] test_sqlite fails on OSX G5 arch if test_ctypes is run
SourceForge.net
noreply at sourceforge.net
Fri Jul 13 17:05:03 CEST 2007
Bugs item #1581906, was opened at 2006-10-21 13:02
Message generated for change (Comment added) made by montanaro
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1581906&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Skip Montanaro (montanaro)
Assigned to: Thomas Heller (theller)
Summary: test_sqlite fails on OSX G5 arch if test_ctypes is run
Initial Comment:
I noticed a test_sqlite test failure on my Mac G5 the other day while trying
to set up a buildbot for sqlalchemy. Everything runs fine on my G4
powerbook. Both machines run Mac OSX 10.4.8 with Apple's gcc 4.0.0,
build 5026.
I whittled the problem down to just having test_ctypes and test_sqlite
enabled, then further whittled it down to just having Lib/ctypes/test/
test_find.py available (all other ctypes tests eliminated). More detailed
problem descriptions are in these two postings to python-dev:
http://article.gmane.org/gmane.comp.python.devel/84478
http://article.gmane.org/gmane.comp.python.devel/84481
Skip
----------------------------------------------------------------------
>Comment By: Skip Montanaro (montanaro)
Date: 2007-07-13 10:05
Message:
Logged In: YES
user_id=44345
Originator: YES
Go for it then. You are, after all, the expert. <wink> I'm crossing
my fingers that this will finally solve the community buildbot
failures I *always* get on mac os x when trying to test out
sqlalchemy.
Skip
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2007-07-13 04:47
Message:
Logged In: YES
user_id=11105
Originator: NO
Skip, IMO loading GLUT does not add any value (I just added it because
GLUT is related to the OpenGL libraries, and because it worked [That's what
I thought at least]). Finding and loading the other libs adds value
because they must be loaded with RTLD_GLOBAL on some platforms, so I would
like to keep that part.
So, I suggest to remove the GLUT part from the tests altogether.
----------------------------------------------------------------------
Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2007-07-12 05:41
Message:
Logged In: YES
user_id=580910
Originator: NO
This certainly sounds like someone is fooling around with the
-force_flat_namespace link flag, which could result in picking up the wrong
version of symbols. I haven't checked yet if this is indeed the case here,
and if so who is doing this.
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-07-04 20:31
Message:
Logged In: YES
user_id=44345
Originator: YES
Certainly the patch is simpler. ;-) If loading GLUT doesn't add any value
on OSX, would it add any value on other platforms? Do other tests have to
find libraries successfully to pass? If so, maybe test_find should be
dispensed with altogether, however if test_find has value we should figure
out a library to load on OSX.
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2007-07-04 15:01
Message:
Logged In: YES
user_id=11105
Originator: NO
Skip, thanks for the analysis and test patch.
IMO loading the GLUT library does not add significant value to the ctypes
tests, so wouldn't it be best to skip the GLUT lib on OSX? See the
attached patch "test_find without GLUT on OSX" (note that I didn't actually
test the patch, and I don't have access to OSX G5 anyway).
Maybe the problem and the analysis should be reported to the PyOpenGL
project...
File Added: test_find-no-osx-glut.diff
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-06-30 08:17
Message:
Logged In: YES
user_id=44345
Originator: YES
Thomas,
Here's a possible modification to ctypes.test.test_find which appears
to be the only place that OpenGL (and thus CoreData on the Mac?) is
accessed. On Darwin I replace the GL find stuff with Tcl find stuff.
It's a bit ugly, but it might solve the problem.
Skip
File Added: test_find.diff
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-06-30 06:15
Message:
Logged In: YES
user_id=44345
Originator: YES
I submitted a bug report to Apple about the problem, though I don't
really expect a response. (The other bug I filed in 2004 has never
received any response as far as I can tell.) Maybe a more recent
version of SQLite will be delivered with Leopard...
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-06-30 05:42
Message:
Logged In: YES
user_id=44345
Originator: YES
I'll be damned... Based on your last response I ran Python
under ktrace then looked at the kdump output. As it's
marching along it sees a mention of /usr/lib/libsqlite3.so
in /System/Library/Frameworks/CoreData.framework/Versions/A/CoreData
then just a few lines later I see
20023 python NAMI "/usr/lib/libsqlite3.0.dylib"
20023 python RET open 3
20023 python CALL fstat(0x3,0xbfffc770)
20023 python RET fstat 0
20023 python CALL pread(0x3,0xbfffcbd0,0x1000,0)
then a bit later I see what I had wanted to see:
20023 python NAMI
"/Users/skip/local/lib/python2.6/lib-dynload/_sqlite3.so"
20023 python RET open 6
20023 python CALL fstat(0x6,0xbfff9c00)
20023 python RET fstat 0
20023 python CALL pread(0x6,0xbfffa060,0x1000,0)
...
20023 python NAMI "/Users/skip/local/lib/libsqlite3.0.dylib"
20023 python RET open 6
20023 python CALL fstat(0x6,0xbfff99b0)
20023 python RET fstat 0
20023 python CALL pread(0x6,0xbfff9e10,0x1000,0)
I can't execute the CDLL call if I've renamed
/usr/lib/libsqlite.3.0.dylib. (I suppose that stands to
reason.)
Now the question becomes whether or not the Apple-provided
library can be replaced with something more modern.
Presumably every application that uses CoreData (most
Apple things) gets it if during startup.
Thomas, for the purposes of the Python regression test
suite do you think you can find something other than
GLUT to dynamically load?
Mark, thanks for the sleuthing.
Skip
----------------------------------------------------------------------
Comment By: Mark Dickinson (marketdickinson)
Date: 2007-06-29 22:56
Message:
Logged In: YES
user_id=703403
Originator: NO
It looks as though GLUT is a red herring. I can reproduce the problem
with a variety of frameworks in place of GLUT, for example Quartz, Cocoa,
AppKit, CoreData, as in the following lines.
import ctypes
ctypes.CDLL('/System/Library/Frameworks/CoreData.framework/CoreData')
import sqlite3
sqlite3.connect(":memory:")
Furthermore, I notice that the GLUT framework has a library dependency on
the AppKit framework, AppKit on CoreData, and CoreData depends on the
Apple-supplied libsqlite3, and that Quartz and Cocoa similarly ultimately
depend on libsqlite3. Coincidence?
I can reproduce the problem with a variety of frameworks in place of GLUT,
for example Quartz, Cocoa, AppKit, CoreData. (The .glutWireTetrahedron is
unnecessary.)
----------------------------------------------------------------------
Comment By: Mark Dickinson (marketdickinson)
Date: 2007-06-29 22:32
Message:
Logged In: YES
user_id=703403
Originator: NO
I think your minor(?) data point is a major data point. I'm using the
Macports version of sqlite3. Furthermore, after hiding the /usr/lib
version with
fibonacci:/usr/lib dickinsm$ sudo mv libsqlite3.0.dylib
libsqlite3.0.dylib.temp
I can't reproduce the problem any more. I tried sprinkling a few printf
statements through the sqlite3 source to see what was going on. Bizarrely,
it looks as though the Apple-supplied sqlite3SafetyCheck() routine is being
called from the Macports createCollation(); or at least that's the only way
that I can understand the results I'm getting. Is this possible?
Mark
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-06-29 21:45
Message:
Logged In: YES
user_id=44345
Originator: YES
One other minor(?) data point. I am using a version of sqlite3 which I
compiled and installed, not Apple's older version:
>>> sqlite3.version, sqlite3.sqlite_version
('2.3.3', '3.3.8')
...
% pwd
/Users/skip/local/lib/python2.6/lib-dynload
% otool -L _sqlite3.so
_sqlite3.so:
/Users/skip/local/lib/libsqlite3.0.dylib (compatibility version
9.0.0, current version 9.6.0)
/usr/lib/libmx.A.dylib (compatibility version 1.0.0, current
version 93.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 88.1.7)
It appears those of us who can reproduce the problem all have slightly
different installs (2.5 v. 2.6, Apple's GCC v. DarwinPorts, Apple's sqlite3
v. DarwinPorts v. homebuilt, G4 v. G5 v. Intel architecture).
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-06-29 21:39
Message:
Logged In: YES
user_id=44345
Originator: YES
Fails for me as well (Python 2.6a0, PowerBook G4, 10.4.10, Apple's GCC
4.0.1). Thomas, can you reproduce with the lines marketdickinson used?
I don't see any directory changing going on:
% python
Python 2.6a0 (trunk:54264M, Mar 10 2007, 15:19:48)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getcwd()
'/Users/skip'
>>> from ctypes import CDLL>>>
CDLL('/System/Library/Frameworks/GLUT.framework/GLUT').glutWireTetrahedron<_FuncPtr
object at 0x106c6c0>
>>> os.getcwd()
'/Users/skip'
>>> import sqlite3
>>> sqlite3.connect(":memory:")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ProgrammingError: library routine called out of sequence
>>> os.getcwd()
'/Users/skip'
Pass it back if you still can't reproduce the problem. Meanwhile I've
subscribed to the pysqlite mailing list and will ask there.
Skip
----------------------------------------------------------------------
Comment By: Mark Dickinson (marketdickinson)
Date: 2007-06-29 18:37
Message:
Logged In: YES
user_id=703403
Originator: NO
And also on Python 2.5, compiled under Apple's GCC. (Still OS X
10.4.10/G4)
The following four lines of Python code are enough to cause the problem
for me.
Interestingly, if the second and third lines are exchanged (so that the
two import commands come first), then no exception is raised.
Python 2.5.1 (r251:54863, May 10 2007, 20:59:25)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import CDLL
>>>
CDLL('/System/Library/Frameworks/GLUT.framework/GLUT').glutWireTetrahedron
<_FuncPtr object at 0x514500>
>>> import sqlite3
>>> sqlite3.connect(":memory:")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: library routine called out of sequence
----------------------------------------------------------------------
Comment By: Mark Dickinson (marketdickinson)
Date: 2007-06-29 14:11
Message:
Logged In: YES
user_id=703403
Originator: NO
I've just noticed the same failure on an iBook G4, with OS X 10.4.10,
sqlite version 3.4.0,
and Python built with gcc 4.2.0 (from macports).
(Python 2.6a0 (trunk:56123M, Jun 29 2007, 13:22:18)
[GCC 4.2.0] on darwin)
----------------------------------------------------------------------
Comment By: ted pollari (incongruity)
Date: 2007-06-13 15:19
Message:
Logged In: YES
user_id=1375728
Originator: NO
FWIW, I'm getting an identical error on a MacBook Pro with a Core Duo
processor
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2007-05-04 09:04
Message:
Logged In: YES
user_id=11105
Originator: NO
FYI, this PyOpenGL bug may be related:
https://sourceforge.net/tracker/?func=detail&atid=105988&aid=1712741&group_id=5988
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2007-03-22 15:18
Message:
Logged In: YES
user_id=11105
Originator: NO
Skip,
I do not have a G5 machine available, only a G4 and the problem doesn't
appear there.
For someone with a G5 it should not be too difficult to reproduce (or
investigate),
I think that running './python Lib/test/regrtest.py test_ctypes
test_sqlite' should suffice.
However, if loading the GLUT library changes the cwd, and sqlite fails
with a changed working dir,
I'm not sure where the problem really is and how it should be fixed. It
could be said that test_ctypes
should restore the cwd after the tests (but maybe this should be done by
the test framework?),
or it could be said that test_sqlite should set the cwd in case the test
relies on it.
I'll assign back to you because I have no way to test it on a machine
where the problem appears.
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2007-03-18 12:20
Message:
Logged In: YES
user_id=44345
Originator: YES
Thomas, I assigned this to you simply so you can cast your eye on the
problem and see if you think there might be anything ctypes-related in the
problem I've reported. I've ignored it up 'til now, but I bumped up
against it again trying to get the Pybots SQLAlchemy test suite running on
my Mac. During my previous encounter with this problem Ronald Oussouren
reported on a possible connection with GLUT:
http://mail.python.org/pipermail/python-dev/2006-October/069523.html
Assuming there is no obvious ctypes connection you can determine (it's
not clear to me that there is any current directory funny business going
on) just assign it back to me and I'll try to find a pysqlite expert to
take a look.
Thx,
Skip
----------------------------------------------------------------------
Comment By: Skip Montanaro (montanaro)
Date: 2006-10-21 13:04
Message:
Logged In: YES
user_id=44345
Another article here:
http://article.gmane.org/gmane.comp.python.devel/84487
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1581906&group_id=5470
More information about the Python-bugs-list
mailing list