numarray-1.2.3 is a bugfix release for numarray-1.2.2 which fixes a problem with universal function setup caching which noticeably impaired 1.2.2 small array performance. Get it if you are new to numarray, haven't upgraded to 1.2.2 yet, or use a lot of small arrays. numarray-1.2.3 is here: http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367 Thanks to Ralf Juengling for quietly reporting this and working with me to identify and fix the problem.
Hi, what is the cvs command to update to the exact same 1.2.3 version using cvs? Also I'm wondering if numarray.__version__ could be more informative about e.g. "1.2" vs. "1.2.2" vs. "1.2.3" ? (What does the 'a' stand for in na.__version__ == '1.2a' ? Does that mean I got it from CVS ? ) Thanks, Sebastian Haase On Thursday 03 March 2005 09:07, Todd Miller wrote:
numarray-1.2.3 is a bugfix release for numarray-1.2.2 which fixes a problem with universal function setup caching which noticeably impaired 1.2.2 small array performance. Get it if you are new to numarray, haven't upgraded to 1.2.2 yet, or use a lot of small arrays.
numarray-1.2.3 is here:
http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367
Thanks to Ralf Juengling for quietly reporting this and working with me to identify and fix the problem.
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Thu, 2005-03-03 at 12:48, Sebastian Haase wrote:
Hi, what is the cvs command to update to the exact same 1.2.3 version using cvs?
% cvs update -r v1_2_3
Also I'm wondering if numarray.__version__ could be more informative about e.g. "1.2" vs. "1.2.2" vs. "1.2.3" ?
They're already OK I think, just like you're showing above. Do you want something else?
(What does the 'a' stand for in na.__version__ == '1.2a' ? Does that mean I got it from CVS ? )
The 'a' in 1.2a stands for "optimism". It actually took 1.2, 1.2.1, 1.2.2 to get to 1.2.3. My original plan was 1.2a, pass go, 1.2... it just didn't work out that way. Regards, Todd
Thanks, Sebastian Haase
On Thursday 03 March 2005 09:07, Todd Miller wrote:
numarray-1.2.3 is a bugfix release for numarray-1.2.2 which fixes a problem with universal function setup caching which noticeably impaired 1.2.2 small array performance. Get it if you are new to numarray, haven't upgraded to 1.2.2 yet, or use a lot of small arrays.
numarray-1.2.3 is here:
http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367
Thanks to Ralf Juengling for quietly reporting this and working with me to identify and fix the problem.
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion --
On Thursday 03 March 2005 10:32, Todd Miller wrote:
On Thu, 2005-03-03 at 12:48, Sebastian Haase wrote:
Hi, what is the cvs command to update to the exact same 1.2.3 version using cvs?
% cvs update -r v1_2_3
I just did this - but comparing with the 1.2.3 from sourceforge I have some files, e.g. Examples/ufunc/Src/airy.h only in the CVS version !? Thanks, Sebastian Haase
On Thu, 2005-03-03 at 14:13, Sebastian Haase wrote:
On Thursday 03 March 2005 10:32, Todd Miller wrote:
On Thu, 2005-03-03 at 12:48, Sebastian Haase wrote:
Hi, what is the cvs command to update to the exact same 1.2.3 version using cvs?
% cvs update -r v1_2_3
I just did this - but comparing with the 1.2.3 from sourceforge I have some files, e.g. Examples/ufunc/Src/airy.h only in the CVS version !?
airy.h exists now for me on both the CVS head and 1.2.3. airy.h did not always exist throughout the entire pre-release lifespan of version 1.2 so if you did a checkout (cvs checkout numarray or cvs update numarray) and saw 1.2, there's no guarantee what the state of airy.h would have been. CVS versions just tend to be stale. I tag CVS and change the numarray version only when I do a tarball or semi-formal tests involving other people. Also note that CVS can be used with dates rather than version numbers or tags, so there is some recourse even when numarray.__version__ isn't telling the whole story. Regards, Todd
Hi, After upgrading from numarray 1.1 (now 1.2.3) We get a Segmentation fault in our C++ program on Linux (python2.2,gcc2.95) , gdb says this: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1087498336 (LWP 8279)] 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 (gdb) where #0 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 #1 0x410f905e in deferred_libnumarray_init () at Src/libnumarraymodule.c:149 #2 0x410f98a8 in NA_NewAllFromBuffer (ndim=3, shape=0xbffff2e4, type=tFloat32, bufferObject=0x8a03988, byteoffset=0, <nl> bytestride=0, byteorder=0, aligned=1, writeable=1) at Src/ libnumarraymodule.c:636 #3 0x0805b159 in MyApp::OnInit (this=0x8108f50) at omx_app.cpp:519 #4 0x4026f616 in wxEntry () from /jws30/haase/PrLin0/wxGtkLibs/ libwx_gtk-2.4.so #5 0x0805a91a in main (argc=1, argv=0xbffff414) at omx_app.cpp:247 To initialize libnumarray I was using this: { // import_libnumarray(); { PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } } } Any idea ? Thanks, Sebastian Haase
From what you're showing me, it looks like libnumarray initialization is failing which makes me suspect a corrupted numarray installation. Here are some things to try:
1. Completely delete your existing site-packages/numarray. Also delete numarray/build then re-install numarray. 2. Delete and re-install your extensions. In principle, numarray-1.2.3 is supposed to be binary compatible with numarray-1.1.1 but maybe I'm mistaken. 3. Hopefully you won't get this far but... a python which works well with gdb can be built from source using ./configure --with-pydebug. So a debug scenario is something like: % tar zxf Python-2.2.3.tar.gz % cd Python-2.2.3 % ./configure --with-pydebug --prefix=$HOME % make % make install % cd .. % tar zxf numarray-1.2.3.tar.gz % cd numarray-1.2.3 % python setup.py install % cd .. % tar zxf your_stuff.tar.gz % cd your_stuff % python setup.py install This makes a debug Python installed in $HOME/bin, $HOME/lib, and $HOME/include. This process is useful for compiling Python itself and extensions with "-g -O0" and hence gdb works better. Besides appropriate compiler switches, debug Python also has more robust object memory management and better tracked reference counting. Debug like this: % setenv PATH $HOME/bin:$PATH # export if you use bash % rehash % gdb python (gdb) run
<do the test> <crash> (gdb) l <startline>,<endline> # to see some code (gdb) p <some_interesting_variable> (gdb) up # Move up the stack frame to see where the bogus value came from
Regards, Todd On Thu, 2005-03-03 at 14:40, Sebastian Haase wrote:
Hi, After upgrading from numarray 1.1 (now 1.2.3) We get a Segmentation fault in our C++ program on Linux (python2.2,gcc2.95) , gdb says this:
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1087498336 (LWP 8279)] 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 (gdb) where #0 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 #1 0x410f905e in deferred_libnumarray_init () at Src/libnumarraymodule.c:149 #2 0x410f98a8 in NA_NewAllFromBuffer (ndim=3, shape=0xbffff2e4, type=tFloat32, bufferObject=0x8a03988, byteoffset=0, <nl> bytestride=0, byteorder=0, aligned=1, writeable=1) at Src/ libnumarraymodule.c:636 #3 0x0805b159 in MyApp::OnInit (this=0x8108f50) at omx_app.cpp:519 #4 0x4026f616 in wxEntry () from /jws30/haase/PrLin0/wxGtkLibs/ libwx_gtk-2.4.so #5 0x0805a91a in main (argc=1, argv=0xbffff414) at omx_app.cpp:247
To initialize libnumarray I was using this: { // import_libnumarray(); { PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } } }
Any idea ?
Thanks, Sebastian Haase
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion --
On Friday 04 March 2005 07:03, Todd Miller wrote:
From what you're showing me, it looks like libnumarray initialization
is failing which makes me suspect a corrupted numarray installation.
It understood it saying it fails in MyApp::OnInit omx_app.cpp:519 while doing: NA_NewAllFromBuffer (ndim=3, shape=0xbffff2e4, type=tFloat32, bufferObject=0x8a03988, byteoffset=0, bytestride=0, byteorder=0, aligned=1, writeable=1) the "initialize libnumarray"-stuff is in the 20 lines above that. Do you use NA_NewAllFromBuffer anywhere ? Thanks, Sebastian Haase
Here are some things to try:
1. Completely delete your existing site-packages/numarray. Also delete numarray/build then re-install numarray.
2. Delete and re-install your extensions. In principle, numarray-1.2.3 is supposed to be binary compatible with numarray-1.1.1 but maybe I'm mistaken.
3. Hopefully you won't get this far but... a python which works well with gdb can be built from source using ./configure --with-pydebug. So a debug scenario is something like:
% tar zxf Python-2.2.3.tar.gz % cd Python-2.2.3 % ./configure --with-pydebug --prefix=$HOME % make % make install
% cd .. % tar zxf numarray-1.2.3.tar.gz % cd numarray-1.2.3 % python setup.py install
% cd .. % tar zxf your_stuff.tar.gz % cd your_stuff % python setup.py install
This makes a debug Python installed in $HOME/bin, $HOME/lib, and $HOME/include. This process is useful for compiling Python itself and extensions with "-g -O0" and hence gdb works better. Besides appropriate compiler switches, debug Python also has more robust object memory management and better tracked reference counting.
Debug like this:
% setenv PATH $HOME/bin:$PATH # export if you use bash % rehash
% gdb python (gdb) run
<do the test>
<crash> (gdb) l <startline>,<endline> # to see some code (gdb) p <some_interesting_variable> (gdb) up # Move up the stack frame to see where the bogus value came from
Regards, Todd
On Thu, 2005-03-03 at 14:40, Sebastian Haase wrote:
Hi, After upgrading from numarray 1.1 (now 1.2.3) We get a Segmentation fault in our C++ program on Linux (python2.2,gcc2.95) , gdb says this:
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1087498336 (LWP 8279)] 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 (gdb) where #0 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 #1 0x410f905e in deferred_libnumarray_init () at Src/libnumarraymodule.c:149 #2 0x410f98a8 in NA_NewAllFromBuffer (ndim=3, shape=0xbffff2e4, type=tFloat32, bufferObject=0x8a03988, byteoffset=0, <nl> bytestride=0, byteorder=0, aligned=1, writeable=1) at Src/ libnumarraymodule.c:636 #3 0x0805b159 in MyApp::OnInit (this=0x8108f50) at omx_app.cpp:519 #4 0x4026f616 in wxEntry () from /jws30/haase/PrLin0/wxGtkLibs/ libwx_gtk-2.4.so #5 0x0805a91a in main (argc=1, argv=0xbffff414) at omx_app.cpp:247
To initialize libnumarray I was using this: { // import_libnumarray(); { PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } } }
Any idea ?
Thanks, Sebastian Haase
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Hi, Now I'm back to this problem. In the meantime I upgraded again:
na.__version__ '1.4.0' All my python2.2 + numarray + my-extensions work. (Debian Woody) Only my C++ program segfaults. It looked to me like a problem with NA_NewAllFromBuffer -- was there any recent change ? (I saw that I was using a bytestride of 0 - but it used to work and giving a 1 for that argument still segfaults -- is 0 OK ??)
Now I noticed in the gdb traceback that I chokes is the function deferred_libnumarray_init (( ((#0 0x406d68d5 in PyObject_GetAttrString () from ((/usr/lib/libpython2.2.so.0.0 #1 0x410f905e in ((deferred_libnumarray_init () at Src/libnumarraymodule.c:149 Is this function the standard function that always get called to init numarray (in embedded Python AND using "normal" python) ? Have you tested this with Python2.2 ? (See for more info below == I did follow Todd's suggestions 1 & 2, but hope not having to recompile python from source.) Thanks Sebastian Haase On Friday 04 March 2005 11:46, Sebastian Haase wrote:
On Friday 04 March 2005 07:03, Todd Miller wrote: From what you're showing me, it looks like libnumarray initialization
is failing which makes me suspect a corrupted numarray installation.
It understood it saying it fails in MyApp::OnInit omx_app.cpp:519 while doing: NA_NewAllFromBuffer (ndim=3, shape=0xbffff2e4, type=tFloat32, bufferObject=0x8a03988, byteoffset=0, bytestride=0, byteorder=0, aligned=1, writeable=1)
the "initialize libnumarray"-stuff is in the 20 lines above that. Do you use NA_NewAllFromBuffer anywhere ?
Thanks, Sebastian Haase
Here are some things to try:
1. Completely delete your existing site-packages/numarray. Also delete numarray/build then re-install numarray.
2. Delete and re-install your extensions. In principle, numarray-1.2.3 is supposed to be binary compatible with numarray-1.1.1 but maybe I'm mistaken.
3. Hopefully you won't get this far but... a python which works well with gdb can be built from source using ./configure --with-pydebug. So a debug scenario is something like:
% tar zxf Python-2.2.3.tar.gz % cd Python-2.2.3 % ./configure --with-pydebug --prefix=$HOME % make % make install
% cd .. % tar zxf numarray-1.2.3.tar.gz % cd numarray-1.2.3 % python setup.py install
% cd .. % tar zxf your_stuff.tar.gz % cd your_stuff % python setup.py install
This makes a debug Python installed in $HOME/bin, $HOME/lib, and $HOME/include. This process is useful for compiling Python itself and extensions with "-g -O0" and hence gdb works better. Besides appropriate compiler switches, debug Python also has more robust object memory management and better tracked reference counting.
Debug like this:
% setenv PATH $HOME/bin:$PATH # export if you use bash % rehash
% gdb python (gdb) run
<do the test>
<crash> (gdb) l <startline>,<endline> # to see some code (gdb) p <some_interesting_variable> (gdb) up # Move up the stack frame to see where the bogus value came from
Regards, Todd
On Thu, 2005-03-03 at 14:40, Sebastian Haase wrote:
Hi, After upgrading from numarray 1.1 (now 1.2.3) We get a Segmentation fault in our C++ program on Linux (python2.2,gcc2.95) , gdb says this:
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1087498336 (LWP 8279)] 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 (gdb) where #0 0x406d68d5 in PyObject_GetAttrString () from /usr/lib/libpython2.2.so.0.0 #1 0x410f905e in deferred_libnumarray_init () at Src/libnumarraymodule.c:149 #2 0x410f98a8 in NA_NewAllFromBuffer (ndim=3, shape=0xbffff2e4, type=tFloat32, bufferObject=0x8a03988, byteoffset=0, <nl> bytestride=0, byteorder=0, aligned=1, writeable=1) at Src/ libnumarraymodule.c:636 #3 0x0805b159 in MyApp::OnInit (this=0x8108f50) at omx_app.cpp:519 #4 0x4026f616 in wxEntry () from /jws30/haase/PrLin0/wxGtkLibs/ libwx_gtk-2.4.so #5 0x0805a91a in main (argc=1, argv=0xbffff414) at omx_app.cpp:247
To initialize libnumarray I was using this: { // import_libnumarray(); { PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } } }
Any idea ?
Thanks, Sebastian Haase
On Mon, 2005-05-09 at 17:58 -0700, Sebastian Haase wrote:
Hi, Now I'm back to this problem. In the meantime I upgraded again:
na.__version__ '1.4.0' All my python2.2 + numarray + my-extensions work. (Debian Woody) Only my C++ program segfaults. It looked to me like a problem with NA_NewAllFromBuffer -- was there any recent change ?
Yes there was. You can undo it in your private CVS checkout like this: % cd numarray/Src % cvs update -j 1.3 -1.2 libnumarray.ch % cd .. % python setup.py install --genapi --selftest
(I saw that I was using a bytestride of 0 - but it used to work and giving a 1 for that argument still segfaults -- is 0 OK ??)
I don't know for sure. It seems like it should be... just leads to 0 strides.
Now I noticed in the gdb traceback that I chokes is the function deferred_libnumarray_init (( ((#0 0x406d68d5 in PyObject_GetAttrString () from ((/usr/lib/libpython2.2.so.0.0 #1 0x410f905e in ((deferred_libnumarray_init () at Src/libnumarraymodule.c:149
Is this function the standard function that always get called to init numarray (in embedded Python AND using "normal" python) ?
Yes and no. deferred_libnumarray_init() is really for stuff that can't be done at libnumarray.so init time. This is because of circular dependencies. The "real" numarray init function is either "import numarray" in Python or "import_array()" in C. After that, certain calls into the C-API trigger the deferred initialization. deferred_libnumarray_init() is never called directly by application code.
Have you tested this with Python2.2 ?
Yes, using the numarray unit tests. NewAllFromBuffer is also used directly by the numarray ufuncs, so it is tested to a degree by numarray itself. My gut feel is you're hitting an embedding problem, but my gut is only so accurate.
(See for more info below == I did follow Todd's suggestions 1 & 2, but hope not having to recompile python from source.)
That's really not hard. A 5-10 minute exercise usually. Regards, Todd
Todd, I would be very thankful if you could compile and run this program: <testnumarrayembedded.cpp> #include <Python.h> #include <numarray/libnumarray.h> int main() { Py_Initialize(); PyObject *pymodMain = PyImport_ImportModule("__main__"); PyObject* globals = PyModule_GetDict(pymodMain); PyEval_InitThreads(); // import_libnumarray(); PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } float *colMapsRGB = new float[4 * 3 * 256]; int shape[3]= { 4, 3, 256 }; PyObject *obj = PyBuffer_FromReadWriteMemory(colMapsRGB, 4*3 *256 * sizeof(float)); PyObject *obj2 =(PyObject*) NA_NewAllFromBuffer( 3, shape, tFloat32, obj, 0, 0, NA_ByteOrder(), 1, 1/*writable*/) ; return 0; } </testnumarrayembedded.cpp> I compile with: g++-2.95 -o testnumarrayembedded -Wall -finline-functions -g -I/usr/include/ python2.2 -I/jws30/haase/myCVS/numarray/Include testnumarrayembedded.cpp -pthread -lpython2.2 It runs through with setting PYTHONPATH to my old numarray(1.1) directory. And now with numarray CVS it now doesn't even find the library... $ ./testnumarrayembedded Fatal Python error: Can't import module 'numarray.libnumarray' Aborted Still, my non-embedded python shell runs all fine - your CVS-update (-j...) suggestion didn't seem to make a difference. Thanks, Sebastian On Tuesday 10 May 2005 05:13, Todd Miller wrote:
On Mon, 2005-05-09 at 17:58 -0700, Sebastian Haase wrote:
Hi,
Now I'm back to this problem. In the meantime I upgraded again:
na.__version__
'1.4.0' All my python2.2 + numarray + my-extensions work. (Debian Woody) Only my C++ program segfaults. It looked to me like a problem with NA_NewAllFromBuffer -- was there any recent change ?
Yes there was. You can undo it in your private CVS checkout like this:
% cd numarray/Src % cvs update -j 1.3 -1.2 libnumarray.ch % cd .. % python setup.py install --genapi --selftest
(I saw that I was using a bytestride of 0 - but it used to work and giving a 1 for that argument still segfaults -- is 0 OK ??)
I don't know for sure. It seems like it should be... just leads to 0 strides.
Now I noticed in the gdb traceback that I chokes is the function deferred_libnumarray_init (( ((#0 0x406d68d5 in PyObject_GetAttrString () from ((/usr/lib/libpython2.2.so.0.0 #1 0x410f905e in ((deferred_libnumarray_init () at Src/libnumarraymodule.c:149
Is this function the standard function that always get called to init numarray (in embedded Python AND using "normal" python) ?
Yes and no. deferred_libnumarray_init() is really for stuff that can't be done at libnumarray.so init time. This is because of circular dependencies.
The "real" numarray init function is either "import numarray" in Python or "import_array()" in C. After that, certain calls into the C-API trigger the deferred initialization. deferred_libnumarray_init() is never called directly by application code.
Have you tested this with Python2.2 ?
Yes, using the numarray unit tests. NewAllFromBuffer is also used directly by the numarray ufuncs, so it is tested to a degree by numarray itself. My gut feel is you're hitting an embedding problem, but my gut is only so accurate.
(See for more info below == I did follow Todd's suggestions 1 & 2, but hope not having to recompile python from source.)
That's really not hard. A 5-10 minute exercise usually.
Regards, Todd
------------------------------------------------------- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the first software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Hi Sebastian, I tried building and running this but I can't even get to main(). I even tried the attached, which I think is completely decoupled from numarray and there is the same problem. BTW, did you leave out -fPIC and -shared on the command line? To simplify things, you might try: a. removing:
PyObject *pymodMain = PyImport_ImportModule("__main__"); PyObject* globals = PyModule_GetDict(pymodMain); PyEval_InitThreads();
b. get rid of threads for now c. downshift from C++ to C Bottom line: I think this is a more fundamental "embedding" setup problem. Regards, Todd On Tue, 2005-05-10 at 13:31, Sebastian Haase wrote:
Todd, I would be very thankful if you could compile and run this program: <testnumarrayembedded.cpp> #include <Python.h> #include <numarray/libnumarray.h> int main() { Py_Initialize(); PyObject *pymodMain = PyImport_ImportModule("__main__"); PyObject* globals = PyModule_GetDict(pymodMain); PyEval_InitThreads();
// import_libnumarray(); PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } float *colMapsRGB = new float[4 * 3 * 256]; int shape[3]= { 4, 3, 256 }; PyObject *obj = PyBuffer_FromReadWriteMemory(colMapsRGB, 4*3 *256 * sizeof(float)); PyObject *obj2 =(PyObject*) NA_NewAllFromBuffer( 3, shape, tFloat32, obj, 0, 0, NA_ByteOrder(), 1, 1/*writable*/) ; return 0; } </testnumarrayembedded.cpp>
I compile with: g++-2.95 -o testnumarrayembedded -Wall -finline-functions -g -I/usr/include/ python2.2 -I/jws30/haase/myCVS/numarray/Include testnumarrayembedded.cpp -pthread -lpython2.2
It runs through with setting PYTHONPATH to my old numarray(1.1) directory. And now with numarray CVS it now doesn't even find the library... $ ./testnumarrayembedded Fatal Python error: Can't import module 'numarray.libnumarray' Aborted
Still, my non-embedded python shell runs all fine - your CVS-update (-j...) suggestion didn't seem to make a difference.
Thanks, Sebastian
On Tuesday 10 May 2005 05:13, Todd Miller wrote:
On Mon, 2005-05-09 at 17:58 -0700, Sebastian Haase wrote:
Hi,
Now I'm back to this problem. In the meantime I upgraded again:
na.__version__
'1.4.0' All my python2.2 + numarray + my-extensions work. (Debian Woody) Only my C++ program segfaults. It looked to me like a problem with NA_NewAllFromBuffer -- was there any recent change ?
Yes there was. You can undo it in your private CVS checkout like this:
% cd numarray/Src % cvs update -j 1.3 -1.2 libnumarray.ch % cd .. % python setup.py install --genapi --selftest
(I saw that I was using a bytestride of 0 - but it used to work and giving a 1 for that argument still segfaults -- is 0 OK ??)
I don't know for sure. It seems like it should be... just leads to 0 strides.
Now I noticed in the gdb traceback that I chokes is the function deferred_libnumarray_init (( ((#0 0x406d68d5 in PyObject_GetAttrString () from ((/usr/lib/libpython2.2.so.0.0 #1 0x410f905e in ((deferred_libnumarray_init () at Src/libnumarraymodule.c:149
Is this function the standard function that always get called to init numarray (in embedded Python AND using "normal" python) ?
Yes and no. deferred_libnumarray_init() is really for stuff that can't be done at libnumarray.so init time. This is because of circular dependencies.
The "real" numarray init function is either "import numarray" in Python or "import_array()" in C. After that, certain calls into the C-API trigger the deferred initialization. deferred_libnumarray_init() is never called directly by application code.
Have you tested this with Python2.2 ?
Yes, using the numarray unit tests. NewAllFromBuffer is also used directly by the numarray ufuncs, so it is tested to a degree by numarray itself. My gut feel is you're hitting an embedding problem, but my gut is only so accurate.
(See for more info below == I did follow Todd's suggestions 1 & 2, but hope not having to recompile python from source.)
That's really not hard. A 5-10 minute exercise usually.
Regards, Todd
------------------------------------------------------- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the first software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the first software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion --
Todd, Thanks for working on this with me. I did not use -fPIC and/or -shared. What is -fPIC anywhy ? I have been using my code + makefile since numarray 0.3.6 (maybe 0.4). I took my gcc/g++ arguments from examples in the wxWindows/wxPython. Trying your modified(attached) c-version I get the exact same behaviour that I had before IF I DON'T use -shared. (I.e. works with numarray1.1; new numarray gives "Can't import module 'numarray.libnumarray'") (I took -pthread out) With -shared I get a segfault before main() - gdb: Program received signal SIGSEGV, Segmentation fault. 0x00000001 in ?? () How can this be ? How can this change from numarray 1.1 to 1.2 (or 1.2.3) ? Thanks, Sebastian On Tuesday 10 May 2005 11:39, Todd Miller wrote:
Hi Sebastian,
I tried building and running this but I can't even get to main(). I even tried the attached, which I think is completely decoupled from numarray and there is the same problem.
BTW, did you leave out -fPIC and -shared on the command line?
To simplify things, you might try:
a. removing:
PyObject *pymodMain = PyImport_ImportModule("__main__"); PyObject* globals = PyModule_GetDict(pymodMain); PyEval_InitThreads();
b. get rid of threads for now
c. downshift from C++ to C
Bottom line: I think this is a more fundamental "embedding" setup problem.
Regards, Todd
On Tue, 2005-05-10 at 13:31, Sebastian Haase wrote:
Todd, I would be very thankful if you could compile and run this program: <testnumarrayembedded.cpp> #include <Python.h> #include <numarray/libnumarray.h> int main() { Py_Initialize(); PyObject *pymodMain = PyImport_ImportModule("__main__"); PyObject* globals = PyModule_GetDict(pymodMain); PyEval_InitThreads();
// import_libnumarray(); PyObject *module = PyImport_ImportModule("numarray.libnumarray"); if (!module) Py_FatalError("Can't import module 'numarray.libnumarray'"); if (module != NULL) { PyObject *module_dict = PyModule_GetDict(module); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); if (PyCObject_Check(c_api_object)) { libnumarray_API = (void **)PyCObject_AsVoidPtr(c_api_object); } else { Py_FatalError("Can't get API for module 'numarray.libnumarray'"); } } float *colMapsRGB = new float[4 * 3 * 256]; int shape[3]= { 4, 3, 256 }; PyObject *obj = PyBuffer_FromReadWriteMemory(colMapsRGB, 4*3 *256 * sizeof(float)); PyObject *obj2 =(PyObject*) NA_NewAllFromBuffer( 3, shape, tFloat32, obj, 0, 0, NA_ByteOrder(), 1, 1/*writable*/) ; return 0; } </testnumarrayembedded.cpp>
I compile with: g++-2.95 -o testnumarrayembedded -Wall -finline-functions -g -I/usr/include/ python2.2 -I/jws30/haase/myCVS/numarray/Include testnumarrayembedded.cpp -pthread -lpython2.2
It runs through with setting PYTHONPATH to my old numarray(1.1) directory. And now with numarray CVS it now doesn't even find the library... $ ./testnumarrayembedded Fatal Python error: Can't import module 'numarray.libnumarray' Aborted
Still, my non-embedded python shell runs all fine - your CVS-update (-j...) suggestion didn't seem to make a difference.
Thanks, Sebastian
On Tuesday 10 May 2005 05:13, Todd Miller wrote:
On Mon, 2005-05-09 at 17:58 -0700, Sebastian Haase wrote:
Hi,
Now I'm back to this problem. In the meantime I upgraded again:
> na.__version__
'1.4.0' All my python2.2 + numarray + my-extensions work. (Debian Woody) Only my C++ program segfaults. It looked to me like a problem with NA_NewAllFromBuffer -- was there any recent change ?
Yes there was. You can undo it in your private CVS checkout like this:
% cd numarray/Src % cvs update -j 1.3 -1.2 libnumarray.ch % cd .. % python setup.py install --genapi --selftest
(I saw that I was using a bytestride of 0 - but it used to work and giving a 1 for that argument still segfaults -- is 0 OK ??)
I don't know for sure. It seems like it should be... just leads to 0 strides.
Now I noticed in the gdb traceback that I chokes is the function deferred_libnumarray_init (( ((#0 0x406d68d5 in PyObject_GetAttrString () from ((/usr/lib/libpython2.2.so.0.0 #1 0x410f905e in ((deferred_libnumarray_init () at Src/libnumarraymodule.c:149
Is this function the standard function that always get called to init numarray (in embedded Python AND using "normal" python) ?
Yes and no. deferred_libnumarray_init() is really for stuff that can't be done at libnumarray.so init time. This is because of circular dependencies.
The "real" numarray init function is either "import numarray" in Python or "import_array()" in C. After that, certain calls into the C-API trigger the deferred initialization. deferred_libnumarray_init() is never called directly by application code.
Have you tested this with Python2.2 ?
Yes, using the numarray unit tests. NewAllFromBuffer is also used directly by the numarray ufuncs, so it is tested to a degree by numarray itself. My gut feel is you're hitting an embedding problem, but my gut is only so accurate.
(See for more info below == I did follow Todd's suggestions 1 & 2, but hope not having to recompile python from source.)
That's really not hard. A 5-10 minute exercise usually.
Regards, Todd
participants (2)
-
Sebastian Haase -
Todd Miller