Re: [Numpy-discussion] Numeric on OS X - Anyone get it to work ?

Paul F Dubois writes:
Such a change breaks other platforms...
Could you give a quick explanation why? I thought the whole point of the "extern" specifier was to flag that this variable was defined elsewhere. Otherwise, doesn't it imply that the API pointer is defined in each file that includes arrayobject.h? i.e. shouldn't headers declare "extern double x" for everything except the file that actually defines x? I should go back and read my copy of K&R... --Johann -- Johann Hibschman johann@physics.berkeley.edu

I've built a compiled Numeric-19.0.0 for Python2.1 on Win32. This should be exactly the same as the other version i built for Python-2.0, just built with a different python version. I've put the file up here, hopefully it can soon be moved into the Numeric sourceforge project. http://pygame.seul.org/ftp/contrib/Numeric-19.0.0-Python-2.1.zip

I haven't been able to get into the SourceForge ftp site all day, it has maxed out its users. I suspect the Python has become too popular...? Anyway, Pete's zip isn't there yet. Pete, could you tell me privately the method you use for making the zips? I have trouble getting the paths right. I have what will probably be 19.1.0 in CVS now. I have added a package, kinds, which is a reference implementation of PEP 0424 for your exploration. It does not install by default on Unix; you have to edit setup_all.py to make it do that. I plan to add a floating 32-bit scalar type to this module before completion. Here is the current text of the PEP: PEP: 242 Title: Numeric Kinds Version: $Revision: 1.1 $ Author: paul@pfdubois.com (Paul F. Dubois) Status: Draft Type: Standards Track Created: 17-Mar-2001 Python-Version: 2.2 Post-History: Abstract This proposal gives the user optional control over the precision and range of numeric computations so that a computation can be written once and run anywhere with at least the desired precision and range. It is backward compatible with existing code. The meaning of decimal literals is clarified. Rationale Currently it is impossible in every language except Fortran 90 to write a program in a portable way that uses floating point and gets roughly the same answer regardless of platform -- or refuses to compile if that is not possible. Python currently has only one floating point type, equal to a C double in the C implementation. No type exists corresponding to single or quad floats. It would complicate the language to try to introduce such types directly and their subsequent use would not be portable. This proposal is similar to the Fortran 90 "kind" solution, adapted to the Python environment. With this facility an entire calculation can be switched from one level of precision to another by changing a single line. If the desired precision does not exist on a particular machine, the program will fail rather than get the wrong answer. Since coding in this style would involve an early call to the routine that will fail, this is the next best thing to not compiling. Supported Kinds of Ints and Floats Complex numbers are treated separately below, since Python can be built without them. Each Python compiler may define as many "kinds" of integer and floating point numbers as it likes, except that it must support at least two kinds of integer corresponding to the existing int and long, and must support at least one kind of floating point number, equivalent to the present float. The range and precision of the these required kinds are processor dependent, as at present, except for the "long integer" kind, which can hold an arbitrary integer. The built-in functions int(), long(), and float() convert inputs to these default kinds as they do at present. (Note that a Unicode string is actually a different "kind" of string and that a sufficiently knowledgeable person might be able to expand this PEP to cover that case.) Within each type (integer, floating) the compiler supports a linearly-ordered set of kinds, with the ordering determined by the ability to hold numbers of an increased range and/or precision. Kind Objects Two new standard functions are defined in a module named "kinds". They return callable objects called kind objects. Each int or floating kind object f has the signature result = f(x), and each complex kind object has the signature result = f(x, y=0.). int_kind(n) For an integer argument n >= 1, return a callable object whose result is an integer kind that will hold an integer number in the open interval (-10**n,10**n). The kind object accepts arguments that are integers including longs. If n == 0, returns the kind object corresponding to the Python literal 0. float_kind(nd, n) For nd >= 0 and n >= 1, return a callable object whose result is a floating point kind that will hold a floating-point number with at least nd digits of precision and a base-10 exponent in the closed interval [-n, n]. The kind object accepts arguments that are integer or float. If nd and n are both zero, returns the kind object corresponding to the Python literal 0.0. The compiler will return a kind object corresponding to the least of its available set of kinds for that type that has the desired properties. If no kind with the desired qualities exists in a given implementation an OverflowError exception is thrown. A kind function converts its argument to the target kind, but if the result does not fit in the target kind's range, an OverflowError exception is thrown. Besides their callable behavior, kind objects have attributes giving the traits of the kind in question. 1. name is the name of the kind. The standard kinds are called int, long, double. 2. typecode is a single-letter string that would be appropriate for use with Numeric or module array to form an array of this kind. The standard types' typecodes are 'i', 'O', 'd' respectively. 3. Integer kinds have these additional attributes: MAX, equal to the maximum permissible integer of this kind, or None for the long kind. MIN, equal to the most negative permissible integer of this kind, or None for the long kind. 4. Float kinds have these additional attributes whose properties are equal to the corresponding value for the corresponding C type in the standard header file "float.h". MAX, MIN, DIG, MANT_DIG, EPSILON, MAX_EXP, MAX_10_EXP, MIN_EXP, MIN_10_EXP, RADIX, ROUNDS (== FLT_RADIX, FLT_ROUNDS in float.h) These values are of type integer except for MAX, MIN, and EPSILON, which are of the Python floating type to which the kind corresponds. Attributes of Module kinds int_kinds is a list of the available integer kinds, sorted from lowest to highest kind. By definition, int_kinds[-1] is the long kind. float_kinds is a list of the available floating point kinds, sorted from lowest to highest kind. default_int_kind is the kind object corresponding to the Python literal 0 default_long_kind is the kind object corresponding to the Python literal 0L default_float_kind is the kind object corresponding to the Python literal 0.0 Complex Numbers If supported, omplex numbers have real and imaginary parts that are floating-point numbers with the same kind. A Python compiler must support a complex analog of each floating point kind it supports, if it supports complex numbers at all. If complex numbers are supported, the following are available in module kinds: complex_kind(nd, n) Return a callable object whose result is a complex kind that will hold a complex number each of whose components (.real, .imag) is of kind float_kind(nd, n). The kind object will accept one argument that is of any integer, real, or complex kind, or two arguments, each integer or real. complex_kinds is a list of the available complex kinds, sorted from lowest to highest kind. default_complex_kind = is the kind object corresponding to the Python literal 0.0j. The name of this kind is doublecomplex, and its typecode is 'D'. Complex kind objects have these addition attributes: floatkind is the kind object of the corresponding float type. Examples In module myprecision.py: import kinds tinyint = kinds.int_kind(1) single = kinds.float_kind(6, 90) double = kinds.float_kind(15, 300) csingle = kinds.complex_kind(6, 90) In the rest of my code: from myprecision import tinyint, single, double, csingle n = tinyint(3) x = double(1.e20) z = 1.2 # builtin float gets you the default float kind, properties unknown w = x * float(x) # but in the following case we know w has kind "double". w = x * double(z) u = csingle(x + z * 1.0j) u2 = csingle(x+z, 1.0) Note how that entire code can then be changed to a higher precision by changing the arguments in myprecision.py. Comment: note that you aren't promised that single != double; but you are promised that double(1.e20) will hold a number with 15 decimal digits of precision and a range up to 10**300 or that the float_kind call will fail. Open Issues No open issues have been raised at this time. Copyright This document has been placed in the public domain.

Hi Everyone, I've had the following done for awhile and hadn't thought to release it for general use. It is the standard Cephesmodule 1.3 that Travis Oliphant has had available for some time at http://pylab.sourceforge.net with the addition of docstrings for all the functions. I modified the original documentation a bit and simply converted it to docstrings. Simply substitute the new cephesmodule.c file with the one in the Cephes distro and compile as per usual. The docstrings have been a huge beneift for me, as I find the cephes function names a bit cryptic... You can find the file here: http://cfa160.harvard.edu/~ransom/cephesmodule.c Hope it will be useful for some of you as well. Scott -- Scott M. Ransom Address: Harvard-Smithsonian CfA Phone: (617) 495-4142 60 Garden St. MS 10 email: ransom@cfa.harvard.edu Cambridge, MA 02138 GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989

Scott Ransom writes:
This message brought me a strange sense of deja vu. A year ago I did the same thing - I submitted a patch which allowed ufuncs to have docstrings, for exactly this reason - the function names in Cephes being a bit cryptic. I also wrote a silly little script to generate the Cephes docstrings from the HTML doc. Please see: http://lists.sourceforge.net/archives/numpy-discussion/2000-March/000091.htm...

Right, but with most platforms' shared library systems, this means "in another source file that is part of the same shared library", not "in another shared library" or "in the main executable".
If a dynamically loaded module consists of more than one source file, then all but one of them (the one which calls import_array()) must define NO_IMPORT_ARRAY before including arrayobject.h. This is also the answer to Kevin Rodgers' question. However, it seems to me that the current arrangement in NumPy has another serious drawback: it should be impossible to link NumPy statically with the Python interpreter, due to multiply defined symbols. And I am rather sure that this was possible many versions ago, since I used NumPy on a Cray T3E, which does not have shared libraries. I checked my own extension modules that export a C API, and they all declare the API pointer static. This is also what the C API export section in the "Extending & Embedding" manual recommends. (OK, I admit that I wrote that section, so that is not a coincidence!) So perhaps the best solution is to make this static. Client modules that consist of more than one source code file with PyArray... calls must then call import_array() once in every such file, or equivalently pass on the PyArray_API pointer explicitly between the files. That sounds quite acceptable to me. BTW, extension modules with more than one source file create a risk of portability problems in any case, as the symbols shared between the files must necessarily be global. On platforms such as MacOS, or with static linking, this means they are global to the interpreter and all extension modules, with a resulting risk of name clashes. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

I've built a compiled Numeric-19.0.0 for Python2.1 on Win32. This should be exactly the same as the other version i built for Python-2.0, just built with a different python version. I've put the file up here, hopefully it can soon be moved into the Numeric sourceforge project. http://pygame.seul.org/ftp/contrib/Numeric-19.0.0-Python-2.1.zip

I haven't been able to get into the SourceForge ftp site all day, it has maxed out its users. I suspect the Python has become too popular...? Anyway, Pete's zip isn't there yet. Pete, could you tell me privately the method you use for making the zips? I have trouble getting the paths right. I have what will probably be 19.1.0 in CVS now. I have added a package, kinds, which is a reference implementation of PEP 0424 for your exploration. It does not install by default on Unix; you have to edit setup_all.py to make it do that. I plan to add a floating 32-bit scalar type to this module before completion. Here is the current text of the PEP: PEP: 242 Title: Numeric Kinds Version: $Revision: 1.1 $ Author: paul@pfdubois.com (Paul F. Dubois) Status: Draft Type: Standards Track Created: 17-Mar-2001 Python-Version: 2.2 Post-History: Abstract This proposal gives the user optional control over the precision and range of numeric computations so that a computation can be written once and run anywhere with at least the desired precision and range. It is backward compatible with existing code. The meaning of decimal literals is clarified. Rationale Currently it is impossible in every language except Fortran 90 to write a program in a portable way that uses floating point and gets roughly the same answer regardless of platform -- or refuses to compile if that is not possible. Python currently has only one floating point type, equal to a C double in the C implementation. No type exists corresponding to single or quad floats. It would complicate the language to try to introduce such types directly and their subsequent use would not be portable. This proposal is similar to the Fortran 90 "kind" solution, adapted to the Python environment. With this facility an entire calculation can be switched from one level of precision to another by changing a single line. If the desired precision does not exist on a particular machine, the program will fail rather than get the wrong answer. Since coding in this style would involve an early call to the routine that will fail, this is the next best thing to not compiling. Supported Kinds of Ints and Floats Complex numbers are treated separately below, since Python can be built without them. Each Python compiler may define as many "kinds" of integer and floating point numbers as it likes, except that it must support at least two kinds of integer corresponding to the existing int and long, and must support at least one kind of floating point number, equivalent to the present float. The range and precision of the these required kinds are processor dependent, as at present, except for the "long integer" kind, which can hold an arbitrary integer. The built-in functions int(), long(), and float() convert inputs to these default kinds as they do at present. (Note that a Unicode string is actually a different "kind" of string and that a sufficiently knowledgeable person might be able to expand this PEP to cover that case.) Within each type (integer, floating) the compiler supports a linearly-ordered set of kinds, with the ordering determined by the ability to hold numbers of an increased range and/or precision. Kind Objects Two new standard functions are defined in a module named "kinds". They return callable objects called kind objects. Each int or floating kind object f has the signature result = f(x), and each complex kind object has the signature result = f(x, y=0.). int_kind(n) For an integer argument n >= 1, return a callable object whose result is an integer kind that will hold an integer number in the open interval (-10**n,10**n). The kind object accepts arguments that are integers including longs. If n == 0, returns the kind object corresponding to the Python literal 0. float_kind(nd, n) For nd >= 0 and n >= 1, return a callable object whose result is a floating point kind that will hold a floating-point number with at least nd digits of precision and a base-10 exponent in the closed interval [-n, n]. The kind object accepts arguments that are integer or float. If nd and n are both zero, returns the kind object corresponding to the Python literal 0.0. The compiler will return a kind object corresponding to the least of its available set of kinds for that type that has the desired properties. If no kind with the desired qualities exists in a given implementation an OverflowError exception is thrown. A kind function converts its argument to the target kind, but if the result does not fit in the target kind's range, an OverflowError exception is thrown. Besides their callable behavior, kind objects have attributes giving the traits of the kind in question. 1. name is the name of the kind. The standard kinds are called int, long, double. 2. typecode is a single-letter string that would be appropriate for use with Numeric or module array to form an array of this kind. The standard types' typecodes are 'i', 'O', 'd' respectively. 3. Integer kinds have these additional attributes: MAX, equal to the maximum permissible integer of this kind, or None for the long kind. MIN, equal to the most negative permissible integer of this kind, or None for the long kind. 4. Float kinds have these additional attributes whose properties are equal to the corresponding value for the corresponding C type in the standard header file "float.h". MAX, MIN, DIG, MANT_DIG, EPSILON, MAX_EXP, MAX_10_EXP, MIN_EXP, MIN_10_EXP, RADIX, ROUNDS (== FLT_RADIX, FLT_ROUNDS in float.h) These values are of type integer except for MAX, MIN, and EPSILON, which are of the Python floating type to which the kind corresponds. Attributes of Module kinds int_kinds is a list of the available integer kinds, sorted from lowest to highest kind. By definition, int_kinds[-1] is the long kind. float_kinds is a list of the available floating point kinds, sorted from lowest to highest kind. default_int_kind is the kind object corresponding to the Python literal 0 default_long_kind is the kind object corresponding to the Python literal 0L default_float_kind is the kind object corresponding to the Python literal 0.0 Complex Numbers If supported, omplex numbers have real and imaginary parts that are floating-point numbers with the same kind. A Python compiler must support a complex analog of each floating point kind it supports, if it supports complex numbers at all. If complex numbers are supported, the following are available in module kinds: complex_kind(nd, n) Return a callable object whose result is a complex kind that will hold a complex number each of whose components (.real, .imag) is of kind float_kind(nd, n). The kind object will accept one argument that is of any integer, real, or complex kind, or two arguments, each integer or real. complex_kinds is a list of the available complex kinds, sorted from lowest to highest kind. default_complex_kind = is the kind object corresponding to the Python literal 0.0j. The name of this kind is doublecomplex, and its typecode is 'D'. Complex kind objects have these addition attributes: floatkind is the kind object of the corresponding float type. Examples In module myprecision.py: import kinds tinyint = kinds.int_kind(1) single = kinds.float_kind(6, 90) double = kinds.float_kind(15, 300) csingle = kinds.complex_kind(6, 90) In the rest of my code: from myprecision import tinyint, single, double, csingle n = tinyint(3) x = double(1.e20) z = 1.2 # builtin float gets you the default float kind, properties unknown w = x * float(x) # but in the following case we know w has kind "double". w = x * double(z) u = csingle(x + z * 1.0j) u2 = csingle(x+z, 1.0) Note how that entire code can then be changed to a higher precision by changing the arguments in myprecision.py. Comment: note that you aren't promised that single != double; but you are promised that double(1.e20) will hold a number with 15 decimal digits of precision and a range up to 10**300 or that the float_kind call will fail. Open Issues No open issues have been raised at this time. Copyright This document has been placed in the public domain.

Hi Everyone, I've had the following done for awhile and hadn't thought to release it for general use. It is the standard Cephesmodule 1.3 that Travis Oliphant has had available for some time at http://pylab.sourceforge.net with the addition of docstrings for all the functions. I modified the original documentation a bit and simply converted it to docstrings. Simply substitute the new cephesmodule.c file with the one in the Cephes distro and compile as per usual. The docstrings have been a huge beneift for me, as I find the cephes function names a bit cryptic... You can find the file here: http://cfa160.harvard.edu/~ransom/cephesmodule.c Hope it will be useful for some of you as well. Scott -- Scott M. Ransom Address: Harvard-Smithsonian CfA Phone: (617) 495-4142 60 Garden St. MS 10 email: ransom@cfa.harvard.edu Cambridge, MA 02138 GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989

Scott Ransom writes:
This message brought me a strange sense of deja vu. A year ago I did the same thing - I submitted a patch which allowed ufuncs to have docstrings, for exactly this reason - the function names in Cephes being a bit cryptic. I also wrote a silly little script to generate the Cephes docstrings from the HTML doc. Please see: http://lists.sourceforge.net/archives/numpy-discussion/2000-March/000091.htm...

Right, but with most platforms' shared library systems, this means "in another source file that is part of the same shared library", not "in another shared library" or "in the main executable".
If a dynamically loaded module consists of more than one source file, then all but one of them (the one which calls import_array()) must define NO_IMPORT_ARRAY before including arrayobject.h. This is also the answer to Kevin Rodgers' question. However, it seems to me that the current arrangement in NumPy has another serious drawback: it should be impossible to link NumPy statically with the Python interpreter, due to multiply defined symbols. And I am rather sure that this was possible many versions ago, since I used NumPy on a Cray T3E, which does not have shared libraries. I checked my own extension modules that export a C API, and they all declare the API pointer static. This is also what the C API export section in the "Extending & Embedding" manual recommends. (OK, I admit that I wrote that section, so that is not a coincidence!) So perhaps the best solution is to make this static. Client modules that consist of more than one source code file with PyArray... calls must then call import_array() once in every such file, or equivalently pass on the PyArray_API pointer explicitly between the files. That sounds quite acceptable to me. BTW, extension modules with more than one source file create a risk of portability problems in any case, as the symbols shared between the files must necessarily be global. On platforms such as MacOS, or with static linking, this means they are global to the interpreter and all extension modules, with a resulting risk of name clashes. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
participants (6)
-
Charles G Waldman
-
Johann Hibschman
-
Konrad Hinsen
-
Paul F. Dubois
-
Pete Shinners
-
Scott Ransom