Mailman 3 July 2003 - SciPy-User

Using NumArray to facilitate Matrix arithmetic
by Colin J. Williams July 17, 2003

July 17, 2003

-------- Original Message -------- [Perry Greenfield] I haven't had much time to respond to these comments and queries, but I'll do my best. [Colin W. >>> NDArray and NumArray are the basic classes of numarray, but >>> the design decision seems to have been made to deprecate the use >>> of class methods in favour of factory functions. This appears >>> to inhibit the development of sub-classes to NumArray. > [Perry Greenfield] I'm not sure what this comment refers to. If it is important, can you be more explicit about the inhibition regarding subclasses? [Colin W.] I feel that it could be important and will try to put together an illustrative example over the next week or so. There are a number of functions in the package which might, with benefit, be replaced with methods of NumArray. Such methods could then be inherited by any sub-classes. In the case of a possible Matrix class, there are some functions/methods which my be represented, in a more user friendly way by class properties. For example, in matrix algebra, one typically represents transpose with a superscript T or an inverse with a superscript -1. Using properties, we could use A.T or B.I to represent the transpose or the inverse of matrices A and B respectively. These are not new ideas. Huaiyu Zhu started implementing them in his MatPy project. [Eric Jones] >> Not sure what this is about because I haven't followed it that closely, >> but I will say that sub-classing arrays isn't all that it is cracked up >> to be in practice. One of the nicest things about arrays is there array >> arithmetic. They allow you to write mathematic formula-like code for >> numerical operations. When you sub-class an array, say with an Image >> class, all mathematic operations work, but they return arrays instead of >> images -- not so nice. You end up having to do a lot of work to get the >> behavior you want from the sub-classing. I'm not saying it isn't >> occasionally useful, it just isn't as nice as it first sounds... > [Colin W.] This notion of being able to write mathematics like code is at the root of this. Suppose that A is a Matrix and B is an array, then A + B ==> a matrix, with coercion, is not that much different from array operations. A * B ==> a matrix is similar but the multiplication implies the operation of linear algebra, rather than a simple element by element multiplication. If we wished to find the common norm for A, it is a little simpler to write A.Norm than to write A.Norm(), using a method or Norm(A), for a function. [Perry Greenfield] My 2 cents on subclassing: It isn't useless, but it is rarely trivial. Eric is right that most subclassing involves overriding all the operators and that will generally involve some work. On the other hand, subclassing is generally far more preferable to writing your own stuff from scratch (I'm guessing anyway). But on the third hand, outside of the customizations referred to below, I haven't seen a good practical example of subclassing numeric arrays yet so I can't comment on how useful it is yet (not to say we won't eventually come up with some). [Colin W.] Let's look at each of these: Overriding: Yes, that's a chore, but a boiler plate could be prepared for the class and then be used by each sub-class builder. Less work: The sub-class constructor is largely riding on the back of the class builder. Labour saving is one of the potential benefits of the object oriented approach. An example: Since Huaiyu Zhu was working largely with Numeric, he could not sub-class. The latest version of Matrix.py that I've seen is of 26-Aug-01. I'll put a simple illustrative example together. [Colin W.] > I have been looking at the sub-classing of NumArray to provide > matrix arithmetic, along the lines of Huaiyu Zhu, but not quite > as ambitious. > > So that A + B, A * B, A / B all make sense, where A and B are matrices. > > Similarly A.T delivers the transpose of A or B.I delivers the > inverse of B. > > Does anyone have any thoughts? > [Perry Greenfield] Yes, I do. Doing this would be (relatively) easy. But it would be wrong (IMHO). If you do this, you will break interoperability with other numarray modules and packages. There will be lots of code that will assume that * means element-by-element multiplication, not matrix multiplication. So by doing this you in effect will be fracturing the numeric community. If the judgement is that there is very little shared code (e.g., FFT, signal processing, linear algebra, etc.) between the two, little harm would be done. I tend to think that isn't the case. Now one has to wrap these modules to work with the matrix arrays, or if there are matrix modules available, then they have to be wrapped to work with numarray. [Colin W.] I agree that interoperability with other numarray modules and packages is essential and doubt that very much would need to be changed in NumArray to facilitate inheritance. For some, the idea of element by element operations takes a little getting use to. For multi-dimensional arrays, it is the only way to go. For two dimensional array, there could well be a choice, which the user makes by choosing to work with a NumArray class or a Matrix class. The latter is based on an extensive literature which has grown over that last 50 and more years. It would seem to be mutually beneficial to have these classes from the same root. [Perry Greenfield] I'd rather see one of the proposals to allow extending operator syntax to allow defining new Python operators so that matrix multiplication and the small number of other matrix operations can be accomodated rather than making incompatible types of arrays/matrices. I believe Guido will accomodate this should enough of the community make a case for it. [Colin W] It would be nice if there were more functional characters available on the keyboard. However, groupings of characters to achieve operations on particular data types would complicate Python and would not likely appeal to our BDFL. The idea of operator overloading is already well established for Python. [Colin W] >>> MatPy is an effort to facilitate matrix operations. It >>> seems to have resulted from a lot of work by Huaiyu Zhu. >>> >>> An extended discussion took place during 2000 to 2001, >>> but there seems to be little current interest. [Eric Jones] >>> >>> Don't know much about this. [Colin W] >>> The project was originally based on Numeric, but appears >>> to be in transition to numarray. Again, the decision >>> to use factory functions, rather than exploiting the >>> full capability of the Python class structure, appears >>> to have been made. > [Perry Greenfield] Again, not sure what this refers to. Also, I'm not aware of any work that that Huaiyu is doing with regard to this or numarray recently. [Colin W.] > Source Forge advertises the availability of version 0.4 > as a downloadable Win 32 package, but it seems to have > regressed to version 0.3. > [Perry Greenfield] numarray? The latest version should be 0.5 (and 0.6 will be released very shortly). [Colin W.] No, my comments here were with regard to MatPy. Please see: http://matpy.sourceforge.net/ The last change appears to be about August 2001.

1 0

finding djbfft
by John Hunter July 16, 2003

July 16, 2003

I am trying to build scipy SciPy-0.2.0_alpha_200.4161 with djbfft on a linux box. I have built djbfft, ran install and instcheck. This created /usr/local/djbfft/lib/ which contains djbfft.a. When I build scipy with > python setup.py build I get djbfft_info: NOT AVAILABLE DJBFFT (http://cr.yp.to/djbfft.html) libraries not found. Directories to search for the libraries can be specified in the scipy_distutils/site.cfg file (section [djbfft]) or by setting the DJBFFT environment variable. I have tried setting setenv DJBFFT /usr/local/djbfft/lib and setenv DJBFFT /usr/local/djbfft as well as placing djbfft.a in /usr/lib and when that failed, renaming it to libdjbfft.a. To no avail. I find no site.cfg file in the scipy src tree to modify. How do I build scipy with djbfft? Thanks, John Hunter

1 0

Enthought package of various Python related stuff
by Colin J. Williams July 16, 2003

July 16, 2003

I have downloaded this, it is massive - 62MB. It includes BTrees and freetype, which are not referenced in the included documentation, also massive - 29MB. Colin W.

2 1

Detecting clusters in an array
by Nils Wagner July 16, 2003

July 16, 2003

Hi all, Let us assume that we have a one-dimensional array, let's say A, of complex and/or real numbers. Now, I am interested in the detection of clusters. Each cluster, if any, consists of the elements of A, which lie inside a circle with fixed radius epsilon) How can I solve this problem most efficiently in scipy. A small example would be appreciated. Thanks in advance Nils nc, nec, cl = cluster(A, epsilon) Output nc : total number of clusters nec[i] : number of elements of cluster i cl[0:sum(nec)] : elements of all clusters

1 0

Python's various numeric packages
by Colin J. Williams July 16, 2003

July 16, 2003

Is there some place where there is an overview of the various packages, their relationships and their present stages of development? I have downloaded SciPy: SciPy-0.2.0_alpha_202.4187.win32-py2.1.exe The install fails. It does not permit the install directory to be entered. It does not detect the registered (XP) Python installation to suggest site-packages as a likely install directory. Does SciPy depend on Numeric? Numeric appears to be fully operational, but no longer being developed as it is being replaced eventually by numarray. The intended time frame for this evolution is not clear. numarray appears to be largely operable, but speed problems have been reported. NDArray and NumArray are the basic classes of numarray, but the design decision seems to have been made to deprecate the use of class methods in favour of factory functions. This appears to inhibit the development of sub-classes to NumArray. MatPy is an effort to facilitate matrix operations. It seems to have resulted from a lot of work by Huaiyu Zhu. An extended discussion took place during 2000 to 2001, but there seems to be little current interest. The project was originally based on Numeric, but appears to be in transition to numarray. Again, the decision to use factory functions, rather than exploiting the full capability of the Python class structure, appears to hav been made. Source Forge advertises the availability of version 0.4 as a downloadable Win 32 package, but it seems to have regressed to version 0.3. I would appreciate any advice. Colin W.

7 9

RE: [SciPy-user] Matrix Arithmetic
by Perry Greenfield July 15, 2003

July 15, 2003

I haven't had much time to respond to these comments and queries, but I'll do my best. > Behalf Of Colin J. Williams > From: "eric jones" <eric(a)enthought.com> > [snip] > >>> numarray appears to be largely operable, but speed problems have been >>> reported. NDArray and NumArray are the basic classes of numarray, but >>> >>> the >>> >>> design decision seems to have been made to deprecate the use of class >>> methods in favour of factory functions. This appears to inhibit the >>> development of sub-classes to NumArray. > I'm not sure what this comment refers to. If it is important, can you be more explicit about the inhibition regarding subclasses? > >> Not sure what this is about because I haven't followed it that closely, >> but I will say that sub-classing arrays isn't all that it is cracked up >> to be in practice. One of the nicest things about arrays is there array >> arithmetic. They allow you to write mathematic formula-like code for >> numerical operations. When you sub-class an array, say with an Image >> class, all mathematic operations work, but they return arrays instead of >> images -- not so nice. You end up having to do a lot of work to get the >> behavior you want from the sub-classing. I'm not saying it isn't >> occasionally useful, it just isn't as nice as it first sounds... > My 2 cents on subclassing: It isn't useless, but it is rarely trivial. Eric is right that most subclassing involves overriding all the operators and that will generally involve some work. On the other hand, subclassing is generally far more preferable to writing your own stuff from scratch (I'm guessing anyway). But on the third hand, outside of the customizations referred to below, I haven't seen a good practical example of subclassing numeric arrays yet so I can't comment on how useful it is yet (not to say we won't eventually come up with some). > I have been looking at the sub-classing of NumArray to provide matrix > arithmetic, along > the lines of Huaiyu Zhu, but not quite as ambitious. > > So that A + B, A * B, A / B all make sense, where A and B are matrices. > > Similarly A.T delivers the transpose of A or B.I delivers the > inverse of B. > > Does anyone have any thoughts? > > Colin W. > Yes, I do. Doing this would be (relatively) easy. But it would be wrong (IMHO). If you do this, you will break interoperability with other numarray modules and packages. There will be lots of code that will assume that * means element-by-element multiplication, not matrix multiplication. So by doing this you in effect will be fracturing the numeric community. If the judgement is that there is very little shared code (e.g., FFT, signal processing, linear algebra, etc.) between the two, little harm would be done. I tend to think that isn't the case. Now one has to wrap these modules to work with the matrix arrays, or if there are matrix modules available, then they have to be wrapped to work with numarray. I'd rather see one of the proposals to allow extending operator syntax to allow defining new Python operators so that matrix multiplication and the small number of other matrix operations can be accomodated rather than making incompatible types of arrays/matrices. I believe Guido will accomodate this should enough of the community make a case for it. > >>> MatPy is an effort to facilitate matrix operations. It seems to have >>> resulted from a lot of work by Huaiyu Zhu. An extended discussion >>> >>> took >>> >>> place during 2000 to 2001, but there seems to be little current >>> >>> interest. > >> Don't know much about this. > > > The project was originally based on Numeric, but appears to be in > transition to > numarray. Again, the decision to use factory functions, rather than > exploiting > the full capability of the Python class structure, appears to hav been > > made. > Again, not sure what this refers to. Also, I'm not aware of any work that that Huaiyu is doing with regard to this or numarray recently. > Source Forge advertises the availability of version 0.4 as a > > downloadable > > Win 32 package, but it seems to have regressed to version 0.3. > numarray? The latest version should be 0.5 (and 0.6 will be released very shortly). Perry Greenfield

1 0

Re: SciPy-user digest, Vol 1 #445 - 6 msgs
by Colin J. Williams July 15, 2003

July 15, 2003

1 0

Debian packages of scipy and f2py
by José Fonseca July 14, 2003

July 14, 2003

I've make Debian packages of scipy and f2py. They are available from my apt-get repository at: http://jrfonseca.dyndns.org/debian/ I referred to Joe Reinhardt's packages http://people.debian.org/~jmr/ but have done these packages from scratch. These come for all python versions in debian unstable (2.1, 2.2 and 2.3). I've run the standard tests successfully but I'd appreciate that problems with these packages to be reported to me [privately] as I'd like to have these on Debian proper at some time. One tricky issue I'd like to address with the scipy and f2py maintainers is the the scipy_distutils. Both scipy and f2py attempt to install to /usr/lib/python*/site-packages/scipy_distutils and at least on Debian this is not admissible so the packages conflict with each other and are mutually exclusive. This is particularly even more annoying for me, since for building scipy f2py is needed, so I need to install and uninstall them as a maniac. Is there any solution that allow both these packages to happily co-exist in a tree without overlapping? Enjoy, José Fonseca PS: Please CC'me on any reply as I'm not subscribed to all these lists.

1 0

Possible bug in linalg.lu
by Nils Wagner July 14, 2003

July 14, 2003

Hi all, >>> C array([[ 3.003+0.j, 1. +0.j, 0. +0.j, 0. +0.j], [ 3. +0.j, 3.003+0.j, 2. +0.j, 0. +0.j], [ 0. +0.j, 2. +0.j, 3.003+0.j, 3. +0.j], [ 0. +0.j, 0. +0.j, 1. +0.j, 2. +0.j]]) >>> linalg.lu(C) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/linalg/decomp.py", line 238, in lu p,l,u,info = flu(a1,permute_l=permute_l,overwrite_a = overwrite_a) TypeError: object of type 'None' is not callable >>> For what reason ? Nils

2 3

PLEA FOR ASSISTANCE
by Mr Mathosa Garba Zhiri July 13, 2003

July 13, 2003

Good Day, With warm heart I offer my friendship, and my greetings, and I hope this letter meet you in good time. It will be surprising to you to receive this proposal from me since you do not know me personally. However, I am sincerely seeking your confidence in this transaction, which I propose with my free mind and as a person of integrity. My name is MATHOSA GARBA,the son of ZHIRI GARBA, a farmer from Zimbabwe, murdered in the land dispute in my country. As led by my instinct, I decided to contact you through email, after searching for contacts via the internet, as it is the only means I can contact anybody since I am cutting off ties with Zimbabwe for now. I apologize if this is not acceptable to you. The purpose of this letter is to seek your most needed assistance in a business venture. Due to the land and political problems in Zimbabwe, as a result of President Robert Mugabe's introduction of new Land Act Reform wholly affecting the rich white farmers and the few rich black farmers, all white farmers were asked to surrender their farms to the government for re-distribution and infact to his political party members and my father though black was the treasurer of the farmers association and a strong member of an opposition party that did not support the president's idea. He then ordered his party members and the police under his pay row to invade my father's farm and burn down everything in the farm. They killed my father and took away a lot of items from his farm. After the death of my father, our local pastor and a close friend of my father handed us over will documents with instructions from my father that we should leave Zimbabwe incase anything happen to him. The will documents has a certificate of deposit, confirming a cash deposit totaling Fifteen million five hundred thousand united state dollars. [15.5m] Kept in custody for us in a security company unknown to the company that the content is money hence it was deposited as personal valuable items. This money was deposited with this Private Security Company for safety and security reasons, and was to be used for the purchase of land, new machines, chemicals and establishment of new farms in Botswana. This violent and barbaric act by Mugabe has since led to the death of my beloved mother and other innocent lives. I was continually threatened to abandon my inheritance from my father after he was murdered. I resisted for a while, but when the danger became unbearable, and I survived two murder attempts, I fled Zimbabwe with the help of my father's close friend Mr. John Casahans from Australia also a farmer who was leaving in Zimbabwe with us but left with his family following this ugly development I have tried to reach him but all to no avail. I am currently staying in the Netherlands where I am seeking political asylum. In fact my decision to come here to seek asylum, is because the security company has a branch here in the netherlands. I need to transfer this money to an account and invest part of the money. Since the law of Netherlands prohibits assylum seeker to open any bank account or to be involved in any financial transaction, this is why I am seeking a genuine and reliable partner, whose account this money can be transferred, hence this proposal to you. You have to understand that this decision taken by me is a very big and brave one, and it entrusts my future in your hands, as a result of the safe keeping of this money. If you accept to assist me, all I want you to do for me, is to assist with arrangements to claim the deposit from the security company. For your assistance, I have two options for you. Firstly you can choose to have 20% of the money for your assistance, and helping me open an account for the money to be deposited here, or you can go into partnership with me for the proper profitable investment of the money in your country. Whichever of the option you choose, please do notify me in your reply. I have also set aside 1 % of this money for all kinds of expenses that come our way in the process of this transaction, and 4% for Charity donation. If you prefer to accept the 20% for your moral and financial assistance then the balance will be left in the account here for me. Please, I want you to maintain absolute secrecy for the purpose of smooth handling of this transaction. You can reach me through my private email box: zhirigarba88(a)netscape.net I look forward to your reply and I thank you in advance as I anticipate your co-operation. Sincerely, Mathosa Garba.

1 0