From sdfrost at UCSD.Edu  Tue Sep  3 07:28:03 2002
From: sdfrost at UCSD.Edu (sdfrost at UCSD.Edu)
Date: Tue Sep  3 07:28:03 2002
Subject: [Numpy-discussion] Comparison between Numpy/Numarray and Ox
Message-ID: <200209031427.HAA20024@smtp.ucsd.edu>

Dear Numpy list,

Has anyone looked at the comparitive speed of Numpy/Numarray vs. Ox?  For 
those that don't know, Ox is an implicitly typed, object oriented matrix 
programming language that is one of the fastest around, and also allows 
linking to C libraries (http://www.nuff.ox.ac.uk/Users/Doornik/index.html).  I 
use Python a lot, but I was wondering what the computational cost would be to 
switch my matrix-y stuff to Numpy.  Ox isn't open source...

Best wishes
Simon






From transue at niehs.nih.gov  Mon Sep  9 14:38:03 2002
From: transue at niehs.nih.gov (Tom Transue)
Date: Mon Sep  9 14:38:03 2002
Subject: [Numpy-discussion] Logical AND on arrays
Message-ID: <3D7D144B.AB8DCB2D@niehs.nih.gov>

If ar1 and ar2 are arrays, are the following two expressions supposed to give
the same result?

  (ar1 and ar2)

  logical_and(ar1,ar2)

The first form seems to return the value of the second array, which isn't very
useful.  It would be nice to map the first expression to do what the first does.

Tom Transue



From a.schmolck at gmx.net  Mon Sep  9 15:25:03 2002
From: a.schmolck at gmx.net (Alexander Schmolck)
Date: Mon Sep  9 15:25:03 2002
Subject: [Numpy-discussion] Logical AND on arrays
In-Reply-To: <3D7D144B.AB8DCB2D@niehs.nih.gov>
References: <3D7D144B.AB8DCB2D@niehs.nih.gov>
Message-ID: <yfsk7lukdll.fsf@black132.ex.ac.uk>

Tom Transue <transue at niehs.nih.gov> writes:

> If ar1 and ar2 are arrays, are the following two expressions supposed to give
> the same result?
> 
>   (ar1 and ar2)
> 
>   logical_and(ar1,ar2)
> 
> The first form seems to return the value of the second array, which isn't very
Only if the boolean value of the first array is true (i.e. iff not all its
values are nonzero).

> useful.  It would be nice to map the first expression to do what the first does.

Impossible. Python's `and` and `or` are shortcircuting operators with a fixed
meaning. ``A and B`` only evaluates both ``A`` and ``B`` iff both have the
boolean value 'True' (e.g. ``0 and 0/1`` is OK because the second part never
gets evaluated), otherwise the first 'False` value is returned.

While each class is free to define a truth testing method (`__nonzero__`, or
`__len__` if `__nonzero__` doesn't exist; the convention is that empty
containers and zero numbers are false and pretty much everything else is
true), there is (luckily) no way to change the meaning of `and` and `or` (just
as one (luckily) can't change what the `if` statement does).

If this is still a bit unclear, I think this question has been raised before
in this list, so you should be able to find a thread in the archive.

cheers,

alex


-- 
Alexander Schmolck     Postgraduate Research Student
                       Department of Computer Science
                       University of Exeter
A.Schmolck at gmx.net     http://www.dcs.ex.ac.uk/people/aschmolc/




From Chris.Barker at noaa.gov  Mon Sep  9 16:56:05 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Mon Sep  9 16:56:05 2002
Subject: [Numpy-discussion] Logical AND on arrays
References: <3D7D144B.AB8DCB2D@niehs.nih.gov> <yfsk7lukdll.fsf@black132.ex.ac.uk>
Message-ID: <3D7D32E4.1BAA1BE5@noaa.gov>

Alexander Schmolck wrote:
> Tom Transue <transue at niehs.nih.gov> writes:
> > useful.  It would be nice to map the first expression to do what the first does.
> 
> Impossible. Python's `and` and `or` are shortcircuting operators with a fixed
> meaning. ``A and B`` only evaluates both ``A`` and ``B`` iff both have the
> boolean value 'True' (e.g. ``0 and 0/1`` is OK because the second part never
> gets evaluated), otherwise the first 'False` value is returned.

true. Another option, however is to use the bitwise and aperator, which
is overloaded:

A & B

This generally will work in the cases you are likely to use it, like:

if (A > 5) & (B < 5):

as you'll be &-ing arrays of zeros and ones.

> there is (luckily) no way to change the meaning of `and` and `or` (just
> as one (luckily) can't change what the `if` statement does).

personally, I don't think it's that lucky. I'd rather lose the slight
benifits of the sometimes confusing ability to short-circuit, and get
full operator overloading. I think "and" and "or" are more like "<" than
"if". However, if I had designed a language it would be a pretty
wretched mess, I'm sure.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From jmiller at stsci.edu  Wed Sep 11 15:08:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 11 15:08:02 2002
Subject: [Numpy-discussion] Todd's NumPy Feature Request tracker surprise!
Message-ID: <3D7FBE61.7060805@stsci.edu>

I was really surprised today to find that Perry Greenfield and I were 
the only two developers listed on the Numeric Feature Requests tracker 
on Source Forge.  Since we work on numarray,  I removed us.  That leaves 
no one.

If you are a NumPy developer who wants to handle Feature Requests for 
Numeric,  you might want to add yourself back onto the list of assignees.  

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / SSG





From perry at stsci.edu  Thu Sep 12 21:17:05 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Sep 12 21:17:05 2002
Subject: [Numpy-discussion] rank-0 arrays
Message-ID: <NEBBIJKBMLDBLNCEEFOCCEEMCOAA.perry@stsci.edu>

Before we implement what we said we would regarding rank-0
arrays in numarray, there became apparent a couple new issues
that didn't really get considered in the first round of 
discussion (at least I don't recall that they did).

To restate the issue: there was a question about whether
an index to an array that identified a single element only
(i.e., not a slice, nor an incomplete index, e.g. x[3] where
x is two dimensional) should return a Python scalar or a
rank-0 array. Currently Numeric is inconsistent on this point.
One usually gets scalars, but on some occasions, rank-0 arrays
are returned. Good arguments are to be had for either alternative.

The primary advantage of returning rank-0 arrays is that they
reduce the need for conditional code checking to see if a result
is a scalar or an array. At the end of the discussion it was
decided to have numarray return rank-0 arrays in all instances of
single item indexing. Since then, a couple potential snags have
arisen. I've already discussed some of these with Paul Dubois
and Eric Jones. I'd like a little wider input before making a
final (or at least experimental) decision.

If we return rank-0 arrays, what should repr return for rank-0
arrays. My initial impression is that the following is highly
undesirable for a interactive session, but maybe it is just me:

>>> x = arange(10)
>>> x[2]
array(2)

We, of course, could arrange __repr__ to return "2" instead,
in other words print the simple scalar for all cases of rank-0
arrays. This would yield the expected output in the above
example. Nevertheless, isn't it violating the intent of repr?
Are there other examples where Python uses repr in a similar,
misleading manner? But perhaps most feel that returning array(2)
is perfectly acceptable and won't annoy users. I am curious
about what people think about this.

The second issue is an efficiency one. Currently numarray uses
Python objects for arrays. If we return rank-0 arrays for
single item indexing, then some naive uses of larger arrays
as sequences may lead to an enormous number of array objects
to be created. True, there will be equivalent means of doing
the same operation that won't result in massive object creations
(such as specifically converting an array to a list, which would 
be done much faster). Is this a serious problem?

These two issues led us to question whether we should indeed
return rank-0 arrays. We can live with either solution. But
we do want to make the right choice. We also know that both
functionalities must exist, e.g., indexing for scalars and 
indexing for rank-0 arrays and we will provide both. The issue
is what indexing syntax returns. One argument is that it is
not a great burden on programmers to use a method (or other
means) to obtain a rank-0 array always if that is important
for the code they are writing and that we should make the
indexing syntax return what most users (especially less expert
ones) intuitively expect (scalars I presume). But others feel
it is just as important for the syntax that a progammer uses
to be as simple as the interactive user expects (instead of
something like x.getindexasarrayalways(2,4,1) [well, with a much
better, and shorter, name])

Do either of these issues change anyone's opinion? If people
still want rank-0 arrays, what should repr do?

Perry Greenfield
 




From pearu at cens.ioc.ee  Fri Sep 13 00:20:03 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Fri Sep 13 00:20:03 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCCEEMCOAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.21.0209130903180.21821-100000@cens.kybi>

On Thu, 12 Sep 2002, Perry Greenfield wrote:

> If we return rank-0 arrays, what should repr return for rank-0
> arrays. My initial impression is that the following is highly
> undesirable for a interactive session, but maybe it is just me:
> 
> >>> x = arange(10)
> >>> x[2]
> array(2)
>
> We, of course, could arrange __repr__ to return "2" instead,
> in other words print the simple scalar for all cases of rank-0
> arrays. This would yield the expected output in the above
> example. Nevertheless, isn't it violating the intent of repr?
> Are there other examples where Python uses repr in a similar,
> misleading manner? But perhaps most feel that returning array(2)
> is perfectly acceptable and won't annoy users. I am curious
> about what people think about this.

I think it would be confusing if the result of repr would be `2' and not
`array(2)' because 2 and array(2) are not equivalent in all usages
but it should be clear from repr results as a first way to learn more
about the objects.

For example, if using array(2) as an index in Python native objects, then
TypeError is raised (as expected). In interactive session the quickest way
to check the type of a variable is just type its name and press enter:

>>> i
array(2)

Now, if repr(array(2)) returns '2', then one firstly assumes that
`i' is an integer. However, this would be very confusing if one sees
>>> some_list[i]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: sequence index must be integer
>>> i
2

So, I think that repr(array(2)) should return 'array(2)'. And users can
always change this behaviour locally by using sys.displayhook. Though, I
would recommend using tools like ipython for interactive sessions.

Btw, note that during an interactive session it would be *seemingly*
desired if also repr(string) would return str(string). For example,
when viewing documentation in interactive sessions. Wouldn't it be nice to
have

>>> sys.displayhook.__doc__
displayhook(object) -> None

Print an object to sys.stdout and also save it in __builtin__._

>>>

instead of the current behaviour:

>>> sys.displayhook.__doc__
'displayhook(object) -> None\n\nPrint an object to sys.stdout and also
save it in __builtin__._\n'
>>>


> The second issue is an efficiency one. Currently numarray uses
> Python objects for arrays. If we return rank-0 arrays for
> single item indexing, then some naive uses of larger arrays
> as sequences may lead to an enormous number of array objects
> to be created. True, there will be equivalent means of doing
> the same operation that won't result in massive object creations
> (such as specifically converting an array to a list, which would 
> be done much faster). Is this a serious problem?

Could array.__getitem_ and __getslice__ detect if their argument is
an array and skip using Python objects when iterating over indices?
If this is technically possible then it is not a good reason to drop
returning rank-0 arrays. The actual implementation may come later, though.

> If people still want rank-0 arrays, what should repr do?

Always return 'array(...)'.

You can also ask from python-dev for advice if numarray is considered to
be included to Python library in future. I am sure that repr issue will be
brought up if repr==str for 0-rank arrays.

Pearu




From huaiyu_zhu at yahoo.com  Fri Sep 13 01:46:03 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Fri Sep 13 01:46:03 2002
Subject: [Numpy-discussion] Tensor-like product with extra indices?
Message-ID: <Pine.LNX.4.33.0209130142080.1276-100000@mars.localdomain>

Two related questions:

1. Suppose  B = ravel(A) and
            B[i] corresponds to A[i0,i1,...].

   Are there named functions that map the indices

        (i0,i0,...) -> i

   and back?  This is assuming A.shape = (m0,m1,...) is known.

2. Suppose the above function is called ravel_index and its
   inverse unravel_index.

   Given a list of matrices [A0, A1, ...] is there a function that
   calculates a matrix B such that

        B[i,j]  = A0[i0,j] * A1[i1,j] * ...

   where i = ravel_index(i0, i1, ...).

   What about the inverse function that turns B into C such that

        C[i0,i1,...,j] = A0[i0,j] * A1[i1,j] * ...


I've considered reshape, ravel, take, put, outerproct but couldn't come
up with a combination to do it without a for-loop on j and a for-loop on
[A0,A1,...].

The above two questions I need answers right now.  But I can imagine a
need for a more general operations in the future.  Given a sequence of
arrays [A0,A1,...], produce a tensor product by picking certain indices
from each array (like i0,i1 above), while keeping some other indices
fixed (like j above).  For example, the above could be written as

        B = tensor_like([A0,A1...], [(0,None,None,...), ...,
                                (None,....,None,0), (1,1,...1)])

As another example, if len(A.shape)==3 then

        transpose(A) == tensor_like([A], [(2,), (1,), (0,)])


Huaiyu




From hinsen at cnrs-orleans.fr  Fri Sep 13 03:22:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 03:22:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.21.0209130903180.21821-100000@cens.kybi>
References: <Pine.LNX.4.21.0209130903180.21821-100000@cens.kybi>
Message-ID: <m3elbychut.fsf@chinon.cnrs-orleans.fr>

Pearu Peterson <pearu at cens.ioc.ee> writes:

> I think it would be confusing if the result of repr would be `2' and not
> `array(2)' because 2 and array(2) are not equivalent in all usages
> but it should be clear from repr results as a first way to learn more
> about the objects.

I agree. There will already be some inevitable confusion with both
rank-0 arrays and scalars around, with similar but not identical
behaviour. Rank-0 arrays shouldn't make it worse by using camouflage.

> > The second issue is an efficiency one. Currently numarray uses
> > Python objects for arrays. If we return rank-0 arrays for
> > single item indexing, then some naive uses of larger arrays
> > as sequences may lead to an enormous number of array objects
> > to be created. True, there will be equivalent means of doing
> > the same operation that won't result in massive object creations
> > (such as specifically converting an array to a list, which would 
> > be done much faster). Is this a serious problem?
> 
> Could array.__getitem_ and __getslice__ detect if their argument is
> an array and skip using Python objects when iterating over indices?

Of course they know that they are indexing an array, they are
defined at the level of the array class/type. However, they cannot
detect an iteration as opposed to a single item access.

I don't know if this efficiency problem could be important in
practice, probably only practice can tell. I have no idea how many
single-item indexing operations into arrays occur in my code, this is
not something I worried about when writing it.

If there will be scalar and rank-0 array returning variants of
indexing anyway, then I suppose that changing the index syntax
to one or the other is not a big effort. So my suggestion is to
make a test release and see what the reactions are.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Fri Sep 13 03:34:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 03:34:02 2002
Subject: [Numpy-discussion] Tensor-like product with extra indices?
In-Reply-To: <Pine.LNX.4.33.0209130142080.1276-100000@mars.localdomain>
References: <Pine.LNX.4.33.0209130142080.1276-100000@mars.localdomain>
Message-ID: <m3admmchab.fsf@chinon.cnrs-orleans.fr>

Huaiyu Zhu <huaiyu_zhu at yahoo.com> writes:

> Two related questions:
> 
> 1. Suppose  B = ravel(A) and
>             B[i] corresponds to A[i0,i1,...].
> 
>    Are there named functions that map the indices
> 
>         (i0,i0,...) -> i
> 
>    and back?  This is assuming A.shape = (m0,m1,...) is known.

Untested, but the idea should work:

  def ravel_index(array, *indices):
     offsets = Numeric.multiply.accumulate(array.shape[1:][::-1])[::-1]
     return Numeric.sum(Numeric.array(indices[:-1])*offsets) + indices[-1]

> 2. Suppose the above function is called ravel_index and its
>    inverse unravel_index.
> 
>    Given a list of matrices [A0, A1, ...] is there a function that
>    calculates a matrix B such that
> 
>         B[i,j]  = A0[i0,j] * A1[i1,j] * ...
> 
>    where i = ravel_index(i0, i1, ...).

Not for all i, which I assume is what you want to do.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From perry at stsci.edu  Fri Sep 13 07:06:14 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 07:06:14 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <m3elbychut.fsf@chinon.cnrs-orleans.fr>
Message-ID: <JFEGLNDJEDNOMPPHDEJFIEFAEAAA.perry@stsci.edu>

Konrad Hinsen writes:
>
> Pearu Peterson <pearu at cens.ioc.ee> writes:
>
> > I think it would be confusing if the result of repr would be `2' and not
> > `array(2)' because 2 and array(2) are not equivalent in all usages
> > but it should be clear from repr results as a first way to learn more
> > about the objects.
>
> I agree. There will already be some inevitable confusion with both
> rank-0 arrays and scalars around, with similar but not identical
> behaviour. Rank-0 arrays shouldn't make it worse by using camouflage.
>
I also agree that having repr hide the fact that it is an array is
a bad thing. But do you find getting "array(2)" when indexing
a single item acceptable when working interactively? I agree that
displayhook is probably the best way of altering this behavior if
a user desires (perhaps we will provide a module function to do so, but
not enable it by default).

> >
> > Could array.__getitem_ and __getslice__ detect if their argument is
> > an array and skip using Python objects when iterating over indices?
>
> Of course they know that they are indexing an array, they are
> defined at the level of the array class/type. However, they cannot
> detect an iteration as opposed to a single item access.
>
I don't know that it is desirable either to have iteration
return different kinds of objects than explicit indexing, even
if possible. Do we really want

for val in x:
    ...

to use scalars while

for i in range(len(x)):
    val = x[i]
    ...

uses rank-0 arrays? I'm inclined not to have different behavior for
what seems like identical iteration.

> I don't know if this efficiency problem could be important in
> practice, probably only practice can tell. I have no idea how many
> single-item indexing operations into arrays occur in my code, this is
> not something I worried about when writing it.
>
> If there will be scalar and rank-0 array returning variants of
> indexing anyway, then I suppose that changing the index syntax
> to one or the other is not a big effort. So my suggestion is to
> make a test release and see what the reactions are.
>
That is our plan (hence the reference to "experimental"), but I
wanted to see if there were strong feelings on this before doing so.

Perry




From jmiller at stsci.edu  Fri Sep 13 08:23:09 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 13 08:23:09 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <JFEGLNDJEDNOMPPHDEJFIEFAEAAA.perry@stsci.edu>
Message-ID: <3D81FBBF.9060703@stsci.edu>

Two other issues come up trying to implement the "rank-0 experiment":

1.  What should be the behavior of subscripting a rank-0 array?

a.  Return the scalar value                         (what numarray does 
now.  seems inconsistent)
b.  Raise an exception
c.  Return a copy  of the rank-0 array

2.  What's a decent notation for .asScalar()?

a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
does now)
b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
__call__)
c.  a[ <subscript_resulting_in_rank0> ].asScalar()  

Any strong opinions?

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / SSG






From hinsen at cnrs-orleans.fr  Fri Sep 13 08:40:06 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 08:40:06 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
References: <JFEGLNDJEDNOMPPHDEJFIEFAEAAA.perry@stsci.edu>
	<3D81FBBF.9060703@stsci.edu>
Message-ID: <m3ofb1c34o.fsf@chinon.cnrs-orleans.fr>

Todd Miller <jmiller at stsci.edu> writes:

> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray
> does now.  seems inconsistent)
> b.  Raise an exception
> c.  Return a copy  of the rank-0 array

I am for b). A rank-0 array is not a sequence and has no elements,
so indexing shouldn't be allowed.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From pearu at cens.ioc.ee  Fri Sep 13 08:48:07 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Fri Sep 13 08:48:07 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
Message-ID: <Pine.LNX.4.21.0209131842420.26238-100000@cens.kybi>

On Fri, 13 Sep 2002, Todd Miller wrote:

> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray does 
> now.  seems inconsistent)

-0.5

> b.  Raise an exception

+1

> c.  Return a copy  of the rank-0 array

-1

> 2.  What's a decent notation for .asScalar()?
> 
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
> does now)

+-0, just be consistent.

> b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
> __call__)

-1

> c.  a[ <subscript_resulting_in_rank0> ].asScalar()  

+0.5


Pearu




From xscottg at yahoo.com  Fri Sep 13 10:17:09 2002
From: xscottg at yahoo.com (Scott Gilbert)
Date: Fri Sep 13 10:17:09 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
Message-ID: <20020913171649.36428.qmail@web40106.mail.yahoo.com>

--- Todd Miller <jmiller at stsci.edu> wrote:
> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray does 
> now.  seems inconsistent)
>

If you think of it as "dereferencing" instead of "indexing", I think this
is most consistent.  It takes 3 elements to dereference a rank-3 array, so
it should take 0 elements to dereference a rank-0 array.


>
> 2.  What's a decent notation for .asScalar()?
> 
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
> does now)
> b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
> __call__)
> c.  a[ <subscript_resulting_in_rank0> ].asScalar()  
> 
> Any strong opinions?
> 

Since subscripting a rank-3 array:

  a[1, 2, 3] 

is very much like 

  a[(1, 2, 3)]

The __getitem__ receives the tuple (1, 2, 3) in both cases.  (Try it!)

So that would imply subscripting (dereferencing) a rank-0 array could be:

  a[] could be represented by a[()]

It's just unfortunate that Python doesn't currently recognize the a[] as a
valid syntax.




Cheers,
    -Scott






__________________________________________________
Do you Yahoo!?
Yahoo! News - Today's headlines
http://news.yahoo.com



From hinsen at cnrs-orleans.fr  Fri Sep 13 12:22:03 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 12:22:03 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <20020913171649.36428.qmail@web40106.mail.yahoo.com>
References: <20020913171649.36428.qmail@web40106.mail.yahoo.com>
Message-ID: <m33csdg0g8.fsf@localhost.localdomain>

Scott Gilbert <xscottg at yahoo.com> writes:

> > a.  Return the scalar value                         (what numarray does 
> > now.  seems inconsistent)
> 
> If you think of it as "dereferencing" instead of "indexing", I think this
> is most consistent.  It takes 3 elements to dereference a rank-3 array, so
> it should take 0 elements to dereference a rank-0 array.

That would be fine with me, but empty index brackets raise a
SyntaxError in Python.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From oliphant at ee.byu.edu  Fri Sep 13 12:59:04 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri Sep 13 12:59:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCCEEMCOAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.33L2.0209131354330.21399-100000@oliphant.ee.byu.edu>

>
> To restate the issue: there was a question about whether
> an index to an array that identified a single element only
> (i.e., not a slice, nor an incomplete index, e.g. x[3] where
> x is two dimensional) should return a Python scalar or a
> rank-0 array. Currently Numeric is inconsistent on this point.
> One usually gets scalars, but on some occasions, rank-0 arrays
> are returned. Good arguments are to be had for either alternative.
>

I think we should implement Python scalars for the other types and then
eliminate rank-0 arrays completely.  I could have a student do this in a
few weeks if it was agreed on.

-Travis Oliphant






From oliphant at ee.byu.edu  Fri Sep 13 13:00:07 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri Sep 13 13:00:07 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
Message-ID: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>

> Two other issues come up trying to implement the "rank-0 experiment":
>
> 1.  What should be the behavior of subscripting a rank-0 array?
>
> a.  Return the scalar value                         (what numarray does
> now.  seems inconsistent)
+0.25

> b.  Raise an exception
+1

> c.  Return a copy  of the rank-0 array
-1

>
> 2.  What's a decent notation for .asScalar()?
>
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray
> does now)
-1

Numeric does this now for backward compatibility.  I don't think this
issue was discussed at length.


> b.  a[ <subscript_resulting_in_rank0> ]()                  (override
> __call__)
-1

> c.  a[ <subscript_resulting_in_rank0> ].asScalar()
+1  just keep the current .toscalar()  notation.

Also int(a) and float(a) and complex(a) would work as well.

If we just implemented Python scalars for the other precisions then much
of this rank 0 array business would be solved as we could always return
the Python scalar rather than an array for rank 0 arrays.

I believe Konrad made this suggestion many moons ago and I would agree
with him.

-Travis Oliphant





From perry at stsci.edu  Fri Sep 13 13:15:09 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 13:15:09 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>
Message-ID: <JFEGLNDJEDNOMPPHDEJFEEFIEAAA.perry@stsci.edu>

> If we just implemented Python scalars for the other precisions then much
> of this rank 0 array business would be solved as we could always return
> the Python scalar rather than an array for rank 0 arrays.
> 
> I believe Konrad made this suggestion many moons ago and I would agree
> with him.
> 
> -Travis Oliphant
> 
Hi Travis,

refresh my memory about how this proposal would work. I've heard
proposals to add new types to Python itself, but that seems out of
the question. Are you talking about adding new scalar types as 
module? E.g.

>>> x = Int8(22) 
>>> arr = arange(10, typecode = Int8)
>>> arr[2]
Int8(2)

Or some other approach? 

In any case, doesn't this still have the problem that Eric complained
about, having to test for whether a result is an array or a scalar
(which was one of the drivers to produce rank-0 results).

Thanks, Perry



From tim.hochberg at ieee.org  Fri Sep 13 13:21:03 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 13 13:21:03 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <JFEGLNDJEDNOMPPHDEJFEEFIEAAA.perry@stsci.edu>
Message-ID: <137601c25b62$ba00c350$6c0b6244@cx781526b>



[Perry wrote about Travis's proposal]

> refresh my memory about how this proposal would work. I've heard
> proposals to add new types to Python itself, but that seems out of
> the question. Are you talking about adding new scalar types as
> module? E.g.
[snip]
> In any case, doesn't this still have the problem that Eric complained
> about, having to test for whether a result is an array or a scalar
> (which was one of the drivers to produce rank-0 results).

Hi Perry,

I like Travis's proposal the best of those I've seen so far, but I don't
recall the details of Eric's problem. Could you refresh us as to the basics
of it?.

-tim





From hinsen at cnrs-orleans.fr  Fri Sep 13 13:34:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 13:34:02 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>
References: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>
Message-ID: <m3wuppeikt.fsf@localhost.localdomain>

Travis Oliphant <oliphant at ee.byu.edu> writes:

> If we just implemented Python scalars for the other precisions then much
> of this rank 0 array business would be solved as we could always return
> the Python scalar rather than an array for rank 0 arrays.
> 
> I believe Konrad made this suggestion many moons ago and I would agree

Right. I still think this is the best solution in that it is the least
confusing. If I remember correctly, the main objection was that
implementing all those types was not as trivial as hoped, but I forgot
the details (and it wasn't me who tried).

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From eric at enthought.com  Fri Sep 13 13:45:01 2002
From: eric at enthought.com (eric jones)
Date: Fri Sep 13 13:45:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <137601c25b62$ba00c350$6c0b6244@cx781526b>
Message-ID: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>

Hey Tim,

Here is a short summary:

Reductions and indexing return different types based on the number of
dimensions of the input array:

>>> b = sum(a)
>>> l = len(b) # or whatever

This code works happily if "a" is 2 or more dimensions, but will fail if
it is 1d because the sum(a) will return a scalar in this case.  To write
generic code, you have to put an if/else statement in to check whether b
is a scalar or an array:

>>> b = sum(a)
>>> if type(b) is ArrayType:
...     l = len(b)
... else:
...     l = 1 # or whatever you need

or less verbose but still unpleasant:  

>>> b = asarray(sum(a))
>>> l = len(b)

eric

> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-
> discussion-admin at lists.sourceforge.net] On Behalf Of Tim Hochberg
> Sent: Friday, September 13, 2002 3:18 PM
> To: numpy-discussion at lists.sourceforge.net
> Subject: Re: [Numpy-discussion] rank-0 arrays
> 
> 
> 
> 
> [Perry wrote about Travis's proposal]
> 
> > refresh my memory about how this proposal would work. I've heard
> > proposals to add new types to Python itself, but that seems out of
> > the question. Are you talking about adding new scalar types as
> > module? E.g.
> [snip]
> > In any case, doesn't this still have the problem that Eric
complained
> > about, having to test for whether a result is an array or a scalar
> > (which was one of the drivers to produce rank-0 results).
> 
> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I
don't
> recall the details of Eric's problem. Could you refresh us as to the
> basics
> of it?.
> 
> -tim
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion




From perry at stsci.edu  Fri Sep 13 13:46:08 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 13:46:08 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <137601c25b62$ba00c350$6c0b6244@cx781526b>
Message-ID: <JFEGLNDJEDNOMPPHDEJFOEFKEAAA.perry@stsci.edu>

> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I don't
> recall the details of Eric's problem. Could you refresh us as to 
> the basics
> of it?.
> 
> -tim
> 
Rather than put keystrokes on Eric's fingertips :-) I think it would
be best if he (or Paul Dubois, who had similar concerns) gave
some examples. But I believe it concerned fairly generic
Numeric routines that had expressions that might or might not
result in scalars. Testing for both cases resulted in 
extra code (Paul said that MA had to deal with this a lot) and
that having to deal with this made the code error prone.

But I'd rather they explained it since they have very specific
experience with this issue. Perhaps they can give a couple
common examples.

Perry



From paul at pfdubois.com  Fri Sep 13 14:07:01 2002
From: paul at pfdubois.com (Paul F Dubois)
Date: Fri Sep 13 14:07:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
Message-ID: <001a01c25b69$5aeadbb0$6901a8c0@NICKLEBY>

This is a perfect example of why the thing is so annoying. 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net 
> [mailto:numpy-discussion-admin at lists.sourceforge.net] On 
> Behalf Of eric jones
> Sent: Friday, September 13, 2002 1:44 PM
> To: 'Tim Hochberg'; numpy-discussion at lists.sourceforge.net
> Subject: RE: [Numpy-discussion] rank-0 arrays
> 
> 
> Hey Tim,
> 
> Here is a short summary:
> 
> Reductions and indexing return different types based on the 
> number of dimensions of the input array:
> 
> >>> b = sum(a)
> >>> l = len(b) # or whatever
> 
> This code works happily if "a" is 2 or more dimensions, but 
> will fail if it is 1d because the sum(a) will return a scalar 
> in this case.  To write generic code, you have to put an 
> if/else statement in to check whether b is a scalar or an array:
> 
> >>> b = sum(a)
> >>> if type(b) is ArrayType:
> ...     l = len(b)
> ... else:
> ...     l = 1 # or whatever you need
> 
> or less verbose but still unpleasant:  
> 
> >>> b = asarray(sum(a))
> >>> l = len(b)
> 
> eric
> 
> > -----Original Message-----
> > From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy- 
> > discussion-admin at lists.sourceforge.net] On Behalf Of Tim Hochberg
> > Sent: Friday, September 13, 2002 3:18 PM
> > To: numpy-discussion at lists.sourceforge.net
> > Subject: Re: [Numpy-discussion] rank-0 arrays
> > 
> > 
> > 
> > 
> > [Perry wrote about Travis's proposal]
> > 
> > > refresh my memory about how this proposal would work. I've heard 
> > > proposals to add new types to Python itself, but that 
> seems out of 
> > > the question. Are you talking about adding new scalar types as 
> > > module? E.g.
> > [snip]
> > > In any case, doesn't this still have the problem that Eric
> complained
> > > about, having to test for whether a result is an array or 
> a scalar 
> > > (which was one of the drivers to produce rank-0 results).
> > 
> > Hi Perry,
> > 
> > I like Travis's proposal the best of those I've seen so far, but I
> don't
> > recall the details of Eric's problem. Could you refresh us 
> as to the 
> > basics of it?.
> > 
> > -tim
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf 
> > _______________________________________________
> > Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 




From perry at stsci.edu  Fri Sep 13 14:07:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 14:07:02 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <137601c25b62$ba00c350$6c0b6244@cx781526b>
Message-ID: <JFEGLNDJEDNOMPPHDEJFGEFMEAAA.perry@stsci.edu>

> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I don't
> recall the details of Eric's problem. Could you refresh us as to 
> the basics
> of it?.
> 
> -tim
> 
On a different front, I'm not sure that this proposal is very useful
for numarray because of the new scalar/array coercion rules that
numarray has. Since the new scalar literals must still use 
type functions (e.g.,, Int8(2)) it doesn't help make expressions
any prettier (or do I misunderstand the proposal?). In returning
scalars from arrays, I don't see that converting the array type
to a Python scalar type helps much. With the exception of long
doubles (and the corresponding complex precision), the conversion
to the python scalar doesn't lose any precision and you can 
restore the exact array type value by assigning that scalar value
back to an array element. As mentioned, the only exception is for
long doubles; for that case, a new scalar type makes sense.

Is there any other compelling reason for new scalar types?

Perry



From tim.hochberg at ieee.org  Fri Sep 13 14:25:14 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 13 14:25:14 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
Message-ID: <13a901c25b6b$9cd1ab10$6c0b6244@cx781526b>

Thanks for the summary Eric.

However, I don't find this example particularly compelling. Probably because
I don't think len(b) should work if b is rank zero (or possibly return
None). Yeah, I know it's worked in NumPy for years, but that doesn't mean I
have to like it. I would favor stripping out as much of the weird special
casing of rank-0 arrays as possible in the transition to numarray.

In the particular example given below, isn't the return value meaningless if
b is rank-0. Or rather only meaningful if  "or whatever you need" happens to
be one. This all smells rather arbitrary to me. However, this isn't a
problem that I run into and I'll concede that it's possible that returning
rank[0] arrays and then treating the rank zeros arrays almost like shape
(1,) arrays most of the time may solve more problems than it causes, but I'd
be interested in seeing more realistic examples. I guess I'll go poke around
MA and see what I can see. Any other suggestions of what to look at?

-tim



> Hey Tim,
>
> Here is a short summary:
>
> Reductions and indexing return different types based on the number of
> dimensions of the input array:
>
> >>> b = sum(a)
> >>> l = len(b) # or whatever
>
> This code works happily if "a" is 2 or more dimensions, but will fail if
> it is 1d because the sum(a) will return a scalar in this case.  To write
> generic code, you have to put an if/else statement in to check whether b
> is a scalar or an array:
>
> >>> b = sum(a)
> >>> if type(b) is ArrayType:
> ...     l = len(b)
> ... else:
> ...     l = 1 # or whatever you need
>
> or less verbose but still unpleasant:
>
> >>> b = asarray(sum(a))
> >>> l = len(b)
>
> eric
>
> > -----Original Message-----
> > From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-
> > discussion-admin at lists.sourceforge.net] On Behalf Of Tim Hochberg
> > Sent: Friday, September 13, 2002 3:18 PM
> > To: numpy-discussion at lists.sourceforge.net
> > Subject: Re: [Numpy-discussion] rank-0 arrays
> >
> >
> >
> >
> > [Perry wrote about Travis's proposal]
> >
> > > refresh my memory about how this proposal would work. I've heard
> > > proposals to add new types to Python itself, but that seems out of
> > > the question. Are you talking about adding new scalar types as
> > > module? E.g.
> > [snip]
> > > In any case, doesn't this still have the problem that Eric
> complained
> > > about, having to test for whether a result is an array or a scalar
> > > (which was one of the drivers to produce rank-0 results).
> >
> > Hi Perry,
> >
> > I like Travis's proposal the best of those I've seen so far, but I
> don't
> > recall the details of Eric's problem. Could you refresh us as to the
> > basics
> > of it?.
> >
> > -tim
> >
> >
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>





From oliphant at ee.byu.edu  Fri Sep 13 15:53:03 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri Sep 13 15:53:03 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFEEFIEAAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu>

> Hi Travis,
>
> refresh my memory about how this proposal would work. I've heard
> proposals to add new types to Python itself, but that seems out of
> the question. Are you talking about adding new scalar types as
> module? E.g.
>
> >>> x = Int8(22)
> >>> arr = arange(10, typecode = Int8)
> >>> arr[2]
> Int8(2)
>
> Or some other approach?

This is the gist of it.  Basically you extend the Python scalars to
include the single precision types (all the types Numeric supports).

Ultimately, in Python it would be nice if all of the scalars had the same
base class.

>
> In any case, doesn't this still have the problem that Eric complained
> about, having to test for whether a result is an array or a scalar
> (which was one of the drivers to produce rank-0 results).

I would have to understand his reason better.  He could be right.  My
reason for rank-0 results as always been more of a type issue.  I'll have
to ask him.

Now I remmber an issue that makes me question the proposal:

Currently the length of a rank-0 array is 1, while the length of a scalar
raises an error --- this is a bad difference.

We could still implement rank-0 arrays as a separate (optimized) type
though (so it doesn't carry around the extra baggage of full-rank arrays).

Now-wanting-rank-0-arrays-all-the-time,

-Travis





From huaiyu_zhu at yahoo.com  Sat Sep 14 00:25:02 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Sat Sep 14 00:25:02 2002
Subject: [Numpy-discussion] Tensor-like product with extra indices?
In-Reply-To: <m3admmchab.fsf@chinon.cnrs-orleans.fr>
Message-ID: <Pine.LNX.4.33.0209140020060.1213-100000@mars.localdomain>

On 13 Sep 2002, Konrad Hinsen wrote:
> 
> Untested, but the idea should work:
> 
>   def ravel_index(array, *indices):
>      offsets = Numeric.multiply.accumulate(array.shape[1:][::-1])[::-1]
>      return Numeric.sum(Numeric.array(indices[:-1])*offsets) + indices[-1]

Aagh!  :-)

> 
> > 2. Suppose the above function is called ravel_index and its
> >    inverse unravel_index.
> > 
> >    Given a list of matrices [A0, A1, ...] is there a function that
> >    calculates a matrix B such that
> > 
> >         B[i,j]  = A0[i0,j] * A1[i1,j] * ...
> > 
> >    where i = ravel_index(i0, i1, ...).
> 
> Not for all i, which I assume is what you want to do.

I end up using two for-loops, one over j and the other over [A0, ...].  
Hoping the speed problem will not be too bad later on.

Huaiyu

-- 
Huaiyu Zhu <huaiyu_zhu at yahoo.com>




From hinsen at cnrs-orleans.fr  Sun Sep 15 03:13:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Sun Sep 15 03:13:02 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
References: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
Message-ID: <m3d6rfmuhy.fsf@localhost.localdomain>

"eric jones" <eric at enthought.com> writes:

> Reductions and indexing return different types based on the number of
> dimensions of the input array:
> 
> >>> b = sum(a)
> >>> l = len(b) # or whatever
> 
> This code works happily if "a" is 2 or more dimensions, but will fail if
> it is 1d because the sum(a) will return a scalar in this case.  To write

And it should fail, because a rank-0 array is not a sequence, so it
doesn't have a length.

But there are valid examples in which it would be nice if scalars
were arrays (but probably if  *all* scalars supported array operations,
not just those that were generated by indexing from arrays):

- a.shape  should return () for a scalar (and (len(a),) for any
           sequence type)

- a.astype(N.Float) should also work for scalars

Similarly, it would be nice if complex operations (real/imaginary
part) would work on integers and floats.

There's one more annoying difference between scalars and arrays of
any rank which I think should be removed in numarray:

  >>> 3 % -2
  -1
  >>> array(3) % 2
  1
  >>> fmod(3, -2)
  1.0

I.e. the mod operation uses fmod() for arrays, but different rules
for standard Python numbers.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Sun Sep 15 03:17:01 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Sun Sep 15 03:17:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFGEFMEAAA.perry@stsci.edu>
References: <JFEGLNDJEDNOMPPHDEJFGEFMEAAA.perry@stsci.edu>
Message-ID: <m38z23mubg.fsf@localhost.localdomain>

"Perry Greenfield" <perry at stsci.edu> writes:

> Is there any other compelling reason for new scalar types?

It solves the problem we are discussing (extracting array elements)
without introducing any incompatibility with existing code that uses
only the standard Python data types, and very little for code that
uses Float32 etc. I'd say it's the best compromise between
consistency and backwards compatibility.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From eric at enthought.com  Sun Sep 15 11:51:01 2002
From: eric at enthought.com (eric jones)
Date: Sun Sep 15 11:51:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <m3d6rfmuhy.fsf@localhost.localdomain>
Message-ID: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>

Hey Konrad,

> 
> "eric jones" <eric at enthought.com> writes:
> 
> > Reductions and indexing return different types based on the number
of
> > dimensions of the input array:
> >
> > >>> b = sum(a)
> > >>> l = len(b) # or whatever
> >
> > This code works happily if "a" is 2 or more dimensions, but will
fail if
> > it is 1d because the sum(a) will return a scalar in this case.  To
write
> 
> And it should fail, because a rank-0 array is not a sequence, so it
> doesn't have a length.
> 

I disagree. You should not have to write special code to check for a
specific case.  It breaks one of the beauties of Numeric -- i.e. you can
write generic code that handles arrays of any size and type.  Any method
that works on a 1 or more d array should also work on 0d arrays.  If you
ask for its shape, it returns a tuple.  If you ask for its size it
returns its length along its "first" axis.  This will always be 1.  It
allows for generic code.

On this note:
I do not see the benefit of making a scalar type object that is separate
for 0d arrays.  It seems to remove instead of enhance capabilities.
What does a scalar object buy that simply using 0d arrays for that
purpose does not?

> 
> But there are valid examples in which it would be nice if scalars
> were arrays (but probably if  *all* scalars supported array
operations,
> not just those that were generated by indexing from arrays):
> 
> - a.shape  should return () for a scalar (and (len(a),) for any
>            sequence type)
> 
> - a.astype(N.Float) should also work for scalars
> 
> Similarly, it would be nice if complex operations (real/imaginary
> part) would work on integers and floats.

Yes, this is needed.  And I think the argument for it is similar as
having len() work on 0d arrays.  It allows for generic code.

> 
> There's one more annoying difference between scalars and arrays of
> any rank which I think should be removed in numarray:
> 
>   >>> 3 % -2
>   -1
>   >>> array(3) % 2
>   1
>   >>> fmod(3, -2)
>   1.0
> 
> I.e. the mod operation uses fmod() for arrays, but different rules
> for standard Python numbers.

I think you meant, 

>   >>> array(3) % -2
>   1

That is unfortunate.  It would be nice to clean this up.

eric




From hinsen at cnrs-orleans.fr  Sun Sep 15 12:41:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Sun Sep 15 12:41:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>
References: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>
Message-ID: <200209151940.g8FJejs03609@localhost.localdomain>

> > And it should fail, because a rank-0 array is not a sequence, so it
> > doesn't have a length.
> 
> I disagree. You should not have to write special code to check for a
> specific case.  It breaks one of the beauties of Numeric -- i.e. you can

It is not a specific case, more like a specific value (for the rank).
1/a fails for a == 0, should that be changed as well?

Let's examine some equivalent code pieces:

- len(a) == a.shape[0]
  the second fails for rank 0, so the first one should fail as well

- for i in range(len(a)):  print a[i]
  works for all sequences. If len(a) doesn't fail (and I assume it would
  then return 1), a[1] shouldn't fail either.

- len(a) == len(list(a))
  for all sequences a. Should list(a) return [a] for a rank-0 array?
  For a scalar it fails.

Actually this might be an argument for not having rank-0 arrays at all.
Arrays are multidimensional sequences, but rank-0 arrays aren't.

> returns its length along its "first" axis.  This will always be 1.  It
> allows for generic code.

Then please give an example where this genericity would be useful.

> On this note:
> I do not see the benefit of making a scalar type object that is separate
> for 0d arrays.  It seems to remove instead of enhance capabilities.
> What does a scalar object buy that simply using 0d arrays for that
> purpose does not?

Compatibility, for example the ability to index a sequence with an
element of an integer array. Also consistency with other Python
sequence types. For example,

   [a][0] == a

so one would expect also

   array([a])[0] == a

but this would not be fully true if the left-hand side is a rank-0
array.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From tim.hochberg at ieee.org  Sun Sep 15 14:30:04 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun Sep 15 14:30:04 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>
Message-ID: <145001c25cfe$b95db550$6c0b6244@cx781526b>

----- Original Message -----
From: "eric jones" <eric at enthought.com>

[Konrad]
> > And it should fail, because a rank-0 array is not a sequence, so it
> > doesn't have a length.

[Eric]
> I disagree. You should not have to write special code to check for a
> specific case.  It breaks one of the beauties of Numeric -- i.e. you can
> write generic code that handles arrays of any size and type.  Any method
> that works on a 1 or more d array should also work on 0d arrays.  If you
> ask for its shape, it returns a tuple.

No problem up to here.

> If you ask for its size it
> returns its length along its "first" axis.

Here's where we part ways. As Konrad already pointed out, it return
anArray.shape[0] except in the case of zero-D arrays where the arbitrary
decision was made that:

> This will always be 1.

Why 1 and not 0 or -1 or 42. If you really had to return something, the
thing to return would be None.

> It allows for generic code.

I don't see how. Even poking around in MA didn't convince me that this would
help although I didn't spend enough time with it to get a great feel for it.
The one function I did come close to working all the way through looked like
it would be about _half_ as long if it didn't have to support the zero-rank
cruft.

> On this note:
> I do not see the benefit of making a scalar type object that is separate
> for 0d arrays.  It seems to remove instead of enhance capabilities.
> What does a scalar object buy that simply using 0d arrays for that
> purpose does not?

Unless somethings changed, you can't index into lists and what not with a
rank-0 array, so returning ints or some subclass from integer arrays would
be convenient. It would be easy to always return subclasses of int, float or
complex (or object for those few who use object arrays) so that the results
would always play nice with the rest of Python.

However, given that the coercion rules have changed in numarray, I don't
really see the point of returning anything other than int, float of complex.
However, I have no objection to allowing the creation rank-0 arrays as long
as they behave consistently with other array objects.

[Konrad]
> > - a.astype(N.Float) should also work for scalars
> >
> > Similarly, it would be nice if complex operations (real/imaginary
> > part) would work on integers and floats.

If these are needed, and I agree they would be nice, it seems that in order
to integrate well with the rest of Python we should supply:

numarray.astype(a, type) -> array or scalar of type type.
numarray.imaginary(a) -> imaginary part of a
numarray.imaginary(a) -> real part of a

I actually had these latter two in JNumeric back when I was working on that,
so I kinda thought they were Numeric, but I musta just added them in because
I liked them.

[Eric and Konrad agree that the inconsistency between Python and Numeric's
mod should be cleaned up]

Me too.

-tim





From Chris.Barker at noaa.gov  Mon Sep 16 11:46:05 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Mon Sep 16 11:46:05 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu>
Message-ID: <3D862463.BB6EED2C@noaa.gov>


Travis Oliphant wrote:

> This is the gist of it.  Basically you extend the Python scalars to
> include the single precision types (all the types Numeric supports).

Would they be recognised as scalars by Python? In particular, could you
use one as an index? Personally, this is what has bit me in the past: I
could use A[3,2] as an index if A was type "Int" but not if it was
"Int16" for example.

In any case, the type of A[3,2] should NOT depend on the precision of
the numbers stored in A.

Frankly, I have no idea what the implimentation details would be, but
could we get rid of rank-0 arrays altogether? I have always simply found
them strange and confusing... What are they really neccesary for
(besides holding scalar values of different precision that standard
Pyton scalars)?

-Chris







-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From hinsen at cnrs-orleans.fr  Mon Sep 16 12:18:14 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Sep 16 12:18:14 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D862463.BB6EED2C@noaa.gov>
References: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu>
	<3D862463.BB6EED2C@noaa.gov>
Message-ID: <m3admhkalh.fsf@localhost.localdomain>

"Chris Barker" <Chris.Barker at noaa.gov> writes:

> Travis Oliphant wrote:
> 
> > This is the gist of it.  Basically you extend the Python scalars to
> > include the single precision types (all the types Numeric supports).
> 
> Would they be recognised as scalars by Python? In particular, could you

There is no "scalar" category in Python. New scalar datatypes would be
types with the same behaviour as the existing Python scalar types, but
different from the. That means that explicitly type-checking code would
not accept them, but everything else would.

> use one as an index? Personally, this is what has bit me in the past: I

At least up to Python 1.5, no, indices have to be of integer type. I
don't know if that condition was extended in later versions.

> could use A[3,2] as an index if A was type "Int" but not if it was
> "Int16" for example.

Ehmmm... Are you sure that is the right example? The restriction is on
the type of the index, not on the type of the array.

> Frankly, I have no idea what the implimentation details would be, but
> could we get rid of rank-0 arrays altogether? I have always simply found

If we introduce additional scalars, yes. Whether or not we would want
to get rid of them is of course another question.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From tim.hochberg at ieee.org  Mon Sep 16 12:39:01 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 16 12:39:01 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu><3D862463.BB6EED2C@noaa.gov> <m3admhkalh.fsf@localhost.localdomain>
Message-ID: <156101c25db8$40872700$6c0b6244@cx781526b>

From: "Konrad Hinsen" <hinsen at cnrs-orleans.fr>
> > Travis Oliphant wrote:
> > Would they be recognised as scalars by Python? In particular, could you
>
> There is no "scalar" category in Python. New scalar datatypes would be
> types with the same behaviour as the existing Python scalar types, but
> different from the. That means that explicitly type-checking code would
> not accept them, but everything else would.
>
> > use one as an index? Personally, this is what has bit me in the past: I
>
> At least up to Python 1.5, no, indices have to be of integer type. I
> don't know if that condition was extended in later versions.

In python 2.2 (I'm not sure about 2.0,2.1), you can subclass int and the
resultant class can be used as an index. So I think this could be done.

> > Frankly, I have no idea what the implimentation details would be, but
> > could we get rid of rank-0 arrays altogether? I have always simply found
>
> If we introduce additional scalars, yes.

Given that numarray has changed its coercion rules, is this still necessary?
>From http://stsdas.stsci.edu/numarray/Userguide.html:

----------start quote----------
*************
Type Coercion
*************

In expressions involving only arrays, the normal coercion rules
apply (i.e., the same as Numeric). However, the same rules do
not apply to binary operations between arrays and Python scalars
in certain cases. If the kind of number is the same for the array
and scalar (e.g., both are integer types or both are float types),
then the type of the output is determined by the precision of the
array, not the scalar. Some examples will best illustrate:

Scalar type  *  Array type        Numeric result type    numarray result
type
   Int            Int16                Int32                    Int16
   Int            Int8                 Int32                    Int8
   Float          Int8                 Float64                  Float64
   Float          Float32              Float64                  Float32

The change in the rules was made so that it would be easy to preserve the
precision of arrays in expressions involving scalars. Previous solutions
with Numeric were either quite awkward (using a function to create a rank-0
array from a scalar with the desired type) or surprising (the savespace
attribute, that never allowed type coercion). The problem arises because
Python has a limited selection of scalar types. This appears to be the
best solution though it admittedy may surprise some who are used to the
classical type coercion model.

----------end quote----------


> Whether or not we would want
> to get rid of them is of course another question.

Agreed. It's possible that rank zero arrays would be useful in certain
unusual situations, although I can't think of any good examples. Regardless
of that choice, if indexing never returns rank-0 arrays most people will
never have to deal with them.

-tim






From tim.hochberg at ieee.org  Mon Sep 16 13:15:01 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 16 13:15:01 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
Message-ID: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>

Can someone refresh my memory as to why some properties of NumArrays that
are conceptually attributes are accessed as atributes (a.shape, a.rank),
while others are accessed through functions (a.type, a.iscontiguous,...).

Thanks,

-tim





From paul at pfdubois.com  Mon Sep 16 13:27:04 2002
From: paul at pfdubois.com (Paul F Dubois)
Date: Mon Sep 16 13:27:04 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
In-Reply-To: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>
Message-ID: <000001c25dbf$5f0dc5b0$6901a8c0@NICKLEBY>

Because that is what Numeric did, and as explained in the position
statement on the numpy website, we are not changing things unless we
have a compelling reason. 

Numeric did some things like a.shape as properties rather than pairs of
get/set functions; there was no special reason for it except a feeling
that it was easier to type or looked more mathematical. E.g.,
x.imaginary. I think the driving reason for shape was interactive use:
x.shape=(4,5)

However we got here, we aren't going to debate changing any of this. The
scalar issue is on the table because it causes actual trouble for real
people rather than philosophical inconsistency. 
> 
> 
> Can someone refresh my memory as to why some properties of 
> NumArrays that are conceptually attributes are accessed as 
> atributes (a.shape, a.rank), while others are accessed 
> through functions (a.type, a.iscontiguous,...).
> 
> Thanks,
> 
> -tim




From tim.hochberg at ieee.org  Mon Sep 16 13:43:04 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 16 13:43:04 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
References: <000001c25dbf$5f0dc5b0$6901a8c0@NICKLEBY>
Message-ID: <159301c25dc1$4c4ab170$6c0b6244@cx781526b>


[Me (Tim Hochberg)]

> > Can someone refresh my memory as to why some properties of
> > NumArrays that are conceptually attributes are accessed as
> > atributes (a.shape, a.rank), while others are accessed
> > through functions (a.type, a.iscontiguous,...).

[Paul F Dubois]

> Because that is what Numeric did, and as explained in the position
> statement on the numpy website, we are not changing things unless we
> have a compelling reason.

Fair enough, but what about attributes new to numarray (rank, type, maybe
others). Is the preference then for function syntax over attribute syntax?
I'd prefer the latter, but I'm mostly trying to figure out what the pattern
is.

-tim





From perry at stsci.edu  Mon Sep 16 14:48:06 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Sep 16 14:48:06 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <m3admhkalh.fsf@localhost.localdomain>
Message-ID: <JFEGLNDJEDNOMPPHDEJFMEGHEAAA.perry@stsci.edu>

Konrad Hinsen writes:
> "Chris Barker" <Chris.Barker at noaa.gov> writes:
> 
> > use one as an index? Personally, this is what has bit me in the past: I
> 
> At least up to Python 1.5, no, indices have to be of integer type. I
> don't know if that condition was extended in later versions.
> 
> > could use A[3,2] as an index if A was type "Int" but not if it was
> > "Int16" for example.
> 
> Ehmmm... Are you sure that is the right example? The restriction is on
> the type of the index, not on the type of the array.
> 
I think Chris is referring to the fact that Numeric returns a rank-0
array for A[3,2] if A is of type Int16, and that value cannot be used
as in index in Python sequences (at least for now; nothing technical
prevents it from being implemented to accept object that have an
__int__ method). This is the odd inconsistency Numeric has now. If
A were a 1-d Int16 array then A[2] would not be a rank-0 array,
nor is A[3,2] a rank-0 array if A is of type Int32 (apparently because
a Python scalar type suffices). Why it gives rank-0 for 2-d and not
1-d I have no idea.
> 



From oliphant at ee.byu.edu  Mon Sep 16 15:51:04 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Mon Sep 16 15:51:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D862463.BB6EED2C@noaa.gov>
Message-ID: <Pine.LNX.4.33L2.0209161643190.20912-100000@oliphant.ee.byu.edu>

>
> Travis Oliphant wrote:
>
> > This is the gist of it.  Basically you extend the Python scalars to
> > include the single precision types (all the types Numeric supports).
>
> Would they be recognised as scalars by Python? In particular, could you
> use one as an index? Personally, this is what has bit me in the past: I
> could use A[3,2] as an index if A was type "Int" but not if it was
> "Int16" for example.
>
> In any case, the type of A[3,2] should NOT depend on the precision of
> the numbers stored in A.
>
> Frankly, I have no idea what the implimentation details would be, but
> could we get rid of rank-0 arrays altogether? I have always simply found
> them strange and confusing... What are they really neccesary for
> (besides holding scalar values of different precision that standard
> Pyton scalars)?

With new coercion rules this becomes a possibility.  Arguments against it
are that special rank-0 arrays behave as more consistent numbers with the
rest of Numeric than Python scalars.  In other words they have a length
and a shape and one can right N-dimensional code that works the same even
when the result is a scalar.

Another advantage of having a Numeric scalar is that we can control the
behavior of floating point operations better.

e.g.

if only Python scalars were available and sum(a) returned 0, then

 1 / sum(a)  would behave as Python behaves (always raises error).

while with our own scalars

1 / sum(a)   could potentially behave however the user wanted.


-Travis







From pearu at cens.ioc.ee  Mon Sep 16 16:10:04 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Mon Sep 16 16:10:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.33L2.0209161643190.20912-100000@oliphant.ee.byu.edu>
Message-ID: <Pine.LNX.4.21.0209170202150.1695-100000@cens.kybi>

On Mon, 16 Sep 2002, Travis Oliphant wrote:

> > Frankly, I have no idea what the implimentation details would be, but
> > could we get rid of rank-0 arrays altogether? I have always simply found
> > them strange and confusing... What are they really neccesary for
> > (besides holding scalar values of different precision that standard
> > Pyton scalars)?
> 
> With new coercion rules this becomes a possibility.  Arguments against it
> are that special rank-0 arrays behave as more consistent numbers with the
> rest of Numeric than Python scalars.  In other words they have a length
> and a shape and one can right N-dimensional code that works the same even
> when the result is a scalar.

In addition, rank-0 arrays are mutable while Python scalars are not.
Mutability is sometimes useful (e.g. when emulating C or Fortran
calls in Python) but often also evil due to its unpythonic side effect.

Pearu




From huaiyu_zhu at yahoo.com  Mon Sep 16 23:40:05 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Mon Sep 16 23:40:05 2002
Subject: [Numpy-discussion] numarray bug: sum, product, alltrue, sometrue
Message-ID: <Pine.LNX.4.33.0209162338270.1534-100000@mars.localdomain>

There is a bug in the dimension-reducing functions (sum, product,
alltrue, sometrue) in numarray: The outside swapaxes should take into
account that one axis is reduced.  The expected behavior should be:

>>> from numarray import *
>>> a = ones((2,3,4))
>>> sum(a, axis=0).shape
(3, 4)
>>> sum(a, axis=1).shape
(2, 4)
>>> sum(a, axis=2).shape
(2, 3)

A patch is attached below.

Huaiyu

*** numarray.py~	Tue Jul  2 11:12:24 2002
--- numarray.py	Mon Sep 16 23:17:27 2002
***************
*** 995,997 ****
      else:
!         return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis)
  
--- 995,997 ----
      else:
!         return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis-1)
  
***************
*** 1014,1016 ****
          return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)),
!                         0, axis)
  
--- 1014,1016 ----
          return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)),
!                         0, axis-1)
  
***************
*** 1034,1036 ****
          return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)),
!                         0, axis)
  def sometrue(a, axis=0):
--- 1034,1036 ----
          return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)),
!                         0, axis-1)
  def sometrue(a, axis=0):
***************
*** 1043,1045 ****
          return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)),
!                         0, axis)
  
--- 1043,1045 ----
          return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)),
!                         0, axis-1)
  





From huaiyu_zhu at yahoo.com  Tue Sep 17 00:05:01 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Tue Sep 17 00:05:01 2002
Subject: [Numpy-discussion] s are mutable scalars
Message-ID: <Pine.LNX.4.33.0209162358310.1534-100000@mars.localdomain>

Some of the choices between rank-0 arrays and new scalar types
might be resolved by enumerating the properties desired of them.

Most properties of rank-0 arrays could be fixed by consistency
requirements alone, using operations that reduce array dimensions.

Let a = ones((2,3,4))
b = sum(a)
c = sum(b)
d = sum(c)

Property 1: the shape of an array is a tuple of integers
	a.shape == (2, 3, 4)
	b.shape == (3, 4)
	c.shape == (4,)
	d.shape == ()
	
Property 2: rank(a) == len(a.shape)
	rank(a) == 3 == len(a.shape)
	rank(b) == 2 == len(b.shape)
	rank(c) == 1 == len(c.shape)
	rank(d) == 0 == len(d.shape)

Property 3: len(a) == a.shape[0]
	len(a) == 2 == a.shape[0]
	len(b) == 3 == b.shape[0]
	len(c) == 4 == c.shape[0]
	len(d) == Exception == d.shape[0]

	# Currently the last is wrong?

Property 4: size(a) == product(a.shape)
	size(a) == 24 == product(a.shape)
	size(b) == 12 == product(b.shape)
	size(c) == 4 == product(c.shape)
	size(d) == 1 == product(d.shape)

	# Currently the last is wrong

Property 5: rank-0 array behaves as mutable numbers when used as value
	array(2)  is similar to 2
	array(2.0)  is similar to 2.0
	array(2j)  is similar to 2j

	# This is a summary of many concrete properties.

Property 6: Indexing reduces rank. Slicing preserves rank.
	a[:,:,:].shape = (2, 3, 4)
	a[1,:,:].shape = (3, 4)
	a[1,1,:].shape = (4,)
	a[1,1,1].shape = ()

Property 7: Indexing by tuple of ints gives scalar.
	a[1,1,1] == 1
	b[1,1] == 2
	c[1,] == 6
	d[()] == 24

	# So rank-0 array indexed by empty tuple should be scalar.
        # Currently the last is wrong

Property 8: Indexing by tuple of slices gives array.
	a[:,:,:] == ones((2,3,4))
	b[:,:] == ones((3,4)) * 2
	c[:] == ones((,4)) * 6
	d[()] == ones(()) * 24

	# So rank-0 array indexed by empty tuple should be rank-0 array.
        # Currently the last is wrong

Property 9: Indexing as lvalues
	a[1,1,1] = 2
	b[1,1] = 2
	c[1,] = 2
	d[()] = 2

Property 10: Indexing and slicing as lvalues
	a[:,:,:] = ones((2, 3, 4))
	a[1,:,:] = ones((3, 4))
	a[1,1,:] = ones((4,))
	a[1,1,1] = ones(())
	
	# But the last is wrong.


Conclusion 1: rank-0 arrays are equivalent to scalars.
	See properties 7 and 8.

Conclusion 2: rank-0 arrays are mutable.
	See property 9.

Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
	should not be defined.

	See conclusion 1 and properties 1, 2, 3, 4.

Conclusion 4: A missing axis is similar to having dimension 1.
	See property 4.

Conclusion 5: rank-0 int arrays should be allowed to act as indices.
	See property 5.

Conclusion 6: rank-0 arrays should not be hashable except by object id.
	See conclusion 2.


Discussions:

  - These properties correspond to the current implementation quite well,
	except a few rough edges.

  - Mutable scalars are useful in their own rights.

  - Is there substantial difference in overhead between rank-0 arrays and
    scalars?

  - How to write literal values?  array(1) is too many characters.

  - For rank-1 and rank-0 arrays, Python notation distinguishes:

	c[1] vs c[1,]
	d[] vs d[()]

    Should these be used to handle semantic difference between indexing 
    and slicing?  Should d[] be syntactically allowed?

Hope these observations help.

Huaiyu




From huaiyu_zhu at yahoo.com  Tue Sep 17 00:12:04 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Tue Sep 17 00:12:04 2002
Subject: [Numpy-discussion] rank-0 arrays are mutable scalars
In-Reply-To: <Pine.LNX.4.33.0209162358310.1534-100000@mars.localdomain>
Message-ID: <Pine.LNX.4.33.0209170008490.1611-100000@mars.localdomain>

Oops, the subject line was somehow cut off.  Please use this one if you 
follow up. - Huaiyu

On Tue, 17 Sep 2002, Huaiyu Zhu wrote:

Some of the choices between rank-0 arrays and new scalar types
might be resolved by enumerating the properties desired of them.

Most properties of rank-0 arrays could be fixed by consistency
requirements alone, using operations that reduce array dimensions.

Let a = ones((2,3,4))
b = sum(a)
c = sum(b)
d = sum(c)

Property 1: the shape of an array is a tuple of integers
	a.shape == (2, 3, 4)
	b.shape == (3, 4)
	c.shape == (4,)
	d.shape == ()
	
Property 2: rank(a) == len(a.shape)
	rank(a) == 3 == len(a.shape)
	rank(b) == 2 == len(b.shape)
	rank(c) == 1 == len(c.shape)
	rank(d) == 0 == len(d.shape)

Property 3: len(a) == a.shape[0]
	len(a) == 2 == a.shape[0]
	len(b) == 3 == b.shape[0]
	len(c) == 4 == c.shape[0]
	len(d) == Exception == d.shape[0]

	# Currently the last is wrong?

Property 4: size(a) == product(a.shape)
	size(a) == 24 == product(a.shape)
	size(b) == 12 == product(b.shape)
	size(c) == 4 == product(c.shape)
	size(d) == 1 == product(d.shape)

	# Currently the last is wrong

Property 5: rank-0 array behaves as mutable numbers when used as value
	array(2)  is similar to 2
	array(2.0)  is similar to 2.0
	array(2j)  is similar to 2j

	# This is a summary of many concrete properties.

Property 6: Indexing reduces rank. Slicing preserves rank.
	a[:,:,:].shape = (2, 3, 4)
	a[1,:,:].shape = (3, 4)
	a[1,1,:].shape = (4,)
	a[1,1,1].shape = ()

Property 7: Indexing by tuple of ints gives scalar.
	a[1,1,1] == 1
	b[1,1] == 2
	c[1,] == 6
	d[()] == 24

	# So rank-0 array indexed by empty tuple should be scalar.
        # Currently the last is wrong

Property 8: Indexing by tuple of slices gives array.
	a[:,:,:] == ones((2,3,4))
	b[:,:] == ones((3,4)) * 2
	c[:] == ones((,4)) * 6
	d[()] == ones(()) * 24

	# So rank-0 array indexed by empty tuple should be rank-0 array.
        # Currently the last is wrong

Property 9: Indexing as lvalues
	a[1,1,1] = 2
	b[1,1] = 2
	c[1,] = 2
	d[()] = 2

Property 10: Indexing and slicing as lvalues
	a[:,:,:] = ones((2, 3, 4))
	a[1,:,:] = ones((3, 4))
	a[1,1,:] = ones((4,))
	a[1,1,1] = ones(())
	
	# But the last is wrong.


Conclusion 1: rank-0 arrays are equivalent to scalars.
	See properties 7 and 8.

Conclusion 2: rank-0 arrays are mutable.
	See property 9.

Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
	should not be defined.

	See conclusion 1 and properties 1, 2, 3, 4.

Conclusion 4: A missing axis is similar to having dimension 1.
	See property 4.

Conclusion 5: rank-0 int arrays should be allowed to act as indices.
	See property 5.

Conclusion 6: rank-0 arrays should not be hashable except by object id.
	See conclusion 2.


Discussions:

  - These properties correspond to the current implementation quite well,
	except a few rough edges.

  - Mutable scalars are useful in their own rights.

  - Is there substantial difference in overhead between rank-0 arrays and
    scalars?

  - How to write literal values?  array(1) is too many characters.

  - For rank-1 and rank-0 arrays, Python notation distinguishes:

	c[1] vs c[1,]
	d[] vs d[()]

    Should these be used to handle semantic difference between indexing 
    and slicing?  Should d[] be syntactically allowed?

Hope these observations help.

Huaiyu






From oliphant at ee.byu.edu  Tue Sep 17 09:10:05 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue Sep 17 09:10:05 2002
Subject: [Numpy-discussion] s are mutable scalars
In-Reply-To: <Pine.LNX.4.33.0209162358310.1534-100000@mars.localdomain>
Message-ID: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>

>
> Some of the choices between rank-0 arrays and new scalar types
> might be resolved by enumerating the properties desired of them.
>

These are helpful observations.

> Most properties of rank-0 arrays could be fixed by consistency
> requirements alone, using operations that reduce array dimensions.
>
> Let a = ones((2,3,4))
> b = sum(a)
> c = sum(b)
> d = sum(c)
>
> Property 1: the shape of an array is a tuple of integers
> 	a.shape == (2, 3, 4)
> 	b.shape == (3, 4)
> 	c.shape == (4,)
> 	d.shape == ()
>
> Property 2: rank(a) == len(a.shape)
> 	rank(a) == 3 == len(a.shape)
> 	rank(b) == 2 == len(b.shape)
> 	rank(c) == 1 == len(c.shape)
> 	rank(d) == 0 == len(d.shape)
>
> Property 3: len(a) == a.shape[0]
> 	len(a) == 2 == a.shape[0]
> 	len(b) == 3 == b.shape[0]
> 	len(c) == 4 == c.shape[0]
> 	len(d) == Exception == d.shape[0]
>
> 	# Currently the last is wrong?

Agreed, but this is because d is an integer and out of Numerics control.
This is a case for returning 0d arrays rather than Python scalars.

>
> Property 4: size(a) == product(a.shape)
> 	size(a) == 24 == product(a.shape)
> 	size(b) == 12 == product(b.shape)
> 	size(c) == 4 == product(c.shape)
> 	size(d) == 1 == product(d.shape)
>
> 	# Currently the last is wrong

I disagree that this is wrong.  This works as described for me.

>
> Property 5: rank-0 array behaves as mutable numbers when used as value
> 	array(2)  is similar to 2
> 	array(2.0)  is similar to 2.0
> 	array(2j)  is similar to 2j
>
> 	# This is a summary of many concrete properties.
>
> Property 6: Indexing reduces rank. Slicing preserves rank.
> 	a[:,:,:].shape = (2, 3, 4)
> 	a[1,:,:].shape = (3, 4)
> 	a[1,1,:].shape = (4,)
> 	a[1,1,1].shape = ()
>
> Property 7: Indexing by tuple of ints gives scalar.
> 	a[1,1,1] == 1
> 	b[1,1] == 2
> 	c[1,] == 6
> 	d[()] == 24
>
> 	# So rank-0 array indexed by empty tuple should be scalar.
>         # Currently the last is wrong

Not sure about this property, but interesting.

>
> Property 8: Indexing by tuple of slices gives array.
> 	a[:,:,:] == ones((2,3,4))
> 	b[:,:] == ones((3,4)) * 2
> 	c[:] == ones((,4)) * 6
> 	d[()] == ones(()) * 24
>
> 	# So rank-0 array indexed by empty tuple should be rank-0 array.
>         # Currently the last is wrong

Not sure about this one either.
>
> Property 9: Indexing as lvalues
> 	a[1,1,1] = 2
> 	b[1,1] = 2
> 	c[1,] = 2
> 	d[()] = 2
>
> Property 10: Indexing and slicing as lvalues
> 	a[:,:,:] = ones((2, 3, 4))
> 	a[1,:,:] = ones((3, 4))
> 	a[1,1,:] = ones((4,))
> 	a[1,1,1] = ones(())
>
> 	# But the last is wrong.
>
>
> Conclusion 1: rank-0 arrays are equivalent to scalars.
> 	See properties 7 and 8.
>
> Conclusion 2: rank-0 arrays are mutable.
> 	See property 9.
>
> Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
> 	should not be defined.

Why is this?  I thought you argued the other way for len(scalar).  Of
course, one solution is that we could overwrite the len() function and
allow it to work for scalars.

>
> 	See conclusion 1 and properties 1, 2, 3, 4.
>
> Conclusion 4: A missing axis is similar to having dimension 1.
> 	See property 4.
>
> Conclusion 5: rank-0 int arrays should be allowed to act as indices.
> 	See property 5.

Can't do this for lists and other builtin sequences.
>
> Conclusion 6: rank-0 arrays should not be hashable except by object id.
> 	See conclusion 2.
>
>
> Discussions:
>
>   - These properties correspond to the current implementation quite well,
> 	except a few rough edges.
>
>   - Mutable scalars are useful in their own rights.
>
>   - Is there substantial difference in overhead between rank-0 arrays and
>     scalars?

Yes.

>
>   - How to write literal values?  array(1) is too many characters.
>
>   - For rank-1 and rank-0 arrays, Python notation distinguishes:
>
> 	c[1] vs c[1,]
> 	d[] vs d[()]
>
>     Should these be used to handle semantic difference between indexing
>     and slicing?  Should d[] be syntactically allowed?
>
> Hope these observations help.

Thanks for the observations.

-Travis O.





From huaiyu_zhu at yahoo.com  Wed Sep 18 00:03:02 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Wed Sep 18 00:03:02 2002
Subject: [Numpy-discussion] rank-0 arrays are mutable scalars
In-Reply-To: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>
Message-ID: <Pine.LNX.4.33.0209172211240.1082-100000@mars.localdomain>

On Tue, 17 Sep 2002, Travis Oliphant wrote:

> > 	len(d) == Exception == d.shape[0]
> >
> > 	# Currently the last is wrong?
> 
> Agreed, but this is because d is an integer and out of Numerics control.
> This is a case for returning 0d arrays rather than Python scalars.

That is one problem.  It can be removed by using shape(d).  

More fundamentally, though, len(d) == shape(d)[0] == ()[0] => IndexError.
I think Konrad made this point a few days back.

> > 	size(d) == 1 == product(d.shape)
> >
> > 	# Currently the last is wrong
> 
> I disagree that this is wrong.  This works as described for me.

Right.  Change d.shape to shape(d) here (and in several other places).

> Why is this?  I thought you argued the other way for len(scalar).  Of
> course, one solution is that we could overwrite the len() function and
> allow it to work for scalars.

Raising exception is the correct behavior, not a problem to be solved.

> >
> > Conclusion 5: rank-0 int arrays should be allowed to act as indices.
> > 	See property 5.
> 
> Can't do this for lists and other builtin sequences.

If numarray defines a consistent set of behaviors for integer types that
is intuitively understandable, it might not be difficult to persuade core
Python to check against an abstract integer type.

> >   - Is there substantial difference in overhead between rank-0 arrays and
> >     scalars?
> 
> Yes.

That would be one major problem.  


However, after giving this some more thoughts, I'm starting to doubt the 
analogy I made.  The problem is that in the end there is still a need to 
index an array and obtain a good old immutable scalar.  So

- What notation should be used for this purpose?  We can use c[0] to get
  immutable scalars and c[0,] for rank-0 arrays / mutable scalars.  But 
  what about other ranks?  Python does not allow distinctions based on  
  a[1,1,1] versus a[(1,1,1)] or d[] versus d[()].

- This weakens the argument that rank-0 arrays are scalars, since that 
  argument is essentially based on sum(c) and c[0] being of the same type.

Huaiyu




From juenglin at informatik.uni-freiburg.de  Wed Sep 18 01:16:02 2002
From: juenglin at informatik.uni-freiburg.de (Ralf Juengling)
Date: Wed Sep 18 01:16:02 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
In-Reply-To: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>
References: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>
Message-ID: <1032336896.15531.429.camel@leto>

On Mon, 2002-09-16 at 22:11, Tim Hochberg wrote:
> 
> Can someone refresh my memory as to why some properties of NumArrays that
> are conceptually attributes are accessed as atributes (a.shape, a.rank),
> while others are accessed through functions (a.type, a.iscontiguous,...).

Yes, I'd like to understand as well; there is even a third. There are
attributes, e.g. a.shape       vs
methods,    e.g. a.type()      vs
functions   e.g. len(a).

Regards,
Ralf



> 
> Thanks,
> 
> -tim
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
-------------------------------------------------------------------------
Ralf J?ngling
Institut f?r Informatik - Lehrstuhl f. Mustererkennung &
Bildverarbeitung
Georges-K?hler-Allee
Geb?ude 52                                       Tel:
+49-(0)761-203-8215
79110 Freiburg                                   Fax:
+49-(0)761-203-8262
-------------------------------------------------------------------------




From aureli at ipk.fhg.de  Thu Sep 19 09:18:06 2002
From: aureli at ipk.fhg.de (Aureli Soria Frisch)
Date: Thu Sep 19 09:18:06 2002
Subject: [Numpy-discussion] Conversion numarray/Numeric objects
In-Reply-To:  <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>
References: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>
Message-ID: <a05111b00b9afa51e3cc0@[153.97.92.109]>

Hi,

I have developed till now all my numerical stuff using Numeric objects.

I want to use some modules using numarray. I have tested some 
conversion functions on Python, e.g.

numarray.array(Numeric_object.tolist())

but it seems to me a bit slow (specially for large arrays as images). 
Is there any more ways of doing this conversion?
Should I write my own conversion in C?
Which advices could you give me in order to make an smoother transition?

Furthermore I found the web site about numarray/Numeric 
incompatibilities very interesting. However I did not understand 
quite well the paragraph:

>Numeric arrays have some public attributes. Numarray arrays
>have none. All changes to an array's state must be made
>through accessor methods. Specifically, Numeric has the following
>attributes:
>
>Numeric        numarray accessor method(s)
>Attribute
>
>shape      --> shape()  (i.e., specify new shape through method arg instead
>of
>                          assigning to shape attribute)
>flat       --> flat()
>real       --> real()   (set capability not implemented yet, but will be)
>imag       --> imag()   (ditto)
>savespace  --> not used, no equivalent functionality (xxx check this)

since interactively typing a.shape() or a.shape((8,8)) raise an 
exception (Tuple object not callable). Am I misunderstanding 
something?

Thanks in advance.
Best regards,
Aureli

-- 
#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail: aureli at ipk.fhg.de
fon: +49 30 39006-143
fax: +49 30 3917517
web: http://vision.fhg.de/~aureli/web-aureli_en.html
#################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20020919/c1cacdb3/attachment.html>

From jmiller at stsci.edu  Thu Sep 19 09:49:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 19 09:49:02 2002
Subject: [Numpy-discussion] Conversion numarray/Numeric objects
References: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu> <a05111b00b9afa51e3cc0@[153.97.92.109]>
Message-ID: <3D89FFDD.6090506@stsci.edu>

Aureli Soria Frisch wrote:

> Hi,
>
>
> I have developed till now all my numerical stuff using Numeric objects.
>
>
> I want to use some modules using numarray. I have tested some 
> conversion functions on Python, e.g.
>
>
> numarray.array(Numeric_object.tolist())
>
Much faster,  but less general, is:

numarray.fromstring(Numeric_object.tostring(), type=XXXX)

>
> but it seems to me a bit slow (specially for large arrays as images). 
> Is there any more ways of doing this conversion?
>
> Should I write my own conversion in C?
>
It depends on how soon you need it to be fast.    numarray 
fromlist/tolist should be implemented in C for the next release, which 
should occur within a couple months, perhaps sooner.  If you want to 
write a C extension for numarray now, the best way to do it is to use 
the Numeric compatability layer,   which essentially uses a subset of 
the Numeric C-API,  and is not likely to change.

> Which advices could you give me in order to make an smoother transition?
>
>
> Furthermore I found the web site about numarray/Numeric 
> incompatibilities very interesting. However I did not understand quite 
> well the paragraph:
>
I think that webpage is out of date.

>
> since interactively typing a.shape() or a.shape((8,8)) raise an 
> exception (Tuple object not callable). Am I misunderstanding something?
>
Actually, for python 2.2 and up,  numarray has all of those attributes 
in a Numeric compatible, Python properties based form.   So ".shape" 
get/sets the shape in numarray, just as in Numeric:

 >>> import numarray
 >>> a = numarray.arange(10)
 >>> a.shape
(10,)
 >>> a.shape = (2,5)
 >>> a.shape
(2,5)

Because of this,  the expression a.shape() that you mentioned tries to 
"call" the shape tuple returned by .shape, resulting in an exception.

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / SSB






From hinsen at cnrs-orleans.fr  Fri Sep 20 10:05:05 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 20 10:05:05 2002
Subject: [Numpy-discussion] A message from Cameron Laird
Message-ID: <200209201704.g8KH4Ie19316@chinon.cnrs-orleans.fr>

I send this on behalf of Cameron Laird <claird at phaseit.net>.
Please reply to him, not to me.

  I have at least a couple of assignments from magazines such
  as IBM's developerWorks to report on matters that involve
  Numeric.  I'd welcome contact from anyone here who wants to
  publicize his or her work with Python and Numeric.

  I have a particular interest in advantages Python and Numeric
  enjoy over such alternatives as Mathematica, IDL, SAS/IML,
  MATLAB, and so on, all of which are more narrowly targeted at
  the kinds of scientific and engineering problems tackled by
  contributors to this mailing list.  What does Python do for
  you that the commercial products don't?

  I suspect that many of you will mention, in one form or
  another, Python's aptness for programming "in the large".
  Do you have specific examples of how this is clumsy in
  MATLAB, Mathematica, and so on?

  Have you tried to interface MATLAB and so on to hardware
  instrumentation or other external data sources?

  How do the scientists and engineers (as opposed to the
  "informaticians" or software developers) on your teams
  accept Python, compared to IDL and friends?  Do scientists
  at your site program?

  Is there anything Python's missing in its competition with
  MATLAB and so on?  


  Cameron Laird       <Cameron at Lairds.com>       +1 281 996 8546 FAX
  http://phaseit.net/claird/misc.writing/publications.html




From travis at enthought.com  Fri Sep 20 10:55:02 2002
From: travis at enthought.com (Travis N. Vaught)
Date: Fri Sep 20 10:55:02 2002
Subject: [Numpy-discussion] A message from Cameron Laird
In-Reply-To: <200209201704.g8KH4Ie19316@chinon.cnrs-orleans.fr>
Message-ID: <KPEIKEPEIBGNGNKHDILIEEIODMAA.travis@enthought.com>

Thought I'd reply to all since I'm including some links that are possibly
interesting to the list:

> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Konrad
> Hinsen
> Sent: Friday, September 20, 2002 12:04 PM
> To: numpy-discussion at lists.sourceforge.net
> Subject: [Numpy-discussion] A message from Cameron Laird
>
>
> I send this on behalf of Cameron Laird <claird at phaseit.net>.
> Please reply to him, not to me.
>
>   I have at least a couple of assignments from magazines such
>   as IBM's developerWorks to report on matters that involve
>   Numeric.  I'd welcome contact from anyone here who wants to
>   publicize his or her work with Python and Numeric.
>

The SciPy Toolkit (http://www.scipy.org) attacks Matlab's functionality
head-on--providing much of the functional interface symantics that Matlab
provides and much much more. It is a stated goal of many in the SciPy
development community to eliminate the need for Matlab altogether by
providing a tool that offers _all_ the functionality and better performance
(among many other things) in an open-source, truly object-oriented package.

>   I have a particular interest in advantages Python and Numeric
>   enjoy over such alternatives as Mathematica, IDL, SAS/IML,
>   MATLAB, and so on, all of which are more narrowly targeted at
>   the kinds of scientific and engineering problems tackled by
>   contributors to this mailing list.  What does Python do for
>   you that the commercial products don't?
>
>   I suspect that many of you will mention, in one form or
>   another, Python's aptness for programming "in the large".
>   Do you have specific examples of how this is clumsy in
>   MATLAB, Mathematica, and so on?
>
>   Have you tried to interface MATLAB and so on to hardware
>   instrumentation or other external data sources?
>

The recent SciPy '02 workshop (http://www.scipy.org/site_content/scipy02)
had some presented material that tangentially addressed interfacing Matlab
and Mathematica (perhaps even in a bof, I've slept since then).  You might
try the scipy-user mailing list as well. (Presentations:
http://www.scipy.org/site_content/scipy02/presentations , Mailing list:
http://www.scipy.org/site_content/MailList)

>   How do the scientists and engineers (as opposed to the
>   "informaticians" or software developers) on your teams
>   accept Python, compared to IDL and friends?  Do scientists
>   at your site program?
>
>   Is there anything Python's missing in its competition with
>   MATLAB and so on?
>

The aforementioned workshop included a survey that addressed these issues as
well.  You can look at the aggregated results by following the link below
(it's a little raw in its layout, let me know if there is a question about
how to interpret some things).
http://www.scipy.org/site_content/scipy02/survey_results.htm

(note: some questions were 'pick one' and others were 'pick all that
apply'--the total number of surveys turned in was 36, I think.)


>
>   Cameron Laird       <Cameron at Lairds.com>       +1 281 996 8546 FAX
>   http://phaseit.net/claird/misc.writing/publications.html
>




From rob at pythonemproject.com  Fri Sep 20 17:41:02 2002
From: rob at pythonemproject.com (Rob)
Date: Fri Sep 20 17:41:02 2002
Subject: [Numpy-discussion] A message from Cameron Laird
References: <200209201704.g8KH4Ie19316@chinon.cnrs-orleans.fr>
Message-ID: <3D8BBA34.AA81A36B@pythonemproject.com>

Konrad Hinsen wrote:
> 
<snip>

What I like about Numeric stems from what I like about Python- the
language does not get in my way of creating algorithms.  I do not have
to constantly refer to some reference book while programming.

The idea of the Numeric Python EM Project (URL below) was to put EM code
into a form where it could be more easily understood.  Python and
Numeric Python were the perfect choice for that task.  As for
Mathematica and other commercial software, I have no use for them. 
Python is free.

Sincerely,  Rob.

-- 
-----------------------------
The Numeric Python EM Project

www.pythonemproject.com



From perry at stsci.edu  Tue Sep 24 13:40:06 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Tue Sep 24 13:40:06 2002
Subject: [Numpy-discussion] numarray rank-0 decisions, rationale, and summary
Message-ID: <JFEGLNDJEDNOMPPHDEJFGEINEAAA.perry@stsci.edu>

[I posted this almost a week ago, but apparently an email problem
prevented it from actually getting posted!]

I think there has been sufficient discussion about rank-0 arrays
to make some decisions about how numarray will handle them.

[If you don't want to wade through the rationale, jump to the
end where there is a short summary of what we plan to do and
what we have questions about]

********************************************************************

First I'd like to take a stab at summarizing the case made for
rank-0 arrays in general and adding some of my own comments 
regarding these points.

1) rank-0 arrays are a useful mechanism to avoid having binary
operations with scalars cause unintended promotion of other arrays
to larger numeric types (e.g. 2*arange(10, typecode=Int16) results
in an Int32 result).

*** For numarray this is a non-issue because the coercion rules
prevent scalars from increasing the type of an array if the scalar
is the same kind of number (e.g., Int, Float, Complex) as the
array.

2) rank-0 arrays preserve the type information instead of converting
scalars to Python scalars.

*** This seems of limited value. With only a couple possible exceptions
in the future (and none now), Python scalars are effectively the 
largest type available so no information is lost. One can convert
to and from Python scalars and not lose any information. The possible
future exceptions are long doubles and UInt32 (supported in Numeric, 
but not numarray yet--but frankly, I'm not yet sure how important
UInt32 is at the moment). It is possible that Python scalars may
move up in size so this may or may not become an issue in the future.
By itself, it does not appear to be a compelling reason.

3) rank-0 arrays allow controlling exceptions (e.g. divide by zero)
in a way different from how Python handles them (exception always)

*** This is a valid concern...maybe. I was more impressed by it
initially, but it occurred to me that most expressions that involve
a scalar exception (NaN, divide-by-zero, overflow, etc.) generally
corrupt everything, unlike exceptions with arrays where only a few
values may be tainted. Unless one is only interested in ignoring some
scalar results in a scalar expression as part of a larger computation,
it seems of very limited use to ignore, or warn on scalar exceptions.
In any event, this is really of no relevance to the use of rank-0
for indexed results or reduction operations.

4) Using rank-0 arrays in place of scalars would promote more
generic programming. This was really the last point of real contention
as far as I was concerned. In the end, it really came down to seeing
good examples of how lacking this caused code to be much worse than
it could be with rank-0 arrays. There really are two cases being
discussed: whether indexing a single item ("complete dereferencing"
in Scott Gilbert's description) returns rank-0, and whether reduction
operations return rank-0.

*** indexing returning rank-0. Amazingly enough no one was able to 
provide even one real code example of where rank-0 returns from indexing
was a problem (this includes MA as far as I can tell). In the end, this
has been much ado about nothing. Henceforth, numarray will return
Python scalars when arrays are indexed (as it does currently).

*** reduction operations. There are good examples of where reduction
operations returning rank-0 are made simpler. However, the situation
is muddied quite a bit by other issues which I will discuss below.
This is an area that deserves a bit more discussion in general.
But before I tackle that, there is a point about rank-0 arrays that
needs to be made which I think is in some respects is an obvious point,
but somehow got lost in much of the discussions.

Even if it were true that rank-0 arrays made for much simpler, generic
code, they are far less useful than might appear in simplifying
code. Why? Because even if array operations (whether indexing, reduction
or other functions) were entirely consistent about never returning 
scalars, it is a general fact that most Numeric/numarray code must
be prepared to handle Python scalars thrown at it in place of
arrays by the user. Since Python scalars can come leaking into
your code at many points, consistency in Numeric/numarray in avoiding
Python scalars really doesn't solve the issue. I would hazard a
guess that the great majority of the conditional code that exist
today would not be eliminated because of this (e.g., this appears
to be the case for MA)

Reduction operations:

There is a good case to be made that reduction operations should
result in rank-0 arrays rather than scalars (after all, they
are reducing dimensions), but not everyone agrees that is what
should be done. But before deciding what is to be done there,
some problems with rank-0 arrays should be discussed. I think 
Konrad and Huaiyu have made very powerful arguments about how
certain operations like indexing, attributes and such should or
shouldn't work. In particular, some current Numeric behaviors
should be changed. Indexing by 0 should not work (instead Scott
Gilbert's suggestion of indexing with an empty tuple sounds 
right, if a bit syntatically clumsy due to Python not accepting
an empty index). len should not return 1, etc.

So even if we return rank-0 values in reduction operations, this
appears to still cause problems with some of the examples given
by Eric that depend on len(rank-0) = 1. What should be done
about that? One possibility is to use different numarray
functions designed to help write generic code (e.g., an alternate
function to len). But there is one aspect to this that ought to be
pointed out. Some have asked for rank-0 arrays that can be indexed
with 0 and produce len of 1. There is such an object that does this
and it is a rank-1 len-1 array. One alternative is to have reduction
operations have as their endpoint a rank-1 len-1 array rather than a
rank-0 array. The rank-0 endpoint is more justified conceptually,
but apparently less practical. If a reduction on a 1-d arrray always
produced a 1-d array, then one can always be guaranteed that it can
be indexed, and that len works on it. The drawback is that it can 
never be used as a scalar directly as rank-0 arrays could be.

I think this is a case where you can't have it both ways. If you 
want a scalar-like object, then some operations that work on higher
rank arrays won't work on it (or shouldn't). If you want something
where these operations do work, don't expect to use it where a
scalar is expected unless you index it.

Is there any interest in this alternate approach to reductions? We plan
to have two reduction methods available, one that results in scalars, and
one in arrays. The main question is which one .reduce maps to, and what
the endpoint is for the method that always returns arrays is.

***********************************************************************
                           SUMMARY
***********************************************************************

1) Indexing returns Python scalars in numarray. No rank-0 arrays
   are ever returned from indexing.
2) rank-0 arrays will be supported, but len(), and indexing will not
   work as they do in Numeric. In particular, to get a scalar, one
   will have to index with an empty tuple (e.g., x[()], x[0] will
   raise an exception), len() will return None.

Questions:

1) given 2, is there still a desire for .reduce() to return
rank-0 arrays (if not, we have .areduce() which is intented to return
arrays always).

2) whichever is the "returns arrays always" reduce method, should the 
endpoint be rank-0 arrays or rank-1 len-1 arrays?
 




From hinsen at cnrs-orleans.fr  Tue Sep 24 14:19:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Tue Sep 24 14:19:04 2002
Subject: [Numpy-discussion] numarray rank-0 decisions, rationale, and summary
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFGEINEAAA.perry@stsci.edu>
References: <JFEGLNDJEDNOMPPHDEJFGEINEAAA.perry@stsci.edu>
Message-ID: <m38z1rqe5v.fsf@localhost.localdomain>

"Perry Greenfield" <perry at stsci.edu> writes:

> Questions:
> 
> 1) given 2, is there still a desire for .reduce() to return
> rank-0 arrays (if not, we have .areduce() which is intented to return
> arrays always).
> 
> 2) whichever is the "returns arrays always" reduce method, should the 
> endpoint be rank-0 arrays or rank-1 len-1 arrays?

I don't really see an application where a reduction operation yielding
rank-1 or higher arrays would be useful. It would be a special case,
not useful for generic programming. So my answer to 2) is rank-0.

As for 1), if indexing doesn't return rank-0 arrays, then standard
reduction shouldn't either. We would then have a system in which
rank-0 arrays are "expert only" stuff, most users would never see
them, and they could safely be ignored in tutorials.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From perry at stsci.edu  Tue Sep 24 14:33:03 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Tue Sep 24 14:33:03 2002
Subject: [Numpy-discussion] numarray rank-0 decisions, rationale, and summary
In-Reply-To: <m38z1rqe5v.fsf@localhost.localdomain>
Message-ID: <JFEGLNDJEDNOMPPHDEJFKEINEAAA.perry@stsci.edu>

Konrad Hinsen writes:
> "Perry Greenfield" <perry at stsci.edu> writes:
> 
> > Questions:
> > 
> > 1) given 2, is there still a desire for .reduce() to return
> > rank-0 arrays (if not, we have .areduce() which is intented to return
> > arrays always).
> > 
> > 2) whichever is the "returns arrays always" reduce method, should the 
> > endpoint be rank-0 arrays or rank-1 len-1 arrays?
> 
> I don't really see an application where a reduction operation yielding
> rank-1 or higher arrays would be useful. It would be a special case,
> not useful for generic programming. So my answer to 2) is rank-0.
> 
What I am wondering is what behavior suits "generic" programming more.
Eric Jones and Paul Dubois have given examples where having to
deal with entities that may be scalars or arrays is a pain. But,
having said that, the behavior that Eric wanted was not consistent
with what many thought rank-0 arrays should have (i.e., len(rank-0)=1,
rank-0[0] = value). The proposal to generate rank-1 len-1 arrays was
made since len() of these arrays is = 1, and indexing with [0]
does work. So for the kinds of examples he gave, rank-1 len-1 arrays
appear to allow for more generic code.

But I'm not trying to speak for everyone; that's why I'm asking
for opinions. Do you have examples where you find rank-0 arrays
make for more generic code in your cases when len() and indexing
on these does not have the behavior Eric wanted? May I see a couple?

> As for 1), if indexing doesn't return rank-0 arrays, then standard
> reduction shouldn't either. We would then have a system in which
> rank-0 arrays are "expert only" stuff, most users would never see
> them, and they could safely be ignored in tutorials.
> 
That's my inclination, but I think that the question of whether there
should be some reduce mechanism that returns arrays always is still
a valid one. I can see that there are good uses for that (or at least
a function to cast a scalar to a rank-1 len-1 array if it isn't already
an array, like what array() does except that it now generates rank-0
arrays).

Perry



From mathew at comma.jpl.nasa.gov  Wed Sep 25 17:24:04 2002
From: mathew at comma.jpl.nasa.gov (Mathew Yeates)
Date: Wed Sep 25 17:24:04 2002
Subject: [Numpy-discussion] Cast problems .... doo dah, doo dah
Message-ID: <3D92538D.8020108@comma.jpl.nasa.gov>

The following code produces an error
---------------------------------------
a=array(([1,2,3,4,5],[10,9,8,7,6])).astype(Float32)
cond=array(([0,0,0,0,0],[1,1,1,1,1])
b=array(([0,0,0,0,0],[0,0,0,0,0]))

choose(cond,(a,b))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: Array can not be safely cast to required type
------------------------------

I suppose I might need to cast b to Float32. But, I have a function 
which is passed a and b
as args. Ideally, I would like a function which casts a and b to the 
highest compatible type since
"choose" doesnt do it. How can I do this simply?

Mathew

-- 
M47h3w `/34735





From mathew at comma.jpl.nasa.gov  Thu Sep 26 16:13:03 2002
From: mathew at comma.jpl.nasa.gov (Mathew Yeates)
Date: Thu Sep 26 16:13:03 2002
Subject: [Numpy-discussion] Casting. Method to the madness?
Message-ID: <3D939474.2030402@comma.jpl.nasa.gov>

Hmmm, aparently, I can cast a long array to a double array but not
to a float array.

Check out the following code from arrayobject.c (Numeric 22.0)
[this is a switch on the type being casted from]

case PyArray_LONG:
    /*Shouldn't allow Longs to be cast to Ints, not safe on 64-bit 
machines!*/
    return (totype >= PyArray_INT) && (totype != PyArray_FLOAT);

This makes absolutely no sense to me. Is this a bug?

Mathew

-- 
M47h3w `/34735





From retype at terra.com.br  Fri Sep 27 10:17:02 2002
From: retype at terra.com.br (Leonardo Santagada)
Date: Fri Sep 27 10:17:02 2002
Subject: [Numpy-discussion] (no subject)
In-Reply-To: <E17uyGi-00050H-00@usw-sf-list2.sourceforge.net>
References: <E17uyGi-00050H-00@usw-sf-list2.sourceforge.net>
Message-ID: <20020927141325.41af5fb8.retype@terra.com.br>

confirm 498554



From szport at fromru.com  Mon Sep 30 02:11:12 2002
From: szport at fromru.com (Zaur Shibzoukhov)
Date: Mon Sep 30 02:11:12 2002
Subject: [Numpy-discussion] P: tofile & fromfile
Message-ID: <20020930130251.0290bb24.szport@fromru.com>

I propose to add two function to multiarray module in Numeric:

tofile(array, openedfile)
array = fromfile(opendfile, dimension, typecode = "l")

Now I use these functions in my module.
Here is an implementation from module:

###############################################################################3

#include "Python.h"
#define PY_ARRAY_UNIQUE_SYMBOL PyArray_arrayfile
#include "Numeric/arrayobject.h"

#define MAX_DIMS 40

static PyObject *
PyArray_FromFile(PyObject *self, PyObject *args)
{
    PyObject *f, *dimobj, *temp;
    PyArray_Descr *descr;
    PyArrayObject *ret = NULL;
    int i, n, dim[40], nt;
    char *typecode = "l";
    FILE *fp;
    int type_num, N = 1;
    size_t nread;

    if (!PyArg_ParseTuple(args, "O!O!|s#:fromfile", \
                          &PyFile_Type, &f, &PyTuple_Type, &dimobj, &typecode, &nt))
        return NULL;

    fp = PyFile_AsFile(f);
    if (fp == NULL) {
        PyErr_SetString(PyExc_TypeError, "1st argument must be open file");
        return NULL;
    }
    n = PyTuple_Size(dimobj);
    if (n > MAX_DIMS) {
        PyErr_SetString(PyExc_TypeError, "dimension is too large");
        return NULL;
    }
    if (n > 0) {
        for (i = 0; i < n; i++) {
            temp = PyTuple_GetItem(dimobj, i);
            dim[i] = (int)PyInt_AsLong(temp);
        }
        
        descr = PyArray_DescrFromType(*typecode);
        type_num = descr -> type_num;
        
        ret = (PyArrayObject *)PyArray_FromDims(n, dim, type_num);
        memcpy(ret->data, descr->zero, N*descr->elsize);
        
        N = 1;
        for (i = 0; i < n; i++) N *= dim[i];
        nread = fread((char *)ret->data, descr->elsize, N, fp);
        if (nread < (size_t)N) {
            PyErr_SetString(PyExc_EOFError, "not enough items in file");
            return NULL;
        }
    }

    return (PyObject *)ret;
}

static char PyArray_FromFile_doc [] =
"fromfile(f, dimension, typecode = \'l\')\n\
\n\
Create array from open file f with given dimension and typecode.\n\
Note, that file must be open in binary mode.";


static PyObject *
PyArray_ToFile(PyObject *self, PyObject *args)
{
    PyObject *f;
    PyArrayObject *A;
    int N;
    FILE *fp;
        
    if (!PyArg_ParseTuple(args, "O!O!:tofile", &PyArray_Type, &A, &PyFile_Type, &f))
        return NULL;
    
    fp = PyFile_AsFile(f);
    if (fp == NULL) {
        PyErr_SetString(PyExc_TypeError, "arg must be open file");
        return NULL;
    }
    
    N = PyArray_SIZE(A);
    if (N > 0) {
        if (fwrite(A->data, A->descr->elsize, N, fp) != (size_t)N) {
            PyErr_SetFromErrno(PyExc_IOError);
            clearerr(fp);
            return NULL;
        }
    }
    Py_INCREF(Py_None);
    return Py_None;
}

static char PyArray_ToFile_doc [] =
"tofile(array, f)\n\
\n\
Write array to file open f.\n\
Note, that file must be open in binary mode.";


static PyMethodDef arrayfile_module_methods[] = {
         {"fromfile",  PyArray_FromFile, 1, PyArray_FromFile_doc},
         {"tofile",    PyArray_ToFile,   1, PyArray_ToFile_doc},
         {NULL,                 NULL}  /* sentinel */
};

############################################################################

Zaur Shibzoukhov






From sdfrost at UCSD.Edu  Tue Sep  3 07:28:03 2002
From: sdfrost at UCSD.Edu (sdfrost at UCSD.Edu)
Date: Tue Sep  3 07:28:03 2002
Subject: [Numpy-discussion] Comparison between Numpy/Numarray and Ox
Message-ID: <200209031427.HAA20024@smtp.ucsd.edu>

Dear Numpy list,

Has anyone looked at the comparitive speed of Numpy/Numarray vs. Ox?  For 
those that don't know, Ox is an implicitly typed, object oriented matrix 
programming language that is one of the fastest around, and also allows 
linking to C libraries (http://www.nuff.ox.ac.uk/Users/Doornik/index.html).  I 
use Python a lot, but I was wondering what the computational cost would be to 
switch my matrix-y stuff to Numpy.  Ox isn't open source...

Best wishes
Simon






From transue at niehs.nih.gov  Mon Sep  9 14:38:03 2002
From: transue at niehs.nih.gov (Tom Transue)
Date: Mon Sep  9 14:38:03 2002
Subject: [Numpy-discussion] Logical AND on arrays
Message-ID: <3D7D144B.AB8DCB2D@niehs.nih.gov>

If ar1 and ar2 are arrays, are the following two expressions supposed to give
the same result?

  (ar1 and ar2)

  logical_and(ar1,ar2)

The first form seems to return the value of the second array, which isn't very
useful.  It would be nice to map the first expression to do what the first does.

Tom Transue



From a.schmolck at gmx.net  Mon Sep  9 15:25:03 2002
From: a.schmolck at gmx.net (Alexander Schmolck)
Date: Mon Sep  9 15:25:03 2002
Subject: [Numpy-discussion] Logical AND on arrays
In-Reply-To: <3D7D144B.AB8DCB2D@niehs.nih.gov>
References: <3D7D144B.AB8DCB2D@niehs.nih.gov>
Message-ID: <yfsk7lukdll.fsf@black132.ex.ac.uk>

Tom Transue <transue at niehs.nih.gov> writes:

> If ar1 and ar2 are arrays, are the following two expressions supposed to give
> the same result?
> 
>   (ar1 and ar2)
> 
>   logical_and(ar1,ar2)
> 
> The first form seems to return the value of the second array, which isn't very
Only if the boolean value of the first array is true (i.e. iff not all its
values are nonzero).

> useful.  It would be nice to map the first expression to do what the first does.

Impossible. Python's `and` and `or` are shortcircuting operators with a fixed
meaning. ``A and B`` only evaluates both ``A`` and ``B`` iff both have the
boolean value 'True' (e.g. ``0 and 0/1`` is OK because the second part never
gets evaluated), otherwise the first 'False` value is returned.

While each class is free to define a truth testing method (`__nonzero__`, or
`__len__` if `__nonzero__` doesn't exist; the convention is that empty
containers and zero numbers are false and pretty much everything else is
true), there is (luckily) no way to change the meaning of `and` and `or` (just
as one (luckily) can't change what the `if` statement does).

If this is still a bit unclear, I think this question has been raised before
in this list, so you should be able to find a thread in the archive.

cheers,

alex


-- 
Alexander Schmolck     Postgraduate Research Student
                       Department of Computer Science
                       University of Exeter
A.Schmolck at gmx.net     http://www.dcs.ex.ac.uk/people/aschmolc/




From Chris.Barker at noaa.gov  Mon Sep  9 16:56:05 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Mon Sep  9 16:56:05 2002
Subject: [Numpy-discussion] Logical AND on arrays
References: <3D7D144B.AB8DCB2D@niehs.nih.gov> <yfsk7lukdll.fsf@black132.ex.ac.uk>
Message-ID: <3D7D32E4.1BAA1BE5@noaa.gov>

Alexander Schmolck wrote:
> Tom Transue <transue at niehs.nih.gov> writes:
> > useful.  It would be nice to map the first expression to do what the first does.
> 
> Impossible. Python's `and` and `or` are shortcircuting operators with a fixed
> meaning. ``A and B`` only evaluates both ``A`` and ``B`` iff both have the
> boolean value 'True' (e.g. ``0 and 0/1`` is OK because the second part never
> gets evaluated), otherwise the first 'False` value is returned.

true. Another option, however is to use the bitwise and aperator, which
is overloaded:

A & B

This generally will work in the cases you are likely to use it, like:

if (A > 5) & (B < 5):

as you'll be &-ing arrays of zeros and ones.

> there is (luckily) no way to change the meaning of `and` and `or` (just
> as one (luckily) can't change what the `if` statement does).

personally, I don't think it's that lucky. I'd rather lose the slight
benifits of the sometimes confusing ability to short-circuit, and get
full operator overloading. I think "and" and "or" are more like "<" than
"if". However, if I had designed a language it would be a pretty
wretched mess, I'm sure.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From jmiller at stsci.edu  Wed Sep 11 15:08:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 11 15:08:02 2002
Subject: [Numpy-discussion] Todd's NumPy Feature Request tracker surprise!
Message-ID: <3D7FBE61.7060805@stsci.edu>

I was really surprised today to find that Perry Greenfield and I were 
the only two developers listed on the Numeric Feature Requests tracker 
on Source Forge.  Since we work on numarray,  I removed us.  That leaves 
no one.

If you are a NumPy developer who wants to handle Feature Requests for 
Numeric,  you might want to add yourself back onto the list of assignees.  

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / SSG





From perry at stsci.edu  Thu Sep 12 21:17:05 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Sep 12 21:17:05 2002
Subject: [Numpy-discussion] rank-0 arrays
Message-ID: <NEBBIJKBMLDBLNCEEFOCCEEMCOAA.perry@stsci.edu>

Before we implement what we said we would regarding rank-0
arrays in numarray, there became apparent a couple new issues
that didn't really get considered in the first round of 
discussion (at least I don't recall that they did).

To restate the issue: there was a question about whether
an index to an array that identified a single element only
(i.e., not a slice, nor an incomplete index, e.g. x[3] where
x is two dimensional) should return a Python scalar or a
rank-0 array. Currently Numeric is inconsistent on this point.
One usually gets scalars, but on some occasions, rank-0 arrays
are returned. Good arguments are to be had for either alternative.

The primary advantage of returning rank-0 arrays is that they
reduce the need for conditional code checking to see if a result
is a scalar or an array. At the end of the discussion it was
decided to have numarray return rank-0 arrays in all instances of
single item indexing. Since then, a couple potential snags have
arisen. I've already discussed some of these with Paul Dubois
and Eric Jones. I'd like a little wider input before making a
final (or at least experimental) decision.

If we return rank-0 arrays, what should repr return for rank-0
arrays. My initial impression is that the following is highly
undesirable for a interactive session, but maybe it is just me:

>>> x = arange(10)
>>> x[2]
array(2)

We, of course, could arrange __repr__ to return "2" instead,
in other words print the simple scalar for all cases of rank-0
arrays. This would yield the expected output in the above
example. Nevertheless, isn't it violating the intent of repr?
Are there other examples where Python uses repr in a similar,
misleading manner? But perhaps most feel that returning array(2)
is perfectly acceptable and won't annoy users. I am curious
about what people think about this.

The second issue is an efficiency one. Currently numarray uses
Python objects for arrays. If we return rank-0 arrays for
single item indexing, then some naive uses of larger arrays
as sequences may lead to an enormous number of array objects
to be created. True, there will be equivalent means of doing
the same operation that won't result in massive object creations
(such as specifically converting an array to a list, which would 
be done much faster). Is this a serious problem?

These two issues led us to question whether we should indeed
return rank-0 arrays. We can live with either solution. But
we do want to make the right choice. We also know that both
functionalities must exist, e.g., indexing for scalars and 
indexing for rank-0 arrays and we will provide both. The issue
is what indexing syntax returns. One argument is that it is
not a great burden on programmers to use a method (or other
means) to obtain a rank-0 array always if that is important
for the code they are writing and that we should make the
indexing syntax return what most users (especially less expert
ones) intuitively expect (scalars I presume). But others feel
it is just as important for the syntax that a progammer uses
to be as simple as the interactive user expects (instead of
something like x.getindexasarrayalways(2,4,1) [well, with a much
better, and shorter, name])

Do either of these issues change anyone's opinion? If people
still want rank-0 arrays, what should repr do?

Perry Greenfield
 




From pearu at cens.ioc.ee  Fri Sep 13 00:20:03 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Fri Sep 13 00:20:03 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCCEEMCOAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.21.0209130903180.21821-100000@cens.kybi>

On Thu, 12 Sep 2002, Perry Greenfield wrote:

> If we return rank-0 arrays, what should repr return for rank-0
> arrays. My initial impression is that the following is highly
> undesirable for a interactive session, but maybe it is just me:
> 
> >>> x = arange(10)
> >>> x[2]
> array(2)
>
> We, of course, could arrange __repr__ to return "2" instead,
> in other words print the simple scalar for all cases of rank-0
> arrays. This would yield the expected output in the above
> example. Nevertheless, isn't it violating the intent of repr?
> Are there other examples where Python uses repr in a similar,
> misleading manner? But perhaps most feel that returning array(2)
> is perfectly acceptable and won't annoy users. I am curious
> about what people think about this.

I think it would be confusing if the result of repr would be `2' and not
`array(2)' because 2 and array(2) are not equivalent in all usages
but it should be clear from repr results as a first way to learn more
about the objects.

For example, if using array(2) as an index in Python native objects, then
TypeError is raised (as expected). In interactive session the quickest way
to check the type of a variable is just type its name and press enter:

>>> i
array(2)

Now, if repr(array(2)) returns '2', then one firstly assumes that
`i' is an integer. However, this would be very confusing if one sees
>>> some_list[i]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: sequence index must be integer
>>> i
2

So, I think that repr(array(2)) should return 'array(2)'. And users can
always change this behaviour locally by using sys.displayhook. Though, I
would recommend using tools like ipython for interactive sessions.

Btw, note that during an interactive session it would be *seemingly*
desired if also repr(string) would return str(string). For example,
when viewing documentation in interactive sessions. Wouldn't it be nice to
have

>>> sys.displayhook.__doc__
displayhook(object) -> None

Print an object to sys.stdout and also save it in __builtin__._

>>>

instead of the current behaviour:

>>> sys.displayhook.__doc__
'displayhook(object) -> None\n\nPrint an object to sys.stdout and also
save it in __builtin__._\n'
>>>


> The second issue is an efficiency one. Currently numarray uses
> Python objects for arrays. If we return rank-0 arrays for
> single item indexing, then some naive uses of larger arrays
> as sequences may lead to an enormous number of array objects
> to be created. True, there will be equivalent means of doing
> the same operation that won't result in massive object creations
> (such as specifically converting an array to a list, which would 
> be done much faster). Is this a serious problem?

Could array.__getitem_ and __getslice__ detect if their argument is
an array and skip using Python objects when iterating over indices?
If this is technically possible then it is not a good reason to drop
returning rank-0 arrays. The actual implementation may come later, though.

> If people still want rank-0 arrays, what should repr do?

Always return 'array(...)'.

You can also ask from python-dev for advice if numarray is considered to
be included to Python library in future. I am sure that repr issue will be
brought up if repr==str for 0-rank arrays.

Pearu




From huaiyu_zhu at yahoo.com  Fri Sep 13 01:46:03 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Fri Sep 13 01:46:03 2002
Subject: [Numpy-discussion] Tensor-like product with extra indices?
Message-ID: <Pine.LNX.4.33.0209130142080.1276-100000@mars.localdomain>

Two related questions:

1. Suppose  B = ravel(A) and
            B[i] corresponds to A[i0,i1,...].

   Are there named functions that map the indices

        (i0,i0,...) -> i

   and back?  This is assuming A.shape = (m0,m1,...) is known.

2. Suppose the above function is called ravel_index and its
   inverse unravel_index.

   Given a list of matrices [A0, A1, ...] is there a function that
   calculates a matrix B such that

        B[i,j]  = A0[i0,j] * A1[i1,j] * ...

   where i = ravel_index(i0, i1, ...).

   What about the inverse function that turns B into C such that

        C[i0,i1,...,j] = A0[i0,j] * A1[i1,j] * ...


I've considered reshape, ravel, take, put, outerproct but couldn't come
up with a combination to do it without a for-loop on j and a for-loop on
[A0,A1,...].

The above two questions I need answers right now.  But I can imagine a
need for a more general operations in the future.  Given a sequence of
arrays [A0,A1,...], produce a tensor product by picking certain indices
from each array (like i0,i1 above), while keeping some other indices
fixed (like j above).  For example, the above could be written as

        B = tensor_like([A0,A1...], [(0,None,None,...), ...,
                                (None,....,None,0), (1,1,...1)])

As another example, if len(A.shape)==3 then

        transpose(A) == tensor_like([A], [(2,), (1,), (0,)])


Huaiyu




From hinsen at cnrs-orleans.fr  Fri Sep 13 03:22:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 03:22:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.21.0209130903180.21821-100000@cens.kybi>
References: <Pine.LNX.4.21.0209130903180.21821-100000@cens.kybi>
Message-ID: <m3elbychut.fsf@chinon.cnrs-orleans.fr>

Pearu Peterson <pearu at cens.ioc.ee> writes:

> I think it would be confusing if the result of repr would be `2' and not
> `array(2)' because 2 and array(2) are not equivalent in all usages
> but it should be clear from repr results as a first way to learn more
> about the objects.

I agree. There will already be some inevitable confusion with both
rank-0 arrays and scalars around, with similar but not identical
behaviour. Rank-0 arrays shouldn't make it worse by using camouflage.

> > The second issue is an efficiency one. Currently numarray uses
> > Python objects for arrays. If we return rank-0 arrays for
> > single item indexing, then some naive uses of larger arrays
> > as sequences may lead to an enormous number of array objects
> > to be created. True, there will be equivalent means of doing
> > the same operation that won't result in massive object creations
> > (such as specifically converting an array to a list, which would 
> > be done much faster). Is this a serious problem?
> 
> Could array.__getitem_ and __getslice__ detect if their argument is
> an array and skip using Python objects when iterating over indices?

Of course they know that they are indexing an array, they are
defined at the level of the array class/type. However, they cannot
detect an iteration as opposed to a single item access.

I don't know if this efficiency problem could be important in
practice, probably only practice can tell. I have no idea how many
single-item indexing operations into arrays occur in my code, this is
not something I worried about when writing it.

If there will be scalar and rank-0 array returning variants of
indexing anyway, then I suppose that changing the index syntax
to one or the other is not a big effort. So my suggestion is to
make a test release and see what the reactions are.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Fri Sep 13 03:34:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 03:34:02 2002
Subject: [Numpy-discussion] Tensor-like product with extra indices?
In-Reply-To: <Pine.LNX.4.33.0209130142080.1276-100000@mars.localdomain>
References: <Pine.LNX.4.33.0209130142080.1276-100000@mars.localdomain>
Message-ID: <m3admmchab.fsf@chinon.cnrs-orleans.fr>

Huaiyu Zhu <huaiyu_zhu at yahoo.com> writes:

> Two related questions:
> 
> 1. Suppose  B = ravel(A) and
>             B[i] corresponds to A[i0,i1,...].
> 
>    Are there named functions that map the indices
> 
>         (i0,i0,...) -> i
> 
>    and back?  This is assuming A.shape = (m0,m1,...) is known.

Untested, but the idea should work:

  def ravel_index(array, *indices):
     offsets = Numeric.multiply.accumulate(array.shape[1:][::-1])[::-1]
     return Numeric.sum(Numeric.array(indices[:-1])*offsets) + indices[-1]

> 2. Suppose the above function is called ravel_index and its
>    inverse unravel_index.
> 
>    Given a list of matrices [A0, A1, ...] is there a function that
>    calculates a matrix B such that
> 
>         B[i,j]  = A0[i0,j] * A1[i1,j] * ...
> 
>    where i = ravel_index(i0, i1, ...).

Not for all i, which I assume is what you want to do.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From perry at stsci.edu  Fri Sep 13 07:06:14 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 07:06:14 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <m3elbychut.fsf@chinon.cnrs-orleans.fr>
Message-ID: <JFEGLNDJEDNOMPPHDEJFIEFAEAAA.perry@stsci.edu>

Konrad Hinsen writes:
>
> Pearu Peterson <pearu at cens.ioc.ee> writes:
>
> > I think it would be confusing if the result of repr would be `2' and not
> > `array(2)' because 2 and array(2) are not equivalent in all usages
> > but it should be clear from repr results as a first way to learn more
> > about the objects.
>
> I agree. There will already be some inevitable confusion with both
> rank-0 arrays and scalars around, with similar but not identical
> behaviour. Rank-0 arrays shouldn't make it worse by using camouflage.
>
I also agree that having repr hide the fact that it is an array is
a bad thing. But do you find getting "array(2)" when indexing
a single item acceptable when working interactively? I agree that
displayhook is probably the best way of altering this behavior if
a user desires (perhaps we will provide a module function to do so, but
not enable it by default).

> >
> > Could array.__getitem_ and __getslice__ detect if their argument is
> > an array and skip using Python objects when iterating over indices?
>
> Of course they know that they are indexing an array, they are
> defined at the level of the array class/type. However, they cannot
> detect an iteration as opposed to a single item access.
>
I don't know that it is desirable either to have iteration
return different kinds of objects than explicit indexing, even
if possible. Do we really want

for val in x:
    ...

to use scalars while

for i in range(len(x)):
    val = x[i]
    ...

uses rank-0 arrays? I'm inclined not to have different behavior for
what seems like identical iteration.

> I don't know if this efficiency problem could be important in
> practice, probably only practice can tell. I have no idea how many
> single-item indexing operations into arrays occur in my code, this is
> not something I worried about when writing it.
>
> If there will be scalar and rank-0 array returning variants of
> indexing anyway, then I suppose that changing the index syntax
> to one or the other is not a big effort. So my suggestion is to
> make a test release and see what the reactions are.
>
That is our plan (hence the reference to "experimental"), but I
wanted to see if there were strong feelings on this before doing so.

Perry




From jmiller at stsci.edu  Fri Sep 13 08:23:09 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 13 08:23:09 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <JFEGLNDJEDNOMPPHDEJFIEFAEAAA.perry@stsci.edu>
Message-ID: <3D81FBBF.9060703@stsci.edu>

Two other issues come up trying to implement the "rank-0 experiment":

1.  What should be the behavior of subscripting a rank-0 array?

a.  Return the scalar value                         (what numarray does 
now.  seems inconsistent)
b.  Raise an exception
c.  Return a copy  of the rank-0 array

2.  What's a decent notation for .asScalar()?

a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
does now)
b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
__call__)
c.  a[ <subscript_resulting_in_rank0> ].asScalar()  

Any strong opinions?

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / SSG






From hinsen at cnrs-orleans.fr  Fri Sep 13 08:40:06 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 08:40:06 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
References: <JFEGLNDJEDNOMPPHDEJFIEFAEAAA.perry@stsci.edu>
	<3D81FBBF.9060703@stsci.edu>
Message-ID: <m3ofb1c34o.fsf@chinon.cnrs-orleans.fr>

Todd Miller <jmiller at stsci.edu> writes:

> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray
> does now.  seems inconsistent)
> b.  Raise an exception
> c.  Return a copy  of the rank-0 array

I am for b). A rank-0 array is not a sequence and has no elements,
so indexing shouldn't be allowed.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From pearu at cens.ioc.ee  Fri Sep 13 08:48:07 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Fri Sep 13 08:48:07 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
Message-ID: <Pine.LNX.4.21.0209131842420.26238-100000@cens.kybi>

On Fri, 13 Sep 2002, Todd Miller wrote:

> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray does 
> now.  seems inconsistent)

-0.5

> b.  Raise an exception

+1

> c.  Return a copy  of the rank-0 array

-1

> 2.  What's a decent notation for .asScalar()?
> 
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
> does now)

+-0, just be consistent.

> b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
> __call__)

-1

> c.  a[ <subscript_resulting_in_rank0> ].asScalar()  

+0.5


Pearu




From xscottg at yahoo.com  Fri Sep 13 10:17:09 2002
From: xscottg at yahoo.com (Scott Gilbert)
Date: Fri Sep 13 10:17:09 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
Message-ID: <20020913171649.36428.qmail@web40106.mail.yahoo.com>

--- Todd Miller <jmiller at stsci.edu> wrote:
> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray does 
> now.  seems inconsistent)
>

If you think of it as "dereferencing" instead of "indexing", I think this
is most consistent.  It takes 3 elements to dereference a rank-3 array, so
it should take 0 elements to dereference a rank-0 array.


>
> 2.  What's a decent notation for .asScalar()?
> 
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
> does now)
> b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
> __call__)
> c.  a[ <subscript_resulting_in_rank0> ].asScalar()  
> 
> Any strong opinions?
> 

Since subscripting a rank-3 array:

  a[1, 2, 3] 

is very much like 

  a[(1, 2, 3)]

The __getitem__ receives the tuple (1, 2, 3) in both cases.  (Try it!)

So that would imply subscripting (dereferencing) a rank-0 array could be:

  a[] could be represented by a[()]

It's just unfortunate that Python doesn't currently recognize the a[] as a
valid syntax.




Cheers,
    -Scott






__________________________________________________
Do you Yahoo!?
Yahoo! News - Today's headlines
http://news.yahoo.com



From hinsen at cnrs-orleans.fr  Fri Sep 13 12:22:03 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 12:22:03 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <20020913171649.36428.qmail@web40106.mail.yahoo.com>
References: <20020913171649.36428.qmail@web40106.mail.yahoo.com>
Message-ID: <m33csdg0g8.fsf@localhost.localdomain>

Scott Gilbert <xscottg at yahoo.com> writes:

> > a.  Return the scalar value                         (what numarray does 
> > now.  seems inconsistent)
> 
> If you think of it as "dereferencing" instead of "indexing", I think this
> is most consistent.  It takes 3 elements to dereference a rank-3 array, so
> it should take 0 elements to dereference a rank-0 array.

That would be fine with me, but empty index brackets raise a
SyntaxError in Python.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From oliphant at ee.byu.edu  Fri Sep 13 12:59:04 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri Sep 13 12:59:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCCEEMCOAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.33L2.0209131354330.21399-100000@oliphant.ee.byu.edu>

>
> To restate the issue: there was a question about whether
> an index to an array that identified a single element only
> (i.e., not a slice, nor an incomplete index, e.g. x[3] where
> x is two dimensional) should return a Python scalar or a
> rank-0 array. Currently Numeric is inconsistent on this point.
> One usually gets scalars, but on some occasions, rank-0 arrays
> are returned. Good arguments are to be had for either alternative.
>

I think we should implement Python scalars for the other types and then
eliminate rank-0 arrays completely.  I could have a student do this in a
few weeks if it was agreed on.

-Travis Oliphant






From oliphant at ee.byu.edu  Fri Sep 13 13:00:07 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri Sep 13 13:00:07 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D81FBBF.9060703@stsci.edu>
Message-ID: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>

> Two other issues come up trying to implement the "rank-0 experiment":
>
> 1.  What should be the behavior of subscripting a rank-0 array?
>
> a.  Return the scalar value                         (what numarray does
> now.  seems inconsistent)
+0.25

> b.  Raise an exception
+1

> c.  Return a copy  of the rank-0 array
-1

>
> 2.  What's a decent notation for .asScalar()?
>
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray
> does now)
-1

Numeric does this now for backward compatibility.  I don't think this
issue was discussed at length.


> b.  a[ <subscript_resulting_in_rank0> ]()                  (override
> __call__)
-1

> c.  a[ <subscript_resulting_in_rank0> ].asScalar()
+1  just keep the current .toscalar()  notation.

Also int(a) and float(a) and complex(a) would work as well.

If we just implemented Python scalars for the other precisions then much
of this rank 0 array business would be solved as we could always return
the Python scalar rather than an array for rank 0 arrays.

I believe Konrad made this suggestion many moons ago and I would agree
with him.

-Travis Oliphant





From perry at stsci.edu  Fri Sep 13 13:15:09 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 13:15:09 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>
Message-ID: <JFEGLNDJEDNOMPPHDEJFEEFIEAAA.perry@stsci.edu>

> If we just implemented Python scalars for the other precisions then much
> of this rank 0 array business would be solved as we could always return
> the Python scalar rather than an array for rank 0 arrays.
> 
> I believe Konrad made this suggestion many moons ago and I would agree
> with him.
> 
> -Travis Oliphant
> 
Hi Travis,

refresh my memory about how this proposal would work. I've heard
proposals to add new types to Python itself, but that seems out of
the question. Are you talking about adding new scalar types as 
module? E.g.

>>> x = Int8(22) 
>>> arr = arange(10, typecode = Int8)
>>> arr[2]
Int8(2)

Or some other approach? 

In any case, doesn't this still have the problem that Eric complained
about, having to test for whether a result is an array or a scalar
(which was one of the drivers to produce rank-0 results).

Thanks, Perry



From tim.hochberg at ieee.org  Fri Sep 13 13:21:03 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 13 13:21:03 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <JFEGLNDJEDNOMPPHDEJFEEFIEAAA.perry@stsci.edu>
Message-ID: <137601c25b62$ba00c350$6c0b6244@cx781526b>



[Perry wrote about Travis's proposal]

> refresh my memory about how this proposal would work. I've heard
> proposals to add new types to Python itself, but that seems out of
> the question. Are you talking about adding new scalar types as
> module? E.g.
[snip]
> In any case, doesn't this still have the problem that Eric complained
> about, having to test for whether a result is an array or a scalar
> (which was one of the drivers to produce rank-0 results).

Hi Perry,

I like Travis's proposal the best of those I've seen so far, but I don't
recall the details of Eric's problem. Could you refresh us as to the basics
of it?.

-tim





From hinsen at cnrs-orleans.fr  Fri Sep 13 13:34:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 13 13:34:02 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>
References: <Pine.LNX.4.33L2.0209131350380.21399-100000@oliphant.ee.byu.edu>
Message-ID: <m3wuppeikt.fsf@localhost.localdomain>

Travis Oliphant <oliphant at ee.byu.edu> writes:

> If we just implemented Python scalars for the other precisions then much
> of this rank 0 array business would be solved as we could always return
> the Python scalar rather than an array for rank 0 arrays.
> 
> I believe Konrad made this suggestion many moons ago and I would agree

Right. I still think this is the best solution in that it is the least
confusing. If I remember correctly, the main objection was that
implementing all those types was not as trivial as hoped, but I forgot
the details (and it wasn't me who tried).

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From eric at enthought.com  Fri Sep 13 13:45:01 2002
From: eric at enthought.com (eric jones)
Date: Fri Sep 13 13:45:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <137601c25b62$ba00c350$6c0b6244@cx781526b>
Message-ID: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>

Hey Tim,

Here is a short summary:

Reductions and indexing return different types based on the number of
dimensions of the input array:

>>> b = sum(a)
>>> l = len(b) # or whatever

This code works happily if "a" is 2 or more dimensions, but will fail if
it is 1d because the sum(a) will return a scalar in this case.  To write
generic code, you have to put an if/else statement in to check whether b
is a scalar or an array:

>>> b = sum(a)
>>> if type(b) is ArrayType:
...     l = len(b)
... else:
...     l = 1 # or whatever you need

or less verbose but still unpleasant:  

>>> b = asarray(sum(a))
>>> l = len(b)

eric

> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-
> discussion-admin at lists.sourceforge.net] On Behalf Of Tim Hochberg
> Sent: Friday, September 13, 2002 3:18 PM
> To: numpy-discussion at lists.sourceforge.net
> Subject: Re: [Numpy-discussion] rank-0 arrays
> 
> 
> 
> 
> [Perry wrote about Travis's proposal]
> 
> > refresh my memory about how this proposal would work. I've heard
> > proposals to add new types to Python itself, but that seems out of
> > the question. Are you talking about adding new scalar types as
> > module? E.g.
> [snip]
> > In any case, doesn't this still have the problem that Eric
complained
> > about, having to test for whether a result is an array or a scalar
> > (which was one of the drivers to produce rank-0 results).
> 
> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I
don't
> recall the details of Eric's problem. Could you refresh us as to the
> basics
> of it?.
> 
> -tim
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion




From perry at stsci.edu  Fri Sep 13 13:46:08 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 13:46:08 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <137601c25b62$ba00c350$6c0b6244@cx781526b>
Message-ID: <JFEGLNDJEDNOMPPHDEJFOEFKEAAA.perry@stsci.edu>

> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I don't
> recall the details of Eric's problem. Could you refresh us as to 
> the basics
> of it?.
> 
> -tim
> 
Rather than put keystrokes on Eric's fingertips :-) I think it would
be best if he (or Paul Dubois, who had similar concerns) gave
some examples. But I believe it concerned fairly generic
Numeric routines that had expressions that might or might not
result in scalars. Testing for both cases resulted in 
extra code (Paul said that MA had to deal with this a lot) and
that having to deal with this made the code error prone.

But I'd rather they explained it since they have very specific
experience with this issue. Perhaps they can give a couple
common examples.

Perry



From paul at pfdubois.com  Fri Sep 13 14:07:01 2002
From: paul at pfdubois.com (Paul F Dubois)
Date: Fri Sep 13 14:07:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
Message-ID: <001a01c25b69$5aeadbb0$6901a8c0@NICKLEBY>

This is a perfect example of why the thing is so annoying. 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net 
> [mailto:numpy-discussion-admin at lists.sourceforge.net] On 
> Behalf Of eric jones
> Sent: Friday, September 13, 2002 1:44 PM
> To: 'Tim Hochberg'; numpy-discussion at lists.sourceforge.net
> Subject: RE: [Numpy-discussion] rank-0 arrays
> 
> 
> Hey Tim,
> 
> Here is a short summary:
> 
> Reductions and indexing return different types based on the 
> number of dimensions of the input array:
> 
> >>> b = sum(a)
> >>> l = len(b) # or whatever
> 
> This code works happily if "a" is 2 or more dimensions, but 
> will fail if it is 1d because the sum(a) will return a scalar 
> in this case.  To write generic code, you have to put an 
> if/else statement in to check whether b is a scalar or an array:
> 
> >>> b = sum(a)
> >>> if type(b) is ArrayType:
> ...     l = len(b)
> ... else:
> ...     l = 1 # or whatever you need
> 
> or less verbose but still unpleasant:  
> 
> >>> b = asarray(sum(a))
> >>> l = len(b)
> 
> eric
> 
> > -----Original Message-----
> > From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy- 
> > discussion-admin at lists.sourceforge.net] On Behalf Of Tim Hochberg
> > Sent: Friday, September 13, 2002 3:18 PM
> > To: numpy-discussion at lists.sourceforge.net
> > Subject: Re: [Numpy-discussion] rank-0 arrays
> > 
> > 
> > 
> > 
> > [Perry wrote about Travis's proposal]
> > 
> > > refresh my memory about how this proposal would work. I've heard 
> > > proposals to add new types to Python itself, but that 
> seems out of 
> > > the question. Are you talking about adding new scalar types as 
> > > module? E.g.
> > [snip]
> > > In any case, doesn't this still have the problem that Eric
> complained
> > > about, having to test for whether a result is an array or 
> a scalar 
> > > (which was one of the drivers to produce rank-0 results).
> > 
> > Hi Perry,
> > 
> > I like Travis's proposal the best of those I've seen so far, but I
> don't
> > recall the details of Eric's problem. Could you refresh us 
> as to the 
> > basics of it?.
> > 
> > -tim
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf 
> > _______________________________________________
> > Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 




From perry at stsci.edu  Fri Sep 13 14:07:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Sep 13 14:07:02 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <137601c25b62$ba00c350$6c0b6244@cx781526b>
Message-ID: <JFEGLNDJEDNOMPPHDEJFGEFMEAAA.perry@stsci.edu>

> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I don't
> recall the details of Eric's problem. Could you refresh us as to 
> the basics
> of it?.
> 
> -tim
> 
On a different front, I'm not sure that this proposal is very useful
for numarray because of the new scalar/array coercion rules that
numarray has. Since the new scalar literals must still use 
type functions (e.g.,, Int8(2)) it doesn't help make expressions
any prettier (or do I misunderstand the proposal?). In returning
scalars from arrays, I don't see that converting the array type
to a Python scalar type helps much. With the exception of long
doubles (and the corresponding complex precision), the conversion
to the python scalar doesn't lose any precision and you can 
restore the exact array type value by assigning that scalar value
back to an array element. As mentioned, the only exception is for
long doubles; for that case, a new scalar type makes sense.

Is there any other compelling reason for new scalar types?

Perry



From tim.hochberg at ieee.org  Fri Sep 13 14:25:14 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 13 14:25:14 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
Message-ID: <13a901c25b6b$9cd1ab10$6c0b6244@cx781526b>

Thanks for the summary Eric.

However, I don't find this example particularly compelling. Probably because
I don't think len(b) should work if b is rank zero (or possibly return
None). Yeah, I know it's worked in NumPy for years, but that doesn't mean I
have to like it. I would favor stripping out as much of the weird special
casing of rank-0 arrays as possible in the transition to numarray.

In the particular example given below, isn't the return value meaningless if
b is rank-0. Or rather only meaningful if  "or whatever you need" happens to
be one. This all smells rather arbitrary to me. However, this isn't a
problem that I run into and I'll concede that it's possible that returning
rank[0] arrays and then treating the rank zeros arrays almost like shape
(1,) arrays most of the time may solve more problems than it causes, but I'd
be interested in seeing more realistic examples. I guess I'll go poke around
MA and see what I can see. Any other suggestions of what to look at?

-tim



> Hey Tim,
>
> Here is a short summary:
>
> Reductions and indexing return different types based on the number of
> dimensions of the input array:
>
> >>> b = sum(a)
> >>> l = len(b) # or whatever
>
> This code works happily if "a" is 2 or more dimensions, but will fail if
> it is 1d because the sum(a) will return a scalar in this case.  To write
> generic code, you have to put an if/else statement in to check whether b
> is a scalar or an array:
>
> >>> b = sum(a)
> >>> if type(b) is ArrayType:
> ...     l = len(b)
> ... else:
> ...     l = 1 # or whatever you need
>
> or less verbose but still unpleasant:
>
> >>> b = asarray(sum(a))
> >>> l = len(b)
>
> eric
>
> > -----Original Message-----
> > From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-
> > discussion-admin at lists.sourceforge.net] On Behalf Of Tim Hochberg
> > Sent: Friday, September 13, 2002 3:18 PM
> > To: numpy-discussion at lists.sourceforge.net
> > Subject: Re: [Numpy-discussion] rank-0 arrays
> >
> >
> >
> >
> > [Perry wrote about Travis's proposal]
> >
> > > refresh my memory about how this proposal would work. I've heard
> > > proposals to add new types to Python itself, but that seems out of
> > > the question. Are you talking about adding new scalar types as
> > > module? E.g.
> > [snip]
> > > In any case, doesn't this still have the problem that Eric
> complained
> > > about, having to test for whether a result is an array or a scalar
> > > (which was one of the drivers to produce rank-0 results).
> >
> > Hi Perry,
> >
> > I like Travis's proposal the best of those I've seen so far, but I
> don't
> > recall the details of Eric's problem. Could you refresh us as to the
> > basics
> > of it?.
> >
> > -tim
> >
> >
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>





From oliphant at ee.byu.edu  Fri Sep 13 15:53:03 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri Sep 13 15:53:03 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFEEFIEAAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu>

> Hi Travis,
>
> refresh my memory about how this proposal would work. I've heard
> proposals to add new types to Python itself, but that seems out of
> the question. Are you talking about adding new scalar types as
> module? E.g.
>
> >>> x = Int8(22)
> >>> arr = arange(10, typecode = Int8)
> >>> arr[2]
> Int8(2)
>
> Or some other approach?

This is the gist of it.  Basically you extend the Python scalars to
include the single precision types (all the types Numeric supports).

Ultimately, in Python it would be nice if all of the scalars had the same
base class.

>
> In any case, doesn't this still have the problem that Eric complained
> about, having to test for whether a result is an array or a scalar
> (which was one of the drivers to produce rank-0 results).

I would have to understand his reason better.  He could be right.  My
reason for rank-0 results as always been more of a type issue.  I'll have
to ask him.

Now I remmber an issue that makes me question the proposal:

Currently the length of a rank-0 array is 1, while the length of a scalar
raises an error --- this is a bad difference.

We could still implement rank-0 arrays as a separate (optimized) type
though (so it doesn't carry around the extra baggage of full-rank arrays).

Now-wanting-rank-0-arrays-all-the-time,

-Travis





From huaiyu_zhu at yahoo.com  Sat Sep 14 00:25:02 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Sat Sep 14 00:25:02 2002
Subject: [Numpy-discussion] Tensor-like product with extra indices?
In-Reply-To: <m3admmchab.fsf@chinon.cnrs-orleans.fr>
Message-ID: <Pine.LNX.4.33.0209140020060.1213-100000@mars.localdomain>

On 13 Sep 2002, Konrad Hinsen wrote:
> 
> Untested, but the idea should work:
> 
>   def ravel_index(array, *indices):
>      offsets = Numeric.multiply.accumulate(array.shape[1:][::-1])[::-1]
>      return Numeric.sum(Numeric.array(indices[:-1])*offsets) + indices[-1]

Aagh!  :-)

> 
> > 2. Suppose the above function is called ravel_index and its
> >    inverse unravel_index.
> > 
> >    Given a list of matrices [A0, A1, ...] is there a function that
> >    calculates a matrix B such that
> > 
> >         B[i,j]  = A0[i0,j] * A1[i1,j] * ...
> > 
> >    where i = ravel_index(i0, i1, ...).
> 
> Not for all i, which I assume is what you want to do.

I end up using two for-loops, one over j and the other over [A0, ...].  
Hoping the speed problem will not be too bad later on.

Huaiyu

-- 
Huaiyu Zhu <huaiyu_zhu at yahoo.com>




From hinsen at cnrs-orleans.fr  Sun Sep 15 03:13:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Sun Sep 15 03:13:02 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
References: <000001c25b66$45b1fec0$6b01a8c0@ericlaptop>
Message-ID: <m3d6rfmuhy.fsf@localhost.localdomain>

"eric jones" <eric at enthought.com> writes:

> Reductions and indexing return different types based on the number of
> dimensions of the input array:
> 
> >>> b = sum(a)
> >>> l = len(b) # or whatever
> 
> This code works happily if "a" is 2 or more dimensions, but will fail if
> it is 1d because the sum(a) will return a scalar in this case.  To write

And it should fail, because a rank-0 array is not a sequence, so it
doesn't have a length.

But there are valid examples in which it would be nice if scalars
were arrays (but probably if  *all* scalars supported array operations,
not just those that were generated by indexing from arrays):

- a.shape  should return () for a scalar (and (len(a),) for any
           sequence type)

- a.astype(N.Float) should also work for scalars

Similarly, it would be nice if complex operations (real/imaginary
part) would work on integers and floats.

There's one more annoying difference between scalars and arrays of
any rank which I think should be removed in numarray:

  >>> 3 % -2
  -1
  >>> array(3) % 2
  1
  >>> fmod(3, -2)
  1.0

I.e. the mod operation uses fmod() for arrays, but different rules
for standard Python numbers.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Sun Sep 15 03:17:01 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Sun Sep 15 03:17:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFGEFMEAAA.perry@stsci.edu>
References: <JFEGLNDJEDNOMPPHDEJFGEFMEAAA.perry@stsci.edu>
Message-ID: <m38z23mubg.fsf@localhost.localdomain>

"Perry Greenfield" <perry at stsci.edu> writes:

> Is there any other compelling reason for new scalar types?

It solves the problem we are discussing (extracting array elements)
without introducing any incompatibility with existing code that uses
only the standard Python data types, and very little for code that
uses Float32 etc. I'd say it's the best compromise between
consistency and backwards compatibility.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From eric at enthought.com  Sun Sep 15 11:51:01 2002
From: eric at enthought.com (eric jones)
Date: Sun Sep 15 11:51:01 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <m3d6rfmuhy.fsf@localhost.localdomain>
Message-ID: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>

Hey Konrad,

> 
> "eric jones" <eric at enthought.com> writes:
> 
> > Reductions and indexing return different types based on the number
of
> > dimensions of the input array:
> >
> > >>> b = sum(a)
> > >>> l = len(b) # or whatever
> >
> > This code works happily if "a" is 2 or more dimensions, but will
fail if
> > it is 1d because the sum(a) will return a scalar in this case.  To
write
> 
> And it should fail, because a rank-0 array is not a sequence, so it
> doesn't have a length.
> 

I disagree. You should not have to write special code to check for a
specific case.  It breaks one of the beauties of Numeric -- i.e. you can
write generic code that handles arrays of any size and type.  Any method
that works on a 1 or more d array should also work on 0d arrays.  If you
ask for its shape, it returns a tuple.  If you ask for its size it
returns its length along its "first" axis.  This will always be 1.  It
allows for generic code.

On this note:
I do not see the benefit of making a scalar type object that is separate
for 0d arrays.  It seems to remove instead of enhance capabilities.
What does a scalar object buy that simply using 0d arrays for that
purpose does not?

> 
> But there are valid examples in which it would be nice if scalars
> were arrays (but probably if  *all* scalars supported array
operations,
> not just those that were generated by indexing from arrays):
> 
> - a.shape  should return () for a scalar (and (len(a),) for any
>            sequence type)
> 
> - a.astype(N.Float) should also work for scalars
> 
> Similarly, it would be nice if complex operations (real/imaginary
> part) would work on integers and floats.

Yes, this is needed.  And I think the argument for it is similar as
having len() work on 0d arrays.  It allows for generic code.

> 
> There's one more annoying difference between scalars and arrays of
> any rank which I think should be removed in numarray:
> 
>   >>> 3 % -2
>   -1
>   >>> array(3) % 2
>   1
>   >>> fmod(3, -2)
>   1.0
> 
> I.e. the mod operation uses fmod() for arrays, but different rules
> for standard Python numbers.

I think you meant, 

>   >>> array(3) % -2
>   1

That is unfortunate.  It would be nice to clean this up.

eric




From hinsen at cnrs-orleans.fr  Sun Sep 15 12:41:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Sun Sep 15 12:41:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>
References: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>
Message-ID: <200209151940.g8FJejs03609@localhost.localdomain>

> > And it should fail, because a rank-0 array is not a sequence, so it
> > doesn't have a length.
> 
> I disagree. You should not have to write special code to check for a
> specific case.  It breaks one of the beauties of Numeric -- i.e. you can

It is not a specific case, more like a specific value (for the rank).
1/a fails for a == 0, should that be changed as well?

Let's examine some equivalent code pieces:

- len(a) == a.shape[0]
  the second fails for rank 0, so the first one should fail as well

- for i in range(len(a)):  print a[i]
  works for all sequences. If len(a) doesn't fail (and I assume it would
  then return 1), a[1] shouldn't fail either.

- len(a) == len(list(a))
  for all sequences a. Should list(a) return [a] for a rank-0 array?
  For a scalar it fails.

Actually this might be an argument for not having rank-0 arrays at all.
Arrays are multidimensional sequences, but rank-0 arrays aren't.

> returns its length along its "first" axis.  This will always be 1.  It
> allows for generic code.

Then please give an example where this genericity would be useful.

> On this note:
> I do not see the benefit of making a scalar type object that is separate
> for 0d arrays.  It seems to remove instead of enhance capabilities.
> What does a scalar object buy that simply using 0d arrays for that
> purpose does not?

Compatibility, for example the ability to index a sequence with an
element of an integer array. Also consistency with other Python
sequence types. For example,

   [a][0] == a

so one would expect also

   array([a])[0] == a

but this would not be fully true if the left-hand side is a rank-0
array.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From tim.hochberg at ieee.org  Sun Sep 15 14:30:04 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun Sep 15 14:30:04 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <000401c25ce8$b0fec590$777ba8c0@ericlaptop>
Message-ID: <145001c25cfe$b95db550$6c0b6244@cx781526b>

----- Original Message -----
From: "eric jones" <eric at enthought.com>

[Konrad]
> > And it should fail, because a rank-0 array is not a sequence, so it
> > doesn't have a length.

[Eric]
> I disagree. You should not have to write special code to check for a
> specific case.  It breaks one of the beauties of Numeric -- i.e. you can
> write generic code that handles arrays of any size and type.  Any method
> that works on a 1 or more d array should also work on 0d arrays.  If you
> ask for its shape, it returns a tuple.

No problem up to here.

> If you ask for its size it
> returns its length along its "first" axis.

Here's where we part ways. As Konrad already pointed out, it return
anArray.shape[0] except in the case of zero-D arrays where the arbitrary
decision was made that:

> This will always be 1.

Why 1 and not 0 or -1 or 42. If you really had to return something, the
thing to return would be None.

> It allows for generic code.

I don't see how. Even poking around in MA didn't convince me that this would
help although I didn't spend enough time with it to get a great feel for it.
The one function I did come close to working all the way through looked like
it would be about _half_ as long if it didn't have to support the zero-rank
cruft.

> On this note:
> I do not see the benefit of making a scalar type object that is separate
> for 0d arrays.  It seems to remove instead of enhance capabilities.
> What does a scalar object buy that simply using 0d arrays for that
> purpose does not?

Unless somethings changed, you can't index into lists and what not with a
rank-0 array, so returning ints or some subclass from integer arrays would
be convenient. It would be easy to always return subclasses of int, float or
complex (or object for those few who use object arrays) so that the results
would always play nice with the rest of Python.

However, given that the coercion rules have changed in numarray, I don't
really see the point of returning anything other than int, float of complex.
However, I have no objection to allowing the creation rank-0 arrays as long
as they behave consistently with other array objects.

[Konrad]
> > - a.astype(N.Float) should also work for scalars
> >
> > Similarly, it would be nice if complex operations (real/imaginary
> > part) would work on integers and floats.

If these are needed, and I agree they would be nice, it seems that in order
to integrate well with the rest of Python we should supply:

numarray.astype(a, type) -> array or scalar of type type.
numarray.imaginary(a) -> imaginary part of a
numarray.imaginary(a) -> real part of a

I actually had these latter two in JNumeric back when I was working on that,
so I kinda thought they were Numeric, but I musta just added them in because
I liked them.

[Eric and Konrad agree that the inconsistency between Python and Numeric's
mod should be cleaned up]

Me too.

-tim





From Chris.Barker at noaa.gov  Mon Sep 16 11:46:05 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Mon Sep 16 11:46:05 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu>
Message-ID: <3D862463.BB6EED2C@noaa.gov>


Travis Oliphant wrote:

> This is the gist of it.  Basically you extend the Python scalars to
> include the single precision types (all the types Numeric supports).

Would they be recognised as scalars by Python? In particular, could you
use one as an index? Personally, this is what has bit me in the past: I
could use A[3,2] as an index if A was type "Int" but not if it was
"Int16" for example.

In any case, the type of A[3,2] should NOT depend on the precision of
the numbers stored in A.

Frankly, I have no idea what the implimentation details would be, but
could we get rid of rank-0 arrays altogether? I have always simply found
them strange and confusing... What are they really neccesary for
(besides holding scalar values of different precision that standard
Pyton scalars)?

-Chris







-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From hinsen at cnrs-orleans.fr  Mon Sep 16 12:18:14 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Sep 16 12:18:14 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D862463.BB6EED2C@noaa.gov>
References: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu>
	<3D862463.BB6EED2C@noaa.gov>
Message-ID: <m3admhkalh.fsf@localhost.localdomain>

"Chris Barker" <Chris.Barker at noaa.gov> writes:

> Travis Oliphant wrote:
> 
> > This is the gist of it.  Basically you extend the Python scalars to
> > include the single precision types (all the types Numeric supports).
> 
> Would they be recognised as scalars by Python? In particular, could you

There is no "scalar" category in Python. New scalar datatypes would be
types with the same behaviour as the existing Python scalar types, but
different from the. That means that explicitly type-checking code would
not accept them, but everything else would.

> use one as an index? Personally, this is what has bit me in the past: I

At least up to Python 1.5, no, indices have to be of integer type. I
don't know if that condition was extended in later versions.

> could use A[3,2] as an index if A was type "Int" but not if it was
> "Int16" for example.

Ehmmm... Are you sure that is the right example? The restriction is on
the type of the index, not on the type of the array.

> Frankly, I have no idea what the implimentation details would be, but
> could we get rid of rank-0 arrays altogether? I have always simply found

If we introduce additional scalars, yes. Whether or not we would want
to get rid of them is of course another question.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From tim.hochberg at ieee.org  Mon Sep 16 12:39:01 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 16 12:39:01 2002
Subject: [Numpy-discussion] rank-0 arrays
References: <Pine.LNX.4.33L2.0209131638520.21637-100000@oliphant.ee.byu.edu><3D862463.BB6EED2C@noaa.gov> <m3admhkalh.fsf@localhost.localdomain>
Message-ID: <156101c25db8$40872700$6c0b6244@cx781526b>

From: "Konrad Hinsen" <hinsen at cnrs-orleans.fr>
> > Travis Oliphant wrote:
> > Would they be recognised as scalars by Python? In particular, could you
>
> There is no "scalar" category in Python. New scalar datatypes would be
> types with the same behaviour as the existing Python scalar types, but
> different from the. That means that explicitly type-checking code would
> not accept them, but everything else would.
>
> > use one as an index? Personally, this is what has bit me in the past: I
>
> At least up to Python 1.5, no, indices have to be of integer type. I
> don't know if that condition was extended in later versions.

In python 2.2 (I'm not sure about 2.0,2.1), you can subclass int and the
resultant class can be used as an index. So I think this could be done.

> > Frankly, I have no idea what the implimentation details would be, but
> > could we get rid of rank-0 arrays altogether? I have always simply found
>
> If we introduce additional scalars, yes.

Given that numarray has changed its coercion rules, is this still necessary?
>From http://stsdas.stsci.edu/numarray/Userguide.html:

----------start quote----------
*************
Type Coercion
*************

In expressions involving only arrays, the normal coercion rules
apply (i.e., the same as Numeric). However, the same rules do
not apply to binary operations between arrays and Python scalars
in certain cases. If the kind of number is the same for the array
and scalar (e.g., both are integer types or both are float types),
then the type of the output is determined by the precision of the
array, not the scalar. Some examples will best illustrate:

Scalar type  *  Array type        Numeric result type    numarray result
type
   Int            Int16                Int32                    Int16
   Int            Int8                 Int32                    Int8
   Float          Int8                 Float64                  Float64
   Float          Float32              Float64                  Float32

The change in the rules was made so that it would be easy to preserve the
precision of arrays in expressions involving scalars. Previous solutions
with Numeric were either quite awkward (using a function to create a rank-0
array from a scalar with the desired type) or surprising (the savespace
attribute, that never allowed type coercion). The problem arises because
Python has a limited selection of scalar types. This appears to be the
best solution though it admittedy may surprise some who are used to the
classical type coercion model.

----------end quote----------


> Whether or not we would want
> to get rid of them is of course another question.

Agreed. It's possible that rank zero arrays would be useful in certain
unusual situations, although I can't think of any good examples. Regardless
of that choice, if indexing never returns rank-0 arrays most people will
never have to deal with them.

-tim






From tim.hochberg at ieee.org  Mon Sep 16 13:15:01 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 16 13:15:01 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
Message-ID: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>

Can someone refresh my memory as to why some properties of NumArrays that
are conceptually attributes are accessed as atributes (a.shape, a.rank),
while others are accessed through functions (a.type, a.iscontiguous,...).

Thanks,

-tim





From paul at pfdubois.com  Mon Sep 16 13:27:04 2002
From: paul at pfdubois.com (Paul F Dubois)
Date: Mon Sep 16 13:27:04 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
In-Reply-To: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>
Message-ID: <000001c25dbf$5f0dc5b0$6901a8c0@NICKLEBY>

Because that is what Numeric did, and as explained in the position
statement on the numpy website, we are not changing things unless we
have a compelling reason. 

Numeric did some things like a.shape as properties rather than pairs of
get/set functions; there was no special reason for it except a feeling
that it was easier to type or looked more mathematical. E.g.,
x.imaginary. I think the driving reason for shape was interactive use:
x.shape=(4,5)

However we got here, we aren't going to debate changing any of this. The
scalar issue is on the table because it causes actual trouble for real
people rather than philosophical inconsistency. 
> 
> 
> Can someone refresh my memory as to why some properties of 
> NumArrays that are conceptually attributes are accessed as 
> atributes (a.shape, a.rank), while others are accessed 
> through functions (a.type, a.iscontiguous,...).
> 
> Thanks,
> 
> -tim




From tim.hochberg at ieee.org  Mon Sep 16 13:43:04 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 16 13:43:04 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
References: <000001c25dbf$5f0dc5b0$6901a8c0@NICKLEBY>
Message-ID: <159301c25dc1$4c4ab170$6c0b6244@cx781526b>


[Me (Tim Hochberg)]

> > Can someone refresh my memory as to why some properties of
> > NumArrays that are conceptually attributes are accessed as
> > atributes (a.shape, a.rank), while others are accessed
> > through functions (a.type, a.iscontiguous,...).

[Paul F Dubois]

> Because that is what Numeric did, and as explained in the position
> statement on the numpy website, we are not changing things unless we
> have a compelling reason.

Fair enough, but what about attributes new to numarray (rank, type, maybe
others). Is the preference then for function syntax over attribute syntax?
I'd prefer the latter, but I'm mostly trying to figure out what the pattern
is.

-tim





From perry at stsci.edu  Mon Sep 16 14:48:06 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Sep 16 14:48:06 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <m3admhkalh.fsf@localhost.localdomain>
Message-ID: <JFEGLNDJEDNOMPPHDEJFMEGHEAAA.perry@stsci.edu>

Konrad Hinsen writes:
> "Chris Barker" <Chris.Barker at noaa.gov> writes:
> 
> > use one as an index? Personally, this is what has bit me in the past: I
> 
> At least up to Python 1.5, no, indices have to be of integer type. I
> don't know if that condition was extended in later versions.
> 
> > could use A[3,2] as an index if A was type "Int" but not if it was
> > "Int16" for example.
> 
> Ehmmm... Are you sure that is the right example? The restriction is on
> the type of the index, not on the type of the array.
> 
I think Chris is referring to the fact that Numeric returns a rank-0
array for A[3,2] if A is of type Int16, and that value cannot be used
as in index in Python sequences (at least for now; nothing technical
prevents it from being implemented to accept object that have an
__int__ method). This is the odd inconsistency Numeric has now. If
A were a 1-d Int16 array then A[2] would not be a rank-0 array,
nor is A[3,2] a rank-0 array if A is of type Int32 (apparently because
a Python scalar type suffices). Why it gives rank-0 for 2-d and not
1-d I have no idea.
> 



From oliphant at ee.byu.edu  Mon Sep 16 15:51:04 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Mon Sep 16 15:51:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <3D862463.BB6EED2C@noaa.gov>
Message-ID: <Pine.LNX.4.33L2.0209161643190.20912-100000@oliphant.ee.byu.edu>

>
> Travis Oliphant wrote:
>
> > This is the gist of it.  Basically you extend the Python scalars to
> > include the single precision types (all the types Numeric supports).
>
> Would they be recognised as scalars by Python? In particular, could you
> use one as an index? Personally, this is what has bit me in the past: I
> could use A[3,2] as an index if A was type "Int" but not if it was
> "Int16" for example.
>
> In any case, the type of A[3,2] should NOT depend on the precision of
> the numbers stored in A.
>
> Frankly, I have no idea what the implimentation details would be, but
> could we get rid of rank-0 arrays altogether? I have always simply found
> them strange and confusing... What are they really neccesary for
> (besides holding scalar values of different precision that standard
> Pyton scalars)?

With new coercion rules this becomes a possibility.  Arguments against it
are that special rank-0 arrays behave as more consistent numbers with the
rest of Numeric than Python scalars.  In other words they have a length
and a shape and one can right N-dimensional code that works the same even
when the result is a scalar.

Another advantage of having a Numeric scalar is that we can control the
behavior of floating point operations better.

e.g.

if only Python scalars were available and sum(a) returned 0, then

 1 / sum(a)  would behave as Python behaves (always raises error).

while with our own scalars

1 / sum(a)   could potentially behave however the user wanted.


-Travis







From pearu at cens.ioc.ee  Mon Sep 16 16:10:04 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Mon Sep 16 16:10:04 2002
Subject: [Numpy-discussion] rank-0 arrays
In-Reply-To: <Pine.LNX.4.33L2.0209161643190.20912-100000@oliphant.ee.byu.edu>
Message-ID: <Pine.LNX.4.21.0209170202150.1695-100000@cens.kybi>

On Mon, 16 Sep 2002, Travis Oliphant wrote:

> > Frankly, I have no idea what the implimentation details would be, but
> > could we get rid of rank-0 arrays altogether? I have always simply found
> > them strange and confusing... What are they really neccesary for
> > (besides holding scalar values of different precision that standard
> > Pyton scalars)?
> 
> With new coercion rules this becomes a possibility.  Arguments against it
> are that special rank-0 arrays behave as more consistent numbers with the
> rest of Numeric than Python scalars.  In other words they have a length
> and a shape and one can right N-dimensional code that works the same even
> when the result is a scalar.

In addition, rank-0 arrays are mutable while Python scalars are not.
Mutability is sometimes useful (e.g. when emulating C or Fortran
calls in Python) but often also evil due to its unpythonic side effect.

Pearu




From huaiyu_zhu at yahoo.com  Mon Sep 16 23:40:05 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Mon Sep 16 23:40:05 2002
Subject: [Numpy-discussion] numarray bug: sum, product, alltrue, sometrue
Message-ID: <Pine.LNX.4.33.0209162338270.1534-100000@mars.localdomain>

There is a bug in the dimension-reducing functions (sum, product,
alltrue, sometrue) in numarray: The outside swapaxes should take into
account that one axis is reduced.  The expected behavior should be:

>>> from numarray import *
>>> a = ones((2,3,4))
>>> sum(a, axis=0).shape
(3, 4)
>>> sum(a, axis=1).shape
(2, 4)
>>> sum(a, axis=2).shape
(2, 3)

A patch is attached below.

Huaiyu

*** numarray.py~	Tue Jul  2 11:12:24 2002
--- numarray.py	Mon Sep 16 23:17:27 2002
***************
*** 995,997 ****
      else:
!         return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis)
  
--- 995,997 ----
      else:
!         return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis-1)
  
***************
*** 1014,1016 ****
          return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)),
!                         0, axis)
  
--- 1014,1016 ----
          return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)),
!                         0, axis-1)
  
***************
*** 1034,1036 ****
          return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)),
!                         0, axis)
  def sometrue(a, axis=0):
--- 1034,1036 ----
          return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)),
!                         0, axis-1)
  def sometrue(a, axis=0):
***************
*** 1043,1045 ****
          return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)),
!                         0, axis)
  
--- 1043,1045 ----
          return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)),
!                         0, axis-1)
  





From huaiyu_zhu at yahoo.com  Tue Sep 17 00:05:01 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Tue Sep 17 00:05:01 2002
Subject: [Numpy-discussion] s are mutable scalars
Message-ID: <Pine.LNX.4.33.0209162358310.1534-100000@mars.localdomain>

Some of the choices between rank-0 arrays and new scalar types
might be resolved by enumerating the properties desired of them.

Most properties of rank-0 arrays could be fixed by consistency
requirements alone, using operations that reduce array dimensions.

Let a = ones((2,3,4))
b = sum(a)
c = sum(b)
d = sum(c)

Property 1: the shape of an array is a tuple of integers
	a.shape == (2, 3, 4)
	b.shape == (3, 4)
	c.shape == (4,)
	d.shape == ()
	
Property 2: rank(a) == len(a.shape)
	rank(a) == 3 == len(a.shape)
	rank(b) == 2 == len(b.shape)
	rank(c) == 1 == len(c.shape)
	rank(d) == 0 == len(d.shape)

Property 3: len(a) == a.shape[0]
	len(a) == 2 == a.shape[0]
	len(b) == 3 == b.shape[0]
	len(c) == 4 == c.shape[0]
	len(d) == Exception == d.shape[0]

	# Currently the last is wrong?

Property 4: size(a) == product(a.shape)
	size(a) == 24 == product(a.shape)
	size(b) == 12 == product(b.shape)
	size(c) == 4 == product(c.shape)
	size(d) == 1 == product(d.shape)

	# Currently the last is wrong

Property 5: rank-0 array behaves as mutable numbers when used as value
	array(2)  is similar to 2
	array(2.0)  is similar to 2.0
	array(2j)  is similar to 2j

	# This is a summary of many concrete properties.

Property 6: Indexing reduces rank. Slicing preserves rank.
	a[:,:,:].shape = (2, 3, 4)
	a[1,:,:].shape = (3, 4)
	a[1,1,:].shape = (4,)
	a[1,1,1].shape = ()

Property 7: Indexing by tuple of ints gives scalar.
	a[1,1,1] == 1
	b[1,1] == 2
	c[1,] == 6
	d[()] == 24

	# So rank-0 array indexed by empty tuple should be scalar.
        # Currently the last is wrong

Property 8: Indexing by tuple of slices gives array.
	a[:,:,:] == ones((2,3,4))
	b[:,:] == ones((3,4)) * 2
	c[:] == ones((,4)) * 6
	d[()] == ones(()) * 24

	# So rank-0 array indexed by empty tuple should be rank-0 array.
        # Currently the last is wrong

Property 9: Indexing as lvalues
	a[1,1,1] = 2
	b[1,1] = 2
	c[1,] = 2
	d[()] = 2

Property 10: Indexing and slicing as lvalues
	a[:,:,:] = ones((2, 3, 4))
	a[1,:,:] = ones((3, 4))
	a[1,1,:] = ones((4,))
	a[1,1,1] = ones(())
	
	# But the last is wrong.


Conclusion 1: rank-0 arrays are equivalent to scalars.
	See properties 7 and 8.

Conclusion 2: rank-0 arrays are mutable.
	See property 9.

Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
	should not be defined.

	See conclusion 1 and properties 1, 2, 3, 4.

Conclusion 4: A missing axis is similar to having dimension 1.
	See property 4.

Conclusion 5: rank-0 int arrays should be allowed to act as indices.
	See property 5.

Conclusion 6: rank-0 arrays should not be hashable except by object id.
	See conclusion 2.


Discussions:

  - These properties correspond to the current implementation quite well,
	except a few rough edges.

  - Mutable scalars are useful in their own rights.

  - Is there substantial difference in overhead between rank-0 arrays and
    scalars?

  - How to write literal values?  array(1) is too many characters.

  - For rank-1 and rank-0 arrays, Python notation distinguishes:

	c[1] vs c[1,]
	d[] vs d[()]

    Should these be used to handle semantic difference between indexing 
    and slicing?  Should d[] be syntactically allowed?

Hope these observations help.

Huaiyu




From huaiyu_zhu at yahoo.com  Tue Sep 17 00:12:04 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Tue Sep 17 00:12:04 2002
Subject: [Numpy-discussion] rank-0 arrays are mutable scalars
In-Reply-To: <Pine.LNX.4.33.0209162358310.1534-100000@mars.localdomain>
Message-ID: <Pine.LNX.4.33.0209170008490.1611-100000@mars.localdomain>

Oops, the subject line was somehow cut off.  Please use this one if you 
follow up. - Huaiyu

On Tue, 17 Sep 2002, Huaiyu Zhu wrote:

Some of the choices between rank-0 arrays and new scalar types
might be resolved by enumerating the properties desired of them.

Most properties of rank-0 arrays could be fixed by consistency
requirements alone, using operations that reduce array dimensions.

Let a = ones((2,3,4))
b = sum(a)
c = sum(b)
d = sum(c)

Property 1: the shape of an array is a tuple of integers
	a.shape == (2, 3, 4)
	b.shape == (3, 4)
	c.shape == (4,)
	d.shape == ()
	
Property 2: rank(a) == len(a.shape)
	rank(a) == 3 == len(a.shape)
	rank(b) == 2 == len(b.shape)
	rank(c) == 1 == len(c.shape)
	rank(d) == 0 == len(d.shape)

Property 3: len(a) == a.shape[0]
	len(a) == 2 == a.shape[0]
	len(b) == 3 == b.shape[0]
	len(c) == 4 == c.shape[0]
	len(d) == Exception == d.shape[0]

	# Currently the last is wrong?

Property 4: size(a) == product(a.shape)
	size(a) == 24 == product(a.shape)
	size(b) == 12 == product(b.shape)
	size(c) == 4 == product(c.shape)
	size(d) == 1 == product(d.shape)

	# Currently the last is wrong

Property 5: rank-0 array behaves as mutable numbers when used as value
	array(2)  is similar to 2
	array(2.0)  is similar to 2.0
	array(2j)  is similar to 2j

	# This is a summary of many concrete properties.

Property 6: Indexing reduces rank. Slicing preserves rank.
	a[:,:,:].shape = (2, 3, 4)
	a[1,:,:].shape = (3, 4)
	a[1,1,:].shape = (4,)
	a[1,1,1].shape = ()

Property 7: Indexing by tuple of ints gives scalar.
	a[1,1,1] == 1
	b[1,1] == 2
	c[1,] == 6
	d[()] == 24

	# So rank-0 array indexed by empty tuple should be scalar.
        # Currently the last is wrong

Property 8: Indexing by tuple of slices gives array.
	a[:,:,:] == ones((2,3,4))
	b[:,:] == ones((3,4)) * 2
	c[:] == ones((,4)) * 6
	d[()] == ones(()) * 24

	# So rank-0 array indexed by empty tuple should be rank-0 array.
        # Currently the last is wrong

Property 9: Indexing as lvalues
	a[1,1,1] = 2
	b[1,1] = 2
	c[1,] = 2
	d[()] = 2

Property 10: Indexing and slicing as lvalues
	a[:,:,:] = ones((2, 3, 4))
	a[1,:,:] = ones((3, 4))
	a[1,1,:] = ones((4,))
	a[1,1,1] = ones(())
	
	# But the last is wrong.


Conclusion 1: rank-0 arrays are equivalent to scalars.
	See properties 7 and 8.

Conclusion 2: rank-0 arrays are mutable.
	See property 9.

Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
	should not be defined.

	See conclusion 1 and properties 1, 2, 3, 4.

Conclusion 4: A missing axis is similar to having dimension 1.
	See property 4.

Conclusion 5: rank-0 int arrays should be allowed to act as indices.
	See property 5.

Conclusion 6: rank-0 arrays should not be hashable except by object id.
	See conclusion 2.


Discussions:

  - These properties correspond to the current implementation quite well,
	except a few rough edges.

  - Mutable scalars are useful in their own rights.

  - Is there substantial difference in overhead between rank-0 arrays and
    scalars?

  - How to write literal values?  array(1) is too many characters.

  - For rank-1 and rank-0 arrays, Python notation distinguishes:

	c[1] vs c[1,]
	d[] vs d[()]

    Should these be used to handle semantic difference between indexing 
    and slicing?  Should d[] be syntactically allowed?

Hope these observations help.

Huaiyu






From oliphant at ee.byu.edu  Tue Sep 17 09:10:05 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue Sep 17 09:10:05 2002
Subject: [Numpy-discussion] s are mutable scalars
In-Reply-To: <Pine.LNX.4.33.0209162358310.1534-100000@mars.localdomain>
Message-ID: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>

>
> Some of the choices between rank-0 arrays and new scalar types
> might be resolved by enumerating the properties desired of them.
>

These are helpful observations.

> Most properties of rank-0 arrays could be fixed by consistency
> requirements alone, using operations that reduce array dimensions.
>
> Let a = ones((2,3,4))
> b = sum(a)
> c = sum(b)
> d = sum(c)
>
> Property 1: the shape of an array is a tuple of integers
> 	a.shape == (2, 3, 4)
> 	b.shape == (3, 4)
> 	c.shape == (4,)
> 	d.shape == ()
>
> Property 2: rank(a) == len(a.shape)
> 	rank(a) == 3 == len(a.shape)
> 	rank(b) == 2 == len(b.shape)
> 	rank(c) == 1 == len(c.shape)
> 	rank(d) == 0 == len(d.shape)
>
> Property 3: len(a) == a.shape[0]
> 	len(a) == 2 == a.shape[0]
> 	len(b) == 3 == b.shape[0]
> 	len(c) == 4 == c.shape[0]
> 	len(d) == Exception == d.shape[0]
>
> 	# Currently the last is wrong?

Agreed, but this is because d is an integer and out of Numerics control.
This is a case for returning 0d arrays rather than Python scalars.

>
> Property 4: size(a) == product(a.shape)
> 	size(a) == 24 == product(a.shape)
> 	size(b) == 12 == product(b.shape)
> 	size(c) == 4 == product(c.shape)
> 	size(d) == 1 == product(d.shape)
>
> 	# Currently the last is wrong

I disagree that this is wrong.  This works as described for me.

>
> Property 5: rank-0 array behaves as mutable numbers when used as value
> 	array(2)  is similar to 2
> 	array(2.0)  is similar to 2.0
> 	array(2j)  is similar to 2j
>
> 	# This is a summary of many concrete properties.
>
> Property 6: Indexing reduces rank. Slicing preserves rank.
> 	a[:,:,:].shape = (2, 3, 4)
> 	a[1,:,:].shape = (3, 4)
> 	a[1,1,:].shape = (4,)
> 	a[1,1,1].shape = ()
>
> Property 7: Indexing by tuple of ints gives scalar.
> 	a[1,1,1] == 1
> 	b[1,1] == 2
> 	c[1,] == 6
> 	d[()] == 24
>
> 	# So rank-0 array indexed by empty tuple should be scalar.
>         # Currently the last is wrong

Not sure about this property, but interesting.

>
> Property 8: Indexing by tuple of slices gives array.
> 	a[:,:,:] == ones((2,3,4))
> 	b[:,:] == ones((3,4)) * 2
> 	c[:] == ones((,4)) * 6
> 	d[()] == ones(()) * 24
>
> 	# So rank-0 array indexed by empty tuple should be rank-0 array.
>         # Currently the last is wrong

Not sure about this one either.
>
> Property 9: Indexing as lvalues
> 	a[1,1,1] = 2
> 	b[1,1] = 2
> 	c[1,] = 2
> 	d[()] = 2
>
> Property 10: Indexing and slicing as lvalues
> 	a[:,:,:] = ones((2, 3, 4))
> 	a[1,:,:] = ones((3, 4))
> 	a[1,1,:] = ones((4,))
> 	a[1,1,1] = ones(())
>
> 	# But the last is wrong.
>
>
> Conclusion 1: rank-0 arrays are equivalent to scalars.
> 	See properties 7 and 8.
>
> Conclusion 2: rank-0 arrays are mutable.
> 	See property 9.
>
> Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
> 	should not be defined.

Why is this?  I thought you argued the other way for len(scalar).  Of
course, one solution is that we could overwrite the len() function and
allow it to work for scalars.

>
> 	See conclusion 1 and properties 1, 2, 3, 4.
>
> Conclusion 4: A missing axis is similar to having dimension 1.
> 	See property 4.
>
> Conclusion 5: rank-0 int arrays should be allowed to act as indices.
> 	See property 5.

Can't do this for lists and other builtin sequences.
>
> Conclusion 6: rank-0 arrays should not be hashable except by object id.
> 	See conclusion 2.
>
>
> Discussions:
>
>   - These properties correspond to the current implementation quite well,
> 	except a few rough edges.
>
>   - Mutable scalars are useful in their own rights.
>
>   - Is there substantial difference in overhead between rank-0 arrays and
>     scalars?

Yes.

>
>   - How to write literal values?  array(1) is too many characters.
>
>   - For rank-1 and rank-0 arrays, Python notation distinguishes:
>
> 	c[1] vs c[1,]
> 	d[] vs d[()]
>
>     Should these be used to handle semantic difference between indexing
>     and slicing?  Should d[] be syntactically allowed?
>
> Hope these observations help.

Thanks for the observations.

-Travis O.





From huaiyu_zhu at yahoo.com  Wed Sep 18 00:03:02 2002
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Wed Sep 18 00:03:02 2002
Subject: [Numpy-discussion] rank-0 arrays are mutable scalars
In-Reply-To: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>
Message-ID: <Pine.LNX.4.33.0209172211240.1082-100000@mars.localdomain>

On Tue, 17 Sep 2002, Travis Oliphant wrote:

> > 	len(d) == Exception == d.shape[0]
> >
> > 	# Currently the last is wrong?
> 
> Agreed, but this is because d is an integer and out of Numerics control.
> This is a case for returning 0d arrays rather than Python scalars.

That is one problem.  It can be removed by using shape(d).  

More fundamentally, though, len(d) == shape(d)[0] == ()[0] => IndexError.
I think Konrad made this point a few days back.

> > 	size(d) == 1 == product(d.shape)
> >
> > 	# Currently the last is wrong
> 
> I disagree that this is wrong.  This works as described for me.

Right.  Change d.shape to shape(d) here (and in several other places).

> Why is this?  I thought you argued the other way for len(scalar).  Of
> course, one solution is that we could overwrite the len() function and
> allow it to work for scalars.

Raising exception is the correct behavior, not a problem to be solved.

> >
> > Conclusion 5: rank-0 int arrays should be allowed to act as indices.
> > 	See property 5.
> 
> Can't do this for lists and other builtin sequences.

If numarray defines a consistent set of behaviors for integer types that
is intuitively understandable, it might not be difficult to persuade core
Python to check against an abstract integer type.

> >   - Is there substantial difference in overhead between rank-0 arrays and
> >     scalars?
> 
> Yes.

That would be one major problem.  


However, after giving this some more thoughts, I'm starting to doubt the 
analogy I made.  The problem is that in the end there is still a need to 
index an array and obtain a good old immutable scalar.  So

- What notation should be used for this purpose?  We can use c[0] to get
  immutable scalars and c[0,] for rank-0 arrays / mutable scalars.  But 
  what about other ranks?  Python does not allow distinctions based on  
  a[1,1,1] versus a[(1,1,1)] or d[] versus d[()].

- This weakens the argument that rank-0 arrays are scalars, since that 
  argument is essentially based on sum(c) and c[0] being of the same type.

Huaiyu




From juenglin at informatik.uni-freiburg.de  Wed Sep 18 01:16:02 2002
From: juenglin at informatik.uni-freiburg.de (Ralf Juengling)
Date: Wed Sep 18 01:16:02 2002
Subject: [Numpy-discussion] Attributes vs functions in numarray.
In-Reply-To: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>
References: <156b01c25dbd$3bb9ce30$6c0b6244@cx781526b>
Message-ID: <1032336896.15531.429.camel@leto>

On Mon, 2002-09-16 at 22:11, Tim Hochberg wrote:
> 
> Can someone refresh my memory as to why some properties of NumArrays that
> are conceptually attributes are accessed as atributes (a.shape, a.rank),
> while others are accessed through functions (a.type, a.iscontiguous,...).

Yes, I'd like to understand as well; there is even a third. There are
attributes, e.g. a.shape       vs
methods,    e.g. a.type()      vs
functions   e.g. len(a).

Regards,
Ralf



> 
> Thanks,
> 
> -tim
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
-------------------------------------------------------------------------
Ralf J?ngling
Institut f?r Informatik - Lehrstuhl f. Mustererkennung &
Bildverarbeitung
Georges-K?hler-Allee
Geb?ude 52                                       Tel:
+49-(0)761-203-8215
79110 Freiburg                                   Fax:
+49-(0)761-203-8262
-------------------------------------------------------------------------




From aureli at ipk.fhg.de  Thu Sep 19 09:18:06 2002
From: aureli at ipk.fhg.de (Aureli Soria Frisch)
Date: Thu Sep 19 09:18:06 2002
Subject: [Numpy-discussion] Conversion numarray/Numeric objects
In-Reply-To:  <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>
References: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu>
Message-ID: <a05111b00b9afa51e3cc0@[153.97.92.109]>

Hi,

I have developed till now all my numerical stuff using Numeric objects.

I want to use some modules using numarray. I have tested some 
conversion functions on Python, e.g.

numarray.array(Numeric_object.tolist())

but it seems to me a bit slow (specially for large arrays as images). 
Is there any more ways of doing this conversion?
Should I write my own conversion in C?
Which advices could you give me in order to make an smoother transition?

Furthermore I found the web site about numarray/Numeric 
incompatibilities very interesting. However I did not understand 
quite well the paragraph:

>Numeric arrays have some public attributes. Numarray arrays
>have none. All changes to an array's state must be made
>through accessor methods. Specifically, Numeric has the following
>attributes:
>
>Numeric        numarray accessor method(s)
>Attribute
>
>shape      --> shape()  (i.e., specify new shape through method arg instead
>of
>                          assigning to shape attribute)
>flat       --> flat()
>real       --> real()   (set capability not implemented yet, but will be)
>imag       --> imag()   (ditto)
>savespace  --> not used, no equivalent functionality (xxx check this)

since interactively typing a.shape() or a.shape((8,8)) raise an 
exception (Tuple object not callable). Am I misunderstanding 
something?

Thanks in advance.
Best regards,
Aureli

-- 
#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail: aureli at ipk.fhg.de
fon: +49 30 39006-143
fax: +49 30 3917517
web: http://vision.fhg.de/~aureli/web-aureli_en.html
#################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20020919/c1cacdb3/attachment-0001.html>

From jmiller at stsci.edu  Thu Sep 19 09:49:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 19 09:49:02 2002
Subject: [Numpy-discussion] Conversion numarray/Numeric objects
References: <Pine.LNX.4.33L2.0209171000430.6629-100000@oliphant.ee.byu.edu> <a05111b00b9afa51e3cc0@[153.97.92.109]>
Message-ID: <3D89FFDD.6090506@stsci.edu>

Aureli Soria Frisch wrote:

> Hi,
>
>
> I have developed till now all my numerical stuff using Numeric objects.
>
>
> I want to use some modules using numarray. I have tested some 
> conversion functions on Python, e.g.
>
>
> numarray.array(Numeric_object.tolist())
>
Much faster,  but less general, is:

numarray.fromstring(Numeric_object.tostring(), type=XXXX)

>
> but it seems to me a bit slow (specially for large arrays as images). 
> Is there any more ways of doing this conversion?
>
> Should I write my own conversion in C?
>
It depends on how soon you need it to be fast.    numarray 
fromlist/tolist should be implemented in C for the next release, which 
should occur within a couple months, perhaps sooner.  If you want to 
write a C extension for numarray now, the best way to do it is to use 
the Numeric compatability layer,   which essentially uses a subset of 
the Numeric C-API,  and is not likely to change.

> Which advices could you give me in order to make an smoother transition?
>
>
> Furthermore I found the web site about numarray/Numeric 
> incompatibilities very interesting. However I did not understand quite 
> well the paragraph:
>
I think that webpage is out of date.

>
> since interactively typing a.shape() or a.shape((8,8)) raise an 
> exception (Tuple object not callable). Am I misunderstanding something?
>
Actually, for python 2.2 and up,  numarray has all of those attributes 
in a Numeric compatible, Python properties based form.   So ".shape" 
get/sets the shape in numarray, just as in Numeric:

 >>> import numarray
 >>> a = numarray.arange(10)
 >>> a.shape
(10,)
 >>> a.shape = (2,5)
 >>> a.shape
(2,5)

Because of this,  the expression a.shape() that you mentioned tries to 
"call" the shape tuple returned by .shape, resulting in an exception.

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / SSB






From hinsen at cnrs-orleans.fr  Fri Sep 20 10:05:05 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Sep 20 10:05:05 2002
Subject: [Numpy-discussion] A message from Cameron Laird
Message-ID: <200209201704.g8KH4Ie19316@chinon.cnrs-orleans.fr>

I send this on behalf of Cameron Laird <claird at phaseit.net>.
Please reply to him, not to me.

  I have at least a couple of assignments from magazines such
  as IBM's developerWorks to report on matters that involve
  Numeric.  I'd welcome contact from anyone here who wants to
  publicize his or her work with Python and Numeric.

  I have a particular interest in advantages Python and Numeric
  enjoy over such alternatives as Mathematica, IDL, SAS/IML,
  MATLAB, and so on, all of which are more narrowly targeted at
  the kinds of scientific and engineering problems tackled by
  contributors to this mailing list.  What does Python do for
  you that the commercial products don't?

  I suspect that many of you will mention, in one form or
  another, Python's aptness for programming "in the large".
  Do you have specific examples of how this is clumsy in
  MATLAB, Mathematica, and so on?

  Have you tried to interface MATLAB and so on to hardware
  instrumentation or other external data sources?

  How do the scientists and engineers (as opposed to the
  "informaticians" or software developers) on your teams
  accept Python, compared to IDL and friends?  Do scientists
  at your site program?

  Is there anything Python's missing in its competition with
  MATLAB and so on?  


  Cameron Laird       <Cameron at Lairds.com>       +1 281 996 8546 FAX
  http://phaseit.net/claird/misc.writing/publications.html




From travis at enthought.com  Fri Sep 20 10:55:02 2002
From: travis at enthought.com (Travis N. Vaught)
Date: Fri Sep 20 10:55:02 2002
Subject: [Numpy-discussion] A message from Cameron Laird
In-Reply-To: <200209201704.g8KH4Ie19316@chinon.cnrs-orleans.fr>
Message-ID: <KPEIKEPEIBGNGNKHDILIEEIODMAA.travis@enthought.com>

Thought I'd reply to all since I'm including some links that are possibly
interesting to the list:

> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Konrad
> Hinsen
> Sent: Friday, September 20, 2002 12:04 PM
> To: numpy-discussion at lists.sourceforge.net
> Subject: [Numpy-discussion] A message from Cameron Laird
>
>
> I send this on behalf of Cameron Laird <claird at phaseit.net>.
> Please reply to him, not to me.
>
>   I have at least a couple of assignments from magazines such
>   as IBM's developerWorks to report on matters that involve
>   Numeric.  I'd welcome contact from anyone here who wants to
>   publicize his or her work with Python and Numeric.
>

The SciPy Toolkit (http://www.scipy.org) attacks Matlab's functionality
head-on--providing much of the functional interface symantics that Matlab
provides and much much more. It is a stated goal of many in the SciPy
development community to eliminate the need for Matlab altogether by
providing a tool that offers _all_ the functionality and better performance
(among many other things) in an open-source, truly object-oriented package.

>   I have a particular interest in advantages Python and Numeric
>   enjoy over such alternatives as Mathematica, IDL, SAS/IML,
>   MATLAB, and so on, all of which are more narrowly targeted at
>   the kinds of scientific and engineering problems tackled by
>   contributors to this mailing list.  What does Python do for
>   you that the commercial products don't?
>
>   I suspect that many of you will mention, in one form or
>   another, Python's aptness for programming "in the large".
>   Do you have specific examples of how this is clumsy in
>   MATLAB, Mathematica, and so on?
>
>   Have you tried to interface MATLAB and so on to hardware
>   instrumentation or other external data sources?
>

The recent SciPy '02 workshop (http://www.scipy.org/site_content/scipy02)
had some presented material that tangentially addressed interfacing Matlab
and Mathematica (perhaps even in a bof, I've slept since then).  You might
try the scipy-user mailing list as well. (Presentations:
http://www.scipy.org/site_content/scipy02/presentations , Mailing list:
http://www.scipy.org/site_content/MailList)

>   How do the scientists and engineers (as opposed to the
>   "informaticians" or software developers) on your teams
>   accept Python, compared to IDL and friends?  Do scientists
>   at your site program?
>
>   Is there anything Python's missing in its competition with
>   MATLAB and so on?
>

The aforementioned workshop included a survey that addressed these issues as
well.  You can look at the aggregated results by following the link below
(it's a little raw in its layout, let me know if there is a question about
how to interpret some things).
http://www.scipy.org/site_content/scipy02/survey_results.htm

(note: some questions were 'pick one' and others were 'pick all that
apply'--the total number of surveys turned in was 36, I think.)


>
>   Cameron Laird       <Cameron at Lairds.com>       +1 281 996 8546 FAX
>   http://phaseit.net/claird/misc.writing/publications.html
>




From rob at pythonemproject.com  Fri Sep 20 17:41:02 2002
From: rob at pythonemproject.com (Rob)
Date: Fri Sep 20 17:41:02 2002
Subject: [Numpy-discussion] A message from Cameron Laird
References: <200209201704.g8KH4Ie19316@chinon.cnrs-orleans.fr>
Message-ID: <3D8BBA34.AA81A36B@pythonemproject.com>

Konrad Hinsen wrote:
> 
<snip>

What I like about Numeric stems from what I like about Python- the
language does not get in my way of creating algorithms.  I do not have
to constantly refer to some reference book while programming.

The idea of the Numeric Python EM Project (URL below) was to put EM code
into a form where it could be more easily understood.  Python and
Numeric Python were the perfect choice for that task.  As for
Mathematica and other commercial software, I have no use for them. 
Python is free.

Sincerely,  Rob.

-- 
-----------------------------
The Numeric Python EM Project

www.pythonemproject.com



From perry at stsci.edu  Tue Sep 24 13:40:06 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Tue Sep 24 13:40:06 2002
Subject: [Numpy-discussion] numarray rank-0 decisions, rationale, and summary
Message-ID: <JFEGLNDJEDNOMPPHDEJFGEINEAAA.perry@stsci.edu>

[I posted this almost a week ago, but apparently an email problem
prevented it from actually getting posted!]

I think there has been sufficient discussion about rank-0 arrays
to make some decisions about how numarray will handle them.

[If you don't want to wade through the rationale, jump to the
end where there is a short summary of what we plan to do and
what we have questions about]

********************************************************************

First I'd like to take a stab at summarizing the case made for
rank-0 arrays in general and adding some of my own comments 
regarding these points.

1) rank-0 arrays are a useful mechanism to avoid having binary
operations with scalars cause unintended promotion of other arrays
to larger numeric types (e.g. 2*arange(10, typecode=Int16) results
in an Int32 result).

*** For numarray this is a non-issue because the coercion rules
prevent scalars from increasing the type of an array if the scalar
is the same kind of number (e.g., Int, Float, Complex) as the
array.

2) rank-0 arrays preserve the type information instead of converting
scalars to Python scalars.

*** This seems of limited value. With only a couple possible exceptions
in the future (and none now), Python scalars are effectively the 
largest type available so no information is lost. One can convert
to and from Python scalars and not lose any information. The possible
future exceptions are long doubles and UInt32 (supported in Numeric, 
but not numarray yet--but frankly, I'm not yet sure how important
UInt32 is at the moment). It is possible that Python scalars may
move up in size so this may or may not become an issue in the future.
By itself, it does not appear to be a compelling reason.

3) rank-0 arrays allow controlling exceptions (e.g. divide by zero)
in a way different from how Python handles them (exception always)

*** This is a valid concern...maybe. I was more impressed by it
initially, but it occurred to me that most expressions that involve
a scalar exception (NaN, divide-by-zero, overflow, etc.) generally
corrupt everything, unlike exceptions with arrays where only a few
values may be tainted. Unless one is only interested in ignoring some
scalar results in a scalar expression as part of a larger computation,
it seems of very limited use to ignore, or warn on scalar exceptions.
In any event, this is really of no relevance to the use of rank-0
for indexed results or reduction operations.

4) Using rank-0 arrays in place of scalars would promote more
generic programming. This was really the last point of real contention
as far as I was concerned. In the end, it really came down to seeing
good examples of how lacking this caused code to be much worse than
it could be with rank-0 arrays. There really are two cases being
discussed: whether indexing a single item ("complete dereferencing"
in Scott Gilbert's description) returns rank-0, and whether reduction
operations return rank-0.

*** indexing returning rank-0. Amazingly enough no one was able to 
provide even one real code example of where rank-0 returns from indexing
was a problem (this includes MA as far as I can tell). In the end, this
has been much ado about nothing. Henceforth, numarray will return
Python scalars when arrays are indexed (as it does currently).

*** reduction operations. There are good examples of where reduction
operations returning rank-0 are made simpler. However, the situation
is muddied quite a bit by other issues which I will discuss below.
This is an area that deserves a bit more discussion in general.
But before I tackle that, there is a point about rank-0 arrays that
needs to be made which I think is in some respects is an obvious point,
but somehow got lost in much of the discussions.

Even if it were true that rank-0 arrays made for much simpler, generic
code, they are far less useful than might appear in simplifying
code. Why? Because even if array operations (whether indexing, reduction
or other functions) were entirely consistent about never returning 
scalars, it is a general fact that most Numeric/numarray code must
be prepared to handle Python scalars thrown at it in place of
arrays by the user. Since Python scalars can come leaking into
your code at many points, consistency in Numeric/numarray in avoiding
Python scalars really doesn't solve the issue. I would hazard a
guess that the great majority of the conditional code that exist
today would not be eliminated because of this (e.g., this appears
to be the case for MA)

Reduction operations:

There is a good case to be made that reduction operations should
result in rank-0 arrays rather than scalars (after all, they
are reducing dimensions), but not everyone agrees that is what
should be done. But before deciding what is to be done there,
some problems with rank-0 arrays should be discussed. I think 
Konrad and Huaiyu have made very powerful arguments about how
certain operations like indexing, attributes and such should or
shouldn't work. In particular, some current Numeric behaviors
should be changed. Indexing by 0 should not work (instead Scott
Gilbert's suggestion of indexing with an empty tuple sounds 
right, if a bit syntatically clumsy due to Python not accepting
an empty index). len should not return 1, etc.

So even if we return rank-0 values in reduction operations, this
appears to still cause problems with some of the examples given
by Eric that depend on len(rank-0) = 1. What should be done
about that? One possibility is to use different numarray
functions designed to help write generic code (e.g., an alternate
function to len). But there is one aspect to this that ought to be
pointed out. Some have asked for rank-0 arrays that can be indexed
with 0 and produce len of 1. There is such an object that does this
and it is a rank-1 len-1 array. One alternative is to have reduction
operations have as their endpoint a rank-1 len-1 array rather than a
rank-0 array. The rank-0 endpoint is more justified conceptually,
but apparently less practical. If a reduction on a 1-d arrray always
produced a 1-d array, then one can always be guaranteed that it can
be indexed, and that len works on it. The drawback is that it can 
never be used as a scalar directly as rank-0 arrays could be.

I think this is a case where you can't have it both ways. If you 
want a scalar-like object, then some operations that work on higher
rank arrays won't work on it (or shouldn't). If you want something
where these operations do work, don't expect to use it where a
scalar is expected unless you index it.

Is there any interest in this alternate approach to reductions? We plan
to have two reduction methods available, one that results in scalars, and
one in arrays. The main question is which one .reduce maps to, and what
the endpoint is for the method that always returns arrays is.

***********************************************************************
                           SUMMARY
***********************************************************************

1) Indexing returns Python scalars in numarray. No rank-0 arrays
   are ever returned from indexing.
2) rank-0 arrays will be supported, but len(), and indexing will not
   work as they do in Numeric. In particular, to get a scalar, one
   will have to index with an empty tuple (e.g., x[()], x[0] will
   raise an exception), len() will return None.

Questions:

1) given 2, is there still a desire for .reduce() to return
rank-0 arrays (if not, we have .areduce() which is intented to return
arrays always).

2) whichever is the "returns arrays always" reduce method, should the 
endpoint be rank-0 arrays or rank-1 len-1 arrays?
 




From hinsen at cnrs-orleans.fr  Tue Sep 24 14:19:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Tue Sep 24 14:19:04 2002
Subject: [Numpy-discussion] numarray rank-0 decisions, rationale, and summary
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFGEINEAAA.perry@stsci.edu>
References: <JFEGLNDJEDNOMPPHDEJFGEINEAAA.perry@stsci.edu>
Message-ID: <m38z1rqe5v.fsf@localhost.localdomain>

"Perry Greenfield" <perry at stsci.edu> writes:

> Questions:
> 
> 1) given 2, is there still a desire for .reduce() to return
> rank-0 arrays (if not, we have .areduce() which is intented to return
> arrays always).
> 
> 2) whichever is the "returns arrays always" reduce method, should the 
> endpoint be rank-0 arrays or rank-1 len-1 arrays?

I don't really see an application where a reduction operation yielding
rank-1 or higher arrays would be useful. It would be a special case,
not useful for generic programming. So my answer to 2) is rank-0.

As for 1), if indexing doesn't return rank-0 arrays, then standard
reduction shouldn't either. We would then have a system in which
rank-0 arrays are "expert only" stuff, most users would never see
them, and they could safely be ignored in tutorials.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From perry at stsci.edu  Tue Sep 24 14:33:03 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Tue Sep 24 14:33:03 2002
Subject: [Numpy-discussion] numarray rank-0 decisions, rationale, and summary
In-Reply-To: <m38z1rqe5v.fsf@localhost.localdomain>
Message-ID: <JFEGLNDJEDNOMPPHDEJFKEINEAAA.perry@stsci.edu>

Konrad Hinsen writes:
> "Perry Greenfield" <perry at stsci.edu> writes:
> 
> > Questions:
> > 
> > 1) given 2, is there still a desire for .reduce() to return
> > rank-0 arrays (if not, we have .areduce() which is intented to return
> > arrays always).
> > 
> > 2) whichever is the "returns arrays always" reduce method, should the 
> > endpoint be rank-0 arrays or rank-1 len-1 arrays?
> 
> I don't really see an application where a reduction operation yielding
> rank-1 or higher arrays would be useful. It would be a special case,
> not useful for generic programming. So my answer to 2) is rank-0.
> 
What I am wondering is what behavior suits "generic" programming more.
Eric Jones and Paul Dubois have given examples where having to
deal with entities that may be scalars or arrays is a pain. But,
having said that, the behavior that Eric wanted was not consistent
with what many thought rank-0 arrays should have (i.e., len(rank-0)=1,
rank-0[0] = value). The proposal to generate rank-1 len-1 arrays was
made since len() of these arrays is = 1, and indexing with [0]
does work. So for the kinds of examples he gave, rank-1 len-1 arrays
appear to allow for more generic code.

But I'm not trying to speak for everyone; that's why I'm asking
for opinions. Do you have examples where you find rank-0 arrays
make for more generic code in your cases when len() and indexing
on these does not have the behavior Eric wanted? May I see a couple?

> As for 1), if indexing doesn't return rank-0 arrays, then standard
> reduction shouldn't either. We would then have a system in which
> rank-0 arrays are "expert only" stuff, most users would never see
> them, and they could safely be ignored in tutorials.
> 
That's my inclination, but I think that the question of whether there
should be some reduce mechanism that returns arrays always is still
a valid one. I can see that there are good uses for that (or at least
a function to cast a scalar to a rank-1 len-1 array if it isn't already
an array, like what array() does except that it now generates rank-0
arrays).

Perry



From mathew at comma.jpl.nasa.gov  Wed Sep 25 17:24:04 2002
From: mathew at comma.jpl.nasa.gov (Mathew Yeates)
Date: Wed Sep 25 17:24:04 2002
Subject: [Numpy-discussion] Cast problems .... doo dah, doo dah
Message-ID: <3D92538D.8020108@comma.jpl.nasa.gov>

The following code produces an error
---------------------------------------
a=array(([1,2,3,4,5],[10,9,8,7,6])).astype(Float32)
cond=array(([0,0,0,0,0],[1,1,1,1,1])
b=array(([0,0,0,0,0],[0,0,0,0,0]))

choose(cond,(a,b))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: Array can not be safely cast to required type
------------------------------

I suppose I might need to cast b to Float32. But, I have a function 
which is passed a and b
as args. Ideally, I would like a function which casts a and b to the 
highest compatible type since
"choose" doesnt do it. How can I do this simply?

Mathew

-- 
M47h3w `/34735





From mathew at comma.jpl.nasa.gov  Thu Sep 26 16:13:03 2002
From: mathew at comma.jpl.nasa.gov (Mathew Yeates)
Date: Thu Sep 26 16:13:03 2002
Subject: [Numpy-discussion] Casting. Method to the madness?
Message-ID: <3D939474.2030402@comma.jpl.nasa.gov>

Hmmm, aparently, I can cast a long array to a double array but not
to a float array.

Check out the following code from arrayobject.c (Numeric 22.0)
[this is a switch on the type being casted from]

case PyArray_LONG:
    /*Shouldn't allow Longs to be cast to Ints, not safe on 64-bit 
machines!*/
    return (totype >= PyArray_INT) && (totype != PyArray_FLOAT);

This makes absolutely no sense to me. Is this a bug?

Mathew

-- 
M47h3w `/34735





From retype at terra.com.br  Fri Sep 27 10:17:02 2002
From: retype at terra.com.br (Leonardo Santagada)
Date: Fri Sep 27 10:17:02 2002
Subject: [Numpy-discussion] (no subject)
In-Reply-To: <E17uyGi-00050H-00@usw-sf-list2.sourceforge.net>
References: <E17uyGi-00050H-00@usw-sf-list2.sourceforge.net>
Message-ID: <20020927141325.41af5fb8.retype@terra.com.br>

confirm 498554



From szport at fromru.com  Mon Sep 30 02:11:12 2002
From: szport at fromru.com (Zaur Shibzoukhov)
Date: Mon Sep 30 02:11:12 2002
Subject: [Numpy-discussion] P: tofile & fromfile
Message-ID: <20020930130251.0290bb24.szport@fromru.com>

I propose to add two function to multiarray module in Numeric:

tofile(array, openedfile)
array = fromfile(opendfile, dimension, typecode = "l")

Now I use these functions in my module.
Here is an implementation from module:

###############################################################################3

#include "Python.h"
#define PY_ARRAY_UNIQUE_SYMBOL PyArray_arrayfile
#include "Numeric/arrayobject.h"

#define MAX_DIMS 40

static PyObject *
PyArray_FromFile(PyObject *self, PyObject *args)
{
    PyObject *f, *dimobj, *temp;
    PyArray_Descr *descr;
    PyArrayObject *ret = NULL;
    int i, n, dim[40], nt;
    char *typecode = "l";
    FILE *fp;
    int type_num, N = 1;
    size_t nread;

    if (!PyArg_ParseTuple(args, "O!O!|s#:fromfile", \
                          &PyFile_Type, &f, &PyTuple_Type, &dimobj, &typecode, &nt))
        return NULL;

    fp = PyFile_AsFile(f);
    if (fp == NULL) {
        PyErr_SetString(PyExc_TypeError, "1st argument must be open file");
        return NULL;
    }
    n = PyTuple_Size(dimobj);
    if (n > MAX_DIMS) {
        PyErr_SetString(PyExc_TypeError, "dimension is too large");
        return NULL;
    }
    if (n > 0) {
        for (i = 0; i < n; i++) {
            temp = PyTuple_GetItem(dimobj, i);
            dim[i] = (int)PyInt_AsLong(temp);
        }
        
        descr = PyArray_DescrFromType(*typecode);
        type_num = descr -> type_num;
        
        ret = (PyArrayObject *)PyArray_FromDims(n, dim, type_num);
        memcpy(ret->data, descr->zero, N*descr->elsize);
        
        N = 1;
        for (i = 0; i < n; i++) N *= dim[i];
        nread = fread((char *)ret->data, descr->elsize, N, fp);
        if (nread < (size_t)N) {
            PyErr_SetString(PyExc_EOFError, "not enough items in file");
            return NULL;
        }
    }

    return (PyObject *)ret;
}

static char PyArray_FromFile_doc [] =
"fromfile(f, dimension, typecode = \'l\')\n\
\n\
Create array from open file f with given dimension and typecode.\n\
Note, that file must be open in binary mode.";


static PyObject *
PyArray_ToFile(PyObject *self, PyObject *args)
{
    PyObject *f;
    PyArrayObject *A;
    int N;
    FILE *fp;
        
    if (!PyArg_ParseTuple(args, "O!O!:tofile", &PyArray_Type, &A, &PyFile_Type, &f))
        return NULL;
    
    fp = PyFile_AsFile(f);
    if (fp == NULL) {
        PyErr_SetString(PyExc_TypeError, "arg must be open file");
        return NULL;
    }
    
    N = PyArray_SIZE(A);
    if (N > 0) {
        if (fwrite(A->data, A->descr->elsize, N, fp) != (size_t)N) {
            PyErr_SetFromErrno(PyExc_IOError);
            clearerr(fp);
            return NULL;
        }
    }
    Py_INCREF(Py_None);
    return Py_None;
}

static char PyArray_ToFile_doc [] =
"tofile(array, f)\n\
\n\
Write array to file open f.\n\
Note, that file must be open in binary mode.";


static PyMethodDef arrayfile_module_methods[] = {
         {"fromfile",  PyArray_FromFile, 1, PyArray_FromFile_doc},
         {"tofile",    PyArray_ToFile,   1, PyArray_ToFile_doc},
         {NULL,                 NULL}  /* sentinel */
};

############################################################################

Zaur Shibzoukhov