On Mon, Sep 8, 2014 at 7:41 AM, Sturla Molden sturla.molden@gmail.com wrote:

Stefan Otte stefan.otte@gmail.com wrote:

...

stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

As far as I can see, it uses hstack and vstack. But that means a and b have to have the same number of rows, c and d must have the same rumber of rows, and hstack((a,b)) and hstack((c,d)) must have the same number of columns.

Thus it requires a regularity like this:

AAAABB AAAABB CCCDDD CCCDDD CCCDDD CCCDDD

What if we just ignore this constraint, and only require the output to be rectangular? Now we have a 'tetris game':

AAAABB AAAABB CCCCBB CCCCBB CCCCDD CCCCDD

or

AAAABB AAAABB CCCCBB CCCCBB CCCCBB CCCCBB

This should be 'stackable', yes? Or perhaps we need another stacking function for this, say numpy.tetris?

And while we're at it, what about higher dimensions? should there be an ndstack function too?

This is starting to look like the second time in a row Stefan tries to extend numpy with a simple convenience function, and he gets tricked into implementing some sophisticated algorithm...

For his next PR I expect nothing less than an NP-complete problem. ;-)

Jaime

Sturla: im not sure if the intention is always unambiguous, for such more flexible arrangements.

Also, I doubt such situations arise often in practice; if the arrays arnt a grid, they are probably a nested grid, and the code would most naturally concatenate them with nested calls to a stacking function.

However, some form of nd-stack function would be neat in my opinion.

On Mon, Sep 8, 2014 at 6:10 PM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:

On Mon, Sep 8, 2014 at 7:41 AM, Sturla Molden sturla.molden@gmail.com wrote:

...

Stefan Otte stefan.otte@gmail.com wrote:

...

stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

As far as I can see, it uses hstack and vstack. But that means a and b have to have the same number of rows, c and d must have the same rumber of rows, and hstack((a,b)) and hstack((c,d)) must have the same number of columns.

Thus it requires a regularity like this:

AAAABB AAAABB CCCDDD CCCDDD CCCDDD CCCDDD

What if we just ignore this constraint, and only require the output to be rectangular? Now we have a 'tetris game':

AAAABB AAAABB CCCCBB CCCCBB CCCCDD CCCCDD

or

AAAABB AAAABB CCCCBB CCCCBB CCCCBB CCCCBB

This should be 'stackable', yes? Or perhaps we need another stacking function for this, say numpy.tetris?

And while we're at it, what about higher dimensions? should there be an ndstack function too?

This is starting to look like the second time in a row Stefan tries to extend numpy with a simple convenience function, and he gets tricked into implementing some sophisticated algorithm...

For his next PR I expect nothing less than an NP-complete problem. ;-)

...

Jaime

-- (__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Btw, on a somewhat related note, whoever can implement ndarray to be able to use views from other ndarrays stitched together would get a fruit basket from me come the holidays and possibly naming rights for the next kid...

Cheers! Ben Root

On Mon, Sep 8, 2014 at 12:55 PM, Benjamin Root ben.root@ou.edu wrote:

A use case would be "image stitching" or even data tiling. I have had to implement something like this at work (so, I can't share it, unfortunately) and it even goes so far as to allow the caller to specify how much the tiles can overlap and such. The specification is ungodly hideous and I doubt I would be willing to share it even if I could lest I release code-thulu upon the world...

I think just having this generalize stack feature would be nice start. Tetris could be built on top of that later. (Although, I do vote for at least 3 or 4 dimensional stacking, if possible).

Cheers! Ben Root

On Mon, Sep 8, 2014 at 12:41 PM, Eelco Hoogendoorn < hoogendoorn.eelco@gmail.com> wrote:

...

Sturla: im not sure if the intention is always unambiguous, for such more flexible arrangements.

Also, I doubt such situations arise often in practice; if the arrays arnt a grid, they are probably a nested grid, and the code would most naturally concatenate them with nested calls to a stacking function.

However, some form of nd-stack function would be neat in my opinion.

On Mon, Sep 8, 2014 at 6:10 PM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:

...

On Mon, Sep 8, 2014 at 7:41 AM, Sturla Molden sturla.molden@gmail.com wrote:

...

Stefan Otte stefan.otte@gmail.com wrote:

...

stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

As far as I can see, it uses hstack and vstack. But that means a and b have to have the same number of rows, c and d must have the same rumber of rows, and hstack((a,b)) and hstack((c,d)) must have the same number of columns.

Thus it requires a regularity like this:

AAAABB AAAABB CCCDDD CCCDDD CCCDDD CCCDDD

What if we just ignore this constraint, and only require the output to be rectangular? Now we have a 'tetris game':

AAAABB AAAABB CCCCBB CCCCBB CCCCDD CCCCDD

or

AAAABB AAAABB CCCCBB CCCCBB CCCCBB CCCCBB

This should be 'stackable', yes? Or perhaps we need another stacking function for this, say numpy.tetris?

And while we're at it, what about higher dimensions? should there be an ndstack function too?

This is starting to look like the second time in a row Stefan tries to extend numpy with a simple convenience function, and he gets tricked into implementing some sophisticated algorithm...

For his next PR I expect nothing less than an NP-complete problem. ;-)

...

Jaime

-- (__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Blaze aims to do something like that; to make the notion of an array and how it stores it's data far more flexible. But if it isn't a single strided ND array, it isn't numpy. This concept lies at its very heart; and for good reasons I would add.

-----Original Message----- From: "Benjamin Root" ben.root@ou.edu Sent: ‎8-‎9-‎2014 19:00 To: "Discussion of Numerical Python" numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] Generalize hstack/vstack --> stack; Blockmatrices like in matlab

Btw, on a somewhat related note, whoever can implement ndarray to be able to use views from other ndarrays stitched together would get a fruit basket from me come the holidays and possibly naming rights for the next kid...

Cheers! Ben Root

On Mon, Sep 8, 2014 at 12:55 PM, Benjamin Root ben.root@ou.edu wrote:

A use case would be "image stitching" or even data tiling. I have had to implement something like this at work (so, I can't share it, unfortunately) and it even goes so far as to allow the caller to specify how much the tiles can overlap and such. The specification is ungodly hideous and I doubt I would be willing to share it even if I could lest I release code-thulu upon the world...

I think just having this generalize stack feature would be nice start. Tetris could be built on top of that later. (Although, I do vote for at least 3 or 4 dimensional stacking, if possible).

Cheers! Ben Root

On Mon, Sep 8, 2014 at 12:41 PM, Eelco Hoogendoorn hoogendoorn.eelco@gmail.com wrote:

Sturla: im not sure if the intention is always unambiguous, for such more flexible arrangements.

Also, I doubt such situations arise often in practice; if the arrays arnt a grid, they are probably a nested grid, and the code would most naturally concatenate them with nested calls to a stacking function.

However, some form of nd-stack function would be neat in my opinion.

On Mon, Sep 8, 2014 at 6:10 PM, Jaime Fernández del Río jaime.frio@gmail.com wrote:

On Mon, Sep 8, 2014 at 7:41 AM, Sturla Molden sturla.molden@gmail.com wrote:

Stefan Otte stefan.otte@gmail.com wrote:

stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

As far as I can see, it uses hstack and vstack. But that means a and b have to have the same number of rows, c and d must have the same rumber of rows, and hstack((a,b)) and hstack((c,d)) must have the same number of columns.

Thus it requires a regularity like this:

AAAABB AAAABB CCCDDD CCCDDD CCCDDD CCCDDD

What if we just ignore this constraint, and only require the output to be rectangular? Now we have a 'tetris game':

AAAABB AAAABB CCCCBB CCCCBB CCCCDD CCCCDD

or

AAAABB AAAABB CCCCBB CCCCBB CCCCBB CCCCBB

This should be 'stackable', yes? Or perhaps we need another stacking function for this, say numpy.tetris?

And while we're at it, what about higher dimensions? should there be an ndstack function too?

This is starting to look like the second time in a row Stefan tries to extend numpy with a simple convenience function, and he gets tricked into implementing some sophisticated algorithm...

For his next PR I expect nothing less than an NP-complete problem. ;-)

Jaime

Hey,

In the last weeks I tested `np.asarray(np.bmat(....))` as `stack` function and it works quite well. So the question persits: If `bmat` already offers something like `stack` should we even bother implementing `stack`? More code leads to more bugs and maintenance work. (However, the current implementation is only 5 lines and by using `bmat` which would reduce that even more.)

Best, Stefan

On Fri, Sep 19, 2014 at 4:47 PM, Paul Hobson pmhobson@gmail.com wrote:

Hey Ben,

Side note: I've had to do the same thing for stitching curvilinear model grid coordinates together. Usings pandas DataFrames indexed by `i` and `j` is really good for this. You can offset the indices directly, unstack the DF, and the pandas will align for you.

Happy to send an example along if you're curious. -p

On Mon, Sep 8, 2014 at 9:55 AM, Benjamin Root ben.root@ou.edu wrote:

...

A use case would be "image stitching" or even data tiling. I have had to implement something like this at work (so, I can't share it, unfortunately) and it even goes so far as to allow the caller to specify how much the tiles can overlap and such. The specification is ungodly hideous and I doubt I would be willing to share it even if I could lest I release code-thulu upon the world...

I think just having this generalize stack feature would be nice start. Tetris could be built on top of that later. (Although, I do vote for at least 3 or 4 dimensional stacking, if possible).

Cheers! Ben Root

On Mon, Sep 8, 2014 at 12:41 PM, Eelco Hoogendoorn hoogendoorn.eelco@gmail.com wrote:

...

Sturla: im not sure if the intention is always unambiguous, for such more flexible arrangements.

Also, I doubt such situations arise often in practice; if the arrays arnt a grid, they are probably a nested grid, and the code would most naturally concatenate them with nested calls to a stacking function.

However, some form of nd-stack function would be neat in my opinion.

On Mon, Sep 8, 2014 at 6:10 PM, Jaime Fernández del Río jaime.frio@gmail.com wrote:

...

On Mon, Sep 8, 2014 at 7:41 AM, Sturla Molden sturla.molden@gmail.com wrote:

...

Stefan Otte stefan.otte@gmail.com wrote:

...

stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

As far as I can see, it uses hstack and vstack. But that means a and b have to have the same number of rows, c and d must have the same rumber of rows, and hstack((a,b)) and hstack((c,d)) must have the same number of columns.

Thus it requires a regularity like this:

AAAABB AAAABB CCCDDD CCCDDD CCCDDD CCCDDD

What if we just ignore this constraint, and only require the output to be rectangular? Now we have a 'tetris game':

AAAABB AAAABB CCCCBB CCCCBB CCCCDD CCCCDD

or

AAAABB AAAABB CCCCBB CCCCBB CCCCBB CCCCBB

This should be 'stackable', yes? Or perhaps we need another stacking function for this, say numpy.tetris?

And while we're at it, what about higher dimensions? should there be an ndstack function too?

This is starting to look like the second time in a row Stefan tries to extend numpy with a simple convenience function, and he gets tricked into implementing some sophisticated algorithm...

For his next PR I expect nothing less than an NP-complete problem. ;-)

Jaime

-- (__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Hey,

there are several ways how to proceed.

- My proposed solution covers the 80% case quite well (at least I use it all the time). I'd convert the doctests into unittests and we're done.

- We could slightly change the interface to leave out the surrounding square brackets, i.e. turning `stack([[a, b], [c, d]])` into `stack([a, b], [c, d])`

- We could extend it even further allowing a "filler value" for non set values and a "shape" argument. This could be done later as well.

- `bmat` is not really matrix specific. We could refactor `bmat` a bit to use the same logic in `stack`. Except the `matrix` calls `bmat` and `_from_string` are pretty agnostic to the input.

I'm in favor of the first or last approach. The first: because it already works and is quite simple. The last: because the logic and tests of both `bmat` and `stack` would be the same and the feature to specify a string representation of the block matrix is nice.

Best, Stefan

On Tue, Oct 28, 2014 at 7:46 PM, Nathaniel Smith njs@pobox.com wrote:

On 28 Oct 2014 18:34, "Stefan Otte" stefan.otte@gmail.com wrote:

...

Hey,

In the last weeks I tested `np.asarray(np.bmat(....))` as `stack` function and it works quite well. So the question persits: If `bmat` already offers something like `stack` should we even bother implementing `stack`? More code leads to more bugs and maintenance work. (However, the current implementation is only 5 lines and by using `bmat` which would reduce that even more.)

In the long run we're trying to reduce usage of np.matrix and ideally deprecate it entirely. So yes, providing ndarray equivalents of matrix functionality (like bmat) is valuable.

-n

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Hey,

Just a quick update. I updated the pull request and renamed `stack` into `block`. Have a look: https://github.com/numpy/numpy/pull/5057

I'm sticking with simple initial implementation because it's simple and does what you think it does.

Cheers, Stefan

On Fri, Oct 31, 2014 at 2:13 PM Stefan Otte stefan.otte@gmail.com wrote:

To make the last point more concrete the implementation could look something like this (note that I didn't test it and that it still takes some work):

def bmat(obj, ldict=None, gdict=None): return matrix(stack(obj, ldict, gdict))

def stack(obj, ldict=None, gdict=None): # the old bmat code minus the matrix calls if isinstance(obj, str): if gdict is None: # get previous frame frame = sys._getframe().f_back glob_dict = frame.f_globals loc_dict = frame.f_locals else: glob_dict = gdict loc_dict = ldict return _from_string(obj, glob_dict, loc_dict)

if isinstance(obj, (tuple, list)):
    # [[A,B],[C,D]]
    arr_rows = []
    for row in obj:
        if isinstance(row, N.ndarray):  # not 2-d
            return concatenate(obj, axis=-1)
        else:
            arr_rows.append(concatenate(row, axis=-1))
    return concatenate(arr_rows, axis=0)

if isinstance(obj, N.ndarray):
    return obj

I basically turned the old `bmat` into `stack` and removed the matrix calls.

Best, Stefan

On Wed, Oct 29, 2014 at 3:59 PM, Stefan Otte stefan.otte@gmail.com wrote:

...

Hey,

there are several ways how to proceed.

• My proposed solution covers the 80% case quite well (at least I use

it all the time). I'd convert the doctests into unittests and we're done.

• We could slightly change the interface to leave out the surrounding

square brackets, i.e. turning `stack([[a, b], [c, d]])` into `stack([a, b], [c, d])`

• We could extend it even further allowing a "filler value" for non

set values and a "shape" argument. This could be done later as well.

• `bmat` is not really matrix specific. We could refactor `bmat` a bit

to use the same logic in `stack`. Except the `matrix` calls `bmat` and `_from_string` are pretty agnostic to the input.

I'm in favor of the first or last approach. The first: because it already works and is quite simple. The last: because the logic and tests of both `bmat` and `stack` would be the same and the feature to specify a string representation of the block matrix is nice.

Best, Stefan

On Tue, Oct 28, 2014 at 7:46 PM, Nathaniel Smith njs@pobox.com wrote:

...

On 28 Oct 2014 18:34, "Stefan Otte" stefan.otte@gmail.com wrote:

...

Hey,

In the last weeks I tested `np.asarray(np.bmat(....))` as `stack` function and it works quite well. So the question persits: If `bmat` already offers something like `stack` should we even bother implementing `stack`? More code leads to more bugs and maintenance work. (However, the current implementation is only 5 lines and by using `bmat` which would reduce that even more.)

In the long run we're trying to reduce usage of np.matrix and ideally deprecate it entirely. So yes, providing ndarray equivalents of matrix functionality (like bmat) is valuable.

-n

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

On 1/9/2016 4:58 AM, Stefan Otte wrote:

Hey,

one of my new year's resolutions is to get my pull requests accepted (or closed). So here we go...

Here is the update pull request: https://github.com/numpy/numpy/pull/5057 Here is the docstring: https://github.com/sotte/numpy/commit/3d4c5d19a8f15b35df50d945b9c8853b683f7a...

The new `block` function is very similar to matlab's `[A, B; C, D]`.

Pros: - it's very useful (in my experiment) - less friction for people coming from matlab - it's conceptually simple - the implementation is simple - it's documented - it's tested

Cons: - the implementation is not super efficient. Temporary copies are created. However, bmat also does that.

Feedback is very welcome!

Best, Stefan

Without commenting on the implementation, I would find this function *very* useful and convenient in my own work.

-gyro

Le 08/09/2014 16:41, Sturla Molden a écrit :

Stefan Otte stefan.otte@gmail.com wrote:

...

 stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

As far as I can see, it uses hstack and vstack. But that means a and b have to have the same number of rows, c and d must have the same rumber of rows, and hstack((a,b)) and hstack((c,d)) must have the same number of columns.

Thus it requires a regularity like this:

AAAABB AAAABB CCCDDD CCCDDD CCCDDD CCCDDD

What if we just ignore this constraint, and only require the output to be rectangular? Now we have a 'tetris game':

AAAABB AAAABB CCCCBB CCCCBB CCCCDD CCCCDD

or

AAAABB AAAABB CCCCBB CCCCBB CCCCBB CCCCBB

stack([stack([[a], [c]]), b])

This should be 'stackable', yes? Or perhaps we need another stacking function for this, say numpy.tetris?

The function should be implemented for its name only ! I like it !

And while we're at it, what about higher dimensions? should there be an ndstack function too?

Sturla

On 08-Sep-14 4:40 PM, Joseph Martinot-Lagarde wrote:

Le 08/09/2014 15:29, Stefan Otte a écrit :

...

Hey,

quite often I work with block matrices. Matlab offers the convenient notation

  [ a b; c d ]

This would appear to be a desirable way to go.

Numpy has something similar for strings. The above is neater.

Colin W.

...

to stack matrices. The numpy equivalent is kinda clumsy:

vstack([hstack([a,b]), hstack([c,d])])

I wrote the little function `stack` that does exactly that:

  stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

Best, Stefan

[1] https://github.com/numpy/numpy/pull/5057

The outside brackets are redundant, stack([[a, b], [c, d]]) should be stack([a, b], [c, d])

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

On Tue, Sep 9, 2014 at 5:42 AM, Stefan Otte stefan.otte@gmail.com wrote:

Hey,

@Josef, I wasn't aware of `bmat` and `np.asarray(np.bmat(....))` does basically what I want and what I'm already using.

I never needed any tetris or anything similar except for the matched block version.

Just to point out two more related functions

scipy.sparse also has `bmat` for sparse block matrices scipy.linalg and scipy.sparse have `block_diag` to complement bmat.

What I sometimes wish for is a sparse pseudo kronecker product as convenience to bmat, where the first (or the second) matrix contains (0,1) flags where the 1's specify where to put the blocks. (I'm not sure what I really mean, similar to block_diag but with a different filling pattern.)

Josef

Regarding the Tetris problem: that never happened to me, but stack, as Josef pointed out, can handle that already :)

I like the idea of removing the redundant square brackets: stack([[a, b], [c, d]]) --> stack([a, b], [c, d]) However, if the brackets are there there is no difference between creating a `np.array` and stacking arrays with `np.stack`.

If we want to get fancy and turn this PR into something bigger (working our way up to a NP-complete problem ;)) then how about this. I sometimes have arrays that look like: AB0 0 C Where 0 is a scalar but is supposed to fill the rest of the array. Having something like 0 in there might lead to ambiguities though. What does ABC 0D0 mean? One could limit the "filler" to appear only on the left or the right: AB0 0CD But even then the shape is not completely determined. So we could require to have one row that only consists of arrays and determines the shape. Alternatively we could have a keyword parameter `shape`: stack([A, B, 0], [0, C, D], shape=(8, 8))

Colin, with `bmat` you can do what you're asking for. Directly taken from the example:

...

...

...

np.bmat('A,B; C,D')

matrix([[1, 1, 2, 2], [1, 1, 2, 2], [3, 4, 7, 8], [5, 6, 9, 0]])

General question: If `bmat` already offers something like `stack` should we even bother implementing `stack`? More code leads to more bugs and maintenance work.

Best, Stefan

On Tue, Sep 9, 2014 at 12:14 AM, cjw cjw@ncf.ca wrote:

...

On 08-Sep-14 4:40 PM, Joseph Martinot-Lagarde wrote:

...

Le 08/09/2014 15:29, Stefan Otte a écrit :

...

Hey,

quite often I work with block matrices. Matlab offers the convenient

notation

...

...

...

  [ a b; c d ]

This would appear to be a desirable way to go.

Numpy has something similar for strings. The above is neater.

Colin W.

...

...

to stack matrices. The numpy equivalent is kinda clumsy:

vstack([hstack([a,b]), hstack([c,d])])

I wrote the little function `stack` that does exactly that:

  stack([[a, b], [c, d]])

In my case `stack` replaced `hstack` and `vstack` almost completely.

If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback!

Best, Stefan

[1] https://github.com/numpy/numpy/pull/5057

The outside brackets are redundant, stack([[a, b], [c, d]]) should be stack([a, b], [c, d])

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion