[Numpy-discussion] Interest to port Numpy to Haskell, with Accelerate/Repa to do the crunching?

[YueCompl]

Yes, I do feel NumExpr does very similar things as Repa, while Repa works rather statically for Haskell code, NumExpr is much more interactive.

Well, honestly as a long time Numpy user, I'm aware of NumExpr since early days I learning Numpy, but never ever resorted to NumExpr, it's neither as complete as Numpy, nor as flexible as Numba.

I'm neither proficient in Haskell at data exploration for now, attempting improving the tooling (e.g. [hadui](https://github.com/complyue/hadui <https://github.com/complyue/hadui>) ), while still working on an ndarray interface for Haskell to an in house array database (mmap'ed virtual large datafile per se). I miss very much the workflow with Pandas/Numpy/Numba, the interactive aspect is crucial to me and my team's daily work. We have to resort to Haskell because some time ago we needed to start processing irregular event streams (no common dimension along time axis), for thousands of series (instruments in the stock market). While in handling this many number of inputs (passing in a tuple of 1d arrays, Numba treats each array as a separate input variable), Numba will swallow 20+GB ram with 10+ hours in jit compilation without a result.

I'm aware of Julia, and feels very pity for my org not able to adopt it for hiring difficulties, while luckily Haskell is agreed to be the base of our next gen internal platform. So I'm currently working hard to bring sufficient interaction abilities with a Haskell tool chain.

> On 2020-01-22, at 01:58, Frederic Bastien <fbastien at nvidia.com> wrote:
> 
> The description you make tell me that this look similar to numexpr[1]. It could be useful that you look it up.
> 
> 
> 
> [1] https://github.com/pydata/numexpr
> 
> -----Original Message-----
> From: NumPy-Discussion <numpy-discussion-bounces+fbastien=nvidia.com at python.org> On Behalf Of Compl Yue
> Sent: Tuesday, January 21, 2020 10:33 AM
> To: numpy-discussion at python.org
> Subject: Re: [Numpy-discussion] Interest to port Numpy to Haskell, with Accelerate/Repa to do the crunching?
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Matthew,
> 
> I hate to say I'm still a noob in the implementation details of Numpy, I just got an interpreter for sth else working, and suddenly realized it can make seemingly better magic methods in Python spirit.
> 
> Repa and Accelerate per my learning so far, are focused on loop-fusion, i.e. optimization after all intended computations have been expressed and composed together, this seems some different from Numpy/Pandas tradition to implement doable things ex ante, and let the users do things interactively  ad-hoc. That said, so I feel Repa will do composed computations more speedy, once properly organized. And with Accelerate, since it does GPU natively, I do feel a sure thing for it to excel CPU implementations.
> 
> btw I'm not a native English speaker, please don't hesitate to ask me to make myself clearer in my wordings, I'm aware of my flaws in English skills.
> 
> btw I'm a believer that Brain-Computer-Interface ought to happen sooner, especially non-intrusive ones, I can't wait to program such interfaces for great good and fun. I see you are pioneering in this area, take my admiring, and I started following you on github, looking forward to hear from you more.
> 
> btw I'm behind China's Great Fire Wall, access to some western sites are banned badly at times, that will explain why I can be unresponsive some times.
> 
> Cheers,
> 
> Compl
> 
> 
> On 2020/1/21 下午10:20, Matthew Brett wrote:
>> Hi,
>> 
>> On Mon, Jan 20, 2020 at 3:00 PM YueCompl <compl.yue at icloud.com> wrote:
>>> Hello folks,
>>> 
>>> Myself a long time Numpy user, and recently done some Haskell stuff,
>>> 
>>> Please have a look at  
>>> https://github.com/e-wrks/edh/tree/master/Tour#defining-more-magic-methods and https://github.com/e-wrks/edh/tree/master/Tour#indexing , any interest to port Numpy to Haskell, with http://hackage.haskell.org/package/accelerate  and/or  http://hackage.haskell.org/package/repa doing the number crunching?
>> Interesting - do you have any feeling for the differences in speed for 
>> Haskell accelerate array processing as compared to Numpy with default 
>> OpenBLAS?
>> 
>> Cheers,
>> 
>> Matthew
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 
> -----------------------------------------------------------------------------------
> This email message is for the sole use of the intended recipient(s) and may contain
> confidential information.  Any unauthorized review, use, disclosure or distribution
> is prohibited.  If you are not the intended recipient, please contact the sender by
> reply email and destroy all copies of the original message.
> -----------------------------------------------------------------------------------
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200122/658cad88/attachment.html>