[Ironpython-users] numpy in IronPython

[Doug Blank]

I've tried to capture all of the links and options towards developing a
numpy for IronPython here:

http://calicoproject.org/Numpy

Please let me know if I have missed something, or if you would like to add
to that page.

My summary so far:

1) a new wrapper (based on IronClad or Cython) would create fast code, and
could utilize the current and future versions of numpy, but would (I
suspect) be low-level, high-maintenance in keeping it working on
multi-platforms/architectures. This wrapper work might also allow other
packages in the SciPy to work. Requires low-level C#, CLR, and
IronPython-specific skills to develop (a small group of people have these
skills; only IronPython users would benefit).

2) A pure-Python version would be a lot of work (perhaps building on PyPy's
RPython version and converting their C) and be slow, but would be little
maintenance as most of the details for the current version of numpy would
be static. Requires generic Python skills to develop (a large group of
people have these skills; any generic Python implementation could use).

-Doug


On Mon, Jun 2, 2014 at 2:09 PM, Doug Blank <doug.blank at gmail.com> wrote:

> On Mon, Jun 2, 2014 at 1:41 PM, Steve Baer <steve at mcneel.com> wrote:
>
>> > That is the code here that I mentioned in the first post:
>> > https://www.enthought.com/repo/.iron/
>>
>> Unless I'm missing something, that's the binary installation and not the
>> actual source code. I believe Enthought used managed C++ which does tie it
>> to Windows, but it would also be good to see since it would give me a good
>> idea about the effort involved in trying to make a C# DLL with pInvokes to
>> a C DLL. I've done a lot of this type of work in the past and it usually
>> isn't very hard once I've gotten things set up correctly (like having an
>> executable to automatically generate the pinvokes from the exported C
>> functions.)
>>
>
> I think you are correct. A bit more googling found:
>
> http://blog.enthought.com/python/scipy-for-net/#.U4y83x92nfE
>
> which states:
>
> """
> The first release of SciPy and NumPy for .NET are available now as binary
> distributions from SciPy.org or directly from Enthought. All of the code
> for these and the supporting projects are open source and available at the
> links below.
>
> NumPy for .NET:https://github.com/numpy/numpy-refactor
> SciPy for .NET:https://github.com/jasonmccampbell/scipy-refactor
> Cython for .NET:
> https://bitbucket.org/jasonmccampbell/cython-for-ironpython
> FWrap extensions:https://github.com/jasonmccampbell/fwrap
> """
>
> So it looks like the Enthought project was using the cython-for-ironpython.
>
> Please let us know what you think about making this work for C#; thanks!
>
> -Doug
>
>
>>
>> -Steve
>>
>> Steve Baer
>> Robert McNeel & Associates
>> www.rhino3d.com
>>
>>
>> On Mon, Jun 2, 2014 at 10:28 AM, Doug Blank <doug.blank at gmail.com> wrote:
>>
>>> On Mon, Jun 2, 2014 at 1:13 PM, Steve Baer <steve at mcneel.com> wrote:
>>>
>>>> We should also check with the Enthought guys to see what they did and
>>>> if they are willing to share.
>>>>
>>>
>>> That is the code here that I mentioned in the first post:
>>>
>>> https://www.enthought.com/repo/.iron/
>>>
>>> I can't tell what license this is done under. Also, it is Windows only.
>>>
>>> It looks like a Cython for IronPython might be closer to working:
>>>
>>> https://bitbucket.org/cwitty/cython-for-ironpython/overview
>>>
>>> Especially here considering these:
>>>
>>>
>>> https://bitbucket.org/cwitty/cython-for-ironpython/src/9dc7e1a2d56a/Cython/Includes/posix/?at=default
>>>
>>> https://bitbucket.org/cwitty/cython-for-ironpython/commits/9dc7e1a2d56a3e3c8a7b3c7a5d23073a54b2b814
>>>
>>> But, I haven't worked much with Python and C libraries.
>>>
>>> -Doug
>>>
>>>
>>>>
>>>> -Steve
>>>>
>>>>
>>>> On Sun, Jun 1, 2014 at 6:29 AM, Doug Blank <doug.blank at gmail.com>
>>>> wrote:
>>>>
>>>>> On Sun, Jun 1, 2014 at 3:05 AM, Jeff Hardy <jdhardy at gmail.com> wrote:
>>>>>
>>>>>> On Sat, May 31, 2014 at 3:23 PM, Doug Blank <doug.blank at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On Fri, May 30, 2014 at 5:22 PM, Steve Baer <steve at mcneel.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I would definitely be interested in helping, but don't exactly know
>>>>>>>> where to start. We have a lot of users who would love to get access to
>>>>>>>> numpy on our OSX and 64bit windows versions of our product.  This is only
>>>>>>>> going to become a bigger problem in the future since we will probably only
>>>>>>>> have a 64bit version for Window in the next version of Rhino.
>>>>>>>>
>>>>>>>
>>>>>>> I agree. Let's start a serious discussion about how to solve the
>>>>>>> lack of numpy in IronPython.
>>>>>>>
>>>>>>> We could look into using ctypes (IronClad?) and wrap what already
>>>>>>> exists.
>>>>>>>
>>>>>>> We could look into a cross-platform DLL drop-in replacement.
>>>>>>>
>>>>>>> Between speed and compatibility, initially I'm most interested in
>>>>>>> compatibility. But speed should be a long term goal.
>>>>>>>
>>>>>>> We could write a pure-python prototype initially, and slowly move
>>>>>>> that to C#, or another CLR language. That would be useful for all
>>>>>>> non-C-based Python implementations, and would probably be quickest to write
>>>>>>> and test.
>>>>>>>
>>>>>>> A related note: Python3 just added a new matrix multiplication
>>>>>>> operator [1]. Hope to see more numpy-related functionality in standard
>>>>>>> Python in the future.
>>>>>>>
>>>>>>> Other ideas? Where to start?
>>>>>>>
>>>>>>
>>>>>> I would start by asking the NumPy team what the best option is, and
>>>>>> seeing what the NumPyPy team are doing - the more work that can be shared,
>>>>>> the better.
>>>>>>
>>>>>
>>>>> That is a good idea, and I started to try to define what we might mean
>>>>> by "best option." Almost all of the work I have seen coming from most
>>>>> related projects would define best in terms of speed. However, this comes
>>>>> at a pretty big cost in terms of maintenance---for example, having wrapped
>>>>> C libraries compiled for each platform, for each bus size (32, 64).
>>>>>
>>>>> In poking around, I think the Jython needs might be closest to ours.
>>>>> Looks like they have two starts: a pure-Java library based on the older
>>>>> Numeric [1], and a native interface for talking directly to C [2]. The
>>>>> native interface is a long term project, and is not yet to the point of
>>>>> working with numpy. As a quick test, I tried to IKVM-convert the pure Java
>>>>> jar file into a DLL. I think that could work (eventually) but would require
>>>>> bringing a lot of Jython, and would probably always be a little wonky. In
>>>>> addition, that API is the older Numeric.
>>>>>
>>>>> A third option, as mentioned by Ivan, is to have a bridge to CPython
>>>>> interfacing with numpy. But that would make CPython a dependency---not
>>>>> really something that many IronPython users would appreciate.
>>>>>
>>>>> Of course, if one is looking for doing numeric operations, you could
>>>>> use a different .NET/Mono math library. But what I am interested in is the
>>>>> numpy API, so that other code will be usable in IronPython.
>>>>>
>>>>> To me, it looks like the best bet at this point in time is to write
>>>>> our own. The next question is: write it in pure Python, or C#/F#/etc.
>>>>>
>>>>> If we write it in pure Python, there is the chance that some Jython
>>>>> developers (and maybe other Python implementation people) might be
>>>>> interested in helping. It would be slow, but we could rely on Python for
>>>>> handling type operations (float times int) to do the right thing. It would
>>>>> also be immediately usable by future CPython users. It could be the case
>>>>> that a future Python implementation could do some JITting to make it run
>>>>> fast enough. (A pure Python version might also be useful for educational
>>>>> uses, as I presume it would be more easily understood by Python students).
>>>>>
>>>>> If we write it as a CLR library, it would be as fast as managed code
>>>>> would allow, and be available for other CLR languages (like F#, Boo, etc).
>>>>> But we would probably be alone in developing it, as it is mostly of
>>>>> interest to Python users using numpy and the CLR.
>>>>>
>>>>> I guess I am leaning towards a pure Python implementation of the
>>>>> latest numpy API. Perhaps followed up by a DLL version.
>>>>>
>>>>> -Doug
>>>>>
>>>>> [1] - https://bitbucket.org/zornslemon/jnumeric-ra/overview
>>>>> [2] - http://jyni.org/
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> - Jeff
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20140603/6ef45380/attachment.html>