[Numpy-discussion] Introduction to Scott, Jason, and (possibly) others from Enthought

Travis Oliphant oliphant at enthought.com
Wed May 26 19:00:18 EDT 2010 

On May 26, 2010, at 3:50 AM, Sebastian Walter wrote:

> I'm a potential user of the C-API and therefore I'm very interested in
> the outcome.
> In the previous discussion
> (http://comments.gmane.org/gmane.comp.python.numeric.general/37409)
> many different views on what the new C-API "should" be were expressed.
>
> Naturally, I wonder if the new C-API will be useful for my purposes.
> So,  I'm not so excited about a "refactoring log" where only the
> progress is reported. I fear that some (potentially minor) design
> decisions would render the new C-API useless for me.
>
> So my question is:
> Does this "refactoring log" also include something like a Numpy
> Enhancement Proposal? Something that can be discussed beforehand?
> I.e., will there be a detailed description (i.e. code examples) what
> the goal of the refactoring is?
>
> If there is any interest, I could provide some simple test examples in
> C that would explain what I'd like to be able to do with the new
> C-API.

Our team would love to see this if possible.

-Travis

>
>
> Sebastian
>
>
>
>
> On Wed, May 26, 2010 at 8:21 AM, David Goldsmith
> <d.l.goldsmith at gmail.com> wrote:
>> On Tue, May 25, 2010 at 9:22 PM, Travis Oliphant <oliphant at enthought.com 
>> >
>> wrote:
>>>
>>> On May 25, 2010, at 4:49 PM, David Goldsmith wrote:
>>>
>>> Travis: do you already have a place on the NumPy Development Wiki  
>>> where
>>> you're (b)logging your design decisions?  Seems like a good way for
>>> concerned parties to monitor your choices in more or less real  
>>> time and thus
>>> provide comment in a timely fashion.
>>>
>>> This is a great idea of course and we will definitely post progess  
>>> there.
>>>
>>
>> Thanks; specific URL please, when available; plus, prominently  
>> feature (a
>> link to) the location on the Development Wiki home page, at the  
>> very least
>> (i.e., if not also on the NumPy home page).
>>
>>>
>>> So far, the code has been reviewed,
>>
>> I.e., the existing code, yes?
>>
>>>
>>> and several functions identified for re-factoring.
>>
>> Please enumerate on the "Wiki Refactoring Log" (name tentative - I  
>> don't
>> care what we call it, just so long as it exists, its purpose is  
>> clear, and
>> we all know where it is).
>>
>>> This is taking place in a github branch of numpy called numpy  
>>> refactor.
>>
>> "This" = the actual creation/modification of code, yes?
>>
>> DG
>>>
>>> -Travis
>>>
>>> DG
>>>
>>> On Tue, May 25, 2010 at 2:19 PM, Charles R Harris
>>> <charlesr.harris at gmail.com> wrote:
>>>>
>>>>
>>>> On Tue, May 25, 2010 at 2:54 PM, Travis Oliphant <oliphant at enthought.com 
>>>> >
>>>> wrote:
>>>>>
>>>>> On May 25, 2010, at 2:50 PM, Charles R Harris wrote:
>>>>>
>>>>>
>>>>> On Tue, May 25, 2010 at 1:37 PM, Travis Oliphant
>>>>> <oliphant at enthought.com> wrote:
>>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> There has been some talk about re-factoring NumPy to separate  
>>>>>> out the
>>>>>> Python C-API layer and make NumPy closer to a C-library.   I know
>>>>>> there are a few different ideas about what this means, and also  
>>>>>> that
>>>>>> people are very busy.  I also know there is a NumPy 2.0 release  
>>>>>> that
>>>>>> is in the works.
>>>>>>
>>>>>> I'm excited to let everyone know that we (at Enthought) have  
>>>>>> been able
>>>>>> to find resources (about 3 man months) to work on this re- 
>>>>>> factoring
>>>>>> project and Scott and Jason (both very experienced C and Python
>>>>>> programmers) are actively pursuing it.    My hope is that NumPy  
>>>>>> 2.0
>>>>>> will contain this re-factoring (which should be finished just  
>>>>>> after
>>>>>> SciPy 2010 --- where I'm going to organize a Sprint on NumPy  
>>>>>> which
>>>>>> will include at least date-time improvements and re-factoring  
>>>>>> work).
>>>>>>
>>>>>> While we have specific goals for the re-factoring, we want this
>>>>>> activity to be fully integrated with the NumPy community and  
>>>>>> Scott and
>>>>>> Jason want to interact with the community as much as feasible  
>>>>>> as they
>>>>>> suggest re-factoring changes (though they both have more  
>>>>>> experience
>>>>>> with phone-conversations to resolve concerns than email chains  
>>>>>> and so
>>>>>> some patience from everybody will be appreciated).
>>>>>>
>>>>>> Because Jason and Scott are new to this mailing list (but not  
>>>>>> new to
>>>>>> NumPy),  I wanted to introduce them so they would feel more
>>>>>> comfortable posting questions and people would have some  
>>>>>> context as to
>>>>>> what they were trying to do.
>>>>>>
>>>>>> Scott and Jason are both very proficient and skilled  
>>>>>> programmers and I
>>>>>> have full confidence in their abilities.   That said, we very  
>>>>>> much
>>>>>> want the input of as many people as possible as we pursue the  
>>>>>> goal of
>>>>>> grouping together more tightly the Python C-API interface layer  
>>>>>> to
>>>>>> NumPy.
>>>>>>
>>>>>> I will be involved in some of the discussions, but am currently  
>>>>>> on a
>>>>>> different project which has tight schedules and so I will only  
>>>>>> be able
>>>>>> to provide limited "mailing-list" visibility.
>>>>>>
>>>>>
>>>>> I think 2.0 would be a bit early for this. Is there any reason it
>>>>> couldn't be done in 2.1? What is the planned policy with regards  
>>>>> to the
>>>>> visible interface for extensions? It would also be nice to have  
>>>>> a rough idea
>>>>> of how the resulting code would be layered, i.e., what is the  
>>>>> design for
>>>>> this re-factoring. Simply having a design would be a major step  
>>>>> forward.
>>>>>
>>>>> The problem with doing it in 2.1 is that this re-factoring will  
>>>>> require
>>>>> extensions to be re-built.   The visible interface to extensions  
>>>>> will not
>>>>> change, but there will likely be ABI incompatibility.    It  
>>>>> seems prudent to
>>>>> do this in NumPy 2.0.   Perhaps we can also put in place the ABI- 
>>>>> protecting
>>>>> indirection approaches that David C. was suggesting earlier.
>>>>> Some aspects of the design are still being fleshed out, but the  
>>>>> basic
>>>>> idea is to separate out a core library that is as independent of  
>>>>> the Python
>>>>> C-API as possible.    There will likely be at least some  
>>>>> dependency on the
>>>>> Python C-API (reference counting and error handling and possibly  
>>>>> others)
>>>>> which any interface would have to provide in a very simple  
>>>>> Python.h --
>>>>> equivalent, for example.
>>>>> Our purpose is to allow NumPy to be integrated with other  
>>>>> languages or
>>>>> other frameworks systems without explicitly relying on  
>>>>> CPython.    There are
>>>>> a lot of questions as to how this will work, and so much of that  
>>>>> is being
>>>>> worked out.   Part of the reason for this mail is to help ensure  
>>>>> that as
>>>>> much of this discussion as possible takes place in public.
>>>>
>>>> Sounds good, but what if it doesn't get finished in a few months?  
>>>> I think
>>>> we should get 2.0.0 out pronto, ideally it would already have  
>>>> been released.
>>>> I think a major refactoring like this proposal should get the  
>>>> 3.0.0 label.
>>>> Admittedly that makes keeping a refactored branch current with  
>>>> fixes going
>>>> into the trunk a hassle, but perhaps that can be worked around  
>>>> somewhat by
>>>> clearly labeling what files will be touched in the refactoring  
>>>> and possibly
>>>> rearranging the content of the existing files. This requires a  
>>>> game plan and
>>>> a clear idea of the goal. Put simply, I think the proposed  
>>>> schedule is too
>>>> ambitious and needs to be fleshed out.  This refactoring isn't  
>>>> going to be
>>>> as straight forward as the python3k port because a lot of design  
>>>> decisions
>>>> need to be made along the way.
>>>>
>>>> Chuck
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>> --
>>> Mathematician: noun, someone who disavows certainty when their  
>>> uncertainty
>>> set is non-empty, even if that set has measure zero.
>>>
>>> Hope: noun, that delusive spirit which escaped Pandora's jar and,  
>>> with her
>>> lies, prevents mankind from committing a general suicide.  (As  
>>> interpreted
>>> by Robert Graves)
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>> ---
>>> Travis Oliphant
>>> Enthought, Inc.
>>> oliphant at enthought.com
>>> 1-512-536-1057
>>> http://www.enthought.com
>>>
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>
>>
>>
>> --
>> Mathematician: noun, someone who disavows certainty when their  
>> uncertainty
>> set is non-empty, even if that set has measure zero.
>>
>> Hope: noun, that delusive spirit which escaped Pandora's jar and,  
>> with her
>> lies, prevents mankind from committing a general suicide.  (As  
>> interpreted
>> by Robert Graves)
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

--
Travis Oliphant
Enthought Inc.
1-512-536-1057
http://www.enthought.com
oliphant at enthought.com

More information about the NumPy-Discussion mailing list