[Numpy-discussion] [SciPy-user] Managing Python with NumPy and many external libraries on multiple Windows machines

[David Cournapeau]

On Wed, Apr 29, 2009 at 12:45 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Tue, Apr 28, 2009 at 9:41 AM, David Cournapeau <cournape at gmail.com>
> wrote:
>>
>> On Tue, Apr 28, 2009 at 11:58 PM, Travis Oliphant
>> <oliphant at enthought.com> wrote:
>> >
>> > On Apr 27, 2009, at 4:05 PM, Wes McKinney wrote:
>> >
>> >> Hello,
>> >>
>> >> I am wondering if anyone can offer some suggestions on this problem.
>> >> Over the last year or so I have been building a number of libraries
>> >> on top of NumPy + SciPy + matplotlib and other libraries which are
>> >> being used for investigative research for my company's problem
>> >> domain in place of, say, Matlab and R (which are more "ready out of
>> >> the box" systems). I have approximately 20 users, all of whom are
>> >> running Windows on a very Microsoft-centric network with databases,
>> >> etc. Has anyone had any luck managing a standardized Python
>> >> environment on lots of Windows machines with a large number of 3rd-
>> >> party Python libraries? Upgrading packages right now involves
>> >> getting 20 people to click through an .exe installer, which is
>> >> hardly a good solution. For example, I was recently forced to
>> >> upgrade everyone's NumPy to 1.3 after I discovered that a DLL I had
>> >> built against 1.3 was incompatible with 1.2.x.
>> >
>> > There has been a good discussion of this with others.   I'm pretty
>> > sure we had fixed this in NumPy so at least you would get a sane error
>> > instead of a segfault.   The solution may not be ideal and it can be
>> > hard to remember to edit the right place in the headers when a change
>> > is made so it may not have been done for 1.3
>>
>> Maybe we could supplement the manual editing: the code generators
>> should guarantee the order of each item of the API, so the number of
>> items in the array is enough to compare to API. So if we implement a
>> guard in the PyArray_API, we could dynamically check the number of
>> items in the array, and check it against the pre-computed number. A
>> different number means incompatible API.
>
> That will work as long as we maintain backward compatibility. If we change
> the ABI at some point we will need something stronger.

I was only suggesting to supplement the current scheme. The current
scheme is necessary in any case. But manually editing some macro in
the header file is not very reliable. More generally, I think we
should have a way to track C API and ABI relatively to the number
version, as well as automatically generate the related documentation.
While working on the split_comp branch, I notice several undocumented
functions, several functions wrongly documented, etc... All this
should be handled as automatically as possible.

cheers,

David