zero-copy array slicing

Alex Martelli aleax at aleax.it
Tue Nov 19 16:59:35 EST 2002


Trevor wrote:

> Alex Martelli <aleax at aleax.it> wrote in message news:<JLqC9.37182
>> > Is a zero-copy array slicing operation possible? [..]
>>
>> [..] if this the behavior you need, the simplest way to get it is exactly
>> with the Numeric extensions
> 
> Interesting, thanks!  However I don't think Numeric or Numarray
> support the buffer interface (or come standard with python), which was
> my reason for trying to use the array module.

Numeric doesn't come standard with Python, but does give you effective ways 
to 'get a grip' as you request.  It also supports the buffer interface:

>>> import Numeric
>>> x=Numeric.arange(5)
>>> buffer(x)
<read-only buffer for 0x81c19d8, ptr 0x81c1778, size 20 at 0x81f5238>
>>>

but I doubt that's really what you want to use for the purpose you explain.


> What I'm trying to do is interface a C cryptography library to python,
> so I can encrypt/hash data in as convenient and efficient a way as
> possible.  I was thinking the buffer interface would be the best way
> to get a grip on the contents of strings (for read-only operations
> like hashing) and arrays.  If arrays had a subsequence() function then
> I could efficiently hash or encrypt-in-place parts of arrays:
> 
> bytes = readArrayFromBigFile()
> hash(bytes.subsequence(128))              #hash all but some header
> encrypt(bytes)                            #encrypt whole array
> encrypt(bytes.subsequence(start, finish)) #encrypt some section

You can do that with slices of Numeric.array instances of course -- Numeric 
IS designed to for power, flexibility, and speed, while the builtin array 
is designed for light-weight, simple, no-overhead operation.  And, the 
buffer interface is not necessarily best for your purposes.


> I'd be willing to try to add this.  But maybe I should explore the
> solution space more.  What would you recommend as the most Pythonic
> approach, given these constraints?
>  - in-place operation on subsequences of bytes, from the C api
>  - convenient application to different types (e.g. strings, arrays)
>  - ideally, no dependency on nonstandard libraries (like Numeric)

Wouldn't it be simpler if your functions also accepted optional parameters 
for offset and size in bytes?  So the user could call

hash(bytes, 128)
encrypt(bytes)
encrypt(bytes, start, finish-start)

...?


Alex




More information about the Python-list mailing list