[Numpy-discussion] out parameter for np.fromfile
mark.harfouche at gmail.com
Mon Nov 5 11:24:57 EST 2018
I tried memap for a few things but it seemed to create an other OS level
buffer in specific situations. I think the `seek` operation in the `memmap`
also caused some performance bottlenecks. Maybe I'll have time to summarize
my findings an other day.
The particular usecase of `ffmpeg` is tricky since it is grabbing a lot of
data from `stdout` which isn't a typical file buffer. Specifically, it is
often `buffered` but `unseekable`.
On Mon, Nov 5, 2018 at 9:38 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:
> Hi Mark,
> Having an `out` might make sense. With present numpy, if you are really
> dealing with a file or file-like object, you might consider using
> `np.memmap` to access the data more directly. If it is something that looks
> more like a buffer, `np.frombuffer` may be useful (that doesn't copy data,
> but points the array at the memory that holds the buffer).
> All the best,
> On Sun, Nov 4, 2018 at 10:35 PM Mark Harfouche <mark.harfouche at gmail.com>
>> I was wondering what would your thoughts be on adding an output parameter
>> to np.fromfile?
>> The advantage would be when interfacing with executables like ffmpeg
>> which are arguably easier to use by calling them as a subprocess compared
>> to a shared library in python.
>> Having the output parameter in np.fromfile would enable pre-allocation
>> of large arrays that are reused during the computation of new image frames
>> when decoding large video files.
>> Thoughts are appreciated!
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion