Thanks Marten.

I tried memap for a few things but it seemed to create an other OS level buffer in specific situations. I think the `seek` operation in the `memmap` also caused some performance bottlenecks. Maybe I'll have time to summarize my findings an other day.

The particular usecase of `ffmpeg` is tricky since it is grabbing a lot of data from `stdout` which isn't a typical file buffer. Specifically, it is often `buffered` but `unseekable`.

On Mon, Nov 5, 2018 at 9:38 AM Marten van Kerkwijk <> wrote:
Hi Mark,

Having an `out` might make sense. With present numpy, if you are really dealing with a file or file-like object, you might consider using `np.memmap` to access the data more directly. If it is something that looks more like a buffer, `np.frombuffer` may be useful (that doesn't copy data, but points the array at the memory that holds the buffer).

All the best,


On Sun, Nov 4, 2018 at 10:35 PM Mark Harfouche <> wrote:

I was wondering what would your thoughts be on adding an output parameter to np.fromfile?

The advantage would be when interfacing with executables like ffmpeg which are arguably easier to use by calling them as a subprocess compared to a shared library in python.

Having the output parameter in np.fromfile would enable pre-allocation of large arrays that are reused during the computation of new image frames when decoding large video files.

Thoughts are appreciated!


NumPy-Discussion mailing list
NumPy-Discussion mailing list