[Python-Dev] Using PEP384 Stable ABI for the lzma extension module

[Antoine Pitrou]

On Wed, 5 Oct 2011 09:38:10 -0700
Toshio Kuratomi <a.badger at gmail.com> wrote:
> On Wed, Oct 05, 2011 at 06:14:08PM +0200, Antoine Pitrou wrote:
> > Le mercredi 05 octobre 2011 à 18:12 +0200, "Martin v. Löwis" a écrit :
> > > >> Not sure what you are using it for. If you need to extend the buffer
> > > >> in case it is too small, there is absolutely no way this could work
> > > >> without copies in the general case because of how computers use
> > > >> address space. Even _PyBytes_Resize will copy the data.
> > > >
> > > > That's not a given. Depending on the memory allocator, a copy can be
> > > > avoided. That's why the "str += str" hack is much more efficient under
> > > > Linux than Windows, AFAIK.
> > > 
> > > Even Linux will have to copy a block on realloc in certain cases, no?
> > 
> > Probably so. How often is totally unknown to me :)
> > 
> http://www.gnu.org/software/libc/manual/html_node/Changing-Block-Size.html
> 
> It depends on whether there's enough free memory after the buffer you
> currently have allocated.  I suppose that this becomes a question of what
> people consider "the general case" :-)

But under certain circumstances (if a large block is requested), the
allocator uses mmap(), no? In which case mremap() should allow to
resize without copying anything.

Regards

Antoine.