[Python-Dev] Using PEP384 Stable ABI for the lzma extension module

["Martin v. Löwis"]

Am 04.10.11 21:06, schrieb Amaury Forgeot d'Arc:
> 2011/10/4 "Martin v. Löwis"<martin at v.loewis.de>:
>>
>>> - _PyBytes_Resize() is missing; I moved it under a Py_LIMITED_API
>>>    section.
>>
>> ??? Are you proposing to add _PyBytes_Resize to the Py_LIMITED_API
>> set of functions? It's not even an API function in the first place
>> (it starts with an underscore), so how can it be a limited API function?
>
> It's not a proposal of any kind; it's just the workaround I used to compile
> and test.
> OTOH, it seems that many modules already use this function. Is there
> another method that does not need to copy data?

Not sure what you are using it for. If you need to extend the buffer
in case it is too small, there is absolutely no way this could work
without copies in the general case because of how computers use
address space. Even _PyBytes_Resize will copy the data.

The only way to avoid copying is to run over the input twice: once
to determine how large the output will have to be, and then another
time to actually produce the output. Whether or not that's actually
faster than copying the output depends on how much work this size
computation requires. It would be nice if LZMA had "output size"
information embedded in it, but it may not.

Regards,
Martin