Hello, On Wed, 1 Apr 2020 20:05:33 -0700 Andrew Barnert <abarnert@yahoo.com> wrote:
On Apr 1, 2020, at 14:47, Paul Sokolovsky <pmiscml@gmail.com> wrote:
At the moment, the message is relatively clear - "build a list and join it" (it's very rare that anyone suggests StringIO currently).
I don't know how much you mix with other Pythonistas, but word "clear" is an exaggeration. From those who don't like it, the usual word is "ugly", though I've seen more vivid epithets, like "repulsive": https://mail.python.org/pipermail/python-list/2006-January/403480.html
More cool-headed guys like me just call it "complete wastage of memory"
Then aren’t you going to be disappointed when you’re told you can now use StringIO instead in 3.10, and you start using it everywhere as soon as you can drop 3.9 support, and then you find that in many Python implementations, including CPython, it takes a little more memory than building a list and joining it, not less?
I accept(ed) your argument. If my proposal would come forward, I would perform a (more) detailed analysis of internal implementations of StringIO in CPython and other implementation(s). In the meantime, to clarify, the memory usage figures (potential 8x space usage of list vs StringIO) do hold do MicroPython/Pycopy. I don't jump into analyzing CPython impl because IMHO the biggest current blocker is a claim that "StringIO wasn't intended to be used like that". So, unless resistance to that is softened, deep memory-digging is premature. So, I'd rather keep discussing that point, for as long as there's a desire to keep this discussion.
The fact that it could theoretically take less memory, and might even do so on some other implementation you aren’t using, is probably not going to be much consolation. Or maybe it is, and you switch to one of those other implementations, because that’s the final straw for CPython for you—and then you discover that, unlike CPython, their StringIO does save memory over str.join, but also unlike CPython it’s also a lot slower than str.join, mainly because its StringIO is encoding every string to UTF-8 when you write and then decoding back at the end.
Thanks for pinpointing another implied matter. Yes, I kinda assume that io.StringIO is actually an optimized implementation, not just TextIOWrapper(BytesIO). We can discuss this point further if useful.
At which point you’re probably ready to give up on Python altogether.
Given that nobody wants to push people away from using "".join() (only invite people to an alternative with cookies^W better interface, and gradually not-worse performance re: speed/memory), I don't think that's a plausible scenario for breaking people's faith in Python. There're certainly bigger risks ;-). -- Best regards, Paul mailto:pmiscml@gmail.com