On Fri, Jun 06, 2014 at 02:05:35AM +1000, Nick Coghlan wrote:

From the "idle speculation" files (inspired by the recent thread on python-dev): has anyone ever experimented with offering string methods like find() on StringIO objects?

I don't work in any sufficiently memory constrained environments these days that that style of API would be worth the hassle relative to a normal string, it just struck me as a potentially interesting approach to the notion of a string manipulation type that didn't generally copy data around and could use different code point sizes internally for different parts of the text data.

Thought about this quite a bit. There are a few ways StringIO/BytesIO/buffers could improve, not sure which approaches are interesting, though.. 1) Not sure if it's the case in Python3.x (pretty sure it isn't in 2.x), but cStringIO could optimize for the case where the IO is discarded after building a single string by using the CPython APIs for doing that (e.g. _PyString_Resize). In that case, getvalue() returns the built string, and sets an internal flag to cause it to be copied to a new private string if any further IO is invoked. This inverts the current behaviour where the normal case of build-and-discard causes a copy. 2) Rather than implement string methods on the StringIO, it might be nicer if those methods could apply to a memoryview, and then make it possible e.g. for BytesIO to be exposed as a memoryview. Right now Python doesn't have much in the way of generic "type safe / memory safe" APIs for doing things to regular memory without first invoking copies/conversions of various sorts. This might be the more useful thing to fix. We have plenty of special cases, like bytearray(), array.array(), StringIO (to some degree), and so on, and various ways to manipulate that memory (ctypes and struct module for example), but they are all somewhat hodge-podges of each other and lack any "one way to do it". I had looked at building some kind of unified 'memory slice' type last year, since I keep bumping into the need for better Python-level support for this stuff when working on 'bit twiddling' projects of various kinds. It's mostly thinking aloud, but here is a rough sketch for the kind of module I had been considering last year, mostly while working with Python 2: https://github.com/dw/memsink/wiki/Memory-Module . The idea was to provide a common 'Slice' adaptor type whose memory could be interpreted using a couple of different abstractions (Vector and File being the obvious). David

Cheers, Nick.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/