On Mar 30, 2015, at 05:52, Joao S. O. Bueno <jsbueno@python.org.br> wrote:

...

On 29 March 2015 at 19:01, Tim Delaney <timothy.c.delaney@gmail.com> wrote:

...

On 30 March 2015 at 04:04, Dmitry Kazakov <jsbfox@gmail.com> wrote:

I propose adding a context manager to contextlib module, which would restore the stream position of its argument at exit:

with contextlib.closing(file_like_object) as file: file.read(1) old = file.tell() with contextlib.keep_stream_position(file) as f: assert file is f assert file_like_object is f # Do something with the file(-like) object e.g. feed it # to `PIL.Image.open` (will consume the whole stream)

assert file.tell() == old

This CM would allow us to reuse streams efficiently, without requiring to call seek() explicitly. I will open a new issue and submit a patch in case of favorable responses.

I like the idea. Might need some bikeshedding on the name (maybe restore_position?).

To test for seeking, I would suggest having a seek to current position during construction. Not perfect (e.g. there may be streams that can only seek forwards, not backwards) but I think it's the best non-destructive option.

If that is being done, anyway, I propose an optional parameter that would allow the context to internally use itertools.tee if the stream can't seek itself.

That sounds cool, but... If you have an actual file object with seek and tell, you're often using other methods besides iterating lines (like read), and tee won't help with those, so it might be a little misleading.

Maybe with even three states: "strict": raises an exception if the stream is not [back]seekable "default": creates a tee cache if the stream is not back-seekable "cache": always cache.

(And maybe a maxbuffer parameter to go along? )

...

Tim Delaney

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

---

Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Mar 30, 2015, at 06:59, Chris Angelico <rosuav@gmail.com> wrote:

On Tue, Mar 31, 2015 at 12:52 AM, Andrew Barnert <abarnert@yahoo.com.dmarc.invalid> wrote:

...

...

If that is being done, anyway, I propose an optional parameter that would allow the context to internally use itertools.tee if the stream can't seek itself.

That sounds cool, but... If you have an actual file object with seek and tell, you're often using other methods besides iterating lines (like read), and tee won't help with those, so it might be a little misleading.

That shouldn't be an issue, though, as those will just use seek/tell.

The whole point of this context manager is to seek at enter and tell at exit. If it's useful to people using files as iterators, it's at least as useful to people using them as files. More so, in fact, because most code that asks for a file-like iterator isn't going to want to assume it's actually a file, while code that asks for something that has file methods doesn't have that problem.

I'm more wondering about the ones that _do_ need the tee'ing; how, after the context manager completes, do you have the file object return something from the other half of the tee?

You're right; tee doesn't do anything to the iterable you give it, it creates two new iterators that share the iterable and a cache. So the caller has to use one of those new iterators, which would be clumsy. Something like: with restoring_position(f, tee_if_needed=True) as f, flocal: # do stuff with flocal as an iterator of lines # do stuff with f, which is an iterator of the same lines Of course this only works if f only lives within this scope, or is a closure or global variable (if it was a normal parameter, as soon as you return to the caller, he's got the original file object seeked to the wrong place). On top of the issue with f now being an iterator rather than whatever kind of file-like object you started with, the potential for confusion and misuse is so high that I don't think I'd do this even internally to my own project, much less suggest it for the stdlib... But at least the implementation is still pretty simple: @contextmanager def restoring_position(t, tee_if_needed=False): try: pos = f.tell() f.seek(pos) except (AttributeError, IForgetTheExceptionForNotSeekable): if not tee_jf_needed: raise f, flocal = tee(f) frestore = None else: flocal = frestore = f try: yield f, flocal finally: if frestore is not None: frestore.seek(pos) A better solution might be to build an actual file tee--a BufferedReader or TextIOWrapper that wraps another object of the same type, doesn't discard used data from the buffer, and can therefore seek back to the start of its buffer without seeking the wrapped file. But even then, I don't think I'd use it this way. I'd construct the wrapper explicitly, and give it a method that returns a pos-restoring context manager.

I'm not sure this mode will work. If the stream can't use seek(), this context manager should simply let the exception bubble IMO.

Agreed. If you really want to, just try the context manager, and fall back to tee if it fails. A couple lines of code, and a lot more explicit and readable, and probably not that commonly needed...

On Mar 30, 2015, at 01:35, Dmitry Kazakov <jsbfox@gmail.com> wrote:

If the final resolution is "we don't need this in the library", that's fine - I agree that the proposal is trivial and doesn't gain much. OTOH the contextlib.closing can also be said to be a simple exercise, because its implementation is 22 lines long and it's equivalent to

@contextmanager def closing(thing): try: yield thing finally: thing.close()

(from the docs)

I suspect that if closing were proposed today (or if you were designing Python from scratch today) it never would have made it into contextlib. At the time, nobody had any experience with context managers. And there used to be tons of types that it was useful for; now, most of those types are context managers and therefore don't need closing. Python didn't come with pip, so having things in the stdlib was a lot more important. And it was one of the motivating examples in the PEP, which made it a look like a no-brainer addition. That doesn't mean there might not be a good argument for this idea. For example, if it turns out to be something that _looks_ trivial to code but is very easy to get subtly wrong, so a lot of people are likely to do it themselves and end up with buggy code, then it sounds like a good candidate. (Well, it would to me, but then I'm not who you have to convince...) But if you agree that it really is trivial, I don't see why it needs to be in the stdlib.