This would only be helpful when the CSV is on the disk but csv.reader() takes a file object so that it can used with anything like a socket for example. json.load() does the same thing.

There has been discussion about adding loading from a “path like” to the JSON lib. See this list about ten months ago and a recent thread on discuss. There resistance, but it ended with the idea that maybe there should be a PEP for a common interface for all “file” readers — eg JSON, CSV, etc.. And that interface could be supported by third party libs. That interface *maybe* would include a single step load from a path-like functionality. It seems to me that it is better to keep a generic interface that are

composable.

No one is suggesting removing the load-from-a-file-like interface. I have no idea why the fact that some people sometimes need a more flexible interface (maybe most people, even) somehow means that we shouldn’t make things easy and obvious for a common use case. -CHB

Cheers, Rémi

? And something similar for ‘csv.reader’? I’m not wedded to the details here.

The two main reasons I think this might be a positive addition are -

* you wouldn’t have to know or remember the right way to open a CSV file (newline=“”). * it elides very common code.

but perhaps there are things I’m missing here?

As a side note, I think ‘csv.reader’ could usefully be renamed to something else (maybe just Reader?), since it’s kind of out of sync with the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at foolish consistency :).

best, —titus

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCT... Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJQF... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Sun, Sep 5, 2021 at 10:32 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

Most Pandas read methods take either a path-like argument or a file-like argument, and figure out which it is by introspection when called. Actually, most of them even accept a URL-like argument as well

I don't think this is a terrible approach. It doesn't make things quite as explicit as the standard library generally does.

The folks in favor of adding this to json are split between overloading load() (my preference) and adding a new, explicit method: loadf() or something like that. Either way, it's useful. I honestly don't get the objection. MY takle is that mosto f the core devs are doing "systsms programming" rather than "scripting" -- which means that they a: Want to be more explicit and robust Need more flexibiilty around various file-like objects Don't mind a bit more expertise required. (e.g. specifying the encoding correctly) Don't mind a couple extra lines of code. So why add a function to make it simple and easy to read a file fro a path-like? I really wish the core devs would remember how useful Python is as a scripting language, and how many people use it that way. Pandas is a good example, I lot of folks use it in a scripting context, so it provided features that make it easy to do so. -CHB On Sun, Sep 5, 2021, 1:19 PM Christopher Barker <pythonchb@gmail.com> wrote:

...

This would only be helpful when the CSV is on the disk but csv.reader()

...

takes a file object so that it can used with anything like a socket for example. json.load() does the same thing.

There has been discussion about adding loading from a “path like” to the JSON lib. See this list about ten months ago and a recent thread on discuss.

There resistance, but it ended with the idea that maybe there should be a PEP for a common interface for all “file” readers — eg JSON, CSV, etc.. And that interface could be supported by third party libs. That interface *maybe* would include a single step load from a path-like functionality.

It seems to me that it is better to keep a generic interface that are

...

composable.

No one is suggesting removing the load-from-a-file-like interface. I have no idea why the fact that some people sometimes need a more flexible interface (maybe most people, even) somehow means that we shouldn’t make things easy and obvious for a common use case.

-CHB

...

Cheers, Rémi

? And something similar for ‘csv.reader’? I’m not wedded to the details here.

The two main reasons I think this might be a positive addition are -

* you wouldn’t have to know or remember the right way to open a CSV file (newline=“”). * it elides very common code.

but perhaps there are things I’m missing here?

As a side note, I think ‘csv.reader’ could usefully be renamed to something else (maybe just Reader?), since it’s kind of out of sync with the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at foolish consistency :).

best, —titus

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCT... Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJQF... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris)

Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GAPBNB... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

Thanks for your comments, everyone! Right, I’m struggling to figure out Greg's example :). Maybe Greg missed that DictReader.open didn’t exist and was in fact what I was asking for! (contextlib.closing is great, thank you for that!) Anyway, just to re-up the original points and add a few - * opening a CSV file with the right newline setting and then applying csv.reader and csv.DictReader is super common. * newline=“” has important ramifications for Windows functionality, as we <ahem> recently discovered when we tried to extend sourmash with windows compat. * yes it’s very easy to write my own utility function to do this, and in fact I have done so …repeatedly. :) * no one is proposing to remove any functionality. * I like Chris Barker’s comment, “”” it ended with the idea that maybe there should be a PEP for a common interface for all “file” readers — eg JSON, CSV, etc.. And that interface could be supported by third party libs. That interface *maybe* would include a single step load from a path-like functionality. “”” I’m +0 on David Mertz’s suggestion, “”" Most Pandas read methods take either a path-like argument or a file-like argument, and figure out which it is by introspection when called. Actually, most of them even accept a URL-like argument as well I don't think this is a terrible approach. It doesn't make things quite as explicit as the standard library generally does. But it's convenient, and there's no real ambiguity. “”” mostly because when we’ve done this in our own packages, I've struggled to figure out the best method to figure out if something is file-like. For example, it looks like ‘csv.DictReader’ will take any iterable, which means passing in a string is problematic; perhaps we could be looking for read or readline instead? Codifying that in some standard way could be nice. best, —titus

On Sep 5, 2021, at 5:30 PM, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:

On Mon, 6 Sept 2021 at 01:13, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

On 6/09/21 3:07 am, C. Titus Brown via Python-ideas wrote:

...

with csv.DictReader.open(filename) as r: for row in r: …

You can do this now:

from contextlib import closing with closing(csv.DictReader.open(filename)) as r: ...

What version of Python are you using?

...

...

...

import csv csv.DictReader.open Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: type object 'DictReader' has no attribute 'open'

...

IMO this is preferable than going around adding context manager methods to everything that has open-like functionality.

I disagree. It would be better if resource acquisition (e.g. opening a file) always took place in an __enter__ method so that it could always be under control of a with statement. Having closing as a separate function negates that because there has to be a separate function that acquires the resource before the closing function is called and hence before __enter__ is called.

-- Oscar _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AR6IHZ... Code of Conduct: http://python.org/psf/codeofconduct/

I think the point here is not the context manager, but rather, having the reader open the file itself, rather than taking an already open file-like object. And if it’s going to do that, it should provide. Context manager. Personally, while we are at it, I’d like to see a “read all” method, analogous to file.readlines(). For scripting use cases, reading a standard file format should be a one liner. -CHB On Sun, Sep 5, 2021 at 5:31 PM Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:

On Mon, 6 Sept 2021 at 01:13, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

On 6/09/21 3:07 am, C. Titus Brown via Python-ideas wrote:

...

with csv.DictReader.open(filename) as r: for row in r: …

You can do this now:

from contextlib import closing with closing(csv.DictReader.open(filename)) as r: ...

What version of Python are you using?

...

...

...

import csv csv.DictReader.open Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: type object 'DictReader' has no attribute 'open'

...

IMO this is preferable than going around adding context manager methods to everything that has open-like functionality.

I disagree. It would be better if resource acquisition (e.g. opening a file) always took place in an __enter__ method so that it could always be under control of a with statement. Having closing as a separate function negates that because there has to be a separate function that acquires the resource before the closing function is called and hence before __enter__ is called.

-- Oscar _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AR6IHZ... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython