Mailman 3 'from os.path import FILE, DIR' or internal structure of filenames - Python-ideas

'from os.path import FILE, DIR' or internal structure of filenames

anatoly techtonik

28 Sep 2013 28 Sep '13

5:19 a.m.

FILE = os.path.abspath(__file__) DIR = os.path.abspath(os.path.dirname(__file__)) ? Repeated pattern for referencing resources relative to your scripts. Ideas about alternative names / locations are welcome. In PHP these are __FILE__ and __DIR__. For Python 3 adding __dir__ is impossible, because the name clashes with __dir__ method (which is not implemented for module object, but should be [ ] for consistency). Also current __file__ is rarely absolute path, because it is never normalized [ ]. So it will be nice to see normalization of Python file name after the import to reduce mess and make its behaviour predictable - http://stackoverflow.com/questions/7116889/python-file-attribute-absolute-or... ----[ possible spec. draft for a beautiful internal structure ]-- The Python interpreter should provide run-time information about: 1. order of import sequence 2. names of imported modules 3. unique location for each imported module which unambiguously identifies it 4. run-time import dependency tree (not sure about this, but it can help with debugging) 5. information about sys.path entry where this module was imported from 6. information about who and when added this sys.path entry -- anatoly t.

Attachments:

attachment.htm (text/html — 1.5 KB)

Show replies by date

Philipp A.

28 Sep 28 Sep

11:34 a.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

as much as i would like the convenience, python has very few magic globals, and they all have names encased in 4 underscores. if we really add more globals, why not __abs_file__ and __abs_dir__ or sth. like that? 2013/9/28 anatoly techtonik

...

FILE = os.path.abspath(__file__) DIR = os.path.abspath(os.path.dirname(__file__)) ?

Repeated pattern for referencing resources relative to your scripts. Ideas about alternative names / locations are welcome.

In PHP these are __FILE__ and __DIR__. For Python 3 adding __dir__ is impossible, because the name clashes with __dir__ method (which is not implemented for module object, but should be [ ] for consistency). Also current __file__ is rarely absolute path, because it is never normalized [ ].

So it will be nice to see normalization of Python file name after the import to reduce mess and make its behaviour predictable - http://stackoverflow.com/questions/7116889/python-file-attribute-absolute-or...

----[ possible spec. draft for a beautiful internal structure ]-- The Python interpreter should provide run-time information about: 1. order of import sequence 2. names of imported modules 3. unique location for each imported module which unambiguously identifies it 4. run-time import dependency tree (not sure about this, but it can help with debugging) 5. information about sys.path entry where this module was imported from 6. information about who and when added this sys.path entry -- anatoly t.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas

MRAB

4:51 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On 28/09/2013 12:34, Philipp A. wrote:

...

as much as i would like the convenience, python has very few magic globals, and they all have names encased in 4 underscores.

if we really add more globals, why not __abs_file__ and __abs_dir__ or sth. like that?

+1 1. Do we need them? 2. If we do, then I agree with __abs_file__ and __abs_dir__.

...

2013/9/28 anatoly techtonik mailto:techtonik@gmail.com>

FILE = os.path.abspath(__file__) DIR = os.path.abspath(os.path.dirname(__file__)) ?

Repeated pattern for referencing resources relative to your scripts. Ideas about alternative names / locations are welcome.

In PHP these are __FILE__ and __DIR__. For Python 3 adding __dir__ is impossible, because the name clashes with __dir__ method (which is not implemented for module object, but should be [ ] for consistency). Also current __file__ is rarely absolute path, because it is never normalized [ ].

So it will be nice to see normalization of Python file name after the import to reduce mess and make its behaviour predictable - http://stackoverflow.com/questions/7116889/python-file-attribute-absolute-or...

----[ possible spec. draft for a beautiful internal structure ]-- The Python interpreter should provide run-time information about: 1. order of import sequence 2. names of imported modules 3. unique location for each imported module which unambiguously identifies it 4. run-time import dependency tree (not sure about this, but it can help with debugging) 5. information about sys.path entry where this module was imported from 6. information about who and when added this sys.path entry

Cameron Simpson

11:26 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On 28Sep2013 08:19, anatoly techtonik wrote: | FILE = os.path.abspath(__file__) | DIR = os.path.abspath(os.path.dirname(__file__)) | ? | | Repeated pattern for referencing resources relative to your scripts. Ideas | about alternative names / locations are welcome. | | In PHP these are __FILE__ and __DIR__. For Python 3 adding __dir__ is | impossible, because the name clashes with __dir__ method (which is not | implemented for module object, but should be [ ] for consistency). Also | current __file__ is rarely absolute path, because it is never normalized [ | ]. Maybe I'm grumpy this morning (though I felt the same reading this yesterday). -1 for any names commencing with __ (or even _). -1 for new globals. -1 because I can imagine wanting different nuances on the definitions above; in particular for DIR I can well imagine wanting bare dirname(abspath(FILE)) - semanticly different to your construction. There's lots of scope for bikeshedding here. -1 because this is trivial trival code. -1 because you can do all this with relative paths anyway, no need for abspath -1 because I can imagine being unable to compute abspath in certain circumstances ( certainly on older UNIX systems you could be inside a directory without sufficient privileges to walk back up the tree for getcwd and its equivalents ) -0 for adding some kind of convenience functions to importlib(?) for this (+0 except that I can see heaps of bikeshedding) Cheers, -- Cameron Simpson In an insane society, the sane man must appear insane. - Keith A. Schauer

Nick Coghlan

29 Sep 29 Sep

12:21 a.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

Note that any remaining occurrences of non-absolute values in __file__ are generally considered bugs in the import system. However, we tend not to fix them in maintenance releases, since converting relative paths to absolute paths runs a risk of breaking user code. We're definitely *not* going to further pollute the module namespace with values that can be trivially and reliably derived from existing values. Cheers, Nick.

INADA Naoki

3:28 a.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

os.path.abspath(__file__) returns wrong path after chdir. So I don't think abspath of module can be trivially and reliably derived from existing values. $ cat foo.py import os print(os.path.abspath(__file__)) os.chdir('work') print(os.path.abspath(__file__)) $ python foo.py /home/inada-n/foo.py /home/inada-n/work/foo.py On Sun, Sep 29, 2013 at 9:21 AM, Nick Coghlan wrote:

...

Note that any remaining occurrences of non-absolute values in __file__ are generally considered bugs in the import system. However, we tend not to fix them in maintenance releases, since converting relative paths to absolute paths runs a risk of breaking user code.

We're definitely *not* going to further pollute the module namespace with values that can be trivially and reliably derived from existing values.

Cheers, Nick.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas

-- INADA Naoki

Nick Coghlan

6:15 a.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On 29 September 2013 13:28, INADA Naoki wrote:

...

os.path.abspath(__file__) returns wrong path after chdir. So I don't think abspath of module can be trivially and reliably derived from existing values.

Hence the part about any remaining instances of non-absolute __file__ values being considered a bug in the import system. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

anatoly techtonik

5:36 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On Sun, Sep 29, 2013 at 9:15 AM, Nick Coghlan wrote:

...

On 29 September 2013 13:28, INADA Naoki wrote:

...
os.path.abspath(__file__) returns wrong path after chdir. So I don't think abspath of module can be trivially and reliably derived from existing values.

Hence the part about any remaining instances of non-absolute __file__ values being considered a bug in the import system.

Bug that will not be fixed, i.e. a wart. And as a result we don't have a way to reliably reference filename of the current script and its directory. Hence the proposal. -- anatoly t.

Terry Reedy

7:13 p.m.

On 9/29/2013 1:36 PM, anatoly techtonik wrote:

...

On Sun, Sep 29, 2013 at 9:15 AM, Nick Coghlan wrote:

...
On 29 September 2013 13:28, INADA Naoki wrote:

...
os.path.abspath(__file__) returns wrong path after chdir. So I don't think abspath of module can be trivially and reliably derived from existing values.

Hence the part about any remaining instances of non-absolute __file__ values being considered a bug in the import system.

Bug that will not be fixed, i.e. a wart.

Nick said "we tend not to fix them in maintenance releases", which I take to mean we can fix in new versions.

...

And as a result we don't have a way to reliably reference filename of the current script and its directory. Hence the proposal.

The proposed addition would not happen in maintenance releases either. -- Terry Jan Reedy

Nick Coghlan

10:18 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On 30 Sep 2013 05:14, "Terry Reedy" wrote:

...

On 9/29/2013 1:36 PM, anatoly techtonik wrote:

...
On Sun, Sep 29, 2013 at 9:15 AM, Nick Coghlan wrote:

...
On 29 September 2013 13:28, INADA Naoki wrote:

...
os.path.abspath(__file__) returns wrong path after chdir. So I don't think abspath of module can be trivially and reliably

derived

...

...
...
...
from existing values.

Hence the part about any remaining instances of non-absolute __file__ values being considered a bug in the import system.

Bug that will not be fixed, i.e. a wart.

Nick said "we tend not to fix them in maintenance releases", which I take to mean we can fix in new versions.

Correct, it's the kind of arguably backwards incompatible bug fix that users will generally tolerate in a feature release but would be justifiably upset about in a maintenance release. Cheers, Nick.

...

...
And as a result we don't have a way to reliably reference filename of the current script and its directory. Hence the proposal.

The proposed addition would not happen in maintenance releases either.

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas

anatoly techtonik

22 Oct 22 Oct

7:12 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On Mon, Sep 30, 2013 at 1:18 AM, Nick Coghlan wrote:

...

On 30 Sep 2013 05:14, "Terry Reedy" wrote:

...
On 9/29/2013 1:36 PM, anatoly techtonik wrote:

...
On Sun, Sep 29, 2013 at 9:15 AM, Nick Coghlan wrote:

...
On 29 September 2013 13:28, INADA Naoki wrote:

...
os.path.abspath(__file__) returns wrong path after chdir. So I don't think abspath of module can be trivially and reliably derived from existing values.

Hence the part about any remaining instances of non-absolute __file__ values being considered a bug in the import system.

Bug that will not be fixed, i.e. a wart.

Nick said "we tend not to fix them in maintenance releases", which I take to mean we can fix in new versions.

Correct, it's the kind of arguably backwards incompatible bug fix that users will generally tolerate in a feature release but would be justifiably upset about in a maintenance release.

Cheers, Nick.

...
...
And as a result we don't have a way to reliably reference filename of the current script and its directory. Hence the proposal.

The proposed addition would not happen in maintenance releases either.

Python 3.4? -- anatoly t.

Terry Reedy

29 Sep 29 Sep

7:19 p.m.

On 9/28/2013 11:28 PM, INADA Naoki wrote:

...

os.path.abspath(__file__) returns wrong path after chdir.

So grab the path before chdir (which most programs do not do).

...

So I don't think abspath of module can be trivially and reliably derived from existing values.

It apparently can if you do so in a timely fashion. Grabbing it as soon as possible is the obvious time to do it.

...

$ cat foo.py import os

foopath = os.path.abspath(__file__) Now print it or do whatever you want with it. -- Terry Jan Reedy

anatoly techtonik

5:39 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On Sun, Sep 29, 2013 at 2:26 AM, Cameron Simpson wrote:

...

On 28Sep2013 08:19, anatoly techtonik wrote: | FILE = os.path.abspath(__file__) | DIR = os.path.abspath(os.path.dirname(__file__)) | ? | | Repeated pattern for referencing resources relative to your scripts. Ideas | about alternative names / locations are welcome. | | In PHP these are __FILE__ and __DIR__. For Python 3 adding __dir__ is | impossible, because the name clashes with __dir__ method (which is not | implemented for module object, but should be [ ] for consistency). Also | current __file__ is rarely absolute path, because it is never normalized [ | ].

Maybe I'm grumpy this morning (though I felt the same reading this yesterday).

-1 for any names commencing with __ (or even _).

-1 for new globals.

-1 because I can imagine wanting different nuances on the definitions above; in particular for DIR I can well imagine wanting bare dirname(abspath(FILE)) - semanticly different to your construction. There's lots of scope for bikeshedding here.

-1 because this is trivial trival code.

-1 because you can do all this with relative paths anyway, no need for abspath

-1 because I can imagine being unable to compute abspath in certain circumstances ( certainly on older UNIX systems you could be inside a directory without sufficient privileges to walk back up the tree for getcwd and its equivalents )

-0 for adding some kind of convenience functions to importlib(?) for this (+0 except that I can see heaps of bikeshedding)

With all -1 above, what is your preferred way to refer to resources that are places into subdirectories of your script directory? -- anatoly t.

Paul Moore

6:16 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On 29 September 2013 18:39, anatoly techtonik wrote:

...

With all -1 above, what is your preferred way to refer to resources that are places into subdirectories of your script directory?

If you are an imported module, pkgutil.get_data (because that handles modules in zipfiles, etc). Otherwise, if you are running a single-file script: with open(os.path.join(os.path.dirname(__file__), 'path', 'to', 'my', 'data')) as f: data = f.read() Why is this a problem? Paul

Cameron Simpson

10:17 p.m.

New subject: 'from os.path import FILE, DIR' or internal structure of filenames

On 29Sep2013 20:39, anatoly techtonik wrote: | On Sun, Sep 29, 2013 at 2:26 AM, Cameron Simpson wrote: | > Maybe I'm grumpy this morning (though I felt the same reading this yesterday). [...] | > -1 because this is trivial trival code. | > -1 because you can do all this with relative paths anyway, no need for abspath [...] | With all -1 above, what is your preferred way to refer to resources | that are places into subdirectories of your script directory? Probably os.path.join(os.path.dirname(__file__), "datafile-here"). I've got some unit tests that want that. No need for abspath at all. Of course, chdir and passing paths to programs-which-are-not-my-children present scope for wanting abspath, but in the very common simple case: unnecessary and therefore undesirable. And I'm aware that modules-inside-zip-files don't work with this; let us ignore that; they won't work with abspath either:-) It is so trite that I can't imagine wanting to bolt it into the stdlib. Cheers, -- Cameron Simpson Thousands of years ago the Egyptians worshipped cats as gods. Cats have never forgotten this. - David Wren-Hardin

3832

Age (days ago)

3856

Last active (days ago)

List overview

Download

14 comments

8 participants

participants (8)

anatoly techtonik
Cameron Simpson
INADA Naoki
MRAB
Nick Coghlan
Paul Moore
Philipp A.
Terry Reedy

'from os.path import FILE, DIR' or internal structure of filenames

Cameron Simpson

Cameron Simpson

tags

participants (8)