Mailman 3 November 2006 - Python-Dev

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by glyph＠divmod.com 30 Nov '06

30 Nov '06

On 09:34 am, jack.jansen(a)cwi.nl wrote: >There's another standard place that is searched on MacOS: a per-user >package directory ~/Library/Python/2.5/site-packages (the name "site- >packages" is a misnomer, really). Standardising something here is >less important than for vendor-packages (as the effect can easily be >gotten by adding things to PYTHONPATH) but it has one advantage: >distutils and such could be taught about it and provide an option to >install either systemwide or for the current user only. Yes, let's do that, please. I've long been annoyed that site.py sets up a local user installation directory, a very useful feature, but _only_ on OS X. I've long since promoted my personal hack to add a local user installation directory into a public project -- divmod's "Combinator" -- but it would definitely be preferable for Python to do something sane by default (and have setuptools et. al. support it). I'd suggest using "~/.local/lib/pythonX.X/site-packages" for the "official" UNIX installation location, since it's what we're already using, and ~/.local seems like a convention being slowly adopted by GNOME and the like. I don't know the cultural equivalent in Windows - "%USERPROFILE%\Application Data\PythonXX" maybe? It would be nice if site.py would do this in the same place as it sets up the "darwin"-specific path, and to set that path as a module global, so packaging tools could use "site.userinstdir" or something. Right now, if it's present, it's just some random entry on sys.path.

6 9

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by Phillip J. Eby 30 Nov '06

30 Nov '06

At 02:46 PM 11/30/2006 -0800, Mike Orr wrote: >Speaking of Virtual Python [1], I've heard some people recommending it >as a general solution to the "this library breaks that other >application" problem and "this app needs a different version of X >library than that other app does". It was actually created to help people who don't have root access (e.g. in shared web hosting accounts) get their "own" Python. It does work for other things, but I haven't been especially recommending it for anything besides that, since eggs take care of multi-version app/library support quite nicely. Indeeed, the virtual-python approach is itself unnecessary now, as easy_install has much better PYTHONPATH support now, than when virtual-python was created.

1 0

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by glyph＠divmod.com 30 Nov '06

30 Nov '06

On 29 Nov, 11:49 pm, barry(a)python.org wrote: >On Nov 29, 2006, at 5:18 AM, glyph(a)divmod.com wrote: >> I'd suggest using "~/.local/lib/pythonX.X/site-packages" for the >> "official" UNIX installation location, ... >+1 from me also for the concept. I'm not sure I like ~/.local though >- -- it seems counter to the app-specific dot-file approach old >schoolers like me are used to. OTOH, if that's a convention being >promoted by GNOME and other frameworks, then I don't have too much >objection. Thanks. I just had a look at the code in Combinator which sets this up and it turns out it's horribly inconsistent and buggy. It doesn't really work on any platform other than Linux. I'll try to clean it up in the next few days so it can serve as an example. GNOME et. al. aren't promoting the concept too hard. It's just the first convention I came across. (Pardon the lack of references here, but it's very hard to google for "~/.local" - I just know that I was looking for a convention when I wrote combinator, and this is the one I found.) The major advantage ~/.local has for *nix systems is the ability to have a parallel *bin* directory, which provides the user one location to set their $PATH to, so that installed scripts work as expected, rather than having to edit a bunch of .foorc files to add to your environment with each additional package. After all, what's the point of a per-user "install" if the software isn't actually installed in any meaningful way, and you have to manually edit your shell startup scripts, log out and log in again anyway? Another nice feature there is that it uses a pre-existing layout convention (bin lib share etc ...) rather than attempting to build a new one, so the only thing that has to change about the package installation is the root. Finally, I know there are quite a few Python developers out there already using Combinator, so at least there it's an established convention :). >I also think that setuptools has the potential to be a big >improvement here because it's much easier to install and use egg >files than it is to get distutils to DTRT with setup.py. (I still >detest the command name 'easy_install' but hey that's still fixable >right? :). What might be nice would be to build a little more >infrastructure into Python to support eggs, by say adding a default >PEP 302 style importer that knows how to search for eggs in >'nests' (a directory containing a bunch of eggs). One of the things that combinator hacks is where distutils thinks it should install to - when *I* type "python setup.py install" nothing tries to insert itself into system directories (those are for Ubuntu, not me) - ~/.local is the *default* install location. I haven't managed to make this feature work with eggs yet, but I haven't done a lot of work with setuptools. On the "easy_install" naming front, how about "layegg"? >What if then that importer were general enough (...) These all sound like interesting ideas, but they're starting to get pretty far afield - I wish I had more time to share ideas about packaging, but I know too well that I'm not going to be able to back them up with any implementation effort. I'd really like Python to use the ~/.local/bin / ~/.local/lib convention for installing packages, though.

7 7

Re: [Python-Dev] Small tweak to tokenize.py?
by Phillip J. Eby 30 Nov '06

30 Nov '06

At 10:28 AM 11/30/2006 -0800, Guido van Rossum wrote: >Are you opposed changing tokenize? If so, why (apart from >compatibility)? Nothing apart from compatibility. I think you should have to explicitly request the new behavior(s), since tools (like detokenize) written to work around the old behavior might behave oddly with the change. Mainly, though, I thought you might find the code useful, given the nature of your project. (Although I suppose you've probably already written something similar.)

2 1

Re: [Python-Dev] Small tweak to tokenize.py?
by python＠rcn.com 30 Nov '06

30 Nov '06

> It would be trivial to add another yield to tokenize.py when > the backslah is detected +1 > I think that it should probably yield a single NL pseudo-token > whose value is a backslash followed by a newline; or perhaps it > should yield the backslash as a comment token, or as a new token. The first option is likely the most compatible with existing uses of tokenize. If a comment token were emitted, an existing colorizer or pretty-printer would markup the continuation as a comment (possibly not what the tool author intended). If a new token were created, it might break if-elif-else chains in tools that thought they knew the universe of possible token types. Raymond

1 0

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by Phillip J. Eby 30 Nov '06

30 Nov '06

At 06:02 PM 11/30/2006 +0000, glyph(a)divmod.com wrote: >On 05:37 pm, janssen(a)parc.com wrote: > >Perhaps "pyinstall"? > >Keep in mind that Python packages will still generally be >*system*-installed with other tools, like dpkg (or apt) and rpm, on >systems which have them. The name of the packaging system we're talking >about is called either "eggs" or "setuptools" depending on the context. Just as an FYI, the (planned) name of the packaging program for setuptools is "nest". It doesn't exist yet, however, except for a whole lot of design notes in my outlining program. You'll be able to use commands like "nest list" to show installed projects, "nest source" to fetch a project's source, "nest rm" or "nest uninstall" to uninstall, etc. It's all 100% vaporware at the moment, but that's the plan. I actually looked at other system package managers written in Python (i.e. yum and smart) to use as a possible base for implementing "nest", but unfortunately these are all GPL'd and thus not compatible with the setuptools or Python licenses, so I didn't actually get very far in my evaluation.

1 0

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by glyph＠divmod.com 30 Nov '06

30 Nov '06

On 05:37 pm, janssen(a)parc.com wrote: >Perhaps "pyinstall"? Keep in mind that Python packages will still generally be *system*-installed with other tools, like dpkg (or apt) and rpm, on systems which have them. The name of the packaging system we're talking about is called either "eggs" or "setuptools" depending on the context. "pyinstall" invites confusion with "the Python installer", which is a different program, used to install Python itself on Windows. It's just a brand. If users can understand that "Excel" means "Spreadsheet", "Outlook" means "E-Mail", and "GIMP" means "Image Editor", then I think we should give them some credit on being able to figure out what the installer program is called. (I don't really care that much in this particular case, but this was one of my pet peeves with GNOME a while back. There was a brief change to the names of everything in the menus to remove all brand-names: "Firefox" became "Web Browser", "Evolution" became "E-Mail", "Rhythmbox" became "Music Player". I remember looking at my applications menu and wondering which of the 3 "music players" that I had installed the menu would run. Thankfully this nonsense stopped and they compromised on names like "Firefox Web Browser" and "GIMP Image Editor".)

1 0

Small tweak to tokenize.py?
by Guido van Rossum 30 Nov '06

30 Nov '06

I've got a small tweak to tokenize.py that I'd like to run by folks here. I'm working on a refactoring tool for Python 2.x-to-3.x conversion, and my approach is to build a full parse tree with annotations that show where the whitespace and comments go. I use the tokenize module to scan the input. This is nearly perfect (I can render code from the parse tree and it will be an exact match of the input) except for continuation lines -- while the tokenize gives me pseudo-tokens for comments and "ignored" newlines, it doesn't give me the backslashes at all (while it does give me the newline following the backslash). It would be trivial to add another yield to tokenize.py when the backslah is detected: --- tokenize.py (revision 52865) +++ tokenize.py (working copy) @@ -370,6 +370,8 @@ elif initial in namechars: # ordinary name yield (NAME, token, spos, epos, line) elif initial == '\\': # continued stmt + # This yield is new; needed for better idempotency: + yield (NL, initial, spos, (spos[0], spos[1]+1), line) continued = 1 else: if initial in '([{': parenlev = parenlev + 1 (Though I think that it should probably yield a single NL pseudo-token whose value is a backslash followed by a newline; or perhaps it should yield the backslash as a comment token, or as a new token. Thoughts?) This wouldn't be 100% backwards compatible, so I'm not dreaming of adding this to 2.5.1, but what about 2.6? (There's another issue with tokenize.py too -- when you use it to parse Python-like source code containing non-Python operators, e.g. '?', it does something bogus.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by Phillip J. Eby 29 Nov '06

29 Nov '06

At 05:10 AM 11/30/2006 +0000, glyph(a)divmod.com wrote: >On 04:36 am, pje(a)telecommunity.com wrote: > > >easy_install uses the standard distutils configuration system, which means > >that you can do e.g. > >Hmm. I thought I knew quite a lot about distutils, but this particular >nugget had evaded me. Thanks! I see that it's mentioned in the >documentation, but I never thought to look in that section. I have an >aversion to .ini files; I tend to assume there's always an equivalent >Python expression, and it's better. Is there an equivalent Python API in >this case? Well, in a setup.py there's an options or some such that can be used to provide effective command-line option overrides in-line, but that doesn't help for systemwide default configurations, like the files I mentioned. It's effectively only a substitute for setup.cfg.

1 0

Re: [Python-Dev] Python and the Linux Standard Base (LSB)
by glyph＠divmod.com 29 Nov '06

29 Nov '06

On 04:36 am, pje(a)telecommunity.com wrote: >easy_install uses the standard distutils configuration system, which means >that you can do e.g. Hmm. I thought I knew quite a lot about distutils, but this particular nugget had evaded me. Thanks! I see that it's mentioned in the documentation, but I never thought to look in that section. I have an aversion to .ini files; I tend to assume there's always an equivalent Python expression, and it's better. Is there an equivalent Python API in this case? I don't know if this is a personal quirk of mine, or a reinforcement of Talin's point about the audience for documentation documentation.

1 0