I have patches up on SF for two ConfigParser bugs, which I'd like to check in ASAP. Any objections? http://www.python.org/sf/1017864: Case sensitivity bug in ConfigParser ====================================================================== This is a simple bug relating to default values passed in as dictionaries, illustrated here:
The patch applies ConfigParser.optionxform to keys of defaults when supplied, to be consistent with the handling of keys of config file entries and runtime-set options. A test patch is also included. http://www.python.org/sf/997050: ConfigParser behavior change ============================================================= This may be more controversial. In Python 2.3 and earlier, ConfigParser implicitly allowed non-string values to be set. http://python.org/sf/810843 asked for clarification, and in rev 1.65 an explicit type check was added that raises TypeError. The problem is that this breaks Docutils code, and I suspect it will break other code as well, which we won't hear about until after Python 2.4 final is released. Setting non-string values worked just fine for application-internal use, as long as interpolation is turned off (RawConfigParser is used or ConfigParser with "raw" parameter set) and config files aren't written. A new SafeConfigParser class was added in Python 2.3: New applications should prefer this version if they don't need to be compatible with older versions of Python. My solution is to add the new string-only restriction to the SafeConfigParser class, leave existing ConfigParser & RawConfigParser behavior alone, and document the conditions under which non-string values work. Code, doc, and test patches included. What say you? -- David Goodger <http://python.net/~goodger>
[David Goodger]
Instead of an explicit type check, it may be better to wrap the interpolation step in a try/except. For non-string objects, raise a TypeError with an informative error message. This would meet the OP's need for an informative error message while leaving the module backwards compatible for internal uses of ConfigParser. Ideally, the docs should discourage further non-string uses and advise that ConfigParser will be string only for Py3.0. Raymond
On Friday 01 October 2004 04:07 pm, Raymond Hettinger wrote:
Ideally, the docs should discourage further non-string uses and advise that ConfigParser will be string only for Py3.0.
Yikes! Can't we just toss it for Py3K? This module just hasn't held up, and exposes a really poor model even for .ini-style configuration files. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
Fred L. Drake, Jr. wrote:
Ditto from me. Personally I would want something that used XML (like property lists on OS X), but I realize people still want a config file style that is easy to modify in a text editor, so I don't see the .ini style going away. But we could stand to come up with a uniform interface that both an XML and .ini config file parsers could use for consistency and thus support both styles. Wasn't there talk for a while of doing an shootout of config file packages like we did for optparse? -Brett
Hear hear.
Well, for me personally, .ini style config files still win over XML every day. And I now have significant experience with both here at ESI. What sucks (relatively) is the specific way that ConfigParser provides access to .ini files; I always end up writing a wrapping layer around it.
Wasn't there talk for a while of doing an shootout of config file packages like we did for optparse?
There are two different axes here: the config file format and the API for accessing it. I'm not sure that attempting to provide a single API for both XML and .ini files will work; the API should reflect the structure of the file (to some extent) and that's just too different. But I may be wrong; I do see some similarities: What I have come to like, both for dealing with XML and with .ini files, is something that lets you map the configuration values (or whatever it is that you're parsing, really) to Python attributes. This is really some kind of simplified DTD support, and there are different ways to go about it; but the net result is that you end up writing e.g. options.client.max_retries (which is an int with a default value) rather than options.getint("client", "max-retries"). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido van Rossum]
Unfortunately ConfigParser discourages such wrappers by hiding too much of its implementation as _private attributes (not __class_privates, so they *are* accessible, but even single-underscore _privacy screams "don't touch this -- subject to change without notice"). Back on topic though, I'd like to see +/-'s on the two patches. The first one (http://www.python.org/sf/1017864) is IMO a no-brainer. The second (http://www.python.org/sf/997050) is what I'm really interested in: do we maintain the existing API and functionality, or is it OK to break existing code by *removing* functionality? -- David Goodger <http://python.net/~goodger>
[Guido] ...
LOL. I wrote a Config wrapper for SpamBayes like that. Then I stopped paying attention, and next thing I knew all the option-accessing code had been changed to look like SB's current prob = options["Classifier", "unknown_word_prob"] instead. But even that's better than options.getint("client", "max-retries")! Encoding the return type at every access point is Plain Wrong.
Guido van Rossum wrote: [SNIP]
OK. Do realize that plists are basically .ini style just expressed in XML:: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Section</key> <dict> <key>key</key> <string>value</string> <key>key2</key> <string>value2</string> </dict> </dict> </plist> I am not thinking of anything fancy or beyond something like this; .ini files expressed in XML. Just thinking that XML might be nice since all of those poor souls who don't use Python have easy access to an XML parser but not necessarily a .ini file parser.
For free-wheeling XML, yes, I would agree, but if we restrict the DTD/schema/spec to be fairly similar to .ini files then I don't think the structure of the files will be that drastic. And as long as people use the API to create/edit/save the files then we can enforce universal rules that makes sure that the difference is really just in the structure of the output.
That is what I am thinking. You have sections and then key/value pairs within the sections in both the XML and the .ini format. We then just agree on how case-sensitivity, naming, etc. works within the file formats and then provide a common-place interface much like the one you lay out above. Is this worth working on now or wait until Py3k? It probably seems like I am playing the PEP drum (not a fan of the harp =) a lot lately in regards to stuff that is going to not be an issue until Py3k, but the longer we have to work on something the better chance we have of having something that everyone likes. Plus for things like modules we can provide it for people now as an external module for experimentation and thus be a little willy-nilly with the API initially until we have it nailed and either integrate into the stdlib beofre Py3k or just wait until then. -Brett
This reveals IMO a big mistake in thinking about configuration files. The most important user of a config file is not the programmer who has to get data out of it; the most important user is the user who has to edit the config file. The outrageous verbosity of XML makes the above example a complete usability liability. Now, if you're talking about config files that represent options that the user edits in a convenient application-specific options dialog, that's a different story; I think XML is well-suited for that; but I'm talking about the classic configuration file pattern where you use your favorite flat-file text editor to edit the options file. In that situation, using XML is insane.
Is this worth working on now or wait until Py3k?
I see no advantage in waiting until Py3K; this is not a language issue and there is no problem with having several library modules (as long as it's clear which one is deprecated). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote: [SNIP]
I am thinking of the app-specific options dialog situation for XML and I do think that using XML for the classic style of editing the flat file by hand is insane. My thinking on the XML idea is that people do write apps where the whole thing is behind a GUI and thus XML makes good sense. So why not make there lives a little easier by giving them a basic API? But it is not a make-or-break thing for me so I am willing to let it go if it is deemed not worth the effort.
OK, so how do people want to proceeed with this? PEP? Shootout? -Brett
People who want to use XML already have all the rope they need. I would be in for adding something like my simplified XML handling class to the library, but I first have to lobby my employer to open-source it (not a problem, I just have to time the request right so it won't be in 2.4). Anyway, this needn't and shouldn't be specific to the problem of storing option values.
OK, so how do people want to proceeed with this? PEP? Shootout?
I'm all for working code in this case, so let's do a shootout. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
OK.
OK, so how do people want to proceeed with this? PEP? Shootout?
I'm all for working code in this case, so let's do a shootout.
OK. Should we do an announcement to python-announce and c.l.py saying we are looking for a new API for .ini file editing/accessing? -Brett
OK. Should we do an announcement to python-announce and c.l.py saying we are looking for a new API for .ini file editing/accessing?
I don't see the point -- it's not like this is the most important issue of the year. I would rather just see if any of the readers here happen to have code lying around that they like. It sounds like maybe the IPython stuff could be used after a thorough rewrite. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido van Rossum]
XML may sometimes be useful _when_ a configuration _has_ to be a tree of thick (contents heavy) nodes. For simpler configurations files, and this covers a flurry of cases, Guido above expresses my feelings exactly. There are XML-fanatics -- the same as there are Unicode-fanatics :-). Not so long ago, I've seen a pushy trend by which `/etc/passwd' and a lot of other simple system files would be reformulated as XML, to be XML all over. (Maybe after an announcement of Microsoft -- real or made-up, I do not know -- that they aim such a thing for the next "Windows".) We should not go overboard with XML. Configuration files with lines and words for structuring of a two-level hierarchy have long proven their value. ConfigParser `.ini' files add a third level to these. and with good sense, we can still go another long way without resorting to XML. Besides, when configuration files are going to have some more complex structure, but explicitly made to drive big applications written in Python -- a common case for most of us presumably -- I found out that Python itself is a very convenient and powerful tool for expressing such parameterisation. Users can easily modify a carefully designed Python-written configuration template without knowing a bit about Python itself. If they know the concepts of the applications they are configuring, in my experience at least, they quickly get what needs to be done, and even grasp by mere mimetism, without much explanations, the bits of syntax to respect and follow. With proper care, this can be made incredibly powerful, while staying _way_ more legible than XML. Also compare speed and simplicity, for the application programmer, for bringing such configuration in memory, type concerns already addressed. Of course, XML better inter-operates with foreign programming languages and systems, and would undoubtedly allow easier communication with alien races from neighbouring galaxies :-). But deep down, I do not think that this need in configuration neutrality is common enough to warrant the added complexity, and loss of legibility, for most practical cases. -- François Pinard http://pinard.progiciels-bpi.ca
Guido van Rossum wrote:
Plist is handy when: - you need a simple storage format for simple data structures - you want to be able to edit a file manually in exceptional situations If you need to *routinely* edit these files by hand than plist is indeed not the right choice. It's not a pretty format, but it's well defined. I find it extremely handy and use it all over the place. I'd be happy to contribute plistlib.py to the std library (ie. move it one level up from Lib/plat-mac/ ;-), but I doubt there's enough interest. Just
All this talk of a replacement for ConfigParser is all well and good (and I agree that a more pythonic interface would be nice)... but. We're under 2 weeks from beta1, and I can't see this new module being designed, implemented, and committed before then. Remember, once b1 is out, we're stuck with the API that we have come up with. I'd much rather see one or more packages developed alongside Python - we can then take the best of them (or maybe a merger of the best ideas for them) for 2.5. Which leaves David's original question about the two patches. While CP isn't perfect, and it would be nice to replace it, for 2.4, I don't think these patches make things any worse, and they add useful functionality (back, in one case) so I'm inclined towards accepting them. Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.
Of course. Other recent proposals for brand new stuff also seem poorly timed.
Right. So it went for optparse and logging.
If you think so, go ahead. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Quoting Guido van Rossum <gvanrossum@gmail.com>:
Of course. Other recent proposals for brand new stuff also seem poorly timed.
I may be guilty of that. Would it be worth adding a Python 2.5 group to the SF Patch tracker for things which shouldn't be considered seriously until after 2.4 final is out the door? Cheers, Nick. -- Nick Coghlan Brisbane, Australia
Of course. Other recent proposals for brand new stuff also seem
[Guido] poorly
timed.
[Nick]
Though ill timed, the -m idea is honking good. If it can be reliably worked out in its simplest form (not trying to be all things to all packages), it's worth going ahead for Py2.4. Command line guys like me need this every day. His patch will address a continual source of irritation and make the tool more pleasant to use. Raymond
I was in fact thinking of the -m proposal when I wrote that... It's so easy to define an alias or use a one-line shell script for invoking Python with a full pathname that I'm really not sure I like the -m idea, and I worry that it would be done the wrong way because it's so close to beta. Half the time the problem is getting the path set in the first place, and there -m doesn't help you. Supporting it with packages seems insanity. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
I was in fact thinking of the -m proposal when I wrote that...
The timing does suck.
My understanding was that it wasn't about a full pathname to python, it was about searching sys.path for the darned script so us poor Windows guys don't have to change directories a million times a day (no aliases or shebangs for us either). I have a big pile of batch files just to invoke timeit, texchecker, profile, etc. It bites the big one. If the patch doesn't get worked out for Py2.4, I hope it doesn't get abandoned.
Supporting it with packages seems insanity.
No doubt. Raymond
Quoting Raymond Hettinger <python@rcn.com>:
Correct. Ilya's original example was wanting to invoke pdb's script behaviour in order to debug another script. At the moment, you have to run: python -c "from inspect import getsourcefile; import pdb; print getsourcefile(pdb)" or python -c "from imp import find_module; print find_module("pdb")[1]" to find out where the file is, and then run python normally with the filename that gets printed by one of the above scripts. (With a Unix-style shell, backticks make that easier. Doesn't help on Windows though). The "-m" option aims to automate this process (using the latter 'find_module' approach in C code)
Supporting it with packages seems insanity.
No doubt.
I'm starting to lean that way myself. The semantics for a straight module are easy, but the package version is horrible (Either we have a package getting imported before "__main__" starts running, or else we don't properly support package __path__ variables. I don't find either option appealing) I think I'll simply drop all package support from the patch and allow modules only. That will still cover the initial use cases (pdb, profile and the like) Adding support for packages later (in a different patch) would still be possible if there was interest. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
On Sat, 2004-10-02 at 23:27, Nick Coghlan wrote:
+1 and I'd still love to see this in Python 2.4. Could you please assign the patch to someone other than me though? I don't honestly think I'll have time to review it before 2.4b1. -Barry
I'm not going to stop you from adding this, but I do think you're overlooking the fact that if you need this a lot, it's trivial to write a tiny Python script that takes the place of "python -m", and putting that python script in your PATH. Something like: #!/usr/bin/env python import sys, imp fn = imp.find_module(sys.argv[1])[1] del sys.argv[0] sys.argv[0] = fn execfile(fn) seems do the trick (though it could use nicer error handling). Wouldn't it be easier to add *this* to the Tools/scripts directory (and the list of scripts that are installed by "make install" or the Windows equivalent) than to burden the Python interpreter with C code to do the same? Why does every handy thing that anyone ever thought of have to be a Python command line option? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Quoting Guido van Rossum <gvanrossum@gmail.com>:
And is basically identical to the C version of this code (which uses _PyImp_FindModule and PyRun_SimpleFileExFlags). I like this idea, as it should automatically work with other versions of the interpreter like Jython and IronPython, whereas the command line option would required reimplementation in each interpreter. The hardest part would be to come up with a name for the script (pymod?, pymodule?, pyrun?, runpy?). It will probably be slightly more verbose to invoke than '-m' would be (particularly on an uninstalled development build), but also easier to enhance (e.g supporting modules inside packages). Since I'm on Linux at the moment, and can't recall how the Windows installer handles things like pydoc, I'll wait and see what Raymond and others think about how well this would work on Windows. Unless they think it won't work for other platforms, I'm inclined to reject my own patch and look at a script-based approach. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
[Guido]
Though not as clean as -m, this script helps. For my own insufficiently advanced windows box, I added a launcher batch file, pythonm.bat: @python \py24\tools\scripts\runpy.py %1 %2 %3 %4 %5 %6 %7 %8 %9 It runs pretty well: C:\tmp> pythonm timeit -s "a=range(20)" "a.append(1)" "a.pop()" 100000 loops, best of 3: 1.75 usec per loop C:\tmp> pythonm regrtest test_string test_string 1 test OK. The combination of batch file and run.py isn't as good as the -m option, but it works and mostly meets my needs. The batch file is rather dumb, doesn't work with other python command line options, and won't work pipes.
Why does every handy thing that anyone ever thought of have to be a Python command line option?
Could Nick's idea be done without an -m option? If a user types, "python abc.py" and "abc.py" is not found, before aborting, try looking for it on sys.path. Raymond
Quoting Raymond Hettinger <python@rcn.com>:
Hmm, that point about command line options is a good one. I know I'd find it handy to be able to throw in a '-i' when I wanted one. Then again for commands with minor variations, I tend to rely on command line history rather than using a batch file (that is, I'll type out a long invocation sequence once a session, then edit it as needed)
Could Nick's idea be done without an -m option?
To be fair to Ilya - his idea, my implementation :)
That's do-able, but the command line option seems cleaner if the interpreter is going to provide C-level support for this. I interpreted Guido's last message as saying he's prepared to tolerate the command line option if there's sufficient support for it. To summarise the differences between the two approaches: Pros (-m over runpy.py, in order of significance as I see it): - easy to combine with other Python command line options - OS & environment independent - easier to use with a non-installed interpreter - no work for Python packaging folks - more concise to invoke - no namespace issues with naming a script - C API for those embedding python Cons: - YACLO (Yet Another Command Line Option) - CPython specific (other interpreters could choose to reimplement) With YACLO seeming to be Guido's main reason for being -0 on the idea. It looks like the script option would require input from those with better knowledge than me of the packaging set up for Windows, *nix and Mac as to how it could be made to work 'nicely' in each environment. And the issue of combining it with other command line options would remain. It's all very well to have "runpy.py" as an executable script, but providing extra options to the interpreter would require: 1. Locate runpy.py (e.g. /usr/bin/local/runpy.py, /usr/bin/runpy.py) 2. Invoke python <options> <dir>/runpy.py <module> Which means we're back to having to find out where at least part of the Python installation lives on each machine (only once per machine, which is an improvement over the status quo, but still not as tidy as a single command line option). I find that persuasive enough that I think it's worthwhile to finish the '-m' patch, and add a documentation patch. Regards, Nick. -- Nick Coghlan Brisbane, Australia
Pros (-m over runpy.py, in order of significance as I see it): - easy to combine with other Python command line options
The script approach can do this too
- OS & environment independent
So is the script -- probably more so, since the script can use Python's OS independence layer.
- easier to use with a non-installed interpreter - no work for Python packaging folks
No contest.
- more concise to invoke
Depends on the length of the name of the script. :-)
- no namespace issues with naming a script
Actually, one-letter options are a much scarcer resource than script names.
- C API for those embedding python
And who needs that?
Additional Pros for using a script: - less code to maintain - can work with older Python versions - shows users how to do a similar thing themselves with additional features (e.g. a common addition I expect will be to hardcode a personal sys.path addition) PS a big honking -1000 on Carlos Ribeiro's suggestion of doing this automatically if the filename path isn't found. Too many chances for accidentally invoking the wrong thing -- including mysyerious silences when the named module happens to exist but doesn't do anything when run as a script (i.e. the majority of modules). (As an aside, I wonder why every single of Carlos' interactions starts of with an idea that is completely misguided and then ends with him vehemently defending it against all opposition. Sounds like a recipe for flame wars to me.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 4 Oct 2004 08:05:01 -0700, Guido van Rossum <gvanrossum@gmail.com> wrote:
Sorry. Really. I may have expressed myself VERY badly on this topic. Please, let me defend myself as I don't like to leave this impression as I'm just starting here: -- The original idea was not mine -- I was just commenting because it seemed good to me, as I'm limited to a DOS box most of the time, and searching modules by hand tend to become an issue in this enviroment; -- I really meant to say this (regarding YACLO): I *don't* think that adding command line options for everything is a good idea for any software. Also, I think that single line options are confusing, and longer option names are better/more readable. It seems that the message was read with the opposite meaning. -- As far as the rest of the comments are concerned, I apologize if I tend to sound so 'flamish'. Actually, I have a tendency to think aloud sometimes, and to explore different ideas. Sometimes it makes for amusing comments, sometimes it's weird, and sometimes it just sounds dumb. Sometimes I get it right. But overall it's not the right thing to do, specially as I'm yet to make some useful contribution. In the end, I think I should learn to keep my mouth/fingers shut. Lesson taken, and really, really, sorry. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com
The problem here is really one of packaging. How should utility scripts be distributed, either with Python, or with extensions? There are many different approaches: 1. a library module with useful behaviour in the __main__ block. 2. a script in sys.prefix + '/scripts' with no extension and a #! line 3. a script as above, but with a .py extension 4. a script as (2) plus a .bat wrapper for Windows 5. a script as (3) plus a .bat wrapper for Windows However, *none* of these give the best behaviour, uniformly across all Windows platforms (the Unix story is far better, as there is a consistent and useful behaviour across all shells). While case (1) is a bit of an anomaly, Guido's script would probably cover it, *if* a really acceptable solution for scripts could be found. Accepting that Unix is generally not an issue, let's look at Windows. On cmd.exe, at least on Windows 2000 and up, the following makes .py files "executable" by python: assoc .py=Python.File ftype Python.File=C:\Python23\python.exe "%1" %* set PATHEXT=%PATHEXT%;.py Can anyone test this on COMMAND.COM on Win9x? It works using the COMMAND.COM supplied with Windows 2000, whatever that proves :-) Of these, the first two are set by the standard Windows install. The final one could be. With that change, simply putting a .py script into C:\Python23\Scripts would be enough (assuming C:\Python23\Scripts was on the user's PATH, which it isn't by default, but the user can add it). Proposal: - Change the Windows Python binary to add .py (and .pyw, maybe) to PATHEXT - Add a variation of Guido's script to the standard install, as something like runmod.py - Document that the way for extensions to include scripts is as .py files in the sys.prefix+'/scripts' directory, with a #! line for Unix - For Unix users who don't like extensions, suggest (again in the docs) that a hardlink can be added without the extension. Maybe include a suitable setup.py snippet to show how. - Possibly add sys.path+'/scripts' to the PATH - I'm not sure about this, as on Windows the installer seems to try hard to *not* modify PATH, so maybe this is a policy issue. I'm punting on the issue of python.exe flags. If someone wants to run a supplied script as python -E -S -O script.py, I don't have a solution (other than doing so "longhand"). But I don't have a use case, either, so I'll leave that one for others... Paul
[Paul Moore]
PATHEXT has no special meaning on 9x; you can set PATHEXT on 9x, but it has no effect. ...
- Change the Windows Python binary to add .py (and .pyw, maybe) to PATHEXT
I don't like that. The Python Windows installer doesn't add, or modify, any environment variables now. It's anti-social to muck with them. Anyone using "a DOS box" should know how to do that themself -- if that's what they want. I'll note that I spend much of my life in a WinXP DOS box running Python programs, but I haven't set PATHEXT. Since cmd.exe's path completion fills in the trailing .py on a .py file all by itself, setting PATHEXT wouldn't save me any typing.
Right. Typing the .py is usually the right thing to do. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Tim Peters <tim.peters@gmail.com> writes:
Right. That means that this isn't a general Windows solution. On the other hand, further experimentation seems to imply that all that PATHEXT actually does is allow the ".py" to be omitted - I hadn't realised that. So as things stand, putting a ".py" script, with a #! line, into sys.prefix+'/scripts', and setting the executable bit on Unix, is already a general way of installing a command - with two provisos: 1. The user must add the directory to his PATH 2. The command must be typed with the .py extension specified So why do existing extensions not do this? Things like wxPython and Twisted seem to jump through a lot of hoops for little benefit beyond allowing users to omit the .py extension... Maybe it would be worth adding a paragraph to section 3.4 of "Distributing Python Modules" (the "Installing Scripts" section) saying something like: Scripts should be given names with a .py extension. This allows them to be executed on both Unix and Windows systems (Windows uses the extension to locate the correct interpreter, in this case Python, in much the same way that Unix uses the #! line). Of course, this will enrage Unix users who dislike file extensions :-) Maybe having a user-specifiable option, something like --strip-ext, to strip the .py extension from scripts when installing, would be useful. (I recall something like this being discussed on the distutils-sig a while back. Maybe this discussion should move there.)
OK, that's the policy I was thinking of. Paul. -- Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind. -- Dr. Seuss
On Monday 04 October 2004 04:03 pm, Paul Moore wrote:
Of course, this will enrage Unix users who dislike file extensions :-)
Yes, indeed it will, as well it should.
We discussed something very similar to this, but it wasn't so much a per-user distinction. As a package author, the name of the executables that get installed as part of my package are part of the user interface; I should be abe to control them on each platform I support. There are enough twists to this that we didn't come up with a general solution at the time, and I've not had time to revisit the topic since then, though I'd like to get something more workable over the next few months. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
On Mon, 04 Oct 2004 21:03:24 +0100, Paul Moore <p.f.moore@gmail.com> wrote:
Twisted jumps through some hoops to let developers run uninstalled versions. The lack of .py extensions is incidental and mostly unimportant. It also does things differently on UNIX than on Windows (it sets things up on Windows so uses can just type "mktap", "twistd", etc (as opposed to path\to\mktap, not as opposed to mktap.py)). Jp
Half the software on my workstation has changed environment variables, social or not. To me, it's less social to leave me to manually change PATHEXT. I'd much rather have Python add its extensions to PATHEXT -- and also have distutils add .py to the extension of Python scripts dropped into the Scripts directory. I used to be a control freak, but now I'm a control freak with kids, and the utter lack of spare time really changes one's perspective on such things. :) Regards, Garth.
[Garth T Kidd]
So contribute installer code to do so -- and don't forget to make the uninstaller smart enough to remove them again, without damaging what other installers may have added in the meantime. Ah, you don't want it enough to do anything to get it <wink>. That's OK. What good would it do you if it were there? As was made plain in the rest of this thread, PATHEXT does nothing except on cmd.exe systems. On cmd.exe systems, it only allows to leave the trailing ".py" off, and since cmd.exe's path completion fills in the ".py" for you, you then have to manually delete the ".py" in order to get a benefit from PATHEXT. Unless you add directories containing Python scripts to your PATH envar too -- in which case you can't credibly claim that adding .py to PATHEXT is beyond your abilities or time. Adding ".py" to scripts is a different issue, but that belongs on the distutils list.
Aside for Carlos: 'm' is short for 'module', and beefing up Python's option parsing to handle things like '--module' is a whole 'nother story. 'python -h' gives a decent description of all the options, though. Guido van Rossum wrote:
Only by spelling out the invocation if we want something non-standard. Using the laptop I'm typing on as an example. . . Python 2.3 is the installed version, so "#!/usr/bin/env python" in a script will invoke Py 2.3 without any extra options If I (or a Python installer) were to create "/usr/local/bin/runpy.py" with that shebang line (i.e. the same as the one you posted), we have the following for profiling a sample script in my current directory with different versions of the interpreter: Installed: runpy.py profile demo.py Prompt After: python -i /usr/local/bin/runpy.py profile demo.py Alt install: python2.4 /usr/local/bin/runpy.py profile demo.py Build dir: ./python /usr/local/bin/runpy.py profile demo.py If we wanted to use the version of the script that came with the relevant version of python, those last two would become: Alt install*: python2.4 /usr/local/lib/python2.4/runpy.py profile demo.py Build dir: ./python Tools/scripts/runpy.py profile demo.py (* This is based on what happens to pydoc under 'make altinstall'. The shebang line is version agnostic, so it tries to use the Py2.3 interpreter with the Py2.4 library modules and it all falls apart. So to run a library module of the altinstall, this is how I have to do it. And this is assuming the script would get installed at all, which it may not, as it isn't actually a library module, unlike pydoc) Using -m, those become: Installed: python -m profile demo.py Prompt After: python -i -m profile demo.py Alt install: python24 -m profile demo.py Build dir: ./python -m profile demo.py
Paul's message goes into detail on what I meant here. The script itself is highly portable, the mechanism for invoking it really isn't. The C code in the patch is platform independent - _PyImport_FindModule (a trivial wrapper around the existing find_module in import.c) and PyRun_SimpleFileExFlags do all the heavy lifting. In fact, sans error-checking, the guts of PyRun_SimpleModuleFlags looks remarkably similar to the Python code in your script. (I initially thought the Python version might handle zip imports, while the C version didn't. However, a quick experiment shows that *neither* of them can handle zip imports. And the source code confirms it - imp.find_module and imp.load_module don't allow zip imports, and PyRun_Module in the patch doesn't allow them either. Amusingly, the current limitations of the imp module make it easier to support zip imports with the *C* version. Allowing imp.find_module to return a 4th value for the module loader would require adding an optional boolean argument to avoid breaking existing code, whereas the patch's new private C API for _PyImport_FindModule already exposes the loader argument)
- more concise to invoke Depends on the length of the name of the script. :-)
See the examples above for what I meant with this one. For the vanilla case you're right, but as soon as we do anything slightly different, the story changes.
- no namespace issues with naming a script Actually, one-letter options are a much scarcer resource than script names.
Well, with '-m' in place, we'd be using 17 out of the available 62 (upper & lower alpha, plus digits). The difference is that we're only competing with ourselves and the other Python interpreter authors for characters to use, whereas we're competing with all and sundry for unique executable names. (Windows isn't immune, either, given enough applications with directories on PATH. Although retaining the '.py' helps a lot there) (I have checked that Jython at least doesn't use '-m' for anything. I don't know about other interpreters)
- C API for those embedding python And who needs that?
There's a reason this one was last on my list :)
Additional Pros for using a script: - less code to maintain
Once we factor in the additional packaging requirements to make the script as easy to use on all target platforms as -m would be, I think this one is at least arguable (script + packaging vs option-parsing and C function).
Certainly, dropping a version of this script into Tools/scripts in CVS couldn't hurt, regardless of whether or not '-m' gets adopted. The same would go for an entry in the Python cookbook.
(e.g. a common addition I expect will be to hardcode a personal sys.path addition)
Except that this feature isn't so much for your *own* scripts, as it is for installed modules that are also usable as scripts (like profile and pdb). In the former case, you *know* where those scripts are (for me ~/script_name usually does the trick on *nix). In the latter case, though, it'd be nice to be able to use these things easily 'out of the box'. For those who want to tweak the search behaviour, all the usual environment variables apply (PYTHONPATH in particular). Heck, there's nothing to stop someone from doing something like the following if they really want to: python -m my_run_module_script some_other_module The command line interface is one of the major holdouts in Python where we really need to care where the source file for a module lives. It'd be nice to change that. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
Quoting Nick Coghlan <ncoghlan@email.com>:
Using -m, those become:
Installed: python -m profile demo.py
OK, of course Python 2.3 doesn't have a '-m' switch. Consider the example to start with the first version to ship with '-m' installed. For actual Python 2.3, this would obviously have to use the absolute path, or some flavour of Guido's script (probably ~/runpy, since that is handiest). Cheers, Nick. -- Nick Coghlan Brisbane, Australia
There is one more small pro: easier to version: e.g if there are multiple installed versions of python on your machine, you could could invoke a module from a particular python version: python2.4 -m pydoc with a wrapper script you will need a wrapper per version...(Or some way to find/invoke the right version of python from the wrapper) Ilya
On Mon, Oct 04, 2004, Raymond Hettinger wrote:
-1 -- too much magic "Explicit is better than implicit" -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson
On Mon, 4 Oct 2004 10:01:10 -0400, Aahz <aahz@pythoncraft.com> wrote:
It's exactly what virtually every shell does. I sincerely don't think this would come up as surprise. On the other hand, I've often became frustrated to have to type long (I mean looooong) paths by hand, without autocompletion to help (welcome to the Windows DOS box!), just to discover that the script wasn't exactly at *that* point, but still it could be found somewhere down the path. (Just to mention, and for the sake of completeness: instead of sys.path, it could search on the shell path... but that's not a good solution either, it seems even more arbitrary) OTOH, I'm also a little bit concerned about YACLO, because there's always more cruft to add. And finally, why '-m'? Just because this letter was available ;-) ? I think that for some stuff, long names are better, even if I have to type them sometimes, and specially if I can have the default stored somewhere. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com
Carlos Ribeiro wrote:
... I've often became frustrated to have to type long (I mean looooong) paths by hand, without autocompletion to help (Windows DOS box!)...
Carlos, if you are using CMD.EXE (that is, on Win2K or XP), try running CMD /F:ON (I have a desktop link to cmd.exe that does this). This gives Ctrl-F to complete filenames and Ctrl-D for directories. You can change the keys used by tweaking the registry, but I've never felt the need. You'll get a description from "help cmd". -- -- Scott David Daniels Scott.Daniels@Acm.Org
Raymond Hettinger wrote:
Sorry for the plug, but I hope you'll forgive it if it actually turns out to be useful. You may want to have a look at ipython (http://ipython.scipy.org). IPython has a 'magic' run command, which allows you to run any python script from its command line: In [1]: cat argv.py #!/usr/bin/env python import sys print 'argv:',sys.argv In [2]: run argv argv: ['argv.py'] In its current form, it is just an execfile() on steroids. But it would be trivial to add to it a -m option so that it would search in sys.path if nothing in the current directory is found matching the given name. Ipython also gives you aliases and 'bookmarks', a persistent list of directories you can change to with a single 'cd' command: ############### 'screenshot' In [1]: pdoc alias Define an alias for a system command. '@alias alias_name cmd' defines 'alias_name' as an alias for 'cmd' Then, typing 'alias_name params' will execute the system command 'cmd params' (from your underlying operating system). [... snip. It's a long docstring] In [6]: pdoc bookmark Manage IPython's bookmark system. @bookmark <name> - set bookmark to current dir @bookmark <name> <dir> - set bookmark to <dir> @bookmark -l - list all bookmarks @bookmark -d <name> - remove bookmark @bookmark -r - remove all bookmarks You can later on access a bookmarked folder with: @cd -b <name> or simply '@cd <name>' if there is no directory called <name> AND there is such a bookmark defined. ################ /screenshot Using ipython's profile system (referred to earlier in this thread), you can use it as a quasi-system shell. The provided 'pysh' profile loads all of your $PATH (with proper extensions in Windows) as ipython aliases, and you can do regular system things, but retaining python syntax. It has also a mechanism for capturing shell output into python variables and expanding back to the shell: ############### 'screenshot' fperez@maqroll[~/test]|4> print 'Yes, this is still %s !' % 'Python' Yes, this is still Python ! fperez@maqroll[~/test]|5> $$files=ls fperez@maqroll[~/test]|6> type files ------------------------> type(files) <6> <type 'list'> fperez@maqroll[~/test]|7> len files ------------------------> len(files) <7> 31 fperez@maqroll[~/test]|8> for f in files: |.> if len(f)>10: |.> wc -l $f |.> 4 inspectbug.py 73 ramptest.py fperez@maqroll[~/test]|9> run argv.py argv: ['argv.py'] ################ /screenshot Note that the 'run' command optionally takes a -p switch, to run code under the profiler's control. Ipython can also automatically activate the pdb debugger inside any uncaught exception, thanks to the @pdb magic. I simply point these things in the hope that they are useful to you. From what I've read on this thread, ipython seems to already give most of the motivations for the -m switch, and it works today with python >= 2.2. Try it, you might like it :) (I'm not trying to say the -m switch is a bad idea, simply that ipython may provide a useful alternative today). Ok, end of blatant plug. Sorry if it was out of line. Cheers, f
[Anthony Baxter]
[Guido van Rossum]
If you think so, go ahead.
Taking this as approval, and since it's easier to ask forgiveness (and probably would've been, in the first place ;-), I've gone ahead and checked in the patches to SF bugs 1017864 & 997050, closed and marked the bug reports "accepted". -- David Goodger <http://python.net/~goodger>
Guido> This reveals IMO a big mistake in thinking about configuration Guido> files. The most important user of a config file is not the Guido> programmer who has to get data out of it; the most important user Guido> is the user who has to edit the config file. The outrageous Guido> verbosity of XML makes the above example a complete usability Guido> liability. Agreed. What about YaML? It's human readable/editable and uses indentation to denote structure instead of <tags>. I'd Google for a reference but I'm off-net at the moment. Skip
At 12:03 PM 10/2/04 -0500, Skip Montanaro wrote:
YaML isn't very friendly to non-techies, IMO. I consider it a very good human *readable* format, but not a human *writable* format. I honestly have an easier time writing XML than YaML, because there are fewer symbols to remember, the format is more regular, and I don't have to think as hard about what the processor is looking for. Of course, I *definitely* prefer .ini files to XML, if they are sufficient for the use case.
On Sun, 03 Oct 2004 22:44:31 -0400, Phillip J. Eby <pje@telecommunity.com> wrote:
Of course, I *definitely* prefer .ini files to XML, if they are sufficient for the use case.
Which means almost always, as far as text configuration files are concerned. (if the configuration is complex enough as to *require* XML, then it's probably better to provide a full fledged user interface for the customization anyway). -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com
Me too. I like the .ini file format (mail header formats will always win over XML for config files, I suspect), but had to wrap it for my latest project. Mainly for two reasons: to describe a hierarchy of config files to process when called, and to provide getint, getbool, and getfloat methods. Bill
Guido van Rossum wrote:
That's exactly what I wrote for ipython's option handler, with a fairly complex (and brittle) code to allow recursive loading of config files, since ipython has the (very useful, it turns out) concept of configuration 'profiles' which can include generic defaults and override them for specialized tasks. This means that options files can specify include statements to pull in other files, recursively. This has proven to be very useful, the rc format is a plain text key/value pair, and internally the options structure is (almost) a simple class with attributes for the options. Because I wrote this in my second week of learning python, the actual implementation is an atrocious mess, and the options class unfortunately has its own methods, which prevents having options with certain reserved names (basically all the methods of a dict). This last part was obviously a big mistake. But the overall idea, despite the shortcomings of my implementation, I think satisfies multiple important (for me) requirements: - flat text for users to edit without pulling them into the mudpit of XML. I simply refuse to ask a user to hand-edit XML, period. From an aesthetic standpoint, I think it would be borderline criminal to do so. - recursive inclusion for handling complex configuration situations. - exposes the options as a simple object with attributes accessible as rc.opt1, rc.opt2, etc. This makes the code which uses all these options (IPython has a _ton_ of cmd line flags) as readable as possible. Anyway, this is just $.02 from yet another one who reinvented this particular wheel... Best, f. ps. On the topic of @decorators and IPython, after some consultations with the ipython user community, the next release will use % for magic functions. This should be out in a few weeks, far before 2.4 is officially out.
[Raymond Hettinger]
Instead of an explicit type check, it may be better to wrap the interpolation step in a try/except.
Problem with this approach: the interpolation is done at .get() time, so any exception raised would be far from the culprit, .set().
Ideally, the docs should discourage further non-string uses
My doc patch makes clear that non-string uses don't work with interpolation & file output.
and advise that ConfigParser will be string only for Py3.0.
Are there any other Python 3.0 advisories in the docs now? Seeing as how there's no definite schedule or plan for Python 3.0, indeed it's "a hypothetical future release" (PEP 3000), that may not be a precedent we want to set. In any case, I don't know that ConfigParser *should* be declared string-only at all. Docutils uses it only to read config files; the non-string values come from interpreting those values. It's quite convenient to plug the interpreted values back into the ConfigParser dicts and use those. -- David Goodger <http://python.net/~goodger>
[David Goodger]
Instead of an explicit type check, it may be better to wrap the interpolation step in a try/except. For non-string objects, raise a TypeError with an informative error message. This would meet the OP's need for an informative error message while leaving the module backwards compatible for internal uses of ConfigParser. Ideally, the docs should discourage further non-string uses and advise that ConfigParser will be string only for Py3.0. Raymond
On Friday 01 October 2004 04:07 pm, Raymond Hettinger wrote:
Ideally, the docs should discourage further non-string uses and advise that ConfigParser will be string only for Py3.0.
Yikes! Can't we just toss it for Py3K? This module just hasn't held up, and exposes a really poor model even for .ini-style configuration files. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
Fred L. Drake, Jr. wrote:
Ditto from me. Personally I would want something that used XML (like property lists on OS X), but I realize people still want a config file style that is easy to modify in a text editor, so I don't see the .ini style going away. But we could stand to come up with a uniform interface that both an XML and .ini config file parsers could use for consistency and thus support both styles. Wasn't there talk for a while of doing an shootout of config file packages like we did for optparse? -Brett
Hear hear.
Well, for me personally, .ini style config files still win over XML every day. And I now have significant experience with both here at ESI. What sucks (relatively) is the specific way that ConfigParser provides access to .ini files; I always end up writing a wrapping layer around it.
Wasn't there talk for a while of doing an shootout of config file packages like we did for optparse?
There are two different axes here: the config file format and the API for accessing it. I'm not sure that attempting to provide a single API for both XML and .ini files will work; the API should reflect the structure of the file (to some extent) and that's just too different. But I may be wrong; I do see some similarities: What I have come to like, both for dealing with XML and with .ini files, is something that lets you map the configuration values (or whatever it is that you're parsing, really) to Python attributes. This is really some kind of simplified DTD support, and there are different ways to go about it; but the net result is that you end up writing e.g. options.client.max_retries (which is an int with a default value) rather than options.getint("client", "max-retries"). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido van Rossum]
Unfortunately ConfigParser discourages such wrappers by hiding too much of its implementation as _private attributes (not __class_privates, so they *are* accessible, but even single-underscore _privacy screams "don't touch this -- subject to change without notice"). Back on topic though, I'd like to see +/-'s on the two patches. The first one (http://www.python.org/sf/1017864) is IMO a no-brainer. The second (http://www.python.org/sf/997050) is what I'm really interested in: do we maintain the existing API and functionality, or is it OK to break existing code by *removing* functionality? -- David Goodger <http://python.net/~goodger>
[Guido] ...
LOL. I wrote a Config wrapper for SpamBayes like that. Then I stopped paying attention, and next thing I knew all the option-accessing code had been changed to look like SB's current prob = options["Classifier", "unknown_word_prob"] instead. But even that's better than options.getint("client", "max-retries")! Encoding the return type at every access point is Plain Wrong.
Guido van Rossum wrote: [SNIP]
OK. Do realize that plists are basically .ini style just expressed in XML:: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Section</key> <dict> <key>key</key> <string>value</string> <key>key2</key> <string>value2</string> </dict> </dict> </plist> I am not thinking of anything fancy or beyond something like this; .ini files expressed in XML. Just thinking that XML might be nice since all of those poor souls who don't use Python have easy access to an XML parser but not necessarily a .ini file parser.
For free-wheeling XML, yes, I would agree, but if we restrict the DTD/schema/spec to be fairly similar to .ini files then I don't think the structure of the files will be that drastic. And as long as people use the API to create/edit/save the files then we can enforce universal rules that makes sure that the difference is really just in the structure of the output.
That is what I am thinking. You have sections and then key/value pairs within the sections in both the XML and the .ini format. We then just agree on how case-sensitivity, naming, etc. works within the file formats and then provide a common-place interface much like the one you lay out above. Is this worth working on now or wait until Py3k? It probably seems like I am playing the PEP drum (not a fan of the harp =) a lot lately in regards to stuff that is going to not be an issue until Py3k, but the longer we have to work on something the better chance we have of having something that everyone likes. Plus for things like modules we can provide it for people now as an external module for experimentation and thus be a little willy-nilly with the API initially until we have it nailed and either integrate into the stdlib beofre Py3k or just wait until then. -Brett
This reveals IMO a big mistake in thinking about configuration files. The most important user of a config file is not the programmer who has to get data out of it; the most important user is the user who has to edit the config file. The outrageous verbosity of XML makes the above example a complete usability liability. Now, if you're talking about config files that represent options that the user edits in a convenient application-specific options dialog, that's a different story; I think XML is well-suited for that; but I'm talking about the classic configuration file pattern where you use your favorite flat-file text editor to edit the options file. In that situation, using XML is insane.
Is this worth working on now or wait until Py3k?
I see no advantage in waiting until Py3K; this is not a language issue and there is no problem with having several library modules (as long as it's clear which one is deprecated). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote: [SNIP]
I am thinking of the app-specific options dialog situation for XML and I do think that using XML for the classic style of editing the flat file by hand is insane. My thinking on the XML idea is that people do write apps where the whole thing is behind a GUI and thus XML makes good sense. So why not make there lives a little easier by giving them a basic API? But it is not a make-or-break thing for me so I am willing to let it go if it is deemed not worth the effort.
OK, so how do people want to proceeed with this? PEP? Shootout? -Brett
People who want to use XML already have all the rope they need. I would be in for adding something like my simplified XML handling class to the library, but I first have to lobby my employer to open-source it (not a problem, I just have to time the request right so it won't be in 2.4). Anyway, this needn't and shouldn't be specific to the problem of storing option values.
OK, so how do people want to proceeed with this? PEP? Shootout?
I'm all for working code in this case, so let's do a shootout. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
OK.
OK, so how do people want to proceeed with this? PEP? Shootout?
I'm all for working code in this case, so let's do a shootout.
OK. Should we do an announcement to python-announce and c.l.py saying we are looking for a new API for .ini file editing/accessing? -Brett
OK. Should we do an announcement to python-announce and c.l.py saying we are looking for a new API for .ini file editing/accessing?
I don't see the point -- it's not like this is the most important issue of the year. I would rather just see if any of the readers here happen to have code lying around that they like. It sounds like maybe the IPython stuff could be used after a thorough rewrite. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido van Rossum]
XML may sometimes be useful _when_ a configuration _has_ to be a tree of thick (contents heavy) nodes. For simpler configurations files, and this covers a flurry of cases, Guido above expresses my feelings exactly. There are XML-fanatics -- the same as there are Unicode-fanatics :-). Not so long ago, I've seen a pushy trend by which `/etc/passwd' and a lot of other simple system files would be reformulated as XML, to be XML all over. (Maybe after an announcement of Microsoft -- real or made-up, I do not know -- that they aim such a thing for the next "Windows".) We should not go overboard with XML. Configuration files with lines and words for structuring of a two-level hierarchy have long proven their value. ConfigParser `.ini' files add a third level to these. and with good sense, we can still go another long way without resorting to XML. Besides, when configuration files are going to have some more complex structure, but explicitly made to drive big applications written in Python -- a common case for most of us presumably -- I found out that Python itself is a very convenient and powerful tool for expressing such parameterisation. Users can easily modify a carefully designed Python-written configuration template without knowing a bit about Python itself. If they know the concepts of the applications they are configuring, in my experience at least, they quickly get what needs to be done, and even grasp by mere mimetism, without much explanations, the bits of syntax to respect and follow. With proper care, this can be made incredibly powerful, while staying _way_ more legible than XML. Also compare speed and simplicity, for the application programmer, for bringing such configuration in memory, type concerns already addressed. Of course, XML better inter-operates with foreign programming languages and systems, and would undoubtedly allow easier communication with alien races from neighbouring galaxies :-). But deep down, I do not think that this need in configuration neutrality is common enough to warrant the added complexity, and loss of legibility, for most practical cases. -- François Pinard http://pinard.progiciels-bpi.ca
Guido van Rossum wrote:
Plist is handy when: - you need a simple storage format for simple data structures - you want to be able to edit a file manually in exceptional situations If you need to *routinely* edit these files by hand than plist is indeed not the right choice. It's not a pretty format, but it's well defined. I find it extremely handy and use it all over the place. I'd be happy to contribute plistlib.py to the std library (ie. move it one level up from Lib/plat-mac/ ;-), but I doubt there's enough interest. Just
All this talk of a replacement for ConfigParser is all well and good (and I agree that a more pythonic interface would be nice)... but. We're under 2 weeks from beta1, and I can't see this new module being designed, implemented, and committed before then. Remember, once b1 is out, we're stuck with the API that we have come up with. I'd much rather see one or more packages developed alongside Python - we can then take the best of them (or maybe a merger of the best ideas for them) for 2.5. Which leaves David's original question about the two patches. While CP isn't perfect, and it would be nice to replace it, for 2.4, I don't think these patches make things any worse, and they add useful functionality (back, in one case) so I'm inclined towards accepting them. Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.
Of course. Other recent proposals for brand new stuff also seem poorly timed.
Right. So it went for optparse and logging.
If you think so, go ahead. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Quoting Guido van Rossum <gvanrossum@gmail.com>:
Of course. Other recent proposals for brand new stuff also seem poorly timed.
I may be guilty of that. Would it be worth adding a Python 2.5 group to the SF Patch tracker for things which shouldn't be considered seriously until after 2.4 final is out the door? Cheers, Nick. -- Nick Coghlan Brisbane, Australia
Of course. Other recent proposals for brand new stuff also seem
[Guido] poorly
timed.
[Nick]
Though ill timed, the -m idea is honking good. If it can be reliably worked out in its simplest form (not trying to be all things to all packages), it's worth going ahead for Py2.4. Command line guys like me need this every day. His patch will address a continual source of irritation and make the tool more pleasant to use. Raymond
I was in fact thinking of the -m proposal when I wrote that... It's so easy to define an alias or use a one-line shell script for invoking Python with a full pathname that I'm really not sure I like the -m idea, and I worry that it would be done the wrong way because it's so close to beta. Half the time the problem is getting the path set in the first place, and there -m doesn't help you. Supporting it with packages seems insanity. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
I was in fact thinking of the -m proposal when I wrote that...
The timing does suck.
My understanding was that it wasn't about a full pathname to python, it was about searching sys.path for the darned script so us poor Windows guys don't have to change directories a million times a day (no aliases or shebangs for us either). I have a big pile of batch files just to invoke timeit, texchecker, profile, etc. It bites the big one. If the patch doesn't get worked out for Py2.4, I hope it doesn't get abandoned.
Supporting it with packages seems insanity.
No doubt. Raymond
Quoting Raymond Hettinger <python@rcn.com>:
Correct. Ilya's original example was wanting to invoke pdb's script behaviour in order to debug another script. At the moment, you have to run: python -c "from inspect import getsourcefile; import pdb; print getsourcefile(pdb)" or python -c "from imp import find_module; print find_module("pdb")[1]" to find out where the file is, and then run python normally with the filename that gets printed by one of the above scripts. (With a Unix-style shell, backticks make that easier. Doesn't help on Windows though). The "-m" option aims to automate this process (using the latter 'find_module' approach in C code)
Supporting it with packages seems insanity.
No doubt.
I'm starting to lean that way myself. The semantics for a straight module are easy, but the package version is horrible (Either we have a package getting imported before "__main__" starts running, or else we don't properly support package __path__ variables. I don't find either option appealing) I think I'll simply drop all package support from the patch and allow modules only. That will still cover the initial use cases (pdb, profile and the like) Adding support for packages later (in a different patch) would still be possible if there was interest. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
On Sat, 2004-10-02 at 23:27, Nick Coghlan wrote:
+1 and I'd still love to see this in Python 2.4. Could you please assign the patch to someone other than me though? I don't honestly think I'll have time to review it before 2.4b1. -Barry
I'm not going to stop you from adding this, but I do think you're overlooking the fact that if you need this a lot, it's trivial to write a tiny Python script that takes the place of "python -m", and putting that python script in your PATH. Something like: #!/usr/bin/env python import sys, imp fn = imp.find_module(sys.argv[1])[1] del sys.argv[0] sys.argv[0] = fn execfile(fn) seems do the trick (though it could use nicer error handling). Wouldn't it be easier to add *this* to the Tools/scripts directory (and the list of scripts that are installed by "make install" or the Windows equivalent) than to burden the Python interpreter with C code to do the same? Why does every handy thing that anyone ever thought of have to be a Python command line option? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Quoting Guido van Rossum <gvanrossum@gmail.com>:
And is basically identical to the C version of this code (which uses _PyImp_FindModule and PyRun_SimpleFileExFlags). I like this idea, as it should automatically work with other versions of the interpreter like Jython and IronPython, whereas the command line option would required reimplementation in each interpreter. The hardest part would be to come up with a name for the script (pymod?, pymodule?, pyrun?, runpy?). It will probably be slightly more verbose to invoke than '-m' would be (particularly on an uninstalled development build), but also easier to enhance (e.g supporting modules inside packages). Since I'm on Linux at the moment, and can't recall how the Windows installer handles things like pydoc, I'll wait and see what Raymond and others think about how well this would work on Windows. Unless they think it won't work for other platforms, I'm inclined to reject my own patch and look at a script-based approach. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
[Guido]
Though not as clean as -m, this script helps. For my own insufficiently advanced windows box, I added a launcher batch file, pythonm.bat: @python \py24\tools\scripts\runpy.py %1 %2 %3 %4 %5 %6 %7 %8 %9 It runs pretty well: C:\tmp> pythonm timeit -s "a=range(20)" "a.append(1)" "a.pop()" 100000 loops, best of 3: 1.75 usec per loop C:\tmp> pythonm regrtest test_string test_string 1 test OK. The combination of batch file and run.py isn't as good as the -m option, but it works and mostly meets my needs. The batch file is rather dumb, doesn't work with other python command line options, and won't work pipes.
Why does every handy thing that anyone ever thought of have to be a Python command line option?
Could Nick's idea be done without an -m option? If a user types, "python abc.py" and "abc.py" is not found, before aborting, try looking for it on sys.path. Raymond
Quoting Raymond Hettinger <python@rcn.com>:
Hmm, that point about command line options is a good one. I know I'd find it handy to be able to throw in a '-i' when I wanted one. Then again for commands with minor variations, I tend to rely on command line history rather than using a batch file (that is, I'll type out a long invocation sequence once a session, then edit it as needed)
Could Nick's idea be done without an -m option?
To be fair to Ilya - his idea, my implementation :)
That's do-able, but the command line option seems cleaner if the interpreter is going to provide C-level support for this. I interpreted Guido's last message as saying he's prepared to tolerate the command line option if there's sufficient support for it. To summarise the differences between the two approaches: Pros (-m over runpy.py, in order of significance as I see it): - easy to combine with other Python command line options - OS & environment independent - easier to use with a non-installed interpreter - no work for Python packaging folks - more concise to invoke - no namespace issues with naming a script - C API for those embedding python Cons: - YACLO (Yet Another Command Line Option) - CPython specific (other interpreters could choose to reimplement) With YACLO seeming to be Guido's main reason for being -0 on the idea. It looks like the script option would require input from those with better knowledge than me of the packaging set up for Windows, *nix and Mac as to how it could be made to work 'nicely' in each environment. And the issue of combining it with other command line options would remain. It's all very well to have "runpy.py" as an executable script, but providing extra options to the interpreter would require: 1. Locate runpy.py (e.g. /usr/bin/local/runpy.py, /usr/bin/runpy.py) 2. Invoke python <options> <dir>/runpy.py <module> Which means we're back to having to find out where at least part of the Python installation lives on each machine (only once per machine, which is an improvement over the status quo, but still not as tidy as a single command line option). I find that persuasive enough that I think it's worthwhile to finish the '-m' patch, and add a documentation patch. Regards, Nick. -- Nick Coghlan Brisbane, Australia
Pros (-m over runpy.py, in order of significance as I see it): - easy to combine with other Python command line options
The script approach can do this too
- OS & environment independent
So is the script -- probably more so, since the script can use Python's OS independence layer.
- easier to use with a non-installed interpreter - no work for Python packaging folks
No contest.
- more concise to invoke
Depends on the length of the name of the script. :-)
- no namespace issues with naming a script
Actually, one-letter options are a much scarcer resource than script names.
- C API for those embedding python
And who needs that?
Additional Pros for using a script: - less code to maintain - can work with older Python versions - shows users how to do a similar thing themselves with additional features (e.g. a common addition I expect will be to hardcode a personal sys.path addition) PS a big honking -1000 on Carlos Ribeiro's suggestion of doing this automatically if the filename path isn't found. Too many chances for accidentally invoking the wrong thing -- including mysyerious silences when the named module happens to exist but doesn't do anything when run as a script (i.e. the majority of modules). (As an aside, I wonder why every single of Carlos' interactions starts of with an idea that is completely misguided and then ends with him vehemently defending it against all opposition. Sounds like a recipe for flame wars to me.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 4 Oct 2004 08:05:01 -0700, Guido van Rossum <gvanrossum@gmail.com> wrote:
Sorry. Really. I may have expressed myself VERY badly on this topic. Please, let me defend myself as I don't like to leave this impression as I'm just starting here: -- The original idea was not mine -- I was just commenting because it seemed good to me, as I'm limited to a DOS box most of the time, and searching modules by hand tend to become an issue in this enviroment; -- I really meant to say this (regarding YACLO): I *don't* think that adding command line options for everything is a good idea for any software. Also, I think that single line options are confusing, and longer option names are better/more readable. It seems that the message was read with the opposite meaning. -- As far as the rest of the comments are concerned, I apologize if I tend to sound so 'flamish'. Actually, I have a tendency to think aloud sometimes, and to explore different ideas. Sometimes it makes for amusing comments, sometimes it's weird, and sometimes it just sounds dumb. Sometimes I get it right. But overall it's not the right thing to do, specially as I'm yet to make some useful contribution. In the end, I think I should learn to keep my mouth/fingers shut. Lesson taken, and really, really, sorry. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com
The problem here is really one of packaging. How should utility scripts be distributed, either with Python, or with extensions? There are many different approaches: 1. a library module with useful behaviour in the __main__ block. 2. a script in sys.prefix + '/scripts' with no extension and a #! line 3. a script as above, but with a .py extension 4. a script as (2) plus a .bat wrapper for Windows 5. a script as (3) plus a .bat wrapper for Windows However, *none* of these give the best behaviour, uniformly across all Windows platforms (the Unix story is far better, as there is a consistent and useful behaviour across all shells). While case (1) is a bit of an anomaly, Guido's script would probably cover it, *if* a really acceptable solution for scripts could be found. Accepting that Unix is generally not an issue, let's look at Windows. On cmd.exe, at least on Windows 2000 and up, the following makes .py files "executable" by python: assoc .py=Python.File ftype Python.File=C:\Python23\python.exe "%1" %* set PATHEXT=%PATHEXT%;.py Can anyone test this on COMMAND.COM on Win9x? It works using the COMMAND.COM supplied with Windows 2000, whatever that proves :-) Of these, the first two are set by the standard Windows install. The final one could be. With that change, simply putting a .py script into C:\Python23\Scripts would be enough (assuming C:\Python23\Scripts was on the user's PATH, which it isn't by default, but the user can add it). Proposal: - Change the Windows Python binary to add .py (and .pyw, maybe) to PATHEXT - Add a variation of Guido's script to the standard install, as something like runmod.py - Document that the way for extensions to include scripts is as .py files in the sys.prefix+'/scripts' directory, with a #! line for Unix - For Unix users who don't like extensions, suggest (again in the docs) that a hardlink can be added without the extension. Maybe include a suitable setup.py snippet to show how. - Possibly add sys.path+'/scripts' to the PATH - I'm not sure about this, as on Windows the installer seems to try hard to *not* modify PATH, so maybe this is a policy issue. I'm punting on the issue of python.exe flags. If someone wants to run a supplied script as python -E -S -O script.py, I don't have a solution (other than doing so "longhand"). But I don't have a use case, either, so I'll leave that one for others... Paul
[Paul Moore]
PATHEXT has no special meaning on 9x; you can set PATHEXT on 9x, but it has no effect. ...
- Change the Windows Python binary to add .py (and .pyw, maybe) to PATHEXT
I don't like that. The Python Windows installer doesn't add, or modify, any environment variables now. It's anti-social to muck with them. Anyone using "a DOS box" should know how to do that themself -- if that's what they want. I'll note that I spend much of my life in a WinXP DOS box running Python programs, but I haven't set PATHEXT. Since cmd.exe's path completion fills in the trailing .py on a .py file all by itself, setting PATHEXT wouldn't save me any typing.
Right. Typing the .py is usually the right thing to do. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Tim Peters <tim.peters@gmail.com> writes:
Right. That means that this isn't a general Windows solution. On the other hand, further experimentation seems to imply that all that PATHEXT actually does is allow the ".py" to be omitted - I hadn't realised that. So as things stand, putting a ".py" script, with a #! line, into sys.prefix+'/scripts', and setting the executable bit on Unix, is already a general way of installing a command - with two provisos: 1. The user must add the directory to his PATH 2. The command must be typed with the .py extension specified So why do existing extensions not do this? Things like wxPython and Twisted seem to jump through a lot of hoops for little benefit beyond allowing users to omit the .py extension... Maybe it would be worth adding a paragraph to section 3.4 of "Distributing Python Modules" (the "Installing Scripts" section) saying something like: Scripts should be given names with a .py extension. This allows them to be executed on both Unix and Windows systems (Windows uses the extension to locate the correct interpreter, in this case Python, in much the same way that Unix uses the #! line). Of course, this will enrage Unix users who dislike file extensions :-) Maybe having a user-specifiable option, something like --strip-ext, to strip the .py extension from scripts when installing, would be useful. (I recall something like this being discussed on the distutils-sig a while back. Maybe this discussion should move there.)
OK, that's the policy I was thinking of. Paul. -- Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind. -- Dr. Seuss
On Monday 04 October 2004 04:03 pm, Paul Moore wrote:
Of course, this will enrage Unix users who dislike file extensions :-)
Yes, indeed it will, as well it should.
We discussed something very similar to this, but it wasn't so much a per-user distinction. As a package author, the name of the executables that get installed as part of my package are part of the user interface; I should be abe to control them on each platform I support. There are enough twists to this that we didn't come up with a general solution at the time, and I've not had time to revisit the topic since then, though I'd like to get something more workable over the next few months. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
On Mon, 04 Oct 2004 21:03:24 +0100, Paul Moore <p.f.moore@gmail.com> wrote:
Twisted jumps through some hoops to let developers run uninstalled versions. The lack of .py extensions is incidental and mostly unimportant. It also does things differently on UNIX than on Windows (it sets things up on Windows so uses can just type "mktap", "twistd", etc (as opposed to path\to\mktap, not as opposed to mktap.py)). Jp
Half the software on my workstation has changed environment variables, social or not. To me, it's less social to leave me to manually change PATHEXT. I'd much rather have Python add its extensions to PATHEXT -- and also have distutils add .py to the extension of Python scripts dropped into the Scripts directory. I used to be a control freak, but now I'm a control freak with kids, and the utter lack of spare time really changes one's perspective on such things. :) Regards, Garth.
[Garth T Kidd]
So contribute installer code to do so -- and don't forget to make the uninstaller smart enough to remove them again, without damaging what other installers may have added in the meantime. Ah, you don't want it enough to do anything to get it <wink>. That's OK. What good would it do you if it were there? As was made plain in the rest of this thread, PATHEXT does nothing except on cmd.exe systems. On cmd.exe systems, it only allows to leave the trailing ".py" off, and since cmd.exe's path completion fills in the ".py" for you, you then have to manually delete the ".py" in order to get a benefit from PATHEXT. Unless you add directories containing Python scripts to your PATH envar too -- in which case you can't credibly claim that adding .py to PATHEXT is beyond your abilities or time. Adding ".py" to scripts is a different issue, but that belongs on the distutils list.
Aside for Carlos: 'm' is short for 'module', and beefing up Python's option parsing to handle things like '--module' is a whole 'nother story. 'python -h' gives a decent description of all the options, though. Guido van Rossum wrote:
Only by spelling out the invocation if we want something non-standard. Using the laptop I'm typing on as an example. . . Python 2.3 is the installed version, so "#!/usr/bin/env python" in a script will invoke Py 2.3 without any extra options If I (or a Python installer) were to create "/usr/local/bin/runpy.py" with that shebang line (i.e. the same as the one you posted), we have the following for profiling a sample script in my current directory with different versions of the interpreter: Installed: runpy.py profile demo.py Prompt After: python -i /usr/local/bin/runpy.py profile demo.py Alt install: python2.4 /usr/local/bin/runpy.py profile demo.py Build dir: ./python /usr/local/bin/runpy.py profile demo.py If we wanted to use the version of the script that came with the relevant version of python, those last two would become: Alt install*: python2.4 /usr/local/lib/python2.4/runpy.py profile demo.py Build dir: ./python Tools/scripts/runpy.py profile demo.py (* This is based on what happens to pydoc under 'make altinstall'. The shebang line is version agnostic, so it tries to use the Py2.3 interpreter with the Py2.4 library modules and it all falls apart. So to run a library module of the altinstall, this is how I have to do it. And this is assuming the script would get installed at all, which it may not, as it isn't actually a library module, unlike pydoc) Using -m, those become: Installed: python -m profile demo.py Prompt After: python -i -m profile demo.py Alt install: python24 -m profile demo.py Build dir: ./python -m profile demo.py
Paul's message goes into detail on what I meant here. The script itself is highly portable, the mechanism for invoking it really isn't. The C code in the patch is platform independent - _PyImport_FindModule (a trivial wrapper around the existing find_module in import.c) and PyRun_SimpleFileExFlags do all the heavy lifting. In fact, sans error-checking, the guts of PyRun_SimpleModuleFlags looks remarkably similar to the Python code in your script. (I initially thought the Python version might handle zip imports, while the C version didn't. However, a quick experiment shows that *neither* of them can handle zip imports. And the source code confirms it - imp.find_module and imp.load_module don't allow zip imports, and PyRun_Module in the patch doesn't allow them either. Amusingly, the current limitations of the imp module make it easier to support zip imports with the *C* version. Allowing imp.find_module to return a 4th value for the module loader would require adding an optional boolean argument to avoid breaking existing code, whereas the patch's new private C API for _PyImport_FindModule already exposes the loader argument)
- more concise to invoke Depends on the length of the name of the script. :-)
See the examples above for what I meant with this one. For the vanilla case you're right, but as soon as we do anything slightly different, the story changes.
- no namespace issues with naming a script Actually, one-letter options are a much scarcer resource than script names.
Well, with '-m' in place, we'd be using 17 out of the available 62 (upper & lower alpha, plus digits). The difference is that we're only competing with ourselves and the other Python interpreter authors for characters to use, whereas we're competing with all and sundry for unique executable names. (Windows isn't immune, either, given enough applications with directories on PATH. Although retaining the '.py' helps a lot there) (I have checked that Jython at least doesn't use '-m' for anything. I don't know about other interpreters)
- C API for those embedding python And who needs that?
There's a reason this one was last on my list :)
Additional Pros for using a script: - less code to maintain
Once we factor in the additional packaging requirements to make the script as easy to use on all target platforms as -m would be, I think this one is at least arguable (script + packaging vs option-parsing and C function).
Certainly, dropping a version of this script into Tools/scripts in CVS couldn't hurt, regardless of whether or not '-m' gets adopted. The same would go for an entry in the Python cookbook.
(e.g. a common addition I expect will be to hardcode a personal sys.path addition)
Except that this feature isn't so much for your *own* scripts, as it is for installed modules that are also usable as scripts (like profile and pdb). In the former case, you *know* where those scripts are (for me ~/script_name usually does the trick on *nix). In the latter case, though, it'd be nice to be able to use these things easily 'out of the box'. For those who want to tweak the search behaviour, all the usual environment variables apply (PYTHONPATH in particular). Heck, there's nothing to stop someone from doing something like the following if they really want to: python -m my_run_module_script some_other_module The command line interface is one of the major holdouts in Python where we really need to care where the source file for a module lives. It'd be nice to change that. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
Quoting Nick Coghlan <ncoghlan@email.com>:
Using -m, those become:
Installed: python -m profile demo.py
OK, of course Python 2.3 doesn't have a '-m' switch. Consider the example to start with the first version to ship with '-m' installed. For actual Python 2.3, this would obviously have to use the absolute path, or some flavour of Guido's script (probably ~/runpy, since that is handiest). Cheers, Nick. -- Nick Coghlan Brisbane, Australia
There is one more small pro: easier to version: e.g if there are multiple installed versions of python on your machine, you could could invoke a module from a particular python version: python2.4 -m pydoc with a wrapper script you will need a wrapper per version...(Or some way to find/invoke the right version of python from the wrapper) Ilya
On Mon, Oct 04, 2004, Raymond Hettinger wrote:
-1 -- too much magic "Explicit is better than implicit" -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson
On Mon, 4 Oct 2004 10:01:10 -0400, Aahz <aahz@pythoncraft.com> wrote:
It's exactly what virtually every shell does. I sincerely don't think this would come up as surprise. On the other hand, I've often became frustrated to have to type long (I mean looooong) paths by hand, without autocompletion to help (welcome to the Windows DOS box!), just to discover that the script wasn't exactly at *that* point, but still it could be found somewhere down the path. (Just to mention, and for the sake of completeness: instead of sys.path, it could search on the shell path... but that's not a good solution either, it seems even more arbitrary) OTOH, I'm also a little bit concerned about YACLO, because there's always more cruft to add. And finally, why '-m'? Just because this letter was available ;-) ? I think that for some stuff, long names are better, even if I have to type them sometimes, and specially if I can have the default stored somewhere. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com
Carlos Ribeiro wrote:
... I've often became frustrated to have to type long (I mean looooong) paths by hand, without autocompletion to help (Windows DOS box!)...
Carlos, if you are using CMD.EXE (that is, on Win2K or XP), try running CMD /F:ON (I have a desktop link to cmd.exe that does this). This gives Ctrl-F to complete filenames and Ctrl-D for directories. You can change the keys used by tweaking the registry, but I've never felt the need. You'll get a description from "help cmd". -- -- Scott David Daniels Scott.Daniels@Acm.Org
Raymond Hettinger wrote:
Sorry for the plug, but I hope you'll forgive it if it actually turns out to be useful. You may want to have a look at ipython (http://ipython.scipy.org). IPython has a 'magic' run command, which allows you to run any python script from its command line: In [1]: cat argv.py #!/usr/bin/env python import sys print 'argv:',sys.argv In [2]: run argv argv: ['argv.py'] In its current form, it is just an execfile() on steroids. But it would be trivial to add to it a -m option so that it would search in sys.path if nothing in the current directory is found matching the given name. Ipython also gives you aliases and 'bookmarks', a persistent list of directories you can change to with a single 'cd' command: ############### 'screenshot' In [1]: pdoc alias Define an alias for a system command. '@alias alias_name cmd' defines 'alias_name' as an alias for 'cmd' Then, typing 'alias_name params' will execute the system command 'cmd params' (from your underlying operating system). [... snip. It's a long docstring] In [6]: pdoc bookmark Manage IPython's bookmark system. @bookmark <name> - set bookmark to current dir @bookmark <name> <dir> - set bookmark to <dir> @bookmark -l - list all bookmarks @bookmark -d <name> - remove bookmark @bookmark -r - remove all bookmarks You can later on access a bookmarked folder with: @cd -b <name> or simply '@cd <name>' if there is no directory called <name> AND there is such a bookmark defined. ################ /screenshot Using ipython's profile system (referred to earlier in this thread), you can use it as a quasi-system shell. The provided 'pysh' profile loads all of your $PATH (with proper extensions in Windows) as ipython aliases, and you can do regular system things, but retaining python syntax. It has also a mechanism for capturing shell output into python variables and expanding back to the shell: ############### 'screenshot' fperez@maqroll[~/test]|4> print 'Yes, this is still %s !' % 'Python' Yes, this is still Python ! fperez@maqroll[~/test]|5> $$files=ls fperez@maqroll[~/test]|6> type files ------------------------> type(files) <6> <type 'list'> fperez@maqroll[~/test]|7> len files ------------------------> len(files) <7> 31 fperez@maqroll[~/test]|8> for f in files: |.> if len(f)>10: |.> wc -l $f |.> 4 inspectbug.py 73 ramptest.py fperez@maqroll[~/test]|9> run argv.py argv: ['argv.py'] ################ /screenshot Note that the 'run' command optionally takes a -p switch, to run code under the profiler's control. Ipython can also automatically activate the pdb debugger inside any uncaught exception, thanks to the @pdb magic. I simply point these things in the hope that they are useful to you. From what I've read on this thread, ipython seems to already give most of the motivations for the -m switch, and it works today with python >= 2.2. Try it, you might like it :) (I'm not trying to say the -m switch is a bad idea, simply that ipython may provide a useful alternative today). Ok, end of blatant plug. Sorry if it was out of line. Cheers, f
[Anthony Baxter]
[Guido van Rossum]
If you think so, go ahead.
Taking this as approval, and since it's easier to ask forgiveness (and probably would've been, in the first place ;-), I've gone ahead and checked in the patches to SF bugs 1017864 & 997050, closed and marked the bug reports "accepted". -- David Goodger <http://python.net/~goodger>
Guido> This reveals IMO a big mistake in thinking about configuration Guido> files. The most important user of a config file is not the Guido> programmer who has to get data out of it; the most important user Guido> is the user who has to edit the config file. The outrageous Guido> verbosity of XML makes the above example a complete usability Guido> liability. Agreed. What about YaML? It's human readable/editable and uses indentation to denote structure instead of <tags>. I'd Google for a reference but I'm off-net at the moment. Skip
At 12:03 PM 10/2/04 -0500, Skip Montanaro wrote:
YaML isn't very friendly to non-techies, IMO. I consider it a very good human *readable* format, but not a human *writable* format. I honestly have an easier time writing XML than YaML, because there are fewer symbols to remember, the format is more regular, and I don't have to think as hard about what the processor is looking for. Of course, I *definitely* prefer .ini files to XML, if they are sufficient for the use case.
On Sun, 03 Oct 2004 22:44:31 -0400, Phillip J. Eby <pje@telecommunity.com> wrote:
Of course, I *definitely* prefer .ini files to XML, if they are sufficient for the use case.
Which means almost always, as far as text configuration files are concerned. (if the configuration is complex enough as to *require* XML, then it's probably better to provide a full fledged user interface for the customization anyway). -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com
Me too. I like the .ini file format (mail header formats will always win over XML for config files, I suspect), but had to wrap it for my latest project. Mainly for two reasons: to describe a hierarchy of config files to process when called, and to provide getint, getbool, and getfloat methods. Bill
Guido van Rossum wrote:
That's exactly what I wrote for ipython's option handler, with a fairly complex (and brittle) code to allow recursive loading of config files, since ipython has the (very useful, it turns out) concept of configuration 'profiles' which can include generic defaults and override them for specialized tasks. This means that options files can specify include statements to pull in other files, recursively. This has proven to be very useful, the rc format is a plain text key/value pair, and internally the options structure is (almost) a simple class with attributes for the options. Because I wrote this in my second week of learning python, the actual implementation is an atrocious mess, and the options class unfortunately has its own methods, which prevents having options with certain reserved names (basically all the methods of a dict). This last part was obviously a big mistake. But the overall idea, despite the shortcomings of my implementation, I think satisfies multiple important (for me) requirements: - flat text for users to edit without pulling them into the mudpit of XML. I simply refuse to ask a user to hand-edit XML, period. From an aesthetic standpoint, I think it would be borderline criminal to do so. - recursive inclusion for handling complex configuration situations. - exposes the options as a simple object with attributes accessible as rc.opt1, rc.opt2, etc. This makes the code which uses all these options (IPython has a _ton_ of cmd line flags) as readable as possible. Anyway, this is just $.02 from yet another one who reinvented this particular wheel... Best, f. ps. On the topic of @decorators and IPython, after some consultations with the ipython user community, the next release will use % for magic functions. This should be out in a few weeks, far before 2.4 is officially out.
[Raymond Hettinger]
Instead of an explicit type check, it may be better to wrap the interpolation step in a try/except.
Problem with this approach: the interpolation is done at .get() time, so any exception raised would be far from the culprit, .set().
Ideally, the docs should discourage further non-string uses
My doc patch makes clear that non-string uses don't work with interpolation & file output.
and advise that ConfigParser will be string only for Py3.0.
Are there any other Python 3.0 advisories in the docs now? Seeing as how there's no definite schedule or plan for Python 3.0, indeed it's "a hypothetical future release" (PEP 3000), that may not be a precedent we want to set. In any case, I don't know that ConfigParser *should* be declared string-only at all. Docutils uses it only to read config files; the non-string values come from interpreting those values. It's quite convenient to plug the interpreted values back into the ConfigParser dicts and use those. -- David Goodger <http://python.net/~goodger>
participants (22)
-
Aahz -
Anthony Baxter -
Barry Warsaw -
Bill Janssen -
Brett C. -
Carlos Ribeiro -
David Goodger -
exarkun＠divmod.com -
Fernando Perez -
François Pinard -
Fred L. Drake, Jr. -
Garth T Kidd -
Guido van Rossum -
Ilya Sandler -
Just van Rossum -
Nick Coghlan -
Paul Moore -
Phillip J. Eby -
Raymond Hettinger -
Scott David Daniels -
Skip Montanaro -
Tim Peters