Re: [Distutils] Proposal: C/C++ compiler identification
At 10:36 15/12/98 -0500, Greg Ward wrote:
*This is not a flame* Please do not interpret it as such!
Quoth John Skaller, on 13 December 1998:
This proposal is intended as a detail of the general requrements document for distutils.
So, first question: are you proposing to throw out my requirements document and replace it with this? Or are you suggesting additions/modifications to mine?
An addition. I though that was clear by describing it as a detail. I'm proposing that, under one of your headings, to add a subheading. OK?
Next, you speak of the "Python build process" creating "run time accessible identification data". The first phrase gives me the impression that you're talking about building Python itself (and its library), which really isn't the province of the distutils suite.
Yes, I AM taking about building Python itself. Sigh. It it that hard to understand: when python is built, it know the name of the compiler. I want a module: import build_compiler cc = build_compiler.cc which is the name of the compiler used to build Python, so I can invoke that same compiler to build an extension. And all the other info needed, such as where various libraries are, what -DMACRO options are needed, etc, (If this SIG produces usable code soon enough, Guido might find it useful to
rework how the standard library is built for Python 1.6. That's the only connection I see between the two.)
Then you are missing the link: the build processes know how to build C. I want the build process to put that information into a standard module. So I can get it out, and call up the compiler to make a C extension FROM PYTHON.
I have no idea what you mean by "run time accessible identification data". Or rather, I can think of a half-dozen things you might mean. Please explain, or rephrase in a more conventional way.
have I explained now? This is the key problem for distutils BECAUSE it REQUIRES Guido to modify the build process to create the information needed to call up the compiler from python script. At least, this is by far the most reliable way to get the information: all that autoconfig and makefile and setup stuff has the information, and it is needed to be able to build C code later as well. That is the connection. Distutils cannot work if Guido doesn't give the information. SOME of the information is already available in some form: the name of the compiler is already encoded in the version string. But it isn't enough to actually emulate the compilation and linkage processes the same way Setup builds things. ------------------------------------------------------- John Skaller email: skaller@maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia
Quoth John Skaller, on 16 December 1998:
Yes, I AM taking about building Python itself. Sigh. It it that hard to understand: when python is built, it know the name of the compiler.
[...deletia...]
I want the build process to put that information into a standard module. So I can get it out, and call up the compiler to make a C extension FROM PYTHON.
A-ha! Thank you -- now it comes clear. As it happens, this was one of the things discussed at the Developer's Day session which I failed to mention in the online summary. Mea culpa. I also forgot to mention it on the requirements list. If I were more weasely, I could squirm out of it by saying "Well, this just follows from the requirement to easily build extension modules from source" or, "That's not a requirement, it's an implementation detail". But I'm not a weasel, and I think at some point I meant to mention this in the requirements document. So I've added it. I also changed the wording of the point that immediately leads into it; here're the changed and new bullet: <!-- the changed one --> <li>It must be possible (and preferably easy) to instead download and build from a source distribution. (Necessary for people who don't happen to be using one of the major platforms, or who don't trust built distributions.) <!-- the new one --> <li>To aid in installing all modules, and in building extension modules from C/C++ source, there must be a standardized way to get Python's configuration data (such as compiler, compiler flags, platform, default library directory, etc.). I'm managing to keep implementation details out of the requirements list, though. In particular, my proposal is for a new 'sys.config' module which would contain everything you might need to know about how Python was configured. The most important items are the compiler, compiler flags, platform, and library installation directories. For an example of the degree of flaming overkill possible in this sort of module, try this (assuming you have a fairly recent [>= 5.003 I think] version of Perl installed): perl -MConfig -e 'map { print "$_: $Config{$_}\n" } sort keys %Config;' You'll note that Perl's Config module stores *every* piece of information, gleaned, gathered, or queried from the user while running Perl's 'Configure' script. I don't think Python's sys.config need be *quite* so exhaustive, but I can personally vouch that the Config module has more utility than simply knowing where to install modules. (F'r instance, if you're wondering "Gee, is Perl's 'syscall' available on this platform?", or, "Was this Perl built with stdio or sfio?", then you can answer your questions with Config. That's pretty cool.) So, perhaps this is a good place to start a new flamewar^H^H^H^H^H^H^H^H discussion. Does anyone violently disagree with the concept of adding a sys.config module? And what should go in it? Here's a very rough start (loosely based on my long experience with building Perl and my very brief experience with building Python, and a bit of poking through Python's makefiles): cc C compiler used to build Python cppflags C preprocessor flags used to build Python (-I, -D, -U) optimize optimization/debugging flags used to build Python ccsharedflags C compiler flags required to build shared objects extra_ccflags any other C compiler flags used to build Python ccflags all compiler flags used to build Python: cppflags + optimize + extra_ccflags ccextflags all compiler flags used to build Python extensions: cppflags + optimize + ccsharedflags + extra_ccflags ldlibdirs -L flags used to build Python ldlibs -l flags used to build Python extra_ldflags any other needed linker flags ldsharedflags linker flags required to build shared objects ldflags all linker flags used to build Python: ldlibdirs + ldlibs + extra_ldflags ldextflags linker flags used to build Python extensions: ldlibdirs + ldlibs + ldsharedflags + extra_ldflags platform hardware/os/vendor (or something along those lines) install_bindir directory to install binary executables install_scriptdir directory to install non-binary executables install_includedir directory to install C header files install_confincludedir directory to install platform- and build-specific C header files install_libdir main Python library directory install_sitelibdir directory for site-specific modules Obviously this is very sketchy. A proper proposal would have examples and/or defaults (eg. "What do I get if I just do "configure ; make ; make install" on a standard POSIX-compliant system?). Any replies should be implicitly prefixed, "Greg, you are a newbie and don't know what you're talking about" because it's true. *But* I have configured and built Perl more times than I care to admit, and I know what works and doesn't work there. All you long-time Python hackers surely know what works and doesn't work *here*, so together we should be able to cobble something together. Go to it... Greg PS. oh, I guess I should briefly address John's remaining points:
This is the key problem for distutils BECAUSE it REQUIRES Guido to modify the build process to create the information needed to call up the compiler from python script. At least, this is by far the most reliable way to get the information: all that autoconfig and makefile and setup stuff has the information, and it is needed to be able to build C code later as well.
That is the connection. Distutils cannot work if Guido doesn't give the information.
I agree, but I hadn't considered it the *key* problem. After all, the first version of distutils will have to work with 1.5.2; one possibility is (hack, blech, gaack) to chew up config.status and/or Makefile from the Python source directory. Obviously, having sys.config available in the standard library is a far nicer solution in every respect. -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 x287 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
Greg Ward wrote:
In particular, my proposal is for a new 'sys.config' module which would contain everything you might need to know about how Python was configured. The most important items are the compiler, compiler flags, platform, and library installation directories.
Good idea, though the module should probably be named sysconfig (without the dot). I'd suggest sticking in the config.h file as dictionary, e.g. {'HAVE_TZNAME':1, 'HAVE_TM_ZONE':0,...} and the information from Makefile (the one in /usr/local/lib/python1.5/config) parsed and stored in a class instance (using the same names as in the Makefile for clarity), e.g. sysconfig.make.CC C compiler .OPT compiler flags etc. Rather than discuss which things to include or not, just put all the information in there you can find by simply parsing Makefile. Note that the parsing mechanism should also allow for fairly decent $(VARIABLE) expansion. .../Setup should also go into a dictionary: {'cPickle.c':'cPickle.c',...} Why dictionaries for config.h and Setup and an instance for make ? Because the Makefile entries will not change to often and the class can provide reasonable defaults. config.h and Setup change a lot between versions and maintaining them as instances would probably cause lot's of compatibility headaches. The remaining problem (?) is where to find the three files. Looking at my Makefile they seem to be at: sys.prefix + '/lib/python' + sys.version[:3] + '/config/' and sys.prefix + '/include/python' + sys.version[:3] + '/' But the install dirs are changeable during configure... we might need some help from Guido here, e.g. the sys module could include the information: sys.configdir, sys.incldir and sys.libdir. Note that the configuration for WinXX is hard-coded, so the sysconfig module could also provide the data on those platforms. Together with the other information in sys and os.environ this should provide a pretty complete picture of how the machine is setup. -- Marc-Andre Lemburg Y2000: 380 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : ---------------------------------------------------------
M.-A. Lemburg wrote:
Greg Ward wrote:
In particular, my proposal is for a new 'sys.config' module which would contain everything you might need to know about how Python was configured. The most important items are the compiler, compiler flags, platform, and library installation directories.
Good idea, though the module should probably be named sysconfig (without the dot).
I'd suggest sticking in the config.h file as dictionary, e.g. {'HAVE_TZNAME':1, 'HAVE_TM_ZONE':0,...} and the information from Makefile (the one in /usr/local/lib/python1.5/config) parsed and stored in a class instance (using the same names as in the Makefile for clarity), e.g.
sysconfig.make.CC C compiler .OPT compiler flags etc.
Rather than discuss which things to include or not, just put all the information in there you can find by simply parsing Makefile. Note that the parsing mechanism should also allow for fairly decent $(VARIABLE) expansion.
../Setup should also go into a dictionary: {'cPickle.c':'cPickle.c',...}
Why dictionaries for config.h and Setup and an instance for make ? Because the Makefile entries will not change to often and the class can provide reasonable defaults. config.h and Setup change a lot between versions and maintaining them as instances would probably cause lot's of compatibility headaches.
The remaining problem (?) is where to find the three files. Looking at my Makefile they seem to be at: ...
Actually, at the discussion, we planned on compiling them right into the interpreter. That would prevent version skews between a compiled Python and the libraries. However, now that I'm thinking about it... screw that. If "import sysconfig" picks up the wrong version, then you have plenty of other issues to deal with. I'd much rather see a .py file that a bunch of stuff compiled into the interpreter. I'd also recommend ignoring all the dictionary vs instance stuff. Just have a bunch of assignments dropped into a sysconfig.py module. IMO, it would look a lot like FCNTL.py (h2py output files). No need to get fancy with defaults and stuff. Just write the dumb file out. I agree with Marc on "just put it all in there". No reason to sort it out. Cheers, -g -- Greg Stein, http://www.lyra.org/
participants (4)
-
Greg Stein
-
Greg Ward
-
John Skaller
-
M.-A. Lemburg