Mailman 3 December 1998 - Distutils-SIG

[Distutils-sig] distutils charter and interscript
by John Skaller April 29, 2018

April 29, 2018

3 2

Re: [Distutils] Anonymous CVS up-and-running
by John Skaller Dec. 28, 1998

Dec. 28, 1998

At 17:43 22/12/98 -0500, Fred L. Drake wrote: > >John Skaller writes: > > I have done this, it is about 5 minutes work for a makefile > > expert. It took me half a day to get working properly :-) > > Cool. Send Guido a patch and rationale as to why it should be in >the standard distribution. I can't. It was a patch to version 1.5.0 which i don't have any more. It got clobbered when I installed 1.5.1 of course. >Frankly, the reason to install pythonpp is >lost on me. The you don't need to invoke it. But other people in the C++ SIG want to use C++ to extend Python. This is not the place to argue about it, they want it. :-) ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

2 1

Re: [Distutils] Access to Python config info
by John Skaller Dec. 28, 1998

Dec. 28, 1998

At 11:58 22/12/98 -0500, Fred L. Drake wrote: > >John Skaller writes: > > This depends on knowing where the makefile and > > the config file are. Which suggest a further factorisation: > > Ok, I've modified distutils.sysconfig to contain: > > get_config_h_filename() > get_makefile_filename() > parse_config_h(fileobject, g=None) > parse_makefile(fileobject, g=None) > > Note that the parse_*() routines take a file object instead of a >filename, Ah, that seems even better, yes! >and the dictionary to update may be passed in. If no >dictionary is provided a new one is used. However the dict is >obtained, it is returned by the function. Right. Seems OK. > Equivalents for non-Unix platforms are still needed; has anyone >thought about Windows or Mac platforms? I don't think we (necessarily) need equivalents for the following reason: the sysconfig data isn't meant for user access: its there to support implementation of functions like 'compile_this_c_as_a_module'. It is these functions which need to be standardised (that is, cross platform). For the VC++ windows build, sysconfig could be omitted entirely, and a hand written 'compile' function provided. For Unix, it is convenient to parameterise such a function with the config data -- for that is the way the autoconf/ make/blah blah process works. none of this implies there shouldn't be an equivalent; but I think the important interfaces is the 'OO' one: <grin> the one characterized by function semantics rather than values of data attributes. > I passed on this for now. I think the cross-platform issues need to >be explored more. I agree. I would like people to explore the interface of my compilers module to see if it is _possible_ for that interface to work on all platforms. It may be that the Unix implementation uses sysconfig data to adapt to various Unix platforms, whereas the Mac or Windows version is hard coded, or uses a different scheme. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

2 1

Re: [Distutils] Access to Python config info
by John Skaller Dec. 24, 1998

Dec. 24, 1998

>Can you send around a URL to the interscript online doc? Yes, but I should say that it has been cited many times, it exists on the www.python.org home page in the 'news' list, and it is in comp.lang.python.announce. http://www.triode.net.au/~skaller/interscript is the home page. >And when you >refer to it, could you please cite a more specific reference? Call it a >character flaw, but I'm not going to just blindly go read a bunch of >doc. A concrete reference helps (me, at least) provide some >focus/structure to why I'm there reading the thing in the first place. I agree, but I want to apologise profusely that I am not able to do this easily at the moment. firstly, my working copy is not the same as the one on the Internet, and secondly the HTML pages are all numbered at the moment, and the numbers are arbitrary and will change completely with every build i do that adds or removes even a single heading. [There are over 500 pages] I am addressing this problem, but at the moment the best I can do is usually to say 'read the section about XXX'. The table of contents is not small, but it should guide finding relevant information quickly. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

1 0

Re: [Distutils] High level options interface
by John Skaller Dec. 24, 1998

Dec. 24, 1998

At 11:26 22/12/98 -0500, Fred L. Drake wrote: > >John Skaller writes: > > BTW: One weakness of this interfaces is that there > > is no provision for creating an object library > > (archive, ar, etc etc). > > Is this needed? I don't know. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

1 0

Enough about feature tests!
by Greg Ward Dec. 23, 1998

Dec. 23, 1998

Hmmm, this seems to be a recurring theme; I guess this is just what a moderator is supposed to do, though. Anyways, it should be obvious but apparently is not: arguing about how best to test for OS features through Python is not really on topic for this SIG. We should be arguing, err discussing, high-level design of the distutils suite; or, failing that, nit-picking implementation details. For example, the sysconfig module, what to call it, when it runs, how it provides its information, etc. are all on-topic. Please stick to relevant things and learn to live with the fact that there's no formal specification for the Python Library, nor is there likely to be one anytime soon, nor will we solve that perceived problem with the distutils. Thanks -- Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 x287 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

1 0

Re: [Distutils] Anonymous CVS up-and-running
by John Skaller Dec. 22, 1998

Dec. 22, 1998

> Agreed. I seem to recall someone had sent patches to support C++, >possibly to the C++-SIG. Does anyone know exactly what's needed? Yes, I know what is 'needed', but not how to get it. What is required is that the python executable be built using C, and another version of it built using C++. Let me call them python.exe and pythonpp.exe. For gcc/g++, three extra steps are required in the makefile: 1) compile python.c using C++ as well as C, name it pythonpp.o(bj). This is the _only_ file that has to be compiled with C++. 2) Link that main object file with the C built python library. The result, pythonpp.exe, works just like python.exe, _except_ that it will accept full C++ dynamic link libraries as python modules: that is, it will support RTTI, exception handling, and proper constructor initialisation of virtual tables, correctly. 3) Install pythonpp.exe as well as python.exe I have done this, it is about 5 minutes work for a makefile expert. It took me half a day to get working properly :-) > > 2) Some of the data, such as CFLAGS, is useless because it reflects > > the python source build tree. On my system > > Misc/Makefile.pre.in manages to get things right in practice; some >time needs to go into figuring out just how it does things and get >sysconfig to do the right thing as well. In my opinion, part of the solution is to factor the 'variables' more logically, and then modify the makefile to use that factorisation. To start with, may I suggest _private_ variables which are not really required for sysconfig, be renamed in the makefile to some convention, such as a leading underscore: these can be ignored by the sysconfig parser as implementation details. Anything with a pathname related to the build directory is likely to be private: for example the current 'CFLAGS' contains -I../../something which is clearly useless to the installed system. So I'd rename that variable _CFLAGS. Then I'd create a real CFLAGS which _only_ has the option flags, no include directories. This allows the flags to be combined with the include directory options obtained for the _installed_ system. (That information is available now in sysconfig -- and indeed directly from Python) > > 3) The directory for config.h is WRONG! > > config.h lives in > > What definition are you looking at? Huh? The code in the CVS archive is wrong (for my system). It creates a directory name using '/usr/local/lib/python1.5/' which is NOT where config.h is installed. It is installed in /usr/local/include/python1.5 I.e. 'lib' should read 'include'. As a result, the module fails on my system. It is a stock standard Linux installation, IMHO it has put config.h in the right place, therefore sysconfig.py is wrong. :-) (Sorry, it is hard for me to quote stuff because I email on NT, but do real work on Linux :-) > > 4) On my system: > > > > SIZEOF_INT = '4' > > Patch attached which gets this right in a general fashion. Thanks! > > 5) I think the module should NOT just put values > > into itself like > > I don't see any need to complicate the interface. Neither do I. Factorisation doesn't complicate the interface, it simplifies it. There are two categories of information, and they come from separate sources. The data strucure should surely reflect this? > > 6) There is no information from Setup. It would be useful to > > get the Tcl/Tk library and headers into the configuration > > if it was built. Hmmm. :( > > Variables defined in Setup & friends are included. Otherwise, >nothing from Setup is included. This can be changed. It may make >sense to use a separate module for this (modconfig?). I think each source of data should be separate, including the makefile info and the autoconf info. The existing architecture is correct, IMHO: its the simplest possible 'pythonic' representation of the raw data. [This is not enough for the high level goal, but it seems correct to build this lower level first] > > 7) _init_posix should NOT modify a dictionary, but return one: > > CHANGE: > > This item is tightly related to item 5; let's deal with that one as >the real issue. The rest will fall out of that result. OK. > > 8) LINKCC = ' gcc' has a bad leading space. > > Fixed in the patch. Thanks. > > 9) I have no idea how to use the available information. > > There are flags all over the place. > > Yes, that's what configure produces. The next layer needs to put >this information together in a usable form. Reading the Makefile >should help a lot in determining how the various bits should be put >together. Yes, I agree. It is clear that while the current set of attributes is not quite right, it contains a lot of information. Therefore, the first step of just getting it into a pythonic format is a good one. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

3 2

Re: [Distutils] Access to Python config info
by John Skaller Dec. 22, 1998

Dec. 22, 1998

>[example has_symlinks() elided] > > But this can be written on top of feature tests. Why not? because it cannot be done. > def has_symlinks(): > global has_symlinks > import os > if hasattr(os, "symlink"): > has_symlinks = true > return 1 > else: > has_symlinks = false > return 0 Obviously, this does not work. It tests if there is a symlinks attribute in os, not what should be tested: 1) if there is an attribute symlinks, it works according to spec 2) if there is not, then the os doesn't support symlinks To perform (1) would require actually creating some symlinks, and seeing if they 'worked'. This is what autoconf does; it tries to compile various features, and it tests them as well (where they're _apparently_ available, and where it is possible to do a short test). It would be much easier to _document_ that the symlinks attribute is present if and only if creation of symbolic links is supported with semantics XXXX (fill in specification here). Then I could rely on the above code. To perform (2) is hard, but might involve using shell scripts. (Assuming you have a reliable way of launching them :-) > Have you created a list of the system aspects that this is desirable >for? Is the list *finite*? No. I just keep running into problems. And, obviously, it is desirable to have a finite list: tradeoffs are necessary. >It may well be, but there will be a lot >of disagreement over what it should be. yes. But there is nothing new here :-) >For now, let's call this the >"features" module. Are you ready to propose an API? No. But I will work on it with others. I do have some architecture developed in interscript for this: several frames for different categories of features: * platform * user * site with some work done on building these frames, but I'm not actually using any of it. You might start by reading the relevant sections in the online interscript doco. Then there is something concrete to discuss. It's quite clear that the separate 'compilers' stuff ought be part of this(but isn't, in interscript 1.0a7-9). >I'm sure several >of us would be willing to read it and carefully consider it (I will, >though it may not sound like it;). I think the biggest problem is >defining it in a way that provides more information or more convenient >access to the information represented. Yes, I agree. One way to help that process is to try to _use_ an actual implementation. I have an implementation, but I'm not using it. I could use help structuring it; doing so requires the combined knowledge of many people with experience on many different platforms. I, by myself, do not have wide enough experience. I do have one comment to guide your thoughts. Consider JPython everywhere. Don't assume CPython. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

3 3

Re: [Distutils] Access to Python config info
by John Skaller Dec. 22, 1998

Dec. 22, 1998

At 14:22 21/12/98 -0500, Fred L. Drake wrote: > >John Skaller writes: > > HOW do you know this works? HOW do you know > > that the method is not, in fact, present, but doesn't work? > > WHERE is the standards document that specifies the > > precise semantics of the 'os' interface?? > > Sounds like I'd better get the documentation for "os" fixed before I >release again; it is on my list. I don't think that is really my point. I'm interested in a STANDARD, not just good documentation. Saying what the library does is a useful hint. Like reading the code. It isn't the same as a specification; which _requires_ certain behaviour. The python _language_ document is written more like a specification. If the language doesn't do what the doco says, its a bug in the implementation (or a fault in the doco). Not an excuse from Guido that it doesn't work on that platform 'because'. I don't want an explanation why something doesn't work, I want an authoritative document that I can cite as evidence of a bug. In other words, relieve myself of the responsibility of fixing it or working around it. I've been shown that my interpretation of what Python does is wrong, by citation of the language specification. I want to be shown wrong about the library (when I am), the same way. [On testing things to see if they work] > > This technique doesn't seem very reliable. :-( > > How so? Because it requires testing every single function in the whole system before using anything. Because there is no definitive specification of what might work or not. Do you see? And just _how_ do I test the functions? Using other functions that are themnselves suspect??? > But this can be written on top of feature tests. Why not? Sigh. Because there is no reliable way to test features. FOR EXAMPLE: consider the module 'commands'. How do I know if it works?? I do not see how to test it .. WITHOUT assuming it works. In fact, you can see by examining the code it will NOT work on NT. So interscript has it's own module called 'commands' that works on both posix and NT. I can't 'test' if it works, because it is one of the modules I need to test if tools such as patch and diff work. I have to rely on it working. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

3 2

Re: [Distutils] Access to Python config info
by John Skaller Dec. 22, 1998

Dec. 22, 1998

At 01:40 23/12/98 +1000, John Skaller wrote: Duh, sorry for replying to my own post again but: >EXECUTIVE SUMMARY > > 1. module parse_configuration, function get_config > returns a dictionary of config data parsed from > the makefile This depends on knowing where the makefile and the config file are. Which suggest a further factorisation: def parse_config_h(filename) def parse_makefile(filename) which would be generally useful! And truly statless. Now, provide: def get_config_h_filename() def get_makefile_filename() and we can wrap them up to get: def get_config(): dict = parse_config_h(get_config_h_filename()) dict.update(parse_config_h(get_makefile_filename())) return dict Again, this factorisation simplifies the design, and also creates added functionality. ------------------------------------------------------- John Skaller email: skaller(a)maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia

3 3