[Patches] [ python-Patches-460751 ] --with-shared: python with shared lib

noreply@sourceforge.net noreply@sourceforge.net
Thu, 20 Sep 2001 12:16:36 -0700


Patches item #460751, was opened at 2001-09-11 14:29
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=460751&group_id=5470

Category: Build
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Frederic Giacometti (giacometti)
Assigned to: Martin v. Löwis (loewis)
Summary: --with-shared: python with shared lib

Initial Comment:
As a follow up to patch 400938
(https://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=400938),
here is an implementation of the --with-shared
configure option for building python on Posix systems
as a shared library.

Building python as shared library is activated with the
--with-shared option on ./configure

The following points are drawn to particular attention:

- A configbuild.py module is generated and placed in
the machine-dependent directory. It records most 
Python/Makefile variables (CC, CFLAG, LDSHARED...), and
make them available in Python.

- The build has been tested on:
   Linux2: OK
   SunOS5: OK
   IRIX64: Build OK, crash on test (bus error)

- Inference with distutils: I found it very difficult
understanding the distutil architecture and finding the
right points of change; I minimized my interaction
there. Furthermore, the availability of the
'configbuild' module overlaps some of the distutils;
but 'configbuild' is very easy to use ...

- Unresolved symbols: I turned on the symbol checking
flags whenever possible, on behalf the its better
generating the error on build than at runtime.

- configure.in: I only edited the configure Bourne
script, and did not bother with configure.in (actually,
I deleted it) and its autoconfig/m4 macro comparses.

Frederic Giacometti





----------------------------------------------------------------------

>Comment By: Frederic Giacometti (giacometti)
Date: 2001-09-20 12:16

Message:
Logged In: YES 
user_id=93657

The patch is against Python 2.1.1.
I'll try to integrate this with the current source. I'll
work with a single checkout across the multiple platforms.

Regarding buildno: I was getting recursive dependency links,
after I made the shared library dependent upon its .o files.
I had to break the recursivity. 

What is dguxR4 ? Is it OSF1 V4.0, a.k.a. Digital Unix 4.0,
a.k.a. Tru64 ? I have now access to an OSF1 v4.0 (a.k.a. DU
4.0). I'll do the build there.
uname returns 'OSF1 ...'

The -lsocket is needed for building the socket module, and
had to be moved there (cf. Python/setup.py, I think).
It is not used in the core, so I dropped it there.
Dito for -ldsl.
No symbols are missing when linking libpython2.1.so, anyway.
These libraries just have now to be linked with the modules
which use them.
The rule I applied for deciding on libraries are:
- link with all libraries necessaries so as to suppress all
unresolved symbol output
- link only with libraries whose absence triggers unresolved
symbol outputs

- -lmpc: You might be right, but then -lmpc should be
present only on systems that need it (and, first of all
these systems should be identified/documented).

- fpectl: dito

FG


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-09-18 10:05

Message:
Logged In: YES 
user_id=21627

I've tried testing your patch, but I get numerous failed
chunks when applying it against the current CVS. What
version does this patch apply to? To simplify the
integration, it would be greatly appreciated if you could
use the current source base.

>From reviewing the patch, I still have a number of concerns:
- it appears that you change the mechanism by which buildno
is incremented. Why?
- There appear to extra white space changes (e.g. in the
distclean chunk); those are naturally unrelated to the patch
- please remove them
- it appears that configuration knowledge for some platforms
is lost when applying this patch, e.g. the dguxR4
special-casing of passing -pic is gone. Please try to
integrate any such knowledge faithfully into your patch,
instead of restricting its applicability to just the
platforms where you've tested it on.
- There is something going on with dropping -lsocket from
the patch. What is that, and why does it happen? Instead, it
appears that you put 'socket' and 'nsl' into setup.py.
According to configure.in, this is wrong on some systems:
they may not have the libraries, or the libraries may be
broken.
- Other libraries appear to suffer as well, such as removing
-lmpc. That might mean that you break thread support on some
systems.
- the chunk to disable fpectl if nothing is known about the
systems appears to be unrelated.


----------------------------------------------------------------------

Comment By: Frederic Giacometti (giacometti)
Date: 2001-09-17 10:36

Message:
Logged In: YES 
user_id=93657

I'm attaching an updated patch where, following Martin
suggestions:
  - I removed the configbuild thing, and replaced it with
distutils.sysconfig.get_conf_var calls
  - I manually cleaned up the patch from unrelated changes
(.cvsignore, @ Makefile modifiers...)

Remains the configure.in thing.

If one looks at autoconf:

- it takes care of plateform idiosyncracies, for platform
which do not follow posix API specs; this was of use in
previous generation Unix OS's, and these legacy systems do
not evolve anymore. On present Unixes, and with regard to
the features used for building the core python engine,
nothing moves, and autoconf updates bring nothing.
 Doesn't the Python 1.5.2's configure still works as it did
the first day it was generated ... ?

- autoconf is of use for configuring particular packages and
libraries (e.g.: for configuring and building python
extensions, Tkinter/Tk/TCL...); unfortunately this never
worked with python, and the task is now assumed by the
distutils.

- In the Python core build, autoconf is of no help in
anything sensible: thread, shared library, debug build,
compiler options... 
All these were programmed manually in Bourne; and for doing
the later, autoconf is a complexifying factor on something
already fairly complex - i.e. a handicap -.

- not to say, autoconf is of no use on non-posix plateforms
(windows...)

- last, and the least: autoconf now requires that perl be
installed...

Frederic Giacometti



----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-12 18:36

Message:
Logged In: YES 
user_id=6380

> What is the point of keeping using the autoconf thing when
> it is easier to maintain configure directly, than going
> through configure.in ?

But it's not. One line of configure.in often expands to
dozens (occasionally hundreds) of lines of configure.
autoconf has arcane knowledge that gets updated with new
autoconf releases.

You are entitled to your opinion, but in this case, it's
wrong. Go fork your own version of Python if you don't want
to learn autoconf.

----------------------------------------------------------------------

Comment By: Frederic Giacometti (giacometti)
Date: 2001-09-12 16:06

Message:
Logged In: YES 
user_id=93657

I am not willing  to make anything difficult for anybody,
but:

What is the point of keeping using the autoconf thing when
it is easier to maintain configure directly, than going
through configure.in ?

After configure has been generated once, it makes more sense
to me to work directly on configure, and forget about
configure.in.
configure (pure Bourne shell) is a easier to understand, to
troubleshoot, and to maintain that configure.in (a mixture
of Bourne shell and autoconf m4 macros).

I'll have a second look at the autoconf thing latter  on -
at present I hardly understand anything to configure.in, and
got 'configure --with-shared' working with satisfaction
across multiple platforms without undue difficulties.






----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-12 11:37

Message:
Logged In: YES 
user_id=6380

> Autoconf is not installed on the commercial Unix
> platforms, it is useless for me to learn it, and I was
> better off directly editing ./configure. However, if
others
> are familiar with the tool, it should be straightforward
for
> them to retrocede the changes from configure to
configure.in

Sure. You'll just have to wait until we have time for that.
You can make it easy for us to integrate your changes, or
you can make it difficult. Your choice.

----------------------------------------------------------------------

Comment By: Frederic Giacometti (giacometti)
Date: 2001-09-12 11:12

Message:
Logged In: YES 
user_id=93657


- I will remove the configbuild; since Martin pointed to me
the existence of  distutils.sysconfig.get_config_vars(). I
did not know about this function... Thanks!
On the fly, you'll also note the following potential problem
with using get_config_vars() instead of configbuild: 
configbuild will report the actual values used for the
build, whereas get_config_vars() will report erroneous
values whenever the Makefile internal variables were
overriden at build time (by either the environment, or by
commandline assignment of the type 'make CC=gxx ...'); not
to mention that configbuild does not need to find and parse
the original Makefile (with all the potential problems
associated).

- The .purify line should be in the CVSROOT/cvsignore CVS
admin file, not in .cvsignore in the root directory of the
source distribution...

- Autoconf is not installed on the commercial Unix
platforms, it is useless for me to learn it, and I was
better off directly editing ./configure. However, if others
are familiar with the tool, it should be straightforward for
them to retrocede the changes from configure to configure.in

- I had to introduce THREADOBJ because you will note that,
in the present implementation, Modules/thread.o appears
multiple times in  LIBOBJS, which causes warning at link
time. Now, Modules/thread.o appears only once




----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-09-12 01:06

Message:
Logged In: YES 
user_id=21627

As for libtool: I strongly advise not to use it. It is
broken in too many ways to enumerate.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-09-12 00:18

Message:
Logged In: YES 
user_id=21627

I recommend to rework this patch to get rid of 
configbuild. I cannot see what you achieve with the 
configbuild module that you couldn't do with 
distutils.sysconfig.get_config_vars (which is capable of 
parsing the Makefile directly).

I also recommend to go through the patch chunk by chunk, 
asking yourself whether this chunk *really* is needed to 
achieve the desired functionality. E.g. I cannot see the 
relationship of the .cvsignore chunk; I also believe that 
removal of .purify is incorrect. Likewise, while I 
sympathize with the unresolved symbols change, I believe 
it is unrelated to the main objective of this patch 
(building Python shared).

What is the rationale for not changing configure.in? 
As-is, this patch is useless: configure will be 
overwritten everytime you run autoconf. Please get a copy 
of autoconf on one of your systems, change configure.in, 
run autoconf, and then provide us with a patch that only 
changes configure.in: everybody reviewing this patch 
should be capable of running autoconf. configure is a 
generated file and should not be edited manually.

What is the rationale for the introduction of the 
THREADOBJ variable?



----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-11 19:03

Message:
Logged In: YES 
user_id=6380

Haven't seen the patch yet, but I like the idea -- the
command line option suggests that it shouldn't hurt
platforms where it can't be done or has to be done
differently.

Alternative: we could also consider using libtool. I didn't
like that much in the past, but maybe it would work. Dunno.
Anybody know about it?

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=460751&group_id=5470