Mailman 3 problems building Python 2.1a1 on QNX 4.25 - Python-Dev

newer
RE: [Python-Dev] problems building...

problems building Python 2.1a1 on QNX 4.25

Goodger, David

26 Jan 2001 26 Jan '01

1:46 p.m.

[CC'ing to Armin Steinhoff, who maintains pyqnx on SourceForge.]

I'm having trouble building Python 2.1a1 on QNX 4.25. Caveat: my C is very rusty (long live Python!), I don't know my way around configure, and am not familiar with Python's Makefile. Python 2.0 compiled fine (with a couple of tweaks), but I'm getting caught by the new way of building things. Please help if you can! Many thanks in advance.

Here's an excerpt of my efforts:

# cd /tmp/py # gunzip -c < python-2.1a1.tgz | tar -rf - # cd Python-2.1a1 # ./configure 2>&1 | tee ../configure.1 # make 2>&1 | tee ../make.1 ... ./python //5/tmp/py/Python-2.1a1/setup.py build 'import site' failed; use -v for traceback Traceback (most recent call last): File "//5/tmp/py/Python-2.1a1/setup.py", line 4, in ? import sys, os, string, getopt ImportError: No module named string

Running ./python results in stack overflow. The old QNX instructions in README recommend editing Modules/Makefile: LDFLAGS= -N 64k

# make 2>&1 | tee ../make.2

Same error as first make. But now the stack doesn't overflow.

# python 'import site' failed; use -v for traceback Python 2.1a1 (#2, Jan 26 2001, 11:38:55) [C] on qnxJ Type "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/local/lib/python', '/home/dgoodger/lib/python', '/5/tmp/py/Python-2.1a1/Lib', '/5/tmp/py/Python-2.1a1/Lib/plat-qnxJ', '/tmp/py/Python-2.1a1/Modules'] >>> ^D

# fullpath . . is //5/tmp/py/Python-2.1a1

The QNX node number prefix '//5' (machine or host number, equivalent to a 'hostname:' prefix for network paths) is being reduced somehow (path normalization?) to '/5', so paths don't resolve. 2 slashes ('//') are required at the head of the path. Is this something that can be fixed?

I added a prefix (QNX virtual-to-real path mapping on the filesystem tree) to correct this:

# prefix -A /5=//5

Now /5 points to //5, similar to a link.

# make 2>&1 | tee ../make.3 ... ./python //5/tmp/py/Python-2.1a1/setup.py build unable to execute ld: No such file or directory running build running build_ext building 'struct' extension creating build creating build/temp.qnx-J-PCI-2.1 cc -O -I. -I/5/tmp/py/Python-2.1a1/./Include -IInclude/ -I/usr/local/include -c /5/tmp/py/Python-2.1a1/Modules/structmodule.c -o build/temp.qnx-J-PCI-2.1/structmodule.o creating build/lib.qnx-J-PCI-2.1 ld build/temp.qnx-J-PCI-2.1/structmodule.o -L/usr/local/lib -o build/lib.qnx-J-PCI-2.1/struct.so error: command 'ld' failed with exit status 1 make: *** [sharedmods] Error 1

QNX doesn't have an 'ld' command. Is configure not getting its info to setup.py? (Is it supposed to?)

What should I check? I have logs of each of the configure & make runs. Should I submit this as a bug on SourceForge?

Hope to hear from somebody soon.

David Goodger Systems Administrator & Programmer, Advanced Systems Automation Tooling Systems Inc., Automation Systems Division direct: (519) 653-4483 ext. 7121 fax: (519) 650-6695 e-mail: dgoodger@atsautomation.com

Show replies by date

Neil Schemenauer

26 Jan 26 Jan

7:26 a.m.

On Fri, Jan 26, 2001 at 04:46:13PM -0500, Goodger, David wrote:

...

Running ./python results in stack overflow. The old QNX instructions in README recommend editing Modules/Makefile: LDFLAGS= -N 64k
# make 2>&1 | tee ../make.2

The README should be changed to say edit the toplevel Makefile. Should those flags be the default? If you can give me the MACHDEP from your Makefile I can add it to configure.in.

...

QNX doesn't have an 'ld' command. Is configure not getting its info to setup.py? (Is it supposed to?)

I'm not sure how distutils figures out what to use for ld. It doesn't appear in the Makefile. It think this is probably some distutils thing. Andrew?

Neil

Andrew M. Kuchling

2:28 p.m.

On Fri, Jan 26, 2001 at 07:26:12AM -0800, Neil Schemenauer wrote:

...

I'm not sure how distutils figures out what to use for ld. It doesn't appear in the Makefile. It think this is probably some distutils thing. Andrew?

It looks at LDSHARED. See customize_compiler in Lib/distutils/sysconfig.py. Looking in Modules/Makefile, LDFLAGS is only used for the final link to produce a Python executable, so I think this is up to the Makefile, not setup.py.

--amk

Neil Schemenauer

7:56 a.m.

On Fri, Jan 26, 2001 at 05:28:03PM -0500, Andrew M. Kuchling wrote:

...

On Fri, Jan 26, 2001 at 07:26:12AM -0800, Neil Schemenauer wrote:

...
I'm not sure how distutils figures out what to use for ld.

It looks at LDSHARED.

Okay. David, what should LDSHARED say for QNX? I can add the magic to configure.in.

Neil

Guido van Rossum

1:52 p.m.

...

[CC'ing to Armin Steinhoff, who maintains pyqnx on SourceForge.]

I'm having trouble building Python 2.1a1 on QNX 4.25. Caveat: my C is very rusty (long live Python!), I don't know my way around configure, and am not familiar with Python's Makefile. Python 2.0 compiled fine (with a couple of tweaks), but I'm getting caught by the new way of building things. Please help if you can! Many thanks in advance.

Here's an excerpt of my efforts:
# cd /tmp/py
# gunzip -c < python-2.1a1.tgz | tar -rf -
# cd Python-2.1a1
# ./configure 2>&1 | tee ../configure.1
# make 2>&1 | tee ../make.1
...
./python //5/tmp/py/Python-2.1a1/setup.py build
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "//5/tmp/py/Python-2.1a1/setup.py", line 4, in ?
    import sys, os, string, getopt
ImportError: No module named string
Running ./python results in stack overflow. The old QNX instructions in README recommend editing Modules/Makefile: LDFLAGS= -N 64k
# make 2>&1 | tee ../make.2
Same error as first make. But now the stack doesn't overflow.
# python
'import site' failed; use -v for traceback
Python 2.1a1 (#2, Jan 26 2001, 11:38:55) [C] on qnxJ
Type "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/python', '/home/dgoodger/lib/python', 
'/5/tmp/py/Python-2.1a1/Lib', '/5/tmp/py/Python-2.1a1/Lib/plat-qnxJ', 
'/tmp/py/Python-2.1a1/Modules']
>>> ^D

# fullpath .
. is //5/tmp/py/Python-2.1a1
The QNX node number prefix '//5' (machine or host number, equivalent to a 'hostname:' prefix for network paths) is being reduced somehow (path normalization?) to '/5', so paths don't resolve. 2 slashes ('//') are required at the head of the path. Is this something that can be fixed?

Aha -- you may need QNX-specific path manipulation functions. What's going on is that site.py normalizes the entries in sys.path, using this function:

def makepath(*paths): dir = os.path.join(*paths) return os.path.normcase(os.path.abspath(dir))

I've got a feeling that os.path.abspath(dir) here is the culprit in posixpath.py:

def abspath(path): """Return an absolute path.""" if not isabs(path): path = join(os.getcwd(), path) return normpath(path)

And here I think that normpath(path) is the routine that actually gets rid of the double leading /.

Feel free to submit a patch that leaves double leading slashes in if on QNX.

...

I added a prefix (QNX virtual-to-real path mapping on the filesystem tree) to correct this:
# prefix -A /5=//5
Now /5 points to //5, similar to a link.
# make 2>&1 | tee ../make.3
...
./python //5/tmp/py/Python-2.1a1/setup.py build
unable to execute ld: No such file or directory
running build
running build_ext
building 'struct' extension
creating build
creating build/temp.qnx-J-PCI-2.1
cc -O -I. -I/5/tmp/py/Python-2.1a1/./Include -IInclude/
-I/usr/local/include -c /5/tmp/py/Python-2.1a1/Modules/structmodule.c -o build/temp.qnx-J-PCI-2.1/structmodule.o creating build/lib.qnx-J-PCI-2.1 ld build/temp.qnx-J-PCI-2.1/structmodule.o -L/usr/local/lib -o build/lib.qnx-J-PCI-2.1/struct.so error: command 'ld' failed with exit status 1 make: *** [sharedmods] Error 1

QNX doesn't have an 'ld' command. Is configure not getting its info to setup.py? (Is it supposed to?)

What should I check? I have logs of each of the configure & make runs. Should I submit this as a bug on SourceForge?

Hope to hear from somebody soon.

This is probably in the realm of the distutils. I have no idea how to teach it to build on QNX, sorry!

--Guido van Rossum (home page: http://www.python.org/~guido/)

Andrew M. Kuchling

2:01 p.m.

On Fri, Jan 26, 2001 at 04:46:13PM -0500, Goodger, David wrote:

...

ImportError: No module named string

The 'import string' in setup.py actually seems to be redundant now, since nothing seems to actually refer to the string module. I've removed it from CVS.

...

The QNX node number prefix '//5' (machine or host number, equivalent to a 'hostname:' prefix for network paths) is being reduced somehow (path normalization?) to '/5', so paths don't resolve. 2 slashes ('//') are required at the head of the path. Is this something that can be fixed?

Ooh, very likely:

...

...
...
os.path.normpath('//5/foo/bar')

'/5/foo/bar'

Isn't // at the root a Unix convention of some sort for some network filesystems? Probably normpath() should just leave it alone.

...

QNX doesn't have an 'ld' command. Is configure not getting its info to setup.py? (Is it supposed to?)

setup.py should be parsing the Makefile. The old QNX instructions say Modules/Makefile should be edited, but with Neil's non-recursive Makefile patch (committed after alpha1's release), editing Modules/Makefile will have no effect. Try editing just the top-level Makefile, which should affect setup.py.

--amk

M.-A. Lemburg

2:15 p.m.

"Andrew M. Kuchling" wrote:

...

...
The QNX node number prefix '//5' (machine or host number, equivalent to a 'hostname:' prefix for network paths) is being reduced somehow (path normalization?) to '/5', so paths don't resolve. 2 slashes ('//') are required at the head of the path. Is this something that can be fixed?

Ooh, very likely:

...
...
...
os.path.normpath('//5/foo/bar')

'/5/foo/bar'

Isn't // at the root a Unix convention of some sort for some network filesystems? Probably normpath() should just leave it alone.

Samba uses //<hostname>/<mountname>/<path>. os.path.normpath() should probably leave the leading '//' untouched (having too many of those in the path doesn't do any harm, AFAIK).

-- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Fredrik Lundh

2:25 p.m.

mal wrote:> > Ooh, very likely:

...

...
...
...
...
os.path.normpath('//5/foo/bar')

'/5/foo/bar'

Isn't // at the root a Unix convention of some sort for some network filesystems? Probably normpath() should just leave it alone.

Samba uses //<hostname>/<mountname>/<path>. os.path.normpath() should probably leave the leading '//' untouched (having too many of those in the path doesn't do any harm, AFAIK).

from 1.5.2's posixpath:

def normpath(path): """Normalize path, eliminating double slashes, etc.""" import string # Treat initial slashes specially slashes = '' while path[:1] == '/': slashes = slashes + '/' path = path[1:] ... return slashes + string.joinfields(comps, '/')

from 2.0's posixpath:

def normpath(path): """Normalize path, eliminating double slashes, etc.""" if path == '': return '.' import string initial_slash = (path[0] == '/') ... if initial_slash: path = '/' + path return path or '.'

interesting...

Cheers /F

M.-A. Lemburg

2:51 p.m.

Fredrik Lundh wrote:

...

mal wrote:> > Ooh, very likely:

...
...
...
...
...
os.path.normpath('//5/foo/bar')

'/5/foo/bar'

Isn't // at the root a Unix convention of some sort for some network filesystems? Probably normpath() should just leave it alone.

Samba uses //<hostname>/<mountname>/<path>. os.path.normpath() should probably leave the leading '//' untouched (having too many of those in the path doesn't do any harm, AFAIK).

from 1.5.2's posixpath:

def normpath(path): """Normalize path, eliminating double slashes, etc.""" import string # Treat initial slashes specially slashes = '' while path[:1] == '/': slashes = slashes + '/' path = path[1:] ... return slashes + string.joinfields(comps, '/')

from 2.0's posixpath:

def normpath(path): """Normalize path, eliminating double slashes, etc.""" if path == '': return '.' import string initial_slash = (path[0] == '/') ... if initial_slash: path = '/' + path return path or '.'

interesting...

Here's the log message:

revision 1.34 date: 2000/07/19 17:09:51; author: montanaro; state: Exp; lines: +18 -23 added rewritten normpath from Moshe Zadka that does the right thing with paths containing ..

and the diff:

diff -r1.34 -r1.33 349,350d348 < if path == '': < return '.' 352,367c350,372 < initial_slash = (path[0] == '/') < comps = string.split(path, '/') < new_comps = [] < for comp in comps: < if comp in ('', '.'): < continue < if (comp != '..' or (not initial_slash and not new_comps) or < (new_comps and new_comps[-1] == '..')): < new_comps.append(comp) < elif new_comps: < new_comps.pop() < comps = new_comps < path = string.join(comps, '/') < if initial_slash: < path = '/' + path < return path or '.' ---

...

# Treat initial slashes specially
slashes = ''
while path[:1] == '/':
    slashes = slashes + '/'
    path = path[1:]
comps = string.splitfields(path, '/')
i = 0
while i < len(comps):
    if comps[i] == '.':
        del comps[i]
        while i < len(comps) and comps[i] == '':
            del comps[i]
    elif comps[i] == '..' and i > 0 and comps[i-1] not in ('', '..'):
        del comps[i-1:i+1]
        i = i-1
    elif comps[i] == '' and i > 0 and comps[i-1] <> '':
        del comps[i]
    else:
        i = i+1
# If the path is now empty, substitute '.'
if not comps and not slashes:
    comps.append('.')
return slashes + string.joinfields(comps, '/')

Revision 1.33 clearly leaves initial slashes untouched. I guess we should restore this...

Guido van Rossum

27 Jan 27 Jan

8:52 a.m.

...

revision 1.34 date: 2000/07/19 17:09:51; author: montanaro; state: Exp; lines: +18 -23 added rewritten normpath from Moshe Zadka that does the right thing with paths containing ..

[...]

...

Revision 1.33 clearly leaves initial slashes untouched. I guess we should restore this...

Yes, please! (Just the "leading extra slashes stay" behavior.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

Fredrik Lundh

28 Jan 28 Jan

1:37 a.m.

guido wrote:

...

...
Revision 1.33 clearly leaves initial slashes untouched. I guess we should restore this...

Yes, please! (Just the "leading extra slashes stay" behavior.)

just looked this up in the specs, and POSIX seem to require that leading slashes are preserved only if there are exactly two of them:

A pathname that begins with two successive slashes may be interpreted in an implementation-dependent manner, although more than two leading slashes are treated as a single slash. (from susv2)

maybe we should add a if len(slashes) > 2: slashes = "/" test to the patch?

Cheers /F

Thomas Wouters

9:39 a.m.

On Sun, Jan 28, 2001 at 10:37:45AM +0100, Fredrik Lundh wrote:

...

guido wrote:

...

...
...
Revision 1.33 clearly leaves initial slashes untouched. I guess we should restore this...

Yes, please! (Just the "leading extra slashes stay" behavior.)

...

just looked this up in the specs, and POSIX seem to require that leading slashes are preserved only if there are exactly two of them:

...

A pathname that begins with two successive slashes
may be interpreted in an implementation-dependent
manner, although more than two leading slashes are
treated as a single slash.
(from susv2)

...

maybe we should add a if len(slashes) > 2: slashes = "/" test to the patch?

How strictly do we need (or want, for that matter) to follow POSIX here ? I'm aware the module is called 'posixpath', but it's used in a bit more than just POSIX environments (or POSIX behaviours) so it might make sense to ignore this particular tidbit. What if there is a system that attaches a special meaning to ///, should we create a new path module for it ?

-- Thomas Wouters thomas@xs4all.net Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

M.-A. Lemburg

29 Jan 29 Jan

3:31 a.m.

Guido van Rossum wrote:

...

...
revision 1.34 date: 2000/07/19 17:09:51; author: montanaro; state: Exp; lines: +18 -23 added rewritten normpath from Moshe Zadka that does the right thing with paths containing ..

[...]

...
Revision 1.33 clearly leaves initial slashes untouched. I guess we should restore this...

Yes, please! (Just the "leading extra slashes stay" behavior.)

Checked in a patch which preserves '/' and '//' but converts more than 3 initial slashes into one (see Fredrik's note about POSIX standard on this).

8272

Age (days ago)

8275

Last active (days ago)

List overview

Download

12 comments

8 participants

tags (0)

participants (8)

Andrew M. Kuchling
Fredrik Lundh
Fredrik Lundh
Goodger, David
Guido van Rossum
M.-A. Lemburg
Neil Schemenauer
Thomas Wouters