[Python-Dev] what is the significance of "plat-linux2" in the python build process?

[Robert P. J. Day]

  (note:  i'm not a python dev subscriber so please make sure you CC
me with any advice and i'm hoping this desperate plea for assistance
is at least enough on-point for this list that someone can help me
out.  and, yes, this is rather verbose but i wanted to supply all of
the relevant details in one shot.)

  i asked about this on the general python help list but i suspect
it's more appropriate to ask developers about this and i'm hoping
someone can clear this up for me.

  i'm building an embedded system using wind river linux 4.2 (WRL
4.2), and part of that build process involves downloading, patching
and compiling python 2.6.2 for the eventual target filesystem.  this
is all being done on my 64-bit ubuntu 11.04 system and, to keep things
simple, i'm not even cross-compiling, i've selected a common 64-bit PC
as the target, so i should be able to ignore any cross compile-related
glitches i might have had.

  this build process works just fine for everyone else on the planet
but it fails for me because i'm doing something apparently no one else
has tried -- i'm running a (hand-rolled) linux 3.x kernel on my build
host and it *seems* that that's what's messing up the python
compilation somewhere in the WRL build scripts.  (as a side note, i
have run across other issues in the WRL build system where it was
simply never imagined that one might want to build on a system running
a 3.x kernel, so that's why i'm suspecting it has something to do with
that.  apparently, i'm out there on the bleeding edge and this is what
might be causing me grief.)

  the symptom seems to be that there is confusion in the python build
process between two directories: "plat-linux2" and "plat-linux3".  my
first simple question would be: what do those names represent and how
should they appear in the build process?

  as a benchmark, i downloaded an *absolutely stock* Python-2.6.2
tarball, untarred it, ran "./configure", then searched for any
references to those strings just so i could have a basis for
comparison.  so, immediately after the configure, here's what i found
for the stock 2.6.2 python tarball:

$ find . -name "plat-linux*"
./Lib/plat-linux2
$

$ grep -r plat-linux *
Doc/install/index.rst:   ['', '/usr/local/lib/python2.3',
'/usr/local/lib/python2.3/plat-linux2',
Doc/install/index.rst:'/www/python/lib/pythonX.Y/plat-linux2', ...]``.
Misc/RPM/python-2.6.spec:%{__prefix}/%{libdirname}/python%{libvers}/plat-linux2
Misc/HISTORY:	* Lib/plat-sunos5/CDIO.py, Lib/plat-linux2/CDROM.py:
Misc/HISTORY:e.g. lib-tk, lib-stdwin, plat-win, plat-linux2,
plat-sunos5, dos-8x3.
$

  so, before the build, there are a few references to plat-linux2 and
*none* to plat-linux3.  i then ran "make" (which seemed to work just
fine) and here's the result of a similar search after the make:

$ find . -name "plat-linux*"
./Lib/plat-linux2
$

  so that's still the same after the build, there is no plat-linux3
file or directory that's been created.  however, if i do a recursive
grep:

$ grep -r plat-linux3 *
Binary file libpython2.6.a matches
Binary file Modules/getpath.o matches
Binary file python matches
$

  then it's clear that the string "plat-linux3" is now embedded in a
small number of the build results.  so what does that string
represent?  why is it there and what does "2" mean compared to "3" in
this context?  and, most importantly, even though it's there, it
didn't stop the build from completing.

  now take a look at the tail end of the output of the WRL build of
python, where things go wrong (what is clearly a packaging step so i'm
well aware that this is somewhat outside the scope of normal python
building):

===== begin output =====
... snip ...
Checking for unpackaged file(s): /home/rpjday/WindRiver/projects/42/python/host-cross/bin/../lib64/rpm/check-files /home/rpjday/WindRiver/projects/42/python/build/INSTALL_STAGE/python-2.6.2
error: Installed (but unpackaged) file(s) found:
   /usr/lib64/python2.6/plat-linux3/IN.py
   /usr/lib64/python2.6/plat-linux3/IN.pyc
   /usr/lib64/python2.6/plat-linux3/IN.pyo
   /usr/lib64/python2.6/plat-linux3/regen


RPM build errors:
    File not found: /home/rpjday/WindRiver/projects/42/python/build/INSTALL_STAGE/python-2.6.2/usr/lib64/python2.6/plat-linux2
    Installed (but unpackaged) file(s) found:
   /usr/lib64/python2.6/plat-linux3/IN.py
   /usr/lib64/python2.6/plat-linux3/IN.pyc
   /usr/lib64/python2.6/plat-linux3/IN.pyo
   /usr/lib64/python2.6/plat-linux3/regen
/home/rpjday/WindRiver/projects/42/python/scripts/packages.mk:2661: *** [python.install] Error 1
/home/rpjday/WindRiver/projects/42/python/scripts/packages.mk:3017: *** [python.buildlogger] Error 2
/home/rpjday/WindRiver/projects/42/python/scripts/packages.mk:3225: *** [python.build] Error 2
make: *** [python] Error 2
make: Leaving directory `/home/rpjday/WindRiver/projects/42/python/build'

===== end output =====

  note the obvious reference to this "plat-linux3" directory that
appears out of nowhere that never existed in the stock build.  and if
i wander down to the WRL python build directory:

$ find . -name plat-linux*
./Lib/plat-linux2
./Lib/plat-linux3
$

$ find Lib/plat-linux[23]
Lib/plat-linux2
Lib/plat-linux2/CDROM.py
Lib/plat-linux2/DLFCN.py
Lib/plat-linux2/IN.py
Lib/plat-linux2/regen
Lib/plat-linux2/TYPES.py
Lib/plat-linux3
Lib/plat-linux3/IN.py
Lib/plat-linux3/regen
$

  and this is as far as i got before confusion set in.  it seems that
the WRL build process is getting confused about which of those two
values to use for the build and ends up scattering generated artifacts
across both directories, at which point the packaging step gets
confused and gives up.

  if anyone can clarify what might be going on here and what *should*
be happening, i'd be grateful.  i realize i'm asking for remote
diagnosis on a proprietary build system, i'm just wondering what a
native python build *should* look like and what it should produce.
thanks muchly for any guidance.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================