[capi-sig] Python for VxWorks

[Gary Kenaley]

Hi Nachaat,

Thank you for the reply and information! I have 2.7.3 mostly up and running under VxWorks 6.9.3.3 RTP. I have the entire cross build system working smoothly for the executable, parser, shared objects, and the install. I have two remaining questions that I hope you will help me with:

(1) What is the purpose of the *.pyc files, and should I be concerned that the installer does not include them in the final build result? On my native Ubuntu system, the *.pyc files are included, but in the VxWorks build they are not. I am not sure why they would be needed when hundreds of python scripts and shared objects are already available.


(2) Can you tell me how to get the 48 shared objects (*.so) to load automatically when the interpreted needs them (i.e. "import math")?

The specifics of the problems I am having with on the shared objects (dlopen(), etc.) are first that I cannot get python to link with dynload_shlib.o:

Currently I have an python RTP executable that is linked -non-static 
with a static library libpython2.7.a, and when it starts it successfully finds libc.so.1 if and only if I have putenv LD_LIBRARY_PATH.

However, to get full python module functionality at run-time, this python vxe needs to
 dynamically load-on-demand any of 48 *.so shared libraries that are built AFTER the python RTP executable
 is linked -non-static. I believe this means
 the 48 shared libraries should be built with the --export-dynamic flag.
 All of this works effortlessly in the native Ubuntu build, using the 
same source code and very similar Makefile.

When I try to enable dynamic loading [dlopen(), etc.] by building and 
linking in dynload_shlib.c (attached) and including it in 
libpython2.7.a, the link of the python RTP fails even though there were 
no compilation warnings) with:

c++pentium -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wl -MD -MP -non-static -Xbind-lazy -ldl -L/home/demo/WindRiver/vxworks-6.9/target/lib/usr/lib/pentium/PENTIUM4/common  -o python Modules/python.o libpython2.7.a
libpython2.7.a(dynload_shlib.o): In function `_PyImport_GetDynLoadFunc':
/romfs/python273/Python/dynload_shlib.c:94: undefined reference to `dlsym'
/romfs/python273/Python/dynload_shlib.c:130: undefined reference to `dlopen'
/romfs/python273/Python/dynload_shlib.c:133: undefined reference to `dlerror'
/romfs/python273/Python/dynload_shlib.c:141: undefined reference to `dlsym'
collect2: ld returned 1 exit status

A slightly more detailed build output for the python RTP is shown about 25 lines below.

I tried dozens of combinations of linker flags, ccpentium,  etc., but I always get exactly the same error. Perhaps the source code (dynload_shlib.c) needs to be changed?
I read through the Application Programmer's guide thoroughly and Googled, etc., but there are no examples 
showing the build process.
This seems to be the case that the Wind River manual refers to as plugins.

Could you please show me how to make this work? I could hardcode 
the dlopenflags for dlopen(pathname, dlopenflags) in dynload_shlib.c if 
you would tell me what the correct values should be. Of course it also 
needs to link (unless these flags can somehow interfere with that).

I was able to link python with  "-Wl,--warn-unresolved-symbols"  (with 4 warnings, instead of errors).
But when I try to access a plugin like math.so:
>>> import math 0xfcfd010 (iPython): RTP 0xfd05010 has been deleted due to signal 11.
Presumably because it tried to blindly call a function that it could not find...

As a possible alternative, now that I have the 48 shared objects built, could I leave the plug-in dynamic loading stubbed out of the libpython2.7.a, and link python.vxe -non-static 
against libc.so (as already done) AND the 48 shared objects? The 48 
shared objects would be built with "c++pentium  -mtune=pentium4 -march=pentium4 
-ansi -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -Wall -MD -MP -fpic 
-D__SO_PICABIRTP__  -shared -Wl,-shared -D_VX_CPU=_VX_PENTIUM4 
-D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu", but without --export-dynamic. Would this work,i.e. would the shared libraries be automatically loadedfrom LD_LIBRARY_PATH whenever the python executable tried to call the functions contained in them?

Thanks,

Gary

More details on the build and link:
demo at gamer-linux:/romfs/python273$ make python

ccpentium -c -fno-strict-aliasing -O0 -g -mtune=pentium4 -march=pentium4 -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wall -MD 
-MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu 
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h 
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip -I/romfs/python273/Include -Wno-parentheses -DNDEBUG -fwrapv -O3 -Wall 
-Wstrict-prototypes  -I. -IInclude -I./Include -O0 -g -mtune=pentium4 
-march=pentium4 -ansi -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -Wall -MD -MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu -I/home/demo/WindRiver/vxworks-6.9/target/usr/h 
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip 
-I/romfs/python273/Include -Wno-parentheses  -DPy_BUILD_CORE -o 
Python/dynload_shlib.o Python/dynload_shlib.c

ccpentium -c -fno-strict-aliasing -O0 -g -mtune=pentium4 -march=pentium4
 -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wall -MD 
-MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu 
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h 
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip
 -I/romfs/python273/Include -Wno-parentheses -DNDEBUG -fwrapv -O3 -Wall 
-Wstrict-prototypes  -I. -IInclude -I./Include -O0 -g -mtune=pentium4 
-march=pentium4 -ansi -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -Wall
 -MD -MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu
 -D_VX_TOOL=gnu -I/home/demo/WindRiver/vxworks-6.9/target/usr/h 
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip 
-I/romfs/python273/Include -Wno-parentheses  -DPy_BUILD_CORE \
      -DSVNVERSION="\"`LC_ALL=C svnversion .`\"" \
      -DHGVERSION="\"`LC_ALL=C `\"" \
      -DHGTAG="\"`LC_ALL=C `\"" \
      -DHGBRANCH="\"`LC_ALL=C `\"" \
      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c

rm -f libpython2.7.a
arpentium rc libpython2.7.a Modules/getbuildinfo.o

arpentium rc libpython2.7.a Parser/acceler.o Parser/grammar1.o 
Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o 
Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o
 Parser/pgen.o Parser/myreadline.o Parser/tokenizer.o

arpentium rc libpython2.7.a Objects/abstract.o Objects/boolobject.o 
Objects/bufferobject.o Objects/bytes_methods.o Objects/bytearrayobject.o
 Objects/capsule.o Objects/cellobject.o Objects/classobject.o 
Objects/cobject.o Objects/codeobject.o Objects/complexobject.o
 Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o 
Objects/genobject.o Objects/fileobject.o Objects/floatobject.o 
Objects/frameobject.o Objects/funcobject.o Objects/intobject.o 
Objects/iterobject.o Objects/listobject.o Objects/longobject.o 
Objects/dictobject.o
 Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o 
Objects/object.o Objects/obmalloc.o Objects/rangeobject.o 
Objects/setobject.o Objects/sliceobject.o Objects/stringobject.o 
Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o 
Objects/weakrefobject.o
 Objects/unicodeobject.o Objects/unicodectype.o

arpentium rc libpython2.7.a Python/_warnings.o Python/Python-ast.o 
Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o 
Python/compile.o Python/codecs.o Python/errors.o Python/frozen.o 
Python/frozenmain.o Python/future.o Python/getargs.o 
Python/getcompiler.o
 Python/getcopyright.o Python/getplatform.o Python/getversion.o 
Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o 
Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o 
Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o
 Python/pymath.o Python/pystate.o Python/pythonrun.o Python/random.o 
Python/structmember.o Python/symtable.o Python/sysmodule.o 
Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o 
Python/dtoa.o Python/formatter_unicode.o Python/formatter_string.o Python/dynload_shlib.o   Python/thread.o

arpentium rc libpython2.7.a Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o 

arpentium rc libpython2.7.a Modules/threadmodule.o  
Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o   
Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  
Modules/zipimport.o  Modules/symtablemodule.o  Modules/xxsubtype.o

ranlibpentium libpython2.7.a

ccpentium -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm 
-Wall -MD -MP 
-L/home/demo/WindRiver/vxworks-6.9/target/lib/usr/lib/pentium/PENTIUM4/common
 -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wall -MD 
-MP -non-static -o python
 Modules/python.o libpython2.7.a

libpython2.7.a(dynload_shlib.o): In function `_PyImport_GetDynLoadFunc':
/romfs/python273/Python/dynload_shlib.c:94: undefined reference to `dlsym'
/romfs/python273/Python/dynload_shlib.c:130: undefined reference to `dlopen'
/romfs/python273/Python/dynload_shlib.c:133: undefined reference to `dlerror'
/romfs/python273/Python/dynload_shlib.c:141: undefined reference to `dlsym'
collect2: ld returned 1 exit status
make: *** [python] Error 1






On Monday, February 17, 2014 12:29 AM, nachaat hassis <nachaat05 at yahoo.fr> wrote:
 
Hi Gary,
I was trying to port Python 3.0 at first .. after porting it, i noticed that Twisted and Buildbot support only Phyton 2.7
So i had to port Python 2.7 ..
I was working with VxWorks 6.9 as well (ARM)..
well, after trying to port the Python as a DKM, i had it working but i had many problems with Twisted and Buildbot .. (they use many signals, pipes, ..) many POSIX stuff .. 
So i took another direction and tried to make it run as an RTP.
With the RTP-Version i had a better result. POSIX stuff is working, my buildbot and twisted are working (with few changes since fork and exec are not supported for example) ..
But i made it work .. I have the feeling that the Python-Interpreter was more stable as a DKM but maybe its just an impression ..

I had the loadable shared objects part done aswell .. If you need help with this part, just ask .. 

and thanks for the replay.

Best regards ..



Gary Kenaley <gkenaley at yahoo.com> schrieb am 22:00 Samstag, 15.Februar 2014:
 
nachaat hassis <nachaat05 at ...> writes:

> 
> Hi Everybody, 
> 
> im trying to port Python on VxWorks or to find binarys for VxWorks.
> My search on the internet was not very successfull till now.
> In this
>
page http://docstore.mik.ua/orelly/other/python/0596001886_pythonian-chp-2-sect-2.html you
> can read:
> The site http://www.python.org/download/ provides links to binary
distributions for Macintosh,
> OS/2, Amiga, RISC OS, QNX, VxWorks, IBM AS/400, Sony PlayStation 2, and
Sharp Zaurus.
> >
> but i cant find any binarys for VxWorks there.
> Is it even possible to port Python on VxWorks? lets say 2.7 ..
> The version is not very important to me .. i just need someone to point me
to the right direction ..
> An old binarys of Python for VxWorks would do it..
> 
> Did someone hear that anybody ported Python on Vxworks?
> 
> My goal is to make BuildBot run on VxWorks ..
> 
> Best regards.
> 

Nachaat,

I am currently working on a Python 2.7
 for VxWorks 6.9.3 x86 user space.
The best resource is: http://wiki.osdev.org/Porting_Python
I am using User Space Real Time Processes (RTPs) because they have POSIX
PSE52+ functionality and full memory protection.
I had the basic 2.6.1 interpreter running this way, but it's fairly useless
without the loadable shared objects (the osdev site should help a lot with
those cross build aspects).

If I get it working satisfactorily, I will post back.

-Gary






_______________________________________________
capi-sig mailing list
capi-sig at python.org
https://mail.python.org/mailman/listinfo/capi-sig