Hello fellow co-developers!
Today I was in contact with a Python user who tried to compile pyprocessing - the ancestor of multiprocessing - on Solaris. It failed to run because Solaris is missing two features (HAVE_FD_TRANSFER and HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to test the settings? Neither Python 2.6 nor my backup have the correct settings for Solaris.
Christian
Today I was in contact with a Python user who tried to compile pyprocessing - the ancestor of multiprocessing - on Solaris. It failed to run because Solaris is missing two features (HAVE_FD_TRANSFER and HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to test the settings? Neither Python 2.6 nor my backup have the correct settings for Solaris.
I don't quite understand what it is that you want tested - what "settings"?
Most likely, the answer is yes, I can test stuff on Solaris (both SPARC and x86/amd64).
Regards, Martin
Martin v. Löwis schrieb:
Today I was in contact with a Python user who tried to compile pyprocessing - the ancestor of multiprocessing - on Solaris. It failed to run because Solaris is missing two features (HAVE_FD_TRANSFER and HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to test the settings? Neither Python 2.6 nor my backup have the correct settings for Solaris.
I don't quite understand what it is that you want tested - what "settings"?
Most likely, the answer is yes, I can test stuff on Solaris (both SPARC and x86/amd64).
According to the user's experience multiprocessing should not compile and run correctly unless this patch is applied. I'm not sure if the value "solaris" for platform is correct. You may also need to change libraries to ['rt'].
Index: setup.py =================================================================== --- setup.py (revision 70478) +++ setup.py (working copy) @@ -1280,6 +1280,14 @@ ) libraries = []
+ elif platform == 'solaris': + macros = dict( + HAVE_SEM_OPEN=1, + HAVE_SEM_TIMEDWAIT=0, + HAVE_FD_TRANSFER=0, + ) + libraries = [] + else: # Linux and other unices macros = dict( HAVE_SEM_OPEN=1,
On Fri, Mar 20, 2009 at 8:50 PM, Christian Heimes lists@cheimes.de wrote:
Martin v. Löwis schrieb:
Today I was in contact with a Python user who tried to compile pyprocessing - the ancestor of multiprocessing - on Solaris. It failed to run because Solaris is missing two features (HAVE_FD_TRANSFER and HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to test the settings? Neither Python 2.6 nor my backup have the correct settings for Solaris.
I don't quite understand what it is that you want tested - what "settings"?
Most likely, the answer is yes, I can test stuff on Solaris (both SPARC and x86/amd64).
According to the user's experience multiprocessing should not compile and run correctly unless this patch is applied. I'm not sure if the value "solaris" for platform is correct. You may also need to change libraries to ['rt'].
Index: setup.py
--- setup.py (revision 70478) +++ setup.py (working copy) @@ -1280,6 +1280,14 @@ ) libraries = []
- elif platform == 'solaris':
- macros = dict(
- HAVE_SEM_OPEN=1,
- HAVE_SEM_TIMEDWAIT=0,
- HAVE_FD_TRANSFER=0,
- )
- libraries = []
else: # Linux and other unices macros = dict( HAVE_SEM_OPEN=1,
If this should be addressed in trunk/3k, we need to track this in the tracker in the bug I cited in the other email. I can't speak for the original pyprocessing code.
-jesse
Jesse Noller wrote:
On Fri, Mar 20, 2009 at 8:50 PM, Christian Heimes lists@cheimes.de wrote:
Martin v. Löwis schrieb:
Today I was in contact with a Python user who tried to compile pyprocessing - the ancestor of multiprocessing - on Solaris. It failed to run because Solaris is missing two features (HAVE_FD_TRANSFER and HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to test the settings? Neither Python 2.6 nor my backup have the correct settings for Solaris.
I don't quite understand what it is that you want tested - what "settings"?
Most likely, the answer is yes, I can test stuff on Solaris (both SPARC and x86/amd64).
According to the user's experience multiprocessing should not compile and run correctly unless this patch is applied. I'm not sure if the value "solaris" for platform is correct. You may also need to change libraries to ['rt'].
Index: setup.py
--- setup.py (revision 70478) +++ setup.py (working copy) @@ -1280,6 +1280,14 @@ ) libraries = []
elif platform == 'solaris':macros = dict(HAVE_SEM_OPEN=1,HAVE_SEM_TIMEDWAIT=0,HAVE_FD_TRANSFER=0,)libraries = []else: # Linux and other unices macros = dict( HAVE_SEM_OPEN=1,If this should be addressed in trunk/3k, we need to track this in the tracker in the bug I cited in the other email. I can't speak for the original pyprocessing code.
I just checked out the trunk on a Sparc Solaris 8 box, and on the trunk, those defines are specified differently:
building '_multiprocessing' extension gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -DHAVE_SEM_OPEN=1 -DHAVE_FD_TRANSFER=1 -DHAVE_SEM_TIMEDWAIT=1 -IModules/_multiprocessing -I. -I./Include -I/usr/local/include -IInclude -I/nfs/nfs2/home/scratch/scodial/python-trunk -c trunk/Modules/_multiprocessing/multiprocessing.c -o build/temp.solaris-2.8-sun4u-2.7/trunk/Modules/_multiprocessing/multiprocessing.o
However, the build is still without issue:
trunk/Modules/_multiprocessing/multiprocessing.c: In function `multiprocessing_sendfd': trunk/Modules/_multiprocessing/multiprocessing.c:100: warning: implicit declaration of function `CMSG_SPACE' trunk/Modules/_multiprocessing/multiprocessing.c:117: warning: implicit declaration of function `CMSG_LEN'
trunk/Modules/_multiprocessing/connection.h: In function `connection_new': trunk/Modules/_multiprocessing/connection.h:51: warning: unknown conversion type character `z' in format trunk/Modules/_multiprocessing/connection.h:51: warning: too many arguments for format trunk/Modules/_multiprocessing/connection.h: In function `connection_repr': trunk/Modules/_multiprocessing/connection.h:401: warning: unknown conversion type character `z' in format
trunk/Modules/_multiprocessing/connection.h: In function `connection_new': trunk/Modules/_multiprocessing/connection.h:51: warning: unknown conversion type character `z' in format trunk/Modules/_multiprocessing/connection.h:51: warning: too many arguments for format trunk/Modules/_multiprocessing/connection.h: In function `connection_repr': trunk/Modules/_multiprocessing/connection.h:401: warning: unknown conversion type character `z' in format
According to the user's experience multiprocessing should not compile and run correctly unless this patch is applied.
Can this please be more qualified? I can confirm Scott's observation: for the trunk, it compiles just fine, using SunPro CC on Solaris 10, on SPARC. Also, test_multiprocessing passes.
IIUC, incorrect setting of HAVE_SEM_TIMEDWAIT to 1 should cause a compile failure, as sem_timedwait would be called but not defined. However, sem_timedwait *is* defined on Solaris.
Likewise, for HAVE_FD_TRANSFER=1 to work, SCM_RIGHTS must be defined (and implemented); it is defined in sys/socket.h, and documented in socket.h(3HEAD).
So there must be going on something else at the user's machine.
Regards, Martin
Martin v. Löwis schrieb:
According to the user's experience multiprocessing should not compile and run correctly unless this patch is applied.
Can this please be more qualified? I can confirm Scott's observation: for the trunk, it compiles just fine, using SunPro CC on Solaris 10, on SPARC. Also, test_multiprocessing passes.
IIUC, incorrect setting of HAVE_SEM_TIMEDWAIT to 1 should cause a compile failure, as sem_timedwait would be called but not defined. However, sem_timedwait *is* defined on Solaris.
Likewise, for HAVE_FD_TRANSFER=1 to work, SCM_RIGHTS must be defined (and implemented); it is defined in sys/socket.h, and documented in socket.h(3HEAD).
So there must be going on something else at the user's machine.
The user doesn't respond to my inquiries anymore. According to his initial message he is using Solaris 5.8 with GCC 3.4.6 on a Sun Fire machine. Here is a link to his mesage: http://permalink.gmane.org/gmane.comp.python.general/615802
Christian
Christian Heimes wrote:
Martin v. Löwis schrieb:
According to the user's experience multiprocessing should not compile and run correctly unless this patch is applied.
Can this please be more qualified? I can confirm Scott's observation: for the trunk, it compiles just fine, using SunPro CC on Solaris 10, on SPARC. Also, test_multiprocessing passes.
Sorry, I mistakenly said "without issue" and then copied the issues below. I meant to say "not without issues." _multiprocessing does *not* build on Solaris 8.
IIUC, incorrect setting of HAVE_SEM_TIMEDWAIT to 1 should cause a compile failure, as sem_timedwait would be called but not defined. However, sem_timedwait *is* defined on Solaris.
Likewise, for HAVE_FD_TRANSFER=1 to work, SCM_RIGHTS must be defined (and implemented); it is defined in sys/socket.h, and documented in socket.h(3HEAD).
So there must be going on something else at the user's machine.
The user doesn't respond to my inquiries anymore. According to his initial message he is using Solaris 5.8 with GCC 3.4.6 on a Sun Fire machine. Here is a link to his mesage: http://permalink.gmane.org/gmane.comp.python.general/615802
I can confirm his build issues on:
$ uname -srvmpi SunOS 5.8 Generic_117350-51 sun4u sparc SUNW,Sun-Fire-280R $ gcc -v Reading specs from /usr/local/gnu/lib/gcc-lib/sparc-sun-solaris2.8/2.95.2/specs gcc version 2.95.2 19991024 (release)
My build output from within the trunk is the same as his modulo the details of being part of a trunk build instead.
Is Solaris 8 really a supported platform? If so, I can investigate the changes he suggested.
Sorry, I mistakenly said "without issue" and then copied the issues below. I meant to say "not without issues." _multiprocessing does *not* build on Solaris 8.
Hmm. They are all warnings - did you omit the actual error message?
The lack of CMSG_LEN seems to suggest that control messages are not supported on Solaris 8, or that you need to include an additional header file to get them.
Is Solaris 8 really a supported platform? If so, I can investigate the changes he suggested.
The concept of "supported platform" doesn't really exist - there is no way to obtain "support". If it works, it works, if it doesn't, it doesn't. So if you want it fixed, provide a patch - else we can drop the issue.
I think multiprocessing is misguided in hard-coding these settings into setup.py. As we can see, the necessary features are available on some versions of Solaris, but not on others. It would be better if autoconf tests were written, and the entire configuration removed from setup.py.
Regards, Martin
Martin v. Löwis wrote:
Sorry, I mistakenly said "without issue" and then copied the issues below. I meant to say "not without issues." _multiprocessing does *not* build on Solaris 8.
Hmm. They are all warnings - did you omit the actual error message?
The lack of CMSG_LEN seems to suggest that control messages are not supported on Solaris 8, or that you need to include an additional header file to get them.
Sorry, I was trying to keep the amount of noise to a minimum, but those particular implicit function declarations ultimately lead to linking errors for lack of those same symbols:
*** WARNING: renaming "_multiprocessing" since importing it failed: ld.so.1: python: fatal: relocation error: file build/lib.solaris-2.8-sun4u-2.7/_multiprocessing.so: symbol CMSG_SPACE: referenced symbol not found
Is Solaris 8 really a supported platform? If so, I can investigate the changes he suggested.
The concept of "supported platform" doesn't really exist - there is no way to obtain "support". If it works, it works, if it doesn't, it doesn't. So if you want it fixed, provide a patch - else we can drop the issue.
I have no personal interest in the matter. I just happened to have access to an older Solaris just for this sort of testing. If someone has a patch, then I would be glad to test it, but otherwise, I am not going to invest any time on this. The patch given by the OP is clearly undesirable since status quo works just fine for a modern Solaris install.
I think multiprocessing is misguided in hard-coding these settings into setup.py. As we can see, the necessary features are available on some versions of Solaris, but not on others. It would be better if autoconf tests were written, and the entire configuration removed from setup.py.
I agree, but I am not familiar enough with the set of features that multiprocessing is implicitly depending on and how to test for them on all of the platforms.
Martin v. Löwis wrote:
I think multiprocessing is misguided in hard-coding these settings into setup.py. As we can see, the necessary features are available on some versions of Solaris, but not on others. It would be better if autoconf tests were written, and the entire configuration removed from setup.py.
I guess multiprocessing doesn't use autoconf tests for historical reasons. It's ancestor -- the pyprocessing package -- was using hard coded values, too.
I've started with a autoconf file for the multiprocessing backport package. The tests should produce the same set of definitions but I'm not able to test it on different platforms expect Linux. http://python-multiprocessing.googlecode.com/svn/trunk/configure.ac
Christian
On Mon, Mar 23, 2009 at 10:31 AM, Christian Heimes lists@cheimes.de wrote:
Martin v. Löwis wrote:
I think multiprocessing is misguided in hard-coding these settings into setup.py. As we can see, the necessary features are available on some versions of Solaris, but not on others. It would be better if autoconf tests were written, and the entire configuration removed from setup.py.
I guess multiprocessing doesn't use autoconf tests for historical reasons. It's ancestor -- the pyprocessing package -- was using hard coded values, too.
I've started with a autoconf file for the multiprocessing backport package. The tests should produce the same set of definitions but I'm not able to test it on different platforms expect Linux. http://python-multiprocessing.googlecode.com/svn/trunk/configure.ac
Christian
Christian is correct, it's purely for historical reasons. I have no particular stake in keeping it as is other than the fact that there are higher priorities in the bug queue (and improving the tests) than the build functionality.
Christian - I would appreciate it if we could coordinate/track this on the tracker too - I had no idea you were doing this in the back port, and I don't think we want the two code bases to diverge that much.
This is definitely something which could be hacked on during the sprints though. There's been a lot of movement in the *BSD space around defining these values and providing the faculties needed for mp to work.
-jesse
Jesse Noller wrote:
Christian - I would appreciate it if we could coordinate/track this on the tracker too - I had no idea you were doing this in the back port, and I don't think we want the two code bases to diverge that much.
You could not have known about the idea because I started the experiment about an hour ago. ;) I needed a playground to test and save the files of the experiment. The multiprocessing backport sounded like a good place.
Christian
I've started with a autoconf file for the multiprocessing backport package. The tests should produce the same set of definitions but I'm not able to test it on different platforms expect Linux. http://python-multiprocessing.googlecode.com/svn/trunk/configure.ac
If you want to see how it fares on the various systems which we have build slaves for, feel free to create a branch, and then manually ask the slaves to build branches/yourbranch.
Or, just commit it into the trunk, and see how it does.
Regards, Martin
Known issue:
http://bugs.python.org/issue3110
I haven't had time to look into it, I was planning on working on many of the mp bugs during the sprint at pycon.
On Fri, Mar 20, 2009 at 8:18 PM, Christian Heimes lists@cheimes.de wrote:
Hello fellow co-developers!
Today I was in contact with a Python user who tried to compile pyprocessing - the ancestor of multiprocessing - on Solaris. It failed to run because Solaris is missing two features (HAVE_FD_TRANSFER and HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to test the settings? Neither Python 2.6 nor my backup have the correct settings for Solaris.
Christian
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
Jesse> Known issue:
Jesse> http://bugs.python.org/issue3110
Jesse> I haven't had time to look into it, I was planning on working on Jesse> many of the mp bugs during the sprint at pycon.
Jesse, I will be at the sprints for a couple days and should be able to test things out on Solaris or let you look over my shoulder as we poke around the machines at work if you need.
Skip
On Fri, Mar 20, 2009 at 9:51 PM, skip@pobox.com wrote:
Jesse> Known issue:
Jesse> http://bugs.python.org/issue3110
Jesse> I haven't had time to look into it, I was planning on working on Jesse> many of the mp bugs during the sprint at pycon.
Jesse, I will be at the sprints for a couple days and should be able to test things out on Solaris or let you look over my shoulder as we poke around the machines at work if you need.
Skip
Sweet, do you think a 64 bit Opensolaris VM would work too?
participants (5)
-
"Martin v. Löwis" -
Christian Heimes -
Jesse Noller -
Scott Dial -
skip＠pobox.com