-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22/11/10 20:12, Brett Cannon wrote:

Are you asking about buildbots only or as a general policy? If you are asking about the buildbots then I definitely think we should use 64 bits. If you are asking about policy I would say it should be an option in case people are using C extensions that are not designed to work with 64 bits.

The point is that building python in 64 bits under Solaris (family) is not easy, because the 64 bits libraries (zlib, openssl, berkeley db, curses, etc., etc., etc) are not is "/usr/lib", "/usr/local/lib", etc., but "/usr/lib/64", "/usr/local/lib/64", etc. Solaris overcomes most of the issue having separate library searchpath in 32 and 64 bits (via the "crle" command). But in some cases python try to find some library in "/usr/local/lib", and my point is that it should search TOO inside "/usr/local/lib/64". - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTOrD8Jlgi5GaxT1NAQJhRQP/dd4q70eXsq5AUFrleqUx3A+AagChpCcp UDHAomaX26cMl0tLFwLOd4SaKizzRMvjdTJc3GhZDIqYrF3QuqZAyLPjr5tyogP8 /4KPM73l5L2cb7IdHdSHpruwMh8f2WJ4S6+ig8DzOj6qBcttXKMymrV/skum4ENJ yb4mbpH9q/0= =Oe2G -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22/11/10 20:42, "Martin v. Löwis" wrote:

Before enabling anything on a build slave, a patch needs to be contributed to make it work in the first place.

I actually agree. I am not sure yet, but I am thinking that adding a "--build-64" parameter to "configure" could be an option under Solaris. Most OSs (let say, Linux) force you to choose 32/64 bits at install time, but Solaris can use both at the same time, and compilers allow to compile both (using -m32 or -m64). Since choosing 32 or 64 bits when compiling python under Solaris change the requirement, paths, etc., automating it should be a goal. PS: Martin, is there any reason to restrict the solaris 10 buildslaves to 32 bits, beside the said problems?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTOrhKZlgi5GaxT1NAQI0cAP+OUFGVDd7UV6MdHzMenBn8fO3h4M1n0dR UZrVyYJhUYvEX9p7MRBdDNFY/6LrUITb3WCVegD3PuOymQP16GgksRfIA/jGDXyl Fe+Ed5amlDgdVPeVVH/55OodrO4SuOrJZ846G6GB1wav2IjR7I9YGxZQ6PA0LR7l 4Iph6HfcMlw= =hTNy -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22/11/10 23:05, "Martin v. Löwis" wrote:

...

PS: Martin, is there any reason to restrict the solaris 10 buildslaves to 32 bits, beside the said problems?.

I don't see that as a restriction. I have to make a choice, and there are sooo many choices to make: - gcc vs. SunPRO - 32-bit vs. 64-bit - GNU make vs. /usr/ccs/bin/make

I picked the combination which was most easy to setup, and is therefore likely to be used by most users (except for those who think 64-bit is somehow "better" than 32-bit, when it is actually the other way 'round - IMO).

Do not think this is a personal attack. Not at all. I am deploying 32 and 64 bits buildslaves (in the same machine) and feeling the pain. You are far more experiences than me with buildbots and python. I want to know if I am missing something.

As for configuration, I personally prefer that setting CC indicates what type of build you want. Set CC to "gcc -m64" to indicate a 64-build. Ideally, you will *not* have to adjust library paths, since the other compiler will know on its own where to search things.

The problem is not with system library paths. Compilers overcome that. The problem is with things like "/usr/local/lib" and hardcoded library paths in Python. For example, checking <http://www.python.org/dev/buildbot/all/builders/AMD64%20OpenIndiana%203.x/builds/8/steps/test/logs/stdio>: """ gcc -shared -m64 build/temp.solaris-2.11-i86pc-3.2-pydebug/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/readline.o - -L/usr/lib/termcap -L/usr/local/lib -lreadline -lncursesw -o build/lib.solaris-2.11-i86pc-3.2-pydebug/readline.so ld: fatal: file /usr/local/lib/libncursesw.so: wrong ELF class: ELFCLASS32 ld: fatal: file processing errors. No output written to build/lib.solaris-2.11-i86pc-3.2-pydebug/readline.so collect2: ld returned 1 exit status """ The "-L/usr/local/lib" should be "-L/usr/local/lib/64". An example of many. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTOr/n5lgi5GaxT1NAQLzogP/Sb2VMe7UwK/YeB8/cQSxhuoKeNRre0pZ XCJDePusysqI3uXBHmH8vitEIILmUKd5kQ6vsFwErPIry7ikl2fbDHe7eQgNr2HK o5Xcul36bqtuKWGkDV+gIyBH/m9k4pkvc7Lfp3mvR7yiYTBB75V/azt64XSTC9si 7QjjetX5wnA= =NCtE -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 23/11/10 01:05, "Martin v. Löwis" wrote:

No offense taken. If you really want to know the historical background: this was the very first build slave (before I actually announced it to python-dev), and I haven't changed much from the initial setup.

I do really want to know. I love trivia :-). Thanks.

I just point out that none of the binaries in /usr/bin is a 64-bit binary; this includes the Sun-provided /usr/sfw/bin/python

...

The "-L/usr/local/lib" should be "-L/usr/local/lib/64". An example of many.

Is that really the case? I.e. will ncurses automatically install into /usr/local/lib/64 if built with a 64-bit compiler? My installation doesn't even have a /usr/local/lib/64 folder.

A fresh Solaris 10 install doesn't even have a "/usr/local" directory :). Sadly today most Open Source code is written like if Linux were the only Unix system out there. I was amazed that OpenSSL 1.0 installs automatically in "/usr/local/ssl/lib" when compiled in 32 bits, and in "/usr/local/ssl/lib/64" when compiled in 64 bits. I almost cry.

In any case: this shouldn't need a configure option. Instead, Python can find out itself whether it's a 64-bit build, and make modifications it considers necessary.

I agree. Python should detect it automatically and update the paths when compiling. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTOsPBZlgi5GaxT1NAQIw+QP/ZuxpWo2WZYUUcDfARRnOtp60n4PbIGMf fqQ4ZnC9JnelzKDU9kBo0yReL2zYAw0ZwezsGwZ98M9i3XyKkFCtcJcM1vXpIsDL eBwga8kPDpab5loP/vuac5kVC0wn0Z0z8x+BRMW6mwoOMHJzd463E8GTQywdx3x1 06FUHwJ0Hv4= =PV43 -----END PGP SIGNATURE-----

But if we say the Python can be compiled as 64 bits under Solaris, would be nice if that was actually true. Now that we have a buildbot (under OpenIndiana) to test, it is doable.

But it is true, and always has been true. The lib/64 issue did not prevent one building Python on Solaris/SPARC64 at all, including the extension modules. Just edit Modules/Setup to suit your needs - that works since 1995 (before distutils was even written).

If not, we could say that Solaris+64 bits is unsupported. I don't think we should go that way. Solaris+64 bits should be a full citizen.

There we go again: "supported". Python builds on many systems which we don't have buildbots for, including obscure systems (although Guido has ruled that we won't specifically accept code for obscure systems anymore, unlike we did before). It is never fully automatic (you always have to at least make sure manually that the dependencies are installed). Regards, Martin

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 23/11/10 13:19, Jesus Cea wrote:

Would be acceptable to change something like:

""" add_library_path("/usr/local/lib") """

to something similar to:

""" if (platform.uname()=="SunOS") and (platform.architecture()[0]=="64bits") : add_library_path("/usr/local/lib/64") else : add_library_path("/usr/local/lib") """

python-dev would consider that change OK?.

Resurrecting an old thread. I think that, under SunOS and 64 bits, python should install the lib directory under "/usr/local/lib/64", following the convention in the platform. In fact, under OpenIndiana 147, I see this: """ - -bash-4.0$ type python python is /usr/bin/python - -bash-4.0$ ldd /usr/bin/python libpython2.6.so.1.0 => /usr/lib/libpython2.6.so.1.0 libc.so.1 => /usr/lib/libc.so.1 libdl.so.1 => /lib/libdl.so.1 libm.so.2 => /lib/libm.so.2 - -bash-4.0$ file /usr/bin/python /usr/bin/python: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, not stripped, no debugging information available - -bash-4.0$ ls -lA /usr/lib/|grep python lrwxrwxrwx 1 root root 19 Oct 1 10:55 libpython2.6.so -> libpython2.6.so.1.0 - -rwxr-xr-x 1 root bin 1788896 Oct 1 10:55 libpython2.6.so.1.0 lrwxrwxrwx 1 root root 22 Oct 1 10:55 libpython2.6_db.so -> libpython2.6_db.so.1.0 - -rwxr-xr-x 1 root bin 10372 Oct 1 10:55 libpython2.6_db.so.1.0 drwxr-xr-x 25 root bin 432 Oct 1 10:55 python2.6 - -bash-4.0$ ls -lA /usr/lib/64/|grep python lrwxrwxrwx 1 root root 19 Oct 1 10:55 libpython2.6.so -> libpython2.6.so.1.0 - -rwxr-xr-x 1 root bin 2217504 Oct 1 10:55 libpython2.6.so.1.0 lrwxrwxrwx 1 root root 22 Oct 1 10:55 libpython2.6_db.so -> libpython2.6_db.so.1.0 - -rwxr-xr-x 1 root bin 13336 Oct 1 10:55 libpython2.6_db.so.1.0 - -bash-4.0$ file /usr/lib/64/libpython2.6.so.1.0 /usr/lib/64/libpython2.6.so.1.0: ELF 64-bit LSB dynamic lib AMD64 Version 1 [SSE2 SSE CMOV], dynamically linked, not stripped, no debugging information available """ - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTYIGSJlgi5GaxT1NAQIrGAP/fBcLKIdFN+O8jNArCs01m30pQC5rNAhp hs1mTqlllpGkSQLDeQvcN/ctwFpR8a+mlC4yCFwYFB8CpB9TjilIKRn0RlerfOlw U0OuhezPTGc75NY95a+x8iZG4FQYytMkv7ClPmyDjG/g68CTS/zArRoKGRA7trZk +WbsLPK4lhg= =ScvG -----END PGP SIGNATURE-----

On Tue, 23 Nov 2010 00:48:06 +0100 Jesus Cea <jcea@jcea.es> wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

I think this is probably trivial, but is there any foolproof way to detect 64 bit builds in python, beside "sys.maxint"?.

sys.maxsize

And any macro useable for conditional compilation in C?.

SIZEOF_VOID_P > 4

Am 23.11.2010 00:48, schrieb Jesus Cea:

I think this is probably trivial, but is there any foolproof way to detect 64 bit builds in python, beside "sys.maxint"?.

The canonical way is to use platform.architecture().

And any macro useable for conditional compilation in C?.

You need to be more specific than that. There are perhaps ten independent properties you may query, depending on what precise problem you try to solve. Most likely, you are looking for SIZEOF_VOID_P (but don't use that unless you literally want to know how many bytes a pointer uses, or whether it uses 4 or 8 bytes). Regards, Martin