So...I've been looking stuff up. Don't have too much free time right now, but it was mostly Googleing.
I have deemed three ideas to be best for Python:
- Write a custom makefile generator. I have some ideas, but they likely won't come to light any time soon.
- Use Boost.Build. It seems ugly and complicated, but it actually has a full configuring system that can detect libraries and try to compile source files. It's written in C89, not C++, and its dependency is the Boehm GC, which probably supports everywhere that Python runs on.
Only issue? It's undergoing a rewrite in Python. However, at the rate it's going, Python will likely be at version 7 by the time they finish.
- Stick with autotools. Trying to avoid this.
Thoughts? Again?
Scons? http://www.scons.org/doc/production/HTML/scons-user.html On Mar 13, 2015 1:44 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
So...I've been looking stuff up. Don't have too much free time right now, but it was mostly Googleing.
I have deemed three ideas to be best for Python:
- Write a custom makefile generator. I have some ideas, but they likely
won't come to light any time soon.
- Use Boost.Build. It seems ugly and complicated, but it actually has a
full configuring system that can detect libraries and try to compile source files. It's written in C89, not C++, and its dependency is the Boehm GC, which probably supports everywhere that Python runs on.
Only issue? It's undergoing a rewrite in Python. However, at the rate it's going, Python will likely be at version 7 by the time they finish.
- Stick with autotools. Trying to avoid this.
Thoughts? Again?
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
That written in Python, so Python would require Python to build which would require Python to build which would require...
Besides, SCons sucks. If I were using a build system written in Python, I'd use fbuild or Waf.
On Fri, Mar 13, 2015 at 2:27 PM, Wes Turner wes.turner@gmail.com wrote:
Scons? http://www.scons.org/doc/production/HTML/scons-user.html On Mar 13, 2015 1:44 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
So...I've been looking stuff up. Don't have too much free time right now, but it was mostly Googleing.
I have deemed three ideas to be best for Python:
- Write a custom makefile generator. I have some ideas, but they likely
won't come to light any time soon.
- Use Boost.Build. It seems ugly and complicated, but it actually has a
full configuring system that can detect libraries and try to compile source files. It's written in C89, not C++, and its dependency is the Boehm GC, which probably supports everywhere that Python runs on.
Only issue? It's undergoing a rewrite in Python. However, at the rate it's going, Python will likely be at version 7 by the time they finish.
- Stick with autotools. Trying to avoid this.
Thoughts? Again?
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
The subject line mentions "Part 2". Is there a link to "Part 1"? What is the rationale for changing?
Make, autoconf, and automake are already widely ported/portable.
What are the criteria for evaluating alternatives? On Mar 13, 2015 2:41 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
Besides, SCons sucks. If I were using a build system written in Python, I'd use fbuild or Waf.
On Fri, Mar 13, 2015 at 2:27 PM, Wes Turner wes.turner@gmail.com wrote:
Scons? http://www.scons.org/doc/production/HTML/scons-user.html On Mar 13, 2015 1:44 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
So...I've been looking stuff up. Don't have too much free time right now, but it was mostly Googleing.
I have deemed three ideas to be best for Python:
- Write a custom makefile generator. I have some ideas, but they likely
won't come to light any time soon.
- Use Boost.Build. It seems ugly and complicated, but it actually has a
full configuring system that can detect libraries and try to compile source files. It's written in C89, not C++, and its dependency is the Boehm GC, which probably supports everywhere that Python runs on.
Only issue? It's undergoing a rewrite in Python. However, at the rate it's going, Python will likely be at version 7 by the time they finish.
- Stick with autotools. Trying to avoid this.
Thoughts? Again?
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
https://mail.python.org/pipermail/python-ideas/2015-March/032381.html
On Fri, Mar 13, 2015 at 2:46 PM, Wes Turner wes.turner@gmail.com wrote:
The subject line mentions "Part 2". Is there a link to "Part 1"? What is the rationale for changing?
Make, autoconf, and automake are already widely ported/portable.
What are the criteria for evaluating alternatives? On Mar 13, 2015 2:41 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
Besides, SCons sucks. If I were using a build system written in Python, I'd use fbuild or Waf.
On Fri, Mar 13, 2015 at 2:27 PM, Wes Turner wes.turner@gmail.com wrote:
Scons? http://www.scons.org/doc/production/HTML/scons-user.html On Mar 13, 2015 1:44 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
So...I've been looking stuff up. Don't have too much free time right now, but it was mostly Googleing.
I have deemed three ideas to be best for Python:
- Write a custom makefile generator. I have some ideas, but they likely
won't come to light any time soon.
- Use Boost.Build. It seems ugly and complicated, but it actually has a
full configuring system that can detect libraries and try to compile source files. It's written in C89, not C++, and its dependency is the Boehm GC, which probably supports everywhere that Python runs on.
Only issue? It's undergoing a rewrite in Python. However, at the rate it's going, Python will likely be at version 7 by the time they finish.
- Stick with autotools. Trying to avoid this.
Thoughts? Again?
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
On Fri, Mar 13, 2015 at 10:46 PM, Wes Turner wes.turner@gmail.com wrote:
The subject line mentions "Part 2". Is there a link to "Part 1"? What is the rationale for changing?
Make, autoconf, and automake are already widely ported/portable.
"Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die." (c) "Autotools must die", ESR. http://esr.ibiblio.org/?p=1877
I couldn't sum up this better. It looks even worse from Windows machine.
What are the criteria for evaluating alternatives?
Python 3 support and beautiful output to console? =)
On Mon, Mar 16, 2015 at 6:24 AM, anatoly techtonik techtonik@gmail.com wrote:
"Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die." (c) "Autotools must die", ESR. http://esr.ibiblio.org/?p=1877
I couldn't sum up this better. It looks even worse from Windows machine.
Is it even used for building Python on Windows (for the "normal" python.org builds)?
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
-CHB
[1] http://www.goodreads.com/quotes/267224-democracy-is-the-worst-form-of-govern...
On Mon, Mar 16, 2015 at 11:31 AM, Chris Barker chris.barker@noaa.gov wrote:
On Mon, Mar 16, 2015 at 6:24 AM, anatoly techtonik techtonik@gmail.com wrote:
"Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die." (c) "Autotools must die", ESR. http://esr.ibiblio.org/?p=1877
I couldn't sum up this better. It looks even worse from Windows machine.
Is it even used for building Python on Windows (for the "normal" python.org builds)?
No. Some random batch file is used. That's all I know.
However, when building without MSVC/VS2008, you need MinGW, which means you'll end up using autotools.
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
-CHB
[1] http://www.goodreads.com/quotes/267224-democracy-is-the-worst-form-of-govern...
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Mar 16, 2015 at 11:33 PM, Ryan Gonzalez rymg19@gmail.com wrote:
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
Is there anything Javascriptey? Python needs to move to web. I heard something about Grunt, which was replaced by Gulp, but no idea if they really mature enough to include "system features detection" and "user configured builds" (or work with C/C++ compilers at all).
* anatoly techtonik techtonik@gmail.com [2015-03-18 10:21:46 +0300]:
On Mon, Mar 16, 2015 at 11:33 PM, Ryan Gonzalez rymg19@gmail.com wrote:
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
Is there anything Javascriptey? Python needs to move to web. I heard something about Grunt, which was replaced by Gulp, but no idea if they really mature enough to include "system features detection" and "user configured builds" (or work with C/C++ compilers at all).
What does "Python needs to move to web" to do with it's build system being "Javascriptey"?
Florian
On Wed, Mar 18, 2015 at 6:21 PM, Florian Bruhin me@the-compiler.org wrote:
- anatoly techtonik techtonik@gmail.com [2015-03-18 10:21:46 +0300]:
On Mon, Mar 16, 2015 at 11:33 PM, Ryan Gonzalez rymg19@gmail.com wrote:
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
Is there anything Javascriptey? Python needs to move to web. I heard something about Grunt, which was replaced by Gulp, but no idea if they really mature enough to include "system features detection" and "user configured builds" (or work with C/C++ compilers at all).
What does "Python needs to move to web" to do with it's build system being "Javascriptey"?
Because things like Twisted are de-facto standard in JavaScript world (Node.js), and Python will benefit more if people look for how things are implemented in non-C worlds.
On Mar 19, 2015, at 4:12 AM, anatoly techtonik techtonik@gmail.com wrote:
On Wed, Mar 18, 2015 at 6:21 PM, Florian Bruhin me@the-compiler.org wrote:
- anatoly techtonik techtonik@gmail.com [2015-03-18 10:21:46 +0300]:
On Mon, Mar 16, 2015 at 11:33 PM, Ryan Gonzalez rymg19@gmail.com wrote:
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
Is there anything Javascriptey? Python needs to move to web. I heard something about Grunt, which was replaced by Gulp, but no idea if they really mature enough to include "system features detection" and "user configured builds" (or work with C/C++ compilers at all).
What does "Python needs to move to web" to do with it's build system being "Javascriptey"?
Because things like Twisted are de-facto standard in JavaScript world (Node.js), and Python will benefit more if people look for how things are implemented in non-C worlds.
Is this a sarcastic comment?
Or are you seriously suggesting that because people use things like the stdlib's asyncio, gevent, Tornado, etc. in Python, which have different paradigms than Twisted, the one Node copied, we need to force some other part of Python to be more like Node in some other way, because "move to the web" means "be more like the latest (but nowhere near most popular) framework for building web services"?
Also, correct me if I'm wrong, but isn't Node implemented in C, with a build system in C and Python? So if we really want to be more like Node for some reason, we should make the build system Pythony instead of Javascripty?
On Thu, Mar 19, 2015 at 1:48 PM, Andrew Barnert < abarnert@yahoo.com.dmarc.invalid> wrote:
On Mar 19, 2015, at 4:12 AM, anatoly techtonik techtonik@gmail.com wrote:
On Wed, Mar 18, 2015 at 6:21 PM, Florian Bruhin me@the-compiler.org
wrote:
- anatoly techtonik techtonik@gmail.com [2015-03-18 10:21:46 +0300]:
On Mon, Mar 16, 2015 at 11:33 PM, Ryan Gonzalez rymg19@gmail.com
wrote:
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
Is there anything Javascriptey? Python needs to move to web. I heard something about Grunt, which was replaced by Gulp, but no idea if they really mature enough to include "system features detection" and "user configured builds" (or work with C/C++ compilers at all).
What does "Python needs to move to web" to do with it's build system being "Javascriptey"?
Because things like Twisted are de-facto standard in JavaScript world (Node.js), and Python will benefit more if people look for how things are implemented in non-C worlds.
Is this a sarcastic comment?
Or are you seriously suggesting that because people use things like the stdlib's asyncio, gevent, Tornado, etc. in Python, which have different paradigms than Twisted, the one Node copied, we need to force some other part of Python to be more like Node in some other way, because "move to the web" means "be more like the latest (but nowhere near most popular) framework for building web services"?
Also, correct me if I'm wrong, but isn't Node implemented in C, with a build system in C and Python? So if we really want to be more like Node for some reason, we should make the build system Pythony instead of Javascripty?
You're not wrong. Node uses Gyp, although, if Chromium switches to GN (written in C++), Node might switch, too.
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Then you need Node.js installed, which is kind of big.
On Wed, Mar 18, 2015 at 2:21 AM, anatoly techtonik techtonik@gmail.com wrote:
On Mon, Mar 16, 2015 at 11:33 PM, Ryan Gonzalez rymg19@gmail.com wrote:
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
Fbuild, Waf, Meson, GN, Boost.Build, Tundra, Bam, ...
Is there anything Javascriptey? Python needs to move to web. I heard something about Grunt, which was replaced by Gulp, but no idea if they really mature enough to include "system features detection" and "user configured builds" (or work with C/C++ compilers at all).
On Mon, Mar 16, 2015 at 7:31 PM, Chris Barker chris.barker@noaa.gov wrote:
On Mon, Mar 16, 2015 at 6:24 AM, anatoly techtonik techtonik@gmail.com wrote:
"Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die." (c) "Autotools must die", ESR. http://esr.ibiblio.org/?p=1877
I couldn't sum up this better. It looks even worse from Windows machine.
Is it even used for building Python on Windows (for the "normal" python.org builds)?
There was some success with MinGW toolchain, but I am not tracking if it was integrated into official source. The FUD was that compiled extensions are not compatible between MSVC builds and MinGW builds, but I could not confirm that. If I understand correctly, the incompatibility is only for C++ programs, because of strange SEH strategy and proprietary name mangling.
But to miss-paraphrase Winston Churchill [1]:
"Autotools is the worst build system except for all the others."
Is there really something better out there?
A lot of attempts are made, but reviewing them requires a lot of time and effort.
On Mar 18, 2015 8:14 AM, "anatoly techtonik" techtonik@gmail.com wrote:
On Mon, Mar 16, 2015 at 7:31 PM, Chris Barker chris.barker@noaa.gov
wrote:
On Mon, Mar 16, 2015 at 6:24 AM, anatoly techtonik techtonik@gmail.com wrote:
"Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die." (c) "Autotools must die", ESR. http://esr.ibiblio.org/?p=1877
I couldn't sum up this better. It looks even worse from Windows
machine.
Is it even used for building Python on Windows (for the "normal"
python.org
builds)?
There was some success with MinGW toolchain, but I am not tracking if it
was
integrated into official source. The FUD was that compiled extensions are
not
compatible between MSVC builds and MinGW builds, but I could not confirm that. If I understand correctly, the incompatibility is only for C++
programs,
because of strange SEH strategy and proprietary name mangling.
This isn't FUD, alas. mingw in almost all of its variants uses effectively a different "libc" than MSVC, so code compiled with two compilers can't even agree on the meaning of file numbers. It is mostly possible to set up a mingw-w64 build that produces MSVC-compatible extensions, but it requires using a configuration that's undocumented and untested upstream (I.e. you will run into mingw bugs). NumPy *may* switch to this at some point -- we've been working on it for like a year now -- but it's extremely non-trivial.
-n
And unfortunately, a lot of that work may need to be redone for 3.5 and VC14... Sorry about that, but once the other compilers have caught up with the new CRT interface then it will (should) all be better in the long run.
Cheers, Steve
Top-posted from my Windows Phone ________________________________ From: Nathaniel Smithmailto:njs@pobox.com Sent: 3/18/2015 10:31 To: anatoly techtonikmailto:techtonik@gmail.com Cc: python-ideas@python.orgmailto:python-ideas@python.org Subject: Re: [Python-ideas] Rewriting the build system (part 2)
On Mar 18, 2015 8:14 AM, "anatoly techtonik" <techtonik@gmail.commailto:techtonik@gmail.com> wrote:
On Mon, Mar 16, 2015 at 7:31 PM, Chris Barker <chris.barker@noaa.govmailto:chris.barker@noaa.gov> wrote:
On Mon, Mar 16, 2015 at 6:24 AM, anatoly techtonik <techtonik@gmail.commailto:techtonik@gmail.com> wrote:
"Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die." (c) "Autotools must die", ESR. http://esr.ibiblio.org/?p=1877
I couldn't sum up this better. It looks even worse from Windows machine.
Is it even used for building Python on Windows (for the "normal" python.orghttp://python.org builds)?
There was some success with MinGW toolchain, but I am not tracking if it was integrated into official source. The FUD was that compiled extensions are not compatible between MSVC builds and MinGW builds, but I could not confirm that. If I understand correctly, the incompatibility is only for C++ programs, because of strange SEH strategy and proprietary name mangling.
This isn't FUD, alas. mingw in almost all of its variants uses effectively a different "libc" than MSVC, so code compiled with two compilers can't even agree on the meaning of file numbers. It is mostly possible to set up a mingw-w64 build that produces MSVC-compatible extensions, but it requires using a configuration that's undocumented and untested upstream (I.e. you will run into mingw bugs). NumPy *may* switch to this at some point -- we've been working on it for like a year now -- but it's extremely non-trivial.
-n
On Fri, Mar 13, 2015 at 12:40 PM, Ryan Gonzalez rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
Well, you can't buid a C compiler without a C compiler either (or SOME compiler). And given that you should be able to do cross-compilation for platforms that don't currently have a python build, I'm not sure that's a killer.
It it so wrong that python eats its own dog food in this manner?
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
On Fri, Mar 13, 2015 at 2:47 PM, Chris Barker chris.barker@noaa.gov wrote:
On Fri, Mar 13, 2015 at 12:40 PM, Ryan Gonzalez rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
Well, you can't buid a C compiler without a C compiler either (or SOME compiler). And given that you should be able to do cross-compilation for platforms that don't currently have a python build, I'm not sure that's a killer.
It it so wrong that python eats its own dog food in this manner?
It's not as much "Python eat its own dog food" as "cross-compiling natively would be even harder than it is right now."
C compilers are available on virtually every UNIX that's ever been created since the 70's, including AIX, Plan 9, Solaris, etc.
Python? Not as much.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
On 14 Mar 2015 09:44, "Chris Barker" chris.barker@noaa.gov wrote:
On Fri, Mar 13, 2015 at 12:40 PM, Ryan Gonzalez rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which
would require Python to build which would require...
Well, you can't buid a C compiler without a C compiler either (or SOME
compiler). And given that you should be able to do cross-compilation for platforms that don't currently have a python build, I'm not sure that's a killer.
It it so wrong that python eats its own dog food in this manner?
Wrong, no. harder yes. C compilers have mature cross compiler build chains including stage aware test suites and the like. When a tool is it's own build dependency bootstrapping it on a new platform gains a hard requirement on cross compiling rather than it being a thing for special cases like platforms that cannot compile the tool. This makes the life of distribution maintainers harder too, since cross arch platform bootstrapping becomes even more manual.
Rob
On Fri, Mar 13, 2015, at 20:07, Robert Collins wrote:
Wrong, no. harder yes. C compilers have mature cross compiler build chains including stage aware test suites and the like. When a tool is it's own build dependency bootstrapping it on a new platform gains a hard requirement on cross compiling rather than it being a thing for special cases like platforms that cannot compile the tool.
That's what bootstrapping is. If you're not doing this, you're not bootstrapping.
On Sat, 14 Mar 2015 13:07:57 +1300 Robert Collins robertc@robertcollins.net wrote:
On 14 Mar 2015 09:44, "Chris Barker" chris.barker@noaa.gov wrote:
On Fri, Mar 13, 2015 at 12:40 PM, Ryan Gonzalez rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which
would require Python to build which would require...
Well, you can't buid a C compiler without a C compiler either (or SOME
compiler). And given that you should be able to do cross-compilation for platforms that don't currently have a python build, I'm not sure that's a killer.
It it so wrong that python eats its own dog food in this manner?
Wrong, no. harder yes. C compilers have mature cross compiler build chains including stage aware test suites and the like.
Not to mention that any decent platform manufacturer or system vendor will do the job of writing or porting a C compiler for the platform. Few of them will port Python for us, though.
Regards
Antoine.
On Sat, Mar 14, 2015 at 6:47 AM, Chris Barker chris.barker@noaa.gov wrote:
On Fri, Mar 13, 2015 at 12:40 PM, Ryan Gonzalez rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
Well, you can't buid a C compiler without a C compiler either (or SOME compiler). And given that you should be able to do cross-compilation for platforms that don't currently have a python build, I'm not sure that's a killer.
It it so wrong that python eats its own dog food in this manner?
-CHB
I've been just skimming this thread rather than reading in detail, so please forgive me if this question is completely unrelated.
Will this affect the commands required to build Python from source, or is all this just for actual development?
Consider two completely different workflows here: Firstly, a core Python dev. S/he wants to be able to efficiently build, edit, etc, and wants to be able to edit any part of the code. and secondly, someone who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
In the first case, it's fine to require that some version of Python be already installed. If editing file XYZ requires that file ABC be rebuilt from it, and the rebuild process requires Python 3.3 or newer, that's not going to hurt people. The modified ABC gets checked in along with the change to XYZ, and all's well.
But in the second case, it'd be much safer to minimize the requirements. If you can untar the source and just "./configure; make; sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
ChrisA
That written in Python, so Python would require Python to build which would require Python to build which would require...
A bit more here:
It seems there are two cases:
1) Development on and for a common, mature platform:
In this case finding a functional python with which to run the build tool sis trivial. We're all set, no need to exclude python-based tools.
2) Development for / porting to a new and/or obscure platform:
In this case, then there is no existing python with which to run the build tools. And there are two ways you might go about this (in general):
a) cross-compiling -- run the compiler on a existing mature platform, and generate a compiled python for the new platform. IN this case, the build tools only need to run on the development platform, so we are back to (1), and all set. Note that the hard part of cross-compiling is still done by the C compiler, not really the build/configuration tools, The build tools need to be able to set up the cross-compile -- but presumable a new tool is chosen be cause it's easier to do things like that with it...
b) Native compiling -- compiling python on the new platform. This is the tricky one. In this case, the entire goal is to get python building/running on the new platform. Clearly you can't just use the nifty python-based build system to do that. But as that's the goal, you'll still need to do the tricky parts anyway. So you could cross-compile the first version -- with maybe as few modules as possible to get started. Another option is a build system that needs python for the configuration stage, but not the compilation stage -- kind of like autotools: autotools is used in a three-step process:
i) run autoconf, etc, to generate a configure script -- this is often distributed with the source tarball, so that end users don't need the autoconf tools
ii) run the configure script to generate Makefiles, etc. configured for the platform at hand.
iii) run make to do the actual build.
Usually the last two steps are run natively -- but it's not too much of a stretch to run i and ii on a known build machine, and then only ned to run make on the final target. That would require the build tools to have a cross-configure ability.
I have no idea if any of the known tools can do this, but if so, it would be pretty usable.
And also worth keeping in mind that truly new platforms are pretty rare -- this kind of bootstrapping would not have to be done often --not a killer if it takes some hand-work.
But the grain of salt : I have neer and am unlikely ever, to try to port a significant project to a new platform. Folks that actually need to do this should make these decisions.
Will this affect the commands required to build Python from source, or
is all this just for actual development?
well, it may or may not be both:
If the system supported a three-step process I outlined above, then makefiles (or the like) could be shipped for the common platforms, though probably easier to run the configure step on the final target. But the vast majority of final targets do have a python ready to run the configure/build machinery.
someone
who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
But in the second case, it'd be much safer to minimize the
requirements. If you can untar the source and just "./configure; make;
sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
In short -- using a python-based build system would make porting to a brand-new platform trickier -- but not really affect most other building.
-CHB
On Tue, Mar 17, 2015 at 3:27 AM, Chris Barker chris.barker@noaa.gov wrote:
someone who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
The tricky part here is defining the oldest supported version. "To build Python X.Y, you must already have Python ??? or later installed" - how do you pick the minimum? The older that minimum, the more warped the build code has to be (especially if you want to support 2.x as well as 3.x - imagine trying to make it possible to use the system Python on RHEL, as well as any 3.[1-4]), but go too far the other way and you end up with a multi-step build process ("first build Python 3.4, then use that to build Python 3.7") which is just going to be a pain.
But in the second case, it'd be much safer to minimize the requirements. If you can untar the source and just "./configure; make; sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
No, that's not what I'm talking about. I'm talking about trying to build Thunderbird and having to do a bunch of steps where I run this program, then run that, then let it chug its way through stuff, and then eventually it'll start building. Without detailed knowledge of the build internals I have no way of knowing what steps I can skip, so every time I want to test the build again, I have to go through them all. It took three parts of forever to figure out what was going on.
I much prefer projects that ship with the standard "./configure; make" pattern. I don't care if the configure script is built by some arcane incantation, as long as it's checked into source control (or at very least is in the downloadable source tarball for any particular version). If I want to build your program, I don't want to have to hunt down your notes file, read it through in detail, and jump through a dozen hoops; I just want to point my compiler at your source, and then go make myself a hot chocolate while it does its work.
ChrisA
On Mon, Mar 16, 2015 at 11:50 AM, Chris Angelico rosuav@gmail.com wrote:
On Tue, Mar 17, 2015 at 3:27 AM, Chris Barker chris.barker@noaa.gov wrote:
someone who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version
--
so probably already has easy access to an older version with which to run the build tools...
The tricky part here is defining the oldest supported version. "To build Python X.Y, you must already have Python ??? or later installed"
- how do you pick the minimum? The older that minimum, the more warped
the build code has to be (especially if you want to support 2.x as well as 3.x - imagine trying to make it possible to use the system Python on RHEL, as well as any 3.[1-4]), but go too far the other way and you end up with a multi-step build process ("first build Python 3.4, then use that to build Python 3.7") which is just going to be a pain.
Sounds like Go is now. It's a pain.
But in the second case, it'd be much safer to minimize the requirements. If you can untar the source and just "./configure; make; sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and
there's a
separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
No, that's not what I'm talking about. I'm talking about trying to build Thunderbird and having to do a bunch of steps where I run this program, then run that, then let it chug its way through stuff, and then eventually it'll start building. Without detailed knowledge of the build internals I have no way of knowing what steps I can skip, so every time I want to test the build again, I have to go through them all. It took three parts of forever to figure out what was going on.
All of Mozilla's build systems need help. They have a build system genius who wrote Tup working for them, and they somehow still have 20,000 stupid mozbuild files with some weird, intensely slow make-based system that everyone complains about.
I much prefer projects that ship with the standard "./configure; make" pattern. I don't care if the configure script is built by some arcane incantation, as long as it's checked into source control (or at very least is in the downloadable source tarball for any particular version). If I want to build your program, I don't want to have to hunt down your notes file, read it through in detail, and jump through a dozen hoops; I just want to point my compiler at your source, and then go make myself a hot chocolate while it does its work.
Most of that is bad documentation. If there's no configure script but a file labeled INSTALLING, then that should contain simple instructions to build.
I hate most tools that ship with configure scripts because 70% of the time some stupid test does some stupid thing wrong and gives the worst errors, 29% of the time the configure system doesn't pick up my options correctly, and 1% of the time it actually *works*.
SCons also constantly gives me trouble.
Of course, you could always do it the plan9port way:
ryan@DevPC-LX:~/stuff/plan9port$ ./configure read the README file. ryan@DevPC-LX:~/stuff/plan9port$
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Ryan Gonzalez <rymg19@...> writes:
I hate most tools that ship with configure scripts because 70% of the time
some stupid test does some stupid thing wrong and gives the worst errors, 29% of the time the configure system doesn't pick up my options correctly, and 1% of the time it actually *works*.
That's just not true. Also, the small build issue that triggered this thread is already tracked here:
http://bugs.python.org/issue22625
It seems more productive to me to fix that rather than rewrite the build system. Users don't have an unlimited tolerance for pointless churn.
Stefan Krah
On Mon, Mar 16, 2015 at 1:33 PM, Stefan Krah stefan@bytereef.org wrote:
Ryan Gonzalez <rymg19@...> writes:
I hate most tools that ship with configure scripts because 70% of the
time some stupid test does some stupid thing wrong and gives the worst errors, 29% of the time the configure system doesn't pick up my options correctly, and 1% of the time it actually *works*.
That's just not true. Also, the small build issue that triggered this thread is already tracked here:
You've evidently had better luck than I usually have.
It seems more productive to me to fix that rather than rewrite the build system. Users don't have an unlimited tolerance for pointless churn.
Lots and lots of projects have moved from autotools to about everything else.
It's slow, annoying, and generates huge files (the 5k-line configure.ac generates a 17k-line configure script). And largely pointless, since nowadays few like it anymore.
Stefan Krah
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 16/03/2015 19:24, Ryan Gonzalez wrote:
On Mon, Mar 16, 2015 at 1:33 PM, Stefan Krah <stefan@bytereef.org mailto:stefan@bytereef.org> wrote:
Ryan Gonzalez <rymg19@...> writes: > I hate most tools that ship with configure scripts because 70% of the time some stupid test does some stupid thing wrong and gives the worst errors, 29% of the time the configure system doesn't pick up my options correctly, and 1% of the time it actually *works*. That's just not true. Also, the small build issue that triggered this thread is already tracked here: http://bugs.python.org/issue22625You've evidently had better luck than I usually have.
Luck doesn't enter in to it. More likely a bad workan always blames his tools?
It seems more productive to me to fix that rather than rewrite the build system. Users don't have an unlimited tolerance for pointless churn.Lots and lots of projects have moved from autotools to about everything else.
It's slow, annoying, and generates huge files (the 5k-line configure.ac http://configure.ac generates a 17k-line configure script). And largely pointless, since nowadays few like it anymore.
Perhaps everybody should move to MSVC, that way at least everything would be consistent :)
On Mon, Mar 16, 2015 at 3:04 PM, Mark Lawrence breamoreboy@yahoo.co.uk wrote:
On 16/03/2015 19:24, Ryan Gonzalez wrote:
On Mon, Mar 16, 2015 at 1:33 PM, Stefan Krah <stefan@bytereef.org mailto:stefan@bytereef.org> wrote:
Ryan Gonzalez <rymg19@...> writes: > I hate most tools that ship with configure scripts because 70% ofthe time some stupid test does some stupid thing wrong and gives the worst errors, 29% of the time the configure system doesn't pick up my options correctly, and 1% of the time it actually *works*.
That's just not true. Also, the small build issue that triggered this thread is already tracked here: http://bugs.python.org/issue22625You've evidently had better luck than I usually have.
Luck doesn't enter in to it. More likely a bad workan always blames his tools?
:P
I never have that much trouble with other build tools, or even hand-written makefiles/configure scripts. Just autotools.
It seems more productive to me to fix that rather than rewrite the build system. Users don't have an unlimited tolerance for pointless churn.Lots and lots of projects have moved from autotools to about everything else.
It's slow, annoying, and generates huge files (the 5k-line configure.ac http://configure.ac generates a 17k-line configure script). And largely pointless, since nowadays few like it anymore.
Perhaps everybody should move to MSVC, that way at least everything would be consistent :)
And non-standards-compliant. :)
-- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language.
Mark Lawrence
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
In article loom.20150316T192520-981@post.gmane.org, Stefan Krah stefan@bytereef.org wrote:
That's just not true. Also, the small build issue that triggered this thread is already tracked here:
http://bugs.python.org/issue22625
It seems more productive to me to fix that rather than rewrite the build system. Users don't have an unlimited tolerance for pointless churn.
+10
How many different platforms and configurations on each platform do we explicitly or implicitly support today for current CPython 2 and CPython 3 releases? I don't know (and I should since I help release them) but it's clearly at least in the dozens. We do not currently have formal tests or test platforms (e.g. buildbots) for many of them like we should and it would be a monumental undertaking to try to migrate the current build system to something substantially different. It's fine to let off steam about frustrations with build systems but talking about it here is not gonna cause it to change. And it won't change unless someone (or, more likely, some big company) is willing to invest an enormous effort in people time and machine resources to do so. Stefan's suggestion is much more practical. Along with it, if someone is motivated, better documenting the current processes for cross-compilation and which pairs of build/target systems are supported would be a welcome and extremely useful improvement. Much of what is there today for cross-compilation has slipped in over the years with little discussion or documentation. It's easy to break because it's not always clear how it is supposed to work and because it isn't regularly tested.
On Mon, Mar 16, 2015 at 3:45 PM, Ned Deily nad@acm.org wrote:
it would be a monumental undertaking to try to migrate the current build system to something substantially different. It's fine to let off steam about frustrations with build systems but talking about it here is not gonna cause it to change. And it won't change unless someone (or, more likely, some big company) is willing to invest an enormous effort in people time and machine resources to do so. Stefan's suggestion is much more practical.
This doesn't seem to me to be a winner takes all proposition -- there is no reason someone couldn't set up a new build system for Python, get it all working nicely on their platform(s) of choice, persuade others how wonderful it is, get contributions, etc.
If it turns out to really be better/easier etc. than autoconf, then , and only then, would it make any sense to talk about replacing the existing official build system. And even then, the two could live in parallel for quite some time...
-Chris
In article CALGmxEJLmU6Jy1LiRo9qbZs09WSsvK2yhGuWWQid3Q=UJC-8tg@mail.gmail.com, Chris Barker chris.barker@noaa.gov wrote:
This doesn't seem to me to be a winner takes all proposition -- there is no reason someone couldn't set up a new build system for Python, get it all working nicely on their platform(s) of choice, persuade others how wonderful it is, get contributions, etc.
If it turns out to really be better/easier etc. than autoconf, then , and only then, would it make any sense to talk about replacing the existing official build system. And even then, the two could live in parallel for quite some time...
Sorry, that's just not realistic. We already have two different build systems, one for Windows and one for all other platforms, and it's difficult enough to keep both of them running. Adding yet another system in parallel is not in the cards without a major infusion of resources. There are frequent minor changes to the current build systems as part of new Python features or bug fixes and those run the risk of breaking things today and too often do. Major changes to build systems impact most parts of the Python ecosphere, for example, development, automated testing, manual testing, release management, downstream distributors of CPython (many of them managing their own sets of local modifications to the current processes), etc etc. To be even able to evaluate the merits of a major build system migration, we would first need to understand what we support today, what are the actual shortcomings of what we have today, and what we want to support in the future. Then, given that, we'd need to decide what are the options and what are their costs. One prerequisite to that is having a much more comprehensive test build environment so we could make even substantial incremental changes with a reasonable degree of confidence that we're not breaking support of a defined set of platforms and configurations. We don't have that today. Working towards that would be a necessary and important step.
Also, note that, in all this talk about autoconf, be aware the CPython build processes don't even make full use of current automake/autoconf features. Much of the current Python build process predates the addition of features to those tools. Just bringing the current configure scripts and Makefiles up-to-date to use all the current features would be a big task that would imply a big testing effort on resources we don't currently have.
Also keep in mind that builds and installations are areas that few core CPython developers are all that interested in and with good reason. It's usually tedious and generally unrewarding work especially when operating in a vacuum. Let's first work towards properly identifying and testing current important environments (for example, from typical installed Python configurations, not build directories and, as noted before, the almost totally undocumented and tested world of cross-building) and then we can start incrementally improving things with some degree of confidence that we're not going to break things for us and our users.
On 16/03/2015 23:45, Chris Barker wrote:
On Mon, Mar 16, 2015 at 3:45 PM, Ned Deily <nad@acm.org mailto:nad@acm.org> wrote:
it would be a monumental undertaking to try to migrate the current build system to something substantially different. It's fine to let off steam about frustrations with build systems but talking about it here is not gonna cause it to change. And it won't change unless someone (or, more likely, some big company) is willing to invest an enormous effort in people time and machine resources to do so. Stefan's suggestion is much more practical.This doesn't seem to me to be a winner takes all proposition -- there is no reason someone couldn't set up a new build system for Python, get it all working nicely on their platform(s) of choice, persuade others how wonderful it is, get contributions, etc.
This sounds really simple to me so when do you start work on it?
On Mon, Mar 16, 2015 at 7:28 PM, Mark Lawrence breamoreboy@yahoo.co.uk wrote:
On Mon, Mar 16, 2015 at 3:45 PM, Ned Deily
This doesn't seem to me to be a winner takes all proposition -- there is no reason someone couldn't set up a new build system for Python, get it all working nicely on their platform(s) of choice, persuade others how wonderful it is, get contributions, etc.
This sounds really simple to me so when do you start work on it?
I think I was misunderstood -- my point was that there is no reason to start all this with trying to persuade the python dev team to accept a new build system up front.
The only possible way it could happen were if someone(s) did the work and got a new system in place that worked well and proved to be better than autoconf in various ways, while being just as capable. And that would be LOT of work to get that far.
But it could be done without having to persuade anyone up front that it 's a good idea.
Frankly, I doubt anyone's going to put that work in -- I know I won't -- remember that I'm one of the folks that's skeptical that this mythical build system exists at all -- never mind the work to transition to it.
-Chris
On Tue, Mar 17, 2015 at 2:45 AM, Chris Barker chris.barker@noaa.gov wrote:
This doesn't seem to me to be a winner takes all proposition -- there is no reason someone couldn't set up a new build system for Python, get it all working nicely on their platform(s) of choice, persuade others how wonderful it is, get contributions, etc.
If it turns out to really be better/easier etc. than autoconf, then , and only then, would it make any sense to talk about replacing the existing official build system. And even then, the two could live in parallel for quite some time...
Both Blender and Wesnoth have SCons and CMake running in parallel for a long time. Both camps are just maintaining their own chains and it works well.
On Mon, Mar 16, 2015 at 8:18 PM, Ryan Gonzalez rymg19@gmail.com wrote:
They have a build system genius who wrote Tup working for them
I like the conceptual brevity of the Tup page http://gittup.org/tup/ It is the core of that I am try to bring from SCons internals to the surface. It is very often that you need to process some DAG and touch nodes based on their changed dependencies.
The problem is so common that it is even worth to spend time on visualization for it. So far no build system has this stuff, and I run of time to complete the SCons plumbing.
On Mon, Mar 16, 2015 at 9:50 AM, Chris Angelico rosuav@gmail.com wrote:
The tricky part here is defining the oldest supported version. "To build Python X.Y, you must already have Python ??? or later installed"
- how do you pick the minimum? The older that minimum, the more warped
the build code has to be
sure -- but it seems what's on the table now is: must be buildable by an old version of C -- not even C99 which is nominally 16 years old...
So yeah, the build system itself would need to support a "old" python -- though probably python2.7 would be OK at this point -- sorry, not py3 yet, but what can you do?
If there is an already build python (like OS-X, any general purpose linux
distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
No, that's not what I'm talking about. I'm talking about trying to build Thunderbird and having to do a bunch of steps where I run this program, then run that, then let it chug its way through stuff, and then eventually it'll start building.
It sounds like that build setup suck, yes -- but not sure why that's relevant. Clearly we don't want something that ugly. We want something that is both easy and robust to configure for new platfroms, etc, and easy on the end user to "jsut build" the source. All I"m proposing that is requireing a not-very-recent python as a first step is only a big hurdle for folks poritn got new platforms -- your use case would be, in INSTALL.txt:
""" 1) You need at least python 2.7 to build this software.
try:
python --version
if it starts up and reports a version greater than 2.7, then you are all set. If not, then It can probably be installed with your system package manager:
apt_get install python
or similar. On Windows, you can download an installer for Python2.7 from python.org
2) Once a python is working, you can build the this latest version from source with:
./buildme """
Of course, there may be options, etc, but that's the basic deal.
And heck, if people REALLY want the ./configure && make && make install dance, then we could ship a little configure script that does nothing but call the python-based one...
Of course, all this is predicated on there BEING some miraculous as-powerful-and-flexible-and-robust-as-autotools-but-much-easier-to-use python-based build system.....
One more note: so adding a python dependency is a pain. But the current system requires separate stems for Windows and *nix-y systems, and who knows what for hypothetical systems that don't support autotools. So maybe a python-based build system that worked on all the common platforms, with an additional makefile that you have to hand-mangle to bootstrap a new platform isn't any more to maintain.
-CHB
On Mon, Mar 16, 2015 at 12:39 PM, Chris Barker chris.barker@noaa.gov wrote:
On Mon, Mar 16, 2015 at 9:50 AM, Chris Angelico rosuav@gmail.com wrote:
The tricky part here is defining the oldest supported version. "To build Python X.Y, you must already have Python ??? or later installed"
- how do you pick the minimum? The older that minimum, the more warped
the build code has to be
sure -- but it seems what's on the table now is: must be buildable by an old version of C -- not even C99 which is nominally 16 years old...
So yeah, the build system itself would need to support a "old" python -- though probably python2.7 would be OK at this point -- sorry, not py3 yet, but what can you do?
If there is an already build python (like OS-X, any general purpose linux
distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
No, that's not what I'm talking about. I'm talking about trying to build Thunderbird and having to do a bunch of steps where I run this program, then run that, then let it chug its way through stuff, and then eventually it'll start building.
It sounds like that build setup suck, yes -- but not sure why that's relevant. Clearly we don't want something that ugly. We want something that is both easy and robust to configure for new platfroms, etc, and easy on the end user to "jsut build" the source. All I"m proposing that is requireing a not-very-recent python as a first step is only a big hurdle for folks poritn got new platforms -- your use case would be, in INSTALL.txt:
"""
- You need at least python 2.7 to build this software.
try:
python --version
if it starts up and reports a version greater than 2.7, then you are all set. If not, then It can probably be installed with your system package manager:
apt_get install python
or similar. On Windows, you can download an installer for Python2.7 from python.org
- Once a python is working, you can build the this latest version from
source with:
./buildme """
Of course, there may be options, etc, but that's the basic deal.
And heck, if people REALLY want the ./configure && make && make install dance, then we could ship a little configure script that does nothing but call the python-based one...
Of course, all this is predicated on there BEING some miraculous as-powerful-and-flexible-and-robust-as-autotools-but-much-easier-to-use python-based build system.....
One more note: so adding a python dependency is a pain. But the current system requires separate stems for Windows and *nix-y systems, and who knows what for hypothetical systems that don't support autotools. So maybe a python-based build system that worked on all the common platforms, with an additional makefile that you have to hand-mangle to bootstrap a new platform isn't any more to maintain.
The best build systems I know in Python are fbuild, Waf, and Meson. Meson generates Ninja scripts, and Ninja's in C++, so that's out of the question.
I have used fbuild a LOT and can say that, after you get through the small amount of command-line boilerplate, it works very well. It's also very easy to extend with host/target compiler differences. Not sure about Waf, but I do know it does about anything you'd guess.
I guess this narrows the result down to Boost.Build or a Python-based build system.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Ryan Gonzalez writes:
I have used fbuild a LOT
Actually, I suspect that in the relevant sense, you haven't used it a lot. Have you used it for a single program on
1. more than 3 CPUs, AND 2. on each CPU, more than one (and preferably more than two) OSes, AND 3. for each CPU-OS combination, more than two different configurations of the program being built, AND 4. for each CPU-OS-configuration combination, at least 3 versions of the program source (preferably spanning a major version bump)?
The number of programs/versions you have built on a single (or small selection of) CPU-OS combinations is irrelevant to the discussion. The problem autotools helps to solve better than any other system I know of is bootstrapping on a new system. It's also quite good at handling foibles (== regressions) across version bumps of the source program.
I agree it's suboptimal (at best :-) to have a PyPI module that depends on the autotools. I've found it to be a real PITA for Windows users who want to develop certain Emacs Lisp applications, for example, which is a similar kind of development environment to Python.
But for bootstrapping Python itself, what else is there, really? Every time I've seen a project move to something like SCons, I've also seen their port rate to new platforms decrease dramatically. (Hard to interpret that trend, though -- if a project has the free cycles to change build systems, maybe they have ported to almost all of the plausible platforms already.) Autotools alternatives just have way fewer modules for dealing with new and/or old ;-) platforms.
Bright! Shiny! New! system use crufty old system :-( + => + missing library :-( library
Caveat: Autotools is the only build system (well, Imake -- now you know how old I am!) I have substantial experience with. But I've heard stories from colleagues about the limitations of Scons and cmake in this respect, and I find it hard to believe that there are systems out there with the accumulated knowledge of platforms that has been assembled in Autotools over the decades.
On Mon, Mar 16, 2015 at 9:27 PM, Stephen J. Turnbull stephen@xemacs.org wrote:
Ryan Gonzalez writes:
I have used fbuild a LOT
Actually, I suspect that in the relevant sense, you haven't used it a lot. Have you used it for a single program on
- more than 3 CPUs, AND
- on each CPU, more than one (and preferably more than two) OSes, AND
- for each CPU-OS combination, more than two different configurations of the program being built, AND
- for each CPU-OS-configuration combination, at least 3 versions of the program source (preferably spanning a major version bump)?
No...but I know that Felix uses it for iPhone, OSX, Linux, ... with separation of host and target and multiple configurations.
The number of programs/versions you have built on a single (or small selection of) CPU-OS combinations is irrelevant to the discussion. The problem autotools helps to solve better than any other system I know of is bootstrapping on a new system. It's also quite good at handling foibles (== regressions) across version bumps of the source program.
I agree it's suboptimal (at best :-) to have a PyPI module that depends on the autotools. I've found it to be a real PITA for Windows users who want to develop certain Emacs Lisp applications, for example, which is a similar kind of development environment to Python.
But for bootstrapping Python itself, what else is there, really? Every time I've seen a project move to something like SCons, I've also seen their port rate to new platforms decrease dramatically. (Hard to interpret that trend, though -- if a project has the free cycles to change build systems, maybe they have ported to almost all of the plausible platforms already.) Autotools alternatives just have way fewer modules for dealing with new and/or old ;-) platforms.
This is also SCons. Everybody loved it until they had to maintain it. :)
Bright! Shiny! New! system use crufty old system :-( + => + missing library :-( library
Caveat: Autotools is the only build system (well, Imake -- now you know how old I am!) I have substantial experience with. But I've heard stories from colleagues about the limitations of Scons and cmake in this respect, and I find it hard to believe that there are systems out there with the accumulated knowledge of platforms that has been assembled in Autotools over the decades.
SCons is very annoying. CMake is weird and only really useful for those huge, 20,000 file C++ projects.
On Tue, Mar 17, 2015 at 8:01 PM, Ryan Gonzalez rymg19@gmail.com wrote:
On Mon, Mar 16, 2015 at 9:27 PM, Stephen J. Turnbull stephen@xemacs.org wrote:
Ryan Gonzalez writes:
I have used fbuild a LOT
Actually, I suspect that in the relevant sense, you haven't used it a lot. Have you used it for a single program on
- more than 3 CPUs, AND
- on each CPU, more than one (and preferably more than two) OSes, AND
- for each CPU-OS combination, more than two different configurations of the program being built, AND
- for each CPU-OS-configuration combination, at least 3 versions of the program source (preferably spanning a major version bump)?
No...but I know that Felix uses it for iPhone, OSX, Linux, ... with separation of host and target and multiple configurations.
What or who is Felix?
This is also SCons. Everybody loved it until they had to maintain it. :)
I know a blind girl, who made a voice synthesizer, which works on Windows, Linux and Android, and she used SCons to build it. That gave me another reason to stick with this build system and try to enhance it.
SCons is very annoying. CMake is weird and only really useful for those huge, 20,000 file C++ projects.
For me the advantage of SCons is that I already know its codebase. There a lot of room for improvement if you can tag yourself as a Python hacker it addition to inherent potential user property.
In any, I'd appreciate the feedback on most annoying SCons feature, because it will give a criteria for comparing it with others and for future development.
On Wed, Mar 18, 2015 at 2:44 AM, anatoly techtonik techtonik@gmail.com wrote:
On Tue, Mar 17, 2015 at 8:01 PM, Ryan Gonzalez rymg19@gmail.com wrote:
On Mon, Mar 16, 2015 at 9:27 PM, Stephen J. Turnbull <stephen@xemacs.org
wrote:
Ryan Gonzalez writes:
I have used fbuild a LOT
Actually, I suspect that in the relevant sense, you haven't used it a lot. Have you used it for a single program on
- more than 3 CPUs, AND
- on each CPU, more than one (and preferably more than two) OSes, AND
- for each CPU-OS combination, more than two different configurations of the program being built, AND
- for each CPU-OS-configuration combination, at least 3 versions of the program source (preferably spanning a major version bump)?
No...but I know that Felix uses it for iPhone, OSX, Linux, ... with separation of host and target and multiple configurations.
What or who is Felix?
A programming language. http://felix-lang.org/. The code base is pretty huge; since it comes with most of its dependencies (re2, Judy, etc.), it consists of about 1600 source files. Fbuild works pretty well, other than the fact that the language's creator is 100% positive that dynamic languages are cruel, evil, and unmaintainable.
This is also SCons. Everybody loved it until they had to maintain it. :)
I know a blind girl, who made a voice synthesizer, which works on Windows, Linux and Android, and she used SCons to build it. That gave me another reason to stick with this build system and try to enhance it.
SCons is very annoying. CMake is weird and only really useful for those huge, 20,000 file C++ projects.
For me the advantage of SCons is that I already know its codebase. There a lot of room for improvement if you can tag yourself as a Python hacker it addition to inherent potential user property.
In any, I'd appreciate the feedback on most annoying SCons feature, because it will give a criteria for comparing it with others and for future development.
Ah, right...you work on SCons...please don't take anything I said personally. :)
It's just that 99% of the time I try to build a project with SCons, some horribly stupid thing goes wrong because of some weird API call. Also, variant directories are WAY to weird for me.
It's also kind of slow. I'm one of those people that gets impatient if I have more than 1.01s for my build to complete, which is why I don't really like make too much.
-- anatoly t.
On Tue, Mar 17, 2015 at 5:27 AM, Stephen J. Turnbull stephen@xemacs.org wrote:
systems out there with the accumulated knowledge of platforms that has been assembled in Autotools over the decades.
Given that Autotools still doesn't have native Windows support, the value of that knowledge is overstated.
On Mon, Mar 16, 2015 at 8:39 PM, Chris Barker chris.barker@noaa.gov wrote:
One more note: so adding a python dependency is a pain. But the current system requires separate stems for Windows and *nix-y systems, and who knows what for hypothetical systems that don't support autotools. So maybe a python-based build system that worked on all the common platforms, with an additional makefile that you have to hand-mangle to bootstrap a new platform isn't any more to maintain.
Manual instruction with lists of files that can be parsed out of it is better than Makefile IMO. Pumped up Markdown or something..
Something nobody's mentioned yet in this thread:
The autotools stuff doesn't just handle different platforms, it also handles local differences (not every FooLinux x.y system has Tcl/Tk; if yours doesn't, you can still build Python, you just won't get tkinter) and user options (disabling features, using a specific OpenSSL instead of the older system one that would get autodetected, etc.).
Autoconf makes this relatively easy (not that anything in autoconf is easy, but if you can write the OpenSSL detection, it's not significantly harder to write OpenSSL detection with a flag to override the path). Is the same thing true for other build configuration systems?
Sent from my iPhone
On Mar 16, 2015, at 9:27 AM, Chris Barker chris.barker@noaa.gov wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
A bit more here:
It seems there are two cases:
- Development on and for a common, mature platform:
In this case finding a functional python with which to run the build tool sis trivial. We're all set, no need to exclude python-based tools.
- Development for / porting to a new and/or obscure platform:
In this case, then there is no existing python with which to run the build tools. And there are two ways you might go about this (in general):
a) cross-compiling -- run the compiler on a existing mature platform, and generate a compiled python for the new platform. IN this case, the build tools only need to run on the development platform, so we are back to (1), and all set. Note that the hard part of cross-compiling is still done by the C compiler, not really the build/configuration tools, The build tools need to be able to set up the cross-compile -- but presumable a new tool is chosen be cause it's easier to do things like that with it...
b) Native compiling -- compiling python on the new platform. This is the tricky one. In this case, the entire goal is to get python building/running on the new platform. Clearly you can't just use the nifty python-based build system to do that. But as that's the goal, you'll still need to do the tricky parts anyway. So you could cross-compile the first version -- with maybe as few modules as possible to get started. Another option is a build system that needs python for the configuration stage, but not the compilation stage -- kind of like autotools: autotools is used in a three-step process:
i) run autoconf, etc, to generate a configure script -- this is often distributed with the source tarball, so that end users don't need the autoconf tools
ii) run the configure script to generate Makefiles, etc. configured for the platform at hand.
iii) run make to do the actual build.
Usually the last two steps are run natively -- but it's not too much of a stretch to run i and ii on a known build machine, and then only ned to run make on the final target. That would require the build tools to have a cross-configure ability.
I have no idea if any of the known tools can do this, but if so, it would be pretty usable.
And also worth keeping in mind that truly new platforms are pretty rare -- this kind of bootstrapping would not have to be done often --not a killer if it takes some hand-work.
But the grain of salt : I have neer and am unlikely ever, to try to port a significant project to a new platform. Folks that actually need to do this should make these decisions.
Will this affect the commands required to build Python from source, or is all this just for actual development?
well, it may or may not be both:
If the system supported a three-step process I outlined above, then makefiles (or the like) could be shipped for the common platforms, though probably easier to run the configure step on the final target. But the vast majority of final targets do have a python ready to run the configure/build machinery.
someone who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
But in the second case, it'd be much safer to minimize the requirements. If you can untar the source and just "./configure; make; sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
In short -- using a python-based build system would make porting to a brand-new platform trickier -- but not really affect most other building.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Most due let you do that easily. For instance, fbuild lets you just trap the raised error and move on.
On Mon, Mar 16, 2015 at 1:05 PM, Andrew Barnert < abarnert@yahoo.com.dmarc.invalid> wrote:
Something nobody's mentioned yet in this thread:
The autotools stuff doesn't just handle different platforms, it also handles local differences (not every FooLinux x.y system has Tcl/Tk; if yours doesn't, you can still build Python, you just won't get tkinter) and user options (disabling features, using a specific OpenSSL instead of the older system one that would get autodetected, etc.).
Autoconf makes this relatively easy (not that anything in autoconf is easy, but if you can write the OpenSSL detection, it's not significantly harder to write OpenSSL detection with a flag to override the path). Is the same thing true for other build configuration systems?
Sent from my iPhone
On Mar 16, 2015, at 9:27 AM, Chris Barker chris.barker@noaa.gov wrote:
That written in Python, so Python would require Python to build which
would require Python to build which would require...
A bit more here:
It seems there are two cases:
- Development on and for a common, mature platform:
In this case finding a functional python with which to run the build tool sis trivial. We're all set, no need to exclude python-based tools.
- Development for / porting to a new and/or obscure platform:
In this case, then there is no existing python with which to run the build tools. And there are two ways you might go about this (in general):
a) cross-compiling -- run the compiler on a existing mature platform, and generate a compiled python for the new platform. IN this case, the build tools only need to run on the development platform, so we are back to (1), and all set. Note that the hard part of cross-compiling is still done by the C compiler, not really the build/configuration tools, The build tools need to be able to set up the cross-compile -- but presumable a new tool is chosen be cause it's easier to do things like that with it...
b) Native compiling -- compiling python on the new platform. This is the tricky one. In this case, the entire goal is to get python building/running on the new platform. Clearly you can't just use the nifty python-based build system to do that. But as that's the goal, you'll still need to do the tricky parts anyway. So you could cross-compile the first version -- with maybe as few modules as possible to get started. Another option is a build system that needs python for the configuration stage, but not the compilation stage -- kind of like autotools: autotools is used in a three-step process:
i) run autoconf, etc, to generate a configure script -- this is often distributed with the source tarball, so that end users don't need the autoconf tools
ii) run the configure script to generate Makefiles, etc. configured for the platform at hand.
iii) run make to do the actual build.
Usually the last two steps are run natively -- but it's not too much of a stretch to run i and ii on a known build machine, and then only ned to run make on the final target. That would require the build tools to have a cross-configure ability.
I have no idea if any of the known tools can do this, but if so, it would be pretty usable.
And also worth keeping in mind that truly new platforms are pretty rare -- this kind of bootstrapping would not have to be done often --not a killer if it takes some hand-work.
But the grain of salt : I have neer and am unlikely ever, to try to port a significant project to a new platform. Folks that actually need to do this should make these decisions.
Will this affect the commands required to build Python from source, or
is all this just for actual development?
well, it may or may not be both:
If the system supported a three-step process I outlined above, then makefiles (or the like) could be shipped for the common platforms, though probably easier to run the configure step on the final target. But the vast majority of final targets do have a python ready to run the configure/build machinery.
someone
who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
But in the second case, it'd be much safer to minimize the
requirements. If you can untar the source and just "./configure; make;
sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
In short -- using a python-based build system would make porting to a brand-new platform trickier -- but not really affect most other building.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mar 16, 2015, at 12:19 PM, Ryan Gonzalez rymg19@gmail.com wrote:
Most due let you do that easily. For instance, fbuild lets you just trap the raised error and move on.
How does trapping an error and moving on give you a user-configurable option to select which OpenSSL to use, whether he wants a fat i386/x86_64 build or just x86_64, whether he wants a framework bundle or a traditional layout...? (These are the first three flags that Homebrew passes to Python's configure aside from the install prefix, which I'd hope everything can handle.)
On Mon, Mar 16, 2015 at 1:05 PM, Andrew Barnert abarnert@yahoo.com.dmarc.invalid wrote: Something nobody's mentioned yet in this thread:
The autotools stuff doesn't just handle different platforms, it also handles local differences (not every FooLinux x.y system has Tcl/Tk; if yours doesn't, you can still build Python, you just won't get tkinter) and user options (disabling features, using a specific OpenSSL instead of the older system one that would get autodetected, etc.).
Autoconf makes this relatively easy (not that anything in autoconf is easy, but if you can write the OpenSSL detection, it's not significantly harder to write OpenSSL detection with a flag to override the path). Is the same thing true for other build configuration systems?
Sent from my iPhone
On Mar 16, 2015, at 9:27 AM, Chris Barker chris.barker@noaa.gov wrote:
That written in Python, so Python would require Python to build which would require Python to build which would require...
A bit more here:
It seems there are two cases:
- Development on and for a common, mature platform:
In this case finding a functional python with which to run the build tool sis trivial. We're all set, no need to exclude python-based tools.
- Development for / porting to a new and/or obscure platform:
In this case, then there is no existing python with which to run the build tools. And there are two ways you might go about this (in general):
a) cross-compiling -- run the compiler on a existing mature platform, and generate a compiled python for the new platform. IN this case, the build tools only need to run on the development platform, so we are back to (1), and all set. Note that the hard part of cross-compiling is still done by the C compiler, not really the build/configuration tools, The build tools need to be able to set up the cross-compile -- but presumable a new tool is chosen be cause it's easier to do things like that with it...
b) Native compiling -- compiling python on the new platform. This is the tricky one. In this case, the entire goal is to get python building/running on the new platform. Clearly you can't just use the nifty python-based build system to do that. But as that's the goal, you'll still need to do the tricky parts anyway. So you could cross-compile the first version -- with maybe as few modules as possible to get started. Another option is a build system that needs python for the configuration stage, but not the compilation stage -- kind of like autotools: autotools is used in a three-step process:
i) run autoconf, etc, to generate a configure script -- this is often distributed with the source tarball, so that end users don't need the autoconf tools
ii) run the configure script to generate Makefiles, etc. configured for the platform at hand.
iii) run make to do the actual build.
Usually the last two steps are run natively -- but it's not too much of a stretch to run i and ii on a known build machine, and then only ned to run make on the final target. That would require the build tools to have a cross-configure ability.
I have no idea if any of the known tools can do this, but if so, it would be pretty usable.
And also worth keeping in mind that truly new platforms are pretty rare -- this kind of bootstrapping would not have to be done often --not a killer if it takes some hand-work.
But the grain of salt : I have neer and am unlikely ever, to try to port a significant project to a new platform. Folks that actually need to do this should make these decisions.
Will this affect the commands required to build Python from source, or is all this just for actual development?
well, it may or may not be both:
If the system supported a three-step process I outlined above, then makefiles (or the like) could be shipped for the common platforms, though probably easier to run the configure step on the final target. But the vast majority of final targets do have a python ready to run the configure/build machinery.
someone who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
But in the second case, it'd be much safer to minimize the requirements. If you can untar the source and just "./configure; make; sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
In short -- using a python-based build system would make porting to a brand-new platform trickier -- but not really affect most other building.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Mar 16, 2015 at 10:38 PM, Andrew Barnert abarnert@yahoo.com wrote:
On Mar 16, 2015, at 12:19 PM, Ryan Gonzalez rymg19@gmail.com wrote:
Most due let you do that easily. For instance, fbuild lets you just trap the raised error and move on.
How does trapping an error and moving on give you a user-configurable option to select which OpenSSL to use, whether he wants a fat i386/x86_64 build or just x86_64, whether he wants a framework bundle or a traditional layout...? (These are the first three flags that Homebrew passes to Python's configure aside from the install prefix, which I'd hope everything can handle.)
Just mix a command-line argument with a try-catch block. It's actually much easier/less-hackish than it sounds.
The nice thing is that fbuild allows multiple builders, so separating target/host stuff and command-line args is super-easy.
On Mon, Mar 16, 2015 at 1:05 PM, Andrew Barnert < abarnert@yahoo.com.dmarc.invalid> wrote:
Something nobody's mentioned yet in this thread:
The autotools stuff doesn't just handle different platforms, it also handles local differences (not every FooLinux x.y system has Tcl/Tk; if yours doesn't, you can still build Python, you just won't get tkinter) and user options (disabling features, using a specific OpenSSL instead of the older system one that would get autodetected, etc.).
Autoconf makes this relatively easy (not that anything in autoconf is easy, but if you can write the OpenSSL detection, it's not significantly harder to write OpenSSL detection with a flag to override the path). Is the same thing true for other build configuration systems?
Sent from my iPhone
On Mar 16, 2015, at 9:27 AM, Chris Barker chris.barker@noaa.gov wrote:
That written in Python, so Python would require Python to build which
would require Python to build which would require...
A bit more here:
It seems there are two cases:
- Development on and for a common, mature platform:
In this case finding a functional python with which to run the build tool sis trivial. We're all set, no need to exclude python-based tools.
- Development for / porting to a new and/or obscure platform:
In this case, then there is no existing python with which to run the build tools. And there are two ways you might go about this (in general):
a) cross-compiling -- run the compiler on a existing mature platform, and generate a compiled python for the new platform. IN this case, the build tools only need to run on the development platform, so we are back to (1), and all set. Note that the hard part of cross-compiling is still done by the C compiler, not really the build/configuration tools, The build tools need to be able to set up the cross-compile -- but presumable a new tool is chosen be cause it's easier to do things like that with it...
b) Native compiling -- compiling python on the new platform. This is the tricky one. In this case, the entire goal is to get python building/running on the new platform. Clearly you can't just use the nifty python-based build system to do that. But as that's the goal, you'll still need to do the tricky parts anyway. So you could cross-compile the first version -- with maybe as few modules as possible to get started. Another option is a build system that needs python for the configuration stage, but not the compilation stage -- kind of like autotools: autotools is used in a three-step process:
i) run autoconf, etc, to generate a configure script -- this is often distributed with the source tarball, so that end users don't need the autoconf tools
ii) run the configure script to generate Makefiles, etc. configured for the platform at hand.
iii) run make to do the actual build.
Usually the last two steps are run natively -- but it's not too much of a stretch to run i and ii on a known build machine, and then only ned to run make on the final target. That would require the build tools to have a cross-configure ability.
I have no idea if any of the known tools can do this, but if so, it would be pretty usable.
And also worth keeping in mind that truly new platforms are pretty rare -- this kind of bootstrapping would not have to be done often --not a killer if it takes some hand-work.
But the grain of salt : I have neer and am unlikely ever, to try to port a significant project to a new platform. Folks that actually need to do this should make these decisions.
Will this affect the commands required to build Python from source, or
is all this just for actual development?
well, it may or may not be both:
If the system supported a three-step process I outlined above, then makefiles (or the like) could be shipped for the common platforms, though probably easier to run the configure step on the final target. But the vast majority of final targets do have a python ready to run the configure/build machinery.
someone
who has little knowledge of Python's internals, and just wants to build the latest version (either from hg or from a source tarball).
That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
But in the second case, it'd be much safer to minimize the
requirements. If you can untar the source and just "./configure; make;
sudo make install" your way to having the latest Python, that's a LOT easier than I've just been doing trying to get my Dad onto a newer Mozilla Thunderbird. (I don't understand what it's doing all along there, but it pulls a whole bunch of separate repositories when I say update, and there's a separate bootstrap step, and ... I dunno.)
requiring _some_, not very recent, python is not exactly
"""a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
In short -- using a python-based build system would make porting to a brand-new platform trickier -- but not really affect most other building.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mar 13, 2015 2:41 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which
would require Python to build which would require...
Besides, SCons sucks. If I were using a build system written in Python,
I'd use fbuild or Waf.
Apparently, Waf started as a fork of SCons. [1][2]
[1] http://stackoverflow.com/questions/12569463/choosing-between-scons-and-waf-i... [2] http://www.scons.org/wiki/SconsVsOtherBuildTools#Waf
I don't know anything about fbuild; others might.
A feature/alternative matrix could be helpful (in addition to a preliminary implementation) for making a cogent argument (in PEP form [3][4]). This is possible in RST with table or list-table syntax.
[3] https://www.python.org/dev/peps/pep-0001/#pep-formats-and-templates [4] https://www.python.org/dev/peps/pep-0009/
On Fri, Mar 13, 2015 at 3:01 PM, Wes Turner wes.turner@gmail.com wrote:
On Mar 13, 2015 2:41 PM, "Ryan Gonzalez" rymg19@gmail.com wrote:
That written in Python, so Python would require Python to build which
would require Python to build which would require...
Besides, SCons sucks. If I were using a build system written in Python,
I'd use fbuild or Waf.
Apparently, Waf started as a fork of SCons. [1][2]
[1] http://stackoverflow.com/questions/12569463/choosing-between-scons-and-waf-i... [2] http://www.scons.org/wiki/SconsVsOtherBuildTools#Waf
But it's now better.
I don't know anything about fbuild; others might.
https://github.com/felix-lang/fbuild
A feature/alternative matrix could be helpful (in addition to a preliminary implementation) for making a cogent argument (in PEP form [3][4]). This is possible in RST with table or list-table syntax.
[3] https://www.python.org/dev/peps/pep-0001/#pep-formats-and-templates [4] https://www.python.org/dev/peps/pep-0009/
-
anatoly techtonik -
Andrew Barnert -
Antoine Pitrou -
Chris Angelico -
Chris Barker -
Florian Bruhin -
Mark Lawrence -
Nathaniel Smith -
Ned Deily -
random832＠fastmail.us -
Robert Collins -
Ryan Gonzalez -
Stefan Krah -
Stephen J. Turnbull -
Steve Dower -
Wes Turner