[Python-Dev] Status of packaging in 3.3

[Tarek Ziadé]

On 6/21/12 10:46 PM, Dag Sverre Seljebotn wrote:
...
>> I think we should, as you proposed, list a few projects w/ compilation
>> needs -- from the simplest to the more complex, then see how a standard
>> *description* could be used by any tool
>
> It's not clear to me what you mean by description. Package metadata, 
> install information or description of what/how to build?
>
> I hope you don't mean the latter, that would be insane...it would 
> effectively amount to creating a build tool that's both more elegant 
> and more powerful than any option that's currently already out there.
>
> Assuming you mean the former, that's what David did to create Bento. 
> Reading and understanding Bento and the design decisions going into it 
> would be a better use of time than redoing a discussion, and would at 
> least be a very good starting point.

What I mean is : what would it take to use Bento (or another tool) as 
the compiler in a distutils-based project, without having to change the 
distutils metadata.


>
> But anyway, some project types from simple to advanced:
>
>  - Simple library using Cython + NumPy C API
>  - Wrappers around HPC codes like mpi4py, petsc4py
>  - NumPy
>  - SciPy (uses Fortran compilers too)
>  - Library using code generation, Cython, NumPy C API, Fortran 90 
> code, some performance tuning with CPU characteristics (instruction 
> set, cache size, optimal loop structure) decided compile-time

I'd add:

- A Distutils project with a few ExtensionsThe other thing is, the folks 
in distutils2 and myself, have zero
>> knowledge about compilers. That's why we got very frustrated not to see
>> people with that knowledge come and help us in this area.
>
> Here's the flip side: If you have zero knowledge about compilers, it's 
> going to be almost impossible to have a meaningful discussion about a 
> compilation PEP. It's very hard to discuss standards unless everybody 
> involved have the necessary prerequisite knowledge. You don't go 
> discussing details of the Linux kernel without some solid C experience 
> either.
Consider me as the end user that want to have his 2 C modules compiled 
in their Python project.

>
> The necessary prerequisites in this case is not merely "knowledge of 
> compilers". To avoid repeating mistakes of the past, the prerequisites 
> for a meaningful discussion is years of hard-worn experience building 
> software in various languages, on different platforms, using different 
> build tools.
>
> Look, these problems are really hard to deal with. Myself I have 
> experience with building 2-3 languages using 2-3 build tools on 2 
> platforms, and I consider myself a complete novice and usually decide 
> to trust David's instincts over trying to make up an opinion of my own 
> -- simply because I know he's got a lot more experience than I have.
>
> Theoretically it is possible to separate and isolate concerns so that 
> one set of people discuss build integration and another set of people 
> discuss installation. Problem is that all the problems tangle -- in 
> particular when the starting point is distutils!
>
> That's why *sometimes*, not always, design by committee is the wrong 
> approach, and one-man-shows is what brings technology forwards.

I am not saying this should be designed by a commitee, but rather - if 
such a tool can be made compatible with simple Distutils project, the 
guy behind this tool can probably help on a PEP with feedback from a 
larger audience than the sci community.

What bugs me is to say that we live in two separate worlds and cannot 
build common pieces. This is not True.

>
>> So, I reiterate my proposal, and it could also be expressed like this:
>>
>> 1/ David writes a PEP where he describes how Bento interact with a
>> project -- metadata, description files, etc
>> 2/ Someone from distutils2 completes the PEP by describing how setup.cfg
>> works wrt Extensions
>> 3/ we see if we can have a common standard even if it's a subset of
>> bento capabilities
>
> bento isn't a build tool, it's a packaging tool, competing directly 
> with distutils2. It can deal with simple distutils-like builds using a 
> bundled build tool, and currently has integration with waf for 
> complicated builds; integration with other build systems will 
> presumably be added later as people need it (the main point is that 
> bento is designed for it).
I am not interested in Bento-the-tool. I am interested in what such a 
tool needs from a project to use it =>

"It can deal with simple distutils-like builds using a bundled build 
tool"  => If I understand this correctly, does that mean that Bento can 
build a distutils project with the distutils Metadata ?

If this is the case it means that there a piece of function that 
translates Distutils metadata into something Bento deals with.

That's the part I am interested in for interoperability.



>
> Dag
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com