[Python-Dev] Distribution tools: What I would like to see

[Talin]

Mike Orr wrote:
> On 11/27/06, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>> Talin schrieb:
>>> As far as rewriting it goes - I can only rewrite things that I understand.
>> So if you want this to change, you obviously need to understand the
>> entire distutils. It's possible to do that; some people have done
>> it (the "understanding" part) - just go ahead and start reading source
>> code.
> 
> You (and Fredrik) are being a little harsh on Talin.  I understand the
> need to encourage people to fix things themselves rather than just
> complaining about stuff they don't like. But people don't have an
> unlimited amount of time and expertise to work on several Python
> projects simultaneously.   Nevertheless, they should be able to offer
> an "It would be good if..." suggestion without being stomped on. The
> suggestion itself can be a contribution if it focuses people's
> attention on a problem and a potential solution.  Just because
> somebody can't learn a big subsystem and write code or docs for it *at
> this moment* doesn't mean they never will.  And even if they don't,
> it's possible to make contributions in one area of Python and
> suggestions in another... or does the karma account not work that way?
> 
> I don't see Talin saying, "You should fix this for me."  He's saying,
> "I'd like this improved and I'm working on it, but it's a big job and
> I need help, ideally from someone with more expertise in distutils."
> Ultimately for Python the question isn't, "Does Talin want this done?"
> but, "Does this dovetail with the direction Python generally wants to
> go?"  From what I've seen of setuptools/distutils evolution, yes, it's
> consistent with what many people want for Python.  So instead of
> saying, "You (Talin) should take on this task alone because you want
> it" as if nobody else did, it would be better to say, "Thank you,
> Talin, for moving this important Python issue along."
> 
> I've privately offered Talin some (unfinished) material I've been
> working on anyway that relates to his vision.  When I get some other
> projects cleared away I'd like to put together that TOC of links I
> mentioned and perhaps collaborate on a Guide with whoever wants to.
> But I also need to learn more about setuptools before I can do that.
> As it happens I need the information anyway because I'm about to
> package an egg....
> 

What you are saying is basically correct, although I have a slightly 
different spin on it.

I've written a lot of documentation over the years, and I know that one 
of the hardest parts of writing documentation is trying to identify your 
own assumptions. To someone who already knows how the system works, its 
hard to understand the mindset of someone who is just learning it. You 
tend to unconsciously assume knowledge of certain things which a new 
user might not know.

To that extent, it can be useful sometimes to have someone who is in the 
process of learning how to use the system, and who is willing to 
carefully analyze and write down their own experiences while doing so. 
Most of the time people are too busy to do this - they want to get their 
immediate problem solved, and they aren't interested in how difficult it 
will be for the next person. This is especially true in cases where the 
problem that is holding them up is three levels down from the level 
where their real goal is - they want to be able to "pop the stack" of 
problems as quickly as possible, so that they can get back to solving 
their *real* problem.

So what I am offering, in this case, is my ignorance -- but a carefully 
described ignorance :) I don't demand that anyone do anything - I'm 
merely pointing out some things that people may or may not care about.

Now, in this particular case, I have actually used distutils before. But 
distutils is one of those systems (like Perl) which tends to leak out of 
your brain if you don't use it regularly - that is, if you only use it 
once every 6 months, at the end of 6 months you have forgotten most of 
what you have learned, and you have to start the learning curve all over 
again. And I am in the middle of that re-learning process right now.

What I am doing right now is creating a new extension project using 
setuputils, and keeping notes on what I do. So for example, I start by 
creating the directory structure:

    mkdir myproject
    cd myproject
    mkdir src
    mkdir test

Next, create a minimal setup.py script. I won't include that here, but 
it's in the notes.

Next, create the myproject.c file for the module in src/, and write the 
'init' function for the module. (again, content omitted but it's in my 
notes). Create a projectname_unittest.py file in test. Add both of these 
to the setup.py file.

At this point, you ought to be able to a "python setup.py test" and have 
it succeed. At this point, you can start adding types and methods, with 
a unit test for each one, testing each one as it is added.

Now, I realize that all of this is "baby steps" to you folks, but it 
took me a day or so to figure out. And its interesting that even these 
few steps cut across a number of tools and libraries - setuptools, 
distutils, unittest, the "extending Python" doc and the "Python C API" doc.

(BTW, I realized another thing that would be really handy is if the 
"extending Python" doc contained hyperlink references to the "Python C 
API" doc, so that when it talks about, say, PyArg_ParseTuple, you could 
go straight to the reference doc for it.)

-- Talin