[Python-ideas] sys.path is a hack - bringing it back under control

[Nick Coghlan]

On Tue, Feb 21, 2012 at 3:39 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> I've read the ToC, but which of these parts answers the question: "How to
> make debugging sys.path problems easier?"

Ah, my apologies. I must confess to having misread your original email
(I paid too much attention to the first half, not enough to the
latter).

PEP 395 aims to avoid people feeling the need to mess with sys.path in
the first place, thus reducing the likelihood of problems occurring in
the first place.

For *debugging* sys.path, as you say, the problem is figuring out who
is messing it up after problems have already occurred and you've found
strange entries in there.

There's definitely a case to be made that sys.path should be a smarter
kind of object by default, one that accepts callbacks to be triggered
when modifications occur. (such behaviour would be useful for updating
namespace package __path__ attributes, for instance).

However, for purely debugging purposes, it should be sufficient to
monkeypatch sys.path with an object that write any changes through to
the original path, while overriding the various mutations methods to
report the source of the modification (via stack introspection). IIt's
the same kind of technique you can use to investigate faults in *any*
kind of container. (Versions based on UserDict and UserList might be
interesting Python Cookbook recipes).

> Now I hope it gives an overview what difficulties a person who is
> out-of-context has while trying to solve one tiny user story of debugging
> sys.path. I just want everything to be as much simplified as possible,
> possibly killing the fun for prose readers. Maybe I don't really want to
> think about complex PEP matters, because the idea is just an episode in the
> daily workflow. I'd also really prefer to keep complicated matters (e.g.
> discussions) around tiny user stories, that don't require much time to load
> into the brain and you can only concentrate on two or three of them that are
> conflicting. Proposal to read 15 page technical paper doesn't work well with
> this scenario, so if you just said - "Yes. You have to read that.", I'd
> reply "Well, ok. Next time then.".

The fault was mine - I didn't understand your suggestion correctly, so
I didn't realise that PEP 395 doesn't actually address it.

>> (But then, it's been suggested many times in the past that
>> you may get better responses if you don't make a habit of effectively
>> calling the current core developers a bunch of incompetent idiots, and
>> that doesn't appear to have had the slightest effect on your style of
>> communication. Why should this be any different?).
>
>
> I am not an English writer, but I am interested to know where did this
> impression of me calling core developers a bunch of incompetent idiots is
> coming from. If anybody can quote concrete example and explain in private -
> I may have a chance to change something. My English is a result of
> learning legal and technical English texts, not love letters, and I may not
> possess the communication skills required to write proper letters in
> informal language (which also I prefer more than business stuff). I can
> write in third person without *you* or *I* other personal pronounce, but it
> takes more time to compose the proper form, so the note like this one can
> take an hour or more (it already took more), and time is that I really lack.
> Not me alone, though, but I may be too obsessed with saving someone else's
> time by placing too much attention to it, indeed.

Anatoly, thanks for taking the time to explain that. The impression
comes from the fact that many of the things you object to within
Python are largely a result of limited availability of development
resources, so even things that are at least arguably good ideas simply
don't get investigated. The universe of good ideas is vast, the
universe of bad ideas is even larger, but the space we have the
capacity to explore is actually relatively tiny.

Since there's such an enormous number of things that *could* be done,
the answer to "why aren't they done?" is almost always going to
"because people don't think they're important enough to do them
instead of all the other things they're doing". Deciding how to spend
our time on Python-related efforts is a matter of perceived priorities
and potential payoffs and those are always going to vary substantially
across individuals.

Being more willing to accept that as a rationale for not doing things
would go a long way towards reducing the negative reactions - I know
mine arise not so much for your initial suggestions (which are often,
although not always, quite reasonable ideas in a world where we had
unlimited development resources), but from subsequently continuing to
push them in the face "because we're simply not interested in doing
that" and "the status quo may not be perfect, but it's good enough
that it isn't worth the hassle of changing" responses.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia