[Python-ideas] Changing the name of __pycache__

[M.-A. Lemburg]

Nick Coghlan wrote:
> On Thu, Feb 10, 2011 at 5:04 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>> Greg Ewing writes:
>>  > On 10/02/11 11:44, Nick Coghlan wrote:
>>  >
>>  > > Changing that will require a patch and advocacy from folks that are
>>  > > actually *fans* of sourceless distribution, as opposed to merely
>>  > > tolerating them as an arguably necessary evil.
>>  >
>>  > Labelling sourceless distributions as a "necessary evil" comes
>>  > across to me as a political stance, not based on any technical
>>  > argument. I don't think that this kind of ideological thinking
>>  > should have any place in deciding what goes into Python.
>>
>> Note that Nick is *not* doing any labeling in that post (I don't know
>> what his actual opinion is).  He is saying that the people doing the
>> work don't want to do this, and it's up to those who want it to do
>> that work, which at this point will include "not screwing up the
>> existing proposal."
> 
> Yeah, I was just intending to relay the tone of the original
> discussion (which definitely acquired an ideological flavour at
> times). As I recall, I was one of those arguing that there are valid,
> practical use cases for sourceless distribution. However, those use
> cases were adequately met by the simple solution in the PEP (i.e.
> retaining support for substituting a bytecode file in place of a
> source file simply by changing the extension), so that compromise
> satisfied all parties involved in the discussion at the time.
> 
> The best reference I found summarising the situation is here:
> http://www.mail-archive.com/python-dev@python.org/msg45924.html

Thanks for the link. Now I know why I didn't spot this... I would
never have assumed such a discussion under a subject line "__file__" :-)

> So apparently there was a fair bit of in person discussion at the
> PyCon 2010 language summit as well. The flurry of PyCon related list
> activity around that time would also explain why several developers
> missed the discussion.
> 
> All that said, you could definitely extend the PEP 3147 idea in 3.3 to
> allow sourceless imports into multiple Python interpreters in a single
> directory, I'm just not sure I see the point.

The point is that creating sourceless distros is quite easy
in Python2 and it's a feature commercial apps often want
to use. They provide a way to protect your code, but are also
useful to trim down the size of a distribution (the source code
is not needed to use a package).

> The whole goal of PEP 3147 is to allow multiple interpreters to share
> a single source file without fighting over a single cache location,
> while still keeping the cached files near the original source files.
> There's no such sharing benefit when it comes to sourceless
> distributions, so why not simply have a separate directory hierarchy
> per interpreter and use the basic file name? Is the desire to do this
> really common enough to add yet-another-stat-call to the import
> sequence?

Oh yes there is :-) People often download the wrong archives
for their Python version and having the possibility to ship
the files for all supported Python versions in one package
would make them very happy - pretty much for the same reasons
the PEP makes distro maintainers happy. (And I don't really
see why Linux distro maintainers are any more special than
people wanting to create sourceless distributions.)

Also note that this won't be another stat call everybody will have
to pay for: the import logic would only fall back to the alternative
pyc location in case it doesn't find the .py file, so sourceful
distributions would not see any extra stat calls.

You do add another stat call before raising the ImportError, but
that's not really all that much of an issue.

Besides, it seems noone is really worried about stat calls anymore
anyway... just check how many stat calls are needed to get an
interpreter started up with a few eggs sitting in site-packages.

Let's add this to Python 3.3.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 10 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/