Nick Coghlan wrote:

Christian Heimes wrote:

...

The solution requires a new attribute in the sys module that contains the name of the implementation. As an alternative we could use the first field of sys.subversion but I prefer an explicit attribute. I'm proposing "sys.name" with a value of "CPython", "IronPython", "Jython" or "PyPy".

My Google skills almost failed me, but searching for "sys.vm" found me what I was after: http://bugs.python.org/issue4242 (a discussion relating to a similar need in the context of flagging implementation specific tests).

Thanks, I knew about the discussion but I didn't know that somebody has already suggested a sys.vm attribute.

As mentioned in that discussion, as of Python 2.6, you can do the following:

...

...

...

import platform platform.python_implementation() 'CPython'

(Although according to the function docstring, PyPy is currently missing from the list of known implementations)

Importing yet-another-module for use in site.py doesn't sound like a great idea, so it may make sense to migrate that information into the sys module is this approach is taken. "sys.name" is a little generic though - something more explicit like "sys.vm" would be better.

The platform modules uses additional modules like the re module. On my system "import platform" loads 7 additional module. The platform modul also uses multiple function calls and regular expression to detect the vm. I'm not going to blame the author of the function for the way the feature is implemented. It's the best way and I'd probably done it the same way, too. But it's an unnecessary slow down of every interpreter startup. 'sys.vm' it is.

Implementation specific user directories sound like a good idea indeed.

An alternative to a lookup table approach might be to be a little more direct and just retrieve the final part of the user specific directory name directly from a new function in the sys module. Then different VM authors can choose any directory name they want without CPython's site.py needing to know anything else about them.

I like your idea. It might be a better idea to turn sys.vm into a struct like sys.float_info. The struct can contains a set of required attributes like an id, a capitalized name, the platform (C, .NET, Java) and the suffix for the user site directory.

...

...

sys.vm sys.vm(id='cpython', name='CPython', platform='C', usersitesuffix='python26') # on win32: Python2.6

...

...

sys.vm sys.vm(id='ironpython', name='IronPython', platform='.NET', usersitesuffix='ironpython26) # on win32: IronPython2.6

Christian

Antoine Pitrou wrote:

`usersitesuffix` should probably be a separate sys attribute, since it doesn't depend on the VM only, but also on the platform and version.

I don't really care and I'm happy to follow the herd in this matter. :) Christian

M.-A. Lemburg wrote:

Christian Heimes wrote:

...

Nick Coghlan wrote:

...

Importing yet-another-module for use in site.py doesn't sound like a great idea, so it may make sense to migrate that information into the sys module is this approach is taken. "sys.name" is a little generic though - something more explicit like "sys.vm" would be better.

The platform modules uses additional modules like the re module. On my system "import platform" loads 7 additional module. The platform modul also uses multiple function calls and regular expression to detect the vm. I'm not going to blame the author of the function for the way the feature is implemented. It's the best way and I'd probably done it the same way, too. But it's an unnecessary slow down of every interpreter startup. 'sys.vm' it is.

...

Implementation specific user directories sound like a good idea indeed.

An alternative to a lookup table approach might be to be a little more direct and just retrieve the final part of the user specific directory name directly from a new function in the sys module. Then different VM authors can choose any directory name they want without CPython's site.py needing to know anything else about them.

I like your idea. It might be a better idea to turn sys.vm into a struct like sys.float_info. The struct can contains a set of required attributes like an id, a capitalized name, the platform (C, .NET, Java) and the suffix for the user site directory.

...

...

...

sys.vm

sys.vm(id='cpython', name='CPython', platform='C', usersitesuffix='python26') # on win32: Python2.6

...

...

...

sys.vm

sys.vm(id='ironpython', name='IronPython', platform='.NET', usersitesuffix='ironpython26) # on win32: IronPython2.6

+1 on adding something like this, though, I'd use sys.python_implementation or just sys.implementation since not all Python implementations are virtual machines - many compile to machine code or some intermediate format.

Agrred.

-1 on importing platform in site.py. Python's startup time is not great to begin with and this would cause the load time to increase even more.

Agrred. (And import time is even worse on IronPython.) Michael -- http://www.ironpythoninaction.com/

Tarek Ziadé wrote:

I have a suggestion though, that completes Nick's answer.

distutils/command/install.py already contains install schemes and imports USER_SITE and USER_BASE to builds user-specific install schemes. I think it wouldn't hurt for clarity to reunite all these schemes in a single place in the stdlib (so in sys maybe?)

Hello Tarek! I'm feeling uncomfortable with sticking so much information into the sys module. Another module with installation schemata wouldn't hurt, though. In my opinion it's usually wise to separate configuration from logic. Christian PS: Thanks for your job with distribute! You are a life safer.

Michael Foord wrote:

I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe for a bug fix release. Christian

On Fri, Oct 9, 2009 at 04:53, Michael Foord wrote:

Christian Heimes wrote:

...

Michael Foord wrote:

...

I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe for a bug fix release.

Right, certainly not adding umpteen new sys attributes. :-)

The problem is that the alternative implementations run well behind Python-trunk, indeed it doesn't really make sense for them to put a lot of effort into implementing a version that is still in development. The result is that they discover incompatibilites after a version has gone into 'bugfix only' mode.

Whilst the fix you have described (add information to sys that is used by site.py and distutils) is ideal it can only go into 2.7. I would *still* like to see a fix in 2.6 - even if it is simple logic in site.py using sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc). That way IronPython 2.6 is able to be compatible with Python 2.6. This logic might need duplicating in distutils (I haven't looked at how distutils works out where the user site-packages folder is), but it is a 'maintenance only' fix.

But it's still a change in semantics. Tossing this into 2.6 would mean that anyone who has worked around the current behaviour is going to have a busted install. -Brett

On Fri, Oct 9, 2009 at 11:32, Michael Foord wrote:

The *only* change in semantics I'm proposing is for users of IronPython 2.6 which is not even at final release yet. CPython users would be unaffected.

Then why can't IronPython patch site.py to do what they want? I still feel uncomfortable changing site.py for this in a micro release.

Sorry for top-posting, mobile device.

Aahz was the most adamant hater of top-posting and he isn't subscribed to python-dev anymore, so whatever. -Brett

Michael

-- http://www.ironpythoninaction.com

On 9 Oct 2009, at 19:00, Brett Cannon wrote:

On Fri, Oct 9, 2009 at 04:53, Michael Foord < fuzzyman@voidspace.org.uk> wrote:

...

Christian Heimes wrote:

...

Michael Foord wrote:

...

I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe for a bug fix release.

Right, certainly not adding umpteen new sys attributes. :-)

The problem is that the alternative implementations run well behind Python-trunk, indeed it doesn't really make sense for them to put a lot of effort into implementing a version that is still in development. The result is that they discover incompatibilites after a version has gone into 'bugfix only' mode.

Whilst the fix you have described (add information to sys that is used by site.py and distutils) is ideal it can only go into 2.7. I would *still* like to see a fix in 2.6 - even if it is simple logic in site.py using sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc). That way IronPython 2.6 is able to be compatible with Python 2.6. This logic might need duplicating in distutils (I haven't looked at how distutils works out where the user site-packages folder is), but it is a 'maintenance only' fix.

But it's still a change in semantics. Tossing this into 2.6 would mean that anyone who has worked around the current behaviour is going to have a busted install.

-Brett

Stephen J. Turnbull wrote:

In the modern environment bandwidth is not really a consideration,

That's no reason to squander it, though. Quoting the entire message every time makes the size of the thread grow as O(n**2), and makes things harder to read as well. That's just senseless. The only possible benefit is to allow latecomers to catch up on the thread without having to consult any archives. But I've never seen any complaints that people who *do* quote responsibly are making this hard, so I can only conclude that it's not a significant problem. -- Greg

On Sun, 11 Oct 2009 03:26:41 pm Stephen J. Turnbull wrote:

Indeed, and that's why I thanked Michael. Trimming can be a PITA if you're using a crummy MUA, and for reasons I have no intention of even trying to remember, let alone understand, a lot of people are very attached to their crummmy MUAs.

Perhaps I've never used a sufficiently crummy MUA, but I don't get this. Particularly for GUI apps on a PC, trimming just means: select text, press delete, repeat until done. How hard can it be? Surely even the crummiest MUA lets you delete text? This might be hard on a PDA or mobile phone, but on a PC with a keyboard and mouse? (I have a vague recollection that Lotus Notes inserts the quoted message *after* you've done editing your reply, so Notes users should be forgiven. Not to mention pitied.) -- Steven D'Aprano

On Sun, Oct 11, 2009 at 1:26 PM, Ben Finney wrote:

"Stephen J. Turnbull" writes:

...

Trimming can be a PITA if you're using a crummy MUA

How so? It merely requires the ability to navigate up and down by lines, and to select and delete text. I've used some very crummy MUAs, but the ability to trim quoted text has never been absent or difficult. Are there MUAs so crummy that this is a PITA, yet still used by any significant portion of email users?

You just can't do it on some mobile device mail clients. For instance Gmail's client on Android. It will just top-post and quote the whole mail for you AFAIK. Tarek

On 11 Oct 2009, at 13:36 , Tarek Ziadé wrote:

On Sun, Oct 11, 2009 at 1:26 PM, Ben Finney

...

wrote: "Stephen J. Turnbull" writes:

...

Trimming can be a PITA if you're using a crummy MUA

How so? It merely requires the ability to navigate up and down by lines, and to select and delete text. I've used some very crummy MUAs, but the ability to trim quoted text has never been absent or difficult. Are there MUAs so crummy that this is a PITA, yet still used by any significant portion of email users?

You just can't do it on some mobile device mail clients. For instance Gmail's client on Android. It will just top-post and quote the whole mail for you AFAIK.

I'll add the iPhone pre 3.0 Mail (also holds for the Touch): while trimming was possible, it was done using the delete key and character per character (jumping to word per word after a few seconds) because there was no selection. This made trimming long mails a fairly long-winded task already, but things were even worse due to the lack of undo: not stopping soon enough (and deleting part of the text you wanted to quote) meant you had to either re-type it manually or restart the whole operation from the start. This has thankfully been fixed (with the addition of a pretty good selection mechanism and an undo). Another iPhone mail "feature", (which hasn't been fixed and is actually just as broken in the desktop version of Mail) is that Mail is hardcoded to top-post: it's not possible to make the insertion point default to bottom the quote, it's always above it with a bunch of white-spaces for the message. Not much of an issue, but it still makes top-posting lower friction than bottom-posting or trimming.

MRAB writes:

[snipped three pages of quoted messages before a one-liner]

Didn't the iPhone also lack cut-and-paste?

Not to sound harsh, but your quoting was a perfect example of wasted visual bandwidth... (are you posting from an iPhone ? ;-)) Antoine.

Ben Finney writes:

"Stephen J. Turnbull" writes:

...

Trimming can be a PITA if you're using a crummy MUA

How so? It merely requires the ability to navigate up and down by lines, and to select and delete text. I've used some very crummy MUAs, but the ability to trim quoted text has never been absent or difficult.

Not difficult, time-consuming. It can take many seconds to scroll to bottom in a 200-line quote with gmail and the webmail system used by the University of California at Santa Cruz. You bet I get down on my knees and give thanks that my other MUA is an Emacs every time I have to use webmail. Please note that I'm not defending the practice of top-posting; I'm trying to understand why anybody would do it, and address the readability issue in that context. A sentinel that says "the next 500 lines are whatever happened to be in the tail of the 16KiB block on the file system" does help. If others are willing to play bad cop, as Aahz did, I'd be very happy to accept the benefit of a cleaned-up list. But I'm not willing to do it myself.

2009/10/11 Jesse Noller :

On Sun, Oct 11, 2009 at 12:38 PM, Stephen J. Turnbull wrote:

...

If others are willing to play bad cop, as Aahz did, I'd be very happy to accept the benefit of a cleaned-up list.  But I'm not willing to do it myself.

Is it really that big of an issue that we have to discuss it ad-infinitum and potentially have a quoting cop? Sometimes top-posting happens. Sometimes people don't trim messages. Sometimes people argue about the color of a shed.

+1 This is really getting so meta that it's off topic. -- Regards, Benjamin

Those who feel diverse can top post. On Sun, Oct 11, 2009 at 11:23 AM, Georg Brandl wrote:

...

...

Is it really that big of an issue that we have to discuss it ad-infinitum and potentially have a quoting cop? Sometimes top-posting happens. Sometimes people don't trim messages. Sometimes people argue about the color of a shed.

+1 This is really getting so meta that it's off topic.

quoting-SIG anyone?

Those who don't can post below.

Fred Drake writes:

Most importantly, insufficient trimming makes many of us start to ignore threads we'd otherwise want to read more carefully or participate in, because the tedium of wading through all the quotes to make sure we catch all the content.

Absolutely. This is a significant reason to publicly, and frequently, respond to request inline-replied, suitably-trimmed messages: to help newcomers reading the list understand what constitutes an easy-to-read message, and to maximise their opportunity to write messages that won't simply be skipped as too much effort to read. -- \ “The WWW is exciting because Microsoft doesn't own it, and | `\ therefore, there's a tremendous amount of innovation | _o__) happening.” —Steve Jobs | Ben Finney

Brett Cannon wrote:

On Fri, Oct 9, 2009 at 11:32, Michael Foord mailto:fuzzyman@voidspace.org.uk> wrote:

The *only* change in semantics I'm proposing is for users of IronPython 2.6 which is not even at final release yet. CPython users would be unaffected.

Then why can't IronPython patch site.py to do what they want? I still feel uncomfortable changing site.py for this in a micro release.

The IronPython team currently have legal issues distributing modified versions of the standard library (Dino can correct me if I'm wrong here). That aside, I think that were a Python feature is implemented in such a way as to preclude compatibility with other implementations then we should see it as a bugfix to correct it. I can't see why you should feel uncomfortable (beyond general principle), it seems clear that this change can't affect users of ClassicPython. It is not only IronPython that will benefit but also Jython when they port to Python 2.6. As a general rule the alternative implementations will only start a serious port once a version is released and into maintenance mode. If we aren't able to fix compatibility issues then the implementations will always have to maintain their own patch sets that may or may not get merged back in (or in the case of implementations with awkward legal departments they'll have to ship bugs). All the best, Michael

On 9 Oct 2009, at 19:00, Brett Cannon mailto:brett@python.org> wrote:

...

On Fri, Oct 9, 2009 at 04:53, Michael Foord mailto:fuzzyman@voidspace.org.uk> wrote:

Christian Heimes wrote:

Michael Foord wrote:

I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe for a bug fix release.

Right, certainly not adding umpteen new sys attributes. :-)

The problem is that the alternative implementations run well behind Python-trunk, indeed it doesn't really make sense for them to put a lot of effort into implementing a version that is still in development. The result is that they discover incompatibilites after a version has gone into 'bugfix only' mode.

Whilst the fix you have described (add information to sys that is used by site.py and distutils) is ideal it can only go into 2.7. I would *still* like to see a fix in 2.6 - even if it is simple logic in site.py using sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc). That way IronPython 2.6 is able to be compatible with Python 2.6. This logic might need duplicating in distutils (I haven't looked at how distutils works out where the user site-packages folder is), but it is a 'maintenance only' fix.

But it's still a change in semantics. Tossing this into 2.6 would mean that anyone who has worked around the current behaviour is going to have a busted install.

-Brett

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

On Sat, Oct 10, 2009 at 10:31, Michael Foord wrote:

Brett Cannon wrote:

...

On Fri, Oct 9, 2009 at 11:32, Michael Foord > wrote:

The *only* change in semantics I'm proposing is for users of IronPython 2.6 which is not even at final release yet. CPython users would be unaffected.

Then why can't IronPython patch site.py to do what they want? I still feel uncomfortable changing site.py for this in a micro release.

The IronPython team currently have legal issues distributing modified versions of the standard library (Dino can correct me if I'm wrong here).

That aside, I think that were a Python feature is implemented in such a way as to preclude compatibility with other implementations then we should see it as a bugfix to correct it.

I can't see why you should feel uncomfortable (beyond general principle), it seems clear that this change can't affect users of ClassicPython. It is not only IronPython that will benefit but also Jython when they port to Python 2.6.

It's just principle. I'm not going to fight this as no one else has spoken up with my view and both Dino and Frank would like this to happen. The one unfortunate thing about this proposal is how this is going to mean I have to install a package potentially four times if I want it available to all possible Python interpreters. Then again, the chances of anyone beyond the people on this mailing list using more than a single interpreter regularly is so small that bothering to add support for yet another user directory that works on any Python interpreter will not be worth it. -Brett

As a general rule the alternative implementations will only start a serious port once a version is released and into maintenance mode. If we aren't able to fix compatibility issues then the implementations will always have to maintain their own patch sets that may or may not get merged back in (or in the case of implementations with awkward legal departments they'll have to ship bugs).

All the best,

Michael

...

On 9 Oct 2009, at 19:00, Brett Cannon mailto:brett@python.org> wrote:

...

On Fri, Oct 9, 2009 at 04:53, Michael Foord mailto:fuzzyman@voidspace.org.uk> wrote:

Christian Heimes wrote:

Michael Foord wrote:

I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe for a bug fix release.

Right, certainly not adding umpteen new sys attributes. :-)

The problem is that the alternative implementations run well behind Python-trunk, indeed it doesn't really make sense for them to put a lot of effort into implementing a version that is still in development. The result is that they discover incompatibilites after a version has gone into 'bugfix only' mode.

Whilst the fix you have described (add information to sys that is used by site.py and distutils) is ideal it can only go into 2.7. I would *still* like to see a fix in 2.6 - even if it is simple logic in site.py using sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc). That way IronPython 2.6 is able to be compatible with Python 2.6. This logic might need duplicating in distutils (I haven't looked at how distutils works out where the user site-packages folder is), but it is a 'maintenance only' fix.

But it's still a change in semantics. Tossing this into 2.6 would mean that anyone who has worked around the current behaviour is going to have a busted install.

-Brett

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

Brett Cannon wrote:

On Sat, Oct 10, 2009 at 10:31, Michael Foord mailto:fuzzyman@voidspace.org.uk> wrote:

Brett Cannon wrote:

On Fri, Oct 9, 2009 at 11:32, Michael Foord mailto:fuzzyman@voidspace.org.uk mailto:fuzzyman@voidspace.org.uk>> wrote:

The *only* change in semantics I'm proposing is for users of IronPython 2.6 which is not even at final release yet. CPython users would be unaffected.

Then why can't IronPython patch site.py to do what they want? I still feel uncomfortable changing site.py for this in a micro release.

The IronPython team currently have legal issues distributing modified versions of the standard library (Dino can correct me if I'm wrong here).

That aside, I think that were a Python feature is implemented in such a way as to preclude compatibility with other implementations then we should see it as a bugfix to correct it.

I can't see why you should feel uncomfortable (beyond general principle), it seems clear that this change can't affect users of ClassicPython. It is not only IronPython that will benefit but also Jython when they port to Python 2.6.

It's just principle. I'm not going to fight this as no one else has spoken up with my view and both Dino and Frank would like this to happen.

The one unfortunate thing about this proposal is how this is going to mean I have to install a package potentially four times if I want it available to all possible Python interpreters. Then again, the chances of anyone beyond the people on this mailing list using more than a single interpreter regularly is so small that bothering to add support for yet another user directory that works on any Python interpreter will not be worth it.

It is unfortunate, there are quite a few modules that I use with both IronPython and ClassicPython. Unfortunately there are many others where I have to make modifications or replace components (especially C extensions). It is these modules that require changes that mean separate site-packages directories are needed. To be honest, at the moment I tend to bundle all third party dependencies in with my IronPython projects. As distutils support for IronPython improves I hope that will change. All the best, Michael

-Brett

As a general rule the alternative implementations will only start a serious port once a version is released and into maintenance mode. If we aren't able to fix compatibility issues then the implementations will always have to maintain their own patch sets that may or may not get merged back in (or in the case of implementations with awkward legal departments they'll have to ship bugs).

All the best,

Michael

On 9 Oct 2009, at 19:00, Brett Cannon mailto:brett@python.org mailto:brett@python.org>> wrote:

On Fri, Oct 9, 2009 at 04:53, Michael Foord mailto:fuzzyman@voidspace.org.uk mailto:fuzzyman@voidspace.org.uk>> wrote:

Christian Heimes wrote:

Michael Foord wrote:

I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True).

I don't think my proposal will land into 2.6. The changes are too severe for a bug fix release.

Right, certainly not adding umpteen new sys attributes. :-)

The problem is that the alternative implementations run well behind Python-trunk, indeed it doesn't really make sense for them to put a lot of effort into implementing a version that is still in development. The result is that they discover incompatibilites after a version has gone into 'bugfix only' mode.

Whilst the fix you have described (add information to sys that is used by site.py and distutils) is ideal it can only go into 2.7. I would *still* like to see a fix in 2.6 - even if it is simple logic in site.py using sys.platform (if sys.platform == 'cli'; elif sys.platform == 'java' etc). That way IronPython 2.6 is able to be compatible with Python 2.6. This logic might need duplicating in distutils (I haven't looked at how distutils works out where the user site-packages folder is), but it is a 'maintenance only' fix.

But it's still a change in semantics. Tossing this into 2.6 would mean that anyone who has worked around the current behaviour is going to have a busted install.

-Brett

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog