FYI, I've bumped the PYC magic in a non-standard way (the old standard broke on 2002-01-01); please review:
import.c: """ /* New way to come up with the low 16 bits of the magic number: (YEAR-1995) * 10000 + MONTH * 100 + DAY where MONTH and DAY are 1-based. XXX Whatever the "old way" may have been isn't documented. XXX This scheme breaks in 2002, as (2002-1995)*10000 = 70000 doesn't fit in 16 bits. XXX Later, sometimes 1 gets added to MAGIC in order to record that the Unicode -U option is in use. IMO (Tim's), that's a Bad Idea (quite apart from that the -U option doesn't work so isn't used anyway).
XXX MAL, 2002-02-07: I had to modify the MAGIC due to a fix of the UTF-8 encoder (it previously produced invalid UTF-8 for unpaired high surrogates), so I simply bumped the month value to 20 (invalid month) and set the day to 1. This should be recognizable by any algorithm relying on the above scheme. Perhaps we should simply start counting in increments of 10 from now on ?!
Known values: Python 1.5: 20121 Python 1.5.1: 20121 Python 1.5.2: 20121 Python 2.0: 50823 Python 2.0.1: 50823 Python 2.1: 60202 Python 2.1.1: 60202 Python 2.1.2: 60202 Python 2.2: 60717 Python 2.3a0: 62001 */ #define MAGIC (62001 | ((long)'\r'<<16) | ((long)'\n'<<24)) """
FYI, I've bumped the PYC magic in a non-standard way (the old standard broke on 2002-01-01); please review:
This is fine. I never intended the algorithm as reversible, just as a way to come up with unique magic numbers. There is no requirement that from the magic number one can calculate the date it was assigned.
--Guido van Rossum (home page: http://www.python.org/~guido/)
[M.-A. Lemburg]
FYI, I've bumped the PYC magic in a non-standard way (the old standard broke on 2002-01-01); please review:
Fine by me, except you should also check in a NEWS blurb about it. The current NEWS file says:
""" - Because Python's magic number scheme broke on January 1st, we decided to stop Python development. Thanks for all the fish! """
That's why PythonLabs hasn't done much of anything on Python since 2.2 was released <wink>.
algorithm relying on the above scheme. Perhaps we should simply start counting in increments of 10 from now on ?!
Why 10? I'd rather see it incremented by 1. If you respond that you want to make room for more hacks akin to -U, my response would be that's exactly what I want to prevent by blessing 1 <0.4 wink>.
Tim Peters wrote:
[M.-A. Lemburg]
FYI, I've bumped the PYC magic in a non-standard way (the old standard broke on 2002-01-01); please review:
Fine by me, except you should also check in a NEWS blurb about it. The current NEWS file says:
"""
- Because Python's magic number scheme broke on January 1st, we decided to stop Python development. Thanks for all the fish!
"""
That's why PythonLabs hasn't done much of anything on Python since 2.2 was released <wink>.
Done.
algorithm relying on the above scheme. Perhaps we should simply start counting in increments of 10 from now on ?!Why 10? I'd rather see it incremented by 1. If you respond that you want to make room for more hacks akin to -U, my response would be that's exactly what I want to prevent by blessing 1 <0.4 wink>.
The reason is that I don't want to break the -U scheme. I know it's a hack, but until someone comes up with a better way to add flags to store PYC compile options, we'll have to stick with it (-U changes the semantics of the language in a pretty nasty way ... nothing works anymore ;-).
[M.-A. Lemburg]
The reason is that I don't want to break the -U scheme.
But -U doesn't work anyway:
(-U changes the semantics of the language in a pretty nasty way ... nothing works anymore ;-).
The only things -U have bought us are a bizarre definition of "magic number", code complication, and complaints from people who see -U in the "python -h" blurb and want to know why everything breaks when they try it. It may be a hack you want to use for internal testing, but stuff that's been broken since the day it was introduced, and makes no progress towards working, doesn't belong in the general release.
I know it's a hack, but until someone comes up with a better way to add flags to store PYC compile options, we'll have to stick with it.
But there is no need to store info about PYC compile options: -U is its only use now, and -U has never worked. Since it's worse than useless, better to throw it out, then dream up a rational way to store PYC compile options if and when (and only if and when) there's an actual need for such.
What would we lose if we tossed the -U support code? I can see what we'd gain.
Tim Peters wrote:
[M.-A. Lemburg]
The reason is that I don't want to break the -U scheme.
But -U doesn't work anyway:
(-U changes the semantics of the language in a pretty nasty way ... nothing works anymore ;-).
The only things -U have bought us are a bizarre definition of "magic number", code complication, and complaints from people who see -U in the "python -h" blurb and want to know why everything breaks when they try it. It may be a hack you want to use for internal testing, but stuff that's been broken since the day it was introduced, and makes no progress towards working, doesn't belong in the general release.
Wait... the -U option was added in order to be able to see how well the 8-bit string / Unicode integration works. It's a know fact that the Python standard lib is not Unicode compatible yet and that's exactly what the -U option allows you to test (in a very simple way).
In the long run, Python's std lib should move into the direction of being Unicode compatible, so I don't really see the need for removing -U altogether. To reduce the noise about Python failing to run with the option set, it may be a good idea to remove the mentioning from the -h blurb, though.
I know it's a hack, but until someone comes up with a better way to add flags to store PYC compile options, we'll have to stick with it.
But there is no need to store info about PYC compile options: -U is its only use now, and -U has never worked. Since it's worse than useless, better to throw it out, then dream up a rational way to store PYC compile options if and when (and only if and when) there's an actual need for such.
The -U option is currently the only application of such a flag. We will definitely have a need for these options in the future to make the runtime aware of certain assumptions which have been made in the compiled byte code, e.g. byte code using special opcodes, byte code compiled for a different Python virtual machine (once we get pluggable Python compiler / VM combos), byte code which was compiled using special literal interpretations (such as in the -U case or when compiling the source code with a different source code encoding assumption).
I would be more than happy to get rid off the current PYC magic hack for -U and have it replaced with a better and extensible alternative, e.g. a combination of PYC version number and marhsalled option dictionary.
(I think we've had this discussion before...)
MAL> Wait... the -U option was added in order to be able to see how well MAL> the 8-bit string / Unicode integration works. It's a know fact that MAL> the Python standard lib is not Unicode compatible yet and that's MAL> exactly what the -U option allows you to test (in a very simple MAL> way).
If -U is really just a "test" flag, I don't think it should show up in "python -h" output.
Skip
Skip Montanaro wrote:
(I think we've had this discussion before...)
MAL> Wait... the -U option was added in order to be able to see how well MAL> the 8-bit string / Unicode integration works. It's a know fact that MAL> the Python standard lib is not Unicode compatible yet and that's MAL> exactly what the -U option allows you to test (in a very simple MAL> way).If -U is really just a "test" flag, I don't think it should show up in "python -h" output.
If noone objects, I'll remove the flag from the -h output. Ok ?
If noone objects, I'll remove the flag from the -h output. Ok ?
+1
--Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
If noone objects, I'll remove the flag from the -h output. Ok ?
+1
Done.
[MAL]
Wait... the -U option was added in order to be able to see how well the 8-bit string / Unicode integration works. It's a know fact that the Python standard lib is not Unicode compatible yet and that's exactly what the -U option allows you to test (in a very simple way).
I don't object to testing hacks provided they don't trip up the innocent; it would help to remove -U from the user-visible docs (which I'll do). Note that, by coincidence, Andreas Jung (at Zope Corp) pissed away time worrying about -U breakage yesterday independent of our thread here: it's doing harm.
If you're the only one who tries -U on purpose (anyone? it's clear that I don't ...), it would be better done via a preprocessor define. How often is this used even by you? If it's once per release just to make sure it's still broken <wink>, a variant build wouldn't be a real burden.
... The -U option is currently the only application of such a flag. We will definitely have a need for these options in the future to make the runtime aware of certain assumptions which have been made in the compiled byte code, e.g. byte code using special opcodes, byte code compiled for a different Python virtual machine (once we get pluggable Python compiler / VM combos), byte code which was compiled using special literal interpretations (such as in the -U case or when compiling the source code with a different source code encoding assumption).
There remains no current use for any of these things. When a real use appears, "magic number" abuse won't be appropriate: imp.get_magic() doesn't return a vector; we're not doing the Unixish /etc/magic database any favors by *ever* changing it; and needing to register umpteen distinct magic numbers per release for Linux binfmt would make Python even more irritating to live with there.
I would be more than happy to get rid off the current PYC magic hack for -U and have it replaced with a better and extensible alternative, e.g. a combination of PYC version number and marhsalled option dictionary.
I agree, except that I still think having -U now is a net loss.
participants (4)
-
Guido van Rossum -
M.-A. Lemburg -
Skip Montanaro -
Tim Peters