Mailman 3 Assigning to __debug__ - Python-Dev

Assigning to debug

Martin v. Loewis

30 Mar 2001 30 Mar '01

7:55 a.m.

After the recent change that assignments to __debug__ are disallowed, I noticed that IDLE stops working (see SF bug report), since it was assigning to __debug__. Simply commenting-out the assignment (to zero) did no good: Inside the __debug__ blocks, IDLE would try to perform print statements, which would write to the re-assigned sys.stdout, which would invoke the code that had the __debug__, which would give up thanks to infinite recursion. So essentially, you either have to remove the __debug__ blocks, or rewrite them to writing to save_stdout - in which case all the ColorDelegator debug message appear in the terminal window. So anybody porting to Python 2.1 will essentially have to remove all __debug__ blocks that were previously disabled by assigning 0 to __debug__. I think this is undesirable. As I recall, in the original description of __debug__, being able to assign to it was reported as one of its main features, so that you still had a run-time option (unless the interpreter was running with -O, which eliminates the __debug__ blocks). So in short, I think this change should be reverted. Regards, Martin P.S. What was the motivation for that change, anyway?

Show replies by thread

M.-A. Lemburg

30 Mar 30 Mar

8:06 a.m.

"Martin v. Loewis" wrote:

...

After the recent change that assignments to __debug__ are disallowed, I noticed that IDLE stops working (see SF bug report), since it was assigning to __debug__.

Simply commenting-out the assignment (to zero) did no good: Inside the __debug__ blocks, IDLE would try to perform print statements, which would write to the re-assigned sys.stdout, which would invoke the code that had the __debug__, which would give up thanks to infinite recursion. So essentially, you either have to remove the __debug__ blocks, or rewrite them to writing to save_stdout - in which case all the ColorDelegator debug message appear in the terminal window.

So anybody porting to Python 2.1 will essentially have to remove all __debug__ blocks that were previously disabled by assigning 0 to __debug__. I think this is undesirable.

As I recall, in the original description of __debug__, being able to assign to it was reported as one of its main features, so that you still had a run-time option (unless the interpreter was running with -O, which eliminates the __debug__ blocks).

So in short, I think this change should be reverted.

+1 from here... I use the same concept for debugging: during development I set __debug__ to 1, in production I change it to 0 (python -O does this for me as well).

...

Regards, Martin

P.S. What was the motivation for that change, anyway?

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

-- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/

Guido van Rossum

1:30 p.m.

...

After the recent change that assignments to __debug__ are disallowed, I noticed that IDLE stops working (see SF bug report), since it was assigning to __debug__.

I checked in a fix to IDLE too, but it seems you were using an externally-installed version of IDLE.

...

Simply commenting-out the assignment (to zero) did no good: Inside the __debug__ blocks, IDLE would try to perform print statements, which would write to the re-assigned sys.stdout, which would invoke the code that had the __debug__, which would give up thanks to infinite recursion. So essentially, you either have to remove the __debug__ blocks, or rewrite them to writing to save_stdout - in which case all the ColorDelegator debug message appear in the terminal window.

IDLE was totally abusing the __debug__ variable -- in the fix, I simply changed all occurrences of __debug__ to DEBUG.

...

So anybody porting to Python 2.1 will essentially have to remove all __debug__ blocks that were previously disabled by assigning 0 to __debug__. I think this is undesirable.

Assigning to __debug__ was never well-defined. You used it at your own risk.

...

As I recall, in the original description of __debug__, being able to assign to it was reported as one of its main features, so that you still had a run-time option (unless the interpreter was running with -O, which eliminates the __debug__ blocks).

The manual has always used words that suggest that there is something special about __debug__. And there was: the compiler assumed it could eliminate blocks started with "if __debug__:" when compiling in -O mode. Also, assert statements have always used LOAD_GLOBAL to retrieve the __debug__ variable.

...

So in short, I think this change should be reverted.

It's possible that it breaks more code, and it's possible that we end up having to change the error into a warning for now. But I insist that assignment to __debug__ should become illegal. You can *use* the variable (to determine whether -O is on or not), but you can't *set* it.

...

Regards, Martin

P.S. What was the motivation for that change, anyway?

To enforce a restriction that was always intended: __debug__ should be a read-only variable. --Guido van Rossum (home page: http://www.python.org/~guido/)

M.-A. Lemburg

1:42 p.m.

Guido van Rossum wrote:

...

...
... So anybody porting to Python 2.1 will essentially have to remove all __debug__ blocks that were previously disabled by assigning 0 to __debug__. I think this is undesirable.

Assigning to __debug__ was never well-defined. You used it at your own risk.

...
As I recall, in the original description of __debug__, being able to assign to it was reported as one of its main features, so that you still had a run-time option (unless the interpreter was running with -O, which eliminates the __debug__ blocks).

The manual has always used words that suggest that there is something special about __debug__. And there was: the compiler assumed it could eliminate blocks started with "if __debug__:" when compiling in -O mode. Also, assert statements have always used LOAD_GLOBAL to retrieve the __debug__ variable.

...
So in short, I think this change should be reverted.

It's possible that it breaks more code, and it's possible that we end up having to change the error into a warning for now. But I insist that assignment to __debug__ should become illegal. You can *use* the variable (to determine whether -O is on or not), but you can't *set* it.

...
Regards, Martin

P.S. What was the motivation for that change, anyway?

To enforce a restriction that was always intended: __debug__ should be a read-only variable.

So you are suggesting that we change all our code to something like: __enable_debug__ = 0 # set to 0 for production mode ... if __debug__ and __enable_debug__: print 'debugging information' ... I don't see the point in having to introduce a new variable just to disable debugging code in Python code which does not run under -O. What does defining __debug__ as read-only variable buy us in the long term ? -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/

Guido van Rossum

2:02 p.m.

...

So you are suggesting that we change all our code to something like:

__enable_debug__ = 0 # set to 0 for production mode

...

if __debug__ and __enable_debug__: print 'debugging information'

...

I can't suggest anything, because I have no idea what semantics you are assuming for __debug__ here, and I have no idea what you want with that code. Maybe you'll want to say "__debug__ = 1" even when you are in -O mode -- that will definitely not work! The form above won't (currently) be optimized out -- only "if __debug__:" is optimized away, nothing more complicated (not even "if (__debug__):". In any case, YOU SHOULD NEVER INTRODUCE VARIABLES USING THE __UNDERSCORE__ CONVENTION! Those names are reserved for the interpreter, and you risk that they will be assigned a different semantics in the future.

...

I don't see the point in having to introduce a new variable just to disable debugging code in Python code which does not run under -O.

What does defining __debug__ as read-only variable buy us in the long term ?

It allows the compiler to assume that __debug__ is a built-in name. In the future, the __debug__ variable may become meaningless, as we develop more differentiated optimization options. The *only* acceptable use for __debug__ is to get rid of code that is essentially an assertion but can't be spelled with just an assertion, e.g. def f(L): if __debug__: # Assert L is a list of integers: for item in L: assert isinstance(item, type(1)) ... --Guido van Rossum (home page: http://www.python.org/~guido/)

Fredrik Lundh

2:07 p.m.

guido wrote:

...

In any case, YOU SHOULD NEVER INTRODUCE VARIABLES USING THE __UNDERSCORE__ CONVENTION!

is the "__version__" convention documented somewhere? Cheers /F

Moshe Zadka

2:21 p.m.

On Fri, 30 Mar 2001, "Fredrik Lundh" wrote:

...

is the "__version__" convention documented somewhere?

Yes. I don't remember where, but the words are something like "the __ names are reserved for use by the infrastructure, loosly defined as the interpreter and the standard library. Code which has aspirations to be part of the infrastructure must use a unique prefix like __bobo_pos__" -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez@debian.org |http://www.{python,debian,gnu}.org

Guido van Rossum

2:40 p.m.

...

guido wrote:

...
In any case, YOU SHOULD NEVER INTRODUCE VARIABLES USING THE __UNDERSCORE__ CONVENTION!

is the "__version__" convention documented somewhere?

This is a trick question, right? :-) __version__ may not be documented but is in de-facto use. Folks introducing other names (e.g. __author__, __credits__) should really consider a PEP before grabbing a piece of the namespace. --Guido van Rossum (home page: http://www.python.org/~guido/)

M.-A. Lemburg

3:10 p.m.

Guido van Rossum wrote:

...

...
So you are suggesting that we change all our code to something like:

__enable_debug__ = 0 # set to 0 for production mode

...

if __debug__ and __enable_debug__: print 'debugging information'

...

I can't suggest anything, because I have no idea what semantics you are assuming for __debug__ here, and I have no idea what you want with that code. Maybe you'll want to say "__debug__ = 1" even when you are in -O mode -- that will definitely not work!

I know, but that's what I'm expecting. The point was to be able to disable debugging code when running Python in non-optimized mode. We'd have to change our code and use a new variable to work around the SyntaxError exception. While this is not so much of a problem for new code, existing code will break (ie. not byte-compile anymore) in Python 2.1. A warning would be OK, but adding yet another SyntaxError for previously perfectly valid code is not going to make the Python users out there very happy... the current situation with two different settings in common use out there (Python 1.5.2 and 2.0) is already a pain to maintain due to the issues on Windows platforms (due to DLL problems). I don't think that introducing even more subtle problems in 2.1 is going to be well accepted by Joe User.

...

The form above won't (currently) be optimized out -- only "if __debug__:" is optimized away, nothing more complicated (not even "if (__debug__):".

Ok, make the code look like this then: if __debug__: if enable_debug: print 'debug info'

...

In any case, YOU SHOULD NEVER INTRODUCE VARIABLES USING THE __UNDERSCORE__ CONVENTION! Those names are reserved for the interpreter, and you risk that they will be assigned a different semantics in the future.

Hey, this was just an example... ;-)

...

...
I don't see the point in having to introduce a new variable just to disable debugging code in Python code which does not run under -O.

What does defining __debug__ as read-only variable buy us in the long term ?

It allows the compiler to assume that __debug__ is a built-in name. In the future, the __debug__ variable may become meaningless, as we develop more differentiated optimization options.

The *only* acceptable use for __debug__ is to get rid of code that is essentially an assertion but can't be spelled with just an assertion, e.g.

def f(L): if __debug__: # Assert L is a list of integers: for item in L: assert isinstance(item, type(1)) ...

Maybe just me, but I use __debug__ a lot to do extra logging or printing in my code too; not just for assertions. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/

barry＠digicool.com

3:38 p.m.

...

...
...
...
...
"GvR" == Guido van Rossum writes:

GvR> The *only* acceptable use for __debug__ is to get rid of code GvR> that is essentially an assertion but can't be spelled with GvR> just an assertion, e.g. Interestingly enough, last night Jim Fulton and I talked about a situation where you might want asserts to survive running under -O, because you want to take advantage of other optimizations, but you still want to assert certain invariants in your code. Of course, you can do this now by just not using the assert statement. So that's what we're doing, and for giggles we're multiply inheriting the exception we raise from AssertionError and our own exception. What I think we'd prefer is a separate switch to control optimization and the disabling of assert. -Barry

Thomas Wouters

3:48 p.m.

On Fri, Mar 30, 2001 at 10:38:48AM -0500, Barry A. Warsaw wrote:

...

What I think we'd prefer is a separate switch to control optimization and the disabling of assert.

You mean something like #!/usr/bin/python -fno-asserts -fno_debug_ -fdocstrings -fdeadbranch Right!-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

M.-A. Lemburg

4:09 p.m.

Thomas Wouters wrote:

...

On Fri, Mar 30, 2001 at 10:38:48AM -0500, Barry A. Warsaw wrote:

...
What I think we'd prefer is a separate switch to control optimization and the disabling of assert.

You mean something like

#!/usr/bin/python -fno-asserts -fno_debug_ -fdocstrings -fdeadbranch

Sounds like a good idea, but how do you tell the interpreter which asserts to leave enabled and which to remove from the code ? In general, I agree, though: a more fine grained control over optimizations would be a Good Thing (even more since we are talking about non-existing code analysis tools here ;-). -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/

Guido van Rossum

5:19 p.m.

...

GvR> The *only* acceptable use for __debug__ is to get rid of code GvR> that is essentially an assertion but can't be spelled with GvR> just an assertion, e.g.

Interestingly enough, last night Jim Fulton and I talked about a situation where you might want asserts to survive running under -O, because you want to take advantage of other optimizations, but you still want to assert certain invariants in your code.

Of course, you can do this now by just not using the assert statement. So that's what we're doing, and for giggles we're multiply inheriting the exception we raise from AssertionError and our own exception. What I think we'd prefer is a separate switch to control optimization and the disabling of assert.

That's one of the things I was alluding to when I talked about more diversified control over optimizations. I guess then the __debug__ variable would indicate whether or not assertions are turned on; something else would let you query the compiler's optimization level. But assigning to __debug__ still wouldn't do what you wanted (unless we decided to *make* this the way to turn assertions on or off in a module -- but since this is a compile-time thing, it would require that the rhs of the assignment was a constant). --Guido van Rossum (home page: http://www.python.org/~guido/)

Martin v. Loewis

6:31 p.m.

...

I checked in a fix to IDLE too, but it seems you were using an externally-installed version of IDLE.

Sorry about that, I used actually one from CVS: with a sticky 2.0 tag :-(

...

Assigning to __debug__ was never well-defined. You used it at your own risk.

When __debug__ was first introduced, the NEWS entry read # Without -O, the assert statement actually generates code that first # checks __debug__; if this variable is false, the assertion is not # checked. __debug__ is a built-in variable whose value is # initialized to track the -O flag (it's true iff -O is not # specified). With -O, no code is generated for assert statements, # nor for code of the form ``if __debug__: <something>''. So it clearly says that it is a variable, and that assert will check its value at runtime. I can't quote any specific messages, but I recall that you've explained it that way also in the public. Regards, Martin

Tim Peters

8:17 p.m.

[Guido]

...

In any case, YOU SHOULD NEVER INTRODUCE VARIABLES USING THE __UNDERSCORE__ CONVENTION!

[/F]

...

is the "__version__" convention documented somewhere?

In the Language Reference manual, section "Reserved classes of identifiers", middle line of the table. It would benefit from more words, though (it just says "System-defined name" now, and hostile users are known to have trouble telling themselves apart from "the system" <wink>).

Tim Peters

8:30 p.m.

Take a trip down memory lane: http://groups.yahoo.com/group/python-list/message/19647 That's the c.l.py msg in which Guido first introduced the idea of __debug__ (and DAMN was searching life easier before DejaNews lost its memory!). The debate immediately following that (cmdline arguments and all) is being reinvented here now. Nothing actually changed from Guido's first proposal (above), except that he gave up his opposition to making "assert" a reserved word (for which far-seeing flexibility I am still most grateful), and he actually implemented the "PS here's a variant" flavor. I wasn't able to find anything in that debate where Guido explicitly said you couldn't bind __debug__ yourself, but neither could I find anything saying you could, and I believe him when he says "no binding" was the *intent* (that's most consistent with everything he said at the time). those-who-don't-remember-the-past-are-doomed-to-read-me-nagging-them- about-it<wink>-ly y'rs - tim

8422

Age (days ago)

8422

Last active (days ago)

List overview

Download

15 comments

8 participants

participants (8)

barry＠digicool.com
Fredrik Lundh
Guido van Rossum
M.-A. Lemburg
Martin v. Loewis
Moshe Zadka
Thomas Wouters
Tim Peters

Assigning to __debug__

Martin v. Loewis

M.-A. Lemburg

Guido van Rossum

M.-A. Lemburg

Guido van Rossum

Fredrik Lundh

Moshe Zadka

Guido van Rossum

M.-A. Lemburg

barry＠digicool.com

Thomas Wouters

M.-A. Lemburg

Guido van Rossum

Martin v. Loewis

Tim Peters

Tim Peters

tags

participants (8)

Assigning to debug