On Tue, Jan 31, 2006 at 06:04:45PM +0100, "Martin v. Löwis" wrote:

Thomas Wouters wrote:

...

I also noticed test_logging is spuriously failing, and not just on my machine (according to buildbot logs.) Is anyone (Vinay?) looking at that yet?

I looked at it, and I couldn't figure it out. It appears that the last line of communication is somehow lost, but I could not find out why that might happen.

Not just the last line, see below. It never happens, as far as I can tell, when the test_logging test is run alone; it never once broke in several hundred solo tests. When I ran all combinations of a non-logging test followed by test_logging, it seems to break fairly randomly with almost all tests. Certainly not just tests that deal with signals, subprocesses, threads or sockets, also with tests like test_dircache, test_cookielib, test_coding, test_filecmp, and others. Just about the only correlation that makes sense is the duration of the first test. Not all of the other tests trigger test_logging's failure in any degree of reliability, although I get the feeling tests that take longer trigger the bug more often -- except that a test consisting of 'time.sleep(30)' never triggered it. All in all, it seems like a timing issue -- the timing is different because the previous test already imported some modules. The test_logging test uses threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at strategic places, but it doesn't seem to matter. Regardless of my pinpointing efforts, much more than just the last line is sometimes missing, though: $ ./python -E -tt Lib/test/regrtest.py -uall test_dircache test_logging test_dircache test_logging test test_logging produced unexpected output: ********************************************************************** *** line 515 of expected output missing: - DEB -> WARNING: Message 16 (via logrecv.tcp.DEB) *** line 521 of expected output missing: - UNDEF -> INFO: Message 22 (via logrecv.tcp.UNDEF) *** line 524 of expected output missing: - INF -> INFO: Finish up, it's closing time. Messages should bear numbers 0 through 24. (via logrecv.tcp.INF) ********************************************************************** I've 'seen' at least all messages between 16 and 22 disappear, although not more than 3 or 4 in total. There's only ever messages missing, never out of order or garbled. I'm starting to think there might actually be a bug in the logging module ;P -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Tue, Jan 31, 2006 at 09:04:39PM +0100, "Martin v. Löwis" wrote:

Thomas Wouters wrote:

...

All in all, it seems like a timing issue -- the timing is different because the previous test already imported some modules. The test_logging test uses threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at strategic places, but it doesn't seem to matter. Regardless of my pinpointing efforts, much more than just the last line is sometimes missing, though:

Can you create a truss/strace dump of a failed run (I tried, but it always succeeded for me when run under strace).

Nope, they always succeed when run under strace. I haven't been able to capture the supposedly-TCP traffic either, not even when binding and connecting to my actual IP address. I wonder if glibc is doing trickery because it sees both endpoints of the socket are in the same process. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On 1/31/06, Tim Peters <tim.peters@gmail.com> wrote:

[Thomas Wouters]

...

Objects/longobject.c:655: warning: 'e' may be used uninitialized in this function

Well, that's pretty bizarre. There's _obviously_ no way to get to a reference to `e` without going through

x = _PyLong_AsScaledDouble(vv, &e);

first. That isn't a useful warning.

But how can the compiler know that it is an output-only argument? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

u guys are way over my head :) bob -- Robert Kim 2611s Highway 101 suite 102 San diego CA 92007 206 984 0880 http://evdo-coverage.com/cellular-repeater.html On 1/31/06, Guido van Rossum <guido@python.org> wrote:

On 1/31/06, Tim Peters <tim.peters@gmail.com> wrote:

...

[Thomas Wouters]

...

Objects/longobject.c:655: warning: 'e' may be used uninitialized in this function

Well, that's pretty bizarre. There's _obviously_ no way to get to a reference to `e` without going through

x = _PyLong_AsScaledDouble(vv, &e);

first. That isn't a useful warning.

But how can the compiler know that it is an output-only argument?

-- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/evdo.hsdpa%40gmail.com

-- Robert Q Kim, Wireless Internet Advisor http://evdo-coverage.com/cellular-repeater.html http://hsdpa-coverage.com 2611 S. Pacific Coast Highway 101 Suite 102 Cardiff by the Sea, CA 92007 206 984 0880

Robert Kim Wireless Internet Advisor <evdo.hsdpa@gmail.com> wrote:

u guys are way over my head :) bob

You seem to be new to the python-dev mailing list. As a heads-up, python-dev is for the development _of_ python. If you are using Python, and want help or want to help others using Python, you should instead join python-list, or the equivalent comp.lang.python newsgroup. Posting as a new user what you just did "u guys are way over my head :)", as well as your earlier post of "anybody here?", is a good and fast way of being placed in everyone's kill file. - Josiah

On Jan 31, 2006, at 8:16 PM, Tim Peters wrote:

[Thomas Wouters]

...

I noticed a few compiler warnings, when I compile Python on my amd64 with gcc 4.0.3:

Objects/longobject.c: In function 'PyLong_AsDouble': Objects/longobject.c:655: warning: 'e' may be used uninitialized in this function

Well, that's pretty bizarre. There's _obviously_ no way to get to a reference to `e` without going through

x = _PyLong_AsScaledDouble(vv, &e);

first. That isn't a useful warning.

Look closer, and it's not quite so obvious. Here's the beginning of PyLong_AsDouble:

double PyLong_AsDouble(PyObject *vv) { int e; double x;

if (vv == NULL || !PyLong_Check(vv)) { PyErr_BadInternalCall(); return -1; } x = _PyLong_AsScaledDouble(vv, &e); if (x == -1.0 && PyErr_Occurred()) return -1.0; if (e > INT_MAX / SHIFT) goto overflow;

Here's the beginning of _PyLong_AsScaledDouble:

_PyLong_AsScaledDouble(PyObject *vv, int *exponent) { #define NBITS_WANTED 57 PyLongObject *v; double x; const double multiplier = (double)(1L << SHIFT); int i, sign; int nbitsneeded;

if (vv == NULL || !PyLong_Check(vv)) { PyErr_BadInternalCall(); return -1; }

Now here's the thing: _PyLong_AsScaledDouble *doesn't* set exponent before returning -1 there, which is where the warning comes from. Now, you might protest, it's impossible to go down that code path, because of two reasons: 1) PyLong_AsDouble has an identical "(vv == NULL || !PyLong_Check (vv))" check, so that codepath in _PyLong_AsScaledDouble cannot possibly be gone down. However, PyLong_Check is a macro which expands to a function call to an external function, "PyType_IsSubtype((vv)-

ob_type, (&PyLong_Type)))", so GCC has no idea it cannot return an error the second time. This is the kind of thing C++'s const

2) There's a guard "(x == -1.0 && PyErr_Occurred())" before "e" is used in PyLong_AsDouble, which checks the conditions that _PyLong_AsScaledDouble set. Thus, e cannot possibly be used, even if the previous codepath *was* possible to go down. However, again, PyErr_BadInternalCall() is an external function, so the compiler has no way of knowing that PyErr_BadInternalCall() causes PyErr_Occurred () to return true. So in conclusion, from all the information the compiler has available to it, it is giving a correct diagnostic. James

[Martin v. Löwis]

It inlines the function to make this determination.

Very cool! Is this a new(ish) behavior?

Now, it's not true that e can be uninitialized then, but there the gcc logic fails:

That's fine -- there are any number of ways a compiler can reach a wrong conclusion by making conservative assumptions, and so long as it's actually staring at code I don't mind that at all. What I would mind is griping about some_func(&a) possibly not setting `a` in the _absence_ of staring at `some_func`'s internals.

If you take the

if (vv == NULL || !PyLong_Check(vv)) { PyErr_BadInternalCall(); return -1; }

case in _PyLong_AsScaledDouble, *exponent won't be initialized.

Certainly, and I don't expect a compiler to realize that this branch is impossible when _PyLong_AsScaledDouble is invoked from the call sites where gcc is complaining.

Then, in PyLong_AsDouble, with

x = _PyLong_AsScaledDouble(vv, &e); if (x == -1.0 && PyErr_Occurred()) return -1.0;

it looks like the return would not be taken if PyErr_Occurred returns false. Of course, it won't, but that is difficult to analyse.

PyLong_AsDouble already did: if (vv == NULL || !PyLong_Check(vv)) { PyErr_BadInternalCall(); return -1; } before calling _PyLong_AsScaledDouble(), and the latter's `x` is the former's `vv`. That is, the check you showed above from _PyLong_AsScaledDouble() is exactly the same as the check PyLong_AsDouble already made. To exploit that, gcc would have to realize PyLong_Check() is a "pure enough" function, and I don't expect gcc to be able to figure that out.

...

I don't know. Is this version of gcc broken in some way relative to other gcc versions, or newer, or ... ? We certainly don't want to see warnings under gcc, since it's heavily used, but I'm not clear on why other versions of gcc aren't producing these warnings (or are they, and people have been ignoring that?).

gcc 4 does inlining in far more cases now.

OK then. Thomas, for these _PyLong_AsScaledDouble()-caller cases, I suggest doing whatever obvious thing manages to silence the warning. For example, in PyLong_AsDouble: int e = -1; /* silence gcc warning */ and then add: assert(e >= 0); after the call.

On Wed, Feb 01, 2006 at 10:15:15AM -0500, Tim Peters wrote:

Thomas, for these _PyLong_AsScaledDouble()-caller cases, I suggest doing whatever obvious thing manages to silence the warning. For example, in PyLong_AsDouble:

int e = -1; /* silence gcc warning */

and then add:

assert(e >= 0);

after the call.

Done, although it was nowhere near obvious to me that -1 would be a sane sentinel value ;) Not that I don't believe you, but it took some actual reading of _PyLong_AsScaledDouble to confirm it. Reading--imagine-that-ly y'rs, -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Tue, Jan 31, 2006 at 08:16:21PM -0500, Tim Peters wrote:

Is this version of gcc broken in some way relative to other gcc versions, or newer, or ... ? We certainly don't want to see warnings under gcc, since it's heavily used, but I'm not clear on why other versions of gcc aren't producing these warnings (or are they, and people have been ignoring that?).

Well, I said 4.0.3, and that was wrong. It's actually a pre-release of 4.0.3 (in Debian's 'unstable' distribution.) However, 4.0.2 (the actual release) behaves the same way. The normal make process shows quite a lot of output on systems that use gcc, so I wouldn't be surprised if people did ignore it, for the most part. My main problem with fixing the warnings is that I don't see the difference between, for example, the 'ssize' variable and the 'nchannels' variable in linuxaudio's lad_obuffree/lad_bufsize/lad_obufcount. 'ssize' gets a warning, 'nchannels' doesn't, yet how they are treated is not particularly different. The ssize output parameter gets set inside a switch, is directly followed by a break, and the switch is directly followed by a set of the nchannels output parameter. The only way through the switch is through the set of ssize. I understand the compiler doesn't "see" it this way, but who knows for how long :) I guess we ignore this until we're closer to a 2.5alpha1 ;P -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

Sjoerd Mullender wrote:

I don't quite understand what's the big deal.

Traditionally, people see two problems with these initializations: - the extra initialization may cause a performance loss. - the initialization might hide real bugs later on. For example, if an additional control flow branch is added which fails to initialize the variable, you don't get the warning anymore, not even from compilers which previously did a correct analysis. Whether this is a big deal, I don't know. Regards, Martin

Scott Dial <scott+python-dev@scottdial.com> writes:

So, either the GCC people have not noticed this problem, or (more likely) have decided that this is acceptable, but clearly it will cause spurious warnings. Hey, after all, they are just warnings.

Well, indeed, but "no warnings" is a useful policy -- it makes new warnings much easier to spot :) The warnings under discussion seem rather excessive to me. Cheers, mwh -- Ignoring the rules in the FAQ: 1" slice in spleen and prevention of immediate medical care. -- Mark C. Langston, asr

Thomas Wouters wrote:

On Wed, Feb 01, 2006 at 01:51:03PM +0000, Michael Hudson wrote:

...

Scott Dial <scott+python-dev@scottdial.com> writes:

...

So, either the GCC people have not noticed this problem, or (more likely) have decided that this is acceptable, but clearly it will cause spurious warnings. Hey, after all, they are just warnings.

...

Well, indeed, but "no warnings" is a useful policy -- it makes new warnings much easier to spot :)

...

The warnings under discussion seem rather excessive to me.

Yes, and more than that; fixing them 'properly' requires more than just initializing them. There is no sane default for some of those warnings, so a proper fix would have to check for a sane value after the function returns. That is, if we take the warning seriously. If we don't take it seriously, initializing the variable may surpress a warning in the future: one of the called functions could change, opening a code path that in fact doesn't initialize the output variable. But initializing to a sentinel value, checking the value before use and handling that case sanely isn't always easy, or efficient. Hence my suggestion to let this wait a bit (since they are, at this time, spurious errors). Fixing the warnings *now* won't fix any bugs, may mask future bugs, and may not be necessary if gcc 4.0.4 grows a better way to surpress these warnings. Or gcc 4.0 may grow more such warnings, in which case we may want to change the 'no warnings' policy or the flags to gcc, or add a 'known spurious warnings' checking thing to the buildbot.

Although it is no consolation, there are two types of unused variable warnings: the known-error ("is used uninitialized in this function") and a probable-error ("may be used uninitialized in this function"). It may be reasonable to ignore the probable-error case. I think someone even mentioned that they really should be an "info" and not a "warning". The points in the code clearly should have attention brought to them because there is a real possibility of error, but as you say, there is no way to rid yourself of this type of warning. Also, note that the phrasing "is"/"may be" is a change from 3.x to 4.x. The old warning was "might" always, and as I understand gcc the "might" of 3.x maps directly to the "is" of 4.x -- leaving "may be" an entirely new thing to 4.x. From gcc/tree-ssa.c: The second pass follows PHI nodes to find uses that are potentially uninitialized. In this case we can't necessarily prove that the use is really uninitialized. This pass is run after most optimizations, so that we thread as many jumps and possible, and delete as much dead code as possible, in order to reduce false positives. We also look again for plain uninitialized variables, since optimization may have changed conditionally uninitialized to unconditionally uninitialized. -- Scott Dial scott@scottdial.com dialsa@rose-hulman.edu

On Wed, Feb 01, 2006 at 11:29:16AM -0500, Tim Peters wrote:

[Thomas Wouters]

...

Well, I said 4.0.3, and that was wrong. It's actually a pre-release of 4.0.3 (in Debian's 'unstable' distribution.) However, 4.0.2 (the actual release) behaves the same way. The normal make process shows quite a lot of output on systems that use gcc, so I wouldn't be surprised if people did ignore it, for the most part.

Does it really? It's completely warning-free on Windows, and that's the intent, and it takes ongoing work to keep it that way. Over at, e.g.,

No, it's mostly warning-free, it just outputs a lot of text. By default, the warnings don't stand out much. And if you have a decent computer, it scrolls by pretty fast, too. ;) -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

Thomas,,,, thanks.. useful string ... bob On 2/1/06, Michael Hudson <mwh@python.net> wrote:

Thomas Wouters <thomas@xs4all.net> writes:

...

On Wed, Feb 01, 2006 at 11:29:16AM -0500, Tim Peters wrote:

...

[Thomas Wouters]

...

Well, I said 4.0.3, and that was wrong. It's actually a pre-release of 4.0.3 (in Debian's 'unstable' distribution.) However, 4.0.2 (the actual release) behaves the same way. The normal make process shows quite a lot of output on systems that use gcc, so I wouldn't be surprised if people did ignore it, for the most part.

...

Does it really? It's completely warning-free on Windows, and that's the intent, and it takes ongoing work to keep it that way. Over at, e.g.,

No, it's mostly warning-free, it just outputs a lot of text. By default, the warnings don't stand out much. And if you have a decent computer, it scrolls by pretty fast, too. ;)

"make -s" is a wonderful thing :)

Cheers, mwh

-- In case you're not a computer person, I should probably point out that "Real Soon Now" is a technical term meaning "sometime before the heat-death of the universe, maybe". -- Scott Fahlman <sef@cs.cmu.edu> _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/evdo.hsdpa%40gmail.com

-- Robert Q Kim, Wireless Internet Advisor http://evdo-coverage.com/cellular-repeater.html http://hsdpa-coverage.com http://evdo-coverage.com/pocket-pc-pda-ppc.html 2611 S. Pacific Coast Highway 101 Suite 102 Cardiff by the Sea, CA 92007 206 984 0880