Benji York wrote:

...

http://www.python.org/dev/buildbot/

If there's interest in slightly nicer buildbot CSS (something like http://buildbot.zope.org/) I'd be glad to contribute.

I personally don't care much about the visual look of web pages. However, people have commented that the buildbot page is ugly, so yes, please do contribute something. Bonus points for visually separating the "trunk" columns from the "2.4" columns. Would a vertical line be appropriate? Bigger spacing? Regards, Martin

Benji York wrote:

Martin v. Löwis wrote:

...

I personally don't care much about the visual look of web pages. However, people have commented that the buildbot page is ugly, so yes, please do contribute something.

See http://www.benjiyork.com/pybb.

It doesn't look quite as good in IE because of the limited HTML the buildbot waterfall display generates and the limitations of IE's CSS support.

...

Bonus points for visually separating the "trunk" columns from the "2.4" columns.

The best I could do without hacking buildbot was to highlight the trunk "builder" links. This only works in Firefox, also because of IE's limited CSS2 support.

More could be done if the HTML generation was modified, but that didn't seem prudent.

I'd like to see vertical lines between the column. Why is everything bold? Bye, Walter Dörwald

Martin v. Löwis wrote:

Walter Dörwald wrote:

...

I'd like to see vertical lines between the column.

Can you please elaborate? Between which columns?

Something like this: http://styx.livinglogic.de/~walter/python/buildbot.gif

...

Why is everything bold?

Not sure.

Bye, Walter Dörwald

On 2/19/06, Benji York <benji@benjiyork.com> wrote:

Walter Dörwald wrote:

...

I'd like to see vertical lines between the column.

I've done a version like that (still at http://www.benjiyork.com/pybb).

I liked your current version better so I installed it. n

On 2/19/06, Walter Dörwald <walter@livinglogic.de> wrote:

Neal Norwitz wrote:

...

On 2/19/06, Benji York <benji@benjiyork.com> wrote:

...

Walter Dörwald wrote:

...

I'd like to see vertical lines between the column. I've done a version like that (still at http://www.benjiyork.com/pybb).

I liked your current version better so I installed it.

How about this one: http://styx.livinglogic.de/~walter/python/BuildBot_%20Python.html

I like it. It's really nice to be able to fit it all on a single screen (at least for me). Seems slightly crisper to me as well. -Brett

Walter Dörwald wrote:

Neal Norwitz wrote:

...

On 2/19/06, Benji York <benji@benjiyork.com> wrote:

...

Walter Dörwald wrote:

...

I'd like to see vertical lines between the column.

I've done a version like that (still at http://www.benjiyork.com/pybb).

I liked your current version better so I installed it.

How about this one: http://styx.livinglogic.de/~walter/python/BuildBot_%20Python.html

All formats would be improved of the headers could be made to float at the top of the page as scrolling took place. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/

Martin v. Löwis wrote:

Steve Holden wrote:

...

All formats would be improved of the headers could be made to float at the top of the page as scrolling took place.

Can this be done in CSS? If so, contributions are welcome.

Not as it is. The big table would have to be split so that there is one table with the heading and one with the rest. But that would make the columns independent, so the header's column widths would differ from the content's. Even then, I don't know if there's a working solution for the headers to stay on top since * floats are only left or right aligned * the header's height is variable * position:absolute doesn't work in MSIE. regards, Georg

Benji York wrote:

Neal Norwitz wrote:

...

http://www.python.org/dev/buildbot/

If there's interest in slightly nicer buildbot CSS (something like http://buildbot.zope.org/) I'd be glad to contribute.

+1. Looks nice! Georg

Georg Brandl wrote:

Don't we have a Windows slave yet?

No; nobody volunteered a machine yet (plus the hand-holding that is always necessary with Windows). Regards, Martin

Manuzhai wrote:

...

No; nobody volunteered a machine yet (plus the hand-holding that is always necessary with Windows).

What exactly is needed for this? Does it need to be a machine dedicated to this stuff, or could I just run the tests once every day or so when I feel like it and have them submitted to buildbot?

"The point" of buildbot (atleast the way we use it) is to see immediately what check-in broke the tests on some platform. So yes, permanent availability would be desirable. However, buildbot runs in the background (atleast on Unix), and gets triggered whenever a checkin occurs. So the machine doesn't have to be *dedicated*; any machine that is always on might do. Regards, Martin

On 2/19/06, Terry Reedy <tjreedy@udel.edu> wrote:

With a couple of more machines added, should there be two separate pages for trunk and 2.4 builds? Or do most checkins affect both?

They don't; I think a separate page would be a fine idea. FWIW, it looks like all the sample templates are still wasting a lot of horizontal space in the first two columns the second is almost always empty. Perhaps the author of the change could be placed *below* the timestamp instead of next to it? Also for all practical purposes we can probably get rid of the seconds in the timestamp. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:

They don't; I think a separate page would be a fine idea.

Ok, I have now split this into three pages.

FWIW, it looks like all the sample templates are still wasting a lot of horizontal space in the first two columns the second is almost always empty. Perhaps the author of the change could be placed *below* the timestamp instead of next to it? Also for all practical purposes we can probably get rid of the seconds in the timestamp.

The latter was easy to do, so I did it. The former is tricky; contributions are welcome. Regards, Martin

""Martin v. Löwis"" <martin@v.loewis.de> wrote in message news:43F96B38.4040004@v.loewis.de...

Terry Reedy wrote:

...

With a couple of more machines added, should there be two separate pages for trunk and 2.4 builds? Or do most checkins affect both?

I'd like to avoid this, assuming that people only look at the "main" page. An individual checkin affects either the trunk or 2.4, but never both; many check-ins come in pairs.

I mentioned the idea because I am not fond of horizontal scrolling, especially when there is a header column down the left and the prospect of several more columns being added, and thought others might feel the same. But this is, of course, for you, Guido, and whoever else is most concerned to decide. *If* someone revises the HTML generation, you might consider a main page that duplicates the column summary headers (but in two block of rows, one for trunk and one for maintenance) with links to the detail pages. If there are ever three simultaneous development tracks, say from 3.0 starting before the last 2.x is released, this might look even more attractive. Terry J. Reedy

Michael Foord wrote:

Has a machine been volunteered ?

Not yet.

I have a spare machine and an always on connection. Would the 'right' development tools be needed ? (In the case of Microsoft they are a touch expensive I believe.)

Any build process would do. I would prefer to see the official tools on the buildbot (i.e. VS.NET 2003), but anything else that can build Python and pass the test suite could do as well. One issue is that you also have to to work with me on defining the build steps: what sequence of commands to send in what order. For Unix, that is easy; for Windows, not so. Regards, Martin

Michael Foord <fuzzyman@voidspace.org.uk> wrote:

Martin v. Löwis wrote:

...

Any build process would do. I would prefer to see the official tools on the buildbot (i.e. VS.NET 2003),

I can get a free academic license for VS.NET 2003 professional with my university (MSDNAA), and I've also got a Windows machine sitting in my office with a few spare cycles.

...

One issue is that you also have to to work with me on defining the build steps: what sequence of commands to send in what order. For Unix, that is easy; for Windows, not so.

If you're up for it, I'm up for it. It'll take me a bit to get the software on the machine. Want me to ping you when I get the toolset installed? - Josiah

If you're willing to commit to running a buildbot, and the only thing preventing you is shelling out $$$ to Microsoft, send me e-mail. I'll compile a list to send to the PSF and we'll either poke Microsoft to provide some more free licenses or pay for it ourselves. This is what the PSF is for! Note the emphasis on the word "commit", please. I'm setting an arbitrary deadline of Saturday Feb 25 so I don't have to monitor indefinitely. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis

On 2/20/06, Tim Peters <tim.peters@gmail.com> wrote:

Speaking as a PSF director, I might not vote for that :-) Fact is I've been keeping the build & tests 100% healthy on WinXP Pro, and that requires more than just running the tests (it also requires repairing compiler warnings and Unixisms).

These are some ways we need buildbot to help us more. IMO compiler warnings should generate emails from buildbot. We would need to filter out a bunch, but it would be desirable to know about warnings on different architectures. Unfortunately, there are a ton of warnings on OS X right now.

Adding -r to the buildbot test recipe is a decent idea. Getting _some_ debug-build test runs would also be good (or do we do that already?).

Buildbot runs "make testall" which does not run the tests in random order. There's nothing to prevent buildbot from making debug builds, though that is not currently done. The builds I run on the x86 box every 12 hours *do* use debug builds (Misc/build.sh). The results are here: http://docs.python.org/dev/results/ I also recently switched the email to go to python-checkins, though there haven't been any failures yet (unless they are sitting in a spam queue). There are some hangs (like right now): Thread 1: Lib/threading.py (204): wait Lib/threading.py (543): join Lib/threading.py (637): __exitfunc Lib/atexit.py (25): _run_exitfuncs Thread 2: Lib/socket.py (170): accept Lib/SocketServer.py (373): get_request Lib/SocketServer.py (218): handle_request Lib/test/test_socketserver.py (33): serve_a_few Lib/test/test_socketserver.py (82): run Lib/threading.py (445): __bootstrap I've seen test_socketserver fail before, this could be due to running 2 tests simultaneously.

Anyway, since XP Pro is effectively covered, I'd be keener to see a Windows buildbot running under a different flavor of Windows.

+1 n

On 2/21/06, Ronald Oussoren <ronaldoussoren@mac.com> wrote:

On 21-feb-2006, at 9:09, Neal Norwitz wrote:

...

Unfortunately, there are a ton of warnings on OS X right now.

How many of those do you see when you ignore the warnings you get while building the Carbon extensions? Those extensions wrap loads of deprecated functions, each of which will give a warning.

RIght: http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/138/step-co... Most but not all of the warnings are due to Carbon AFAICT. I'd like to fix those that are important, but it's so far down on the priority list. :-( n

Neal Norwitz wrote:

...

How many of those do you see when you ignore the warnings you get while building the Carbon extensions? Those extensions wrap loads of deprecated functions, each of which will give a warning.

RIght: http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/138/step-co...

Most but not all of the warnings are due to Carbon AFAICT. I'd like to fix those that are important, but it's so far down on the priority list. :-(

Should we build with -Wno-deprecated (or whatever it is spelled) on OSX? In general, "deprecated" warnings are useless for Python. We *know* we are providing wrappers around many deprecated functions. We will (nearly automatically) discontinue wrapping the functions when they get removed. Regards, Martin

Ronald> How many of those do you see when you ignore the warnings you Ronald> get while building the Carbon extensions? skip> I see a bunch related to Py_ssize_t. Those have nothing to do skip> with Carbon. I don't see them on the gentoo build, so I assume skip> they just haven't been tackled yet. Let me rephrase that. I assume the people digging through Py_ssize_t issues have been looking at compilation warnings for platforms other than Mac OSX. Skip

>> Let me rephrase that. I assume the people digging through Py_ssize_t >> issues have been looking at compilation warnings for platforms other >> than Mac OSX. Martin> In the buildbot log, I see only a single one of these, and only Martin> in an OSX-specific module. So no - "we" don't look into fixing Martin> them, as they don't occur on Linux at all (as _Qdmodule isn't Martin> built on Linux). Sure looks like core to me: Objects/bufferobject.c: In function `buffer_repr': Objects/bufferobject.c:250: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/bufferobject.c:258: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/bufferobject.c:258: warning: signed size_t format, Py_ssize_t arg (arg 5) ... Objects/funcobject.c: In function `func_set_code': Objects/funcobject.c:254: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/funcobject.c:254: warning: signed size_t format, Py_ssize_t arg (arg 5) Objects/funcobject.c: In function `func_new': Objects/funcobject.c:406: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/funcobject.c:406: warning: signed size_t format, Py_ssize_t arg (arg 5) ... Objects/listobject.c: In function `list_ass_subscript': Objects/listobject.c:2604: warning: signed size_t format, Py_ssize_t arg (arg 3) Objects/listobject.c:2604: warning: signed size_t format, Py_ssize_t arg (arg 4) ... Objects/dictobject.c: In function `PyDict_MergeFromSeq2': Objects/dictobject.c:1152: warning: signed size_t format, Py_ssize_t arg (arg 4) ... Objects/methodobject.c: In function `PyCFunction_Call': Objects/methodobject.c:85: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/methodobject.c:96: warning: signed size_t format, Py_ssize_t arg (arg 4) ... Objects/structseq.c: In function `structseq_new': Objects/structseq.c:129: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/structseq.c:129: warning: signed size_t format, Py_ssize_t arg (arg 5) Objects/structseq.c:137: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/structseq.c:137: warning: signed size_t format, Py_ssize_t arg (arg 5) Objects/structseq.c:146: warning: signed size_t format, Py_ssize_t arg (arg 4) Objects/structseq.c:146: warning: signed size_t format, Py_ssize_t arg (arg 5) ... Objects/typeobject.c: In function `check_num_args': Objects/typeobject.c:3378: warning: signed size_t format, Py_ssize_t arg (arg 4) ... Objects/unicodeobject.c: In function `unicode_decode_call_errorhandler': Objects/unicodeobject.c:794: warning: signed size_t format, Py_ssize_t arg (arg 3) Objects/unicodeobject.c: In function `unicode_encode_call_errorhandler': Objects/unicodeobject.c:2475: warning: signed size_t format, int arg (arg 3) Objects/unicodeobject.c: In function `unicode_translate_call_errorhandler': Objects/unicodeobject.c:3374: warning: signed size_t format, int arg (arg 3) ... This from the build on g5 osx.3 trunk from 22:54 today (21 Feb). Skip

skip@pobox.com wrote:

Neal> IMO compiler warnings should generate emails from buildbot.

It doesn't generate emails for any other condition. I think it should just turn the compilation section yellow.

It would be easy to run the builds with -Werror, making warnings let the compilation fail, which in turn is flagged red. Regards, Martin

"Neal Norwitz" <nnorwitz@gmail.com> wrote in message news:ee2a432c0602210009x2f4d1fffl3d49037b9b084d1e@mail.gmail.com...

There's nothing to prevent buildbot from making debug builds, though that is not currently done.

Now that there are separate report pages for 2.4 and 2.5, you could add pages for debug builds, perhaps with a lower frequency (once a day?), without cluttering up the main two pages.

Tim Peters wrote:

Speaking of which, a number of test failures over the past few weeks were provoked here only under -r (run tests in random order) or under a debug build, and didn't look like those were specific to Windows. Adding -r to the buildbot test recipe is a decent idea. Getting _some_ debug-build test runs would also be good (or do we do that already?).

So what is your recipe: Add -r to all buildbots? Only to those which have an 'a' in their name? Only to every third build? Duplicating the number of builders? Same question for --with-pydebug. Combining this with -r would multiply the number of builders by 4 already. I'm not keen on deciding this for myself. Somebody else please decide for me. Regards, Martin

skip@pobox.com wrote:

Now that you've broken the buildbot page into two (trunk and 2.4) I assume breaking it down even further wouldn't be a major undertaking. If we can recruit a suitable number of boxes I see no particular reason you can't support a 2x, 4x, 8x or more increase in the number of buildbot slaves.

Let me explain the procedure for breaking it down, then: - there are builder objects in buildbot, each displayed as a single lane. - each builder now gets a "category" attribute; currently, the categories are "trunk" and "2.4". - for each page, there is an instance of the Waterfall object, constructed with the list of categories to display. Each Waterfall gets its own port number (currently 9010, 9011, and 9012). - There are reverse proxy rules in Apache's httpd.conf, each page requiring 2 lines (giving currently 6 lines of Apache configuration). So for multiplying this by 8, I would have to create 48 lines of Apache configuration, and use 24 TCP ports. This can be done, but it would take some time to implement. And who is going to look at the 24 pages? Regards, Martin

Martin> So for multiplying this by 8, I would have to create 48 lines of Martin> Apache configuration, and use 24 TCP ports. This can be done, Martin> but it would take some time to implement. I'm not too worried about that because it's a one-time cost. I'd be willing to help out. Just shoot me the httpd config file and other necessary bits and I'll return you the modified stuff. Martin> And who is going to look at the 24 pages? This is, of course, the bigger problem since it's ongoing. If we solicit buildbot slaves we should solicit a pair of eyeballs for each slave as well. That doesn't need to be the owner of the box, but the owner is the likely first candidate to trick^H^H^H^H^Hask. Skip

On 2/21/06, Tim Peters <tim.peters@gmail.com> wrote:

Since the most fruitful variations (IME) for finding code errors are using -r and running a debug build too, I'd change the current run-all-the-time recipes to:

- Stop doing the second "without deleting .py[co]" run. - Do one run with a release build. - Do one run with a debug build. - Use -uall -r for both.

I agree with this, but don't know a clean way to do 2 builds. I modified buildbot to: - Stop doing the second "without deleting .py[co]" run. - Do one run with a debug build. - Use -uall -r for both. Buildbot does *not* also do a release build. That's the only difference between your request above. I agree that it would be desirable, but I think the debug build is more important than the release build right now. We don't have to make this perfect right now. We can talk about this at PyCon and resolve the remaining issues. One thing that would be nice is to have the master.cfg checked in somewhere so we can track changes. n

[Neal Norwitz]

... I also think I know how to do the "double builds" (one release and one debug). But it's too late for me to change it tonight without screwing it up.

I'm not mad :-). The debug build is more fruitful than the release build for finding problems, so doing two debug-build runs is an improvement (keeping in mind that some bugs only show up in release builds, though -- for example, subtly incorrect C code that works differently depending on whether compiler optimization is in effect).

The good/bad news after this change is:

http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/145/step-te...

A seg fault on Mac OS when running with -r. :-(

Yay! That's certainly good/bad news. Since I always run with -r, I've had the fun of tracking most of these down. Sometimes it's very hard, sometimes not. regrtest's -f option is usually needed, to force running the tests in exactly the same order, then commenting test names out in binary-search fashion to get a minimal subset. Alas, half the time the cause for a -r segfault turns out to be an error in refcounting or in setting up gc'able containers, and has nothing in particular to do with the specific tests being run. Those are the "very hard" ones ;-) Setting the gc threshold to 1 (do a full collection on every allocation) can sometimes provoke such problems easily.