Hi, people. I just subscribed to py-dev@codespeak. The page at: http://agiletesting.blogspot.com/2005/06/keyword-based-logging-with-py-libra... says: If you think you can use the keyword-based approach to logging in your own application code, or if you have a use case that you would like to see implemented via py.log, or if you have any suggestions at all, please leave a comment here of send email to py-dev at codespeak dot net. So here! :-) I'm preparing to clean up a big Python application which, among other things, (ab)uses the concept of logging, with many complex, over-loaded logging routines. My guess is that cleaning this single aspect, if done properly, would significantly reduce the amount of noise in the source code of this application, and so, somehow increase its legibility. I'm always starving for clean, simple design, and this is how I got interested in `py.log'. Am I welcome at discussing logging issues here, in the context of `py.log', as they will pop up in our adventure? -- François Pinard http://pinard.progiciels-bpi.ca
Hi Francois! On Sun, Oct 02, 2005 at 15:58 -0400, François Pinard wrote:
Hi, people. I just subscribed to py-dev@codespeak. The page at:
http://agiletesting.blogspot.com/2005/06/keyword-based-logging-with-py-libra...
says:
If you think you can use the keyword-based approach to logging in your own application code, or if you have a use case that you would like to see implemented via py.log, or if you have any suggestions at all, please leave a comment here of send email to py-dev at codespeak dot net.
So here! :-)
:-)
I'm preparing to clean up a big Python application which, among other things, (ab)uses the concept of logging, with many complex, over-loaded logging routines. My guess is that cleaning this single aspect, if done properly, would significantly reduce the amount of noise in the source code of this application, and so, somehow increase its legibility.
I'm always starving for clean, simple design, and this is how I got interested in `py.log'. Am I welcome at discussing logging issues here, in the context of `py.log', as they will pop up in our adventure?
Yes, it is. Actually the logging API is still quite experimental so your feedback is most welcome. In fact, there are two logging APIs. The current approach has a global dispatching mechanism that maps produced messages to consumers. Grig has described that in his blog. The new experimental approach is less dynamic and has a single entry "py.log.get(loggername, chan1=consumer, ...)" but it is not documented, unfortunately. I guess i need to make my mind up about the right approach. But for the time being, i'd suggest just forgetting about py.log.get() and sticking to the more documented approach. cheers, holger
[holger krekel]
Hi Francois!
Hi Holger, and thanks for replying!
Actually the logging API is still quite experimental so your feedback is most welcome.
You did open the door! :-) I fetched py-dev@codespeak.net archives a few hours ago, but it will take some time before I become acquainted with them, so I may raise issues which have already been discussed, I do not know yet.
[...] But for the time being, i'd suggest just forgetting about py.log.get() and sticking to the more documented approach.
OK. In any case, the code base for our big application is likely to be boiling at least for the next six months or so, so changes are always possible on our side for quite a while, without being hurtful. Let me start the discussion on the following issue. It is a common and useful idiom, where I work most of the time, to pass a ``write`` argument to various functions, letting the caller decide what ``write`` will do in any particular case. It could be an opened file, the input side of a service process, the ``append`` method of a list, or God knows what else. It would be convenient being able to merely pass a logger for that ``write`` argument, hoping this would yield a sensible behaviour. This comes with problems, however. Currently, py.log and the standard logging module both provide a newline after the text to be logged. If I provide a newline at the end of a text given to a py.log logger, this will create a spurious white line. Whether a logger should remove trailing newlines or not, is debatable. For one, I would be willing and ready to always provide the trailing newline in my calls, given I expect the logger to add none, so logging functions and ``write`` functions are more fully interchangeable. However, tradition is that logging functions turn incomplete lines into complete lines. I do not know where lies the greater wisdom... Another point, which is pretty factual in our code base, is that the application often logs multi-line texts, which are then (of course) displayed on many lines. I really dislike runs of stars or hyphens for creating visual blocks out of such log messages. Currently, py.log merely transmits embedded newlines as is, so the output is a bit unclear about if a line is part of the log message, or not. A multi-line logged message might neatly insert a white margin to the left of all lines except the first, with the margin length being equal to the size of the prefix (shown between square brackets) found on the first line. This (different prefix, different margins) would be more legible, I think, than a fixed margin size -- yet we might theoretically imagine bad effects from very long prefixes, all unlikely that they can be. -- François Pinard http://pinard.progiciels-bpi.ca
Hi, [Francois Pinard]
[holger krekel]
Actually the logging API is still quite experimental so your feedback is most welcome.
You did open the door! :-)
I fetched py-dev@codespeak.net archives a few hours ago, but it will take some time before I become acquainted with them, so I may raise issues which have already been discussed, I do not know yet.
Thanks for taking care.
[...] But for the time being, i'd suggest just forgetting about py.log.get() and sticking to the more documented approach.
OK. In any case, the code base for our big application is likely to be boiling at least for the next six months or so, so changes are always possible on our side for quite a while, without being hurtful.
Good. There also is a growing number of pypy-usages of the log functionality but doing changes there should not be a big problem either.
Let me start the discussion on the following issue.
It is a common and useful idiom, where I work most of the time, to pass a ``write`` argument to various functions, letting the caller decide what ``write`` will do in any particular case. It could be an opened file, the input side of a service process, the ``append`` method of a list, or God knows what else. It would be convenient being able to merely pass a logger for that ``write`` argument, hoping this would yield a sensible behaviour. This comes with problems, however.
I think it's a very valid concern. So there should be some code path that makes this easy.
Currently, py.log and the standard logging module both provide a newline after the text to be logged. If I provide a newline at the end of a text given to a py.log logger, this will create a spurious white line. Whether a logger should remove trailing newlines or not, is debatable. For one, I would be willing and ready to always provide the trailing newline in my calls, given I expect the logger to add none, so logging functions and ``write`` functions are more fully interchangeable. However, tradition is that logging functions turn incomplete lines into complete lines. I do not know where lies the greater wisdom...
Maybe we could add e.g. "autocr=True|False" somewhere to make this configurable.
Another point, which is pretty factual in our code base, is that the application often logs multi-line texts, which are then (of course) displayed on many lines. I really dislike runs of stars or hyphens for creating visual blocks out of such log messages.
I share that dislike.
Currently, py.log merely transmits embedded newlines as is, so the output is a bit unclear about if a line is part of the log message, or not. A multi-line logged message might neatly insert a white margin to the left of all lines except the first, with the margin length being equal to the size of the prefix (shown between square brackets) found on the first line. This (different prefix, different margins) would be more legible, I think, than a fixed margin size -- yet we might theoretically imagine bad effects from very long prefixes, all unlikely that they can be.
The suggestion makes sense (inserting " " * len(prefix) after newlines in messages). Actually I would like to sit together with you for some hours sorting out usages/scenarios and designing the API accordingly. I also have a couple of more ideas of how py.log should fit with other usages like sending things remotely over py.execnet channels. Just to mention it already: i am thinking of doing a py lib hacking/sprinting session around Pycon US 2006. For the time being, I'd suggest that you get commit rights and we work commonly on some (non-main-page linked for now) feature descriptions/api scenarios in py/documentation/log.txt to sort out the goals. Implementing it is less of a deal, i think and we can first do it by introducing py.log2 until the API is settled/tested by real code. If you agree just send me your account name/ssh-key. Maybe Grig is also interested in participating in that effort. cheers, holger
Hi, Francois and Holger (have been pretty busy during the weekend, so I'm only now reading this thread...) --- holger krekel <hpk@trillke.net> wrote:
Hi,
[Francois Pinard]
[holger krekel]
Actually the logging API is still quite experimental so your feedback is most welcome.
You did open the door! :-)
I fetched py-dev@codespeak.net archives a few hours ago, but it will take some time before I become acquainted with them, so I may raise issues which have already been discussed, I do not know yet.
Thanks for taking care.
[...] But for the time being, i'd suggest just forgetting about py.log.get() and sticking to the more documented approach.
OK. In any case, the code base for our big application is likely to be boiling at least for the next six months or so, so changes are always possible on our side for quite a while, without being hurtful.
Good. There also is a growing number of pypy-usages of the log functionality but doing changes there should not be a big problem either.
Let me start the discussion on the following issue.
It is a common and useful idiom, where I work most of the time, to pass a ``write`` argument to various functions, letting the caller decide what ``write`` will do in any particular case. It could be an opened file, the input side of a service process, the ``append`` method of a list, or God knows what else. It would be convenient being able to merely pass a logger for that ``write`` argument, hoping this would yield a sensible behaviour. This comes with problems, however.
I think it's a very valid concern. So there should be some code path
that makes this easy.
Currently, py.log and the standard logging module both provide a newline after the text to be logged. If I provide a newline at the end of a text given to a py.log logger, this will create a spurious white line. Whether a logger should remove trailing newlines or not, is debatable. For one, I would be willing and ready to always provide the trailing newline in my calls, given I expect the logger to add none, so logging functions and ``write`` functions are more fully interchangeable. However, tradition is that logging functions turn incomplete lines into complete lines. I do not know where lies the greater wisdom...
Maybe we could add e.g. "autocr=True|False" somewhere to make this configurable.
This should indeed be pretty easy to implement.
Another point, which is pretty factual in our code base, is that the application often logs multi-line texts, which are then (of course) displayed on many lines. I really dislike runs of stars or hyphens for creating visual blocks out of such log messages.
I share that dislike.
Currently, py.log merely transmits embedded newlines as is, so the output is a bit unclear about if a line is part of the log message, or not. A multi-line logged message might neatly insert a white margin to the left of all lines except the first, with the margin length being equal to the size of the prefix (shown between square brackets) found on the first line. This (different prefix, different margins) would be more legible, I think, than a fixed margin size -- yet we might theoretically imagine bad effects from very long prefixes, all unlikely that they can be.
The suggestion makes sense (inserting " " * len(prefix) after newlines in messages).
I agree.
Actually I would like to sit together with you for some hours sorting out usages/scenarios and designing the API accordingly. I also have a couple of more ideas of how py.log should fit with other usages like sending things remotely over py.execnet channels.
I think the more usage scenarios we have, the better the py.log API will get. Maybe we can chat on IRC one day.
Just to mention it already: i am thinking of doing a py lib hacking/sprinting session around Pycon US 2006.
For the time being, I'd suggest that you get commit rights and we work commonly on some (non-main-page linked for now) feature descriptions/api scenarios in py/documentation/log.txt to sort out the goals. Implementing it is less of a deal, i think and we can first do it by introducing py.log2 until the API is settled/tested by real code. If you agree just send me your account name/ssh-key. Maybe Grig is also interested in participating in that effort.
I am very interested in participating. Thanks for the interest, Francois, we need more people like you who want to use py.log in their projects :-) Grig
Grig Gheorghiu wrote:
I am very interested in participating. Thanks for the interest, Francois, we need more people like you who want to use py.log in their projects :-)
Some Schevo folks might be interested in participating as well... We need a good logging mechanism for Schevo, but also need to minimize our dependencies and make sure anything we do use will work well with setuptools and eggs. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org
--- "Patrick K. O'Brien" <pobrien@orbtech.com> wrote:
I am very interested in participating. Thanks for the interest, Francois, we need more people like you who want to use py.log in
Grig Gheorghiu wrote: their
projects :-)
Some Schevo folks might be interested in participating as well...
We need a good logging mechanism for Schevo, but also need to minimize our dependencies and make sure anything we do use will work well with setuptools and eggs.
That's great to hear. Maybe Holger can coordinate some group discussions. We can also start by people writing down their wishlist for py.log and then prioritize the features we want to implement. Grig
On Mon, Oct 03, 2005 at 07:20 -0700, Grig Gheorghiu wrote:
--- "Patrick K. O'Brien" <pobrien@orbtech.com> wrote:
I am very interested in participating. Thanks for the interest, Francois, we need more people like you who want to use py.log in
Grig Gheorghiu wrote: their
projects :-)
Some Schevo folks might be interested in participating as well...
We need a good logging mechanism for Schevo, but also need to minimize our dependencies and make sure anything we do use will work well with setuptools and eggs.
That's great to hear. Maybe Holger can coordinate some group discussions. We can also start by people writing down their wishlist for py.log and then prioritize the features we want to implement.
Good idea. I can try that but i am leaving for Paris (for a PyPy sprint plus other meetings) tomorrow and will be on the road until 5th November. So it's not yet clear for me when i would get to it. OTOH, there is no big hurry so i hope i can work on logging - among other things - from my work-holiday flat in Marroco second half October :) This should not keep us from writing down thoughts and ideas sooner already. cheers, holger
On 03/10/05, holger krekel <hpk@trillke.net> wrote:
Just to mention it already: i am thinking of doing a py lib hacking/sprinting session around Pycon US 2006.
That would be fabulous. I'll likely be able to go to PyCon again, and hacking on py lib would be a great sprint. ;) Take care, -Brian
[holger krekel]
Actually I would like to sit together with you for some hours sorting out usages/scenarios and designing the API accordingly.
I spent the afternoon with the application designer, staring at the logging calls within only one of our numerous Python modules. It looks like the mere inventory of logging cases is going to be a big job. So, I'm not ready to give you a complete picture of the problem here. On the other hand, even an incomplete picture could be useful to shake a few ideas, if it does not cast decisions in stone prematurely.
Just to mention it already: i am thinking of doing a py lib hacking/sprinting session around Pycon US 2006.
I'm of course willing to sit with you, and anybody else interested. Presumably each of us at our terminal, as I'm not much of a traveller, and I also guess you do not live in Montreal or suburbs :-). You're thinking of IRC, probably?
we work commonly on some (non-main-page linked for now) feature descriptions/api scenarios in py/documentation/log.txt to sort out the goals.
Sounds good.
Implementing it is less of a deal, i think and we can first do it by introducing py.log2 until the API is settled/tested by real code.
Besides, py.log is currently not such a big thing. It likely have enough flexibility already that I can make things work for us here, before they really get available in the mainstream. I'm ready to adapt.
Maybe Grig is also interested in participating in that effort.
I surely thank him for having provided a visible pointer to py.log! The fun thing is that we started using py.test in that project not so long ago (the administrators accepted that I use experimental software), so now pushing for py.log was much easier, given the proximity! :-) -- François Pinard http://pinard.progiciels-bpi.ca
On Mon, Oct 03, 2005 at 13:39 -0700, Brian Dorsey wrote:
On 03/10/05, holger krekel <hpk@trillke.net> wrote:
Just to mention it already: i am thinking of doing a py lib hacking/sprinting session around Pycon US 2006.
That would be fabulous. I'll likely be able to go to PyCon again, and hacking on py lib would be a great sprint. ;)
hey, i just re-looked at this mail ... see my other mail to py-dev ... i am looking into doing a post-pycon sprint on the py lib ... what do you think? cheers & hope everything is going well on your side! holger
--- holger krekel <hpk@trillke.net> wrote:
On Mon, Oct 03, 2005 at 13:39 -0700, Brian Dorsey wrote:
On 03/10/05, holger krekel <hpk@trillke.net> wrote:
Just to mention it already: i am thinking of doing a py lib hacking/sprinting session around Pycon US 2006.
That would be fabulous. I'll likely be able to go to PyCon again, and hacking on py lib would be a great sprint. ;)
hey, i just re-looked at this mail ... see my other mail to py-dev ... i am looking into doing a post-pycon sprint on the py lib ... what do you think?
cheers & hope everything is going well on your side!
holger _______________________________________________ py-dev mailing list py-dev@codespeak.net http://codespeak.net/mailman/listinfo/py-dev
Unfortunately I won't be able to stay for the sprint, but we can always go to a pub during the conference to do some serious py.lib brainstorming! (with whatever brain is left :-) Grig
participants (6)
-
Brian Dorsey -
François Pinard -
Grig Gheorghiu -
holger krekel -
hpk＠trillke.net
-
Patrick K. O'Brien