From loris.bennett at fu-berlin.de  Fri Nov  1 02:52:32 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Fri, 01 Nov 2024 07:52:32 +0100
Subject: Printing UTF-8 mail to terminal
References: <878qu49tii.fsf@zedat.fu-berlin.de>
 <CAJQBtgmwNbjYNr-LWYCia-9+CoRzaLj22YxzyP_EhwSspRD8_g@mail.gmail.com>
 <mailman.61.1730392745.4695.python-list@python.org>
Message-ID: <87v7x7o37z.fsf@zedat.fu-berlin.de>

Left Right <olegsivokon at gmail.com> writes:

> There's quite a lot of misuse of terminology around terminal / console
> / shell.  Please, correct me if I'm wrong, but it looks like you are
> printing that on MS Windows, right?  MS Windows doesn't have or use
> terminals (that's more of a Unix-related concept). And, by "terminal"
> I mean terminal emulator (i.e. a program that emulates the behavior of
> a physical terminal). You can, of course, find some terminal programs
> for windows (eg. mintty), but I doubt that that's what you are dealing
> with.
>
> What MS Windows users usually end up using is the console.  If you
> run, eg. cmd.exe, it will create a process that displays a graphical
> console.  The console uses an encoding scheme to represent the text
> output.  I believe that the default on MS Windows is to use some
> single-byte encoding. This answer from SE family site tells you how to
> set the console encoding to UTF-8 permanently:
> https://superuser.com/questions/269818/change-default-code-page-of-windows-console-to-utf-8
> , which, I believe, will solve your problem with how the text is
> displayed.

I'm not using MS Windows.  I am using a Gnome terminal on Debian 12
locally and connecting via SSH to a AlmaLinux 8 server, where I start a
tmux session. 

> On Thu, Oct 31, 2024 at 5:19?PM Loris Bennett via Python-list
> <python-list at python.org> wrote:
>>
>> Hi,
>>
>> I have a command-line program which creates an email containing German
>> umlauts.  On receiving the mail, my mail client displays the subject and
>> body correctly:
>>
>>   Subject: ?bung
>>
>>   Sehr geehrter Herr Dr. Bennett,
>>
>>   Dies ist eine ?bung.
>>
>> So far, so good.  However, when I use the --verbose option to print
>> the mail to the terminal via
>>
>>   if args.verbose:
>>       print(mail)
>>
>> I get:
>>
>>   Subject: ?bungsbetreff
>>
>>   Sehr geehrter Herr Dr. Bennett,
>>
>>   Dies ist eine =C3=9Cbung.
>>
>> What do I need to do to prevent the body from getting mangled?
>>
>> I seem to remember that I had issues in the past with a Perl version of
>> a similar program.  As far as I recall there was an issue with fact the
>> greeting is generated by querying a server, whereas the body is being
>> read from a file, which lead to oddities when the two bits were
>> concatenated.  But that might just have been a Perl thing.
>>
>> Cheers,
>>
>> Loris
>>
>> --
>> This signature is currently under constuction.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
-- 
Dr. Loris Bennett (Herr/Mr)
FUB-IT, Freie Universit?t Berlin

From loris.bennett at fu-berlin.de  Fri Nov  1 03:11:30 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Fri, 01 Nov 2024 08:11:30 +0100
Subject: Printing UTF-8 mail to terminal
References: <878qu49tii.fsf@zedat.fu-berlin.de>
 <ZyPtsLSme7IJ-q4j@cskk.homeip.net>
 <mailman.63.1730408232.4695.python-list@python.org>
Message-ID: <87msijo2cd.fsf@zedat.fu-berlin.de>

Cameron Simpson <cs at cskk.id.au> writes:

> On 31Oct2024 16:33, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>>I have a command-line program which creates an email containing German
>>umlauts.  On receiving the mail, my mail client displays the subject and
>>body correctly:
> [...]
>>So far, so good.  However, when I use the --verbose option to print
>>the mail to the terminal via
>>
>>  if args.verbose:
>>      print(mail)
>>
>>I get:
>>
>>  Subject: ?bungsbetreff
>>
>>  Sehr geehrter Herr Dr. Bennett,
>>
>>  Dies ist eine =C3=9Cbung.
>>
>>What do I need to do to prevent the body from getting mangled?
>
> That looks to me like quoted-printable. This is an encoding for binary
> transport of text to make it robust against not 8-buit clean
> transports.  So your Unicode text is encodings as UTF-8, and then that
> is encoded in quoted-printable for transport through the email system.

As I mentioned, I think the problem is to do with the way the salutation
text provided by the "salutation server" and the mail body from a file
are encoded.  This seems to be different.  

> Your terminal probably accepts UTF-8 - I imagine other German text
> renders corectly?

Yes, it does.

> You need to get the text and undo the quoted-printable encoding.
>
> If you're using the Python email module to parse (or construct) the
> message as a `Message` object I'd expect that to happen automatically.

I am using

  email.message.EmailMessage

as, from the Python documentation

  https://docs.python.org/3/library/email.examples.html

I gathered that that is the standard approach.

And you are right that encoding for the actual mail which is received is
automatically sorted out.  If I display the raw email in my client I get
the following:

  Content-Type: text/plain; charset="utf-8"
  Content-Transfer-Encoding: quoted-printable
  ...
  Subject: =?utf-8?q?=C3=9Cbungsbetreff?=
  ...
  Dies ist eine =C3=9Cbung.

I would interpret that as meaning that the subject and body are encoded
in the same way.

The problem just occurs with the unsent string representation printed to
the terminal.

Cheers,

Loris

-- 
This signature is currently under constuction.

From loris.bennett at fu-berlin.de  Fri Nov  1 05:10:03 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Fri, 01 Nov 2024 10:10:03 +0100
Subject: Printing UTF-8 mail to terminal
References: <878qu49tii.fsf@zedat.fu-berlin.de>
 <ZyPtsLSme7IJ-q4j@cskk.homeip.net>
 <mailman.63.1730408232.4695.python-list@python.org>
 <87msijo2cd.fsf@zedat.fu-berlin.de>
Message-ID: <875xp7nwus.fsf@zedat.fu-berlin.de>

"Loris Bennett" <loris.bennett at fu-berlin.de> writes:

> Cameron Simpson <cs at cskk.id.au> writes:
>
>> On 31Oct2024 16:33, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>>>I have a command-line program which creates an email containing German
>>>umlauts.  On receiving the mail, my mail client displays the subject and
>>>body correctly:
>> [...]
>>>So far, so good.  However, when I use the --verbose option to print
>>>the mail to the terminal via
>>>
>>>  if args.verbose:
>>>      print(mail)
>>>
>>>I get:
>>>
>>>  Subject: ?bungsbetreff
>>>
>>>  Sehr geehrter Herr Dr. Bennett,
>>>
>>>  Dies ist eine =C3=9Cbung.
>>>
>>>What do I need to do to prevent the body from getting mangled?
>>
>> That looks to me like quoted-printable. This is an encoding for binary
>> transport of text to make it robust against not 8-buit clean
>> transports.  So your Unicode text is encodings as UTF-8, and then that
>> is encoded in quoted-printable for transport through the email system.
>
> As I mentioned, I think the problem is to do with the way the salutation
> text provided by the "salutation server" and the mail body from a file
> are encoded.  This seems to be different.  
>
>> Your terminal probably accepts UTF-8 - I imagine other German text
>> renders corectly?
>
> Yes, it does.
>
>> You need to get the text and undo the quoted-printable encoding.
>>
>> If you're using the Python email module to parse (or construct) the
>> message as a `Message` object I'd expect that to happen automatically.
>
> I am using
>
>   email.message.EmailMessage
>
> as, from the Python documentation
>
>   https://docs.python.org/3/library/email.examples.html
>
> I gathered that that is the standard approach.
>
> And you are right that encoding for the actual mail which is received is
> automatically sorted out.  If I display the raw email in my client I get
> the following:
>
>   Content-Type: text/plain; charset="utf-8"
>   Content-Transfer-Encoding: quoted-printable
>   ...
>   Subject: =?utf-8?q?=C3=9Cbungsbetreff?=
>   ...
>   Dies ist eine =C3=9Cbung.
>
> I would interpret that as meaning that the subject and body are encoded
> in the same way.
>
> The problem just occurs with the unsent string representation printed to
> the terminal.

If I log the body like this

  body = f"{salutation},\n\n{text}\n{signature}"
  logger.debug("body: " + body)
 
and look at the log file in my terminal I see 

  2024-11-01 09:59:12,318 - DEBUG - mailer:create_body - body: Sehr geehrter Herr Dr. Bennett,

  Dies ist eine ?bung.
 
  ...

as expected.  The non-UTF-8 text occurs when I do

  mail = EmailMessage()
  mail.set_content(body, cte="quoted-printable")
  ...

  if args.verbose:   
      print(mail)

which is presumably also correct.

The question is: What conversion is necessary in order to print the
EmailMessage object to the terminal, such that the quoted-printable
parts are turned (back) into UTF-8?

Cheers,

Loris

-- 
This signature is currently under constuction.

From dieter.maurer at online.de  Fri Nov  1 12:32:04 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Fri, 1 Nov 2024 17:32:04 +0100
Subject: Correct module for site customization of path
In-Reply-To: <adfac553-cac7-4441-b70a-9197723279d8@gmail.com>
References: <adfac553-cac7-4441-b70a-9197723279d8@gmail.com>
Message-ID: <26405.644.968594.532579@ixdm.fritz.box>

> ...
>After the recent upgrades I had to install youtube_dl with pipx for the
>new python version.
>When I ran the script which imported youtube_dl, I got an import error
>as it appears the path to the module
>was not in sys.path....

I see at several options:

 * install `youtoube_dl` where Python looks for it
   (you can print `sys.path` to find out all the places
   normally checked for importable modules)

 * put a (symbolic) link to `youtoube_dl` at a place
   where Python looks for modules

 * use Pythons' `.pth' feature to tell Python additional
   places where to look for modules.
   You can place `.pth` files where Python looks for modules
   to be imported

From dieter.maurer at online.de  Fri Nov  1 12:38:01 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Fri, 1 Nov 2024 17:38:01 +0100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <875xp7nwus.fsf@zedat.fu-berlin.de>
References: <878qu49tii.fsf@zedat.fu-berlin.de>
 <ZyPtsLSme7IJ-q4j@cskk.homeip.net>
 <mailman.63.1730408232.4695.python-list@python.org>
 <87msijo2cd.fsf@zedat.fu-berlin.de>
 <875xp7nwus.fsf@zedat.fu-berlin.de>
Message-ID: <26405.1001.772245.235696@ixdm.fritz.box>

Loris Bennett wrote at 2024-11-1 10:10 +0100:
> ...
>  mail.set_content(body, cte="quoted-printable")

In the line above, you request the content to use
the "cte" (= "Content-Transfer-Encoding") "quoted-printable"
and consequently, the content is encoded with `quoted-printable`.
Maybe, you do not need to pass `cte`?

From cs at cskk.id.au  Fri Nov  1 17:44:18 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 2 Nov 2024 08:44:18 +1100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <87msijo2cd.fsf@zedat.fu-berlin.de>
References: <87msijo2cd.fsf@zedat.fu-berlin.de>
Message-ID: <ZyVLsn2l9jJ2Fikl@cskk.homeip.net>

On 01Nov2024 08:11, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>Cameron Simpson <cs at cskk.id.au> writes:
>> If you're using the Python email module to parse (or construct) the
>> message as a `Message` object I'd expect that to happen automatically.
>
>I am using
>  email.message.EmailMessage

Noted. That seems like the correct approach to me.

>And you are right that encoding for the actual mail which is received 
>is
>automatically sorted out.  If I display the raw email in my client I get
>the following:
>
>  Content-Type: text/plain; charset="utf-8"
>  Content-Transfer-Encoding: quoted-printable
>  ...
>  Subject: =?utf-8?q?=C3=9Cbungsbetreff?=
>  ...
>  Dies ist eine =C3=9Cbung.

Right. Quoted-printable encoding for the transport.

>I would interpret that as meaning that the subject and body are encoded
>in the same way.

Yes.

>The problem just occurs with the unsent string representation printed to
>the terminal.

Yes, and I was thinking abut this yesterday. I suspect that 
`print(some_message_object)` is intended to transcribe it for transport.  
For example, one could write to an mbox file and just print() the 
message into it and get correct transport/storage formatting, which 
includes the qp encoding.

Can you should the code (or example code) which leads to the qp output?  
I suspect there's a straight forward way to get the decoded Unicode, but 
I'd need to see how what you've got was obtained.

From cs at cskk.id.au  Fri Nov  1 17:47:39 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 2 Nov 2024 08:47:39 +1100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <875xp7nwus.fsf@zedat.fu-berlin.de>
References: <875xp7nwus.fsf@zedat.fu-berlin.de>
Message-ID: <ZyVMe3Jspc0fJrel@cskk.homeip.net>

On 01Nov2024 10:10, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>as expected.  The non-UTF-8 text occurs when I do
>
>  mail = EmailMessage()
>  mail.set_content(body, cte="quoted-printable")
>  ...
>
>  if args.verbose:
>      print(mail)
>
>which is presumably also correct.
>
>The question is: What conversion is necessary in order to print the
>EmailMessage object to the terminal, such that the quoted-printable
>parts are turned (back) into UTF-8?

Do you still have access to `body` ? That would be the original message 
text? Otherwise maybe:

     print(mail.get_content())

The objective is to obtain the message body Unicode text (i.e. a regular 
Python string with the original text, unencoded). And to print that.

From cs at cskk.id.au  Fri Nov  1 17:52:22 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 2 Nov 2024 08:52:22 +1100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <9f754662-4b73-4d3d-a9b8-ac0a40762143@yahoo.co.uk>
References: <9f754662-4b73-4d3d-a9b8-ac0a40762143@yahoo.co.uk>
Message-ID: <ZyVNlhl5Ee6Iq1Xj@cskk.homeip.net>

On 31Oct2024 21:53, alan.gauld at yahoo.co.uk <alan.gauld at yahoo.co.uk> wrote:
>On 31/10/2024 20:50, Cameron Simpson via Python-list wrote:
>> If you're just dealing with this directly, use the `quopri` stdlib
>> module: https://docs.python.org/3/library/quopri.html
>
>One of the things I love about this list are these little features
>that I didn't know existed. Despite having used Python for over 25
>years, I've never noticed that module before! :-)

Heh. And James Parrott caused me to discover the 
`subprocess.run(executable_path)` mode of `run()/Popen()`: a string with 
`shell=False` (the default) is an executable name.

From barry at barrys-emacs.org  Fri Nov  1 18:10:43 2024
From: barry at barrys-emacs.org (Barry)
Date: Fri, 1 Nov 2024 22:10:43 +0000
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <CAJQBtgmwNbjYNr-LWYCia-9+CoRzaLj22YxzyP_EhwSspRD8_g@mail.gmail.com>
References: <CAJQBtgmwNbjYNr-LWYCia-9+CoRzaLj22YxzyP_EhwSspRD8_g@mail.gmail.com>
Message-ID: <5D77331F-91F4-4000-BF82-DB7A8EA7CD4F@barrys-emacs.org>



> On 31 Oct 2024, at 16:42, Left Right via Python-list <python-list at python.org> wrote:
> 
> MS Windows doesn't have or use
> terminals (that's more of a Unix-related concept).

Windows does now. They implemented this feature over the last few years.
Indeed they took inspiration from how linux does this.

You might find https://devblogs.microsoft.com/commandline/ has interesting articles about this.

They also have implemented utf-8 as code page 65001.

Barry



From olegsivokon at gmail.com  Fri Nov  1 18:57:00 2024
From: olegsivokon at gmail.com (Left Right)
Date: Fri, 1 Nov 2024 23:57:00 +0100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <5D77331F-91F4-4000-BF82-DB7A8EA7CD4F@barrys-emacs.org>
References: <CAJQBtgmwNbjYNr-LWYCia-9+CoRzaLj22YxzyP_EhwSspRD8_g@mail.gmail.com>
 <5D77331F-91F4-4000-BF82-DB7A8EA7CD4F@barrys-emacs.org>
Message-ID: <CAJQBtgmQbJf8CDLx3Ty0DkqAU7LDqC_5zO87Lrz0yOOuOdGAsQ@mail.gmail.com>

> Windows does now. They implemented this feature over the last few years.
> Indeed they took inspiration from how linux does this.
>
> You might find https://devblogs.microsoft.com/commandline/ has interesting articles about this.

I don't have MS Windows. My wife does, but I don't want to bother her
with this kind of testing. Does this Windows Terminal support the use
of programs like tmux? Last time I searched for a way to run tmux on
Windows, best I could find was mintty.  None of the "native" MS
"solutions" couldn't do it.

Anyways, OP said they were using an actual terminal (emulator) on
Ubuntu, and it looks like their problem is more with extracting
information from the email message rather than with the terminal
capabilities. Also, looks like there was an answer already wrt.
message.get_body()

From thjmmj15 at gmail.com  Fri Nov  1 20:50:12 2024
From: thjmmj15 at gmail.com (Tim Johnson)
Date: Fri, 1 Nov 2024 16:50:12 -0800
Subject: Correct module for site customization of path
In-Reply-To: <26405.644.968594.532579@ixdm.fritz.box>
References: <adfac553-cac7-4441-b70a-9197723279d8@gmail.com>
 <26405.644.968594.532579@ixdm.fritz.box>
Message-ID: <5e2ee6d2-ee0d-4dbe-b6ab-ad640bce4a17@gmail.com>

On 11/1/24 08:32, dieter.maurer at online.de wrote:
>> ...
>> After the recent upgrades I had to install youtube_dl with pipx for the
>> new python version.
>> When I ran the script which imported youtube_dl, I got an import error
>> as it appears the path to the module
>> was not in sys.path....
> I see at several options:
>
>   * install `youtoube_dl` where Python looks for it
>     (you can print `sys.path` to find out all the places
>     normally checked for importable modules)
>
>   * put a (symbolic) link to `youtoube_dl` at a place
>     where Python looks for modules
>
>   * use Pythons' `.pth' feature to tell Python additional
>     places where to look for modules.
>     You can place `.pth` files where Python looks for modules
>     to be imported
In? /usr/lib/python3.12/sitecustomize.py I put the following:
import sys
sys.path.append("path to youtube_dl")
# In /usr/lib/python3.12/sitecustomize.py
# Thanks seems to work for me
cheers


-- 
Tim
thjmmj15 at gmail.com


From barry at barrys-emacs.org  Sat Nov  2 09:19:40 2024
From: barry at barrys-emacs.org (Barry)
Date: Sat, 2 Nov 2024 13:19:40 +0000
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <CAJQBtgmQbJf8CDLx3Ty0DkqAU7LDqC_5zO87Lrz0yOOuOdGAsQ@mail.gmail.com>
References: <CAJQBtgmQbJf8CDLx3Ty0DkqAU7LDqC_5zO87Lrz0yOOuOdGAsQ@mail.gmail.com>
Message-ID: <7B2B2A31-4F79-4333-A941-71727924694D@barrys-emacs.org>



> On 1 Nov 2024, at 22:57, Left Right <olegsivokon at gmail.com> wrote:
> 
> Does this Windows Terminal support the use
> of programs like tmux?

I have not tried, but should work.

Best to install the terminal app from the MS app store.
Most use I make is to ssh into linux systems and stuff like editors.
Colour output and cursor movement all work.

 Barry


From * at eli.users.panix.com  Fri Nov  1 16:55:20 2024
From: * at eli.users.panix.com (Eli the Bearded)
Date: Fri, 1 Nov 2024 20:55:20 -0000 (UTC)
Subject: Printing UTF-8 mail to terminal
References: <9f754662-4b73-4d3d-a9b8-ac0a40762143@yahoo.co.uk>
 <mailman.64.1730411614.4695.python-list@python.org>
 <cIaVO.1746331$v0Jd.1554563@fx06.ams4>
Message-ID: <eli$2411011655@qaz.wtf>

In comp.lang.python, Gilmeh Serda  <gilmeh.serda at nothing.here.invalid> wrote:
> Python 3.12.6 (main, Sep  8 2024, 13:18:56) [GCC 14.2.1 20240805] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> help('modules')
> 
> Please wait a moment while I gather a list of all available modules...
> 
> AssemblyApp         apparmor            io                  pyzipper
> AssemblyGui         appdirs             ipaddress           qrtools
> CAMSimulator        application_utility isodate             queue
> Cheetah             apprise             isort               quopri
> [...]
> """
> 
> Put it in a list, unmangle it, sort it and you should have an alphabetical 
> list of all modules on your system.

As someone who has done a lot of work with email in other languages,
"quopri" is not a name I'd expect or look for first pass for dealing
with MIME quoted-printable encoding. (Me, being me, I'd probably just
write it for myself if I didn't quickly find it while working with
email.)

Elijah
------
MIME: multipurpose Internet mail extensions

From jon+usenet at unequivocal.eu  Fri Nov  1 17:05:54 2024
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Fri, 1 Nov 2024 21:05:54 -0000 (UTC)
Subject: Printing UTF-8 mail to terminal
References: <9f754662-4b73-4d3d-a9b8-ac0a40762143@yahoo.co.uk>
 <mailman.64.1730411614.4695.python-list@python.org>
 <cIaVO.1746331$v0Jd.1554563@fx06.ams4> <eli$2411011655@qaz.wtf>
Message-ID: <slrnviagli.5qt.jon+usenet@raven.unequivocal.eu>

On 2024-11-01, Eli the Bearded <*@eli.users.panix.com> wrote:
> In comp.lang.python, Gilmeh Serda  <gilmeh.serda at nothing.here.invalid> wrote:
>> Python 3.12.6 (main, Sep  8 2024, 13:18:56) [GCC 14.2.1 20240805] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> help('modules')
>> 
>> Please wait a moment while I gather a list of all available modules...
>> 
>> AssemblyApp         apparmor            io                  pyzipper
>> AssemblyGui         appdirs             ipaddress           qrtools
>> CAMSimulator        application_utility isodate             queue
>> Cheetah             apprise             isort               quopri
>> [...]
>> """
>> 
>> Put it in a list, unmangle it, sort it and you should have an alphabetical 
>> list of all modules on your system.
>
> As someone who has done a lot of work with email in other languages,
> "quopri" is not a name I'd expect or look for first pass for dealing
> with MIME quoted-printable encoding. (Me, being me, I'd probably just
> write it for myself if I didn't quickly find it while working with
> email.)

Python went through a period of time where lots of things just got stuck
in the standard library without any particula taxonomy. Hence ending up
with base64, binascii, binhex, quopri, and uu all being separate
top-level modules, only some of which got tidied up in Python 3.

From songofacandy at gmail.com  Sat Nov  2 23:08:41 2024
From: songofacandy at gmail.com (Inada Naoki)
Date: Sun, 3 Nov 2024 12:08:41 +0900
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <87v7x7o37z.fsf@zedat.fu-berlin.de>
References: <878qu49tii.fsf@zedat.fu-berlin.de>
 <CAJQBtgmwNbjYNr-LWYCia-9+CoRzaLj22YxzyP_EhwSspRD8_g@mail.gmail.com>
 <mailman.61.1730392745.4695.python-list@python.org>
 <87v7x7o37z.fsf@zedat.fu-berlin.de>
Message-ID: <CAEfz+Ty9LNyHWzouvxxSxmtjQJ=zGhBWBxq3NdH=bJLACBpmBg@mail.gmail.com>

Try PYTHONUTF8=1 envver.

2024?11?2?(?) 0:36 Loris Bennett via Python-list <python-list at python.org>:

> Left Right <olegsivokon at gmail.com> writes:
>
> > There's quite a lot of misuse of terminology around terminal / console
> > / shell.  Please, correct me if I'm wrong, but it looks like you are
> > printing that on MS Windows, right?  MS Windows doesn't have or use
> > terminals (that's more of a Unix-related concept). And, by "terminal"
> > I mean terminal emulator (i.e. a program that emulates the behavior of
> > a physical terminal). You can, of course, find some terminal programs
> > for windows (eg. mintty), but I doubt that that's what you are dealing
> > with.
> >
> > What MS Windows users usually end up using is the console.  If you
> > run, eg. cmd.exe, it will create a process that displays a graphical
> > console.  The console uses an encoding scheme to represent the text
> > output.  I believe that the default on MS Windows is to use some
> > single-byte encoding. This answer from SE family site tells you how to
> > set the console encoding to UTF-8 permanently:
> >
> https://superuser.com/questions/269818/change-default-code-page-of-windows-console-to-utf-8
> > , which, I believe, will solve your problem with how the text is
> > displayed.
>
> I'm not using MS Windows.  I am using a Gnome terminal on Debian 12
> locally and connecting via SSH to a AlmaLinux 8 server, where I start a
> tmux session.
>
> > On Thu, Oct 31, 2024 at 5:19?PM Loris Bennett via Python-list
> > <python-list at python.org> wrote:
> >>
> >> Hi,
> >>
> >> I have a command-line program which creates an email containing German
> >> umlauts.  On receiving the mail, my mail client displays the subject and
> >> body correctly:
> >>
> >>   Subject: ?bung
> >>
> >>   Sehr geehrter Herr Dr. Bennett,
> >>
> >>   Dies ist eine ?bung.
> >>
> >> So far, so good.  However, when I use the --verbose option to print
> >> the mail to the terminal via
> >>
> >>   if args.verbose:
> >>       print(mail)
> >>
> >> I get:
> >>
> >>   Subject: ?bungsbetreff
> >>
> >>   Sehr geehrter Herr Dr. Bennett,
> >>
> >>   Dies ist eine =C3=9Cbung.
> >>
> >> What do I need to do to prevent the body from getting mangled?
> >>
> >> I seem to remember that I had issues in the past with a Perl version of
> >> a similar program.  As far as I recall there was an issue with fact the
> >> greeting is generated by querying a server, whereas the body is being
> >> read from a file, which lead to oddities when the two bits were
> >> concatenated.  But that might just have been a Perl thing.
> >>
> >> Cheers,
> >>
> >> Loris
> >>
> >> --
> >> This signature is currently under constuction.
> >> --
> >> https://mail.python.org/mailman/listinfo/python-list
> --
> Dr. Loris Bennett (Herr/Mr)
> FUB-IT, Freie Universit?t Berlin
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From aotto1968 at t-online.de  Sun Nov  3 04:49:07 2024
From: aotto1968 at t-online.de (aotto1968)
Date: Sun, 3 Nov 2024 10:49:07 +0100
Subject: =?UTF-8?B?W0FOTl0gKHByZXZpZXcpIOKAnHB5bXNncXVl4oCdIGlzIHRoZSBjb25u?=
 =?UTF-8?Q?ection_between_Tcl_and_the_=E2=80=9CProgramming_Language_Micro-Ke?=
 =?UTF-8?B?cm5lbOKAnSAoUExNSyku?=
Message-ID: <vg7guj$a8p6$1@dont-email.me>

ANNOUNCEMENT

"pymsgque" is the project to integrate the Programming-Language-Micro-Kernel (*PLMK*) into *Python*.
Together with C, C++, Java, Ruby and Tcl, a growing language community is emerging that will combine *all* existing programming 
languages with *PLMK* technology in the future.

: http://thedev.nhi1.de/NHI1/main/

?pymkkernel? extends *Python* with a ?code integrator? so that a 3rd party library can be connected to *Python* without 
additional programming effort.

: http://thedev.nhi1.de/theKernel/main/index.htm

?pymqmsgque? extends *Python* with a ?code parallelizer? so that existing *Python* code can be parallelized and integrated into 
a multiprocessor environment.

: http://thedev.nhi1.de/theLink/main/index.htm


PERFORMANCE
-----------

: http://thedev.nhi1.de/theLink/main/md_docs_2main_2README__PERFORMANCE.htm#README_PERFORMANCE

*Python* has good performance, compared to *TclOO* and *Ruby*, *Python* is on the same level as *Ruby*.

             |   send     send     send     send    create    create    data     data
             |  NOTHING   END    CALLBACK   WAIT    PARENT    CHILD     BUS      BFL
             | -------- -------- -------- -------- --------- -------- -------- --------

  R: Tcl     |   332380   190834   120565    61112      132    23589    43077    42926
  R: Python  |   493313   315040   160869    75802      103    21982    68504    65800
  R: Ruby    |   436564   301587   165921    77032       52    16330    71040    63967


occasional updates to the project are available as screenshots on social media.

: Facebook -> https://www.facebook.com/profile.php?id=100069563501101

From loris.bennett at fu-berlin.de  Mon Nov  4 05:44:03 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 04 Nov 2024 11:44:03 +0100
Subject: Printing UTF-8 mail to terminal
References: <875xp7nwus.fsf@zedat.fu-berlin.de>
 <ZyVMe3Jspc0fJrel@cskk.homeip.net>
 <mailman.69.1730497664.4695.python-list@python.org>
Message-ID: <87ed3rmg7g.fsf@zedat.fu-berlin.de>

Cameron Simpson <cs at cskk.id.au> writes:

> On 01Nov2024 10:10, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>>as expected.  The non-UTF-8 text occurs when I do
>>
>>  mail = EmailMessage()
>>  mail.set_content(body, cte="quoted-printable")
>>  ...
>>
>>  if args.verbose:
>>      print(mail)
>>
>>which is presumably also correct.
>>
>>The question is: What conversion is necessary in order to print the
>>EmailMessage object to the terminal, such that the quoted-printable
>>parts are turned (back) into UTF-8?
>
> Do you still have access to `body` ? That would be the original
> message text? Otherwise maybe:
>
>     print(mail.get_content())
>
> The objective is to obtain the message body Unicode text (i.e. a
> regular Python string with the original text, unencoded). And to print
> that.

With the following:

######################################################################

import email.message

m = email.message.EmailMessage()

m['Subject'] = '?bung'

m.set_content('Dies ist eine ?bung')
print('== cte: default == \n')
print(m)

print('-- full mail ---')
print(m)
print('-- just content--')
print(m.get_content())

m.set_content('Dies ist eine ?bung', cte='quoted-printable')
print('== cte: quoted-printable ==\n')
print('-- full mail --')
print(m)
print('-- just content --')
print(m.get_content())

######################################################################

I get the following output:

######################################################################

== cte: default == 

Subject: ?bung
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0

RGllcyBpc3QgZWluZSDDnGJ1bmcK

-- full mail ---
Subject: ?bung
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0

RGllcyBpc3QgZWluZSDDnGJ1bmcK

-- just content--
Dies ist eine ?bung

== cte: quoted-printable ==

-- full mail --
Subject: ?bung
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Dies ist eine =C3=9Cbung

-- just content --
Dies ist eine ?bung

######################################################################

So in both cases the subject is fine, but it is unclear to me how to
print the body.  Or rather, I know how to print the body OK, but I don't
know how to print the headers separately - there seems to be nothing
like 'get_headers()'.  I can use 'get('Subject) etc. and reconstruct the
headers, but that seems a little clunky.  

Cheers,

Loris

-- 
This signature is currently under constuction.

From loris.bennett at fu-berlin.de  Mon Nov  4 05:48:15 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 04 Nov 2024 11:48:15 +0100
Subject: Printing UTF-8 mail to terminal
References: <878qu49tii.fsf@zedat.fu-berlin.de>
 <CAJQBtgmwNbjYNr-LWYCia-9+CoRzaLj22YxzyP_EhwSspRD8_g@mail.gmail.com>
 <mailman.61.1730392745.4695.python-list@python.org>
 <87v7x7o37z.fsf@zedat.fu-berlin.de>
 <CAEfz+Ty9LNyHWzouvxxSxmtjQJ=zGhBWBxq3NdH=bJLACBpmBg@mail.gmail.com>
 <mailman.75.1730603335.4695.python-list@python.org>
Message-ID: <87a5efmg0g.fsf@zedat.fu-berlin.de>

Inada Naoki <songofacandy at gmail.com> writes:

> 2024?11?2?(?) 0:36 Loris Bennett via Python-list <python-list at python.org>:
>
>> Left Right <olegsivokon at gmail.com> writes:
>>
>> > There's quite a lot of misuse of terminology around terminal / console
>> > / shell.  Please, correct me if I'm wrong, but it looks like you are
>> > printing that on MS Windows, right?  MS Windows doesn't have or use
>> > terminals (that's more of a Unix-related concept). And, by "terminal"
>> > I mean terminal emulator (i.e. a program that emulates the behavior of
>> > a physical terminal). You can, of course, find some terminal programs
>> > for windows (eg. mintty), but I doubt that that's what you are dealing
>> > with.
>> >
>> > What MS Windows users usually end up using is the console.  If you
>> > run, eg. cmd.exe, it will create a process that displays a graphical
>> > console.  The console uses an encoding scheme to represent the text
>> > output.  I believe that the default on MS Windows is to use some
>> > single-byte encoding. This answer from SE family site tells you how to
>> > set the console encoding to UTF-8 permanently:
>> >
>> https://superuser.com/questions/269818/change-default-code-page-of-windows-console-to-utf-8
>> > , which, I believe, will solve your problem with how the text is
>> > displayed.
>>
>> I'm not using MS Windows.  I am using a Gnome terminal on Debian 12
>> locally and connecting via SSH to a AlmaLinux 8 server, where I start a
>> tmux session.
>>
>> > On Thu, Oct 31, 2024 at 5:19?PM Loris Bennett via Python-list
>> > <python-list at python.org> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I have a command-line program which creates an email containing German
>> >> umlauts.  On receiving the mail, my mail client displays the subject and
>> >> body correctly:
>> >>
>> >>   Subject: ?bung
>> >>
>> >>   Sehr geehrter Herr Dr. Bennett,
>> >>
>> >>   Dies ist eine ?bung.
>> >>
>> >> So far, so good.  However, when I use the --verbose option to print
>> >> the mail to the terminal via
>> >>
>> >>   if args.verbose:
>> >>       print(mail)
>> >>
>> >> I get:
>> >>
>> >>   Subject: ?bungsbetreff
>> >>
>> >>   Sehr geehrter Herr Dr. Bennett,
>> >>
>> >>   Dies ist eine =C3=9Cbung.
>> >>
>> >> What do I need to do to prevent the body from getting mangled?
>> >>
>> >> I seem to remember that I had issues in the past with a Perl version of
>> >> a similar program.  As far as I recall there was an issue with fact the
>> >> greeting is generated by querying a server, whereas the body is being
>> >> read from a file, which lead to oddities when the two bits were
>> >> concatenated.  But that might just have been a Perl thing.
>> >>
>
> Try PYTHONUTF8=1 envver.
>

This does not seem to affect the way the email body is printed.

Cheers,

Loris

-- 
This signature is currently under constuction.

From loris.bennett at fu-berlin.de  Mon Nov  4 05:57:37 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 04 Nov 2024 11:57:37 +0100
Subject: Printing UTF-8 mail to terminal
References: <875xp7nwus.fsf@zedat.fu-berlin.de>
 <ZyVMe3Jspc0fJrel@cskk.homeip.net>
 <mailman.69.1730497664.4695.python-list@python.org>
 <87ed3rmg7g.fsf@zedat.fu-berlin.de>
Message-ID: <875xp3mfku.fsf@zedat.fu-berlin.de>

"Loris Bennett" <loris.bennett at fu-berlin.de> writes:

> Cameron Simpson <cs at cskk.id.au> writes:
>
>> On 01Nov2024 10:10, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>>>as expected.  The non-UTF-8 text occurs when I do
>>>
>>>  mail = EmailMessage()
>>>  mail.set_content(body, cte="quoted-printable")
>>>  ...
>>>
>>>  if args.verbose:
>>>      print(mail)
>>>
>>>which is presumably also correct.
>>>
>>>The question is: What conversion is necessary in order to print the
>>>EmailMessage object to the terminal, such that the quoted-printable
>>>parts are turned (back) into UTF-8?
>>
>> Do you still have access to `body` ? That would be the original
>> message text? Otherwise maybe:
>>
>>     print(mail.get_content())
>>
>> The objective is to obtain the message body Unicode text (i.e. a
>> regular Python string with the original text, unencoded). And to print
>> that.
>
> With the following:
>
> ######################################################################
>
> import email.message
>
> m = email.message.EmailMessage()
>
> m['Subject'] = '?bung'
>
> m.set_content('Dies ist eine ?bung')
> print('== cte: default == \n')
> print(m)
>
> print('-- full mail ---')
> print(m)
> print('-- just content--')
> print(m.get_content())
>
> m.set_content('Dies ist eine ?bung', cte='quoted-printable')
> print('== cte: quoted-printable ==\n')
> print('-- full mail --')
> print(m)
> print('-- just content --')
> print(m.get_content())
>
> ######################################################################
>
> I get the following output:
>
> ######################################################################
>
> == cte: default == 
>
> Subject: ?bung
> Content-Type: text/plain; charset="utf-8"
> Content-Transfer-Encoding: base64
> MIME-Version: 1.0
>
> RGllcyBpc3QgZWluZSDDnGJ1bmcK
>
> -- full mail ---
> Subject: ?bung
> Content-Type: text/plain; charset="utf-8"
> Content-Transfer-Encoding: base64
> MIME-Version: 1.0
>
> RGllcyBpc3QgZWluZSDDnGJ1bmcK
>
> -- just content--
> Dies ist eine ?bung
>
> == cte: quoted-printable ==
>
> -- full mail --
> Subject: ?bung
> MIME-Version: 1.0
> Content-Type: text/plain; charset="utf-8"
> Content-Transfer-Encoding: quoted-printable
>
> Dies ist eine =C3=9Cbung
>
> -- just content --
> Dies ist eine ?bung
>
> ######################################################################
>
> So in both cases the subject is fine, but it is unclear to me how to
> print the body.  Or rather, I know how to print the body OK, but I don't
> know how to print the headers separately - there seems to be nothing
> like 'get_headers()'.  I can use 'get('Subject) etc. and reconstruct the
> headers, but that seems a little clunky.  

Sorry, I am confusing the terminology here.  The 'body' seems to be the
headers plus the 'content'.  So I can print the *content* without the
headers OK, but I can't easily print all the headers separately.  If
just print the body, i.e. headers plus content, the umlauts in the
content are not resolved.

-- 
This signature is currently under constuction.

From ml at fam-goebel.de  Mon Nov  4 10:32:48 2024
From: ml at fam-goebel.de (Ulrich Goebel)
Date: Mon, 4 Nov 2024 16:32:48 +0100
Subject: TkInter Scrolled Listbox class?
Message-ID: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>

Hi,

I would like to build a class ScrolledListbox, which can be packed somewhere in ttk.Frames. What I did is to build not really a scrolled Listbox but a Frame containing a Listbox and a Scrollbar:

class FrameScrolledListbox(ttk.Frame):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        #
        # build Listbox and Scrollbar
        self.Listbox = tk.Listbox(self)
        self.Scrollbar = ttk.Scrollbar(self)
        #
        # configure these two
        self.Listbox.config(yscrollcommand=self.Scrollbar.set)
        self.Scrollbar.config(command=self.Listbox.yview)
        #
        # pack them in Frame
        self.Listbox.pack(side=tk.LEFT, fill=tk.BOTH)
        self.Scrollbar.pack(side=tk.RIGHT, fill=tk.BOTH)

That works, so instances of FrameScrolledListbox can be packed and the tk.Listbox itself is accessible via an attribute:

frmScrolledListbox = FrameScrolledListbox(main)
frmScrolledListbox.Listbox.config(...)

But it would be a bit nicer to get a class like

class ScrolledListbox(tk.Listbox):
    ...

So it would be used that way:

scrolledListbox = ScrolledListbox(main)
scrolledListbox.config(...)

Is that possible? The problem which I can't handle is to handle the Frame which seems to be needed to place the Scrollbar somewhere.

Best regards
Ulrich


-- 
Ulrich Goebel <ml at fam-goebel.de>

From loris.bennett at fu-berlin.de  Mon Nov  4 07:02:21 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 04 Nov 2024 13:02:21 +0100
Subject: Printing UTF-8 mail to terminal
References: <875xp7nwus.fsf@zedat.fu-berlin.de>
 <ZyVMe3Jspc0fJrel@cskk.homeip.net>
 <mailman.69.1730497664.4695.python-list@python.org>
 <87ed3rmg7g.fsf@zedat.fu-berlin.de>
 <875xp3mfku.fsf@zedat.fu-berlin.de>
Message-ID: <871pzrmcky.fsf@zedat.fu-berlin.de>

"Loris Bennett" <loris.bennett at fu-berlin.de> writes:

> "Loris Bennett" <loris.bennett at fu-berlin.de> writes:
>
>> Cameron Simpson <cs at cskk.id.au> writes:
>>
>>> On 01Nov2024 10:10, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>>>>as expected.  The non-UTF-8 text occurs when I do
>>>>
>>>>  mail = EmailMessage()
>>>>  mail.set_content(body, cte="quoted-printable")
>>>>  ...
>>>>
>>>>  if args.verbose:
>>>>      print(mail)
>>>>
>>>>which is presumably also correct.
>>>>
>>>>The question is: What conversion is necessary in order to print the
>>>>EmailMessage object to the terminal, such that the quoted-printable
>>>>parts are turned (back) into UTF-8?
>>>
>>> Do you still have access to `body` ? That would be the original
>>> message text? Otherwise maybe:
>>>
>>>     print(mail.get_content())
>>>
>>> The objective is to obtain the message body Unicode text (i.e. a
>>> regular Python string with the original text, unencoded). And to print
>>> that.
>>
>> With the following:
>>
>> ######################################################################
>>
>> import email.message
>>
>> m = email.message.EmailMessage()
>>
>> m['Subject'] = '?bung'
>>
>> m.set_content('Dies ist eine ?bung')
>> print('== cte: default == \n')
>> print(m)
>>
>> print('-- full mail ---')
>> print(m)
>> print('-- just content--')
>> print(m.get_content())
>>
>> m.set_content('Dies ist eine ?bung', cte='quoted-printable')
>> print('== cte: quoted-printable ==\n')
>> print('-- full mail --')
>> print(m)
>> print('-- just content --')
>> print(m.get_content())
>>
>> ######################################################################
>>
>> I get the following output:
>>
>> ######################################################################
>>
>> == cte: default == 
>>
>> Subject: ?bung
>> Content-Type: text/plain; charset="utf-8"
>> Content-Transfer-Encoding: base64
>> MIME-Version: 1.0
>>
>> RGllcyBpc3QgZWluZSDDnGJ1bmcK
>>
>> -- full mail ---
>> Subject: ?bung
>> Content-Type: text/plain; charset="utf-8"
>> Content-Transfer-Encoding: base64
>> MIME-Version: 1.0
>>
>> RGllcyBpc3QgZWluZSDDnGJ1bmcK
>>
>> -- just content--
>> Dies ist eine ?bung
>>
>> == cte: quoted-printable ==
>>
>> -- full mail --
>> Subject: ?bung
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset="utf-8"
>> Content-Transfer-Encoding: quoted-printable
>>
>> Dies ist eine =C3=9Cbung
>>
>> -- just content --
>> Dies ist eine ?bung
>>
>> ######################################################################
>>
>> So in both cases the subject is fine, but it is unclear to me how to
>> print the body.  Or rather, I know how to print the body OK, but I don't
>> know how to print the headers separately - there seems to be nothing
>> like 'get_headers()'.  I can use 'get('Subject) etc. and reconstruct the
>> headers, but that seems a little clunky.  
>
> Sorry, I am confusing the terminology here.  The 'body' seems to be the
> headers plus the 'content'.  So I can print the *content* without the
> headers OK, but I can't easily print all the headers separately.  If
> just print the body, i.e. headers plus content, the umlauts in the
> content are not resolved.

OK, so I can do:

######################################################################
if args.verbose:
    for k in mail.keys():
        print(f"{k}: {mail.get(k)}")
    print('')
    print(mail.get_content())
######################################################################

prints what I want and is not wildly clunky, but I am a little surprised
that I can't get a string representation of the whole email in one go.

Cheers,

Loris


-- 
Dr. Loris Bennett (Herr/Mr)
FUB-IT, Freie Universit?t Berlin

From cs at cskk.id.au  Mon Nov  4 16:21:28 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 5 Nov 2024 08:21:28 +1100
Subject: TkInter Scrolled Listbox class?
In-Reply-To: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>
References: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>
Message-ID: <Zyk62DZerPWhqFd1@cskk.homeip.net>

On 04Nov2024 16:32, Ulrich Goebel <ml at fam-goebel.de> wrote:
>I would like to build a class ScrolledListbox, which can be packed 
>somewhere in ttk.Frames. What I did is to build not really a scrolled 
>Listbox but a Frame containing a Listbox and a Scrollbar:

That's what I would build too.

>class FrameScrolledListbox(ttk.Frame):
>    def __init__(self, *args, **kwargs):
>        super().__init__(*args, **kwargs)
>        # build Listbox and Scrollbar
>        self.Listbox = tk.Listbox(self)
>        self.Scrollbar = ttk.Scrollbar(self)
[...]
>But it would be a bit nicer to get a class like
>
>class ScrolledListbox(tk.Listbox):
>    ...
>
>So it would be used that way:
>
>scrolledListbox = ScrolledListbox(main)
>scrolledListbox.config(...)

Probably you want to proxy various methods to the enclosed widgets.  
Possibly you want to do that with parameters in `__init__` also.

Example:

     class FrameScrolledListbox(ttk.Frame):
         def __init__(self, *frame_args, *, height=None, jump=None, **frame_kw):
             super().__init__(*frame_args, **frame_kw)
             self.Listbox = tk.Listbox(self, height=height)
             self.Scrollbar = ttk.Scrollbar(self, jump=jump)
             ........

         def config(self, *a, **kw):
             return self.Listbox.config(*a, **kw)

and so forth for the various listbox methods you want to proxy to the 
listbox itself. You could pass scroll specific methods to the scrollbar 
as well.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From learn2program at gmail.com  Mon Nov  4 19:06:56 2024
From: learn2program at gmail.com (Alan Gauld)
Date: Tue, 5 Nov 2024 00:06:56 +0000
Subject: TkInter Scrolled Listbox class?
In-Reply-To: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>
References: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>
Message-ID: <9aa48794-2dff-419a-b83a-0f5ac76332e0@yahoo.co.uk>

On 04/11/2024 15:32, Ulrich Goebel via Python-list wrote:

> I would like to build a class ScrolledListbox, 

I assume like the one that used to be available via the Tix module?
It's a great shame that Tix is gone, it had a lot of these useful
widgets, but they were all wrappers around Tcl/Tk's Tix module and
that is the thing that has gone.

If you really want to replace it I'd suggest looking at the IDLE
code and see what they did because the IDLE settings dialogues
have scrolled lists and the IDLE guys have probably done a decent
job in building their widgets. (Based on the fact that IDLE has
had a tone of work done in recent years using ttk)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From vpaijc at gmail.com  Tue Nov  5 13:28:38 2024
From: vpaijc at gmail.com (Vishal Chandratreya)
Date: Tue, 5 Nov 2024 23:58:38 +0530
Subject: TkInter Scrolled Listbox class?
In-Reply-To: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>
References: <20241104163248.108d895a431837a246a22fe4@fam-goebel.de>
Message-ID: <6B6251E9-0E4E-4021-A244-2D4E26E7C1AB@gmail.com>

   This is what I created ScrollableContainers for. Its usage deviates a bit
   from standard Tkinter practices in that you add widgets to the frame
   attribute of a ScrollableFrameTk instance.?

       [1]twitter.abaf4b19.webp
   [2]ScrollableContainers
   pypi.org

   For your use case, you could populate your list box and then put it inside
   that frame. I?ve not tested this scenario, so I?d appreciate feedback!
   Thanks.?

     On 4 Nov 2024, at 21:28, Ulrich Goebel via Python-list
     <python-list at python.org> wrote:

     ?Hi,
     I would like to build a class ScrolledListbox, which can be packed
     somewhere in ttk.Frames. What I did is to build not really a scrolled
     Listbox but a Frame containing a Listbox and a Scrollbar:
     class FrameScrolledListbox(ttk.Frame):
     ???def __init__(self, *args, **kwargs):
     ???????super().__init__(*args, **kwargs)
     ???????#
     ???????# build Listbox and Scrollbar
     ???????self.Listbox = tk.Listbox(self)
     ???????self.Scrollbar = ttk.Scrollbar(self)
     ???????#
     ???????# configure these two
     ???????self.Listbox.config(yscrollcommand=self.Scrollbar.set)
     ???????self.Scrollbar.config(command=self.Listbox.yview)
     ???????#
     ???????# pack them in Frame
     ???????self.Listbox.pack(side=tk.LEFT, fill=tk.BOTH)
     ???????self.Scrollbar.pack(side=tk.RIGHT, fill=tk.BOTH)
     That works, so instances of FrameScrolledListbox can be packed and the
     tk.Listbox itself is accessible via an attribute:
     frmScrolledListbox = FrameScrolledListbox(main)
     frmScrolledListbox.Listbox.config(...)
     But it would be a bit nicer to get a class like
     class ScrolledListbox(tk.Listbox):
     ???...
     So it would be used that way:
     scrolledListbox = ScrolledListbox(main)
     scrolledListbox.config(...)
     Is that possible? The problem which I can't handle is to handle the
     Frame which seems to be needed to place the Scrollbar somewhere.
     Best regards
     Ulrich
     --
     Ulrich Goebel <ml at fam-goebel.de>
     --
     https://mail.python.org/mailman/listinfo/python-list

References

   Visible links
   1. https://pypi.org/project/ScrollableContainers/
   2. https://pypi.org/project/ScrollableContainers/

From raymond.boute at pandora.be  Tue Nov  5 09:48:07 2024
From: raymond.boute at pandora.be (Raymond Boute)
Date: Tue, 5 Nov 2024 15:48:07 +0100
Subject: Two python issues
Message-ID: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>

L.S.,

Python seem to suffer from a few poor design decisions regarding strings 
and lists that affect the elegance of the language.

(a) An error-prone "feature" is returning -1 if a substring is not found 
by "find", since -1 currently refers to the last item. An example:

 >>> s = 'qwertyuiop'
 >>> s[s.find('r')]
'r'
 >>> s[s.find('p')]
'p'
 >>> s[s.find('a')]
'p'
 >>>

If "find" is unsuccessful, an error message is the only clean option.
Moreover, using index -1 for the last item is a bad choice: it should be 
len(s) - 1 (no laziness!).
Negative indices should be reserved for elements preceding the element 
with index 0 (currently not implemented, but a must for orthogonal 
design supporting general sequences).

(b) When using assignment for slices, only lists with the same length as 
the slice should be acceptable, otherwise an error should be given.? 
Anything that re-indexes items not covered by the slice is against the 
essential idea of assignment. For changes that imply re-indexing (e.g., 
inserting a list longer than the slice), Python offers cleaner solutions.

Comments are welcome.

With best regards,

Raymond


From loris.bennett at fu-berlin.de  Tue Nov  5 10:13:33 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 05 Nov 2024 16:13:33 +0100
Subject: Specifying local dependency with Poetry
Message-ID: <87fro5k92a.fsf@zedat.fu-berlin.de>

Hi,

I have installed a package, 'first-package, built using Poetry and
installed it as root in /usr/local.  The package is on sys.path so root
can successfully import it.

I have now developed a second package which depends on 'first-package'.
I would have expected that I could specify this dependency in
pyproj.toml via

[tool.poetry.dependencies]
python = "^3.6"
first-package = "^1.6.0"

However, if I do that and then attempt to install the second package, I
get the error

  Could not find a version that satisfies the requirement first-package<2.0.0,>=1.6.0 (from second-package==0.5.0) (from versions: )
No matching distribution found for first-package<2.0.0,>=1.6.0 (from second-package==0.5.0)

How should I define the dependency?

Cheers,

Loris

--
This signature is currently under constuction.

From hjp-python at hjp.at  Tue Nov  5 15:39:32 2024
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Tue, 5 Nov 2024 21:39:32 +0100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <871pzrmcky.fsf@zedat.fu-berlin.de>
References: <875xp7nwus.fsf@zedat.fu-berlin.de>
 <ZyVMe3Jspc0fJrel@cskk.homeip.net>
 <mailman.69.1730497664.4695.python-list@python.org>
 <87ed3rmg7g.fsf@zedat.fu-berlin.de>
 <875xp3mfku.fsf@zedat.fu-berlin.de>
 <871pzrmcky.fsf@zedat.fu-berlin.de>
Message-ID: <20241105203932.rclo4j5g763cfnmh@hjp.at>

On 2024-11-04 13:02:21 +0100, Loris Bennett via Python-list wrote:
> "Loris Bennett" <loris.bennett at fu-berlin.de> writes:
> > "Loris Bennett" <loris.bennett at fu-berlin.de> writes:
> >> Cameron Simpson <cs at cskk.id.au> writes:
> >>> On 01Nov2024 10:10, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
> >>>>as expected.  The non-UTF-8 text occurs when I do
> >>>>
> >>>>  mail = EmailMessage()
> >>>>  mail.set_content(body, cte="quoted-printable")
> >>>>  ...
> >>>>
> >>>>  if args.verbose:
> >>>>      print(mail)
> >>>>
> >>>>which is presumably also correct.
> >>>>
> >>>>The question is: What conversion is necessary in order to print the
> >>>>EmailMessage object to the terminal, such that the quoted-printable
> >>>>parts are turned (back) into UTF-8?
[...]
> OK, so I can do:
> 
> ######################################################################
> if args.verbose:
>     for k in mail.keys():
>         print(f"{k}: {mail.get(k)}")
>     print('')
>     print(mail.get_content())
> ######################################################################
> 
> prints what I want and is not wildly clunky, but I am a little surprised
> that I can't get a string representation of the whole email in one go.

Mails can contain lots of stuff, so there is in general no suitable
human readable string representation of a whole email. You have to go
through it part by part and decide what you want to do with each. For
example, if you have a multipart/alternative with a text/plain and a
text/html part what should the "string representation" be? For some uses
the text/plain part might be sufficient. For some you might want the
HTML part or some rendering of it. Or what would you do with an image?
Omit it completely? Just use the filename (if any)? Try to convert it to
ASCII-Art? Use an AI to describe it?

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20241105/2c992772/attachment.sig>

From cs at cskk.id.au  Tue Nov  5 16:06:12 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Wed, 6 Nov 2024 08:06:12 +1100
Subject: Two python issues
In-Reply-To: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
References: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
Message-ID: <ZyqIxGMojbeDzq63@cskk.homeip.net>

On 05Nov2024 15:48, Raymond Boute <raymond.boute at pandora.be> wrote:
>Python seem to suffer from a few poor design decisions regarding 
>strings and lists that affect the elegance of the language.
>
>(a) An error-prone "feature" is returning -1 if a substring is not 
>found by "find", since -1 currently refers to the last item.

`find` is a pretty old API interface. It is what it is. It may obtain 
some of its design choices from C style calls where returning -1 for 
failure was a common idiom.

>If "find" is unsuccessful, an error message is the only clean option.

This is not true. Often we want to look for something, and act one way 
or another depending on whether it is found. I've got plenty of loops 
and other tests which more or less go "run until this is not found". It 
is not an error, it is just a circumstance to accomodate.

>Moreover, using index -1 for the last item is a bad choice: it should 
>be len(s) - 1 (no laziness!).
>Negative indices should be reserved for elements preceding the element 
>with index 0 (currently not implemented, but a must for orthogonal 
>design supporting general sequences).

It is _far_ too late to propose such a change.

Plenty of us are quite hapoy with negative indices. We just view them as 
counting backwarss from the end of the string or sequence instead of 
forwards from the beginning.

>(b) When using assignment for slices, only lists with the same length 
>as the slice should be acceptable, otherwise an error should be 
>given.

There are many many circumstances where we replace a subsequence with a 
different subsequence of different length. Outlawing such a thing would 
remove and extremely useful feature.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From jsf80238 at gmail.com  Tue Nov  5 16:08:45 2024
From: jsf80238 at gmail.com (Jason Friedman)
Date: Tue, 5 Nov 2024 14:08:45 -0700
Subject: Two python issues
In-Reply-To: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
References: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
Message-ID: <CANy1k1iML9=cKxhQfASBZDwqc_4N_E66MgXhddPf6zna3G-ckw@mail.gmail.com>

>
> (a) An error-prone "feature" is returning -1 if a substring is not found
> by "find", since -1 currently refers to the last item. An example:
>
>  >>> s = 'qwertyuiop'
>  >>> s[s.find('r')]
> 'r'
>  >>> s[s.find('p')]
> 'p'
>  >>> s[s.find('a')]
> 'p'
>  >>>
>
> If "find" is unsuccessful, an error message is the only clean option.
> Moreover, using index -1 for the last item is a bad choice: it should be
> len(s) - 1 (no laziness!).
>

I'm not sure if this answers your objection but the note in the
documentation (https://docs.python.org/3/library/stdtypes.html#str.find)
says:

The find() method should be used only if you need to know the position of
sub.

I think the use case above is a little bit different.

From cs at cskk.id.au  Tue Nov  5 16:20:44 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Wed, 6 Nov 2024 08:20:44 +1100
Subject: Printing UTF-8 mail to terminal
In-Reply-To: <871pzrmcky.fsf@zedat.fu-berlin.de>
References: <871pzrmcky.fsf@zedat.fu-berlin.de>
Message-ID: <ZyqMLOUxvnwARS2e@cskk.homeip.net>

On 04Nov2024 13:02, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>OK, so I can do:
>
>######################################################################
>if args.verbose:
>    for k in mail.keys():
>        print(f"{k}: {mail.get(k)}")
>    print('')
>    print(mail.get_content())
>######################################################################
>
>prints what I want and is not wildly clunky, but I am a little surprised
>that I can't get a string representation of the whole email in one go.

A string representation of the whole message needs to be correctly 
encoded so that its components can be identified mechanically. So it 
needs to be a syntacticly valid RFC5322 message. Thus the encoding.

As an example (slightly contrived) of why this is important, multipart 
messages are delimited with distinct lines, and their content may not 
present such a line (even f it's in the "raw" original data).

So printing a whole message transcribes it in the encoded form so that 
it can be decoded mechanically. And conservativly, this is usually an 
ASCII compatibly encoding so that it can traverse various systems 
undamaged. This means the text requiring UTF8 encoding get further 
encoded as quoted printable to avoid ambiguity about the meaning of 
bytes/octets which have their high bit set.

BTW, doesn't this:

     for k in mail.keys():
         print(f"{k}: {mail.get(k)}")

print the quoted printable (i.e. not decoded) form of subject lines?

Cheers,
Cameron Simpson <cs at cskk.id.au>

From PythonList at DancesWithMice.info  Tue Nov  5 16:41:59 2024
From: PythonList at DancesWithMice.info (dn)
Date: Wed, 6 Nov 2024 10:41:59 +1300
Subject: Two python issues
In-Reply-To: <CANy1k1iML9=cKxhQfASBZDwqc_4N_E66MgXhddPf6zna3G-ckw@mail.gmail.com>
References: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
 <CANy1k1iML9=cKxhQfASBZDwqc_4N_E66MgXhddPf6zna3G-ckw@mail.gmail.com>
Message-ID: <8faeaf6f-0110-41bd-9e21-1fc4d57b42ae@DancesWithMice.info>

On 6/11/24 10:08, Jason Friedman via Python-list wrote:
>>
>> (a) An error-prone "feature" is returning -1 if a substring is not found
>> by "find", since -1 currently refers to the last item. An example:
>>
>>   >>> s = 'qwertyuiop'
>>   >>> s[s.find('r')]
>> 'r'
>>   >>> s[s.find('p')]
>> 'p'
>>   >>> s[s.find('a')]
>> 'p'
>>   >>>
>>
>> If "find" is unsuccessful, an error message is the only clean option.
>> Moreover, using index -1 for the last item is a bad choice: it should be
>> len(s) - 1 (no laziness!).
>>
> 
> I'm not sure if this answers your objection but the note in the
> documentation (https://docs.python.org/3/library/stdtypes.html#str.find)
> says:
> 
> The find() method should be used only if you need to know the position of
> sub.
> 
> I think the use case above is a little bit different.


Not really, there are two questions:

1. is x in sequence (or in this case "not in")
2. where is x within sequence (find())

There are situations where one might be used, similarly where the other 
will be used, and still more where both apply.

That said, and with @Cameron's observation, the idea that a function's 
return-value (appears to) performs two functionalities is regarded as a 
'code-smell' in today's world - either it indicates "found" or it 
indicates "where found" (see also various APIs which return both a 
boolean: success/fail, and a value: None/valid-info).

The problem with the third scenario being that purity suggests we should 
use both (1) and (2) which seems like duplication - and is certainly 
going to take more CPU time.
(will such be noticeable in your use-case?)

Backward-compatibility... ('nuff said!)


With reference to the OP content about slicing:
- Python's memory-addressing is different from many other languages. 
Thus, requires study before comparison/criticism
- there are major differences in what can be accomplished with mutable 
and immutable objects

-- 
Regards,
=dn

From greg.ewing at canterbury.ac.nz  Wed Nov  6 00:35:47 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 6 Nov 2024 18:35:47 +1300
Subject: Specifying local dependency with Poetry
In-Reply-To: <87fro5k92a.fsf@zedat.fu-berlin.de>
References: <87fro5k92a.fsf@zedat.fu-berlin.de>
Message-ID: <lp0dhmF6pnuU1@mid.individual.net>

On 6/11/24 4:13 am, Loris Bennett wrote:
> [tool.poetry.dependencies]
> python = "^3.6"
> first-package = "^1.6.0"
> 
>    Could not find a version that satisfies the requirement first-package<2.0.0,>=1.6.0 (from second-package==0.5.0) (from versions: )
> No matching distribution found for first-package<2.0.0,>=1.6.0 (from second-package==0.5.0)

What version number does first-package have?

The caret in the specification "^1.6.0" restricts it to 1.x.x versions.

If you don't want that restriction, use ">=" instead.

-- 
Greg


From roel at roelschroeven.net  Wed Nov  6 04:36:01 2024
From: roel at roelschroeven.net (Roel Schroeven)
Date: Wed, 6 Nov 2024 10:36:01 +0100
Subject: Two python issues
In-Reply-To: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
References: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
Message-ID: <0779f355-ed05-43cb-aa05-1afded82246c@roelschroeven.net>

Op 5/11/2024 om 15:48 schreef Raymond Boute via Python-list:
> L.S.,
>
> Python seem to suffer from a few poor design decisions regarding 
> strings and lists that affect the elegance of the language.
>
> (a) An error-prone "feature" is returning -1 if a substring is not 
> found by "find", since -1 currently refers to the last item. An example:
This is IMO indeed not the best design decision. Fortunately there's an 
alternative: the "index" method on strings, which raises exception 
ValueError when the substring is not found, instead of returning -1. An 
example:

 >>> s = 'qwertyuiop'
 >>> s[s.index('p')]
'p'
 >>> s[s.index('a')]
Traceback (most recent call last):
 ? File "<pyshell#3>", line 1, in <module>
 ??? s[s.index('a')]
ValueError: substring not found

> Moreover, using index -1 for the last item is a bad choice: it should 
> be len(s) - 1 (no laziness!).
> Negative indices should be reserved for elements preceding the element 
> with index 0 (currently not implemented, but a must for orthogonal 
> design supporting general sequences).
I don't agree, I think this is a case of "practicality beats purity". 
Being able to use negative indices to index from the end is often very 
useful in my experience, and leads to code that is much easier to grok. 
General sequences on the other hand is a concept that I don't see ever 
implemented in Python (or most other programming languages AFAIK). I 
think it would be wrong to avoid implementing a feature that's very 
useful in practice in order to keep the door open for a theoretical 
feature that's probably not even wanted in the language.
> (b) When using assignment for slices, only lists with the same length 
> as the slice should be acceptable, otherwise an error should be 
> given.? Anything that re-indexes items not covered by the slice is 
> against the essential idea of assignment. For changes that imply 
> re-indexing (e.g., inserting a list longer than the slice), Python 
> offers cleaner solutions.
Again I don't agree. I don't see anything wrong with replacing a part of 
a list with something that's longer, or shorter, or even empty. It's 
very practical, and I don't see how it's against the essential idea of 
assignment (or actually I'm not even sure what you mean by that).

Two closing remarks:

(1) I think that Python indeed has some warts, inconsistencies, gotchas. 
I think this is unavoidable in any complex system. Python got rid of a 
number of those in the transition from Python 2 to Python 3; others 
might remain forever. Overall though I feel Python is more consistent 
than most other programming languages I come in contact with.

(2) Design decisions are not necessarily wrong when they don't match 
your mental model, or don't match what you know from other languages. 
Often there are different valid options, each with their own tradeoffs.

-- 
"Programming today is a race between software engineers striving to build bigger
and better idiot-proof programs, and the Universe trying to produce bigger and
better idiots. So far, the Universe is winning."
         -- Douglas Adams


From piergiorgio.sartor.this.should.not.be.used at nexgo.REMOVETHIS.de  Tue Nov  5 16:27:53 2024
From: piergiorgio.sartor.this.should.not.be.used at nexgo.REMOVETHIS.de (Piergiorgio Sartor)
Date: Tue, 5 Nov 2024 22:27:53 +0100
Subject: Two python issues
In-Reply-To: <mailman.80.1730839406.4695.python-list@python.org>
References: <700403c2-e052-4670-b2ec-eaf9b4babada@pandora.be>
 <mailman.80.1730839406.4695.python-list@python.org>
Message-ID: <p7fqvk-jt4.ln1@lazy.lzy>

On 05/11/2024 15.48, Raymond Boute wrote:
> L.S.,
> 
> Python seem to suffer from a few poor design decisions regarding strings 
> and lists that affect the elegance of the language.
> 
> (a) An error-prone "feature" is returning -1 if a substring is not found 
> by "find", since -1 currently refers to the last item. An example:
> 
>  >>> s = 'qwertyuiop'
>  >>> s[s.find('r')]
> 'r'
>  >>> s[s.find('p')]
> 'p'
>  >>> s[s.find('a')]
> 'p'
>  >>>
> 
> If "find" is unsuccessful, an error message is the only clean option.
> Moreover, using index -1 for the last item is a bad choice: it should be 
> len(s) - 1 (no laziness!).
> Negative indices should be reserved for elements preceding the element 
> with index 0 (currently not implemented, but a must for orthogonal 
> design supporting general sequences).
> 
> (b) When using assignment for slices, only lists with the same length as 
> the slice should be acceptable, otherwise an error should be given. 
> Anything that re-indexes items not covered by the slice is against the 
> essential idea of assignment. For changes that imply re-indexing (e.g., 
> inserting a list longer than the slice), Python offers cleaner solutions.
> 
> Comments are welcome.

To write the nested expression, s[s.find(...)] it
means you're 200% sure of what happens in case of
not found.
It could be -1 or None or [] or anything.

So, the really correct thing to do, since you know
what will happen in case of not found, is *not* to
write the nested form, but explicitly state what it
will happen.

r = s.find(...)
if r is good:
     s[r]
else:
     print('not found')

Which is much easier to read, to debug, etc.

To paraphrase someone: "If the length of a
program would be measured by the time needed
to understand it, some programs are too short
to be short."

bye,

-- 

piergiorgio

From loris.bennett at fu-berlin.de  Wed Nov  6 01:57:09 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Wed, 06 Nov 2024 07:57:09 +0100
Subject: Specifying local dependency with Poetry
References: <87fro5k92a.fsf@zedat.fu-berlin.de>
 <lp0dhmF6pnuU1@mid.individual.net>
Message-ID: <87cyj8x322.fsf@zedat.fu-berlin.de>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> On 6/11/24 4:13 am, Loris Bennett wrote:
>> [tool.poetry.dependencies]
>> python = "^3.6"
>> first-package = "^1.6.0"
>>    Could not find a version that satisfies the requirement
>> first-package<2.0.0,>=1.6.0 (from second-package==0.5.0) (from
>> versions: )
>> No matching distribution found for first-package<2.0.0,>=1.6.0 (from second-package==0.5.0)
>
> What version number does first-package have?
>
> The caret in the specification "^1.6.0" restricts it to 1.x.x versions.
>
> If you don't want that restriction, use ">=" instead.

The first package has the version number 1.6.0.

-- 
This signature is currently under constuction.

From greg.ewing at canterbury.ac.nz  Thu Nov  7 17:15:38 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 8 Nov 2024 11:15:38 +1300
Subject: Two aces up Python's sleeve
In-Reply-To: <vgihe5$9tsc$1@solani.org>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgihe5$9tsc$1@solani.org>
Message-ID: <lp4sgdFg90oU1@mid.individual.net>

On 8/11/24 3:04 am, Mild Shock wrote:
> This only works for small integers. I guess
> this is because tagged pointers are used
> nowadays ?

No, it's because integers in a certain small range are cached. Not sure 
what the actual range is nowadays, it used to be something like -5 to 
256 I think.

BTW you have to be careful testing this, because the compiler sometimes 
does constant folding, so you need to be sure it's actually computing 
the numbers at run time.

-- 
Greg

From PythonList at DancesWithMice.info  Thu Nov  7 19:07:48 2024
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 8 Nov 2024 13:07:48 +1300
Subject: Two aces up Python's sleeve
In-Reply-To: <lp4sgdFg90oU1@mid.individual.net>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgihe5$9tsc$1@solani.org> <lp4sgdFg90oU1@mid.individual.net>
Message-ID: <0c94311a-f3e4-4aac-93e8-9bf95c065dcf@DancesWithMice.info>

On 8/11/24 11:15, Greg Ewing via Python-list wrote:
> On 8/11/24 3:04 am, Mild Shock wrote:
>> This only works for small integers. I guess
>> this is because tagged pointers are used
>> nowadays ?
> 
> No, it's because integers in a certain small range are cached. Not sure 
> what the actual range is nowadays, it used to be something like -5 to 
> 256 I think.
> 
> BTW you have to be careful testing this, because the compiler sometimes 
> does constant folding, so you need to be sure it's actually computing 
> the numbers at run time.

Haven't seen the OP. Is the Newsgroup link forwarding to the email-list 
correctly?


Integer interning is indeed valid for -5 <= i <= 256
("it works on my machine"! see below)

 >>> a = 0; b = 0; c = 0; d = 0
 >>> while a is b:
...     print( a, b, end=" ", )
...     print( c, d, ) if c is d else print()
...     a += 1; b += 1; c -= 1; d -= 1
...
0 0 0 0
1 1 -1 -1
2 2 -2 -2
3 3 -3 -3
4 4 -4 -4
5 5 -5 -5
6 6
7 7
8 8
9 9
...
254 254
255 255
256 256
 >>>

Be aware that this is implementation-dependent and not guaranteed to 
hold forever.

  dn ? ~ ? python
Python 3.12.7 (main, Oct  1 2024, 00:00:00) [GCC 13.3.1 20240913 (Red 
Hat 13.3.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.


See also https://docs.python.org/3/library/sys.html#sys.intern

Thus could decide what is interned for yourself:

a_string = sys.intern( str( 1000 ) )

-- 
Regards,
=dn

From annada at tilde.green  Thu Nov  7 02:25:53 2024
From: annada at tilde.green (Annada Behera)
Date: Thu, 07 Nov 2024 12:55:53 +0530
Subject: Two aces up Python's sleeve
In-Reply-To: <vgg5dn$ij48$1@solani.org>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
Message-ID: <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>

>Then please explain why I have to write:
>
>??? i += 1
>
>Instead of the shorter:
>
>??? i ++
>
>My short-term memory is really stressed.

I heard this behavior is because python's integers are immutable.
For example:

    >>> x,y = 5,5
    >>> id(x) == id(y)
    True

5 is a object that x and y points to. ++x or x++ will redefine 5 to
6, which the interpreter forbids to keep it's state mathematically
consistent. Also, by not supporting x++ and ++x, it avoids the pre-
and post-increment (substitute-increment v. increment-substitute) bugs
that plagues C and it's children.



From janburse at fastmail.fm  Thu Nov  7 09:04:53 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Thu, 7 Nov 2024 15:04:53 +0100
Subject: Two aces up Python's sleeve
In-Reply-To: <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
Message-ID: <vgihe5$9tsc$1@solani.org>

This only works for small integers. I guess
this is because tagged pointers are used
nowadays ? For large integers, also known

as bigint, it doesn't work:

Python 3.13.0a1 (tags/v3.13.0a1:ad056f0, Oct 13 2023, 09:51:17)

 >>> x, y = 5, 4+1
 >>> id(x) == id(y)
True

 >>> x, y = 10**200, 10**199*10
 >>> x == y
True
 >>> id(x) == id(y)
False

In tagged pointers a small integer is
directly inlined into the pointer. The
pointer has usually some higher bits,

that identify the type and when masking
to see the lower bits, one gets the
integer value.

But I don't know for sure whats going on,
would need to find a CPython documentation.

P.S.: I also tested PyPy it doesn't show
the same behaviour, because it computes
an exaberated id():

Python 3.10.14 (39dc8d3c85a7, Aug 27 2024, 14:33:33)
[PyPy 7.3.17 with MSC v.1929 64 bit (AMD64)]
 >>>> x, y = 5, 4+1
 >>>> id(x) == id(y)
True

 >>>> x, y = 10**200, 10**199*10
 >>>> id(x) == id(y)
True
 >>>> id(x)
1600000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000
000000000000000001

Quite funny!

Annada Behera schrieb:
>> Then please explain why I have to write:
>>
>>  ??? i += 1
>>
>> Instead of the shorter:
>>
>>  ??? i ++
>>
>> My short-term memory is really stressed.
> 
> I heard this behavior is because python's integers are immutable.
> For example:
> 
>      >>> x,y = 5,5
>      >>> id(x) == id(y)
>      True
> 
> 5 is a object that x and y points to. ++x or x++ will redefine 5 to
> 6, which the interpreter forbids to keep it's state mathematically
> consistent. Also, by not supporting x++ and ++x, it avoids the pre-
> and post-increment (substitute-increment v. increment-substitute) bugs
> that plagues C and it's children.
> 
> 


From janburse at fastmail.fm  Thu Nov  7 19:25:52 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Fri, 8 Nov 2024 01:25:52 +0100
Subject: Two aces up Python's sleeve
In-Reply-To: <lp4sgdFg90oU1@mid.individual.net>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgihe5$9tsc$1@solani.org> <lp4sgdFg90oU1@mid.individual.net>
Message-ID: <vgjlqg$ki9n$1@solani.org>

Hi,

In Java its possible to work this way
with the Integer datatype, just call
Integer.valueOf().

I am not sure whether CPython does the
same. Because it shows me the same behaviour
for small integers that are more than

only in the range -128 to 128. You can try yourself:

Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
 >>> x,y = 10**10, 10**9*10
 >>> id(x) == id(y)
True

Maybe the idea that objects have an address
that can be accessed via id() has been abandoned.
This is already seen in PyPy. So maybe we

are falsly assuming that id() gives na object address.

Greg Ewing schrieb:
> On 8/11/24 3:04 am, Mild Shock wrote:
>> This only works for small integers. I guess
>> this is because tagged pointers are used
>> nowadays ?
> 
> No, it's because integers in a certain small range are cached. Not sure 
> what the actual range is nowadays, it used to be something like -5 to 
> 256 I think.
> 
> BTW you have to be careful testing this, because the compiler sometimes 
> does constant folding, so you need to be sure it's actually computing 
> the numbers at run time.
> 


From janburse at fastmail.fm  Thu Nov  7 19:29:48 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Fri, 8 Nov 2024 01:29:48 +0100
Subject: Two aces up Python's sleeve
In-Reply-To: <vgjlqg$ki9n$1@solani.org>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgihe5$9tsc$1@solani.org> <lp4sgdFg90oU1@mid.individual.net>
 <vgjlqg$ki9n$1@solani.org>
Message-ID: <vgjm1s$ki9n$2@solani.org>


For example this article:

https://www.codementor.io/@arpitbhayani/python-caches-integers-16jih595jk

about the integer singletons claims:

 >>> x, y = 257, 257
 >>> id(x) == id(y)
False

But on Windows my recent CPython doesn't do that:

Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
 >>> x, y = 257, 257
 >>> id(x) == id(y)
True

Mild Shock schrieb:
> Hi,
> 
> In Java its possible to work this way
> with the Integer datatype, just call
> Integer.valueOf().
> 
> I am not sure whether CPython does the
> same. Because it shows me the same behaviour
> for small integers that are more than
> 
> only in the range -128 to 128. You can try yourself:
> 
> Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
>  >>> x,y = 10**10, 10**9*10
>  >>> id(x) == id(y)
> True
> 
> Maybe the idea that objects have an address
> that can be accessed via id() has been abandoned.
> This is already seen in PyPy. So maybe we
> 
> are falsly assuming that id() gives na object address.
> 
> Greg Ewing schrieb:
>> On 8/11/24 3:04 am, Mild Shock wrote:
>>> This only works for small integers. I guess
>>> this is because tagged pointers are used
>>> nowadays ?
>>
>> No, it's because integers in a certain small range are cached. Not 
>> sure what the actual range is nowadays, it used to be something like 
>> -5 to 256 I think.
>>
>> BTW you have to be careful testing this, because the compiler 
>> sometimes does constant folding, so you need to be sure it's actually 
>> computing the numbers at run time.
>>
> 


From janburse at fastmail.fm  Thu Nov  7 19:47:14 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Fri, 8 Nov 2024 01:47:14 +0100
Subject: Two aces up Python's sleeve
In-Reply-To: <vgjm1s$ki9n$2@solani.org>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgihe5$9tsc$1@solani.org> <lp4sgdFg90oU1@mid.individual.net>
 <vgjlqg$ki9n$1@solani.org> <vgjm1s$ki9n$2@solani.org>
Message-ID: <vgjn2h$kiom$1@solani.org>

The wiked brain of ChatGPT gives me a lead:

PEP 659
Storing data caches before the bytecode.

Maybe its an effect of constant folding
and constant pooling by the compiler?

Mild Shock schrieb:
> 
> For example this article:
> 
> https://www.codementor.io/@arpitbhayani/python-caches-integers-16jih595jk
> 
> about the integer singletons claims:
> 
>  >>> x, y = 257, 257
>  >>> id(x) == id(y)
> False
> 
> But on Windows my recent CPython doesn't do that:
> 
> Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
>  >>> x, y = 257, 257
>  >>> id(x) == id(y)
> True
> 
> Mild Shock schrieb:
>> Hi,
>>
>> In Java its possible to work this way
>> with the Integer datatype, just call
>> Integer.valueOf().
>>
>> I am not sure whether CPython does the
>> same. Because it shows me the same behaviour
>> for small integers that are more than
>>
>> only in the range -128 to 128. You can try yourself:
>>
>> Python 3.14.0a1 (tags/v3.14.0a1:8cdaca8, Oct 15 2024, 20:08:21)
>> ?>>> x,y = 10**10, 10**9*10
>> ?>>> id(x) == id(y)
>> True
>>
>> Maybe the idea that objects have an address
>> that can be accessed via id() has been abandoned.
>> This is already seen in PyPy. So maybe we
>>
>> are falsly assuming that id() gives na object address.
>>
>> Greg Ewing schrieb:
>>> On 8/11/24 3:04 am, Mild Shock wrote:
>>>> This only works for small integers. I guess
>>>> this is because tagged pointers are used
>>>> nowadays ?
>>>
>>> No, it's because integers in a certain small range are cached. Not 
>>> sure what the actual range is nowadays, it used to be something like 
>>> -5 to 256 I think.
>>>
>>> BTW you have to be careful testing this, because the compiler 
>>> sometimes does constant folding, so you need to be sure it's actually 
>>> computing the numbers at run time.
>>>
>>
> 


From janburse at fastmail.fm  Thu Nov  7 20:40:56 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Fri, 8 Nov 2024 02:40:56 +0100
Subject: Two aces up Python's sleeve (Posting On Python-List Prohibited)
In-Reply-To: <vgjoe9$2stii$2@dont-email.me>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgjoe9$2stii$2@dont-email.me>
Message-ID: <vgjq77$aj8c$1@solani.org>

Well you can use your Browser, since
JavaScript understand post and pre increment:

 > x = 5
5
 > x ++
5
 > x = 5
5
 > ++ x
6

So we have x ++ equals in Python:

     x + = 1
     x - 1

And ++ x equals in Python:

     x += 1
     x

But I don't know how to combine an
assignment and an expression into one
expession. In JavaScript one can use

the comma:

 > x = 5
5
 > y = (x += 1, x - 1)
5
 > x = 5
5
 > y = (x += 1, x)
6

But in Python the comma would create a tuple.

Lawrence D'Oliveiro schrieb:
> On Thu, 07 Nov 2024 12:55:53 +0530, Annada Behera wrote:
> 
>> I heard this behavior is because python's integers are immutable.
> 
> Nothing to do with that.
> 
>> ++x or x++ will redefine 5 to 6, which the interpreter forbids ...
> 
> One of those is actually syntactically valid.
> 
> It just won?t do what you expect it to do.
> 


From PythonList at DancesWithMice.info  Fri Nov  8 14:09:49 2024
From: PythonList at DancesWithMice.info (dn)
Date: Sat, 9 Nov 2024 08:09:49 +1300
Subject: Two aces up Python's sleeve (Posting On Python-List Prohibited)
In-Reply-To: <vgjq77$aj8c$1@solani.org>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgjoe9$2stii$2@dont-email.me> <vgjq77$aj8c$1@solani.org>
Message-ID: <59a78893-d96a-4f5b-921f-5ad862449ca2@DancesWithMice.info>

On 8/11/24 14:40, Mild Shock via Python-list wrote:
> Well you can use your Browser, since
> JavaScript understand post and pre increment:

Question: are we talking Python or JavaScript?


> So we have x ++ equals in Python:

Trying to find a word-for-word translation serves as badly in 
computer-programming languages as it does in human spoken-languages. 
Learn how to adapt and embrace the differences...


>  ??? x + = 1
>  ??? x - 1

The above probably only 'works' (the way you expect) in the REPL.


> But I don't know how to combine an
> assignment and an expression into one
> expession. In JavaScript one can use

Again!

"Everything should be made as simple as possible, but no simpler."

Check out "The Zen of Python" and PEP-0008 for Python idioms.


> the comma:
> 
>  > x = 5
> 5
>  > y = (x += 1, x - 1)
> 5
>  > x = 5
> 5
>  > y = (x += 1, x)
> 6
> 
> But in Python the comma would create a tuple.

Exactly, just as driving on the left side of the road will be fine in 
some countries but cause a crash in others. Learn the local rules FIRST!


The 'walrus operator' could be applied:

 >>> x = 5
 >>> y = (x := x + 1); x
6
 >>> x, y
(6, 6)

However, if such were submitted for Code Review, unhappiness would result.


Was the question re-phrased to: how to ... in Python, we'd end-up with 
something more like this:

 >>> x = 5  # define
 >>> x += 1  # increment
 >>> y = x  # alias
 >>> x, y
(6, 6)

-- 
Regards,
=dn

From list1 at tompassin.net  Fri Nov  8 17:00:31 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 8 Nov 2024 17:00:31 -0500
Subject: Two aces up Python's sleeve (Posting On Python-List Prohibited)
In-Reply-To: <59a78893-d96a-4f5b-921f-5ad862449ca2@DancesWithMice.info>
References: <seven-20241106014329@ram.dialup.fu-berlin.de>
 <vgg5dn$ij48$1@solani.org>
 <050c2ce9efd8442fb902ecc926afb1ee42fe6c34.camel@tilde.green>
 <vgjoe9$2stii$2@dont-email.me> <vgjq77$aj8c$1@solani.org>
 <59a78893-d96a-4f5b-921f-5ad862449ca2@DancesWithMice.info>
Message-ID: <5b6c9d39-2a1a-4554-8cd8-da123411eb81@tompassin.net>

On 11/8/2024 2:09 PM, dn via Python-list wrote:
> On 8/11/24 14:40, Mild Shock via Python-list wrote:
>> Well you can use your Browser, since
>> JavaScript understand post and pre increment:
> 
> Question: are we talking Python or JavaScript?
> 
> 
>> So we have x ++ equals in Python:
> 
> Trying to find a word-for-word translation serves as badly in computer- 
> programming languages as it does in human spoken-languages. Learn how to 
> adapt and embrace the differences...
> 
> 
>> ???? x + = 1
>> ???? x - 1
> 
> The above probably only 'works' (the way you expect) in the REPL.
> 
> 
>> But I don't know how to combine an
>> assignment and an expression into one
>> expession. In JavaScript one can use
> 
> Again!
> 
> "Everything should be made as simple as possible, but no simpler."
> 
> Check out "The Zen of Python" and PEP-0008 for Python idioms.
> 
> 
>> the comma:
>>
>> ?> x = 5
>> 5
>> ?> y = (x += 1, x - 1)
>> 5
>> ?> x = 5
>> 5
>> ?> y = (x += 1, x)
>> 6
>>
>> But in Python the comma would create a tuple.
> 
> Exactly, just as driving on the left side of the road will be fine in 
> some countries but cause a crash in others. Learn the local rules FIRST!
> 
> 
> The 'walrus operator' could be applied:
> 
>  >>> x = 5
>  >>> y = (x := x + 1); x
> 6
>  >>> x, y
> (6, 6)
> 
> However, if such were submitted for Code Review, unhappiness would result.
> 
> 
> Was the question re-phrased to: how to ... in Python, we'd end-up with 
> something more like this:
> 
>  >>> x = 5? # define
>  >>> x += 1? # increment
>  >>> y = x? # alias
>  >>> x, y
> (6, 6)

Or, still Pythonic but simpler:

 >>> x = 5
 >>> y = x = x + 1
 >>> x, y
(6, 6)


From research at johnohagan.com  Mon Nov 11 05:46:16 2024
From: research at johnohagan.com (John O'Hagan)
Date: Mon, 11 Nov 2024 21:46:16 +1100
Subject: SOLVED: Tkinter button-motion event behaves differently under
 Windows and Linux
Message-ID: <290a97ea3b1843d786daddd43869d851f604695c.camel@johnohagan.com>

I'm posting this in case anyone else encounters the same problem, and
to ask for suggestions, if any, about a better way to do it.

I'm implementing a method for dragging embedded widgets on a Text
widget. When the left mouse button is held down over an embedded widget
and the mouse is dragged across other widgets embedded in the same
parent, the original widget moves to the new positions until the button
is released.

Here's some minimal code:

-------

from tkinter import *
text = Text(Tk())
text.pack()

def drag(e):
    print(e.widget)
    target = e.widget.winfo_containing(e.x_root, e.y_root)
    if target and target not in (text, e.widget):
        if text.compare(target, '>', e.widget):
            target = f'{target} + 1 char'
        text.window_create(target, window=e.widget)

for n in ('1', '2', '3'):
    l=Label(text, text=n, width=10)
    #l.bind('<Button-1>', lambda e:e.widget.grab_set())
    l.bind('<B1-Motion>', drag)
    #l.bind('<ButtonRelease-1>', lambda e:e.widget.grab_release())
    text.window_create(END, window=l)

mainloop()

-------

This works as intended for me on Windows (11). The print statement
always shows the name of the first selected widget while the button is
held down, regardless of where the mouse is dragged to.

But on Linux (Debian testing with Gnome), for me the above code only
moves the widget to the first new position it is dragged to, any
further dragging is ineffective, and the print statement shows the
names of the subsequently-traversed widgets.

There is a further issue in the real code (not shown here) also on
Linux only, where if the cursor traverses any other widgets bound to
'<B1-Motion>', they are also triggered, which is not intended.

Just a guess, but it seems that on Linux, the focus switches to
whatever widget is under the cursor even during dragging, so any
bindings on the originally-clicked widget are no longer triggered,
whereas Windows maintains focus on the originally-clicked widget during
dragging until the button is released.

If that's the case (and I never thought I'd say this), I think Windows
is right! But for all I know it might be the window manager or
something else.

I eventually figured out that the commented lines calling grab_set and
grab_release solved the issue for me on Linux.

I haven't found this documented anywhere and I'm interested to know if
anyone can cast any light on it.

Thanks 



From loris.bennett at fu-berlin.de  Mon Nov 11 09:05:56 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 11 Nov 2024 15:05:56 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
Message-ID: <87v7wt986z.fsf@zedat.fu-berlin.de>

Hi,

I have the following in my program:

    try:
        logging.config.fileConfig(args.config_file)
        config = configparser.ConfigParser()
        config.read(args.config_file)
        if args.verbose:
            print(f"Configuration file: {args.config_file}")
    except FileNotFoundError:
        print(f"Error: configuration file {args.config_file} not found.  Exiting.")
        sys.exit(0)

and when I ran the program I got the error

  Error: configuration file /usr/local/etc/sc_mailer not found.  Exiting.

However, this file *does* exist and *can* be read.  By checking the
'filename' attribute of the exception I discovered that the problem was
the log file defined *in* the config file, namely

  [handler_fileHandler]
  class=FileHandler
  level=DEBUG
  formatter=defaultFormatter
  args=('/var/log/my_prog.log', 'a')

This log file did not exist.  The exception is thrown by 

  logging.config.fileConfig(args.config_file)

My questions are:

1. Should I be surprised by this behaviour?
2. In terms of generating a helpful error message, how should one
   distinguish between the config file not existing and the log file not
   existing?

Cheers,

Loris

-- 
This signature is currently under constuction.

From jdroth1003 at gmail.com  Sun Nov 10 17:28:05 2024
From: jdroth1003 at gmail.com (jdroth1003 at gmail.com)
Date: Sun, 10 Nov 2024 17:28:05 -0500
Subject: Seeking Assistance with Python's IDLE for Blind Users
Message-ID: <003501db33bf$cfca66c0$6f5f3440$@gmail.com>

Dear Python Users Group,

 

I am currently learning Python. I am blind and use the JAWS screen reader to
assist me. I am trying to use Python's IDLE editor but find it quite
challenging. When I move my cursor to a line of code, it reads out the
letters or words from the line above, which makes it very difficult to edit
the code accurately.

 

I am reaching out to see if anyone knows how to configure IDLE to call out
the letters or words that the cursor is actually on. Additionally, I have
searched extensively online but have not been able to find any documentation
specifically designed for blind users on how to effectively use Python's
IDLE.

 

Any guidance or resources you could provide would be greatly appreciated.

 

Thank you for your assistance.

 

Best regards,

Jeff


From olegsivokon at gmail.com  Mon Nov 11 11:04:52 2024
From: olegsivokon at gmail.com (Left Right)
Date: Mon, 11 Nov 2024 17:04:52 +0100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <87v7wt986z.fsf@zedat.fu-berlin.de>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
Message-ID: <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>

Poor error reporting is a very common problem in programming.  Python
is not anything special in this case.  Of course, it would've been
better if the error reported what file wasn't found.  But, usually
these problems are stacking, like in your code.  Unfortunately, it's
your duty, as the language user, to anticipate those problems and act
accordingly. Now you've learned that the one file you believe that
could be the source for the error isn't the only one--well, adjust
your code to differentiate between those two (and potentially other?)
cases.  There's very little else you can do beside that.

NB. On the system level, the error has no information about what file
wasn't found.  It simply returns some numeric value (the famous
ENOENT) in case when the system call to open a file fails.  Python
could've been more helpful by figuring out what path caused the
problem and printing that in the error message, but it doesn't...
That's why I, myself, never use the vanilla FileNotFoundError, I
always re-rise it with a customized version that incorporates the
information about the missing file in the error message.

NB2. It's always a bad idea to print logs to files.  Any sysadmin /
ops / infra person worth their salt will tell you that.  The only
place the logs should go to is the standard error.  There are true and
tried tools that can pick up logs from that point on, and do with them
whatever your heart desires.  That is, of course, unless you are
creating system tools for universal log management (in which case, I'd
question the choice of Python as a suitable language for such a task).
Unfortunately, even though this has been common knowledge for decades,
it's still elusive in the world of application development :|

On Mon, Nov 11, 2024 at 4:00?PM Loris Bennett via Python-list
<python-list at python.org> wrote:
>
> Hi,
>
> I have the following in my program:
>
>     try:
>         logging.config.fileConfig(args.config_file)
>         config = configparser.ConfigParser()
>         config.read(args.config_file)
>         if args.verbose:
>             print(f"Configuration file: {args.config_file}")
>     except FileNotFoundError:
>         print(f"Error: configuration file {args.config_file} not found.  Exiting.")
>         sys.exit(0)
>
> and when I ran the program I got the error
>
>   Error: configuration file /usr/local/etc/sc_mailer not found.  Exiting.
>
> However, this file *does* exist and *can* be read.  By checking the
> 'filename' attribute of the exception I discovered that the problem was
> the log file defined *in* the config file, namely
>
>   [handler_fileHandler]
>   class=FileHandler
>   level=DEBUG
>   formatter=defaultFormatter
>   args=('/var/log/my_prog.log', 'a')
>
> This log file did not exist.  The exception is thrown by
>
>   logging.config.fileConfig(args.config_file)
>
> My questions are:
>
> 1. Should I be surprised by this behaviour?
> 2. In terms of generating a helpful error message, how should one
>    distinguish between the config file not existing and the log file not
>    existing?
>
> Cheers,
>
> Loris
>
> --
> This signature is currently under constuction.
> --
> https://mail.python.org/mailman/listinfo/python-list

From loris.bennett at fu-berlin.de  Mon Nov 11 10:31:54 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 11 Nov 2024 16:31:54 +0100
Subject: Seeking Assistance with Python's IDLE for Blind Users
References: <003501db33bf$cfca66c0$6f5f3440$@gmail.com>
 <mailman.91.1731337099.4695.python-list@python.org>
Message-ID: <87h68d947p.fsf@zedat.fu-berlin.de>

Dear Jeff,

<jdroth1003 at gmail.com> writes:

> Dear Python Users Group,
>
>  
>
> I am currently learning Python. I am blind and use the JAWS screen reader to
> assist me. I am trying to use Python's IDLE editor but find it quite
> challenging. When I move my cursor to a line of code, it reads out the
> letters or words from the line above, which makes it very difficult to edit
> the code accurately.
>
>  
>
> I am reaching out to see if anyone knows how to configure IDLE to call out
> the letters or words that the cursor is actually on. Additionally, I have
> searched extensively online but have not been able to find any documentation
> specifically designed for blind users on how to effectively use Python's
> IDLE.
>
>  
>
> Any guidance or resources you could provide would be greatly appreciated.
>
>  
>
> Thank you for your assistance.
>
>  
>
> Best regards,
>
> Jeff
>

I can't make any suggestions regarding IDLE as I don't use it.  I also
have the impression that most people, rather than IDLE, use some other
form of IDE or editor.  So it might be worth looking for more generally
programming environments with appropriate support.

I personally use Emacs for many activities, including programming in
Python.  On various Emacs-related mailing lists I have come across blind
and visually impaired people, most of whom I assume are using Emacsspeak 

  https://emacspeak.sourceforge.net/

which has been around for 30 years, so that may be worth having a look
at.

Cheers,

Loris

-- 
This signature is currently under constuction.

From dieter.maurer at online.de  Mon Nov 11 12:24:44 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Mon, 11 Nov 2024 18:24:44 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
In-Reply-To: <87v7wt986z.fsf@zedat.fu-berlin.de>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
Message-ID: <26418.15836.335097.984240@ixdm.fritz.box>

Loris Bennett wrote at 2024-11-11 15:05 +0100:
>I have the following in my program:
>    try:
>        logging.config.fileConfig(args.config_file)
>        config = configparser.ConfigParser()
>        config.read(args.config_file)
>        if args.verbose:
>            print(f"Configuration file: {args.config_file}")
>    except FileNotFoundError:
>        print(f"Error: configuration file {args.config_file} not found.  Exiting.")

Do not replace full error information (including a traceback)
with your own reduced error message.
If you omit your "try ... except FileNotFoundError`
(or start the `except` clause with a `raise`), you
will learn where in the code the exception has been raised
and likely as well what was not found (Python is quite good
with such error details).

> ...
>My questions are:
>
>1. Should I be surprised by this behaviour?

Your code contains a major weakness (see above); thus surprises
are not unlikely.

>2. In terms of generating a helpful error message, how should one
>   distinguish between the config file not existing and the log file not
>   existing?

You look at the error information provided by Python
(and its library) rather than hiding it.

From cs at cskk.id.au  Mon Nov 11 16:17:46 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 12 Nov 2024 08:17:46 +1100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <26418.15836.335097.984240@ixdm.fritz.box>
References: <26418.15836.335097.984240@ixdm.fritz.box>
Message-ID: <ZzJ0eoWZds3xSKWn@cskk.homeip.net>

On 11Nov2024 18:24, dieter.maurer at online.de <dieter.maurer at online.de> wrote:
>Loris Bennett wrote at 2024-11-11 15:05 +0100:
>>I have the following in my program:
>>    try:
>>        logging.config.fileConfig(args.config_file)
>>        config = configparser.ConfigParser()
>>        config.read(args.config_file)
>>        if args.verbose:
>>            print(f"Configuration file: {args.config_file}")
>>    except FileNotFoundError:
>>        print(f"Error: configuration file {args.config_file} not found.  Exiting.")
>
>Do not replace full error information (including a traceback)
>with your own reduced error message.
>If you omit your "try ... except FileNotFoundError`
>(or start the `except` clause with a `raise`), you
>will learn where in the code the exception has been raised
>and likely as well what was not found (Python is quite good
>with such error details).

Actually, file-not-found is pretty well defined - the except action 
itself is fine in that regard.

[...]
>>2. In terms of generating a helpful error message, how should one
>>   distinguish between the config file not existing and the log file not
>>   existing?

Generally you should put a try/except around the smallest possible piece 
of code. So:

     config = configparser.ConfigParser()
     try:
         config.read(args.config_file)
     except FileNotFoundError as e:
         print(f"Error: configuration file {args.config_file} not found: {e}")

This way you know that the config file was missing.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From PythonList at DancesWithMice.info  Mon Nov 11 20:10:01 2024
From: PythonList at DancesWithMice.info (dn)
Date: Tue, 12 Nov 2024 14:10:01 +1300
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
References: <26418.15836.335097.984240@ixdm.fritz.box>
 <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
Message-ID: <635ef074-15b3-4084-ad34-c350b150a794@DancesWithMice.info>

On 12/11/24 10:17, Cameron Simpson via Python-list wrote:
> On 11Nov2024 18:24, dieter.maurer at online.de <dieter.maurer at online.de> 
> wrote:
>> Loris Bennett wrote at 2024-11-11 15:05 +0100:
>>> I have the following in my program:
>>> ?? try:
>>> ?????? logging.config.fileConfig(args.config_file)
>>> ?????? config = configparser.ConfigParser()
>>> ?????? config.read(args.config_file)
>>> ?????? if args.verbose:
>>> ?????????? print(f"Configuration file: {args.config_file}")
>>> ?? except FileNotFoundError:
>>> ?????? print(f"Error: configuration file {args.config_file} not 
>>> found.? Exiting.")
>>
> My questions are:
> 
> 1. Should I be surprised by this behaviour?

No. Python has behaved as-programmed.


>>> 2. In terms of generating a helpful error message, how should one
>>> ? distinguish between the config file not existing and the log file not
>>> ? existing?
> 
> Generally you should put a try/except around the smallest possible piece 
> of code. So:
> 
>  ??? config = configparser.ConfigParser()
>  ??? try:
>  ??????? config.read(args.config_file)
>  ??? except FileNotFoundError as e:
>  ??????? print(f"Error: configuration file {args.config_file} not found: 
> {e}")
> 
> This way you know that the config file was missing.

Augmenting @Cameron's excellent advice: please research "Separation of 
Concerns", eg https://en.wikipedia.org/wiki/Separation_of_concerns 
(which overlaps with one interpretation of SOLID's "Single 
Responsibility Principle" and the *nix philosophy of "do one thing, and 
do it well").

If you were to explain the code-snippet in English (or ...) it has 
several parts/concerns:

- configure the log
- instantiate ConfigParser()
- read the env.file
- advise verbosity-setting
- handle file-error
- (and, but not appearing) terminate execution

A block of code (see indentation for rough definition of "block") should 
achieve one thing ("concern"). Thus, the advice to separate-out the 
file-read and attendant defensive-coding.

This anticipates the problem (2) of distinguishing the subject of any 
one error/stack-trace from any others - and, arguably, makes the code 
easier to read.


-- 
Regards,
=dn

From rosuav at gmail.com  Mon Nov 11 20:17:27 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 12 Nov 2024 12:17:27 +1100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <87v7wt986z.fsf@zedat.fu-berlin.de>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
Message-ID: <CAPTjJmoJErFTE-oS2X_Z+2yC6bMXvHZi0=CxkGhWnVtd4Gacvw@mail.gmail.com>

On Tue, 12 Nov 2024 at 01:59, Loris Bennett via Python-list
<python-list at python.org> wrote:
> 2. In terms of generating a helpful error message, how should one
>    distinguish between the config file not existing and the log file not
>    existing?

By looking at the exception's attributes rather than assuming and
hard-coding the path in your message? Or, even better, just let the
exception bubble.

ChrisA

From jacob.kruger.work at gmail.com  Tue Nov 12 03:03:00 2024
From: jacob.kruger.work at gmail.com (Jacob Kruger)
Date: Tue, 12 Nov 2024 10:03:00 +0200
Subject: Seeking Assistance with Python's IDLE for Blind Users
In-Reply-To: <003501db33bf$cfca66c0$6f5f3440$@gmail.com>
References: <003501db33bf$cfca66c0$6f5f3440$@gmail.com>
Message-ID: <2d519c9f-5258-4739-a74f-8dc9993959f6@gmail.com>

Jeff, there are a few specific programmer's resources for blind/VI 
programmers that might help more - I am myself a 100% blind programmer, 
and, would not necessarily recommend idol from our perspective, since 
most people use VS code, or things like edSharp or even pycharm, but, I 
am a user of NVDA, which is itself written in python, and not a jaws 
user, so you might do better to ask some of the other people in that 
context - general blind programmer's mailing list:

https://www.freelists.org/list/program-l


And, the pythonvis blind python programmer's mailing list:

https://programmingforeveryone.groups.io/g/pythonvis


But, besides that, I am a 100%/full-time python web-developer, who uses 
python for other things as well, and, besides VS code, edSharp 
programmer's text editor is still probably my code editor of choice.


Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."


On 2024/11/11 00:28, Jeff via Python-list wrote:
> Dear Python Users Group,
>
>   
>
> I am currently learning Python. I am blind and use the JAWS screen reader to
> assist me. I am trying to use Python's IDLE editor but find it quite
> challenging. When I move my cursor to a line of code, it reads out the
> letters or words from the line above, which makes it very difficult to edit
> the code accurately.
>
>   
>
> I am reaching out to see if anyone knows how to configure IDLE to call out
> the letters or words that the cursor is actually on. Additionally, I have
> searched extensively online but have not been able to find any documentation
> specifically designed for blind users on how to effectively use Python's
> IDLE.
>
>   
>
> Any guidance or resources you could provide would be greatly appreciated.
>
>   
>
> Thank you for your assistance.
>
>   
>
> Best regards,
>
> Jeff
>

From dieter.maurer at online.de  Tue Nov 12 11:18:57 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Tue, 12 Nov 2024 17:18:57 +0100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
References: <26418.15836.335097.984240@ixdm.fritz.box>
 <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
Message-ID: <26419.32753.211549.746227@ixdm.fritz.box>

Cameron Simpson wrote at 2024-11-12 08:17 +1100:
>On 11Nov2024 18:24, dieter.maurer at online.de <dieter.maurer at online.de> wrote:
>>Loris Bennett wrote at 2024-11-11 15:05 +0100:
>>>I have the following in my program:
>>>    try:
>>>        logging.config.fileConfig(args.config_file)
>>>        config = configparser.ConfigParser()
>>>        config.read(args.config_file)
>>>        if args.verbose:
>>>            print(f"Configuration file: {args.config_file}")
>>>    except FileNotFoundError:
>>>        print(f"Error: configuration file {args.config_file} not found.  Exiting.")
>>
>>Do not replace full error information (including a traceback)
>>with your own reduced error message.
>>If you omit your "try ... except FileNotFoundError`
>>(or start the `except` clause with a `raise`), you
>>will learn where in the code the exception has been raised
>>and likely as well what was not found (Python is quite good
>>with such error details).
>
>Actually, file-not-found is pretty well defined - the except action
>itself is fine in that regard.

The original exception likely tells us which file was not found.

From loris.bennett at fu-berlin.de  Tue Nov 12 03:52:31 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 12 Nov 2024 09:52:31 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
References: <26418.15836.335097.984240@ixdm.fritz.box>
 <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
 <mailman.94.1731359879.4695.python-list@python.org>
Message-ID: <87o72kygts.fsf@zedat.fu-berlin.de>

Cameron Simpson <cs at cskk.id.au> writes:

> On 11Nov2024 18:24, dieter.maurer at online.de <dieter.maurer at online.de> wrote:
>>Loris Bennett wrote at 2024-11-11 15:05 +0100:
>>>I have the following in my program:
>>>    try:
>>>        logging.config.fileConfig(args.config_file)
>>>        config = configparser.ConfigParser()
>>>        config.read(args.config_file)
>>>        if args.verbose:
>>>            print(f"Configuration file: {args.config_file}")
>>>    except FileNotFoundError:
>>>        print(f"Error: configuration file {args.config_file} not found.  Exiting.")
>>
>>Do not replace full error information (including a traceback)
>>with your own reduced error message.
>>If you omit your "try ... except FileNotFoundError`
>>(or start the `except` clause with a `raise`), you
>>will learn where in the code the exception has been raised
>>and likely as well what was not found (Python is quite good
>>with such error details).
>
> Actually, file-not-found is pretty well defined - the except action
> itself is fine in that regard.
>
> [...]
>>>2. In terms of generating a helpful error message, how should one
>>>   distinguish between the config file not existing and the log file not
>>>   existing?
>
> Generally you should put a try/except around the smallest possible
> piece of code. So:
>
>     config = configparser.ConfigParser()
>     try:
>         config.read(args.config_file)
>     except FileNotFoundError as e:
>         print(f"Error: configuration file {args.config_file} not found: {e}")
>
> This way you know that the config file was missing.

I appreciate the point you make about the smallest possible piece of
code, although I can imagine that this could potentially create a lot of
try/except clutter and I might just want a single block and then try to
catch various exceptions.

Regarding your example above, if 'missingfile.py' contains the following

  import configparser

  config = configparser.ConfigParser()

  try:
      config.read('/foo/bar')
  except FileNotFoundError as e:
      print(f"Error: configuration file {config_file} not found: {e}")

them

  python3 missingfile.py

does not produce an any output for me and so does not seem to be a
reliable way of handling the case where the config file does not exist.

Cheers,

Loris

-- 
This signature is currently under constuction.

From loris.bennett at fu-berlin.de  Tue Nov 12 04:00:36 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 12 Nov 2024 10:00:36 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAPTjJmoJErFTE-oS2X_Z+2yC6bMXvHZi0=CxkGhWnVtd4Gacvw@mail.gmail.com>
 <mailman.96.1731374262.4695.python-list@python.org>
Message-ID: <87h68cyggb.fsf@zedat.fu-berlin.de>

Chris Angelico <rosuav at gmail.com> writes:

> On Tue, 12 Nov 2024 at 01:59, Loris Bennett via Python-list
> <python-list at python.org> wrote:
>> 2. In terms of generating a helpful error message, how should one
>>    distinguish between the config file not existing and the log file not
>>    existing?
>
> By looking at the exception's attributes rather than assuming and
> hard-coding the path in your message? Or, even better, just let the
> exception bubble.

As Dieter also pointed out, I obviously made a mistake in assuming that
I knew what 'FileNotFound' was referring to.

However, it strikes me as not immediately obvious that the logging file
must exist at this point.  I can imagine a situation in which I want to
configure a default log file and create it if it missing.

Cheers,

Loris

-- 
This signature is currently under constuction.

From loris.bennett at fu-berlin.de  Tue Nov 12 04:03:10 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 12 Nov 2024 10:03:10 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAPTjJmoJErFTE-oS2X_Z+2yC6bMXvHZi0=CxkGhWnVtd4Gacvw@mail.gmail.com>
 <mailman.96.1731374262.4695.python-list@python.org>
Message-ID: <87a5e4ygc1.fsf@zedat.fu-berlin.de>

Chris Angelico <rosuav at gmail.com> writes:

> On Tue, 12 Nov 2024 at 01:59, Loris Bennett via Python-list
> <python-list at python.org> wrote:
>> 2. In terms of generating a helpful error message, how should one
>>    distinguish between the config file not existing and the log file not
>>    existing?
>
> By looking at the exception's attributes rather than assuming and
> hard-coding the path in your message? Or, even better, just let the
> exception bubble.

I didn't consider letting the exception bubble as is the top of the
code for a CLI program.  I was hoping to just catch what I thought might
be a common error of the config file being missing.

-- 
This signature is currently under constuction.

From loris.bennett at fu-berlin.de  Tue Nov 12 04:15:47 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 12 Nov 2024 10:15:47 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
Message-ID: <875xosyfr0.fsf@zedat.fu-berlin.de>

Left Right <olegsivokon at gmail.com> writes:

> Poor error reporting is a very common problem in programming.  Python
> is not anything special in this case.  Of course, it would've been
> better if the error reported what file wasn't found.  But, usually
> these problems are stacking, like in your code.  Unfortunately, it's
> your duty, as the language user, to anticipate those problems and act
> accordingly. Now you've learned that the one file you believe that
> could be the source for the error isn't the only one--well, adjust
> your code to differentiate between those two (and potentially other?)
> cases.  There's very little else you can do beside that.
>
> NB. On the system level, the error has no information about what file
> wasn't found.  It simply returns some numeric value (the famous
> ENOENT) in case when the system call to open a file fails.  Python
> could've been more helpful by figuring out what path caused the
> problem and printing that in the error message, but it doesn't...
> That's why I, myself, never use the vanilla FileNotFoundError, I
> always re-rise it with a customized version that incorporates the
> information about the missing file in the error message.

That sounds like a good idea.

> NB2. It's always a bad idea to print logs to files.  Any sysadmin /
> ops / infra person worth their salt will tell you that.  The only
> place the logs should go to is the standard error.  There are true and
> tried tools that can pick up logs from that point on, and do with them
> whatever your heart desires.  That is, of course, unless you are
> creating system tools for universal log management (in which case, I'd
> question the choice of Python as a suitable language for such a task).
> Unfortunately, even though this has been common knowledge for decades,
> it's still elusive in the world of application development :|

I am not entirely convinced by NB2.  I am, in fact, a sort of sysadmin
person and most of my programs write to a log file.  The programs are
also moderately complex, so a single program might access a database,
query an LDAP server, send email etc., so potentially quite a lot can go
wrong.  They are also not programs whose output I would pipe to another
command.  What would be the advantage of logging to stderr?  Quite apart
from that, I find having a log file a useful for debugging when I am
developing.

Cheers,

Loris


> On Mon, Nov 11, 2024 at 4:00?PM Loris Bennett via Python-list
> <python-list at python.org> wrote:
>>
>> Hi,
>>
>> I have the following in my program:
>>
>>     try:
>>         logging.config.fileConfig(args.config_file)
>>         config = configparser.ConfigParser()
>>         config.read(args.config_file)
>>         if args.verbose:
>>             print(f"Configuration file: {args.config_file}")
>>     except FileNotFoundError:
>>         print(f"Error: configuration file {args.config_file} not found.  Exiting.")
>>         sys.exit(0)
>>
>> and when I ran the program I got the error
>>
>>   Error: configuration file /usr/local/etc/sc_mailer not found.  Exiting.
>>
>> However, this file *does* exist and *can* be read.  By checking the
>> 'filename' attribute of the exception I discovered that the problem was
>> the log file defined *in* the config file, namely
>>
>>   [handler_fileHandler]
>>   class=FileHandler
>>   level=DEBUG
>>   formatter=defaultFormatter
>>   args=('/var/log/my_prog.log', 'a')
>>
>> This log file did not exist.  The exception is thrown by
>>
>>   logging.config.fileConfig(args.config_file)
>>
>> My questions are:
>>
>> 1. Should I be surprised by this behaviour?
>> 2. In terms of generating a helpful error message, how should one
>>    distinguish between the config file not existing and the log file not
>>    existing?
>>
>> Cheers,
>>
>> Loris
>>
>> --
>> This signature is currently under constuction.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
-- 
Dr. Loris Bennett (Herr/Mr)
FUB-IT, Freie Universit?t Berlin

From Karsten.Hilbert at gmx.net  Tue Nov 12 12:47:37 2024
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 12 Nov 2024 18:47:37 +0100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <87o72kygts.fsf@zedat.fu-berlin.de>
References: <26418.15836.335097.984240@ixdm.fritz.box>
 <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
 <mailman.94.1731359879.4695.python-list@python.org>
 <87o72kygts.fsf@zedat.fu-berlin.de>
Message-ID: <ZzOUuVy6s0Ao3ZKq@hermes.hilbert.loc>

Am Tue, Nov 12, 2024 at 09:52:31AM +0100 schrieb Loris Bennett via Python-list:

> Regarding your example above, if 'missingfile.py' contains the following
>
>   import configparser
>
>   config = configparser.ConfigParser()
>
>   try:
>       config.read('/foo/bar')
>   except FileNotFoundError as e:
>       print(f"Error: configuration file {config_file} not found: {e}")
>
> them
>
>   python3 missingfile.py
>
> does not produce an any output for me and so does not seem to be a
> reliable way of handling the case where the config file does not exist.

Well:

	>>> help(config.read)
	Help on method read in module configparser:

	read(filenames, encoding=None) method of configparser.ConfigParser instance
	    Read and parse a filename or an iterable of filenames.

	    Files that cannot be opened are silently ignored; this is
	    designed so that you can specify an iterable of potential
	    configuration file locations (e.g. current directory, user's
	    home directory, systemwide directory), and all existing
	    configuration files in the iterable will be read.  A single
	    filename may also be given.

	    Return list of successfully read files.

So, the very fact that it does not return any output AND
returns an empty list is the (intended) way of knowing the
error state.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

From olegsivokon at gmail.com  Tue Nov 12 14:10:55 2024
From: olegsivokon at gmail.com (Left Right)
Date: Tue, 12 Nov 2024 20:10:55 +0100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <875xosyfr0.fsf@zedat.fu-berlin.de>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
Message-ID: <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>

> I am not entirely convinced by NB2.  I am, in fact, a sort of sysadmin
> person and most of my programs write to a log file.  The programs are
> also moderately complex, so a single program might access a database,
> query an LDAP server, send email etc., so potentially quite a lot can go
> wrong.  They are also not programs whose output I would pipe to another
> command.  What would be the advantage of logging to stderr?  Quite apart
> from that, I find having a log file a useful for debugging when I am
> developing.

First, the problem with writing to files is that there is no way to
make these logs reliable.  This is what I mean by saying these are
unreliable: since logs are designed to grow indefinitely, the natural
response to this design property is log rotation.  But, it's
impossible to reliably rotate a log file.  There's always a chance
that during the rotation some log entries will be written to the file
past the point of rotation, but prior to the point where the next logs
volume starts.

There are similar reliability problems with writing to Unix or
Internet sockets, databases etc.  For different reasons, but at the
end of the day, whoever wants logs, they want them to be reliable.
Both simplicity and convention selected for stderr as the only and the
best source of logging output.

Programs that write their output to log files will always irritate
their users because users will have to do some detective work to
figure out where those files are, and in some cases they will have to
do administrative works to make sure that the location where the
program wants to store the log files is accessible, has enough free
space, is speedy enough etc.  So, from the ops perspective, whenever I
come across a program that tries to write logs to anything other than
stderr, I make an earnest effort to throw that program into the gutter
and never touch it again.  It's too much headache to babysit every
such program, to remember the location of the log files of every such
program, the required permissions, to provision storage.  If you are
in that line of work, you just want all logs to go to the same place
(journal), where you can later filter / aggregate / correlate and
perform other BI tasks as your heart desires.

Of course, if you only administer your own computer, and you have low
single digits programs to run, and their behavior doesn't change
frequently, and you don't care to drop some records every now and
then... it's OK to log to files directly from a program.  But then you
aren't really in the sysadmin / infra / ops category, as you are more
of a hobby enthusiast.

Finally, if you want your logs to go to a file, and currently, your
only option is stderr, your shell gives you a really, really simple
way of redirecting stderr to a file.  So, really, there aren't any
excuses to do that.

From mats at wichmann.us  Tue Nov 12 15:28:04 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 12 Nov 2024 13:28:04 -0700
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
Message-ID: <f60479e0-b3a9-487e-9625-804d4aab26bf@wichmann.us>

On 11/12/24 12:10, Left Right via Python-list wrote:

> Finally, if you want your logs to go to a file, and currently, your
> only option is stderr, your shell gives you a really, really simple
> way of redirecting stderr to a file.  So, really, there aren't any
> excuses to do that.

an awful lot of the programs that need to do extensive logging don't run 
under control of a shell, and thus "shell redirection" helps not at all.



From rosuav at gmail.com  Tue Nov 12 15:34:10 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 Nov 2024 07:34:10 +1100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <f60479e0-b3a9-487e-9625-804d4aab26bf@wichmann.us>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <f60479e0-b3a9-487e-9625-804d4aab26bf@wichmann.us>
Message-ID: <CAPTjJmpTngd26KUymwbi6LJn1yunCEpwMbZRT11Bi1=S2nuR-g@mail.gmail.com>

On Wed, 13 Nov 2024 at 07:29, Mats Wichmann via Python-list
<python-list at python.org> wrote:
>
> On 11/12/24 12:10, Left Right via Python-list wrote:
>
> > Finally, if you want your logs to go to a file, and currently, your
> > only option is stderr, your shell gives you a really, really simple
> > way of redirecting stderr to a file.  So, really, there aren't any
> > excuses to do that.
>
> an awful lot of the programs that need to do extensive logging don't run
> under control of a shell, and thus "shell redirection" helps not at all.
>

Redirection's still a thing. "Shell redirection" is just the shell
syntax to request redirection.

ChrisA

From rob.cliffe at btinternet.com  Tue Nov 12 16:04:02 2024
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 12 Nov 2024 21:04:02 +0000
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <87o72kygts.fsf@zedat.fu-berlin.de>
References: <26418.15836.335097.984240@ixdm.fritz.box>
 <ZzJ0eoWZds3xSKWn@cskk.homeip.net>
 <mailman.94.1731359879.4695.python-list@python.org>
 <87o72kygts.fsf@zedat.fu-berlin.de>
Message-ID: <343ddd3d-65ec-4985-80a6-61e3c1e52353@btinternet.com>




On 12/11/2024 08:52, Loris Bennett via Python-list wrote:
> Cameron Simpson<cs at cskk.id.au> writes:
>
>> Generally you should put a try/except around the smallest possible
>> piece of code.
That is excellent advice.
Best wishes
Rob Cliffe
>>   So:
>>
>>      config = configparser.ConfigParser()
>>      try:
>>          config.read(args.config_file)
>>      except FileNotFoundError as e:
>>          print(f"Error: configuration file {args.config_file} not found: {e}")
>>
>>

From greg.ewing at canterbury.ac.nz  Tue Nov 12 20:04:33 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Nov 2024 14:04:33 +1300
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <mailman.100.1731438669.4695.python-list@python.org>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <mailman.100.1731438669.4695.python-list@python.org>
Message-ID: <lpic93Fi985U1@mid.individual.net>

On 13/11/24 8:10 am, Left Right wrote:
> since logs are designed to grow indefinitely, the natural
> response to this design property is log rotation.

I don't see how writing logs to stderr solves that problem in any way.
Whatever stderr is sent to still has a potentially unlimited amount
of data to deal with.

> But, it's
> impossible to reliably rotate a log file.  There's always a chance
> that during the rotation some log entries will be written to the file
> past the point of rotation, but prior to the point where the next logs
> volume starts.

Not sure I follow you there. You seem to be thinking of a particular
way of rotating log files, where an external process tries to swap
the program's log file out from under it without its knowledge. That
could be vulnerable to race conditions. But if the program doing the
logging handles the rotation itself, there's no reason it has to
lose data.

-- 
Greg


From roel at roelschroeven.net  Wed Nov 13 04:12:07 2024
From: roel at roelschroeven.net (Roel Schroeven)
Date: Wed, 13 Nov 2024 10:12:07 +0100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
Message-ID: <5f972e7d-e72d-459b-9148-0af977a82ead@roelschroeven.net>

Op 12/11/2024 om 20:10 schreef Left Right via Python-list:
> > I am not entirely convinced by NB2.  I am, in fact, a sort of sysadmin
> > person and most of my programs write to a log file.  The programs are
> > also moderately complex, so a single program might access a database,
> > query an LDAP server, send email etc., so potentially quite a lot can go
> > wrong.  They are also not programs whose output I would pipe to another
> > command.  What would be the advantage of logging to stderr?  Quite apart
> > from that, I find having a log file a useful for debugging when I am
> > developing.
>
> First, the problem with writing to files is that there is no way to
> make these logs reliable.  This is what I mean by saying these are
> unreliable: since logs are designed to grow indefinitely, the natural
> response to this design property is log rotation.  But, it's
> impossible to reliably rotate a log file.  There's always a chance
> that during the rotation some log entries will be written to the file
> past the point of rotation, but prior to the point where the next logs
> volume starts.

What I most often do is use one logfile per day, with the date in the 
filename. Then simply delete all files older than 7 days, or 30 days, or 
whatever is useful for the task at hand. Not only does that sidestep any 
issues with rotating logs, but I also find it's very useful to have the 
date in the filename.
> Of course, if you only administer your own computer, and you have low
> single digits programs to run, and their behavior doesn't change
> frequently, and you don't care to drop some records every now and
> then... it's OK to log to files directly from a program.  But then you
> aren't really in the sysadmin / infra / ops category, as you are more
> of a hobby enthusiast.
I would not use my scheme for something released to a wider audience. 
For in-house software though, I like that I can easily put each 
application's logs next to its other data files, and that I don't have 
to figure out how to get the system's own log infrastructure to work is 
I want it to.
> Finally, if you want your logs to go to a file, and currently, your
> only option is stderr, your shell gives you a really, really simple
> way of redirecting stderr to a file.
I feel this is the worst of both worlds. Now your program doesn't have 
any control over filename or log expiration, and neither does your 
system's logging infrastructure. You just get one indefinitely growing 
log file.

-- 
"You can fool some of the people all the time, and all of the people some
of the time, but you cannot fool all of the people all of the time."
         -- Abraham Lincoln
"You can fool too many of the people too much of the time."
         -- James Thurber


From dieter.maurer at online.de  Wed Nov 13 13:36:04 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Wed, 13 Nov 2024 19:36:04 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
In-Reply-To: <87h68cyggb.fsf@zedat.fu-berlin.de>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAPTjJmoJErFTE-oS2X_Z+2yC6bMXvHZi0=CxkGhWnVtd4Gacvw@mail.gmail.com>
 <mailman.96.1731374262.4695.python-list@python.org>
 <87h68cyggb.fsf@zedat.fu-berlin.de>
Message-ID: <26420.61844.597011.839900@ixdm.fritz.box>

Loris Bennett wrote at 2024-11-12 10:00 +0100:
> ...
>However, it strikes me as not immediately obvious that the logging file
>must exist at this point.  I can imagine a situation in which I want to
>configure a default log file and create it if it missing.

This is what happens usually:
if you open a file with mode `a` or `w`, the file is created
if it does not yet exist.

Thus, a missing log file should not give you the `FileNotFound`
exception.
Look at the exception details: they should tell you what really
was not found (maybe the directory for the logfile).

From kushal at locationd.net  Wed Nov 13 17:40:57 2024
From: kushal at locationd.net (Kushal Kumaran)
Date: Wed, 13 Nov 2024 14:40:57 -0800
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <26420.61844.597011.839900@ixdm.fritz.box> (dieter maurer's
 message of "Wed, 13 Nov 2024 19:36:04 +0100")
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAPTjJmoJErFTE-oS2X_Z+2yC6bMXvHZi0=CxkGhWnVtd4Gacvw@mail.gmail.com>
 <mailman.96.1731374262.4695.python-list@python.org>
 <87h68cyggb.fsf@zedat.fu-berlin.de>
 <26420.61844.597011.839900@ixdm.fritz.box>
Message-ID: <87msi2eozq.fsf@copper>

On Wed, Nov 13 2024 at 07:36:04 PM, dieter.maurer at online.de wrote:
> Loris Bennett wrote at 2024-11-12 10:00 +0100:
>> ...
>>However, it strikes me as not immediately obvious that the logging file
>>must exist at this point.  I can imagine a situation in which I want to
>>configure a default log file and create it if it missing.
>
> This is what happens usually:
> if you open a file with mode `a` or `w`, the file is created
> if it does not yet exist.
>
> Thus, a missing log file should not give you the `FileNotFound`
> exception.
> Look at the exception details: they should tell you what really
> was not found (maybe the directory for the logfile).

It is possible a directory along the path does not exist.

-- 
regards,
kushal

From torriem at gmail.com  Wed Nov 13 23:07:56 2024
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 13 Nov 2024 21:07:56 -0700
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
Message-ID: <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>

On 11/12/24 12:10 PM, Left Right via Python-list wrote:
> But, it's
> impossible to reliably rotate a log file.  There's always a chance
> that during the rotation some log entries will be written to the file
> past the point of rotation, but prior to the point where the next logs
> volume starts.

On any Unix system this is untrue.  Rotating a log file is quite simple:
simply rename the log file, then send a signal to the process to close
the log file handle and open a new one.  After that perhaps compress the
rotated log file.  Nothing is lost.  This is standard practice in Unix.
It is reliable.

Perhaps the scenario you posit would happen on Windows.

From olegsivokon at gmail.com  Thu Nov 14 02:03:32 2024
From: olegsivokon at gmail.com (Left Right)
Date: Thu, 14 Nov 2024 08:03:32 +0100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>
Message-ID: <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>

> On any Unix system this is untrue.  Rotating a log file is quite simple:

I realized I posted this without cc'ing the list:
http://jdebp.info/FGA/do-not-use-logrotate.html .

The link above gives a more detailed description of why log rotation
on the Unix system is not only not simple, but is, in fact,
unreliable.

NB. Also, it really rubs me the wrong way when the word "standard" is
used to mean "common" (instead of "as described in a standard
document").  And when it comes to popular tools, oftentimes "common"
is wrong because commonly the tool is used by amateurs rather than
experts.  In other words, you only reinforced what I wrote initially:
plenty of application developers don't know how to do logging well.
It also appears that they would lecture infra / ops people on how to
do something that they aren't experts on, while the latter are :)

From rosuav at gmail.com  Thu Nov 14 03:13:54 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 14 Nov 2024 19:13:54 +1100
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>
 <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
Message-ID: <CAPTjJmpaAcHAhoRS=yW+faR4nFoaX_6syTQX1XSRL9vnUd2Sng@mail.gmail.com>

On Thu, 14 Nov 2024 at 18:05, Left Right via Python-list
<python-list at python.org> wrote:
>
> > On any Unix system this is untrue.  Rotating a log file is quite simple:
>
> I realized I posted this without cc'ing the list:
> http://jdebp.info/FGA/do-not-use-logrotate.html .
>
> The link above gives a more detailed description of why log rotation
> on the Unix system is not only not simple, but is, in fact,
> unreliable.

You're assuming a very specific tool here. Log rotation isn't
necessarily performed by that one tool. There are many ways to do it.

Log to stderr. That puts the power in the hands of the sysadmin,
rather than forcing trickery like setting the log file name to be
/proc/self/fd/2 to get around it.

ChrisA

From loris.bennett at fu-berlin.de  Wed Nov 13 02:11:01 2024
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Wed, 13 Nov 2024 08:11:01 +0100
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
Message-ID: <87iksrehh6.fsf@zedat.fu-berlin.de>

Left Right <olegsivokon at gmail.com> writes:

>> I am not entirely convinced by NB2.  I am, in fact, a sort of sysadmin
>> person and most of my programs write to a log file.  The programs are
>> also moderately complex, so a single program might access a database,
>> query an LDAP server, send email etc., so potentially quite a lot can go
>> wrong.  They are also not programs whose output I would pipe to another
>> command.  What would be the advantage of logging to stderr?  Quite apart
>> from that, I find having a log file a useful for debugging when I am
>> developing.
>
> First, the problem with writing to files is that there is no way to
> make these logs reliable.  This is what I mean by saying these are
> unreliable: since logs are designed to grow indefinitely, the natural
> response to this design property is log rotation.  But, it's
> impossible to reliably rotate a log file.  There's always a chance
> that during the rotation some log entries will be written to the file
> past the point of rotation, but prior to the point where the next logs
> volume starts.

> There are similar reliability problems with writing to Unix or
> Internet sockets, databases etc.  For different reasons, but at the
> end of the day, whoever wants logs, they want them to be reliable.
> Both simplicity and convention selected for stderr as the only and the
> best source of logging output.

If I understand correctly you are not saying that logrotate is less
reliable that the other methods mentioned above.  But in that case,
reliability seems no more of a reason not to log to files than it is a
reason not to write to a socket or to a database.

> Programs that write their output to log files will always irritate
> their users because users will have to do some detective work to
> figure out where those files are, and in some cases they will have to
> do administrative works to make sure that the location where the
> program wants to store the log files is accessible, has enough free
> space, is speedy enough etc.

All your points regarding the work involved are valid, but many
programs, such as MariaDB, OpenLDAP or SSSD, do write to a log file (and
it is usually under /var/log or /var/log/something.  So it seems like a
common approach.

Besides, I define the location of the logfile in the config file for the
program (the problem in my original question arose from this filename in
the config file not existing).  So finding the location is not an issue.
You have to find the config file, of course, but I think /etc or
/usr/local/etc are fairly standard and my programs generally have an
option '--config-file' anyway.

>  So, from the ops perspective, whenever I
> come across a program that tries to write logs to anything other than
> stderr, I make an earnest effort to throw that program into the gutter
> and never touch it again.  It's too much headache to babysit every
> such program, to remember the location of the log files of every such
> program, the required permissions, to provision storage.  If you are
> in that line of work, you just want all logs to go to the same place
> (journal), where you can later filter / aggregate / correlate and
> perform other BI tasks as your heart desires.

That may be true in many cases, but those I am dealing with don't
require much filtering beyond 'grep' and also don't require aggregation
or correlation.

> Of course, if you only administer your own computer, and you have low
> single digits programs to run, and their behavior doesn't change
> frequently, and you don't care to drop some records every now and
> then... it's OK to log to files directly from a program.  But then you
> aren't really in the sysadmin / infra / ops category, as you are more
> of a hobby enthusiast.

What I do is indeed a bit of a niche, but I do get paid for this, so I
would not consider myself a 'hobby enthusiast'.

> Finally, if you want your logs to go to a file, and currently, your
> only option is stderr, your shell gives you a really, really simple
> way of redirecting stderr to a file.  So, really, there aren't any
> excuses to do that.

I don't quite understand what your suggestion is.  Do you mean that I
should log to stderr and then run my program as

  my_program ... 2>&1 | logger

?

Cheers,

Loris

-- 
This signature is currently under constuction.

From info at egenix.com  Wed Nov 13 05:22:41 2024
From: info at egenix.com (eGenix Team)
Date: Wed, 13 Nov 2024 11:22:41 +0100
Subject: ANN: eGenix PyRun - One file Python Runtime 2.6.0
Message-ID: <9f38c14b-23af-4fb2-95e0-db84d2657693@egenix.com>

*ANNOUNCING*


    eGenix PyRun - One file Python Runtime

Version 2.6.0

Python runtime taking up just 4-6MB on disk

This announcement is also available on our web-site for online reading:
https://www.egenix.com/company/news/eGenix-PyRun-2.6.0-GA.html

------------------------------------------------------------------------
*INTRODUCTION*

*eGenix PyRun*? <https://www.egenix.com/company/legal/trademarks.html> 
is our open source, one file, no installation version of Python, making 
the distribution of a Python interpreter to run Python based scripts and 
applications to Unix based systems simple and efficient.

eGenix PyRun's executable only needs 4-6MB on disk, but still supports 
most Python applications and scripts.

Compared to a regular Python installation of typically 100MB on disk, 
eGenix PyRun is ideal for applications and scripts that need to be 
distributed to containers, VMs, clusters, client installations, 
customers or end-users.

It makes "installing" Python on a Unix based system as simple as copying 
a single file.

eGenix has been using eGenix PyRun as run-time for the Linux version of 
mxODBC Connect Server 
<https://www.egenix.com/products/python/mxODBCConnect/> product since 
2008 with great success and decided to make it available as a 
stand-alone open-source product.

We provide the source archive to build your own *eGenix PyRun on Github* 
<https://github.com/eGenix/egenix-pyrun>, as well as a few binary 
distributions to get you started on Linux x86_64. In the future, we will 
set up automated builds for several other platforms.

Please see the product page for more details:

 >>> eGenix PyRun - One file Python Runtime 
<https://www.egenix.com/products/python/PyRun/>


------------------------------------------------------------------------
*NEWS*

This major release of eGenix PyRun 
<https://www.egenix.com/products/python/PyRun> comes with the following 
enhancements:

  * Added support for *Python 3.12*
  * Added support for LTO release builds
  * Added dev build targets for development; these don't use PGO and
    thus build faster

For a complete list of changes, please see the *eGenix PyRun Changelog 
<https://www.egenix.com/products/python/PyRun/changelog.html>*.


Enjoy,

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 13 2024)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::
     
    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611
                https://www.egenix.com/company/contact/
                      https://www.malemburg.com/

From darcy at VybeNetworks.com  Wed Nov 13 08:37:47 2024
From: darcy at VybeNetworks.com (D'Arcy Cain)
Date: Wed, 13 Nov 2024 06:37:47 -0700
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <5f972e7d-e72d-459b-9148-0af977a82ead@roelschroeven.net>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <5f972e7d-e72d-459b-9148-0af977a82ead@roelschroeven.net>
Message-ID: <79074400-022a-42d8-b1c7-2fe53c5c7007@VybeNetworks.com>

On 11/13/24 02:12, Roel Schroeven via Python-list wrote:
> What I most often do is use one logfile per day, with the date in the 
> filename. Then simply delete all files older than 7 days, or 30 days, or 
> whatever is useful for the task at hand. Not only does that sidestep any 
> issues with rotating logs, but I also find it's very useful to have the 
> date in the filename.

I do something similar for my system logs except that I let the system 
use the usual names and, at midnight, I rename the file appending the 
previous day's date to it and restarting services.

-- 
D'Arcy J.M. Cain
Vybe Networks Inc.
http://www.VybeNetworks.com/
IM:darcy at VybeNetworks.com VoIP: sip:darcy at VybeNetworks.com

From barry at barrys-emacs.org  Thu Nov 14 11:01:42 2024
From: barry at barrys-emacs.org (Barry)
Date: Thu, 14 Nov 2024 16:01:42 +0000
Subject: FileNotFoundError thrown due to file name in file,
 rather than file itself
In-Reply-To: <87iksrehh6.fsf@zedat.fu-berlin.de>
References: <87iksrehh6.fsf@zedat.fu-berlin.de>
Message-ID: <C4D98083-D2F1-4490-B342-5AA94DB9E4B1@barrys-emacs.org>



> On 14 Nov 2024, at 14:07, Loris Bennett via Python-list <python-list at python.org> wrote:
> 
> I don't quite understand what your suggestion is.  Do you mean that I
> should log to stderr and then run my program as
> 
>  my_program ... 2>&1 | logger

On almost all Linux distros you would run a long running program as a systemd service and let it put logs into the journal. I wonder if that was what was being hinted at?

Barry



From ethan at stoneleaf.us  Thu Nov 14 12:32:32 2024
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 14 Nov 2024 09:32:32 -0800
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>
 <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
Message-ID: <606e45ca-7d2c-38f2-7850-e04b22b697f2@stoneleaf.us>

On 11/13/24 23:03, Left Right via Python-list wrote:

 >> On any Unix system this is untrue.  Rotating a log file is quite simple:
 >
 > I realized I posted this without cc'ing the list:
 > http://jdebp.info/FGA/do-not-use-logrotate.html .
 >
 > The link above gives a more detailed description of why log rotation
 > on the Unix system is not only not simple, but is, in fact,
 > unreliable.

Having read the linked article, I see it is not relevant to Python, as Python's logging tool is
the writer/rotator program, thus no window for lost entries exists.

 > NB. Also, it really rubs me the wrong way when the word "standard" is
 > used to mean "common" (instead of "as described in a standard
 > document").

Yes, that is irritating.

 >  And when it comes to popular tools, oftentimes "common"
 > is wrong because commonly the tool is used by amateurs rather than
 > experts.  In other words, you only reinforced what I wrote initially:
 > plenty of application developers don't know how to do logging well.
 > It also appears that they would lecture infra / ops people on how to
 > do something that they aren't experts on, while the latter are :)

Well, since this is a Python list, perhaps you could make sure your advice is also Python appropriate.  I appreciate 
diversions into general areas and learning new things, but your general claims were untrue when it comes to Python 
specifically, and that was unclear until I read your linked post.

--
~Ethan~

From torriem at gmail.com  Thu Nov 14 10:44:56 2024
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 14 Nov 2024 08:44:56 -0700
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
In-Reply-To: <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>
 <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
Message-ID: <541261c9-5d96-b52f-536a-a60bec679d63@gmail.com>

On 11/14/24 12:03 AM, Left Right wrote:
>> On any Unix system this is untrue.  Rotating a log file is quite simple:
> 
> I realized I posted this without cc'ing the list:
> http://jdebp.info/FGA/do-not-use-logrotate.html .
> 
> The link above gives a more detailed description of why log rotation
> on the Unix system is not only not simple, but is, in fact,
> unreliable.

Nothing in that article contradicts what I said about renaming log
files.  His argument is that renaming log files messes with tail -F, and
therefore broken and unreliable.  Which a pretty strange argument.  tail
-F might not see some data during the rotation, but the log files
themselves don't miss anything, which was my contention.  In all my
years of sysadmin-ing I have never once worried about problems GNU tail
might have with a file that gets rotated out from under you.  Not sure
why the author is so fixated on it.

There are actual legitimate issues at play, such as a mechanism for
informing the process to close the file (rotate usually requires
processes to respond to SIGHUP).  And of course the disk can fill up
causing a denial of service of one kind or another.  The latter is the
biggest source of problems.

Of course you could just log using the standard libc syslog facility.
Or better yet, start your process from a systemd unit file and let the
journal automatically log stderr.  In both cases that would satisfy the
technical objections of the author of that little treatise.


From jon+usenet at unequivocal.eu  Thu Nov 14 13:12:28 2024
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Thu, 14 Nov 2024 18:12:28 -0000 (UTC)
Subject: FileNotFoundError thrown due to file name in file, rather than
 file itself
References: <87v7wt986z.fsf@zedat.fu-berlin.de>
 <CAJQBtg=UOiOmmHa25EUZtrZO19F1O0_VxCO6gWjZ5ebAMHnXCA@mail.gmail.com>
 <mailman.92.1731341107.4695.python-list@python.org>
 <875xosyfr0.fsf@zedat.fu-berlin.de>
 <CAJQBtg=nr+KC5iz_WY7nqkGK+YBEWAPq40ksaWJCAOT_+=pHxg@mail.gmail.com>
 <235f9d53-f784-8998-f201-1c01dbbf14ba@gmail.com>
 <CAJQBtgmQ0MLh2xYpUTwsSWhO1ra=QVuJRddn=17Pd7gKA=aM+g@mail.gmail.com>
 <541261c9-5d96-b52f-536a-a60bec679d63@gmail.com>
 <mailman.114.1731606301.4695.python-list@python.org>
Message-ID: <slrnvjcfcc.5qt.jon+usenet@raven.unequivocal.eu>

On 2024-11-14, Michael Torrie <torriem at gmail.com> wrote:
> On 11/14/24 12:03 AM, Left Right wrote:
>>> On any Unix system this is untrue.  Rotating a log file is quite simple:
>> 
>> I realized I posted this without cc'ing the list:
>> http://jdebp.info/FGA/do-not-use-logrotate.html .
>> 
>> The link above gives a more detailed description of why log rotation
>> on the Unix system is not only not simple, but is, in fact,
>> unreliable.
>
> Nothing in that article contradicts what I said about renaming log
> files.  His argument is that renaming log files messes with tail -F, and
> therefore broken and unreliable.  Which a pretty strange argument.  tail
> -F might not see some data during the rotation, but the log files
> themselves don't miss anything, which was my contention.  In all my
> years of sysadmin-ing I have never once worried about problems GNU tail
> might have with a file that gets rotated out from under you.  Not sure
> why the author is so fixated on it.

I really wouldn't worry about anything Jonathan de Boyne Pollard says.

From hugo at python.org  Tue Nov 19 16:17:29 2024
From: hugo at python.org (Hugo van Kemenade)
Date: Tue, 19 Nov 2024 23:17:29 +0200
Subject: [RELEASE] Python 3.14.0 alpha 2 is out
Message-ID: <CANKMJBMBna9R6h9oKx3+-jDGetXzd1b3DT0rkJsdPsHB-qdZ+g@mail.gmail.com>

Alpha 2? But Alpha 1 only just came out!

https://www.python.org/downloads/release/python-3140a2/

This is an early developer preview of Python 3.14

Major new features of the 3.14 series, compared to 3.13:

Python 3.14 is still in development. This release, 3.14.0a2 is the second
of seven planned alpha releases.

Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.

During the alpha phase, features may be added up until the start of the
beta phase (2025-05-06) and, if necessary, may be modified or deleted up
until the release candidate phase (2025-07-22). Please keep in mind that
this is a preview release and its use is not recommended for production
environments.

Many new features for Python 3.14 are still being planned and written.
Among the new major new features and changes so far:

* PEP 649: deferred evaluation of annotations
* PEP 741: Python configuration C API
* PEP 761: Python 3.14 and onwards no longer provides PGP signatures for
release artifacts. Instead, Sigstore is recommended for verifiers.
* Improved error messages
* (Hey, fellow core developer, if a feature you find important is missing
from this list, let Hugo know.)

The next pre-release of Python 3.14 will be 3.14.0a3, currently scheduled
for 2024-12-17.

More resources

* Online documentation: https://docs.python.org/3.14/
* PEP 745, 3.14 Release Schedule: https://peps.python.org/pep-0719/
* Report bugs at https://github.com/python/cpython/issues
* Help fund Python and its community: https://www.python.org/psf/donations/

And now for something completely different

Ludolph van Ceulen (1540-1610) was a fencing and mathematics teacher in
Leiden, Netherlands, and spent around 25 years calculating ? (or pi), using
essentially the same methods Archimedes employed some seventeen hundred
years earlier.

Archimedes estimated ? by calculating the circumferences of polygons that
fit just inside and outside of a circle, reasoning the circumference of the
circle lies between these two values. Archimedes went up to polygons with
96 sides, for a value between 3.1408 and 3.1428, which is accurate to two
decimal places.

Van Ceulen used a polygon with half a billion sides. He published a
20-decimal value in his 1596 book Vanden Circkel (?On the Circle?), and
later expanded it to 35 decimals:

3.14159265358979323846264338327950288

Van Ceulen?s 20 digits is more than enough precision for any conceivable
practical purpose. For example, even if a printed circle was perfect down
to the atomic scale, the thermal vibrations of the molecules of ink would
make most of those digits physically meaningless. NASA Jet Propulsion
Laboratory?s highest accuracy calculations, for interplanetary navigation,
uses 15 decimals: 3.141592653589793.

At Van Ceulen?s request, his upper and lower bounds for ? were engraved on
his tombstone in Leiden. The tombstone was eventually lost but restored in
2000. In the Netherlands and Germany, ? is sometimes referred to as the
?Ludolphine number?, after Van Ceulen.

Enjoy the new release

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organisation contributions to the Python
Software Foundation.

Regards from a chilly Helsinki with snow on the way,

Your release team,
Hugo van Kemenade
Ned Deily
Steve Dower
?ukasz Langa

From mk1853387 at gmail.com  Wed Nov 27 07:28:06 2024
From: mk1853387 at gmail.com (marc nicole)
Date: Wed, 27 Nov 2024 13:28:06 +0100
Subject: How to break while loop based on events raised in a thread (Python
 2.7)
Message-ID: <CAGJtH9SuFdPOF80BW2x+obM7-CbaqRuHOnraSmxmD+-yLqOOPw@mail.gmail.com>

I am using the below class1 code to detect some event through the method

check_for_the_event()
      # Some class1def check_for_the_event(self):
thread1 = threading.Timer(2, self.check_for_the_event)
thread1.start()# some processingif event is detected:
   self.event_ok = Trueelse:
   self.event_ok = False

then I pass an instance of that class to the below class to know when the
event is on or off and act upon its value accordingly using the following
below code:

              # execution of other part of the program (where
self.another_class_instance.event_ok = False and
self.some_other_condition is true)
              current_time = current_time_some_cond = current_max_time
= time.time()
              while not self.another_class_instance.event_ok and
self.some_other_condition:
                while self.some_other_condition and not
self.another_class_instance.event_ok:#self.some_other_condition takes
up to 10 secs to be checked (minimum 5 secs)
                    if time.time() > current_time + 5:
                        current_time = time.time()
                        # some processing
                else:
                    while not self.another_class_instance.event_ok:
                        #processing
                        if time.time() > current_max_time + 60 * 2 and
not self.another_class_instance.event_ok:
                            #some other processing
                        if time.time() > current_time_some_cond + 10
and not self.cond1 and not self.another_class_instance.event_ok:
                            # some processing that takes 2-3 seconds
                            self.cond1 = True
                            current_time_some_cond = time.time()
                        elif self.cond1 and time.time() >
current_time_some_cond + 10 and not
self.another_class_instance.event_ok:
                            current_time_some_cond = time.time()
                            #some other processing that takes 2-3 seconds
                    else:
                        pass
            else:
                pass

The problem is the real time execution of the program (in class2) requires
an instant check to be performed and that cannot wait until the current
loop instructions finish, and I want is to break out of the outer while
loop if the event is on without further checking any other condition.

for now I perform multiple checks at each if or while statement, but is
there a IO async based method that breaks out of the loop when the event is
raised in the thread?