From mk1853387 at gmail.com Mon Sep 2 10:00:15 2024 From: mk1853387 at gmail.com (marc nicole) Date: Mon, 2 Sep 2024 16:00:15 +0200 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 Message-ID: Hello, I am using Python 2.7 on Windows 10 and I want to launch a process independently of the rest of the code so that the execution continues while the started process proceeds. I am using Process().start() from Python 2.7 as follows: from multiprocessing import Process def do_something(text): print(text) if __name__ == "__main__": q = Process(target=do_something,args=("somecmd") ) q.start() # following code should execute right after the q.start() call (not until it returns) ..... But getting the error at the call of Process().start(): pickle.PicklingError: Can't pickle : it's not found as __builtin__.module anybody could provide an alternative to call the function do_something() in a separate thread ? From barry at barrys-emacs.org Mon Sep 2 11:36:20 2024 From: barry at barrys-emacs.org (Barry Scott) Date: Mon, 2 Sep 2024 16:36:20 +0100 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: References: Message-ID: > On 2 Sep 2024, at 15:00, marc nicole via Python-list wrote: > > I am using Python 2.7 on Windows 10 Why? Install Python 3.12 and it will be easier to get help and support. If you have legacy that still needs porting then you can install 3.12 along side the unsupported 3.12. Barry From leamhall at gmail.com Mon Sep 2 20:47:10 2024 From: leamhall at gmail.com (Leam Hall) Date: Mon, 2 Sep 2024 19:47:10 -0500 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: References: Message-ID: <5ba5ba23-6636-af68-0f00-6be190471277@gmail.com> On 9/2/24 10:36, Barry Scott wrote: > > >> On 2 Sep 2024, at 15:00, marc nicole via Python-list wrote: >> >> I am using Python 2.7 on Windows 10 > > Why? Install Python 3.12 and it will be easier to get help and support. > If you have legacy that still needs porting then you can install 3.12 along side > the unsupported 3.12. > > > Barry There are a lot of reasons to not upgrade, please don't exclude people who are not in a position to do so. Leam -- DevOps Engineer (reuel.net/resume) Scribe: The Domici War (domiciwar.net) General Ne'er-do-well (github.com/LeamHall) From mats at wichmann.us Mon Sep 2 21:41:42 2024 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 02 Sep 2024 19:41:42 -0600 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: <5ba5ba23-6636-af68-0f00-6be190471277@gmail.com> References: <5ba5ba23-6636-af68-0f00-6be190471277@gmail.com> Message-ID: On September 2, 2024 6:47:10 PM MDT, Leam Hall wrote: >On 9/2/24 10:36, Barry Scott wrote: >> >> >>> On 2 Sep 2024, at 15:00, marc nicole via Python-list wrote: >>> >>> I am using Python 2.7 on Windows 10 >> >> Why? Install Python 3.12 and it will be easier to get help and support. >> If you have legacy that still needs porting then you can install 3.12 along side >> the unsupported 3.12. >> >> >> Barry > >There are a lot of reasons to not upgrade, please don't exclude people who are not in a position to do so. > >Leam > well, in an open source project, "unsupported" means whatever the various volunteers think it means. Personally, I'm not spending any time thinking about issues on Python 2, and I suspect that's the case for many others. there are people you can pay to care... -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From alan.gauld at yahoo.co.uk Tue Sep 3 04:41:15 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 3 Sep 2024 09:41:15 +0100 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: References: Message-ID: On 02/09/2024 15:00, marc nicole via Python-list wrote: > Hello, > > I am using Python 2.7 on Windows 10 Others have pointed out that 2.7 is unsupported and has been for many years now. Its also inferior in most respects including its error reporting. If possible you should upgrade to 3.X > from multiprocessing import Process > def do_something(text): > print(text) > if __name__ == "__main__": > q = Process(target=do_something,args=("somecmd") ) > q.start() > # following code should execute right after the q.start() call So what does happen? If you put a print statement here does it execute before or after the error message? It might make things easier to debug(clearer error traceback) if you put the code to create the thread into a separate function? def do_Something(text)... def start(fn): q = Process.... q.start() if __name_.... start(do_something) print('Something here') > But getting the error at the call of Process().start(): > pickle.PicklingError: Can't pickle : it's not found as > __builtin__.module But please show us the full error trace even if its not much. Also check your module naming, is there a possibility you've named your file do_something.py or similar? (I'm guessing the function is what is being pickled?) > anybody could provide an alternative to call the function do_something() in > a separate thread ? Why not just use the Threading module? If it's as simple as just running something in a thread multiprocessing is probably not needed. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mk1853387 at gmail.com Tue Sep 3 05:34:53 2024 From: mk1853387 at gmail.com (marc nicole) Date: Tue, 3 Sep 2024 11:34:53 +0200 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: References: Message-ID: Hello Alan, Thanks for the reply, Here's the code I tested for the debug: import time from multiprocessing import Process def do_Something(): print('hello world!') def start(fn): p = Process(target=fn, args=()) p.start() def ghello(): print ("hello world g") def fhello(): print('hello world f') if __name__ == "__main__": start(do_something) print("executed") exit(0) but neither "Hello World" or "Executed" are displayed in the console which finishes normally without returning any message. Module naming is OK and don't think it is a problem related to that. Now the question, when to use Process/Multiprocess and when to use Threading in Python?.Thread is there a distinctive use case that can showcase when to use either? are they interchangeable? to note that using Threading the console DID display the messages correctly! Thanks. Le mar. 3 sept. 2024 ? 10:48, Alan Gauld via Tutor a ?crit : > On 02/09/2024 15:00, marc nicole via Python-list wrote: > > Hello, > > > > I am using Python 2.7 on Windows 10 > > Others have pointed out that 2.7 is unsupported and has > been for many years now. Its also inferior in most > respects including its error reporting. > If possible you should upgrade to 3.X > > > from multiprocessing import Process > > def do_something(text): > > print(text) > > if __name__ == "__main__": > > q = Process(target=do_something,args=("somecmd") ) > > q.start() > > # following code should execute right after the q.start() call > > So what does happen? If you put a print statement here does it execute > before or after the error message? It might make things easier to > debug(clearer error traceback) if you put the code to create the thread > into a separate function? > > def do_Something(text)... > > def start(fn): > q = Process.... > q.start() > > if __name_.... > start(do_something) > print('Something here') > > > > But getting the error at the call of Process().start(): > > pickle.PicklingError: Can't pickle : it's not found as > > __builtin__.module > > But please show us the full error trace even if its not much. > > Also check your module naming, is there a possibility > you've named your file do_something.py or similar? > (I'm guessing the function is what is being pickled?) > > > anybody could provide an alternative to call the function do_something() > in > > a separate thread ? > > Why not just use the Threading module? > If it's as simple as just running something in a > thread multiprocessing is probably not needed. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From mats at wichmann.us Tue Sep 3 09:54:51 2024 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 3 Sep 2024 07:54:51 -0600 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: References: Message-ID: <0673d602-68a1-44b0-a51b-f64f8da6dd63@wichmann.us> On 9/3/24 03:34, marc nicole wrote: > Hello Alan, > > Thanks for the reply, Here's the code I tested for the debug: > > import time > from multiprocessing import Process > > def do_Something(): > print('hello world!') > > def start(fn): > p = Process(target=fn, args=()) > p.start() > > def ghello(): > print ("hello world g") > > def fhello(): > print('hello world f') > > if __name__ == "__main__": > start(do_something) > print("executed") > exit(0) > > but neither "Hello World" or "Executed" are displayed in the console which > finishes normally without returning any message. > > Module naming is OK and don't think it is a problem related to that. > > Now the question, when to use Process/Multiprocess and when to use > Threading in Python?.Thread is there a distinctive use case that can > showcase when to use either? are they interchangeable? to note that using > Threading the console DID display the messages correctly! Just generically, threading is good when you have discrete tasks that don't require executing a lot of Python code (the canonical example seems to be fetching things from webservers over the internet), everything happens in one process, but a thread can start a request and the rest of the program can get on with its work ad collect the results when they're avaiable. This is a class of work called I/O Bound. Multiprocessing is better when you have discrete tasks but they're compute intensive. Python (currently) uses a lock around the execution of microcode, so it doesn't matter how many threads are doing big computations, they can't run concurrently (they *appear* to run concurrently, but they're actually taking turns), so here you can instead start multiple processes, and each can run independently. The downside is if those processes need to share information, it's quite a bit more complicated to do so. The final alternative is your problem isn't easily partitioned, and then you don't use either threading or multiprocessing :-) threading and multiprocessing are written to use a very similar API for convenience. The error you were originally seeing (do you still get the pickle error?) comes from an oddity. There are different ways to start a new process, as multiprocessing has to do. On UNIX systems, there's a fork() system call which makes an exact copy of the running process (arranging the code to be shared, and the data to be copy-on-write, so it's fairly efficient). That has long been a little controversial because usually what happens right after fork() is exec() to run a new program, so the previous work is just thrown away - but that's ideal for Python multiprocessing, a copy of the already set up process can just start running (or *would* be ideal, if it wasn't rather tricky to do safely). If that model is not available, you have to spawn a new process and then it has to be set up - this is where Python serializes the important objects (using pickle) and sends them over to the new process so initial data setup can be shared. But not everything can be pickled easily, so multiprocessing has some limitations. Windows doesn't have the fork strategy and uses spawn method by default, thus why pickling gets involved. There's more on that here: https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods Hope this at least helps understanding, even if it doesn't help your specific problem. From avi.e.gross at gmail.com Wed Sep 4 01:19:12 2024 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Wed, 4 Sep 2024 01:19:12 -0400 Subject: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 In-Reply-To: References: Message-ID: <059301dafe89$fac57470$f0505d50$@gmail.com> Unfortunately, Alan, even though 2.7 was considered pickled, people keep taking it back out of the bottle and wondering why it does not work so well! There are companies like Microsoft and Samsung that let people know their OS on their devices will no longer be supported with updates and some apps may no longer work if downloaded. And, yet, I bet for years afterwards, people will refuse to upgrade because they don't want to replace equipment or even learn a new slightly different interface. Having said that, I understand many people are stuck for various reasons and are required to use whatever version is officially allowed. For some questions, answers may still be provided. There are some workarounds or even newer packages designed to do what is not otherwise available. But many of us here may not be answering the questions as we have no reason to be able to access the old software or interest. -----Original Message----- From: Tutor On Behalf Of Alan Gauld via Tutor Sent: Tuesday, September 3, 2024 4:41 AM To: tutor at python.org Cc: python-list at python.org Subject: Re: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7 On 02/09/2024 15:00, marc nicole via Python-list wrote: > Hello, > > I am using Python 2.7 on Windows 10 Others have pointed out that 2.7 is unsupported and has been for many years now. Its also inferior in most respects including its error reporting. If possible you should upgrade to 3.X > from multiprocessing import Process > def do_something(text): > print(text) > if __name__ == "__main__": > q = Process(target=do_something,args=("somecmd") ) > q.start() > # following code should execute right after the q.start() call So what does happen? If you put a print statement here does it execute before or after the error message? It might make things easier to debug(clearer error traceback) if you put the code to create the thread into a separate function? def do_Something(text)... def start(fn): q = Process.... q.start() if __name_.... start(do_something) print('Something here') > But getting the error at the call of Process().start(): > pickle.PicklingError: Can't pickle : it's not found as > __builtin__.module But please show us the full error trace even if its not much. Also check your module naming, is there a possibility you've named your file do_something.py or similar? (I'm guessing the function is what is being pickled?) > anybody could provide an alternative to call the function do_something() in > a separate thread ? Why not just use the Threading module? If it's as simple as just running something in a thread multiprocessing is probably not needed. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From sjeik_appie at hotmail.com Thu Sep 19 16:40:43 2024 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Thu, 19 Sep 2024 22:40:43 +0200 Subject: [Tutor] Side effect and return value Message-ID: Hi, I've just read "Code that fits into your Brain". The author says a function with a side effect should not have a return value. But how would one write this function in order to do that? This looks totally fine to me: from uuid import UUID from requests import post def create_user(info: dict) -> UUID: resp = post("https://some-thing/user", json=info) resp.raise_for_status() return UUID(resp.json()["id"]) Thanks! AJ From cs at cskk.id.au Thu Sep 19 17:16:51 2024 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 20 Sep 2024 07:16:51 +1000 Subject: [Tutor] Side effect and return value In-Reply-To: References: Message-ID: On 19Sep2024 22:40, Albert-Jan Roskam wrote: > I've just read "Code that fits into your Brain". > The author says a function with a side effect should not have a return > value. This is a rule of thumb partitioning functions into those which "do something" and those which "compute something". The typical examples might be `list.append`, which modifies a list and returns `None`, and `list.__len__` aka `len(list)`, which returns the length of the list and changes nothing. This is a good design rule - it broadly lets you use "compute" functions in expressions without worrying that they have side effects. Things with side effects are not provided as useful-in-expressions (i.e. they're not written to return a useful value, which precludes them being enticing in expressions). But not everything is _quite_ that cut and dry as you've discovered. The rule is a way of thinking about functions. So, your example: >But how would one write this function in order to do that? This looks totally fine to me: > > from uuid import UUID > from requests import post > def create_user(info: dict) -> UUID: > resp = post("https://some-thing/user", json=info) > resp.raise_for_status() > return UUID(resp.json()["id"]) This function creates something (the user record) and returns its key for later use. It looks like it "does something", but that's mostly an artifact of its interaction with a storage system: the web server which holds the database records for the users. it isn't very different from a function like this: def list_of_characters(s: str) -> List[str]: return list(s) We're making a new list and returning a reference to the list. You can think of that reference as like the user UUID key above. We might use it in some computation: letters = [ c for c in list_of_characters(s) if c.isalpha() ] It doesn['t really have much of a side effect other than allocating storage for the result of the computation (the new list). Your `create_user` function is similar - it creates a user record and returns a reference. That the reference is a UUID isn't very different. Any "compute" function has side effects of a sort - storage must be allocated for intermediate values and for its result. Your example just brings that into the light. From PythonList at DancesWithMice.info Thu Sep 19 20:06:47 2024 From: PythonList at DancesWithMice.info (dn) Date: Fri, 20 Sep 2024 12:06:47 +1200 Subject: [Tutor] Side effect and return value In-Reply-To: References: Message-ID: <226c8dbf-3537-476e-af60-4837057dbe49@DancesWithMice.info> On 20/09/24 08:40, Albert-Jan Roskam wrote: > Hi, > I've just read "Code that fits into your Brain". > The author says a function with a side effect should not have a return > value. But how would one write this function in order to do that? This > looks totally fine to me: > from uuid import UUID > from requests import post > def create_user(info: dict) -> UUID: > resp = post("https://some-thing/user", json=info) > resp.raise_for_status() > return UUID(resp.json()["id"]) It took me a moment to realise that what I was (trying to) remembering is Mark Seemann's book, "Code that fits in your Head". It does have a chapter with such as its title. However, a quick skim didn't readily-reveal discussion of side-effects, etc. Perplexity.ai saved me some time and typing, offering the following for your reading pleasure: ? Key Characteristics of Functional Programming 1 Pure Functions: A pure function is one where the output is determined solely by its input values, without observable side effects. This means that calling a pure function with the same arguments will always produce the same result, and it does not alter any external state or interact with outside systems (e.g., no modifying global variables or performing I/O operations). 2 Avoiding Side Effects: Functional programming aims to minimize side effects, which are changes in state that occur outside a given function's scope. By avoiding side effects, programs become easier to understand, test, and debug. This is because functions behave predictably and independently of the program's state. 3 Immutability: Data is immutable in functional programming, meaning once a data structure is created, it cannot be changed. Instead, new data structures are created when modifications are needed. This immutability helps prevent unintended side effects. 4 Higher-Order Functions: These are functions that can take other functions as arguments or return them as results. This allows for greater abstraction and code reuse. 5 Declarative Style: Functional programming focuses on what to solve rather than how to solve it, using expressions and declarations instead of statements. ? Sadly, the difference between 'theory' and 'practice' is that it is impossible to write an application which contains ONLY "pure functions". The example code (above) MUST perform I/O and step outside of the application. So, and particularly in Python, Functional Programming involves acceptance and compromise. The specific question: ? In functional programming, it's often advised that "a function with a side effect should not have a return value." This guideline helps maintain clear boundaries between functions that produce values and those that perform actions (side effects). By separating these concerns, the code becomes more modular and easier to reason about. ? @Cameron's post illustrates such very neatly. Accordingly, the 'compromise': - if the routine is all about a side-effect (the POST and response), forget that convention - if the routine is 'pure', then follow it. ie it's good advice worth following, but maybe with some 'situational adherence'. - or does "nuanced approach" sound more judicious? -- Regards, =dn From mats at wichmann.us Fri Sep 20 11:31:23 2024 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 20 Sep 2024 09:31:23 -0600 Subject: [Tutor] Side effect and return value In-Reply-To: <226c8dbf-3537-476e-af60-4837057dbe49@DancesWithMice.info> References: <226c8dbf-3537-476e-af60-4837057dbe49@DancesWithMice.info> Message-ID: On 9/19/24 18:06, dn via Tutor wrote: > Key Characteristics of Functional Programming > > 1 Pure Functions: A pure function is one where the output is determined > solely by its input values, without observable side effects. This means > that calling a pure function with the same arguments will always produce > the same result, and it does not alter any external state or interact > with outside systems (e.g., no modifying global variables or performing > I/O operations). > > 2 Avoiding Side Effects: Functional programming aims to minimize side > effects, which are changes in state that occur outside a given > function's scope. By avoiding side effects, programs become easier to > understand, test, and debug. This is because functions behave > predictably and independently of the program's state. > > 3 Immutability: Data is immutable in functional programming, meaning > once a data structure is created, it cannot be changed. Instead, new > data structures are created when modifications are needed. This > immutability helps prevent unintended side effects. I think this is a clear example that while Python can be used for some level of functional programming it is not designed as such. Many builtin Python data structures, and thus many "works like" implementations that take advantage of the familiarity with those basics, are mutable, and their methods often have side effects AND return values. Consider the pop() method of a dict: pop(...) method of builtins.dict instance D.pop(k[,d]) -> v, remove specified key and return the corresponding value. From sjeik_appie at hotmail.com Fri Sep 20 11:02:45 2024 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Fri, 20 Sep 2024 17:02:45 +0200 Subject: [Tutor] Side effect and return value In-Reply-To: <226c8dbf-3537-476e-af60-4837057dbe49@DancesWithMice.info> Message-ID: On Sep 20, 2024 02:06, dn via Tutor wrote: On 20/09/24 08:40, Albert-Jan Roskam wrote: > Hi, > I've just read "Code that fits into your Brain". > The author says a function with a side effect should not have a return > value. But how would one write this function in order to do that? It took me a moment to realise that what I was (trying to) remembering is Mark Seemann's book, "Code that fits in your Head". === That's indeed the book I meant. Loved reading it. He also mentioned that learning/using functional principles had also been very useful for his OOP skills. But I think you're right, one has to be pragmatic sometimes. 1 Pure Functions: A pure function is one where the output is determined solely by its input values, without observable side effects. 2 Avoiding Side Effects: Functional programming aims to minimize side effects, which are changes in state that occur outside a given function's scope. ==== Hmmm, the function in my example was not pure in two ways: the GUIDs in the json make the return value impossible to predict AND it has a side effect (a record is INSERTed in a table). Oh well. :-) @Cameron's post illustrates such very neatly. === Did Cameron also reply to my post? I didn't see anything. Is there also a news group for Python Tutor? The mailing lists have been so quiet lately. Comp.lang.python has more traffic the mirrored mailing list From PythonList at DancesWithMice.info Fri Sep 20 12:07:09 2024 From: PythonList at DancesWithMice.info (dn) Date: Sat, 21 Sep 2024 04:07:09 +1200 Subject: [Tutor] Side effect and return value In-Reply-To: References: Message-ID: On 21/09/24 03:02, Albert-Jan Roskam wrote: ... > @Cameron's post illustrates such very neatly. > > > === > Did Cameron also reply to my post? I didn't see anything. Is there also > a news group for Python Tutor? The mailing lists have been so quiet > lately. Comp.lang.python has more traffic the mirrored mailing list Indeed he did - To: your address Cc: the list See also: https://mail.python.org/pipermail/tutor/2024-September/thread.html -- Regards, =dn From sjeik_appie at hotmail.com Fri Sep 20 13:46:49 2024 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Fri, 20 Sep 2024 19:46:49 +0200 Subject: [Tutor] Side effect and return value In-Reply-To: Message-ID: On Sep 20, 2024 18:07, dn wrote: On 21/09/24 03:02, Albert-Jan Roskam wrote: ... > @Cameron's post illustrates such very neatly. > > > === > Did Cameron also reply to my post? I didn't see anything. Is there also > a news group for Python Tutor? The mailing lists have been so quiet > lately. Comp.lang.python has more traffic the mirrored mailing list Indeed he did - To: your address Cc: the list See also: https://mail.python.org/pipermail/tutor/2024-September/thread.html ===== Thanks for the url. Strange, I checked & double-checked but I didn't receive Cameron's mail. I now included him manually in the list of reply-to addresses. Something may have changed in the listserv settings. I'll ask Alan. @Cameron: thanks for your reply, I appreciate it. From cs at cskk.id.au Fri Sep 20 18:10:08 2024 From: cs at cskk.id.au (Cameron Simpson) Date: Sat, 21 Sep 2024 08:10:08 +1000 Subject: [Tutor] Side effect and return value In-Reply-To: References: Message-ID: On 20Sep2024 19:46, Albert-Jan Roskam wrote: > See also: > https://mail.python.org/pipermail/tutor/2024-September/thread.html > ===== > Thanks for the url. Strange, I checked & double-checked but I didn't > receive Cameron's mail. I now included him manually in the list of > reply-to addresses. Something may have changed in the listserv > settings. I'll ask Alan. This won't help you :-( I got a bounce from hotmail saying my mail server was in a IP range block list. I'll try to chastise them. > @Cameron: thanks for your reply, I appreciate it. Any time. - Cameron Simpson From avi.e.gross at gmail.com Fri Sep 20 21:26:52 2024 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Fri, 20 Sep 2024 21:26:52 -0400 Subject: [Tutor] Side effect and return value In-Reply-To: References: <226c8dbf-3537-476e-af60-4837057dbe49@DancesWithMice.info> Message-ID: <035301db0bc5$56b7cab0$04276010$@gmail.com> The following is OPINION except where it is FACT. And this is long enough that some of you should hit NEXT. The topic is about one form of purity or, frankly, religion, in Computer Science and related fields. There is a school of thought that makes a fairly valid point. If you want a high probability of code that does not cause odd errors, and can mathematically be shown to in some sense be optimal, then you can restrict a language drastically so it does a very few things well and other things with difficulty and others not at all. In the real world people rapidly move on if they can as these are toy languages in many ways. It is true that many things in CS can be written in ways that work decently and mostly for small samples. Consider something as simple as adding things to a list. In some languages, that means you slide forward one link at a time to the last item which points to something like NIL and add an item. By the time you are adding thousands, this slows down. Someone suggest an idea. Build the list backwards as inserting a new front item takes constant time. When done, reverse the list just once! That is an example of twisting the algorithm when you could have had a form of linear list that keeps a pointer both to the head and tail and can be added to trivially. That would make the operation fast but opens possible vulnerabilities. Along come purists and suggest every addition to a new list must be done by asking some central code to make a COPY with the added item. The old version now can be garbage collected as nothing else may be allowed to "point" to it. Therefore, your code typically requires a call to a function to result in receiving the new list and at the same time, the old list is no longer available or in scope. Often you have to stand on your head in these scenarios to get things done before winter. Or take recursion. Some languages want to do almost everything recursively. Nice but it can eat memory having a hundred function calls on the stack when some iterative solution takes up no space at all. But, if you can somehow rewrite the algorithm to use tail recursion, all is fine as each function call replaces the previous one on the stack. Too many other examples like this make you realize that purists are too pure to be allowed to program. So, consider the opposite. A language like Python supports objects and a way to program using object-oriented paradigms. Many, probably most, languages now have such ideas to one degree or another. But objects have all kinds of "state" that is or becomes a part of them. All kinds of operations change them internally. They can become quite large and contain other objects within them or off to the side. Talk about side effects because so much of what you do to them just changes the internals and the results may be seen later or not at all. Imagine an object that holds some statistical analysis and there is a way to specify options later, such as to use a different method of sorting by providing a function to use so that filenames like "S3E22" will sort before "S12E123". This is a real example I recently worked on. When you set this, nothing much happens. But later, when the object has a need to sort the data it has been given, it now does something different than before and sorts the lists ["S", 3, "E", 22] versus ["S", 12, "E", 123] which sorts differently than just alphabetically would have. If objects were immutable, and some are, then I see a need for massive amounts of copying as any change to an object cannot happen unless it is done anew. Older versions may persist. I see a potential mess. You almost want to construct an object all at once. So, just being object oriented may make the concept of a function doing either a side-effect or return something a bit fuzzy. But python has MANY other ideas and constructs that simply do not easily go away and keep changing. One is closures where you can create functions that somehow hold on to variables so that calling them again and again allows them to retrieve and change those variables. They act a tad like objects. Another is generators of several kinds that do not finish but yield something and sleep till called again and yield some more. Some can also receive something. Calling a generator to keep returning the next prime will thus change the generator to hold on to the last rime it calculated so it can next time start from right after that. It sounds like a variant of a closure or it can look like an object that can be called and stores the change each time. I will stop but assure you there are so many more ideas like currying and other functional programming methods that are very useful and yet seem to violate all kinds of idealistic rules. Bring in parallel programming and the mess gets messier. But, that does not mean you should not follow the advice when feasible. If you do not need a mutable list but can use a tuple, then you probably should. Some mistakes would then perhaps be caught. My opinion is that python has been way too lax compared to some other languages and that can be good. The attempts at adding variations on TYPING to the language as options has some merit as it allows programmers to state clearly their intent so that linters can detect many anomalies. This may be a reasonable compromise between purity and practicality, if done well. -----Original Message----- From: Tutor On Behalf Of Albert-Jan Roskam Sent: Friday, September 20, 2024 11:03 AM To: dn Cc: tutor at python.org Subject: Re: [Tutor] Side effect and return value On Sep 20, 2024 02:06, dn via Tutor wrote: On 20/09/24 08:40, Albert-Jan Roskam wrote: > Hi, > I've just read "Code that fits into your Brain". > The author says a function with a side effect should not have a return > value. But how would one write this function in order to do that? It took me a moment to realise that what I was (trying to) remembering is Mark Seemann's book, "Code that fits in your Head". === That's indeed the book I meant. Loved reading it. He also mentioned that learning/using functional principles had also been very useful for his OOP skills. But I think you're right, one has to be pragmatic sometimes. 1 Pure Functions: A pure function is one where the output is determined solely by its input values, without observable side effects. 2 Avoiding Side Effects: Functional programming aims to minimize side effects, which are changes in state that occur outside a given function's scope. ==== Hmmm, the function in my example was not pure in two ways: the GUIDs in the json make the return value impossible to predict AND it has a side effect (a record is INSERTed in a table). Oh well. :-) @Cameron's post illustrates such very neatly. === Did Cameron also reply to my post? I didn't see anything. Is there also a news group for Python Tutor? The mailing lists have been so quiet lately. Comp.lang.python has more traffic the mirrored mailing list _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From xingyuanshi17 at gmail.com Tue Sep 24 22:23:21 2024 From: xingyuanshi17 at gmail.com (Xingyuan Shi) Date: Wed, 25 Sep 2024 10:23:21 +0800 Subject: [Tutor] Clarification on .format() Method and Negative Indices in Python Message-ID: Dear Python Tutor Team, I hope you are having a great day. I have been learning more about Python?s string formatting, and I came across an interesting issue regarding the .format() method. Specifically, I was curious as to why negative indices like {-1} are not supported in this method, while negative indexing works perfectly with lists and other sequences in Python. For instance, I attempted to use the following code: print("{-1},{-1},{-1}".format(20, "example", 3.14)) I expected the placeholders {-1} to behave similarly to negative indices in lists, but it resulted in an error. I understand that .format() relies on positional arguments, but I wonder if there was a specific design rationale behind not allowing negative indices here. Could you shed some light on this design choice? Thank you very much for your time and for developing such an incredible language that I thoroughly enjoy learning. Best regards, Stone From alan.gauld at yahoo.co.uk Wed Sep 25 11:03:27 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 25 Sep 2024 16:03:27 +0100 Subject: [Tutor] Clarification on .format() Method and Negative Indices in Python In-Reply-To: References: Message-ID: On 25/09/2024 03:23, Xingyuan Shi via Tutor wrote: > was curious as to why negative indices like {-1} are not supported in this > method, while negative indexing works perfectly with lists and other > sequences in Python. Yes, it is odd. I can see why it might be useful too. OTOH its not something I've ever looked for so maybe whoever designed it just didn't think it necessary. However, when it comes to design decisions you are probably better asking on the main python mailing list as more of the core team hang out there. You could also try the Python dev mailing list and look for the PEP that covers string formatting(3101). You might find a discussion there. Here is the original PEP: https://peps.python.org/pep-3101/ > For instance, I attempted to use the following code: > > print("{-1},{-1},{-1}".format(20, "example", 3.14)) > > I expected the placeholders {-1} to behave similarly to negative indices -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Wed Sep 25 12:28:43 2024 From: mats at wichmann.us (Mats Wichmann) Date: Wed, 25 Sep 2024 10:28:43 -0600 Subject: [Tutor] Clarification on .format() Method and Negative Indices in Python In-Reply-To: References: Message-ID: <6518ccb1-b5ab-41d7-8fda-b574f2e75129@wichmann.us> On 9/25/24 09:03, Alan Gauld via Tutor wrote: > On 25/09/2024 03:23, Xingyuan Shi via Tutor wrote: > >> was curious as to why negative indices like {-1} are not supported in this >> method, while negative indexing works perfectly with lists and other >> sequences in Python. > > Yes, it is odd. I can see why it might be useful too. > OTOH its not something I've ever looked for so maybe whoever > designed it just didn't think it necessary. > > However, when it comes to design decisions you are probably > better asking on the main python mailing list as more of > the core team hang out there. > > You could also try the Python dev mailing list and look for > the PEP that covers string formatting(3101). You might find > a discussion there. the Python dev list is dead - there was (grudging) agreement to move all that to discuss.python.org, and that's probably the best place to ask the question, as things are fairly well categorized and thus tend to reach the right people (python-dev was kind of like a firehose for some of the developers who didn't have much time, part of the reason for its demise). If you find anything useful it might be fun to post it here, too! > > Here is the original PEP: > > https://peps.python.org/pep-3101/ From mk1853387 at gmail.com Wed Sep 25 13:24:49 2024 From: mk1853387 at gmail.com (marc nicole) Date: Wed, 25 Sep 2024 19:24:49 +0200 Subject: [Tutor] How to stop a specific thread in Python 2.7? Message-ID: Hello guys, I want to know how to kill a specific running thread (say by its id) for now I run and kill a thread like the following: # start thread thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) thread1.start() # kill the thread event_thread1 = threading.Event() event_thread1.set() I know that set() will kill all running threads, but if there was thread2 as well and I want to kill only thread1? Thanks! From mats at wichmann.us Wed Sep 25 14:25:36 2024 From: mats at wichmann.us (Mats Wichmann) Date: Wed, 25 Sep 2024 12:25:36 -0600 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: Message-ID: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> On 9/25/24 11:24, marc nicole via Tutor wrote: > Hello guys, > > I want to know how to kill a specific running thread (say by its id) > > for now I run and kill a thread like the following: > # start thread > thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) > thread1.start() > # kill the thread > event_thread1 = threading.Event() > event_thread1.set() > > I know that set() will kill all running threads, but if there was thread2 > as well and I want to kill only thread1? There's no official way - as usual we'll come back with a question (well, I will): Why Would You Want To Do That? Threads often hold critical-section locks and other important things and killing such a thread will put you in an undefined, possibly unstable state. That may not be your case... it would be useful if you described a use case that causes you to want to kill an individual thread. Or are you just curious? From mats at wichmann.us Wed Sep 25 14:33:40 2024 From: mats at wichmann.us (Mats Wichmann) Date: Wed, 25 Sep 2024 12:33:40 -0600 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> References: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> Message-ID: <05245899-744d-4b6e-bc08-399560065bc3@wichmann.us> On 9/25/24 12:25, Mats Wichmann wrote: > On 9/25/24 11:24, marc nicole via Tutor wrote: >> Hello guys, >> >> I want to know how to kill a specific running thread (say by its id) >> >> for now I run and kill a thread like the following: >> # start thread >> thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) >> thread1.start() >> # kill the thread >> event_thread1 = threading.Event() >> event_thread1.set() >> >> I know that set() will kill all running threads, but if there was thread2 >> as well and I want to kill only thread1? > There's no official way - as usual we'll come back with a question > (well, I will): Why Would You Want To Do That??? Threads often hold > critical-section locks and other important things and killing such a > thread will put you in an undefined, possibly unstable state.? That may > not be your case... it would be useful if you described a use case that > causes you to want to kill an individual thread.? Or are you just curious? eh, should have added, but sent too soon: it's absolutely fine to set a flag so that the thread knows to kill itself - that way things can happen cleanly. From mats at wichmann.us Wed Sep 25 16:26:42 2024 From: mats at wichmann.us (Mats Wichmann) Date: Wed, 25 Sep 2024 14:26:42 -0600 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> Message-ID: On 9/25/24 13:57, marc nicole wrote: > Why i want to kill a thread? > > At first, i want to look for two values through a thread, if i got them > before an event happens, that's cool, the thread will end itself and i > will use the values in the subsequent code, if not i want to kill the > background running function/thread, and relaunch the function of thread > (blocking call this time) to get the values and then continue the program. > > The context is robotics: So to start out, from the documentation: "currently, there are no priorities, no thread groups, and threads cannot be destroyed, stopped, suspended, resumed, or interrupted." (see the bits just above this heading: https://docs.python.org/3.12/library/threading.html#thread-local-data) So you need to find a way to extend your model to the negative case as well, so the thread stops itself even if the ideal case didn't occur. I think one approach is to use Event, as you've already started to think about, I'd just change the naming to be more clear: should_stop = threading.Event() # then at some point: should_stop.set() and in your thread: while not should_stop.wait(1): # do whatever work you can do I'm guessing you've already been here and this isn't new info. Maybe someone else has more experience... From stuff I've heard, robotics folks are making use of async I/O to solve these situations, rather than threads. Unfortunately, Python's async is a bit of a pain as it's kind of like a virus: a function that calls async routines must itself be async, and so on... it kind of spreads throughout your program this way, and can be a lot of work unless you built it from scratch as async. From cs at cskk.id.au Wed Sep 25 16:44:09 2024 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 26 Sep 2024 06:44:09 +1000 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: Message-ID: On 25Sep2024 19:24, marc nicole wrote: >I want to know how to kill a specific running thread (say by its id) > >for now I run and kill a thread like the following: ># start thread >thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) >thread1.start() ># kill the thread >event_thread1 = threading.Event() >event_thread1.set() > >I know that set() will kill all running threads, but if there was thread2 >as well and I want to kill only thread1? No, `set()` doesn't kill a thread at all. It sets the `Event`, and each thread must be checking that event regularly, and quitting if it becomes set. You just need a per-thred vent instead of a single Event for all the threads. Cheers, Cameron Simpson From mk1853387 at gmail.com Wed Sep 25 15:57:48 2024 From: mk1853387 at gmail.com (marc nicole) Date: Wed, 25 Sep 2024 21:57:48 +0200 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> References: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> Message-ID: Why i want to kill a thread? At first, i want to look for two values through a thread, if i got them before an event happens, that's cool, the thread will end itself and i will use the values in the subsequent code, if not i want to kill the background running function/thread, and relaunch the function of thread (blocking call this time) to get the values and then continue the program. The context is robotics: The robot should run the thread in the state idle to get values needed for a task, but if the user talks to him and the values weren't captured yet, the robot should run the function again in a blocking manner until he gets the values for the task, while stopping the initial call to it through the thread. On Wed, 25 Sept 2024, 20:27 Mats Wichmann, wrote: > On 9/25/24 11:24, marc nicole via Tutor wrote: > > Hello guys, > > > > I want to know how to kill a specific running thread (say by its id) > > > > for now I run and kill a thread like the following: > > # start thread > > thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) > > thread1.start() > > # kill the thread > > event_thread1 = threading.Event() > > event_thread1.set() > > > > I know that set() will kill all running threads, but if there was thread2 > > as well and I want to kill only thread1? > There's no official way - as usual we'll come back with a question > (well, I will): Why Would You Want To Do That? Threads often hold > critical-section locks and other important things and killing such a > thread will put you in an undefined, possibly unstable state. That may > not be your case... it would be useful if you described a use case that > causes you to want to kill an individual thread. Or are you just curious? > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From mk1853387 at gmail.com Wed Sep 25 16:56:58 2024 From: mk1853387 at gmail.com (marc nicole) Date: Wed, 25 Sep 2024 22:56:58 +0200 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: Message-ID: How to create a per-thread event in Python 2.7? On Wed, 25 Sept 2024, 22:47 Cameron Simpson via Python-list, < python-list at python.org> wrote: > On 25Sep2024 19:24, marc nicole wrote: > >I want to know how to kill a specific running thread (say by its id) > > > >for now I run and kill a thread like the following: > ># start thread > >thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) > >thread1.start() > ># kill the thread > >event_thread1 = threading.Event() > >event_thread1.set() > > > >I know that set() will kill all running threads, but if there was thread2 > >as well and I want to kill only thread1? > > No, `set()` doesn't kill a thread at all. It sets the `Event`, and each > thread must be checking that event regularly, and quitting if it becomes > set. > > You just need a per-thred vent instead of a single Event for all the > threads. > > Cheers, > Cameron Simpson > -- > https://mail.python.org/mailman/listinfo/python-list > From olegsivokon at gmail.com Wed Sep 25 16:14:43 2024 From: olegsivokon at gmail.com (Left Right) Date: Wed, 25 Sep 2024 22:14:43 +0200 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: Message-ID: That's one of the "disadvantages" of threads: you cannot safely stop a thread. Of course you could try, but that's never a good idea. The reason for this is that threads share memory. They might be holding locks that, if killed, will never be unlocked. They might (partially) modify the shared state observed by other threads in such a way that it becomes unusable to other threads. So... if you want to kill a thread, I'm sorry to say this: you will have to bring down the whole process, there's really no other way, and that's not Python-specific, this is just the design of threads. On Wed, Sep 25, 2024 at 7:26?PM marc nicole via Python-list wrote: > > Hello guys, > > I want to know how to kill a specific running thread (say by its id) > > for now I run and kill a thread like the following: > # start thread > thread1 = threading.Thread(target= self.some_func(), args=( ...,), ) > thread1.start() > # kill the thread > event_thread1 = threading.Event() > event_thread1.set() > > I know that set() will kill all running threads, but if there was thread2 > as well and I want to kill only thread1? > > Thanks! > -- > https://mail.python.org/mailman/listinfo/python-list From cs at cskk.id.au Wed Sep 25 21:06:03 2024 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 26 Sep 2024 11:06:03 +1000 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: Message-ID: On 25Sep2024 22:56, marc nicole wrote: >How to create a per-thread event in Python 2.7? Every time you make a Thread, make an Event. Pass it to the thread worker function and keep it to hand for your use outside the thread. From alan.gauld at yahoo.co.uk Thu Sep 26 05:41:11 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 26 Sep 2024 10:41:11 +0100 Subject: [Tutor] Clarification on .format() Method and Negative Indices in Python In-Reply-To: <6518ccb1-b5ab-41d7-8fda-b574f2e75129@wichmann.us> References: <6518ccb1-b5ab-41d7-8fda-b574f2e75129@wichmann.us> Message-ID: On 25/09/2024 17:28, Mats Wichmann wrote: > On 9/25/24 09:03, Alan Gauld via Tutor wrote: >> You could also try the Python dev mailing list and look for >> the PEP that covers string formatting(3101). You might find >> a discussion there. > > the Python dev list is dead - there was (grudging) agreement to move all > that to discuss.python.org, Yes, but the archive is still available and that's what I was thinking of searching. To see if there was a discussion at the time it was implemented. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From sjeik_appie at hotmail.com Thu Sep 26 12:57:15 2024 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Thu, 26 Sep 2024 18:57:15 +0200 Subject: [Tutor] Type hint question Message-ID: Hi, I just started using type hints and mypy so this is probably a basic question. Why doesn't like mypy the code below? Also, I noticed that mypy seems to favour LBYL (if isinstance) to EAFP (try-except), in the sense that doesn't understand the latter well. Is this true? I'm still learning to use mypy, but sometimes it feels a lot like "pleasing mypy". Then again, it has already helped me find some errors. from typing import TypeVar T = TypeVar("T") def try_int(value: T) -> T: try: return int(value.strip(" ")) except (TypeError, AttributeError, ValueError): return value n = try_int(" 1") I get this message: main.py:7: error: Incompatible return value type (got "int", expected "T") [return-value] main.py:7: error: "T" has no attribute "strip" [attr-defined] Found 2 errors in 1 file (checked 1 source file) Thanks! Albert-Jan From oscar.j.benjamin at gmail.com Thu Sep 26 13:50:25 2024 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 26 Sep 2024 18:50:25 +0100 Subject: [Tutor] Type hint question In-Reply-To: References: Message-ID: On Thu, 26 Sept 2024 at 18:12, Albert-Jan Roskam wrote: > > Hi, > I just started using type hints and mypy so this is probably a basic > question. Why doesn't like mypy the code below? Also, I noticed that mypy > seems to favour LBYL (if isinstance) to EAFP (try-except), in the sense > that doesn't understand the latter well. Is this true? Sort of if by EAFP you mean that you'll call code that is not necessarily valid for the type and then catch the exceptions for the wrong type. The type checker checks that the arguments are used as expected though so if you mean to catch TypeError somewhere then it suggests that your types are not consistent somehow. > I'm still learning > to use mypy, but sometimes it feels a lot like "pleasing mypy". Then > again, it has already helped me find some errors. > > from typing import TypeVar > T = TypeVar("T") > def try_int(value: T) -> T: > try: > return int(value.strip(" ")) > except (TypeError, AttributeError, ValueError): > return value > > n = try_int(" 1") > I get this message: > main.py:7: error: Incompatible return value type (got "int", expected "T") > [return-value] > main.py:7: error: "T" has no attribute "strip" [attr-defined] > Found 2 errors in 1 file (checked 1 source file) Those errors are reasonable. You have declared the parameter and return types to both be T. That means that this should return the same type as the input although note that "type" here is not exactly the same as "type" in the usual Python sense although usually it means the same thing. You are calling the function with a string and it will return an int. This is not consistent with saying that the return type is the same as the parameter type unless T is a sufficiently general type that both str and int are subtypes of T. Your function also returns different types, int or T, depending on the codepath which is broadly not the kind of thing that you should expect a typechecker to be happy with. I think here that mypy would expect you to change this a bit like: def try_int(value: T) -> T | int: if isinstance(value, str): try: return int(value.strip()) except ValueError: pass return value So the return value type is either the same as the input (T) or it is an int. You also don't call the .strip() method without first checking that you actually have a string which can be checked by the parameter type hint or isinstance() here since the parameter is T and not str. Personally I wouldn't use "EAFP" like this for TypeError. Catching the ValueError for a string whose value might not be valid seems reasonable but types I would check explicitly whether with a TypeChecker or isinstance() and I would only use isinstance() at the boundaries. Internal code should be able to work with the expected types without the need for runtime checks (a typechecker helps to ensure this). I would say then that the problem you have with this function is not mypy being picky but you defining a poorly typed function. Clearly the argument is supposed to be a string but it is being called with things that are not strings. That suggests other problems elsewhere like the code calling this function also should be changed. There are ways to make the typechecker accept this but really the idea would be to avoid writing code like this. -- Oscar From mats at wichmann.us Thu Sep 26 14:10:58 2024 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 26 Sep 2024 12:10:58 -0600 Subject: [Tutor] Type hint question In-Reply-To: References: Message-ID: <167f871e-39d0-4317-b306-a06297c92385@wichmann.us> On 9/26/24 10:57, Albert-Jan Roskam wrote: > Hi, > I just started using type hints and mypy so this is probably a basic > question. Why doesn't like mypy the code below? Also, I noticed that mypy > seems to favour LBYL (if isinstance) to EAFP (try-except), in the sense > that doesn't understand the latter well. Is this true? I'm still learning > to use mypy, but sometimes it feels a lot like "pleasing mypy". Then > again, it has already helped me find some errors. > from typing import TypeVar > T = TypeVar("T") > def try_int(value: T) -> T: so that says *value* can be any type, but the return value must be the same type > try: > return int(value.strip(" ")) and here, if *value* was a string, you perform a string op on it and won't get an exception, then turn that into an int for returning - which clearly isn't the same type as was passed in. Ergo, some complaint should be expected. > except (TypeError, AttributeError, ValueError): > return value > The TypeVar will probably be more useful if it applies some contraint... From sjeik_appie at hotmail.com Thu Sep 26 15:05:49 2024 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Thu, 26 Sep 2024 21:05:49 +0200 Subject: [Tutor] Type hint question In-Reply-To: Message-ID: On Sep 26, 2024 19:50, Oscar Benjamin via Tutor wrote: On Thu, 26 Sept 2024 at 18:12, Albert-Jan Roskam wrote: > > > n = try_int(" 1") > I get this message: > main.py:7: error: Incompatible return value type (got "int", expected "T") > [return-value] > main.py:7: error: "T" has no attribute "strip" [attr-defined] > Found 2 errors in 1 file (checked 1 source file) Those errors are reasonable. You have declared the parameter and return types to both be T. That means that this should return the same type as the input although note that "type" here is not exactly the same as "type" in the usual Python sense although usually it means the same thing. You are calling the function with a string and it will return an int. This is not consistent with saying that the return type is the same ==== Arrgh. Of course! How stupid of me. **** That makes perfect sense. Thank you. Personally I wouldn't use "EAFP" like this for TypeError. Catching the ValueError for a string whose value might not be valid seems reasonable but types I would check explicitly whether with a TypeChecker or isinstance() and I would only use isinstance() at the boundaries. ==== Isn't try-except cheaper if the "except" is relatively rare? Also, I thought try/except wss considered to be more pythonic... but I don't know where I read that. From sjeik_appie at hotmail.com Thu Sep 26 15:10:53 2024 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Thu, 26 Sep 2024 21:10:53 +0200 Subject: [Tutor] Type hint question In-Reply-To: <167f871e-39d0-4317-b306-a06297c92385@wichmann.us> Message-ID: On Sep 26, 2024 20:10, Mats Wichmann wrote: On 9/26/24 10:57, Albert-Jan Roskam wrote: > Hi, > I just started using type hints and mypy so this is probably a basic > question. Why doesn't like mypy the code below? Also, I noticed that mypy > seems to favour LBYL (if isinstance) to EAFP (try-except), in the sense > that doesn't understand the latter well. Is this true? I'm still learning > to use mypy, but sometimes it feels a lot like "pleasing mypy". Then > again, it has already helped me find some errors. > from typing import TypeVar > T = TypeVar("T") > def try_int(value: T) -> T: so that says *value* can be any type, but the return value must be the same type ==== Thanks! Maybe it was fatigue but I didn't see this earlier, even though it's so obvious! > try: > return int(value.strip(" ")) and here, if *value* was a string, you perform a string op on it and won't get an exception, then turn that into an int for returning - which clearly isn't the same type as was passed in. Ergo, some complaint should be expected. > except (TypeError, AttributeError, ValueError): > return value > The TypeVar will probably be more useful if it applies some contraint... === You mean TypeVar("T", str, int) or something similar? Or "if isinstance(value, str)"? From mats at wichmann.us Thu Sep 26 15:32:28 2024 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 26 Sep 2024 13:32:28 -0600 Subject: [Tutor] Type hint question In-Reply-To: References: Message-ID: <1a7a1184-d9b5-47a7-ac36-b2f4b4d46e93@wichmann.us> On 9/26/24 13:10, Albert-Jan Roskam wrote: > The TypeVar will probably be more useful if it applies some contraint... > > > === > You mean TypeVar("T", str, int) or something similar? Or "if > isinstance(value, str)"? I meant the former. From avi.e.gross at gmail.com Wed Sep 25 20:50:56 2024 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Wed, 25 Sep 2024 20:50:56 -0400 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> Message-ID: <00a601db0fae$2565b430$70311c90$@gmail.com> There is a low tech approach if you are willing to let the thread run a little longer or use up some time checking. I see others suggesting similar things. There are many ways other parts of your program can set a flag that the thread can occasionally check and then quit. Some are as simple as creating a temporary file with a known name and the thread would check periodically and when found, delete it and quit. And, of course, various methods can be used for a sort of messaging in memory. But as noted by Mats, the best kill is suicide after settling your affairs. In the real world, I suspect this is a solved problem. Imagine for example if you decide to simultaneously attack a problem in multiple ways and the first one to finish means the others need not continue. An example might be trying to determine what language a paragraph is in that will start threads checking to see if it matches one or another language and returning if the likelihood is very high. -----Original Message----- From: Tutor On Behalf Of Mats Wichmann Sent: Wednesday, September 25, 2024 4:27 PM To: marc nicole Cc: tutor at python.org Subject: Re: [Tutor] How to stop a specific thread in Python 2.7? On 9/25/24 13:57, marc nicole wrote: > Why i want to kill a thread? > > At first, i want to look for two values through a thread, if i got them > before an event happens, that's cool, the thread will end itself and i > will use the values in the subsequent code, if not i want to kill the > background running function/thread, and relaunch the function of thread > (blocking call this time) to get the values and then continue the program. > > The context is robotics: So to start out, from the documentation: "currently, there are no priorities, no thread groups, and threads cannot be destroyed, stopped, suspended, resumed, or interrupted." (see the bits just above this heading: https://docs.python.org/3.12/library/threading.html#thread-local-data) So you need to find a way to extend your model to the negative case as well, so the thread stops itself even if the ideal case didn't occur. I think one approach is to use Event, as you've already started to think about, I'd just change the naming to be more clear: should_stop = threading.Event() # then at some point: should_stop.set() and in your thread: while not should_stop.wait(1): # do whatever work you can do I'm guessing you've already been here and this isn't new info. Maybe someone else has more experience... From stuff I've heard, robotics folks are making use of async I/O to solve these situations, rather than threads. Unfortunately, Python's async is a bit of a pain as it's kind of like a virus: a function that calls async routines must itself be async, and so on... it kind of spreads throughout your program this way, and can be a lot of work unless you built it from scratch as async. _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From mk1853387 at gmail.com Wed Sep 25 21:34:05 2024 From: mk1853387 at gmail.com (marc nicole) Date: Thu, 26 Sep 2024 03:34:05 +0200 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: Message-ID: Could you show a python code example of this? On Thu, 26 Sept 2024, 03:08 Cameron Simpson, wrote: > On 25Sep2024 22:56, marc nicole wrote: > >How to create a per-thread event in Python 2.7? > > Every time you make a Thread, make an Event. Pass it to the thread > worker function and keep it to hand for your use outside the thread. > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From mk1853387 at gmail.com Thu Sep 26 06:00:54 2024 From: mk1853387 at gmail.com (marc nicole) Date: Thu, 26 Sep 2024 12:00:54 +0200 Subject: [Tutor] How to stop a specific thread in Python 2.7? In-Reply-To: References: <30e63957-bc36-42fd-9fe0-d15e6b9a920a@wichmann.us> Message-ID: To note that i dont want to achieve concurrent execution but rather launch a background task On Wed, 25 Sept 2024, 22:26 Mats Wichmann, wrote: > On 9/25/24 13:57, marc nicole wrote: > > Why i want to kill a thread? > > > > At first, i want to look for two values through a thread, if i got them > > before an event happens, that's cool, the thread will end itself and i > > will use the values in the subsequent code, if not i want to kill the > > background running function/thread, and relaunch the function of thread > > (blocking call this time) to get the values and then continue the > program. > > > > The context is robotics: > > So to start out, from the documentation: > > "currently, there are no priorities, no thread groups, and threads > cannot be destroyed, stopped, suspended, resumed, or interrupted." > > (see the bits just above this heading: > https://docs.python.org/3.12/library/threading.html#thread-local-data) > > So you need to find a way to extend your model to the negative case as > well, so the thread stops itself even if the ideal case didn't occur. I > think one approach is to use Event, as you've already started to think > about, I'd just change the naming to be more clear: > > > should_stop = threading.Event() > # then at some point: > should_stop.set() > > > and in your thread: > > while not should_stop.wait(1): > # do whatever work you can do > > > I'm guessing you've already been here and this isn't new info. Maybe > someone else has more experience... > > From stuff I've heard, robotics folks are making use of async I/O to > solve these situations, rather than threads. Unfortunately, Python's > async is a bit of a pain as it's kind of like a virus: a function that > calls async routines must itself be async, and so on... it kind of > spreads throughout your program this way, and can be a lot of work > unless you built it from scratch as async. > > > >