From lenjaffe at jaffesystems.com Wed Jul 3 16:32:16 2024 From: lenjaffe at jaffesystems.com (Len Jaffe) Date: Wed, 3 Jul 2024 16:32:16 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output Message-ID: I have a method that present a user prompt, and returns the respond, and method tests using pytest. First draft using print(), input() and pytest capsys.out worked fine. But now I want to move the prompt into another method which is called inside a FileInput context manager. The FileInput is instantiated with inplace=True, so I can no longer use stdout for the use prompt since fileinput uses it for the in-place processing. I have no problem opening /dev/tty on new fds to do the prompt and response, but I'm stumped trying to figure out an alternative solution for capturing that output in pytest. My web searches found nothing of value. Please help. Suggestions and advice would be greatly appreciated. Len -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at intellovations.com Thu Jul 4 16:19:08 2024 From: eric at intellovations.com (Eric Floehr) Date: Thu, 4 Jul 2024 16:19:08 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output In-Reply-To: References: Message-ID: Len, This may be a long-shot and it's a bit of a hack, but what about using stderr for output? stderr doesn't get captured by inplace=True. Though I'm not an expert with pytest and what it might do. Or could you create a "prompt_user" function that could take a StringIO object for input and output and pass that through the test since it has a stream-like interface? Here is a quick test I tried, you could in process_file create and pass in StringIO's in a pytest test as well. ------------------- import fileinput import sys def prompt_user(prompt, input_stream=sys.stdin, output_stream=sys.stderr): output_stream.write(prompt) output_stream.flush() return input_stream.readline() def process_file(filename): with fileinput.input(files=(filename,), inplace=True) as f: for line in f: user_input = prompt_user(f"Edit line '{line.strip()}': ") print(user_input if user_input else line, end='') process_file('test.txt') ------------------- Cheers, Eric On Wed, Jul 3, 2024 at 4:32?PM Len Jaffe wrote: > I have a method that present a user prompt, and returns the respond, and > method tests using pytest. > > First draft using print(), input() and pytest capsys.out worked fine. > > But now I want to move the prompt into another method which is called > inside a FileInput context manager. The FileInput is instantiated with > inplace=True, so I can no longer use stdout for the use prompt since > fileinput uses it for the in-place processing. > > I have no problem opening /dev/tty on new fds to do the prompt and > response, but I'm stumped trying to figure out an alternative solution for > capturing that output in pytest. > > My web searches found nothing of value. Please help. > > Suggestions and advice would be greatly appreciated. > > Len > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > https://mail.python.org/mailman/listinfo/centraloh > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenjaffe at jaffesystems.com Fri Jul 5 12:03:27 2024 From: lenjaffe at jaffesystems.com (Len Jaffe) Date: Fri, 5 Jul 2024 12:03:27 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output In-Reply-To: References: Message-ID: Thank Eric. I did something similar, explicitly opening a /dev/tty for output - hadn't thought about just using stderr. I'll try that out on Monday. On Thu, Jul 4, 2024, 4:19 PM Eric Floehr wrote: > Len, > > This may be a long-shot and it's a bit of a hack, but what about using > stderr for output? stderr doesn't get captured by inplace=True. Though I'm > not an expert with pytest and what it might do. Or could you create a > "prompt_user" function that could take a StringIO object for input and > output and pass that through the test since it has a stream-like interface? > > Here is a quick test I tried, you could in process_file create and pass in > StringIO's in a pytest test as well. > > ------------------- > > import fileinput > import sys > > def prompt_user(prompt, input_stream=sys.stdin, output_stream=sys.stderr): > output_stream.write(prompt) > output_stream.flush() > return input_stream.readline() > > def process_file(filename): > with fileinput.input(files=(filename,), inplace=True) as f: > for line in f: > user_input = prompt_user(f"Edit line '{line.strip()}': ") > print(user_input if user_input else line, end='') > > process_file('test.txt') > > ------------------- > > Cheers, > Eric > > > On Wed, Jul 3, 2024 at 4:32?PM Len Jaffe > wrote: > >> I have a method that present a user prompt, and returns the respond, and >> method tests using pytest. >> >> First draft using print(), input() and pytest capsys.out worked fine. >> >> But now I want to move the prompt into another method which is called >> inside a FileInput context manager. The FileInput is instantiated with >> inplace=True, so I can no longer use stdout for the use prompt since >> fileinput uses it for the in-place processing. >> >> I have no problem opening /dev/tty on new fds to do the prompt and >> response, but I'm stumped trying to figure out an alternative solution for >> capturing that output in pytest. >> >> My web searches found nothing of value. Please help. >> >> Suggestions and advice would be greatly appreciated. >> >> Len >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> https://mail.python.org/mailman/listinfo/centraloh >> > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > https://mail.python.org/mailman/listinfo/centraloh > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.costlow at gmail.com Sat Jul 6 21:40:07 2024 From: brian.costlow at gmail.com (Brian Costlow) Date: Sat, 6 Jul 2024 21:40:07 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output In-Reply-To: References: Message-ID: A couple of things come to mind when looking over this exchange. If you are just handling a single file, and not a list of files, then all fileinput buys you is the in-place editing. And all that really does is move the original file to a backup, writes to a new file using the original name, and deletes the backup if everything succeeds. You could do that yourself, and then by using the standard open call, and some careful refactoring, you get much more testable code that doesn't rely on https://rhodesmill.org/brandon/slides/2015-05-pywaw/hoist/ Or, you can keep using fileinput, and maybe use https://pytest-pyfakefs.readthedocs.io/en/latest/index.html to set up a mocked in memory filesystem. On Fri, Jul 5, 2024 at 12:03?PM Len Jaffe wrote: > Thank Eric. > I did something similar, explicitly opening a /dev/tty for output - hadn't > thought about just using stderr. > > I'll try that out on Monday. > > On Thu, Jul 4, 2024, 4:19 PM Eric Floehr wrote: > >> Len, >> >> This may be a long-shot and it's a bit of a hack, but what about using >> stderr for output? stderr doesn't get captured by inplace=True. Though I'm >> not an expert with pytest and what it might do. Or could you create a >> "prompt_user" function that could take a StringIO object for input and >> output and pass that through the test since it has a stream-like interface? >> >> Here is a quick test I tried, you could in process_file create and pass >> in StringIO's in a pytest test as well. >> >> ------------------- >> >> import fileinput >> import sys >> >> def prompt_user(prompt, input_stream=sys.stdin, output_stream=sys.stderr): >> output_stream.write(prompt) >> output_stream.flush() >> return input_stream.readline() >> >> def process_file(filename): >> with fileinput.input(files=(filename,), inplace=True) as f: >> for line in f: >> user_input = prompt_user(f"Edit line '{line.strip()}': ") >> print(user_input if user_input else line, end='') >> >> process_file('test.txt') >> >> ------------------- >> >> Cheers, >> Eric >> >> >> On Wed, Jul 3, 2024 at 4:32?PM Len Jaffe >> wrote: >> >>> I have a method that present a user prompt, and returns the respond, >>> and method tests using pytest. >>> >>> First draft using print(), input() and pytest capsys.out worked fine. >>> >>> But now I want to move the prompt into another method which is called >>> inside a FileInput context manager. The FileInput is instantiated with >>> inplace=True, so I can no longer use stdout for the use prompt since >>> fileinput uses it for the in-place processing. >>> >>> I have no problem opening /dev/tty on new fds to do the prompt and >>> response, but I'm stumped trying to figure out an alternative solution for >>> capturing that output in pytest. >>> >>> My web searches found nothing of value. Please help. >>> >>> Suggestions and advice would be greatly appreciated. >>> >>> Len >>> _______________________________________________ >>> CentralOH mailing list >>> CentralOH at python.org >>> https://mail.python.org/mailman/listinfo/centraloh >>> >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> https://mail.python.org/mailman/listinfo/centraloh >> > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > https://mail.python.org/mailman/listinfo/centraloh > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.costlow at gmail.com Sat Jul 6 21:41:46 2024 From: brian.costlow at gmail.com (Brian Costlow) Date: Sat, 6 Jul 2024 21:41:46 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output In-Reply-To: References: Message-ID: Whoops, typo. Hacked off the send of a sentence while pasting. you get much more testable code that doesn't rely on *redirecting writes to std.out.* On Sat, Jul 6, 2024 at 9:40?PM Brian Costlow wrote: > A couple of things come to mind when looking over this exchange. > > If you are just handling a single file, and not a list of files, then all > fileinput buys you is the in-place editing. And all that really does is > move the original file to a backup, writes to a new file using the original > name, and deletes the backup if everything succeeds. > You could do that yourself, and then by using the standard open call, and > some careful refactoring, you get much more testable code that doesn't rely > on > > https://rhodesmill.org/brandon/slides/2015-05-pywaw/hoist/ > > Or, you can keep using fileinput, and maybe use > https://pytest-pyfakefs.readthedocs.io/en/latest/index.html to set up a > mocked in memory filesystem. > > On Fri, Jul 5, 2024 at 12:03?PM Len Jaffe > wrote: > >> Thank Eric. >> I did something similar, explicitly opening a /dev/tty for output - >> hadn't thought about just using stderr. >> >> I'll try that out on Monday. >> >> On Thu, Jul 4, 2024, 4:19 PM Eric Floehr wrote: >> >>> Len, >>> >>> This may be a long-shot and it's a bit of a hack, but what about using >>> stderr for output? stderr doesn't get captured by inplace=True. Though I'm >>> not an expert with pytest and what it might do. Or could you create a >>> "prompt_user" function that could take a StringIO object for input and >>> output and pass that through the test since it has a stream-like interface? >>> >>> Here is a quick test I tried, you could in process_file create and pass >>> in StringIO's in a pytest test as well. >>> >>> ------------------- >>> >>> import fileinput >>> import sys >>> >>> def prompt_user(prompt, input_stream=sys.stdin, >>> output_stream=sys.stderr): >>> output_stream.write(prompt) >>> output_stream.flush() >>> return input_stream.readline() >>> >>> def process_file(filename): >>> with fileinput.input(files=(filename,), inplace=True) as f: >>> for line in f: >>> user_input = prompt_user(f"Edit line '{line.strip()}': ") >>> print(user_input if user_input else line, end='') >>> >>> process_file('test.txt') >>> >>> ------------------- >>> >>> Cheers, >>> Eric >>> >>> >>> On Wed, Jul 3, 2024 at 4:32?PM Len Jaffe >>> wrote: >>> >>>> I have a method that present a user prompt, and returns the respond, >>>> and method tests using pytest. >>>> >>>> First draft using print(), input() and pytest capsys.out worked fine. >>>> >>>> But now I want to move the prompt into another method which is called >>>> inside a FileInput context manager. The FileInput is instantiated with >>>> inplace=True, so I can no longer use stdout for the use prompt since >>>> fileinput uses it for the in-place processing. >>>> >>>> I have no problem opening /dev/tty on new fds to do the prompt and >>>> response, but I'm stumped trying to figure out an alternative solution for >>>> capturing that output in pytest. >>>> >>>> My web searches found nothing of value. Please help. >>>> >>>> Suggestions and advice would be greatly appreciated. >>>> >>>> Len >>>> _______________________________________________ >>>> CentralOH mailing list >>>> CentralOH at python.org >>>> https://mail.python.org/mailman/listinfo/centraloh >>>> >>> _______________________________________________ >>> CentralOH mailing list >>> CentralOH at python.org >>> https://mail.python.org/mailman/listinfo/centraloh >>> >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> https://mail.python.org/mailman/listinfo/centraloh >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenjaffe at jaffesystems.com Mon Jul 8 08:34:38 2024 From: lenjaffe at jaffesystems.com (Len Jaffe) Date: Mon, 8 Jul 2024 08:34:38 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output In-Reply-To: References: Message-ID: Yes. Disappointingly, I probably will reimplement FileInput. I don't really blame it for using stdout. If I wasy implementing a per-file prompt, I would too. Thanks for your advice. Len. On Sat, Jul 6, 2024, 9:42 PM Brian Costlow wrote: > Whoops, typo. Hacked off the send of a sentence while pasting. > > you get much more testable code that doesn't rely on *redirecting writes > to std.out.* > > On Sat, Jul 6, 2024 at 9:40?PM Brian Costlow > wrote: > >> A couple of things come to mind when looking over this exchange. >> >> If you are just handling a single file, and not a list of files, then all >> fileinput buys you is the in-place editing. And all that really does is >> move the original file to a backup, writes to a new file using the original >> name, and deletes the backup if everything succeeds. >> You could do that yourself, and then by using the standard open call, and >> some careful refactoring, you get much more testable code that doesn't rely >> on >> >> https://rhodesmill.org/brandon/slides/2015-05-pywaw/hoist/ >> >> Or, you can keep using fileinput, and maybe use >> https://pytest-pyfakefs.readthedocs.io/en/latest/index.html to set up a >> mocked in memory filesystem. >> >> On Fri, Jul 5, 2024 at 12:03?PM Len Jaffe >> wrote: >> >>> Thank Eric. >>> I did something similar, explicitly opening a /dev/tty for output - >>> hadn't thought about just using stderr. >>> >>> I'll try that out on Monday. >>> >>> On Thu, Jul 4, 2024, 4:19 PM Eric Floehr >>> wrote: >>> >>>> Len, >>>> >>>> This may be a long-shot and it's a bit of a hack, but what about using >>>> stderr for output? stderr doesn't get captured by inplace=True. Though I'm >>>> not an expert with pytest and what it might do. Or could you create a >>>> "prompt_user" function that could take a StringIO object for input and >>>> output and pass that through the test since it has a stream-like interface? >>>> >>>> Here is a quick test I tried, you could in process_file create and pass >>>> in StringIO's in a pytest test as well. >>>> >>>> ------------------- >>>> >>>> import fileinput >>>> import sys >>>> >>>> def prompt_user(prompt, input_stream=sys.stdin, >>>> output_stream=sys.stderr): >>>> output_stream.write(prompt) >>>> output_stream.flush() >>>> return input_stream.readline() >>>> >>>> def process_file(filename): >>>> with fileinput.input(files=(filename,), inplace=True) as f: >>>> for line in f: >>>> user_input = prompt_user(f"Edit line '{line.strip()}': ") >>>> print(user_input if user_input else line, end='') >>>> >>>> process_file('test.txt') >>>> >>>> ------------------- >>>> >>>> Cheers, >>>> Eric >>>> >>>> >>>> On Wed, Jul 3, 2024 at 4:32?PM Len Jaffe >>>> wrote: >>>> >>>>> I have a method that present a user prompt, and returns the respond, >>>>> and method tests using pytest. >>>>> >>>>> First draft using print(), input() and pytest capsys.out worked fine. >>>>> >>>>> But now I want to move the prompt into another method which is called >>>>> inside a FileInput context manager. The FileInput is instantiated with >>>>> inplace=True, so I can no longer use stdout for the use prompt since >>>>> fileinput uses it for the in-place processing. >>>>> >>>>> I have no problem opening /dev/tty on new fds to do the prompt and >>>>> response, but I'm stumped trying to figure out an alternative solution for >>>>> capturing that output in pytest. >>>>> >>>>> My web searches found nothing of value. Please help. >>>>> >>>>> Suggestions and advice would be greatly appreciated. >>>>> >>>>> Len >>>>> _______________________________________________ >>>>> CentralOH mailing list >>>>> CentralOH at python.org >>>>> https://mail.python.org/mailman/listinfo/centraloh >>>>> >>>> _______________________________________________ >>>> CentralOH mailing list >>>> CentralOH at python.org >>>> https://mail.python.org/mailman/listinfo/centraloh >>>> >>> _______________________________________________ >>> CentralOH mailing list >>> CentralOH at python.org >>> https://mail.python.org/mailman/listinfo/centraloh >>> >> _______________________________________________ > CentralOH mailing list > CentralOH at python.org > https://mail.python.org/mailman/listinfo/centraloh > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenjaffe at jaffesystems.com Sat Jul 13 00:59:14 2024 From: lenjaffe at jaffesystems.com (Len Jaffe) Date: Sat, 13 Jul 2024 00:59:14 -0400 Subject: [CentralOH] Pytest, FileInput, and captured output In-Reply-To: References: Message-ID: Just a quick follow-up. Work pulled me into a different project for a little while. Probably be able to follow-up on this in Monday or Tuesday, Ave I'll probably implement my own tiny version of File input. On Mon, Jul 8, 2024, 8:34 AM Len Jaffe wrote: > Yes. Disappointingly, I probably will reimplement FileInput. > > I don't really blame it for using stdout. If I wasy implementing a > per-file prompt, I would too. > > Thanks for your advice. > > Len. > > On Sat, Jul 6, 2024, 9:42 PM Brian Costlow > wrote: > >> Whoops, typo. Hacked off the send of a sentence while pasting. >> >> you get much more testable code that doesn't rely on *redirecting writes >> to std.out.* >> >> On Sat, Jul 6, 2024 at 9:40?PM Brian Costlow >> wrote: >> >>> A couple of things come to mind when looking over this exchange. >>> >>> If you are just handling a single file, and not a list of files, then >>> all fileinput buys you is the in-place editing. And all that really does is >>> move the original file to a backup, writes to a new file using the original >>> name, and deletes the backup if everything succeeds. >>> You could do that yourself, and then by using the standard open call, >>> and some careful refactoring, you get much more testable code that doesn't >>> rely on >>> >>> https://rhodesmill.org/brandon/slides/2015-05-pywaw/hoist/ >>> >>> Or, you can keep using fileinput, and maybe use >>> https://pytest-pyfakefs.readthedocs.io/en/latest/index.html to set up a >>> mocked in memory filesystem. >>> >>> On Fri, Jul 5, 2024 at 12:03?PM Len Jaffe >>> wrote: >>> >>>> Thank Eric. >>>> I did something similar, explicitly opening a /dev/tty for output - >>>> hadn't thought about just using stderr. >>>> >>>> I'll try that out on Monday. >>>> >>>> On Thu, Jul 4, 2024, 4:19 PM Eric Floehr >>>> wrote: >>>> >>>>> Len, >>>>> >>>>> This may be a long-shot and it's a bit of a hack, but what about using >>>>> stderr for output? stderr doesn't get captured by inplace=True. Though I'm >>>>> not an expert with pytest and what it might do. Or could you create a >>>>> "prompt_user" function that could take a StringIO object for input and >>>>> output and pass that through the test since it has a stream-like interface? >>>>> >>>>> Here is a quick test I tried, you could in process_file create and >>>>> pass in StringIO's in a pytest test as well. >>>>> >>>>> ------------------- >>>>> >>>>> import fileinput >>>>> import sys >>>>> >>>>> def prompt_user(prompt, input_stream=sys.stdin, >>>>> output_stream=sys.stderr): >>>>> output_stream.write(prompt) >>>>> output_stream.flush() >>>>> return input_stream.readline() >>>>> >>>>> def process_file(filename): >>>>> with fileinput.input(files=(filename,), inplace=True) as f: >>>>> for line in f: >>>>> user_input = prompt_user(f"Edit line '{line.strip()}': ") >>>>> print(user_input if user_input else line, end='') >>>>> >>>>> process_file('test.txt') >>>>> >>>>> ------------------- >>>>> >>>>> Cheers, >>>>> Eric >>>>> >>>>> >>>>> On Wed, Jul 3, 2024 at 4:32?PM Len Jaffe >>>>> wrote: >>>>> >>>>>> I have a method that present a user prompt, and returns the respond, >>>>>> and method tests using pytest. >>>>>> >>>>>> First draft using print(), input() and pytest capsys.out worked fine. >>>>>> >>>>>> But now I want to move the prompt into another method which is called >>>>>> inside a FileInput context manager. The FileInput is instantiated with >>>>>> inplace=True, so I can no longer use stdout for the use prompt since >>>>>> fileinput uses it for the in-place processing. >>>>>> >>>>>> I have no problem opening /dev/tty on new fds to do the prompt and >>>>>> response, but I'm stumped trying to figure out an alternative solution for >>>>>> capturing that output in pytest. >>>>>> >>>>>> My web searches found nothing of value. Please help. >>>>>> >>>>>> Suggestions and advice would be greatly appreciated. >>>>>> >>>>>> Len >>>>>> _______________________________________________ >>>>>> CentralOH mailing list >>>>>> CentralOH at python.org >>>>>> https://mail.python.org/mailman/listinfo/centraloh >>>>>> >>>>> _______________________________________________ >>>>> CentralOH mailing list >>>>> CentralOH at python.org >>>>> https://mail.python.org/mailman/listinfo/centraloh >>>>> >>>> _______________________________________________ >>>> CentralOH mailing list >>>> CentralOH at python.org >>>> https://mail.python.org/mailman/listinfo/centraloh >>>> >>> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> https://mail.python.org/mailman/listinfo/centraloh >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jocassid at gmail.com Wed Jul 24 23:58:53 2024 From: jocassid at gmail.com (John Cassidy) Date: Wed, 24 Jul 2024 23:58:53 -0400 Subject: [CentralOH] Day trip to PyOhio this Saturday Message-ID: I'm planning to make a day trip to PyOhio this Saturday in Cleveland. I have room for 4 more passengers in my van. My wife and I have dogs so if you're allergic you might want to take a separate vehicle. Chipping in for gas and parking would be appreciated. If we want to make it in time for the keynote we should probably leave my neighborhood around 6AM. We'll work out details once we know who all are interested. -John Cassidy -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.lairson at gmail.com Tue Jul 30 08:05:12 2024 From: john.lairson at gmail.com (J L) Date: Tue, 30 Jul 2024 08:05:12 -0400 Subject: [CentralOH] Google Colab Repo Link Message-ID: Hi All, As promised here is the link to the github repo for the talk I gave yesterday evening. Thank you for having me. https://github.com/pleasewaitinthequeue/GoogleColab -------------- next part -------------- An HTML attachment was scrubbed... URL: