From atharvasakhala5445 at gmail.com Wed Sep 1 00:25:16 2021 From: atharvasakhala5445 at gmail.com (ABCCDE921) Date: Tue, 31 Aug 2021 21:25:16 -0700 (PDT) Subject: Request for argmax(list) and argmin(list) Message-ID: I dont want to import numpy argmax(list) returns index of (left most) max element argmin(list) returns index of (left most) min element From auriocus at gmx.de Wed Sep 1 00:50:45 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 1 Sep 2021 06:50:45 +0200 Subject: Create a real-time interactive TUI using Python. In-Reply-To: References: Message-ID: Am 31.08.21 um 18:49 schrieb Chris Angelico: > On Wed, Sep 1, 2021 at 1:59 AM hongy... at gmail.com wrote: >> >> I want to know whether python can be used to create real-time interactive TUI, as hstr [1] does. >> >> [1] https://github.com/dvorka/hstr >> > > Yes. I think he also would like to know, how to achieve this ;) This kind of interface is usually done with the "curses"-library. There is a Python version of it, see e.g. here https://docs.python.org/3/howto/curses.html Christian From pbryan at anode.ca Wed Sep 1 02:32:06 2021 From: pbryan at anode.ca (Paul Bryan) Date: Tue, 31 Aug 2021 23:32:06 -0700 Subject: Request for argmax(list) and argmin(list) In-Reply-To: References: Message-ID: <7031d0cbb8b2bfa31edb16995b5b6f2280c51cb8.camel@anode.ca> Why not: >>> l = [1, 3, 5, 9, 2, 7] >>> l.index(max(l)) 3 >>> l.index(min(l)) 0 On Tue, 2021-08-31 at 21:25 -0700, ABCCDE921 wrote: > I dont want to import numpy > > argmax(list) > ?? returns index of (left most) max element > > ?argmin(list) > ?? returns index of (left most) min element From __peter__ at web.de Wed Sep 1 03:21:30 2021 From: __peter__ at web.de (Peter Otten) Date: Wed, 1 Sep 2021 09:21:30 +0200 Subject: Request for argmax(list) and argmin(list) In-Reply-To: References: Message-ID: On 01/09/2021 06:25, ABCCDE921 wrote: > I dont want to import numpy > > argmax(list) > returns index of (left most) max element >>> import operator >>> second = operator.itemgetter(1) >>> def argmax(values): return max(enumerate(values), key=second)[0] >>> argmax([1, 2, 3, 0]) 2 > argmin(list) > returns index of (left most) min element This is left as an exercise;) From gisle.vanem at gmail.com Wed Sep 1 04:18:50 2021 From: gisle.vanem at gmail.com (Gisle Vanem) Date: Wed, 1 Sep 2021 10:18:50 +0200 Subject: Create a real-time interactive TUI using Python. In-Reply-To: <3019e9c8-fcef-43e4-adca-0ee39b1902dcn@googlegroups.com> References: <3019e9c8-fcef-43e4-adca-0ee39b1902dcn@googlegroups.com> Message-ID: hongy... at gmail.com wrote: > The following are some python TUI framework libraries/projects I have discovered so far: > > https://github.com/pfalcon/picotui > https://github.com/peterbrittain/asciimatics > https://github.com/bczsalba/pytermgui > https://github.com/GeorgeFilipkin/pulsemixer > https://github.com/jwlodek/py_cui > https://github.com/saulpw/visidata > https://github.com/willmcgugan/textual > https://github.com/urwid/urwid Tried some of these. But only Asciimatics and Py_CUI works on Windows. -- --gv From cspealma at redhat.com Wed Sep 1 09:31:02 2021 From: cspealma at redhat.com (Calvin Spealman) Date: Wed, 1 Sep 2021 09:31:02 -0400 Subject: PEP Idea: Real private attribute In-Reply-To: References: Message-ID: On Tue, Aug 31, 2021 at 1:19 PM Mehrzad Saremi wrote: > Calvin, even if the language offered truly private members? > I'm saying I don't think they're necessary, especially not for the use case posited here. Private members in other languages are about things internal to the class of the object itself. If we *did* have them in Python, you couldn't use private members of an object from a decorator on it because the decorator is external to the target class. > On Tue, 31 Aug 2021 at 17:31, Calvin Spealman wrote: > >> The right way for those decorators to hold some private information, >> imho, isn't to put anything on the decorated object at all, but to use a >> weak-ref dictionary using the target object as a key. >> >> On Sat, Aug 28, 2021 at 5:42 PM Mehrzad Saremi >> wrote: >> >>> Python currently uses name mangling for double-underscore attributes. >>> Name >>> mangling is not an ideal method to avoid name conflicting. There are >>> various normal programming patterns that can simply cause name >>> conflicting >>> in double-underscore members. A typical example is when a class is >>> re-decorated using the same decorator. The decorator can not take >>> double-underscore members without name conflicts. For example: >>> >>> ``` >>> @custom_decorator("a") >>> @custom_decorator("b") >>> class C: >>> pass >>> ``` >>> >>> The `@custom_decorator` wrapper may need to hold private members, but >>> Python's current name conflict resolution does not provide any solution >>> and >>> the decorator cannot hold private members without applying tricky >>> programming methods. >>> >>> Another example is when a class inherits from a base class of the same >>> name. >>> >>> ``` >>> class View: >>> """A class representing a view of an object; similar to >>> numpy.ndarray.view""" >>> pass >>> >>> class Object: >>> class View(View): >>> """A view class costumized for objects of type Object""" >>> pass >>> ``` >>> >>> Again, in this example, class `Object.View` can't take double-underscore >>> names without conflicting with `View`'s. >>> >>> My idea is to introduce real private members (by which I do not mean to >>> be >>> inaccessible from outside the class, but to be guaranteed not to conflict >>> with other private members of the same object). These private members are >>> started with triple underscores and are stored in a separate dictionary >>> named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__` >>> will be a double layer dictionary that takes 'type' keys in the first >>> level, and 'str' keys in the second level. >>> >>> For example, assume that the user runs the following code: >>> ``` >>> class C: >>> def __init__(self, value): >>> self.___member = value >>> >>> c = C("my value") >>> ``` >>> >>> On the last line, Python's attribute setter creates a new entry in the >>> dictionary with key `C`, adds the value "my value" to a new entry with >>> the >>> key 'member'. >>> >>> The user can then retrieve `c.___member` by invoking the `__privs__` >>> dictionary: >>> >>> ``` >>> print(c.__privs__[C]['member']) # prints 'my value' >>> ``` >>> >>> Note that, unlike class names, class objects are unique and there will >>> not >>> be any conflicts. Python classes are hashable and can be dictionary keys. >>> Personally, I do not see any disadvantage of using __privs__ over name >>> mangling/double-underscores. While name mangling does not truly guarantee >>> conflict resolution, __privs__ does. >>> >>> Please discuss the idea, let me know what you think about it, whether >>> there >>> are possible disadvantages, and if you think it will be approved as a >>> PEP. >>> >>> Thanks, >>> Mehrzad Saremi >>> >>> AI M.Sc. grad. from AUT >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >>> >> >> -- >> >> CALVIN SPEALMAN >> >> SENIOR QUALITY ENGINEER >> >> calvin.spealman at redhat.com M: +1.336.210.5107 >> [image: https://red.ht/sig] >> TRIED. TESTED. TRUSTED. >> > -- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman at redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] TRIED. TESTED. TRUSTED. From loris.bennett at fu-berlin.de Wed Sep 1 07:48:26 2021 From: loris.bennett at fu-berlin.de (Loris Bennett) Date: Wed, 01 Sep 2021 13:48:26 +0200 Subject: Trouble propagating logging configuration References: <87eea9zqy1.fsf@hornfels.zedat.fu-berlin.de> <24878.24705.612533.264970@ixdm.fritz.box> Message-ID: <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> "Dieter Maurer" writes: > Loris Bennett wrote at 2021-8-31 15:25 +0200: >>I am having difficulty getting the my logging configuration passed on >>to imported modules. >> >>My initial structure was as follows: >> >> $ tree blorp/ >> blorp/ >> |-- blorp >> | |-- __init__.py >> | |-- bar.py >> | |-- foo.py >> | `-- main.py >> `-- pyproject.toml >> >>whereby the logging configuration is done in main.py. >> >>After thinking about it, I decided maybe the inheritance wasn't working >>because main.py is in the same directory as the other files. > > Should you speak about Python's `logging` module, then > the "inheritance" does not depend on the source layout. > Instead, it is based on the hierarchy of dotted names. > It is completely up to you which dotted names you are using > in your `getLogger` calls. Yes, but to quote from https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial: A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows: logger = logging.getLogger(__name__) This means that logger names track the package/module hierarchy, and it?s intuitively obvious where events are logged just from the logger name. so in this case the source layout is relevant, isn't it? > Furthermore, the place of the configuration (and where in the > code it is activated) is completely irrelevant for the "inheritance". OK, so one issue is that I was getting confused by the *order* in which modules are being called. If I have two modules, 'foo' and 'bar', in the same directory, configure the logging just in 'foo' and then call foo.some_method() bar.some_method() then both methods will be logged. If I do bar.some_method() foo.some_method() then only the method in 'foo' will be logged. However, I still have the following problem. With the structure $ tree . . |-- log_test | |-- __init__.py | |-- first.py | `-- second.py |-- pyproject.toml |-- README.rst |-- run.py `-- tests |-- __init__.py |-- config `-- test_log_test.py I have __name__ variables as follows: __file__: /home/loris/log_test/log_test/first.py, __name__: log_test.first __file__: /home/loris/log_test/log_test/second.py, __name__: log_test.second __file__: ./run.py, __name__: __main__ If I have [loggers] keys=root,main,log_test in my logging configuration and initialise the logging in run.py with logging.config.fileConfig("/home/loris/log_test/tests/config") logger = logging.getLogger() or logging.config.fileConfig("/home/loris/log_test/tests/config") logger = logging.getLogger("log_test") then only calls in 'run.py' are logged. I can obviously initialise the logging within the subordinate package, i.e. in 'log_test/__init__.py', but that seems wrong to me. So what is the correct way to initialise logging from a top-level script such that logging is activated in all modules requested in the logging configuration? > For details, read the Python documentation for the `logging` module. If they were sufficient, I wouldn't need the newsgroup :-) Thanks for the help, Loris -- This signature is currently under construction. From hongyi.zhao at gmail.com Wed Sep 1 08:09:21 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 1 Sep 2021 05:09:21 -0700 (PDT) Subject: Mutually exclusive options with argparse. Message-ID: See the following code snippets [1] for implementation of the exclusive options with argparse: def query_builder(args): if args.r and args.s: sys.exit(Term.FAIL + 'Only one of -re and -sql should be set' + Term.ENDC) sum_status = sum(1 for x in [args.failure, args.code != -1] if x) if sum_status > 1: sys.exit(Term.FAIL + ('Only one of --failure and --code has to be set') + Term.ENDC) [1] https://github.com/hongyi-zhao/recent2/blob/5486afbd56a6b06bb149a3ea969fb33d9d8b288f/recent2.py#L391 It seems that the above method is awkward, but I'm out of idea to work out more graceful solutions. Any comment/suggestion/enhancement will be highly appreciated. Regards, HY From atharvasakhala5445 at gmail.com Wed Sep 1 03:20:18 2021 From: atharvasakhala5445 at gmail.com (ABCCDE921) Date: Wed, 1 Sep 2021 00:20:18 -0700 (PDT) Subject: Request for argmax(list) and argmin(list) In-Reply-To: References: <7031d0cbb8b2bfa31edb16995b5b6f2280c51cb8.camel@anode.ca> Message-ID: Because that does 2 passes over the entire array when you only need one and there is no option to specify if you want the leftmost or rightmost element On Wednesday, September 1, 2021 at 12:02:29 PM UTC+5:30, Paul Bryan wrote: > Why not: > > >>> l = [1, 3, 5, 9, 2, 7] > >>> l.index(max(l)) > 3 > >>> l.index(min(l)) > 0 > On Tue, 2021-08-31 at 21:25 -0700, ABCCDE921 wrote: > > I dont want to import numpy > > > > argmax(list) > > returns index of (left most) max element > > > > argmin(list) > > returns index of (left most) min element From songbird at anthive.com Wed Sep 1 02:56:11 2021 From: songbird at anthive.com (songbird) Date: Wed, 1 Sep 2021 02:56:11 -0400 Subject: my python progress Message-ID: i've had some fun recently when i've had a chance to work on it. i needed some kind of project that would encourage me to learn more python. i still consider myself very new to the language and a long ways to go (and i don't consider the code as a great example, but it is progress to me in my own understanding of python). pyglet the opengl graphics modules have been providing some fun too. here is my recent update to this project: https://github.com/flowerbug/npath which took the following code and then changes it into a more general framework and then as a practical result i was able to use this framework to create and test the first version of the class (posted at the bottom). https://github.com/flowerbug/ngfp one thing i would like to have is a version of the GIMP button border code converted from the GIMP version of lisp to python. i used to be fluent enough in lisp that i can actually figure out most of what the code does but i am not familiar enough with GIMP and graphics to fully understand the code to be able to convert it. if you are up for the challenge i would appreciate it. ===== ; GIMP - The GNU Image Manipulation Program ; Copyright (C) 1995 Spencer Kimball and Peter Mattis ; ; This program is free software: you can redistribute it and/or modify ; it under the terms of the GNU General Public License as published by ; the Free Software Foundation; either version 3 of the License, or ; (at your option) any later version. ; ; This program is distributed in the hope that it will be useful, ; but WITHOUT ANY WARRANTY; without even the implied warranty of ; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ; GNU General Public License for more details. ; ; You should have received a copy of the GNU General Public License ; along with this program. If not, see . ; ; Copyright (C) 1997 Andy Thomas alt at picnic.demon.co.uk ; ; Version 0.2 10.6.97 Changed to new script-fu interface in 0.99.10 ; Delta the color by the given amount. Check for boundary conditions ; If < 0 set to zero ; If > 255 set to 255 ; Return the new value (define (script-fu-addborder aimg adraw xsize ysize color dvalue) (define (deltacolor col delta) (let* ((newcol (+ col delta))) (if (< newcol 0) (set! newcol 0)) (if (> newcol 255) (set! newcol 255)) newcol ) ) (define (adjcolor col delta) (mapcar (lambda (x) (deltacolor x delta)) col) ) (define (gen_top_array xsize ysize owidth oheight width height) (let* ((n_array (cons-array 10 'double))) (aset n_array 0 0 ) (aset n_array 1 0 ) (aset n_array 2 xsize) (aset n_array 3 ysize) (aset n_array 4 (+ xsize owidth)) (aset n_array 5 ysize) (aset n_array 6 width) (aset n_array 7 0 ) (aset n_array 8 0 ) (aset n_array 9 0 ) n_array) ) (define (gen_left_array xsize ysize owidth oheight width height) (let* ((n_array (cons-array 10 'double))) (aset n_array 0 0 ) (aset n_array 1 0 ) (aset n_array 2 xsize) (aset n_array 3 ysize) (aset n_array 4 xsize) (aset n_array 5 (+ ysize oheight)) (aset n_array 6 0 ) (aset n_array 7 height ) (aset n_array 8 0 ) (aset n_array 9 0 ) n_array) ) (define (gen_right_array xsize ysize owidth oheight width height) (let* ((n_array (cons-array 10 'double))) (aset n_array 0 width ) (aset n_array 1 0 ) (aset n_array 2 (+ xsize owidth)) (aset n_array 3 ysize) (aset n_array 4 (+ xsize owidth)) (aset n_array 5 (+ ysize oheight)) (aset n_array 6 width) (aset n_array 7 height) (aset n_array 8 width ) (aset n_array 9 0 ) n_array) ) (define (gen_bottom_array xsize ysize owidth oheight width height) (let* ((n_array (cons-array 10 'double))) (aset n_array 0 0 ) (aset n_array 1 height) (aset n_array 2 xsize) (aset n_array 3 (+ ysize oheight)) (aset n_array 4 (+ xsize owidth)) (aset n_array 5 (+ ysize oheight)) (aset n_array 6 width) (aset n_array 7 height) (aset n_array 8 0 ) (aset n_array 9 height) n_array) ) (let* ((img (car (gimp-item-get-image adraw))) (owidth (car (gimp-image-get-width img))) (oheight (car (gimp-image-get-height img))) (width (+ owidth (* 2 xsize))) (height (+ oheight (* 2 ysize))) (layer (car (gimp-layer-new img width height (car (gimp-drawable-type-with-alpha adraw)) _"Border Layer" 100 LAYER-MODE-NORMAL)))) (gimp-context-push) (gimp-context-set-paint-mode LAYER-MODE-NORMAL) (gimp-context-set-opacity 100.0) (gimp-context-set-antialias FALSE) (gimp-context-set-feather FALSE) (gimp-image-undo-group-start img) (gimp-image-resize img width height xsize ysize) (gimp-image-insert-layer img layer 0 0) (gimp-drawable-fill layer FILL-TRANSPARENT) (gimp-context-set-background (adjcolor color dvalue)) (gimp-image-select-polygon img CHANNEL-OP-REPLACE 10 (gen_top_array xsize ysize owidth oheight width height)) (gimp-drawable-edit-fill layer FILL-BACKGROUND) (gimp-context-set-background (adjcolor color (/ dvalue 2))) (gimp-image-select-polygon img CHANNEL-OP-REPLACE 10 (gen_left_array xsize ysize owidth oheight width height)) (gimp-drawable-edit-fill layer FILL-BACKGROUND) (gimp-context-set-background (adjcolor color (- 0 (/ dvalue 2)))) (gimp-image-select-polygon img CHANNEL-OP-REPLACE 10 (gen_right_array xsize ysize owidth oheight width height)) (gimp-drawable-edit-fill layer FILL-BACKGROUND) (gimp-context-set-background (adjcolor color (- 0 dvalue))) (gimp-image-select-polygon img CHANNEL-OP-REPLACE 10 (gen_bottom_array xsize ysize owidth oheight width height)) (gimp-drawable-edit-fill layer FILL-BACKGROUND) (gimp-selection-none img) (gimp-image-undo-group-end img) (gimp-displays-flush) (gimp-context-pop) ) ) (script-fu-register "script-fu-addborder" _"Add _Border..." _"Add a border around an image" "Andy Thomas " "Andy Thomas" "6/10/97" "*" SF-IMAGE "Input image" 0 SF-DRAWABLE "Input drawable" 0 SF-ADJUSTMENT _"Border X size" '(12 1 250 1 10 0 1) SF-ADJUSTMENT _"Border Y size" '(12 1 250 1 10 0 1) SF-COLOR _"Border color" '(38 31 207) SF-ADJUSTMENT _"Delta value on color" '(25 1 255 1 10 0 1) ) (script-fu-menu-register "script-fu-addborder" "/Filters/Decor") ===== my own simpler version of a bordered button class is below, but i have not fully tested it yet, but it works for the square version that i'm using at the moment. i'll have to update my progam a bit some more to test out the rectangular version (that's my next bit of challenge). ===== from math import sqrt import pyglet from pyglet.image import SolidColorImagePattern, ImageData class SolidColorButtonImagePattern(SolidColorImagePattern): """Creates a beveled button image filled with a solid color.""" def __init__(self, color=(0, 0, 0, 0), border_color=(0,0,0,255)): """Create a beveled image pattern with color and blend the border towards border_color. :Parameters: `color` : (int, int, int, int) 4-tuple of ints in range [0,255] giving RGBA components of color to fill with. `border_color` : (int, int, int, int) 4-tuple of ints in range [0,255] giving RGBA components of the border color to blend towards. """ if len(color) != 4: raise TypeError("color is expected to have 4 components") self.color = list(color) self.border_color = list(border_color) def create_image(self, width, height): data = self.color * width * height if (width < 3) or (height < 3): print ("width or height < 3 : ", width, height, " No Border Applied to image.") img_data = ImageData(width, height, 'RGBA', bytes(data)) return img_data x_border = int(width / sqrt(width)) y_border = int(height / sqrt(height)) if (x_border == 0): x_border = 1 if (y_border == 0): y_border = 1 # make the borders are uniform make sure they are the same size # even if the width and height of the tiles are different if (x_border != y_border): y_border = (x_border + x_border) // 2 x_border = y_border # how many gradient steps to use for each part of the color and alpha step_x_red = (self.border_color[0] - self.color[0]) / x_border step_y_red = (self.border_color[0] - self.color[0]) / y_border step_x_green = (self.border_color[1] - self.color[1]) / x_border step_y_green = (self.border_color[1] - self.color[1]) / y_border step_x_blue = (self.border_color[2] - self.color[2]) / x_border step_y_blue = (self.border_color[2] - self.color[2]) / y_border step_x_alpha = (self.border_color[3] - self.color[3]) / x_border step_y_alpha = (self.border_color[3] - self.color[3]) / y_border # bottom and top row(s) red_x = self.border_color[0] green_x = self.border_color[1] blue_x = self.border_color[2] alpha_x = self.border_color[3] for x in range(x_border): for y in range(width-(x*2)): indx_lower = (x*width*4)+((y+x)*4) indx_upper = ((height-(x+1))*width*4)+((y+x)*4) data[indx_lower] = int(red_x) data[indx_upper] = int(red_x) data[indx_lower+1] = int(green_x) data[indx_upper+1] = int(green_x) data[indx_lower+2] = int(blue_x) data[indx_upper+2] = int(blue_x) data[indx_lower+3] = int(alpha_x) data[indx_upper+3] = int(alpha_x) red_x -= step_x_red green_x -= step_x_green blue_x -= step_x_blue alpha_x -= step_x_alpha # left and right col(s) red_y = self.border_color[0] green_y = self.border_color[1] blue_y = self.border_color[2] alpha_y = self.border_color[3] for x in range(y_border): for y in range(height-(x*2)): indx_left = (((y+x)*height)+(x))*4 indx_right = (((y+x)*height)+(width-x-1))*4 data[indx_left] = int(red_y) data[indx_right] = int(red_y) data[indx_left+1] = int(green_y) data[indx_right+1] = int(green_y) data[indx_left+2] = int(blue_y) data[indx_right+2] = int(blue_y) data[indx_left+3] = int(alpha_y) data[indx_right+3] = int(alpha_y) red_y -= step_y_red green_y -= step_y_green blue_y -= step_y_blue alpha_y -= step_y_alpha img_data = ImageData(width, height, 'RGBA', bytes(data)) return img_data ===== so never give up hope of teaching an old dog some new tricks. :) songbird From __peter__ at web.de Wed Sep 1 10:10:54 2021 From: __peter__ at web.de (Peter Otten) Date: Wed, 1 Sep 2021 16:10:54 +0200 Subject: Trouble propagating logging configuration In-Reply-To: <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> References: <87eea9zqy1.fsf@hornfels.zedat.fu-berlin.de> <24878.24705.612533.264970@ixdm.fritz.box> <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> Message-ID: On 01/09/2021 13:48, Loris Bennett wrote: > "Dieter Maurer" writes: > >> Loris Bennett wrote at 2021-8-31 15:25 +0200: >>> I am having difficulty getting the my logging configuration passed on >>> to imported modules. >>> >>> My initial structure was as follows: >>> >>> $ tree blorp/ >>> blorp/ >>> |-- blorp >>> | |-- __init__.py >>> | |-- bar.py >>> | |-- foo.py >>> | `-- main.py >>> `-- pyproject.toml >>> >>> whereby the logging configuration is done in main.py. >>> >>> After thinking about it, I decided maybe the inheritance wasn't working >>> because main.py is in the same directory as the other files. >> >> Should you speak about Python's `logging` module, then >> the "inheritance" does not depend on the source layout. >> Instead, it is based on the hierarchy of dotted names. >> It is completely up to you which dotted names you are using >> in your `getLogger` calls. > > Yes, but to quote from https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial: > > A good convention to use when naming loggers is to use a module-level > logger, in each module which uses logging, named as follows: > > logger = logging.getLogger(__name__) > > This means that logger names track the package/module hierarchy, and > it?s intuitively obvious where events are logged just from the logger > name. > > so in this case the source layout is relevant, isn't it? In most cases you will only add handlers to the root logger. If you want special treatment of a specific logger you need to know its name including its package, i. e. log_test.first, not first. If you have the log_test directory in your path and use import first instead of import log_test.first then not only the __name__ will be wrong/unexpected, relative imports will fail, too. In other words, this is generally a bad idea. >> Furthermore, the place of the configuration (and where in the >> code it is activated) is completely irrelevant for the "inheritance". > > OK, so one issue is that I was getting confused by the *order* in which > modules are being called. If I have two modules, 'foo' and 'bar', in > the same directory, configure the logging just in 'foo' and then call > > > foo.some_method() > bar.some_method() > > then both methods will be logged. If I do > > bar.some_method() > foo.some_method() > > then only the method in 'foo' will be logged. > > However, I still have the following problem. With the structure > > $ tree . > . > |-- log_test > | |-- __init__.py > | |-- first.py > | `-- second.py > |-- pyproject.toml > |-- README.rst > |-- run.py > `-- tests > |-- __init__.py > |-- config > `-- test_log_test.py > > I have __name__ variables as follows: > > __file__: /home/loris/log_test/log_test/first.py, __name__: log_test.first > __file__: /home/loris/log_test/log_test/second.py, __name__: log_test.second > __file__: ./run.py, __name__: __main__ > > If I have > > [loggers] > keys=root,main,log_test > > in my logging configuration and initialise the logging in run.py with > > logging.config.fileConfig("/home/loris/log_test/tests/config") > logger = logging.getLogger() > > or > > logging.config.fileConfig("/home/loris/log_test/tests/config") > logger = logging.getLogger("log_test") > > then only calls in 'run.py' are logged. > > I can obviously initialise the logging within the subordinate package, > i.e. in 'log_test/__init__.py', but that seems wrong to me. > > So what is the correct way to initialise logging from a top-level script > such that logging is activated in all modules requested in the logging > configuration? > >> For details, read the Python documentation for the `logging` module. > > If they were sufficient, I wouldn't need the newsgroup :-) > > Thanks for the help, Perhaps you can start with logging.basicConfig() in run.py. If the imported modules contain module-level logging messages you have to put the import statements after the basicConfig() invocation. You should see that simple changes like setting the logging level affect messages from all loggers. If that's not enough to get the granularity you want give your loggers fixed names # in test_log/first.py logger = logging.getLogger("test_log.first") or make sure that __name__ is what you expect # in test_log/first.py assert __name__ == "test_log.first" during the debugging phase. From cspealma at redhat.com Wed Sep 1 10:46:07 2021 From: cspealma at redhat.com (Calvin Spealman) Date: Wed, 1 Sep 2021 10:46:07 -0400 Subject: Request for argmax(list) and argmin(list) In-Reply-To: References: <7031d0cbb8b2bfa31edb16995b5b6f2280c51cb8.camel@anode.ca> Message-ID: If only there were a library that already provides exactly the functions you're asking for... ? On Wed, Sep 1, 2021 at 9:54 AM ABCCDE921 wrote: > Because that does 2 passes over the entire array when you only need one > and there is no option to specify if you want the leftmost or rightmost > element > > > On Wednesday, September 1, 2021 at 12:02:29 PM UTC+5:30, Paul Bryan wrote: > > Why not: > > > > >>> l = [1, 3, 5, 9, 2, 7] > > >>> l.index(max(l)) > > 3 > > >>> l.index(min(l)) > > 0 > > On Tue, 2021-08-31 at 21:25 -0700, ABCCDE921 wrote: > > > I dont want to import numpy > > > > > > argmax(list) > > > returns index of (left most) max element > > > > > > argmin(list) > > > returns index of (left most) min element > -- > https://mail.python.org/mailman/listinfo/python-list > > -- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman at redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] TRIED. TESTED. TRUSTED. From lee at lcongdon.com Wed Sep 1 11:19:48 2021 From: lee at lcongdon.com (Lee Congdon) Date: Wed, 1 Sep 2021 11:19:48 -0400 Subject: Mutually exclusive options with argparse. In-Reply-To: References: Message-ID: Does a mutually exclusive group, as described in "Mutual exclusion" at https://docs.python.org/3/library/argparse.html meet your needs? On Wed, Sep 1, 2021 at 9:48 AM hongy... at gmail.com wrote: > See the following code snippets [1] for implementation of the exclusive > options with argparse: > > def query_builder(args): > if args.r and args.s: > sys.exit(Term.FAIL + 'Only one of -re and -sql should be set' + > Term.ENDC) > sum_status = sum(1 for x in [args.failure, args.code != -1] if x) > if sum_status > 1: > sys.exit(Term.FAIL + ('Only one of --failure and --code has to be > set') + Term.ENDC) > > [1] > https://github.com/hongyi-zhao/recent2/blob/5486afbd56a6b06bb149a3ea969fb33d9d8b288f/recent2.py#L391 > > It seems that the above method is awkward, but I'm out of idea to work out > more graceful solutions. Any comment/suggestion/enhancement will be highly > appreciated. > > Regards, > HY > -- > https://mail.python.org/mailman/listinfo/python-list > -- +1-202-507-9867, Twitter @lcongdon From info at wingware.com Wed Sep 1 12:21:10 2021 From: info at wingware.com (Wingware) Date: Wed, 1 Sep 2021 12:21:10 -0400 Subject: ANN: Wing Python IDE 8.0.3 has been released Message-ID: <6dabdfde-f21a-80b9-1e08-99eafcf8338c@wingware.com> Wing 8.0.3 allows specifying the Django settings module for unit tests with --settings= in Run Args on the Testing page of Project Properties, fixes using an Activated Env that contains spaces in its path, prevents failure to reformat code on remote hosts and containers, fixes searching in files with non-ascii characters, and makes several other improvements. Details:????????? https://wingware.com/news/2021-08-31 Downloads:?? https://wingware.com/downloads == About Wing == Wing is a light-weight but full-featured Python IDE designed specifically for Python, with powerful editing, code inspection, testing, and debugging capabilities. Wing's deep code analysis provides auto-completion, auto-editing, and refactoring that speed up development. Its top notch debugger works with any Python code, locally or on a remote host, container, or cluster. Wing also supports test-driven development, version control, UI color and layout customization, and includes extensive documentation and support. Wing is available in three product levels:? Wing Pro is the full-featured Python IDE for professional developers, Wing Personal is a free Python IDE for students and hobbyists (omits some features), and Wing 101 is a very simplified free Python IDE for beginners (omits many features). Learn more at https://wingware.com/ From dieter at handshake.de Wed Sep 1 12:47:53 2021 From: dieter at handshake.de (Dieter Maurer) Date: Wed, 1 Sep 2021 18:47:53 +0200 Subject: Trouble propagating logging configuration In-Reply-To: <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> References: <87eea9zqy1.fsf@hornfels.zedat.fu-berlin.de> <24878.24705.612533.264970@ixdm.fritz.box> <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> Message-ID: <24879.44729.220204.596891@ixdm.fritz.box> Loris Bennett wrote at 2021-9-1 13:48 +0200: > ... >Yes, but to quote from https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial: > > A good convention to use when naming loggers is to use a module-level > logger, in each module which uses logging, named as follows: > > logger = logging.getLogger(__name__) > > This means that logger names track the package/module hierarchy, and > it?s intuitively obvious where events are logged just from the logger > name. > >so in this case the source layout is relevant, isn't it? Relevant in this case is the package/module hierarchy. Often the package/module hierarchy follows the source layout **BUT** this is not necessarily the case. In particular, the "start" module of a script is called `__main__` indepently of its location. Furthermore, so called "namespace packages" consist of decentralized (i.e. located at different places in the file system) subpackages. Thus, in general, the connection between pachage/module hierarchy and the source layout is loose. >> Furthermore, the place of the configuration (and where in the >> code it is activated) is completely irrelevant for the "inheritance". > >OK, so one issue is that I was getting confused by the *order* in which >modules are being called. If I have two modules, 'foo' and 'bar', in >the same directory, configure the logging just in 'foo' and then call > > > foo.some_method() > bar.some_method() > >then both methods will be logged. If I do > > bar.some_method() > foo.some_method() > >then only the method in 'foo' will be logged. Usually, log configuration is considered a (global) application (not a (local) package/module) concern: The components (modules) decide what to log at what level and global configuration decides what to do with those messages. Thus, typically, you configure the complete logging in your main application module and then start to import modules and call their functions. > ... >If I have > > [loggers] > keys=root,main,log_test > >in my logging configuration and initialise the logging in run.py ... This logging configuration is obviously not complete (thus, I cannot check what happens in your case). You may have made errors at other places in your configuration which may explain your observations. At your place, I would look at the `logging` source code to find out where you can see (in an interactive Python session) which loggers have been created by your configuration and how they are configured. For the last part you use "vars(logger)". -- Dieter From wlfraed at ix.netcom.com Wed Sep 1 10:58:59 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 01 Sep 2021 10:58:59 -0400 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <575f2891-aee0-7973-60ea-1782d36efbbb@DancesWithMice.info> Message-ID: On Wed, 1 Sep 2021 11:19:03 +1200, dn via Python-list declaimed the following: >time-zones. (Yes, you did say "CA", but easily 'missed' - by author and >reader alike) Or possibly misread as the Peoples Republic of CALIFORNIA (even if there isn't a "New Brunswick" in the PRCa) Though https://roadsidethoughts.com/nb/lower-california-profile.htm doesn't help in deconfusion -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Wed Sep 1 11:06:37 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 01 Sep 2021 11:06:37 -0400 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. References: <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <575f2891-aee0-7973-60ea-1782d36efbbb@DancesWithMice.info> Message-ID: On Wed, 1 Sep 2021 09:38:30 +1000, Chris Angelico declaimed the following: >Yeah, I have no idea where the "farmers" explanation came from. >Growing up, we had some friends in countrified areas, including a >sheep farmer who showed us a ton of stuff about how shearing worked, >how moronic sheep are, and how farming life actually operates. It's >not uncommon for such people to have two clocks: Government Time and >Sun Time. Like you say, no cow wears a watch! > Not only do the cattle not have timepieces -- but they probably expect to be milked /n/-hours after the prior milking. I think the DST was meant for industrialization -- factories wouldn't have to run expensive artificial lighting for part of the day-shift; there would be more natural light falling in through the south facing windows on the roof of the building. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Wed Sep 1 11:11:14 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 01 Sep 2021 11:11:14 -0400 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Tue, 31 Aug 2021 16:53:14 -0500, 2QdxY4RzWzUUiLuE at potatochowder.com declaimed the following: >On 2021-09-01 at 07:32:43 +1000, >Chris Angelico wrote: >> If we could abolish DST world-wide, life would be far easier. All the >> rest of it would be easy enough to handle. > >Agreed. > Unfortunately, most of the proposals in the US seem to be that /standard time/ would be abolished, and DST would rule year-round. Hence putting the center of the time zone one hour off from solar mean noon; which is what the time zones were originally based upon. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From loris.bennett at fu-berlin.de Thu Sep 2 03:06:53 2021 From: loris.bennett at fu-berlin.de (Loris Bennett) Date: Thu, 02 Sep 2021 09:06:53 +0200 Subject: Trouble propagating logging configuration References: <87eea9zqy1.fsf@hornfels.zedat.fu-berlin.de> <24878.24705.612533.264970@ixdm.fritz.box> <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> <24879.44729.220204.596891@ixdm.fritz.box> Message-ID: <87lf4f4fsi.fsf@hornfels.zedat.fu-berlin.de> "Dieter Maurer" writes: > Loris Bennett wrote at 2021-9-1 13:48 +0200: >> ... >>Yes, but to quote from https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial: >> >> A good convention to use when naming loggers is to use a module-level >> logger, in each module which uses logging, named as follows: >> >> logger = logging.getLogger(__name__) >> >> This means that logger names track the package/module hierarchy, and >> it?s intuitively obvious where events are logged just from the logger >> name. >> >>so in this case the source layout is relevant, isn't it? > > Relevant in this case is the package/module hierarchy. > > Often the package/module hierarchy follows the source layout > **BUT** this is not necessarily the case. > > In particular, the "start" module of a script is called `__main__` > indepently of its location. > Furthermore, so called "namespace packages" consist of > decentralized (i.e. located at different places in the file system) > subpackages. > > Thus, in general, the connection between pachage/module hierarchy > and the source layout is loose. > > >>> Furthermore, the place of the configuration (and where in the >>> code it is activated) is completely irrelevant for the "inheritance". >> >>OK, so one issue is that I was getting confused by the *order* in which >>modules are being called. If I have two modules, 'foo' and 'bar', in >>the same directory, configure the logging just in 'foo' and then call >> >> >> foo.some_method() >> bar.some_method() >> >>then both methods will be logged. If I do >> >> bar.some_method() >> foo.some_method() >> >>then only the method in 'foo' will be logged. > > Usually, log configuration is considered a (global) application > (not a (local) package/module) concern: > The components (modules) decide what to log at what level > and global configuration decides what to do with those messages. > > Thus, typically, you configure the complete logging in > your main application module and then start to import modules > and call their functions. > >> ... >>If I have >> >> [loggers] >> keys=root,main,log_test >> >>in my logging configuration and initialise the logging in run.py ... > > This logging configuration is obviously not complete (thus, I cannot > check what happens in your case). > You may have made errors at other places in your configuration > which may explain your observations. > > At your place, I would look at the `logging` source code > to find out where you can see (in an interactive Python session) > which loggers have been created by your configuration and > how they are configured. For the last part you use "vars(logger)". Thanks Peter and Dieter for all the help. I have finally figured out what my problem was. If in a module 'mylibs.mylib' I have import logging logger = logging.getLogger(__name__) and in my main script have import logger import logger.config import mylibs.mylib logging.config.fileConfig("/home/loris/config") logger = logging.getLogger(__name__) then 'logger' in 'mylibs.mylib' is defined before 'logging.config.fileConfig' is called and the logger in the module is not configured. If change the order and write import logger import logger.config logging.config.fileConfig("/home/loris/config") logger = logging.getLogger(__name__) import mylibs.mylib then the 'logger' in 'mylibs.mylibs' does get configured properly. I'm still thinking what implications this has if I want to load a configuration file via a command-line option. Cheers, Loris -- This signature is currently under construction. From alan.gauld at yahoo.co.uk Thu Sep 2 03:19:16 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 2 Sep 2021 08:19:16 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 31/08/2021 22:13, Chris Angelico wrote: > But ultimately, it all just means that timezones are too hard for > humans to handle, and we MUST handle them using IANA's database. It is > the only way. Except for the places that don't follow the IANA scheme and/or dynamically change their time settings on a whim. To be complete you need the ability to manually override too. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Thu Sep 2 03:32:36 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 2 Sep 2021 08:32:36 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 31/08/2021 22:32, Chris Angelico wrote: > If we could abolish DST world-wide, life would be far easier. All the > rest of it would be easy enough to handle. We tried that in the UK for 2 years back in the '70s and very quickly reverted to DST when they realized that the number of fatalities among young children going to school doubled during those two years. Without DST the schools opened in the dark so all the kids had to travel to school in the dark and the number of traffic accidents while crossing roads jumped. In fact during WW2 they increased DST to 2 hours (that was for the farmers!) because it meant that farm labourers would start work at first light, effectively extending the working day. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Thu Sep 2 03:40:01 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 2 Sep 2021 08:40:01 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <4d5d265c-59f0-8559-4b35-22695ad4a3e2@mrabarnett.plus.com> Message-ID: On 31/08/2021 23:31, Chris Angelico wrote: > Ah, good to know. I think that actually makes a lot of sense; in the > US, they try to let everyone pretend that the rest of the world > doesn't exist ("we always change at 2AM"), but in Europe, they try to > synchronize for the convenience of commerce ("everyone changes at 1AM > UTC"). There's another gotcha with DST changes. The EU and USA have different dates on which they change to DST. In one of them (I can't recall which is which) they change on the 4th weekend of October/March in the other they change on the last weekend. That means on some years (when there are 5 weekends) there is a week when one has changed and the other hasn't. That caused us a lot of head scratching the first time we encountered it because our service centres in the US and EU were getting inconsistent time reporting and some updates showing as having happened in the future! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From hrouselle at jevedi.com Thu Sep 2 09:18:59 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 10:18:59 -0300 Subject: on the popularity of loops while and for References: <86eeadlmpg.fsf@jevedi.com> Message-ID: <86y28fm7y4.fsf@jevedi.com> Chris Angelico writes: > On Sun, Aug 29, 2021 at 7:40 AM Hope Rouselle wrote: >> >> I'd like get a statistic of how often each loop is used in practice. >> >> I was trying to take a look at the Python's standard libraries --- those >> included in a standard installation of Python 3.9.6, say --- to see >> which loops are more often used among while and for loops. Of course, >> since English use the preposition ``for'' a lot, that makes my life >> harder. Removing comments is easy, but removing strings is harder. So >> I don't know yet what I'll do. >> >> Have you guys ever measured something like that in a casual or serious >> way? I'd love to know. Thank you! > > For analysis like this, I recommend using the Abstract Syntax Tree: > > https://docs.python.org/3/library/ast.html > > You can take a Python source file, parse it to the AST, and then walk > that tree to see what it's using. That will avoid any false positives > from the word "for" coming up in the wrong places. Wonderful. Someone here actually posted a nice ``routine'' that does the job marvelously well. I had the impression the poster posted only to solve my request --- impressive and appreciated. From hrouselle at jevedi.com Thu Sep 2 09:23:29 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 10:23:29 -0300 Subject: on the popularity of loops while and for References: <86eeadlmpg.fsf@jevedi.com> Message-ID: <86sfynm7qm.fsf@jevedi.com> Terry Reedy writes: > On 8/28/2021 9:31 AM, Hope Rouselle wrote: >> I'd like get a statistic of how often each loop is used in practice. > > My guess is that for loops are at least twice as common as while loops. Scanning just the Python 3.9.6's Lib/ directory --- using the ast module and the program someone kindly posted here ---, I found for loop appearing 598% more times than while loops. The ratio was something like that 6.98..., excluding files that due to encoding errors (or whatever) could not be parsed. (I reported these numbers elsewhere in this thread.) >> I was trying to take a look at the Python's standard libraries --- those >> included in a standard installation of Python 3.9.6, say --- to see >> which loops are more often used among while and for loops. Of course, >> since English use the preposition ``for'' a lot, that makes my life >> harder. Removing comments is easy, but removing strings is harder. So >> I don't know yet what I'll do. > > Try something like > > fors = 0 > for file in files: > for line in file: > if re.match(r"/s*for .* in ", line): > fors += 1 > > This excludes comprehensions, which have replaced some for loops. That's nice too. Thanks for the alternative. From hrouselle at jevedi.com Thu Sep 2 09:27:15 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 10:27:15 -0300 Subject: on the popularity of loops while and for References: Message-ID: <86k0jzm7kc.fsf@jevedi.com> Barry writes: >> On 28 Aug 2021, at 22:42, Hope Rouselle wrote: >> >> ?I'd like get a statistic of how often each loop is used in practice. >> >> I was trying to take a look at the Python's standard libraries --- those >> included in a standard installation of Python 3.9.6, say --- to see >> which loops are more often used among while and for loops. Of course, >> since English use the preposition ``for'' a lot, that makes my life >> harder. Removing comments is easy, but removing strings is harder. So >> I don't know yet what I'll do. >> >> Have you guys ever measured something like that in a casual or serious >> way? I'd love to know. Thank you! > > I am interesting in why you think that choice of while vs. for is > about popularity? Perhaps you think my choice of "popular" had a special judgement about it. Let's define "popular" in the way that I used it. --8<---------------cut here---------------start------------->8--- Definition. We say x is more popular than y if x appears more times relative to y when considered in a sample. Example. For loops are more popular than while loops in a certain sample because they appeared 17 times compared to 5 times the while loop appeared. --8<---------------cut here---------------end--------------->8--- So it's just a numeric comparison. [...] From hrouselle at jevedi.com Thu Sep 2 09:51:03 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 10:51:03 -0300 Subject: on floating-point numbers Message-ID: <86bl5bm6go.fsf@jevedi.com> Just sharing a case of floating-point numbers. Nothing needed to be solved or to be figured out. Just bringing up conversation. (*) An introduction to me I don't understand floating-point numbers from the inside out, but I do know how to work with base 2 and scientific notation. So the idea of expressing a number as mantissa * base^{power} is not foreign to me. (If that helps you to perhaps instruct me on what's going on here.) (*) A presentation of the behavior >>> import sys >>> sys.version '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]' >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>> sum(ls) 39.599999999999994 >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>> sum(ls) 39.60000000000001 All I did was to take the first number, 7.23, and move it to the last position in the list. (So we have a violation of the commutativity of addition.) Let me try to reduce the example. It's not so easy. Although I could display the violation of commutativity by moving just a single number in the list, I also see that 7.23 commutes with every other number in the list. (*) My request I would like to just get some clarity. I guess I need to translate all these numbers into base 2 and perform the addition myself to see the situation coming up? From hrouselle at jevedi.com Thu Sep 2 10:28:21 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 11:28:21 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> Message-ID: <864kb3m4qi.fsf@jevedi.com> dn writes: > On 29/08/2021 08.46, Hope Rouselle wrote: >> Here's my solution: >> >> --8<---------------cut here---------------start------------->8--- >> def how_many_times(): >> x, y = 0, 1 >> c = 0 >> while x != y: >> c = c + 1 >> x, y = roll() >> return c, (x, y) > >> >> Why am I unhappy? I'm wish I could confine x, y to the while loop. The >> introduction of ``x, y = 0, 1'' must feel like a trick to a novice. How >> would you write this? > >> ram at zedat.fu-berlin.de (Stefan Ram) writes: >>> """Rolls two dice until both yield the same value. >>> Returns the number of times the two dice were rolled >>> and the final value yielded.""" >>> roll_count = 0 >>> while True: >>> outcome = roll_two_dice() >>> roll_count += 1 >>> if outcome[ 0 ]== outcome[ 1 ]: break >>> return roll_count, outcome[ 0 ] >> >> You totally convinced me. Thanks. > > On the other hand... > whilst you expressed concern about the apparently disconnected 'set up' > necessary before the loop, this solution adds a "True/Forever" and a > "Break" construct, which some may deem not that much better (if at all) > > The idea of abrogating the while-condition but then adding another > (disconnected) condition to break, seems to hold equal potential for > confusion. or the type of dissatisfaction which motivated the original > question! Pretty well observed! Hats to you. > Looking at that from the inside-out, the loop's contents perform two > functions: the rolling and counting (per Statement of Requirements), but > also a loop-controlling element. Thus the reader's question: "what does > this loop do?" is conflated with "how many times does it do it?". Well put. > Let's go completely off-the-rails, why not use a never-ending range() to > fuel a for-loop 'counter', and within that loop perform the dice-roll(s) > and decide if it is time to 'break'. The range replaces the "True". The > for-loops index or 'counter' will deliver the desired result. > > Neat? No! > Readable? No! > An improvement over the while-True? Definitely not! > Yet, the mechanism is the same AND offers a built-in counter. Hmmm... Yeah. Here's a little context. I came across this by processing a list of exercises. (I'm teaching a course --- you know that by now, I guess.) So the first thing I observed was the equal volume of work dedicated to while loops and for loops --- so I decided to compared which appeared more often in a certain sample of well-written Python code. It turns out the for loop was much more frequent. Students have been reporting too much work in too little time, so I decided to reduce the number of exercises involving while loops. When I began to look at the exercises, to see which ones I'd exclude, I decided to exclude them all --- lol! --- except for one. The one that remained was this one about rolling dice until a satisfying result would appear. (All other ones were totally more naturally written with a for loop.) So if I were to also write this with a for-loop, it'd defeat the purpose of the course's moment. Besides, I don't think a for-loop would improve the readability here. But I thought your protest against the while-True was very well put: while-True is not too readable for a novice. Surely what's readable or more-natural /to someone/ is, well, subjective (yes, by definition). But perhaps we may agree that while rolling dice until a certain success, we want to roll them while something happens or doesn't happen. One of the two. So while-True is a bit of a jump. Therefore, in this case, the easier and more natural option is to say while-x-not-equal-y. But this approach seems to force me into initializing x, y with different values. > Returning to the concern: > > x, y = 0, 1 > c = 0 > > The first line is purely to ensure that the loop executes at least once, > ie the two assigned-values are not 'real'. Hence the disquiet! > > Initiating the counter is unavoidable (@Chris' suggestion notwithstanding). > > However, remember that Python (like decent DBs) has a concept (and an > idiom) of a value to be used when we don't (yet) know what the value > is/should be! Further that Python allows such a value to be used in > comparisons: > >>>> None != None > False >>>> None == None > True > > Leading to: > > c, x, y = 0, None, None > while ... > > > Which solution reverts to the original loop-contents. which seem more > obvious and thus more readable. (YMMV!) > > Simplicity over 'being clever'... I don't see it. You seem to have found what we seem to agree that it would be the more natural way to write the strategy. But I can't see it. It certainly isn't --8<---------------cut here---------------start------------->8--- def how_many_times_1(): c, x, y = 0, None, None while x != y: c = c + 1 x, y = roll() return c, x, y --8<---------------cut here---------------end--------------->8--- nor --8<---------------cut here---------------start------------->8--- def how_many_times_2(): c, x, y = 0, None, None while x == y: c = c + 1 x, y = dados() return c, x, y --8<---------------cut here---------------end--------------->8--- What do you have in mind? I couldn't see it. From hrouselle at jevedi.com Thu Sep 2 10:42:24 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 11:42:24 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> Message-ID: <86tuj3kpin.fsf@jevedi.com> David Raymond writes: >> def how_many_times(): >> x, y = 0, 1 >> c = 0 >> while x != y: >> c = c + 1 >> x, y = roll() >> return c, (x, y) > > Since I haven't seen it used in answers yet, here's another option using our new walrus operator > > def how_many_times(): > roll_count = 1 > while (rolls := roll())[0] != rolls[1]: > roll_count += 1 > return (roll_count, rolls) That's nice, although it doesn't seem more readable to a novice seeing a while for the first time, seeing a loop for the first time, than that while-True version. In fact, I think the while-True is the clearest so far. But it's always nice to spot a walrus in the wild! (If you're somewhere safe, that is.) From hrouselle at jevedi.com Thu Sep 2 10:43:57 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 11:43:57 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> Message-ID: <86k0jzkpg2.fsf@jevedi.com> Chris Angelico writes: > On Mon, Aug 30, 2021 at 11:13 PM David Raymond wrote: >> >> > def how_many_times(): >> > x, y = 0, 1 >> > c = 0 >> > while x != y: >> > c = c + 1 >> > x, y = roll() >> > return c, (x, y) >> >> Since I haven't seen it used in answers yet, here's another option using our new walrus operator >> >> def how_many_times(): >> roll_count = 1 >> while (rolls := roll())[0] != rolls[1]: >> roll_count += 1 >> return (roll_count, rolls) >> > > Since we're creating solutions that use features in completely > unnecessary ways, here's a version that uses collections.Counter: > > def how_many_times(): > return next((count, rolls) for count, rolls in > enumerate(iter(roll, None)) if len(Counter(rolls)) == 1) > > Do I get bonus points for it being a one-liner that doesn't fit in > eighty characters? Lol. You do not. In fact, this should be syntax error :-D --- as I guess it would be if it were a lambda expression? From michael.stemper at gmail.com Thu Sep 2 10:56:38 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 2 Sep 2021 09:56:38 -0500 Subject: urgent In-Reply-To: References: Message-ID: On 31/08/2021 18.02, Barry wrote: > The big problem with >>> is that it means a third level quote in email clients. > So when people cut-n-paste REPL output it?s formatted badly by email clients. > A prompt that avoided that issue would be nice. A little bit of piping fixes that: username at hostname$ cat text username at hostname$ python3 Python 3.5.2 (default, Jan 26 2021, 13:30:48) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys # in the original, this line will be messed up >>> sys.exit(0) # this one, too username at hostname$ sed 's/^>>> /REPL> /' < text username at hostname$ python3 Python 3.5.2 (default, Jan 26 2021, 13:30:48) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. REPL> import sys # in the original, this line will be messed up REPL> sys.exit(0) # this one, too username at hostname$ -- Michael F. Stemper What happens if you play John Cage's "4'33" at a slower tempo? From hrouselle at jevedi.com Thu Sep 2 11:08:21 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Thu, 02 Sep 2021 12:08:21 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: <86a6kvkobe.fsf@jevedi.com> Hope Rouselle writes: > Just sharing a case of floating-point numbers. Nothing needed to be > solved or to be figured out. Just bringing up conversation. > > (*) An introduction to me > > I don't understand floating-point numbers from the inside out, but I do > know how to work with base 2 and scientific notation. So the idea of > expressing a number as > > mantissa * base^{power} > > is not foreign to me. (If that helps you to perhaps instruct me on > what's going on here.) > > (*) A presentation of the behavior > >>>> import sys >>>> sys.version > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 > bit (AMD64)]' > >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>> sum(ls) > 39.599999999999994 > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>> sum(ls) > 39.60000000000001 > > All I did was to take the first number, 7.23, and move it to the last > position in the list. (So we have a violation of the commutativity of > addition.) Suppose these numbers are prices in dollar, never going beyond cents. Would it be safe to multiply each one of them by 100 and therefore work with cents only? For instance --8<---------------cut here---------------start------------->8--- >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>> sum(map(lambda x: int(x*100), ls)) / 100 39.6 >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>> sum(map(lambda x: int(x*100), ls)) / 100 39.6 --8<---------------cut here---------------end--------------->8--- Or multiplication by 100 isn't quite ``safe'' to do with floating-point numbers either? (It worked in this case.) I suppose that if I multiply it by a power of two, that would be an operation that I can be sure will not bring about any precision loss with floating-point numbers. Do you agree? From auriocus at gmx.de Thu Sep 2 10:47:45 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 2 Sep 2021 16:47:45 +0200 Subject: on floating-point numbers In-Reply-To: <86bl5bm6go.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> Message-ID: Am 02.09.21 um 15:51 schrieb Hope Rouselle: > Just sharing a case of floating-point numbers. Nothing needed to be > solved or to be figured out. Just bringing up conversation. > > (*) An introduction to me > > I don't understand floating-point numbers from the inside out, but I do > know how to work with base 2 and scientific notation. So the idea of > expressing a number as > > mantissa * base^{power} > > is not foreign to me. (If that helps you to perhaps instruct me on > what's going on here.) > > (*) A presentation of the behavior > >>>> import sys >>>> sys.version > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]' > >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>> sum(ls) > 39.599999999999994 > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>> sum(ls) > 39.60000000000001 > > All I did was to take the first number, 7.23, and move it to the last > position in the list. (So we have a violation of the commutativity of > addition.) I believe it is not commutativity, but associativity, that is violated. Even for floating point, a+b=b+a except for maybe some extreme cases like denormliazed numbers etc. But in general (a+b)+c != a+ (b+c) Consider decimal floating point with 2 digits. a=1 b=c=0.04 Then you get LHS; (1 + 0.04) + 0.04 = 1 + 0.04 = 1 RHS: 1 + (0.04 + 0.04) = 1 + 0.08 = 1.1 Your sum is evaluated like (((a + b) + c) + ....) and hence, if you permute the numbers, it can be unequal. If you need better accuracy, there is the Kahan summation algorithm and other alternatives: https://en.wikipedia.org/wiki/Kahan_summation_algorithm Christian From auriocus at gmx.de Thu Sep 2 10:49:03 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 2 Sep 2021 16:49:03 +0200 Subject: on floating-point numbers In-Reply-To: <4105d35c-cb9a-42d1-9985-f3c5b6ff6209n@googlegroups.com> References: <86bl5bm6go.fsf@jevedi.com> <4105d35c-cb9a-42d1-9985-f3c5b6ff6209n@googlegroups.com> Message-ID: Am 02.09.21 um 16:49 schrieb Julio Di Egidio: > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote: >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote: > >>> 39.60000000000001 >> >> Welcome to the exciting world of roundoff error: > > Welcome to the exiting world of Usenet. > > *Plonk* Pretty harsh, isn't it? He gave a concise example of the same inaccuracy right afterwards. Christian From hongyi.zhao at gmail.com Wed Sep 1 22:15:57 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 1 Sep 2021 19:15:57 -0700 (PDT) Subject: Mutually exclusive options with argparse. In-Reply-To: References: Message-ID: On Wednesday, September 1, 2021 at 11:20:21 PM UTC+8, Lee Congdon wrote: > Does a mutually exclusive group, as described in "Mutual exclusion" at > https://docs.python.org/3/library/argparse.html meet your needs? Thanks for letting me know about this feature. I'll give it a try. > > On Wed, Sep 1, 2021 at 9:48 AM hongy... at gmail.com > wrote: > > See the following code snippets [1] for implementation of the exclusive > > options with argparse: > > > > def query_builder(args): > > if args.r and args.s: > > sys.exit(Term.FAIL + 'Only one of -re and -sql should be set' + > > Term.ENDC) > > sum_status = sum(1 for x in [args.failure, args.code != -1] if x) > > if sum_status > 1: > > sys.exit(Term.FAIL + ('Only one of --failure and --code has to be > > set') + Term.ENDC) > > > > [1] > > https://github.com/hongyi-zhao/recent2/blob/5486afbd56a6b06bb149a3ea969fb33d9d8b288f/recent2.py#L391 > > > > It seems that the above method is awkward, but I'm out of idea to work out > > more graceful solutions. Any comment/suggestion/enhancement will be highly > > appreciated. > > > > Regards, > > HY > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > -- > +1-202-507-9867, Twitter @lcongdon From hongyi.zhao at gmail.com Wed Sep 1 23:06:21 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 1 Sep 2021 20:06:21 -0700 (PDT) Subject: Mutually exclusive options with argparse. In-Reply-To: References: Message-ID: <02cc23ff-ee8e-4d06-a6b8-d9e48a2d81a0n@googlegroups.com> On Thursday, September 2, 2021 at 10:16:09 AM UTC+8, hongy... at gmail.com wrote: > On Wednesday, September 1, 2021 at 11:20:21 PM UTC+8, Lee Congdon wrote: > > Does a mutually exclusive group, as described in "Mutual exclusion" at > > https://docs.python.org/3/library/argparse.html meet your needs? > Thanks for letting me know about this feature. I'll give it a try. The exact document URL is located at here [1]. [1] https://docs.python.org/3/library/argparse.html#mutual-exclusion > > > > On Wed, Sep 1, 2021 at 9:48 AM hongy... at gmail.com > > wrote: > > > See the following code snippets [1] for implementation of the exclusive > > > options with argparse: > > > > > > def query_builder(args): > > > if args.r and args.s: > > > sys.exit(Term.FAIL + 'Only one of -re and -sql should be set' + > > > Term.ENDC) > > > sum_status = sum(1 for x in [args.failure, args.code != -1] if x) > > > if sum_status > 1: > > > sys.exit(Term.FAIL + ('Only one of --failure and --code has to be > > > set') + Term.ENDC) > > > > > > [1] > > > https://github.com/hongyi-zhao/recent2/blob/5486afbd56a6b06bb149a3ea969fb33d9d8b288f/recent2.py#L391 > > > > > > It seems that the above method is awkward, but I'm out of idea to work out > > > more graceful solutions. Any comment/suggestion/enhancement will be highly > > > appreciated. > > > > > > Regards, > > > HY > > > -- > > > https://mail.python.org/mailman/listinfo/python-list > > > > > > > > > -- > > +1-202-507-9867, Twitter @lcongdon From mystirk at gmail.com Wed Sep 1 11:46:19 2021 From: mystirk at gmail.com (Alex Kaye) Date: Wed, 1 Sep 2021 08:46:19 -0700 Subject: ( AI retail store inventory control.) Message-ID: Looking for Python package to develop positive inventory control for a chain of stores. Alex kaye From pkpearson at nowhere.invalid Thu Sep 2 10:41:17 2021 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 2 Sep 2021 14:41:17 GMT Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote: > >>>> import sys >>>> sys.version > '3.8.10 (tags/... > >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>> sum(ls) > 39.599999999999994 > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>> sum(ls) > 39.60000000000001 Welcome to the exciting world of roundoff error: Python 3.5.3 (default, Jul 9 2020, 13:00:10) [GCC 6.3.0 20170516] on linux >>> 0.1 + 0.2 + 9.3 == 0.1 + 9.3 + 0.2 False >>> -- To email me, substitute nowhere->runbox, invalid->com. From janburse at fastmail.fm Thu Sep 2 12:39:35 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Thu, 2 Sep 2021 18:39:35 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: More best kept secrets of Prolog: Pattern Matching Everybody loves pattern matching. Languages like Python, release 3.10, even provide it now. There is now a match/case statement in Python. But Prolog users will scratch their head. Will my if-then-else be as fast as a imperative switch jump table lookup? Dogelog runtime has stepped up its game concerning pattern matching. It now provides ECLiPSe Prolog disjunction and if-then-else indexing. Take this example: ?- [user]. foo(X,Y) :- X=baz, Y=2; X=bar -> Y=1. SWI-Prolog leaves a choice point, so no clause indexing used: /* SWI-Prolog 8.3.26 */ ?- foo(baz,Z). Z = 2 ; %%% Spurious Choice Point false. Dogelog doesn't leave a choice point, since it can index the disjunction and if-then-else: /* Dogelog Runtime 0.9.3 */ ?- foo(baz,Z). Z = 2. %%% No Choice Point See also: Preview: Dogelog disjunction and if-then-else indexing. (Jekejeke) https://twitter.com/dogelogch/status/1433446729974796293 Preview: Dogelog disjunction and if-then-else indexing. (Jekejeke) https://www.facebook.com/groups/dogelog From tedpfspencer at gmail.com Thu Sep 2 13:02:39 2021 From: tedpfspencer at gmail.com (Edward Spencer) Date: Thu, 2 Sep 2021 10:02:39 -0700 (PDT) Subject: Add a method to list the current named logging levels Message-ID: <69fe3a05-45cb-4bb5-b2ec-56ca8423f7ebn@googlegroups.com> Sometimes I like to pass the logging level up to the command line params so my user can specific what level of logging they want. However there is no easy method for pulling the named logging level names. Looking into the code, it would actually be incredibly easy to implement; in `logging.__init__.py`; def listLevelNames(): return _nameToLevel.keys() You could obviously add some other features, like listing only the defaults, sorted by numerical level or alphabetically, etc. But really this basic implementation would be enough to expose the internal variables which shouldn't be accessed because they change (and in fact, between python 2 and 3, they did). Any thoughts? Thanks, Ed Spencer From rosuav at gmail.com Thu Sep 2 14:22:20 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 04:22:20 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Fri, Sep 3, 2021 at 4:18 AM Dennis Lee Bieber wrote: > > On Tue, 31 Aug 2021 16:53:14 -0500, 2QdxY4RzWzUUiLuE at potatochowder.com > declaimed the following: > > >On 2021-09-01 at 07:32:43 +1000, > >Chris Angelico wrote: > >> If we could abolish DST world-wide, life would be far easier. All the > >> rest of it would be easy enough to handle. > > > >Agreed. > > > > Unfortunately, most of the proposals in the US seem to be that > /standard time/ would be abolished, and DST would rule year-round. Hence > putting the center of the time zone one hour off from solar mean noon; > which is what the time zones were originally based upon. > I'd be fine with that, honestly. It's a bit 'off', but it isn't THAT big a deal. If some city in the US decides that it wants its timezone to be a few hours AHEAD of UTC, more power to them, just as long as it's consistent year-round. ChrisA From rosuav at gmail.com Thu Sep 2 14:28:42 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 04:28:42 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Fri, Sep 3, 2021 at 4:22 AM Alan Gauld via Python-list wrote: > > On 31/08/2021 22:13, Chris Angelico wrote: > > > But ultimately, it all just means that timezones are too hard for > > humans to handle, and we MUST handle them using IANA's database. It is > > the only way. > > Except for the places that don't follow the IANA scheme and/or > dynamically change their time settings on a whim. To be complete > you need the ability to manually override too. > What places are those? IANA maintains the database by noticing changes and announcements, and updating the database. I don't think governments need to "opt in" or anything. Stuff happens because people do stuff, and people do stuff because they want to be able to depend on timezone conversions. There ARE times when a government makes a change too quickly to get updates out to everyone, especially those who depend on an OS-provided copy of tzdata, so I agree with the "on a whim" part. Though, fortunately, that's rare. ChrisA From rosuav at gmail.com Thu Sep 2 14:43:02 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 04:43:02 +1000 Subject: on floating-point numbers In-Reply-To: <86bl5bm6go.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle wrote: > > Just sharing a case of floating-point numbers. Nothing needed to be > solved or to be figured out. Just bringing up conversation. > > (*) An introduction to me > > I don't understand floating-point numbers from the inside out, but I do > know how to work with base 2 and scientific notation. So the idea of > expressing a number as > > mantissa * base^{power} > > is not foreign to me. (If that helps you to perhaps instruct me on > what's going on here.) > > (*) A presentation of the behavior > > >>> import sys > >>> sys.version > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]' > > >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >>> sum(ls) > 39.599999999999994 > > >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >>> sum(ls) > 39.60000000000001 > > All I did was to take the first number, 7.23, and move it to the last > position in the list. (So we have a violation of the commutativity of > addition.) > It's not about the commutativity of any particular pair of operands - that's always guaranteed. What you're seeing here is the results of intermediate rounding. Try this: >>> def sum(stuff): ... total = 0 ... for thing in stuff: ... total += thing ... print(thing, "-->", total) ... return total ... >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>> sum(ls) 7.23 --> 7.23 8.41 --> 15.64 6.15 --> 21.79 2.31 --> 24.099999999999998 7.73 --> 31.83 7.77 --> 39.599999999999994 39.599999999999994 >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>> sum(ls) 8.41 --> 8.41 6.15 --> 14.56 2.31 --> 16.87 7.73 --> 24.6 7.77 --> 32.370000000000005 7.23 --> 39.60000000000001 39.60000000000001 >>> Nearly all floating-point confusion stems from an assumption that the input values are exact. They usually aren't. Consider: >>> from fractions import Fraction >>> for n in ls: print(n, Fraction(*n.as_integer_ratio())) ... 8.41 2367204554136617/281474976710656 6.15 3462142213541069/562949953421312 2.31 5201657569612923/2251799813685248 7.73 2175801569973371/281474976710656 7.77 2187060569041797/281474976710656 7.23 2035064081618043/281474976710656 Those are the ACTUAL values you're adding. Do the same exercise with the partial sums, and see where the rounding happens. It's probably happening several times, in fact. The naive summation algorithm used by sum() is compatible with a variety of different data types - even lists, although it's documented as being intended for numbers - but if you know for sure that you're working with floats, there's a more accurate algorithm available to you. >>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77]) 39.6 >>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23]) 39.6 It seeks to minimize loss to repeated rounding and is, I believe, independent of data order. ChrisA From rosuav at gmail.com Thu Sep 2 14:58:35 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 04:58:35 +1000 Subject: on writing a while loop for rolling two dice In-Reply-To: <864kb3m4qi.fsf@jevedi.com> References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> Message-ID: On Fri, Sep 3, 2021 at 4:33 AM Hope Rouselle wrote: > Yeah. Here's a little context. I came across this by processing a list > of exercises. (I'm teaching a course --- you know that by now, I > guess.) So the first thing I observed was the equal volume of work > dedicated to while loops and for loops --- so I decided to compared > which appeared more often in a certain sample of well-written Python > code. It turns out the for loop was much more frequent. Students have > been reporting too much work in too little time, so I decided to reduce > the number of exercises involving while loops. When I began to look at > the exercises, to see which ones I'd exclude, I decided to exclude them > all --- lol! --- except for one. The one that remained was this one > about rolling dice until a satisfying result would appear. (All other > ones were totally more naturally written with a for loop.) > > So if I were to also write this with a for-loop, it'd defeat the purpose > of the course's moment. Besides, I don't think a for-loop would improve > the readability here. It's on the cusp. When you ask someone to express the concept of "do this until this happens", obviously that's a while loop; but as soon as you introduce the iteration counter, it becomes less obvious, since "iterate over counting numbers until this happens" is a quite viable way to express this. However, if the students don't know itertools.count(), they'll most likely put in an arbitrary limit (like "for c in range(100000000)"), which you can call them out for. > But I thought your protest against the while-True was very well put: > while-True is not too readable for a novice. Surely what's readable or > more-natural /to someone/ is, well, subjective (yes, by definition). > But perhaps we may agree that while rolling dice until a certain > success, we want to roll them while something happens or doesn't happen. > One of the two. So while-True is a bit of a jump. Therefore, in this > case, the easier and more natural option is to say while-x-not-equal-y. That may be the case, but in Python, I almost never write "while True". Consider the two while loops in this function: https://github.com/Rosuav/shed/blob/master/autohost_manager.py#L92 Thanks to Python's flexibility and efficient compilation, these loops are as descriptive as those with actual conditions, while still behaving exactly like "while True". (The inner loop, "more pages", looks superficially like it should be a for loop - "for page in pages:" - but the data is coming from successive API calls, so it can't know.) > I don't see it. You seem to have found what we seem to agree that it > would be the more natural way to write the strategy. But I can't see > it. It certainly isn't > > --8<---------------cut here---------------start------------->8--- > def how_many_times_1(): > c, x, y = 0, None, None > while x != y: > c = c + 1 > x, y = roll() > return c, x, y > --8<---------------cut here---------------end--------------->8--- > > nor > > --8<---------------cut here---------------start------------->8--- > def how_many_times_2(): > c, x, y = 0, None, None > while x == y: > c = c + 1 > x, y = dados() > return c, x, y > --8<---------------cut here---------------end--------------->8--- > > What do you have in mind? I couldn't see it. You're overlaying two loops here. One is iterating "c" up from zero, the other is calling a function and testing its results. It's up to you which of these should be considered the more important, and which is a bit of extra work added onto it. With the counter as primary, you get something like this: for c in itertools.count(): x, y = roll() if x == y: return c, x, y With the roll comparison as primary, you get this: c, x, y = 0, 0, 1 while x != y: x, y = roll() c += 1 return c, x, y Reworking the second into a do-while style (Python doesn't have that, so we have to write it manually): c = 0 while "x and y differ": x, y = roll() c += 1 if x == y: break return c, x, y And at this point, it's looking pretty much identical to the for loop version. Ultimately, they're all the same and you can pick and choose elements from each of them. ChrisA From rosuav at gmail.com Thu Sep 2 15:08:56 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 05:08:56 +1000 Subject: on writing a while loop for rolling two dice In-Reply-To: <86k0jzkpg2.fsf@jevedi.com> References: <86r1edlqxw.fsf@jevedi.com> <86k0jzkpg2.fsf@jevedi.com> Message-ID: On Fri, Sep 3, 2021 at 4:51 AM Hope Rouselle wrote: > > Chris Angelico writes: > > > On Mon, Aug 30, 2021 at 11:13 PM David Raymond wrote: > >> > >> > def how_many_times(): > >> > x, y = 0, 1 > >> > c = 0 > >> > while x != y: > >> > c = c + 1 > >> > x, y = roll() > >> > return c, (x, y) > >> > >> Since I haven't seen it used in answers yet, here's another option using our new walrus operator > >> > >> def how_many_times(): > >> roll_count = 1 > >> while (rolls := roll())[0] != rolls[1]: > >> roll_count += 1 > >> return (roll_count, rolls) > >> > > > > Since we're creating solutions that use features in completely > > unnecessary ways, here's a version that uses collections.Counter: > > > > def how_many_times(): > > return next((count, rolls) for count, rolls in > > enumerate(iter(roll, None)) if len(Counter(rolls)) == 1) > > > > Do I get bonus points for it being a one-liner that doesn't fit in > > eighty characters? > > Lol. You do not. In fact, this should be syntax error :-D --- as I > guess it would be if it were a lambda expression? It got split across lines when I posted it, but if I did this in a program, I'd make it a single long line. That said, though - Python doesn't mind if you mess up the indentation inside a parenthesized expression. Even broken like this, it WILL work. It just looks even uglier than it does with proper indentation :) BTW, this sort of thing is great as an anti-plagiarism check. If a student ever turns in an abomination like this, you can be extremely confident that it was copied from some programming site/list. Especially since I've used the two-arg version of iter() in there - that's quite a rarity. Hmmmmm. My mind is straying to evil things. The two-arg iter can do SO much more than I'm using it for here. By carefully designing the second argument, we could make something that is equal to anything whose two elements are equal, which would then terminate the loop. This... could be a lot worse than it seems. I'll leave it as an exercise for the reader to figure out how to capture the matching elements for return. ChrisA From python at mrabarnett.plus.com Thu Sep 2 15:11:23 2021 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 2 Sep 2021 20:11:23 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <4d5d265c-59f0-8559-4b35-22695ad4a3e2@mrabarnett.plus.com> Message-ID: <30880c73-863a-ef94-e3a3-235aa589f070@mrabarnett.plus.com> On 2021-09-02 08:40, Alan Gauld via Python-list wrote: > On 31/08/2021 23:31, Chris Angelico wrote: > >> Ah, good to know. I think that actually makes a lot of sense; in the >> US, they try to let everyone pretend that the rest of the world >> doesn't exist ("we always change at 2AM"), but in Europe, they try to >> synchronize for the convenience of commerce ("everyone changes at 1AM >> UTC"). > > There's another gotcha with DST changes. The EU and USA have different > dates on which they change to DST. > > In one of them (I can't recall which is which) they change on the 4th > weekend of October/March in the other they change on the last weekend. > > That means on some years (when there are 5 weekends) there is a > week when one has changed and the other hasn't. That caused us > a lot of head scratching the first time we encountered it because > our service centres in the US and EU were getting inconsistent > time reporting and some updates showing as having happened in > the future! > In the EU (and UK) it's the last Sunday in March/October. In the US it's second Sunday in March and the first Sunday in November. I know which one I find easier to remember! From rosuav at gmail.com Thu Sep 2 15:24:56 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 05:24:56 +1000 Subject: on floating-point numbers In-Reply-To: <86a6kvkobe.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle wrote: > > Hope Rouselle writes: > > > Just sharing a case of floating-point numbers. Nothing needed to be > > solved or to be figured out. Just bringing up conversation. > > > > (*) An introduction to me > > > > I don't understand floating-point numbers from the inside out, but I do > > know how to work with base 2 and scientific notation. So the idea of > > expressing a number as > > > > mantissa * base^{power} > > > > is not foreign to me. (If that helps you to perhaps instruct me on > > what's going on here.) > > > > (*) A presentation of the behavior > > > >>>> import sys > >>>> sys.version > > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 > > bit (AMD64)]' > > > >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >>>> sum(ls) > > 39.599999999999994 > > > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >>>> sum(ls) > > 39.60000000000001 > > > > All I did was to take the first number, 7.23, and move it to the last > > position in the list. (So we have a violation of the commutativity of > > addition.) > > Suppose these numbers are prices in dollar, never going beyond cents. > Would it be safe to multiply each one of them by 100 and therefore work > with cents only? For instance Yes and no. It absolutely *is* safe to always work with cents, but to do that, you have to be consistent: ALWAYS work with cents, never with floating point dollars. (Or whatever other unit you choose to use. Most currencies have a smallest-normally-used-unit, with other currency units (where present) being whole number multiples of that minimal unit. Only in forex do you need to concern yourself with fractional cents or fractional yen.) But multiplying a set of floats by 100 won't necessarily solve your problem; you may have already fallen victim to the flaw of assuming that the numbers are represented accurately. > --8<---------------cut here---------------start------------->8--- > >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >>> sum(map(lambda x: int(x*100), ls)) / 100 > 39.6 > > >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >>> sum(map(lambda x: int(x*100), ls)) / 100 > 39.6 > --8<---------------cut here---------------end--------------->8--- > > Or multiplication by 100 isn't quite ``safe'' to do with floating-point > numbers either? (It worked in this case.) You're multiplying and then truncating, which risks a round-down error. Try adding a half onto them first: int(x * 100 + 0.5) But that's still not a perfect guarantee. Far safer would be to consider monetary values to be a different type of value, not just a raw number. For instance, the value $7.23 could be stored internally as the integer 723, but you also know that it's a value in USD, not a simple scalar. It makes perfect sense to add USD+USD, it makes perfect sense to multiply USD*scalar, but it doesn't make sense to multiply USD*USD. > I suppose that if I multiply it by a power of two, that would be an > operation that I can be sure will not bring about any precision loss > with floating-point numbers. Do you agree? Assuming you're nowhere near 2**53, yes, that would be safe. But so would multiplying by a power of five. The problem isn't precision loss from the multiplication - the problem is that your input numbers aren't what you think they are. That number 7.23, for instance, is really.... >>> 7.23.as_integer_ratio() (2035064081618043, 281474976710656) ... the rational number 2035064081618043 / 281474976710656, which is very close to 7.23, but not exactly so. (The numerator would have to be ...8042.88 to be exactly correct.) There is nothing you can do at this point to regain the precision, although a bit of multiplication and rounding can cheat it and make it appear as if you did. Floating point is a very useful approximation to real numbers, but real numbers aren't the best way to represent financial data. Integers are. ChrisA From rosuav at gmail.com Thu Sep 2 14:30:19 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 04:30:19 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Fri, Sep 3, 2021 at 4:26 AM Alan Gauld via Python-list wrote: > > On 31/08/2021 22:32, Chris Angelico wrote: > > > If we could abolish DST world-wide, life would be far easier. All the > > rest of it would be easy enough to handle. > We tried that in the UK for 2 years back in the '70s and very > quickly reverted to DST when they realized that the number > of fatalities among young children going to school doubled > during those two years. > > Without DST the schools opened in the dark so all the kids > had to travel to school in the dark and the number of > traffic accidents while crossing roads jumped. How do they manage in winter? Can that be solved with better street lighting? That was fifty years ago now, and the negative consequences of DST are far stronger now. ChrisA From rosuav at gmail.com Thu Sep 2 15:11:16 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 05:11:16 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <4d5d265c-59f0-8559-4b35-22695ad4a3e2@mrabarnett.plus.com> Message-ID: On Fri, Sep 3, 2021 at 4:40 AM Alan Gauld via Python-list wrote: > > On 31/08/2021 23:31, Chris Angelico wrote: > > > Ah, good to know. I think that actually makes a lot of sense; in the > > US, they try to let everyone pretend that the rest of the world > > doesn't exist ("we always change at 2AM"), but in Europe, they try to > > synchronize for the convenience of commerce ("everyone changes at 1AM > > UTC"). > > There's another gotcha with DST changes. The EU and USA have different > dates on which they change to DST. > > In one of them (I can't recall which is which) they change on the 4th > weekend of October/March in the other they change on the last weekend. > > That means on some years (when there are 5 weekends) there is a > week when one has changed and the other hasn't. That caused us > a lot of head scratching the first time we encountered it because > our service centres in the US and EU were getting inconsistent > time reporting and some updates showing as having happened in > the future! > I live in Australia. You folks all change in *the wrong direction*. Twice a year, there's a roughly two-month period that I call "DST season", when different countries (or groups of countries) switch DST. It is a nightmare to schedule anything during that time. The ONLY way is to let the computer handle it. Don't try to predict ANYTHING about DST manually. ChrisA From alan.gauld at yahoo.co.uk Thu Sep 2 16:20:13 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 2 Sep 2021 21:20:13 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 02/09/2021 19:28, Chris Angelico wrote: >> Except for the places that don't follow the IANA scheme and/or >> dynamically change their time settings on a whim. To be complete >> you need the ability to manually override too. >> > > What places are those? Mainly small non-tech oriented places such as small pacific islands or principalities with local governance - such as by a group of tribal elders. I mentioned earlier the example of Andorra announcing on the Friday night before a DST change that they were deferring it for a week to preserve the skiing conditions. But we came across several similar situations in dealing with multi-national service centres. > IANA maintains the database by noticing changes > and announcements, and updating the database. But don;t those have to be electronic in nature? How, for example would it pick up the radio news announcement mentioned above? > governments need to "opt in" or anything. Stuff happens because people > do stuff, and people do stuff because they want to be able to depend > on timezone conversions. Umm, they do DST because it makes their lives easier - more daylight, extra work time. etc. The needs of, or impact on, computers in these kinds of small localities and communities are way down the pecking order. > There ARE times when a government makes a change too quickly to get > updates out to everyone, especially those who depend on an OS-provided > copy of tzdata, so I agree with the "on a whim" part. Though, > fortunately, that's rare. I agree it is very rare and if you only operate in mainstream localities you probably never see it as an issue, it's only when you need to support "off grid" locations that manual control becomes important. Also the problems we had were about 15 years ago, things may be better ordered nowadays. (I've been retired for 7 years so can't speak of more recent events) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Thu Sep 2 16:25:27 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 2 Sep 2021 21:25:27 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: <30880c73-863a-ef94-e3a3-235aa589f070@mrabarnett.plus.com> References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <4d5d265c-59f0-8559-4b35-22695ad4a3e2@mrabarnett.plus.com> <30880c73-863a-ef94-e3a3-235aa589f070@mrabarnett.plus.com> Message-ID: On 02/09/2021 20:11, MRAB wrote: >> In one of them (I can't recall which is which) they change on the 4th >> weekend of October/March in the other they change on the last weekend. >> >> > In the EU (and UK) it's the last Sunday in March/October. > > In the US it's second Sunday in March and the first Sunday in November. > > I know which one I find easier to remember! Interesting. I remember it as closer than that. The bugs we found were due to differences in the DST settings of the BIOS in the PCs. (They were deliberately all sourced from DELL but the EU PCs had a slightly different BIOS). The differences you cite should have thrown up issues every year. I must see if I can find my old log books... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From wlfraed at ix.netcom.com Thu Sep 2 16:25:35 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 02 Sep 2021 16:25:35 -0400 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle declaimed the following: >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>> sum(ls) >39.599999999999994 > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>> sum(ls) >39.60000000000001 > >All I did was to take the first number, 7.23, and move it to the last >position in the list. (So we have a violation of the commutativity of >addition.) > https://www.amazon.com/Real-Computing-Made-Engineering-Calculations-dp-B01K0Q03AA/dp/B01K0Q03AA/ref=mt_other?_encoding=UTF8&me=&qid= -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Thu Sep 2 16:48:57 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 02 Sep 2021 16:48:57 -0400 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: On Thu, 02 Sep 2021 12:08:21 -0300, Hope Rouselle declaimed the following: >Suppose these numbers are prices in dollar, never going beyond cents. >Would it be safe to multiply each one of them by 100 and therefore work >with cents only? For instance > A lot of software with a "monetary" data type uses scaled INTEGERS for that... M$ Excel uses four decimal places, internally scaled. The Ada language has both FIXED and FLOAT data types; for FIXED one specifies the delta between adjacent values that must be met (the compiler is free to use something with more resolution internally). Money should never be treated as a floating value. >I suppose that if I multiply it by a power of two, that would be an >operation that I can be sure will not bring about any precision loss >with floating-point numbers. Do you agree? Are we talking IEEE floats? Or some of the ancient formats used for computers that may not have had hardware floating point units, or predate the IEEE standard. Normalized with suppressed leading bit? (If normalization always puts the most significant bit at the binary point... Why store that bit?, shift it out and gain another bit at the small end) Xerox Sigma floats used an exponent based on radix 16. A normalized mantissa could have up to three leading 0 bits. Motorola Fast Floating Point (software float implementation used on base Amiga systems -- the exponent was in the low byte) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Thu Sep 2 16:53:48 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 02 Sep 2021 16:53:48 -0400 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico declaimed the following: > >The naive summation algorithm used by sum() is compatible with a >variety of different data types - even lists, although it's documented >as being intended for numbers - but if you know for sure that you're >working with floats, there's a more accurate algorithm available to >you. > >>>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77]) >39.6 >>>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23]) >39.6 > >It seeks to minimize loss to repeated rounding and is, I believe, >independent of data order. > Most likely it sorts the data so the smallest values get summed first, and works its way up to the larger values. That way it minimizes the losses that occur when denormalizing a value (to set the exponent equal to that of the next larger value). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From grant.b.edwards at gmail.com Thu Sep 2 17:41:59 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 2 Sep 2021 21:41:59 -0000 (UTC) Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: On 2021-09-02, Hope Rouselle wrote: > Suppose these numbers are prices in dollar, never going beyond cents. > Would it be safe to multiply each one of them by 100 and therefore work > with cents only? The _practical_ answer is that no, it's not safe to use floating point when doing normal bookeeping type stuff with money. At least not if you want everything to balance correctly at the end of the day (week, month, quarter, year or etc.). Use integer cents, or mills or whatever. If you have to use floating point to calculate a payment or credit/debit amount, always round or truncate the result back to an integer value in your chosen units before actually using that amount for anything. In theory, decimal floating point should be usable, but I've never personally worked with it. Back in the day (1980's) microcomputers didn't have floating point hardware, and many compilers allowed you to choose between base-2 floating point and base-10 (BCD) floating point. The idea was that if you were doing financial stuff, you could use BCD floating point. -- Grant From medina.rc at gmail.com Thu Sep 2 15:28:31 2021 From: medina.rc at gmail.com (Richard Medina) Date: Thu, 2 Sep 2021 12:28:31 -0700 (PDT) Subject: Select columns based on dates - Pandas Message-ID: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com> Hello, forum, I have a data frame with covid-19 cases per month from 2019 - 2021 like a header like this: Country, 01/01/2019, 2/01/2019, 01/02/2019, 3/01/2019, ... 01/01/2021, 2/01/2021, 01/02/2021, 3/01/2021 I want to filter my data frame for columns of a specific month range of march to September of 2019, 2020, and 2021 only (three data frames). Any ideas? Thank you From drsalists at gmail.com Thu Sep 2 18:14:33 2021 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 2 Sep 2021 15:14:33 -0700 Subject: Request for argmax(list) and argmin(list) In-Reply-To: References: <7031d0cbb8b2bfa31edb16995b5b6f2280c51cb8.camel@anode.ca> Message-ID: How about this?: python3 -c 'list_ = [1, 3, 5, 4, 2]; am = max((value, index) for index, value in enumerate(list_)); print(am)' On Wed, Sep 1, 2021 at 6:51 AM ABCCDE921 wrote: > Because that does 2 passes over the entire array when you only need one > and there is no option to specify if you want the leftmost or rightmost > element > > > On Wednesday, September 1, 2021 at 12:02:29 PM UTC+5:30, Paul Bryan wrote: > > Why not: > > > > >>> l = [1, 3, 5, 9, 2, 7] > > >>> l.index(max(l)) > > 3 > > >>> l.index(min(l)) > > 0 > > On Tue, 2021-08-31 at 21:25 -0700, ABCCDE921 wrote: > > > I dont want to import numpy > > > > > > argmax(list) > > > returns index of (left most) max element > > > > > > argmin(list) > > > returns index of (left most) min element > -- > https://mail.python.org/mailman/listinfo/python-list > From lucas at bourneuf.net Thu Sep 2 18:25:23 2021 From: lucas at bourneuf.net (lucas) Date: Fri, 3 Sep 2021 00:25:23 +0200 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> Message-ID: <12c5f226-6aef-5551-b60b-eb8d5bef445c@bourneuf.net> >> def how_many_times(): >> x, y = 0, 1 >> c = 0 >> while x != y: >> c = c + 1 >> x, y = roll() >> return c, (x, y) > > Since I haven't seen it used in answers yet, here's another option using our new walrus operator > > def how_many_times(): > roll_count = 1 > while (rolls := roll())[0] != rolls[1]: > roll_count += 1 > return (roll_count, rolls) > I would go even further, saying there is no need to ?roll dices?: def how_many_times(): nb_times = random.choice([n for n in range(50) for _ in range(round(10000*(1/6)*(5/6)**(n-1)))]) return nb_times, (random.randint(1, 6),) * 2 If i had more time on my hands, i would do something with bissect to get nb_times with more precision, as i have (mis)calculated that the probability of having nb_times = N is N = (1/6) * (5/6) ** (N-1) Something like this may work: nb_times = [random.random() < (1/6) * (5/6) ** (N-1) for N in range(1, 50)].index(True)+1 From dieter at handshake.de Thu Sep 2 18:35:40 2021 From: dieter at handshake.de (Dieter Maurer) Date: Fri, 3 Sep 2021 00:35:40 +0200 Subject: Add a method to list the current named logging levels In-Reply-To: <69fe3a05-45cb-4bb5-b2ec-56ca8423f7ebn@googlegroups.com> References: <69fe3a05-45cb-4bb5-b2ec-56ca8423f7ebn@googlegroups.com> Message-ID: <24881.20924.937699.552812@ixdm.fritz.box> Edward Spencer wrote at 2021-9-2 10:02 -0700: >Sometimes I like to pass the logging level up to the command line params so my user can specific what level of logging they want. However there is no easy method for pulling the named logging level names. > >Looking into the code, it would actually be incredibly easy to implement; > >in `logging.__init__.py`; > >def listLevelNames(): > return _nameToLevel.keys() > >You could obviously add some other features, like listing only the defaults, sorted by numerical level or alphabetically, etc. But really this basic implementation would be enough to expose the internal variables which shouldn't be accessed because they change (and in fact, between python 2 and 3, they did). > >Any thoughts? Usually, you use 5 well known log levels: "DEBUG", "INFO", "WARNING", "ERROR" and "CRITICAL". No need to provide a special function listing those levels. -- Dieter From msuginoo at reversalpoint.com Thu Sep 2 19:12:19 2021 From: msuginoo at reversalpoint.com (Michio Suginoo) Date: Thu, 2 Sep 2021 20:12:19 -0300 Subject: Scraping of Google Map to get the address of locations for given geocodes. Message-ID: Dear all, I have the following question regarding how to scrape Google Map to get address based on a given list of geocodes. Given a list of geocodes (latitude; longitude) of locations, I would like to scrape municipalities of all the spots in the list. How can I do that? For example, I have lat lon 1) -34.5722317 -58.4314464 2) -34.553906 -58.4520949 3) -34.5661444 -58.4964289 4) -34.5648053 -58.4431567 5) -34.5664089 -58.4323004 6) -34.5664089 -58.4323004 And I want to get the name of municipality (city/town), or alternatively the address, of each location. I would appreciate it, if anyone can advise me how to do that. Thanks Best regards Michio From rosuav at gmail.com Thu Sep 2 19:20:12 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 09:20:12 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Fri, Sep 3, 2021 at 8:01 AM Alan Gauld via Python-list wrote: > > On 02/09/2021 19:28, Chris Angelico wrote: > > >> Except for the places that don't follow the IANA scheme and/or > >> dynamically change their time settings on a whim. To be complete > >> you need the ability to manually override too. > >> > > > > What places are those? > > Mainly small non-tech oriented places such as small pacific islands > or principalities with local governance - such as by a group of > tribal elders. I mentioned earlier the example of Andorra announcing > on the Friday night before a DST change that they were deferring > it for a week to preserve the skiing conditions. But we came across > several similar situations in dealing with multi-national service centres. > > > IANA maintains the database by noticing changes > > and announcements, and updating the database. > > But don;t those have to be electronic in nature? How, for example > would it pick up the radio news announcement mentioned above? Can't find the specific example of Andorra, but there have been plenty of times when someone's reported a time change to the IANA list and it's resulted in the zonedata being updated. "It" picks up changes by people reporting them, because IANA is, ultimately, a group of people. > > governments need to "opt in" or anything. Stuff happens because people > > do stuff, and people do stuff because they want to be able to depend > > on timezone conversions. > > Umm, they do DST because it makes their lives easier - more daylight, > extra work time. etc. The needs of, or impact on, computers in these > kinds of small localities and communities are way down the pecking order. Yes, and what happens when those changes make other people's lives harder because tzdata is out of date? People change tzdata to be up-to-date. It's not the local government that maintains tzdata. > > There ARE times when a government makes a change too quickly to get > > updates out to everyone, especially those who depend on an OS-provided > > copy of tzdata, so I agree with the "on a whim" part. Though, > > fortunately, that's rare. > > I agree it is very rare and if you only operate in mainstream > localities you probably never see it as an issue, it's only > when you need to support "off grid" locations that manual > control becomes important. Also the problems we had were about > 15 years ago, things may be better ordered nowadays. (I've been > retired for 7 years so can't speak of more recent events) Oh, fifteen years ago. That explains why searching the tz-announce list didn't help - the archive doesn't go back past 2012. The actual tzdata files can be found going all the way back [1] but you'd have to dig through to find the announcement in question. In any case, I'm sure you'll find an update there; if not immediately, then after the event, because tzdata cares a LOT about historical times. For instance, this recent announcement [2] had one change to upcoming times, but a number of corrections to past times. So a manual override is ONLY necessary when (a) tzdata hasn't caught up yet, or (b) tzdata has been updated, but you're using an old version (maybe your OS hasn't caught up yet, and you can't get it from PyPI). ChrisA [1] https://data.iana.org/time-zones/releases/ [2] https://mm.icann.org/pipermail/tz-announce/2020-December/000063.html From rosuav at gmail.com Thu Sep 2 19:21:56 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 09:21:56 +1000 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber wrote: > > On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico > declaimed the following: > > > > >The naive summation algorithm used by sum() is compatible with a > >variety of different data types - even lists, although it's documented > >as being intended for numbers - but if you know for sure that you're > >working with floats, there's a more accurate algorithm available to > >you. > > > >>>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77]) > >39.6 > >>>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23]) > >39.6 > > > >It seeks to minimize loss to repeated rounding and is, I believe, > >independent of data order. > > > > Most likely it sorts the data so the smallest values get summed first, > and works its way up to the larger values. That way it minimizes the losses > that occur when denormalizing a value (to set the exponent equal to that of > the next larger value). > I'm not sure, but that sounds familiar. It doesn't really matter though - the docs just say that it is an "accurate floating point sum", so the precise algorithm is an implementation detail. ChrisA From 2QdxY4RzWzUUiLuE at potatochowder.com Thu Sep 2 19:46:19 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Thu, 2 Sep 2021 18:46:19 -0500 Subject: Scraping of Google Map to get the address of locations for given geocodes. In-Reply-To: References: Message-ID: On 2021-09-02 at 20:12:19 -0300, Michio Suginoo wrote: > I have the following question regarding how to scrape Google Map to get > address based on a given list of geocodes. > Given a list of geocodes (latitude; longitude) of locations, I would like > to scrape municipalities of all the spots in the list. How can I do that? > > For example, I have > > lat lon > > 1) -34.5722317 -58.4314464 > > 2) -34.553906 -58.4520949 > > 3) -34.5661444 -58.4964289 > > 4) -34.5648053 -58.4431567 > > 5) -34.5664089 -58.4323004 > > 6) -34.5664089 -58.4323004 > > > > And I want to get the name of municipality (city/town), or alternatively > the address, of each location. > > > I would appreciate it, if anyone can advise me how to do that. Is scraping Google Map a hard requirement? Both geopy? and geocoder? can translate those coordinates into street addresses directly, and their documentation even explains how to bypass the libraries and go straight to the service providers. ? https://geopy.readthedocs.io/en/latest/ ? https://geocoder.readthedocs.io/ From bvvkrishna at gmail.com Fri Sep 3 01:13:03 2021 From: bvvkrishna at gmail.com (Krishna) Date: Fri, 3 Sep 2021 10:43:03 +0530 Subject: Python Package documentation error In-Reply-To: References: Message-ID: Hi Python Team, I think the statement "The __init__.py files are required to make Python treat directories containing the file as packages" is wrong in the documentation[1] because it is valid only for Python 2.x version not Python 3.x version. Even though it is good practice to have this file, it is not mandatory to have in Python 3x. to treat directories as packages. So I think we should correct it. [1] https://docs.python.org/3/tutorial/modules.html?highlight=packages#packages Thanks, Krishna On Fri, Sep 3, 2021 at 10:13 AM Krishna wrote: > Hi Python Team, > I think the statement "The __init__.py files are required to make Python > treat directories containing the file as packages" is wrong in the > documentation[1] because it is valid only for Python 2.x version not Python > 3.x version. > > Even though it is good practice to have this file, it is not mandatory to > have in Python 3x. to treat directories as packages. So I think we should > correct it. > > [1] > https://docs.python.org/3/tutorial/modules.html?highlight=packages#packages > > Thanks, > Krishna > From nospam at dfs.com Thu Sep 2 21:38:45 2021 From: nospam at dfs.com (DFS) Date: Thu, 2 Sep 2021 21:38:45 -0400 Subject: Connecting python to DB2 database Message-ID: Having a problem with the DB2 connector test.py ---------------------------------------------------------------- import ibm_db_dbi connectstring = 'DATABASE=xxx;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;' conn = ibm_db_dbi.connect(connectstring,'','') curr = conn.cursor print(curr) cSQL = "SELECT * FROM TEST" curr.execute(cSQL) rows = curr.fetchall() print(len(rows)) ---------------------------------------------------------------- $python test.py Traceback (most recent call last): File "temp.py", line 9, in curr.execute(cSQL) AttributeError: 'function' object has no attribute 'execute' The ibm_db_dbi library supposedly adheres to PEP 249 (DB-API Spec 2.0), but it ain't happening here. Googling got me nowhere. Any ideas? python 3.8.2 on Windows 10 pip install ibm_db From rosuav at gmail.com Fri Sep 3 01:47:34 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 15:47:34 +1000 Subject: Connecting python to DB2 database In-Reply-To: References: Message-ID: On Fri, Sep 3, 2021 at 3:42 PM DFS wrote: > > Having a problem with the DB2 connector > > test.py > ---------------------------------------------------------------- > import ibm_db_dbi > connectstring = > 'DATABASE=xxx;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;' > conn = ibm_db_dbi.connect(connectstring,'','') > > curr = conn.cursor > print(curr) According to PEP 249, what you want is conn.cursor() not conn.cursor. I'm a bit surprised as to the repr of that function though, which seems to be this line from your output: I'd have expected it to say something like "method cursor of Connection object", which would have been an immediate clue as to what needs to be done. Not sure why the repr is so confusing, and that might be something to report upstream. ChrisA From roel at roelschroeven.net Fri Sep 3 03:45:04 2021 From: roel at roelschroeven.net (Roel Schroeven) Date: Fri, 3 Sep 2021 09:45:04 +0200 Subject: on floating-point numbers In-Reply-To: <86a6kvkobe.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: <7e90fecd-8db1-e0e1-fd3f-f1f52c450a7b@roelschroeven.net> Op 2/09/2021 om 17:08 schreef Hope Rouselle: > >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >>>> sum(ls) > > 39.599999999999994 > > > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >>>> sum(ls) > > 39.60000000000001 > > > > All I did was to take the first number, 7.23, and move it to the last > > position in the list. (So we have a violation of the commutativity of > > addition.) > > Suppose these numbers are prices in dollar, never going beyond cents. > Would it be safe to multiply each one of them by 100 and therefore work > with cents only? For working with monetary values, or any value that needs to accurate correspondence to 10-based values, best use Python's Decimal; see the documentation: https://docs.python.org/3.8/library/decimal.html Example: from decimal import Decimal as D ls1 = [D('7.23'), D('8.41'), D('6.15'), D('2.31'), D('7.73'), D('7.77')] ls2 = [D('8.41'), D('6.15'), D('2.31'), D('7.73'), D('7.77'), D('7.23')] print(sum(ls1), sum(ls2)) Output: 39.60 39.60 (Note that I initialized the values with strings instead of numbers, to allow Decimal access to the exact number without it already being converted to a float that doesn't necessarily exactly correspond to the decimal value) -- "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should" -- Dr. Ian Malcolm From bob.martin at excite.com Fri Sep 3 02:14:14 2021 From: bob.martin at excite.com (Bob Martin) Date: 3 Sep 2021 06:14:14 GMT Subject: The sqlite3 timestamp conversion between unixepoch and localtime In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <4d5d265c-59f0-8559-4b35-22695ad4a3e2@mrabarnett.plus.com> <30880c73-863a-ef94-e3a3-235aa589f070@mrabarnett.plus.com> Message-ID: On 2 Sep 2021 at 20:25:27, Alan Gauld wrote: > On 02/09/2021 20:11, MRAB wrote: > >>> In one of them (I can't recall which is which) they change on the 4th >>> weekend of October/March in the other they change on the last weekend. >>> >>> >> In the EU (and UK) it's the last Sunday in March/October. >> >> In the US it's second Sunday in March and the first Sunday in November. >> >> I know which one I find easier to remember! > > Interesting. I remember it as closer than that. The bugs we found were > due to differences in the DST settings of the BIOS in the PCs. (They > were deliberately all sourced from DELL but the EU PCs had a slightly > different BIOS). > > The differences you cite should have thrown up issues every year. > I must see if I can find my old log books... > ISTR that the USA changes were the same as the EU until a few years ago. I remember thinking at the time it changed "why would they do that?" From auriocus at gmx.de Fri Sep 3 04:11:25 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 3 Sep 2021 10:11:25 +0200 Subject: on floating-point numbers In-Reply-To: <846feb19-dbd4-4af9-a74d-f484fc310a0bn@googlegroups.com> References: <86bl5bm6go.fsf@jevedi.com> <846feb19-dbd4-4af9-a74d-f484fc310a0bn@googlegroups.com> Message-ID: Am 02.09.21 um 21:02 schrieb Julio Di Egidio: > On Thursday, 2 September 2021 at 20:43:36 UTC+2, Chris Angelico wrote: >> On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle wrote: > >>> All I did was to take the first number, 7.23, and move it to the last >>> position in the list. (So we have a violation of the commutativity of >>> addition.) >>> >> It's not about the commutativity of any particular pair of operands - >> that's always guaranteed. > > Nope, that is rather *not* guaranteed, as I have quite explained up thread. > No, you haven't explained that. You linked to the famous Goldberg paper. Where in the paper does it say that operations on floats are not commutative? I'd be surprised because it is generally wrong. Unless you have special numbers like NaN or signed zeros etc., a+b=b+a and a*b=b*a holds also for floats. Christiah From nospam at please.ty Fri Sep 3 05:18:15 2021 From: nospam at please.ty (jak) Date: Fri, 3 Sep 2021 11:18:15 +0200 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: Il 03/09/2021 09:07, Julio Di Egidio ha scritto: > On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote: >> On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber wrote: >>> On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico >>> declaimed the following: >>> >>>> The naive summation algorithm used by sum() is compatible with a >>>> variety of different data types - even lists, although it's documented >>>> as being intended for numbers - but if you know for sure that you're >>>> working with floats, there's a more accurate algorithm available to >>>> you. >>>> >>>>>>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77]) >>>> 39.6 >>>>>>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23]) >>>> 39.6 >>>> >>>> It seeks to minimize loss to repeated rounding and is, I believe, >>>> independent of data order. >>> >>> Most likely it sorts the data so the smallest values get summed first, >>> and works its way up to the larger values. That way it minimizes the losses >>> that occur when denormalizing a value (to set the exponent equal to that of >>> the next larger value). >>> >> I'm not sure, but that sounds familiar. It doesn't really matter >> though - the docs just say that it is an "accurate floating point >> sum", so the precise algorithm is an implementation detail. > > The docs are quite misleading there, it is not accurate without further qualifications. > > > > > That said, fucking pathetic, when Dunning-Kruger is a compliment... > > *Plonk* > > Julio > https://en.wikipedia.org/wiki/IEEE_754 From rosuav at gmail.com Fri Sep 3 08:45:57 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 22:45:57 +1000 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Fri, Sep 3, 2021 at 10:42 PM jak wrote: > > Il 03/09/2021 09:07, Julio Di Egidio ha scritto: > > On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote: > >> On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber wrote: > >>> On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico > >>> declaimed the following: > >>> > >>>> The naive summation algorithm used by sum() is compatible with a > >>>> variety of different data types - even lists, although it's documented > >>>> as being intended for numbers - but if you know for sure that you're > >>>> working with floats, there's a more accurate algorithm available to > >>>> you. > >>>> > >>>>>>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77]) > >>>> 39.6 > >>>>>>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23]) > >>>> 39.6 > >>>> > >>>> It seeks to minimize loss to repeated rounding and is, I believe, > >>>> independent of data order. > >>> > >>> Most likely it sorts the data so the smallest values get summed first, > >>> and works its way up to the larger values. That way it minimizes the losses > >>> that occur when denormalizing a value (to set the exponent equal to that of > >>> the next larger value). > >>> > >> I'm not sure, but that sounds familiar. It doesn't really matter > >> though - the docs just say that it is an "accurate floating point > >> sum", so the precise algorithm is an implementation detail. > > > > The docs are quite misleading there, it is not accurate without further qualifications. > > > > > > > > > > https://en.wikipedia.org/wiki/IEEE_754 I believe the definition of "accurate" here is that, if you take all of the real numbers represented by those floats, add them all together with mathematical accuracy, and then take the nearest representable float, that will be the exact value that fsum will return. In other words, its accuracy is exactly as good as the final result can be. ChrisA From oscar.j.benjamin at gmail.com Fri Sep 3 09:15:18 2021 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 3 Sep 2021 14:15:18 +0100 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> Message-ID: On Fri, 3 Sept 2021 at 13:48, Chris Angelico wrote: > > On Fri, Sep 3, 2021 at 10:42 PM jak wrote: > > > > Il 03/09/2021 09:07, Julio Di Egidio ha scritto: > > > On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote: > > >> On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber wrote: > > >>> On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico > > >>> declaimed the following: > > >>> > > >>>> The naive summation algorithm used by sum() is compatible with a > > >>>> variety of different data types - even lists, although it's documented > > >>>> as being intended for numbers - but if you know for sure that you're > > >>>> working with floats, there's a more accurate algorithm available to > > >>>> you. > > >>>> > > >>>>>>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77]) > > >>>> 39.6 > > >>>>>>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23]) > > >>>> 39.6 > > >>>> > > >>>> It seeks to minimize loss to repeated rounding and is, I believe, > > >>>> independent of data order. > > >>> > > >>> Most likely it sorts the data so the smallest values get summed first, > > >>> and works its way up to the larger values. That way it minimizes the losses > > >>> that occur when denormalizing a value (to set the exponent equal to that of > > >>> the next larger value). > > >>> > > >> I'm not sure, but that sounds familiar. It doesn't really matter > > >> though - the docs just say that it is an "accurate floating point > > >> sum", so the precise algorithm is an implementation detail. > > > > > > The docs are quite misleading there, it is not accurate without further qualifications. > > > > > > > > > > > > > > > > https://en.wikipedia.org/wiki/IEEE_754 > > I believe the definition of "accurate" here is that, if you take all > of the real numbers represented by those floats, add them all together > with mathematical accuracy, and then take the nearest representable > float, that will be the exact value that fsum will return. In other > words, its accuracy is exactly as good as the final result can be. It's as good as it can be if the result must fit into a single float. Actually the algorithm itself maintains an exact result for the sum internally using a list of floats whose exact sum is the same as that of the input list. In essence it compresses a large list of floats to a small list of say 2 or 3 floats while preserving the exact value of the sum. Unfortunately fsum does not give any way to access the internal exact list so using fsum repeatedly suffers the same problems as plain float arithmetic e.g.: >>> x = 10**20 >>> fsum([fsum([1, x]), -x]) 0.0 -- Oscar From nospam at dfs.com Fri Sep 3 09:29:20 2021 From: nospam at dfs.com (DFS) Date: Fri, 3 Sep 2021 09:29:20 -0400 Subject: Connecting python to DB2 database In-Reply-To: References: Message-ID: On 9/3/2021 1:47 AM, Chris Angelico wrote: > On Fri, Sep 3, 2021 at 3:42 PM DFS wrote: >> >> Having a problem with the DB2 connector >> >> test.py >> ---------------------------------------------------------------- >> import ibm_db_dbi >> connectstring = >> 'DATABASE=xxx;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;' >> conn = ibm_db_dbi.connect(connectstring,'','') >> >> curr = conn.cursor >> print(curr) > > According to PEP 249, what you want is conn.cursor() not conn.cursor. > > I'm a bit surprised as to the repr of that function though, which > seems to be this line from your output: > > > > I'd have expected it to say something like "method cursor of > Connection object", which would have been an immediate clue as to what > needs to be done. Not sure why the repr is so confusing, and that > might be something to report upstream. > > ChrisA Thanks. I must've done it right, using conn.cursor(), 500x. Bleary-eyed from staring at code too long I guess. Now can you get DB2 to accept ; as a SQL statement terminator like the rest of the world? They call it "An unexpected token"... From rosuav at gmail.com Fri Sep 3 09:50:25 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 3 Sep 2021 23:50:25 +1000 Subject: Connecting python to DB2 database In-Reply-To: References: Message-ID: On Fri, Sep 3, 2021 at 11:37 PM DFS wrote: > > On 9/3/2021 1:47 AM, Chris Angelico wrote: > > On Fri, Sep 3, 2021 at 3:42 PM DFS wrote: > >> > >> Having a problem with the DB2 connector > >> > >> test.py > >> ---------------------------------------------------------------- > >> import ibm_db_dbi > >> connectstring = > >> 'DATABASE=xxx;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;' > >> conn = ibm_db_dbi.connect(connectstring,'','') > >> > >> curr = conn.cursor > >> print(curr) > > > > According to PEP 249, what you want is conn.cursor() not conn.cursor. > > > > I'm a bit surprised as to the repr of that function though, which > > seems to be this line from your output: > > > > > > > > I'd have expected it to say something like "method cursor of > > Connection object", which would have been an immediate clue as to what > > needs to be done. Not sure why the repr is so confusing, and that > > might be something to report upstream. > > > > ChrisA > > > Thanks. I must've done it right, using conn.cursor(), 500x. > Bleary-eyed from staring at code too long I guess. Cool cool! Glad that's working. > Now can you get DB2 to accept ; as a SQL statement terminator like the > rest of the world? They call it "An unexpected token"... > Hmm, I don't know that the execute() method guarantees to allow semicolons. Some implementations will strip a trailing semi, but they usually won't allow interior ones, because that's a good way to worsen SQL injection vulnerabilities. It's entirely possible - and within the PEP 249 spec, I believe - for semicolons to be simply rejected. ChrisA From o1bigtenor at gmail.com Fri Sep 3 10:08:21 2021 From: o1bigtenor at gmail.com (o1bigtenor) Date: Fri, 3 Sep 2021 09:08:21 -0500 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: On Thu, Sep 2, 2021 at 2:27 PM Chris Angelico wrote: > On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle wrote: > > > > Hope Rouselle writes: > > > > > Just sharing a case of floating-point numbers. Nothing needed to be > > > solved or to be figured out. Just bringing up conversation. > > > > > > (*) An introduction to me > > > > > > I don't understand floating-point numbers from the inside out, but I do > > > know how to work with base 2 and scientific notation. So the idea of > > > expressing a number as > > > > > > mantissa * base^{power} > > > > > > is not foreign to me. (If that helps you to perhaps instruct me on > > > what's going on here.) > > > > > > (*) A presentation of the behavior > > > > > >>>> import sys > > >>>> sys.version > > > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 > > > bit (AMD64)]' > > > > > >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > > >>>> sum(ls) > > > 39.599999999999994 > > > > > >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > > >>>> sum(ls) > > > 39.60000000000001 > > > > > > All I did was to take the first number, 7.23, and move it to the last > > > position in the list. (So we have a violation of the commutativity of > > > addition.) > > > > Suppose these numbers are prices in dollar, never going beyond cents. > > Would it be safe to multiply each one of them by 100 and therefore work > > with cents only? For instance > > Yes and no. It absolutely *is* safe to always work with cents, but to > do that, you have to be consistent: ALWAYS work with cents, never with > floating point dollars. > > (Or whatever other unit you choose to use. Most currencies have a > smallest-normally-used-unit, with other currency units (where present) > being whole number multiples of that minimal unit. Only in forex do > you need to concern yourself with fractional cents or fractional yen.) > > But multiplying a set of floats by 100 won't necessarily solve your > problem; you may have already fallen victim to the flaw of assuming > that the numbers are represented accurately. > > > --8<---------------cut here---------------start------------->8--- > > >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > > >>> sum(map(lambda x: int(x*100), ls)) / 100 > > 39.6 > > > > >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > > >>> sum(map(lambda x: int(x*100), ls)) / 100 > > 39.6 > > --8<---------------cut here---------------end--------------->8--- > > > > Or multiplication by 100 isn't quite ``safe'' to do with floating-point > > numbers either? (It worked in this case.) > > You're multiplying and then truncating, which risks a round-down > error. Try adding a half onto them first: > > int(x * 100 + 0.5) > > But that's still not a perfect guarantee. Far safer would be to > consider monetary values to be a different type of value, not just a > raw number. For instance, the value $7.23 could be stored internally > as the integer 723, but you also know that it's a value in USD, not a > simple scalar. It makes perfect sense to add USD+USD, it makes perfect > sense to multiply USD*scalar, but it doesn't make sense to multiply > USD*USD. > > > I suppose that if I multiply it by a power of two, that would be an > > operation that I can be sure will not bring about any precision loss > > with floating-point numbers. Do you agree? > > Assuming you're nowhere near 2**53, yes, that would be safe. But so > would multiplying by a power of five. The problem isn't precision loss > from the multiplication - the problem is that your input numbers > aren't what you think they are. That number 7.23, for instance, is > really.... > > >>> 7.23.as_integer_ratio() > (2035064081618043, 281474976710656) > > ... the rational number 2035064081618043 / 281474976710656, which is > very close to 7.23, but not exactly so. (The numerator would have to > be ...8042.88 to be exactly correct.) There is nothing you can do at > this point to regain the precision, although a bit of multiplication > and rounding can cheat it and make it appear as if you did. > > Floating point is a very useful approximation to real numbers, but > real numbers aren't the best way to represent financial data. Integers > are. > > Hmmmmmmm - - - ZI would suggest that you haven't looked into taxation yet! In taxation you get a rational number that MUST be multiplied by the amount in currency. The error rate here is stupendous. Some organizations track each transaction with its taxes rounded. Then some track using use untaxed and then calculate the taxes on the whole (when you have 2 or 3 or 4 (dunno about more but who knows there are some seriously tax loving jurisdictions out there)) the differences between adding amounts and then calculating taxes and calculating taxes on each amount and then adding all items together can have some 'interesting' differences. So financial data MUST be able to handle rational numbers. (I have been bit by the differences enumerated in the previous!) Regards From rosuav at gmail.com Fri Sep 3 11:13:23 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 4 Sep 2021 01:13:23 +1000 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: On Sat, Sep 4, 2021 at 12:08 AM o1bigtenor wrote: > Hmmmmmmm - - - ZI would suggest that you haven't looked into > taxation yet! > In taxation you get a rational number that MUST be multiplied by > the amount in currency. (You can, of course, multiply a currency amount by any scalar. Just not by another currency amount.) > The error rate here is stupendous. > Some organizations track each transaction with its taxes rounded. > Then some track using use untaxed and then calculate the taxes > on the whole (when you have 2 or 3 or 4 (dunno about more but > who knows there are some seriously tax loving jurisdictions out there)) > the differences between adding amounts and then calculating taxes > and calculating taxes on each amount and then adding all items > together can have some 'interesting' differences. > > So financial data MUST be able to handle rational numbers. > (I have been bit by the differences enumerated in the previous!) The worst problem is knowing WHEN to round. Sometimes you have to do intermediate rounding in order to make something agree with something else :( But if you need finer resolution than the cent, I would still recommend trying to use fixed-point arithmetic. The trouble is figuring out exactly how much precision you need. Often, 1c precision is actually sufficient. ChrisA From python at mrabarnett.plus.com Fri Sep 3 11:39:07 2021 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 3 Sep 2021 16:39:07 +0100 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: <2d9c1be7-f19d-3c9d-b779-6b1c1b6e13ab@mrabarnett.plus.com> On 2021-09-03 16:13, Chris Angelico wrote: > On Sat, Sep 4, 2021 at 12:08 AM o1bigtenor wrote: >> Hmmmmmmm - - - ZI would suggest that you haven't looked into >> taxation yet! >> In taxation you get a rational number that MUST be multiplied by >> the amount in currency. > > (You can, of course, multiply a currency amount by any scalar. Just > not by another currency amount.) > >> The error rate here is stupendous. >> Some organizations track each transaction with its taxes rounded. >> Then some track using use untaxed and then calculate the taxes >> on the whole (when you have 2 or 3 or 4 (dunno about more but >> who knows there are some seriously tax loving jurisdictions out there)) >> the differences between adding amounts and then calculating taxes >> and calculating taxes on each amount and then adding all items >> together can have some 'interesting' differences. >> >> So financial data MUST be able to handle rational numbers. >> (I have been bit by the differences enumerated in the previous!) > > The worst problem is knowing WHEN to round. Sometimes you have to do > intermediate rounding in order to make something agree with something > else :( > > But if you need finer resolution than the cent, I would still > recommend trying to use fixed-point arithmetic. The trouble is > figuring out exactly how much precision you need. Often, 1c precision > is actually sufficient. > At some point, some finance/legal person has to specify how any fractional currency should be handled. From Joseph.Schachner at Teledyne.com Fri Sep 3 11:55:05 2021 From: Joseph.Schachner at Teledyne.com (Schachner, Joseph) Date: Fri, 3 Sep 2021 15:55:05 +0000 Subject: on floating-point numbers In-Reply-To: <86bl5bm6go.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> Message-ID: What's really going on is that you are printing out more digits than you are entitled to. 39.60000000000001 : 16 decimal digits. 4e16 should require 55 binary bits (in the mantissa) to represent, at least as I calculate it. Double precision floating point has 52 bits in the mantissa, plus one assumed due to normalization. So 53 bits. The actual minor difference in sums that you see is because when you put the largest value 1st it makes a difference in the last few bits of the mantissa. I recommend that you print out double precision values to at most 14 digits. Then you will never see this kind of issue. If you don't like that suggestion, you can create your own floating point representation using a Python integer as the mantissa, so it can grow as large as you have memory to represent the value; and a sign and an exponent. It would be slow, but it could have much more accuracy (if implemented to preserve accuracy). By the way, this is why banks and other financial institutions use BCD (binary coded decimal). They cannot tolerate sums that have fraction of a cent errors. I should also point out another float issue: subtractive cancellation. Try 1e14 + 0.1 - 1e14. The result clearly should be 0.1, but it won't be. That's because 0.1 cannot be accurately represented in binary, and it was only represented in the bottom few bits. I just tried it: I got 0.09375 This is not a Python issue. This is a well known issue when using binary floating point. So, when you sum a large array of data, to avoid these issues, you could either 1) sort the data smallest to largest ... may be helpful, but maybe not. 2) Create multiple sums of a few of the values. Next layer: Sum a few of the sums. Top layer: Sum the sum of sums to get the final sum. This is much more likely to work accurately than adding up all the values in one summation except the last, and then adding the last (which could be a relatively small value). --- Joseph S. Teledyne Confidential; Commercially Sensitive Business Data -----Original Message----- From: Hope Rouselle Sent: Thursday, September 2, 2021 9:51 AM To: python-list at python.org Subject: on floating-point numbers Just sharing a case of floating-point numbers. Nothing needed to be solved or to be figured out. Just bringing up conversation. (*) An introduction to me I don't understand floating-point numbers from the inside out, but I do know how to work with base 2 and scientific notation. So the idea of expressing a number as mantissa * base^{power} is not foreign to me. (If that helps you to perhaps instruct me on what's going on here.) (*) A presentation of the behavior >>> import sys >>> sys.version '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]' >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>> sum(ls) 39.599999999999994 >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>> sum(ls) 39.60000000000001 All I did was to take the first number, 7.23, and move it to the last position in the list. (So we have a violation of the commutativity of addition.) Let me try to reduce the example. It's not so easy. Although I could display the violation of commutativity by moving just a single number in the list, I also see that 7.23 commutes with every other number in the list. (*) My request I would like to just get some clarity. I guess I need to translate all these numbers into base 2 and perform the addition myself to see the situation coming up? From Joseph.Schachner at Teledyne.com Fri Sep 3 12:05:00 2021 From: Joseph.Schachner at Teledyne.com (Schachner, Joseph) Date: Fri, 3 Sep 2021 16:05:00 +0000 Subject: on floating-point numbers In-Reply-To: <86bl5bm6go.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> Message-ID: Actually, Python has an fsum function meant to address this issue. >>> math.fsum([1e14, 1, -1e14]) 1.0 >>> Wow it works. --- Joseph S. Teledyne Confidential; Commercially Sensitive Business Data -----Original Message----- From: Hope Rouselle Sent: Thursday, September 2, 2021 9:51 AM To: python-list at python.org Subject: on floating-point numbers Just sharing a case of floating-point numbers. Nothing needed to be solved or to be figured out. Just bringing up conversation. (*) An introduction to me I don't understand floating-point numbers from the inside out, but I do know how to work with base 2 and scientific notation. So the idea of expressing a number as mantissa * base^{power} is not foreign to me. (If that helps you to perhaps instruct me on what's going on here.) (*) A presentation of the behavior >>> import sys >>> sys.version '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]' >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>> sum(ls) 39.599999999999994 >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>> sum(ls) 39.60000000000001 All I did was to take the first number, 7.23, and move it to the last position in the list. (So we have a violation of the commutativity of addition.) Let me try to reduce the example. It's not so easy. Although I could display the violation of commutativity by moving just a single number in the list, I also see that 7.23 commutes with every other number in the list. (*) My request I would like to just get some clarity. I guess I need to translate all these numbers into base 2 and perform the addition myself to see the situation coming up? From martinp.dipaola at gmail.com Fri Sep 3 12:56:44 2021 From: martinp.dipaola at gmail.com (Martin Di Paola) Date: Fri, 3 Sep 2021 16:56:44 +0000 Subject: Select columns based on dates - Pandas In-Reply-To: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com> References: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com> Message-ID: <20210903165644.kpgyvpits5hjlmbc@gmail.com> You may want to reshape the dataset to a tidy format: Pandas works better with that format. Let's assume the following dataset (this is what I understood from your message): In [34]: df = pd.DataFrame({ ...: 'Country': ['us', 'uk', 'it'], ...: '01/01/2019': [10, 20, 30], ...: '02/01/2019': [12, 22, 32], ...: '03/01/2019': [14, 24, 34], ...: }) In [35]: df Out[35]: Country 01/01/2019 02/01/2019 03/01/2019 0 us 10 12 14 1 uk 20 22 24 2 it 30 32 34 Then, reshape it to a tidy format. Notice how each row now represents a single measure. In [43]: pd.melt(df, id_vars=['Country'], var_name='Date', value_name='Cases') Out[43]: Country Date Cases 0 us 01/01/2019 10 1 uk 01/01/2019 20 2 it 01/01/2019 30 3 us 02/01/2019 12 4 uk 02/01/2019 22 5 it 02/01/2019 32 6 us 03/01/2019 14 7 uk 03/01/2019 24 8 it 03/01/2019 34 I used strings to represent the dates but it is much handy work with real date objects. In [44]: df2 = _ In [45]: df2['Date'] = pd.to_datetime(df2['Date']) Now we can filter by date: In [50]: df2[df2['Date'] < '2019-03-01'] Out[50]: Country Date Cases 0 us 2019-01-01 10 1 uk 2019-01-01 20 2 it 2019-01-01 30 3 us 2019-02-01 12 4 uk 2019-02-01 22 5 it 2019-02-01 32 With that you could create three dataframes, one per month. Thanks, Martin. On Thu, Sep 02, 2021 at 12:28:31PM -0700, Richard Medina wrote: >Hello, forum, >I have a data frame with covid-19 cases per month from 2019 - 2021 like a header like this: > >Country, 01/01/2019, 2/01/2019, 01/02/2019, 3/01/2019, ... 01/01/2021, 2/01/2021, 01/02/2021, 3/01/2021 > >I want to filter my data frame for columns of a specific month range of march to September of 2019, 2020, and 2021 only (three data frames). > >Any ideas? >Thank you > > >-- >https://mail.python.org/mailman/listinfo/python-list From michael.stemper at gmail.com Fri Sep 3 10:55:51 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Fri, 3 Sep 2021 09:55:51 -0500 Subject: The sqlite3 timestamp conversion between unixepoch and localtime In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> <4d5d265c-59f0-8559-4b35-22695ad4a3e2@mrabarnett.plus.com> <30880c73-863a-ef94-e3a3-235aa589f070@mrabarnett.plus.com> Message-ID: On 03/09/2021 01.14, Bob Martin wrote: > On 2 Sep 2021 at 20:25:27, Alan Gauld wrote: >> On 02/09/2021 20:11, MRAB wrote: >> >>>> In one of them (I can't recall which is which) they change on the 4th >>>> weekend of October/March in the other they change on the last weekend. >>>> >>>> >>> In the EU (and UK) it's the last Sunday in March/October. >>> >>> In the US it's second Sunday in March and the first Sunday in November. >>> >>> I know which one I find easier to remember! >> >> Interesting. I remember it as closer than that. The bugs we found were >> due to differences in the DST settings of the BIOS in the PCs. (They >> were deliberately all sourced from DELL but the EU PCs had a slightly >> different BIOS). >> >> The differences you cite should have thrown up issues every year. >> I must see if I can find my old log books... >> > > ISTR that the USA changes were the same as the EU until a few years ago. > > I remember thinking at the time it changed "why would they do that?" It was part of the Energy Policy Act of 2005[1]. However, saying that doesn't explain "why". The explanation given at the time was that it would save energy because people wouldn't need to turn on their lights as early. This ignored the fact that we needed to have them on later in the morning. The required studies were inconclusive, but some reported that since it was light later in the day, people went driving around in the evening, causing aggregate energy consumption to increase rather than decrease. One of the bill's sponsors said that having it be light later in the day would "make people happy". Talk at the time (which I never verified or refuted) said that he got significant campaign contributions from a trade group for outdoor cooking (grills, charcoal, usw) and that they wanted it so that the grilling season would be longer, leading to more revenue for them. At the time, I was product manager for real-time control systems for critical infrastructure. Having to collect the changes to zoneinfo, whatever the equivalent file for Oracle was, revalidate our entire system, and get information/patches to our North American customers was annoying. [1] , Sec 110 -- Michael F. Stemper If you take cranberries and stew them like applesauce they taste much more like prunes than rhubarb does. From alan.gauld at yahoo.co.uk Fri Sep 3 11:33:08 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 3 Sep 2021 16:33:08 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 02/09/2021 19:30, Chris Angelico wrote: >> Without DST the schools opened in the dark so all the kids >> had to travel to school in the dark and the number of >> traffic accidents while crossing roads jumped. > > How do they manage in winter? That was the winter. Sunrise wasn't till 10:00 or so and the schools open at 9. With DST sunrise became 9:00 and with pre-dawn light it is enough to see by. Its a recurring theme. Every now and then some smart young politician from the south of the UK suggests doing away with DST and a large crowd of northerners jump up and say no way! > Can that be solved with better street> lighting? They had street lighting but it casts dark shadows etc. In fact modern LED based street lighting is worse in that respect that the old yellow sodium lights were. But where it doesn't exist the cost of installing street lighting in small villages is too high compared to just changing the clocks. And of course street lighting has a real running cost that would be reflected in the local council taxes, and nobody wants to pay more of them! After all street lighting has been available for over 150 years, if they haven't installed it by now.... (I mean, nearly everywhere has some lighting, at least on the main roads, it's just the smaller back streets that tend to be dark.) > That was fifty years ago now, and the negative consequences > of DST are far stronger now. But not apparent to most people. Most still see it as a benefit because they get longer working daylight. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From pkpearson at nowhere.invalid Fri Sep 3 11:39:25 2021 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 3 Sep 2021 15:39:25 GMT Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <4105d35c-cb9a-42d1-9985-f3c5b6ff6209n@googlegroups.com> Message-ID: On Thu, 2 Sep 2021 07:54:27 -0700 (PDT), Julio Di Egidio wrote: > On Thursday, 2 September 2021 at 16:51:24 UTC+2, Christian Gollwitzer wrote: >> Am 02.09.21 um 16:49 schrieb Julio Di Egidio: >> > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote: >> >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote: >> > >> >>> 39.60000000000001 >> >> >> >> Welcome to the exciting world of roundoff error: >> > >> > Welcome to the exiting world of Usenet. >> > >> > *Plonk* >> >> Pretty harsh, isn't it? He gave a concise example of the same inaccuracy >> right afterwards. > > And I thought you were not seeing my posts... > > Given that I have already given a full explanation, you guys, that you > realise it or not, are simply adding noise for the usual pub-level > discussion I must most charitably guess. > > Anyway, just my opinion. (EOD.) Although we are in the world of Usenet, comp.lang.python is by no means typical of Usenet. This is a positive, helpful, welcoming community in which "Plonk", "EOD", and "RTFM" (appearing in another post) are seldom seen, and in which I have never before seen the suggestion that everybody else should be silent so that the silver voice of the chosen one can be heard. -- To email me, substitute nowhere->runbox, invalid->com. From wlfraed at ix.netcom.com Fri Sep 3 12:20:35 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Fri, 03 Sep 2021 12:20:35 -0400 Subject: Connecting python to DB2 database References: Message-ID: <5bi4jgdh9lcv51fmeila9j2aa14t5odc9h@4ax.com> On Fri, 3 Sep 2021 09:29:20 -0400, DFS declaimed the following: > >Now can you get DB2 to accept ; as a SQL statement terminator like the >rest of the world? They call it "An unexpected token"... I've never seen a semi-colon used for SQL statements via a db-api adapter. The semi-colon is, in my experience, only required by basic interactive query utilities -- to tell the utility that the statement is fully entered, and can be sent to the server. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From rosuav at gmail.com Fri Sep 3 13:37:04 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 4 Sep 2021 03:37:04 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Sat, Sep 4, 2021 at 3:33 AM Alan Gauld via Python-list wrote: > > On 02/09/2021 19:30, Chris Angelico wrote: > > >> Without DST the schools opened in the dark so all the kids > >> had to travel to school in the dark and the number of > >> traffic accidents while crossing roads jumped. > > > > How do they manage in winter? > > That was the winter. Sunrise wasn't till 10:00 or so > and the schools open at 9. With DST sunrise became > 9:00 and with pre-dawn light it is enough to see by. Are you saying that you had DST in winter, or that, when summer *and* DST came into effect, there was more light at dawn? Because a *lot* of people confuse summer and DST, and credit DST with the natural effects of the season change. ChrisA From barry at barrys-emacs.org Fri Sep 3 13:49:44 2021 From: barry at barrys-emacs.org (Barry) Date: Fri, 3 Sep 2021 18:49:44 +0100 Subject: Add a method to list the current named logging levels In-Reply-To: <24881.20924.937699.552812@ixdm.fritz.box> References: <24881.20924.937699.552812@ixdm.fritz.box> Message-ID: > On 2 Sep 2021, at 23:38, Dieter Maurer wrote: > > ?Edward Spencer wrote at 2021-9-2 10:02 -0700: >> Sometimes I like to pass the logging level up to the command line params so my user can specific what level of logging they want. However there is no easy method for pulling the named logging level names. >> >> Looking into the code, it would actually be incredibly easy to implement; >> >> in `logging.__init__.py`; >> >> def listLevelNames(): >> return _nameToLevel.keys() >> >> You could obviously add some other features, like listing only the defaults, sorted by numerical level or alphabetically, etc. But really this basic implementation would be enough to expose the internal variables which shouldn't be accessed because they change (and in fact, between python 2 and 3, they did). >> >> Any thoughts? > > Usually, you use 5 well known log levels: "DEBUG", "INFO", "WARNING", > "ERROR" and "CRITICAL". > No need to provide a special function listing those levels. I add my own levels, but then I know I did it. Barry > > > > -- > Dieter > -- > https://mail.python.org/mailman/listinfo/python-list > From barry at barrys-emacs.org Fri Sep 3 17:53:34 2021 From: barry at barrys-emacs.org (Barry) Date: Fri, 3 Sep 2021 22:53:34 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime In-Reply-To: References: Message-ID: <6E13D2EA-A8B5-4689-AF27-20C8FDBFB257@barrys-emacs.org> > On 3 Sep 2021, at 13:40, Bob Martin wrote: > > ?On 2 Sep 2021 at 20:25:27, Alan Gauld wrote: >> On 02/09/2021 20:11, MRAB wrote: >> >>>> In one of them (I can't recall which is which) they change on the 4th >>>> weekend of October/March in the other they change on the last weekend. >>>> >>>> >>> In the EU (and UK) it's the last Sunday in March/October. >>> >>> In the US it's second Sunday in March and the first Sunday in November. >>> >>> I know which one I find easier to remember! >> >> Interesting. I remember it as closer than that. The bugs we found were >> due to differences in the DST settings of the BIOS in the PCs. (They >> were deliberately all sourced from DELL but the EU PCs had a slightly >> different BIOS). >> >> The differences you cite should have thrown up issues every year. >> I must see if I can find my old log books... >> > > ISTR that the USA changes were the same as the EU until a few years ago. I recall that DST changes have been at least 1 week different between the UK and USA since the 80?s. Barry > > I remember thinking at the time it changed "why would they do that?" > > -- > https://mail.python.org/mailman/listinfo/python-list > From hjp-python at hjp.at Sat Sep 4 06:05:32 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 12:05:32 +0200 Subject: on writing a while loop for rolling two dice In-Reply-To: <864kb3m4qi.fsf@jevedi.com> References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> Message-ID: On 2021-09-02 11:28:21 -0300, Hope Rouselle wrote: > dn writes: > > On 29/08/2021 08.46, Hope Rouselle wrote: > >> Here's my solution: > >> > >> --8<---------------cut here---------------start------------->8--- > >> def how_many_times(): > >> x, y = 0, 1 > >> c = 0 > >> while x != y: > >> c = c + 1 > >> x, y = roll() > >> return c, (x, y) > > > >> > >> Why am I unhappy? I'm wish I could confine x, y to the while loop. > >> The introduction of ``x, y = 0, 1'' must feel like a trick to a > >> novice. How would you write this? [...] > But perhaps we may agree that while rolling dice until a certain > success, we want to roll them while something happens or doesn't happen. > One of the two. So while-True is a bit of a jump. Therefore, in this > case, the easier and more natural option is to say while-x-not-equal-y. > > But this approach seems to force me into initializing x, y with > different values. You can get around this by using NaN: def how_many_times(): c, x, y = 0, math.nan, math.nan while x != y: c = c + 1 x, y = roll() return c, x, y Not sure if this is an improvement. Now you have to explain to your students why math.nan != math.nan. When I need a guaranteed unique value I often just use object(): def how_many_times(): c, x, y = 0, object(), object() while x != y: c = c + 1 x, y = roll() return c, x, y Of course now you are back to two different values, but they look the same. Which may be better or worse for your students. Plus x and y are now bound to objects of different types during their lifetime, which may be a bit dicey. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 4 06:21:14 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 12:21:14 +0200 Subject: on the popularity of loops while and for In-Reply-To: References: Message-ID: On 2021-08-29 10:04:47 +0100, Barry wrote: > > I'd like get a statistic of how often each loop is used in practice. > > > > I was trying to take a look at the Python's standard libraries --- those > > included in a standard installation of Python 3.9.6, say --- to see > > which loops are more often used among while and for loops. Of course, > > since English use the preposition ``for'' a lot, that makes my life > > harder. Removing comments is easy, but removing strings is harder. So > > I don't know yet what I'll do. > > > > Have you guys ever measured something like that in a casual or serious > > way? I'd love to know. Thank you! > > I am interesting in why you think that choice of while vs. for is > about popularity? > > Surely the choice is made in most cases by the algorithm? The context here is an introductory course for Python. So there is not "the algorithm", there are all the algorithms that a novice is likely to encounter. For me it makes absolute sense to base the contents of the course on popularity. Constructs which a novice programmer is very likely to use or encounter in other people's code should be covered more thoroughly than constructs that will be used only rarely. Some are so rare that they can be safely omitted. The while loop is certainly not in that category, but it probably makes sense to spend less time on it than on the for loop. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 4 06:24:32 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 12:24:32 +0200 Subject: urgent In-Reply-To: References: Message-ID: On 2021-09-02 09:56:38 -0500, Michael F. Stemper wrote: > On 31/08/2021 18.02, Barry wrote: > > The big problem with >>> is that it means a third level quote in > > email clients. So when people cut-n-paste REPL output it?s formatted > > badly by some > > email clients. A prompt that avoided that issue would be nice. > > A little bit of piping fixes that: [...] > username at hostname$ sed 's/^>>> /REPL> /' < text > username at hostname$ python3 > Python 3.5.2 (default, Jan 26 2021, 13:30:48) > [GCC 5.4.0 20160609] on linux > Type "help", "copyright", "credits" or "license" for more information. > REPL> import sys # in the original, this line will be messed up > REPL> sys.exit(0) # this one, too > username at hostname$ Yes, but then it doesn't look like the Python prompt anymore which may confuse the reader even more. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From arj.python at gmail.com Sat Sep 4 07:54:51 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sat, 4 Sep 2021 15:54:51 +0400 Subject: urgent In-Reply-To: References: Message-ID: You need an IDE Check out: PyCharm Wing IDE Spyder ^^ Very few people use the in-built IDE From ikorot01 at gmail.com Sat Sep 4 14:27:32 2021 From: ikorot01 at gmail.com (Igor Korot) Date: Sat, 4 Sep 2021 13:27:32 -0500 Subject: Problem with python Message-ID: Hi, ALL, [code] igor at WaylandGnome ~/bakefile $ python Python 3.9.6 (default, Aug 8 2021, 17:26:32) [GCC 10.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from distutils import sysconfig >>> print sysconfig.get_python_inc() File "", line 1 print sysconfig.get_python_inc() ^ SyntaxError: invalid syntax >>> [/code] What is the proper way to fix this? Thank you. From joel.goldstick at gmail.com Sat Sep 4 14:39:27 2021 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 4 Sep 2021 14:39:27 -0400 Subject: Problem with python In-Reply-To: References: Message-ID: On Sat, Sep 4, 2021 at 2:29 PM Igor Korot wrote: > > Hi, ALL, > > [code] > igor at WaylandGnome ~/bakefile $ python > Python 3.9.6 (default, Aug 8 2021, 17:26:32) > [GCC 10.3.0] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from distutils import sysconfig > >>> print sysconfig.get_python_inc() > File "", line 1 > print sysconfig.get_python_inc() print( sysconfig.get_python_inc()) Since python3 print is a function. > ^ > SyntaxError: invalid syntax > >>> > [/code] > > What is the proper way to fix this? > > Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick From mats at wichmann.us Sat Sep 4 14:38:20 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sat, 4 Sep 2021 12:38:20 -0600 Subject: Problem with python In-Reply-To: References: Message-ID: <33d0a8dd-8100-fc9b-1623-3cc0597f6bdf@wichmann.us> On 9/4/21 12:27 PM, Igor Korot wrote: > Hi, ALL, > > [code] > igor at WaylandGnome ~/bakefile $ python > Python 3.9.6 (default, Aug 8 2021, 17:26:32) > [GCC 10.3.0] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> from distutils import sysconfig >>>> print sysconfig.get_python_inc() > File "", line 1 > print sysconfig.get_python_inc() > ^ > SyntaxError: invalid syntax >>>> > [/code] > > What is the proper way to fix this? print is a function in Python 3. print(sysconfig.get_python_inc()) Try: >>> help(print) for a quick check. From PythonList at DancesWithMice.info Sat Sep 4 14:41:33 2021 From: PythonList at DancesWithMice.info (dn) Date: Sun, 5 Sep 2021 06:41:33 +1200 Subject: Problem with python In-Reply-To: References: Message-ID: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> On 05/09/2021 06.27, Igor Korot wrote: > Hi, ALL, > > [code] > igor at WaylandGnome ~/bakefile $ python > Python 3.9.6 (default, Aug 8 2021, 17:26:32) > [GCC 10.3.0] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> from distutils import sysconfig >>>> print sysconfig.get_python_inc() > File "", line 1 > print sysconfig.get_python_inc() > ^ > SyntaxError: invalid syntax >>>> > [/code] > > What is the proper way to fix this? Remember that in Python3 print became a function: print( sysconfig.get_python_inc() ) -- Regards, =dn From ikorot01 at gmail.com Sat Sep 4 14:49:49 2021 From: ikorot01 at gmail.com (Igor Korot) Date: Sat, 4 Sep 2021 13:49:49 -0500 Subject: Problem with python In-Reply-To: References: Message-ID: Thx guys. I submitted a bug report for the project that uses it. On Sat, Sep 4, 2021 at 1:42 PM Joel Goldstick wrote: > > On Sat, Sep 4, 2021 at 2:29 PM Igor Korot wrote: > > > > Hi, ALL, > > > > [code] > > igor at WaylandGnome ~/bakefile $ python > > Python 3.9.6 (default, Aug 8 2021, 17:26:32) > > [GCC 10.3.0] on linux > > Type "help", "copyright", "credits" or "license" for more information. > > >>> from distutils import sysconfig > > >>> print sysconfig.get_python_inc() > > File "", line 1 > > print sysconfig.get_python_inc() > > print( sysconfig.get_python_inc()) > > Since python3 print is a function. > > ^ > > SyntaxError: invalid syntax > > >>> > > [/code] > > > > What is the proper way to fix this? > > > > Thank you. > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > -- > Joel Goldstick > -- > https://mail.python.org/mailman/listinfo/python-list From ikorot01 at gmail.com Sat Sep 4 15:29:47 2021 From: ikorot01 at gmail.com (Igor Korot) Date: Sat, 4 Sep 2021 14:29:47 -0500 Subject: Problem with python In-Reply-To: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: Hi, Will this syntax work in python 2? Thank you. On Sat, Sep 4, 2021 at 1:52 PM dn via Python-list wrote: > > On 05/09/2021 06.27, Igor Korot wrote: > > Hi, ALL, > > > > [code] > > igor at WaylandGnome ~/bakefile $ python > > Python 3.9.6 (default, Aug 8 2021, 17:26:32) > > [GCC 10.3.0] on linux > > Type "help", "copyright", "credits" or "license" for more information. > >>>> from distutils import sysconfig > >>>> print sysconfig.get_python_inc() > > File "", line 1 > > print sysconfig.get_python_inc() > > ^ > > SyntaxError: invalid syntax > >>>> > > [/code] > > > > What is the proper way to fix this? > > Remember that in Python3 print became a function: > > print( sysconfig.get_python_inc() ) > > -- > Regards, > =dn > -- > https://mail.python.org/mailman/listinfo/python-list From hjp-python at hjp.at Sat Sep 4 15:48:14 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 21:48:14 +0200 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 2021-09-02 08:32:36 +0100, Alan Gauld via Python-list wrote: > On 31/08/2021 22:32, Chris Angelico wrote: > > If we could abolish DST world-wide, life would be far easier. All the > > rest of it would be easy enough to handle. > We tried that in the UK for 2 years back in the '70s and very > quickly reverted to DST when they realized that the number > of fatalities among young children going to school doubled > during those two years. That makes no sense. With DST the clocks are changed in summer, and they are set forward, so it's darker at the same time in the morning. Maybe you are talking about switching to DST for the whole year or just moving to CET? That would have the effect of it being noticably darker in the morning in winter. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 4 15:50:19 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 21:50:19 +0200 Subject: Problem with python In-Reply-To: References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: On 2021-09-04 14:29:47 -0500, Igor Korot wrote: > Will this syntax work in python 2? Yes. It's just a redundant pair of parentheses. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From rob.cliffe at btinternet.com Sat Sep 4 16:07:11 2021 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Sat, 4 Sep 2021 21:07:11 +0100 Subject: Problem with python In-Reply-To: References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: Well, up to a point. In Python 2 the output from ??? print 1, 2 is '1 2' In Python 3 if you add brackets: ??? print(1, 2) the output is the same. But if you transplant that syntax back into Python 2, the output from ??? print(1, 2) is '(1, 2)'.? The brackets have turned two separate items into a single tuple. If you want Python 2- and 3-compatibility you must find a work-around such as ??? print('%d %d' % (1,2)) ??? print('%s %s' % (1,2)) ??? print(str(1) + ' ' + str(2)) Similarly ??? 'print' in Python 2 outputs a blank line. ??? 'print()' in Python 3 outputs a blank line.? In python 2 it prints a line containing a blank tuple: '()'. A 2/3-compatible way of outputting a blank line is ??? print('') Best wishes Rob Cliffe On 04/09/2021 20:50, Peter J. Holzer wrote: > On 2021-09-04 14:29:47 -0500, Igor Korot wrote: >> Will this syntax work in python 2? > Yes. It's just a redundant pair of parentheses. > > hp > > From hjp-python at hjp.at Sat Sep 4 16:31:12 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 22:31:12 +0200 Subject: Trouble propagating logging configuration In-Reply-To: <87lf4f4fsi.fsf@hornfels.zedat.fu-berlin.de> References: <87eea9zqy1.fsf@hornfels.zedat.fu-berlin.de> <24878.24705.612533.264970@ixdm.fritz.box> <87h7f4a54l.fsf@hornfels.zedat.fu-berlin.de> <24879.44729.220204.596891@ixdm.fritz.box> <87lf4f4fsi.fsf@hornfels.zedat.fu-berlin.de> Message-ID: On 2021-09-02 09:06:53 +0200, Loris Bennett wrote: > Thanks Peter and Dieter for all the help. I have finally figured out > what my problem was. If in a module 'mylibs.mylib' I have > > import logging > > logger = logging.getLogger(__name__) > > and in my main script have > > import logger > import logger.config > > import mylibs.mylib > > logging.config.fileConfig("/home/loris/config") > logger = logging.getLogger(__name__) > > then 'logger' in 'mylibs.mylib' is defined before > 'logging.config.fileConfig' is called and the logger in the module is > not configured. > > If change the order and write > > import logger > import logger.config > > logging.config.fileConfig("/home/loris/config") > logger = logging.getLogger(__name__) > > import mylibs.mylib > > then the 'logger' in 'mylibs.mylibs' does get configured properly. That normally shouldn't make any difference, since any functions in mylibs.mylib would be called only after the logger is initialized. Does your mylibs.mylib contain code which is run on import (other than class, function and variable definitions)? hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 4 16:41:12 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 22:41:12 +0200 Subject: Problem with python In-Reply-To: References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: On 2021-09-04 21:07:11 +0100, Rob Cliffe via Python-list wrote: > Well, up to a point. > In Python 2 the output from > ??? print 1, 2 > is '1 2' > In Python 3 if you add brackets: > ??? print(1, 2) > the output is the same. > But if you transplant that syntax back into Python 2, the output from > ??? print(1, 2) > is '(1, 2)'.? The brackets have turned two separate items into a single > tuple. Yes. I was just talking about that specific case with a single function call. I do not consider explaining the differences between Python 2 and Python 3 to be time well spent in 2021, especially not to someone who apparently just wants to report a bug to some unnamed project (whose maintainers may or may not care about Python2 compatibility). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 4 16:48:11 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 4 Sep 2021 22:48:11 +0200 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 2021-09-04 21:48:14 +0200, Peter J. Holzer wrote: > On 2021-09-02 08:32:36 +0100, Alan Gauld via Python-list wrote: > > On 31/08/2021 22:32, Chris Angelico wrote: > > > If we could abolish DST world-wide, life would be far easier. All the > > > rest of it would be easy enough to handle. > > We tried that in the UK for 2 years back in the '70s and very > > quickly reverted to DST when they realized that the number > > of fatalities among young children going to school doubled > > during those two years. > > That makes no sense. With DST the clocks are changed in summer, and they > are set forward, so it's darker at the same time in the morning. > > Maybe you are talking about switching to DST for the whole year or just > moving to CET? That would have the effect of it being noticably darker > in the morning in winter. Found it. Between 1968-02-18 and 1971-10-30 was continuously on UTC+0100 (probably to be in the same timezone as Germany and France (which didn't have DST at that time). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From nad at python.org Sat Sep 4 18:30:50 2021 From: nad at python.org (Ned Deily) Date: Sat, 4 Sep 2021 18:30:50 -0400 Subject: [RELEASE] Python 3.7.12 and 3.6.15 security updates now available Message-ID: <752577B1-4634-4BC2-B0CD-8CCE3F9B054A@python.org> Python 3.7.12 and 3.6.15, the lastest security fix rollups for Python 3.7 and Python 3.6, are now available. You can find the release files, links to the changelogs, and more information here: https://www.python.org/downloads/release/python-3712/ https://www.python.org/downloads/release/python-3615/ These releases are source code only; Windows and macOS binary installers are not provided for security fix releases. Note that Python 3.9 is now the latest feature release series of Python 3. You should consider upgrading to 3.9 as soon as practical. Get the latest release of 3.9.x here: https://www.python.org/downloads/ Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://www.python.org/psf-landing/ -- Ned Deily nad at python.org -- [] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From michael.stemper at gmail.com Sat Sep 4 13:27:55 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Sat, 4 Sep 2021 12:27:55 -0500 Subject: on writing a while loop for rolling two dice In-Reply-To: <8635qkh2ga.fsf@jevedi.com> References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> Message-ID: On 04/09/2021 08.53, Hope Rouselle wrote: > Chris Angelico writes: >> And at this point, it's looking pretty much identical to the for loop >> version. Ultimately, they're all the same and you can pick and choose >> elements from each of them. > > I see. That's why C must have added the do-while, but yeah --- it's not > really worth it. Kernighan and Ritchie agree(d) with you. Per _The C Programming Language__: Experience shows that do-while is much less used that while and for. (Second edition, Section 3.6, page 63) -- Michael F. Stemper Outside of a dog, a book is man's best friend. Inside of a dog, it's too dark to read. From nospam at please.ty Fri Sep 3 16:24:31 2021 From: nospam at please.ty (jak) Date: Fri, 3 Sep 2021 22:24:31 +0200 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: Il 03/09/2021 14:45, Chris Angelico ha scritto: > I believe the definition of "accurate" here is that, if you take all > of the real numbers represented by those floats, add them all together > with mathematical accuracy, and then take the nearest representable > float, that will be the exact value that fsum will return. In other > words, its accuracy is exactly as good as the final result can be. yup, I agree and this is the because of the link. From alan.gauld at yahoo.co.uk Fri Sep 3 19:15:48 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 4 Sep 2021 00:15:48 +0100 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On 03/09/2021 18:37, Chris Angelico wrote: >>>> Without DST the schools opened in the dark so all the kids >>>> had to travel to school in the dark and the number of >>>> traffic accidents while crossing roads jumped. > > Are you saying that you had DST in winter, or that, when summer *and* > DST came into effect, there was more light at dawn? Because a *lot* of > people confuse summer and DST, and credit DST with the natural effects > of the season change. OK, I see the confusion. What I should point out was that the experiment involved us staying on DST and not reverting to UTC in the winter - that unified us with most of the EU apparently... So although I'm saying DST it was really the non-reversion from DST to UTC that caused problems. Arguably, if we just stayed on UTC and didn't have DST at all there would be no issue - except we'd be an hour out of sync with the EU. (Post Brexit that may not be seen as a problem!! :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From greg.ewing at canterbury.ac.nz Fri Sep 3 20:45:58 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 4 Sep 2021 12:45:58 +1200 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <846feb19-dbd4-4af9-a74d-f484fc310a0bn@googlegroups.com> Message-ID: On 3/09/21 8:11 pm, Christian Gollwitzer wrote: > Unless you have special numbers like NaN or signed zeros etc., a+b=b+a > and a*b=b*a holds also for floats. The only exception I'm aware of is for NaNs, and it's kind of pendantic: you can't say that x + NaN == NaN + x, but only because NaNs never compare equal. You still get a NaN either way, so for all practical purposes it's commutative. -- Greg From tjreedy at udel.edu Fri Sep 3 22:32:29 2021 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 3 Sep 2021 22:32:29 -0400 Subject: How to include insertion order in dict equality Message-ID: In https://bugs.python.org/issue45093, Michael Rans suggested adding a dict method that would include the insertion order in comparing dicts for equality. He wanted this for testing. The proposal is rejected because there are already multiple good methods. To make them more visible and searchable, I list them here. Steven D'Aprano: d1 == d2 and all(k1 == k2 for k1, k2 in zip(d1, d2)) Sethiy Storchaka: list(d1.items()) == list(d2.items()) # When using unittest, produces nicer report on failure. Raymond Hettinger: assert OrderedDict(d) == OrderedDict(e) -- Terry Jan Reedy From hrouselle at jevedi.com Sat Sep 4 08:31:17 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 09:31:17 -0300 Subject: on the popularity of loops while and for References: Message-ID: <86wnnwiktm.fsf@jevedi.com> "Peter J. Holzer" writes: > On 2021-08-29 10:04:47 +0100, Barry wrote: >> > I'd like get a statistic of how often each loop is used in practice. >> > >> > I was trying to take a look at the Python's standard libraries --- those >> > included in a standard installation of Python 3.9.6, say --- to see >> > which loops are more often used among while and for loops. Of course, >> > since English use the preposition ``for'' a lot, that makes my life >> > harder. Removing comments is easy, but removing strings is harder. So >> > I don't know yet what I'll do. >> > >> > Have you guys ever measured something like that in a casual or serious >> > way? I'd love to know. Thank you! >> >> I am interesting in why you think that choice of while vs. for is >> about popularity? >> >> Surely the choice is made in most cases by the algorithm? > > The context here is an introductory course for Python. So there is not > "the algorithm", there are all the algorithms that a novice is likely > to encounter. > > For me it makes absolute sense to base the contents of the course on > popularity. Constructs which a novice programmer is very likely to use > or encounter in other people's code should be covered more thoroughly > than constructs that will be used only rarely. Some are so rare that > they can be safely omitted. The while loop is certainly not in that > category, but it probably makes sense to spend less time on it than on > the for loop. ``A man after my own heart.'' From hrouselle at jevedi.com Sat Sep 4 08:34:25 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 09:34:25 -0300 Subject: on perhaps unloading modules? References: <86k0kghtim.fsf@jevedi.com> <867dgfinp6.fsf@jevedi.com> <86a6lahoml.fsf@jevedi.com> <868s0uftbh.fsf@jevedi.com> <7st4ig986242i36gqemgr5e1kub6s8t6ld@4ax.com> <86o89p9t4j.fsf@jevedi.com> Message-ID: <86mtosikoe.fsf@jevedi.com> Dennis Lee Bieber writes: > On Sun, 22 Aug 2021 16:28:12 -0300, Hope Rouselle > declaimed the following: > > >>That's wild. :-) Was this created by Brian Kernighan? It's hard to >>believe. Oh, I think he wrote AMPL, wasn't it? A Mathematical >>Programming Language, or something like that. > > Kenneth Iverson, early 1960s for release, though he started in the late > 50s (so a decade before Kernighan). I believe it started life as a notation > for publishing reports, not as an actual implemented language. > > > {Hmmm, supposed to have influenced Matlab, S, and Wolfram/Mathematica} > > One of the quirks is that one reads APL from right to left... cf: > > > You do not want to look down at the one-liner for Conway's Game of > Life. I really do not. But I did. That's really wild. Dramatic. Speechless. From hrouselle at jevedi.com Sat Sep 4 08:48:40 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 09:48:40 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: <86a6ksik0n.fsf@jevedi.com> Christian Gollwitzer writes: > Am 02.09.21 um 15:51 schrieb Hope Rouselle: >> Just sharing a case of floating-point numbers. Nothing needed to be >> solved or to be figured out. Just bringing up conversation. >> (*) An introduction to me >> I don't understand floating-point numbers from the inside out, but I >> do >> know how to work with base 2 and scientific notation. So the idea of >> expressing a number as >> mantissa * base^{power} >> is not foreign to me. (If that helps you to perhaps instruct me on >> what's going on here.) >> (*) A presentation of the behavior >> >>>>> import sys >>>>> sys.version >> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >> bit (AMD64)]' >> >>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>> sum(ls) >> 39.599999999999994 >> >>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>> sum(ls) >> 39.60000000000001 >> All I did was to take the first number, 7.23, and move it to the >> last >> position in the list. (So we have a violation of the commutativity of >> addition.) > > I believe it is not commutativity, but associativity, that is > violated. Shall we take this seriously? (I will disagree, but that doesn't mean I am not grateful for your post. Quite the contary.) It in general violates associativity too, but the example above couldn't be referring to associativity because the second sum above could not be obtained from associativity alone. Commutativity is required, applied to five pairs of numbers. How can I go from 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 to 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? Perhaps only through various application of commutativity, namely the ones below. (I omit the parentheses for less typing. I suppose that does not create much trouble. There is no use of associativity below, except for the intented omission of parentheses.) 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77 = 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77 = 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77 = 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77 = 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23. > Even for floating point, a+b=b+a except for maybe some extreme cases > like denormliazed numbers etc. > > But in general (a+b)+c != a+ (b+c) > > Consider decimal floating point with 2 digits. > a=1 > b=c=0.04 > > Then you get LHS; > (1 + 0.04) + 0.04 = 1 + 0.04 = 1 > > RHS: > > 1 + (0.04 + 0.04) = 1 + 0.08 = 1.1 > > Your sum is evaluated like (((a + b) + c) + ....) and hence, if you > permute the numbers, it can be unequal. If you need better accuracy, > there is the Kahan summation algorithm and other alternatives: > https://en.wikipedia.org/wiki/Kahan_summation_algorithm Thanks! From hrouselle at jevedi.com Sat Sep 4 09:04:41 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 10:04:41 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> Message-ID: <861r64ij9y.fsf@jevedi.com> Chris Angelico writes: > On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle wrote: >> >> Just sharing a case of floating-point numbers. Nothing needed to be >> solved or to be figured out. Just bringing up conversation. >> >> (*) An introduction to me >> >> I don't understand floating-point numbers from the inside out, but I do >> know how to work with base 2 and scientific notation. So the idea of >> expressing a number as >> >> mantissa * base^{power} >> >> is not foreign to me. (If that helps you to perhaps instruct me on >> what's going on here.) >> >> (*) A presentation of the behavior >> >> >>> import sys >> >>> sys.version >> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >> bit (AMD64)]' >> >> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >> >>> sum(ls) >> 39.599999999999994 >> >> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >> >>> sum(ls) >> 39.60000000000001 >> >> All I did was to take the first number, 7.23, and move it to the last >> position in the list. (So we have a violation of the commutativity of >> addition.) > > It's not about the commutativity of any particular pair of operands - > that's always guaranteed. Shall we take this seriously? It has to be about the commutativity of at least one particular pair because it is involved with the commutavitity of a set of pairs. If various pairs are involved, then at least one is involved. IOW, it is about the commutativity of some pair of operands and so it could not be the case that it's not about the commutativity of any. (Lol. I hope that's not too insubordinate. I already protested against a claim for associativity in this thread and now I'm going for the king of the hill, for whom I have always been so grateful!) > What you're seeing here is the results of intermediate rounding. Try > this: Indeed. Your post here is really great, as usual. It offers a nice tool for investigation. Thank you so much. >>>> def sum(stuff): > ... total = 0 > ... for thing in stuff: > ... total += thing > ... print(thing, "-->", total) > ... return total > ... >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>> sum(ls) > 7.23 --> 7.23 > 8.41 --> 15.64 > 6.15 --> 21.79 > 2.31 --> 24.099999999999998 Alright. Thanks so much for this example. Here's a new puzzle for me. The REPL makes me think that both 21.79 and 2.31 *are* representable exactly in Python's floating-point datatype because I see: >>> 2.31 2.31 >>> 21.79 21.79 When I add them, the result obtained makes me think that the sum is *not* representable exactly in Python's floating-point number: >>> 21.79 + 2.31 24.099999999999998 However, when I type 24.10 explicitly, the REPL makes me think that 24.10 *is* representable exactly: >>> 24.10 24.1 I suppose I cannot trust the appearance of the representation? What's really going on there? (Perhaps the trouble appears while Python is computing the sum of the numbers 21.79 and 2.31?) Thanks so much! From hrouselle at jevedi.com Sat Sep 4 09:40:35 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 10:40:35 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> Message-ID: <86eea4h31o.fsf@jevedi.com> Chris Angelico writes: > On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle wrote: >> >> Hope Rouselle writes: >> >> > Just sharing a case of floating-point numbers. Nothing needed to be >> > solved or to be figured out. Just bringing up conversation. >> > >> > (*) An introduction to me >> > >> > I don't understand floating-point numbers from the inside out, but I do >> > know how to work with base 2 and scientific notation. So the idea of >> > expressing a number as >> > >> > mantissa * base^{power} >> > >> > is not foreign to me. (If that helps you to perhaps instruct me on >> > what's going on here.) >> > >> > (*) A presentation of the behavior >> > >> >>>> import sys >> >>>> sys.version >> > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >> > bit (AMD64)]' >> > >> >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >> >>>> sum(ls) >> > 39.599999999999994 >> > >> >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >> >>>> sum(ls) >> > 39.60000000000001 >> > >> > All I did was to take the first number, 7.23, and move it to the last >> > position in the list. (So we have a violation of the commutativity of >> > addition.) >> >> Suppose these numbers are prices in dollar, never going beyond cents. >> Would it be safe to multiply each one of them by 100 and therefore work >> with cents only? For instance > > Yes and no. It absolutely *is* safe to always work with cents, but to > do that, you have to be consistent: ALWAYS work with cents, never with > floating point dollars. > > (Or whatever other unit you choose to use. Most currencies have a > smallest-normally-used-unit, with other currency units (where present) > being whole number multiples of that minimal unit. Only in forex do > you need to concern yourself with fractional cents or fractional yen.) > > But multiplying a set of floats by 100 won't necessarily solve your > problem; you may have already fallen victim to the flaw of assuming > that the numbers are represented accurately. Hang on a second. I see it's always safe to work with cents, but I'm only confident to say that when one gives me cents to start with. In other words, if one gives me integers from the start. (Because then, of course, I don't even have floats to worry about.) If I'm given 1.17, say, I am not confident that I could turn this number into 117 by multiplying it by 100. And that was the question. Can I always multiply such IEEE 754 dollar amounts by 100? Considering your last paragraph above, I should say: if one gives me an accurate floating-point representation, can I assume a multiplication of it by 100 remains accurately representable in IEEE 754? >> --8<---------------cut here---------------start------------->8--- >> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >> >>> sum(map(lambda x: int(x*100), ls)) / 100 >> 39.6 >> >> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >> >>> sum(map(lambda x: int(x*100), ls)) / 100 >> 39.6 >> --8<---------------cut here---------------end--------------->8--- >> >> Or multiplication by 100 isn't quite ``safe'' to do with floating-point >> numbers either? (It worked in this case.) > > You're multiplying and then truncating, which risks a round-down > error. Try adding a half onto them first: > > int(x * 100 + 0.5) > > But that's still not a perfect guarantee. Far safer would be to > consider monetary values to be a different type of value, not just a > raw number. For instance, the value $7.23 could be stored internally > as the integer 723, but you also know that it's a value in USD, not a > simple scalar. It makes perfect sense to add USD+USD, it makes perfect > sense to multiply USD*scalar, but it doesn't make sense to multiply > USD*USD. Because of the units? That would be USD squared? (Nice analysis.) >> I suppose that if I multiply it by a power of two, that would be an >> operation that I can be sure will not bring about any precision loss >> with floating-point numbers. Do you agree? > > Assuming you're nowhere near 2**53, yes, that would be safe. But so > would multiplying by a power of five. The problem isn't precision loss > from the multiplication - the problem is that your input numbers > aren't what you think they are. That number 7.23, for instance, is > really.... Hm, I think I see what you're saying. You're saying multiplication and division in IEEE 754 is perfectly safe --- so long as the numbers you start with are accurately representable in IEEE 754 and assuming no overflow or underflow would occur. (Addition and subtraction are not safe.) >>>> 7.23.as_integer_ratio() > (2035064081618043, 281474976710656) > > ... the rational number 2035064081618043 / 281474976710656, which is > very close to 7.23, but not exactly so. (The numerator would have to > be ...8042.88 to be exactly correct.) There is nothing you can do at > this point to regain the precision, although a bit of multiplication > and rounding can cheat it and make it appear as if you did. > > Floating point is a very useful approximation to real numbers, but > real numbers aren't the best way to represent financial data. Integers > are. I'm totally persuaded. Thanks. From hrouselle at jevedi.com Sat Sep 4 09:53:25 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 10:53:25 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> Message-ID: <8635qkh2ga.fsf@jevedi.com> Chris Angelico writes: > On Fri, Sep 3, 2021 at 4:33 AM Hope Rouselle wrote: >> Yeah. Here's a little context. I came across this by processing a list >> of exercises. (I'm teaching a course --- you know that by now, I >> guess.) So the first thing I observed was the equal volume of work >> dedicated to while loops and for loops --- so I decided to compared >> which appeared more often in a certain sample of well-written Python >> code. It turns out the for loop was much more frequent. Students have >> been reporting too much work in too little time, so I decided to reduce >> the number of exercises involving while loops. When I began to look at >> the exercises, to see which ones I'd exclude, I decided to exclude them >> all --- lol! --- except for one. The one that remained was this one >> about rolling dice until a satisfying result would appear. (All other >> ones were totally more naturally written with a for loop.) >> >> So if I were to also write this with a for-loop, it'd defeat the purpose >> of the course's moment. Besides, I don't think a for-loop would improve >> the readability here. > > It's on the cusp. When you ask someone to express the concept of "do > this until this happens", obviously that's a while loop; but as soon > as you introduce the iteration counter, it becomes less obvious, since > "iterate over counting numbers until this happens" is a quite viable > way to express this. However, if the students don't know > itertools.count(), they'll most likely put in an arbitrary limit (like > "for c in range(100000000)"), which you can call them out for. > >> But I thought your protest against the while-True was very well put: >> while-True is not too readable for a novice. Surely what's readable or >> more-natural /to someone/ is, well, subjective (yes, by definition). >> But perhaps we may agree that while rolling dice until a certain >> success, we want to roll them while something happens or doesn't happen. >> One of the two. So while-True is a bit of a jump. Therefore, in this >> case, the easier and more natural option is to say while-x-not-equal-y. > > That may be the case, but in Python, I almost never write "while > True". Consider the two while loops in this function: > > https://github.com/Rosuav/shed/blob/master/autohost_manager.py#L92 > > Thanks to Python's flexibility and efficient compilation, these loops > are as descriptive as those with actual conditions, while still > behaving exactly like "while True". (The inner loop, "more pages", > looks superficially like it should be a for loop - "for page in > pages:" - but the data is coming from successive API calls, so it > can't know.) That's pretty nice. I did suggest the same to my students, showing your code to them, actually. The course explicitly avoided talking about regular values being considered True, but now I couldn't keep the truth from them. >> I don't see it. You seem to have found what we seem to agree that it >> would be the more natural way to write the strategy. But I can't see >> it. It certainly isn't >> >> --8<---------------cut here---------------start------------->8--- >> def how_many_times_1(): >> c, x, y = 0, None, None >> while x != y: >> c = c + 1 >> x, y = roll() >> return c, x, y >> --8<---------------cut here---------------end--------------->8--- >> >> nor >> >> --8<---------------cut here---------------start------------->8--- >> def how_many_times_2(): >> c, x, y = 0, None, None >> while x == y: >> c = c + 1 >> x, y = dados() >> return c, x, y >> --8<---------------cut here---------------end--------------->8--- >> >> What do you have in mind? I couldn't see it. > > You're overlaying two loops here. One is iterating "c" up from zero, > the other is calling a function and testing its results. It's up to > you which of these should be considered the more important, and which > is a bit of extra work added onto it. With the counter as primary, you > get something like this: > > for c in itertools.count(): > x, y = roll() > if x == y: return c, x, y > > With the roll comparison as primary, you get this: > > c, x, y = 0, 0, 1 > while x != y: > x, y = roll() > c += 1 > return c, x, y > > Reworking the second into a do-while style (Python doesn't have that, > so we have to write it manually): > > c = 0 > while "x and y differ": > x, y = roll() > c += 1 > if x == y: break > return c, x, y > > And at this point, it's looking pretty much identical to the for loop > version. Ultimately, they're all the same and you can pick and choose > elements from each of them. I see. That's why C must have added the do-while, but yeah --- it's not really worth it. (An educational investigation. Thank you.) From hrouselle at jevedi.com Sat Sep 4 09:56:15 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 10:56:15 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> Message-ID: <86v93gfnr4.fsf@jevedi.com> "Peter J. Holzer" writes: > On 2021-09-02 11:28:21 -0300, Hope Rouselle wrote: >> dn writes: >> > On 29/08/2021 08.46, Hope Rouselle wrote: >> >> Here's my solution: >> >> >> >> --8<---------------cut here---------------start------------->8--- >> >> def how_many_times(): >> >> x, y = 0, 1 >> >> c = 0 >> >> while x != y: >> >> c = c + 1 >> >> x, y = roll() >> >> return c, (x, y) >> > >> >> >> >> Why am I unhappy? I'm wish I could confine x, y to the while loop. >> >> The introduction of ``x, y = 0, 1'' must feel like a trick to a >> >> novice. How would you write this? > [...] >> But perhaps we may agree that while rolling dice until a certain >> success, we want to roll them while something happens or doesn't happen. >> One of the two. So while-True is a bit of a jump. Therefore, in this >> case, the easier and more natural option is to say while-x-not-equal-y. >> >> But this approach seems to force me into initializing x, y with >> different values. > > You can get around this by using NaN: > > def how_many_times(): > c, x, y = 0, math.nan, math.nan > while x != y: > c = c + 1 > x, y = roll() > return c, x, y > > Not sure if this is an improvement. Now you have to explain to your > students why math.nan != math.nan. > > When I need a guaranteed unique value I often just use object(): > > def how_many_times(): > c, x, y = 0, object(), object() > while x != y: > c = c + 1 > x, y = roll() > return c, x, y > > Of course now you are back to two different values, but they look the > same. Which may be better or worse for your students. Plus x and y are > now bound to objects of different types during their lifetime, which may > be a bit dicey. Lol. I would argue that it's quite appropriate to the event (``of rolling dice'' --- clarity-obsession). :D Pretty nice alternatives. Thank you so much. From hrouselle at jevedi.com Sat Sep 4 09:57:18 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 10:57:18 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <3c2985dd-103c-43ef-b0e3-ea545a3f9c97n@googlegroups.com> Message-ID: <86pmtofnpd.fsf@jevedi.com> Julio Di Egidio writes: > On Thursday, 2 September 2021 at 15:52:02 UTC+2, Hope Rouselle wrote: > >> I don't understand floating-point numbers from the inside out, but I do >> know how to work with base 2 and scientific notation. So the idea of >> expressing a number as >> >> mantissa * base^{power} > > That's the basic idea, but the actual (ISO) floating-point *encoding* > is more complicated than that. > >> is not foreign to me. (If that helps you to perhaps instruct me on >> what's going on here.) > > This is the "classic": > DAVID GOLDBER > What Every Computer Scientist Should Know About Floating-Point Arithmetic > > > Here is some more introductory stuff: > > Rozman's was pretty concise and nice. Thank you. From hrouselle at jevedi.com Sat Sep 4 10:07:36 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 11:07:36 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <4105d35c-cb9a-42d1-9985-f3c5b6ff6209n@googlegroups.com> Message-ID: <868s0cfn87.fsf@jevedi.com> Julio Di Egidio writes: > On Thursday, 2 September 2021 at 16:51:24 UTC+2, Christian Gollwitzer wrote: >> Am 02.09.21 um 16:49 schrieb Julio Di Egidio: >> > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote: >> >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote: >> > >> >>> 39.60000000000001 >> >> >> >> Welcome to the exciting world of roundoff error: >> > >> > Welcome to the exiting world of Usenet. >> > >> > *Plonk* >> >> Pretty harsh, isn't it? He gave a concise example of the same inaccuracy >> right afterwards. > > And I thought you were not seeing my posts... > > Given that I have already given a full explanation, you guys, that you > realise it or not, are simply adding noise for the usual pub-level > discussion I must most charitably guess. > > Anyway, just my opinion. (EOD.) Which is certainly appreciated --- as a rule. Pub-level noise is pretty much unavoidable in investigation, education. Being wrong is, too, unavoidable in investigation, education. There is a point we eventually publish at the most respected journals, but that's a whole other interval of the time-line. IOW, chill out! :-D (Give us a C-k and meet us up in the next thread. Oh, my, you're not a Gnus user: you are a G2/1.0 user. That's pretty scary.) By the way, how's sci.logic going? I, too, lost my patience there. :-) From hrouselle at jevedi.com Sat Sep 4 10:42:38 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 11:42:38 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> Message-ID: <86o898e71d.fsf@jevedi.com> Richard Damon writes: > On 9/4/21 9:40 AM, Hope Rouselle wrote: >> Chris Angelico writes: >> >>> On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle wrote: >>>> >>>> Hope Rouselle writes: >>>> >>>>> Just sharing a case of floating-point numbers. Nothing needed to be >>>>> solved or to be figured out. Just bringing up conversation. >>>>> >>>>> (*) An introduction to me >>>>> >>>>> I don't understand floating-point numbers from the inside out, but I do >>>>> know how to work with base 2 and scientific notation. So the idea of >>>>> expressing a number as >>>>> >>>>> mantissa * base^{power} >>>>> >>>>> is not foreign to me. (If that helps you to perhaps instruct me on >>>>> what's going on here.) >>>>> >>>>> (*) A presentation of the behavior >>>>> >>>>>>>> import sys >>>>>>>> sys.version >>>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >>>>> bit (AMD64)]' >>>>> >>>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>>>>> sum(ls) >>>>> 39.599999999999994 >>>>> >>>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>>>>> sum(ls) >>>>> 39.60000000000001 >>>>> >>>>> All I did was to take the first number, 7.23, and move it to the last >>>>> position in the list. (So we have a violation of the commutativity of >>>>> addition.) >>>> >>>> Suppose these numbers are prices in dollar, never going beyond cents. >>>> Would it be safe to multiply each one of them by 100 and therefore work >>>> with cents only? For instance >>> >>> Yes and no. It absolutely *is* safe to always work with cents, but to >>> do that, you have to be consistent: ALWAYS work with cents, never with >>> floating point dollars. >>> >>> (Or whatever other unit you choose to use. Most currencies have a >>> smallest-normally-used-unit, with other currency units (where present) >>> being whole number multiples of that minimal unit. Only in forex do >>> you need to concern yourself with fractional cents or fractional yen.) >>> >>> But multiplying a set of floats by 100 won't necessarily solve your >>> problem; you may have already fallen victim to the flaw of assuming >>> that the numbers are represented accurately. >> >> Hang on a second. I see it's always safe to work with cents, but I'm >> only confident to say that when one gives me cents to start with. In >> other words, if one gives me integers from the start. (Because then, of >> course, I don't even have floats to worry about.) If I'm given 1.17, >> say, I am not confident that I could turn this number into 117 by >> multiplying it by 100. And that was the question. Can I always >> multiply such IEEE 754 dollar amounts by 100? >> >> Considering your last paragraph above, I should say: if one gives me an >> accurate floating-point representation, can I assume a multiplication of >> it by 100 remains accurately representable in IEEE 754? >> > > Multiplication by 100 might not be accurate if the number you are > starting with is close to the limit of precision, because 100 is > 1.1001 x 64 so multiplying by 100 adds about 5 more 'bits' to the > representation of the number. In your case, the numbers are well below > that point. Alright. That's clear now. Thanks so much! >>>> --8<---------------cut here---------------start------------->8--- >>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>>>> sum(map(lambda x: int(x*100), ls)) / 100 >>>> 39.6 >>>> >>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>>>> sum(map(lambda x: int(x*100), ls)) / 100 >>>> 39.6 >>>> --8<---------------cut here---------------end--------------->8--- >>>> >>>> Or multiplication by 100 isn't quite ``safe'' to do with floating-point >>>> numbers either? (It worked in this case.) >>> >>> You're multiplying and then truncating, which risks a round-down >>> error. Try adding a half onto them first: >>> >>> int(x * 100 + 0.5) >>> >>> But that's still not a perfect guarantee. Far safer would be to >>> consider monetary values to be a different type of value, not just a >>> raw number. For instance, the value $7.23 could be stored internally >>> as the integer 723, but you also know that it's a value in USD, not a >>> simple scalar. It makes perfect sense to add USD+USD, it makes perfect >>> sense to multiply USD*scalar, but it doesn't make sense to multiply >>> USD*USD. >> >> Because of the units? That would be USD squared? (Nice analysis.) >> >>>> I suppose that if I multiply it by a power of two, that would be an >>>> operation that I can be sure will not bring about any precision loss >>>> with floating-point numbers. Do you agree? >>> >>> Assuming you're nowhere near 2**53, yes, that would be safe. But so >>> would multiplying by a power of five. The problem isn't precision loss >>> from the multiplication - the problem is that your input numbers >>> aren't what you think they are. That number 7.23, for instance, is >>> really.... >> >> Hm, I think I see what you're saying. You're saying multiplication and >> division in IEEE 754 is perfectly safe --- so long as the numbers you >> start with are accurately representable in IEEE 754 and assuming no >> overflow or underflow would occur. (Addition and subtraction are not >> safe.) >> > > Addition and Subtraction are just as safe, as long as you stay within > the precision limits. Multiplication and division by powers of two are > the safest, not needing to add any precision, until you hit the limits > of the magnitude of numbers that can be expressed. > > The problem is that a number like 0.1 isn't precisely represented, so it > ends up using ALL available precision to get the closest value to it so > ANY operations on it run the danger of precision loss. Got it. That's clear now. It should've been before, but my attention is that of a beginner, so some extra iterations turn up. As long as the numbers involved are accurately representable, floating-points have no other problems. I may, then, conclude that the whole difficulty with floating-point is nothing but going beyond the reserved space for the number. However, I still lack an easy method to detect when a number is not accurately representable by the floating-point datatype in use. For instance, 0.1 is not representable accurately in IEEE 754. But I don't know how to check that >>> 0.1 0.1 # no clue >>> 0.1 + 0.1 0.2 # no clue >>> 0.1 + 0.1 + 0.1 0.30000000000000004 # there is the clue How can I get a clearer and quicker evidence that 0.1 is not accurately representable --- using the REPL? I know 0.1 = 1/10 = 1 * 10^-1 and in base 2 that would have to be represented as... Let me calculate it with my sophisticated skills: 0.1: 0 + 0.2 --> 0 + 0.4 --> 0 + 0.8 --> 1 + 0.6 --> 1 + 0.2, closing a cycle. So 0.1 is representable poorly as 0.00011... In other words, 1/10 in base 10 equals 1/2^4 + 1/2^5 + 1/2^9 + 1/2^10 + ... The same question in other words --- what's a trivial way for the REPL to show me such cycles occur? >>>>>> 7.23.as_integer_ratio() >>> (2035064081618043, 281474976710656) Here's what I did on this case. The REPL is telling me that 7.23 = 2035064081618043/281474976710656 If that were true, then 7.23 * 281474976710656 would have to equal 2035064081618043. So I typed: >>> 7.23 * 281474976710656 2035064081618043.0 That agrees with the falsehood. I'm getting no evidence of the problem. When take control of my life out of the hands of misleading computers, I calculate the sum: 844424930131968 + 5629499534213120 197032483697459200 ================== 203506408161804288 =/= 203506408161804300 How I can save the energy spent on manual verification? Thanks very much. From hrouselle at jevedi.com Sat Sep 4 11:09:53 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 12:09:53 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <4105d35c-cb9a-42d1-9985-f3c5b6ff6209n@googlegroups.com> <868s0cfn87.fsf@jevedi.com> <056efcd4-5928-4675-b6de-6f0266412843n@googlegroups.com> Message-ID: <86ilzge5ry.fsf@jevedi.com> Julio Di Egidio writes: [...] >> I, too, lost my patience there. :-) > > As if I didn't know who's trolling... I never trolled you. When we had our conversations in sci.logic, I was Boris Dorestand --- you would remember if you have very good memory. We talked for just a few days, I guess. The people trolling you there were not me. So, though that's no proof, *I* know you didn't know. :-) From hrouselle at jevedi.com Sat Sep 4 12:07:54 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 13:07:54 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6ksik0n.fsf@jevedi.com> Message-ID: <86zgsscoit.fsf@jevedi.com> Christian Gollwitzer writes: > Am 04.09.21 um 14:48 schrieb Hope Rouselle: >> Christian Gollwitzer writes: >> >>> Am 02.09.21 um 15:51 schrieb Hope Rouselle: >>>> Just sharing a case of floating-point numbers. Nothing needed to be >>>> solved or to be figured out. Just bringing up conversation. >>>> (*) An introduction to me >>>> I don't understand floating-point numbers from the inside out, but I >>>> do >>>> know how to work with base 2 and scientific notation. So the idea of >>>> expressing a number as >>>> mantissa * base^{power} >>>> is not foreign to me. (If that helps you to perhaps instruct me on >>>> what's going on here.) >>>> (*) A presentation of the behavior >>>> >>>>>>> import sys >>>>>>> sys.version >>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >>>> bit (AMD64)]' >>>> >>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>>>> sum(ls) >>>> 39.599999999999994 >>>> >>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>>>> sum(ls) >>>> 39.60000000000001 >>>> All I did was to take the first number, 7.23, and move it to the >>>> last >>>> position in the list. (So we have a violation of the commutativity of >>>> addition.) >>> >>> I believe it is not commutativity, but associativity, that is >>> violated. >> Shall we take this seriously? (I will disagree, but that doesn't >> mean I >> am not grateful for your post. Quite the contary.) It in general >> violates associativity too, but the example above couldn't be referring >> to associativity because the second sum above could not be obtained from >> associativity alone. Commutativity is required, applied to five pairs >> of numbers. How can I go from >> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 >> to >> 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? >> Perhaps only through various application of commutativity, namely >> the >> ones below. (I omit the parentheses for less typing. I suppose that >> does not create much trouble. There is no use of associativity below, >> except for the intented omission of parentheses.) > > With the parens it will become more obvious. > >> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 >> = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77 > > The sum is evaluated as > > (((7.23 + 8.41) + 6.15 + ...) > > For the first shift, you are correct that commutativity will result in > > (((8.41 + 7.23) + 6.15 + ...) > > But you can't go in one step to > > (((8.41 + 6.15) + 7.23 + ...) > > with the commutativity law alone. Instead, a sequence of > associativity and commutativity is required to move the 7.23 out of > the first pair of parentheses. > > And what I was trying to say, the commutative steps *are* equal in > floating point arithmetics, whereas the associative steps are not. Oh, I see it. Very good point! Lesson learned. From hrouselle at jevedi.com Sat Sep 4 12:22:27 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 13:22:27 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: <86pmtocnuk.fsf@jevedi.com> Greg Ewing writes: > On 5/09/21 2:42 am, Hope Rouselle wrote: >> Here's what I did on this case. The REPL is telling me that >> 7.23 = 2035064081618043/281474976710656 > > If 7.23 were exactly representable, you would have got > 723/1000. > > Contrast this with something that *is* exactly representable: > >>>> 7.875.as_integer_ratio() > (63, 8) > > and observe that 7875/1000 == 63/8: > >>>> from fractions import Fraction >>>> Fraction(7875,1000) > Fraction(63, 8) > > In general, to find out whether a decimal number is exactly > representable in binary, represent it as a ratio of integers > where the denominator is a power of 10, reduce that to lowest > terms, and compare with the result of as_integer_ratio(). That makes perfect sense and answers my question. I appreciate it. From hrouselle at jevedi.com Sat Sep 4 13:32:55 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 14:32:55 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> <86pmtocnuk.fsf@jevedi.com> Message-ID: <86k0jwckl4.fsf@jevedi.com> Hope Rouselle writes: > Greg Ewing writes: > >> On 5/09/21 2:42 am, Hope Rouselle wrote: >>> Here's what I did on this case. The REPL is telling me that >>> 7.23 = 2035064081618043/281474976710656 >> >> If 7.23 were exactly representable, you would have got >> 723/1000. >> >> Contrast this with something that *is* exactly representable: >> >>>>> 7.875.as_integer_ratio() >> (63, 8) >> >> and observe that 7875/1000 == 63/8: >> >>>>> from fractions import Fraction >>>>> Fraction(7875,1000) >> Fraction(63, 8) >> >> In general, to find out whether a decimal number is exactly >> representable in binary, represent it as a ratio of integers >> where the denominator is a power of 10, reduce that to lowest >> terms, and compare with the result of as_integer_ratio(). > > That makes perfect sense and answers my question. I appreciate it. Here's my homework in high-precision. Thoughts? Thank you! --8<---------------cut here---------------start------------->8--- def is_representable(s): return in_lowest_terms(rat_power_of_10(s)) == float(s).as_integer_ratio() # >>> is_representable("1.5") # True # # >>> is_representable("0.1") # False def rat_power_of_10(s): """I assume s is a numeric string in the format .""" if "." not in s: return int(s), 1 integral, fractional = s.split(".") return int(integral + fractional), 10**(len(fractional)) # >>> rat_power_of_10("72.100") # (72100, 1000) def in_lowest_terms(rat): from math import gcd n, d = rat return n//gcd(n, d), d//gcd(n, d) # >>> in_lowest_terms( (72100, 1000) ) # (721, 10) --8<---------------cut here---------------end--------------->8--- From hrouselle at jevedi.com Sat Sep 4 15:46:27 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 16:46:27 -0300 Subject: Problem with python References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: <86czpoceek.fsf@jevedi.com> Igor Korot writes: > Hi, > Will this syntax work in python 2? If you say print(something) it works in both. So, stick to this syntax. From hrouselle at jevedi.com Sat Sep 4 15:52:41 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sat, 04 Sep 2021 16:52:41 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> Message-ID: <867dfwce46.fsf@jevedi.com> "Michael F. Stemper" writes: > On 04/09/2021 08.53, Hope Rouselle wrote: >> Chris Angelico writes: > >>> And at this point, it's looking pretty much identical to the for loop >>> version. Ultimately, they're all the same and you can pick and choose >>> elements from each of them. >> I see. That's why C must have added the do-while, but yeah --- it's >> not >> really worth it. > > Kernighan and Ritchie agree(d) with you. Per _The C Programming > Language__: > Experience shows that do-while is much less used that while > and for. > > (Second edition, Section 3.6, page 63) Special thanks for the reference! Here's what they say on the first edition. ``As might be expected, do-while is much less used than while and for, accounting for perhaps five percent of all loops.'' (First edition, section 3.6, page 59.) They looked into a sample and decided to remove the statistic. :-) From nacnud_uk27 at yahoo.com Fri Sep 3 16:09:40 2021 From: nacnud_uk27 at yahoo.com (Nacnud Nac) Date: Fri, 3 Sep 2021 20:09:40 +0000 (UTC) Subject: CPython / Decimal and bit length of value. References: <1771811707.3518631.1630699780851.ref@mail.yahoo.com> Message-ID: <1771811707.3518631.1630699780851@mail.yahoo.com> Hi, Is there a quick way to get the number of bits required to store the value in a Decimal class?? What obvious thing am I missing? I'm working with really large integers, say, in the order of 5_000_000 of ASCII base 10 digits.? It seems the function mpd_sizeinbase would be a nice thing to be able to call..... It'd be nice to just be able to tell the "size of data", or something like that, that was stored in the *data? I note there is len "field" in the structure mpd_t Sorry for the dumb question.... Thanks,Duncan From Richard at Damon-Family.org Sat Sep 4 10:01:23 2021 From: Richard at Damon-Family.org (Richard Damon) Date: Sat, 4 Sep 2021 10:01:23 -0400 Subject: on floating-point numbers In-Reply-To: <86eea4h31o.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> Message-ID: On 9/4/21 9:40 AM, Hope Rouselle wrote: > Chris Angelico writes: > >> On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle wrote: >>> >>> Hope Rouselle writes: >>> >>>> Just sharing a case of floating-point numbers. Nothing needed to be >>>> solved or to be figured out. Just bringing up conversation. >>>> >>>> (*) An introduction to me >>>> >>>> I don't understand floating-point numbers from the inside out, but I do >>>> know how to work with base 2 and scientific notation. So the idea of >>>> expressing a number as >>>> >>>> mantissa * base^{power} >>>> >>>> is not foreign to me. (If that helps you to perhaps instruct me on >>>> what's going on here.) >>>> >>>> (*) A presentation of the behavior >>>> >>>>>>> import sys >>>>>>> sys.version >>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >>>> bit (AMD64)]' >>>> >>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>>>> sum(ls) >>>> 39.599999999999994 >>>> >>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>>>> sum(ls) >>>> 39.60000000000001 >>>> >>>> All I did was to take the first number, 7.23, and move it to the last >>>> position in the list. (So we have a violation of the commutativity of >>>> addition.) >>> >>> Suppose these numbers are prices in dollar, never going beyond cents. >>> Would it be safe to multiply each one of them by 100 and therefore work >>> with cents only? For instance >> >> Yes and no. It absolutely *is* safe to always work with cents, but to >> do that, you have to be consistent: ALWAYS work with cents, never with >> floating point dollars. >> >> (Or whatever other unit you choose to use. Most currencies have a >> smallest-normally-used-unit, with other currency units (where present) >> being whole number multiples of that minimal unit. Only in forex do >> you need to concern yourself with fractional cents or fractional yen.) >> >> But multiplying a set of floats by 100 won't necessarily solve your >> problem; you may have already fallen victim to the flaw of assuming >> that the numbers are represented accurately. > > Hang on a second. I see it's always safe to work with cents, but I'm > only confident to say that when one gives me cents to start with. In > other words, if one gives me integers from the start. (Because then, of > course, I don't even have floats to worry about.) If I'm given 1.17, > say, I am not confident that I could turn this number into 117 by > multiplying it by 100. And that was the question. Can I always > multiply such IEEE 754 dollar amounts by 100? > > Considering your last paragraph above, I should say: if one gives me an > accurate floating-point representation, can I assume a multiplication of > it by 100 remains accurately representable in IEEE 754? > Multiplication by 100 might not be accurate if the number you are starting with is close to the limit of precision, because 100 is 1.1001 x 64 so multiplying by 100 adds about 5 more 'bits' to the representation of the number. In your case, the numbers are well below that point. >>> --8<---------------cut here---------------start------------->8--- >>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>>> sum(map(lambda x: int(x*100), ls)) / 100 >>> 39.6 >>> >>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>>> sum(map(lambda x: int(x*100), ls)) / 100 >>> 39.6 >>> --8<---------------cut here---------------end--------------->8--- >>> >>> Or multiplication by 100 isn't quite ``safe'' to do with floating-point >>> numbers either? (It worked in this case.) >> >> You're multiplying and then truncating, which risks a round-down >> error. Try adding a half onto them first: >> >> int(x * 100 + 0.5) >> >> But that's still not a perfect guarantee. Far safer would be to >> consider monetary values to be a different type of value, not just a >> raw number. For instance, the value $7.23 could be stored internally >> as the integer 723, but you also know that it's a value in USD, not a >> simple scalar. It makes perfect sense to add USD+USD, it makes perfect >> sense to multiply USD*scalar, but it doesn't make sense to multiply >> USD*USD. > > Because of the units? That would be USD squared? (Nice analysis.) > >>> I suppose that if I multiply it by a power of two, that would be an >>> operation that I can be sure will not bring about any precision loss >>> with floating-point numbers. Do you agree? >> >> Assuming you're nowhere near 2**53, yes, that would be safe. But so >> would multiplying by a power of five. The problem isn't precision loss >> from the multiplication - the problem is that your input numbers >> aren't what you think they are. That number 7.23, for instance, is >> really.... > > Hm, I think I see what you're saying. You're saying multiplication and > division in IEEE 754 is perfectly safe --- so long as the numbers you > start with are accurately representable in IEEE 754 and assuming no > overflow or underflow would occur. (Addition and subtraction are not > safe.) > Addition and Subtraction are just as safe, as long as you stay within the precision limits. Multiplication and division by powers of two are the safest, not needing to add any precision, until you hit the limits of the magnitude of numbers that can be expressed. The problem is that a number like 0.1 isn't precisely represented, so it ends up using ALL available precision to get the closest value to it so ANY operations on it run the danger of precision loss. >>>>> 7.23.as_integer_ratio() >> (2035064081618043, 281474976710656) >> >> ... the rational number 2035064081618043 / 281474976710656, which is >> very close to 7.23, but not exactly so. (The numerator would have to >> be ...8042.88 to be exactly correct.) There is nothing you can do at >> this point to regain the precision, although a bit of multiplication >> and rounding can cheat it and make it appear as if you did. >> >> Floating point is a very useful approximation to real numbers, but >> real numbers aren't the best way to represent financial data. Integers >> are. > > I'm totally persuaded. Thanks. > From auriocus at gmx.de Sat Sep 4 11:25:29 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 4 Sep 2021 17:25:29 +0200 Subject: on floating-point numbers In-Reply-To: <86a6ksik0n.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6ksik0n.fsf@jevedi.com> Message-ID: Am 04.09.21 um 14:48 schrieb Hope Rouselle: > Christian Gollwitzer writes: > >> Am 02.09.21 um 15:51 schrieb Hope Rouselle: >>> Just sharing a case of floating-point numbers. Nothing needed to be >>> solved or to be figured out. Just bringing up conversation. >>> (*) An introduction to me >>> I don't understand floating-point numbers from the inside out, but I >>> do >>> know how to work with base 2 and scientific notation. So the idea of >>> expressing a number as >>> mantissa * base^{power} >>> is not foreign to me. (If that helps you to perhaps instruct me on >>> what's going on here.) >>> (*) A presentation of the behavior >>> >>>>>> import sys >>>>>> sys.version >>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 >>> bit (AMD64)]' >>> >>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>>>>> sum(ls) >>> 39.599999999999994 >>> >>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>>>>> sum(ls) >>> 39.60000000000001 >>> All I did was to take the first number, 7.23, and move it to the >>> last >>> position in the list. (So we have a violation of the commutativity of >>> addition.) >> >> I believe it is not commutativity, but associativity, that is >> violated. > > Shall we take this seriously? (I will disagree, but that doesn't mean I > am not grateful for your post. Quite the contary.) It in general > violates associativity too, but the example above couldn't be referring > to associativity because the second sum above could not be obtained from > associativity alone. Commutativity is required, applied to five pairs > of numbers. How can I go from > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > > to > > 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? > > Perhaps only through various application of commutativity, namely the > ones below. (I omit the parentheses for less typing. I suppose that > does not create much trouble. There is no use of associativity below, > except for the intented omission of parentheses.) With the parens it will become more obvious. > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77 The sum is evaluated as (((7.23 + 8.41) + 6.15 + ...) For the first shift, you are correct that commutativity will result in (((8.41 + 7.23) + 6.15 + ...) But you can't go in one step to (((8.41 + 6.15) + 7.23 + ...) with the commutativity law alone. Instead, a sequence of associativity and commutativity is required to move the 7.23 out of the first pair of parentheses. And what I was trying to say, the commutative steps *are* equal in floating point arithmetics, whereas the associative steps are not. Christian From greg.ewing at canterbury.ac.nz Sat Sep 4 11:38:55 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 5 Sep 2021 03:38:55 +1200 Subject: on floating-point numbers In-Reply-To: <86o898e71d.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: On 5/09/21 2:42 am, Hope Rouselle wrote: > Here's what I did on this case. The REPL is telling me that > > 7.23 = 2035064081618043/281474976710656 If 7.23 were exactly representable, you would have got 723/1000. Contrast this with something that *is* exactly representable: >>> 7.875.as_integer_ratio() (63, 8) and observe that 7875/1000 == 63/8: >>> from fractions import Fraction >>> Fraction(7875,1000) Fraction(63, 8) In general, to find out whether a decimal number is exactly representable in binary, represent it as a ratio of integers where the denominator is a power of 10, reduce that to lowest terms, and compare with the result of as_integer_ratio(). -- Greg From nospam at dfs.com Sat Sep 4 12:31:31 2021 From: nospam at dfs.com (DFS) Date: Sat, 4 Sep 2021 12:31:31 -0400 Subject: Connecting python to DB2 database In-Reply-To: References: Message-ID: On 9/3/2021 9:50 AM, Chris Angelico wrote: > On Fri, Sep 3, 2021 at 11:37 PM DFS wrote: >> >> On 9/3/2021 1:47 AM, Chris Angelico wrote: >>> On Fri, Sep 3, 2021 at 3:42 PM DFS wrote: >>>> >>>> Having a problem with the DB2 connector >>>> >>>> test.py >>>> ---------------------------------------------------------------- >>>> import ibm_db_dbi >>>> connectstring = >>>> 'DATABASE=xxx;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;' >>>> conn = ibm_db_dbi.connect(connectstring,'','') >>>> >>>> curr = conn.cursor >>>> print(curr) >>> >>> According to PEP 249, what you want is conn.cursor() not conn.cursor. >>> >>> I'm a bit surprised as to the repr of that function though, which >>> seems to be this line from your output: >>> >>> >>> >>> I'd have expected it to say something like "method cursor of >>> Connection object", which would have been an immediate clue as to what >>> needs to be done. Not sure why the repr is so confusing, and that >>> might be something to report upstream. >>> >>> ChrisA >> >> >> Thanks. I must've done it right, using conn.cursor(), 500x. >> Bleary-eyed from staring at code too long I guess. > > Cool cool! Glad that's working. > >> Now can you get DB2 to accept ; as a SQL statement terminator like the >> rest of the world? They call it "An unexpected token"... >> > > Hmm, I don't know that the execute() method guarantees to allow > semicolons. Some implementations will strip a trailing semi, but they > usually won't allow interior ones, because that's a good way to worsen > SQL injection vulnerabilities. It's entirely possible - and within the > PEP 249 spec, I believe - for semicolons to be simply rejected. The default in the DB2 'Command Line Plus' tool is semicolons aren't "allowed". db2 => connect to SAMPLE db2 => SELECT COUNT(*) FROM STAFF; SQL0104N An unexpected token ";" was found following "COUNT(*) FROM STAFF". Expected tokens may include: "END-OF-STATEMENT". SQLSTATE=42601 db2 => SELECT COUNT(*) FROM STAFF 1 ----------- 35 1 record(s) selected. But I should've known you can set the terminator value: https://www.ibm.com/docs/en/db2/11.1?topic=clp-options Option : -t Description: This option tells the command line processor to use a semicolon (;) as the statement termination character. Default : OFF $ db2 -t turns it on in CommandLinePlus - and the setting applies to the DB-API code too. From tjreedy at udel.edu Sat Sep 4 15:22:17 2021 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 4 Sep 2021 15:22:17 -0400 Subject: Problem with python In-Reply-To: References: Message-ID: On 9/4/2021 2:27 PM, Igor Korot wrote: > Hi, ALL, > > [code] > igor at WaylandGnome ~/bakefile $ python > Python 3.9.6 (default, Aug 8 2021, 17:26:32) > [GCC 10.3.0] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> from distutils import sysconfig In 3.10, distutils and d.sysconfig are deprecated, with suggestions to use setuptools or sysconfig modules instead. >>>> print sysconfig.get_python_inc() > File "", line 1 > print sysconfig.get_python_inc() > ^ > SyntaxError: invalid syntax In interactive mode, print is not usually needed. >>> sysconfig.get_python_inc() 'C:\\Programs\\Python310\\include' -- Terry Jan Reedy From nospam at dfs.com Sat Sep 4 17:55:52 2021 From: nospam at dfs.com (DFS) Date: Sat, 4 Sep 2021 17:55:52 -0400 Subject: Help me split a string into elements Message-ID: Typical cases: lines = [('one\ntwo\nthree\n')] print(str(lines[0]).splitlines()) ['one', 'two', 'three'] lines = [('one two three\n')] print(str(lines[0]).split()) ['one', 'two', 'three'] That's the result I'm wanting, but I get data in a slightly different format: lines = [('one\ntwo\nthree\n',)] Note the comma after the string data, but inside the paren. splitlines() doesn't work on it: print(str(lines[0]).splitlines()) ["('one\\ntwo\\nthree\\n',)"] I've banged my head enough - can someone spot an easy fix? Thanks From nddtwentyone at gmail.com Sat Sep 4 18:15:12 2021 From: nddtwentyone at gmail.com (Neil) Date: 04 Sep 2021 22:15:12 GMT Subject: Help me split a string into elements References: Message-ID: <6133efee$0$704$14726298@news.sunsite.dk> DFS wrote: > Typical cases: > lines = [('one\ntwo\nthree\n')] > print(str(lines[0]).splitlines()) > ['one', 'two', 'three'] > > lines = [('one two three\n')] > print(str(lines[0]).split()) > ['one', 'two', 'three'] > > > That's the result I'm wanting, but I get data in a slightly different > format: > > lines = [('one\ntwo\nthree\n',)] > > Note the comma after the string data, but inside the paren. > splitlines() doesn't work on it: > > print(str(lines[0]).splitlines()) > ["('one\\ntwo\\nthree\\n',)"] > > > I've banged my head enough - can someone spot an easy fix? > > Thanks lines[0][0].splitlines() (You have a list containing a tuple containing the string you want to split up.) From nospam at dfs.com Sat Sep 4 18:16:46 2021 From: nospam at dfs.com (DFS) Date: Sat, 4 Sep 2021 18:16:46 -0400 Subject: Help me split a string into elements In-Reply-To: References: Message-ID: On 9/4/2021 5:55 PM, DFS wrote: > Typical cases: > ?lines = [('one\ntwo\nthree\n')] > ?print(str(lines[0]).splitlines()) > ?['one', 'two', 'three'] > > ?lines = [('one two three\n')] > ?print(str(lines[0]).split()) > ?['one', 'two', 'three'] > > > That's the result I'm wanting, but I get data in a slightly different > format: > > lines = [('one\ntwo\nthree\n',)] > > Note the comma after the string data, but inside the paren. splitlines() > doesn't work on it: > > print(str(lines[0]).splitlines()) > ["('one\\ntwo\\nthree\\n',)"] > > > I've banged my head enough - can someone spot an easy fix? > > Thanks I got it: lines = [('one\ntwo\nthree\n',)] print(str(lines[0][0]).splitlines()) ['one', 'two', 'three'] From wlfraed at ix.netcom.com Sat Sep 4 19:06:13 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sat, 04 Sep 2021 19:06:13 -0400 Subject: Problem with python References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: On Sat, 4 Sep 2021 22:41:12 +0200, "Peter J. Holzer" declaimed the following: >Python 3 to be time well spent in 2021, especially not to someone who >apparently just wants to report a bug to some unnamed project (whose >maintainers may or may not care about Python2 compatibility). > Given the nature of the error reported by the OP... It may be closer to state "whose maintainers may or may not care about" /Python3/ "compatibility"; the code appears to already be Python2 compatible -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From medina.rc at gmail.com Sat Sep 4 19:06:27 2021 From: medina.rc at gmail.com (Richard Medina) Date: Sat, 4 Sep 2021 16:06:27 -0700 (PDT) Subject: Select columns based on dates - Pandas In-Reply-To: References: <20210903165644.kpgyvpits5hjlmbc@gmail.com> <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com> Message-ID: On Friday, September 3, 2021 at 11:57:12 AM UTC-5, Martin Di Paola wrote: > You may want to reshape the dataset to a tidy format: Pandas works > better with that format. > > Let's assume the following dataset (this is what I understood from your > message): > > In [34]: df = pd.DataFrame({ > ...: 'Country': ['us', 'uk', 'it'], > ...: '01/01/2019': [10, 20, 30], > ...: '02/01/2019': [12, 22, 32], > ...: '03/01/2019': [14, 24, 34], > ...: }) > > In [35]: df > Out[35]: > Country 01/01/2019 02/01/2019 03/01/2019 > 0 us 10 12 14 > 1 uk 20 22 24 > 2 it 30 32 34 > > Then, reshape it to a tidy format. Notice how each row now represents > a single measure. > > In [43]: pd.melt(df, id_vars=['Country'], var_name='Date', > value_name='Cases') > Out[43]: > Country Date Cases > 0 us 01/01/2019 10 > 1 uk 01/01/2019 20 > 2 it 01/01/2019 30 > 3 us 02/01/2019 12 > 4 uk 02/01/2019 22 > 5 it 02/01/2019 32 > 6 us 03/01/2019 14 > 7 uk 03/01/2019 24 > 8 it 03/01/2019 34 > > I used strings to represent the dates but it is much handy work > with real date objects. > > In [44]: df2 = _ > In [45]: df2['Date'] = pd.to_datetime(df2['Date']) > > Now we can filter by date: > > In [50]: df2[df2['Date'] < '2019-03-01'] > Out[50]: > Country Date Cases > 0 us 2019-01-01 10 > 1 uk 2019-01-01 20 > 2 it 2019-01-01 30 > 3 us 2019-02-01 12 > 4 uk 2019-02-01 22 > 5 it 2019-02-01 32 > > With that you could create three dataframes, one per month. > > Thanks, > Martin. > On Thu, Sep 02, 2021 at 12:28:31PM -0700, Richard Medina wrote: > >Hello, forum, > >I have a data frame with covid-19 cases per month from 2019 - 2021 like a header like this: > > > >Country, 01/01/2019, 2/01/2019, 01/02/2019, 3/01/2019, ... 01/01/2021, 2/01/2021, 01/02/2021, 3/01/2021 > > > >I want to filter my data frame for columns of a specific month range of march to September of 2019, 2020, and 2021 only (three data frames). > > > >Any ideas? > >Thank you > > > > > >-- > >https://mail.python.org/mailman/listinfo/python-list Thank you, Martin, Is it possible to select/filter the dates (like columns from a date range) from the columns without reshaping them? Thank you for your answer. From grant.b.edwards at gmail.com Sat Sep 4 22:11:50 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 5 Sep 2021 02:11:50 -0000 (UTC) Subject: Problem with python References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> Message-ID: On 2021-09-04, Peter J. Holzer wrote: > On 2021-09-04 14:29:47 -0500, Igor Korot wrote: >> Will this syntax work in python 2? > > Yes. It's just a redundant pair of parentheses. Not really. With the parens, it doesn't produce the same results in 2.x unless you import the print function from the future: Python 3.9.6 (default, Aug 9 2021, 12:35:39) [GCC 10.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print(1,2) 1 2 Python 2.7.18 (default, Jul 18 2021, 14:51:54) [GCC 10.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print(1,2) (1, 2) Python 2.7.18 (default, Jul 18 2021, 14:51:54) [GCC 10.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from __future__ import print_function >>> print(1,2) 1 2 From rosuav at gmail.com Sat Sep 4 22:41:32 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 12:41:32 +1000 Subject: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically. In-Reply-To: References: <715e310e-56e3-4c97-a454-770c709ea706n@googlegroups.com> <82b49bdc-36f3-2429-06cd-bd370ba3dfd7@DancesWithMice.info> <5453b2a9-1a77-1aa1-e18e-f08904aaeabb@DancesWithMice.info> Message-ID: On Sun, Sep 5, 2021 at 12:39 PM Alan Gauld via Python-list wrote: > > On 03/09/2021 18:37, Chris Angelico wrote: > > >>>> Without DST the schools opened in the dark so all the kids > >>>> had to travel to school in the dark and the number of > >>>> traffic accidents while crossing roads jumped. > > > > Are you saying that you had DST in winter, or that, when summer *and* > > DST came into effect, there was more light at dawn? Because a *lot* of > > people confuse summer and DST, and credit DST with the natural effects > > of the season change. > > OK, I see the confusion. What I should point out was that the > experiment involved us staying on DST and not reverting to UTC > in the winter - that unified us with most of the EU apparently... > > So although I'm saying DST it was really the non-reversion from > DST to UTC that caused problems. Arguably, if we just stayed on > UTC and didn't have DST at all there would be no issue - except > we'd be an hour out of sync with the EU. (Post Brexit that may > not be seen as a problem!! :-) Oh, I see what you mean. When I complain about DST, I'm complaining about the repeated changes of UTC offset. Whether you either stay on UTC+0 or stay on UTC+1, it's basically the same, doesn't make a lot of difference. "Abolishing DST" and "staying on summer time permanently" are effectively the same. ChrisA From rosuav at gmail.com Sat Sep 4 22:49:11 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 12:49:11 +1000 Subject: on floating-point numbers In-Reply-To: <861r64ij9y.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <861r64ij9y.fsf@jevedi.com> Message-ID: On Sun, Sep 5, 2021 at 12:44 PM Hope Rouselle wrote: > > Chris Angelico writes: > > > On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle wrote: > >> > >> Just sharing a case of floating-point numbers. Nothing needed to be > >> solved or to be figured out. Just bringing up conversation. > >> > >> (*) An introduction to me > >> > >> I don't understand floating-point numbers from the inside out, but I do > >> know how to work with base 2 and scientific notation. So the idea of > >> expressing a number as > >> > >> mantissa * base^{power} > >> > >> is not foreign to me. (If that helps you to perhaps instruct me on > >> what's going on here.) > >> > >> (*) A presentation of the behavior > >> > >> >>> import sys > >> >>> sys.version > >> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 > >> bit (AMD64)]' > >> > >> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >> >>> sum(ls) > >> 39.599999999999994 > >> > >> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >> >>> sum(ls) > >> 39.60000000000001 > >> > >> All I did was to take the first number, 7.23, and move it to the last > >> position in the list. (So we have a violation of the commutativity of > >> addition.) > > > > It's not about the commutativity of any particular pair of operands - > > that's always guaranteed. > > Shall we take this seriously? It has to be about the commutativity of > at least one particular pair because it is involved with the > commutavitity of a set of pairs. If various pairs are involved, then at > least one is involved. IOW, it is about the commutativity of some pair > of operands and so it could not be the case that it's not about the > commutativity of any. (Lol. I hope that's not too insubordinate. I > already protested against a claim for associativity in this thread and > now I'm going for the king of the hill, for whom I have always been so > grateful!) No, that is not the case. Look at the specific pairs of numbers that get added. ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] >>> 7.23 + 8.41 15.64 >>> _ + 6.15 21.79 >>> _ + 2.31 24.099999999999998 >>> _ + 7.73 31.83 >>> _ + 7.77 39.599999999999994 And with the other list: ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] >>> 8.41 + 6.15 14.56 >>> _ + 2.31 16.87 >>> _ + 7.73 24.6 >>> _ + 7.77 32.370000000000005 >>> _ + 7.23 39.60000000000001 If commutativity is being violated, then there should be some situation where you could have written "7.73 + _" instead of "_ + 7.73" or equivalent, and gotten a different result. But that is simply not the case. What you are seeing is NOT commutativity, but the consequences of internal rounding, which is a matter of associativity. > Alright. Thanks so much for this example. Here's a new puzzle for me. > The REPL makes me think that both 21.79 and 2.31 *are* representable > exactly in Python's floating-point datatype because I see: > > >>> 2.31 > 2.31 > >>> 21.79 > 21.79 > > When I add them, the result obtained makes me think that the sum is > *not* representable exactly in Python's floating-point number: > > >>> 21.79 + 2.31 > 24.099999999999998 > > However, when I type 24.10 explicitly, the REPL makes me think that > 24.10 *is* representable exactly: > > >>> 24.10 > 24.1 > > I suppose I cannot trust the appearance of the representation? What's > really going on there? (Perhaps the trouble appears while Python is > computing the sum of the numbers 21.79 and 2.31?) Thanks so much! The representation is a conversion from the internal format into decimal digits. It is rounded for convenience of display, because you don't want it to look like this: >>> print(Fraction(24.10)) 3391773469363405/140737488355328 Since that's useless, the repr of a float rounds it to the shortest plausible number as represented in decimal digits. This has nothing to do with whether it is exactly representable, and everything to do with displaying things usefully in as many situations as possible :) ChrisA From rosuav at gmail.com Sat Sep 4 22:52:02 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 12:52:02 +1000 Subject: on floating-point numbers In-Reply-To: <86eea4h31o.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> Message-ID: On Sun, Sep 5, 2021 at 12:48 PM Hope Rouselle wrote: > > Chris Angelico writes: > > > On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle wrote: > >> > >> Hope Rouselle writes: > >> > >> > Just sharing a case of floating-point numbers. Nothing needed to be > >> > solved or to be figured out. Just bringing up conversation. > >> > > >> > (*) An introduction to me > >> > > >> > I don't understand floating-point numbers from the inside out, but I do > >> > know how to work with base 2 and scientific notation. So the idea of > >> > expressing a number as > >> > > >> > mantissa * base^{power} > >> > > >> > is not foreign to me. (If that helps you to perhaps instruct me on > >> > what's going on here.) > >> > > >> > (*) A presentation of the behavior > >> > > >> >>>> import sys > >> >>>> sys.version > >> > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 > >> > bit (AMD64)]' > >> > > >> >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >> >>>> sum(ls) > >> > 39.599999999999994 > >> > > >> >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >> >>>> sum(ls) > >> > 39.60000000000001 > >> > > >> > All I did was to take the first number, 7.23, and move it to the last > >> > position in the list. (So we have a violation of the commutativity of > >> > addition.) > >> > >> Suppose these numbers are prices in dollar, never going beyond cents. > >> Would it be safe to multiply each one of them by 100 and therefore work > >> with cents only? For instance > > > > Yes and no. It absolutely *is* safe to always work with cents, but to > > do that, you have to be consistent: ALWAYS work with cents, never with > > floating point dollars. > > > > (Or whatever other unit you choose to use. Most currencies have a > > smallest-normally-used-unit, with other currency units (where present) > > being whole number multiples of that minimal unit. Only in forex do > > you need to concern yourself with fractional cents or fractional yen.) > > > > But multiplying a set of floats by 100 won't necessarily solve your > > problem; you may have already fallen victim to the flaw of assuming > > that the numbers are represented accurately. > > Hang on a second. I see it's always safe to work with cents, but I'm > only confident to say that when one gives me cents to start with. In > other words, if one gives me integers from the start. (Because then, of > course, I don't even have floats to worry about.) If I'm given 1.17, > say, I am not confident that I could turn this number into 117 by > multiplying it by 100. And that was the question. Can I always > multiply such IEEE 754 dollar amounts by 100? > > Considering your last paragraph above, I should say: if one gives me an > accurate floating-point representation, can I assume a multiplication of > it by 100 remains accurately representable in IEEE 754? Humans usually won't give you IEEE 754 floats. What they'll usually give you is a text string. Let's say you ask someone to type in the prices of various items, the quantities thereof, and the shipping. You take strings like "1.17" (or "$1.17"), and you parse that into the integer 117. > Hm, I think I see what you're saying. You're saying multiplication and > division in IEEE 754 is perfectly safe --- so long as the numbers you > start with are accurately representable in IEEE 754 and assuming no > overflow or underflow would occur. (Addition and subtraction are not > safe.) > All operations are equally valid. Anything that causes rounding can cause loss of data, and that can happen with multiplication/division as well as addition/subtraction. But yes, with the caveats you give, everything is safe. ChrisA From rosuav at gmail.com Sat Sep 4 22:53:54 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 12:53:54 +1000 Subject: on floating-point numbers In-Reply-To: <86a6ksik0n.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6ksik0n.fsf@jevedi.com> Message-ID: On Sun, Sep 5, 2021 at 12:50 PM Hope Rouselle wrote: > > Christian Gollwitzer writes: > > > Am 02.09.21 um 15:51 schrieb Hope Rouselle: > >> Just sharing a case of floating-point numbers. Nothing needed to be > >> solved or to be figured out. Just bringing up conversation. > >> (*) An introduction to me > >> I don't understand floating-point numbers from the inside out, but I > >> do > >> know how to work with base 2 and scientific notation. So the idea of > >> expressing a number as > >> mantissa * base^{power} > >> is not foreign to me. (If that helps you to perhaps instruct me on > >> what's going on here.) > >> (*) A presentation of the behavior > >> > >>>>> import sys > >>>>> sys.version > >> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 > >> bit (AMD64)]' > >> > >>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >>>>> sum(ls) > >> 39.599999999999994 > >> > >>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >>>>> sum(ls) > >> 39.60000000000001 > >> All I did was to take the first number, 7.23, and move it to the > >> last > >> position in the list. (So we have a violation of the commutativity of > >> addition.) > > > > I believe it is not commutativity, but associativity, that is > > violated. > > Shall we take this seriously? (I will disagree, but that doesn't mean I > am not grateful for your post. Quite the contary.) It in general > violates associativity too, but the example above couldn't be referring > to associativity because the second sum above could not be obtained from > associativity alone. Commutativity is required, applied to five pairs > of numbers. How can I go from > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > > to > > 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? > > Perhaps only through various application of commutativity, namely the > ones below. (I omit the parentheses for less typing. I suppose that > does not create much trouble. There is no use of associativity below, > except for the intented omission of parentheses.) > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77 > = 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77 > = 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77 > = 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77 > = 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23. > Show me the pairs of numbers. You'll find that they are not the same numbers. Commutativity is specifically that a+b == b+a and you won't find any situation where that is violated. As soon as you go to three or more numbers, what you're doing is changing which numbers get added first, which is this: a + (b + c) != (a + b) + c and this can most certainly be violated due to intermediate rounding. ChrisA From rosuav at gmail.com Sat Sep 4 22:59:12 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 12:59:12 +1000 Subject: on floating-point numbers In-Reply-To: <868s0cfn87.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <4105d35c-cb9a-42d1-9985-f3c5b6ff6209n@googlegroups.com> <868s0cfn87.fsf@jevedi.com> Message-ID: On Sun, Sep 5, 2021 at 12:55 PM Hope Rouselle wrote: > > Julio Di Egidio writes: > > > On Thursday, 2 September 2021 at 16:51:24 UTC+2, Christian Gollwitzer wrote: > >> Am 02.09.21 um 16:49 schrieb Julio Di Egidio: > >> > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote: > >> >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote: > >> > > >> >>> 39.60000000000001 > >> >> > >> >> Welcome to the exciting world of roundoff error: > >> > > >> > Welcome to the exiting world of Usenet. > >> > > >> > *Plonk* > >> > >> Pretty harsh, isn't it? He gave a concise example of the same inaccuracy > >> right afterwards. > > > > And I thought you were not seeing my posts... > > > > Given that I have already given a full explanation, you guys, that you > > realise it or not, are simply adding noise for the usual pub-level > > discussion I must most charitably guess. > > > > Anyway, just my opinion. (EOD.) > > Which is certainly appreciated --- as a rule. Pub-level noise is pretty > much unavoidable in investigation, education. Being wrong is, too, > unavoidable in investigation, education. There is a point we eventually > publish at the most respected journals, but that's a whole other > interval of the time-line. IOW, chill out! :-D (Give us a C-k and meet > us up in the next thread. Oh, my, you're not a Gnus user: you are a > G2/1.0 user. That's pretty scary.) > I'm not a fan of the noise level in a pub, but I have absolutely no problem with arguing these points out. And everyone (mostly) in this thread is being respectful. I don't mind when someone else is wrong, especially since - a lot of the time - I'm wrong too (or maybe I'm the only one who's wrong). ChrisA From rosuav at gmail.com Sat Sep 4 23:00:19 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 13:00:19 +1000 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: On Sun, Sep 5, 2021 at 12:58 PM Greg Ewing wrote: > > On 5/09/21 2:42 am, Hope Rouselle wrote: > > Here's what I did on this case. The REPL is telling me that > > > > 7.23 = 2035064081618043/281474976710656 > > If 7.23 were exactly representable, you would have got > 723/1000. > > Contrast this with something that *is* exactly representable: > > >>> 7.875.as_integer_ratio() > (63, 8) > > and observe that 7875/1000 == 63/8: > > >>> from fractions import Fraction > >>> Fraction(7875,1000) > Fraction(63, 8) > > In general, to find out whether a decimal number is exactly > representable in binary, represent it as a ratio of integers > where the denominator is a power of 10, reduce that to lowest > terms, and compare with the result of as_integer_ratio(). > Or let Python do that work for you! >>> from fractions import Fraction >>> Fraction("7.875") == Fraction(7.875) True >>> Fraction("7.8") == Fraction(7.8) False ChrisA From rosuav at gmail.com Sat Sep 4 23:38:03 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 13:38:03 +1000 Subject: on floating-point numbers In-Reply-To: <86o898e71d.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: On Sun, Sep 5, 2021 at 1:04 PM Hope Rouselle wrote: > The same question in other words --- what's a trivial way for the REPL > to show me such cycles occur? > > >>>>>> 7.23.as_integer_ratio() > >>> (2035064081618043, 281474976710656) > > Here's what I did on this case. The REPL is telling me that > > 7.23 = 2035064081618043/281474976710656 > > If that were true, then 7.23 * 281474976710656 would have to equal > 2035064081618043. So I typed: > > >>> 7.23 * 281474976710656 > 2035064081618043.0 > > That agrees with the falsehood. I'm getting no evidence of the problem. > > When take control of my life out of the hands of misleading computers, I > calculate the sum: > > 844424930131968 > + 5629499534213120 > 197032483697459200 > ================== > 203506408161804288 > =/= 203506408161804300 > > How I can save the energy spent on manual verification? > What you've stumbled upon here is actually a neat elegance of floating-point, and an often-forgotten fundamental of it: rounding occurs exactly the same regardless of the scale. The number 7.23 is represented with a certain mantissa, and multiplying it by some power of two doesn't change the mantissa, only the exponent. So the rounding happens exactly the same, and it comes out looking equal! The easiest way, in Python, to probe this sort of thing is to use either fractions.Fraction or decimal.Decimal. I prefer Fraction, since a float is fundamentally a rational number, and you can easily see what's happening. You can construct a Fraction from a string, and it'll do what you would expect; or you can construct one from a float, and it'll show you what that float truly represents. It's often cleanest to print fractions out rather than just dumping them to the console, since the str() of a fraction looks like a fraction, but the repr() looks like a constructor call. >>> Fraction(0.25) Fraction(1, 4) >>> Fraction(0.1) Fraction(3602879701896397, 36028797018963968) If it looks like the number you put in, it was perfectly representable. If it looks like something of roughly that many digits, it's probably not the number you started with. ChrisA From rosuav at gmail.com Sat Sep 4 23:39:33 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 5 Sep 2021 13:39:33 +1000 Subject: Connecting python to DB2 database In-Reply-To: References: Message-ID: On Sun, Sep 5, 2021 at 1:26 PM DFS wrote: > > On 9/3/2021 9:50 AM, Chris Angelico wrote: > > On Fri, Sep 3, 2021 at 11:37 PM DFS wrote: > >> > >> On 9/3/2021 1:47 AM, Chris Angelico wrote: > >>> On Fri, Sep 3, 2021 at 3:42 PM DFS wrote: > >>>> > >>>> Having a problem with the DB2 connector > >>>> > >>>> test.py > >>>> ---------------------------------------------------------------- > >>>> import ibm_db_dbi > >>>> connectstring = > >>>> 'DATABASE=xxx;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;' > >>>> conn = ibm_db_dbi.connect(connectstring,'','') > >>>> > >>>> curr = conn.cursor > >>>> print(curr) > >>> > >>> According to PEP 249, what you want is conn.cursor() not conn.cursor. > >>> > >>> I'm a bit surprised as to the repr of that function though, which > >>> seems to be this line from your output: > >>> > >>> > >>> > >>> I'd have expected it to say something like "method cursor of > >>> Connection object", which would have been an immediate clue as to what > >>> needs to be done. Not sure why the repr is so confusing, and that > >>> might be something to report upstream. > >>> > >>> ChrisA > >> > >> > >> Thanks. I must've done it right, using conn.cursor(), 500x. > >> Bleary-eyed from staring at code too long I guess. > > > > Cool cool! Glad that's working. > > > >> Now can you get DB2 to accept ; as a SQL statement terminator like the > >> rest of the world? They call it "An unexpected token"... > >> > > > > Hmm, I don't know that the execute() method guarantees to allow > > semicolons. Some implementations will strip a trailing semi, but they > > usually won't allow interior ones, because that's a good way to worsen > > SQL injection vulnerabilities. It's entirely possible - and within the > > PEP 249 spec, I believe - for semicolons to be simply rejected. > > > The default in the DB2 'Command Line Plus' tool is semicolons aren't > "allowed". > Yeah, but that's a REPL feature. In Python, the end of a command is signalled by the end of the string; if you want a multiline command, you have a string with multiple lines in it. Command line SQL has to either force you to one line, or have some means of distinguishing between "more text coming" and "here's a command, run it". ChrisA From hjp-python at hjp.at Sun Sep 5 17:41:40 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 5 Sep 2021 23:41:40 +0200 Subject: on floating-point numbers In-Reply-To: <86a6ksik0n.fsf@jevedi.com> References: <86bl5bm6go.fsf@jevedi.com> <86a6ksik0n.fsf@jevedi.com> Message-ID: On 2021-09-04 09:48:40 -0300, Hope Rouselle wrote: > Christian Gollwitzer writes: > > Am 02.09.21 um 15:51 schrieb Hope Rouselle: > >>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77] > >>>>> sum(ls) > >> 39.599999999999994 > >> > >>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23] > >>>>> sum(ls) > >> 39.60000000000001 > >> All I did was to take the first number, 7.23, and move it to the > >> last > >> position in the list. (So we have a violation of the commutativity of > >> addition.) > > > > I believe it is not commutativity, but associativity, that is > > violated. I agree. > Shall we take this seriously? (I will disagree, but that doesn't mean I > am not grateful for your post. Quite the contary.) It in general > violates associativity too, but the example above couldn't be referring > to associativity because the second sum above could not be obtained from > associativity alone. Commutativity is required, applied to five pairs > of numbers. How can I go from > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > > to > > 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? Simple: >>> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 39.599999999999994 >>> 7.23 + (8.41 + 6.15 + 2.31 + 7.73 + 7.77) 39.60000000000001 Due to commutativity, this is the same as >>> (8.41 + 6.15 + 2.31 + 7.73 + 7.77) + 7.23 39.60000000000001 So commutativity is preserved but associativity is lost. (Of course a single example doesn't prove that this is always the case, but it can be seen from the guarantees that IEEE-754 arithmetic gives you that this is actually the case). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sun Sep 5 18:13:16 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 6 Sep 2021 00:13:16 +0200 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: On 2021-09-05 03:38:55 +1200, Greg Ewing wrote: > If 7.23 were exactly representable, you would have got > 723/1000. > > Contrast this with something that *is* exactly representable: > > >>> 7.875.as_integer_ratio() > (63, 8) > > and observe that 7875/1000 == 63/8: > > >>> from fractions import Fraction > >>> Fraction(7875,1000) > Fraction(63, 8) > > In general, to find out whether a decimal number is exactly > representable in binary, represent it as a ratio of integers > where the denominator is a power of 10, reduce that to lowest > terms, ... and check if the denominator is a power of two. If it isn't (e.g. 1000 == 2**3 * 5**3) then the number is not exactly representable as a binary floating point number. More generally, if the prime factorization of the denominator only contains prime factors which are also prime factors of your base, then the number can be exactle represented (unless either the denominator or the enumerator get too big). So, for base 10 (2*5), all numbers which have only powers of 2 and 5 in the denominator (e.g 1/10 == 1/(2*5), 1/8192 == 1/2**13, 1/1024000 == 1/(2**13 * 5**3)) can represented exactly, but those with other prime factors (e.g. 1/3, 1/7, 1/24576 == 1/(2**13 * 3), 1/1024001 == 1/(11 * 127 * 733)) cannot. Similarly, for base 12 (2*2*3) numbers with 2 and 3 in the denominator can be represented and for base 60 (2*2*3*5), numbers with 2, 3 and 5. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sun Sep 5 18:19:37 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 6 Sep 2021 00:19:37 +0200 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> Message-ID: On 2021-09-04 10:01:23 -0400, Richard Damon wrote: > On 9/4/21 9:40 AM, Hope Rouselle wrote: > > Hm, I think I see what you're saying. You're saying multiplication and > > division in IEEE 754 is perfectly safe --- so long as the numbers you > > start with are accurately representable in IEEE 754 and assuming no > > overflow or underflow would occur. (Addition and subtraction are not > > safe.) > > > > Addition and Subtraction are just as safe, as long as you stay within > the precision limits. That depends a lot on what you call "safe", a * b / a will always be very close to b (unless there's an over- or underflow), but a + b - a can be quite different from b. In general when analyzing a numerical algorithm you have to pay a lot more attention to addition and subtraction than to multiplication and division. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From Richard at damon-family.org Sun Sep 5 23:21:14 2021 From: Richard at damon-family.org (Richard Damon) Date: Sun, 5 Sep 2021 23:21:14 -0400 Subject: on floating-point numbers In-Reply-To: References: Message-ID: <479224BA-0EFA-4DC3-893B-EBF1ED4C3CD1@damon-family.org> > On Sep 5, 2021, at 6:22 PM, Peter J. Holzer wrote: > > ?On 2021-09-04 10:01:23 -0400, Richard Damon wrote: >>> On 9/4/21 9:40 AM, Hope Rouselle wrote: >>> Hm, I think I see what you're saying. You're saying multiplication and >>> division in IEEE 754 is perfectly safe --- so long as the numbers you >>> start with are accurately representable in IEEE 754 and assuming no >>> overflow or underflow would occur. (Addition and subtraction are not >>> safe.) >>> >> >> Addition and Subtraction are just as safe, as long as you stay within >> the precision limits. > > That depends a lot on what you call "safe", > > a * b / a will always be very close to b (unless there's an over- or > underflow), but a + b - a can be quite different from b. > > In general when analyzing a numerical algorithm you have to pay a lot > more attention to addition and subtraction than to multiplication and > division. > > hp > > -- Yes, it depends on your definition of safe. If ?close? is good enough then multiplication is probably safer as the problems are in more extreme cases. If EXACT is the question, addition tends to be better. To have any chance, the numbers need to be somewhat low ?precision?, which means the need to avoid arbitrary decimals. Once past that, as long as the numbers are of roughly the same magnitude, and are the sort of numbers you are apt to just write, you can tend to add a lot of them before you get enough bits to accumulate to have a problem. With multiplication, every multiply roughly adds the number of bits of precision, so you quickly run out, and one divide will have a chance to just end the process. Remember, the question came up because the sum was?t associative because of fractional bits. That points to thinking of exact operations, and addition does better at that. From avigross at verizon.net Mon Sep 6 11:22:07 2021 From: avigross at verizon.net (Avi Gross) Date: Mon, 6 Sep 2021 11:22:07 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: <86tuj3kpin.fsf@jevedi.com> References: <86r1edlqxw.fsf@jevedi.com> <86tuj3kpin.fsf@jevedi.com> Message-ID: <057c01d7a332$f45b37b0$dd11a710$@verizon.net> For some people the "while true" method seems reasonable but it has a problem if the internal body does not have some guarantee of an exit. And that exit can be subtle. Many have mentioned ways an end condition can fail due to rounding errors not being exactly equal to what you are looking for, or an asymptotic process that approaches zero with a cutoff that is thus never reached. But I have seen some cases where you want a process or thread to run indefinitely until it is killed by another process or you reboot. You could of course use something like flag="green" and have your while keep testing that value but that is just wasted motion. Of course, properly used, it does help explain to the reader what your intention is, if done properly. It also can confuse if you put in "while 5 < 6" or other weird ways to say do this indefinitely. I think part of this conversation is about programming styles and what we should teach new students versus those who are trying to find more creative ways to do things or more efficiently or more generality. Sometimes a more direct method may be reasonable. On another forum, I saw someone write somewhat lengthy and redundant code on how to deal with partitioning data into three clusters randomly. My algorithm was general and handled any number of N clusters and since N often does not cleanly divide the number of items, has to place the remaining items into one of the N clusters or distribute them one at a time into some to make them almost even in size. His solution was to repeat large sections of code, three times, with some modification, for the cases where the remainder (modulo N) was 0, 1 or 2. The latter method was longer but in an odd way, much easier to understand. It boiled down to, if you have one extra put it here. If you have two, put one here and one there. My method had all kinds of bells and whistles that this specific case did not need. Let me end with asking if anyone has ever seen a mild variant of this: ... # code while false apologize() ... # more code -----Original Message----- From: Python-list On Behalf Of Hope Rouselle Sent: Thursday, September 2, 2021 10:42 AM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice David Raymond writes: >> def how_many_times(): >> x, y = 0, 1 >> c = 0 >> while x != y: >> c = c + 1 >> x, y = roll() >> return c, (x, y) > > Since I haven't seen it used in answers yet, here's another option > using our new walrus operator > > def how_many_times(): > roll_count = 1 > while (rolls := roll())[0] != rolls[1]: > roll_count += 1 > return (roll_count, rolls) That's nice, although it doesn't seem more readable to a novice seeing a while for the first time, seeing a loop for the first time, than that while-True version. In fact, I think the while-True is the clearest so far. But it's always nice to spot a walrus in the wild! (If you're somewhere safe, that is.) -- https://mail.python.org/mailman/listinfo/python-list From grant.b.edwards at gmail.com Sun Sep 5 10:47:05 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 5 Sep 2021 14:47:05 -0000 (UTC) Subject: Problem with python References: <66abcd73-b5eb-8646-a107-fcd673c2f3fb@DancesWithMice.info> <86czpoceek.fsf@jevedi.com> Message-ID: On 2021-09-04, Hope Rouselle wrote: > Igor Korot writes: > >> Hi, >> Will this syntax work in python 2? > > If you say > > print(something) > > it works in both. But it doesn't always work the _same_ in both. If you're expecting some particular output, then one or the other might not won't "work". > So, stick to this syntax. Only if you import print_function from __future__ in 2.x -- Grant From wlfraed at ix.netcom.com Sun Sep 5 12:50:21 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 05 Sep 2021 12:50:21 -0400 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> Message-ID: On Sat, 4 Sep 2021 12:27:55 -0500, "Michael F. Stemper" declaimed the following: > >Kernighan and Ritchie agree(d) with you. Per _The C Programming >Language__: > Experience shows that do-while is much less used that while > and for. > And just for confusion, consider languages with "repeat / until"... "do / while" repeats so long as the condition evaluates to "true"; "repeat / until" /exits/ when the condition evaluates to "true". Then... there is Ada... While one is most likely to encounter constructs: for ix in start..end loop ... end loop; and while condition loop ... end loop; the core construct is just a bare loop ... end loop; which, with the "exit when condition" statement allows low-level emulation of any loop... (same as Python "if condition: break" loop -- "while" loop exit when not condition; ... end loop; loop -- "repeat / until" ... exit when condition; end loop; loop -- split ... exit when condition; ... end loop; {I'm not going to do the "for" loop, but one can easily manage initialization/increment/test statements}. To really get your mind blown, look at loops in REXX... do while condition ... end do until condition /* note that the termination parameter is listed at top, */ /* but takes effect at the bottom, so always one pass */ ... end do forever ... if condition then leave ... end do idx = start to end /* optional: by increment */ ... end do idx = 1 by increment for repetitions ... end AND worse! You can combine them... do idx = start for repetitions while condition1 until condition2 ... end {I need to check if both while and until can be in the same statement} -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Sun Sep 5 13:22:58 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 05 Sep 2021 13:22:58 -0400 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> Message-ID: On Sat, 04 Sep 2021 10:40:35 -0300, Hope Rouselle declaimed the following: >course, I don't even have floats to worry about.) If I'm given 1.17, >say, I am not confident that I could turn this number into 117 by >multiplying it by 100. And that was the question. Can I always >multiply such IEEE 754 dollar amounts by 100? > HOW are you "given" that 1.17? If that is coming from some user-readable source (keyboard entry, text-based file [CSV/TSV, even SYLK]) you do NOT have a number -- you have a string, which needs to be converted by some algorithm. For money, the best solution, again, is to use the Decimal module and feed the /string/ to the initialization call. If you want to do it yourself, to get a scaled integer, you will have to code a parser/converter. * strip extraneous punctuation ($, etc -- but not comma, decimal point, or + and -) * strip any grouping separators (commas, but beware, some countries group using a period -- "1.234,56" vs "1,234.56"). "1,234.56" => "1234.56" * ensure there is a decimal point (again, you may have to convert a comma to decimal point), if not append a "." to the input * append enough 0s to the end to ensure you have whatever scale factor you are using behind the decimal point (as mentioned M$ Excel money type uses four places) "1234.56" => "1234.5600" * remove the decimal marker. "1234.5600" => "12345600" * convert to native integer. int("12345600") => 12345600 [as integer] {may fail if +/- are not in the required position for Python} If the number is coming from some binary file format, it is already too late. And you might need to study any DBMS being used. Some transfer values as text strings, and reconvert to DBMS numeric types on receipt. Make sure the DBMS is using a decimal number type and not a floating type for the field (which leaves out SQLite3 which will store anything in any field, but uses some slightly obscure logic to determine what conversion is done) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From grant.b.edwards at gmail.com Sun Sep 5 18:32:51 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 5 Sep 2021 22:32:51 -0000 (UTC) Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: On 2021-09-05, Peter J. Holzer wrote: > On 2021-09-05 03:38:55 +1200, Greg Ewing wrote: >> If 7.23 were exactly representable, you would have got >> 723/1000. >> >> Contrast this with something that *is* exactly representable: >> >> >>> 7.875.as_integer_ratio() >> (63, 8) >> >> and observe that 7875/1000 == 63/8: >> >> >>> from fractions import Fraction >> >>> Fraction(7875,1000) >> Fraction(63, 8) >> >> In general, to find out whether a decimal number is exactly >> representable in binary, represent it as a ratio of integers where >> the denominator is a power of 10, reduce that to lowest terms, > > ... and check if the denominator is a power of two. If it isn't > (e.g. 1000 == 2**3 * 5**3) then the number is not exactly > representable as a binary floating point number. > > More generally, if the prime factorization of the denominator only > contains prime factors which are also prime factors of your base, > then the number can be exactle represented (unless either the > denominator or the enumerator get too big). And once you understand that, ignore it and write code under the assumumption that nothing can be exactly represented in floating point. If you like, you can assume that 0 can be exactly represented without getting into too much trouble as long as it's a literal constant value and not the result of any run-time FP operations. If you want to live dangerously, you can assume that integers with magnitude less than a million can be exactly represented. That assumption is true for all the FP representations I've ever used, but once you start depending on it, you're one stumble from the edge of the cliff. -- Grant From hrouselle at jevedi.com Mon Sep 6 12:58:16 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Mon, 06 Sep 2021 13:58:16 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: <86eea1bpzr.fsf@jevedi.com> "Peter J. Holzer" writes: > On 2021-09-05 03:38:55 +1200, Greg Ewing wrote: >> If 7.23 were exactly representable, you would have got >> 723/1000. >> >> Contrast this with something that *is* exactly representable: >> >> >>> 7.875.as_integer_ratio() >> (63, 8) >> >> and observe that 7875/1000 == 63/8: >> >> >>> from fractions import Fraction >> >>> Fraction(7875,1000) >> Fraction(63, 8) >> >> In general, to find out whether a decimal number is exactly >> representable in binary, represent it as a ratio of integers >> where the denominator is a power of 10, reduce that to lowest >> terms, > > ... and check if the denominator is a power of two. If it isn't (e.g. > 1000 == 2**3 * 5**3) then the number is not exactly representable as a > binary floating point number. > > More generally, if the prime factorization of the denominator only > contains prime factors which are also prime factors of your base, then > the number can be exactle represented (unless either the denominator or > the enumerator get too big). So, for base 10 (2*5), all numbers which > have only powers of 2 and 5 in the denominator (e.g 1/10 == 1/(2*5), > 1/8192 == 1/2**13, 1/1024000 == 1/(2**13 * 5**3)) can represented > exactly, but those with other prime factors (e.g. 1/3, 1/7, > 1/24576 == 1/(2**13 * 3), 1/1024001 == 1/(11 * 127 * 733)) cannot. > Similarly, for base 12 (2*2*3) numbers with 2 and 3 in the denominator > can be represented and for base 60 (2*2*3*5), numbers with 2, 3 and 5. Very grateful to these paragraphs. They destroy all the mystery. From hongyi.zhao at gmail.com Sun Sep 5 03:36:44 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Sun, 5 Sep 2021 00:36:44 -0700 (PDT) Subject: Re-design the position of the RPROMPT string. Message-ID: I forked and made some improvements to the ariadne package [1]. I noticed that the current RPROMPT line is composed by percol.view.PROMPT [2] and percol.view.__class__.RPROMPT [3], as shown below: X10DAi-00 (M-h/M-n)> M-m:string Path:C-d Local:C-l Unique:M-r Exit0:M-t Fold:F1,F2,F3 But this leaves very little room for the search string used to filter the history. So, I want to split the RPROMPT line into two lines as follows: M-m:string Path:C-d Local:C-l Unique:M-r Exit0:M-t Fold:F1,F2,F3 X10DAi-00 (M-h/M-n)> But I've tried a lot and still don't know how to do it. Any hints will be highly appreciated. Regareds, HY [1] https://github.com/hongyi-zhao/ariadne [2] https://github.com/hongyi-zhao/ariadne/blob/ec864434b8a947b801f019e84c827c3a96dcf7e4/rc.py#L56 [3] https://github.com/hongyi-zhao/ariadne/blob/ec864434b8a947b801f019e84c827c3a96dcf7e4/rc.py#L73 From nospam at please.ty Sun Sep 5 07:41:01 2021 From: nospam at please.ty (jak) Date: Sun, 5 Sep 2021 13:41:01 +0200 Subject: CPython / Decimal and bit length of value. References: <1771811707.3518631.1630699780851.ref@mail.yahoo.com> <1771811707.3518631.1630699780851@mail.yahoo.com> Message-ID: Il 03/09/2021 22:09, Nacnud Nac ha scritto: > Hi, > Is there a quick way to get the number of bits required to store the value in a Decimal class? > What obvious thing am I missing? I'm working with really large integers, say, in the order of 5_000_000 of ASCII base 10 digits. > It seems the function mpd_sizeinbase would be a nice thing to be able to call..... > It'd be nice to just be able to tell the "size of data", or something like that, that was stored in the *data? I note there is len "field" in the structure mpd_t > Sorry for the dumb question.... > Thanks,Duncan > to semplfy the example I'll use the value 1000000: value="1000000" exponent in base 10 is len(value) - 1 # (1 * 10^6) now need change from base 10 to base 2: newexp = 6 / log(2) # 19 (more or less) you will need newexp + 1 bits to represent the number: 2^20 = 1.048.576 hope helps From hrouselle at jevedi.com Mon Sep 6 17:10:21 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Mon, 06 Sep 2021 18:10:21 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: <86pmtl9zr6.fsf@jevedi.com> Chris Angelico writes: > On Sun, Sep 5, 2021 at 1:04 PM Hope Rouselle wrote: >> The same question in other words --- what's a trivial way for the REPL >> to show me such cycles occur? >> >> >>>>>> 7.23.as_integer_ratio() >> >>> (2035064081618043, 281474976710656) >> >> Here's what I did on this case. The REPL is telling me that >> >> 7.23 = 2035064081618043/281474976710656 >> >> If that were true, then 7.23 * 281474976710656 would have to equal >> 2035064081618043. So I typed: >> >> >>> 7.23 * 281474976710656 >> 2035064081618043.0 >> >> That agrees with the falsehood. I'm getting no evidence of the problem. >> >> When take control of my life out of the hands of misleading computers, I >> calculate the sum: >> >> 844424930131968 >> + 5629499534213120 >> 197032483697459200 >> ================== >> 203506408161804288 >> =/= 203506408161804300 >> >> How I can save the energy spent on manual verification? > > What you've stumbled upon here is actually a neat elegance of > floating-point, and an often-forgotten fundamental of it: rounding > occurs exactly the same regardless of the scale. The number 7.23 is > represented with a certain mantissa, and multiplying it by some power > of two doesn't change the mantissa, only the exponent. So the rounding > happens exactly the same, and it comes out looking equal! That's insightful. Thanks! > The easiest way, in Python, to probe this sort of thing is to use > either fractions.Fraction or decimal.Decimal. I prefer Fraction, since > a float is fundamentally a rational number, and you can easily see > what's happening. You can construct a Fraction from a string, and > it'll do what you would expect; or you can construct one from a float, > and it'll show you what that float truly represents. > > It's often cleanest to print fractions out rather than just dumping > them to the console, since the str() of a fraction looks like a > fraction, but the repr() looks like a constructor call. > >>>> Fraction(0.25) > Fraction(1, 4) >>>> Fraction(0.1) > Fraction(3602879701896397, 36028797018963968) > > If it looks like the number you put in, it was perfectly > representable. If it looks like something of roughly that many digits, > it's probably not the number you started with. That's pretty, pretty nice. It was really what I was looking for. -- You're the best ``little lord of local nonsense'' I've ever met! :-D (Lol. The guy is kinda stressed out! Plonk, plonk, plonk. EOD.) From avigross at verizon.net Mon Sep 6 19:38:17 2021 From: avigross at verizon.net (Avi Gross) Date: Mon, 6 Sep 2021 19:38:17 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> Message-ID: <004401d7a378$44d43580$ce7ca080$@verizon.net> I actually like it if a language lets you spell out your intention, although adding many keywords is not a plus. So, yes something like: loop ... end loop; Is appealing as it makes clear the decision on when to exit the loop must be within the loop (or till something kills ...) In languages like C/C++ there are people who make up macros like: #define INDEFINITELY_LOOP while (true) Or something like that and then allow the preprocessor to replace INDEFINITELY_LOOP with valid C code. So, how to do something like that in python, is a challenge left to the user ? Bottom line I find is it is a great idea to write comments as even if you are the one reviewing the code much later, you may struggle to figure out what you did and why. -----Original Message----- From: Python-list On Behalf Of Dennis Lee Bieber Sent: Sunday, September 5, 2021 12:50 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice On Sat, 4 Sep 2021 12:27:55 -0500, "Michael F. Stemper" declaimed the following: > >Kernighan and Ritchie agree(d) with you. Per _The C Programming >Language__: > Experience shows that do-while is much less used that while > and for. > And just for confusion, consider languages with "repeat / until"... "do / while" repeats so long as the condition evaluates to "true"; "repeat / until" /exits/ when the condition evaluates to "true". Then... there is Ada... While one is most likely to encounter constructs: for ix in start..end loop ... end loop; and while condition loop ... end loop; the core construct is just a bare loop ... end loop; which, with the "exit when condition" statement allows low-level emulation of any loop... (same as Python "if condition: break" loop -- "while" loop exit when not condition; ... end loop; loop -- "repeat / until" ... exit when condition; end loop; loop -- split ... exit when condition; ... end loop; {I'm not going to do the "for" loop, but one can easily manage initialization/increment/test statements}. To really get your mind blown, look at loops in REXX... do while condition ... end do until condition /* note that the termination parameter is listed at top, */ /* but takes effect at the bottom, so always one pass */ ... end do forever ... if condition then leave ... end do idx = start to end /* optional: by increment */ ... end do idx = 1 by increment for repetitions ... end AND worse! You can combine them... do idx = start for repetitions while condition1 until condition2 ... end {I need to check if both while and until can be in the same statement} -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list From avigross at verizon.net Mon Sep 6 20:11:41 2021 From: avigross at verizon.net (Avi Gross) Date: Mon, 6 Sep 2021 20:11:41 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86tuj3kpin.fsf@jevedi.com> <057c01d7a332$f45b37b0$dd11a710$@verizon.net> Message-ID: <006401d7a37c$ef9301a0$ceb904e0$@verizon.net> Let me add something, Stefan. Some people just want to get a job done. For this person, he had a very specific need for a one-time project where the rest of it was understood but one small step was confusing. Frankly, he could have done it faster by opening a text editor on something like a CSV file and flipped some (actual) three-sided coins while manually doing a cut and past of lines into three new files and near the end, make sure the last few only went into files with a missing row or two. I have too often had people ask me for programming help and when I suggested they learn how to do it or let me make a more careful piece of code that checks for errors or will easily work under various changed conditions, guess what I usually encounter? Many just want it DONE. They often sheepishly come back some time later with some variant of the same need that mysteriously does not work with code they asked to be designed to just work in one case! Often they hack away at the code in odd ways first and then come to me to have it fixed. But there are costs to generality and creating functions with dozens of optional arguments and that handle a wide variety of inputs and thus constantly are checking what types they are working with and perhaps converting to others or need to create objects with lots of dunders, can also make it error prone. Looking at our forever loop discussion, how often are the contents of the loop written with multiple terminating parts within the loop body that get complex? I mean lots of break or continue statements all over the place with multi-part if statements. Some can be rewritten as a more standard loop with a condition like "while (a < 5 && found_it == FALSE || ( ... )) ..." and often in the body you need if statements that let you skip the rest if found_it is true. It can be a mess either way. At times you need to consider rewriting it from scratch. This especially happens as requirements keep changing. Ages ago we had code that processed MTA headers and every time we had a meeting of standards bodies, we kept adding ever more headers and keywords that often required some existing code to also look at other headers that now might be present as they might change what you did in existing code locations. I eventually suggested a rewrite and it turned out to be more compact now that we evaluated some things first, rather than within many existing parts. Has anyone mentioned the really stupid looking forever loop in languages with the other style of for loop? for ( ; ;) All that generally was, was an initialization command before a while and so on but here all three parts were null, so why use it? And in the python version, has anyone made a generator that returned NULL or the like so you can say uselessly: for ( _ in forever() ) ... -----Original Message----- From: Python-list On Behalf Of Stefan Ram Sent: Monday, September 6, 2021 12:34 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice "Avi Gross" writes: >For some people the "while true" method seems reasonable but it has a >problem if the internal body does not have some guarantee of an exit. >And A programming error where the condition used does not express the intent of the programmer can happen with any kind of while loop. >help explain to the reader what your intention is, if done properly. It >also can confuse if you put in "while 5 < 6" or other weird ways to say >do this indefinitely. If any reader cannot understand "while True:", it's not a problem with the source code. When someone writes "while True:", it is clearly his intention that the following indented code is repeated until it's exited by some means, which means is not the while condition. >His solution was to repeat large sections of code, three times, with >some modification, for the cases where the remainder (modulo N) was 0, 1 or 2. This is typical of - beginners who just do not know the means of the language to reduce redundancy yet, - less experience programmers, who know the language but still are not able to apply features to reduce redundancy always, - premature publication of unfinished code where the redundancy has not yet been reduced, or - programmers who do not have the personal means to climb to the level of abstraction needed to remove the redundancy in a specific case. And, I think you wrote that too, sometimes less smart code is more readable or maintainable. Maybe three very similar blocks are repeated literally because future modifications are expected that will make them less similar. -- https://mail.python.org/mailman/listinfo/python-list From 2QdxY4RzWzUUiLuE at potatochowder.com Mon Sep 6 20:27:52 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Mon, 6 Sep 2021 17:27:52 -0700 Subject: on writing a while loop for rolling two dice In-Reply-To: <006401d7a37c$ef9301a0$ceb904e0$@verizon.net> References: <86r1edlqxw.fsf@jevedi.com> <86tuj3kpin.fsf@jevedi.com> <057c01d7a332$f45b37b0$dd11a710$@verizon.net> <006401d7a37c$ef9301a0$ceb904e0$@verizon.net> Message-ID: On 2021-09-06 at 20:11:41 -0400, Avi Gross via Python-list wrote: > And in the python version, has anyone made a generator that returned > NULL or the like so you can say uselessly: > > for ( _ in forever() ) ... while "forever": ... From avigross at verizon.net Mon Sep 6 20:41:20 2021 From: avigross at verizon.net (Avi Gross) Date: Mon, 6 Sep 2021 20:41:20 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86tuj3kpin.fsf@jevedi.com> <057c01d7a332$f45b37b0$dd11a710$@verizon.net> <006401d7a37c$ef9301a0$ceb904e0$@verizon.net> Message-ID: <008b01d7a381$13c671c0$3b535540$@verizon.net> I hate to quibble but as almost anything in Python can evaluate to being truthy, a command like while "never" evaluates to true as the string is not empty. I meant a generator like >>> def boring(): while True: yield() >>> for _ in boring(): print("repeating ...") The above gives me a nice infinite loop, with the second one looking like a normal loop but actually doing nothing much. -----Original Message----- From: Python-list On Behalf Of 2QdxY4RzWzUUiLuE at potatochowder.com Sent: Monday, September 6, 2021 8:28 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice On 2021-09-06 at 20:11:41 -0400, Avi Gross via Python-list wrote: > And in the python version, has anyone made a generator that returned > NULL or the like so you can say uselessly: > > for ( _ in forever() ) ... while "forever": ... -- https://mail.python.org/mailman/listinfo/python-list From wlfraed at ix.netcom.com Mon Sep 6 22:48:17 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Mon, 06 Sep 2021 22:48:17 -0400 Subject: Non sequitur: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86tuj3kpin.fsf@jevedi.com> <057c01d7a332$f45b37b0$dd11a710$@verizon.net> <006401d7a37c$ef9301a0$ceb904e0$@verizon.net> Message-ID: On Mon, 6 Sep 2021 20:11:41 -0400, Avi Gross via Python-list declaimed the following: >changing. Ages ago we had code that processed MTA headers and every time we >had a meeting of standards bodies, we kept adding ever more headers and Why did my mind immediately flash on "The Man Who Never Returned"? (Especially as you describe just the opposite -- many returns ) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Mon Sep 6 22:55:40 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Mon, 06 Sep 2021 22:55:40 -0400 Subject: Non sequitur: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86tuj3kpin.fsf@jevedi.com> <057c01d7a332$f45b37b0$dd11a710$@verizon.net> <006401d7a37c$ef9301a0$ceb904e0$@verizon.net> Message-ID: RESEND with clarification! On Mon, 6 Sep 2021 20:11:41 -0400, Avi Gross via Python-list declaimed the following: >changing. Ages ago we had code that processed MTA headers and every time we >had a meeting of standards bodies, we kept adding ever more headers and Why did my mind immediately flash on "The Man Who Never Returned"? (AKA: "M.T.A.", Kingston Trio song) (Especially as you describe just the opposite -- many returns ) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From avigross at verizon.net Tue Sep 7 00:05:36 2021 From: avigross at verizon.net (Avi Gross) Date: Tue, 7 Sep 2021 00:05:36 -0400 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: <017701d7a39d$9ccd1d40$d66757c0$@verizon.net> It has been nearly three decades since I have had to write in C, Stefan, but what I suggested jokingly is quite mild compared to what the winners of the obfuscated C Contest do: https://www.ioccc.org/ Time for me to drop out of this thread. Personally I fully agree uses of "while' as described are perfectly understandable. Features that sophisticated programmers love tend to confuse novices. I recall my exposure to PERL where weird things seemed to just happen with no rhyme or reason or connections. Turned out just about everything puts things into or takes them out of hidden variables so much of the time, a string of commands just does what might be expected. Another variant on the elusive concept of a pipeline. But all the nice gimmicks and tricks make novices a bit puzzled. On the other hand, you can still write most programs the old fashioned way and sort of start normal then head off into hyperspace at warp speed. Python too has a way to write fairly unsophisticated programs as well as idioms and methods that rapidly become hard to comprehend. -----Original Message----- From: Python-list On Behalf Of Stefan Ram Sent: Monday, September 6, 2021 7:49 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice "Avi Gross" writes: > In languages like C/C++ there are people who make up macros like: >#define INDEFINITELY_LOOP while (true) >Or something like that and then allow the preprocessor to replace >INDEFINITELY_LOOP with valid C code. Those usually are beginners. >So, how to do something like that in python, is a challenge left to the >user Such a use of macros is frowned upon by most C programmers, because it renders the code unreadable. "while(1)" in C or "while True:" in Python is perfectly clear. Don't fix it if it ain't broke! -- https://mail.python.org/mailman/listinfo/python-list From greg.ewing at canterbury.ac.nz Tue Sep 7 00:08:04 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 7 Sep 2021 16:08:04 +1200 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On 7/09/21 11:38 am, Avi Gross wrote: > #define INDEFINITELY_LOOP while (true) > > So, how to do something like that in python, is a challenge left to the user ? def hell_frozen_over(): return False while not hell_frozen_over(): .... -- Greg From grant.b.edwards at gmail.com Tue Sep 7 10:53:29 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 7 Sep 2021 14:53:29 -0000 (UTC) Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On 2021-09-06, Stefan Ram wrote: > "Avi Gross" writes: >> In languages like C/C++ there are people who make up macros like: >>#define INDEFINITELY_LOOP while (true) >>Or something like that and then allow the preprocessor to replace >>INDEFINITELY_LOOP with valid C code. > > Those usually are beginners. > >>So, how to do something like that in python, is a challenge left to the >>user > > Such a use of macros is frowned upon by most C programmers, > because it renders the code unreadable. I remember engineering manager I worked with about 35 years ago who used a set of C macros to try to make his code look as much like BASIC as possible: #define IF if ( #define THEN ) { #define ELSE } else { #define ENDIF } ... IIRC he copied them out of a magazine article. He then proceeded to try to implement a tree search algorithm (he didn't actually know that's what he was doing) using his new "language" without using recursion (which he had never heard of and couldn't grok) by keeping track of state using an array. It did not go well and made him a bit of a laughingstock. IIRC, he had first tried to write it in actual BASIC, but gave up on that before switching to C and his ridiculous macro set. From alister.ware at ntlworld.com Tue Sep 7 14:58:16 2021 From: alister.ware at ntlworld.com (alister) Date: Tue, 7 Sep 2021 18:58:16 -0000 (UTC) Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On Tue, 07 Sep 2021 14:53:29 +0000, Grant Edwards wrote: > On 2021-09-06, Stefan Ram wrote: >> "Avi Gross" writes: >>> In languages like C/C++ there are people who make up macros like: >>>#define INDEFINITELY_LOOP while (true) >>>Or something like that and then allow the preprocessor to replace >>>INDEFINITELY_LOOP with valid C code. >> >> Those usually are beginners. >> >>>So, how to do something like that in python, is a challenge left to the >>>user >> >> Such a use of macros is frowned upon by most C programmers, >> because it renders the code unreadable. > > I remember engineering manager I worked with about 35 years ago who used > a set of C macros to try to make his code look as much like BASIC as > possible: > > #define IF if ( #define THEN ) { #define ELSE } else { > #define ENDIF } > ... > > IIRC he copied them out of a magazine article. > > He then proceeded to try to implement a tree search algorithm (he didn't > actually know that's what he was doing) using his new "language" without > using recursion (which he had never heard of and couldn't grok) by > keeping track of state using an array. It did not go well and made him a > bit of a laughingstock. IIRC, he had first tried to write it in actual > BASIC, but gave up on that before switching to C and his ridiculous > macro set. 1 Simple rule, if you are programming in language 'a' then write Language 'a' Code it. Just because you can do something doesn't mean you should -- Help! I'm trapped in a Chinese computer factory! From hrouselle at jevedi.com Tue Sep 7 14:58:29 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Tue, 07 Sep 2021 15:58:29 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: <864kaw9pre.fsf@jevedi.com> Grant Edwards writes: > On 2021-09-06, Stefan Ram wrote: >> "Avi Gross" writes: >>> In languages like C/C++ there are people who make up macros like: >>>#define INDEFINITELY_LOOP while (true) >>>Or something like that and then allow the preprocessor to replace >>>INDEFINITELY_LOOP with valid C code. >> >> Those usually are beginners. [...] >> Such a use of macros is frowned upon by most C programmers, >> because it renders the code unreadable. > > I remember engineering manager I worked with about 35 years ago who > used a set of C macros to try to make his code look as much like BASIC > as possible: > > #define IF if ( > #define THEN ) { > #define ELSE } else { > #define ENDIF } > ... > > IIRC he copied them out of a magazine article. > > He then proceeded to try to implement a tree search algorithm (he > didn't actually know that's what he was doing) using his new > "language" without using recursion (which he had never heard of and > couldn't grok) by keeping track of state using an array. It did not go > well and made him a bit of a laughingstock. IIRC, he had first tried > to write it in actual BASIC, but gave up on that before switching to C > and his ridiculous macro set. LOL! (Had fun reading this.) From wlfraed at ix.netcom.com Tue Sep 7 15:15:37 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Tue, 07 Sep 2021 15:15:37 -0400 Subject: on writing a while loop for rolling two dice References: <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On Tue, 7 Sep 2021 16:08:04 +1200, Greg Ewing declaimed the following: >On 7/09/21 11:38 am, Avi Gross wrote: > >> #define INDEFINITELY_LOOP while (true) >> >> So, how to do something like that in python, is a challenge left to the user ? > >def hell_frozen_over(): > return False > >while not hell_frozen_over(): > .... Hell typically freezes every January (scroll down to monthly average): https://www.worldweatheronline.com/hell-weather-averages/michigan/us.aspx -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From avigross at verizon.net Tue Sep 7 15:51:49 2021 From: avigross at verizon.net (Avi Gross) Date: Tue, 7 Sep 2021 15:51:49 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: <05c601d7a421$cc45c700$64d15500$@verizon.net> Although I sort of agree with Alister, I also note that many languages deliberately provide you with the means to customize in ways that make your personal life more amenable while making it perhaps harder for others. Consider the humble import statement frequently used as: import numpy as np import pandas as pd The above is perfectly legal and in some circles is expected. But I suspect many people do not care about being able to type a slightly abbreviated np.name(..) rather than numpy.name(...) and yet others import specific functions from a package like: from numpy import arrange The bottom line is that there are many ways the same code can be called, including quite a few other variations I am not describing. People can customize their code in many ways including making it look more like the way they might program in some other computer language/environment. Anyone reading their code with a different viewpoint may have some adjustment issues. I am wondering if anyone has written programs that would take some complete body of code and not prettify it or reformat it as some tools do, but sort of make it into a standard format. In the above example, it might undo things like renaming numpy to np and change all the code that looks like np.name to numpy.name and similarly changes any function imported directly to also be fully qualified. Of course, some code I have seen changes things mid-stream or has some if statement that sets one of several function calls to be the one to use beyond that point. So, I am not sanguine on trying to enforce some standards to make your code easy to read by others and am more a fan of suggesting enough comments in the code to guide anyone on your own idiosyncratic uses. -----Original Message----- From: Python-list On Behalf Of alister via Python-list Sent: Tuesday, September 7, 2021 2:58 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice On Tue, 07 Sep 2021 14:53:29 +0000, Grant Edwards wrote: > On 2021-09-06, Stefan Ram wrote: >> "Avi Gross" writes: >>> In languages like C/C++ there are people who make up macros like: >>>#define INDEFINITELY_LOOP while (true) Or something like that and >>>then allow the preprocessor to replace INDEFINITELY_LOOP with valid C >>>code. >> >> Those usually are beginners. >> >>>So, how to do something like that in python, is a challenge left to >>>the user >> >> Such a use of macros is frowned upon by most C programmers, >> because it renders the code unreadable. > > I remember engineering manager I worked with about 35 years ago who > used a set of C macros to try to make his code look as much like BASIC > as > possible: > > #define IF if ( #define THEN ) { #define ELSE } else { > #define ENDIF } > ... > > IIRC he copied them out of a magazine article. > > He then proceeded to try to implement a tree search algorithm (he > didn't actually know that's what he was doing) using his new > "language" without using recursion (which he had never heard of and > couldn't grok) by keeping track of state using an array. It did not go > well and made him a bit of a laughingstock. IIRC, he had first tried > to write it in actual BASIC, but gave up on that before switching to C > and his ridiculous macro set. 1 Simple rule, if you are programming in language 'a' then write Language 'a' Code it. Just because you can do something doesn't mean you should -- Help! I'm trapped in a Chinese computer factory! -- https://mail.python.org/mailman/listinfo/python-list From pfeiffer at cs.nmsu.edu Tue Sep 7 15:54:08 2021 From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer) Date: Tue, 07 Sep 2021 13:54:08 -0600 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6ksik0n.fsf@jevedi.com> Message-ID: <1b1r60go0v.fsf@pfeifferfamily.net> Hope Rouselle writes: > Christian Gollwitzer writes: >> >> I believe it is not commutativity, but associativity, that is >> violated. > > Shall we take this seriously? (I will disagree, but that doesn't mean I > am not grateful for your post. Quite the contary.) It in general > violates associativity too, but the example above couldn't be referring > to associativity because the second sum above could not be obtained from > associativity alone. Commutativity is required, applied to five pairs > of numbers. How can I go from > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > > to > > 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? > > Perhaps only through various application of commutativity, namely the > ones below. (I omit the parentheses for less typing. I suppose that > does not create much trouble. There is no use of associativity below, > except for the intented omission of parentheses.) > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77 > = 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77 > = 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77 > = 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77 > = 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23. But these transformations depend on both commutativity and associativity, precisely due to those omitted parentheses. When you transform 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 into 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23. it isn't just assuming commutativity, it's also assuming associativity since it is changing from (7.23 + 8.41 + 6.15 + 2.31 + 7.73) + 7.77 to (8.41 + 6.15 + 2.31 + 7.73 + 7.77) + 7.23. If I use parentheses to modify the order of operations of the first line to match that of the last, I get 7.23 + (8.41 + 6.15 + 2.31 + 7.73 + 7.77) Now, I get 39.60000000000001 evaluating either of them. From pablogsal at gmail.com Tue Sep 7 19:28:26 2021 From: pablogsal at gmail.com (Pablo Galindo Salgado) Date: Wed, 8 Sep 2021 00:28:26 +0100 Subject: [RELEASE] Python 3.10.0rc2 is available Message-ID: Python 3.10 is one month away, can you believe it? This snake is still trying to bite as it has been an interesting day of fighting fires, release blockers, and a bunch of late bugs but your friendly release team always delivers :) You can get this new release while is still fresh here: https://www.python.org/downloads/release/python-3100rc2/ This is the second release candidate of Python 3.10 This release, **3.10.0rc2** , is the last preview before the final release of Python 3.10.0 on 2021-10-04. Entering the release candidate phase, only reviewed code changes which are clear bug fixes are allowed between release candidates and the final release. There will be no ABI changes from this point forward in the 3.10 series and the goal is that there will be as few code changes as possible. *The next release will be the final release of Python 3.10.0, which is currently scheduled for Monday, 2021-10-04.* Call to action ?????????????? *The 3.10 branch is now accepting changes for 3.10.**1*. To maximize stability, the final release will be cut from the v3.10.0rc2 tag. If you need the release manager (me) to cherry-pick any critical fixes, mark issues as release blockers, and/or add me as a reviewer on a critical backport PR on GitHub. To see which changes are currently cherry-picked for inclusion in 3.10.0, look at the short-lived branch-v3.10.0 https://github.com/python/cpython/tree/branch-v3.10.0 on GitHub. ?????????????? *Core developers: all eyes on the docs now* - Are all your changes properly documented? - Did you notice other changes you know of to have insufficient documentation? *Community members* We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.10 compatibilities during this phase. As always, report any issues to [the Python bug tracker ](https://bugs.python.org/). Please keep in mind that this is a preview release and its use is **not** recommended for production environments. And now for something completely different Maxwell's demon is a thought experiment that would hypothetically violate the second law of thermodynamics. It was proposed by the physicist James Clerk Maxwell in 1867. In the thought experiment, a demon controls a small massless door between two chambers of gas. As individual gas molecules (or atoms) approach the door, the demon quickly opens and closes the door to allow only fast-moving molecules to pass through in one direction, and only slow-moving molecules to pass through in the other. Because the kinetic temperature of a gas depends on the velocities of its constituent molecules, the demon's actions cause one chamber to warm up and the other to cool down. This would decrease the total entropy of the two gases, without applying any work, thereby violating the second law of thermodynamics. We hope you enjoy those new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from a plane going to Malaga, Your friendly release team, Pablo Galindo @pablogsal Ned Deily @nad Steve Dower @steve.dower From alan.gauld at yahoo.co.uk Tue Sep 7 19:24:44 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 8 Sep 2021 00:24:44 +0100 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On 07/09/2021 15:53, Grant Edwards wrote: > I remember engineering manager I worked with about 35 years ago who > used a set of C macros to try to make his code look as much like BASIC > as possible: > > #define IF if ( > #define THEN ) { > #define ELSE } else { > #define ENDIF } > ... > > IIRC he copied them out of a magazine article. That was quite common in C before it became popular(early/mid 80s). I've seen Pascal, Algol and Coral macro sets in use. You could even download pre-written ones from various bulletin boards (remember them?!) for a while. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From greg.ewing at canterbury.ac.nz Tue Sep 7 21:07:47 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 8 Sep 2021 13:07:47 +1200 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On 8/09/21 2:53 am, Grant Edwards wrote: > #define IF if ( > #define THEN ) { > #define ELSE } else { > #define ENDIF } > ... I gather that early versions of some of the Unix utilities were written by someone who liked using macros to make C resemble Algol. I guess you can get away with that sort of thing if you're a Really Smart Person. -- Greg From avigross at verizon.net Tue Sep 7 21:20:13 2021 From: avigross at verizon.net (Avi Gross) Date: Tue, 7 Sep 2021 21:20:13 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: <018401d7a44f$accc70d0$06655270$@verizon.net> Greg, Yes, a smart person may come up with such tricks but a really smart person, in my view, adjusts. With some exceptions, such as when trying to port existing code to a new language quickly, someone who is not too obsessive will try to pick up the goals and spirit of a new language and use them when it seems reasonable. And, a smart person, if they see nothing new, might just go back to their old language or ... Pick a language that easily supports regular expressions and object creation and functional programming and so on, like python, and ask why you might want to use it to simulate a really old version of BASIC when you can just use BASIC. Admittedly, most people are not flexible. I find that with human languages too that some learn another language just enough to recognize words but not to use the changed grammar or the different idioms and never become fluent. I am amused though at the fact that python, by using indentation rather than things like curly braces, would make some of the games like shown below quite a bit more difficult. -----Original Message----- From: Python-list On Behalf Of Greg Ewing Sent: Tuesday, September 7, 2021 9:08 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice On 8/09/21 2:53 am, Grant Edwards wrote: > #define IF if ( > #define THEN ) { > #define ELSE } else { > #define ENDIF } > ... I gather that early versions of some of the Unix utilities were written by someone who liked using macros to make C resemble Algol. I guess you can get away with that sort of thing if you're a Really Smart Person. -- Greg -- https://mail.python.org/mailman/listinfo/python-list From Richard at Damon-Family.org Tue Sep 7 22:08:43 2021 From: Richard at Damon-Family.org (Richard Damon) Date: Tue, 7 Sep 2021 22:08:43 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: <05c601d7a421$cc45c700$64d15500$@verizon.net> References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> <05c601d7a421$cc45c700$64d15500$@verizon.net> Message-ID: On 9/7/21 3:51 PM, Avi Gross via Python-list wrote: > and similarly changes any function imported directly > to also be fully qualified. One danger with this is that it can actual change the behavior of the program. Maybe more likely with global objects than functions, but still an issue. Remember, "from module import fun" will bind the name fun in this modules namespace to the current definiton of the object fun in the other modules name space. If after that point, something rebinds the name in the other module, the module that did the from import won't see that change, but if the reference is changed to module.fun, it will. Since that rebinding might even be some third module doing a 'monkey patch', detecting if it is safe is basically impossible. Admittedly, if this is an issue, the sensitivity to timing makes the difference something very fussy to exactly the order you do things, the cases where the difference is intended is likely fairly small. -- Richard Damon From avigross at verizon.net Wed Sep 8 00:12:29 2021 From: avigross at verizon.net (Avi Gross) Date: Wed, 8 Sep 2021 00:12:29 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> <05c601d7a421$cc45c700$64d15500$@verizon.net> Message-ID: <01a401d7a467$bdbb3b70$3931b250$@verizon.net> I think I already agreed with much of your point. That is indeed the problem. You are allowed to do some possibly non-standard things. A static evaluation/replacement may show the wrong function being called and a dynamic one may still mess up as there are many ways to do indirection. I mention this partially because of a recent discussion on another computer where the question was why they got an error message that the object being given a function was not a data.frame or something compatible. What was being passed was a data structure meant to hold some simulation of an EXCEL spreadsheet. Not quite the same. But they insisted it had to be a data.frame because they called functionA() and it is documented to return a data.frame. It turned out that they had loaded lots of packages (modules but not) and had not paid attention when they got a message that one function called functionA was now being covered by another with the same name. Each such package loaded often gets a new namespace/environment and when searching for a function by name, the new package was ahead of the older package, hence the warning. So one solution was to change his code in one of several ways, but more generally to call any functions that may be hidden this way in a fully qualified manner as in packageA::functionA(...) do you do not accidentally get package::functionA(...) Now note this is not so much a bug as a feature in that language and It is quite common to sort of redefine a function name to do what you want. But if you have developed your own package and want to guarantee the user does not undo your work in specific cases, you should also call some things as explicitly within your own namespace. Back to Python, it is a tad too flexible in some places and my point was that it would be interesting, along the lines of a linter, to have some tools that help explain what the program might be doing a tad more explicitly. But as an interpreted language, so much can happen at runtime that this may not be very effective or accurate. The language is full of places where slipping in another object means you over-rode the meaning of += for instance. -----Original Message----- From: Python-list On Behalf Of Richard Damon Sent: Tuesday, September 7, 2021 10:09 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice On 9/7/21 3:51 PM, Avi Gross via Python-list wrote: > and similarly changes any function imported directly to also be fully > qualified. One danger with this is that it can actual change the behavior of the program. Maybe more likely with global objects than functions, but still an issue. Remember, "from module import fun" will bind the name fun in this modules namespace to the current definiton of the object fun in the other modules name space. If after that point, something rebinds the name in the other module, the module that did the from import won't see that change, but if the reference is changed to module.fun, it will. Since that rebinding might even be some third module doing a 'monkey patch', detecting if it is safe is basically impossible. Admittedly, if this is an issue, the sensitivity to timing makes the difference something very fussy to exactly the order you do things, the cases where the difference is intended is likely fairly small. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list From chottel at earthlink.net Tue Sep 7 23:31:27 2021 From: chottel at earthlink.net (charles hottel) Date: Tue, 7 Sep 2021 23:31:27 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> <018401d7a44f$accc70d0$06655270$@verizon.net> Message-ID: On 9/7/2021 9:20 PM, Avi Gross wrote: > Greg, > > Yes, a smart person may come up with such tricks but a really smart person, > in my view, adjusts. With some exceptions, such as when trying to port > existing code to a new language quickly, someone who is not too obsessive > will try to pick up the goals and spirit of a new language and use them when > it seems reasonable. And, a smart person, if they see nothing new, might > just go back to their old language or ... > > Pick a language that easily supports regular expressions and object creation > and functional programming and so on, like python, and ask why you might > want to use it to simulate a really old version of BASIC when you can just > use BASIC. > > Admittedly, most people are not flexible. I find that with human languages > too that some learn another language just enough to recognize words but not > to use the changed grammar or the different idioms and never become fluent. > > I am amused though at the fact that python, by using indentation rather than > things like curly braces, would make some of the games like shown below > quite a bit more difficult. > > -----Original Message----- > From: Python-list On > Behalf Of Greg Ewing > Sent: Tuesday, September 7, 2021 9:08 PM > To: python-list at python.org > Subject: Re: on writing a while loop for rolling two dice > > On 8/09/21 2:53 am, Grant Edwards wrote: >> #define IF if ( >> #define THEN ) { >> #define ELSE } else { >> #define ENDIF } >> ... > > I gather that early versions of some of the Unix utilities were written by > someone who liked using macros to make C resemble Algol. > > I guess you can get away with that sort of thing if you're a Really Smart > Person. > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > So what do yoy think or feel about a language like RATFOR (Rational FORTRAN) which was implemented as macros? Should they instead have simply adapted themselves to FORTRAN? From hongyi.zhao at gmail.com Wed Sep 8 01:05:58 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Tue, 7 Sep 2021 22:05:58 -0700 (PDT) Subject: combine multiple xlsx files which include dropdown columns in them. Message-ID: <97e5e97f-7b86-41f4-8b8b-3e3baeadd127n@googlegroups.com> I've some xlsx files which include dropdown columns in them. I want to know whether I can combine all the lines into one xlsx file. Any hints for doing this job with python programmatically will be highly appreciated. Regards, HY From hrouselle at jevedi.com Wed Sep 8 09:20:27 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Wed, 08 Sep 2021 10:20:27 -0300 Subject: on floating-point numbers References: <86bl5bm6go.fsf@jevedi.com> <86a6ksik0n.fsf@jevedi.com> <1b1r60go0v.fsf@pfeifferfamily.net> Message-ID: <86wnnr8aqs.fsf@jevedi.com> Joe Pfeiffer writes: > Hope Rouselle writes: >> Christian Gollwitzer writes: >>> >>> I believe it is not commutativity, but associativity, that is >>> violated. >> >> Shall we take this seriously? (I will disagree, but that doesn't mean I >> am not grateful for your post. Quite the contary.) It in general >> violates associativity too, but the example above couldn't be referring >> to associativity because the second sum above could not be obtained from >> associativity alone. Commutativity is required, applied to five pairs >> of numbers. How can I go from >> >> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 >> >> to >> >> 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23? >> >> Perhaps only through various application of commutativity, namely the >> ones below. (I omit the parentheses for less typing. I suppose that >> does not create much trouble. There is no use of associativity below, >> except for the intented omission of parentheses.) >> >> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 >> = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77 >> = 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77 >> = 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77 >> = 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77 >> = 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23. > > But these transformations depend on both commutativity and > associativity, precisely due to those omitted parentheses. When you > transform > > 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77 > > into > > 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23. > > it isn't just assuming commutativity, it's also assuming associativity > since it is changing from > > (7.23 + 8.41 + 6.15 + 2.31 + 7.73) + 7.77 > > to > > (8.41 + 6.15 + 2.31 + 7.73 + 7.77) + 7.23. > > If I use parentheses to modify the order of operations of the first line > to match that of the last, I get > 7.23 + (8.41 + 6.15 + 2.31 + 7.73 + 7.77) > > Now, I get 39.60000000000001 evaluating either of them. I need to go slow. If I have just two numbers, then I don't need to talk about associativity: I can send 7.23 to the rightmost place with a single application of commutativity. In symbols, 7.23 + 8.41 = 8.41 + 7.23. But if I have three numbers and I want to send the leftmost to the rightmost place, I need to apply associativity 7.23 + 8.41 + 6.15 = (7.23 + 8.41) + 6.15 -- clarifying that I go left to right = 7.23 + (8.41 + 6.15) -- associativity = (8.41 + 6.15) + 7.23 -- commutativity I see it. Cool. Thanks. From grant.b.edwards at gmail.com Wed Sep 8 10:46:28 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 8 Sep 2021 14:46:28 -0000 (UTC) Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> <018401d7a44f$accc70d0$06655270$@verizon.net> Message-ID: On 2021-09-08, charles hottel wrote: > So what do yoy think or feel about a language like RATFOR (Rational > FORTRAN) which was implemented as macros? Should they instead have > simply adapted themselves to FORTRAN? That's an interesting question. If the langauge is complete, well-defined, and well-documented then it's not that much different than any other source language than gets translated into a lower level language (e.g. C -> assembly). My recollection of RATFOR was that it provided enough signifcant "features" that weren't available in the underlying FORTRAN to make it worthwhile. That seems to me to be qualitatively different than a set of macros that simply make one language look (somewhat) like a different language with a similar level of abstraction -- without providing anything other than cosmetic changes. -- Grant From grant.b.edwards at gmail.com Wed Sep 8 10:58:04 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 8 Sep 2021 14:58:04 -0000 (UTC) Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> <018401d7a44f$accc70d0$06655270$@verizon.net> Message-ID: On 2021-09-08, charles hottel wrote: > So what do yoy think or feel about a language like RATFOR (Rational > FORTRAN) which was implemented as macros? The RATFOR implementations I've seen weren't done using macros. It was a preprocessor, yes. But it generates code for the various structured statement types (while, for, if/then/else) and source structures (open/close brace) the way a compiler does, even though the generated code is FORTRAN66 rather than assembly or bytecode or whatever. > Should they instead have simply adapted themselves to FORTRAN? From wlfraed at ix.netcom.com Wed Sep 8 11:46:55 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 08 Sep 2021 11:46:55 -0400 Subject: on writing a while loop for rolling two dice References: <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On Wed, 8 Sep 2021 00:24:44 +0100, Alan Gauld via Python-list declaimed the following: > >That was quite common in C before it became popular(early/mid 80s). >I've seen Pascal, Algol and Coral macro sets in use. >You could even download pre-written ones from various >bulletin boards (remember them?!) for a while. And if one wants to expose themselves to real horror -- look at the output of one of those "structured FORTRAN" preprocessors (RATFOR, TEXTFOR, others) that were cropping up in the late 70s to generate FORTRAN-IV from code written with block IF, WHILE, etc. I spent close to 20 years (80s-90s) maintaining the /output/ of such a preprocessor. The software had apparently originated with a sub-contractor, and we did not have access to their preprocessor (and/or no one ported it to the VAX-11 from PDP-11). At first my practice was to study a file in detail, and then rewrite it using F77 with separately compiled subroutines (the preprocessor was "one file per program" and implemented BASIC-style GOSUB, relying on shared variables and ASSIGNed GOTO for the return address -- I had to move the shared variables to COMMON blocks to make separate subroutines usable). By the end of those decades, I'd been exposed to the preprocessor enough to hand-write smaller patches within its style. The only good thing is that is left the structured source as comments, and tended to right-justify the F-IV output, so easy to locate... -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Wed Sep 8 12:20:02 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 08 Sep 2021 12:20:02 -0400 Subject: combine multiple xlsx files which include dropdown columns in them. References: <97e5e97f-7b86-41f4-8b8b-3e3baeadd127n@googlegroups.com> Message-ID: <4smhjg9ggqv04ut5uh176cbmc22caa9nta@4ax.com> On Tue, 7 Sep 2021 22:05:58 -0700 (PDT), "hongy... at gmail.com" declaimed the following: >I've some xlsx files which include dropdown columns in them. I want to know whether I can combine all the lines into one xlsx file. Any hints for doing this job with python programmatically will be highly appreciated. I don't recall ANY Python Excel reader/writer that can handle embedded scripting. They all only read the last value active in a cell, not the code behind the cell. They mostly just bypass the need to convert to/from CSV/TSV. Presuming you are doing this on Windows, you might be able to use either the pywin32/win32py extension library, or ctypes, to invoke the Excel COM interface -- which may allow you to read the underlying script and/or form objects. If this is a one-time operation, it may be faster to just open the files and use cut&paste Or write a large VBA script in Excel. This appears to be the current "favored" package from Python: https://openpyxl.readthedocs.io/en/stable/ You may still have problems: """ keep_vba controls whether any Visual Basic elements are preserved or not (default). If they are preserved they are still not editable. """ If not editable, you won't be able to copy from one file to another. Also: """ openpyxl does currently not read all possible items in an Excel file so images and charts will be lost from existing files if they are opened and saved with the same name. """ It does, however, grant access to the formulas in cells, rather than just last value; along with style information... -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From avigross at verizon.net Wed Sep 8 13:24:32 2021 From: avigross at verizon.net (Avi Gross) Date: Wed, 8 Sep 2021 13:24:32 -0400 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> <86a6l1thz7.fsf@jevedi.com> <4b8ed591-2f2f-6c76-4606-b431f52cf847@DancesWithMice.info> <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> <018401d7a44f$accc70d0$06655270$@verizon.net> Message-ID: <0e7601d7a4d6$6395b070$2ac11150$@verizon.net> Charles, This forum is for python discussions so I will clarify that there is nothing wrong with making up a new language, including by bootstrapping an old language. But why not call it new and do not try to confuse people using the old. Python has grown and added much over the years and even had a fairly negative experience in a transition from versions 2 to 3 as some people complain their old software no longer works. If you look at a language like C which was remade into C++, guess what? It has a new name and makes no claims to be fully compatible. If RATFOR was a more pleasant programming environment for some people and purposes, fine. I note Python has many modules that for some people become the most used parts even when the base language might allow some similar functionality. Someone who learns python but has never experienced those, may feel a bit lost. And you can customize Python to a point where it does things in fairly hidden ways too. There are no promises of purity. What this particular discussion began with was much lower level and about various ways of writing a while loop within very basic Python and seems to be heading away into discussions of other languages. I happen to enjoy learning and contrasting lots of languages but doubt many others do and most people want to learn just a few good tools and use them to get their job done. My point was if you have a tool you like and you try another you don't, then rather than trying to force the new one, just use what works and leave others that like it alone. Make your own religion and let others keep theirs. However, improvements and increased functionality may be more welcome, than examples where the interface was changed with no benefits. -----Original Message----- From: Python-list On Behalf Of charles hottel Sent: Tuesday, September 7, 2021 11:31 PM To: python-list at python.org Subject: Re: on writing a while loop for rolling two dice So what do yoy think or feel about a language like RATFOR (Rational FORTRAN) which was implemented as macros? Should they instead have simply adapted themselves to FORTRAN? -- https://mail.python.org/mailman/listinfo/python-list From grant.b.edwards at gmail.com Wed Sep 8 12:32:45 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 8 Sep 2021 16:32:45 -0000 (UTC) Subject: on writing a while loop for rolling two dice References: <864kb3m4qi.fsf@jevedi.com> <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On 2021-09-08, Dennis Lee Bieber wrote: > I spent close to 20 years (80s-90s) maintaining the /output/ of such > a preprocessor. Ouch. I hope it paid well. ;) Back when I did a lot of TeX/LaTeX stuff on VMS, I used to make occasional fixes and add (very minor) features to one of the dvi handling executables (I don't remember if it was the VT2xx previewer, or the laser-printer "driver") using the VMS "PATCH" utility. I also can't remember how we ended up without source code for that piece of the toolchain when we build all of the reset of it from source. > The software had apparently originated with a sub-contractor, and we > did not have access to their preprocessor Allowing that to happen is definitely a major management F*%k-up for which somebody should have had their career ended. > (and/or no one ported it to the VAX-11 from PDP-11). I would have though that would have been pretty trivial compared to maintaining the output source for 20 years. -- Grant From wlfraed at ix.netcom.com Wed Sep 8 12:59:11 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 08 Sep 2021 12:59:11 -0400 Subject: on writing a while loop for rolling two dice References: <004401d7a378$44d43580$ce7ca080$@verizon.net> <018401d7a44f$accc70d0$06655270$@verizon.net> Message-ID: On Wed, 8 Sep 2021 14:46:28 -0000 (UTC), Grant Edwards declaimed the following: >On 2021-09-08, charles hottel wrote: > >> So what do yoy think or feel about a language like RATFOR (Rational >> FORTRAN) which was implemented as macros? Should they instead have >> simply adapted themselves to FORTRAN? > >That's an interesting question. If the langauge is complete, >well-defined, and well-documented then it's not that much different >than any other source language than gets translated into a lower level >language (e.g. C -> assembly). My recollection of RATFOR was that it >provided enough signifcant "features" that weren't available in the >underlying FORTRAN to make it worthwhile. > Primarily providing block structured IF/ELSE and loops -- in a language that only really provided IF/GOTO... My college tried using one of these (for some reason the name TextFOR sticks with me)... The experiment only lasted one term. The preprocessor ate CPU and I/O time -- with the result that the FORTRAN class used two or three times the times of the COBOL class! (The native compilers were designed as re-entrant, allowing multiple compiles to share one in-core image; the preprocessor no doubt ran as one image per compile, triggering lots of page swapping to disk) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Wed Sep 8 15:54:58 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 08 Sep 2021 15:54:58 -0400 Subject: on writing a while loop for rolling two dice References: <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: <1b4ijglpmc9ladcijv8jv71ui4p61oiqvk@4ax.com> On Wed, 8 Sep 2021 16:32:45 -0000 (UTC), Grant Edwards declaimed the following: >On 2021-09-08, Dennis Lee Bieber wrote: > >> I spent close to 20 years (80s-90s) maintaining the /output/ of such >> a preprocessor. > >Ouch. I hope it paid well. ;) Only if one ignores the bloody cost of apartments (given the rent paid over the 30 years I spent in that 1BR 680sq.ft. unit I should have owned it, if not half the facility ) > >> The software had apparently originated with a sub-contractor, and we >> did not have access to their preprocessor > >Allowing that to happen is definitely a major management F*%k-up for >which somebody should have had their career ended. > For all I know, someone might have... Back when the application was running on PDP-11 (I joined the program early 80s, when the "mission planning" software was being ported to the VAX-11). Most of the "mission planning" software was local-grown, so was DEC FORTRAN (may have started as F-IV, and developed F77 during maintenance and the porting to the VAX). Not the part I maintained -- which drove a Ramtek 9300 graphics engine (only one engine could be installed into a VAX, it was that archaic). It got to the point where I think Ramtek service was recycling the parts when ever we had a service call. Eventually I proposed that a VAXstation (DECWindows) and GKS could replace most of the display side. We weren't approved to rewrite the application to directly display -- but rewriting the Ramtek library to send the plotting data to a separate program, which managed the display window, was in the budget. > >I would have though that would have been pretty trivial compared to >maintaining the output source for 20 years. I'm not so sure -- that "one file per executable" would have led to long compile times (first for the preprocessor, then for the generated F-IV output), vs my rewrites into multiple files, where only isolated changes needed to be recompiled. Humorously, the graphics application suite was known as "MESS" (I can't go deeper into the acronym without violating security classification). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From kushal at locationd.net Wed Sep 8 21:05:16 2021 From: kushal at locationd.net (Kushal Kumaran) Date: Wed, 08 Sep 2021 18:05:16 -0700 Subject: combine multiple xlsx files which include dropdown columns in them. In-Reply-To: <97e5e97f-7b86-41f4-8b8b-3e3baeadd127n@googlegroups.com> (hongyi.zhao@gmail.com's message of "Tue, 7 Sep 2021 22:05:58 -0700 (PDT)") References: <97e5e97f-7b86-41f4-8b8b-3e3baeadd127n@googlegroups.com> Message-ID: <87pmtiftir.fsf@locationd.net> On Tue, Sep 07 2021 at 10:05:58 PM, "hongy... at gmail.com" wrote: > I've some xlsx files which include dropdown columns in them. I want to > know whether I can combine all the lines into one xlsx file. Any hints > for doing this job with python programmatically will be highly > appreciated. > The dropdown is *probably* implemented using Excel's data validation mechanism. openpyxl's documentation says it can read[1]/write Excel files and also mentions data validation[2]. You might be able to use it to read the existing files, combine the rows and write to a different file. [1] https://openpyxl.readthedocs.io/en/stable/usage.html#read-an-existing-workbook [2] https://openpyxl.readthedocs.io/en/stable/validation.html -- regards, kushal From rickcarta12 at gmail.com Wed Sep 8 21:54:14 2021 From: rickcarta12 at gmail.com (Ricardo) Date: Wed, 8 Sep 2021 21:54:14 -0400 Subject: Problems with running Python in Npp Message-ID: Hey Python and crew I'm having difficulties installing and running Python on my computer. I've seen plenty YouTube videos on how to set it up, but none of them have worked. Any help or guidance will be greatly appreciated. Sincerely, Ricardo Sent from [1]Mail for Windows References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From hongyi.zhao at gmail.com Wed Sep 8 20:20:03 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 8 Sep 2021 17:20:03 -0700 (PDT) Subject: Change the display style of the text on the STACKLINE. Message-ID: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> I'm using the following code in my forked project [1]: percol.view.STACKLINE = 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s Dir:M-d Dircmd:M-b' I would like to change the display style of the text mentioned above, for example, to underline some characters in it, as shown below: _D_ir:M-d How to achieve this purpose? [1] https://github.com/hongyi-zhao/ariadne/blob/838179bb4275ac85f5342d9e7d086d6ade3be1de/rc.py#L55 Regards, HY From hongyi.zhao at gmail.com Wed Sep 8 22:31:09 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 8 Sep 2021 19:31:09 -0700 (PDT) Subject: combine multiple xlsx files which include dropdown columns in them. In-Reply-To: References: <87pmtiftir.fsf@locationd.net> <97e5e97f-7b86-41f4-8b8b-3e3baeadd127n@googlegroups.com> Message-ID: <20129c02-4542-4280-bebf-bb8e033349fcn@googlegroups.com> On Thursday, September 9, 2021 at 9:15:23 AM UTC+8, Kushal Kumaran wrote: > On Tue, Sep 07 2021 at 10:05:58 PM, "hongy... at gmail.com" wrote: > > I've some xlsx files which include dropdown columns in them. I want to > > know whether I can combine all the lines into one xlsx file. Any hints > > for doing this job with python programmatically will be highly > > appreciated. > > > The dropdown is *probably* implemented using Excel's data validation > mechanism. openpyxl's documentation says it can read[1]/write Excel > files and also mentions data validation[2]. You might be able to use it > to read the existing files, combine the rows and write to a different > file. > > [1] https://openpyxl.readthedocs.io/en/stable/usage.html#read-an-existing-workbook > [2] https://openpyxl.readthedocs.io/en/stable/validation.html Thank you for your comments. For a related discussion on this topic, please refer to [1]. [1] https://groups.google.com/g/openpyxl-users/c/k2xnPZS2kbo/m/m2YmEO9ZBgAJ Regards, HY From wlfraed at ix.netcom.com Thu Sep 9 07:20:46 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 09 Sep 2021 07:20:46 -0400 Subject: Problems with running Python in Npp References: Message-ID: <83rjjg9hf7b6spifh41h48iakl3dkl6i3t@4ax.com> On Wed, 8 Sep 2021 21:54:14 -0400, Ricardo declaimed the following: > Hey Python and crew I'm having difficulties installing and running Python > on my computer. I've seen plenty YouTube videos on how to set it up, but > none of them have worked. Any help or guidance will be greatly > appreciated. > INFORMATION... We want... information. "None of them have worked" is as useful to us as "I went out to a football field looking for baseballs" (actually, the latter is more informative and useful for debugging). What is "Npp" -- Wikipedia shows a dozen political parties, a weather satellite, a number of chemical compounds, an Australian electronic payment system, and a form of rocket propulsion. What exactly is happening when you attempt to run Python? HOW are you attempting to run it? The most common problem is that people keep clicking on the INSTALLER file -- once the installer has run, just stuff it some place safe should you need to fix an install. Python is not some massive IDE like Visual Studio. It is a language interpreter, commonly run from a command shell by typing "Python some-script-file-name" (though newer Python's include a so-called "launcher" invoked as just "py"). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From roland.em0001 at googlemail.com Thu Sep 9 08:57:02 2021 From: roland.em0001 at googlemail.com (Roland Mueller) Date: Thu, 9 Sep 2021 15:57:02 +0300 Subject: Change the display style of the text on the STACKLINE. In-Reply-To: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> Message-ID: Hello to 9. syysk. 2021 klo 6.53 hongy... at gmail.com (hongyi.zhao at gmail.com) kirjoitti: > I'm using the following code in my forked project [1]: > > percol.view.STACKLINE = 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s Dir:M-d > Dircmd:M-b' > > I would like to change the display style of the text mentioned above, for > example, to underline some characters in it, as shown below: > > _D_ir:M-d > > You can use e.g. str.replace() or re.sub() >>> percol.view.STACKLINE = percol.view.STACKLINE.replace('D', '_D_') Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d _D_ircmd:M-b' >>> import re Replace D with _D_ >>> percol.view.STACKLINE = re.sub(r'([D])', r'_\1_', percol.view.STACKLINE) Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d _D_ircmd:M-b' Replace D and M with _D_, _M_ >>> percol.view.STACKLINE = re.sub(r'([DM])', r'_\1_', percol.view.STACKLINE) 'Fold:F1,F2,F3 Push:C-p Pop:_M_-p Script:_M_-s _D_ir:_M_-d _D_ircmd:_M_-b' Regards, Roland > How to achieve this purpose? > > [1] > https://github.com/hongyi-zhao/ariadne/blob/838179bb4275ac85f5342d9e7d086d6ade3be1de/rc.py#L55 > > Regards, > HY > -- > https://mail.python.org/mailman/listinfo/python-list > From PythonList at DancesWithMice.info Thu Sep 9 17:36:36 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 10 Sep 2021 09:36:36 +1200 Subject: Friday Finking: Contorted loops Message-ID: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Why does Python not have a repeat-until loop construct? (or should that be 'modern programming languages'?) This is a perennial question (one contributor calling it "immemorial"), but there seem to be reasons why the Python Interpreter would find such a construct awkward, or is otherwise unable to comply. If so, what does one need to understand, in order to comprehend the (apparent) omission? NB I'm not asking 'how to do this with while?'. TLDR; - wherein the historical background is explored, a possible 'gap in knowledge' exposed, alternative implementations discussed, PEP-proposals critiqued, and related-questions (re-)asked at the end... If the question itself doesn't appeal to you, perhaps some of the discussion and web.refs (below) will. Happy Friday. Happy thinking! The term "Structured Programming" was coined by Edsger W Dijkstra. It proposed a number of "control structures" (which were largely unavailable in the programming languages of that time): - sequence: a series of statements/routines to be executed in sequence - selection: if...then, if...then...else..., case - iteration: while, repeat (do...until), for - recursion: a routine 'calling itself' as a cascade The 'content' or 'process' of each structure was a block (or in Python terminology: a "suite") consisting of any/all of the above (thus "nesting"). Python's indentation practice, today likely descended from this concept. Much of the development of the ideas behind Structured Programming that followed the crystallisation of this list of constructs, were attempts to mathematically (logically) 'prove' code as "correct". One of the ideas to (help?) make things more prove-able, was that each block and construct have only one way in (entry), and one way out (exit), eg (from Wikipedia) "The conditional statement should have at least one true condition and each condition should have one exit point at max ... Often it is recommended that each loop should only have one entry point (and in the original structural programming, also only one exit point, and a few languages enforce this)" which as they say, was an idea later dropped/felt to be somewhat impracticable (but to which theme I shall return...) Even in fairly modest Python constructs, we quickly repeal the one-in, one-out philosophy because try...except operates by providing another exit-path. The 'structures' (or "constructs") of Structured Programming were fore-runners of the Software Patterns and SOLID Principles commonly-practised today. These ideas still hold the same goal of trading a degree of abstraction for programming simplicity, possibly testability, and improved quality. Today, Python offers almost all of the SP constructs. A form of case/select is expected in v3.10. The continuing omission is repeat-until. If you have not met such a code-component before, the idea of a repeat...until (or do...until) might look like this: repeat: code-suite until condition Thus, the code-suite will be executed as many times as necessary, until the condition is met. In Python, we are used to while-loops, which can be expressed in the same style as: while condition: code-suite What's the difference? The answer is that the repeat's code-block MUST be executed at least once. Whereas a while's code-suite could be totally ignored and not executed at all! An analogy is to RegEx and its * and + repetitions: * means zero, one, or more matches + means (at least) one, or more matches During the last weeks 'here', writing a while-loop was a topic of conversation. A solution offered to the OP, can be described as: loop-init-code while True: #in other words, loop forever code-suite if condition: break Note three things: 1 the while condition has been bastardised - there is no meaningful condition, it is designed to loop without thought or control* 2 the control condition within and ending the loop's suite exactly replaces the until-condition of a repeat-until construct 3 the cyclomatic-complexity of the multi-faceted construct is much higher than of a 'pure' while-loop (or for-loop) NB "cyclomatic complexity" is an attempt to measure a program's complexity based on the number of distinct paths or branches in the code (please recall earlier comment about 'entry and exit'). * in one of the web.ref discussions, our own @Chris suggests taking advantage of the 'truthiness' of data, and inserting some documentation: while 'there is data to process': Which is a considerable improvement over the bland 'loop forever' or 'loop until I tell you otherwise, according to criteria I won't reveal until later' (an operating mode every?no teenager would accept, on-principle! - including this one...) This form is a regularly recommended as a 'solution' (see first Answer to SO question). However, it is likely to require some set-up (which is technically preceding, and therefore outside of the construct, yet the construct is highly-dependent upon it. (this may be unavoidable, regardless) Most importantly (regretfully), another construct has been added at the (middle or) end of the loop to perform work which has been displaced from the while-condition. So, whereas a repeat...until is a single construct encapsulating its code-suite, the while+True...if+condition-break, forms two constructs 'around' the code-suite - and in some circumstances the code-suite may be effectively split into two by the positioning of the added if+condition. None of this is calculated to lead to 'the simple life' and soothe minds into the Zen of Python! Whereas most of this discussion is at the theoretical level, I have spent several 'happy hours' hacking-away in the hope of finding a practical solution to, or work-around for, this issue. Mostly disappearing down the ast-and-exec rabbit-hole. In a sub-set of possible-solutions "the time has come [for] the walrus [operator]" ('improving' a line from a Lewis Carroll poem). However, most solutions will require some retention of 'state'. Accordingly, generators - which will also work in simpler cases. They in-turn led me all the way to a class. I'm still playing with that progression... Sadly, am of the feeling that the 'cure' may be (as much criticised and) more painful than 'the disease'... Returning to an earlier point: for the 'pure' while-loop there is exactly one way 'in' (entry), and one way 'out' (exit). The above, loop-forever idea complicates matters, because when reading the code, one's first understanding is that the while will control the indented code-suite beneath - be its (single) exit. However, further reading reveals that there is a second way 'out'. I should say, "theoretically", because while True offers no escape - that implies it is no "exit" (the "Hotel California" clause). So, reading the "while" creates an expectation, but that must be immediately discarded when our eyes reach the True-condition! In the military, we were taught to always have a (current and applicable) Escape Plan. If you've travelled by airplane/aeroplane you will remember the crew giving a Safety Drill, and asking you to be aware of your nearest exit should it be necessary to rapidly depart the plane - and that "the closest may be behind you". These plans have virtue, because in the chaos of an emergency, the time it takes to work-it-out on-the-fly (hah!) may cost your life (no joke!). To be sure, the extra time and effort required to read a bastardised Python while-loop is hardly likely to be a matter of life or death (I sincerely hope!), but it is grounds for complaint or 'muttering'. When introducing trainees to recursion, I repeat over-and-over (and -over) that the very first thing to do, is to code the termination condition (the Escape Plan)! If you've ever coded recursive constructs, you've almost certainly seen someone who hasn't followed this (simple) plan - in your bath-room mirror... The same principle applies to any while+True construct. Left alone, it will not stop. In this situation, having to prepare by thinking about an escape-route is a fundamental flaw. When you start coding the loop, your mind is more interested in the code-suite - the business of the loop! Once that is coded we will (in the normal course) be ready to think-about 'what happens next'. Sure, if one forgets the termination-clause, Python will save your life - and it is likely that no-one else will notice. Does that make it 'right'? Doesn't it indicate that there's a 'smell' of something wrong? In programming, consideration of "cognitive load" very much applies. We are unlikely to ever need to escape from a smoke-filled development environment, but we do have (more than) enough to think-about when coding. Indeed the essential virtue of Structured Programming, SOLID, software patterns, etc, is to reduce cognitive load by offering tried-and-tested solutions, templates/abstractions, re-usable code, etc. Am I the first to query the omission of repeat-until? No - not by a long-shot! Raymond Hettinger and Isaac Carroll proposed PEP 315 back in 2003. The BDFL said: ?Please reject the PEP. More variations along these lines won't make the language more elegant or easier to learn. They'd just save a few hasty folks some typing while making others who have to read/maintain their code wonder what it means.? It was rejected. Reading it now, the proposed syntax seems more than a little clumsy; but makes me wonder why a more 'usual' repeat-until format wasn't, or perhaps couldn't, be used (see Parser question). @Raymond had (at least one) another 'go' in 2009. His comments included: ?The challenge has been finding a syntax that fits well with the patterns in the rest of the language. It seems that every approach has it's own strengths and weaknesses...These seem syntactically weird to me and feel more like typos than real python code. I'm sure there are many ways to spell the last line, but in the two years since I first worked on the PEP, I haven't found any condition-at-the-end syntax that FeelsRight(tm).? Curiously, the last approach illustrated therein was to open the loop with do: and terminate it with while+condition. I'd certainly concur that the idea of using "while" at the 'head' of a while-loop and also at the 'foot' of a repeat-until construct, seems "weird" (also (Tm)?). Am not sure, perhaps didn't research far-enough, to see why another construct-name, eg "until" was not considered. Can you point-out where that is discussed, or give a reason 'why'? As recently as 2017, David Murray addressed this issue with PEP 548 "More Flexible Loop Control". Again, with all the benefits conferred by hind-sight, the idea seems clumsy: replacing the if+condition-break with break-if+condition (in similar fashion to ternary conditional operators, list-comprehensions, etc). It comes across as syntactic-sugar. It does not address the 'bastardisation' and extra-construct's cyclomatic-complexity rise. Many points raised 'here' appear in another post at about that time (see web.refs FYI). In many cases, another common recommendation follows the lines of: do-something while the-something-is-ok: do-more do-something This deserves criticism due to its code-repetition and thus contravention of the DRY principle (Do not Repeat Yourself) - the cyclomatic-complexity of the 'bastardisation' has been removed, but I'm still discomforted. Are you? As mentioned earlier, a major criticism is that something that is only being done to establish the looping mechanism (is "closely-coupled") is separate from, not (initially at least) an integral-part of the construct. That said, please recall earlier allowance, that some 'initialisation' may be necessary. Perhaps I missed it as life unfolded: has there been a paper/line of research which discounted the need for repeat-until, assuming a while construct was available? Is a repeat-until construct missing from other modern-languages, or is that a particular choice made in Python's design? A paper from Duke (University) makes reference to a "Loop and a Half" structure, with the particular example of completing an action until some "sentinel-value" is reached. They presage my comments (below) about "priming" the loop, the "sentinel" text as an additional construct, and/or duplicate code - and how all that adds-up to making "the loop body harder to understand since it turns a read-and-process loop into a process-and-read loop." With sundry further admissions, they rewrite into the bastardised form which has become Python's accepted-solution. Perhaps it was not possible before, but will it become feasible under Python's new (v3.9+) PEG parser? Web.Refs: https://en.wikipedia.org/wiki/Structured_programming Dijkstra "Notes on Structured Programming" https://dl.acm.org/doi/pdf/10.5555/1243380 Single-Entry, Single-Exit https://web.archive.org/web/20121114195652/http://msmvps.com/blogs/peterritchie/archive/2008/03/07/single-entry-single-exit-should-it-still-be-applicable-in-object-oriented-languages.aspx https://en.wikipedia.org/wiki/Cyclomatic_complexity Ned Batchelder's McCabe plug-in https://pypi.org/project/mccabe/ The Walrus and the Carpenter by Lewis Carroll https://poets.org/poem/walrus-and-carpenter Python's Parser https://www.python.org/dev/peps/pep-0617/ PEP 315 https://www.python.org/dev/peps/pep-0315/ BDFL Rejection https://mail.python.org/pipermail/python-ideas/2013-June/021610.html Later discussion https://mail.python.org/pipermail/python-ideas/2009-April/004306.html and https://mail.python.org/archives/list/python-ideas at python.org/thread/2VUZ3J6C4GSHGBZJW62AY4HPEEBMXAT6/#2VUZ3J6C4GSHGBZJW62AY4HPEEBMXAT6 PEP 548 https://www.python.org/dev/peps/pep-0548/ BDFL Rejection https://mail.python.org/pipermail/python-dev/2017-September/149232.html Python-Ideas post https://mail.python.org/archives/list/python-ideas at python.org/thread/EDNARFL2RGOE53SLWPTD5ZLJQOYSVDCR/#EDNARFL2RGOE53SLWPTD5ZLJQOYSVDCR Duke Paper https://users.cs.duke.edu/~ola/patterns/plopd/loops.html#loop-and-a-half RegEx in Python https://docs.python.org/3/library/re.html and https://docs.python.org/3/howto/regex.html "bastardise" (meaning 1) https://www.dictionary.com/browse/bastardize https://stackoverflow.com/questions/743164/how-to-emulate-a-do-while-loop DRY https://code.tutsplus.com/tutorials/3-key-software-principles-you-must-understand--net-25161 -- Regards, =dn From wlfraed at ix.netcom.com Thu Sep 9 19:07:49 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 09 Sep 2021 19:07:49 -0400 Subject: Friday Finking: Contorted loops References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On Fri, 10 Sep 2021 09:36:36 +1200, dn via Python-list declaimed the following: >Why does Python not have a repeat-until loop construct? >(or should that be 'modern programming languages'?) > I would suspect Python's indentation for block structure would be the major hindrance. After all, all existing block constructs /open/ the block. if ...: block else: block elif ...: block try: block except ...: block for ...: block while ...: block def ...: block class ...: block so how would repeat: block until ... fit the language. The alternative would be repeat until ...: block putting the condition at the top, even though it is only tested at the bottom (after processing at least once). Granted, that IS the style used in REXX, where DO/END are generic block boundary marks, with the DO accepting all the loop constructs (FOR, WHILE, UNTIL) as optional parts. >This is a perennial question (one contributor calling it "immemorial"), >but there seem to be reasons why the Python Interpreter would find such >a construct awkward, or is otherwise unable to comply. If so, what does >one need to understand, in order to comprehend the (apparent) omission? > >NB I'm not asking 'how to do this with while?'. > > >TLDR; >- wherein the historical background is explored, a possible 'gap in >knowledge' exposed, alternative implementations discussed, PEP-proposals >critiqued, and related-questions (re-)asked at the end... > > >If the question itself doesn't appeal to you, perhaps some of the >discussion and web.refs (below) will. Happy Friday. Happy thinking! > > >The term "Structured Programming" was coined by Edsger W Dijkstra. It >proposed a number of "control structures" (which were largely >unavailable in the programming languages of that time): > >- sequence: a series of statements/routines to be executed in sequence >- selection: if...then, if...then...else..., case >- iteration: while, repeat (do...until), for >- recursion: a routine 'calling itself' as a cascade > >The 'content' or 'process' of each structure was a block (or in Python >terminology: a "suite") consisting of any/all of the above (thus >"nesting"). Python's indentation practice, today likely descended from >this concept. > > >Much of the development of the ideas behind Structured Programming that >followed the crystallisation of this list of constructs, were attempts >to mathematically (logically) 'prove' code as "correct". > >One of the ideas to (help?) make things more prove-able, was that each >block and construct have only one way in (entry), and one way out >(exit), eg (from Wikipedia) "The conditional statement should have at >least one true condition and each condition should have one exit point >at max ... Often it is recommended that each loop should only have one >entry point (and in the original structural programming, also only one >exit point, and a few languages enforce this)" which as they say, was an >idea later dropped/felt to be somewhat impracticable (but to which theme >I shall return...) > >Even in fairly modest Python constructs, we quickly repeal the one-in, >one-out philosophy because try...except operates by providing another >exit-path. > > >The 'structures' (or "constructs") of Structured Programming were >fore-runners of the Software Patterns and SOLID Principles >commonly-practised today. These ideas still hold the same goal of >trading a degree of abstraction for programming simplicity, possibly >testability, and improved quality. > >Today, Python offers almost all of the SP constructs. A form of >case/select is expected in v3.10. The continuing omission is repeat-until. > > >If you have not met such a code-component before, the idea of a >repeat...until (or do...until) might look like this: > > repeat: > code-suite > until condition > >Thus, the code-suite will be executed as many times as necessary, until >the condition is met. > > >In Python, we are used to while-loops, which can be expressed in the >same style as: > > while condition: > code-suite > >What's the difference? > >The answer is that the repeat's code-block MUST be executed at least >once. Whereas a while's code-suite could be totally ignored and not >executed at all! > >An analogy is to RegEx and its * and + repetitions: > >* means zero, one, or more matches >+ means (at least) one, or more matches > > >During the last weeks 'here', writing a while-loop was a topic of >conversation. A solution offered to the OP, can be described as: > > loop-init-code > while True: #in other words, loop forever > code-suite > if condition: > break > >Note three things: > >1 the while condition has been bastardised - there is no meaningful >condition, it is designed to loop without thought or control* > >2 the control condition within and ending the loop's suite exactly >replaces the until-condition of a repeat-until construct > >3 the cyclomatic-complexity of the multi-faceted construct is much >higher than of a 'pure' while-loop (or for-loop) > >NB "cyclomatic complexity" is an attempt to measure a program's >complexity based on the number of distinct paths or branches in the code >(please recall earlier comment about 'entry and exit'). > >* in one of the web.ref discussions, our own @Chris suggests taking >advantage of the 'truthiness' of data, and inserting some documentation: > > while 'there is data to process': > >Which is a considerable improvement over the bland 'loop forever' or >'loop until I tell you otherwise, according to criteria I won't reveal >until later' (an operating mode every?no teenager would accept, >on-principle! - including this one...) > > >This form is a regularly recommended as a 'solution' (see first Answer >to SO question). However, it is likely to require some set-up (which is >technically preceding, and therefore outside of the construct, yet the >construct is highly-dependent upon it. (this may be unavoidable, regardless) > >Most importantly (regretfully), another construct has been added at the >(middle or) end of the loop to perform work which has been displaced >from the while-condition. > >So, whereas a repeat...until is a single construct encapsulating its >code-suite, the while+True...if+condition-break, forms two constructs >'around' the code-suite - and in some circumstances the code-suite may >be effectively split into two by the positioning of the added if+condition. > >None of this is calculated to lead to 'the simple life' and soothe minds >into the Zen of Python! > > >Whereas most of this discussion is at the theoretical level, I have >spent several 'happy hours' hacking-away in the hope of finding a >practical solution to, or work-around for, this issue. Mostly >disappearing down the ast-and-exec rabbit-hole. > >In a sub-set of possible-solutions "the time has come [for] the walrus >[operator]" ('improving' a line from a Lewis Carroll poem). > >However, most solutions will require some retention of 'state'. >Accordingly, generators - which will also work in simpler cases. > >They in-turn led me all the way to a class. I'm still playing with that >progression... > >Sadly, am of the feeling that the 'cure' may be (as much criticised and) >more painful than 'the disease'... > > >Returning to an earlier point: for the 'pure' while-loop there is >exactly one way 'in' (entry), and one way 'out' (exit). The above, >loop-forever idea complicates matters, because when reading the code, >one's first understanding is that the while will control the indented >code-suite beneath - be its (single) exit. However, further reading >reveals that there is a second way 'out'. I should say, "theoretically", >because while True offers no escape - that implies it is no "exit" (the >"Hotel California" clause). So, reading the "while" creates an >expectation, but that must be immediately discarded when our eyes reach >the True-condition! > > >In the military, we were taught to always have a (current and >applicable) Escape Plan. If you've travelled by airplane/aeroplane you >will remember the crew giving a Safety Drill, and asking you to be aware >of your nearest exit should it be necessary to rapidly depart the plane >- and that "the closest may be behind you". These plans have virtue, >because in the chaos of an emergency, the time it takes to work-it-out >on-the-fly (hah!) may cost your life (no joke!). > >To be sure, the extra time and effort required to read a bastardised >Python while-loop is hardly likely to be a matter of life or death (I >sincerely hope!), but it is grounds for complaint or 'muttering'. > >When introducing trainees to recursion, I repeat over-and-over (and >-over) that the very first thing to do, is to code the termination >condition (the Escape Plan)! If you've ever coded recursive constructs, >you've almost certainly seen someone who hasn't followed this (simple) >plan - in your bath-room mirror... > >The same principle applies to any while+True construct. Left alone, it >will not stop. In this situation, having to prepare by thinking about an >escape-route is a fundamental flaw. When you start coding the loop, your >mind is more interested in the code-suite - the business of the loop! >Once that is coded we will (in the normal course) be ready to >think-about 'what happens next'. > >Sure, if one forgets the termination-clause, Python will save your life >- and it is likely that no-one else will notice. Does that make it >'right'? Doesn't it indicate that there's a 'smell' of something wrong? > >In programming, consideration of "cognitive load" very much applies. We >are unlikely to ever need to escape from a smoke-filled development >environment, but we do have (more than) enough to think-about when >coding. Indeed the essential virtue of Structured Programming, SOLID, >software patterns, etc, is to reduce cognitive load by offering >tried-and-tested solutions, templates/abstractions, re-usable code, etc. > > >Am I the first to query the omission of repeat-until? No - not by a >long-shot! Raymond Hettinger and Isaac Carroll proposed PEP 315 back in >2003. The BDFL said: ?Please reject the PEP. More variations along these >lines won't make the language more elegant or easier to learn. They'd >just save a few hasty folks some typing while making others who have to >read/maintain their code wonder what it means.? It was rejected. > >Reading it now, the proposed syntax seems more than a little clumsy; but >makes me wonder why a more 'usual' repeat-until format wasn't, or >perhaps couldn't, be used (see Parser question). > > >@Raymond had (at least one) another 'go' in 2009. His comments included: >?The challenge has been finding a syntax that fits well with the >patterns in the rest of the language. It seems that every approach has >it's own strengths and weaknesses...These seem syntactically weird to me >and feel more like typos than real python code. I'm sure there are many >ways to spell the last line, but in the two years since I first worked >on the PEP, I haven't found any condition-at-the-end syntax that >FeelsRight(tm).? > >Curiously, the last approach illustrated therein was to open the loop >with do: and terminate it with while+condition. I'd certainly concur >that the idea of using "while" at the 'head' of a while-loop and also at >the 'foot' of a repeat-until construct, seems "weird" (also (Tm)?). Am >not sure, perhaps didn't research far-enough, to see why another >construct-name, eg "until" was not considered. Can you point-out where >that is discussed, or give a reason 'why'? > > >As recently as 2017, David Murray addressed this issue with PEP 548 >"More Flexible Loop Control". Again, with all the benefits conferred by >hind-sight, the idea seems clumsy: replacing the if+condition-break with >break-if+condition (in similar fashion to ternary conditional operators, >list-comprehensions, etc). It comes across as syntactic-sugar. It does >not address the 'bastardisation' and extra-construct's >cyclomatic-complexity rise. > >Many points raised 'here' appear in another post at about that time (see >web.refs FYI). > > >In many cases, another common recommendation follows the lines of: > > do-something > while the-something-is-ok: > do-more > do-something > >This deserves criticism due to its code-repetition and thus >contravention of the DRY principle (Do not Repeat Yourself) - the >cyclomatic-complexity of the 'bastardisation' has been removed, but I'm >still discomforted. Are you? > >As mentioned earlier, a major criticism is that something that is only >being done to establish the looping mechanism (is "closely-coupled") is >separate from, not (initially at least) an integral-part of the >construct. That said, please recall earlier allowance, that some >'initialisation' may be necessary. > > >Perhaps I missed it as life unfolded: has there been a paper/line of >research which discounted the need for repeat-until, assuming a while >construct was available? > >Is a repeat-until construct missing from other >modern-languages, or is that a particular choice made in Python's design? > > >A paper from Duke (University) makes reference to a "Loop and a Half" >structure, with the particular example of completing an action until >some "sentinel-value" is reached. They presage my comments (below) about >"priming" the loop, the "sentinel" text as an additional construct, >and/or duplicate code - and how all that adds-up to making "the loop >body harder to understand since it turns a read-and-process loop into a >process-and-read loop." With sundry further admissions, they rewrite >into the bastardised form which has become Python's accepted-solution. > > >Perhaps it was not possible before, but will it become feasible under >Python's new (v3.9+) PEG parser? > > > >Web.Refs: >https://en.wikipedia.org/wiki/Structured_programming >Dijkstra "Notes on Structured Programming" >https://dl.acm.org/doi/pdf/10.5555/1243380 >Single-Entry, Single-Exit >https://web.archive.org/web/20121114195652/http://msmvps.com/blogs/peterritchie/archive/2008/03/07/single-entry-single-exit-should-it-still-be-applicable-in-object-oriented-languages.aspx >https://en.wikipedia.org/wiki/Cyclomatic_complexity >Ned Batchelder's McCabe plug-in https://pypi.org/project/mccabe/ >The Walrus and the Carpenter by Lewis Carroll >https://poets.org/poem/walrus-and-carpenter >Python's Parser https://www.python.org/dev/peps/pep-0617/ >PEP 315 https://www.python.org/dev/peps/pep-0315/ >BDFL Rejection >https://mail.python.org/pipermail/python-ideas/2013-June/021610.html >Later discussion >https://mail.python.org/pipermail/python-ideas/2009-April/004306.html >and >https://mail.python.org/archives/list/python-ideas at python.org/thread/2VUZ3J6C4GSHGBZJW62AY4HPEEBMXAT6/#2VUZ3J6C4GSHGBZJW62AY4HPEEBMXAT6 >PEP 548 https://www.python.org/dev/peps/pep-0548/ >BDFL Rejection >https://mail.python.org/pipermail/python-dev/2017-September/149232.html >Python-Ideas post >https://mail.python.org/archives/list/python-ideas at python.org/thread/EDNARFL2RGOE53SLWPTD5ZLJQOYSVDCR/#EDNARFL2RGOE53SLWPTD5ZLJQOYSVDCR >Duke Paper >https://users.cs.duke.edu/~ola/patterns/plopd/loops.html#loop-and-a-half >RegEx in Python https://docs.python.org/3/library/re.html >and https://docs.python.org/3/howto/regex.html >"bastardise" (meaning 1) https://www.dictionary.com/browse/bastardize >https://stackoverflow.com/questions/743164/how-to-emulate-a-do-while-loop >DRY >https://code.tutsplus.com/tutorials/3-key-software-principles-you-must-understand--net-25161 > >-- >Regards, >=dn -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From tjreedy at udel.edu Thu Sep 9 19:47:10 2021 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 9 Sep 2021 19:47:10 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 9/9/2021 5:36 PM, dn via Python-list wrote: > Why does Python not have a repeat-until loop construct? 1. It is not needed. You covered that. 2. It is rare useful. For loops are common. While loops are occasional (nearly an order of magnitude less common than for loops. Fractional loop constructs are rare. ("loop-and-a-half" is a mislabel since at not even one loop is guaranteed.) "do-while" or "repeat-until is even rarer since fractional-loop include this as a special case. 3. Adding 'until' as a keyword *now*, rather than in 1.0 or at least several versions ago, has cost so far judged to outweigh the small benefit. The PEP parser makes contextual keywords much more easily possible but there is a cost to having a context dependent grammar. Consider this 3.10.0 snippet: >>> match, case = 1, 1 >>> match match: ... case case: ... print('matched') ... ... matched >>> match case: ... case match: ... print('matched') ... ... matched To me, having a word sometimes be a keyword and sometime not make code harder to read. In IDLE, it is a bit easier as the keyword uses of 'match' and 'case' above are correctly highlighted as keywords, and the non-keywords uses not highlighted. But this is harder that for full-time keywords with sane code that works in an re-based highlighter. Underscore, not used above, but also a new contextual keyword, is even harder. Three of us could not get all the cases we tested correct and I suspect doing so without running the PEG parser may be impossible. Since highlighting is redone with each keystroke, I suspect doing the latter would add a noticeable and unacceptable lag between keystrokes and display. -- Terry Jan Reedy From 2QdxY4RzWzUUiLuE at potatochowder.com Thu Sep 9 21:19:45 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Thu, 9 Sep 2021 18:19:45 -0700 Subject: Friday Finking: Contorted loops In-Reply-To: References: Message-ID: On 2021-09-09 at 22:33:16 +0000, Stefan Ram wrote: > One can think of a language where every loop is exited this > way, the only loop construct would be > > loop > ... > > and it would /always/ have to be exited via enclosed breaks. I'm not quite sure what you mean by "one can," but that's how the simple form of Common Lisp's loop macro works. Friendlier looping constructs are built with other macros on top of that one and/or its underlying mechanism.? ? The underlying mechanism is the moral equivalent of a Python suite that can also contain tags and unconditional jumps to those tags, aka "goto"s. From wlfraed at ix.netcom.com Thu Sep 9 23:33:33 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 09 Sep 2021 23:33:33 -0400 Subject: Friday Finking: Contorted loops References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On Thu, 09 Sep 2021 19:07:49 -0400, Dennis Lee Bieber declaimed the following: >On Fri, 10 Sep 2021 09:36:36 +1200, dn via Python-list > declaimed the following: Someone, please shoot me now... >>This is a perennial question (one contributor calling it "immemorial"), >>but there seem to be reasons why the Python Interpreter would find such >>a construct awkward, or is otherwise unable to comply. If so, what does >>one need to understand, in order to comprehend the (apparent) omission? >> How did I ever let that whole quote pass through without trimming... -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From hongyi.zhao at gmail.com Fri Sep 10 01:04:40 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Thu, 9 Sep 2021 22:04:40 -0700 (PDT) Subject: Change the display style of the text on the STACKLINE. In-Reply-To: References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> Message-ID: <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> On Thursday, September 9, 2021 at 8:57:37 PM UTC+8, Roland Mueller wrote: > Hello > > to 9. syysk. 2021 klo 6.53 hongy... at gmail.com (hongy... at gmail.com) > kirjoitti: > > I'm using the following code in my forked project [1]: > > > > percol.view.STACKLINE = 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s Dir:M-d > > Dircmd:M-b' > > > > I would like to change the display style of the text mentioned above, for > > example, to underline some characters in it, as shown below: > > > > _D_ir:M-d > > > > > You can use e.g. str.replace() or re.sub() > > >>> percol.view.STACKLINE = percol.view.STACKLINE.replace('D', '_D_') > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d _D_ircmd:M-b' > > >>> import re > > Replace D with _D_ > >>> percol.view.STACKLINE = re.sub(r'([D])', > r'_\1_', percol.view.STACKLINE) > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d _D_ircmd:M-b' > > Replace D and M with _D_, _M_ > >>> percol.view.STACKLINE = re.sub(r'([DM])', r'_\1_', > percol.view.STACKLINE) > 'Fold:F1,F2,F3 Push:C-p Pop:_M_-p Script:_M_-s _D_ir:_M_-d _D_ircmd:_M_-b' > > Regards, > Roland I tried with the following, but failed to achieve the expected effect: class Term: HEADER = '\033[95m' OKBLUE = '\033[94m' OKGREEN = '\033[92m' WARNING = '\033[93m' FAIL = '\033[91m' ENDC = '\033[0m' LIGHTCYAN = '\033[1;36m' LIGHTGRAY = '\033[0;37m' YELLOW = '\033[0;33m' BOLD = '\033[1m' UNDERLINE = '\033[4m' [...] percol.view.STACKLINE = percol.view.STACKLINE.replace('D', Term.UNDERLINE + 'D' + Term.ENDC) The result will look like this: Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s ?[4mD?[0mir:M-d ?[4mD?[0mircmd:M-b Regards, HY > > How to achieve this purpose? > > > > [1] > > https://github.com/hongyi-zhao/ariadne/blob/838179bb4275ac85f5342d9e7d086d6ade3be1de/rc.py#L55 > > > > Regards, > > HY > > -- > > https://mail.python.org/mailman/listinfo/python-list > > From roland.em0001 at googlemail.com Fri Sep 10 02:11:58 2021 From: roland.em0001 at googlemail.com (Roland Mueller) Date: Fri, 10 Sep 2021 09:11:58 +0300 Subject: Change the display style of the text on the STACKLINE. In-Reply-To: <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> Message-ID: pe 10. syysk. 2021 klo 8.53 hongy... at gmail.com (hongyi.zhao at gmail.com) kirjoitti: > On Thursday, September 9, 2021 at 8:57:37 PM UTC+8, Roland Mueller wrote: > > Hello > > > > to 9. syysk. 2021 klo 6.53 hongy... at gmail.com (hongy... at gmail.com) > > kirjoitti: > > > I'm using the following code in my forked project [1]: > > > > > > percol.view.STACKLINE = 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s > Dir:M-d > > > Dircmd:M-b' > > > > > > I would like to change the display style of the text mentioned above, > for > > > example, to underline some characters in it, as shown below: > > > > > > _D_ir:M-d > > > > > > > > You can use e.g. str.replace() or re.sub() > > > > >>> percol.view.STACKLINE = percol.view.STACKLINE.replace('D', '_D_') > > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d > _D_ircmd:M-b' > > > > >>> import re > > > > Replace D with _D_ > > >>> percol.view.STACKLINE = re.sub(r'([D])', > > r'_\1_', percol.view.STACKLINE) > > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d > _D_ircmd:M-b' > > > > Replace D and M with _D_, _M_ > > >>> percol.view.STACKLINE = re.sub(r'([DM])', r'_\1_', > > percol.view.STACKLINE) > > 'Fold:F1,F2,F3 Push:C-p Pop:_M_-p Script:_M_-s _D_ir:_M_-d > _D_ircmd:_M_-b' > > > > Regards, > > Roland > > I tried with the following, but failed to achieve the expected effect: > > class Term: > HEADER = '\033[95m' > OKBLUE = '\033[94m' > OKGREEN = '\033[92m' > WARNING = '\033[93m' > FAIL = '\033[91m' > ENDC = '\033[0m' > LIGHTCYAN = '\033[1;36m' > LIGHTGRAY = '\033[0;37m' > YELLOW = '\033[0;33m' > BOLD = '\033[1m' > UNDERLINE = '\033[4m' > > [...] > > percol.view.STACKLINE = percol.view.STACKLINE.replace('D', Term.UNDERLINE > + 'D' + Term.ENDC) > > The result will look like this: > > Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s ?[4mD?[0mir:M-d > ?[4mD?[0mircmd:M-b > > I cannot repeat that. Are you sure that the '?' shown in your output are not due to your terminal settings that influence how strings printed by Python or inside used terminal are shown? Python 3.9.6 (default, Jul 16 2021, 00:00:00) [GCC 11.1.1 20210531 (Red Hat 11.1.1-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> UL = '\033[4m' >>> UL '\x1b[4m' >>> ENDC = '\033[0m' >>> ENDC '\x1b[0m' >>> s = UL + 'D' + ENDC >>> s '\x1b[4mD\x1b[0m' >>> s = 'ABCDE' >>> s = s.replace('D', UL + 'D' + ENDC) >>> s 'ABC\x1b[4mD\x1b[0mE' When I call print(s) it even shows ABCD and D is underscored. But copying the output to mail looses the underscore ... [image: image.png] BR, Roland > Regards, > HY > > > > > How to achieve this purpose? > > > > > > [1] > > > > https://github.com/hongyi-zhao/ariadne/blob/838179bb4275ac85f5342d9e7d086d6ade3be1de/rc.py#L55 > > > > > > Regards, > > > HY > > > -- > > > https://mail.python.org/mailman/listinfo/python-list > > > > -- > https://mail.python.org/mailman/listinfo/python-list > From greg.ewing at canterbury.ac.nz Fri Sep 10 02:39:57 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 10 Sep 2021 18:39:57 +1200 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 10/09/21 11:47 am, Terry Reedy wrote: > 2. It is rare useful.? For loops are common.? While loops are occasional > (nearly an order of magnitude less common than for loops.? Fractional > loop constructs are rare. I would say that fractional loops are more common than loops which truly need to execute completely at least once, and aren't bugs waiting to be triggered by an edge case such as empty input. I seem to remember someone - maybe Wirth? - long ago expressing the opinion that repeat-until loops often tended to be error prone, but I can't provide a reference, sorry. -- Greg From greg.ewing at canterbury.ac.nz Fri Sep 10 02:48:20 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 10 Sep 2021 18:48:20 +1200 Subject: Change the display style of the text on the STACKLINE. In-Reply-To: References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> Message-ID: On 10/09/21 6:11 pm, Roland Mueller wrote: > When I call print(s) it even shows ABCD and D is underscored. But copying > the output to mail looses the underscore ... If the terminal understands unicode, Combining Low Line: U+0332 might help -- Greg From loris.bennett at fu-berlin.de Fri Sep 10 07:24:03 2021 From: loris.bennett at fu-berlin.de (Loris Bennett) Date: Fri, 10 Sep 2021 13:24:03 +0200 Subject: Making command-line args available to deeply-nested functions References: <87czq8tpay.fsf@hornfels.zedat.fu-berlin.de> <230149ee-f6ed-4e2f-8ac3-c12388ca45acn@googlegroups.com> <878s0wtljf.fsf@hornfels.zedat.fu-berlin.de> <87sfz0y0gx.fsf@hornfels.zedat.fu-berlin.de> <87sfyw7cvn.fsf@hornfels.zedat.fu-berlin.de> Message-ID: <877dfok71o.fsf@hornfels.zedat.fu-berlin.de> George Fischhof writes: > George Fischhof ezt ?rta (id?pont: 2021. aug. 29., V, > 21:27): > >> >> >> Loris Bennett ezt ?rta (id?pont: 2021. aug. >> 26., Cs, 16:02): >> >>> George Fischhof writes: >>> >>> [snip (79 lines)] >>> >>> >> > Hi, >>> >> > >>> >> > Also you can give a try to click and / or typer packages. >>> >> > Putting args into environment variables can be a solution too >>> >> > All of these depends on several things: personal preferences, >>> colleagues >>> >> / >>> >> > firm standards, the program, readability, variable accessibility (IDE >>> >> > support, auto completition) (env vars not supported by IDEs as they >>> are >>> >> not >>> >> > part of code) >>> >> >>> >> Thanks for the pointers, although I have only just got my head around >>> >> argparse/configargparse, so click is something I might have a look at >>> >> for future project. >>> >> >>> >> However, the question of how to parse the arguments is somewhat >>> separate >>> >> from that of how to pass (or not pass) the arguments around within a >>> >> program. >>> >>> [snip (16 lines)] >>> > >>> > Hi, >>> > I thought not just parsing, but the usage method: you add a decorator to >>> > the function where you want to use the parameters. This way you do not >>> have >>> > to pass the value through the calling hierarchy. >>> > >>> > Note: typer is a newer package, it contains click and leverages command >>> > line parsing even more. >>> >>> Do you have an example of how this is done? From a cursory reading of >>> the documentation, it didn't seem obvious to me how to do this, but then >>> I don't have much understanding of how decorators work. >>> >>> Cheers, >>> >>> Loris >>> >>> >>> -- >>> This signature is currently under construction. >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >> >> >> Hi, >> >> will create a sample code on Monday - Tuesday >> >> BR, >> George >> > > > Hi, > > here is the program ;-) (see below) > typer does not uses decorators, to solve this problem they advice to use > click's decorators, mixing typer and click. > Practically I prefer not to mix them, also the parts for easiest way to do > this just available in latest click, which is not supported in typer. > > So I created all the stuff in click, 8.x should be used > > BR, > George > > > import click > > > # read command line parameters > @click.command() > @click.option('--level_1', help='Level 1') > @click.option('--level_2', help='Level 2') > def main(level_1, level_2): > # put command line parameters into global context > ctx = click.get_current_context() > ctx.meta['level_1'] = level_1 > ctx.meta['level_2'] = level_2 > > level_1_function() > > > # pass / inject level_1 parameter to this function > @click.decorators.pass_meta_key('level_1') > def level_1_function(level_1): > print(f'level 1 variable: {level_1}') > level_2_function() > > > # pass / inject level_2 parameter to this function > @click.decorators.pass_meta_key('level_2') > def level_2_function(level_2): > print(f'level 2 variable: {level_2}') > > > if __name__ == "__main__": > main() Thanks for the example - that's very interesting. However, after a bit of reflection I think I am going to stick to explicit argument passing, so that I can have more generic modules that can be used by other programs. I'll then just encapsulate the argument parsing in a single function corresponding to the command line tool. Cheers, Loris -- This signature is currently under construction. From alan.gauld at yahoo.co.uk Fri Sep 10 07:26:24 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 10 Sep 2021 12:26:24 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 10/09/2021 00:47, Terry Reedy wrote: > even one loop is guaranteed.) "do-while" or "repeat-until is even rarer > since fractional-loop include this as a special case. Is there any empirical evidence to support this? Or is it just a case of using the tools that are available? In my experience of using Pascal (and much later with Delphi) that I used repeat loops at least as often as while loops, possibly more. But using Python and to a lesser extent C (which has a rather horrible do/while) construct I use while loops (often with an if-break) simply because that's what the language offers. So is it the case that the "need" for repeat loops is rare, simply a result of there being no native repeat loop available? After all we could have done without a for loop too and just used a while loop for everything (as was done in Oberon(?) ) Would we then state that the use of for loops was rare? But I would hope that any empirical research would look at the wider function of the loop and its purpose rather than merely analyzing the syntax and keywords. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Fri Sep 10 07:38:26 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 10 Sep 2021 12:38:26 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 09/09/2021 22:36, dn via Python-list wrote: > Even in fairly modest Python constructs, we quickly repeal the one-in, > one-out philosophy because try...except operates by providing another > exit-path. Exceptions are exceptional by their nature (or should be!) As such they can arguably be excused from the SP strictures. But python complicates this tenet still further by adding an else clause to its loops. And complicating this still more is that these else clauses have almost exactly opposite effects. while...else... executes the else if the body of the loop does NOT get executed. for...else... executes the else iff ALL iterations of the for loop DO complete. This confuses beginners immensely - and quite a few non beginners too; which is probably why they are not often seen "in the wild". This adds to the question of where exactly does a Python loop end? Is it after the code-suite following the loop construct? Or is it after the else code-suite, where such exists? Returning to the specific case of a repeat structure. In the case of a while loop the else offers another option: while condition loop-body else loop-body Now loop-body always gets executed at least once. But at the cost of duplicating the loop-body code, thus violating DRY. Just another thought... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From python at mrabarnett.plus.com Fri Sep 10 11:36:39 2021 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 10 Sep 2021 16:36:39 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> On 2021-09-10 12:38, Alan Gauld via Python-list wrote: > On 09/09/2021 22:36, dn via Python-list wrote: > >> Even in fairly modest Python constructs, we quickly repeal the one-in, >> one-out philosophy because try...except operates by providing another >> exit-path. > > Exceptions are exceptional by their nature (or should be!) As such > they can arguably be excused from the SP strictures. > > But python complicates this tenet still further by adding an else > clause to its loops. And complicating this still more is that these > else clauses have almost exactly opposite effects. > > while...else... > > executes the else if the body of the loop does NOT get executed. > > for...else... > > executes the else iff ALL iterations of the for loop DO complete. > [snip] In both cases, it executes the 'else' part if it didn't break out of the loop. That's it. If all of the iterations completed, then there was no break, so the 'else' part is executed. If there were no iterations, then there was no break, so the 'else' part is executed. It's the same for both of them. From alan.gauld at yahoo.co.uk Fri Sep 10 11:57:39 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 10 Sep 2021 16:57:39 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> Message-ID: On 10/09/2021 16:36, MRAB wrote: >> while...else... >> >> executes the else if the body of the loop does NOT get executed. >> >> for...else... >> >> executes the else iff ALL iterations of the for loop DO complete. >> > [snip] > > In both cases, it executes the 'else' part if it didn't break out of the > loop. That's it. OK, That's a useful perspective that is at least consistent. Unfortunately it's not how beginners perceive it and it causes regular confusion about how/when they should use else with a loop. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From roland.em0001 at googlemail.com Fri Sep 10 16:54:22 2021 From: roland.em0001 at googlemail.com (Roland Mueller) Date: Fri, 10 Sep 2021 23:54:22 +0300 Subject: Change the display style of the text on the STACKLINE. In-Reply-To: References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> Message-ID: pe 10. syysk. 2021 klo 17.22 Greg Ewing (greg.ewing at canterbury.ac.nz) kirjoitti: > On 10/09/21 6:11 pm, Roland Mueller wrote: > > When I call print(s) it even shows ABCD and D is underscored. But > copying > > the output to mail looses the underscore ... > > If the terminal understands unicode, Combining Low Line: U+0332 > might help > > > The problem was to get the string to this mail. In my terminal it was OK. From pfeiffer at cs.nmsu.edu Fri Sep 10 17:08:19 2021 From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer) Date: Fri, 10 Sep 2021 15:08:19 -0600 Subject: Friday Finking: Contorted loops References: Message-ID: <1blf44f8ak.fsf@pfeifferfamily.net> ram at zedat.fu-berlin.de (Stefan Ram) writes: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >>can be misleading, because the "..." part can still contain >>"break", "raise", "continue", and "return" statement. So one >>better should always be on the watch when reading source code >>of a language like Python than relying only on the condition >>behind the "while". > > The existence of statements like "break" renders > proof techniques for loops (such as Hoare's) with > their invariants and inference rules unapplicable. Also the reason to avoid repeat-until loops: the loop "invariant" isn't the same on the first iteration as on subsequent iterations. From avigross at verizon.net Fri Sep 10 17:27:22 2021 From: avigross at verizon.net (Avi Gross) Date: Fri, 10 Sep 2021 17:27:22 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> Message-ID: <023701d7a68a$a481d9f0$ed858dd0$@verizon.net> So why use the word "else" when it really does not mean what users consider else? Again, we have words like "finally" used in some places to mean it should be done no matter what, like closing a file that may be open. What phrase used either in one or all contexts might have been better, if longer? I mean in the case where your while is NOT entered, "else" almost makes sense. But it could also have been loop_skipped or something. In other cases, it could be some name like "loop_completed_normally" or whatever. Reusing the keyword "else" over and over for "if" statements and perhaps some kind of case/switch statement and while loops and so on, may be parsimonious but ... -----Original Message----- From: Python-list On Behalf Of Alan Gauld via Python-list Sent: Friday, September 10, 2021 11:58 AM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 10/09/2021 16:36, MRAB wrote: >> while...else... >> >> executes the else if the body of the loop does NOT get executed. >> >> for...else... >> >> executes the else iff ALL iterations of the for loop DO complete. >> > [snip] > > In both cases, it executes the 'else' part if it didn't break out of > the loop. That's it. OK, That's a useful perspective that is at least consistent. Unfortunately it's not how beginners perceive it and it causes regular confusion about how/when they should use else with a loop. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list From avigross at verizon.net Fri Sep 10 17:36:22 2021 From: avigross at verizon.net (Avi Gross) Date: Fri, 10 Sep 2021 17:36:22 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: <026001d7a68b$e6865ff0$b3931fd0$@verizon.net> In this discussion, I shudder to mention that people often simply use all kinds of additional logic to get a result. How much code have you seen that has some variable like "completed = False" that can be set to True in multiple places and many areas inside the loop are enclosed in an IF statement that only executes when completed remains False. So there is one clean exit in the sense that even when only half a loop has been done, it continues to the end of the loop and leaves before the next iteration. True, there is no break or return from the middle of the loop but logically there is if not for the convoluted code to avoid it. Similarly, can most "while" loops that you want to be "until" loops not be made with a bit of code? I mean set first_time to True before starting. Set your while condition to while first_time OR condition or some other such logic. That guarantees you go into the loop even when condition is False. Within the loop, negate first_time. Does that look more like a simulated repeat until, with extra overhead? As I see it, there are many viewpoints here. From a programming perspective, it is nice to be able to state the overall shape of what you are doing in an upfront-way and also something others can read. Some things like the C-style for loop provide a bit of this but in a way I think outsiders may stare at as in for (initialize; compare-condition; iterate-change) { ... } That is sort of compact but I have seen it get quite complex. If designed for readers, it might be designed a bit like what we do with keywords in functions headers where you might specify the "names" of each such section to make it clearer, and not just positional. But some forms of loops like do {...} until ... Make you have to scan forward to see what makes them end. That is not necessarily bad as you may need to read the code to see how it sets up the variables controlling the exit condition. But if you want a wide open setup, where the conditions for the loop being entered can be specified, then the condition for it to be repeated (if different) can be specified and the condition at the end that makes you exit without trying the condition on top, you can go nuts. As mentioned, some languages have else clauses or finally clauses and error handling with things like try() or some kind of on.exit() can cause weird things. Some languages may even want you to be able to test some condition automatically after every single statement and exit immediately. Even if you disagree with the idea of picking a few constructs that are commonly used or encouraged, you may want to consider what happens when you make a language so bloated that compiling or interpreting it becomes a big challenge and it has too many keywords. When it comes to other perspectives like having algorithms able to evaluate a program and prove it has no bugs, you may end up with a very restricted programming language and still fail. Throw in the reality that your loop may use variable manipulated in parallel by other threads and that your thread may be killed as it runs and I wonder. -----Original Message----- From: Python-list On Behalf Of Greg Ewing Sent: Friday, September 10, 2021 2:40 AM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 10/09/21 11:47 am, Terry Reedy wrote: > 2. It is rare useful. For loops are common. While loops are > occasional (nearly an order of magnitude less common than for loops. > Fractional loop constructs are rare. I would say that fractional loops are more common than loops which truly need to execute completely at least once, and aren't bugs waiting to be triggered by an edge case such as empty input. I seem to remember someone - maybe Wirth? - long ago expressing the opinion that repeat-until loops often tended to be error prone, but I can't provide a reference, sorry. -- Greg -- https://mail.python.org/mailman/listinfo/python-list From flaskee at protonmail.com Fri Sep 10 18:49:14 2021 From: flaskee at protonmail.com (flaskee) Date: Fri, 10 Sep 2021 22:49:14 +0000 Subject: leave Message-ID: Sent with [ProtonMail](https://protonmail.com/) Secure Email. From 2QdxY4RzWzUUiLuE at potatochowder.com Fri Sep 10 19:30:50 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Fri, 10 Sep 2021 16:30:50 -0700 Subject: Friday Finking: Contorted loops In-Reply-To: <1blf44f8ak.fsf@pfeifferfamily.net> References: <1blf44f8ak.fsf@pfeifferfamily.net> Message-ID: On 2021-09-10 at 15:08:19 -0600, Joe Pfeiffer wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: > > The existence of statements like "break" renders > > proof techniques for loops (such as Hoare's) with > > their invariants and inference rules unapplicable. > > Also the reason to avoid repeat-until loops: the loop "invariant" isn't > the same on the first iteration as on subsequent iterations. I am by no means an expert, nor likely even a neophyte, but why would the loop invariant not be the same on the first iteration? I can certainly see that the exit condition may not make sense at the beginning of the first iteration (e.g., there is not yet any data to compare to the sentinel), but ISTM that claiming that the exit condition is a loop invariant isn't kosher (because all you're claiming is that the compiler works). I can also see that certain state information may not be captured until the end of the first iteration. But presumably said state information can change from iteration to iteration, so I can't see how you'd derive an invariant involving it. From hongyi.zhao at gmail.com Fri Sep 10 19:43:30 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Fri, 10 Sep 2021 16:43:30 -0700 (PDT) Subject: Change the display style of the text on the STACKLINE. In-Reply-To: References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> Message-ID: <2f95335c-6ebc-4917-9bb5-ed9991586a82n@googlegroups.com> On Friday, September 10, 2021 at 2:12:31 PM UTC+8, Roland Mueller wrote: > pe 10. syysk. 2021 klo 8.53 hongy... at gmail.com (hongy... at gmail.com) > kirjoitti: > > On Thursday, September 9, 2021 at 8:57:37 PM UTC+8, Roland Mueller wrote: > > > Hello > > > > > > to 9. syysk. 2021 klo 6.53 hongy... at gmail.com (hongy... at gmail.com) > > > kirjoitti: > > > > I'm using the following code in my forked project [1]: > > > > > > > > percol.view.STACKLINE = 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s > > Dir:M-d > > > > Dircmd:M-b' > > > > > > > > I would like to change the display style of the text mentioned above, > > for > > > > example, to underline some characters in it, as shown below: > > > > > > > > _D_ir:M-d > > > > > > > > > > > You can use e.g. str.replace() or re.sub() > > > > > > >>> percol.view.STACKLINE = percol.view.STACKLINE.replace('D', '_D_') > > > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d > > _D_ircmd:M-b' > > > > > > >>> import re > > > > > > Replace D with _D_ > > > >>> percol.view.STACKLINE = re.sub(r'([D])', > > > r'_\1_', percol.view.STACKLINE) > > > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d > > _D_ircmd:M-b' > > > > > > Replace D and M with _D_, _M_ > > > >>> percol.view.STACKLINE = re.sub(r'([DM])', r'_\1_', > > > percol.view.STACKLINE) > > > 'Fold:F1,F2,F3 Push:C-p Pop:_M_-p Script:_M_-s _D_ir:_M_-d > > _D_ircmd:_M_-b' > > > > > > Regards, > > > Roland > > > > I tried with the following, but failed to achieve the expected effect: > > > > class Term: > > HEADER = '\033[95m' > > OKBLUE = '\033[94m' > > OKGREEN = '\033[92m' > > WARNING = '\033[93m' > > FAIL = '\033[91m' > > ENDC = '\033[0m' > > LIGHTCYAN = '\033[1;36m' > > LIGHTGRAY = '\033[0;37m' > > YELLOW = '\033[0;33m' > > BOLD = '\033[1m' > > UNDERLINE = '\033[4m' > > > > [...] > > > > percol.view.STACKLINE = percol.view.STACKLINE.replace('D', Term.UNDERLINE > > + 'D' + Term.ENDC) > > > > The result will look like this: > > > > Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s ?[4mD?[0mir:M-d > > ?[4mD?[0mircmd:M-b > > > > I cannot repeat that. Are you sure that the '?' shown in your output are > not due to your terminal settings that influence how strings printed by > Python or inside used terminal are shown? > > Python 3.9.6 (default, Jul 16 2021, 00:00:00) > [GCC 11.1.1 20210531 (Red Hat 11.1.1-3)] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> UL = '\033[4m' > >>> UL > '\x1b[4m' > >>> ENDC = '\033[0m' > >>> ENDC > '\x1b[0m' > > >>> s = UL + 'D' + ENDC > >>> s > '\x1b[4mD\x1b[0m' > > >>> s = 'ABCDE' > >>> s = s.replace('D', UL + 'D' + ENDC) > >>> s > 'ABC\x1b[4mD\x1b[0mE' > > When I call print(s) it even shows ABCD and D is underscored. If I test the code snippet above with ipython/ptpython/python, I got the same result as you described. But the problem I reported here is triggered by running the command line wrapper of the project by `Ctrl-r`, which is worked with curses library. And I also noticed the following file [1] used by the project, which may be pertinent to the problem discussed here. But till now I still can't think of a solution. [1] https://github.com/hongyi-zhao/ariadne/blob/master/percol/ansi.py > But copying > the output to mail looses the underscore ... > [image: image.png] > > BR, > Roland > > Regards, > > HY > > > > > > > > > > > > How to achieve this purpose? > > > > > > > > [1] > > > > > > https://github.com/hongyi-zhao/ariadne/blob/838179bb4275ac85f5342d9e7d086d6ade3be1de/rc.py#L55 > > > > > > > > Regards, > > > > HY > > > > -- > > > > https://mail.python.org/mailman/listinfo/python-list > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > From hongyi.zhao at gmail.com Fri Sep 10 19:48:55 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Fri, 10 Sep 2021 16:48:55 -0700 (PDT) Subject: Change the display style of the text on the STACKLINE. In-Reply-To: <2f95335c-6ebc-4917-9bb5-ed9991586a82n@googlegroups.com> References: <49f73502-68a0-444f-baa5-33e6e136055fn@googlegroups.com> <71240eba-d7ed-46e5-a98b-9a8fd2fc5169n@googlegroups.com> <2f95335c-6ebc-4917-9bb5-ed9991586a82n@googlegroups.com> Message-ID: <281ff3f5-5e0c-4052-9f04-473a9c96a872n@googlegroups.com> On Saturday, September 11, 2021 at 7:43:44 AM UTC+8, hongy... at gmail.com wrote: > On Friday, September 10, 2021 at 2:12:31 PM UTC+8, Roland Mueller wrote: > > pe 10. syysk. 2021 klo 8.53 hongy... at gmail.com (hongy... at gmail.com) > > kirjoitti: > > > On Thursday, September 9, 2021 at 8:57:37 PM UTC+8, Roland Mueller wrote: > > > > Hello > > > > > > > > to 9. syysk. 2021 klo 6.53 hongy... at gmail.com (hongy... at gmail.com) > > > > kirjoitti: > > > > > I'm using the following code in my forked project [1]: > > > > > > > > > > percol.view.STACKLINE = 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s > > > Dir:M-d > > > > > Dircmd:M-b' > > > > > > > > > > I would like to change the display style of the text mentioned above, > > > for > > > > > example, to underline some characters in it, as shown below: > > > > > > > > > > _D_ir:M-d > > > > > > > > > > > > > > You can use e.g. str.replace() or re.sub() > > > > > > > > >>> percol.view.STACKLINE = percol.view.STACKLINE.replace('D', '_D_') > > > > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d > > > _D_ircmd:M-b' > > > > > > > > >>> import re > > > > > > > > Replace D with _D_ > > > > >>> percol.view.STACKLINE = re.sub(r'([D])', > > > > r'_\1_', percol.view.STACKLINE) > > > > Result: 'Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s _D_ir:M-d > > > _D_ircmd:M-b' > > > > > > > > Replace D and M with _D_, _M_ > > > > >>> percol.view.STACKLINE = re.sub(r'([DM])', r'_\1_', > > > > percol.view.STACKLINE) > > > > 'Fold:F1,F2,F3 Push:C-p Pop:_M_-p Script:_M_-s _D_ir:_M_-d > > > _D_ircmd:_M_-b' > > > > > > > > Regards, > > > > Roland > > > > > > I tried with the following, but failed to achieve the expected effect: > > > > > > class Term: > > > HEADER = '\033[95m' > > > OKBLUE = '\033[94m' > > > OKGREEN = '\033[92m' > > > WARNING = '\033[93m' > > > FAIL = '\033[91m' > > > ENDC = '\033[0m' > > > LIGHTCYAN = '\033[1;36m' > > > LIGHTGRAY = '\033[0;37m' > > > YELLOW = '\033[0;33m' > > > BOLD = '\033[1m' > > > UNDERLINE = '\033[4m' > > > > > > [...] > > > > > > percol.view.STACKLINE = percol.view.STACKLINE.replace('D', Term.UNDERLINE > > > + 'D' + Term.ENDC) > > > > > > The result will look like this: > > > > > > Fold:F1,F2,F3 Push:C-p Pop:M-p Script:M-s ?[4mD?[0mir:M-d > > > ?[4mD?[0mircmd:M-b > > > > > > I cannot repeat that. Are you sure that the '?' shown in your output are > > not due to your terminal settings that influence how strings printed by > > Python or inside used terminal are shown? > > > > Python 3.9.6 (default, Jul 16 2021, 00:00:00) > > [GCC 11.1.1 20210531 (Red Hat 11.1.1-3)] on linux > > Type "help", "copyright", "credits" or "license" for more information. > > >>> UL = '\033[4m' > > >>> UL > > '\x1b[4m' > > >>> ENDC = '\033[0m' > > >>> ENDC > > '\x1b[0m' > > > > >>> s = UL + 'D' + ENDC > > >>> s > > '\x1b[4mD\x1b[0m' > > > > >>> s = 'ABCDE' > > >>> s = s.replace('D', UL + 'D' + ENDC) > > >>> s > > 'ABC\x1b[4mD\x1b[0mE' > > > > When I call print(s) it even shows ABCD and D is underscored. > If I test the code snippet above with ipython/ptpython/python, I got the same result as you described. But the problem I reported here is triggered by running the command line wrapper of the project by `Ctrl-r`, which is worked with curses library. And I also noticed the following file [1] used by the project, which may be pertinent to the problem discussed here. > But till now I still can't think of a solution. > > [1] https://github.com/hongyi-zhao/ariadne/blob/master/percol/ansi.py And refer to the following file used in the project: https://github.com/hongyi-zhao/ariadne/blob/master/percol/display.py Regards, HY From tjreedy at udel.edu Fri Sep 10 21:30:32 2021 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 10 Sep 2021 21:30:32 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 9/10/2021 7:38 AM, Alan Gauld via Python-list wrote: > But python complicates this tenet still further by adding an else > clause to its loops. And complicating this still more is that these > else clauses have almost exactly opposite effects. To the contrary... if...else executes the else part if the condition is false. > > while...else... > > executes the else if the body of the loop does NOT get executed. IE, executes the else part if the condition is false. A while statement is, or can be viewed as, an if statement with a goto ending the if part. > for...else... > > executes the else iff ALL iterations of the for loop DO complete. IE, executes the else part of the condition is false. A for loop is, or can be viewed as syntactic sugar for a while loop. The condition is that next(iterable) yields a value. It is possible that the doc could be improved. I have not looked for a while. Or maybe it needs to be read more. -- Terry Jan Reedy From pfeiffer at cs.nmsu.edu Fri Sep 10 22:48:22 2021 From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer) Date: Fri, 10 Sep 2021 20:48:22 -0600 Subject: Friday Finking: Contorted loops References: <1blf44f8ak.fsf@pfeifferfamily.net> Message-ID: <1bh7erg749.fsf@pfeifferfamily.net> 2QdxY4RzWzUUiLuE at potatochowder.com writes: > On 2021-09-10 at 15:08:19 -0600, > Joe Pfeiffer wrote: > >> ram at zedat.fu-berlin.de (Stefan Ram) writes: > >> > The existence of statements like "break" renders >> > proof techniques for loops (such as Hoare's) with >> > their invariants and inference rules unapplicable. >> >> Also the reason to avoid repeat-until loops: the loop "invariant" isn't >> the same on the first iteration as on subsequent iterations. > > I am by no means an expert, nor likely even a neophyte, but why would > the loop invariant not be the same on the first iteration? > > I can certainly see that the exit condition may not make sense at the > beginning of the first iteration (e.g., there is not yet any data to > compare to the sentinel), but ISTM that claiming that the exit condition > is a loop invariant isn't kosher (because all you're claiming is that > the compiler works). Precisely because you've got knowledge of the exit condition on iterations after the first, but not the first one. So, unlike a while loop, you don't have the same knowledge on every pass. > I can also see that certain state information may not be captured until > the end of the first iteration. But presumably said state information > can change from iteration to iteration, so I can't see how you'd derive > an invariant involving it. From PythonList at DancesWithMice.info Sat Sep 11 01:24:47 2021 From: PythonList at DancesWithMice.info (dn) Date: Sat, 11 Sep 2021 17:24:47 +1200 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> Message-ID: On 31/08/2021 01.50, Chris Angelico wrote: > On Mon, Aug 30, 2021 at 11:13 PM David Raymond wrote: >> >>> def how_many_times(): >>> x, y = 0, 1 >>> c = 0 >>> while x != y: >>> c = c + 1 >>> x, y = roll() >>> return c, (x, y) >> >> Since I haven't seen it used in answers yet, here's another option using our new walrus operator >> >> def how_many_times(): >> roll_count = 1 >> while (rolls := roll())[0] != rolls[1]: >> roll_count += 1 >> return (roll_count, rolls) >> > > Since we're creating solutions that use features in completely > unnecessary ways, here's a version that uses collections.Counter: > > def how_many_times(): > return next((count, rolls) for count, rolls in > enumerate(iter(roll, None)) if len(Counter(rolls)) == 1) > > Do I get bonus points for it being a one-liner that doesn't fit in > eighty characters? Herewith my claim to one-liner fame (assuming such leads in any way to virtue or fame) It retains @Peter's preference for a more re-usable roll_die() which returns a single event, cf the OP's roll() which returns two results). import itertools, random def roll_die(): while True: yield random.randrange(1, 7) def how_many_times(): return list( itertools.takewhile( lambda r:r[ 0 ] != r[ 1 ], zip( roll_die(), roll_die() ) ) ) Also, a claim for 'bonus points' because the one-liner will fit within 80-characters - if only I didn't have that pernicious and vile habit of coding a more readable layout. It doesn't use a two-arg iter, but still rates because it does use a relatively-obscure member of the itertools library... https://docs.python.org/3.8/library/itertools.html#itertools.takewhile -- Regards, =dn From rosuav at gmail.com Sat Sep 11 02:03:26 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 11 Sep 2021 16:03:26 +1000 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> Message-ID: On Sat, Sep 11, 2021 at 3:26 PM dn via Python-list wrote: > > On 31/08/2021 01.50, Chris Angelico wrote: > > On Mon, Aug 30, 2021 at 11:13 PM David Raymond wrote: > >> > >>> def how_many_times(): > >>> x, y = 0, 1 > >>> c = 0 > >>> while x != y: > >>> c = c + 1 > >>> x, y = roll() > >>> return c, (x, y) > >> > >> Since I haven't seen it used in answers yet, here's another option using our new walrus operator > >> > >> def how_many_times(): > >> roll_count = 1 > >> while (rolls := roll())[0] != rolls[1]: > >> roll_count += 1 > >> return (roll_count, rolls) > >> > > > > Since we're creating solutions that use features in completely > > unnecessary ways, here's a version that uses collections.Counter: > > > > def how_many_times(): > > return next((count, rolls) for count, rolls in > > enumerate(iter(roll, None)) if len(Counter(rolls)) == 1) > > > > Do I get bonus points for it being a one-liner that doesn't fit in > > eighty characters? > > > Herewith my claim to one-liner fame (assuming such leads in any way to > virtue or fame) > > It retains @Peter's preference for a more re-usable roll_die() which > returns a single event, cf the OP's roll() which returns two results). > > > import itertools, random > > def roll_die(): > while True: > yield random.randrange(1, 7) > > def how_many_times(): > return list( itertools.takewhile( lambda r:r[ 0 ] != r[ 1 ], > zip( roll_die(), roll_die() ) > ) > ) > > Also, a claim for 'bonus points' because the one-liner will fit within > 80-characters - if only I didn't have that pernicious and vile habit of > coding a more readable layout. > > It doesn't use a two-arg iter, but still rates because it does use a > relatively-obscure member of the itertools library... > Nice, but that's only going to give you the ones that don't match. You can then count those, and that's a start, but how do you capture the matching rolls? I smell another opportunity for gratuitous use of a language feature: nonlocal. In a lambda function. Which may require shenanigans of epic proportions. ChrisA From vinay_sajip at yahoo.co.uk Sat Sep 11 04:28:01 2021 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 11 Sep 2021 08:28:01 +0000 (UTC) Subject: ANN: Version 0.5.1 of the Python config module has been released. References: <1871841610.7293037.1631348881810.ref@mail.yahoo.com> Message-ID: <1871841610.7293037.1631348881810@mail.yahoo.com> What Does It Do? ================ The CFG configuration format is a text format for configuration files which is similar to, and a superset of, the JSON format.It has the following aims: * Allow a hierarchical configuration scheme with support for key-value mappings and ? lists. * Support cross-references between one part of the configuration and another. * Provide a string interpolation facility to easily build up configuration values from ? other configuration values. * Provide the ability to compose configurations (using include and merge facilities). * Provide the ability to access real application objects safely. * Be completely declarative. It overcomes a number of drawbacks of JSON when used as a configuration format: * JSON is more verbose than necessary. * JSON doesn?t allow comments. * JSON doesn?t provide first-class support for dates and multi-line strings. * JSON doesn?t allow trailing commas in lists and mappings. * JSON doesn?t provide easy cross-referencing, interpolation, or composition. The Python config module provides an interface to work with configuration files written in the CFG format. Comprehensive documentation is available at https://docs.red-dove.com/cfg/index.html and you can report issues / enhancement requests at https://github.com/vsajip/py-cfg-lib/issues As always, your feedback is most welcome (especially bug reports, patches and suggestions for improvement). Enjoy! Cheers, Vinay Sajip From PythonList at DancesWithMice.info Sat Sep 11 05:09:38 2021 From: PythonList at DancesWithMice.info (dn) Date: Sat, 11 Sep 2021 21:09:38 +1200 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <86r1edlqxw.fsf@jevedi.com> Message-ID: <854aa6b9-558f-b280-4d7e-e9ed1d0a9aeb@DancesWithMice.info> On 11/09/2021 18.03, Chris Angelico wrote: > On Sat, Sep 11, 2021 at 3:26 PM dn via Python-list > wrote: >> >> On 31/08/2021 01.50, Chris Angelico wrote: >>> On Mon, Aug 30, 2021 at 11:13 PM David Raymond wrote: >>>> >>>>> def how_many_times(): >>>>> x, y = 0, 1 >>>>> c = 0 >>>>> while x != y: >>>>> c = c + 1 >>>>> x, y = roll() >>>>> return c, (x, y) >>>> >>>> Since I haven't seen it used in answers yet, here's another option using our new walrus operator >>>> >>>> def how_many_times(): >>>> roll_count = 1 >>>> while (rolls := roll())[0] != rolls[1]: >>>> roll_count += 1 >>>> return (roll_count, rolls) >>>> >>> >>> Since we're creating solutions that use features in completely >>> unnecessary ways, here's a version that uses collections.Counter: >>> >>> def how_many_times(): >>> return next((count, rolls) for count, rolls in >>> enumerate(iter(roll, None)) if len(Counter(rolls)) == 1) >>> >>> Do I get bonus points for it being a one-liner that doesn't fit in >>> eighty characters? >> >> >> Herewith my claim to one-liner fame (assuming such leads in any way to >> virtue or fame) >> >> It retains @Peter's preference for a more re-usable roll_die() which >> returns a single event, cf the OP's roll() which returns two results). >> >> >> import itertools, random >> >> def roll_die(): >> while True: >> yield random.randrange(1, 7) >> >> def how_many_times(): >> return list( itertools.takewhile( lambda r:r[ 0 ] != r[ 1 ], >> zip( roll_die(), roll_die() ) >> ) >> ) >> >> Also, a claim for 'bonus points' because the one-liner will fit within >> 80-characters - if only I didn't have that pernicious and vile habit of >> coding a more readable layout. >> >> It doesn't use a two-arg iter, but still rates because it does use a >> relatively-obscure member of the itertools library... >> > > Nice, but that's only going to give you the ones that don't match. You > can then count those, and that's a start, but how do you capture the > matching rolls? > > I smell another opportunity for gratuitous use of a language feature: > nonlocal. In a lambda function. Which may require shenanigans of epic > proportions. The stated requirement is: "I'd like to get the number of times I tried". Given such: why bother with returning any of the pairs of values? Further, if you look at the OP's original solution, it only publishes the last pair, ie the match, without mention of the list of non-matches. Was it perhaps only a means of testing the solution? Regret that I'll settle for (or continue to seek) 'fame'. I don't play guitar, so have no use for epic. -- Regards, =dn From hjp-python at hjp.at Sat Sep 11 08:40:00 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 11 Sep 2021 14:40:00 +0200 Subject: on writing a while loop for rolling two dice In-Reply-To: References: <8635qkh2ga.fsf@jevedi.com> <004401d7a378$44d43580$ce7ca080$@verizon.net> Message-ID: On 2021-09-08 13:07:47 +1200, Greg Ewing wrote: > On 8/09/21 2:53 am, Grant Edwards wrote: > > #define IF if ( > > #define THEN ) { > > #define ELSE } else { > > #define ENDIF } > > I gather that early versions of some of the Unix utilities were > written by someone who liked using macros to make C resemble Algol. Steve Bourne, the author of the eponymous shell. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 11 10:41:59 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 11 Sep 2021 16:41:59 +0200 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 2021-09-10 12:26:24 +0100, Alan Gauld via Python-list wrote: > On 10/09/2021 00:47, Terry Reedy wrote: > > even one loop is guaranteed.) "do-while" or "repeat-until is even rarer > > since fractional-loop include this as a special case. > > Is there any empirical evidence to support this? > Or is it just a case of using the tools that are available? > In my experience of using Pascal (and much later with Delphi) > that I used repeat loops at least as often as while loops, > possibly more. > > But using Python and to a lesser extent C (which has a > rather horrible do/while) construct How is C's do/while loop more horrible than Pascal's repeat/until? They seem almost exactly the same to me (the differences I see are the inverted condition (debatable which is better) and the added block delimiters (which I actually like)). > So is it the case that the "need" for repeat loops is > rare, simply a result of there being no native repeat > loop available? A tiny non-representative data point: In an old collection of small C programs of mine I find: 35 regular for loops 28 while loops 2 infinite for loops 1 "infinite" for loop (i.e. it exits somewhere in the middle) 0 do/while loops. So even though do/while loops are available in C (and I don't find them horrible) I apparently found very little use for them (I'm sure if I look through more of my C programs I'll find a few examples, but this small samples shows they are rare. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 11 10:57:06 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 11 Sep 2021 16:57:06 +0200 Subject: on floating-point numbers In-Reply-To: References: <86bl5bm6go.fsf@jevedi.com> <86a6kvkobe.fsf@jevedi.com> <86eea4h31o.fsf@jevedi.com> <86o898e71d.fsf@jevedi.com> Message-ID: On 2021-09-05 22:32:51 -0000, Grant Edwards wrote: > On 2021-09-05, Peter J. Holzer wrote: [on the representability of fractional numbers as floating point numbers] > And once you understand that, ignore it and write code under the > assumumption that nothing can be exactly represented in floating > point. In almost all cases even the input values aren't exact. > If you like, you can assume that 0 can be exactly represented without > getting into too much trouble as long as it's a literal constant value > and not the result of any run-time FP operations. > > If you want to live dangerously, you can assume that integers with > magnitude less than a million can be exactly represented. That > assumption is true for all the FP representations I've ever used, If you know nothing about the FP representation you use you could do that (however, there is half-precision (16-bit) floating-point which has an even shorter mantissa). But if you are that conservative, you should be equally conservative with your integers, which probably means you can't depend on more than 16 bits (?32767). However, we are using Python here which means we have at least 9 decimal digits of useable mantissa (https://docs.python.org/3/library/stdtypes.html#numeric-types-int-float-complex somewhat unhelpfully states that "[f]loating point numbers are usually implemented using double in C", but refers to https://docs.python.org/3/library/sys.html#sys.float_info which in turn refers directly to the DBL_* constants from C99. So DBL_EPSILON is at most 1E-9. in practice almost certainly less than 1E-15). > but once you start depending on it, you're one stumble from the edge > of the cliff. I think this attitude will prevent you from using floating point numbers when you could, reinventing the wheel, probably badly. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Sep 11 11:06:09 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 11 Sep 2021 17:06:09 +0200 Subject: on floating-point numbers In-Reply-To: <479224BA-0EFA-4DC3-893B-EBF1ED4C3CD1@damon-family.org> References: <479224BA-0EFA-4DC3-893B-EBF1ED4C3CD1@damon-family.org> Message-ID: On 2021-09-05 23:21:14 -0400, Richard Damon wrote: > > On Sep 5, 2021, at 6:22 PM, Peter J. Holzer wrote: > > On 2021-09-04 10:01:23 -0400, Richard Damon wrote: > >>> On 9/4/21 9:40 AM, Hope Rouselle wrote: > >>> Hm, I think I see what you're saying. You're saying multiplication and > >>> division in IEEE 754 is perfectly safe --- so long as the numbers you > >>> start with are accurately representable in IEEE 754 and assuming no > >>> overflow or underflow would occur. (Addition and subtraction are not > >>> safe.) > >>> > >> > >> Addition and Subtraction are just as safe, as long as you stay within > >> the precision limits. > > > > That depends a lot on what you call "safe", > > > > a * b / a will always be very close to b (unless there's an over- or > > underflow), but a + b - a can be quite different from b. > > > > In general when analyzing a numerical algorithm you have to pay a lot > > more attention to addition and subtraction than to multiplication and > > division. > > > Yes, it depends on your definition of safe. If ?close? is good enough > then multiplication is probably safer as the problems are in more > extreme cases. If EXACT is the question, addition tends to be better. > To have any chance, the numbers need to be somewhat low ?precision?, > which means the need to avoid arbitrary decimals. If you have any "decimals" (i.e decimal digits to the right of your decimal point) then the input values won't be exactly representable and the nearest representation will use all available bits, thus losing some precision with most additions. > Once past that, as long as the numbers are of roughly the same > magnitude, and are the sort of numbers you are apt to just write, you > can tend to add a lot of them before you get enough bits to accumulate > to have a problem. But they won't be exact. You may not care about rounding errors in the tenth digit after the point, but you are only close, not exact. So if you are fine with a tiny rounding error here, why are you upset about equally tiny rounding errors on multiplication? > With multiplication, every multiply roughly adds the number of bits of > precision, so you quickly run out, and one divide will have a chance > to just end the process. Nope. The relative error stays the same unlike for addition where is can get very large very quickly. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From rosuav at gmail.com Sat Sep 11 11:40:12 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 Sep 2021 01:40:12 +1000 Subject: on floating-point numbers In-Reply-To: References: <479224BA-0EFA-4DC3-893B-EBF1ED4C3CD1@damon-family.org> Message-ID: On Sun, Sep 12, 2021 at 1:07 AM Peter J. Holzer wrote: > If you have any "decimals" (i.e decimal digits to the right of your > decimal point) then the input values won't be exactly representable and > the nearest representation will use all available bits, thus losing some > precision with most additions. That's an oversimplification, though - numbers like 12345.03125 can be perfectly accurately represented, since the fractional part is a (negative) power of two. The perceived inaccuracy of floating point numbers comes from an assumption that a string of decimal digits is exact, and the computer's representation of it is not. If I put this in my code: ONE_THIRD = 0.33333 then you know full well that it's not accurate, and that's nothing to do with IEEE floating-point! The confusion comes from the fact that one fifth (0.2) can be represented precisely in decimal, and not in binary. Once you accept that "perfectly representable numbers" aren't necessarily the ones you expect them to be, 64-bit floats become adequate for a huge number of tasks. Even 32-bit floats are pretty reliable for most tasks, although I suspect that there's little reason to use them now - would be curious to see if there's any performance benefit from restricting to the smaller format, given that most FPUs probably have 80-bit or wider internal registers. ChrisA From alan.gauld at yahoo.co.uk Sat Sep 11 03:58:58 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 11 Sep 2021 08:58:58 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> Message-ID: On 10/09/2021 19:49, Stefan Ram wrote: > Alan Gauld writes: >> OK, That's a useful perspective that is at least consistent. >> Unfortunately it's not how beginners perceive it > ... > > Beginners perceive it the way it is explained to them by > their teacher. I'm not sure that's true. Most beginners, in my experience, learn the syntax from their teachers and then go off and play. What they observe happening is what sticks. And python loop 'else' constructs appear inconsistent to them. As teachers we like to think we are passing on our wisdom to our students but in reality everyone learns from their own experience. The teachers advice is just the starting point. Hopefully, that starting point sends them in the right direction but that's the best we can hope for. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From a.h.jaffe at gmail.com Sat Sep 11 06:12:20 2021 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Sat, 11 Sep 2021 11:12:20 +0100 Subject: on writing a while loop for rolling two dice In-Reply-To: <854aa6b9-558f-b280-4d7e-e9ed1d0a9aeb@DancesWithMice.info> References: <86r1edlqxw.fsf@jevedi.com> <854aa6b9-558f-b280-4d7e-e9ed1d0a9aeb@DancesWithMice.info> Message-ID: On 11/09/2021 10:09, dn via Python-list wrote: > > > > The stated requirement is: "I'd like to get the number of times I > tried". Given such: why bother with returning any of the pairs of values? Indeed, if that's the requirement, then you can do even better, noting that the probability of getting a matched pair is 1/6 (6 matches out of 6*6 possibilities). So the answer to the problem is exactly the same as rolling a single die until you get any particular number (e.g., 1). This is somewhat easier to simulate than the two-dice problem (and the number of throws until a match is also a known, analytic distribution that you could sample from, but this is probably easier). From hjp-python at hjp.at Sat Sep 11 15:07:09 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 11 Sep 2021 21:07:09 +0200 Subject: on floating-point numbers In-Reply-To: References: <479224BA-0EFA-4DC3-893B-EBF1ED4C3CD1@damon-family.org> Message-ID: On 2021-09-12 01:40:12 +1000, Chris Angelico wrote: > On Sun, Sep 12, 2021 at 1:07 AM Peter J. Holzer wrote: > > If you have any "decimals" (i.e decimal digits to the right of your > > decimal point) then the input values won't be exactly representable and > > the nearest representation will use all available bits, thus losing some > > precision with most additions. > > That's an oversimplification, though - numbers like 12345.03125 can be > perfectly accurately represented, since the fractional part is a > (negative) power of two. Yes. I had explained that earlier in this thread. > The perceived inaccuracy of floating point numbers comes from an > assumption that a string of decimal digits is exact, and the > computer's representation of it is not. If I put this in my code: > > ONE_THIRD = 0.33333 > > then you know full well that it's not accurate, and that's nothing to > do with IEEE floating-point! The confusion comes from the fact that > one fifth (0.2) can be represented precisely in decimal, and not in > binary. Exactly. > Once you accept that "perfectly representable numbers" aren't > necessarily the ones you expect them to be, 64-bit floats become > adequate for a huge number of tasks. Yep. That's what I was trying to convey. > Even 32-bit floats are pretty reliable for most tasks, although I > suspect that there's little reason to use them now - would be curious > to see if there's any performance benefit from restricting to the > smaller format, given that most FPUs probably have 80-bit or wider > internal registers. AFAIK C compilers on 64-bit AMD/Intel architecture don't use the x87 ABI any more, they use the various vector extensions (SSE, etc.) instead. Those have hardware support for 64 and 32 bit FP values, so 32 bit are probably faster, if only because you can cram more of them into a register. Modern GPUs now have 16 bit FP numbers - those are perfectly adequate for neural networks and also some graphics tasks and you can transfer twice as many per memory cycle ... hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From avigross at verizon.net Sat Sep 11 16:29:36 2021 From: avigross at verizon.net (Avi Gross) Date: Sat, 11 Sep 2021 16:29:36 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> Message-ID: <041f01d7a74b$bd3d83b0$37b88b10$@verizon.net> Alan and others, I think human languages used to make computer languages will often cause confusion. Some languages have an IF .. ELSE construct but also an EITHER ... OR and a NEITHER ... NOR and other twists and turns like words that sometimes come apart and you end up having to dangle a part that was in the front of a word to later in the sentence and so on. But I suspect many languages do NOT naturally have a construct like: WHILE ... ELSE. The might have a sentence like "While it is sunny you should use sunscreen but when it rains use an umbrella." It probably is even a tad deceptive to use WHILE in one part and not in the other. Perfectly valid sentences are "When going outside if it is sunny use sunscreen but if it is rainy use an umbrella" or skip the while and use a more standard if/else. The world "while" just does not feel like a partner for "else". So say you want to have a loop starting with WHILE and FOLLOWED by a single ELSE clause. Arguably you could make WHILE as a construct return a status of sorts if it runs at all or perhaps if it exits after at least one iteration because the condition evaluates to FALSE. It would either return false if you exit with a BREAK or by an error or perhaps not exit at all if you do a return from within. So if you made up a syntax like: IF (WHILE condition {...}) ELSE {...} Then what would that mean? Again, this is a make-believe construct. In the above, if WHILE returned a True of some sort, the else is skipped. Otherwise, no matter what has been done within the while loop, it is done. But as noted we have odd choices here potentially. Could we differentiate between a BREAK statement within and something like BREAK OK variant that means the while is to be treated as succeeded and please do not do the trailing ELSE? I can see many possible ways to design things and cannot expect humans to automatically assume the specific nomenclature will be meaningful to them. There is an alternative that people who are not damn sure what the meaning is can do. Create a variable that is set to False or True to represent something before the WHILE is entered. Then make sure your code flips that value in cased you want to make sure a trailing statement is run. Then following the while, you place an IF statement that tests that variable and does what the ELSE cluse would have done, or not. Looking at other constructs, look at this code with a try: i=0 while i<5: try: assert(i!=3) #Raises an AssertionError if i==3 print("i={0}".format(i)) except: continue finally: i+= 1; #Increment i Now attach an ELSE clause to the WHILE, LOL! At some point, van some humans decide just not to write the code this way? What about code that uses CONTINUE to the point where you enter the WHILE statement and get a secondary IF or something that keeps triggering a CONTINUE to start the next iteration. Arguably, this can effectively mean the WHILE loop did nothing. An example would be evaluating the contents of a data structure like a list and adding all numeric items together and ignoring any that are character strings. Given all characters, no summation is done. The first statement in the loop tests a list item and does a CONTINUE. But by the rules as I see them, the loop was entered. Yet, a similar loop written where the WHILE condition simply tests if ANY item is numeric, might drop right through to an ELSE clause. Bottom line is humans do not all think alike and language constructs that are clear and logical to one may be confusing or mean the opposite to others. I can even imagine designing an interface like this: WHILE (condition): ... IF_NOT_RUN: ... IF_EXITED_EARLY: ... IF_ERROR_THROWN: ... ON_PREMATURE_RETURN_DO_THIS: ... I am not suggesting we need critters like that, simply that ELSE is a grab bag case that can mean many things to many people. But if the specific meaning is clearly documented, use it. Lots of people who program in languages like Python do not necessarily even speak much English and just memorize the keywords. We can come up with ever more interesting or even bizarre constructs like multiple WHILE in a row with each one being called only if the previous one failed to process the data. An example might be if each tests the data type and refuses to work on it so the next one in line is called. That could perhaps be done by having multiple ELSE statements each with another WHILE. But is that an ideal way to do this or perhaps instead use some variant of a switch statement or a dictionary pointing to functions to invoke or something. Time to go do something lese of even minor usefulness! -----Original Message----- From: Python-list On Behalf Of Alan Gauld via Python-list Sent: Saturday, September 11, 2021 3:59 AM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 10/09/2021 19:49, Stefan Ram wrote: > Alan Gauld writes: >> OK, That's a useful perspective that is at least consistent. >> Unfortunately it's not how beginners perceive it > ... > > Beginners perceive it the way it is explained to them by > their teacher. I'm not sure that's true. Most beginners, in my experience, learn the syntax from their teachers and then go off and play. What they observe happening is what sticks. And python loop 'else' constructs appear inconsistent to them. As teachers we like to think we are passing on our wisdom to our students but in reality everyone learns from their own experience. The teachers advice is just the starting point. Hopefully, that starting point sends them in the right direction but that's the best we can hope for. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list From avigross at verizon.net Sat Sep 11 21:38:02 2021 From: avigross at verizon.net (Avi Gross) Date: Sat, 11 Sep 2021 21:38:02 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: <059a01d7a776$d3daa460$7b8fed20$@verizon.net> Peter, in your own personal finite sample, I am wondering what you might do TODAY if you looked at your loops again and considered redoing them for an assortment of reasons ranging from using the code for teaching to efficiency to just fitting your mood better? I have seen seasoned authors go back to their early work and groan. Some have even reissued earlier work with a partial rewrite often with a long additional preface explaining why and even mentioned what was changed and bemoaning how they thought differently back then. My guess is that many of us (meaning myself included) often approach a problem and go with the first thing that comes to mind. If it fits well enough, we move on to the next thing we can do. If not, we may step back and evaluate multiple additional options and try another tack. I have seen not of sort-of redundant code because someone did not plan ahead and realize something very similar might be needed later and thus did not make a general function they could re-use. Occasionally they may later go back and re-do but often, not so much and just keep copying lines and making minor modifications. Same general idea. And perhaps worse, you may write a loop and later have to keep adding code to deal with new requirements and special cases and rather than pause and analyze and perhaps start again with a cleaner or more easily extendable solution, just keep grafting on things to make the darn current code work. Code that has many ways to exit a loop is often an example of this happening. So if you looked at your own code now, in the context of the rest of your code, would you change things? in python, I suspect I would seriously change an amazing number of things for older code including code being ported. It supports quite a few programming constructs and styles and has access to plenty of modules that mean you need not re-invent all the time. How many formal loops might you replace with a list comprehension or use a generator, NOW? How many problems you once solved by doing things like looping and searching for an element being present in a list when now you might use a set or dictionary? The reality is many people learn the basics of a language and write using fairly basic constructs and only later master the more advanced topics. But their mature work may then often heavily use those later and more effective methods. Functional programming often uses constructs where loops become invisible. Objects often hide loops in all kinds of methods. Sometimes recursion effectively does a loop. It is sometimes easy to write programs with no visible loops. So when counting the various kinds, are you looking for direct or indirect methods too like map/reduce or vectorized operations? -----Original Message----- From: Python-list On Behalf Of Peter J. Holzer Sent: Saturday, September 11, 2021 10:42 AM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 2021-09-10 12:26:24 +0100, Alan Gauld via Python-list wrote: > On 10/09/2021 00:47, Terry Reedy wrote: > > even one loop is guaranteed.) "do-while" or "repeat-until is even > > rarer since fractional-loop include this as a special case. > > Is there any empirical evidence to support this? > Or is it just a case of using the tools that are available? > In my experience of using Pascal (and much later with Delphi) that I > used repeat loops at least as often as while loops, possibly more. > > But using Python and to a lesser extent C (which has a rather horrible > do/while) construct How is C's do/while loop more horrible than Pascal's repeat/until? They seem almost exactly the same to me (the differences I see are the inverted condition (debatable which is better) and the added block delimiters (which I actually like)). > So is it the case that the "need" for repeat loops is rare, simply a > result of there being no native repeat loop available? A tiny non-representative data point: In an old collection of small C programs of mine I find: 35 regular for loops 28 while loops 2 infinite for loops 1 "infinite" for loop (i.e. it exits somewhere in the middle) 0 do/while loops. So even though do/while loops are available in C (and I don't find them horrible) I apparently found very little use for them (I'm sure if I look through more of my C programs I'll find a few examples, but this small samples shows they are rare. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" From hjp-python at hjp.at Sun Sep 12 05:43:58 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 12 Sep 2021 11:43:58 +0200 Subject: Friday Finking: Contorted loops In-Reply-To: <059a01d7a776$d3daa460$7b8fed20$@verizon.net> References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <059a01d7a776$d3daa460$7b8fed20$@verizon.net> Message-ID: On 2021-09-11 21:38:02 -0400, Avi Gross via Python-list wrote: > Peter, in your own personal finite sample, I am wondering what you might do > TODAY if you looked at your loops again and considered redoing them for an > assortment of reasons ranging from using the code for teaching to efficiency > to just fitting your mood better? > > I have seen seasoned authors go back to their early work and groan. Yeah, I do that. (Un)fortunately I also have other people's code to groan about so I won't despair too much about the stupidity of my younger self. > My guess is that many of us (meaning myself included) often approach a > problem and go with the first thing that comes to mind. If it fits well > enough, we move on to the next thing we can do. If not, we may step back and > evaluate multiple additional options and try another tack. > > I have seen not of sort-of redundant code because someone did not plan ahead > and realize something very similar might be needed later and thus did not > make a general function they could re-use. Occasionally they may later go > back and re-do but often, not so much and just keep copying lines and making > minor modifications. Same general idea. That certainly happens. I am a bit overly conservative and try to get away with minimal code changes even if a complete reimplementation of that unit would be clearly better. Especially if it's someone else's code and there are no unit tests. But also for my own code. (As an aside, I notice the same tendency when changing text: Altering an existing paragraph is hard, especially if someone else wrote it. Also, while I think I can express myself quite clearly in both German and English, I'm rarely satisfied when I try to translate between those languages. I always stick too close to the original). > And perhaps worse, you may write a loop and later have to keep adding code > to deal with new requirements and special cases and rather than pause and > analyze and perhaps start again with a cleaner or more easily extendable > solution, just keep grafting on things to make the darn current code work. > Code that has many ways to exit a loop is often an example of this > happening. That too. Those little C utilities I mentioned are probably a bad example because they are so small and had little reason to evolve. But I do have Perl scripts which I originally wrote 20 years ago and which are still in use and have been adapted to changing business requirements again and again in that time. Those do contain some gnarly code. > So if you looked at your own code now, in the context of the rest of your > code, would you change things? Almost certainly. Especially in C I would probably be more cautious about undefined behaviour now and for different reasons. Back in the 90's I mostly worried about portability: That code could one day run on a 36-bit ones-complement machine with 9-bit chars. These I days I worry more about overly aggressive optimizations: That pointer is accessed here so it can't be null, so it can't be null here either so that check can be optimized away. I started using Python only 7 years ago, when I had already been using Perl for almost 20 and C for over 25 years. So my older Python code probably looks a bit "perly". So they use dicts and map and filter but not list comprehensions for example. Also some of that code was partially inherited from other Python programmers who adhered to the "a real programmer can write Fortran in any language" mindset. > So when counting the various kinds, are you looking for direct or indirect > methods too like map/reduce or vectorized operations? No, because that wasn't the question I was trying to answer. The question was "do people use do/while loops frequently in languages which provide them"? I chose C (mostly because it is easier to get useful numbers with tools like grep and wc than with Perl) and therefore only the types of loops available in C. (Methodically the main flaw in my approach is that I only looked at a single language and a single person and only a tinly sample from that person. To really answer that question you would have to look at a sizable sample from Github or something like that). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From grant.b.edwards at gmail.com Sat Sep 11 13:01:15 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 11 Sep 2021 17:01:15 -0000 (UTC) Subject: on floating-point numbers References: <479224BA-0EFA-4DC3-893B-EBF1ED4C3CD1@damon-family.org> Message-ID: On 2021-09-11, Chris Angelico wrote: > Once you accept that "perfectly representable numbers" aren't > necessarily the ones you expect them to be, 64-bit floats become > adequate for a huge number of tasks. Even 32-bit floats are pretty > reliable for most tasks, although I suspect that there's little reason > to use them now - would be curious to see if there's any performance > benefit from restricting to the smaller format, given that most FPUs > probably have 80-bit or wider internal registers. Not all CPUs have FPUs. Most of my development time is spent writing code for processors without FPUs. A soft implementation of 32-bit FP on a 32-bit processors is way, way faster than for 64-bit FP. Not to mention the fact that 32-bit FP data takes up half the memory of 64-bit. There are probably not many people using Python on 32-bit CPUs w/o FP. -- Grant From alan.gauld at yahoo.co.uk Sat Sep 11 13:21:17 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 11 Sep 2021 18:21:17 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 11/09/2021 15:41, Peter J. Holzer wrote: > How is C's do/while loop more horrible than Pascal's repeat/until? Because it is very hard to spot or distinguish from a normal while loop. while condition ; Is a valid (and fairly common) loop in C so code that has do{ code } while condition; Looks, for non-trivial cases, like a lot of code followed by an empty while loop. The do is easy to miss and the while loop disguised as a repeat termination is confusing. repeat code until condition Is far clearer to comprehend since there is no ambiguity. > In an old collection of small C programs of mine I find: > > 35 regular for loops > 28 while loops > 2 infinite for loops > 1 "infinite" for loop (i.e. it exits somewhere in the middle) > 0 do/while loops. That wouldn't surprise me, I've only used do/while in C a handful of times. But in Pascal I use it regularly. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From pfeiffer at cs.nmsu.edu Sat Sep 11 17:58:29 2021 From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer) Date: Sat, 11 Sep 2021 15:58:29 -0600 Subject: Friday Finking: Contorted loops References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> Message-ID: <1bczpeg4fu.fsf@pfeifferfamily.net> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Alan Gauld writes: >>OK, That's a useful perspective that is at least consistent. >>Unfortunately it's not how beginners perceive it > ... > > Beginners perceive it the way it is explained to them by > their teacher. My life as a professor would have been *so* much easier if that were true... From nospam at please.ty Sun Sep 12 04:11:15 2021 From: nospam at please.ty (jak) Date: Sun, 12 Sep 2021 10:11:15 +0200 Subject: Friday Finking: Contorted loops References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> <041f01d7a74b$bd3d83b0$37b88b10$@verizon.net> Message-ID: -- snip -- > > An inconsistency that I have been able to notice is this: > someone suggests to remedy the absence of the do-while with: > while True: > ???? ... > ???? if condition: > ???????? break > the problem arises if the while has an else of its own because the break > not only blocks the while loop but will also ignore the relative else. > I will try to make my doubt clearer: if the only way to terminate a 'while True' loop is by using the 'break' statement, why is it allowed to add the 'else' statement which will only contain dead code? while True: break else: print('dead code') From alister.ware at ntlworld.com Sun Sep 12 10:05:05 2021 From: alister.ware at ntlworld.com (alister) Date: Sun, 12 Sep 2021 14:05:05 -0000 (UTC) Subject: Friday Finking: Contorted loops References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> <041f01d7a74b$bd3d83b0$37b88b10$@verizon.net> Message-ID: On Sun, 12 Sep 2021 10:11:15 +0200, jak wrote: > -- snip -- >> >> An inconsistency that I have been able to notice is this: >> someone suggests to remedy the absence of the do-while with: >> while True: >> ???? ... >> ???? if condition: >> ???????? break >> the problem arises if the while has an else of its own because the >> break not only blocks the while loop but will also ignore the relative >> else. >> >> > I will try to make my doubt clearer: > if the only way to terminate a 'while True' loop is by using the 'break' > statement, why is it allowed to add the 'else' statement which will only > contain dead code? > > while True: > break > else: > print('dead code') Because adjusting the parser for one specific special case is not worth the effort. it is not the job of the interpreter to sanitise stupid programming "Special Cases aren't special enough the break the rules" -- If you stew apples like cranberries, they taste more like prunes than rhubarb does. -- Groucho Marx From nospam at please.ty Sun Sep 12 02:56:56 2021 From: nospam at please.ty (jak) Date: Sun, 12 Sep 2021 08:56:56 +0200 Subject: Friday Finking: Contorted loops References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> <041f01d7a74b$bd3d83b0$37b88b10$@verizon.net> Message-ID: Il 11/09/2021 22:29, Avi Gross ha scritto: > > Alan and others, > > I think human languages used to make computer languages will often cause > confusion. > > Some languages have an IF .. ELSE construct but also an EITHER ... OR and a > NEITHER ... NOR and other twists and turns like words that sometimes come > apart and you end up having to dangle a part that was in the front of a word > to later in the sentence and so on. > > But I suspect many languages do NOT naturally have a construct like: > > WHILE ... ELSE. > > The might have a sentence like "While it is sunny you should use sunscreen > but when it rains use an umbrella." It probably is even a tad deceptive to > use WHILE in one part and not in the other. Perfectly valid sentences are > "When going outside if it is sunny use sunscreen but if it is rainy use an > umbrella" or skip the while and use a more standard if/else. The world > "while" just does not feel like a partner for "else". > > So say you want to have a loop starting with WHILE and FOLLOWED by a single > ELSE clause. Arguably you could make WHILE as a construct return a status of > sorts if it runs at all or perhaps if it exits after at least one iteration > because the condition evaluates to FALSE. It would either return false if > you exit with a BREAK or by an error or perhaps not exit at all if you do a > return from within. > > So if you made up a syntax like: > > IF (WHILE condition {...}) > ELSE {...} > > Then what would that mean? Again, this is a make-believe construct. In the > above, if WHILE returned a True of some sort, the else is skipped. > Otherwise, no matter what has been done within the while loop, it is done. > > But as noted we have odd choices here potentially. Could we differentiate > between a BREAK statement within and something like BREAK OK variant that > means the while is to be treated as succeeded and please do not do the > trailing ELSE? I can see many possible ways to design things and cannot > expect humans to automatically assume the specific nomenclature will be > meaningful to them. > > There is an alternative that people who are not damn sure what the meaning > is can do. Create a variable that is set to False or True to represent > something before the WHILE is entered. Then make sure your code flips that > value in cased you want to make sure a trailing statement is run. Then > following the while, you place an IF statement that tests that variable and > does what the ELSE cluse would have done, or not. > > Looking at other constructs, look at this code with a try: > > i=0 > while i<5: > try: > assert(i!=3) #Raises an AssertionError if i==3 > print("i={0}".format(i)) > except: > continue > finally: > i+= 1; #Increment i > > Now attach an ELSE clause to the WHILE, LOL! > > At some point, van some humans decide just not to write the code this way? > > What about code that uses CONTINUE to the point where you enter the WHILE > statement and get a secondary IF or something that keeps triggering a > CONTINUE to start the next iteration. Arguably, this can effectively mean > the WHILE loop did nothing. An example would be evaluating the contents of a > data structure like a list and adding all numeric items together and > ignoring any that are character strings. Given all characters, no summation > is done. The first statement in the loop tests a list item and does a > CONTINUE. But by the rules as I see them, the loop was entered. Yet, a > similar loop written where the WHILE condition simply tests if ANY item is > numeric, might drop right through to an ELSE clause. > > Bottom line is humans do not all think alike and language constructs that > are clear and logical to one may be confusing or mean the opposite to > others. > > I can even imagine designing an interface like this: > > WHILE (condition): > ... > IF_NOT_RUN: > ... > IF_EXITED_EARLY: > ... > IF_ERROR_THROWN: > ... > ON_PREMATURE_RETURN_DO_THIS: > ... > > I am not suggesting we need critters like that, simply that ELSE is a grab > bag case that can mean many things to many people. > > But if the specific meaning is clearly documented, use it. Lots of people > who program in languages like Python do not necessarily even speak much > English and just memorize the keywords. > > We can come up with ever more interesting or even bizarre constructs like > multiple WHILE in a row with each one being called only if the previous one > failed to process the data. An example might be if each tests the data type > and refuses to work on it so the next one in line is called. That could > perhaps be done by having multiple ELSE statements each with another WHILE. > But is that an ideal way to do this or perhaps instead use some variant of a > switch statement or a dictionary pointing to functions to invoke or > something. > > Time to go do something lese of even minor usefulness! > > -----Original Message----- > From: Python-list On > Behalf Of Alan Gauld via Python-list > Sent: Saturday, September 11, 2021 3:59 AM > To: python-list at python.org > Subject: Re: Friday Finking: Contorted loops > > On 10/09/2021 19:49, Stefan Ram wrote: >> Alan Gauld writes: >>> OK, That's a useful perspective that is at least consistent. >>> Unfortunately it's not how beginners perceive it >> ... >> >> Beginners perceive it the way it is explained to them by >> their teacher. > > I'm not sure that's true. Most beginners, in my experience, learn the > syntax from their teachers and then go off and play. > What they observe happening is what sticks. And python loop 'else' > constructs appear inconsistent to them. > > As teachers we like to think we are passing on our wisdom to our students > but in reality everyone learns from their own experience. The teachers > advice is just the starting point. Hopefully, that starting point sends them > in the right direction but that's the best we can hope for. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > An inconsistency that I have been able to notice is this: someone suggests to remedy the absence of the do-while with: while True: ... if condition: break the problem arises if the while has an else of its own because the break not only blocks the while loop but will also ignore the relative else. From arj.python at gmail.com Sun Sep 12 13:11:07 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 12 Sep 2021 21:11:07 +0400 Subject: ANN: Version 0.5.1 of the Python config module has been released. In-Reply-To: <1871841610.7293037.1631348881810@mail.yahoo.com> References: <1871841610.7293037.1631348881810.ref@mail.yahoo.com> <1871841610.7293037.1631348881810@mail.yahoo.com> Message-ID: Used by 4.8k but only ... 4 stars From 2QdxY4RzWzUUiLuE at potatochowder.com Sun Sep 12 13:28:22 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Sun, 12 Sep 2021 10:28:22 -0700 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 2021-09-11 at 18:21:17 +0100, Alan Gauld via Python-list wrote: > On 11/09/2021 15:41, Peter J. Holzer wrote: > > > How is C's do/while loop more horrible than Pascal's repeat/until? > > Because it is very hard to spot or distinguish from a normal > while loop. > > while condition ; > > Is a valid (and fairly common) loop in C > > so code that has > > do{ > code > } > while condition; > > Looks, for non-trivial cases, like a lot of code followed > by an empty while loop. > > The do is easy to miss and the while loop disguised as > a repeat termination is confusing. Well, yeah, except that only a macro would ever write it that way. :-) At the very least, the code would be indented (making it easier to spot the "do," before or after you see the while; and my apologies if you intended it that way and it got lost somewhere between your intent and my monitor): do { code; } while(condition); (Side question: why put the "{" next to the "do," but the "}" and the "while" on separate lines?) And I would put the while on the same line as the closing brace (which is also where I put the "else" in an if statement): do { code; } while(condition); From avigross at verizon.net Sun Sep 12 15:08:04 2021 From: avigross at verizon.net (Avi Gross) Date: Sun, 12 Sep 2021 15:08:04 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <059a01d7a776$d3daa460$7b8fed20$@verizon.net> Message-ID: <009101d7a809$83f48090$8bdd81b0$@verizon.net> The topic of looping and our current discussion stimulates me to ask if someone has categorized the uses of loops and it seems something obvious. Once you know what kinds of loopy looping there are, it can get easier to decide which, if any, of the methods to set up a loop make more sense. Computer languages differ in all kinds of ways including more subtle ideas of when a variable is within view or not and that can impact looping methods. Years ago, in many languages, loops were used simply to COPY something stored in some array-like format. Languages like C might have a character string stored in a null-terminated array and the accepted way to copy it might be to use pointers called p (the source) and q(the destination) in weird and compact code like: while (*q++ = *p++); Now Python does not have some of those operators but copying strings is fairly trivial with no visible loops but copying some objects (I mean a new and even deep copy) can use loops, albeit for many objects, there is a copy functionality available to do it quietly. If you only want a subset of the values to be taken into a copy, many languages have a subletting method like var[1:5] and often you have functions like map/reduce you can pass things to that return an arbitrary subset based on applying a function. Python has thus quite a few ways to hide any loops involved in copying so you will often not see a WHILE or FOR in sight. For simple (and even sometimes more complex) scenarios, comprehensions of many types can be used in in-line situations. Generators can supply what can amount to a sort of deferred and thus distributed loop that is run interspersed with other things happening. One reason many applications bring in add-on modules like numpy and pandas is because they partially fill some gaps and make many vectorized operations easy. Any operation that works simultaneously on every member in a way that has no conflict with calculations on other members is an example. If I have two vector like objects such as A and B, then being able to write 3*A means modifying all the elements of A to be triple what they were and it does not necessarily need to happen in any order or even be all done by the same process. If A and B are the same length (or logically adjusted to be) then A+B is similarly a vector operation. Quite a few such things are done routinely with no loops visible in the code that once littered my C code. Languages like R (and the original S) were built from the ground up so everything starts with a vector and even a singleton variable is merely a vector of length 1. Many programs simply do not need loops as it is done for you. I have mentioned ways to use objects in Python as a way to hide loops. Of course that implies an object may use methods that contain small loops, such as to search and internal list to see if something is already being held, or to find the end and append another and so on. But you main program may thus often be written without such looks. Functional programming techniques, again, can be used to say apply a function to every item in a list and collect the results in another list or scalar. You will not see a WHILE or a FOR but a loop happens. I have sometimes written a program and then after it was working, took another look and redesigned it in ways that often shorten it substantially and even speed it up. I admit people reading the code often have no clue what it does. So especially for teaching Computer Science, many loops remain a good place to start. My guess is that if, like me, you often avoid using loops in trivial cases, you may end up using them in cases that are more complex. You may end up with mainly cases where you end up having to BREAK or CONTINUE or RETURN from within a loop, perhaps with multiple exit points. Some of those cases may well be much more difficult using the hidden loops. So I was not in any way being negative with Peter about his admittedly restricted sample of coding practices, or of others who searched a body of code by many authors. I am saying that people who write code can evolve and not go back and change older code. I once inherited code that had nested loops about 9 units deep. Something like for in in ... for j in ... for k in ... It was a tad more complex, of course as it tested all variations of categorical variables. There were additional loops in selected places within there, as well. It ran slowly in interpreted form. What I ended up doing was generating a data structure (a data.frame) that contained all combinations and handed that to a function called pmap that did one row at a time by setting variables. The problem became way more tractable and quite a bit faster. And, in this case, I suspect it may have been more readable without getting lost in all the nesting. But not all languages and problems are amenable to some approaches, and some play games with how variables are kept and used. Python has some nice features that allow a single loop to replace more complex arrangements by say allowing multiple variables to be instantiated each time around so something like my deeply nested version can be run in a straightforward way. I suspect in many cases, a little though of what to feed a loop might be a great way to simplify the innards of the loop and minimize some of the concerns about multiple exit methods and so on. But you can add bells and whistles like the ELSE clause but not get many to actually use it as there are more standard ways to do that without confusion, especially if it is confusing to some. -----Original Message----- From: Python-list On Behalf Of Peter J. Holzer Sent: Sunday, September 12, 2021 5:44 AM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 2021-09-11 21:38:02 -0400, Avi Gross via Python-list wrote: > Peter, in your own personal finite sample, I am wondering what you > might do TODAY if you looked at your loops again and considered > redoing them for an assortment of reasons ranging from using the code > for teaching to efficiency to just fitting your mood better? > > I have seen seasoned authors go back to their early work and groan. Yeah, I do that. (Un)fortunately I also have other people's code to groan about so I won't despair too much about the stupidity of my younger self. > My guess is that many of us (meaning myself included) often approach a > problem and go with the first thing that comes to mind. If it fits > well enough, we move on to the next thing we can do. If not, we may > step back and evaluate multiple additional options and try another tack. > > I have seen not of sort-of redundant code because someone did not plan > ahead and realize something very similar might be needed later and > thus did not make a general function they could re-use. Occasionally > they may later go back and re-do but often, not so much and just keep > copying lines and making minor modifications. Same general idea. That certainly happens. I am a bit overly conservative and try to get away with minimal code changes even if a complete reimplementation of that unit would be clearly better. Especially if it's someone else's code and there are no unit tests. But also for my own code. (As an aside, I notice the same tendency when changing text: Altering an existing paragraph is hard, especially if someone else wrote it. Also, while I think I can express myself quite clearly in both German and English, I'm rarely satisfied when I try to translate between those languages. I always stick too close to the original). > And perhaps worse, you may write a loop and later have to keep adding > code to deal with new requirements and special cases and rather than > pause and analyze and perhaps start again with a cleaner or more > easily extendable solution, just keep grafting on things to make the darn current code work. > Code that has many ways to exit a loop is often an example of this > happening. That too. Those little C utilities I mentioned are probably a bad example because they are so small and had little reason to evolve. But I do have Perl scripts which I originally wrote 20 years ago and which are still in use and have been adapted to changing business requirements again and again in that time. Those do contain some gnarly code. > So if you looked at your own code now, in the context of the rest of > your code, would you change things? Almost certainly. Especially in C I would probably be more cautious about undefined behaviour now and for different reasons. Back in the 90's I mostly worried about portability: That code could one day run on a 36-bit ones-complement machine with 9-bit chars. These I days I worry more about overly aggressive optimizations: That pointer is accessed here so it can't be null, so it can't be null here either so that check can be optimized away. I started using Python only 7 years ago, when I had already been using Perl for almost 20 and C for over 25 years. So my older Python code probably looks a bit "perly". So they use dicts and map and filter but not list comprehensions for example. Also some of that code was partially inherited from other Python programmers who adhered to the "a real programmer can write Fortran in any language" mindset. > So when counting the various kinds, are you looking for direct or > indirect methods too like map/reduce or vectorized operations? No, because that wasn't the question I was trying to answer. The question was "do people use do/while loops frequently in languages which provide them"? I chose C (mostly because it is easier to get useful numbers with tools like grep and wc than with Perl) and therefore only the types of loops available in C. (Methodically the main flaw in my approach is that I only looked at a single language and a single person and only a tinly sample from that person. To really answer that question you would have to look at a sizable sample from Github or something like that). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" From avigross at verizon.net Sun Sep 12 15:31:48 2021 From: avigross at verizon.net (Avi Gross) Date: Sun, 12 Sep 2021 15:31:48 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <059a01d7a776$d3daa460$7b8fed20$@verizon.net> Message-ID: <00b501d7a80c$d43de8e0$7cb9baa0$@verizon.net> Stefan, Agreed that writing code to handle all possible eventualities is usually overkill and results in bloated software delivered very late or not at all. My point is that often OTHERS start adding requests afterward that seem trivial to THEM as they have no idea what it takes. I have often done some kind of data analysis for someone and made a few graphs and suddenly they ask if I can add something else to the graph such as a few horizontal lines showing where a danger zone lies, or some kind of average. No problem but to do that means the new info has to have been made available or can be calculated and often even means my function needs to take more arguments or a wider data.frame. Then they reconsider and ask if instead of a line, can I color the background above that point. Well, yeah, but now I might need to calculate another column to use to guide that feature. Ah, but can you show a series of related such graphs as a unit, or perhaps combine several unrelated graphs in a 3 by 2 matrix. Argh! Sure, I can do that but you did not ask me to before I started. I now might toss out much of my original code and rewrite something so all the things needed are made first and then the graphs are made and recombined in the right output format. This means that what used to make a graph will now make a data structure to return that can be used later to recombine into a bigger consolidated graphic. Does the story end here? Nope. Tons more requests like removing color and using shades of gray or dotted lines so it can be printed on any printer, changing the point size of text and other characteristics and introduce mathematical symbols and equations along the axes and I swear an amazing number of such fine tunings including taking a series of these things into one multi-page PDF. If this was a paid gig and someone offered me a fixed sum, should I tolerate almost any changes? If this was a regular paid job and this made me late and not get other things done? My bottom line is that it may not be reasonable to make a detailed top-down design and stock with it BUT that code written by ever-changing requirements can end up badly too. I shudder at times I wrote decent code full of comments explaining well and a while later had a mess where the comments lagged behind changes in the code as there was no point in bothering to update them unless it stopped changing. And, often, by then, I was no longer interested in spending any more time and sometimes just removed all the comments and moved on! Good luck to anyone coming along to maintain or improve the code. I have to think about when to make a function. Something trivial is often not worth is. And making a very abstract function that can do a dozen things if invoked just right with many arguments is sometimes a tad too much when a few simpler functions might do as well with less overhead and especially when the uses have fairly little in common. Some languages may discourage you if the repeated code needs to do things in the current environment and thus only part of the functionality can be moved away. -----Original Message----- From: Python-list On Behalf Of Stefan Ram Sent: Saturday, September 11, 2021 10:56 PM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops "Avi Gross" writes: >I have seen not of sort-of redundant code because someone did not plan >ahead From my experience, the "plan ahead" approach (waterfall model) often is less applicable than the "code is design" (Reeve) + "refactoring" (Fowler) approach. (However, in some fields, planning ahead is a requirement). >and realize something very similar might be needed later and thus did >not make a general function they could re-use. Occasionally they may >later go back and re-do but often, not so much and just keep copying >lines and making minor modifications. Same general idea. I remember having read a discussion in the Web. The question was something like: How many times do you have to write a piece of code, before you create a function for it? I believe I still remember two answers: - One time. - Three times. The justification I can't remember, but what I would come up with now would be: (for "one time":) Functions structure your code. You don't have to wait for repetitions as an "excuse" to create them. (for "three times:) Relax. Don't overengineer. You need to have at least /three/ repetitions to be able to see a clear pattern. -- https://mail.python.org/mailman/listinfo/python-list From vinay_sajip at yahoo.co.uk Sun Sep 12 16:18:48 2021 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 12 Sep 2021 20:18:48 +0000 (UTC) Subject: ANN: Version 0.5.1 of the Python config module has been released. In-Reply-To: References: <1871841610.7293037.1631348881810.ref@mail.yahoo.com> <1871841610.7293037.1631348881810@mail.yahoo.com> Message-ID: <1031600355.7762988.1631477928991@mail.yahoo.com> Many of those 4.8K "users" might be using indirectly via some other dependency - I'm not sure how GitHub calculates "used by", but even if it were a direct dependency, one has no idea if it's actually being used or not. so I tend not to worry about such things. My distlib library has only 2 stars, and is used by pip and therefore by many developers every day, and yet it doesn't merit a "used by" according to GitHub. I've had a few users contact me about the config library and they're happy with it and they find it useful, and that's good enough for me :-) On Sunday, 12 September 2021, 18:11:21 BST, Abdur-Rahmaan Janhangeer wrote: Used by 4.8k but only ... 4 stars From hjp-python at hjp.at Sun Sep 12 16:49:29 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 12 Sep 2021 22:49:29 +0200 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: On 2021-09-12 10:28:22 -0700, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-11 at 18:21:17 +0100, > Alan Gauld via Python-list wrote: > > On 11/09/2021 15:41, Peter J. Holzer wrote: > > > How is C's do/while loop more horrible than Pascal's repeat/until? [...] > > so code that has > > > > do{ > > code > > } > > while condition; > > > > Looks, for non-trivial cases, like a lot of code followed > > by an empty while loop. > > > > The do is easy to miss and the while loop disguised as > > a repeat termination is confusing. [...] > (Side question: why put the "{" next to the "do," but the "}" and the > "while" on separate lines?) > > And I would put the while on the same line as the closing brace (which > is also where I put the "else" in an if statement): > > do { > code; > } while(condition); Me too. I also checked two C books from "?e olde times" (K&R, 1st editiion, German translation; and "A Book on C" by Kelley/Pohl) and both nestle the while on the same line as the closing brace. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From avigross at verizon.net Sun Sep 12 17:11:58 2021 From: avigross at verizon.net (Avi Gross) Date: Sun, 12 Sep 2021 17:11:58 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> Message-ID: <022501d7a81a$d308bbe0$791a33a0$@verizon.net> Some of what I read makes me chuckle. Yes, large units of code, and even smaller ones, may be a chore to figure out. Arguably harder when you use indentation and the next/last parts are not even on the same screen as the rest. Sometimes you want to use a split-screen in some editor to line up the two parts or some other technique. But my suggestion is to COMMENT things well and I mean too much! do { } while Why not add a comment at the top like: # The following loop has a WHILE clause controlling it at the end. do { # until the while clause below } while # End of the do loop. My code tends to have brief comments especially when I have nested constructs such as multiple nested loops or in sequence, or if statements inside others. The comment often looks like ... # END of inner if ... # END of outer if The point is that places where the way of writing a program may not be as obvious as you want, may be the places you comment to make up for that. Do people read the comments? Are they extra verbiage or in the way? Who knows. And, of course, as I noted earlier, it is one more thing that gets in the way. There are languages which allow you to add some kind of labels in the code and you can label a loop with something like "doo_wop:" and inside a nested loop, you can break or continue to the named label and thus jump out multiple levels if needed. The point is not to have that feature, but perhaps have an option like: do label { code } label while ... Something that uniquely allows you to associate the end of the loop right next to the end. Since the label can be anything allowed, it could by something that suggest it is a while loop. I appreciate programming environments that let you do complex, often nested, things. But with great power can come great responsibility to use it well and make sure others can figure it out. -----Original Message----- From: Python-list On Behalf Of Peter J. Holzer Sent: Sunday, September 12, 2021 4:49 PM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 2021-09-12 10:28:22 -0700, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-11 at 18:21:17 +0100, > Alan Gauld via Python-list wrote: > > On 11/09/2021 15:41, Peter J. Holzer wrote: > > > How is C's do/while loop more horrible than Pascal's repeat/until? [...] > > so code that has > > > > do{ > > code > > } > > while condition; > > > > Looks, for non-trivial cases, like a lot of code followed by an > > empty while loop. > > > > The do is easy to miss and the while loop disguised as a repeat > > termination is confusing. [...] > (Side question: why put the "{" next to the "do," but the "}" and the > "while" on separate lines?) > > And I would put the while on the same line as the closing brace (which > is also where I put the "else" in an if statement): > > do { > code; > } while(condition); Me too. I also checked two C books from "?e olde times" (K&R, 1st editiion, German translation; and "A Book on C" by Kelley/Pohl) and both nestle the while on the same line as the closing brace. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" From alan.gauld at yahoo.co.uk Sun Sep 12 13:25:05 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 12 Sep 2021 18:25:05 +0100 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <05c4abc0-70b6-b2ea-7cfa-1121d75d2126@mrabarnett.plus.com> <041f01d7a74b$bd3d83b0$37b88b10$@verizon.net> Message-ID: On 12/09/2021 09:11, jak wrote: > if the only way to terminate a 'while True' loop is by using the 'break' > statement, why is it allowed to add the 'else' statement which will only > contain dead code? > > while True: > break > else: > print('dead code') > Because to the interpreter the condition is not part of the language. It is syntactically correct. An optimiser OTOH might welkl determine that the condition will never fail and therefore the else clause never be reached, in which case it would remove the dead code (possibly emitting a warning in the process?). A linter likewise might identify the redundant code. I don't use any python linters, does anyone know if they do detect such dead spots? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From srittau at rittau.biz Mon Sep 13 04:35:33 2021 From: srittau at rittau.biz (Sebastian Rittau) Date: Mon, 13 Sep 2021 10:35:33 +0200 Subject: Discussion forum for typing Q&A and review requests Message-ID: <23d801fa-9c46-07cc-31c9-be7b60e848f0@rittau.biz> Typing with Python is hard and constantly evolving. This is why we set up a forum to help users with typing at https://github.com/python/typing/discussions . It's fairly new, but at the moment we have two categories for general Q&A and for asking for reviews. If you have questions about typing, please join us there or on our Gitter channel (https://gitter.im/python/typing ). If you maintain a typing-related package or typing-related documentation, we would be happy if you could add a link to the discussions forum to your documentation. Contributors to the forum and suggestions on how to improve it are most welcome. The forum is intended as a help forum. Discussions about new features, improvements, and implementation should continue to go to the typing-sig list or to the appropriate issue trackers. - Sebastian From janburse at fastmail.fm Mon Sep 13 08:46:36 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Mon, 13 Sep 2021 14:46:36 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: The Standard Python version of Dogelog runtime is annoyingly slow. So we gave it a try with andother Python, and it was 6x times faster. We could test GraalVM. We worked around the missing match in Python 3.8 by replacing it with if-then-else. Performance is a little better, we find: /* Standard Python Version, Warm Run */ ?- time(fibo(23,X)). % Wall 3865 ms, gc 94 ms, 71991 lips X = 46368. /* GraalVM Python Version, Warm Warm Run */ ?- time(fibo(23,X)). % Wall 695 ms, gc 14 ms, 400356 lips X = 46368. See also: JDK 1.8 GraalVM Python is 6x faster than Standard Python https://twitter.com/dogelogch/status/1437395917167112193 JDK 1.8 GraalVM Python is 6x faster than Standard Python https://www.facebook.com/groups/dogelog Mostowski Collapse schrieb: > Yesterday we went into a little programming binge, despite there > was a free parade in Zurich. We could now already implement a transpiler > that targets Python. We simply took the transpiler main.p that targets > > JavaScript and moded it into a new transpiler mainpy.p that targets > Python. The code is already on GitHub and we present it also here > as the Python code mainpy.p. We were also busy > > on machine.py and special.py. The progress is now: > > +------------+?? cross????? +-------------+ > | loader.p?? |?? compile??? | loader.py?? | 100% > | compiler.p | -----------> | compiler.py | 100% > +------------+????????????? +-------------+ > ??????????????????????????? | machine.py? |? 66% > ??????????????????????????? | special.py? |? 33% > ??????????????????????????? +-------------+ > > See also: > > Python Version of Dogelog Runtime special > https://twitter.com/dogelogch/status/1426884473988292617 > > Python Version of Dogelog Runtime special > https://www.facebook.com/groups/dogelog From hjp-python at hjp.at Mon Sep 13 15:27:57 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 13 Sep 2021 21:27:57 +0200 Subject: Friday Finking: Contorted loops In-Reply-To: <022501d7a81a$d308bbe0$791a33a0$@verizon.net> References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <022501d7a81a$d308bbe0$791a33a0$@verizon.net> Message-ID: On 2021-09-12 17:11:58 -0400, Avi Gross via Python-list wrote: > Yes, large units of code, and even smaller ones, may be a chore to figure > out. Arguably harder when you use indentation and the next/last parts are > not even on the same screen as the rest. Sometimes you want to use a > split-screen in some editor to line up the two parts or some other > technique. > > But my suggestion is to COMMENT things well and I mean too much! > > do { > > } while > > Why not add a comment at the top like: > > # The following loop has a WHILE clause controlling it at the end. > > do { # until the while clause below > > } while # End of the do loop. Because those comments don't tell me anything that I as a C programmer don't already know. Even though do/while loops are relatively rare, I've seen hundreds of them. Seeing "do {" and recognizing it as the top of a do/while loop takes absolutely no conscious thought - reading a comment does. > My code tends to have brief comments especially when I have nested > constructs such as multiple nested loops or in sequence, or if statements > inside others. The comment often looks like > > ... # END of inner if > > ... # END of outer if Those are a bit better, but they still don't help much. What is the "inner if", what is the "outer if"? Chances are that I have to find the corresponding if to find out - I can do that without a comment, too. There are end-of-construct comments which I do find useful (at least occasionally) but those are semantic. Imagine a somewhat longish function which read several files which are comprised of records which are comprised of field. for file in files: some setup for each file here for record in file: some setup for each record here for field in record: process field do some postprocessing for the record do some postprocessing for the file If each of these blocks is longer than a few lines you might lose track where you are, so some comments might help: for file in files: some setup for each file here for record in file: some setup for each record here for field in record: process field # end of field loop do some postprocessing for the record # end of record loop do some postprocessing for the file # end of file loop Note that the comments say which part of the input the loop is processing, not just that it is the end of a loop or "the outer loop", "the intermediate loop" and "the inner loop" or some other purely syntactic information. (In most cases I would just break up such a function into smaller functions instead of adding comments, though) hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From tjreedy at udel.edu Mon Sep 13 17:07:22 2021 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 13 Sep 2021 17:07:22 -0400 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On 9/13/2021 8:46 AM, Mostowski Collapse wrote: > The Standard Python version of Dogelog runtime > is annoyingly slow. So we gave it a try with > andother Python, and it was 6x times faster. > > We could test GraalVM. We worked around the missing > match in Python 3.8 by replacing it with if-then-else. > Performance is a little better, we find: > > /* Standard Python Version, Warm Run */ > ?- time(fibo(23,X)). > % Wall 3865 ms, gc 94 ms, 71991 lips > X = 46368. > > /* GraalVM Python Version, Warm Warm Run */ > ?- time(fibo(23,X)). > % Wall 695 ms, gc 14 ms, 400356 lips > X = 46368. > > See also: > > JDK 1.8 GraalVM Python is 6x faster than Standard Python > https://twitter.com/dogelogch/status/1437395917167112193 > > JDK 1.8 GraalVM Python is 6x faster than Standard Python > https://www.facebook.com/groups/dogelog You need to test more than fibonacci to make that claim. There is a benchmark test that times around 40 different similarly small benchmarks. -- Terry Jan Reedy From avigross at verizon.net Mon Sep 13 17:50:48 2021 From: avigross at verizon.net (Avi Gross) Date: Mon, 13 Sep 2021 17:50:48 -0400 Subject: Friday Finking: Contorted loops In-Reply-To: References: <097bf31d-44e8-da35-df41-3834524716c9@DancesWithMice.info> <022501d7a81a$d308bbe0$791a33a0$@verizon.net> Message-ID: <03a101d7a8e9$69e80a60$3db81f20$@verizon.net> Peter, Choosong when and how to comment is an invidual thing and certainly the interpreter does not care. Yes, I gave an example where my comment used inner/outer as the words but more often I am more explicit along the lines you mention. Sometimes I include most of the text into the comment such as if x <10 { ... MANY LINES } # END if x <10 But please understand the goal is many fold. One simple goal is that by having to comment the code, you sort of have to explain it to yourself and use the opportunity to see if it makes sense and matches any requirements or perhaps even adds new functionality not already documented. Secondarily, it is for you to read much later when returning to the code to refresh your memory. In many organizations, including where I worked, there can be levels of code review that work better if others have a clue what they are reviewing or why. And of course, others often take over your code or examine it. At one point I was doing field support on a project, way back when it meant something to carry a beeper. I had written some of the code but the vast majority was done by others on the team. When I got beeped at 3 AM from a place like Japan, I often had to make rapid decisions on how big a problem was and whether I should wake someone up, including developers and their management or even rarely upper management. I often dove into the code to find the likely locus of a problem and even to see if I could spot the place that might be patched. That often let me wake up the right person and work with them to expedite an emergency change. Even in less chaotic or major times, I had to work with getting a problem acknowledged and scheduled for inclusion in a future release. Being able to read all kinds of source code and figuring it out quickly was important and sometimes comments were helpful. In particular, some people did not need to be woken up if I was able to see the problem was not in their code! Comments do often have a rather weird purpose. Sometimes we collected metrics on how many lines of code were new or changed between releases and unless the tools removed comments, ... But I have found over the years that too many comments are badly done. For example, rather than ending every line of a function definition with all kinds of things that clutter things, it is often better to place a few block comments above a function definition explaining the general purpose of the function, what inputs are expected on the command line, what assumptions are made about them, whether it uses any other resources (including dreaded global variables) and what it returns or what errors it may generate and so on. You can also discuss the overall idea of how the function does the job. Then, within the function, you can have smaller mainly one-liners like: # This section checks if the variables are all within expected bounds. ... # Combine ... ... # quit if not satisfied ... # Make the changes requested. ... # return successfully with a structure holding ... ... Again individual taste. My personal commenting style has evolved and varies from language to language. In many cases, I have used well-chosen variable names that are meaningful to me but not keywords. Sadly for some others, I sometimes choose variable names in other human languages, sometimes transliterated as sticking to ASCII requires. Most people are not able to even guess, so say commenting in Hungarian or German or Hebrew may only be of use to me and other weird people. I do find that for frequent enough usage of something, such as an object like a deque, you get to know it so well that you may stop commenting on how you are using some built-in feature as it seems routine. -----Original Message----- From: Python-list On Behalf Of Peter J. Holzer Sent: Monday, September 13, 2021 3:28 PM To: python-list at python.org Subject: Re: Friday Finking: Contorted loops On 2021-09-12 17:11:58 -0400, Avi Gross via Python-list wrote: > Yes, large units of code, and even smaller ones, may be a chore to > figure out. Arguably harder when you use indentation and the next/last > parts are not even on the same screen as the rest. Sometimes you want > to use a split-screen in some editor to line up the two parts or some > other technique. > > But my suggestion is to COMMENT things well and I mean too much! > > do { > > } while > > Why not add a comment at the top like: > > # The following loop has a WHILE clause controlling it at the end. > > do { # until the while clause below > > } while # End of the do loop. Because those comments don't tell me anything that I as a C programmer don't already know. Even though do/while loops are relatively rare, I've seen hundreds of them. Seeing "do {" and recognizing it as the top of a do/while loop takes absolutely no conscious thought - reading a comment does. > My code tends to have brief comments especially when I have nested > constructs such as multiple nested loops or in sequence, or if > statements inside others. The comment often looks like > > ... # END of inner if > > ... # END of outer if Those are a bit better, but they still don't help much. What is the "inner if", what is the "outer if"? Chances are that I have to find the corresponding if to find out - I can do that without a comment, too. There are end-of-construct comments which I do find useful (at least occasionally) but those are semantic. Imagine a somewhat longish function which read several files which are comprised of records which are comprised of field. for file in files: some setup for each file here for record in file: some setup for each record here for field in record: process field do some postprocessing for the record do some postprocessing for the file If each of these blocks is longer than a few lines you might lose track where you are, so some comments might help: for file in files: some setup for each file here for record in file: some setup for each record here for field in record: process field # end of field loop do some postprocessing for the record # end of record loop do some postprocessing for the file # end of file loop Note that the comments say which part of the input the loop is processing, not just that it is the end of a loop or "the outer loop", "the intermediate loop" and "the inner loop" or some other purely syntactic information. (In most cases I would just break up such a function into smaller functions instead of adding comments, though) hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" From janburse at fastmail.fm Tue Sep 14 08:56:16 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Tue, 14 Sep 2021 14:56:16 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: I am testing a Prolog interpreter written in Python. So fibonacci number routine is written in Prolog and I am running the fibonnaci number routine inside the Prolog interpreter that is written in Python. The factor 6x times faster of GraalVM can be reproduced also for other Prolog programs running inside the Prolog interpreter that is written in Python. I have a benchmark suite, where I get, the figures are milliseconds: Test Standard GraalVM Total 170'996 28'523 This means the factor is: 170'996 / 28'523 = 5.9950 The test harness, test cases and individual results for all test cases are found here: And we could test GraalVM Python, results are from 14.09.2021, tested with Dogelog Runtime 0.9.5, Python Version: https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587 Terry Reedy wrote: > On 9/13/2021 8:46 AM, Mostowski Collapse wrote: >> The Standard Python version of Dogelog runtime >> is annoyingly slow. So we gave it a try with >> andother Python, and it was 6x times faster. >> >> We could test GraalVM. We worked around the missing >> match in Python 3.8 by replacing it with if-then-else. >> Performance is a little better, we find: >> >> /* Standard Python Version, Warm Run */ >> ?- time(fibo(23,X)). >> % Wall 3865 ms, gc 94 ms, 71991 lips >> X = 46368. >> >> /* GraalVM Python Version, Warm Warm Run */ >> ?- time(fibo(23,X)). >> % Wall 695 ms, gc 14 ms, 400356 lips >> X = 46368. >> >> See also: >> >> JDK 1.8 GraalVM Python is 6x faster than Standard Python >> https://twitter.com/dogelogch/status/1437395917167112193 >> >> JDK 1.8 GraalVM Python is 6x faster than Standard Python >> https://www.facebook.com/groups/dogelog > > You need to test more than fibonacci to make that claim.? There is a > benchmark test that times around 40 different similarly small benchmarks. > > From janburse at fastmail.fm Tue Sep 14 09:01:58 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Tue, 14 Sep 2021 15:01:58 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: But even when using GraalVM Python, the execution is still slow. Much slower then the same Prolog interpreter written in JavaScript. For JavaScript node.exe I get much better results. I get these results comparing to a few other new Prolog systems as well: Test Dogelog Scryer Trealla Total 4427 7325 1824 The test harness, test cases and individual results for all test cases are found here: Meanwhile could also test Trealla, results are from 12.09.2021, tested with Dogelog Runtime 0.9.5, JavaScript Version: https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3890013 But this is all only a moment in time. The Prolog interpreter itself is evolving, and the programming languages also, so this is a moving target. Benchmark results will look different tomorrow. Mostowski Collapse wrote: > I am testing a Prolog interpreter written > in Python. So fibonacci number routine is > written in Prolog and I am running the > > fibonnaci number routine inside the > Prolog interpreter that is written in > Python. The factor 6x times faster of > > GraalVM can be reproduced also for other > Prolog programs running inside the Prolog > interpreter that is written in Python. > > I have a benchmark suite, where I get, > the figures are milliseconds: > > Test??? Standard??? GraalVM > Total???? 170'996????? 28'523 > > This means the factor is: > > 170'996 / 28'523 = 5.9950 > > The test harness, test cases and individual > results for all test cases are found here: > > And we could test GraalVM Python, results are from 14.09.2021, > tested with Dogelog Runtime 0.9.5, Python Version: > https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587 > > > Terry Reedy wrote: >> On 9/13/2021 8:46 AM, Mostowski Collapse wrote: >>> The Standard Python version of Dogelog runtime >>> is annoyingly slow. So we gave it a try with >>> andother Python, and it was 6x times faster. >>> >>> We could test GraalVM. We worked around the missing >>> match in Python 3.8 by replacing it with if-then-else. >>> Performance is a little better, we find: >>> >>> /* Standard Python Version, Warm Run */ >>> ?- time(fibo(23,X)). >>> % Wall 3865 ms, gc 94 ms, 71991 lips >>> X = 46368. >>> >>> /* GraalVM Python Version, Warm Warm Run */ >>> ?- time(fibo(23,X)). >>> % Wall 695 ms, gc 14 ms, 400356 lips >>> X = 46368. >>> >>> See also: >>> >>> JDK 1.8 GraalVM Python is 6x faster than Standard Python >>> https://twitter.com/dogelogch/status/1437395917167112193 >>> >>> JDK 1.8 GraalVM Python is 6x faster than Standard Python >>> https://www.facebook.com/groups/dogelog >> >> You need to test more than fibonacci to make that claim.? There is a >> benchmark test that times around 40 different similarly small benchmarks. >> >> > From Joseph.Schachner at Teledyne.com Tue Sep 14 12:30:12 2021 From: Joseph.Schachner at Teledyne.com (Schachner, Joseph) Date: Tue, 14 Sep 2021 16:30:12 +0000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: Opinion: Anyone who is counting on Python for truly fast compute speed is probably using Python for the wrong purpose. Here, we use Python to control Test Equipment, to set up the equipment and ask for a measurement, get it, and proceed to the next measurement; and at the end produce a nice formatted report. If we wrote the test script in C or Rust or whatever it could not run substantially faster because it is communicating with the test equipment, setting it up and waiting for responses, and that is where the vast majority of the time goes. Especially if the measurement result requires averaging it can take a while. In my opinion this is an ideal use for Python, not just because the speed of Python is not important, but also because we can easily find people who know Python, who like coding in Python, and will join the company to program in Python ... and stay with us. --- Joseph S. Teledyne Confidential; Commercially Sensitive Business Data -----Original Message----- From: Mostowski Collapse Sent: Tuesday, September 14, 2021 8:56 AM To: python-list at python.org Subject: Re: ANN: Dogelog Runtime, Prolog to the Moon (2021) I am testing a Prolog interpreter written in Python. So fibonacci number routine is written in Prolog and I am running the fibonnaci number routine inside the Prolog interpreter that is written in Python. The factor 6x times faster of GraalVM can be reproduced also for other Prolog programs running inside the Prolog interpreter that is written in Python. I have a benchmark suite, where I get, the figures are milliseconds: Test Standard GraalVM Total 170'996 28'523 This means the factor is: 170'996 / 28'523 = 5.9950 The test harness, test cases and individual results for all test cases are found here: And we could test GraalVM Python, results are from 14.09.2021, tested with Dogelog Runtime 0.9.5, Python Version: https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587 Terry Reedy wrote: > On 9/13/2021 8:46 AM, Mostowski Collapse wrote: >> The Standard Python version of Dogelog runtime is annoyingly slow. So >> we gave it a try with andother Python, and it was 6x times faster. >> >> We could test GraalVM. We worked around the missing match in Python >> 3.8 by replacing it with if-then-else. >> Performance is a little better, we find: >> >> /* Standard Python Version, Warm Run */ >> ?- time(fibo(23,X)). >> % Wall 3865 ms, gc 94 ms, 71991 lips >> X = 46368. >> >> /* GraalVM Python Version, Warm Warm Run */ >> ?- time(fibo(23,X)). >> % Wall 695 ms, gc 14 ms, 400356 lips >> X = 46368. >> >> See also: >> >> JDK 1.8 GraalVM Python is 6x faster than Standard Python >> https://twitter.com/dogelogch/status/1437395917167112193 >> >> JDK 1.8 GraalVM Python is 6x faster than Standard Python >> https://www.facebook.com/groups/dogelog > > You need to test more than fibonacci to make that claim.? There is a > benchmark test that times around 40 different similarly small benchmarks. > > From arj.python at gmail.com Wed Sep 15 09:39:06 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 15 Sep 2021 17:39:06 +0400 Subject: The code version of python -i Message-ID: Greetings, If i have a file name flower.py and i add x = 1 in it. When i run python -i flower.py i get a shell >>> If type x i get 1 >>> x 1 The values are auto injected. How do i start a shell by code with values already injected? Thanks Kind Regards, Abdur-Rahmaan Janhangeer about | blog github Mauritius From __peter__ at web.de Wed Sep 15 10:08:32 2021 From: __peter__ at web.de (Peter Otten) Date: Wed, 15 Sep 2021 16:08:32 +0200 Subject: The code version of python -i In-Reply-To: References: Message-ID: On 15/09/2021 15:39, Abdur-Rahmaan Janhangeer wrote: > Greetings, > > If i have a file name flower.py and i add x = 1 in it. > When i run python -i flower.py i get a shell >>>> > > If type x i get 1 >>>> x > 1 > > The values are auto injected. > > How do i start a shell by code with values already injected? Thanks > > Kind Regards, I tried import code x = 42 code.interact(local=globals()) but didn't bother to read the documentation at https://docs.python.org/3/library/code.html and thus do not know what the limitations might be. From janburse at fastmail.fm Wed Sep 15 09:36:58 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Wed, 15 Sep 2021 15:36:58 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: I am not testing this use-case. But a related use-case might highlight why speed did never hurt anybody. Lets say you program a flying drone with Python, and the measurement is from the drone sensor and communication systems. Lets say you are using the idle time between measurements for some complex planning. It is then not true that you have anyway to wait for the measurement. Hope this helps! BTW: If somebody knows another Python implementation I am happy to test this implementation as well. I am assuming that the standard Python python.exe I tested amounts to CPython? Not sure. And the GraalVM is practically the same as JPython? Not sure either. > Opinion: Anyone who is counting on Python > for truly fast compute speed is probably using > Python for the wrong purpose. > Here, we use Python to control Test Equipment, > to set up the equipment and ask for a measurement, > get it, and proceed to the next measurement; and > at the end produce a nice formatted report. > If we wrote the test script in C or Rust or > whatever it could not run substantially faster > because it is communicating with the test equipment, > setting it up and waiting for responses, and > that is where the vast majority of the time goes. > Especially if the measurement result requires > averaging it can take a while. In my opinion > this is an ideal use for Python, not just because > the speed of Python is not important, but also > because we can easily find people who know Python, > who like coding in Python, and will join the > company to program in Python ... and stay with us. > > --- Joseph S. From janburse at fastmail.fm Wed Sep 15 10:02:36 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Wed, 15 Sep 2021 16:02:36 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: Oops "speed did never hurt anybody". Don't be evil, I am talking about unarmed drones. See also: Drone Programming With Python Course https://www.youtube.com/watch?v=LmEcyQnfpDA Mostowski Collapse schrieb: > I am not testing this use-case. But a related > use-case might highlight why speed did never > hurt anybody. > > Lets say you program a flying drone with Python, > and the measurement is from the drone sensor > and communication systems. > > Lets say you are using the idle time between > measurements for some complex planning. It > is then not true that you have anyway > > to wait for the measurement. > > Hope this helps! > > BTW: If somebody knows another Python implementation > I am happy to test this implementation as well. > I am assuming that the standard Python python.exe > > I tested amounts to CPython? Not sure. And the > GraalVM is practically the same as JPython? Not > sure either. > >> Opinion:?? Anyone who is counting on Python for truly fast compute >> speed is probably using Python for the wrong purpose. Here, we use >> Python to control Test Equipment, to set up the equipment and ask for >> a measurement, get it, and proceed to the next measurement; and at the >> end produce a nice formatted report. If we wrote the test script in C >> or Rust or whatever it could not run substantially faster because it >> is communicating with the test equipment, setting it up and waiting >> for responses, and that is where the vast majority of the time goes. >> Especially if the measurement result requires averaging it can take a >> while.? In my opinion this is an ideal use for Python, not just >> because the speed of Python is not important, but also because we can >> easily find people who know Python, who like coding in Python, and >> will join the company to program in Python ... and stay with us. >> --- Joseph S. > From janburse at fastmail.fm Wed Sep 15 12:23:10 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Wed, 15 Sep 2021 18:23:10 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: I really wonder why my Python implementation is a factor 40 slower than my JavaScript implementation. Structurally its the same code. You can check yourself: Python Version: https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py JavaScript Version: https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js Its the same while, if-then-else, etc.. its the same classes Variable, Compound etc.. Maybe I could speed it up by some details. For example to create an array of length n, I use in Python: temp = [NotImplemented] * code[pos] pos += 1 Whereas in JavaScript I use, also in exec_build2(): temp = new Array(code[pos++]); So I hear Guido doesn't like ++. So in Python I use += and a separate statement as a workaround. But otherwise, what about the creation of an array, is the the idiom [_] * _ slow? I am assuming its compiled away. Or does it really first create an array of size 1 and then enlarge it? Julio Di Egidio wrote: > On Wednesday, 15 September 2021 at 15:37:19 UTC+2, Mostowski Collapse wrote: > >>> Opinion: Anyone who is counting on Python >>> for truly fast compute speed is probably using >>> Python for the wrong purpose. > > You just don't know anything about this environment: those who need fast computation rather use *libraries* where all the performance critical parts are written in native code... and that's pretty customary in Python. > > By that I don't mean Python is flawless, indeed (IMO) it isn't in so many ways: to the point that, for more professional solutions in the maths/statistics realms in particular, people rather use R: yet, the primary reason is not so much performance but really the solidity/structure of the language per se... > > Julio > From rosuav at gmail.com Wed Sep 15 13:26:39 2021 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 16 Sep 2021 03:26:39 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse wrote: > > I really wonder why my Python implementation > is a factor 40 slower than my JavaScript implementation. > Structurally its the same code. > Very hard to know. Your code is detailed and complicated. Do they produce identical results? Are you using the same sort of floating-point data everywhere, or is one integer and the other float? What's going on with all the globals, the continuations, etc? My suspicion is that you're trying to write weird, wonky Python code, and then are surprised that it doesn't perform well. ChrisA From bursejan at gmail.com Wed Sep 15 14:13:23 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:13:23 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: If you find a "wonky" spot, I can replace it by "non-wonky" code. I noticed some differences between Python Dicts and JavaScript objects. Python tends to throw more exceptions. So in Python I now do the following: peek = kb.get(functor, NotImplemented) if peek is not NotImplemented: In JavaScript I can directly do: peek = kb[functor]; if (peek !== undefined) But if get() in Python is implemented under the hood with exception handling. i.e. using the exception prone [] and then in case an exception is thrown, returning the default value, then Python get() will probably be quite slow. Since usually exceptions are slow. Chris Angelico schrieb am Mittwoch, 15. September 2021 um 19:27:13 UTC+2: > On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse wrote: > > > > I really wonder why my Python implementation > > is a factor 40 slower than my JavaScript implementation. > > Structurally its the same code. > > > Very hard to know. Your code is detailed and complicated. Do they > produce identical results? Are you using the same sort of > floating-point data everywhere, or is one integer and the other float? > What's going on with all the globals, the continuations, etc? My > suspicion is that you're trying to write weird, wonky Python code, and > then are surprised that it doesn't perform well. > > ChrisA From alister.ware at ntlworld.com Wed Sep 15 14:17:03 2021 From: alister.ware at ntlworld.com (alister) Date: Wed, 15 Sep 2021 18:17:03 -0000 (UTC) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) References: Message-ID: On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: > I really wonder why my Python implementation is a factor 40 slower than > my JavaScript implementation. > Structurally its the same code. > > You can check yourself: > > Python Version: > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ machine.py > > JavaScript Version: > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ machine.js > > Its the same while, if-then-else, etc.. its the same classes Variable, > Compound etc.. Maybe I could speed it up by some details. For example to > create an array of length n, I use in Python: > > temp = [NotImplemented] * code[pos] > pos += 1 > > Whereas in JavaScript I use, also in exec_build2(): > > temp = new Array(code[pos++]); > > So I hear Guido doesn't like ++. So in Python I use += > and a separate statement as a workaround. But otherwise, > what about the creation of an array, > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or does > it really first create an array of size 1 and then enlarge it? > > Julio Di Egidio wrote: this is probably a string contender i = 0 while i < len(term.args) - 1: mark_term(term.args[i]) i += 1 term = term.args[i] try replacing with something more pythonic for index,term in enumerate(term.args): mark_term(term.args[i]) & possibly go all the way to changing it into a comprehension there are other similar anti patterns throughout this code. any time you are manually keeping a counter as an index into a list,tupple other iterable YOU ARE DOING IT WRONG! Do not write javascript in python, write python -- Two percent of zero is almost nothing. -- Whoever dies with the most toys wins. From alister.ware at ntlworld.com Wed Sep 15 14:22:37 2021 From: alister.ware at ntlworld.com (alister) Date: Wed, 15 Sep 2021 18:22:37 -0000 (UTC) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) References: Message-ID: On Thu, 16 Sep 2021 03:26:39 +1000, Chris Angelico wrote: > On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse > wrote: >> >> I really wonder why my Python implementation is a factor 40 slower than >> my JavaScript implementation. >> Structurally its the same code. >> >> > Very hard to know. Your code is detailed and complicated. Do they > produce identical results? Are you using the same sort of floating-point > data everywhere, or is one integer and the other float? > What's going on with all the globals, the continuations, etc? My > suspicion is that you're trying to write weird, wonky Python code, and > then are surprised that it doesn't perform well. > > ChrisA And this demonstrates how an experienced Python programmer can make an almost spot on diagnosis without even checking the source code. @ this stage I would recommend watching some presentations on you tube this one https://www.youtube.com/watch?v=wf-BqAjZb8M by Raymond Hettinger is brilliant as it highlights there is more to checking code than just making sure it looks nice & runs correctly. -- Lemmings don't grow older, they just die. From bursejan at gmail.com Wed Sep 15 14:22:39 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:22:39 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> Do you mean, replace this: i = 0 while i < len(term.args) - 1: ____mark_term(term.args[i]) ____i += 1 term = term.args[i] By this: for i,term in enumerate(term.args): ____mark_term(term.args[i]) This wouldn't be correct anymore. The recursive call is only for the arguments except for the last one one. alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2: > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: > > > I really wonder why my Python implementation is a factor 40 slower than > > my JavaScript implementation. > > Structurally its the same code. > > > > You can check yourself: > > > > Python Version: > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ > machine.py > > > > JavaScript Version: > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ > machine.js > > > > Its the same while, if-then-else, etc.. its the same classes Variable, > > Compound etc.. Maybe I could speed it up by some details. For example to > > create an array of length n, I use in Python: > > > > temp = [NotImplemented] * code[pos] > > pos += 1 > > > > Whereas in JavaScript I use, also in exec_build2(): > > > > temp = new Array(code[pos++]); > > > > So I hear Guido doesn't like ++. So in Python I use += > > and a separate statement as a workaround. But otherwise, > > what about the creation of an array, > > > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or does > > it really first create an array of size 1 and then enlarge it? > > > > Julio Di Egidio wrote: > > > this is probably a string contender > > i = 0 > while i < len(term.args) - 1: > mark_term(term.args[i]) > i += 1 > term = term.args[i] > > try replacing with something more pythonic > > for index,term in enumerate(term.args): > mark_term(term.args[i]) > > > & possibly go all the way to changing it into a comprehension > > there are other similar anti patterns throughout this code. > > any time you are manually keeping a counter as an index into a list,tupple > other iterable YOU ARE DOING IT WRONG! > > Do not write javascript in python, write python > > > > -- > Two percent of zero is almost nothing. > > > > > -- > Whoever dies with the most toys wins. From bursejan at gmail.com Wed Sep 15 14:26:54 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:26:54 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: <5452f15f-857b-42df-891a-364862841cc1n@googlegroups.com> Well I would be more than happy if an experienced programmer can fine tune my code. My programming experience in Python is only 4 weeks. Mostowski Collapse schrieb am Dienstag, 14. September 2021 um 14:56:35 UTC+2: > The test harness, test cases and individual > results for all test cases are found here: > > And we could test GraalVM Python, results are from 14.09.2021, > tested with Dogelog Runtime 0.9.5, Python Version: > https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587 If you follow the above link, you also find: Test Harness https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#file-suite2-pl Test Cases https://github.com/jburse/jekejeke-samples/tree/master/jekrun_bench/core/tests CPU / RAM: Intel(R) Core(TM) i7-6700HQ, 32 GB Standard Python: python.exe Python 3.10.0rc1 GraalVM Python: WSL 2, Python 3.8.5 (GraalVM CE Native 21.2.0) alister schrieb am Mittwoch, 15. September 2021 um 20:22:56 UTC+2: > On Thu, 16 Sep 2021 03:26:39 +1000, Chris Angelico wrote: > > > On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse > > wrote: > >> > >> I really wonder why my Python implementation is a factor 40 slower than > >> my JavaScript implementation. > >> Structurally its the same code. > >> > >> > > Very hard to know. Your code is detailed and complicated. Do they > > produce identical results? Are you using the same sort of floating-point > > data everywhere, or is one integer and the other float? > > What's going on with all the globals, the continuations, etc? My > > suspicion is that you're trying to write weird, wonky Python code, and > > then are surprised that it doesn't perform well. > > > > ChrisA > And this demonstrates how an experienced Python programmer can make an > almost spot on diagnosis without even checking the source code. > > @ this stage I would recommend watching some presentations on you tube > > this one https://www.youtube.com/watch?v=wf-BqAjZb8M by Raymond Hettinger > is brilliant as it highlights there is more to checking code than just > making sure it looks nice & runs correctly. > > > > -- > Lemmings don't grow older, they just die. From wlfraed at ix.netcom.com Wed Sep 15 14:32:48 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 15 Sep 2021 14:32:48 -0400 Subject: The code version of python -i References: Message-ID: On Wed, 15 Sep 2021 16:08:32 +0200, Peter Otten <__peter__ at web.de> declaimed the following: > >I tried > >import code > >x = 42 >code.interact(local=globals()) > >but didn't bother to read the documentation at > >https://docs.python.org/3/library/code.html > >and thus do not know what the limitations might be. Well, offhand, I'd say you need to provide a function to be used for raw_input (which will have to provide prompts, etc.). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From bursejan at gmail.com Wed Sep 15 14:31:48 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:31:48 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> Message-ID: <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> There is a further problem with this: > for i,term in enumerate(term.args): > ____mark_term(term.args[i]) It should read: for i,help in enumerate(term.args): ____mark_term(help) But then i isn't need. Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50 UTC+2: > Do you mean, replace this: > i = 0 > while i < len(term.args) - 1: > ____mark_term(term.args[i]) > ____i += 1 > term = term.args[i] > > By this: > > for i,term in enumerate(term.args): > ____mark_term(term.args[i]) > > This wouldn't be correct anymore. The > recursive call is only for the arguments > except for the last one one. > alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2: > > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: > > > > > I really wonder why my Python implementation is a factor 40 slower than > > > my JavaScript implementation. > > > Structurally its the same code. > > > > > > You can check yourself: > > > > > > Python Version: > > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ > > machine.py > > > > > > JavaScript Version: > > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ > > machine.js > > > > > > Its the same while, if-then-else, etc.. its the same classes Variable, > > > Compound etc.. Maybe I could speed it up by some details. For example to > > > create an array of length n, I use in Python: > > > > > > temp = [NotImplemented] * code[pos] > > > pos += 1 > > > > > > Whereas in JavaScript I use, also in exec_build2(): > > > > > > temp = new Array(code[pos++]); > > > > > > So I hear Guido doesn't like ++. So in Python I use += > > > and a separate statement as a workaround. But otherwise, > > > what about the creation of an array, > > > > > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or does > > > it really first create an array of size 1 and then enlarge it? > > > > > > Julio Di Egidio wrote: > > > > > > this is probably a string contender > > > > i = 0 > > while i < len(term.args) - 1: > > mark_term(term.args[i]) > > i += 1 > > term = term.args[i] > > > > try replacing with something more pythonic > > > > for index,term in enumerate(term.args): > > mark_term(term.args[i]) > > > > > > & possibly go all the way to changing it into a comprehension > > > > there are other similar anti patterns throughout this code. > > > > any time you are manually keeping a counter as an index into a list,tupple > > other iterable YOU ARE DOING IT WRONG! > > > > Do not write javascript in python, write python > > > > > > > > -- > > Two percent of zero is almost nothing. > > > > > > > > > > -- > > Whoever dies with the most toys wins. From alister.ware at ntlworld.com Wed Sep 15 14:40:52 2021 From: alister.ware at ntlworld.com (alister) Date: Wed, 15 Sep 2021 18:40:52 -0000 (UTC) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> Message-ID: On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote: > There is a further problem with this: > >> for i,term in enumerate(term.args): >> ____mark_term(term.args[i]) > > It should read: > > for i,help in enumerate(term.args): > ____mark_term(help) > > But then i isn't need. even Better (i had only skimmed the code as I was certain I would find this, it is probably the No. 1 thing new python programmers get wrong if your example is correct the it can be simplified even further to for help in term.args: mark_term(help) & if help does not get used after this loop then a comprehension is even better _ == [mark_term(help) for help in term.args] the underscore character is python convention for an unneeded place- holder variable. > > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50 > UTC+2: >> Do you mean, replace this: >> i = 0 while i < len(term.args) - 1: >> ____mark_term(term.args[i]) >> ____i += 1 term = term.args[i] >> >> By this: >> >> for i,term in enumerate(term.args): >> ____mark_term(term.args[i]) >> >> This wouldn't be correct anymore. The recursive call is only for the >> arguments except for the last one one. >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2: >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: >> > >> > > I really wonder why my Python implementation is a factor 40 slower >> > > than my JavaScript implementation. >> > > Structurally its the same code. >> > > >> > > You can check yourself: >> > > >> > > Python Version: >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ >> > machine.py >> > > >> > > JavaScript Version: >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ >> > machine.js >> > > >> > > Its the same while, if-then-else, etc.. its the same classes >> > > Variable, Compound etc.. Maybe I could speed it up by some details. >> > > For example to create an array of length n, I use in Python: >> > > >> > > temp = [NotImplemented] * code[pos] >> > > pos += 1 >> > > >> > > Whereas in JavaScript I use, also in exec_build2(): >> > > >> > > temp = new Array(code[pos++]); >> > > >> > > So I hear Guido doesn't like ++. So in Python I use += >> > > and a separate statement as a workaround. But otherwise, >> > > what about the creation of an array, >> > > >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or >> > > does it really first create an array of size 1 and then enlarge it? >> > > >> > > Julio Di Egidio wrote: >> > >> > >> > this is probably a string contender >> > >> > i = 0 while i < len(term.args) - 1: >> > mark_term(term.args[i]) >> > i += 1 term = term.args[i] >> > >> > try replacing with something more pythonic >> > >> > for index,term in enumerate(term.args): >> > mark_term(term.args[i]) >> > >> > >> > & possibly go all the way to changing it into a comprehension >> > >> > there are other similar anti patterns throughout this code. >> > >> > any time you are manually keeping a counter as an index into a >> > list,tupple other iterable YOU ARE DOING IT WRONG! >> > >> > Do not write javascript in python, write python >> > >> > >> > >> > -- >> > Two percent of zero is almost nothing. >> > >> > >> > >> > >> > -- >> > Whoever dies with the most toys wins. -- Pie are not square. Pie are round. Cornbread are square. From alister.ware at ntlworld.com Wed Sep 15 14:45:39 2021 From: alister.ware at ntlworld.com (alister) Date: Wed, 15 Sep 2021 18:45:39 -0000 (UTC) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> Message-ID: On Wed, 15 Sep 2021 18:40:52 +0000, alister wrote: > On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote: > >> There is a further problem with this: >> >>> for i,term in enumerate(term.args): >>> ____mark_term(term.args[i]) >> >> It should read: >> >> for i,help in enumerate(term.args): >> ____mark_term(help) >> >> But then i isn't need. > even Better (i had only skimmed the code as I was certain I would find > this, it is probably the No. 1 thing new python programmers get wrong if > your example is correct the it can be simplified even further to > > for help in term.args: > mark_term(help) > > & if help does not get used after this loop then a comprehension is even > better _ == [mark_term(help) for help in term.args] > > > the underscore character is python convention for an unneeded place- > holder variable. > > I also notice you are using match case - this was only introduced in python 10 so it is very new (i had not seen it before) the break statements are probably not necessary as if ii understand this feature correctly python does not fall-through & execute every subsequent case after a successful match. & finally the netiquette in this forum is to interleave of bottom post rather than top post, it makes it easier to follow the thread. -- Don't quit now, we might just as well lock the door and throw away the key. From bursejan at gmail.com Wed Sep 15 14:48:18 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:48:18 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> Message-ID: And how do you iterate over the first n-1 elements of a list with n elements? This is what my code does: i = 0 while i < len(term.args) - 1: ____mark_term(term.args[i]) ____i += 1 term = term.args[i] You can try yourself: % python3 >>> foo = ["a", "b", "c"] >>> i = 0 >>> while i < len(foo) - 1: ... print("mark_term", foo[i]) ... i += 1 ... mark_term a mark_term b >>> foo = foo[i] >>> foo 'c' alister schrieb am Mittwoch, 15. September 2021 um 20:41:12 UTC+2: > On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote: > > > There is a further problem with this: > > > >> for i,term in enumerate(term.args): > >> ____mark_term(term.args[i]) > > > > It should read: > > > > for i,help in enumerate(term.args): > > ____mark_term(help) > > > > But then i isn't need. > even Better (i had only skimmed the code as I was certain I would find > this, it is probably the No. 1 thing new python programmers get wrong > if your example is correct the it can be simplified even further to > > for help in term.args: > mark_term(help) > > & if help does not get used after this loop then a comprehension is even > better > _ == [mark_term(help) for help in term.args] > > > the underscore character is python convention for an unneeded place- > holder variable. > > > > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50 > > UTC+2: > >> Do you mean, replace this: > >> i = 0 while i < len(term.args) - 1: > >> ____mark_term(term.args[i]) > >> ____i += 1 term = term.args[i] > >> > >> By this: > >> > >> for i,term in enumerate(term.args): > >> ____mark_term(term.args[i]) > >> > >> This wouldn't be correct anymore. The recursive call is only for the > >> arguments except for the last one one. > >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2: > >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: > >> > > >> > > I really wonder why my Python implementation is a factor 40 slower > >> > > than my JavaScript implementation. > >> > > Structurally its the same code. > >> > > > >> > > You can check yourself: > >> > > > >> > > Python Version: > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ > >> > machine.py > >> > > > >> > > JavaScript Version: > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ > >> > machine.js > >> > > > >> > > Its the same while, if-then-else, etc.. its the same classes > >> > > Variable, Compound etc.. Maybe I could speed it up by some details. > >> > > For example to create an array of length n, I use in Python: > >> > > > >> > > temp = [NotImplemented] * code[pos] > >> > > pos += 1 > >> > > > >> > > Whereas in JavaScript I use, also in exec_build2(): > >> > > > >> > > temp = new Array(code[pos++]); > >> > > > >> > > So I hear Guido doesn't like ++. So in Python I use += > >> > > and a separate statement as a workaround. But otherwise, > >> > > what about the creation of an array, > >> > > > >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or > >> > > does it really first create an array of size 1 and then enlarge it? > >> > > > >> > > Julio Di Egidio wrote: > >> > > >> > > >> > this is probably a string contender > >> > > >> > i = 0 while i < len(term.args) - 1: > >> > mark_term(term.args[i]) > >> > i += 1 term = term.args[i] > >> > > >> > try replacing with something more pythonic > >> > > >> > for index,term in enumerate(term.args): > >> > mark_term(term.args[i]) > >> > > >> > > >> > & possibly go all the way to changing it into a comprehension > >> > > >> > there are other similar anti patterns throughout this code. > >> > > >> > any time you are manually keeping a counter as an index into a > >> > list,tupple other iterable YOU ARE DOING IT WRONG! > >> > > >> > Do not write javascript in python, write python > >> > > >> > > >> > > >> > -- > >> > Two percent of zero is almost nothing. > >> > > >> > > >> > > >> > > >> > -- > >> > Whoever dies with the most toys wins. > -- > Pie are not square. Pie are round. Cornbread are square. From bursejan at gmail.com Wed Sep 15 14:51:48 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:51:48 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> Message-ID: <22aba2fe-5b58-4274-a8b6-d1b2e137084bn@googlegroups.com> There is a Python 3.8 compatible version here: https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine2.py I have replaced match by if-then-else. So as to be able to test with GraalVM. GraalVM is still faster despite using if-then-else. But GraalVM needs some time to JIT the code. You need to make some cold runs before you see kicking it being fast. Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:48:31 UTC+2: > And how do you iterate over the first n-1 elements > of a list with n elements? This is what my code does: > i = 0 > while i < len(term.args) - 1: > ____mark_term(term.args[i]) > ____i += 1 > term = term.args[i] > You can try yourself: > > % python3 > >>> foo = ["a", "b", "c"] > >>> i = 0 > >>> while i < len(foo) - 1: > ... print("mark_term", foo[i]) > ... i += 1 > ... > mark_term a > mark_term b > >>> foo = foo[i] > >>> foo > 'c' > alister schrieb am Mittwoch, 15. September 2021 um 20:41:12 UTC+2: > > On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote: > > > > > There is a further problem with this: > > > > > >> for i,term in enumerate(term.args): > > >> ____mark_term(term.args[i]) > > > > > > It should read: > > > > > > for i,help in enumerate(term.args): > > > ____mark_term(help) > > > > > > But then i isn't need. > > even Better (i had only skimmed the code as I was certain I would find > > this, it is probably the No. 1 thing new python programmers get wrong > > if your example is correct the it can be simplified even further to > > > > for help in term.args: > > mark_term(help) > > > > & if help does not get used after this loop then a comprehension is even > > better > > _ == [mark_term(help) for help in term.args] > > > > > > the underscore character is python convention for an unneeded place- > > holder variable. > > > > > > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50 > > > UTC+2: > > >> Do you mean, replace this: > > >> i = 0 while i < len(term.args) - 1: > > >> ____mark_term(term.args[i]) > > >> ____i += 1 term = term.args[i] > > >> > > >> By this: > > >> > > >> for i,term in enumerate(term.args): > > >> ____mark_term(term.args[i]) > > >> > > >> This wouldn't be correct anymore. The recursive call is only for the > > >> arguments except for the last one one. > > >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2: > > >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: > > >> > > > >> > > I really wonder why my Python implementation is a factor 40 slower > > >> > > than my JavaScript implementation. > > >> > > Structurally its the same code. > > >> > > > > >> > > You can check yourself: > > >> > > > > >> > > Python Version: > > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ > > >> > machine.py > > >> > > > > >> > > JavaScript Version: > > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ > > >> > machine.js > > >> > > > > >> > > Its the same while, if-then-else, etc.. its the same classes > > >> > > Variable, Compound etc.. Maybe I could speed it up by some details. > > >> > > For example to create an array of length n, I use in Python: > > >> > > > > >> > > temp = [NotImplemented] * code[pos] > > >> > > pos += 1 > > >> > > > > >> > > Whereas in JavaScript I use, also in exec_build2(): > > >> > > > > >> > > temp = new Array(code[pos++]); > > >> > > > > >> > > So I hear Guido doesn't like ++. So in Python I use += > > >> > > and a separate statement as a workaround. But otherwise, > > >> > > what about the creation of an array, > > >> > > > > >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or > > >> > > does it really first create an array of size 1 and then enlarge it? > > >> > > > > >> > > Julio Di Egidio wrote: > > >> > > > >> > > > >> > this is probably a string contender > > >> > > > >> > i = 0 while i < len(term.args) - 1: > > >> > mark_term(term.args[i]) > > >> > i += 1 term = term.args[i] > > >> > > > >> > try replacing with something more pythonic > > >> > > > >> > for index,term in enumerate(term.args): > > >> > mark_term(term.args[i]) > > >> > > > >> > > > >> > & possibly go all the way to changing it into a comprehension > > >> > > > >> > there are other similar anti patterns throughout this code. > > >> > > > >> > any time you are manually keeping a counter as an index into a > > >> > list,tupple other iterable YOU ARE DOING IT WRONG! > > >> > > > >> > Do not write javascript in python, write python > > >> > > > >> > > > >> > > > >> > -- > > >> > Two percent of zero is almost nothing. > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Whoever dies with the most toys wins. > > -- > > Pie are not square. Pie are round. Cornbread are square. From bursejan at gmail.com Wed Sep 15 14:56:47 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 11:56:47 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> Message-ID: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> What could be slow, repeatedly requesting the "args" field. Maybe I should do: help = term.args i = 0 while i < len(help) - 1: ____mark_term(help[i]) ____i += 1 term = help[i] Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:48:31 UTC+2: > And how do you iterate over the first n-1 elements > of a list with n elements? This is what my code does: > i = 0 > while i < len(term.args) - 1: > ____mark_term(term.args[i]) > ____i += 1 > term = term.args[i] > You can try yourself: > > % python3 > >>> foo = ["a", "b", "c"] > >>> i = 0 > >>> while i < len(foo) - 1: > ... print("mark_term", foo[i]) > ... i += 1 > ... > mark_term a > mark_term b > >>> foo = foo[i] > >>> foo > 'c' > alister schrieb am Mittwoch, 15. September 2021 um 20:41:12 UTC+2: > > On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote: > > > > > There is a further problem with this: > > > > > >> for i,term in enumerate(term.args): > > >> ____mark_term(term.args[i]) > > > > > > It should read: > > > > > > for i,help in enumerate(term.args): > > > ____mark_term(help) > > > > > > But then i isn't need. > > even Better (i had only skimmed the code as I was certain I would find > > this, it is probably the No. 1 thing new python programmers get wrong > > if your example is correct the it can be simplified even further to > > > > for help in term.args: > > mark_term(help) > > > > & if help does not get used after this loop then a comprehension is even > > better > > _ == [mark_term(help) for help in term.args] > > > > > > the underscore character is python convention for an unneeded place- > > holder variable. > > > > > > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50 > > > UTC+2: > > >> Do you mean, replace this: > > >> i = 0 while i < len(term.args) - 1: > > >> ____mark_term(term.args[i]) > > >> ____i += 1 term = term.args[i] > > >> > > >> By this: > > >> > > >> for i,term in enumerate(term.args): > > >> ____mark_term(term.args[i]) > > >> > > >> This wouldn't be correct anymore. The recursive call is only for the > > >> arguments except for the last one one. > > >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2: > > >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote: > > >> > > > >> > > I really wonder why my Python implementation is a factor 40 slower > > >> > > than my JavaScript implementation. > > >> > > Structurally its the same code. > > >> > > > > >> > > You can check yourself: > > >> > > > > >> > > Python Version: > > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/ > > >> > machine.py > > >> > > > > >> > > JavaScript Version: > > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/ > > >> > machine.js > > >> > > > > >> > > Its the same while, if-then-else, etc.. its the same classes > > >> > > Variable, Compound etc.. Maybe I could speed it up by some details. > > >> > > For example to create an array of length n, I use in Python: > > >> > > > > >> > > temp = [NotImplemented] * code[pos] > > >> > > pos += 1 > > >> > > > > >> > > Whereas in JavaScript I use, also in exec_build2(): > > >> > > > > >> > > temp = new Array(code[pos++]); > > >> > > > > >> > > So I hear Guido doesn't like ++. So in Python I use += > > >> > > and a separate statement as a workaround. But otherwise, > > >> > > what about the creation of an array, > > >> > > > > >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or > > >> > > does it really first create an array of size 1 and then enlarge it? > > >> > > > > >> > > Julio Di Egidio wrote: > > >> > > > >> > > > >> > this is probably a string contender > > >> > > > >> > i = 0 while i < len(term.args) - 1: > > >> > mark_term(term.args[i]) > > >> > i += 1 term = term.args[i] > > >> > > > >> > try replacing with something more pythonic > > >> > > > >> > for index,term in enumerate(term.args): > > >> > mark_term(term.args[i]) > > >> > > > >> > > > >> > & possibly go all the way to changing it into a comprehension > > >> > > > >> > there are other similar anti patterns throughout this code. > > >> > > > >> > any time you are manually keeping a counter as an index into a > > >> > list,tupple other iterable YOU ARE DOING IT WRONG! > > >> > > > >> > Do not write javascript in python, write python > > >> > > > >> > > > >> > > > >> > -- > > >> > Two percent of zero is almost nothing. > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Whoever dies with the most toys wins. > > -- > > Pie are not square. Pie are round. Cornbread are square. From rosuav at gmail.com Wed Sep 15 15:29:35 2021 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 16 Sep 2021 05:29:35 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On Thu, Sep 16, 2021 at 5:15 AM Mostowski Collapse wrote: > > If you find a "wonky" spot, I can replace it by "non-wonky" > code. I noticed some differences between Python Dicts > and JavaScript objects. Python tends to throw more exceptions. > > So in Python I now do the following: > > peek = kb.get(functor, NotImplemented) > if peek is not NotImplemented: > > In JavaScript I can directly do: > > peek = kb[functor]; > if (peek !== undefined) > > But if get() in Python is implemented under the hood with > exception handling. i.e. using the exception prone [] and > then in case an exception is thrown, returning the > > default value, then Python get() will probably be quite slow. > Since usually exceptions are slow. > No, you're thinking in terms of microoptimizations. Exception handling isn't THAT slow. I'm talking more about how everything's getting packaged up and then unpackaged (the repeated use of the "Compound" class looks highly suboptimal), rather than reworking your algorithm to behave more cleanly. ChrisA From arj.python at gmail.com Wed Sep 15 16:15:16 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Thu, 16 Sep 2021 00:15:16 +0400 Subject: The code version of python -i In-Reply-To: References: Message-ID: Thanks folks will update with progress soon > From alister.ware at ntlworld.com Wed Sep 15 15:15:50 2021 From: alister.ware at ntlworld.com (alister) Date: Wed, 15 Sep 2021 19:15:50 -0000 (UTC) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> Message-ID: On Wed, 15 Sep 2021 11:48:18 -0700, Mostowski Collapse wrote: > And how do you iterate over the first n-1 elements of a list with n > elements? This is what my code does: > > i = 0 while i < len(term.args) - 1: > ____mark_term(term.args[i]) > ____i += 1 term = term.args[i] > > You can try yourself: use a slice as a generic example you can try in the interactive interpreter (aka REPL): >>>items = [1,2,3,4,5,6,7,8,9,0] >>>for item in items[:-1]: >>> print(item) 1 2 3 4 5 6 7 8 9 I should state for the record that I am no Python expert, I mainly visit this news group to learn. sometimes from reading solutions & somtimes by trying to work out what the solution might be. most productively by working out a solution & seeing if other posters agree (& working out why they don't) -- The universe seems neither benign nor hostile, merely indifferent. -- Sagan From alister.ware at ntlworld.com Wed Sep 15 16:00:11 2021 From: alister.ware at ntlworld.com (alister) Date: Wed, 15 Sep 2021 20:00:11 -0000 (UTC) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> Message-ID: On Wed, 15 Sep 2021 11:56:47 -0700, Mostowski Collapse wrote: > What could be slow, repeatedly requesting the "args" > field. Maybe I should do: > > help = term.args i = 0 while i < len(help) - 1: > ____mark_term(help[i]) > ____i += 1 term = help[i] > No this construct is a common error in new python programmers the next progression they make is when they discover the range function items = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0] for x in range(len(list)): print (items[x]) but that is equally inefficient the pythonic way is as previously suggested for item in items: print(item) then the management of the index is being handled by the python interpreter internally & is effectively machine code. every time you increment your own pointer the interpreter has to process reading the next line, reading the variable , incrementing it & then using it. this is what makes your current code slow. if you ever find your self creating a variable purely to use as a pointer into a list then you are almost certainly taking the wrong approach. more usefull links https://www.youtube.com/watch?v=zdJEYhA2AZQ -- "Show me a good loser, and I'll show you a loser." -- Vince Lombardi, football coach From bursejan at gmail.com Wed Sep 15 17:10:36 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 14:10:36 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> Message-ID: <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> And how do you only iterate over n-1 elements? I don't need a loop over all elements. With array slicing? Someting like: for item in items[0:len(items)-2]: ___print(item) Or with negative slicing indexes? Problem is my length can be equal to one. And when I have length equal to one, the slice might not do the right thing? LoL alister schrieb am Mittwoch, 15. September 2021 um 22:00:30 UTC+2: > for item in items: > print(item) From bursejan at gmail.com Wed Sep 15 17:13:00 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 14:13:00 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> Message-ID: <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> Ok you suggested: >>>items = [1,2,3,4,5,6,7,8,9,0] >>>for item in items[:-1]: >>> print(item) 1 2 3 4 5 6 7 8 9 Does this also work for length = 1? Ok let me try: >>> foo = ["a","b","c"] >>> for x in foo[:-1]: ... print(x) ... a b >>> foo = ["a"] >>> for x in foo[:-1]: ... print(x) ... Oki Doki Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:10:50 UTC+2: > And how do you only iterate over n-1 elements? > I don't need a loop over all elements. > > With array slicing? > > Someting like: > > for item in items[0:len(items)-2]: > ___print(item) > > Or with negative slicing indexes? Problem > is my length can be equal to one. > > And when I have length equal to one, the > slice might not do the right thing? > > LoL > alister schrieb am Mittwoch, 15. September 2021 um 22:00:30 UTC+2: > > for item in items: > > print(item) From bursejan at gmail.com Wed Sep 15 17:19:02 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 14:19:02 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> Message-ID: <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> BTW: I could already make it faster, by not repeatedly accessing .arg anymore. It went down from ca.: 171'000 ms To this here: 140'000 ms But only in the cold run. In the warm run it went back to 171'000 ms. Possibly when my code is faster, it will create objects more faster, and kill the Python GC. Or it was because my Laptop went into screen black? And throttled the CPU. Not sure. Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:13:12 UTC+2: > Ok you suggested: > >>>items = [1,2,3,4,5,6,7,8,9,0] > >>>for item in items[:-1]: > >>> print(item) > > 1 > 2 > 3 > 4 > 5 > 6 > 7 > 8 > 9 > Does this also work for length = 1? Ok let me try: > >>> foo = ["a","b","c"] > >>> for x in foo[:-1]: > ... print(x) > ... > a > b > >>> foo = ["a"] > >>> for x in foo[:-1]: > ... print(x) > ... > > Oki Doki > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:10:50 UTC+2: > > And how do you only iterate over n-1 elements? > > I don't need a loop over all elements. > > > > With array slicing? > > > > Someting like: > > > > for item in items[0:len(items)-2]: > > ___print(item) > > > > Or with negative slicing indexes? Problem > > is my length can be equal to one. > > > > And when I have length equal to one, the > > slice might not do the right thing? > > > > LoL > > alister schrieb am Mittwoch, 15. September 2021 um 22:00:30 UTC+2: > > > for item in items: > > > print(item) From bursejan at gmail.com Wed Sep 15 17:29:36 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 14:29:36 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <9nt0J.15689$4X4.14148@fx27.iad> References: <9nt0J.15689$4X4.14148@fx27.iad> Message-ID: <6a7734b2-0e15-4789-8abe-d0bee0b5b634n@googlegroups.com> Thank you for the suggestion. The test harness is invoked as follows. So it does already do time/1, thats also how I did the comparison Standard Python and GraalVM Python, a file dogelog.py: import sys # sys.path.append("\jekrun_bench\core\harness2\libpy") sys.path.append("/mnt/c//jekrun_bench/core/harness2/libpy") from index import init, consult init() consult(":- ['suite2.p']. " ":- time(suite). " ":- nl. " ":- time(suite). ") Here you see a GraalVM cold and warm run.The warm run is faster. If you do a warm warm run, it even gets more faster, because of JIT-ing, Just-in-Time machine compilation, via the GraalVM Truffles framework: $ export PATH=/graalvm-ce-java8-21.2.0/bin:$PATH $ cd /mnt/c//jekrun_bench/core/harness2 $ graalpython /mnt/c//jekrun_bench/core/harness2/dogelog.py nrev % Wall 6175 ms, gc 212 ms, 154473 lips crypt % Wall 9327 ms, gc 63 ms, 112838 lips deriv % Wall 4101 ms, gc 90 ms, 321890 lips poly % Wall 3594 ms, gc 415 ms, 216299 lips sortq % Wall 3427 ms, gc 67 ms, 290070 lips tictac % Wall 2770 ms, gc 51 ms, 136580 lips queens % Wall 3287 ms, gc 64 ms, 325617 lips query % Wall 1432 ms, gc 77 ms, 382969 lips mtak % Wall 2532 ms, gc 95 ms, 533881 lips perfect % Wall 3980 ms, gc 76 ms, 290382 lips % Wall 40745 ms, gc 1212 ms, 235751 lips nrev % Wall 4508 ms, gc 112 ms, 211595 lips crypt % Wall 6063 ms, gc 61 ms, 173584 lips deriv % Wall 3150 ms, gc 42 ms, 419070 lips poly % Wall 3549 ms, gc 432 ms, 219042 lips sortq % Wall 3196 ms, gc 63 ms, 311036 lips tictac % Wall 2670 ms, gc 52 ms, 141695 lips queens % Wall 3087 ms, gc 60 ms, 346713 lips query % Wall 1434 ms, gc 25 ms, 382435 lips mtak % Wall 2596 ms, gc 90 ms, 520719 lips perfect % Wall 3521 ms, gc 43 ms, 328236 lips % Wall 33810 ms, gc 980 ms, 284108 lips DFS schrieb am Mittwoch, 15. September 2021 um 23:15:07 UTC+2: > On 9/15/2021 12:23 PM, Mostowski Collapse wrote: > > I really wonder why my Python implementation > > is a factor 40 slower than my JavaScript implementation. > > Structurally its the same code. > > > > You can check yourself: > > > > Python Version: > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py > > > > JavaScript Version: > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js > > > > Its the same while, if-then-else, etc.. its the same > > classes Variable, Compound etc.. Maybe I could speed > > it up by some details. For example to create an array > > of length n, I use in Python: > > > > temp = [NotImplemented] * code[pos] > > pos += 1 > > > > Whereas in JavaScript I use, also > > in exec_build2(): > > > > temp = new Array(code[pos++]); > > > > So I hear Guido doesn't like ++. So in Python I use += > > and a separate statement as a workaround. But otherwise, > > what about the creation of an array, > > > > is the the idiom [_] * _ slow? I am assuming its > > compiled away. Or does it really first create an > > array of size 1 and then enlarge it? > I'm sure you know you can put in timing statements to find bottlenecks. > > import time > startTime = time.perf_counter() > [code block] > print("%.2f" % (time.perf_counter() - startTime)) From bursejan at gmail.com Wed Sep 15 17:33:03 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 14:33:03 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <6a7734b2-0e15-4789-8abe-d0bee0b5b634n@googlegroups.com> References: <9nt0J.15689$4X4.14148@fx27.iad> <6a7734b2-0e15-4789-8abe-d0bee0b5b634n@googlegroups.com> Message-ID: <34df815d-09a2-4923-afdb-27a847727e26n@googlegroups.com> But the end-result is still very weak: % Wall 33810 ms, gc 980 ms, 284108 lips This is below 1 million LIPS. The JavaScript version of Dogelog does currently around 2 million LIPS. And SWI-Prolog can do around 20 million LIPS. Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:29:48 UTC+2: > Thank you for the suggestion. The test harness > is invoked as follows. So it does already do time/1, > thats also how I did the comparison Standard Python > > and GraalVM Python, a file dogelog.py: > > import sys > # sys.path.append("\jekrun_bench\core\harness2\libpy") > sys.path.append("/mnt/c//jekrun_bench/core/harness2/libpy") > from index import init, consult > > init() > consult(":- ['suite2.p']. " > ":- time(suite). " > ":- nl. " > ":- time(suite). ") > > Here you see a GraalVM cold and warm run.The warm run is faster. > If you do a warm warm run, it even gets more faster, because of > JIT-ing, Just-in-Time machine compilation, > > via the GraalVM Truffles framework: > > $ export PATH=/graalvm-ce-java8-21.2.0/bin:$PATH > $ cd /mnt/c//jekrun_bench/core/harness2 > $ graalpython /mnt/c//jekrun_bench/core/harness2/dogelog.py > nrev % Wall 6175 ms, gc 212 ms, 154473 lips > crypt % Wall 9327 ms, gc 63 ms, 112838 lips > deriv % Wall 4101 ms, gc 90 ms, 321890 lips > poly % Wall 3594 ms, gc 415 ms, 216299 lips > sortq % Wall 3427 ms, gc 67 ms, 290070 lips > tictac % Wall 2770 ms, gc 51 ms, 136580 lips > queens % Wall 3287 ms, gc 64 ms, 325617 lips > query % Wall 1432 ms, gc 77 ms, 382969 lips > mtak % Wall 2532 ms, gc 95 ms, 533881 lips > perfect % Wall 3980 ms, gc 76 ms, 290382 lips > % Wall 40745 ms, gc 1212 ms, 235751 lips > > nrev % Wall 4508 ms, gc 112 ms, 211595 lips > crypt % Wall 6063 ms, gc 61 ms, 173584 lips > deriv % Wall 3150 ms, gc 42 ms, 419070 lips > poly % Wall 3549 ms, gc 432 ms, 219042 lips > sortq % Wall 3196 ms, gc 63 ms, 311036 lips > tictac % Wall 2670 ms, gc 52 ms, 141695 lips > queens % Wall 3087 ms, gc 60 ms, 346713 lips > query % Wall 1434 ms, gc 25 ms, 382435 lips > mtak % Wall 2596 ms, gc 90 ms, 520719 lips > perfect % Wall 3521 ms, gc 43 ms, 328236 lips > % Wall 33810 ms, gc 980 ms, 284108 lips > DFS schrieb am Mittwoch, 15. September 2021 um 23:15:07 UTC+2: > > On 9/15/2021 12:23 PM, Mostowski Collapse wrote: > > > I really wonder why my Python implementation > > > is a factor 40 slower than my JavaScript implementation. > > > Structurally its the same code. > > > > > > You can check yourself: > > > > > > Python Version: > > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py > > > > > > JavaScript Version: > > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js > > > > > > Its the same while, if-then-else, etc.. its the same > > > classes Variable, Compound etc.. Maybe I could speed > > > it up by some details. For example to create an array > > > of length n, I use in Python: > > > > > > temp = [NotImplemented] * code[pos] > > > pos += 1 > > > > > > Whereas in JavaScript I use, also > > > in exec_build2(): > > > > > > temp = new Array(code[pos++]); > > > > > > So I hear Guido doesn't like ++. So in Python I use += > > > and a separate statement as a workaround. But otherwise, > > > what about the creation of an array, > > > > > > is the the idiom [_] * _ slow? I am assuming its > > > compiled away. Or does it really first create an > > > array of size 1 and then enlarge it? > > I'm sure you know you can put in timing statements to find bottlenecks. > > > > import time > > startTime = time.perf_counter() > > [code block] > > print("%.2f" % (time.perf_counter() - startTime)) From nospam at dfs.com Wed Sep 15 17:14:44 2021 From: nospam at dfs.com (DFS) Date: Wed, 15 Sep 2021 17:14:44 -0400 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: <9nt0J.15689$4X4.14148@fx27.iad> On 9/15/2021 12:23 PM, Mostowski Collapse wrote: > I really wonder why my Python implementation > is a factor 40 slower than my JavaScript implementation. > Structurally its the same code. > > You can check yourself: > > Python Version: > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py > > JavaScript Version: > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js > > Its the same while, if-then-else, etc.. its the same > classes Variable, Compound etc.. Maybe I could speed > it up by some details. For example to create an array > of length n, I use in Python: > > ? temp = [NotImplemented] * code[pos] > ? pos += 1 > > Whereas in JavaScript I use, also > in exec_build2(): > > ? temp = new Array(code[pos++]); > > So I hear Guido doesn't like ++. So in Python I use += > and a separate statement as a workaround. But otherwise, > what about the creation of an array, > > is the the idiom [_] * _ slow? I am assuming its > compiled away. Or does it really first create an > array of size 1 and then enlarge it? I'm sure you know you can put in timing statements to find bottlenecks. import time startTime = time.perf_counter() [code block] print("%.2f" % (time.perf_counter() - startTime)) From nospam at dfs.com Wed Sep 15 17:26:17 2021 From: nospam at dfs.com (DFS) Date: Wed, 15 Sep 2021 17:26:17 -0400 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> Message-ID: <_xt0J.81744$rl3.50879@fx45.iad> On 9/15/2021 5:10 PM, Mostowski Collapse wrote: > And how do you only iterate over n-1 elements? > I don't need a loop over all elements. > > With array slicing? > > Someting like: > > for item in items[0:len(items)-2]: > ___print(item) > > Or with negative slicing indexes? Problem > is my length can be equal to one. > > And when I have length equal to one, the > slice might not do the right thing? > > LoL From the python command prompt: items = [1,2,3,4] for itm in items: print(itm) 1 2 3 4 for itm in items[:-2]: print(itm) 1 2 for itm in items[:-3]: print(itm) 1 for itm in items[:-4]: print(itm) (no result, no error thrown) for itm in items[:-5]: print(itm) (no result, no error thrown) From rosuav at gmail.com Wed Sep 15 18:02:21 2021 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 16 Sep 2021 08:02:21 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> Message-ID: On Thu, Sep 16, 2021 at 7:59 AM Mostowski Collapse wrote: > > BTW: I could already make it faster, by not repeatedly > accessing .arg anymore. It went down from ca.: > > 171'000 ms > > To this here: > > 140'000 ms > > But only in the cold run. In the warm run it went back > to 171'000 ms. Possibly when my code is faster, > it will create objects more faster, and kill the Python GC. > > Or it was because my Laptop went into screen black? > And throttled the CPU. Not sure. > Instead of worrying about all these details, start by simplifying your code. Focus on clean, simple, readable code, and don't microoptimize. Specifically, focus on the core arithmetic that you're trying to do, and get rid of all the bookkeeping overhead; most of that is a waste of time. I mentioned earlier the repeated boxing and unboxing in "Compound" objects - have you changed anything with those? ChrisA From Ak.82009 at hotmail.com Thu Sep 16 01:50:17 2021 From: Ak.82009 at hotmail.com (af kh) Date: Thu, 16 Sep 2021 05:50:17 +0000 Subject: Question again Message-ID: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> Hello, I was doing some coding on a website called replit then I extracted the file, and opened it in Python. For some reason, after answering 'no' or 'yes' after the last sentence I wrote, the Python window shut off, in replit I added one more sentence, but it isn't shown on Python, it just shuts off. Why is that? please reply to me soon since I need to submit it as an assignment for my class. Code on replit: #Title: Week 2: Chatbot with personality #Author: Afnan Khan #Date:9/15/21 #Description: Ask at least 3 questions to the user #and create a creative topic #Gives greetings to the user import random greetings = ["Hello, I'm Mr. ChatBot!", "Hi, I'm Mr. ChatBot!", "Hey~, I'm Mr. ChatBot!"] comment = random.choice(greetings) print(comment) #Learn about the user's Name: First question name = input("What is your name? ") #Greet the User print("Nice to meet you, " + name) #Ask the user about their day: Second question print("How is your day going? ") #The user replies reply = input() #If user says 'amazing', reply with 'I am glad!' if reply == "amazing" : print("I am glad!") #If user says 'Alright', reply with 'that's good' elif reply == "alright" : print("that's good") #If user says 'bad', reply with 'Do not worry things will get better' elif reply == "bad" : print("Do not worry things will get better") #Else than that type 'I see' else : print("I see!") #Ask to pick between numbers 1~10 to see if you will get lucky today: Third question number = input("Please pick between numbers 1~10 to see your luck for today: ") #From number 1~3 and an answer if number == "1" or number == "2" or number == "3" : print("You're in greeeeat luck today!") #From number 4~7 and an answer elif number == "4" or number == "5" or number == "6" : print("damn, bad luck is coming your way") #From number 8~10 and an answer elif number == "7" or number == "8" or number == "9" or number == "10" : print("I cannot sense any luck today, try again next time") #Add a statement and question: Fourth question print("That will be all for today's chitchat, woohooo! would you like to exit the chat?") #User says 'yes' reply = input() #If user says 'yes' reply 'wait hold on! are you really leaving??': Fifth question if reply == "yes" : print("Wait hold on! are you really leaving??") #User answers answer = input() #If user says 'yes' again, reply 'fine! bye then!' if answer == "yes" : print("Fine! bye then!") #Other than that if user says 'no', reply 'just kidding we're done here haha' elif answer == "no" : print("just kidding we're done here haha") Regards, Aya From bursejan at gmail.com Wed Sep 15 22:56:39 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Wed, 15 Sep 2021 19:56:39 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> Message-ID: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Compound is not used for boxing. Integers and floats are represented directly. Also integers are not mapped to floats. But maybe compound could be a little flattened, like using directly an array. But then you cannot assure anymore "clean, simple, readable code". For example now I have clean, simple and readable code, since I can access the functor of a compound via: obj.functor but when I flatten Compound into arrays, it would become: obj[0] Should I declare a constant FUNCTOR = 0? Some of your requirements have a trade-off, not all of them can be sustained simultaneously so easy. I am rather expecting languages like Python and JavaScript to offer the same comfort as C or Java, except that I don't need to write types for all the varianbles and fields all the time. But this requires smart language compiler and language runtime. V8 Chrome has interesting articles how they optimize access like .functor. Chris Angelico schrieb am Donnerstag, 16. September 2021 um 00:02:54 UTC+2: > On Thu, Sep 16, 2021 at 7:59 AM Mostowski Collapse wrote: > > > > BTW: I could already make it faster, by not repeatedly > > accessing .arg anymore. It went down from ca.: > > > > 171'000 ms > > > > To this here: > > > > 140'000 ms > > > > But only in the cold run. In the warm run it went back > > to 171'000 ms. Possibly when my code is faster, > > it will create objects more faster, and kill the Python GC. > > > > Or it was because my Laptop went into screen black? > > And throttled the CPU. Not sure. > > > Instead of worrying about all these details, start by simplifying your > code. Focus on clean, simple, readable code, and don't microoptimize. > Specifically, focus on the core arithmetic that you're trying to do, > and get rid of all the bookkeeping overhead; most of that is a waste > of time. I mentioned earlier the repeated boxing and unboxing in > "Compound" objects - have you changed anything with those? > > ChrisA From rosuav at gmail.com Thu Sep 16 13:29:12 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 17 Sep 2021 03:29:12 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Fri, Sep 17, 2021 at 3:20 AM Mostowski Collapse wrote: > > Compound is not used for boxing. Integers and floats > are represented directly. Also integers are not mapped to > floats. But maybe compound could be a little flattened, > "Boxing" in this case isn't about ints and floats, since Java-like bizarrenesses simply don't happen in Python; I'm talking about the way that you frequently build up a Compound object for various situations (even for throwing an error - you have a function that constructs a generic Exception, and then buries a Compound inside it), and then you're frequently checking if something is an instance of Compound. All these constant back-and-forths are extremely expensive, since they're not part of your algorithm at all. At very least, use tuples instead of Compounds, but it would be far better to ask less questions about your data and do more things by tidying up your algorithm. Unfortunately, I can't really advise with any detail, because you have code like this: ### # Mark a term. # # @param term The term. ## def mark_term(term): What does that even mean?! I get it, you have a term, and you're marking it. Whatever that mark means. The comments add absolutely nothing that the function header didn't tell me. Are you implementing your own garbage collection on top of Python's? Or something else? It's extremely hard to give any sort of recommendations when your code is hard to read, and nearly all of the comments are nothing more than restating what can be seen in the next line of code. Also, with the number of globals you're using, tracing the purpose of your functions is not easy. ChrisA From nospam at dfs.com Thu Sep 16 13:26:44 2021 From: nospam at dfs.com (DFS) Date: Thu, 16 Sep 2021 13:26:44 -0400 Subject: Question again In-Reply-To: References: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> Message-ID: On 9/16/2021 1:50 AM, af kh wrote: > Hello, > I was doing some coding on a website called replit then I extracted the file, and opened it in Python. For some reason, after answering 'no' or 'yes' after the last sentence I wrote, the Python window shut off, in replit I added one more sentence, but it isn't shown on Python, it just shuts off. Why is that? please reply to me soon since I need to submit it as an assignment for my class. > > Code on replit: > #Title: Week 2: Chatbot with personality > #Author: Afnan Khan > #Date:9/15/21 > #Description: Ask at least 3 questions to the user > #and create a creative topic > > #Gives greetings to the user > import random > > greetings = ["Hello, I'm Mr. ChatBot!", "Hi, I'm Mr. ChatBot!", "Hey~, I'm Mr. ChatBot!"] > comment = random.choice(greetings) > print(comment) > #Learn about the user's Name: First question > name = input("What is your name? ") > > #Greet the User > print("Nice to meet you, " + name) > > #Ask the user about their day: Second question > print("How is your day going? ") > > #The user replies > reply = input() > > #If user says 'amazing', reply with 'I am glad!' > if reply == "amazing" : > print("I am glad!") > > #If user says 'Alright', reply with 'that's good' > elif reply == "alright" : > print("that's good") > > #If user says 'bad', reply with 'Do not worry things will get better' > elif reply == "bad" : > print("Do not worry things will get better") > > #Else than that type 'I see' > else : > print("I see!") > > #Ask to pick between numbers 1~10 to see if you will get lucky today: Third question > number = input("Please pick between numbers 1~10 to see your luck for today: ") > > #From number 1~3 and an answer > if number == "1" or number == "2" or number == "3" : > print("You're in greeeeat luck today!") > > #From number 4~7 and an answer > elif number == "4" or number == "5" or number == "6" : > print("damn, bad luck is coming your way") > > #From number 8~10 and an answer > elif number == "7" or number == "8" or number == "9" or number == "10" : > print("I cannot sense any luck today, try again next time") > > #Add a statement and question: Fourth question > print("That will be all for today's chitchat, woohooo! would you like to exit the chat?") > > #User says 'yes' > reply = input() > > #If user says 'yes' reply 'wait hold on! are you really leaving??': Fifth question > if reply == "yes" : > print("Wait hold on! are you really leaving??") > > #User answers > answer = input() > > #If user says 'yes' again, reply 'fine! bye then!' > if answer == "yes" : > print("Fine! bye then!") > > #Other than that if user says 'no', reply 'just kidding we're done here haha' > elif answer == "no" : > print("just kidding we're done here haha") > > > Regards, > Aya I don't understand your issue, but this code runs fine for me. From janburse at fastmail.fm Thu Sep 16 15:56:49 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Thu, 16 Sep 2021 21:56:49 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: About Exceptions: Thats just building ISO core standard Prolog error terms. About Garbage Collection: Thats just Prolog garbage collection, which does shrink some single linked lists, which ordinary programmig language GC cannot do, or maybe some Weak Pointer magic can do it? The use case is very simple. A Prolog system has a so called trail. The trail in Dogelog Runtime is a single linked list: -->[ A ]-->[ B ]-->[ C ]--> Now if B becomes unused, you need to rewire the trail, it should then look like: -->[ A ]---------->[ C ]--> If a programming language has a means to communicate this to the Garbage Collector, I happy to apply it. The challenge is many fold, the pointer from A to B for example needs not to be accounted to determine whether B is reachable. So all the links in the trail are weak pointers. But they are weak pointers that need to be able to adapt. Chris Angelico wrote: > On Fri, Sep 17, 2021 at 3:20 AM Mostowski Collapse wrote: >> >> Compound is not used for boxing. Integers and floats >> are represented directly. Also integers are not mapped to >> floats. But maybe compound could be a little flattened, >> > > "Boxing" in this case isn't about ints and floats, since Java-like > bizarrenesses simply don't happen in Python; I'm talking about the way > that you frequently build up a Compound object for various situations > (even for throwing an error - you have a function that constructs a > generic Exception, and then buries a Compound inside it), and then > you're frequently checking if something is an instance of Compound. > All these constant back-and-forths are extremely expensive, since > they're not part of your algorithm at all. > > At very least, use tuples instead of Compounds, but it would be far > better to ask less questions about your data and do more things by > tidying up your algorithm. Unfortunately, I can't really advise with > any detail, because you have code like this: > > ### > # Mark a term. > # > # @param term The term. > ## > def mark_term(term): > > What does that even mean?! I get it, you have a term, and you're > marking it. Whatever that mark means. The comments add absolutely > nothing that the function header didn't tell me. Are you implementing > your own garbage collection on top of Python's? Or something else? > It's extremely hard to give any sort of recommendations when your code > is hard to read, and nearly all of the comments are nothing more than > restating what can be seen in the next line of code. Also, with the > number of globals you're using, tracing the purpose of your functions > is not easy. > > ChrisA > From janburse at fastmail.fm Thu Sep 16 15:58:48 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Thu, 16 Sep 2021 21:58:48 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: Here is a challenge for Python. Can Python solve Sudoku? Mostowski Collapse wrote: > I am not testing this use-case. But a related > use-case might highlight why speed did never > hurt anybody. > > Lets say you program a flying drone with Python, > and the measurement is from the drone sensor > and communication systems. > > Lets say you are using the idle time between > measurements for some complex planning. It > is then not true that you have anyway > > to wait for the measurement. > > Hope this helps! > > BTW: If somebody knows another Python implementation > I am happy to test this implementation as well. > I am assuming that the standard Python python.exe > > I tested amounts to CPython? Not sure. And the > GraalVM is practically the same as JPython? Not > sure either. > >> Opinion:?? Anyone who is counting on Python for truly fast compute >> speed is probably using Python for the wrong purpose. Here, we use >> Python to control Test Equipment, to set up the equipment and ask for >> a measurement, get it, and proceed to the next measurement; and at the >> end produce a nice formatted report. If we wrote the test script in C >> or Rust or whatever it could not run substantially faster because it >> is communicating with the test equipment, setting it up and waiting >> for responses, and that is where the vast majority of the time goes. >> Especially if the measurement result requires averaging it can take a >> while.? In my opinion this is an ideal use for Python, not just >> because the speed of Python is not important, but also because we can >> easily find people who know Python, who like coding in Python, and >> will join the company to program in Python ... and stay with us. >> --- Joseph S. > From bursejan at gmail.com Thu Sep 16 16:27:18 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Thu, 16 Sep 2021 13:27:18 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: <40aa358a-a74f-4983-823d-3d97068ce874n@googlegroups.com> A friend just sent me a Web Sudoku made with Dogelog Runtime https://gist.github.com/jburse/c85297e97091caf22d306dd8c8be12fe#gistcomment-3895696 LoL Mostowski Collapse schrieb am Donnerstag, 16. September 2021 um 21:59:05 UTC+2: > Here is a challenge for Python. > Can Python solve Sudoku? > > Mostowski Collapse wrote: > > I am not testing this use-case. But a related > > use-case might highlight why speed did never > > hurt anybody. > > > > Lets say you program a flying drone with Python, > > and the measurement is from the drone sensor > > and communication systems. > > > > Lets say you are using the idle time between > > measurements for some complex planning. It > > is then not true that you have anyway > > > > to wait for the measurement. > > > > Hope this helps! > > > > BTW: If somebody knows another Python implementation > > I am happy to test this implementation as well. > > I am assuming that the standard Python python.exe > > > > I tested amounts to CPython? Not sure. And the > > GraalVM is practically the same as JPython? Not > > sure either. > > > >> Opinion: Anyone who is counting on Python for truly fast compute > >> speed is probably using Python for the wrong purpose. Here, we use > >> Python to control Test Equipment, to set up the equipment and ask for > >> a measurement, get it, and proceed to the next measurement; and at the > >> end produce a nice formatted report. If we wrote the test script in C > >> or Rust or whatever it could not run substantially faster because it > >> is communicating with the test equipment, setting it up and waiting > >> for responses, and that is where the vast majority of the time goes. > >> Especially if the measurement result requires averaging it can take a > >> while. In my opinion this is an ideal use for Python, not just > >> because the speed of Python is not important, but also because we can > >> easily find people who know Python, who like coding in Python, and > >> will join the company to program in Python ... and stay with us. > >> --- Joseph S. > > From avigross at verizon.net Thu Sep 16 17:42:30 2021 From: avigross at verizon.net (Avi Gross) Date: Thu, 16 Sep 2021 17:42:30 -0400 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: <062001d7ab43$c0229790$4067c6b0$@verizon.net> Some questions make no sense to me. Can a kind of snake solve Sudoku? Do you mean a specific puzzle, or any puzzle or even a puzzle with no solution? Can a programming language do it? Well, in my experience, programming languages are tools to be used by humans, or sometimes by other programming languages. They are not sentient and cannot be asked to solve much of anything. So is the question whether someone can program using only Python to solve an arbitrary sudoku problem? Short answer is you can do that in just about ANY language. I mean by brute force, if you have a 9 by 9 matrix with some of the 81 locations already filled in, then you can try every darn combination of the other spots using digits 1 to 9 and then ignore any where the rows and columns and the 9 3x3 submatrices do not follow the rules. At least one solution is guaranteed to pop out if there is one. Sure, such methods may run out of memory or take a while, but many can use little memory and some can speed things up by not going down blind alleys such as placing a number in a position where there already is the same number on the same row or column or sub-matrix. So is the real question whether a human has already made a decent implementation in Python available? Sure, do a little searching and there are plenty of such things including some that use interesting features of python and some that are just translations from a more primitive language. -----Original Message----- From: Python-list On Behalf Of Mostowski Collapse Sent: Thursday, September 16, 2021 3:59 PM To: python-list at python.org Subject: Re: ANN: Dogelog Runtime, Prolog to the Moon (2021) Here is a challenge for Python. Can Python solve Sudoku? Mostowski Collapse wrote: > I am not testing this use-case. But a related use-case might highlight > why speed did never hurt anybody. > > Lets say you program a flying drone with Python, and the measurement > is from the drone sensor and communication systems. > > Lets say you are using the idle time between measurements for some > complex planning. It is then not true that you have anyway > > to wait for the measurement. > > Hope this helps! > > BTW: If somebody knows another Python implementation I am happy to > test this implementation as well. > I am assuming that the standard Python python.exe > > I tested amounts to CPython? Not sure. And the GraalVM is practically > the same as JPython? Not sure either. > >> Opinion: Anyone who is counting on Python for truly fast compute >> speed is probably using Python for the wrong purpose. Here, we use >> Python to control Test Equipment, to set up the equipment and ask for >> a measurement, get it, and proceed to the next measurement; and at >> the end produce a nice formatted report. If we wrote the test script >> in C or Rust or whatever it could not run substantially faster >> because it is communicating with the test equipment, setting it up >> and waiting for responses, and that is where the vast majority of the time goes. >> Especially if the measurement result requires averaging it can take a >> while. In my opinion this is an ideal use for Python, not just >> because the speed of Python is not important, but also because we can >> easily find people who know Python, who like coding in Python, and >> will join the company to program in Python ... and stay with us. >> --- Joseph S. > -- https://mail.python.org/mailman/listinfo/python-list From bursejan at gmail.com Thu Sep 16 17:46:47 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Thu, 16 Sep 2021 14:46:47 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <062001d7ab43$c0229790$4067c6b0$@verizon.net> Message-ID: The new release 0.9.6 is quite speedy: "Maailman vaikein" 850002400720000009004000000000107002305000900040000000000080070017000000000036040 time(solve(Puzzle)) % Wall 41354 ms, gc 520 ms, 3143029 lips in Browser See also: Preview: New para/1 instruction for Dogelog runtime. (Jekejeke) https://twitter.com/dogelogch/status/1438586282502983682 Preview: New para/1 instruction for Dogelog runtime. (Jekejeke) https://www.facebook.com/groups/dogelog Avi Gross schrieb am Donnerstag, 16. September 2021 um 23:43:10 UTC+2: > Some questions make no sense to me. > > Can a kind of snake solve Sudoku? Do you mean a specific puzzle, or any puzzle or even a puzzle with no solution? > > Can a programming language do it? Well, in my experience, programming languages are tools to be used by humans, or sometimes by other programming languages. They are not sentient and cannot be asked to solve much of anything. > > So is the question whether someone can program using only Python to solve an arbitrary sudoku problem? Short answer is you can do that in just about ANY language. I mean by brute force, if you have a 9 by 9 matrix with some of the 81 locations already filled in, then you can try every darn combination of the other spots using digits 1 to 9 and then ignore any where the rows and columns and the 9 3x3 submatrices do not follow the rules. At least one solution is guaranteed to pop out if there is one. Sure, such methods may run out of memory or take a while, but many can use little memory and some can speed things up by not going down blind alleys such as placing a number in a position where there already is the same number on the same row or column or sub-matrix. > > So is the real question whether a human has already made a decent implementation in Python available? Sure, do a little searching and there are plenty of such things including some that use interesting features of python and some that are just translations from a more primitive language. > -----Original Message----- > From: Python-list On Behalf Of Mostowski Collapse > Sent: Thursday, September 16, 2021 3:59 PM > To: pytho... at python.org > Subject: Re: ANN: Dogelog Runtime, Prolog to the Moon (2021) > > Here is a challenge for Python. > Can Python solve Sudoku? > > Mostowski Collapse wrote: > > I am not testing this use-case. But a related use-case might highlight > > why speed did never hurt anybody. > > > > Lets say you program a flying drone with Python, and the measurement > > is from the drone sensor and communication systems. > > > > Lets say you are using the idle time between measurements for some > > complex planning. It is then not true that you have anyway > > > > to wait for the measurement. > > > > Hope this helps! > > > > BTW: If somebody knows another Python implementation I am happy to > > test this implementation as well. > > I am assuming that the standard Python python.exe > > > > I tested amounts to CPython? Not sure. And the GraalVM is practically > > the same as JPython? Not sure either. > > > >> Opinion: Anyone who is counting on Python for truly fast compute > >> speed is probably using Python for the wrong purpose. Here, we use > >> Python to control Test Equipment, to set up the equipment and ask for > >> a measurement, get it, and proceed to the next measurement; and at > >> the end produce a nice formatted report. If we wrote the test script > >> in C or Rust or whatever it could not run substantially faster > >> because it is communicating with the test equipment, setting it up > >> and waiting for responses, and that is where the vast majority of the time goes. > >> Especially if the measurement result requires averaging it can take a > >> while. In my opinion this is an ideal use for Python, not just > >> because the speed of Python is not important, but also because we can > >> easily find people who know Python, who like coding in Python, and > >> will join the company to program in Python ... and stay with us. > >> --- Joseph S. > > > -- > https://mail.python.org/mailman/listinfo/python-list From alan.gauld at yahoo.co.uk Thu Sep 16 20:11:08 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 17 Sep 2021 01:11:08 +0100 Subject: Question again In-Reply-To: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> References: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> Message-ID: On 16/09/2021 06:50, af kh wrote: > Hello, > I was doing some coding on a website called replit I have no idea what that is but... > after answering 'no' or 'yes' after the last sentence I wrote, > the Python window shut off, That's what you told it to do in the code. Regardless of which answer the user gives the program reaches the end and stops. > in replit I added one more sentence, but it isn't shown on Python, I dn;t know what this means. > #Gives greetings to the user > import random ... > #Ask to pick between numbers 1~10 to see if you will get lucky today: Third question > number = input("Please pick between numbers 1~10 to see your luck for today: ") > > #From number 1~3 and an answer > if number == "1" or number == "2" or number == "3" : > print("You're in greeeeat luck today!") > > #From number 4~7 and an answer > elif number == "4" or number == "5" or number == "6" : > print("damn, bad luck is coming your way") The cde and comment are not consistent. > #From number 8~10 and an answer > elif number == "7" or number == "8" or number == "9" or number == "10" : > print("I cannot sense any luck today, try again next time") Same here. > #Add a statement and question: Fourth question > print("That will be all for today's chitchat, woohooo! would you like to exit the chat?") > #User says 'yes' > reply = input() > > #If user says 'yes' reply 'wait hold on! are you really leaving??': Fifth question > if reply == "yes" : > print("Wait hold on! are you really leaving??") > > #User answers > answer = input() > #If user says 'yes' again, reply 'fine! bye then!' > if answer == "yes" : > print("Fine! bye then!") Shouldn't those lines be indented as part of the if statement above? > #Other than that if user says 'no', reply 'just kidding we're done here haha' > elif answer == "no" : > print("just kidding we're done here haha") But the code always gets to the end, there is nothing to stop it exiting. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From avigross at verizon.net Thu Sep 16 20:25:44 2021 From: avigross at verizon.net (Avi Gross) Date: Thu, 16 Sep 2021 20:25:44 -0400 Subject: Question again In-Reply-To: References: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> Message-ID: <070401d7ab5a$8df92240$a9eb66c0$@verizon.net> Alan, I wonder if this is yet another case when a pop-up window closes rapidly when done and any last text written is just not perceived. Good design in such cases makes a final pause till the user acknowledges in some way that they are done and then no more messages! Avi -----Original Message----- From: Python-list On Behalf Of Alan Gauld via Python-list Sent: Thursday, September 16, 2021 8:11 PM To: python-list at python.org Subject: Re: Question again On 16/09/2021 06:50, af kh wrote: > Hello, > I was doing some coding on a website called replit I have no idea what that is but... > after answering 'no' or 'yes' after the last sentence I wrote, the > Python window shut off, That's what you told it to do in the code. Regardless of which answer the user gives the program reaches the end and stops. > in replit I added one more sentence, but it isn't shown on Python, I dn;t know what this means. > #Gives greetings to the user > import random ... > #Ask to pick between numbers 1~10 to see if you will get lucky today: > Third question number = input("Please pick between numbers 1~10 to see > your luck for today: ") > > #From number 1~3 and an answer > if number == "1" or number == "2" or number == "3" : > print("You're in greeeeat luck today!") > > #From number 4~7 and an answer > elif number == "4" or number == "5" or number == "6" : > print("damn, bad luck is coming your way") The cde and comment are not consistent. > #From number 8~10 and an answer > elif number == "7" or number == "8" or number == "9" or number == "10" : > print("I cannot sense any luck today, try again next time") Same here. > #Add a statement and question: Fourth question print("That will be all > for today's chitchat, woohooo! would you like to exit the chat?") > #User says 'yes' > reply = input() > > #If user says 'yes' reply 'wait hold on! are you really leaving??': > Fifth question if reply == "yes" : > print("Wait hold on! are you really leaving??") > > #User answers > answer = input() > #If user says 'yes' again, reply 'fine! bye then!' > if answer == "yes" : > print("Fine! bye then!") Shouldn't those lines be indented as part of the if statement above? > #Other than that if user says 'no', reply 'just kidding we're done here haha' > elif answer == "no" : > print("just kidding we're done here haha") But the code always gets to the end, there is nothing to stop it exiting. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list From PythonList at DancesWithMice.info Thu Sep 16 21:00:51 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 17 Sep 2021 13:00:51 +1200 Subject: The code version of python -i In-Reply-To: References: Message-ID: Abdur-Rahmaan, Apologies for delay: several last-minute tasks were landed on me, so I haven't been able to 'read the list' since last week. > If i have a file name flower.py and i add x = 1 in it. > When i run python -i flower.py i get a shell >>>> > > If type x i get 1 >>>> x > 1 > > The values are auto injected. > > How do i start a shell by code with values already injected? Thanks What do you want to achieve with this? At this week's local PUG meeting, I was 'the warm-up act' for (our own) @Chris Angelico giving a talk about 'the walrus operator', his part in that PEP, and how anyone-and-everyone can contribute to the Python eco-system. My contribution was to start at the Python-Apprentice level, talking about different types of 'operators', and preparing the stage?beach for the more advanced 'walrus'. It culminated in a live-coding demo of Conditional expressions/the "ternary operator". Live-coding demos are always better (for the audience) than passively watching a progression of slides! However, that territory is labelled 'here be dragons', for presenters! (continuing the tradition of slide projectors which don't accept my cartridge/carousel, white boards lacking pens with ink, over-head projectors with blown lamps (and a used spare), RGB projectors which only/don't accept VGA cables, ...) Further: we've all suffered YouTube wannabes telling us how easy it is to do 'something', typing and mis-typing and re-typing, and generally wasting our time... (have they not heard of video-editing?) Those of us who present 'live' (unlike my video-lectures), can't rely upon a later 'cosmetic' stage to apply lip-stick on any 'pigs' in our typing! Rather than using a "script" document (as-in 'speaker's notes') to prompt me with what to type next, the remote presentation tool (BigBlueButton web-conferencing) allows the 'projection' of a (desktop) terminal window - whilst hiding my text-editor. Thus, at the appropriate moment, I was copy-pasting from my 'script' in xed, into the terminal's Python REPL. Worked very well - it appears as if I'm a very fast typist! Of course, sometimes I was copying one line of code at a time, and at others, multiple lines - thus (*still*) giving opportunity to 'fluff my lines'. Hah! For live-demos, I like to use pedagogical (teaching) 'patterns' of a structured narrative (as one would with a lecture) but trying to encourage involvement (if not "active learning"), using a "worked example" (learning through demonstration), and "discovery" (which because it is effectively a 'presentation', really means 'gradual revelation'). (apologies for any non-obvious, non-Python, jargon - cognitive psychology (learning how we learn) is my research field) Thus, the demo proceeds on a step-by-step basis. At each step the process is: 1 present a problem/ask a question 2 ensure understanding/invite comments/accept suggestions 3 present code snippet* and (presumably) execute same to produce an 'answer' 4 comment/discuss/critique - which, unless 'the end', should lead us back to nr1 for the 'next step'... * in other scenarios, this might involve use of code suggested by the audience! Before reading this thread, I had been thinking that Jupyter might empower the combination of Python, 'steps', and 'live-coding'. However, a notebook will likely display future steps on-screen, before the current one has been completed (which defeats 'gradual revelation' - somewhat like jumping to 'the last page to find-out who dunnit') - unlike (say) presentation slides where no-one can see 'what comes next'/jump ahead, before the earlier 'lesson' has been learned. NB I'm not seeking 'information hiding' or encapsulation by its Java definition; but do want people to consider (only) one sub-problem at a time, and thus building-up to the whole solution! If one first understands the problem(s) and then contributes to generating a solution, you will learn far more effectively than if I simply lecture/read the Python manual-entry to you! Courtesy of this discussion, am wondering if it might be possible to structure what is currently a 'teaching notes' or "script" document (from which I was copy-pasting), and build a 'projector' program which will run a sub-interpreter* to run exactly one 'step' of the 'live-coding' demo at a time (whilst also retaining the option of REPL access, to be able prove or expose short-comings (nr 4, above), and without revealing the 'what comes next'? * rather than python -i; am considering Lib.runpy, and/or Lib.code How might any of this relate to your interest? -- Regards, =dn From rosuav at gmail.com Thu Sep 16 21:29:29 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 17 Sep 2021 11:29:29 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse wrote: > > About Exceptions: Thats just building ISO core > standard Prolog error terms. > > About Garbage Collection: Thats just Prolog > garbage collection, which does shrink some > single linked lists, which ordinary > programmig language GC cannot do, > Okay, so.... you're building your own garbage collection on top of Python's, and you're wondering why it's slow? Change your code to not try to implement one language inside another, and you'll see a massive performance improvement. ChrisA From greg.ewing at canterbury.ac.nz Thu Sep 16 21:24:01 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Sep 2021 13:24:01 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On 16/09/21 4:23 am, Mostowski Collapse wrote: > I really wonder why my Python implementation > is a factor 40 slower than my JavaScript implementation. There are Javascript implementations around nowadays that are blazingly fast. Partly that's because a lot of effort has been put into them, but it's also because Javascript is a different language. There are many dynamic aspects to Python that make fast implementations difficult. > I use in Python: > > ? temp = [NotImplemented] * code[pos] > ? pos += 1 > > is the the idiom [_] * _ slow? No, on the contrary it's probably the fastest way to do it in Python. You could improve it a bit by precomputing [NotImplemented]: # once at the module level NotImplementedList = [NotImplemented] # whenever you want a new list temp = NotImplementedList * code[pos] That's probably at least as fast as built-in function for creating lists would be. > does it really first create an > array of size 1 and then enlarge it? It does: >>> def f(code, pos): ... return [NotImplemented] * code[pos] ... >>> from dis import dis >>> dis(f) 2 0 LOAD_GLOBAL 0 (NotImplemented) 2 BUILD_LIST 1 4 LOAD_FAST 0 (code) 6 LOAD_FAST 1 (pos) 8 BINARY_SUBSCR 10 BINARY_MULTIPLY 12 RETURN_VALUE BTW, the Python terminology is "list", not "array". (There *is* something in the stdlib called an array, but it's rarely used or needed.) -- Greg From greg.ewing at canterbury.ac.nz Thu Sep 16 21:34:35 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Sep 2021 13:34:35 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On 16/09/21 2:56 pm, Mostowski Collapse wrote: > I can access the functor of a compound via: > > obj.functor > > but when I flatten Compound into arrays, it would become: > > obj[0] > > Should I declare a constant FUNCTOR = 0? You could, but keep in mind that access to a global in Python is somewhat expensive, since it requires a dictionary lookup. There's always a tradeoff between clarity and efficiency. In this case, I think obj[0] is clear enough -- the reader will be well aware that the first item of a term is the functor. Especially if you use a name that makes it clear what kind of object it is, rather than a generic name such as "obj". -- Greg From PythonList at DancesWithMice.info Thu Sep 16 22:25:56 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 17 Sep 2021 14:25:56 +1200 Subject: Question again In-Reply-To: <070401d7ab5a$8df92240$a9eb66c0$@verizon.net> References: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> <070401d7ab5a$8df92240$a9eb66c0$@verizon.net> Message-ID: On 17/09/2021 12.25, Avi Gross via Python-list wrote: > I wonder if this is yet another case when a pop-up window closes rapidly > when done and any last text written is just not perceived. > > Good design in such cases makes a final pause till the user acknowledges in > some way that they are done and then no more messages! > -----Original Message----- > From: Python-list On > Behalf Of Alan Gauld via Python-list > Sent: Thursday, September 16, 2021 8:11 PM > To: python-list at python.org > Subject: Re: Question again > > On 16/09/2021 06:50, af kh wrote: >> after answering 'no' or 'yes' after the last sentence I wrote, the Are you using MS-Windows? Are you executing the program directly/from the menu? (or do you first start a terminal window, and then run Python within that) -- Regards, =dn From greg.ewing at canterbury.ac.nz Thu Sep 16 21:44:18 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Sep 2021 13:44:18 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> Message-ID: On 16/09/21 6:56 am, Mostowski Collapse wrote: > What could be slow, repeatedly requesting the "args" > field. Maybe I should do: > > help = term.args > i = 0 > while i < len(help) - 1: > ____mark_term(help[i]) > ____i += 1 > term = help[i] Yes, that will certainly help. But you're still evaluating len(help) - 1 every time around the loop, so this is even better: help = term.args n = len(help) - 1 i = 0 while i < n: mark_term(help[i]) i += 1 term = help[i] Some general principles to be aware of: * Almost nothing is free -- Python very literally does what you tell it to do, even if it looks dumb. * Access to attributes and global variables is expensive (requires at least one dict lookup, often more in the case of attributes). * Access to *local* variables, on the other hand, is very cheap (essentially an array lookup). * Function calls are expensive -- both to look up the name, if it's global, which it usually is, and the machinery of the call itself. * Creating objects is expensive. Creating instances of user-defined objects is more expensive than built-in ones. -- Greg From janburse at fastmail.fm Fri Sep 17 04:41:27 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Fri, 17 Sep 2021 10:41:27 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: Thanks for your response, will have a look. Ok, dis() is all that is need to disassemble. Very cool! A long term goal could be indeed to have a Prolog interpreter produce 20MLips, like SWI-Prolog, but tightly integrated into Python. So that it directly makes use of the Python objects and the Python garbage collection like Dogelog Runtime. Although Dogelog Runtime has its own garbage collection, its only used to help the native Python garbage collection. The result is that you can enjoy bi-directly calling Python. For example the Prolog adding of two numbers is realized as: ### # +(A, B, C): [ISO 9.1.7] # The predicate succeeds in C with the sum of A and B. ## def eval_add(alpha, beta): check_number(alpha) check_number(beta) try: return alpha + beta except OverflowError: raise make_error(Compound("evaluation_error", ["float_overflow"])) And then register it: add("+", 3, make_dispatch(eval_add, MASK_MACH_FUNC)) Could also map the exception to a Prolog term later. Thats not so much an issue for speed. The sunshine case is straight forward. But I might try dis() on eval_add(). Are exceptions blocks in Python cheap or expensive? Are they like in Java, some code annotation, or like in Go programming language pushing some panic handler? Greg Ewing schrieb: > On 16/09/21 4:23 am, Mostowski Collapse wrote: >> I really wonder why my Python implementation >> is a factor 40 slower than my JavaScript implementation. > > There are Javascript implementations around nowadays that are > blazingly fast. Partly that's because a lot of effort has been > put into them, but it's also because Javascript is a different > language. There are many dynamic aspects to Python that make > fast implementations difficult. > >> I use in Python: >> >> ?? temp = [NotImplemented] * code[pos] >> ?? pos += 1 >> >> is the the idiom [_] * _ slow? > > No, on the contrary it's probably the fastest way to do it > in Python. You could improve it a bit by precomputing > [NotImplemented]: > > # once at the module level > NotImplementedList = [NotImplemented] > > # whenever you want a new list > temp = NotImplementedList * code[pos] > > That's probably at least as fast as built-in function for > creating lists would be. > >> does it really first create an >> array of size 1 and then enlarge it? > > It does: > > >>> def f(code, pos): > ...? return [NotImplemented] * code[pos] > ... > >>> from dis import dis > >>> dis(f) > ? 2?????????? 0 LOAD_GLOBAL????????????? 0 (NotImplemented) > ????????????? 2 BUILD_LIST?????????????? 1 > ????????????? 4 LOAD_FAST??????????????? 0 (code) > ????????????? 6 LOAD_FAST??????????????? 1 (pos) > ????????????? 8 BINARY_SUBSCR > ???????????? 10 BINARY_MULTIPLY > ???????????? 12 RETURN_VALUE > > BTW, the Python terminology is "list", not "array". > (There *is* something in the stdlib called an array, but > it's rarely used or needed.) > From janburse at fastmail.fm Fri Sep 17 04:51:29 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Fri, 17 Sep 2021 10:51:29 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: No its cooperative. Usually objects do get garbage collected by the native garbage collector of the host language in Dogelog runtime. The Prolog garbage collection is only to help the host language garbage collector when you have a deep recursion in Prolog. You can then reclaim intermediate variables. A simple example to test the slightly idio- syncratic Prolog garbage collection is: fibo(0, 1) :- !. fibo(1, 1) :- !. fibo(N, X) :- M is N-1, fibo(M, Y), L is M-1, fibo(L, Z), X is Y+Z. When running fibo(30,X) SWI-Prolog does around 800 garbage collections to keep the environment small. But SWI-Prolog allocates all its objects only very seldom on the heap. It uses its own stack. On the other hand Dogelog runtime creates everything on the heap. And completely relies on the host language garbage collection. It only helps the host language garbage collection it that it performs from time to time this movement: Before: -->[ A ]-->[ B ]-->[ C ]--> After: -->[ A ]---------->[ C ]--> A,B,C are objects of type Variable. The above movement only happens for objects of type Variables from time to time. For objects of type Compound no modifications are done during Prolog garbage collection. The Prolog garbage collection aggressively nulls the Variable object B, and the host language later will garbage collect what the Variable object B was pointing to. But the Variable object B might nevertheless have point to something shared with some other Variable object or a local or a global Python variable, or a Compound. This is then all courtesy of the host language to decide reachability. Chris Angelico schrieb: > On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse wrote: >> >> About Exceptions: Thats just building ISO core >> standard Prolog error terms. >> >> About Garbage Collection: Thats just Prolog >> garbage collection, which does shrink some >> single linked lists, which ordinary >> programmig language GC cannot do, >> > > Okay, so.... you're building your own garbage collection on top of > Python's, and you're wondering why it's slow? > > Change your code to not try to implement one language inside another, > and you'll see a massive performance improvement. > > ChrisA > From janburse at fastmail.fm Fri Sep 17 04:58:38 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Fri, 17 Sep 2021 10:58:38 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: The Prolog garbage collection that does the movement on the variable trail is only a very small fraction of the runtime. The garbage collection time is measured. Some measurements with version 0.9.5 took the following values: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Standard Python Version, Warm Run % ?- time(fibo(23,X)). % % Wall 3865 ms, gc 94 ms, 71991 lips % X = 46368. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % GraalVM Python Version, Warm Warm Run % ?- time(fibo(23,X)). % % Wall 695 ms, gc 14 ms, 400356 lips % X = 46368. The "gc" timing measures Prolog garbage collection. So you get the following percentage of time spent in Prolog garbage collection: Standard Python: 94 / 3865 = 2.4% GraalVM Python: 14 / 695 = 2.0% I consider this a good result. The Prolog garbage collection is not utterly expensive. The detecting the movement and performing the variable movement on the trail, doesn't take so much time. Currently the garbage collection in Dogelog runtime is configured towards synchronization with 60FPS, it does around 60-120 garbage collections per second. This is less than what SWI-Prolog does. SWI-Prolog has a much higher GC rate. But I did not yet measure new version 0.9.6. Mostowski Collapse schrieb: > No its cooperative. Usually objects do get > garbage collected by the native garbage collector > of the host language in Dogelog runtime. > > The Prolog garbage collection is only to help > the host language garbage collector when you have > a deep recursion in Prolog. > > You can then reclaim intermediate variables. > A simple example to test the slightly idio- > syncratic Prolog garbage collection is: > > fibo(0, 1) :- !. > fibo(1, 1) :- !. > fibo(N, X) :- > ?? M is N-1, fibo(M, Y), > ?? L is M-1, fibo(L, Z), > ?? X is Y+Z. > > When running fibo(30,X) SWI-Prolog does around > 800 garbage collections to keep the environment > small. But SWI-Prolog allocates all its objects > > only very seldom on the heap. It uses its own > stack. On the other hand Dogelog runtime creates > everything on the heap. And completely relies on > > the host language garbage collection. It only > helps the host language garbage collection it > that it performs from time to time this movement: > > Before: > > ??? -->[ A ]-->[ B ]-->[ C ]--> > > After: > > ??? -->[ A ]---------->[ C ]--> > > A,B,C are objects of type Variable. The above > movement only happens for objects of type Variables > from time to time. For objects of type Compound > > no modifications are done during Prolog garbage > collection. The Prolog garbage collection aggressively > nulls the Variable object B, and the host language > > later will garbage collect what the Variable object B > was pointing to. But the Variable object B might > nevertheless have point to something shared with > > some other Variable object or a local or a global > Python variable, or a Compound. This is then all > courtesy of the host language to decide reachability. > > Chris Angelico schrieb: >> On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse >> wrote: >>> >>> About Exceptions: Thats just building ISO core >>> standard Prolog error terms. >>> >>> About Garbage Collection: Thats just Prolog >>> garbage collection, which does shrink some >>> single linked lists, which ordinary >>> programmig language GC cannot do, >>> >> >> Okay, so.... you're building your own garbage collection on top of >> Python's, and you're wondering why it's slow? >> >> Change your code to not try to implement one language inside another, >> and you'll see a massive performance improvement. >> >> ChrisA >> > From bursejan at gmail.com Fri Sep 17 07:05:26 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Fri, 17 Sep 2021 04:05:26 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: <4b6a3ba7-3c91-4a7d-89e1-231c87cc573en@googlegroups.com> Concerning garbage collection, did a long term measurement for the first time. I measured LIPS for fibonacci numbers, i.e. time(fibo(23,X)). Doing the same thing 16 times, how long does it take? Here is a depiction how the LIPS relatively differ in each run: https://gist.github.com/jburse/c85297e97091caf22d306dd8c8be12fe#gistcomment-3896343 I can also calculate the mean and standard deviation. >From this we see that Python has a 5% deviation, whereas GraalVM has a 1% deviation. So the GraalVM garbage collector works more evenly? Disclaimer, I measured different time spans, the GraalVM is now 7x times faster than Standard Python, so this is inconclusive. Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2: > The Prolog garbage collection that does > the movement on the variable trail is only > a very small fraction of the runtime. > > The garbage collection time is measured. > Some measurements with version 0.9.5 > took the following values: > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % Standard Python Version, Warm Run > % ?- time(fibo(23,X)). > % % Wall 3865 ms, gc 94 ms, 71991 lips > % X = 46368. > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % GraalVM Python Version, Warm Warm Run > % ?- time(fibo(23,X)). > % % Wall 695 ms, gc 14 ms, 400356 lips > % X = 46368. > > The "gc" timing measures Prolog garbage > collection. So you get the following percentage > of time spent in Prolog garbage collection: > > Standard Python: 94 / 3865 = 2.4% > > GraalVM Python: 14 / 695 = 2.0% > > I consider this a good result. The Prolog > garbage collection is not utterly expensive. > The detecting the movement and performing > > the variable movement on the trail, doesn't > take so much time. Currently the garbage collection > in Dogelog runtime is configured towards > > synchronization with 60FPS, it does around > 60-120 garbage collections per second. This > is less than what SWI-Prolog does. > > SWI-Prolog has a much higher GC rate. > > But I did not yet measure new version 0.9.6. > > Mostowski Collapse schrieb: > > No its cooperative. Usually objects do get > > garbage collected by the native garbage collector > > of the host language in Dogelog runtime. > > > > The Prolog garbage collection is only to help > > the host language garbage collector when you have > > a deep recursion in Prolog. > > > > You can then reclaim intermediate variables. > > A simple example to test the slightly idio- > > syncratic Prolog garbage collection is: > > > > fibo(0, 1) :- !. > > fibo(1, 1) :- !. > > fibo(N, X) :- > > M is N-1, fibo(M, Y), > > L is M-1, fibo(L, Z), > > X is Y+Z. > > > > When running fibo(30,X) SWI-Prolog does around > > 800 garbage collections to keep the environment > > small. But SWI-Prolog allocates all its objects > > > > only very seldom on the heap. It uses its own > > stack. On the other hand Dogelog runtime creates > > everything on the heap. And completely relies on > > > > the host language garbage collection. It only > > helps the host language garbage collection it > > that it performs from time to time this movement: > > > > Before: > > > > -->[ A ]-->[ B ]-->[ C ]--> > > > > After: > > > > -->[ A ]---------->[ C ]--> > > > > A,B,C are objects of type Variable. The above > > movement only happens for objects of type Variables > > from time to time. For objects of type Compound > > > > no modifications are done during Prolog garbage > > collection. The Prolog garbage collection aggressively > > nulls the Variable object B, and the host language > > > > later will garbage collect what the Variable object B > > was pointing to. But the Variable object B might > > nevertheless have point to something shared with > > > > some other Variable object or a local or a global > > Python variable, or a Compound. This is then all > > courtesy of the host language to decide reachability. > > > > Chris Angelico schrieb: > >> On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse > >> wrote: > >>> > >>> About Exceptions: Thats just building ISO core > >>> standard Prolog error terms. > >>> > >>> About Garbage Collection: Thats just Prolog > >>> garbage collection, which does shrink some > >>> single linked lists, which ordinary > >>> programmig language GC cannot do, > >>> > >> > >> Okay, so.... you're building your own garbage collection on top of > >> Python's, and you're wondering why it's slow? > >> > >> Change your code to not try to implement one language inside another, > >> and you'll see a massive performance improvement. > >> > >> ChrisA > >> > > From Marco.Sulla.Python at gmail.com Fri Sep 17 16:03:42 2021 From: Marco.Sulla.Python at gmail.com (Marco Sulla) Date: Fri, 17 Sep 2021 22:03:42 +0200 Subject: How to support annotations for a custom type in a C extension? Message-ID: I created a custom dict in a C extension. Name it `promethea`. How can I implement `promethea[str, str]`? Now I get: TypeError: 'type' object is not subscriptable From python at mrabarnett.plus.com Fri Sep 17 20:54:59 2021 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 18 Sep 2021 01:54:59 +0100 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: References: Message-ID: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> On 2021-09-17 21:03, Marco Sulla wrote: > I created a custom dict in a C extension. Name it `promethea`. How can > I implement `promethea[str, str]`? Now I get: > > TypeError: 'type' object is not subscriptable > Somewhere you'll have a table of the class's methods. It needs an entry like this: static PyMethodDef customdict_methods[] = { ... {"__class_getitem__", (PyCFunction)Py_GenericAlias, METH_CLASS | METH_O | METH_COEXIST, PyDoc_STR("See PEP 585")}, ... }; Note the flags: METH_CLASS says that it's a class method and METH_COEXIST says that it should use this method instead of the slot. From Marco.Sulla.Python at gmail.com Sat Sep 18 02:40:08 2021 From: Marco.Sulla.Python at gmail.com (Marco Sulla) Date: Sat, 18 Sep 2021 08:40:08 +0200 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> References: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> Message-ID: Ooook..... I have a question. Why is this code not present in dictobject.c? Where are the dict annotations implemented? On Sat, 18 Sept 2021 at 03:00, MRAB wrote: > > On 2021-09-17 21:03, Marco Sulla wrote: > > I created a custom dict in a C extension. Name it `promethea`. How can > > I implement `promethea[str, str]`? Now I get: > > > > TypeError: 'type' object is not subscriptable > > > Somewhere you'll have a table of the class's methods. It needs an entry > like this: > > > static PyMethodDef customdict_methods[] = { > ... > {"__class_getitem__", (PyCFunction)Py_GenericAlias, METH_CLASS | > METH_O | METH_COEXIST, PyDoc_STR("See PEP 585")}, > ... > }; > > > Note the flags: METH_CLASS says that it's a class method and > METH_COEXIST says that it should use this method instead of the slot. > -- > https://mail.python.org/mailman/listinfo/python-list From boblatest at yahoo.com Fri Sep 17 16:15:42 2021 From: boblatest at yahoo.com (Robert Latest) Date: 17 Sep 2021 20:15:42 GMT Subject: How to "cast" an object to a derived class? Message-ID: Hi all, let's assume I'm using a module that defines some class "Opaque" and also a function that creates objects of that type. In my program I subclass that type because I want some extra functionality. But how can I "promote" a given Opaque instance to the derived class? Of course I could just create an instance of the derived type and copy all attributes from the original object to the new one, but that either breaks when the opaque.py module changes, or it requires introspection. It's easily done of course but seems overly clumsy. Is there a more Pythonic way? # this is the module opaque.py class Opaque(): def __init__(self, x): assert isinstance(x, int) self.n = x def make_opaque(): return Opaque(0) # this is my program import opaque class MyClass(opaque.Opaque): # generic __init__ b/c I don't want to have to know anything about that # class def __init__(self, *a, **k): super().__init__(*a, *k) self.__something = 0 def get_something(self): return self.something op = opaque.make_opaque() # But I want to have this as type MyClass. This obviously doesn't work: my_object = MyClass(op) # But what does? From joel.dcosta83 at gmail.com Fri Sep 17 16:36:41 2021 From: joel.dcosta83 at gmail.com (Python 4 Fun) Date: Fri, 17 Sep 2021 13:36:41 -0700 (PDT) Subject: =?UTF-8?B?wqHCocKhdW/JpcqHyo7qk5IgdeG0iSDKh3jHncqHIHXKjW/qk7cgx51w4bSJc2Tqk7Ug?= =?UTF-8?B?x53Kh8mQx53JueKGgyBvyocgyo1vzpc=?= Message-ID: <7fe8e50e-7c8a-433e-a2dd-ecc2909eeecen@googlegroups.com> How to Create Upside Down text in Python!!! This is a very simple yet very interesting program in python. We are going to make our text, word or string UPSIDE DOWN which will be still readable on phone or desktop if you stand on your head lol. But any ways... lets do a little research first. I am going to collect upside down texts using google as it cannot be written by yourself and if you do it would be time consuming. To make this quick here are some text that I have found from following sources: Read more... ? https://pysnakeblog.blogspot.com/2021/09/create-upside-down-text-for-instagram.html From michael.stemper at gmail.com Fri Sep 17 17:38:38 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Fri, 17 Sep 2021 16:38:38 -0500 Subject: Question again In-Reply-To: References: <2c7126a0bc3d4085830a1b434264c2dbPA4P190MB116625A98478F0A4B5670267D8DC9@PA4P190MB1166.EURP190.PROD.OUTLOOK.COM> Message-ID: On 16/09/2021 19.11, Alan Gauld wrote: > On 16/09/2021 06:50, af kh wrote: >> Hello, >> I was doing some coding on a website called replit > > I have no idea what that is but... > >> after answering 'no' or 'yes' after the last sentence I wrote, >> the Python window shut off, > > That's what you told it to do in the code. > Regardless of which answer the user gives the program > reaches the end and stops. > >> in replit I added one more sentence, but it isn't shown on Python, > > I dn;t know what this means. > >> #Gives greetings to the user >> import random > ... >> #Ask to pick between numbers 1~10 to see if you will get lucky today: Third question >> number = input("Please pick between numbers 1~10 to see your luck for today: ") >> >> #From number 1~3 and an answer >> if number == "1" or number == "2" or number == "3" : >> print("You're in greeeeat luck today!") >> >> #From number 4~7 and an answer >> elif number == "4" or number == "5" or number == "6" : >> print("damn, bad luck is coming your way") > > The cde and comment are not consistent. > >> #From number 8~10 and an answer >> elif number == "7" or number == "8" or number == "9" or number == "10" : >> print("I cannot sense any luck today, try again next time") > > Same here. This is, of course, why comments shouldn't just translate the code into English. They'll get out of synch. Instead, comments should say *why* the code is doing what it does. -- Michael F. Stemper Exodus 22:21 From greg.ewing at canterbury.ac.nz Fri Sep 17 23:49:16 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 18 Sep 2021 15:49:16 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On 17/09/21 7:56 am, Mostowski Collapse wrote: > The trail in Dogelog > Runtime is a single linked list: > > ??? -->[ A ]-->[ B ]-->[ C ]--> > > Now if B becomes unused, you need to rewire > the trail, it should then look like: > > ??? -->[ A ]---------->[ C ]--> Python has a way of creating weak references (see the weakref module). You could set the trail up like this: -->[*]-->[*]-->[*]--> | | | v v v [w] [w] [w] : : : v v v [A] [B] [C] where [w] is a weak reference object. Then you could periodically scan the trail looking for dead weakref objects and remove the corresponding [*] node from the list. You can also attach callbacks to weakref objects that are triggered when the referenced object dies. You might be able to make use of that to remove items from the trail instead of the periodic scanning. -- Greg From greg.ewing at canterbury.ac.nz Fri Sep 17 23:53:25 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 18 Sep 2021 15:53:25 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On 16/09/21 6:13 am, Mostowski Collapse wrote: > So in Python I now do the following: > > peek = kb.get(functor, NotImplemented) > if peek is not NotImplemented: If you're able to use None instead of NotImplemented, you could simplify that to peek = kb.get(functor) if peek: ... > But if get() in Python is implemented under the hood with > exception handling. ... then Python get() will probably be quite slow. No, it doesn't use exceptions as far as I know. It should be fairly fast. -- Greg From greg.ewing at canterbury.ac.nz Sat Sep 18 00:29:03 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 18 Sep 2021 16:29:03 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On 17/09/21 8:41 pm, Mostowski Collapse wrote: > Are exceptions > blocks in Python cheap or expensive? Are they like > in Java, some code annotation, or like in Go > programming language pushing some panic handler? They're not free, but they're not hugely expensive either. Don't be afraid to use them when it makes sense to do so. I'm not sure what you mean by "some code annotations", so I don't know how to answer that question. I suspect that the way exceptions are implemented in the JVM is similar to the way CPython does it, but I don't know a lot about the JVM. -- Greg From boblatest at yahoo.com Sat Sep 18 09:03:54 2021 From: boblatest at yahoo.com (Robert Latest) Date: 18 Sep 2021 13:03:54 GMT Subject: How to "cast" an object to a derived class? References: Message-ID: Stefan Ram wrote: > Robert Latest writes: But how can I "promote" a >>given Opaque instance to the derived class? > > Sometimes, one can use containment instead of inheritance. Nah, doesn't work in my case. I'm trying to write a wrapper around xml.etree.ElemenTree and .Element to circumvent its idiotic namespace handling. For that I need inheritance since I want to override the find..() functions to return my derived MyElement classes. I think it could work but I somehow need to convert the root Element to a MyElement. From storchaka at gmail.com Sat Sep 18 11:09:26 2021 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 18 Sep 2021 18:09:26 +0300 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> References: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> Message-ID: 18.09.21 03:54, MRAB ????: > static PyMethodDef customdict_methods[] = { > ... > ??? {"__class_getitem__", (PyCFunction)Py_GenericAlias, METH_CLASS | > METH_O | METH_COEXIST, PyDoc_STR("See PEP 585")}, > ... > }; > > > Note the flags: METH_CLASS says that it's a class method and > METH_COEXIST says that it should use this method instead of the slot. "(PyCFunction)" is redundant, Py_GenericAlias already has the right type. Overuse of casting to PyCFunction can hide actual bugs. METH_COEXIST is not needed. There is no slot for __class_getitem__, and even if it was, there would be no reasons to prohibit using it. From storchaka at gmail.com Sat Sep 18 11:10:30 2021 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 18 Sep 2021 18:10:30 +0300 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: References: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> Message-ID: 18.09.21 09:40, Marco Sulla ????: > Ooook..... I have a question. Why is this code not present in > dictobject.c? Where are the dict annotations implemented? In dictobject.c. From bursejan at gmail.com Sat Sep 18 17:28:54 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Sat, 18 Sep 2021 14:28:54 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: Yeah, it seems weak references could indeed spare me mark_term(). But then I am stil left with sweep_trail(). I did not yet measure what takes more time mark_term() or sweep_trail(). The displayed "gc" is the sum of both. >From what I have seen, very large trail practically reduced to a zero trail during Prolog GC, I am assuming that mark_term() is not the working horse. Usually mark_term() only marks what is not-Garbage, and sweep_trail() has to deal with Garbage and not-Garbage. And there is usually a lot of Garbage, much more than not-Garbage. Finding the objects that survive, is like finding the needle in the haystack, except we do not have to scan the haystack, the needles are on the goal list. But afterwards, the second pass, scanning the trail is the work of Heracles cleaning the Augeas stables. This scan is trowing away the hay and keeping the needles. Greg Ewing schrieb am Samstag, 18. September 2021 um 05:49:37 UTC+2: > On 17/09/21 7:56 am, Mostowski Collapse wrote: > > The trail in Dogelog > > Runtime is a single linked list: > > > > -->[ A ]-->[ B ]-->[ C ]--> > > > > Now if B becomes unused, you need to rewire > > the trail, it should then look like: > > > > -->[ A ]---------->[ C ]--> > Python has a way of creating weak references (see the weakref > module). You could set the trail up like this: > > -->[*]-->[*]-->[*]--> > | | | > v v v > [w] [w] [w] > : : : > v v v > [A] [B] [C] > > where [w] is a weak reference object. Then you could periodically > scan the trail looking for dead weakref objects and remove the > corresponding [*] node from the list. > > You can also attach callbacks to weakref objects that are triggered > when the referenced object dies. You might be able to make use of > that to remove items from the trail instead of the periodic scanning. > > -- > Greg From rosuav at gmail.com Sat Sep 18 21:57:29 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 19 Sep 2021 11:57:29 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <36358b16-6985-4e39-8b1e-6bdb62c8725an@googlegroups.com> <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Sun, Sep 19, 2021 at 11:46 AM Mostowski Collapse wrote: > > Yeah, it seems weak references could indeed spare > me mark_term(). But then I am stil left with sweep_trail(). > I did not yet measure what takes more time mark_term() > or sweep_trail(). The displayed "gc" is the sum of both. > > From what I have seen, very large trail practically reduced > to a zero trail during Prolog GC, I am assuming that > mark_term() is not the working horse. Usually mark_term() > only marks what is not-Garbage, and sweep_trail() > > has to deal with Garbage and not-Garbage. And there > is usually a lot of Garbage, much more than not-Garbage. > Finding the objects that survive, is like finding the needle > in the haystack, except we do not have to scan the If you stop referring to something, it is garbage. Python will dispose of it. You literally need to do nothing at all, and let the language take care of things. ChrisA From python at mrabarnett.plus.com Sat Sep 18 22:59:31 2021 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 19 Sep 2021 03:59:31 +0100 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: References: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> Message-ID: <0638d578-f322-0b59-3af5-3d0927c1eeb2@mrabarnett.plus.com> On 2021-09-18 16:09, Serhiy Storchaka wrote: > 18.09.21 03:54, MRAB ????: >> static PyMethodDef customdict_methods[] = { >> ... >> ??? {"__class_getitem__", (PyCFunction)Py_GenericAlias, METH_CLASS | >> METH_O | METH_COEXIST, PyDoc_STR("See PEP 585")}, >> ... >> }; >> >> >> Note the flags: METH_CLASS says that it's a class method and >> METH_COEXIST says that it should use this method instead of the slot. > > "(PyCFunction)" is redundant, Py_GenericAlias already has the right > type. Overuse of casting to PyCFunction can hide actual bugs. > I borrowed that from listobject.c, which does have the cast. > METH_COEXIST is not needed. There is no slot for __class_getitem__, and > even if it was, there would be no reasons to prohibit using it. > From python at mrabarnett.plus.com Sat Sep 18 23:01:50 2021 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 19 Sep 2021 04:01:50 +0100 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: References: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> Message-ID: <09349370-5c8a-6cc5-51e9-06ac316b90fe@mrabarnett.plus.com> On 2021-09-18 16:10, Serhiy Storchaka wrote: > 18.09.21 09:40, Marco Sulla ????: >> Ooook..... I have a question. Why is this code not present in >> dictobject.c? Where are the dict annotations implemented? > > In dictobject.c. > I just had a look at dictobject.c. It too has a cast to PyCFunction. From dieter at handshake.de Sun Sep 19 12:22:55 2021 From: dieter at handshake.de (Dieter Maurer) Date: Sun, 19 Sep 2021 18:22:55 +0200 Subject: How to "cast" an object to a derived class? In-Reply-To: References: Message-ID: <24903.25567.608277.813590@ixdm.fritz.box> Robert Latest wrote at 2021-9-18 13:03 GMT: >Stefan Ram wrote: >> Robert Latest writes: But how can I "promote" a >>>given Opaque instance to the derived class? >> >> Sometimes, one can use containment instead of inheritance. > >Nah, doesn't work in my case. I'm trying to write a wrapper around >xml.etree.ElemenTree and .Element to circumvent its idiotic namespace >handling. For that I need inheritance since I want to override the find..() >functions to return my derived MyElement classes. I think it could work but I >somehow need to convert the root Element to a MyElement. If the class is a pure Python class, then the `__class__` attribute of its instances is writable. You could try to assign the derived class -- at your own risk. -- Dieter From janburse at fastmail.fm Sun Sep 19 04:50:40 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Sun, 19 Sep 2021 10:50:40 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: I am refering to: Greg Ewing schrieb: > where [w] is a weak reference object. Then you could periodically > scan the trail looking for dead weakref objects and remove the > corresponding [*] node from the list. > > You can also attach callbacks to weakref objects that are triggered > when the referenced object dies. You might be able to make use of > that to remove items from the trail instead of the periodic scanning. Question to Chris Angelico: If I stay with my sweep_trail(), which is the periodically scanning, I can use a single linked list. On the other hand if I would use the trigger from Python, I possibly would need a double linked list, to remove an element. Chris Angelico, is there a third option, that I have overlooked? Single linked list uses less space than double linked list, this why I go with scan. Chris Angelico schrieb: > On Sun, Sep 19, 2021 at 11:46 AM Mostowski Collapse wrote: >> >> Yeah, it seems weak references could indeed spare >> me mark_term(). But then I am stil left with sweep_trail(). >> I did not yet measure what takes more time mark_term() >> or sweep_trail(). The displayed "gc" is the sum of both. >> >> From what I have seen, very large trail practically reduced >> to a zero trail during Prolog GC, I am assuming that >> mark_term() is not the working horse. Usually mark_term() >> only marks what is not-Garbage, and sweep_trail() >> >> has to deal with Garbage and not-Garbage. And there >> is usually a lot of Garbage, much more than not-Garbage. >> Finding the objects that survive, is like finding the needle >> in the haystack, except we do not have to scan the > > If you stop referring to something, it is garbage. Python will dispose of it. > > You literally need to do nothing at all, and let the language take > care of things. > > ChrisA > From bursejan at gmail.com Sun Sep 19 05:03:01 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Sun, 19 Sep 2021 02:03:01 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: <7497b77f-f4cc-4b36-99e4-c1cff741e4c4n@googlegroups.com> The trail itself can possibly not be eliminated. Its like a database logfile. The trail is used during backtracking to undo variable bindings. Which is like a database rollback. Here is an example where a tail is used: /* X equals 1 or X equals 2 */ ?- X=1; X=2. X = 1; X = 2. In the first answer the trail will have recorded that X was bound to 1. When the second answer is requested, the trail is used to unbind X, so that it can be bound to 2. The Prolog garbage collection is a compactification of the trail. Maybe databases can do the same with their logfiles, for example if a logfile contains an insert and then a delete of the same row, then these two logentries can be merged. The only difference here is that Prolog garbage collection primarily compactes towards trail entries that have become irrelevant. This is part of intelligent bracktracking, and backjumping over multiple goal invocations, that didn't record a choice point. The Prolog garbage collection can compact the trail before backjumping happens. So that Prolog search has more space available. Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 10:51:03 UTC+2: > I am refering to: > > Greg Ewing schrieb: > > where [w] is a weak reference object. Then you could periodically > > scan the trail looking for dead weakref objects and remove the > > corresponding [*] node from the list. > > > > You can also attach callbacks to weakref objects that are triggered > > when the referenced object dies. You might be able to make use of > > that to remove items from the trail instead of the periodic scanning. > Question to Chris Angelico: If I stay with my > sweep_trail(), which is the periodically scanning, > I can use a single linked list. > > On the other hand if I would use the trigger > from Python, I possibly would need a double linked > list, to remove an element. > > Chris Angelico, is there a third option, that I have > overlooked? Single linked list uses less space > than double linked list, this why I go with scan. > > Chris Angelico schrieb: > > On Sun, Sep 19, 2021 at 11:46 AM Mostowski Collapse wrote: > >> > >> Yeah, it seems weak references could indeed spare > >> me mark_term(). But then I am stil left with sweep_trail(). > >> I did not yet measure what takes more time mark_term() > >> or sweep_trail(). The displayed "gc" is the sum of both. > >> > >> From what I have seen, very large trail practically reduced > >> to a zero trail during Prolog GC, I am assuming that > >> mark_term() is not the working horse. Usually mark_term() > >> only marks what is not-Garbage, and sweep_trail() > >> > >> has to deal with Garbage and not-Garbage. And there > >> is usually a lot of Garbage, much more than not-Garbage. > >> Finding the objects that survive, is like finding the needle > >> in the haystack, except we do not have to scan the > > > > If you stop referring to something, it is garbage. Python will dispose of it. > > > > You literally need to do nothing at all, and let the language take > > care of things. > > > > ChrisA > > From storchaka at gmail.com Sun Sep 19 11:06:17 2021 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 19 Sep 2021 18:06:17 +0300 Subject: How to support annotations for a custom type in a C extension? In-Reply-To: <0638d578-f322-0b59-3af5-3d0927c1eeb2@mrabarnett.plus.com> References: <74a8da49-6966-62e1-d481-db350b69bed2@mrabarnett.plus.com> <0638d578-f322-0b59-3af5-3d0927c1eeb2@mrabarnett.plus.com> Message-ID: 19.09.21 05:59, MRAB ????: > On 2021-09-18 16:09, Serhiy Storchaka wrote: >> "(PyCFunction)" is redundant, Py_GenericAlias already has the right >> type. Overuse of casting to PyCFunction can hide actual bugs. >> > I borrowed that from listobject.c, which does have the cast. Fixed. https://github.com/python/cpython/pull/28450 From hrouselle at jevedi.com Sun Sep 19 11:31:50 2021 From: hrouselle at jevedi.com (Hope Rouselle) Date: Sun, 19 Sep 2021 12:31:50 -0300 Subject: on writing a while loop for rolling two dice References: <86r1edlqxw.fsf@jevedi.com> <854aa6b9-558f-b280-4d7e-e9ed1d0a9aeb@DancesWithMice.info> Message-ID: <86ilywbmzd.fsf@jevedi.com> dn writes: [...] > Further, if you look at the OP's original solution, it only publishes > the last pair, ie the match, without mention of the list of non-matches. > Was it perhaps only a means of testing the solution? It was a means of showing the student that indeed they obtained a match. If the exercise asked for returning a single die or no die at all, they would make mistakes and there'd be no sign of them being wrong. For instance, one trouble a lot of them went through was to start counting from zero and so their number of rolls was off by one. (I didn't see that coming!) The reason they fall for this is that they also test little --- for some it never occurred a match on the first roll, so they never saw the zero counter coming out. From shashwatpandey2303 at gmail.com Sun Sep 19 08:42:11 2021 From: shashwatpandey2303 at gmail.com (Shashwat Pandey) Date: Sun, 19 Sep 2021 18:12:11 +0530 Subject: query In-Reply-To: References: Message-ID: ---------- Forwarded message --------- From: Date: Sun, Sep 19, 2021, 13:20 Subject: query To: Hello! I see you want to post a message to the Python List. We would be happy to help, but you must subscribe first: https://mail.python.org/mailman/listinfo/python-list After you have subscribed, please send your message to python-list at python.org again. Please do not include images as this is a text-only forum -- instead, copy/paste or transcribe the information from the image that you want us to know. Alternatively, this list is mirrored both ways with the comp.lang.python newsgroup (news:comp.lang.python). Some people find it easier to follow this and other lists via gmane (http://news.gmane.org/gmane.comp.python.general), a service which offers a newsgroup interface to many online mailing lists. *NB all posts to the mailing list are publicly archived at:* https://mail.python.org/pipermail/python-list ---------- Forwarded message ---------- From: Shashwat Pandey To: python-list at python.org Cc: Bcc: Date: Sun, 19 Sep 2021 13:19:31 +0530 Subject: query Hello, One thing I want to ask... How to Fix Installation Error of PyAudio in VS Code ( Windows 32 Bit ) ?? Please Answer Me.... It's very essential to complete my concept digitalization system project. the error is given below please solve this problem PS C:\Users\PRIYA> pip install pyaudio Collecting pyaudio Using cached PyAudio-0.2.11.tar.gz (37 kB) Using legacy 'setup.py install' for pyaudio, since package 'wheel' is not installed. Installing collected packages: pyaudio Running setup.py install for pyaudio ... error ERROR: Command errored out with exit status 1: command: 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"'; __file__='"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\PRIYA\AppData\Local\Temp\pip-record-bdr6wfd9\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\Include\pyaudio' cwd: C:\Users\PRIYA\AppData\Local\Temp\pip-install-v8hphebk\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\ Complete output (9 lines): running install running build running build_py creating build creating build\lib.win32-3.9 copying src\pyaudio.py -> build\lib.win32-3.9 running build_ext building '_portaudio' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ ---------------------------------------- ERROR: Command errored out with exit status 1: 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"'; __file__='"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\PRIYA\AppData\Local\Temp\pip-record-bdr6wfd9\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\Include\pyaudio' Check the logs for full command output. WARNING: You are using pip version 21.2.3; however, version 21.2.4 is available. You should consider upgrading via the 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe -m pip install --upgrade pip' command. From mats at wichmann.us Sun Sep 19 13:47:45 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sun, 19 Sep 2021 11:47:45 -0600 Subject: query In-Reply-To: References: Message-ID: <468fb509-48fb-28d9-fb08-d04d6203bb0a@wichmann.us> On 9/19/21 06:42, Shashwat Pandey wrote: > How to Fix Installation Error of PyAudio in VS Code ( Windows 32 Bit ) ?? > > Please Answer Me.... > It's very essential to complete my concept digitalization system project. This comes up somewhat often - pyaudio has not released new binary installer packages for some time. When the system you install on doesn't find a matching binary package, it tries to build it from the from source. This doesn't work if you don't have a compiler: > ERROR: Command errored out with exit status 1: > command: > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe' -u -c > 'import io, os, sys, setuptools, tokenize; sys.argv[0] = > '"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"'; > __file__='"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"';f > = getattr(tokenize, '"'"'open'"'"', open)(__file__) if > os.path.exists(__file__) else io.StringIO('"'"'from setuptools import > setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record > 'C:\Users\PRIYA\AppData\Local\Temp\pip-record-bdr6wfd9\install-record.txt' > --single-version-externally-managed --compile --install-headers > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\Include\pyaudio' > cwd: > C:\Users\PRIYA\AppData\Local\Temp\pip-install-v8hphebk\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\ > Complete output (9 lines): > running install > running build > running build_py > creating build > creating build\lib.win32-3.9 > copying src\pyaudio.py -> build\lib.win32-3.9 > running build_ext > building '_portaudio' extension > error: Microsoft Visual C++ 14.0 or greater is required. Get it with > "Microsoft C++ Build Tools": quite honestly, even if you do install msvc on your Windows system, it will probably still fail to build as there are usually some addditional setup steps. This kind of thing is fairly easy to prospect for - got to https://pypi.org, search for your package, and when found, click on Download Files and look at what's available. In the case of the "official" pyaudio, there haven't been versions since Python 3.6. In this case there's an alternate build you can download, if you look here: https://pypi.org/project/pyaudio-wheels/ (in other words, instead of installing "pyaudio" try installing "pyaudio-wheels"). Hopefully that will work out for you. From python at mrabarnett.plus.com Sun Sep 19 13:54:09 2021 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 19 Sep 2021 18:54:09 +0100 Subject: query In-Reply-To: References: Message-ID: On 2021-09-19 13:42, Shashwat Pandey wrote: > ---------- Forwarded message --------- > From: > Date: Sun, Sep 19, 2021, 13:20 > Subject: query > To: > > > Hello! I see you want to post a message to the Python List. We would > be happy to help, but you must subscribe first: > > https://mail.python.org/mailman/listinfo/python-list > > After you have subscribed, please send your message to > python-list at python.org again. Please do not include images as this is > a text-only forum -- instead, copy/paste or transcribe the information > from the image that you want us to know. > > Alternatively, this list is mirrored both ways with the > comp.lang.python newsgroup (news:comp.lang.python). > > Some people find it easier to follow this and other lists via gmane > (http://news.gmane.org/gmane.comp.python.general), a service which > offers a newsgroup interface to many online mailing lists. > > *NB all posts to the mailing list are publicly archived at:* > > https://mail.python.org/pipermail/python-list > > > > > > ---------- Forwarded message ---------- > From: Shashwat Pandey > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 19 Sep 2021 13:19:31 +0530 > Subject: query > Hello, > One thing I want to ask... > > How to Fix Installation Error of PyAudio in VS Code ( Windows 32 Bit ) ?? > > Please Answer Me.... > It's very essential to complete my concept digitalization system project. > > the error is given below please solve this problem > > > PS C:\Users\PRIYA> pip install pyaudio > Collecting pyaudio > Using cached PyAudio-0.2.11.tar.gz (37 kB) > Using legacy 'setup.py install' for pyaudio, since package 'wheel' is not > installed. > Installing collected packages: pyaudio > Running setup.py install for pyaudio ... error > ERROR: Command errored out with exit status 1: > command: > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe' -u -c > 'import io, os, sys, setuptools, tokenize; sys.argv[0] = > '"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"'; > __file__='"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"';f > = getattr(tokenize, '"'"'open'"'"', open)(__file__) if > os.path.exists(__file__) else io.StringIO('"'"'from setuptools import > setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record > 'C:\Users\PRIYA\AppData\Local\Temp\pip-record-bdr6wfd9\install-record.txt' > --single-version-externally-managed --compile --install-headers > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\Include\pyaudio' > cwd: > C:\Users\PRIYA\AppData\Local\Temp\pip-install-v8hphebk\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\ > Complete output (9 lines): > running install > running build > running build_py > creating build > creating build\lib.win32-3.9 > copying src\pyaudio.py -> build\lib.win32-3.9 > running build_ext > building '_portaudio' extension > error: Microsoft Visual C++ 14.0 or greater is required. Get it with > "Microsoft C++ Build Tools": > https://visualstudio.microsoft.com/visual-cpp-build-tools/ > ---------------------------------------- > ERROR: Command errored out with exit status 1: > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe' -u -c > 'import io, os, sys, setuptools, tokenize; sys.argv[0] = > '"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"'; > __file__='"'"'C:\\Users\\PRIYA\\AppData\\Local\\Temp\\pip-install-v8hphebk\\pyaudio_4240cdb9492c47abb6d002cdb32aa86c\\setup.py'"'"';f > = getattr(tokenize, '"'"'open'"'"', open)(__file__) if > os.path.exists(__file__) else io.StringIO('"'"'from setuptools import > setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record > 'C:\Users\PRIYA\AppData\Local\Temp\pip-record-bdr6wfd9\install-record.txt' > --single-version-externally-managed --compile --install-headers > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\Include\pyaudio' > Check the logs for full command output. > WARNING: You are using pip version 21.2.3; however, version 21.2.4 is > available. > You should consider upgrading via the > 'C:\Users\PRIYA\AppData\Local\Programs\Python\Python39-32\python.exe -m pip > install --upgrade pip' command. > By default, pip looks for packages on PyPI. Unfortunately, it looks like PyAudio is not available on PyPI for that version of Python. Fortunately, it _is_ available from Christoph Gohlke's site: https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio The one you want is: PyAudio?0.2.11?cp39?cp39?win32.whl From arj.python at gmail.com Sun Sep 19 14:21:55 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 19 Sep 2021 22:21:55 +0400 Subject: The code version of python -i In-Reply-To: References: Message-ID: Oh thanks a lot, i was way from list this week Just a note: it's pretty amazing to have at Chris Angelico at your local PUG Else, Well, let's say you have a lib that you need to provide users with a shell for them to try out functions, no need to translate functions to cli args. Like Django provides a shell for users to use. The aim of injection is to allow code usage without imports. The presentation write-up is pretty new to me as I never approached clean code presentations. My presentations have always been hard text on screen . As all coders my natural instinct is to ... code a prez tool from scratch! Kind Regards, Abdur-Rahmaan Janhangeer about | blog github Mauritius On Fri, Sep 17, 2021 at 5:04 AM dn via Python-list wrote: > Abdur-Rahmaan, > Apologies for delay: several last-minute tasks were landed on me, so I > haven't been able to 'read the list' since last week. > > > > If i have a file name flower.py and i add x = 1 in it. > > When i run python -i flower.py i get a shell > >>>> > > > > If type x i get 1 > >>>> x > > 1 > > > > The values are auto injected. > > > > How do i start a shell by code with values already injected? Thanks > > > What do you want to achieve with this? > > > At this week's local PUG meeting, I was 'the warm-up act' for (our own) > @Chris Angelico giving a talk about 'the walrus operator', his part in > that PEP, and how anyone-and-everyone can contribute to the Python > eco-system. > > My contribution was to start at the Python-Apprentice level, talking > about different types of 'operators', and preparing the stage?beach for > the more advanced 'walrus'. It culminated in a live-coding demo of > Conditional expressions/the "ternary operator". > > Live-coding demos are always better (for the audience) than passively > watching a progression of slides! However, that territory is labelled > 'here be dragons', for presenters! (continuing the tradition of slide > projectors which don't accept my cartridge/carousel, white boards > lacking pens with ink, over-head projectors with blown lamps (and a used > spare), RGB projectors which only/don't accept VGA cables, ...) > > Further: we've all suffered YouTube wannabes telling us how easy it is > to do 'something', typing and mis-typing and re-typing, and generally > wasting our time... (have they not heard of video-editing?) Those of us > who present 'live' (unlike my video-lectures), can't rely upon a later > 'cosmetic' stage to apply lip-stick on any 'pigs' in our typing! > > Rather than using a "script" document (as-in 'speaker's notes') to > prompt me with what to type next, the remote presentation tool > (BigBlueButton web-conferencing) allows the 'projection' of a (desktop) > terminal window - whilst hiding my text-editor. Thus, at the appropriate > moment, I was copy-pasting from my 'script' in xed, into the terminal's > Python REPL. Worked very well - it appears as if I'm a very fast typist! > Of course, sometimes I was copying one line of code at a time, and at > others, multiple lines - thus (*still*) giving opportunity to 'fluff my > lines'. Hah! > > For live-demos, I like to use pedagogical (teaching) 'patterns' of a > structured narrative (as one would with a lecture) but trying to > encourage involvement (if not "active learning"), using a "worked > example" (learning through demonstration), and "discovery" (which > because it is effectively a 'presentation', really means 'gradual > revelation'). > > (apologies for any non-obvious, non-Python, jargon - cognitive > psychology (learning how we learn) is my research field) > > Thus, the demo proceeds on a step-by-step basis. At each step the > process is: > 1 present a problem/ask a question > 2 ensure understanding/invite comments/accept suggestions > 3 present code snippet* > and (presumably) execute same to produce an 'answer' > 4 comment/discuss/critique > > - which, unless 'the end', should lead us back to nr1 for the 'next > step'... > > * in other scenarios, this might involve use of code suggested by the > audience! > > Before reading this thread, I had been thinking that Jupyter might > empower the combination of Python, 'steps', and 'live-coding'. However, > a notebook will likely display future steps on-screen, before the > current one has been completed (which defeats 'gradual revelation' - > somewhat like jumping to 'the last page to find-out who dunnit') - > unlike (say) presentation slides where no-one can see 'what comes > next'/jump ahead, before the earlier 'lesson' has been learned. > > NB I'm not seeking 'information hiding' or encapsulation by its Java > definition; but do want people to consider (only) one sub-problem at a > time, and thus building-up to the whole solution! If one first > understands the problem(s) and then contributes to generating a > solution, you will learn far more effectively than if I simply > lecture/read the Python manual-entry to you! > > Courtesy of this discussion, am wondering if it might be possible to > structure what is currently a 'teaching notes' or "script" document > (from which I was copy-pasting), and build a 'projector' program which > will run a sub-interpreter* to run exactly one 'step' of the > 'live-coding' demo at a time (whilst also retaining the option of REPL > access, to be able prove or expose short-comings (nr 4, above), and > without revealing the 'what comes next'? > > * rather than python -i; am considering Lib.runpy, and/or Lib.code > > > How might any of this relate to your interest? > -- > Regards, > =dn > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Sun Sep 19 14:33:39 2021 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 20 Sep 2021 04:33:39 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <3b41cebb-7f8e-4250-b290-0734f59dbd63n@googlegroups.com> <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: > > I am refering to: > > Greg Ewing schrieb: > > where [w] is a weak reference object. Then you could periodically > > scan the trail looking for dead weakref objects and remove the > > corresponding [*] node from the list. > > > > You can also attach callbacks to weakref objects that are triggered > > when the referenced object dies. You might be able to make use of > > that to remove items from the trail instead of the periodic scanning. > > Question to Chris Angelico: If I stay with my > sweep_trail(), which is the periodically scanning, > I can use a single linked list. > > On the other hand if I would use the trigger > from Python, I possibly would need a double linked > list, to remove an element. > > Chris Angelico, is there a third option, that I have > overlooked? Single linked list uses less space > than double linked list, this why I go with scan. > I don't know. I don't understand your code well enough to offer advice like that, because *your code is too complicated* and not nearly clear enough. But however it is that you're doing things, the best way is almost always to directly refer to objects. Don't fiddle around with creating your own concept of a doubly-linked list and a set of objects; just refer directly to the objects. Let Python be Python, don't try to build your own language on top of it. ChrisA From hjp-python at hjp.at Mon Sep 20 07:49:31 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 20 Sep 2021 13:49:31 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On 2021-09-20 04:33:39 +1000, Chris Angelico wrote: > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: > > Question to Chris Angelico: If I stay with my > > sweep_trail(), which is the periodically scanning, > > I can use a single linked list. > > > > On the other hand if I would use the trigger > > from Python, I possibly would need a double linked > > list, to remove an element. > > > > Chris Angelico, is there a third option, that I have > > overlooked? Single linked list uses less space > > than double linked list, this why I go with scan. > > > > I don't know. I don't understand your code well enough to offer advice > like that, because *your code is too complicated* and not nearly clear > enough. > > But however it is that you're doing things, the best way is almost > always to directly refer to objects. Don't fiddle around with creating > your own concept of a doubly-linked list and a set of objects; just > refer directly to the objects. And almost certainly: Just use the builtin list type if you need a list. Don't build one yourself. > Let Python be Python, don't try to build your own language on top of > it. Well, he's writing a Prolog interpreter, so building his own language on top of Python is sort of the point. I think a better way to put it is "Don't try to write Python as if it was C". A C operation may be compiled to a single machine instruction which is executed in a fraction of a nanosecond. A Python instruction (in CPython) always includes at least the interpreter overhead and often several method lookups and method calls. You want to amortize that overhead over as much work as possible. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From rosuav at gmail.com Mon Sep 20 08:24:40 2021 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 20 Sep 2021 22:24:40 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer wrote: > > Let Python be Python, don't try to build your own language on top of > > it. > > Well, he's writing a Prolog interpreter, so building his own language on > top of Python is sort of the point. I think a better way to put it is > "Don't try to write Python as if it was C". Fair point. Or combining them both: Writing a language interpreter in Python as if you were writing it in C, and then complaining that it is slow, is only going to elicit "well uhh yes?" responses. Languages like NetRexx (and, I think, Jython, although I can't find any definitive and current answers) are slightly different from their "parent" languages, because they make good use of their implementation languages' features. This Prolog interpreter might not even need to be different in functionality, but its implementation would be different, and it could take advantage of the underlying garbage collection. ChrisA From janburse at fastmail.fm Sun Sep 19 15:46:01 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Sun, 19 Sep 2021 21:46:01 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: sympy also builds a language on top of Python. pandas also builds a language on top of Python. Is there some pope that says this wouldn't be allowed, I dont think so, otherwise sympy, pandas, etc.. wouldn't exist. I dont understand your argument. Chris Angelico schrieb: > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: >> >> I am refering to: >> >> Greg Ewing schrieb: >> > where [w] is a weak reference object. Then you could periodically >> > scan the trail looking for dead weakref objects and remove the >> > corresponding [*] node from the list. >> > >> > You can also attach callbacks to weakref objects that are triggered >> > when the referenced object dies. You might be able to make use of >> > that to remove items from the trail instead of the periodic scanning. >> >> Question to Chris Angelico: If I stay with my >> sweep_trail(), which is the periodically scanning, >> I can use a single linked list. >> >> On the other hand if I would use the trigger >> from Python, I possibly would need a double linked >> list, to remove an element. >> >> Chris Angelico, is there a third option, that I have >> overlooked? Single linked list uses less space >> than double linked list, this why I go with scan. >> > > I don't know. I don't understand your code well enough to offer advice > like that, because *your code is too complicated* and not nearly clear > enough. > > But however it is that you're doing things, the best way is almost > always to directly refer to objects. Don't fiddle around with creating > your own concept of a doubly-linked list and a set of objects; just > refer directly to the objects. Let Python be Python, don't try to > build your own language on top of it. > > ChrisA > From bursejan at gmail.com Sun Sep 19 16:02:31 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Sun, 19 Sep 2021 13:02:31 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: <775ec5c8-8211-42ee-99c1-86d6e536fb71n@googlegroups.com> Also I nowhere wrote that I would use doubly-linked list or a set of objects. The trail is neither, its a single linked list. I also refer directly to objects, I do not need the trail to refer to objects. The Prolog trail is only a logbook. The Prolog trail has similarity to a database log: transaction journal, database log, binary log or audit **trail** https://en.wikipedia.org/wiki/Transaction_log Do you say Python should not be used to implement such things? In my opinion, Python has a high potential to implement Prolog, because it has also ast.parse() and compile(). But I do not yet have a showcase that uses these features of Python to compile Prolog. I dont work 24/7 and I cannot clone myself. Currently the Prolog code is interpreted. I have a prototype where Prolog code is compiled into JavaScript, but I did not yet try this approach with Python. Here you see how JavaScript closures are generated, a first prototype: const alt4 = make_defined([new Clause(1, [0, 0], function( display, actual, cont) {return(new Compound(".", [new Compound( "==", [deref(actual.args[0]), "end_of_file"]), new Compound( ".", [new Compound("$CUT", [deref(display[0])]), cont ])]))}, 0, undefined), new Clause(1, [0, 0], function( display, actual, cont) {return(new Compound(".", [new Compound( "expand_include", [deref(actual.args[0]), deref(actual.args[1] ), display[0] = new Variable()]), new Compound(".", [new Compound("handle_term", [deref(display[0])]), new Compound( ".", ["fail", cont])])]))}, -1, undefined)]); add("next_term", 1, new Clause(2, [0], function(display, actual, cont) {return(new Compound(".", [new Compound("read_term", [deref(actual.args[0]), display[0] = new Variable(), new Compound(".", [new Compound("variable_names", [ display[1] = new Variable()]), "[]"])]), new Compound( ".", [new Compound(alt4, [deref(display[0]), deref( display[1])]), cont])]))}, -1, undefined)); https://github.com/jburse/dogelog-moon/issues/184 Will do the same for Python in the next weeks. Then later this approach will be combined with a few planned optimizations. So far got a 25% speed increase for JavaScript with this new compilation scheme, but there is no official release out yet, that features this approach. And there should be much more in it, also for Python. Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 21:46:20 UTC+2: > sympy also builds a language on top of Python. > pandas also builds a language on top of Python. > > Is there some pope that says this wouldn't be > allowed, I dont think so, otherwise sympy, pandas, etc.. > > wouldn't exist. I dont understand your argument. > > Chris Angelico schrieb: > > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: > >> > >> I am refering to: > >> > >> Greg Ewing schrieb: > >> > where [w] is a weak reference object. Then you could periodically > >> > scan the trail looking for dead weakref objects and remove the > >> > corresponding [*] node from the list. > >> > > >> > You can also attach callbacks to weakref objects that are triggered > >> > when the referenced object dies. You might be able to make use of > >> > that to remove items from the trail instead of the periodic scanning. > >> > >> Question to Chris Angelico: If I stay with my > >> sweep_trail(), which is the periodically scanning, > >> I can use a single linked list. > >> > >> On the other hand if I would use the trigger > >> from Python, I possibly would need a double linked > >> list, to remove an element. > >> > >> Chris Angelico, is there a third option, that I have > >> overlooked? Single linked list uses less space > >> than double linked list, this why I go with scan. > >> > > > > I don't know. I don't understand your code well enough to offer advice > > like that, because *your code is too complicated* and not nearly clear > > enough. > > > > But however it is that you're doing things, the best way is almost > > always to directly refer to objects. Don't fiddle around with creating > > your own concept of a doubly-linked list and a set of objects; just > > refer directly to the objects. Let Python be Python, don't try to > > build your own language on top of it. > > > > ChrisA > > From bursejan at gmail.com Sun Sep 19 16:22:18 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Sun, 19 Sep 2021 13:22:18 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: <775ec5c8-8211-42ee-99c1-86d6e536fb71n@googlegroups.com> References: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> <775ec5c8-8211-42ee-99c1-86d6e536fb71n@googlegroups.com> Message-ID: <3dafd08a-ff7e-438f-bbd9-25559cbc6ef8n@googlegroups.com> Please be patient. A big problem with development can be burnout. So I am trying to slow down things at the moment. The ideas are easy to sketch, but implementation can take weeks. Here is the idea again in a nutshell: use ast.parse() and compile(). Or build directly an AST, not using the string detour as in ast.parse(). How long will it take to have a working solution? 11 years ago there was: Pyrolog: Prolog written in Python using PyPy's RPython tool chain https://www.reddit.com/r/prolog/comments/fbuz1/pyrolog_prolog_written_in_python_using_pypys/ RPython is a framework for implementing interpreters and virtual machines for programming languages, especially dynamic languages. https://rpython.readthedocs.io/en/latest/faq.html Currently I am not planning to use RPython, want to to use standard Python AST and compile(). Might have a look at RPython or similar stuff later. Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 22:02:41 UTC+2: > Also I nowhere wrote that I would use doubly-linked list or > a set of objects. The trail is neither, its a single linked list. I > also refer directly to objects, I do not need the trail to refer > > to objects. The Prolog trail is only a logbook. The Prolog trail > has similarity to a database log: > > transaction journal, database log, binary log or audit **trail** > https://en.wikipedia.org/wiki/Transaction_log > > Do you say Python should not be used to implement > such things? In my opinion, Python has a high potential > to implement Prolog, because it has also ast.parse() > > and compile(). But I do not yet have a showcase that uses > these features of Python to compile Prolog. I dont work 24/7 > and I cannot clone myself. Currently the Prolog code is interpreted. > > I have a prototype where Prolog code is compiled into JavaScript, > but I did not yet try this approach with Python. Here you see how > JavaScript closures are generated, a first prototype: > > const alt4 = make_defined([new Clause(1, [0, 0], function( > display, actual, cont) {return(new Compound(".", [new Compound( > "==", [deref(actual.args[0]), "end_of_file"]), new Compound( > ".", [new Compound("$CUT", [deref(display[0])]), cont > ])]))}, 0, undefined), new Clause(1, [0, 0], function( > display, actual, cont) {return(new Compound(".", [new Compound( > "expand_include", [deref(actual.args[0]), deref(actual.args[1] > ), display[0] = new Variable()]), new Compound(".", > [new Compound("handle_term", [deref(display[0])]), new Compound( > ".", ["fail", cont])])]))}, -1, undefined)]); > > add("next_term", 1, new Clause(2, [0], function(display, actual, > cont) {return(new Compound(".", [new Compound("read_term", > [deref(actual.args[0]), display[0] = new Variable(), > new Compound(".", [new Compound("variable_names", [ > display[1] = new Variable()]), "[]"])]), new Compound( > ".", [new Compound(alt4, [deref(display[0]), deref( > display[1])]), cont])]))}, -1, undefined)); > > https://github.com/jburse/dogelog-moon/issues/184 > > Will do the same for Python in the next weeks. Then later this approach > will be combined with a few planned optimizations. So far got a 25% > speed increase for JavaScript with this new compilation scheme, but > > there is no official release out yet, that features this approach. And > there should be much more in it, also for Python. > Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 21:46:20 UTC+2: > > sympy also builds a language on top of Python. > > pandas also builds a language on top of Python. > > > > Is there some pope that says this wouldn't be > > allowed, I dont think so, otherwise sympy, pandas, etc.. > > > > wouldn't exist. I dont understand your argument. > > > > Chris Angelico schrieb: > > > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: > > >> > > >> I am refering to: > > >> > > >> Greg Ewing schrieb: > > >> > where [w] is a weak reference object. Then you could periodically > > >> > scan the trail looking for dead weakref objects and remove the > > >> > corresponding [*] node from the list. > > >> > > > >> > You can also attach callbacks to weakref objects that are triggered > > >> > when the referenced object dies. You might be able to make use of > > >> > that to remove items from the trail instead of the periodic scanning. > > >> > > >> Question to Chris Angelico: If I stay with my > > >> sweep_trail(), which is the periodically scanning, > > >> I can use a single linked list. > > >> > > >> On the other hand if I would use the trigger > > >> from Python, I possibly would need a double linked > > >> list, to remove an element. > > >> > > >> Chris Angelico, is there a third option, that I have > > >> overlooked? Single linked list uses less space > > >> than double linked list, this why I go with scan. > > >> > > > > > > I don't know. I don't understand your code well enough to offer advice > > > like that, because *your code is too complicated* and not nearly clear > > > enough. > > > > > > But however it is that you're doing things, the best way is almost > > > always to directly refer to objects. Don't fiddle around with creating > > > your own concept of a doubly-linked list and a set of objects; just > > > refer directly to the objects. Let Python be Python, don't try to > > > build your own language on top of it. > > > > > > ChrisA > > > From greg.ewing at canterbury.ac.nz Sun Sep 19 18:53:30 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 20 Sep 2021 10:53:30 +1200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: >> >> On the other hand if I would use the trigger >> from Python, I possibly would need a double linked >> list, to remove an element. Here's another idea: Put weak references in the trail, but don't bother with the scanning -- just skip over dead ones during backtracking. Seems to me you would be doing about the same amount of work either way, just doing it at different times. -- Greg From janburse at fastmail.fm Mon Sep 20 02:44:27 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Mon, 20 Sep 2021 08:44:27 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: This strategy works if you use failure driven loops. It doesn't work you program recursive loops that go on forever. Like Erlang processes. foo(X) :- bar(X,Y), foo(Y). Typically Y is a fresh variable. A good Prolog system with good Garbage Collection can run such a process for days and months. If you don't clean up the trail, you exhaust the memory after a few minutes or seconds. Greg Ewing schrieb: >> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse >> wrote: >>> >>> On the other hand if I would use the trigger >>> from Python, I possibly would need a double linked >>> list, to remove an element. > > Here's another idea: Put weak references in the trail, but > don't bother with the scanning -- just skip over dead ones > during backtracking. > > Seems to me you would be doing about the same amount of > work either way, just doing it at different times. > From bursejan at gmail.com Mon Sep 20 06:04:38 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Mon, 20 Sep 2021 03:04:38 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: But I dont see any utility in investing too much time with the Prolog garbage collection of Dogelog runtime. It is only a small share of the execution time: Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2: > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % Standard Python Version, Warm Run > % ?- time(fibo(23,X)). > % % Wall 3865 ms, gc 94 ms, 71991 lips > % X = 46368. > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % GraalVM Python Version, Warm Warm Run > % ?- time(fibo(23,X)). > % % Wall 695 ms, gc 14 ms, 400356 lips > % X = 46368. > > The "gc" timing measures Prolog garbage > collection. So you get the following percentage > of time spent in Prolog garbage collection: > > Standard Python: 94 / 3865 = 2.4% > > GraalVM Python: 14 / 695 = 2.0% If you spare these 2-3% it will not speed-up Dogelog runtime. The Prolog garbage collection is there to allow Erlang processes. And its also there to allow more Prolog search without exhausting the memory. But it cost you only 2-3% judging from the Fibonnacci Numbers example. Need to check with other examples whether its higher. But since the ratio between Garbage and non-Garbage is usually high, and non-Garbage is easily identified, and the Garbage is also easily removed, the time will possibly not exceed much more than the same 2-3% for other examples. So in conclusion I am least worried about the Prolog garbage collection. You guys are only worried because its something new. But its nothing that does any harm and costs a lot, it only does good and is very cheap in terms of extra runtime effort. Mostowski Collapse schrieb am Montag, 20. September 2021 um 08:44:49 UTC+2: > This strategy works if you use failure driven loops. > It doesn't work you program recursive loops that go > on forever. Like Erlang processes. > > foo(X) :- > bar(X,Y), foo(Y). > > Typically Y is a fresh variable. A good Prolog system > with good Garbage Collection can run such a process > for days and months. > > If you don't clean up the trail, you exhaust the > memory after a few minutes or seconds. > > Greg Ewing schrieb: > >> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse > >> wrote: > >>> > >>> On the other hand if I would use the trigger > >>> from Python, I possibly would need a double linked > >>> list, to remove an element. > > > > Here's another idea: Put weak references in the trail, but > > don't bother with the scanning -- just skip over dead ones > > during backtracking. > > > > Seems to me you would be doing about the same amount of > > work either way, just doing it at different times. > > From bursejan at gmail.com Mon Sep 20 08:08:55 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Mon, 20 Sep 2021 05:08:55 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: I read the following, and you should also know: > Python's [] is implemented as an array, not a linked list. > Although resizing is O(n), appending to it is amortized O(1), > because resizes happen very rarely. https://stackoverflow.com/a/5932364/502187 The list type doesn't have an O(1) operation to remove an element during sweep. The list type, not like its name would suggest, in Python is an array. These arrays are not so expensive when you append() an element. Because they are allocated with some excess capacity. And they grow exponentially. So amortisized you can append() a lot of elements to a Python list, which is an array. But you cannot poke so cheaply holes into it. So if you have this scenario: Before: - [ A1, .., An , B, C1, .., Cm ] After: - [ A1, .., An , C1, .., Cm ] You have to copy C1,..,Cm one position down. On the other hand, when scanning the single list, removing the element is just pointer swizzling. The code is here, the positive if-then-else branch keeps the element, the negative if-then-else branch drops the element. Thats quite standard algorithm for linked lists: /* pointer swizzling */ while temp is not None: term = temp temp = term.tail if (term.flags & MASK_VAR_MARK) != 0: term.flags &= ~MASK_VAR_MARK if back is not None: back.tail = term else: trail = term back = term else: term.instantiated = NotImplemented term.tail = None count -= 1 https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163 There is nothing wrong with implementing a single list in Python. Only you never saw that one maybe. If you would indeed use Python lists which are arrays, you would maybe get a much slower sweep_trail() and this would be seen. But currently its not seen. It happens that 1000000 of elements are sweeped, if you would do this with copy inside a Python list which are arrays, it would get much more expensive, and the extremly cheap Prolog garbage collection, as it stands now, wouldn't be that cheap anymore. You can try yourself. My sweep_trail() needs frequent resize, which would be O(n) each, so that sweep_trail becomes O(n^2). Which the current implementation its only O(n). Peter J. Holzer schrieb am Montag, 20. September 2021 um 13:49:49 UTC+2: > On 2021-09-20 04:33:39 +1000, Chris Angelico wrote: > > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: > > > Question to Chris Angelico: If I stay with my > > > sweep_trail(), which is the periodically scanning, > > > I can use a single linked list. > > > > > > On the other hand if I would use the trigger > > > from Python, I possibly would need a double linked > > > list, to remove an element. > > > > > > Chris Angelico, is there a third option, that I have > > > overlooked? Single linked list uses less space > > > than double linked list, this why I go with scan. > > > > > > > I don't know. I don't understand your code well enough to offer advice > > like that, because *your code is too complicated* and not nearly clear > > enough. > > > > But however it is that you're doing things, the best way is almost > > always to directly refer to objects. Don't fiddle around with creating > > your own concept of a doubly-linked list and a set of objects; just > > refer directly to the objects. > And almost certainly: Just use the builtin list type if you need a list. > Don't build one yourself. > > Let Python be Python, don't try to build your own language on top of > > it. > Well, he's writing a Prolog interpreter, so building his own language on > top of Python is sort of the point. I think a better way to put it is > "Don't try to write Python as if it was C". A C operation may be > compiled to a single machine instruction which is executed in a fraction > of a nanosecond. A Python instruction (in CPython) always includes at > least the interpreter overhead and often several method lookups and method > calls. You want to amortize that overhead over as much work as possible. > > hp > > -- > _ | Peter J. Holzer | Story must make more sense than reality. > |_|_) | | > | | | h... at hjp.at | -- Charles Stross, "Creative writing > __/ | http://www.hjp.at/ | challenge!" From janburse at fastmail.fm Mon Sep 20 08:16:37 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Mon, 20 Sep 2021 14:16:37 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: What would be maybe possible, is to scan the trail from the other side, and use a first pass to determine the new size, and use a second pass to fill a new array with the remaining elments. This would be two times O(n), so it would be linear and not quadratic O(n^2) as when you scan from the top and poke holes. But then something else doesn't work so easily. Namely: def adjust_mark(temp): while temp is not None: if (temp.flags & MASK_VAR_MARK) != 0: return temp else: temp = temp.tail return temp https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L151 This is needed to adjust the choice points. If you have an array instead of a linked listed as I have now, you would need to adjust array indexes instead pointers into linked list elements. I havent come up with an array solution yet for the trail, since I dont see any utility in investing too much time with the Prolog garbage collection of Dogelog runtime. It is only a small share of the execution time: Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2: > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % Standard Python Version, Warm Run > % ?- time(fibo(23,X)). > % % Wall 3865 ms, gc 94 ms, 71991 lips > % X = 46368. > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % GraalVM Python Version, Warm Warm Run > % ?- time(fibo(23,X)). > % % Wall 695 ms, gc 14 ms, 400356 lips > % X = 46368. Mostowski Collapse wrote: > I read the following, and you should also know: > >> Python's [] is implemented as an array, not a linked list. >> Although resizing is O(n), appending to it is amortized O(1), >> because resizes happen very rarely. > https://stackoverflow.com/a/5932364/502187 > > The list type doesn't have an O(1) operation to remove > an element during sweep. The list type, not like its name > would suggest, in Python is an array. > > These arrays are not so expensive when you append() > an element. Because they are allocated with some excess > capacity. And they grow exponentially. > > So amortisized you can append() a lot of elements to > a Python list, which is an array. But you cannot poke so > cheaply holes into it. So if you have this scenario: > > Before: > - [ A1, .., An , B, C1, .., Cm ] > > After: > - [ A1, .., An , C1, .., Cm ] > > You have to copy C1,..,Cm one position down. On the other > hand, when scanning the single list, removing the > element is just pointer swizzling. > > The code is here, the positive if-then-else branch keeps > the element, the negative if-then-else branch drops the > element. Thats quite standard algorithm for linked lists: > > /* pointer swizzling */ > while temp is not None: > term = temp > temp = term.tail > if (term.flags & MASK_VAR_MARK) != 0: > term.flags &= ~MASK_VAR_MARK > if back is not None: > back.tail = term > else: > trail = term > back = term > else: > term.instantiated = NotImplemented > term.tail = None > count -= 1 > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163 > > There is nothing wrong with implementing a single list > in Python. Only you never saw that one maybe. If you would > indeed use Python lists which are arrays, you would > > maybe get a much slower sweep_trail() and this would > be seen. But currently its not seen. It happens that 1000000 > of elements are sweeped, if you would do this with copy > > inside a Python list which are arrays, it would get much > more expensive, and the extremly cheap Prolog garbage > collection, as it stands now, wouldn't be that cheap anymore. > > You can try yourself. My sweep_trail() needs frequent resize, > which would be O(n) each, so that sweep_trail becomes O(n^2). > Which the current implementation its only O(n). > > Peter J. Holzer schrieb am Montag, 20. September 2021 um 13:49:49 UTC+2: >> On 2021-09-20 04:33:39 +1000, Chris Angelico wrote: >>> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse wrote: >>>> Question to Chris Angelico: If I stay with my >>>> sweep_trail(), which is the periodically scanning, >>>> I can use a single linked list. >>>> >>>> On the other hand if I would use the trigger >>>> from Python, I possibly would need a double linked >>>> list, to remove an element. >>>> >>>> Chris Angelico, is there a third option, that I have >>>> overlooked? Single linked list uses less space >>>> than double linked list, this why I go with scan. >>>> >>> >>> I don't know. I don't understand your code well enough to offer advice >>> like that, because *your code is too complicated* and not nearly clear >>> enough. >>> >>> But however it is that you're doing things, the best way is almost >>> always to directly refer to objects. Don't fiddle around with creating >>> your own concept of a doubly-linked list and a set of objects; just >>> refer directly to the objects. >> And almost certainly: Just use the builtin list type if you need a list. >> Don't build one yourself. >>> Let Python be Python, don't try to build your own language on top of >>> it. >> Well, he's writing a Prolog interpreter, so building his own language on >> top of Python is sort of the point. I think a better way to put it is >> "Don't try to write Python as if it was C". A C operation may be >> compiled to a single machine instruction which is executed in a fraction >> of a nanosecond. A Python instruction (in CPython) always includes at >> least the interpreter overhead and often several method lookups and method >> calls. You want to amortize that overhead over as much work as possible. >> >> hp >> >> -- >> _ | Peter J. Holzer | Story must make more sense than reality. >> |_|_) | | >> | | | h... at hjp.at | -- Charles Stross, "Creative writing >> __/ | http://www.hjp.at/ | challenge!" From janburse at fastmail.fm Mon Sep 20 08:21:54 2021 From: janburse at fastmail.fm (Mostowski Collapse) Date: Mon, 20 Sep 2021 14:21:54 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: In general the algorithm I am using is from a paper by Matts Carlson from SICStus Prolog. Its this paper here: Garbage Collection for Prolog Based on WAM January 1986 Karen Appleby, Mats Carlsson, Seif Haridi, Dan Sahlin https://www.researchgate.net/publication/279463524 But since you guys are so facinated with the Prolog garbage collection aspect, which is not the locus where you can do a lot of optimizations, feel free to investigate and come up with a solution. It will not change the performance of Dogelog runtime, but it could be an academic experiment neverthless. There is nothing wrong with the simgle linked list as it stands, since it has O(n) sweep_trail(). It uses a litte more storage than an array would do. Mostowski Collapse wrote: > What would be maybe possible, is to > scan the trail from the other side, and > use a first pass to determine the new > > size, and use a second pass to fill a new > array with the remaining elments. This would > be two times O(n), so it would be linear > > and not quadratic O(n^2) as when you scan > from the top and poke holes. But then something > else doesn't work so easily. Namely: > > ?? def adjust_mark(temp): > ?????? while temp is not None: > ??????? if (temp.flags & MASK_VAR_MARK) != 0: > ??????????? return temp > ??????? else: > ??????????? temp = temp.tail > ??? return temp > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L151 > > > This is needed to adjust the choice points. > If you have an array instead of a linked > listed as I have now, you would need > > to adjust array indexes instead pointers > into linked list elements. I havent come > up with an array solution yet for the trail, > > since I dont see any utility in investing too > much time with the Prolog garbage collection of > Dogelog runtime. It is only a small share > > of the execution time: > > Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 > UTC+2: > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > % Standard Python Version, Warm Run > > % ?- time(fibo(23,X)). > > % % Wall 3865 ms, gc 94 ms, 71991 lips > > % X = 46368. > > > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > % GraalVM Python Version, Warm Warm Run > > % ?- time(fibo(23,X)). > > % % Wall 695 ms, gc 14 ms, 400356 lips > > % X = 46368. > > Mostowski Collapse wrote: >> I read the following, and you should also know: >> >>> Python's [] is implemented as an array, not a linked list. >>> Although resizing is O(n), appending to it is amortized O(1), >>> because resizes happen very rarely. >> https://stackoverflow.com/a/5932364/502187 >> >> The list type doesn't have an O(1) operation to remove >> an element during sweep. The list type, not like its name >> would suggest, in Python is an array. >> >> These arrays are not so expensive when you append() >> an element. Because they are allocated with some excess >> capacity. And they grow exponentially. >> >> So amortisized you can append() a lot of elements to >> a Python list, which is an array. But you cannot poke so >> cheaply holes into it. So if you have this scenario: >> >> Before: >> ????? - [ A1, .., An , B, C1, .., Cm ] >> >> After: >> ????? - [ A1, .., An , C1, .., Cm ] >> You have to copy C1,..,Cm one position down. On the other >> hand, when scanning the single list, removing the >> element is just pointer swizzling. >> >> The code is here, the positive if-then-else branch keeps >> the element, the negative if-then-else branch drops the >> element. Thats quite standard algorithm for linked lists: >> >> ????? /* pointer swizzling */ >> ???? while temp is not None: >> ???????? term = temp >> ???????? temp = term.tail >> ???????? if (term.flags & MASK_VAR_MARK) != 0: >> ???????????? term.flags &= ~MASK_VAR_MARK >> ???????????? if back is not None: >> ???????????????? back.tail = term >> ???????????? else: >> ???????????????? trail = term >> ???????????? back = term >> ???????? else: >> ???????????? term.instantiated = NotImplemented >> ???????????? term.tail = None >> ???????????? count -= 1 >> >> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163 >> >> >> There is nothing wrong with implementing a single list >> in Python. Only you never saw that one maybe. If you would >> indeed use Python lists which are arrays, you would >> >> maybe get a much slower sweep_trail() and this would >> be seen. But currently its not seen. It happens that 1000000 >> of elements are sweeped, if you would do this with copy >> >> inside a Python list which are arrays, it would get much >> more expensive, and the extremly cheap Prolog garbage >> collection, as it stands now, wouldn't be that cheap anymore. >> >> You can try yourself. My sweep_trail() needs frequent resize, >> which would be O(n) each, so that sweep_trail becomes O(n^2). >> Which the current implementation its only O(n). >> >> Peter J. Holzer schrieb am Montag, 20. September 2021 um 13:49:49 UTC+2: >>> On 2021-09-20 04:33:39 +1000, Chris Angelico wrote: >>>> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse >>>> wrote: >>>>> Question to Chris Angelico: If I stay with my >>>>> sweep_trail(), which is the periodically scanning, >>>>> I can use a single linked list. >>>>> >>>>> On the other hand if I would use the trigger >>>>> from Python, I possibly would need a double linked >>>>> list, to remove an element. >>>>> >>>>> Chris Angelico, is there a third option, that I have >>>>> overlooked? Single linked list uses less space >>>>> than double linked list, this why I go with scan. >>>>> >>>> >>>> I don't know. I don't understand your code well enough to offer advice >>>> like that, because *your code is too complicated* and not nearly clear >>>> enough. >>>> >>>> But however it is that you're doing things, the best way is almost >>>> always to directly refer to objects. Don't fiddle around with creating >>>> your own concept of a doubly-linked list and a set of objects; just >>>> refer directly to the objects. >>> And almost certainly: Just use the builtin list type if you need a list. >>> Don't build one yourself. >>>> Let Python be Python, don't try to build your own language on top of >>>> it. >>> Well, he's writing a Prolog interpreter, so building his own language on >>> top of Python is sort of the point. I think a better way to put it is >>> "Don't try to write Python as if it was C". A C operation may be >>> compiled to a single machine instruction which is executed in a fraction >>> of a nanosecond. A Python instruction (in CPython) always includes at >>> least the interpreter overhead and often several method lookups and >>> method >>> calls. You want to amortize that overhead over as much work as possible. >>> >>> hp >>> >>> -- >>> _ | Peter J. Holzer | Story must make more sense than reality. >>> |_|_) | | >>> | | | h... at hjp.at | -- Charles Stross, "Creative writing >>> __/ | http://www.hjp.at/ | challenge!" > From bursejan at gmail.com Mon Sep 20 08:35:49 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Mon, 20 Sep 2021 05:35:49 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: The sweep_trail() is not an issue. There must be a bottleneck somewhere else in Python. The sweep_trail() respectively the paremt call gc() only takes a very small fraction of the runtime: Check the "gc" timing, the bottleneck is somewhere else? Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2: > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % Standard Python Version, Warm Run > % ?- time(fibo(23,X)). > % % Wall 3865 ms, gc 94 ms, 71991 lips > % X = 46368. > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > % GraalVM Python Version, Warm Warm Run > % ?- time(fibo(23,X)). > % % Wall 695 ms, gc 14 ms, 400356 lips > % X = 46368. Also my code is not C-style. If I would use C-style code, I would use address calculations and the adress operator &. But you don't find and according C-style in the Python or JavaScript code. Also there is no substitute for such coding style in the for of value holders or some such. Its all plain Python respectively JavaScript not at all inspired by the C programming language. The single linked list is not some indicative of C programming language style. With C programming language sytle one would do other tricks, you cannot read off from my Python or JavaScript code, since I cannot apply them to Python or JavaScript. Among the other C programming language tricks not available in Python or JavaScript would for example be inlining the args in Compound and so on. But I am not sure whether this is the bottleneck. Chris Angelico schrieb am Montag, 20. September 2021 um 14:25:12 UTC+2: > On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer wrote: > > > Let Python be Python, don't try to build your own language on top of > > > it. > > > > Well, he's writing a Prolog interpreter, so building his own language on > > top of Python is sort of the point. I think a better way to put it is > > "Don't try to write Python as if it was C". > Fair point. Or combining them both: Writing a language interpreter in > Python as if you were writing it in C, and then complaining that it is > slow, is only going to elicit "well uhh yes?" responses. > > Languages like NetRexx (and, I think, Jython, although I can't find > any definitive and current answers) are slightly different from their > "parent" languages, because they make good use of their implementation > languages' features. This Prolog interpreter might not even need to be > different in functionality, but its implementation would be different, > and it could take advantage of the underlying garbage collection. > > ChrisA From bursejan at gmail.com Mon Sep 20 08:42:49 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Mon, 20 Sep 2021 05:42:49 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: Also I am not a C programmer. The last time I was programming C was 30 years ago. I am mostly a Java programmer the recent years. Dogelog Runtime adopts a lot of implementation details from Jekejeke Prolog which is a Prolog written in Java. The difference betweeen the two is that Jekejeke Prolog has another Prolog term model where Prolog terms are passed around as molecs, which are pairs of skeleton and display. On the other in Dogelog Runtime uses a simpler representation, Prolog terms are passed around as only one object. Programming language wise the difference between using Java and JavaScript or Python, is that Java has types. So variables need a type declaration. Otherwise Java is very similar to JavaScript or Python, it also provides a runtime with a garbage collection. The idea I would use C-style is a little absurd. It would also require that a free() objects manually. But sweep_trail() has nothing to do with freeing objects manually, its the anti-thesis to freeing objects manually, its a Prolog garbage collector after all! Mostowski Collapse schrieb am Montag, 20. September 2021 um 14:36:01 UTC+2: > The sweep_trail() is not an issue. There must be a bottleneck > somewhere else in Python. The sweep_trail() respectively the > paremt call gc() only takes a very small fraction of the runtime: > > Check the "gc" timing, the bottleneck is somewhere else? > Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2: > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > % Standard Python Version, Warm Run > > % ?- time(fibo(23,X)). > > % % Wall 3865 ms, gc 94 ms, 71991 lips > > % X = 46368. > > > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > % GraalVM Python Version, Warm Warm Run > > % ?- time(fibo(23,X)). > > % % Wall 695 ms, gc 14 ms, 400356 lips > > % X = 46368. > Also my code is not C-style. If I would use C-style code, I would > use address calculations and the adress operator &. But you > don't find and according C-style in the Python or JavaScript code. > > Also there is no substitute for such coding style in the for > of value holders or some such. Its all plain Python respectively > JavaScript not at all inspired by the C programming language. > > The single linked list is not some indicative of C programming > language style. With C programming language sytle one would > do other tricks, you cannot read off from my Python or JavaScript > > code, since I cannot apply them to Python or JavaScript. Among > the other C programming language tricks not available in Python > or JavaScript would for example be inlining the args in Compound > > and so on. But I am not sure whether this is the bottleneck. > Chris Angelico schrieb am Montag, 20. September 2021 um 14:25:12 UTC+2: > > On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer wrote: > > > > Let Python be Python, don't try to build your own language on top of > > > > it. > > > > > > Well, he's writing a Prolog interpreter, so building his own language on > > > top of Python is sort of the point. I think a better way to put it is > > > "Don't try to write Python as if it was C". > > Fair point. Or combining them both: Writing a language interpreter in > > Python as if you were writing it in C, and then complaining that it is > > slow, is only going to elicit "well uhh yes?" responses. > > > > Languages like NetRexx (and, I think, Jython, although I can't find > > any definitive and current answers) are slightly different from their > > "parent" languages, because they make good use of their implementation > > languages' features. This Prolog interpreter might not even need to be > > different in functionality, but its implementation would be different, > > and it could take advantage of the underlying garbage collection. > > > > ChrisA From bursejan at gmail.com Mon Sep 20 08:52:08 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Mon, 20 Sep 2021 05:52:08 -0700 (PDT) Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: Now I am expecting somebody telling me I should not program Java style in Python. So I wonder what would have happened if I would tell you I am FORTRAN programmer. Then somebody would tell me I should not program FORTRAN style in Python. Hopefully this makes it evident that this argumentation is moot. Better would be more facts on the table based on the existing code of Dogelog runtime. The future Dogelog runtime code will possibly use ast.parse() and compile(). Thats why I am conducting this experiment which has only been half way completed. The experiment is conducted because Python belongs to the champ of dynamic languages: Dynamic programming language https://en.wikipedia.org/wiki/Dynamic_programming_language The experiment will be completed when the ast.parse() and compile() thingy was explored as well. As it happens I am conducting the experiment in parallel for JavaScript and Python, both being dynamic programming languages. Mostowski Collapse schrieb am Montag, 20. September 2021 um 14:43:01 UTC+2: > Also I am not a C programmer. The last time I was programming C > was 30 years ago. I am mostly a Java programmer the recent years. > Dogelog Runtime adopts a lot of implementation details from > > Jekejeke Prolog which is a Prolog written in Java. The difference > betweeen the two is that Jekejeke Prolog has another Prolog term > model where Prolog terms are passed around as molecs, which > > are pairs of skeleton and display. On the other in Dogelog Runtime > uses a simpler representation, Prolog terms are passed around > as only one object. Programming language wise the difference > > between using Java and JavaScript or Python, is that Java has > types. So variables need a type declaration. Otherwise Java is > very similar to JavaScript or Python, it also provides a runtime with > > a garbage collection. The idea I would use C-style is a little absurd. It > would also require that a free() objects manually. But sweep_trail() has > nothing to do with freeing objects manually, its the anti-thesis to > > freeing objects manually, its a Prolog garbage collector after all! > Mostowski Collapse schrieb am Montag, 20. September 2021 um 14:36:01 UTC+2: > > The sweep_trail() is not an issue. There must be a bottleneck > > somewhere else in Python. The sweep_trail() respectively the > > paremt call gc() only takes a very small fraction of the runtime: > > > > Check the "gc" timing, the bottleneck is somewhere else? > > Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2: > > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > % Standard Python Version, Warm Run > > > % ?- time(fibo(23,X)). > > > % % Wall 3865 ms, gc 94 ms, 71991 lips > > > % X = 46368. > > > > > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > % GraalVM Python Version, Warm Warm Run > > > % ?- time(fibo(23,X)). > > > % % Wall 695 ms, gc 14 ms, 400356 lips > > > % X = 46368. > > Also my code is not C-style. If I would use C-style code, I would > > use address calculations and the adress operator &. But you > > don't find and according C-style in the Python or JavaScript code. > > > > Also there is no substitute for such coding style in the for > > of value holders or some such. Its all plain Python respectively > > JavaScript not at all inspired by the C programming language. > > > > The single linked list is not some indicative of C programming > > language style. With C programming language sytle one would > > do other tricks, you cannot read off from my Python or JavaScript > > > > code, since I cannot apply them to Python or JavaScript. Among > > the other C programming language tricks not available in Python > > or JavaScript would for example be inlining the args in Compound > > > > and so on. But I am not sure whether this is the bottleneck. > > Chris Angelico schrieb am Montag, 20. September 2021 um 14:25:12 UTC+2: > > > On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer wrote: > > > > > Let Python be Python, don't try to build your own language on top of > > > > > it. > > > > > > > > Well, he's writing a Prolog interpreter, so building his own language on > > > > top of Python is sort of the point. I think a better way to put it is > > > > "Don't try to write Python as if it was C". > > > Fair point. Or combining them both: Writing a language interpreter in > > > Python as if you were writing it in C, and then complaining that it is > > > slow, is only going to elicit "well uhh yes?" responses. > > > > > > Languages like NetRexx (and, I think, Jython, although I can't find > > > any definitive and current answers) are slightly different from their > > > "parent" languages, because they make good use of their implementation > > > languages' features. This Prolog interpreter might not even need to be > > > different in functionality, but its implementation would be different, > > > and it could take advantage of the underlying garbage collection. > > > > > > ChrisA From ast at invalid Mon Sep 20 10:08:41 2021 From: ast at invalid (ast) Date: Mon, 20 Sep 2021 16:08:41 +0200 Subject: Inheriting from str Message-ID: <614895e9$0$28598$426a74cc@news.free.fr> Hello class NewStr(str): def __init__(self, s): self.l = len(s) Normaly str is an immutable type so it can't be modified after creation with __new__ But the previous code is working well obj = NewStr("qwerty") obj.l 6 I don't understand why it's working ? (python 3.9) From jon+usenet at unequivocal.eu Mon Sep 20 10:27:13 2021 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 20 Sep 2021 14:27:13 -0000 (UTC) Subject: Inheriting from str References: <614895e9$0$28598$426a74cc@news.free.fr> Message-ID: On 2021-09-20, ast wrote: > Hello > > class NewStr(str): > def __init__(self, s): > self.l = len(s) > > Normaly str is an immutable type so it can't be modified > after creation with __new__ > > But the previous code is working well > > obj = NewStr("qwerty") > obj.l > 6 > > I don't understand why it's working ? The string itself is immutable. If it's a subclass then it may have attributes that are not immutable: >>> s = 'hello' >>> s.jam = 3 Traceback (most recent call last): File "", line 1, in AttributeError: 'str' object has no attribute 'jam' >>> class foo(str): pass ... >>> s = foo('hello') >>> s.jam = 3 >>> s.jam 3 There's a lot of places in Python where you can break standard assumptions with sufficiently evil or badly-written classes. e.g.: >>> class intt(int): ... def __add__(self, other): ... return int(self) + int(other) * 2 ... >>> a = intt(2) >>> b = intt(3) >>> a + b 8 >>> b + a 7 From tjreedy at udel.edu Mon Sep 20 12:46:24 2021 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 20 Sep 2021 12:46:24 -0400 Subject: Explaining exec(globals, separate_locals) Message-ID: The second paragraph of the current exec entry https://docs.python.org/3.11/library/functions.html#exec ends with a sentence I wrote. "If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition." Would the following would be clearer? "If exec gets two separate objects as globals and locals, the code will be executed in two namespaces, the same as is done with the suite of a class statements." In either case, does the following help even more? "(The difference is that the locals for exec is not passed to type() to become a class dict.)" I am asking because the current sentence engenders questions. Today I got a private email with "This behavior seems really peculiar to me, and I would love to know if you know about why this behavior was chosen to be as it is." The answer is that the choice is made by the user by what the user passes. Top level code is executed in one namespace serving as both globals and locals. Class definition code is executed in two separate namespaces serving the two role. The populated local namespace is then passed to type() to become the __dict__ of the resulting class. An interpreter written in Python could use exec for the two cases. IDLE, for instance, uses exec, with *one* namespace, to execute user code as toplevel code. The third type of code is function code, which executes in 2 *or more* namespaces, with peculiar locals behavior, and with 'return', 'yield', and 'nonlocal' valid. It would make no sense to treat the code passed to exec as a function suite. -- Terry Jan Reedy From rosuav at gmail.com Mon Sep 20 13:53:10 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 21 Sep 2021 03:53:10 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <90e326fc-dc3d-43ff-a8ea-630f4740c8e2n@googlegroups.com> <7232cc7f-52d3-4af8-99c2-d952752cdf09n@googlegroups.com> <58398d5e-6b73-4a50-b52b-0e943bc543f4n@googlegroups.com> <924df9eb-fb4f-4773-b81c-23e7209368ean@googlegroups.com> <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Tue, Sep 21, 2021 at 3:51 AM Mostowski Collapse wrote: > > sympy also builds a language on top of Python. > pandas also builds a language on top of Python. > > Is there some pope that says this wouldn't be > allowed, I dont think so, otherwise sympy, pandas, etc.. > > wouldn't exist. I dont understand your argument. > That's not the same thing as reimplementing your own low-level features on top of a high level language. ChrisA From rosuav at gmail.com Mon Sep 20 14:09:22 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 21 Sep 2021 04:09:22 +1000 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: <6a2ecba4-0558-4784-a9ae-edefca830351n@googlegroups.com> Message-ID: On Tue, Sep 21, 2021 at 3:58 AM Mostowski Collapse wrote: > > I read the following, and you should also know: > > > Python's [] is implemented as an array, not a linked list. > > Although resizing is O(n), appending to it is amortized O(1), > > because resizes happen very rarely. > https://stackoverflow.com/a/5932364/502187 > > The list type doesn't have an O(1) operation to remove > an element during sweep. The list type, not like its name > would suggest, in Python is an array. > > These arrays are not so expensive when you append() > an element. Because they are allocated with some excess > capacity. And they grow exponentially. > > So amortisized you can append() a lot of elements to > a Python list, which is an array. But you cannot poke so > cheaply holes into it. So if you have this scenario: > > Before: > - [ A1, .., An , B, C1, .., Cm ] > > After: > - [ A1, .., An , C1, .., Cm ] > > You have to copy C1,..,Cm one position down. On the other > hand, when scanning the single list, removing the > element is just pointer swizzling. > > The code is here, the positive if-then-else branch keeps > the element, the negative if-then-else branch drops the > element. Thats quite standard algorithm for linked lists: > > /* pointer swizzling */ > while temp is not None: > term = temp > temp = term.tail > if (term.flags & MASK_VAR_MARK) != 0: > term.flags &= ~MASK_VAR_MARK > if back is not None: > back.tail = term > else: > trail = term > back = term > else: > term.instantiated = NotImplemented > term.tail = None > count -= 1 > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163 > > There is nothing wrong with implementing a single list > in Python. Only you never saw that one maybe. If you would > indeed use Python lists which are arrays, you would > maybe get a much slower sweep_trail() and this would > be seen. But currently its not seen. It happens that 1000000 > of elements are sweeped, if you would do this with copy > inside a Python list which are arrays, it would get much > more expensive, and the extremly cheap Prolog garbage > collection, as it stands now, wouldn't be that cheap anymore. > > You can try yourself. My sweep_trail() needs frequent resize, > which would be O(n) each, so that sweep_trail becomes O(n^2). > Which the current implementation its only O(n). > How about, instead: Use a Python list, and instead of removing individual items one by one, filter out the ones you don't want, using a list comprehension? That would be O(n) to completely remove all the ones you don't want, instead of O(n) for each individual removal. Also, have you actually benchmarked a version that uses Python's lists, or are you simply assuming that the removals will be slow? Implementing your own singly-linked list was clearly suboptimal, but have you tried writing simpler code and seeing if it is also faster? ChrisA From hjp-python at hjp.at Mon Sep 20 15:51:44 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 20 Sep 2021 21:51:44 +0200 Subject: ANN: Dogelog Runtime, Prolog to the Moon (2021) In-Reply-To: References: Message-ID: On 2021-09-20 05:08:55 -0700, Mostowski Collapse wrote: > You have to copy C1,..,Cm one position down. On the other > hand, when scanning the single list, removing the > element is just pointer swizzling. That's the problem. You think that pointer swizzling is fast because you are used to languages where this is compiled into a few machine instructions. (Even there it depends a lot on locality: A random memory access can take hundreds of clock cycles.) In Python something like ?a.next = b.next? normally does a dict lookup on both objects, plus it also has to check for the existence of special methods on both objects. You can probably copy quite a few contiguous bytes while this is happening. Here is a simple demo: ------------------------------------------------------------------------ #!/usr/bin/python3 import time n = 10000 def build_linked_list(n): head = [None, None] tail = head for i in range(n): tail[1] = [i, None] tail = tail[1] return head def empty_list(ls): while ls[1]: ls[1] = ls[1][1] t0 = time.monotonic() ll = build_linked_list(n) t1 = time.monotonic() empty_list(ll) t2 = time.monotonic() print(n, t1 - t0, t2 - t1) ------------------------------------------------------------------------ #!/usr/bin/python3 import time n = 10000 def build_list(n): return list(range(n)) def empty_list(ls): while ls: ls.pop(0) t0 = time.monotonic() ll = build_list(n) t1 = time.monotonic() empty_list(ll) t2 = time.monotonic() print(n, t1 - t0, t2 - t1) ------------------------------------------------------------------------ Both scripts create a list of n integers, then repeatedly pop off the first element until the list is empty. The first uses a linked list (where each element is a Python list - I guess this is faster than an object although I haven't timed it) the second a python list. Popping off the first element is the worst case for a python list since it is implemented as an array of pointers and n-1 pointers have to be copied for each operation and the best case for a linked list. Still, on my laptop with CPython 3.8, the builtin list is faster for lists of less than 100 elements, they are about on par between 100 and 1000 elements and only for longer lists does the linked list become faster. With Pypy 6.0 (yes, I know it's old), the linked list seems to be always a bit faster although the difference is very small for lists shorter than 100 elements - GraalVM is probably closer to PyPy than to CPython in its performance characteristics, but the point is that the dynamic nature of Python makes some seemingly cheap operations much more costly than one would expect - a good optimizing compiler may be able to detect that all that is not needed in a specific case and produce straightforward code. But unlike for JavaScript (which has a very similar problem) no company has spent millions of dollars on an optimizing JIT compiler for Python yet. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From eryksun at gmail.com Mon Sep 20 16:23:59 2021 From: eryksun at gmail.com (Eryk Sun) Date: Mon, 20 Sep 2021 15:23:59 -0500 Subject: Explaining exec(globals, separate_locals) In-Reply-To: References: Message-ID: On 9/20/21, Terry Reedy wrote: > > "If exec gets two separate objects as globals and locals, the code will > be executed as if it were embedded in a class definition." Note that, unlike exec(), the body of a class definition can modify closure variables in nonlocal function scopes. For example: >>> def f(): ... x = 'spam' ... class C: ... nonlocal x ... x = 'eggs' ... return x ... >>> f() 'eggs' From joel.dcosta83 at gmail.com Mon Sep 20 16:50:57 2021 From: joel.dcosta83 at gmail.com (Python 4 Fun) Date: Mon, 20 Sep 2021 13:50:57 -0700 (PDT) Subject: Generate Popular Tags for instagram or facebook Message-ID: Before we proceed let me make it clear we are not scraping tags from any website. We will "Generate" them using simple code in python. So, How does it work? First, I have collected all popular tags and saved it in a text file as list. [Read More] https://pysnakeblog.blogspot.com/2021/09/generate-popular-tags-for-instagram-or.html From fabdelmalk at thecontactcentre.ae Tue Sep 21 00:42:55 2021 From: fabdelmalk at thecontactcentre.ae (Fady Victor Mikhael Abdelmalk) Date: Tue, 21 Sep 2021 04:42:55 +0000 Subject: issue for setup pandas In-Reply-To: References: Message-ID: Dear Python Team, I got the below issue when trying to install python on my user. Kindly assist to know how can I solved. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /simple/pandas/ WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /simple/pandas/ WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /simple/pandas/ Thanks, Fady Victor 0508510414 ________________________________ Emirates Telecommunications Group Company PJSC (Etisalat) is a Public Joint Stock Company registered in the UAE. This email (and its attachments) contains confidential information. Do not use, copy or disclose the information contained in the email or in any attachment. If you are not the intended recipient, please permanently delete it. Etisalat does not conclude agreements through email communications and nothing in this email shall be construed or interpreted as binding Etisalat or creating any obligation on Etisalat. Although Etisalat has taken reasonable precautions to ensure no viruses are present in this email, Etisalat cannot accept liability and responsibility for any loss or damage sustained as a result of computer viruses or malwares and the recipient must ensure that the email (and attachments) are virus free. From mohsen.owzar at gmail.com Tue Sep 21 04:36:24 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Tue, 21 Sep 2021 01:36:24 -0700 (PDT) Subject: Free OCR package in Python and selecting appropriate widget for the GUI Message-ID: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> Hi Guys Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. Now I try to write this GUI with Python with PyQt5 or TKinter. First question is: Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. Second: Because, I can not attach a picture to this post, I try to describe my picture of my GUI. It is a 3x3 block / matrix (one third of the whole Sudoku 9x9 block). This block must be placed three times in row and columns. Each square of this 3x3 block has 3x3 digits from 1 to 9 at the initialization time. These digits are spread out in 3x3 equal distances. These are small fonts and black. The values given by the puzzle are red big fonts and not changeable The digits typed by the user are black big fonts and changeable If there is a big font from the Puzzle / User, these must be removed from the neighboring fields from the 3x3 matrix (1-9). Now my question is, actually a hint from your side: What should I take as widget for one of these small squares to have different content as I described above? I thought of a QLineEdit in PyQt5 or LineEdit in TKinter, because the user must type some values in there and therefore can not be labels. So again, for the sake of the clarity. ? When the fields are empty, each square has to show digits from 1 to 9 with small fonts in 3x3 matrix order. ? Given values in the puzzle are big fonts and red. ? Digits which are typed by the user are big fonts and black. Now the question is, can I take only one LineEdit for all these tree situations, or I have to implement 9 small squares for small fonts and a big one for the big fonts? Any help and suggestion is welcome and appreciated. Best regards Mohsen From robin at reportlab.com Tue Sep 21 08:08:00 2021 From: robin at reportlab.com (Robin Becker) Date: Tue, 21 Sep 2021 13:08:00 +0100 Subject: c extension finding the module in object initialization Message-ID: <05588d73-29c0-825e-82ae-7989d90cf31f@everest.reportlab.co.uk> I have a c extension which is intended to implement a module the looks structurally like this ############ a = 1 b = 2 class T: def __init__(self): self.a = a self.b = b ############ so when an object of type T is instantiated it can set up defaults based on the current module values of a and b. In the past using old style single phase module creation the init function for the type could look up the module by using the PyState_FindModule function. In the new world where we can implement c extensions which might work with multiple interpreters (multi-phase creation). The docs say PyState_FindModule won't work for those and indeed it returns NULL so is useless. Is there a way I can set the current module onto the type definition during the module's exec function? I thought it would be easy to add at the point in the exec where I'm doing this to the module, m, TType.tp_base = &PyBaseObject_Type; if(PyType_Ready(&TType)<0) goto fail; if(PyModule_AddObject(m,"T", (PyObject *)&TType)<0) goto fail; but I don't see the place in the type where I can add these sorts of dynamic attributes. The basic_size is for the created objects. The created type does have a __dict__ which is a mappingproxy. I wondered when that actually gets instantiated. I can see that the type has a __module__ attribute after the module is imported and I suppose if I can find the 'current' interpreter I might use the __module__ to locate the module object via PyImport_GetModuleDict. Any expertise or advice gratefully received. -- Robin Becker From nospam at dfs.com Tue Sep 21 09:45:17 2021 From: nospam at dfs.com (DFS) Date: Tue, 21 Sep 2021 09:45:17 -0400 Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> Message-ID: On 9/21/2021 4:36 AM, Mohsen Owzar wrote: > Hi Guys > Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. > Now I try to write this GUI with Python with PyQt5 or TKinter. > First question is: > Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. > Second: > Because, I can not attach a picture to this post, I try to describe my picture of my GUI. Draw your GUI in PyQt designer or other graphics tool, then upload a screenshot of it to imgur, then post the link to the picture. From rosuav at gmail.com Tue Sep 21 09:59:12 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 21 Sep 2021 23:59:12 +1000 Subject: issue for setup pandas In-Reply-To: References: Message-ID: On Tue, Sep 21, 2021 at 11:53 PM Fady Victor Mikhael Abdelmalk wrote: > > > Dear Python Team, > > I got the below issue when trying to install python on my user. Kindly assist to know how can I solved. > > > WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /simple/pandas/ > WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /simple/pandas/ > WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /simple/pandas/ > Looks like a problem with your internet connection. When you try to install things like pandas, they have to be downloaded from the internet. If your firewall is blocking this, you'll have to grant permission before the installation can continue. ChrisA From dieter at handshake.de Tue Sep 21 14:00:25 2021 From: dieter at handshake.de (Dieter Maurer) Date: Tue, 21 Sep 2021 20:00:25 +0200 Subject: Explaining exec(globals, separate_locals) In-Reply-To: References: Message-ID: <24906.7609.611924.329425@ixdm.fritz.box> Terry Reedy wrote at 2021-9-20 12:46 -0400: >The second paragraph of the current exec entry >https://docs.python.org/3.11/library/functions.html#exec >ends with a sentence I wrote. > >"If exec gets two separate objects as globals and locals, the code will >be executed as if it were embedded in a class definition." > >Would the following would be clearer? > >"If exec gets two separate objects as globals and locals, the code will >be executed in two namespaces, the same as is done with the suite of a >class statements." It is better but the reference to class statements does not help (at least me) very much. Instead, I would try to explain how those namespaces are used, e.g. bindings are stored in the "local" namespace; lookups are first performed in the "local", then the "global" namespace, etc. (e.g. about the effect of `global`, `nonlocal`). -- Dieter From travisgriggs at gmail.com Tue Sep 21 14:58:31 2021 From: travisgriggs at gmail.com (Travis Griggs) Date: Tue, 21 Sep 2021 18:58:31 +0000 Subject: Polymorphic imports Message-ID: I guess this is kind of like mocking for testing. I have a simple module that's imported in a number of other spots in my program. There's a condition in the OS/filesystem where I'd like to import a polymorphically compatible variant of the same module. Can this be accomplished in a sort of once-and-only once spot? For example, consider something like this: client/ module_a module_a_prime lib/ paths lib_a lib_b ... model/ model_a model_b ... top_level_a top_level_b ... I have a number of imports of module_a. I have a paths module that isolates all of my file system access, and that's where the determination can be made which one to use, so I tried to do something like: def dynamic_client_module(): return client.module_a_prime if the_condition_occurs else client.module_a Hoping that I could do something like from lib import paths import paths.dynamic_client_module() But this seems to not work. Import can only take real modules? Not programatic ones? Is there a Not-Too-Evil-Way(tm) to add a level of programmatic indirection in the import declarations? Or some other trick from a different angle? From rosuav at gmail.com Tue Sep 21 15:10:02 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 22 Sep 2021 05:10:02 +1000 Subject: Polymorphic imports In-Reply-To: References: Message-ID: On Wed, Sep 22, 2021 at 4:59 AM Travis Griggs wrote: > > I guess this is kind of like mocking for testing. I have a simple module that's imported in a number of other spots in my program. There's a condition in the OS/filesystem where I'd like to import a polymorphically compatible variant of the same module. Can this be accomplished in a sort of once-and-only once spot? > > For example, consider something like this: > > client/ > module_a > module_a_prime > lib/ > paths > lib_a > lib_b > ... > model/ > model_a > model_b > ... > top_level_a > top_level_b > ... > > > I have a number of imports of module_a. I have a paths module that isolates all of my file system access, and that's where the determination can be made which one to use, so I tried to do something like: > > def dynamic_client_module(): > return client.module_a_prime if the_condition_occurs else client.module_a > > > Hoping that I could do something like > > from lib import paths > import paths.dynamic_client_module() > > But this seems to not work. Import can only take real modules? Not programatic ones? > > Is there a Not-Too-Evil-Way(tm) to add a level of programmatic indirection in the import declarations? Or some other trick from a different angle? You can dynamically import modules using importlib.import_module(), but an easier way might just be a conditional import: # client/__init__.py if some_condition: import module_a_default as module_a else: import module_a_prime as module_a Now everything that refers to client.module_a.whatever will get the appropriate one, either the original or the alternate. Alternatively, since you are talking about paths, it might be easiest to give everything the same name, and then use sys.path to control your import directories. Not sure which would work out best. ChrisA From 2QdxY4RzWzUUiLuE at potatochowder.com Tue Sep 21 15:42:30 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Tue, 21 Sep 2021 12:42:30 -0700 Subject: Polymorphic imports In-Reply-To: References: Message-ID: On 2021-09-22 at 05:10:02 +1000, Chris Angelico wrote: > You can dynamically import modules using importlib.import_module(), > but an easier way might just be a conditional import: > > # client/__init__.py > if some_condition: > import module_a_default as module_a > else: > import module_a_prime as module_a > > Now everything that refers to client.module_a.whatever will get the > appropriate one, either the original or the alternate. +1 > Alternatively, since you are talking about paths, it might be easiest > to give everything the same name, and then use sys.path to control > your import directories. Not sure which would work out best. -1 Please don't do that. Mutable shared and/or global state (i.e., sys.paths) is the root of all evil. And homegrown crypto and date libraries. And those funny red hats. From rosuav at gmail.com Tue Sep 21 16:23:43 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 22 Sep 2021 06:23:43 +1000 Subject: Polymorphic imports In-Reply-To: References: Message-ID: On Wed, Sep 22, 2021 at 6:05 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > > On 2021-09-22 at 05:10:02 +1000, > Chris Angelico wrote: > > > You can dynamically import modules using importlib.import_module(), > > but an easier way might just be a conditional import: > > > > # client/__init__.py > > if some_condition: > > import module_a_default as module_a > > else: > > import module_a_prime as module_a > > > > Now everything that refers to client.module_a.whatever will get the > > appropriate one, either the original or the alternate. > > +1 > > > Alternatively, since you are talking about paths, it might be easiest > > to give everything the same name, and then use sys.path to control > > your import directories. Not sure which would work out best. > > -1 > > Please don't do that. Mutable shared and/or global state (i.e., > sys.paths) is the root of all evil. And homegrown crypto and date > libraries. And those funny red hats. All depends on whether this is a script/application or a library. If it's a library, then I agree, don't mutate sys.path, don't change the working directory, etc, etc, etc. But applications are free to do those sorts of things. I don't know what the OP's purpose here is, and it's entirely possible that sys.path switching is the cleanest way to do it. ChrisA From michael.stemper at gmail.com Tue Sep 21 14:12:10 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 21 Sep 2021 13:12:10 -0500 Subject: XML Considered Harmful Message-ID: On the prolog thread, somebody posted a link to: One thing that it tangentially says is "XML is not the answer." I read this page right when I was about to write an XML parser to get data into the code for a research project I'm working on. It seems to me that XML is the right approach for this sort of thing, especially since the data is hierarchical in nature. Does the advice on that page mean that I should find some other way to get data into my programs, or does it refer to some kind of misuse/abuse of XML for something that it wasn't designed for? If XML is not the way to package data, what is the recommended approach? -- Michael F. Stemper Life's too important to take seriously. From jon+usenet at unequivocal.eu Tue Sep 21 14:42:33 2021 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 21 Sep 2021 18:42:33 -0000 (UTC) Subject: XML Considered Harmful References: Message-ID: On 2021-09-21, Michael F. Stemper wrote: > On the prolog thread, somebody posted a link to: > > > One thing that it tangentially says is "XML is not the answer." > > I read this page right when I was about to write an XML parser > to get data into the code for a research project I'm working on. > It seems to me that XML is the right approach for this sort of > thing, especially since the data is hierarchical in nature. > > Does the advice on that page mean that I should find some other > way to get data into my programs, or does it refer to some kind > of misuse/abuse of XML for something that it wasn't designed > for? > > If XML is not the way to package data, what is the recommended > approach? I'd agree that you should not use XML unless the data is being supplied already in XML format or perhaps if there is already a schema defined in XML for exactly your purpose. If there is nothing pre-existing to build upon then I'd suggest JSON. If anyone suggests YAML, then you should just back slowly away while speaking in a low calm voice until you have reached sufficient safe distance, then turn and run. From alister.ware at ntlworld.com Tue Sep 21 14:49:30 2021 From: alister.ware at ntlworld.com (alister) Date: Tue, 21 Sep 2021 18:49:30 -0000 (UTC) Subject: XML Considered Harmful References: Message-ID: On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote: > On the prolog thread, somebody posted a link to: > > > One thing that it tangentially says is "XML is not the answer." > > I read this page right when I was about to write an XML parser to get > data into the code for a research project I'm working on. > It seems to me that XML is the right approach for this sort of thing, > especially since the data is hierarchical in nature. > > Does the advice on that page mean that I should find some other way to > get data into my programs, or does it refer to some kind of misuse/abuse > of XML for something that it wasn't designed for? > > If XML is not the way to package data, what is the recommended approach? 1'st can I say don't write your own XML parser, there are already a number of existing parsers that should do everything you will need. This is a wheel that does not need re-inventing. 2nd if you are not generating the data then you have to use whatever data format you are supplied as far as I can see the main issue with XML is bloat, it tries to do too many things & is a very verbose format, often the quantity of mark-up can easily exceed the data contained within it. other formats such a JSON & csv have far less overhead, although again not always suitable. As in all such cases it is a matter of choosing the most apropriate tool for the job in hand. -- Antonym, n.: The opposite of the word you're trying to think of. From michael.stemper at gmail.com Tue Sep 21 15:22:52 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 21 Sep 2021 14:22:52 -0500 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 21/09/2021 13.49, alister wrote: > On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote: > >> On the prolog thread, somebody posted a link to: >> >> >> One thing that it tangentially says is "XML is not the answer." >> >> I read this page right when I was about to write an XML parser to get >> data into the code for a research project I'm working on. >> It seems to me that XML is the right approach for this sort of thing, >> especially since the data is hierarchical in nature. >> >> Does the advice on that page mean that I should find some other way to >> get data into my programs, or does it refer to some kind of misuse/abuse >> of XML for something that it wasn't designed for? >> >> If XML is not the way to package data, what is the recommended approach? > > 1'st can I say don't write your own XML parser, there are already a > number of existing parsers that should do everything you will need. This > is a wheel that does not need re-inventing. I was going to build it on top of xml.etree.ElementTree > 2nd if you are not generating the data then you have to use whatever data > format you are supplied It's my own research, so I can give myself the data in any format that I like. > as far as I can see the main issue with XML is bloat, it tries to do too > many things & is a very verbose format, often the quantity of mark-up can > easily exceed the data contained within it. > > other formats such a JSON & csv have far less overhead, although again > not always suitable. I've heard of JSON, but never done anything with it. How does CSV handle hierarchical data? For instance, I have generators[1], each of which has a name, a fuel and one or more incremental heat rate curves. Each fuel has a name, UOM, heat content, and price. Each incremental cost curve has a name, and a series of ordered pairs (representing a piecewise linear curve). Can CSV files model this sort of situation? > As in all such cases it is a matter of choosing the most apropriate tool > for the job in hand. Naturally. That's what I'm exploring. [1] The kind made of tons of iron and copper, filled with oil, and rotating at 1800 rpm. -- Michael F. Stemper This sentence no verb. From petef4+usenet at gmail.com Tue Sep 21 17:21:43 2021 From: petef4+usenet at gmail.com (Pete Forman) Date: Tue, 21 Sep 2021 22:21:43 +0100 Subject: XML Considered Harmful References: Message-ID: <87ilytzkt4.fsf@gmail.com> "Michael F. Stemper" writes: > On 21/09/2021 13.49, alister wrote: >> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote: > It's my own research, so I can give myself the data in any format that I > like. > >> as far as I can see the main issue with XML is bloat, it tries to do >> too many things & is a very verbose format, often the quantity of >> mark-up can easily exceed the data contained within it. other formats >> such a JSON & csv have far less overhead, although again not always >> suitable. > > I've heard of JSON, but never done anything with it. Then you should certainly try to get a basic understanding of it. One thing JSON shares with XML is that it is best left to machines to produce and consume. Because both can be viewed in a text editor there is a common misconception that they are easy to edit. Not so, commas are a common bugbear in JSON and non-trivial edits in (XML unaware) text editors are tricky. Consider what overhead you should worry about. If you are concerned about file sizes then XML, JSON and CSV should all compress to a similar size. > How does CSV handle hierarchical data? For instance, I have > generators[1], each of which has a name, a fuel and one or more > incremental heat rate curves. Each fuel has a name, UOM, heat content, > and price. Each incremental cost curve has a name, and a series of > ordered pairs (representing a piecewise linear curve). > > Can CSV files model this sort of situation? The short answer is no. CSV files represent spreadsheet row-column values with nothing fancier such as formulas or other redirections. CSV is quite good as a lowest common denominator exchange format. I say quite because I would characterize it by 8 attributes and you need to pick a dialect such as MS Excel which sets out what those are. XML and JSON are controlled much better. You can easily verify that you conform to those and guarantee that *any* conformant parser can read your content. XML is more powerful in that repect than JSON in that you can define and enforce schemas. In your case the fuel name, UOM, etc. can be validated with standard tools. In JSON all that checking is entirely handled by the consuming program(s). >> As in all such cases it is a matter of choosing the most apropriate tool >> for the job in hand. > > Naturally. That's what I'm exploring. You might also like to consider HDF5. It is targeted at large volumes of scientific data and its capabilities are well above what you need. MATLAB, Octave and Scilab use it as their native format. PyTables and h2py provide Python/NumPy bindings to it. -- Pete Forman From alister.ware at ntlworld.com Tue Sep 21 18:30:40 2021 From: alister.ware at ntlworld.com (alister) Date: Tue, 21 Sep 2021 22:30:40 -0000 (UTC) Subject: XML Considered Harmful References: Message-ID: On Tue, 21 Sep 2021 14:22:52 -0500, Michael F. Stemper wrote: > On 21/09/2021 13.49, alister wrote: >> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote: >> >>> On the prolog thread, somebody posted a link to: >>> >>> >>> One thing that it tangentially says is "XML is not the answer." >>> >>> I read this page right when I was about to write an XML parser to get >>> data into the code for a research project I'm working on. >>> It seems to me that XML is the right approach for this sort of thing, >>> especially since the data is hierarchical in nature. >>> >>> Does the advice on that page mean that I should find some other way to >>> get data into my programs, or does it refer to some kind of >>> misuse/abuse of XML for something that it wasn't designed for? >>> >>> If XML is not the way to package data, what is the recommended >>> approach? >> >> 1'st can I say don't write your own XML parser, there are already a >> number of existing parsers that should do everything you will need. >> This is a wheel that does not need re-inventing. > > I was going to build it on top of xml.etree.ElementTree > so not writing a parser, using one, that's ok >> 2nd if you are not generating the data then you have to use whatever >> data format you are supplied > > It's my own research, so I can give myself the data in any format that I > like. > >> as far as I can see the main issue with XML is bloat, it tries to do >> too many things & is a very verbose format, often the quantity of >> mark-up can easily exceed the data contained within it. >> >> other formats such a JSON & csv have far less overhead, although again >> not always suitable. > > I've heard of JSON, but never done anything with it. the python json library makes it simple. it was originally invented for javascript, it looks very much like the repl for a list/dictionary but if you are using std libraries you don't really need to know except for academic interst > > How does CSV handle hierarchical data? It dosn't, if you have heirachiacl data it is not a suitable format > For instance, I have > generators[1], each of which has a name, a fuel and one or more > incremental heat rate curves. Each fuel has a name, UOM, heat content, > and price. Each incremental cost curve has a name, and a series of > ordered pairs (representing a piecewise linear curve). > > Can CSV files model this sort of situation? > >> As in all such cases it is a matter of choosing the most apropriate >> tool for the job in hand. > > Naturally. That's what I'm exploring. > > > [1] The kind made of tons of iron and copper, filled with oil, and > rotating at 1800 rpm. -- Riches cover a multitude of woes. -- Menander From pfeiffer at cs.nmsu.edu Tue Sep 21 18:49:53 2021 From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer) Date: Tue, 21 Sep 2021 16:49:53 -0600 Subject: XML Considered Harmful References: Message-ID: <1b1r5hfsry.fsf@pfeifferfamily.net> ram at zedat.fu-berlin.de (Stefan Ram) writes: > - S expressions (i.e., LISP notation) If you're looking at hierarchical data and you don't have some good reason to use something else, this is very likely to be your simplest option. From jon+usenet at unequivocal.eu Tue Sep 21 18:58:17 2021 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 21 Sep 2021 22:58:17 -0000 (UTC) Subject: XML Considered Harmful References: <87ilytzkt4.fsf@gmail.com> Message-ID: On 2021-09-21, Pete Forman wrote: > CSV is quite good as a lowest common denominator exchange format. I say > quite because I would characterize it by 8 attributes and you need to > pick a dialect such as MS Excel which sets out what those are. XML and > JSON are controlled much better. You can easily verify that you conform > to those and guarantee that *any* conformant parser can read your > content. XML is more powerful in that repect than JSON in that you can > define and enforce schemas. In your case the fuel name, UOM, etc. can be > validated with standard tools. In JSON all that checking is entirely > handled by the consuming program(s). That's not true. You can use "JSON Schema" to create a schema for validating JSON files, and there appear to be at least four implementations in Python. From * at eli.users.panix.com Tue Sep 21 20:30:35 2021 From: * at eli.users.panix.com (Eli the Bearded) Date: Wed, 22 Sep 2021 00:30:35 -0000 (UTC) Subject: XML Considered Harmful References: Message-ID: In comp.lang.python, Michael F. Stemper wrote: > I've heard of JSON, but never done anything with it. You probably have used it inadvertantly on a regular basis over the past few years. Websites live on it. > How does CSV handle hierarchical data? For instance, I have > generators[1], each of which has a name, a fuel and one or more > incremental heat rate curves. Each fuel has a name, UOM, heat content, > and price. Each incremental cost curve has a name, and a series of > ordered pairs (representing a piecewise linear curve). > > Can CSV files model this sort of situation? Can a string of ones and zeros encode the sounds of Bach, the images of his sheet music, the details to reproduce his bust in melted plastic extruded from nozzle under the control of machines? Yes, CSV files can model that. But it would not be my first choice of data format. (Neither would JSON.) I'd probably use XML. I rather suspect that all (many) of those genomes that end up in Microsoft Excel files get there via a CSV export from a command line tool. Once you can model life in CSV, everything seems possible. > [1] The kind made of tons of iron and copper, filled with oil, and > rotating at 1800 rpm. Those are rather hard to model in CSV, too, but I'm sure it could be done. Elijah ------ for bonus round, use punched holes in paper to encode the ones and zeros From pfeiffer at cs.nmsu.edu Tue Sep 21 21:27:40 2021 From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer) Date: Tue, 21 Sep 2021 19:27:40 -0600 Subject: XML Considered Harmful References: Message-ID: <1bpmt1e6wj.fsf@pfeifferfamily.net> Eli the Bearded <*@eli.users.panix.com> writes: > In comp.lang.python, Michael F. Stemper wrote: >> I've heard of JSON, but never done anything with it. > > You probably have used it inadvertantly on a regular basis over the > past few years. Websites live on it. If the user has any interaction whatever with the formats being used to transfer data then something is very, very wrong. Someone using a website built on JSON isn't using JSON in any meaningful sense of the term. >> How does CSV handle hierarchical data? For instance, I have >> generators[1], each of which has a name, a fuel and one or more >> incremental heat rate curves. Each fuel has a name, UOM, heat content, >> and price. Each incremental cost curve has a name, and a series of >> ordered pairs (representing a piecewise linear curve). >> >> Can CSV files model this sort of situation? > > Can a string of ones and zeros encode the sounds of Bach, the images > of his sheet music, the details to reproduce his bust in melted plastic > extruded from nozzle under the control of machines? > > Yes, CSV files can model that. But it would not be my first choice of > data format. (Neither would JSON.) I'd probably use XML. > > I rather suspect that all (many) of those genomes that end up in > Microsoft Excel files get there via a CSV export from a command line > tool. Once you can model life in CSV, everything seems possible. Whenever someone asks "can this be done?" in any sort of computer related question, the real question is "is this practical?" I have hazy memories of seeing a Turing Machine implemented in an Excel spreadsheet, so *anything* can, with sufficiently ridiculous amounts of work. That's not really helpful here. >> [1] The kind made of tons of iron and copper, filled with oil, and >> rotating at 1800 rpm. > > Those are rather hard to model in CSV, too, but I'm sure it could be > done. So let's try to point him at representations that are easy. From ethan at stoneleaf.us Tue Sep 21 22:36:14 2021 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 21 Sep 2021 19:36:14 -0700 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 9/21/21 11:12 AM, Michael F. Stemper wrote: > It seems to me that XML is the right approach for this sort of > thing, especially since the data is hierarchical in nature. If you're looking for a format that you can read (as a human) and possibly hand-edit, check out NestedText: https://nestedtext.org/en/stable/ -- ~Ethan~ From drsalists at gmail.com Tue Sep 21 22:46:19 2021 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 21 Sep 2021 19:46:19 -0700 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On Tue, Sep 21, 2021 at 7:26 PM Michael F. Stemper < michael.stemper at gmail.com> wrote: > If XML is not the way to package data, what is the recommended > approach? > I prefer both JSON and YAML over XML. XML has both elements and tags, but it didn't really need both. This results in more complexity than necessary. Also, XSLT and XPath are not really all that simple. But there's hope. If you're stuck with XML, you can use xmltodict, which makes XML almost as easy as JSON. HTH. From mohsen.owzar at gmail.com Tue Sep 21 22:38:44 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Tue, 21 Sep 2021 19:38:44 -0700 (PDT) Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> Message-ID: <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: > On 9/21/2021 4:36 AM, Mohsen Owzar wrote: > > Hi Guys > > Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. > > Now I try to write this GUI with Python with PyQt5 or TKinter. > > First question is: > > Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. > > Second: > > Because, I can not attach a picture to this post, I try to describe my picture of my GUI. > Draw your GUI in PyQt designer or other graphics tool, then upload a > screenshot of it to imgur, then post the link to the picture. Thanks, for your answer. But, what is "imgur"? I'm not so familiar with handling of pictures in this group. How can I call "imgur" or how can I get there? Regards Mohsen From ikorot01 at gmail.com Tue Sep 21 23:05:24 2021 From: ikorot01 at gmail.com (Igor Korot) Date: Tue, 21 Sep 2021 22:05:24 -0500 Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> Message-ID: imgur.com Then you upload your image and post a perma-link here in reply. On Tue, Sep 21, 2021, 22:03 Mohsen Owzar wrote: > DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: > > On 9/21/2021 4:36 AM, Mohsen Owzar wrote: > > > Hi Guys > > > Long time ago I've written a program in Malab a GUI for solving Sudoku > puzzles, which worked not so bad. > > > Now I try to write this GUI with Python with PyQt5 or TKinter. > > > First question is: > > > Is there any free OCR software, packages or code in Python, which I > can use to recognize the given digits and their positions in the puzzle > square. > > > Second: > > > Because, I can not attach a picture to this post, I try to describe my > picture of my GUI. > > Draw your GUI in PyQt designer or other graphics tool, then upload a > > screenshot of it to imgur, then post the link to the picture. > Thanks, for your answer. > But, what is "imgur"? > I'm not so familiar with handling of pictures in this group. > How can I call "imgur" or how can I get there? > > Regards > Mohsen > -- > https://mail.python.org/mailman/listinfo/python-list > From roland.em0001 at googlemail.com Wed Sep 22 03:38:12 2021 From: roland.em0001 at googlemail.com (Roland Mueller) Date: Wed, 22 Sep 2021 10:38:12 +0300 Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> Message-ID: Hello, ti 21. syysk. 2021 klo 16.53 Mohsen Owzar (mohsen.owzar at gmail.com) kirjoitti: > Hi Guys > Long time ago I've written a program in Malab a GUI for solving Sudoku > puzzles, which worked not so bad. > Now I try to write this GUI with Python with PyQt5 or TKinter. > First question is: > Is there any free OCR software, packages or code in Python, which I can > use to recognize the given digits and their positions in the puzzle square. > to my knowledge there is no Python package for OCR. Using a free OCR package that has a command line interface one could integrate this into a Python script that makes a call to this external OCR e.g. using Python module subprocess. BR, Roland > Second: > Because, I can not attach a picture to this post, I try to describe my > picture of my GUI. > It is a 3x3 block / matrix (one third of the whole Sudoku 9x9 block). > This block must be placed three times in row and columns. > Each square of this 3x3 block has 3x3 digits from 1 to 9 at the > initialization time. These digits are spread out in 3x3 equal distances. > These are small fonts and black. > The values given by the puzzle are red big fonts and not changeable > The digits typed by the user are black big fonts and changeable > If there is a big font from the Puzzle / User, these must be removed from > the neighboring fields from the 3x3 matrix (1-9). > Now my question is, actually a hint from your side: > What should I take as widget for one of these small squares to have > different content as I described above? > I thought of a QLineEdit in PyQt5 or LineEdit in TKinter, because the user > must type some values in there and therefore can not be labels. So again, > for the sake of the clarity. > ? When the fields are empty, each square has to show digits from 1 > to 9 with small fonts in 3x3 matrix order. > ? Given values in the puzzle are big fonts and red. > ? Digits which are typed by the user are big fonts and black. > Now the question is, can I take only one LineEdit for all these tree > situations, or I have to implement 9 small squares for small fonts and a > big one for the big fonts? > Any help and suggestion is welcome and appreciated. > > Best regards > Mohsen > -- > https://mail.python.org/mailman/listinfo/python-list > From vinay_sajip at yahoo.co.uk Wed Sep 22 04:10:53 2021 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 22 Sep 2021 08:10:53 +0000 (UTC) Subject: ANN: distlib 0.3.3 released on PyPI References: <21569820.115373.1632298253392.ref@mail.yahoo.com> Message-ID: <21569820.115373.1632298253392@mail.yahoo.com> I've recently released version 0.3.3 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #152: Removed splituser() function which wasn't used and is deprecated. * Fixed #149: Handle version comparisons correctly in environment markers. * Add ARM-64 launchers and support code to use them. Thanks to Niyas Sait and ? Adrian Vladu for their contributions. * Fixed #148: Handle a single trailing comma following a version. Thanks to Blazej ? Floch for the report and suggested fix. * Fixed #150: Fix incorrect handling of epochs. * Reverted handling of tags for Python >= 3.10 (use 310 rather than 3_10). This is ? because PEP 641 was rejected. * Added a GitHub Actions workflow to perform tests. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.3/ [2] https://distlib.readthedocs.io/en/0.3.3/ [3] https://bitbucket.org/pypa/distlib/issues/new From davidcaul774 at gmail.com Wed Sep 22 13:08:54 2021 From: davidcaul774 at gmail.com (David Caul) Date: Wed, 22 Sep 2021 22:38:54 +0530 Subject: [Hint]: python simulator http Message-ID: Hi All, Making any simulator http request , response base simulator with some UI on browser , what are all things python package can support br David From nospam at dfs.com Tue Sep 21 23:10:18 2021 From: nospam at dfs.com (DFS) Date: Tue, 21 Sep 2021 23:10:18 -0400 Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> Message-ID: On 9/21/2021 10:38 PM, Mohsen Owzar wrote: > DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: >> On 9/21/2021 4:36 AM, Mohsen Owzar wrote: >>> Hi Guys >>> Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. >>> Now I try to write this GUI with Python with PyQt5 or TKinter. >>> First question is: >>> Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. >>> Second: >>> Because, I can not attach a picture to this post, I try to describe my picture of my GUI. >> Draw your GUI in PyQt designer or other graphics tool, then upload a >> screenshot of it to imgur, then post the link to the picture. > Thanks, for your answer. > But, what is "imgur"? > I'm not so familiar with handling of pictures in this group. > How can I call "imgur" or how can I get there? > > Regards > Mohsen www.imgur.com It's a website you can upload image files or screenshots to. Then you can copy a link to your picture and post the link here. From mohsen.owzar at gmail.com Wed Sep 22 01:54:18 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Tue, 21 Sep 2021 22:54:18 -0700 (PDT) Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> Message-ID: <39c154b6-ccd2-4f42-9e2f-a8b86431cc23n@googlegroups.com> DFS schrieb am Mittwoch, 22. September 2021 um 05:10:30 UTC+2: > On 9/21/2021 10:38 PM, Mohsen Owzar wrote: > > DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: > >> On 9/21/2021 4:36 AM, Mohsen Owzar wrote: > >>> Hi Guys > >>> Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. > >>> Now I try to write this GUI with Python with PyQt5 or TKinter. > >>> First question is: > >>> Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. > >>> Second: > >>> Because, I can not attach a picture to this post, I try to describe my picture of my GUI. > >> Draw your GUI in PyQt designer or other graphics tool, then upload a > >> screenshot of it to imgur, then post the link to the picture. > > Thanks, for your answer. > > But, what is "imgur"? > > I'm not so familiar with handling of pictures in this group. > > How can I call "imgur" or how can I get there? > > > > Regards > > Mohsen > www.imgur.com > > It's a website you can upload image files or screenshots to. Then you > can copy a link to your picture and post the link here. I have already posted the link, but I can not see it anywhere. Now, I post it again: https://imgur.com/a/Vh8P2TE I hope that you can see my two images. Regards Mohsen From nospam at dfs.com Wed Sep 22 03:41:31 2021 From: nospam at dfs.com (DFS) Date: Wed, 22 Sep 2021 03:41:31 -0400 Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: <39c154b6-ccd2-4f42-9e2f-a8b86431cc23n@googlegroups.com> References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> <39c154b6-ccd2-4f42-9e2f-a8b86431cc23n@googlegroups.com> Message-ID: On 9/22/2021 1:54 AM, Mohsen Owzar wrote: > DFS schrieb am Mittwoch, 22. September 2021 um 05:10:30 UTC+2: >> On 9/21/2021 10:38 PM, Mohsen Owzar wrote: >>> DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: >>>> On 9/21/2021 4:36 AM, Mohsen Owzar wrote: >>>>> Hi Guys >>>>> Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. >>>>> Now I try to write this GUI with Python with PyQt5 or TKinter. >>>>> First question is: >>>>> Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. >>>>> Second: >>>>> Because, I can not attach a picture to this post, I try to describe my picture of my GUI. >>>> Draw your GUI in PyQt designer or other graphics tool, then upload a >>>> screenshot of it to imgur, then post the link to the picture. >>> Thanks, for your answer. >>> But, what is "imgur"? >>> I'm not so familiar with handling of pictures in this group. >>> How can I call "imgur" or how can I get there? >>> >>> Regards >>> Mohsen >> www.imgur.com >> >> It's a website you can upload image files or screenshots to. Then you >> can copy a link to your picture and post the link here. > I have already posted the link, but I can not see it anywhere. > Now, I post it again: > https://imgur.com/a/Vh8P2TE > I hope that you can see my two images. > Regards > Mohsen Got it. I haven't used tkinter. In PyQt5 designer I think you should use one QTextEdit control for each square. Each square with the small black font can be initially populated with 1 2 3 4 5 6 7 8 9 https://imgur.com/lTcEiML some starter python code (maybe save as sudoku.py) ===================================================================== from PyQt5 import Qt, QtCore, QtGui, QtWidgets, uic from PyQt5.Qt import * from PyQt5.QtCore import * from PyQt5.QtGui import * from PyQt5.QtWidgets import * #objects app = QtWidgets.QApplication([]) frm = uic.loadUi("sudoku.ui") #grid = a collection of squares grids = 1 #squares = number of squares per grid squares = 9 #fill the squares with 1-9 def populateSquares(): for i in range(grids,grids+1): for j in range(1,squares+1): widget = frm.findChild(QtWidgets.QTextEdit, "txt{}_{}".format(i,j)) widget.setText("1 2 3 4 5 6 7 8 9") #read data from squares def readSquares(): for i in range(grids,grids+1): for j in range(1,squares+1): print("txt%d_%d contains: %s" % (i,j,frm.findChild(QtWidgets.QTextEdit, "txt{}_{}".format(i,j)).toPlainText())) #connect pushbuttons to code frm.btnPopulate.clicked.connect(populateSquares) frm.btnReadContents.clicked.connect(readSquares) #show main form frm.show() #initiate application app.exec() ===================================================================== .ui file (ie save as sudoku.ui) ===================================================================== MainWindow 0 0 325 288 Sudoku 32 22 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 114 22 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 196 22 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 32 86 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 114 86 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 196 86 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 32 150 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 114 150 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 32 228 121 35 Populate with numbers 196 150 83 65 Courier 12 50 false false color: rgb(0, 0, 127); background-color: rgb(255, 255, 127); QFrame::StyledPanel QFrame::Sunken Qt::ScrollBarAlwaysOff true 170 228 109 35 Read contents ===================================================================== From petef4+usenet at gmail.com Wed Sep 22 03:56:48 2021 From: petef4+usenet at gmail.com (Pete Forman) Date: Wed, 22 Sep 2021 08:56:48 +0100 Subject: XML Considered Harmful References: <87ilytzkt4.fsf@gmail.com> Message-ID: <87bl4lyren.fsf@gmail.com> Jon Ribbens writes: > On 2021-09-21, Pete Forman wrote: >> CSV is quite good as a lowest common denominator exchange format. I >> say quite because I would characterize it by 8 attributes and you >> need to pick a dialect such as MS Excel which sets out what those >> are. XML and JSON are controlled much better. You can easily verify >> that you conform to those and guarantee that *any* conformant parser >> can read your content. XML is more powerful in that repect than JSON >> in that you can define and enforce schemas. In your case the fuel >> name, UOM, etc. can be validated with standard tools. In JSON all >> that checking is entirely handled by the consuming program(s). > > That's not true. You can use "JSON Schema" to create a schema for > validating JSON files, and there appear to be at least four > implementations in Python. Fair point. It has been a while since I looked at JSON schemas and they were rather less mature then. -- Pete Forman From michael.stemper at gmail.com Wed Sep 22 10:40:48 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Wed, 22 Sep 2021 09:40:48 -0500 Subject: XML Considered Harmful In-Reply-To: <87ilytzkt4.fsf@gmail.com> References: <87ilytzkt4.fsf@gmail.com> Message-ID: On 21/09/2021 16.21, Pete Forman wrote: > "Michael F. Stemper" writes: >> On 21/09/2021 13.49, alister wrote: >>> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote: >> It's my own research, so I can give myself the data in any format that I >> like. >> >>> as far as I can see the main issue with XML is bloat, it tries to do >>> too many things & is a very verbose format, often the quantity of >>> mark-up can easily exceed the data contained within it. other formats >>> such a JSON & csv have far less overhead, although again not always >>> suitable. >> >> I've heard of JSON, but never done anything with it. > > Then you should certainly try to get a basic understanding of it. One > thing JSON shares with XML is that it is best left to machines to > produce and consume. Because both can be viewed in a text editor there > is a common misconception that they are easy to edit. Not so, commas are > a common bugbear in JSON and non-trivial edits in (XML unaware) text > editors are tricky. Okay, after playing around with the example in Lubanovic's book[1] I've managed to create a dict of dicts of dicts and write it to a json file. It seems to me that this is how json handles hierarchical data. Is that understanding correct? Is this then the process that I would use to create a *.json file to provide data to my various programs? Copy and paste the current hard-coded assignment statements into REPL, use json.dump(dict,fp) to write it to a file, and then read the file into each program with json.load(fp)? (Actually, I'd write a function to do that, just as I would with XML.) > Consider what overhead you should worry about. If you are concerned > about file sizes then XML, JSON and CSV should all compress to a similar > size. Not a concern at all for my current application. >> How does CSV handle hierarchical data? For instance, I have >> generators[1], each of which has a name, a fuel and one or more >> incremental heat rate curves. Each fuel has a name, UOM, heat content, >> and price. Each incremental cost curve has a name, and a series of >> ordered pairs (representing a piecewise linear curve). >> >> Can CSV files model this sort of situation? > > The short answer is no. CSV files represent spreadsheet row-column > values with nothing fancier such as formulas or other redirections. Okay, that was what I suspected. > CSV is quite good as a lowest common denominator exchange format. I say > quite because I would characterize it by 8 attributes and you need to > pick a dialect such as MS Excel which sets out what those are. XML and > JSON are controlled much better. You can easily verify that you conform > to those and guarantee that *any* conformant parser can read your > content. XML is more powerful in that repect than JSON in that you can > define and enforce schemas. In your case the fuel name, UOM, etc. can be > validated with standard tools. Yeah, validating against a DTD is pretty easy, since lxml.etree does all of the work. > In JSON all that checking is entirely > handled by the consuming program(s). Well, the consumer's (almost) always going to need to do *some* validation. For instance, as far as I can tell, a DTD can't specify that there must be at least two of a particular item. The designers of DTD seem to have taken the advice of MacLennan[2]: "The only reasonable numbers are zero, one, or infinity." Which is great until you need to make sure that you have enough points to define at least one line segment. >>> As in all such cases it is a matter of choosing the most apropriate tool >>> for the job in hand. >> >> Naturally. That's what I'm exploring. > > You might also like to consider HDF5. It is targeted at large volumes of > scientific data and its capabilities are well above what you need. Yeah, I won't be looking at more than five or ten generators at most. A small number is enough to confirm or refute the behavior that I'm testing. [1] _Introducing Python: Modern Computing in Simple Packages_, Second Release, (c) 2015, Bill Lubanovic, O'Reilly Media, Inc. [2] _Principles of Programming Languages: Design, Evaluation, and Implementation_, Second Edition, (c) 1987, Bruce J. MacLennan, Holt, Rinehart, & Winston -- Michael F. Stemper No animals were harmed in the composition of this message. From michael.stemper at gmail.com Wed Sep 22 10:52:59 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Wed, 22 Sep 2021 09:52:59 -0500 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 21/09/2021 19.30, Eli the Bearded wrote: > In comp.lang.python, Michael F. Stemper wrote: >> I've heard of JSON, but never done anything with it. > > You probably have used it inadvertantly on a regular basis over the > past few years. Websites live on it. I used to use javascript when I was running Windows (up until 2009), since it was the only programming language to which I had ready access. Then I got a linux box and quickly discovered python. I dropped javascript like a hot potato. >> How does CSV handle hierarchical data? For instance, I have >> generators[1], each of which has a name, a fuel and one or more >> incremental heat rate curves. Each fuel has a name, UOM, heat content, >> and price. Each incremental cost curve has a name, and a series of >> ordered pairs (representing a piecewise linear curve). >> >> Can CSV files model this sort of situation? > > Can a string of ones and zeros encode the sounds of Bach, the images > of his sheet music, the details to reproduce his bust in melted plastic > extruded from nozzle under the control of machines? > > Yes, CSV files can model that. But it would not be my first choice of > data format. (Neither would JSON.) I'd probably use XML. Okay. 'Go not to the elves for counsel, for they will say both no and yes.' (I'm not actually surprised to find differences of opinion.) >> [1] The kind made of tons of iron and copper, filled with oil, and >> rotating at 1800 rpm. > > Those are rather hard to model in CSV, too, but I'm sure it could be > done. > for bonus round, use punched holes in paper to encode the ones and zeros I've done cardboard. -- Michael F. Stemper No animals were harmed in the composition of this message. From wlfraed at ix.netcom.com Wed Sep 22 12:31:22 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 22 Sep 2021 12:31:22 -0400 Subject: XML Considered Harmful References: Message-ID: On Tue, 21 Sep 2021 13:12:10 -0500, "Michael F. Stemper" declaimed the following: >On the prolog thread, somebody posted a link to: > > >One thing that it tangentially says is "XML is not the answer." > >I read this page right when I was about to write an XML parser >to get data into the code for a research project I'm working on. >It seems to me that XML is the right approach for this sort of >thing, especially since the data is hierarchical in nature. > >Does the advice on that page mean that I should find some other >way to get data into my programs, or does it refer to some kind >of misuse/abuse of XML for something that it wasn't designed >for? There are some that try to use XML as a /live/ data /storage/ format (such as http://www.drivehq.com/web/brana/pandora.htm which has to parse XML files for all configuration data and filter definitions on start-up, and update those files on any changes). If you control both the data generation and the data consumption, finding some format with less overhead than XML is probably to be recommended. XML is more a self-documented (in theory) means of packaging data for transport between widely disparate applications, which are likely written by different teams, if not different companies, who only interface via the definition of the data as seen by XML. > >If XML is not the way to package data, what is the recommended >approach? Again, if you control both generation and consumption... I'd probably use an RDBM. SQLite tends to be packaged with Python [Windows] or, at the least, the DB-API adapter [Linux tends to expect SQLite as a standard installed item]. SQLite is a "file server" model (as is the JET engine used by M$ Access) -- each application (instance) is directly accessing the database file; there is no server process mediating access. Hierarchical (since you mention that in later posts) would be represented by relations (terminology from relational theory -- a "table" to most) linked by foreign keys. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Wed Sep 22 12:41:47 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 22 Sep 2021 12:41:47 -0400 Subject: Polymorphic imports References: Message-ID: On Tue, 21 Sep 2021 18:58:31 +0000, Travis Griggs declaimed the following: >from lib import paths >import paths.dynamic_client_module() > >But this seems to not work. Import can only take real modules? Not programatic ones? Consider "import" to be equivalent to a compile-time operation. https://docs.python.org/3/using/cmdline.html#environment-variables cf PYTHONPATH https://docs.python.org/3/library/sys.html#sys.path """ sys.path A list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default. As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH. A program is free to modify this list for its own purposes. Only strings and bytes should be added to sys.path; all other data types are ignored during import. """ So... Putting the module variants into separate directories, and modifying the import path to reference the directory of a specific variant, might do what you want -- as long as the module name itself remains fixed. The other alternative may be https://docs.python.org/3/library/functions.html#__import__ -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From rosuav at gmail.com Wed Sep 22 15:52:31 2021 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 23 Sep 2021 05:52:31 +1000 Subject: Polymorphic imports In-Reply-To: References: Message-ID: On Thu, Sep 23, 2021 at 4:20 AM Dennis Lee Bieber wrote: > > The other alternative may be > https://docs.python.org/3/library/functions.html#__import__ > I wouldn't recommend calling a dunder. If you just want to pass a text string and get back a module, importlib is a better choice. ChrisA From wlfraed at ix.netcom.com Wed Sep 22 18:37:18 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 22 Sep 2021 18:37:18 -0400 Subject: XML Considered Harmful References: Message-ID: On Wed, 22 Sep 2021 09:52:59 -0500, "Michael F. Stemper" declaimed the following: >On 21/09/2021 19.30, Eli the Bearded wrote: >> In comp.lang.python, Michael F. Stemper wrote: >>> How does CSV handle hierarchical data? For instance, I have >>> generators[1], each of which has a name, a fuel and one or more >>> incremental heat rate curves. Each fuel has a name, UOM, heat content, >>> and price. Each incremental cost curve has a name, and a series of >>> ordered pairs (representing a piecewise linear curve). >>> >>> Can CSV files model this sort of situation? >> >> Yes, CSV files can model that. But it would not be my first choice of >> data format. (Neither would JSON.) I'd probably use XML. > >Okay. 'Go not to the elves for counsel, for they will say both no >and yes.' (I'm not actually surprised to find differences of opinion.) > You'd have to include a "level" (and/or data type if multiple objects can be at the same level) field (as the first field) in CSV which identifies how to parse the rest of the CSV data (well, technically, the CSV module has "parsed" it -- in terms of splitting at commas, handling quoted strings (which may contain commas which are not split points, etc.). 1-generator, name 2-fuel, name, UOM, heat-content, price 2-curve, name 3-point, X, Y 3-point, X, Y ... 2-curve, name 3-point, X, Y 3-point, X, Y ... You extract objects at each level; if the level is the same or "lower" (numerically -- higher in hierarchy) you attach the "previously" extracted object to the parent object... Whether list or dictionary, or class instance(s): class Point(): #Point may be overkill, easier to just use a tuple (X, Y) def __init__(self, X, Y): self.X = X self.Y = Y class Curve(): def __init__(self, name): self.name = name self.points = [] #use as aCurve.points.append(currentPoint) class Fuel(): def __init__(self, name, ..., price): self.name = name ... self.price = price class Generator(): def __init__(self, name): self.name = name self.fuel = None self.curves = [] #aGenerator.fuel = currentCurve #aGenerator.curves.append(currentCurve) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From mats at wichmann.us Thu Sep 23 08:53:10 2021 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 23 Sep 2021 06:53:10 -0600 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> On 9/22/21 10:31, Dennis Lee Bieber wrote: > If you control both the data generation and the data consumption, > finding some format ... This is really the key. I rant at people seeming to believe that csv is THE data interchange format, and it's about as bad as it gets at that, if you have a choice. xml is noisy but at least (potentially) self-documenting, and ought to be able to recover from certain errors. The problem with csv is that a substantial chunk of the world seems to live inside Excel, and so data is commonly both generated in csv so it can be imported into excel and generated in csv as a result of exporting from excel, so the parts often are *not* in your control. Sigh. From rosuav at gmail.com Thu Sep 23 09:27:13 2021 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 23 Sep 2021 23:27:13 +1000 Subject: XML Considered Harmful In-Reply-To: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> References: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> Message-ID: On Thu, Sep 23, 2021 at 10:55 PM Mats Wichmann wrote: > > On 9/22/21 10:31, Dennis Lee Bieber wrote: > > > If you control both the data generation and the data consumption, > > finding some format ... > > This is really the key. I rant at people seeming to believe that csv is > THE data interchange format, and it's about as bad as it gets at that, > if you have a choice. xml is noisy but at least (potentially) > self-documenting, and ought to be able to recover from certain errors. > The problem with csv is that a substantial chunk of the world seems to > live inside Excel, and so data is commonly both generated in csv so it > can be imported into excel and generated in csv as a result of exporting > from excel, so the parts often are *not* in your control. > > Sigh. The only people who think that CSV is *the* format are people who habitually live in spreadsheets. People who move data around the internet, from program to program, are much more likely to assume that JSON is the sole format. Of course, there is no single ultimate data interchange format, but JSON is a lot closer to one than CSV is. (Or to be more precise: any such thing as a "single ultimate data interchange format" will be so generic that it isn't enough to define everything. For instance, "a stream of bytes" is a universal data interchange format, but that's not ultimately a very useful claim.) ChrisA From olivier.perch333 at gmail.com Thu Sep 23 04:16:38 2021 From: olivier.perch333 at gmail.com (olivier Perchet) Date: Thu, 23 Sep 2021 10:16:38 +0200 Subject: =?utf-8?Q?t=C3=A9l=C3=A9chargement_python?= Message-ID: Envoye `a partir de [1]Courrier pour Windows [2][IMG] Garanti sans virus. [3]www.avast.com References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 2. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient 3. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient From mohsen.owzar at gmail.com Thu Sep 23 02:53:04 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Wed, 22 Sep 2021 23:53:04 -0700 (PDT) Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> <39c154b6-ccd2-4f42-9e2f-a8b86431cc23n@googlegroups.com> Message-ID: DFS schrieb am Mittwoch, 22. September 2021 um 09:41:42 UTC+2: > On 9/22/2021 1:54 AM, Mohsen Owzar wrote: > > DFS schrieb am Mittwoch, 22. September 2021 um 05:10:30 UTC+2: > >> On 9/21/2021 10:38 PM, Mohsen Owzar wrote: > >>> DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: > >>>> On 9/21/2021 4:36 AM, Mohsen Owzar wrote: > >>>>> Hi Guys > >>>>> Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. > >>>>> Now I try to write this GUI with Python with PyQt5 or TKinter. > >>>>> First question is: > >>>>> Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. > >>>>> Second: > >>>>> Because, I can not attach a picture to this post, I try to describe my picture of my GUI. > >>>> Draw your GUI in PyQt designer or other graphics tool, then upload a > >>>> screenshot of it to imgur, then post the link to the picture. > >>> Thanks, for your answer. > >>> But, what is "imgur"? > >>> I'm not so familiar with handling of pictures in this group. > >>> How can I call "imgur" or how can I get there? > >>> > >>> Regards > >>> Mohsen > >> www.imgur.com > >> > >> It's a website you can upload image files or screenshots to. Then you > >> can copy a link to your picture and post the link here. > > I have already posted the link, but I can not see it anywhere. > > Now, I post it again: > > https://imgur.com/a/Vh8P2TE > > I hope that you can see my two images. > > Regards > > Mohsen > Got it. > > I haven't used tkinter. In PyQt5 designer I think you should use one > QTextEdit control for each square. > > > Each square with the small black font can be initially populated with > > 1 2 3 > 4 5 6 > 7 8 9 > > > > https://imgur.com/lTcEiML > > > > some starter python code (maybe save as sudoku.py) > > ===================================================================== > from PyQt5 import Qt, QtCore, QtGui, QtWidgets, uic > from PyQt5.Qt import * > from PyQt5.QtCore import * > from PyQt5.QtGui import * > from PyQt5.QtWidgets import * > > #objects > app = QtWidgets.QApplication([]) > frm = uic.loadUi("sudoku.ui") > > > #grid = a collection of squares > grids = 1 > > #squares = number of squares per grid > squares = 9 > > #fill the squares with 1-9 > def populateSquares(): > for i in range(grids,grids+1): > for j in range(1,squares+1): > widget = frm.findChild(QtWidgets.QTextEdit, "txt{}_{}".format(i,j)) > widget.setText("1 2 3 4 5 6 7 8 9") > > #read data from squares > def readSquares(): > for i in range(grids,grids+1): > for j in range(1,squares+1): > print("txt%d_%d contains: %s" % > (i,j,frm.findChild(QtWidgets.QTextEdit, > "txt{}_{}".format(i,j)).toPlainText())) > > > #connect pushbuttons to code > frm.btnPopulate.clicked.connect(populateSquares) > frm.btnReadContents.clicked.connect(readSquares) > > #show main form > frm.show() > > #initiate application > app.exec() > ===================================================================== > > > > > > .ui file (ie save as sudoku.ui) > ===================================================================== > > > > MainWindow > > > > 0 > 0 > 325 > 288 > > > > Sudoku > > > > > > 32 > 22 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 114 > 22 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 196 > 22 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 32 > 86 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 114 > 86 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 196 > 86 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 32 > 150 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 114 > 150 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 32 > 228 > 121 > 35 > > > > Populate with numbers > > > > > > 196 > 150 > 83 > 65 > > > > > Courier > 12 > 50 > false > > > > false > > > color: rgb(0, 0, 127); > background-color: rgb(255, 255, 127); > > > QFrame::StyledPanel > > > QFrame::Sunken > > > Qt::ScrollBarAlwaysOff > > > true > > > > > > 170 > 228 > 109 > 35 > > > > Read contents > > > > > > > > > > ===================================================================== Thank you, I'll try to use this QTextEdit, to see if I'm able to manage my needs. Regards Mohsen From auriocus at gmx.de Thu Sep 23 05:21:27 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 23 Sep 2021 11:21:27 +0200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: Am 22.09.21 um 16:52 schrieb Michael F. Stemper: > On 21/09/2021 19.30, Eli the Bearded wrote: >> Yes, CSV files can model that. But it would not be my first choice of >> data format. (Neither would JSON.) I'd probably use XML. > > Okay. 'Go not to the elves for counsel, for they will say both no > and yes.' (I'm not actually surprised to find differences of opinion.) It is wrong, CSV has no model of hierarchical data. A CSV file is a 2d table, just like a database table or an Excel sheet. You can /layer/ high-dimensional data on top of a 2D table, there is the relational algebra theory behind this, but it is wrong (or misleading at best) to say that CSV can model hierarchical data. It's the same as saying "CSV supports images". Of course it doesn't, its a textfile, but you could encode a JPEG as base64 and then put this string into the cell of a CSV table. That definitely isn't what a sane person would understand as "support". Christian From michael.stemper at gmail.com Thu Sep 23 13:23:06 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 23 Sep 2021 12:23:06 -0500 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 22/09/2021 17.37, Dennis Lee Bieber wrote: > On Wed, 22 Sep 2021 09:52:59 -0500, "Michael F. Stemper" > declaimed the following: >> On 21/09/2021 19.30, Eli the Bearded wrote: >>> In comp.lang.python, Michael F. Stemper wrote: >>>> How does CSV handle hierarchical data? For instance, I have >>>> Can CSV files model this sort of situation? >>> > >>> Yes, CSV files can model that. But it would not be my first choice of >>> data format. (Neither would JSON.) I'd probably use XML. >> >> Okay. 'Go not to the elves for counsel, for they will say both no >> and yes.' (I'm not actually surprised to find differences of opinion.) >> > You'd have to include a "level" (and/or data type if multiple objects > can be at the same level) field (as the first field) in CSV which > identifies how to parse the rest of the CSV data (well, technically, the > CSV module has "parsed" it -- in terms of splitting at commas, handling > quoted strings (which may contain commas which are not split points, etc.). > > 1-generator, name > 2-fuel, name, UOM, heat-content, price > 2-curve, name > 3-point, X, Y > 3-point, X, Y > ... > 2-curve, name > 3-point, X, Y > 3-point, X, Y This reminds me of how my (former) employer imported data models into our systems from the 1970s until the mid-2000s. We had 80-column records (called "card images"), that would have looked like: FUEL0 LIGNITE TON 13.610 043.581 UNIT1 COAL CREK1 UNIT2 ... The specific columns for the start and end of each field on each record were defined in a thousand-plus page document. (We modeled all of a power system, not just economic data about generators.) However, this doesn't seem like it would fit too well with the csv module, since it requires a lot more logic on the part of the consuming program. Interesting flashback, though. -- Michael F. Stemper Deuteronomy 24:17 From * at eli.users.panix.com Thu Sep 23 13:51:45 2021 From: * at eli.users.panix.com (Eli the Bearded) Date: Thu, 23 Sep 2021 17:51:45 -0000 (UTC) Subject: XML Considered Harmful References: Message-ID: In comp.lang.python, Christian Gollwitzer wrote: > Am 22.09.21 um 16:52 schrieb Michael F. Stemper: >> On 21/09/2021 19.30, Eli the Bearded wrote: >>> Yes, CSV files can model that. But it would not be my first choice of >>> data format. (Neither would JSON.) I'd probably use XML. >> Okay. 'Go not to the elves for counsel, for they will say both no >> and yes.' (I'm not actually surprised to find differences of opinion.) Well, I have a recommendation with my answer. > It's the same as saying "CSV supports images". Of course it doesn't, its > a textfile, but you could encode a JPEG as base64 and then put this > string into the cell of a CSV table. That definitely isn't what a sane > person would understand as "support". I'd use one of the netpbm formats instead of JPEG. PBM for one bit bitmaps, PGM for one channel (typically grayscale), PPM for three channel RGB, and PAM for anything else (two channel gray plus alpha, CMYK, RGBA, HSV, YCbCr, and more exotic formats). JPEG is tricky to map to CSV since it is a three channel format (YCbCr), where the channels are typically not at the same resolution. Usually Y is full size and the Cb and Cr channels are one quarter size ("4:2:0 chroma subsampling"). The unequal size of the channels does not lend itself to CSV, but I can't say it's impossible. But maybe you meant the whole JFIF or Exif JPEG file format base64 encoded with no attempt to understand the image. That sort of thing is common in JSON, and I've seen it in YAML, too. It wouldn't surprise me if people do that in CSV or XML, but I have so far avoided seeing that. I used that method for sticking a tiny PNG in a CSS file just earlier this month. The whole PNG was smaller than the typical headers of an HTTP/1.1 request and response, so I figured "don't make it a separate file". Elijah ------ can at this point recegnize a bunch of "magic numbers" in base64 From michael.stemper at gmail.com Thu Sep 23 16:06:24 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 23 Sep 2021 15:06:24 -0500 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 23/09/2021 12.51, Eli the Bearded wrote: >> Am 22.09.21 um 16:52 schrieb Michael F. Stemper: >>> On 21/09/2021 19.30, Eli the Bearded wrote: >>>> Yes, CSV files can model that. But it would not be my first choice of >>>> data format. (Neither would JSON.) I'd probably use XML. >>> Okay. 'Go not to the elves for counsel, for they will say both no >>> and yes.' (I'm not actually surprised to find differences of opinion.) > > Well, I have a recommendation with my answer. Sorry, didn't mean that to be disparaging. -- Michael F. Stemper This post contains greater than 95% post-consumer bytes by weight. From avigross at verizon.net Thu Sep 23 17:26:44 2021 From: avigross at verizon.net (Avi Gross) Date: Thu, 23 Sep 2021 17:26:44 -0400 Subject: XML Considered Harmful In-Reply-To: References: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> Message-ID: <01f701d7b0c1$b5646030$202d2090$@verizon.net> Can we agree that there are way more general ways to store data than anything currently in common use and that in some ways, CSV and cousins like TSV are a subset of the others in a sense? There are trees and arbitrary graphs and many complex data structures often encountered while a program is running as in-memory objects. Many are not trivial to store. But some are if all you see is table-like constructs including matrices and data.frames. I mean any rectangular data format with umpteen rows and N columns can trivially be stored in many other formats and especially when it allows some columns to have NA values. The other format would simply have major categories that contain components with one per column, and if missing, represents an NA. Is there any reason JSON or XML cannot include the contents of any CSV with headers and without loss of info? Going the other way is harder. Note that a data.frame type of structure often imposes restrictions on a CSV and requires everything in a column to be of the same type, or coercible to a common type. (well, not always true as in using list columns in R.) But given some arbitrary structure in XML, can you look at all possible labels and if it is not too complex, make a CSV with one or more columns for every possible need? It can be a problem if say a record for an Author allows multiple actual co-authors. Normal books may let you get by with multiple columns (mostly containing an NA) with names like author1, author2, author3, ... But scientific papers seemingly allow oodles of authors and any time you update the data, you may need yet another column. And, of course, processing data where many columns have the same meaning is a bit of a pain. Data structures can also often be nested multiple levels and at some point, CSV is not a reasonable fit unless you play database games and make multiple tables you can store and retrieve to make complex queries, as in many relational database systems. Yes, each such table can be a CSV. But if you give someone a hammer, they tend to stop using thumbtacks or other tools. The real question is what kind of data makes good sense for an application. If a nice rectangular format works, great. Even if not, the Author problem above can fairly easily be handled by making the author column something like a character string you compose as "Last1, First1; Last2, First2; Last3, First3" and that fits fine in a CSV but can be taken apart in your software if looking for any book by a particular author. Not optimal, but a workaround I am sure is used. But using the most abstract and complex storage method is very often overkill and unless you are very good at it, may well be a fairly slow and even error-prone way to solve a problem. -----Original Message----- From: Python-list On Behalf Of Chris Angelico Sent: Thursday, September 23, 2021 9:27 AM To: Python Subject: Re: XML Considered Harmful On Thu, Sep 23, 2021 at 10:55 PM Mats Wichmann wrote: > > On 9/22/21 10:31, Dennis Lee Bieber wrote: > > > If you control both the data generation and the data > > consumption, finding some format ... > > This is really the key. I rant at people seeming to believe that csv > is THE data interchange format, and it's about as bad as it gets at > that, if you have a choice. xml is noisy but at least (potentially) > self-documenting, and ought to be able to recover from certain errors. > The problem with csv is that a substantial chunk of the world seems to > live inside Excel, and so data is commonly both generated in csv so it > can be imported into excel and generated in csv as a result of > exporting from excel, so the parts often are *not* in your control. > > Sigh. The only people who think that CSV is *the* format are people who habitually live in spreadsheets. People who move data around the internet, from program to program, are much more likely to assume that JSON is the sole format. Of course, there is no single ultimate data interchange format, but JSON is a lot closer to one than CSV is. (Or to be more precise: any such thing as a "single ultimate data interchange format" will be so generic that it isn't enough to define everything. For instance, "a stream of bytes" is a universal data interchange format, but that's not ultimately a very useful claim.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list From avigross at verizon.net Thu Sep 23 17:59:41 2021 From: avigross at verizon.net (Avi Gross) Date: Thu, 23 Sep 2021 17:59:41 -0400 Subject: XML Considered Harmful In-Reply-To: References: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> <01f701d7b0c1$b5646030$202d2090$@verizon.net> Message-ID: <024501d7b0c6$4ffad210$eff07630$@verizon.net> What you are describing Stephen, is what I meant by emulating a relational database with tables. And, FYI, There is no guarantee that two authors with the same name will not be assumed to be the same person. Besides the lack of any one official CSV format, there are oodles of features I have seen that are normally external to the CSV. For example, I have often read in data from a CSV or similar, where you could tell the software to consider a blank or 999 to mean NA and what denotes a line in the file to be ignored as a comment and whether a separator is a space or any combination of whitespace and what quotes something so say you can hide a comma and how to handle escapes and whether to skip blank lines and more. Now a really good design might place some metadata into the file that can be used to set defaults for things like that or incorporate them into the format unambiguously. It might calculate the likely data type for various fields and store that in the metadata. So even if you stored rectangular data in a CSV file, perhaps the early lines would be in some format that can be read as comments and supply some info like the above. Are any of the CSV variants more like that? -----Original Message----- From: Python-list On Behalf Of Stefan Ram Sent: Thursday, September 23, 2021 5:43 PM To: python-list at python.org Subject: Re: XML Considered Harmful "Avi Gross" writes: >But scientific papers seemingly allow oodles of authors and any time >you update the data, you may need yet another column. You can use three CSV files: papers, persons, and authors: papers.csv 1, "Is the accelerated expansion evidence of a change of signature?" persons.csv 1, Marc Mars authors.csv 1, 1 I.e., paper 1 is authored by person 1. Now, when we learn that Jos? M. M. Senovilla also is a co-author of "Is the accelerated expansion evidence of a forthcoming change of signature?", we do only have to add new rows, no new colums. papers.csv 1, "Is the accelerated expansion evidence of a change of signature?" persons.csv 1, "Marc Mars" 2, "Jos? M. M. Senovilla" authors.csv 1, 1 1, 2 The real problem with CSV is that there is no CSV. This is not a specific data language with a specific specification. Instead it is a vague designation for a plethora of CSV dialects, which usually dot not even have a specification. Compare this with XML. XML has a sole specification managed by the W3C. -- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Thu Sep 23 18:02:46 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 24 Sep 2021 08:02:46 +1000 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On Fri, Sep 24, 2021 at 7:11 AM Eli the Bearded <*@eli.users.panix.com> wrote: > > In comp.lang.python, Christian Gollwitzer wrote: > > Am 22.09.21 um 16:52 schrieb Michael F. Stemper: > >> On 21/09/2021 19.30, Eli the Bearded wrote: > >>> Yes, CSV files can model that. But it would not be my first choice of > >>> data format. (Neither would JSON.) I'd probably use XML. > >> Okay. 'Go not to the elves for counsel, for they will say both no > >> and yes.' (I'm not actually surprised to find differences of opinion.) > > Well, I have a recommendation with my answer. > > > It's the same as saying "CSV supports images". Of course it doesn't, its > > a textfile, but you could encode a JPEG as base64 and then put this > > string into the cell of a CSV table. That definitely isn't what a sane > > person would understand as "support". > > I'd use one of the netpbm formats instead of JPEG. PBM for one bit > bitmaps, PGM for one channel (typically grayscale), PPM for three > channel RGB, and PAM for anything else (two channel gray plus alpha, > CMYK, RGBA, HSV, YCbCr, and more exotic formats). JPEG is tricky to > map to CSV since it is a three channel format (YCbCr), where the > channels are typically not at the same resolution. Usually Y is full > size and the Cb and Cr channels are one quarter size ("4:2:0 chroma > subsampling"). The unequal size of the channels does not lend itself > to CSV, but I can't say it's impossible. > Examine prior art, and I truly do mean art, from Matt Parker: https://www.youtube.com/watch?v=UBX2QQHlQ_I ChrisA From jon+usenet at unequivocal.eu Thu Sep 23 18:55:31 2021 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Thu, 23 Sep 2021 22:55:31 -0000 (UTC) Subject: XML Considered Harmful References: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> <01f701d7b0c1$b5646030$202d2090$@verizon.net> Message-ID: On 2021-09-23, Stefan Ram wrote: > The real problem with CSV is that there is no CSV. > > This is not a specific data language with a specific > specification. Instead it is a vague designation for > a plethora of CSV dialects, which usually dot not even > have a specification. Indeed. For example, at least at some points in its history, Excel has been unable to import CSV written by itself, because its importer was incompatible with its own exporter. > Compare this with XML. XML has a sole specification managed > by the W3C. Other well-defined formats are also available ;-) From PythonList at DancesWithMice.info Thu Sep 23 21:02:29 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 24 Sep 2021 13:02:29 +1200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: <250ad800-665e-1b6b-a671-0dbec1e03360@DancesWithMice.info> On 22/09/2021 07.22, Michael F. Stemper wrote: > On 21/09/2021 13.49, alister wrote: >> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote: >> >>> On the prolog thread, somebody posted a link to: >>> Given the source, shouldn't one take any criticism of Python (or Java) with at least the proverbial grain of salt! >>> One thing that it tangentially says is "XML is not the answer." "tangential" as in 'spinning off'? ... > It's my own research, so I can give myself the data in any format that I > like. ... With that, why not code it as Python expressions, and include the module? -- Regards, =dn From rosuav at gmail.com Thu Sep 23 23:11:48 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 24 Sep 2021 13:11:48 +1000 Subject: XML Considered Harmful In-Reply-To: References: <250ad800-665e-1b6b-a671-0dbec1e03360@DancesWithMice.info> Message-ID: On Fri, Sep 24, 2021 at 12:22 PM Stefan Ram wrote: > > dn writes: > >With that, why not code it as Python expressions, and include the module? > > This might create a code execution vulnerability if such > files are exchanged between multiple parties. > > If code execution vulnerabilities and human-readability are > not an issue, then one could also think about using pickle. > > If one ignores security concerns for a moment, serialization into > a text format and subsequent deserialization can be a easy as: > > |>>> eval( str( [1, (2, 3)] )) > |[1, (2, 3)] > One good hybrid is to take a subset of Python syntax (so it still looks like a Python script for syntax highlighting etc), and then parse that yourself, using the ast module. For instance, you can strip out comments, then look for "VARNAME = ...", and parse the value using ast.literal_eval(), which will give you a fairly flexible file format that's still quite safe. ChrisA From drsalists at gmail.com Thu Sep 23 23:44:32 2021 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 23 Sep 2021 20:44:32 -0700 Subject: XML Considered Harmful In-Reply-To: References: <250ad800-665e-1b6b-a671-0dbec1e03360@DancesWithMice.info> Message-ID: On Thu, Sep 23, 2021 at 8:12 PM Chris Angelico wrote: > One good hybrid is to take a subset of Python syntax (so it still > looks like a Python script for syntax highlighting etc), and then > parse that yourself, using the ast module. For instance, you can strip > out comments, then look for "VARNAME = ...", and parse the value using > ast.literal_eval(), which will give you a fairly flexible file format > that's still quite safe. > Restricting Python with the ast module is interesting, but I don't think I'd want to bet my career on the actual safety of such a thing. Given that Java bytecode was a frequent problem inside web browsers, imagine all the messiness that could accidentally happen with a subset of Python syntax from untrusted sources. ast.literal_eval might be a little better - or a list of such, actually. Better still to use JSON or ini format - IOW something designed for the purpose. From rosuav at gmail.com Thu Sep 23 23:49:04 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 24 Sep 2021 13:49:04 +1000 Subject: XML Considered Harmful In-Reply-To: References: <250ad800-665e-1b6b-a671-0dbec1e03360@DancesWithMice.info> Message-ID: On Fri, Sep 24, 2021 at 1:44 PM Dan Stromberg wrote: > > > On Thu, Sep 23, 2021 at 8:12 PM Chris Angelico wrote: >> >> One good hybrid is to take a subset of Python syntax (so it still >> looks like a Python script for syntax highlighting etc), and then >> parse that yourself, using the ast module. For instance, you can strip >> out comments, then look for "VARNAME = ...", and parse the value using >> ast.literal_eval(), which will give you a fairly flexible file format >> that's still quite safe. > > > Restricting Python with the ast module is interesting, but I don't think I'd want to bet my career on the actual safety of such a thing. Given that Java bytecode was a frequent problem inside web browsers, imagine all the messiness that could accidentally happen with a subset of Python syntax from untrusted sources. > > ast.literal_eval might be a little better - or a list of such, actually. Uhh, I specifically mention literal_eval in there :) Simple text parsing followed by literal_eval for the bulk of it is a level of safety that I *would* bet my career on. > Better still to use JSON or ini format - IOW something designed for the purpose. It all depends on how human-editable it needs to be. JSON has several problems in that respect, including some rigidities, and a lack of support for comments. INI format doesn't have enough data types for many purposes. YAML might be closer, but it's not for every situation either. That's why we have options. ChrisA From mohsen.owzar at gmail.com Thu Sep 23 23:38:50 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Thu, 23 Sep 2021 20:38:50 -0700 (PDT) Subject: When should I use "parent=None" in __ini__ and "parent" in super() Message-ID: Hi Guys I'm writing since almost one-year codes in Python, using TKinter and PyQt5. I'm somehow able to writes GUIs in both of them. But since I'm using more Pyqt5 and using classes with initialization and super() constructs, and also I saw lots of videos and examples of coding them, I still don?t know exactly how and when should I use the parent in __init()__ and super() construct. For example, the following lines use "parent=None" in the __init__ construct and "parent" in the super() construct. And sometimes I do not see any of them in these two constructs. :::::::::::::::::::::::::::::::::::::::::::::::: class MyClass1(QWidget): def __init__(self, name, parent=None): super(MyClass1, self).__init__(parent) print(self.parent().test) :::::::::::::::::::::::::::::::::::::::::::::::: And sometimes without "parent" like the following: :::::::::::::::::::::::::::::::::::::::::::::::: class MyClass2(QWidget): def __init__(self, name): super(MyClass2, self).__init__() print(self.parent().test) :::::::::::::::::::::::::::::::::::::::::::::::: Could anyone explain this to me when and for which action I have to use the first one "MyClass1" and when the second one "MyClass2"? Regards Mohsen From mohsen.owzar at gmail.com Fri Sep 24 00:46:09 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Thu, 23 Sep 2021 21:46:09 -0700 (PDT) Subject: Flush / update GUIs in PyQt5 during debugging in PyCharm Message-ID: Hi Guys I've written a GUI using PyQt5 and in there I use StyleSheets (css) for the buttons and labels to change their background- and foreground-colors and their states as well. Because my program doesn't function correctly, I try to debug it in my IDE (PyCharm). The problem is that during debugging, when I change some attributes of a button or label, let say its background-color, I can not see this modification of the color until the whole method or function is completed. I believe that I have seen somewhere during my searches and googling that one can flush or update the GUI after each step/action is done. But until now I couldn't manage it and I don't know where I have to invoke flush/update command in PyCharm. If anyone has done this before and knows about it, I would very appreciate seeing his solution. Regards Mohsen From PythonList at DancesWithMice.info Fri Sep 24 01:40:09 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 24 Sep 2021 17:40:09 +1200 Subject: XML Considered Harmful In-Reply-To: References: <250ad800-665e-1b6b-a671-0dbec1e03360@DancesWithMice.info> Message-ID: <064e615c-3ff8-68ff-43b0-df0fb93f8315@DancesWithMice.info> On 24/09/2021 14.07, Stefan Ram wrote: > dn writes: >> With that, why not code it as Python expressions, and include the module? > > This might create a code execution vulnerability if such > files are exchanged between multiple parties. The OP's spec, as quoted earlier(!), reads: "It's my own research, so I can give myself the data in any format that I like." Whither "files are exchanged" and/or "multiple parties"? Are these anticipations of problems that may/won't ever apply? aka YAGNI. Concern about such an approach *is* warranted. However, the preceding question to be considered during the design-stage is: 'does such concern apply?'. The OP describes full and unique agency. Accordingly, "KISS"! NB my personal choice would likely be JSON or YAML, but see reservations (eg @Chris) - and with greater relevance: shouldn't we consider the OP's 'learning curve'? (such deduced only from OP's subsequent reactions/responses 'here' - with any and all due apologies) -- Regards, =dn From boblatest at yahoo.com Fri Sep 24 01:48:19 2021 From: boblatest at yahoo.com (Robert Latest) Date: 24 Sep 2021 05:48:19 GMT Subject: Type annotation pitfall Message-ID: Hi all, this just caused me several hours of my life until I could whittle it down to this minimal example. Simple question: Why is the x member of object "foo" modified by initializing "bar"? Obviously, initializing foo with None doesn't set foo.x at all. So I guess x stays a class property, not an instance property. And instantiating bar just added an item to the class property but didn't instantiate a new set. So basically, neither foo nor bar ever had their "own" x, right? Oooohh, dangerous! Never use mutable types in type hint, unless it's in explicit dataclasses (which automatically add an appropriate __init__()?) Now I must fine-comb all my code for more of these. class Foo(): x : set = set() def __init__(self, s): if s: self.x.add(s) foo = Foo(None) print(foo.x) # prints 'set()' bar = Foo('abc') print(foo.x) # prints '{'abc'} From miked at dewhirst.com.au Fri Sep 24 02:33:32 2021 From: miked at dewhirst.com.au (Mike Dewhirst) Date: Fri, 24 Sep 2021 16:33:32 +1000 Subject: XML Considered Harmful In-Reply-To: <064e615c-3ff8-68ff-43b0-df0fb93f8315@DancesWithMice.info> Message-ID: <4HG2X95DNMzpFyW@mail.python.org> I had to use XML once because that was demanded by the receiving machine over which I had no say.I wouldn't use it otherwise because staring at it makes you dizzy.I would want to know how the data are derived from the multiple sources and transmitted to the collating platform before pontificating.Then I would ignore any potential future enhancements and choose the easiest possible mechanism.?I have used json with python and been delighted at the ease of converting data into dicts and even arbitrary nesting where data values can also be dicts etc.Good luck--(Unsigned mail from my phone) -------- Original message --------From: dn via Python-list Date: 24/9/21 15:42 (GMT+10:00) To: python-list at python.org Subject: Re: XML Considered Harmful On 24/09/2021 14.07, Stefan Ram wrote:> dn writes:>> With that, why not code it as Python expressions, and include the module?> >?? This might create a code execution vulnerability if such >?? files are exchanged between multiple parties.The OP's spec, as quoted earlier(!), reads:"It's my own research, so I can give myself the data in any format thatI like."Whither "files are exchanged" and/or "multiple parties"? Are theseanticipations of problems that may/won't ever apply? aka YAGNI.Concern about such an approach *is* warranted.However, the preceding question to be considered during the design-stageis: 'does such concern apply?'. The OP describes full and unique agency.Accordingly, "KISS"!NB my personal choice would likely be JSON or YAML, but see reservations(eg @Chris) - and with greater relevance: shouldn't we consider the OP's'learning curve'?(such deduced only from OP's subsequent reactions/responses 'here' -with any and all due apologies)-- Regards,=dn-- https://mail.python.org/mailman/listinfo/python-list From petersaalbrink at gmail.com Fri Sep 24 07:28:29 2021 From: petersaalbrink at gmail.com (Peter Saalbrink) Date: Fri, 24 Sep 2021 13:28:29 +0200 Subject: Type annotation pitfall In-Reply-To: References: Message-ID: I don't think this has anything to do with typing or providing type hints. The type hint is the `: set` part, not the `= set()` part. You can declare the type without assigning to the variable. Indeed, as you already said, `x` is a class property here, and is shared amongst instances of the class. It might be a good idea to move the attribute assignment to the `__init__` method. In the following way, you can safely provide the type hint: ```python class Foo: x: set def __init__(self, s): self.x = set() if s: self.x.add(s) ``` Or, even shorter: ```python class Foo: def __init__(self, s: str): self.x: set[str] = {s} if s else set() print(reveal_type(Foo.x)) # mypy only ``` On Fri, Sep 24, 2021 at 7:58 AM Robert Latest via Python-list < python-list at python.org> wrote: > Hi all, > > this just caused me several hours of my life until I could whittle it down > to > this minimal example. Simple question: Why is the x member of object "foo" > modified by initializing "bar"? > > Obviously, initializing foo with None doesn't set foo.x at all. So I guess > x > stays a class property, not an instance property. And instantiating bar > just > added an item to the class property but didn't instantiate a new set. So > basically, neither foo nor bar ever had their "own" x, right? > > Oooohh, dangerous! Never use mutable types in type hint, unless it's in > explicit dataclasses (which automatically add an appropriate __init__()?) > > Now I must fine-comb all my code for more of these. > > class Foo(): > x : set = set() > > def __init__(self, s): > if s: > self.x.add(s) > > foo = Foo(None) > print(foo.x) # prints 'set()' > bar = Foo('abc') > print(foo.x) # prints '{'abc'} > > -- > https://mail.python.org/mailman/listinfo/python-list > From greg.ewing at canterbury.ac.nz Fri Sep 24 02:24:29 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 24 Sep 2021 18:24:29 +1200 Subject: Type annotation pitfall In-Reply-To: References: Message-ID: On 24/09/21 5:48 pm, Robert Latest wrote: > Never use mutable types in type hint, No, the lesson is: Don't mutate a shared object if you don't want the changes to be shared. If you want each instance to have its own set object, you need to create one for it in the __init__ method, e.g. class Foo(): x : set def __init__(self, s): self.x = set() if s: self.x.add(s) -- Greg From keller.steve at gmx.de Fri Sep 24 05:48:47 2021 From: keller.steve at gmx.de (Steve Keller) Date: Fri, 24 Sep 2021 11:48:47 +0200 Subject: Different "look and feel" of some built-in functions Message-ID: Why do some built-in Python functions feel so differently: For example sum(), all(), any() expect exactly one argument which is a sequence to operate on, i.e. a list, an iterator or a generator etc. sum([1,2,3,4]) sum(range(1, 101)) sum(2**i for i in range(10)) all([True, False]) any(even, {1,2,3,4}) while other functions like set.union() and set.intersection() work on a list of arguments but not on a sequence: set.intersection({1,2,3}, {3,4,5}) but set.union(map(...)) # does not work set.intersection(list(...)) # does not work and you have to use a * instead set.union(*map(...)) etc. Is this just for historical reason? And wouldn't it be possible and desirable to have more consistency? Steve From nospam at dfs.com Fri Sep 24 08:52:27 2021 From: nospam at dfs.com (DFS) Date: Fri, 24 Sep 2021 08:52:27 -0400 Subject: Flush / update GUIs in PyQt5 during debugging in PyCharm In-Reply-To: References: Message-ID: On 9/24/2021 12:46 AM, Mohsen Owzar wrote: > Hi Guys > I've written a GUI using PyQt5 and in there I use StyleSheets (css) for the buttons and labels to change their background- and foreground-colors and their states as well. > Because my program doesn't function correctly, I try to debug it in my IDE (PyCharm). > The problem is that during debugging, when I change some attributes of a button or label, let say its background-color, I can not see this modification of the color until the whole method or function is completed. > I believe that I have seen somewhere during my searches and googling that one can flush or update the GUI after each step/action is done. > But until now I couldn't manage it and I don't know where I have to invoke flush/update command in PyCharm. > If anyone has done this before and knows about it, I would very appreciate seeing his solution. > > Regards > Mohsen screen: form.repaint() individual widgets: form.widget.repaint() From rosuav at gmail.com Fri Sep 24 09:47:53 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 24 Sep 2021 23:47:53 +1000 Subject: Type annotation pitfall In-Reply-To: References: Message-ID: On Fri, Sep 24, 2021 at 11:43 PM Peter Saalbrink wrote: > > I don't think this has anything to do with typing or providing type hints. > The type hint is the `: set` part, not the `= set()` part. > You can declare the type without assigning to the variable. > Indeed, as you already said, `x` is a class property here, and is shared > amongst instances of the class. > It might be a good idea to move the attribute assignment to the `__init__` > method. > > In the following way, you can safely provide the type hint: > > ```python > class Foo: > x: set > > def __init__(self, s): > self.x = set() > if s: > self.x.add(s) > ``` > To be clear, this isn't a case of "never use mutables as class attributes"; often you *want* a single mutable object to be shared among instances of a class (so they can all find each other, perhaps). If you want each instance to have its own set, you construct a new set every object initialization; if you want them to all use the same set, you construct a single set and attach it to the class. Neither is wrong, they just have different meanings. ChrisA From rosuav at gmail.com Fri Sep 24 09:53:00 2021 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 24 Sep 2021 23:53:00 +1000 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: Message-ID: On Fri, Sep 24, 2021 at 11:47 PM Steve Keller wrote: > > Why do some built-in Python functions feel so differently: > > For example sum(), all(), any() expect exactly one argument which is a > sequence to operate on, i.e. a list, an iterator or a generator etc. > > sum([1,2,3,4]) > sum(range(1, 101)) > sum(2**i for i in range(10)) > all([True, False]) > any(even, {1,2,3,4}) > > while other functions like set.union() and set.intersection() work on > a list of arguments but not on a sequence: > > set.intersection({1,2,3}, {3,4,5}) > > but > > set.union(map(...)) # does not work > set.intersection(list(...)) # does not work > > and you have to use a * instead > > set.union(*map(...)) > > etc. > > Is this just for historical reason? And wouldn't it be possible and > desirable to have more consistency? > The ones you're calling off the set class are actually meant to be methods. >>> s = {1,2,3} >>> s.intersection({3,4,5}) {3} They expect a set, specifically, as the first argument, because normally that one goes before the dot. If you want to call the unbound method with two arguments, that's fine, but it's not the intended use, so you have to basically fudge it to look like a normal method call :) That's why it doesn't take a sequence. ChrisA From bursejan at gmail.com Fri Sep 24 09:46:13 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Fri, 24 Sep 2021 06:46:13 -0700 (PDT) Subject: XML Considered Harmful In-Reply-To: <755414c0-2148-4eba-b777-9f369b158493n@googlegroups.com> References: <755414c0-2148-4eba-b777-9f369b158493n@googlegroups.com> Message-ID: BTW: I think its problematic to associate Java with XML. Michael F. Stemper schrieb am Dienstag, 21. September 2021 um 20:12:33 UTC+2: > On the prolog thread, somebody posted a link to: > The above linke is very old, from 2004, and might apply how Java presented itself back in those days. But since the Jigsaw project, XML has practically left Java. Its all not anymore part of the javax.* or java.* namespace, Oracle got rid of XML technologies housing in these namespaces, and there is now the jakarta.* namespace. Example JAXB: Jakarta XML Binding (JAXB; formerly Java Architecture for XML Binding) https://de.wikipedia.org/wiki/Jakarta_XML_Binding If I remember well, also XML never went into the Java Language Specification, unlike the Scala programming language, where you can have XML literals: XML literals in scala https://tuttlem.github.io/2015/02/24/xml-literals-in-scala.html An easy protection against tampered XML data vulnerabilities is DTD or some other XML schema language. It can at least catch problems that are in the scope of the schema language. From bursejan at gmail.com Fri Sep 24 10:55:34 2021 From: bursejan at gmail.com (Mostowski Collapse) Date: Fri, 24 Sep 2021 07:55:34 -0700 (PDT) Subject: XML Considered Harmful In-Reply-To: References: <755414c0-2148-4eba-b777-9f369b158493n@googlegroups.com> Message-ID: <1ba7e98b-5f1b-45b0-9570-126d7ca96131n@googlegroups.com> Or then use cryptographic methods to protect your XML file when in transit. Like encryption and/or signatures. Mostowski Collapse schrieb am Freitag, 24. September 2021 um 15:46:27 UTC+2: > BTW: I think its problematic to associate Java with XML. > Michael F. Stemper schrieb am Dienstag, 21. September 2021 um 20:12:33 UTC+2: > > On the prolog thread, somebody posted a link to: > > > The above linke is very old, from 2004, and might apply > how Java presented itself back in those days. But since > the Jigsaw project, XML has practically left Java. > > Its all not anymore part of the javax.* or java.* namespace, > Oracle got rid of XML technologies housing in these > namespaces, and there is now the jakarta.* namespace. > > Example JAXB: > Jakarta XML Binding (JAXB; formerly Java Architecture for XML Binding) > https://de.wikipedia.org/wiki/Jakarta_XML_Binding > > If I remember well, also XML never went into the Java > Language Specification, unlike the Scala programming > language, where you can have XML literals: > > XML literals in scala > https://tuttlem.github.io/2015/02/24/xml-literals-in-scala.html > > An easy protection against tampered XML data vulnerabilities > is DTD or some other XML schema language. It can at least catch > problems that are in the scope of the schema language. From dieter at handshake.de Fri Sep 24 12:14:56 2021 From: dieter at handshake.de (Dieter Maurer) Date: Fri, 24 Sep 2021 18:14:56 +0200 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: Message-ID: <24909.63872.434489.521344@ixdm.fritz.box> Steve Keller wrote at 2021-9-24 11:48 +0200: >Why do some built-in Python functions feel so differently: Because the typical use cases are different >For example sum(), all(), any() expect exactly one argument which is a >sequence to operate on, i.e. a list, an iterator or a generator etc. > > sum([1,2,3,4]) > sum(range(1, 101)) > sum(2**i for i in range(10)) > all([True, False]) > any(even, {1,2,3,4}) You use those functions typically on a large number of operands, typically already collected together via some form of iterator. If you want to compute the sum of a few operands, you would usually not use `sum` but `o1 + o2 + ...`. >while other functions like set.union() and set.intersection() work on >a list of arguments but not on a sequence: > > set.intersection({1,2,3}, {3,4,5}) Those operations are typically applied to a small number of operands. You would need to build an iterator in that case should the functions only accept iterators. -- Dieter From dieter at handshake.de Fri Sep 24 12:18:00 2021 From: dieter at handshake.de (Dieter Maurer) Date: Fri, 24 Sep 2021 18:18:00 +0200 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: Message-ID: <24909.64056.735078.892613@ixdm.fritz.box> Stefan Ram wrote at 2021-9-24 14:53 GMT: >Steve Keller writes: >>Why do some built-in Python functions feel so differently: > >|>>> s = set() >|>>> s.add( 1 ) >|>>> > > >|>>> l = [] >|>>> l.add( 1 ) > >| >|Traceback (most recent call last): >| File "", line 1, in >|AttributeError: 'list' object has no attribute 'add' >| >|>>> l.append( 1 ) A list is ordered. Therefore, it is important where in this order an element is added. Thus, for a list, `append` is a better name than `add` -- because it already tells us in the name where it adds the new element. A set is unordered. Therefore, the name `append` does not make sense for a set. -- Dieter From rosuav at gmail.com Fri Sep 24 13:48:27 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 25 Sep 2021 03:48:27 +1000 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.64056.735078.892613@ixdm.fritz.box> Message-ID: On Sat, Sep 25, 2021 at 3:42 AM Stefan Ram wrote: > > "Dieter Maurer" writes: > >A list is ordered. Therefore, it is important where > >in this order an element is added. Thus, for a list, > >`append` is a better name than `add` -- because it already > >tells us in the name where it adds the new element. > > In a collection of texts, which is not large but mixed from > many different fields and genres, I find (using a Python > script, of course) eight hits for "added to the list" : > > |s and rock n roll can be added to the list. As - Taylor, 2012 > | of opinion was probably added to the list tow - from a dictionary > |gg and his wife might be added to the list of - Sir Walter Scott > |ships when your name was added to the list. In - Percy Bysshe Shelley > |em that wealth should be added to the list. No - Henry > |n was discovered and was added to the list of - from a dictionary > |nd said his name must be added to the list, or - Mark Twain > > . There was no hit for "appended to the list". > > When one says "add something to a list", it is usually understood > that one adds it at the /end/. In the case of traditional written > lists it is not possible in any other way. > When I add something to the shopping list, it is not added to the end. It is added anywhere that there is room. If you care about the sequence, you would say "add to the end". Or, using the more technical and briefer word, "append". Most of the lists you're seeing there are not being added to the end of; for instance, I would guess that quite a few of them are inherently sorted lists, so you would be adding a person's name in affabeck lauder, or adding something to a particular ranked position, or whatever else. This is not the same thing. ChrisA From hjp-python at hjp.at Fri Sep 24 14:29:17 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Fri, 24 Sep 2021 20:29:17 +0200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 2021-09-21 19:46:19 -0700, Dan Stromberg wrote: > On Tue, Sep 21, 2021 at 7:26 PM Michael F. Stemper < > michael.stemper at gmail.com> wrote: > > If XML is not the way to package data, what is the recommended > > approach? > > > > I prefer both JSON and YAML over XML. > > XML has both elements and tags, but it didn't really need both. I think you meant "both elements and attributes". Tags are how you denote elements, so they naturally go together. I agree that for representing data (especially object-oriented data) the distiction between (sub-)elements and attributes seems moot (should represent that field as an attribute or a field?), but don't forget that XML was intended to replace SGML, and that SGML was intended to mark up text, not represent any data. Would you really want to write

Mr. Smiths point was corroborated by Ms. Jones point that bla, bla, which seemed more plausibe than Mr. Willam claim that blub, blub. as

Mr. Smiths point was corroborated by Ms. Jones point that bla, bla, which seemed more plausibe than Mr. Willam claim that blub, blub. or

Mr. Smith<(defendant>s point was corroborated by Ms. Jones point that bla, bla, which seemed more plausibe than Mr. Willam claim that blub, blub. ? I probably chose an example (no doubt influenced by the fact that SGML was originally invented to digitize court decisions) which is too simple (in HTML I often see many attributes on a single element, even with CSS), but even here you can see that attributes add clarity. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Fri Sep 24 14:34:10 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Fri, 24 Sep 2021 20:34:10 +0200 Subject: XML Considered Harmful In-Reply-To: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> References: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> Message-ID: On 2021-09-23 06:53:10 -0600, Mats Wichmann wrote: > The problem with csv is that a substantial chunk of the world seems to > live inside Excel, This is made sp much worse by Excel being exceptionally bad at reading CSV. Several hundred genes were recently renamed because Excel was unable to read their names as simply strings and insisted on interpreting them as something else (e.g. dates). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Fri Sep 24 14:59:23 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Fri, 24 Sep 2021 20:59:23 +0200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 2021-09-21 13:12:10 -0500, Michael F. Stemper wrote: > I read this page right when I was about to write an XML parser > to get data into the code for a research project I'm working on. > It seems to me that XML is the right approach for this sort of > thing, especially since the data is hierarchical in nature. > > Does the advice on that page mean that I should find some other > way to get data into my programs, or does it refer to some kind > of misuse/abuse of XML for something that it wasn't designed > for? > > If XML is not the way to package data, what is the recommended > approach? There are a gazillion formats and depending on your needs one of them might be perfect. Or you may have to define you own bespoke format (I mean, nobody (except Matt Parker) tries to represent images or videos as CSVs: There's PNG and JPEG and WEBP and H.264 and AV1 and whatever for that). Of the three formats discussed here my take is: CSV: Good for tabular data of a single data type (strings). As soon as there's a second data type (numbers, dates, ...) you leave standard territory and are into "private agreements". JSON: Has a few primitive data types (bool, number, string) and a two compound types (list, dict(string -> any)). Still missing many frequently used data types (e.g. dates) and has no standard way to denote composite types. But its simple and if it's sufficient for your needs, use it. XML: Originally invented for text markup, and that shows. Can represent different types (via tags), can define those types (via DTD and/or schemas), can identify schemas in a globally-unique way and you can mix them all in a single document (and there are tools available to validate your files). But those features make it very complex (you almost certainly don't want to write your own parser) and you really have to understand the data model (especiall namespaces) to use it. You can of course represent any data in any format if you jump through enough hoops, but the real question is "does the data I have fit naturally within the data model of the format I'm trying to use". If it doesn't, look for something else. For me, CSV, JSON and XML form a hierarchy where each can naturally represent all the data of its predecessors, but not vice versa. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From keller.steve at gmx.de Fri Sep 24 18:15:10 2021 From: keller.steve at gmx.de (Steve Keller) Date: Sat, 25 Sep 2021 00:15:10 +0200 Subject: Different "look and feel" of some built-in functions References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: "Dieter Maurer" writes: > Steve Keller wrote at 2021-9-24 11:48 +0200: > >Why do some built-in Python functions feel so differently: > > Because the typical use cases are different > > [...] > > >while other functions like set.union() and set.intersection() work on > >a list of arguments but not on a sequence: > > > > set.intersection({1,2,3}, {3,4,5}) > > Those operations are typically applied to a small number > of operands. You would need to build an iterator in that > case should the functions only accept iterators. In my experience I need intersection and union on a list of sets, set of sets or a map() returning sets much more often. E.g. in some mathematical problems, and in automaton theory (IIRC, computing of LR or LALR sets, converting NFA to DFA, minimizing DFA), many graph algorithms traversing the nodes (shortest path, ...), and so on). Intersection and union of two sets is actually often seen in na?ve programming in loops like for t in (some_sequence): s.union(t) where set.union(*(some_sequence)) would be much more elegant. BTW, I like how the min() and max() functions allow both ways of being called. Steve From PythonList at DancesWithMice.info Fri Sep 24 18:51:36 2021 From: PythonList at DancesWithMice.info (dn) Date: Sat, 25 Sep 2021 10:51:36 +1200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> On 25/09/2021 06.59, Peter J. Holzer wrote: > There are a gazillion formats and depending on your needs one of them > might be perfect. Or you may have to define you own bespoke format (I > mean, nobody (except Matt Parker) tries to represent images or videos as > CSVs: There's PNG and JPEG and WEBP and H.264 and AV1 and whatever for > that). > > Of the three formats discussed here my take is: > > CSV: Good for tabular data of a single data type (strings). As soon as > there's a second data type (numbers, dates, ...) you leave standard > territory and are into "private agreements". > > JSON: Has a few primitive data types (bool, number, string) and a two > compound types (list, dict(string -> any)). Still missing many > frequently used data types (e.g. dates) and has no standard way to > denote composite types. But its simple and if it's sufficient for your > needs, use it. > > XML: Originally invented for text markup, and that shows. Can represent > different types (via tags), can define those types (via DTD and/or > schemas), can identify schemas in a globally-unique way and you can mix > them all in a single document (and there are tools available to validate > your files). But those features make it very complex (you almost > certainly don't want to write your own parser) and you really have to > understand the data model (especiall namespaces) to use it. and YAML? -- Regards, =dn From rosuav at gmail.com Fri Sep 24 19:00:06 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 25 Sep 2021 09:00:06 +1000 Subject: XML Considered Harmful In-Reply-To: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: On Sat, Sep 25, 2021 at 8:53 AM dn via Python-list wrote: > > On 25/09/2021 06.59, Peter J. Holzer wrote: > > There are a gazillion formats and depending on your needs one of them > > might be perfect. Or you may have to define you own bespoke format (I > > mean, nobody (except Matt Parker) tries to represent images or videos as > > CSVs: There's PNG and JPEG and WEBP and H.264 and AV1 and whatever for > > that). > > > > Of the three formats discussed here my take is: > > > > CSV: Good for tabular data of a single data type (strings). As soon as > > there's a second data type (numbers, dates, ...) you leave standard > > territory and are into "private agreements". > > > > JSON: Has a few primitive data types (bool, number, string) and a two > > compound types (list, dict(string -> any)). Still missing many > > frequently used data types (e.g. dates) and has no standard way to > > denote composite types. But its simple and if it's sufficient for your > > needs, use it. > > > > XML: Originally invented for text markup, and that shows. Can represent > > different types (via tags), can define those types (via DTD and/or > > schemas), can identify schemas in a globally-unique way and you can mix > > them all in a single document (and there are tools available to validate > > your files). But those features make it very complex (you almost > > certainly don't want to write your own parser) and you really have to > > understand the data model (especiall namespaces) to use it. > > and YAML? Invented because there weren't enough markup languages, so we needed another? ChrisA From DomainAdmin at DancesWithMice.info Fri Sep 24 19:26:44 2021 From: DomainAdmin at DancesWithMice.info (David L Neil) Date: Sat, 25 Sep 2021 11:26:44 +1200 Subject: XML Considered Harmful In-Reply-To: References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: <1092b736-985e-6e1a-9b4c-534bbcd674de@DancesWithMice.info> On 25/09/2021 11.00, Chris Angelico wrote: > Invented because there weren't enough markup languages, so we needed another? Anything You Can Do I Can Do Better https://www.youtube.com/watch?v=_UB1YAsPD6U -- Regards =dn From greg.ewing at canterbury.ac.nz Fri Sep 24 19:15:01 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 25 Sep 2021 11:15:01 +1200 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: On 25/09/21 10:15 am, Steve Keller wrote: > BTW, I like how the min() and max() functions allow both ways of being > called. That wouldn't work for set.union and set.intersection, because as was pointed out, they're actually methods, so set.union(some_seq) is a type error: >>> a = {1, 2} >>> b = {3, 4} >>> set.union([a, b]) Traceback (most recent call last): File "", line 1, in TypeError: descriptor 'union' for 'set' objects doesn't apply to a 'list' object I suppose they could be fiddled somehow to make it possible, but that would be turning them into special cases that break the rules. It would be better to provide separate functions, as was done with sum(). -- Greg From jon+usenet at unequivocal.eu Fri Sep 24 19:32:47 2021 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Fri, 24 Sep 2021 23:32:47 -0000 (UTC) Subject: XML Considered Harmful References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: On 2021-09-24, Chris Angelico wrote: > On Sat, Sep 25, 2021 at 8:53 AM dn via Python-list > wrote: >> On 25/09/2021 06.59, Peter J. Holzer wrote: >> > CSV: Good for tabular data of a single data type (strings). As soon as >> > there's a second data type (numbers, dates, ...) you leave standard >> > territory and are into "private agreements". CSV is not good for strings, as there is no one specification of how to encode things like newlines and commas within the strings, so you may find that your CSV data transfer fails or even silently corrupts data. >> > JSON: Has a few primitive data types (bool, number, string) and a two >> > compound types (list, dict(string -> any)). Still missing many >> > frequently used data types (e.g. dates) and has no standard way to >> > denote composite types. But its simple and if it's sufficient for your >> > needs, use it. JSON Schema provides a way to denote composite types. >> > XML: Originally invented for text markup, and that shows. Can represent >> > different types (via tags), can define those types (via DTD and/or >> > schemas), can identify schemas in a globally-unique way and you can mix >> > them all in a single document (and there are tools available to validate >> > your files). But those features make it very complex (you almost >> > certainly don't want to write your own parser) and you really have to >> > understand the data model (especiall namespaces) to use it. >> >> and YAML? > > Invented because there weren't enough markup languages, so we needed > another? Invented as a drunken bet that got out of hand, and used by people who don't realise this. From oscar.j.benjamin at gmail.com Fri Sep 24 20:54:58 2021 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 25 Sep 2021 01:54:58 +0100 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: On Sat, 25 Sept 2021 at 00:37, Greg Ewing wrote: > On 25/09/21 10:15 am, Steve Keller wrote: > > BTW, I like how the min() and max() functions allow both ways of being > > called. > > That wouldn't work for set.union and set.intersection, because as > was pointed out, they're actually methods, so set.union(some_seq) > is a type error: > > >>> a = {1, 2} > >>> b = {3, 4} > >>> set.union([a, b]) > Traceback (most recent call last): > File "", line 1, in > TypeError: descriptor 'union' for 'set' objects doesn't apply to a > 'list' object > > I suppose they could be fiddled somehow to make it possible, but > that would be turning them into special cases that break the rules. > It would be better to provide separate functions, as was done with > sum(). > A separate union function would be good. Even in a situation where all inputs are assured to be sets the set.union method fails the base case: >>> sets = [] >>> set.union(*sets) Traceback (most recent call last): File "", line 1, in TypeError: descriptor 'union' of 'set' object needs an argument In the case of intersection perhaps the base case should be undefined. -- Oscar From rosuav at gmail.com Fri Sep 24 20:58:05 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 25 Sep 2021 10:58:05 +1000 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin wrote: > > On Sat, 25 Sept 2021 at 00:37, Greg Ewing > wrote: > > > On 25/09/21 10:15 am, Steve Keller wrote: > > > BTW, I like how the min() and max() functions allow both ways of being > > > called. > > > > That wouldn't work for set.union and set.intersection, because as > > was pointed out, they're actually methods, so set.union(some_seq) > > is a type error: > > > > >>> a = {1, 2} > > >>> b = {3, 4} > > >>> set.union([a, b]) > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: descriptor 'union' for 'set' objects doesn't apply to a > > 'list' object > > > > I suppose they could be fiddled somehow to make it possible, but > > that would be turning them into special cases that break the rules. > > It would be better to provide separate functions, as was done with > > sum(). > > > > A separate union function would be good. Even in a situation where all > inputs are assured to be sets the set.union method fails the base case: > > >>> sets = [] > >>> set.union(*sets) > Traceback (most recent call last): > File "", line 1, in > TypeError: descriptor 'union' of 'set' object needs an argument > > In the case of intersection perhaps the base case should be undefined. > Rather than calling the unbound method, why not just call it on an empty set? That defines your base case as well. set().union(*sets) ChrisA From oscar.j.benjamin at gmail.com Fri Sep 24 21:11:16 2021 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 25 Sep 2021 02:11:16 +0100 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: On Sat, 25 Sept 2021 at 02:01, Chris Angelico wrote: > On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin > wrote: > > > > On Sat, 25 Sept 2021 at 00:37, Greg Ewing > > wrote: > > > I suppose they could be fiddled somehow to make it possible, but > > > that would be turning them into special cases that break the rules. > > > It would be better to provide separate functions, as was done with > > > sum(). > > > > > > > A separate union function would be good. Even in a situation where all > > inputs are assured to be sets the set.union method fails the base case: > > > > >>> sets = [] > > >>> set.union(*sets) > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: descriptor 'union' of 'set' object needs an argument > > > > In the case of intersection perhaps the base case should be undefined. > > > > Rather than calling the unbound method, why not just call it on an > empty set? That defines your base case as well. > > set().union(*sets) > That is indeed what I do but it seems unnatural compared to simply union(*sets). It shouldn't be necessary to create a redundant empty set just to compute the union of some other sets. If there was a union function then I don't think I would ever use the union method. -- Oscar From rosuav at gmail.com Fri Sep 24 21:14:08 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 25 Sep 2021 11:14:08 +1000 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: On Sat, Sep 25, 2021 at 11:11 AM Oscar Benjamin wrote: > > On Sat, 25 Sept 2021 at 02:01, Chris Angelico wrote: >> >> On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin >> wrote: >> > >> > On Sat, 25 Sept 2021 at 00:37, Greg Ewing >> > wrote: >> > > I suppose they could be fiddled somehow to make it possible, but >> > > that would be turning them into special cases that break the rules. >> > > It would be better to provide separate functions, as was done with >> > > sum(). >> > > >> > >> > A separate union function would be good. Even in a situation where all >> > inputs are assured to be sets the set.union method fails the base case: >> > >> > >>> sets = [] >> > >>> set.union(*sets) >> > Traceback (most recent call last): >> > File "", line 1, in >> > TypeError: descriptor 'union' of 'set' object needs an argument >> > >> > In the case of intersection perhaps the base case should be undefined. >> > >> >> Rather than calling the unbound method, why not just call it on an >> empty set? That defines your base case as well. >> >> set().union(*sets) > > > That is indeed what I do but it seems unnatural compared to simply union(*sets). It shouldn't be necessary to create a redundant empty set just to compute the union of some other sets. If there was a union function then I don't think I would ever use the union method. > Maybe, but if you start with a set, then you define the base case, and it also is quite happy to take non-set arguments: >>> set().union([1,2,3], map(int, "234"), {3:"three",4:"four",5:"five"}) {1, 2, 3, 4, 5} ChrisA From greg.ewing at canterbury.ac.nz Fri Sep 24 19:46:14 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 25 Sep 2021 11:46:14 +1200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 25/09/21 6:29 am, Peter J. Holzer wrote: > don't forget that > XML was intended to replace SGML, and that SGML was intended to mark up > text, not represent any data. And for me this is the number one reason why XML is the wrong tool for almost everything it's used for nowadays. It's bizarre. It's as though there were a large community of professional builders who insisted on using hammers to drive scews, and extolled the advantages of doing so. -- Greg From greg.ewing at canterbury.ac.nz Fri Sep 24 20:01:10 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 25 Sep 2021 12:01:10 +1200 Subject: XML Considered Harmful In-Reply-To: References: <9dba6f39-64c8-9077-a866-e0335eabf79b@wichmann.us> Message-ID: On 25/09/21 6:34 am, Peter J. Holzer wrote: > Several hundred genes were recently renamed because Excel was unable to > read their names as simply strings and insisted on interpreting them as > something else (e.g. dates). Another fun one I've come across is interpreting phone numbers as floating point and writing them out again with exponents... -- Greg From greg.ewing at canterbury.ac.nz Fri Sep 24 20:14:01 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 25 Sep 2021 12:14:01 +1200 Subject: XML Considered Harmful In-Reply-To: References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: On 25/09/21 10:51 am, dn wrote: >> XML: Originally invented for text markup, and that shows. Can represent >> different types (via tags), can define those types (via DTD and/or >> schemas), can identify schemas in a globally-unique way and you can mix >> them all in a single document (and there are tools available to validate >> your files). But those features make it very complex And for all that complexity, it still doesn't map very well onto the kinds of data structures used inside programs (lists, structs, etc.), so you end up having to build those structures on top of it, and everyone does that in a different way. -- Greg From greg.ewing at canterbury.ac.nz Fri Sep 24 20:16:11 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 25 Sep 2021 12:16:11 +1200 Subject: XML Considered Harmful In-Reply-To: References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: On 25/09/21 11:00 am, Chris Angelico wrote: > On Sat, Sep 25, 2021 at 8:53 AM dn via Python-list > wrote: >> >> and YAML? > > Invented because there weren't enough markup languages, so we needed another? There were *too many* markup languages, so we invented another! -- Greg From oscar.j.benjamin at gmail.com Sat Sep 25 04:56:18 2021 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 25 Sep 2021 09:56:18 +0100 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: On Sat, 25 Sept 2021 at 02:16, Chris Angelico wrote: > On Sat, Sep 25, 2021 at 11:11 AM Oscar Benjamin > wrote: > > > > On Sat, 25 Sept 2021 at 02:01, Chris Angelico wrote: > >> > >> On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin > >> wrote: > >> > > >> > On Sat, 25 Sept 2021 at 00:37, Greg Ewing < > greg.ewing at canterbury.ac.nz> > >> > wrote: > >> > > I suppose they could be fiddled somehow to make it possible, but > >> > > that would be turning them into special cases that break the rules. > >> > > It would be better to provide separate functions, as was done with > >> > > sum(). > >> > > > >> > > >> > A separate union function would be good. Even in a situation where all > >> > inputs are assured to be sets the set.union method fails the base > case: > >> > > >> > >>> sets = [] > >> > >>> set.union(*sets) > >> > Traceback (most recent call last): > >> > File "", line 1, in > >> > TypeError: descriptor 'union' of 'set' object needs an argument > >> > > >> > In the case of intersection perhaps the base case should be undefined. > >> > > >> > >> Rather than calling the unbound method, why not just call it on an > >> empty set? That defines your base case as well. > >> > >> set().union(*sets) > > > > > > That is indeed what I do but it seems unnatural compared to simply > union(*sets). It shouldn't be necessary to create a redundant empty set > just to compute the union of some other sets. If there was a union function > then I don't think I would ever use the union method. > > > > Maybe, but if you start with a set, then you define the base case, and > it also is quite happy to take non-set arguments: > > >>> set().union([1,2,3], map(int, "234"), {3:"three",4:"four",5:"five"}) > {1, 2, 3, 4, 5} Actually it should be union(sets) rather than union(*sets). A more realistic example is when you need the union of sets given by something like a generator expression so it looks like: items = set().union(*(f(x) for x in stuff)) With a union function it should look like: items = union(f(x) for x in stuff) -- Oscar From hjp-python at hjp.at Sat Sep 25 06:46:55 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 25 Sep 2021 12:46:55 +0200 Subject: XML Considered Harmful In-Reply-To: References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: On 2021-09-24 23:32:47 -0000, Jon Ribbens via Python-list wrote: > On 2021-09-24, Chris Angelico wrote: > > On Sat, Sep 25, 2021 at 8:53 AM dn via Python-list > > wrote: > >> On 25/09/2021 06.59, Peter J. Holzer wrote: > >> > CSV: Good for tabular data of a single data type (strings). As soon as > >> > there's a second data type (numbers, dates, ...) you leave standard > >> > territory and are into "private agreements". > > CSV is not good for strings, as there is no one specification of how to > encode things like newlines and commas within the strings, so you may > find that your CSV data transfer fails or even silently corrupts data. Those two cases are actually pretty straightforward: Just enclose the field in quotes. Handling quotes is less standardized. I think doubling quotes is much more common than an escape character, but I've certainly seen both. But if you get down to it, the problems with CSV start at a much lower level: 1) The encoding is not defined. These days UTF-8 (with our without BOM) is pretty common, but I still regularly get files in Windows-1252 encoding and occasionally something else. 2) The record separator isn't defined. CRLF is most common, followed by LF. But just recently I got a file with CR (Does Eurostat still use some Macs with MacOS 9?) 3) The field separator isn't defined. Officially the format is known as "comma separated values", but in my neck of the woods it's actually semicolon-separated in the vast majority of cases. So even for the most simple files there are three parameters the sender and the receiver have to agree on. > >> > JSON: Has a few primitive data types (bool, number, string) and a two > >> > compound types (list, dict(string -> any)). Still missing many > >> > frequently used data types (e.g. dates) and has no standard way to > >> > denote composite types. But its simple and if it's sufficient for your > >> > needs, use it. > > JSON Schema provides a way to denote composite types. I probably wasn't clear what I meant. In XML, every element has a tag, which is basically its type. So by looking at an XML file (without reference to a schema) you can tell what each element is. And a validator can say something like "expected a 'product' or 'service' element here but found a 'person'". In JSON everything is just an object or a list. You may guess that an object with a field "product_id" is a product, but is one with "name": "Billy" a person or a piece of furniture? I'm not familiar with JSON schema (I know that it exists and I've read a tutorial or two but I've never used it in a real project), but as far as I know it doesn't change that. It describes the structure of a JSON document but it doesn't add type information to that document. So a validator can at best guess what the malformed thing it just found was supposed to be. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From Karsten.Hilbert at gmx.net Sat Sep 25 09:24:57 2021 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sat, 25 Sep 2021 15:24:57 +0200 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: Am Fri, Sep 24, 2021 at 08:59:23PM +0200 schrieb Peter J. Holzer: > JSON: Has a few primitive data types (bool, number, string) and a two > compound types (list, dict(string -> any)). Still missing many > frequently used data types (e.g. dates) But that (dates) at least has a well-known mapping to string, which makes it usable within JSON. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B From mal at europython.eu Sat Sep 25 11:42:02 2021 From: mal at europython.eu (Marc-Andre Lemburg) Date: Sat, 25 Sep 2021 17:42:02 +0200 Subject: EuroPython Society: General Assembly 2021 Message-ID: <2c733060-d652-22e3-7492-191ac3acf7c6@europython.eu> As last year, we are holding the General Assembly (GA) of the EuroPython Society (EPS) online for this year. General Assembly ---------------- In accordance with our bylaws, we are calling for the EuroPython Society General Assembly to be held on Sunday, October 10th 2020, from 19:00 - 21:00 CEST. We will use a Zoom meeting to hold the event and send around the URL closer to the event. All EPS members are welcome to join and vote at the meeting. Please be aware that we cannot allow non-EPS members to join, as we often do at the in-person GAs we hold at the conference, since we would then not be able to control access to the Zoom call. Board Nominations ----------------- As every year, we will vote in a new board. We have already sent out the list of board nominations in a separate blog post on 2021-09-23. Please see that post for details on the candidates and the nomination process. Motions by the Members ---------------------- EuroPython Society Members can propose motions to be put forward and voted on at the General Assembly. If you want to put forward a motion, please send this to board at europython.eu no later than Sunday, 2021-10-03, so that we can add them to the agenda. The bylaws require that any such motions be announced no later than 5 days before the GA and we will need time to clarify details and prepare the agenda. Agenda ------ We will publish the agenda with all motions put forward by the board and the members on Tuesday, 2020-10-05. The agenda will follow the template set out in our bylaws under section 8. https://www.europython-society.org/bylaws Reports ------- All reports for the GA will be published on Friday, 2020-10-08, to give the members enough time to read them and prepare questions. We?ll then answer any questions at the GA. Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://www.europython-society.org/europython-society-general-assembly-2021/ Tweet: https://twitter.com/europythons/status/1441788705514065921 Thanks, -- EuroPython Society https://www.europython-society.org/ From dieter at handshake.de Sat Sep 25 12:26:21 2021 From: dieter at handshake.de (Dieter Maurer) Date: Sat, 25 Sep 2021 18:26:21 +0200 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.64056.735078.892613@ixdm.fritz.box> Message-ID: <24911.19885.904633.336077@ixdm.fritz.box> Stefan Ram wrote at 2021-9-24 16:48 GMT: >"Dieter Maurer" writes: >>A list is ordered. Therefore, it is important where >>in this order an element is added. Thus, for a list, >>`append` is a better name than `add` -- because it already >>tells us in the name where it adds the new element. > > In a collection of texts, which is not large but mixed from > many different fields and genres, I find (using a Python > script, of course) eight hits for "added to the list" : > >|s and rock n roll can be added to the list. As - Taylor, 2012 >| of opinion was probably added to the list tow - from a dictionary >|gg and his wife might be added to the list of - Sir Walter Scott >|ships when your name was added to the list. In - Percy Bysshe Shelley >|em that wealth should be added to the list. No - Henry >|n was discovered and was added to the list of - from a dictionary >|nd said his name must be added to the list, or - Mark Twain While a list is ordered, applications using a list may not be interested in the particular order and thus just speak of "add to the list" rather than "append to the list". Nevertheless, I find the name `append` far better than `add` for the list type - because it describes better what it is doing. I am a big fan of "speaking names". > . There was no hit for "appended to the list". > > When one says "add something to a list", it is usually understood > that one adds it at the /end/. In the case of traditional written > lists it is not possible in any other way. Really? Prepending should be as possible as appending (if one disregards implementation details). -- Dieter From rosuav at gmail.com Sat Sep 25 12:38:30 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 26 Sep 2021 02:38:30 +1000 Subject: Different "look and feel" of some built-in functions In-Reply-To: <24911.19885.904633.336077@ixdm.fritz.box> References: <24909.64056.735078.892613@ixdm.fritz.box> <24911.19885.904633.336077@ixdm.fritz.box> Message-ID: On Sun, Sep 26, 2021 at 2:27 AM Dieter Maurer wrote: > > Stefan Ram wrote at 2021-9-24 16:48 GMT: > >"Dieter Maurer" writes: > >>A list is ordered. Therefore, it is important where > >>in this order an element is added. Thus, for a list, > >>`append` is a better name than `add` -- because it already > >>tells us in the name where it adds the new element. > > > > In a collection of texts, which is not large but mixed from > > many different fields and genres, I find (using a Python > > script, of course) eight hits for "added to the list" : > > > >|s and rock n roll can be added to the list. As - Taylor, 2012 > >| of opinion was probably added to the list tow - from a dictionary > >|gg and his wife might be added to the list of - Sir Walter Scott > >|ships when your name was added to the list. In - Percy Bysshe Shelley > >|em that wealth should be added to the list. No - Henry > >|n was discovered and was added to the list of - from a dictionary > >|nd said his name must be added to the list, or - Mark Twain > > While a list is ordered, > applications using a list may not be interested in the particular > order and thus just speak of "add to the list" > rather than "append to the list". > > Nevertheless, I find the name `append` far better than `add` for > the list type - because it describes better what it is doing. > I am a big fan of "speaking names". Yeah, so I would say it makes perfectly good sense to do something like this: def add_customer(...): cust = ... customers.append(cust) where it's "add" in your application, but "append" in Python's list object. ChrisA From dieter at handshake.de Sat Sep 25 12:40:41 2021 From: dieter at handshake.de (Dieter Maurer) Date: Sat, 25 Sep 2021 18:40:41 +0200 Subject: Different "look and feel" of some built-in functions In-Reply-To: References: <24909.63872.434489.521344@ixdm.fritz.box> Message-ID: <24911.20745.950224.626585@ixdm.fritz.box> Steve Keller wrote at 2021-9-25 00:15 +0200: >"Dieter Maurer" writes: > >> Steve Keller wrote at 2021-9-24 11:48 +0200: >> >Why do some built-in Python functions feel so differently: >> >> Because the typical use cases are different >> >> [...] >> >> >while other functions like set.union() and set.intersection() work on >> >a list of arguments but not on a sequence: >> > >> > set.intersection({1,2,3}, {3,4,5}) >> >> Those operations are typically applied to a small number >> of operands. You would need to build an iterator in that >> case should the functions only accept iterators. > >In my experience I need intersection and union on a list of sets, set >of sets or a map() returning sets much more often. E.g. in some >mathematical problems, and in automaton theory (IIRC, computing of LR >or LALR sets, converting NFA to DFA, minimizing DFA), many graph >algorithms traversing the nodes (shortest path, ...), and so on). I, too, occasionally work with set operations on many operands -- in the context of `Zope`, more precisely `BTrees`. There, I have both `union(op1, op2)` and `multiunion(iterator)` (`multiunion` is there primarily for efficiency reasons). I you often have operations on large sets of operands, you could define corresponding "convernience functions". -- Dieter From jon+usenet at unequivocal.eu Sat Sep 25 07:49:58 2021 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Sat, 25 Sep 2021 11:49:58 -0000 (UTC) Subject: XML Considered Harmful References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> Message-ID: On 2021-09-25, Peter J. Holzer wrote: > On 2021-09-24 23:32:47 -0000, Jon Ribbens via Python-list wrote: >> JSON Schema provides a way to denote composite types. > > I probably wasn't clear what I meant. In XML, every element has a tag, > which is basically its type. So by looking at an XML file (without > reference to a schema) you can tell what each element is. And a > validator can say something like "expected a 'product' or 'service' > element here but found a 'person'". > > In JSON everything is just an object or a list. You may guess that an > object with a field "product_id" is a product, but is one with "name": > "Billy" a person or a piece of furniture? > > I'm not familiar with JSON schema (I know that it exists and I've read a > tutorial or two but I've never used it in a real project), but as far as > I know it doesn't change that. It describes the structure of a JSON > document but it doesn't add type information to that document. So a > validator can at best guess what the malformed thing it just found was > supposed to be. JSON Schema absolutely does change that. You can create named types and specify where they may appear in the document. With a well-defined schema you do not need to make any guesses about what type something is. From michael.stemper at gmail.com Sat Sep 25 16:20:19 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Sat, 25 Sep 2021 15:20:19 -0500 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 21/09/2021 13.12, Michael F. Stemper wrote: > If XML is not the way to package data, what is the recommended > approach? Well, there have been a lot of ideas put forth on this thread, many more than I expected. I'd like to thank everyone who took the time to contribute. Most of the reasons given for avoiding XML appear to be along the lines of "XML has all of these different options that it supports." However, it seems that I could ignore 99% of those things and just use a teeny subset of its capabilities. For instance, if I modeled a fuel like this: ton 21.96 18.2 and a generating unit like this: why would the fact that I could have chosen, instead, to model the unit of measure as an attribute of the fuel, or its name as a sub-element matter? Once the modeling decision has been made, all of the decisions that might have been would seem to be irrelevant. Some years back, IEC's TC57 came up with CIM[1]. This nailed down a lot of decisions. The fact that other decisions could have been made doesn't seem to keep utilities from going forward with it as an enterprise-wide data model. My current interests are not anywhere so expansive, but it seems that the situations are at least similar: 1. Look at an endless range of options for a data model. 2. Pick one. 3. Run with it. To clearly state my (revised) question: Why does the existence of XML's many options cause a problem for my use case? Other reactions: Somebody pointed out that some approaches would require that I climb a learning curve. That's appreciated, although learning new things is always good. NestedText looks cool, and a lot like YAML. Having not gotten around to playing with YAML yet, I was surprised to learn that it tries to guess data types. This sounds as if it could lead to the same type of problems that led to the names of some genes being turned into dates. It was suggested that I use an RDBMS, such as sqlite3, for the input data. I've used sqlite3 for real-time data exchange between concurrently-running programs. However, I don't see syntax like: sqlite> INSERT INTO Fuels ...> (name,uom,price,heat_content) ...> VALUES ("Montana Sub-Bituminous", "ton", 21.96, 13.65); as being nearly as readable as the XML that I've sketched above. Yeah, I could write a program to do this, but that doesn't really change anything, since I'd still need to get the data into the program. (Changing a value would be even worse, requiring the dreaded UPDATE INTO statement, instead of five seconds in vi.) Many of the problems listed for CSV, which come from its lack of standardization, seem similar to those given for XML. "Commas or tabs?" "How are new-lines represented?" If I was to use CSV, I'd be able to just pick answers. However, fitting hierarchical data into rows/columns just seems wrong, so I doubt that I'll end up going that way. As far as disambiguating authors, I believe that most journals are now expecting an ORCID[2] (which doesn't help with papers published before that came around). As far as use of XML to store program state, I wouldn't ever consider that. As noted above, I've used an RDBMS to do so. It handles all of the concurrency issues for me. The current use case is specifically for raw, static input. Fascinating to find out that XML was originally designed to mark up text, especially legal text. It was nice to be reminded of what Matt Parker looked like when he had hair. [1] [2] -- Michael F. Stemper Psalm 82:3-4 From nomail at adres.net Sat Sep 25 17:16:51 2021 From: nomail at adres.net (Sir Real) Date: Sat, 25 Sep 2021 16:16:51 -0500 Subject: Could not initilalize crash reporting DB Message-ID: I have a script that chooses a paragraph at random from a text file then uses that paragraph to generate and send an email message. It's set up to run on Windows 7 startup. It has run without issue more than 400 times. Recently two consecutive runs produced the following messages... Could not initialize crash reporting DB Cannot init crashpad with status: CRASHPAD_INIT_DB_ERROR Despite the messages the script otherwise ran as expected. Google searches turned up nothing useful. I was hoping that maybe someone here could tell me what these messages mean. From avigross at verizon.net Sat Sep 25 17:39:29 2021 From: avigross at verizon.net (Avi Gross) Date: Sat, 25 Sep 2021 17:39:29 -0400 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: <00a401d7b255$d27d72c0$77785840$@verizon.net> Michael, I don't care what you choose. Whatever works is fine for an internal use. But is the data scheme you share representative of your actual application? >From what I see below, unless the number of "point" variables is not always exactly four, the application might be handled well by any format that handles rectangular data, perhaps even CSV. You show a I mean anything like a data.frame can contain data columns like p1,p2,p3,p4 and a categorical one like IHRcurve_name. Or do you have a need for more variability such as an undetermined number of similar units in ways that might require more flexibility or be more efficient done another way? MOST of the discussion I am seeing here seems peripheral to getting you what you need for your situation and may require a learning curve to learn to use properly. Are you planning on worrying about how to ship your data encrypted, for example? Any file format you use for storage can presumably be encrypted and send and decrypted if that matters. So, yes, from an abstract standpoint we can discuss the merits of various approaches. If it matters that humans can deal with your data in a file or that it be able to be imported into a program like EXCEL, those are considerations. But if not, there are quite a few relatively binary formats where your program can save a snapshot of the data into a file and read it back in next time. I often do that in another language that lets me share variable including nested components such as the complex structures that come out of a statistical analysis or the components needed to make one or more graphs later. If you write the program that creates the darn things as well as the one that later reads them back in, you can do what you want. Or, did I miss something and others have already produced the data using other tools, in which case you have to read it in at least once/ -----Original Message----- From: Python-list On Behalf Of Michael F. Stemper Sent: Saturday, September 25, 2021 4:20 PM To: python-list at python.org Subject: Re: XML Considered Harmful On 21/09/2021 13.12, Michael F. Stemper wrote: > If XML is not the way to package data, what is the recommended > approach? Well, there have been a lot of ideas put forth on this thread, many more than I expected. I'd like to thank everyone who took the time to contribute. Most of the reasons given for avoiding XML appear to be along the lines of "XML has all of these different options that it supports." However, it seems that I could ignore 99% of those things and just use a teeny subset of its capabilities. For instance, if I modeled a fuel like this: ton 21.96 18.2 and a generating unit like this: why would the fact that I could have chosen, instead, to model the unit of measure as an attribute of the fuel, or its name as a sub-element matter? Once the modeling decision has been made, all of the decisions that might have been would seem to be irrelevant. Some years back, IEC's TC57 came up with CIM[1]. This nailed down a lot of decisions. The fact that other decisions could have been made doesn't seem to keep utilities from going forward with it as an enterprise-wide data model. My current interests are not anywhere so expansive, but it seems that the situations are at least similar: 1. Look at an endless range of options for a data model. 2. Pick one. 3. Run with it. To clearly state my (revised) question: Why does the existence of XML's many options cause a problem for my use case? Other reactions: Somebody pointed out that some approaches would require that I climb a learning curve. That's appreciated, although learning new things is always good. NestedText looks cool, and a lot like YAML. Having not gotten around to playing with YAML yet, I was surprised to learn that it tries to guess data types. This sounds as if it could lead to the same type of problems that led to the names of some genes being turned into dates. It was suggested that I use an RDBMS, such as sqlite3, for the input data. I've used sqlite3 for real-time data exchange between concurrently-running programs. However, I don't see syntax like: sqlite> INSERT INTO Fuels ...> (name,uom,price,heat_content) ...> VALUES ("Montana Sub-Bituminous", "ton", 21.96, 13.65); as being nearly as readable as the XML that I've sketched above. Yeah, I could write a program to do this, but that doesn't really change anything, since I'd still need to get the data into the program. (Changing a value would be even worse, requiring the dreaded UPDATE INTO statement, instead of five seconds in vi.) Many of the problems listed for CSV, which come from its lack of standardization, seem similar to those given for XML. "Commas or tabs?" "How are new-lines represented?" If I was to use CSV, I'd be able to just pick answers. However, fitting hierarchical data into rows/columns just seems wrong, so I doubt that I'll end up going that way. As far as disambiguating authors, I believe that most journals are now expecting an ORCID[2] (which doesn't help with papers published before that came around). As far as use of XML to store program state, I wouldn't ever consider that. As noted above, I've used an RDBMS to do so. It handles all of the concurrency issues for me. The current use case is specifically for raw, static input. Fascinating to find out that XML was originally designed to mark up text, especially legal text. It was nice to be reminded of what Matt Parker looked like when he had hair. [1] [2] -- Michael F. Stemper Psalm 82:3-4 -- https://mail.python.org/mailman/listinfo/python-list From 2QdxY4RzWzUUiLuE at potatochowder.com Sat Sep 25 17:48:48 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Sat, 25 Sep 2021 14:48:48 -0700 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 2021-09-25 at 15:20:19 -0500, "Michael F. Stemper" wrote: > ... For instance, if > I modeled a fuel like this: > > > ton > 21.96 > 18.2 > > > and a generating unit like this: > > > > > > > > > > > > > > > > > why would the fact that I could have chosen, instead, to model > the unit of measure as an attribute of the fuel, or its name > as a sub-element matter? Once the modeling decision has been > made, all of the decisions that might have been would seem to > be irrelevant. Disclaimer: I am not a big XML fan, for a number of reasons already stated in this thread. That said, please do include units in elements like heat_content, whether or not it's Joules/kilogram/K, and price, even if is the local currency in the only country to which your data applies. If there's a standard for your industry, or your company, or on some other level, then at least document what it is and that you're using it, so that the next person (which may be you a year from now) doesn't have to guess. You also never know when someone else on the other side of the planet will notice your work and try to duplicate it and/or update it (again, even if it's you). The fewer assumptions that person has to make, the better. From PythonList at DancesWithMice.info Sat Sep 25 17:56:04 2021 From: PythonList at DancesWithMice.info (dn) Date: Sun, 26 Sep 2021 10:56:04 +1300 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: <83e94330-d6fd-649d-800a-d35d8f20a919@DancesWithMice.info> On 26/09/2021 10.07, Stefan Ram wrote: > "Michael F. Stemper" writes: >> fitting hierarchical >> data into rows/columns just seems wrong > > There were hierarchical database management systems like > IMS by IBM based on that point of view. Today, almost all > hierarchical data that is stored in databases is stored > in relational databases. Maybe, the relational model has > proven superior to the hierarchical data model after all. Back in the days of mainframes (and when the Flintstones was 'filmed before a live studio audience') hierarchical DBs were considerably faster than RDBMS. Because of this, we used to take a daily 'snapshot' of the transaction DBs (in IMS) and make a 'copy' as DB2 relational DBs, which were (supposedly) used for MIS (Management Information Systems - as distinct from TPS (Transaction Processing Systems)). These days RDBMS are (a lot!) faster - much of which would be better expressed as: the hardware these days is a lot faster. Therefore an RDBMS is sufficiently responsive, and we no-longer need to maintain separate, 'parallel' systems (and multiple mainframes)! Cue: NoSQL justifications... Today's best example of an hierarchical DB is probably LDAP. It is most commonly used within the 'directory' of communications systems, eg email. Such waters muddied considerably by MSFT's attempts to 'improve' international 'standards' and integrate AD with Exchange (so don't go there!). There have been some well-engineered systems based on LDAP, eg organisational/personnel and part/component break-downs. That said, unless looking at something such as just-mentioned, overlaying hierarchy onto 3NF and using an RDBMS would be my first thought - but because of the recursive JOINs, I recommend something more capable than SQLite. -- Regards, =dn From PythonList at DancesWithMice.info Sat Sep 25 18:01:42 2021 From: PythonList at DancesWithMice.info (dn) Date: Sun, 26 Sep 2021 11:01:42 +1300 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: <393f27d0-c407-8330-f8a1-807b42a2b98b@DancesWithMice.info> On 26/09/2021 10.48, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-25 at 15:20:19 -0500, > "Michael F. Stemper" wrote: > >> ... For instance, if >> I modeled a fuel like this: >> >> >> ton >> 21.96 >> 18.2 >> ... > Disclaimer: I am not a big XML fan, for a number of reasons > already stated in this thread. > > That said, please do include units in elements like heat_content, > whether or not it's Joules/kilogram/K, and price, even if is the > local currency in the only country to which your data applies. > If there's a standard for your industry, or your company, or on > some other level, then at least document what it is and that > you're using it, so that the next person (which may be you a > year from now) doesn't have to guess. +1 *always* add unit attributes -- Regards, =dn From grant.b.edwards at gmail.com Sat Sep 25 18:31:04 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 25 Sep 2021 17:31:04 -0500 Subject: Posts from gmane no longer allowed? Message-ID: I've been reading (and posting to) this list for many years by pointing an NNTP client at news://gmane.comp.python.general. Sometime in the past few days posts started being refused: You have tried posting to gmane.comp.python.general, which is a unidirectional mailing list. Gmane can therefore not send this message to that mailing list. Was this a change made by the mailing list admins? If so, is it permanent? [Trying to send a plaintext e-mail via Gmail, but not sure if it's working.] -- Grant From * at eli.users.panix.com Sat Sep 25 18:46:59 2021 From: * at eli.users.panix.com (Eli the Bearded) Date: Sat, 25 Sep 2021 22:46:59 -0000 (UTC) Subject: XML Considered Harmful References: Message-ID: In comp.lang.python, Chris Angelico wrote: > Eli the Bearded <*@eli.users.panix.com> wrote: >> I'd use one of the netpbm formats instead of JPEG. PBM for one bit >> bitmaps, PGM for one channel (typically grayscale), PPM for three >> channel RGB, and PAM for anything else (two channel gray plus alpha, >> CMYK, RGBA, HSV, YCbCr, and more exotic formats). JPEG is tricky to >> map to CSV since it is a three channel format (YCbCr), where the >> channels are typically not at the same resolution. Usually Y is full >> size and the Cb and Cr channels are one quarter size ("4:2:0 chroma >> subsampling"). The unequal size of the channels does not lend itself >> to CSV, but I can't say it's impossible. > Examine prior art, and I truly do mean art, from Matt Parker: > https://www.youtube.com/watch?v=UBX2QQHlQ_I His spreadsheet is a PPM file, not a JPEG. You can tell because all of the cells are the same size. He also ignores vector graphics when considering digital images. Often they are rendered in what he calls "spreadsheets" but not always. I have a Vectrex, for example. Elijah ------ then there's typewriter art with non-square "pixels" From rosuav at gmail.com Sat Sep 25 19:14:15 2021 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 26 Sep 2021 09:14:15 +1000 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On Sun, Sep 26, 2021 at 9:09 AM Eli the Bearded <*@eli.users.panix.com> wrote: > > In comp.lang.python, Chris Angelico wrote: > > Eli the Bearded <*@eli.users.panix.com> wrote: > >> I'd use one of the netpbm formats instead of JPEG. PBM for one bit > >> bitmaps, PGM for one channel (typically grayscale), PPM for three > >> channel RGB, and PAM for anything else (two channel gray plus alpha, > >> CMYK, RGBA, HSV, YCbCr, and more exotic formats). JPEG is tricky to > >> map to CSV since it is a three channel format (YCbCr), where the > >> channels are typically not at the same resolution. Usually Y is full > >> size and the Cb and Cr channels are one quarter size ("4:2:0 chroma > >> subsampling"). The unequal size of the channels does not lend itself > >> to CSV, but I can't say it's impossible. > > Examine prior art, and I truly do mean art, from Matt Parker: > > https://www.youtube.com/watch?v=UBX2QQHlQ_I > > His spreadsheet is a PPM file, not a JPEG. You can tell because all of > the cells are the same size. > > He also ignores vector graphics when considering digital images. Often > they are rendered in what he calls "spreadsheets" but not always. I have > a Vectrex, for example. > > Elijah > ------ > then there's typewriter art with non-square "pixels" Ah, I remember playing around with line printer art. We mostly had Epsons and IBMs that did have some measure of graphical capabilities, but it was WAY faster to print text, so we sometimes did things the hacky and elegant way instead. ChrisA From grant.b.edwards at gmail.com Sun Sep 26 09:35:57 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 08:35:57 -0500 Subject: Posts from gmane no longer allowed? Message-ID: On 2021-09-25, Grant Edwards wrote: > I've been reading (and posting to) this list for many years by > pointing an NNTP client at > news://gmane.comp.python.general. Sometime in the past few days > posts started being refused: > > You have tried posting to gmane.comp.python.general, which is a > unidirectional mailing list. Gmane can therefore not send this > message to that mailing list. > > Was this a change made by the mailing list admins? > > If so, is it permanent? > > [Trying to send a plaintext e-mail via Gmail, but not sure if it's > working.] When composing plaintext, Gmail really should switch to a fixed font. I'll try writing the post in slrn (which calls emacs in mail-mode) and then inserting that file into gmail. What a colossal PITA. Another gmane user posted a reply to my message, and I see it got rejected also. Were posts from gmane causing problems? I've been trying to figure out how to set up mutt with oauth2 for gmail, but have run into a wall there too: Google doesn't want to let me create an "application" unless I have my own domain pre-registered with Google. Perhaps after 20+ years participating here, it's time to call it quits. -- Grant From rosuav at gmail.com Sun Sep 26 10:07:01 2021 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 27 Sep 2021 00:07:01 +1000 Subject: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On Sun, Sep 26, 2021 at 11:37 PM Grant Edwards wrote: > > I've been trying to figure out how to set up mutt with oauth2 for > gmail, but have run into a wall there too: Google doesn't want to let > me create an "application" unless I have my own domain pre-registered > with Google. > > Perhaps after 20+ years participating here, it's time to call it > quits. > Not sure what the significance of the "application" is - Google has different services for where you're using it with your own domain, but that shouldn't be relevant. If you want to use Gmail with mutt, you should be able to do that, regardless. (Or you can just use some other email address to post from, that would also work.) ChrisA From mohsen.owzar at gmail.com Sat Sep 25 23:12:43 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Sat, 25 Sep 2021 20:12:43 -0700 (PDT) Subject: Flush / update GUIs in PyQt5 during debugging in PyCharm In-Reply-To: References: Message-ID: DFS schrieb am Freitag, 24. September 2021 um 14:52:41 UTC+2: > On 9/24/2021 12:46 AM, Mohsen Owzar wrote: > > Hi Guys > > I've written a GUI using PyQt5 and in there I use StyleSheets (css) for the buttons and labels to change their background- and foreground-colors and their states as well. > > Because my program doesn't function correctly, I try to debug it in my IDE (PyCharm). > > The problem is that during debugging, when I change some attributes of a button or label, let say its background-color, I can not see this modification of the color until the whole method or function is completed. > > I believe that I have seen somewhere during my searches and googling that one can flush or update the GUI after each step/action is done. > > But until now I couldn't manage it and I don't know where I have to invoke flush/update command in PyCharm. > > If anyone has done this before and knows about it, I would very appreciate seeing his solution. > > > > Regards > > Mohsen > screen: > form.repaint() > > individual widgets: > form.widget.repaint() @DFS Hi I tried to use your suggestion in my code, to see changes in the background-color of the widgets (buttons and labels). Below is the link of two screenshots of my GUI: https://imgur.com/a/5LIefN7 They show the start phase and after clicking the ?All Active? button. As I wrote before, I use Stylesheets for configuring widgets. Here is the code snipped of the relevant part: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: if state == 'All Active': . . . . . . for i in range(7): for j in range(6): self.buttons_active[j].setText('Inactive') self.buttons_active[j].setEnabled(True) self.buttons_active[j].setChecked(True) self.buttons_active[j].setStyleSheet(css_pushButton()) ###################################################### self.buttons_reset[j].setChecked(True) self.buttons_reset[j].setEnabled(False) self.buttons_reset[j].setStyleSheet(css_pushButton()) ##################################################### if i > 0 and j < 6: ################################################ self.labels[i, j].setProperty('orange', True) # self.labels[i, j].repaint() # aaaa = 4 else: ################################################ self.labels[i, j].setProperty('yellow', True) # self.labels[i, j].repaint() # aaaa = 4 self.labels[i, j].setStyleSheet(css_Label()) self.labels[i, j].repaint() aaa = 4 ##################################################### else: . . . . . . for i in range(7): for j in range(6): self.buttons_active[j].setText('Active') self.buttons_active[j].setEnabled(True) self.buttons_active[j].setStyleSheet(css_pushButton()) ###################################################### ###################################################### self.buttons_reset[j].setChecked(False) self.buttons_reset[j].setEnabled(True) self.buttons_reset[j].setStyleSheet(css_pushButton()) ###################################################### if i > 0: ################################################## self.labels[i, j].setProperty('gray', True) # self.labels[i, j].repaint() # aaaa = 4 else: self.labels[i, j].setProperty('yellow', True) # self.labels[i, j].repaint() # aaaa = 4 self.labels[i, j].setStyleSheet(css_Label()) self.labels[i, j].repaint() aaa = 4 :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: As you can see, the if-else part has to change the background-color of the gray labels (at the start) and the buttons ?Active?, ?Reset?, ?All Active? and ?Exit? to orange. The problem is, after clicking on the ?All Active? button all ?Resets? and the gray labels change their background-color to orange as desired. But when I afterwards click again on this button (Now its name is ?All Inactive?), the ?Resets? and ?Exit? buttons change their color and not the 36 labels. They remain in orange. Therefore I wanted to debug the code in PyCharm and used your suggestion, at first with ?form.widget.repaint()? and then with ?form.repaint()?. It didn?t help. I put after the repaint() command line a dummy line to set a break point there. I have put the repair() attribute everywhere in the loop to see if it makes any difference. Nothing was changed. As long as I was in the ?for-loop? and the indices ?I? and ?j? did not have their end value, I couldn?t see any changes at the break point ?aaa = 4?. Do you know why it didn?t update the GUI? Perhaps it is not possible to see any changes or updates in the debug mode!! The CSS part is as below: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: def css_Label(): css = ''' QLabel {background-color: #E0E0E0; color: black; border-style: outset; border-width: 2px; border-radius: 5px; border-color: blue; font: bold 16px; min-width: 5em; padding: 4px; } QLabel[gray='true'] {background-color: #E0E0E0; color: black; border-style: outset; border-width: 2px; border-radius: 5px; border-color: black; font: bold 16px; min-width: 5em; padding: 4px; } QLabel[yellow='true'] {color: black; background-color: yellow; border-color: black; font: bold 16px;} QLabel[orange='true'] {color: black; background-color: orange; border-color: black; font: bold 16px;} ''' return css def css_pushButton(): css = ''' QPushButton { background-color: cyan; border-style: outset; border-width: 2px; border-radius: 5px; border-color: black; font: bold 16px; min-width: 5em; padding: 4px; } QPushButton:disabled {color: blue; background-color: orange; font: bold 16px;} QPushButton:hover {color: black; background-color: lightgreen;} QPushButton:hover:enabled {color: blue; background-color: magenta;} QPushButton:enabled {background-color: cyan;} QPushButton:checked {background-color: '#00FF00'; color: red; font: bold 16px;} ''' return css :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Regards Mohsen From grant.b.edwards at gmail.com Sun Sep 26 11:08:38 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 10:08:38 -0500 Subject: Posts from gmane no longer allowed? Message-ID: On 2021-09-26, Chris Angelico wrote: > Not sure what the significance of the "application" is - Google has > different services for where you're using it with your own domain, but > that shouldn't be relevant. If you want to use Gmail with mutt, you > should be able to do that, regardless. (Or you can just use some other > email address to post from, that would also work.) I have other addresses from which I could post, but I don't want them visible on the list. There are three ways to use mutt (or any other IMAP/SMTP) client with GMail. 1. Plain old username/password authentication: This requires you to enable the "less secure apps" option on your Gmail account. I used to use that, and it worked. But, it's frowned upon, and I wouldn't be surprised if it went away soon. It requires you to either put your password in a plaintext config file or enter it every time your connect to Gmail. 2. Application-specific password: Creates a unique 16-digit application password that allows access to selected capabilities. This requires that you have two-factor authentication enabled on your Google account. I probably should do that, but I haven't figured out a convenient way to do so. 3. OAUTH2: This requires that you "register an application" with Google. That application is then issued revokable credentials. The application uses those credentials to send an access request to Google. The account's owner then goes to a specified URL to autorize that access. The application is then issued revokable access and refresh tokens. The access token allows access to specific APIs for a short period of time (maybe an hour). After the access token expires, the refresh token can be used to obtain a new access token. The "register an application" step is where I got stuck. Other mutt users seem to have been able to go to their GMail account's cloud services page, create a project, create/register an application, and then download OAUTH2 credentials which they then use (via an external utility) with programs like mutt and msmtp. When I tried to "register an application" it demanded support and privacy policy URLs for my application. Those URLs had to be using domains that had bee pre-registered with Google. The only domain where I have a web page is at panix.com, and that domain isn't pre- registered with Google. I don't own that domain, so I'm not going to try to pre-register it with Google. I could continue to read the list with slrn, but post using something like Thunderbird, but do I really want to set up a whole new MUA just for one mailing list? [The other 20+ mailing lists I follow are all happy with posts from gmane.] -- Grant From rosuav at gmail.com Sun Sep 26 11:22:56 2021 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 27 Sep 2021 01:22:56 +1000 Subject: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On Mon, Sep 27, 2021 at 1:10 AM Grant Edwards wrote: > > On 2021-09-26, Chris Angelico wrote: > > > Not sure what the significance of the "application" is - Google has > > different services for where you're using it with your own domain, but > > that shouldn't be relevant. If you want to use Gmail with mutt, you > > should be able to do that, regardless. (Or you can just use some other > > email address to post from, that would also work.) > > I have other addresses from which I could post, but I don't want them > visible on the list. > > There are three ways to use mutt (or any other IMAP/SMTP) client with > GMail. > > 1. Plain old username/password authentication: This requires you to > enable the "less secure apps" option on your Gmail account. I used > to use that, and it worked. But, it's frowned upon, and I wouldn't > be surprised if it went away soon. It requires you to either put > your password in a plaintext config file or enter it every time > your connect to Gmail. Agreed, I wouldn't recommend that. > 2. Application-specific password: Creates a unique 16-digit > application password that allows access to selected > capabilities. This requires that you have two-factor > authentication enabled on your Google account. I probably should > do that, but I haven't figured out a convenient way to do so. That would normally be the most convenient for personal usage like this. It should be possible to use a simple TOTP tool, maybe even one written in Python, like this: https://github.com/Rosuav/shed/blob/master/2fa > 3. OAUTH2: This requires that you "register an application" with > Google. That application is then issued revokable credentials. The > application uses those credentials to send an access request to > Google. The account's owner then goes to a specified URL to > autorize that access. The application is then issued revokable > access and refresh tokens. The access token allows access to > specific APIs for a short period of time (maybe an hour). After > the access token expires, the refresh token can be used to obtain > a new access token. This is primarily aimed at third-party tools, where the user owning the account isn't the same person as the creator of the tool. That's why there's all the extra layers. It can certainly be used in a more personal setup, but you'll probably need to do some otherwise-unnecessary work. Google in particular is a bit of a hassle for OAuth (compared to other OAuth providers), due to the wide variety of services that they provide, some of which cost money. So there are extra steps to choose which services to activate. > The "register an application" step is where I got stuck. Other mutt > users seem to have been able to go to their GMail account's cloud > services page, create a project, create/register an application, and > then download OAUTH2 credentials which they then use (via an external > utility) with programs like mutt and msmtp. > > When I tried to "register an application" it demanded support and > privacy policy URLs for my application. Those URLs had to be using > domains that had bee pre-registered with Google. The only domain where > I have a web page is at panix.com, and that domain isn't pre- > registered with Google. I don't own that domain, so I'm not going to > try to pre-register it with Google. The most important address is the redirect URL. If you set that to something on localhost, Google will know that you're building an internal-only tool, and won't require anything much for the support and privacy policy URLs (you could probably just set them to http://localhost/support and http://localhost/privacy, or maybe even leave them blank). (But I'd still recommend an app password. Much easier.) > I could continue to read the list with slrn, but post using something > like Thunderbird, but do I really want to set up a whole new MUA just > for one mailing list? [The other 20+ mailing lists I follow are all > happy with posts from gmane.] > I'm not sure whether the policy change happened on python-list, or at gmane. From the look of the error message you got, it may have actually been gmane's decision. Haven't heard anything from the list admins here about it, either way, so I have no idea. ChrisA From grant.b.edwards at gmail.com Sun Sep 26 12:21:08 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 11:21:08 -0500 Subject: Subject: Re: Posts from gmane no longer allowed? Message-ID: On 2021-09-26, Chris Angelico wrote: Thanks for the tips on registering an application for oauth2 credentials. It sounds like I should be able to do that if I practice my hoop-jumping a bit more. > (But I'd still recommend an app password. Much easier.) Yes, I really should go with the 2FA and app-password option. Do you need the 2nd factor every time you connect to GMail via a browser or Android Gmail app? Or just the first time for each browser/device? A bit of studying seems to be in order no matter what. :) I had initially thought that oauth2 was going to be the easier row to hoe, but it had a lot more roots and rocks than I thought. >> I could continue to read the list with slrn, but post using something >> like Thunderbird, but do I really want to set up a whole new MUA just >> for one mailing list? [The other 20+ mailing lists I follow are all >> happy with posts from gmane.] > > I'm not sure whether the policy change happened on python-list, or at > gmane. From the look of the error message you got, it may have > actually been gmane's decision. Haven't heard anything from the list > admins here about it, either way, so I have no idea. I'm just guessing, but I can't imagine that gmane's owner would have done that unless python-list itself started to refuse posts (or the python-list admins requested that gmane stop sending posts). -- Grant From 2QdxY4RzWzUUiLuE at potatochowder.com Sun Sep 26 12:38:32 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Sun, 26 Sep 2021 09:38:32 -0700 Subject: Subject: Re: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On 2021-09-26 at 11:21:08 -0500, Grant Edwards wrote: > [...] Do you need the 2nd factor every time you connect to GMail via a > browser or Android Gmail app? Or just the first time for each > browser/device? A bit of studying seems to be in order no matter > what. :) No. I use mbsync (formerly isync) to synchronize my gmail account with a local maildir folder, and while mbsync does send the app password (over TLS) to google every few minutes, it doesn't need the second factor. Or at least it hasn't so far (it's been a couple of years). I do get periodic nags from google because I'm using a "less secure" method than their web page to access my mail, but I ignore them. From ethan at stoneleaf.us Sun Sep 26 12:56:16 2021 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 26 Sep 2021 09:56:16 -0700 Subject: Subject: Re: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On 9/26/21 9:21 AM, Grant Edwards wrote: > On 2021-09-26, Chris Angelico wrote: >> I'm not sure whether the policy change happened on python-list, or at >> gmane. From the look of the error message you got, it may have >> actually been gmane's decision. Haven't heard anything from the list >> admins here about it, either way, so I have no idea. > > I'm just guessing, but I can't imagine that gmane's owner would have > done that unless python-list itself started to refuse posts (or the > python-list admins requested that gmane stop sending posts). I am unaware of a change in the newsgroup <--> mailing list policy, and other newsgroup posts were coming through last week (it's been a light weekend). -- ~Ethan~ From mohsen.owzar at gmail.com Sun Sep 26 12:54:16 2021 From: mohsen.owzar at gmail.com (Mohsen Owzar) Date: Sun, 26 Sep 2021 09:54:16 -0700 (PDT) Subject: Free OCR package in Python and selecting appropriate widget for the GUI In-Reply-To: References: <44aad2fa-366d-42fb-8012-363794b94fdbn@googlegroups.com> <00d55cd0-0a3e-483b-87a9-52155877f56en@googlegroups.com> <39c154b6-ccd2-4f42-9e2f-a8b86431cc23n@googlegroups.com> Message-ID: <5b91d630-a9f5-453f-916a-b91d40246327n@googlegroups.com> Mohsen Owzar schrieb am Donnerstag, 23. September 2021 um 08:53:15 UTC+2: > DFS schrieb am Mittwoch, 22. September 2021 um 09:41:42 UTC+2: > > On 9/22/2021 1:54 AM, Mohsen Owzar wrote: > > > DFS schrieb am Mittwoch, 22. September 2021 um 05:10:30 UTC+2: > > >> On 9/21/2021 10:38 PM, Mohsen Owzar wrote: > > >>> DFS schrieb am Dienstag, 21. September 2021 um 15:45:38 UTC+2: > > >>>> On 9/21/2021 4:36 AM, Mohsen Owzar wrote: > > >>>>> Hi Guys > > >>>>> Long time ago I've written a program in Malab a GUI for solving Sudoku puzzles, which worked not so bad. > > >>>>> Now I try to write this GUI with Python with PyQt5 or TKinter. > > >>>>> First question is: > > >>>>> Is there any free OCR software, packages or code in Python, which I can use to recognize the given digits and their positions in the puzzle square. > > >>>>> Second: > > >>>>> Because, I can not attach a picture to this post, I try to describe my picture of my GUI. > > >>>> Draw your GUI in PyQt designer or other graphics tool, then upload a > > >>>> screenshot of it to imgur, then post the link to the picture. > > >>> Thanks, for your answer. Hi > > >>> But, what is "imgur"? > > >>> I'm not so familiar with handling of pictures in this group. > > >>> How can I call "imgur" or how can I get there? > > >>> > > >>> Regards > > >>> Mohsen > > >> www.imgur.com > > >> > > >> It's a website you can upload image files or screenshots to. Then you > > >> can copy a link to your picture and post the link here. > > > I have already posted the link, but I can not see it anywhere. > > > Now, I post it again: > > > https://imgur.com/a/Vh8P2TE > > > I hope that you can see my two images. > > > Regards > > > Mohsen > > Got it. > > > > I haven't used tkinter. In PyQt5 designer I think you should use one > > QTextEdit control for each square. > > > > > > Each square with the small black font can be initially populated with > > > > 1 2 3 > > 4 5 6 > > 7 8 9 > > > > > > > > https://imgur.com/lTcEiML > > > > > > > > some starter python code (maybe save as sudoku.py) > > > > ===================================================================== > > from PyQt5 import Qt, QtCore, QtGui, QtWidgets, uic > > from PyQt5.Qt import * > > from PyQt5.QtCore import * > > from PyQt5.QtGui import * > > from PyQt5.QtWidgets import * > > > > #objects > > app = QtWidgets.QApplication([]) > > frm = uic.loadUi("sudoku.ui") > > > > > > #grid = a collection of squares > > grids = 1 > > > > #squares = number of squares per grid > > squares = 9 > > > > #fill the squares with 1-9 > > def populateSquares(): > > for i in range(grids,grids+1): > > for j in range(1,squares+1): > > widget = frm.findChild(QtWidgets.QTextEdit, "txt{}_{}".format(i,j)) > > widget.setText("1 2 3 4 5 6 7 8 9") > > > > #read data from squares > > def readSquares(): > > for i in range(grids,grids+1): > > for j in range(1,squares+1): > > print("txt%d_%d contains: %s" % > > (i,j,frm.findChild(QtWidgets.QTextEdit, > > "txt{}_{}".format(i,j)).toPlainText())) > > > > > > #connect pushbuttons to code > > frm.btnPopulate.clicked.connect(populateSquares) > > frm.btnReadContents.clicked.connect(readSquares) > > > > #show main form > > frm.show() > > > > #initiate application > > app.exec() > > ===================================================================== > > > > > > > > > > > > .ui file (ie save as sudoku.ui) > > ===================================================================== > > > > > > > > MainWindow > > > > > > > > 0 > > 0 > > 325 > > 288 > > > > > > > > Sudoku > > > > > > > > > > > > 32 > > 22 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 114 > > 22 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 196 > > 22 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 32 > > 86 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 114 > > 86 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 196 > > 86 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 32 > > 150 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 114 > > 150 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 32 > > 228 > > 121 > > 35 > > > > > > > > Populate with numbers > > > > > > > > > > > > 196 > > 150 > > 83 > > 65 > > > > > > > > > > Courier > > 12 > > 50 > > false > > > > > > > > false > > > > > > color: rgb(0, 0, 127); > > background-color: rgb(255, 255, 127); > > > > > > QFrame::StyledPanel > > > > > > QFrame::Sunken > > > > > > Qt::ScrollBarAlwaysOff > > > > > > true > > > > > > > > > > > > 170 > > 228 > > 109 > > 35 > > > > > > > > Read contents > > > > > > > > > > > > > > > > > > > > ===================================================================== > Thank you, > > I'll try to use this QTextEdit, to see if I'm able to manage my needs. > > Regards > Mohsen Hi DFS, I?ve copied your code and ran it. It is almost that what I wanted except the fact, the 9 digits are not behave as a 3x3 matrix. When I click on the button ?Populate with numbers?, on my screen looks like below in each square: 1 2 3 4 5 6 7 8 9 I want to get the numbers in a matrix form as: 1 2 3 4 5 6 7 8 9 And they should be placed centered in each square. And when a digit is missing, let say 4, 5 doesn?t have to take its place , so that: 1 2 3 5 6 7 8 9 And when the real number / digit is found, it should be a big one digit in the center of the square. I don?t know how to bring these two different font sizes and digit placements under one hat. Do you have any idea? Regards Mohsen From grant.b.edwards at gmail.com Sun Sep 26 13:34:16 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 12:34:16 -0500 Subject: Posts from gmane no longer allowed? Message-ID: On 2021-09-26, Ethan Furman wrote: > On 9/26/21 9:21 AM, Grant Edwards wrote: > > On 2021-09-26, Chris Angelico wrote: > > >> I'm not sure whether the policy change happened on python-list, or at > >> gmane. From the look of the error message you got, it may have > >> actually been gmane's decision. Haven't heard anything from the list > >> admins here about it, either way, so I have no idea. > > > > I'm just guessing, but I can't imagine that gmane's owner would have > > done that unless python-list itself started to refuse posts (or the > > python-list admins requested that gmane stop sending posts). > > I am unaware of a change in the newsgroup <--> mailing list policy, > and other newsgroup posts were coming through last week (it's been a > light weekend). We're not talking about the usenet<-->list gateway. Gmane.io is a server that subscribes to and archives many thousands of email lists. It provides NNTP access to that archive. It submits postings made by NNTP clients via e-mail. It stopped allowing posting a few days ago for python-list. At least one other gmane user has also reported being no longer able to post to python-list. Posting to other lists via gmane still works fine. My first _guess_ would be that Mailman started refusing emailed posts from the gmane server to python-list, and that triggered the gmane server to stop accepting posts for python-list. -- Grant From aucampia at gmail.com Sun Sep 26 15:46:02 2021 From: aucampia at gmail.com (Iwan Aucamp) Date: Sun, 26 Sep 2021 12:46:02 -0700 (PDT) Subject: io.TextIOWrapper and io.RawIOBase Message-ID: Documentation for `io.TextIOWrapper` [[1]] suggests that the buffer supplied to the constructor should be of `io.BufferedIOBase` type: > A buffered text stream over a `BufferedIOBase` binary stream. Looking at the implementation of `io.TextIOWrapper` in both `Modules/_io/textio.c` [[2]] and `Lib/_pyio.py` [[3]], it seems like `io.TextIOWrapper` will only use the following attributes and functions on the supplied `buffer` object: ```python buffer.closed buffer.close() buffer._dealloc_warn() # ^ only from Modules/_io/textio.c, looks like failures will be ignored buffer.fileno() buffer.flush() buffer.isatty() buffer.name # ^ only when TextIOWrapper.buffer is accessed buffer.raw() # ^ only if buffer is instance of BufferedReader, BufferedWriter # or BufferedRandom, also only from Modules/_io/textio.c buffer.read() buffer.read1() # only if buffer has a read1 method buffer.readable() buffer.seek() buffer.seekable() buffer.tell() buffer.truncate() buffer.writable() buffer.write() ``` More specifically, `io.TextIOWrapper` looks like it will work fine with `buffer` objects that does not have any of the following attributes and methods that only exists in `io.BufferedIOBase`, but not in `io.RawIOBase`: ```python buffer.raw # ^ this is only called if buffer is an instance of # explicit subclasses of `io.BufferedIOBase` buffer.detatch() # this is never called buffer.read1() # this is only used if it exists buffer.readinto() # this is never called buffer.readinto1() # this is never called ``` Or stated differently, it looks like `io.TextIOWrapper` will work equally well with buffer objects that are either `io.RawIOBase` or `io.BufferedIOBase` types. So the question is, if my assessment is correct, should the documentation not be updated to clearly state that the `buffer` can be either a `io.RawIOBase` or `io.BufferedIOBase` object? (text written for python 3.7.12) [1]: https://docs.python.org/3.7/library/io.html#io.TextIOWrapper [2]: https://github.com/python/cpython/blob/v3.7.12/Modules/_io/textio.c [3]: https://github.com/python/cpython/blob/v3.7.12/Lib/_pyio.py From PythonList at DancesWithMice.info Sun Sep 26 16:17:43 2021 From: PythonList at DancesWithMice.info (dn) Date: Mon, 27 Sep 2021 09:17:43 +1300 Subject: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On 27/09/2021 06.34, Grant Edwards wrote: > On 2021-09-26, Ethan Furman wrote: >> On 9/26/21 9:21 AM, Grant Edwards wrote: >>> On 2021-09-26, Chris Angelico wrote: >> >>>> I'm not sure whether the policy change happened on python-list, or at >>>> gmane. From the look of the error message you got, it may have >>>> actually been gmane's decision. Haven't heard anything from the list >>>> admins here about it, either way, so I have no idea. ... > My first _guess_ would be that Mailman started refusing emailed posts > from the gmane server to python-list, and that triggered the gmane server > to stop accepting posts for python-list. > 1 Google are not the most reliable when it comes to maintaining policy/services, nor for advising their decision to make changes. CI/CD breakage doesn't 'count' when you're big-enough not to care... 2 Every message from the OP in this 'thread' (not others) has broken the thread, which indicates a wider problem/change. I'm ignorant of such things. Why not subscribe directly to this list with your dedicated/public-facing gmail address? (if the concern relates to having too many email addresses 'open to the world', is Google the best 'gatekeeper' and privacy guard?) -- Regards, =dn From grant.b.edwards at gmail.com Sun Sep 26 16:55:59 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 15:55:59 -0500 Subject: Posts from gmane no longer allowed? References: Message-ID: On 2021-09-26, dn via Python-list wrote: > On 27/09/2021 06.34, Grant Edwards wrote: >> On 2021-09-26, Ethan Furman wrote: >>> On 9/26/21 9:21 AM, Grant Edwards wrote: >>>> On 2021-09-26, Chris Angelico wrote: >>> >>>>> I'm not sure whether the policy change happened on python-list, >>>>> or at gmane. From the look of the error message you got, it may >>>>> have actually been gmane's decision. Haven't heard anything from >>>>> the list admins here about it, either way, so I have no idea. > ... > >> My first _guess_ would be that Mailman started refusing emailed >> posts from the gmane server to python-list, and that triggered the >> gmane server to stop accepting posts for python-list. >> > > > 1 Google are not the most reliable when it comes to maintaining > policy/services, nor for advising their decision to make changes. CI/CD > breakage doesn't 'count' when you're big-enough not to care... I'm not really sure what Google has to do with it other than me wanting to post using my GMail address, since that's the one that everybody knows. I could be using any other e-mail address, and it wouldn't make any difference. > 2 Every message from the OP in this 'thread' (not others) has broken the > thread, which indicates a wider problem/change. And I apologize for that. It's because I'm reading the list using an NNTP client (slrn) connected to an NNTP server at gmane.io and then posting via e-mail. Doing that doesn't include the correct References header. Posting via gmane stopped working a few days ago (after having worked for 20 years). I'm working on a solution so that slrn can read using NNTP and post using email (including the proper headers), but it's going to take a few days. In the meanwhile, I'll try to manually insert the proper References: header when I post. > I'm ignorant of such things. Why not subscribe directly to this list > with your dedicated/public-facing gmail address? I find that following mailing lists using a news reader is far, far more efficient than using an e-mail program. Efficiently sifting through thousands and thousands of posts in dozens of groups/lists is want newsreaders are designed to do, and they're very good at it. One of the big advantages of slrn is that I can create a "score file" so I always see things I want to see, and not see things I don't want to see. I can also search through decades of articles almost instantly without having to have any of them stored locally. -- Grant From ethan at stoneleaf.us Sun Sep 26 17:06:32 2021 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 26 Sep 2021 14:06:32 -0700 Subject: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On 9/26/21 10:34 AM, Grant Edwards wrote: > On 2021-09-26, Ethan Furman wrote: >> I am unaware of a change in the newsgroup <--> mailing list policy, >> and other newsgroup posts were coming through last week (it's been a >> light weekend). > > We're not talking about the usenet<-->list gateway. > My first _guess_ would be that Mailman started refusing emailed posts > from the gmane server to python-list, and that triggered the gmane server > to stop accepting posts for python-list. I readily admit I may not understand, or know, all the usenet jargon, but looking at the Mailman server for Python list I see it is still fully configured to talk with News.FU-Berlin.DE, newsgroup comp.lang.python. I'll forward the problem to the maintainers -- hopefully they'll have some insight. -- ~Ethan~ From mats at wichmann.us Sun Sep 26 17:09:17 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sun, 26 Sep 2021 15:09:17 -0600 Subject: Subject: Re: Posts from gmane no longer allowed? In-Reply-To: References: Message-ID: On 9/26/21 10:38, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-26 at 11:21:08 -0500, > No. I use mbsync (formerly isync) to synchronize my gmail account with > a local maildir folder, and while mbsync does send the app password > (over TLS) to google every few minutes, it doesn't need the second > factor. Or at least it hasn't so far (it's been a couple of years). I > do get periodic nags from google because I'm using a "less secure" > method than their web page to access my mail, but I ignore them. Just be aware, because It's Google, this will change again at some point and you'll lose access. I had a working setup that started triggering the less-secure warnings, but was able to ignore it after setting the account to allow such "insecure" access (several times, because it kept "forgetting" that setting, probably quite intentionally). And then after a while, there was no way around it, and now if my infrequently used setup gets triggered to sync with gmail I get "critical security alert" messages and nothing goes through. And it's unchangeable. I'm going to have to go clean it out, just been too lazy to do so. From grant.b.edwards at gmail.com Sun Sep 26 20:28:33 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 17:28:33 -0700 (PDT) Subject: Posts from gmane no longer allowed? References: Message-ID: <61511031.1c69fb81.127c6.0222@mx.google.com> On 2021-09-26, Grant Edwards wrote: > >> 2 Every message from the OP in this 'thread' (not others) has broken the >> thread, which indicates a wider problem/change. > > And I apologize for that. It's because I'm reading the list using an > NNTP client (slrn) connected to an NNTP server at gmane.io and then > posting via e-mail. Doing that doesn't include the correct References > header. Posting via gmane stopped working a few days ago (after > having worked for 20 years). I'm working on a solution so that slrn > can read using NNTP and post using email (including the proper > headers), but it's going to take a few days. It was easier than I thought. It only took about a half hour to write my own 'inews' utility that I can tell slrn to use for posting articles. It looks at the article and decides based on the destination whether to use NNTP or SMTP to send it (and then sends it using the appropriate protocol). It's a total of 50 lines of Python (including logging stuff to syslog). This is my first "real" post using it, so hopefully this shows up with proper headers. -- Grant From grant.b.edwards at gmail.com Sun Sep 26 20:38:12 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 17:38:12 -0700 (PDT) Subject: Posts from gmane no longer allowed? References: Message-ID: <61511274.1c69fb81.7ebab.f590@mx.google.com> On 2021-09-26, Ethan Furman wrote: > On 9/26/21 10:34 AM, Grant Edwards wrote: > > On 2021-09-26, Ethan Furman wrote: > >>> I am unaware of a change in the newsgroup <--> mailing list policy, >>> and other newsgroup posts were coming through last week (it's been a >>> light weekend). >> >> We're not talking about the usenet<-->list gateway. > >> My first _guess_ would be that Mailman started refusing emailed posts >> from the gmane server to python-list, and that triggered the gmane server >> to stop accepting posts for python-list. > > I readily admit I may not understand, or know, all the usenet > jargon, Once again, there is no Usenet involved. Gmane.io is an NNTP server, and NNTP is _also_ used by Usenet servers, but Gmane isn't part of Usenet and doesn't implement any of the Usenet peering "news" protocols. > but looking at the Mailman server for Python list I see it is still > fully configured to talk with News.FU-Berlin.DE, newsgroup > comp.lang.python. This has nothing to do with Usenet servers or their connections to python-list. >From the list's POV, gmane.io is a "normal" email subscriber who just happens to archive all the articles it receives. I should never have mentioned that gmane.io does NNTP -- it just seems to have confused everybody. When a gmane.io user submits a post, the post arrives at python-list by normal email channels having been sent via SMTP by the gmane.io server "From:" the user (who, IIRC, has to be subscribed to the list). -- Grant From grant.b.edwards at gmail.com Sun Sep 26 20:40:18 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 17:40:18 -0700 (PDT) Subject: Subject: Re: Posts from gmane no longer allowed? References: Message-ID: <615112f2.1c69fb81.96ca6.3498@mx.google.com> On 2021-09-26, Mats Wichmann wrote: > On 9/26/21 10:38, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: >> On 2021-09-26 at 11:21:08 -0500, > >> No. I use mbsync (formerly isync) to synchronize my gmail account with >> a local maildir folder, and while mbsync does send the app password >> (over TLS) to google every few minutes, it doesn't need the second >> factor. Or at least it hasn't so far (it's been a couple of years). I >> do get periodic nags from google because I'm using a "less secure" >> method than their web page to access my mail, but I ignore them. Are you using an app-specific password? > Just be aware, because It's Google, this will change again at some point > and you'll lose access. Of course. That's half the fun of using Google's services. ;) From grant.b.edwards at gmail.com Sun Sep 26 20:47:52 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 17:47:52 -0700 (PDT) Subject: Posts from gmane no longer allowed? References: <61511274.1c69fb81.7ebab.f590@mx.google.com> Message-ID: <615114b8.1c69fb81.9e5f5.aae7@mx.google.com> On 2021-09-27, Grant Edwards wrote: > >From the list's POV, gmane.io is a "normal" email subscriber who just > happens to archive all the articles it receives. I should never have > mentioned that gmane.io does NNTP -- it just seems to have confused > everybody. Did SMTP.send_message() add that '>' escape when it saw a 'From' at the beginning of the line? -- Grant From 2QdxY4RzWzUUiLuE at potatochowder.com Sun Sep 26 22:31:30 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Sun, 26 Sep 2021 19:31:30 -0700 Subject: Subject: Re: Posts from gmane no longer allowed? In-Reply-To: <615112f2.1c69fb81.96ca6.3498@mx.google.com> References: <615112f2.1c69fb81.96ca6.3498@mx.google.com> Message-ID: On 2021-09-26 at 17:40:18 -0700, Grant Edwards wrote: > On 2021-09-26, Mats Wichmann wrote: > > On 9/26/21 10:38, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > >> On 2021-09-26 at 11:21:08 -0500, > > > >> No. I use mbsync (formerly isync) to synchronize my gmail account with > >> a local maildir folder, and while mbsync does send the app password > >> (over TLS) to google every few minutes, it doesn't need the second > >> factor. Or at least it hasn't so far (it's been a couple of years). I > >> do get periodic nags from google because I'm using a "less secure" > >> method than their web page to access my mail, but I ignore them. > > Are you using an app-specific password? Yes. It was a hoop, but not a big one. No, wait, I mean it wasn't a small, flaming, poison spiked hoop; i.e., it was fairly simple to jump through. > > Just be aware, because It's Google, this will change again at some > > point and you'll lose access. > > Of course. That's half the fun of using Google's services. ;) If it changes, I'll just forward incoming mail to my main (non-google) account and not look back. If I didn't need the account to access the google store on my android phone, I'd would have abandoned it long ago. From grant.b.edwards at gmail.com Sun Sep 26 22:57:39 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 19:57:39 -0700 (PDT) Subject: Why does SMTP.send_message() do from mangling? Message-ID: <61513323.1c69fb81.78da5.a7ca@mx.google.com> Why does SMTP.send_message(msg) do from mangling even though msg's policy has mangle_from_ set to False? The msg policy is email.policy.SMTP which has mangle_from_ disabled. One might expect that SMTP.send_message(msg) would use either msg's policy or email.policy.SMTP to send the message, but it does neither. Don't you think that email.policy.SMTP would be the policy that ought to be used to serialize a message for sending via SMTP? None of the other SMTP clients I've checked do from mangling. AFAICT, in order to send message without from mangling, I have to do something like this: s.sendmail(msg['From'], msg['To'], msg.as_bytes()) Instead of s.send_message(msg) For simple cases, it doesn't matter much, but if you have multiple recipients, bcc, and cc, it's a pain that send_message() can't be used. What is the purpose of from mangling in an SMTP context? I understand why it's done when writing messages to mbox files, but what does that have to do with SMTP.send_message()? -- Grant From grant.b.edwards at gmail.com Mon Sep 27 00:07:00 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 21:07:00 -0700 (PDT) Subject: Bug in email.generator.BytesGenerator() [was: Why does SMTP.send_message() do from mangling?] References: <61513323.1c69fb81.78da5.a7ca@mx.google.com> Message-ID: <61514364.1c69fb81.e6837.218b@mx.google.com> On 2021-09-27, Grant Edwards wrote: > Why does SMTP.send_message(msg) do from mangling even though msg's > policy has mangle_from_ set to False? The msg policy is > email.policy.SMTP which has mangle_from_ disabled. > > One might expect that SMTP.send_message(msg) would use either msg's > policy or email.policy.SMTP to send the message, but it does neither. I've been looking at the smtplib.py sources, and the problem appears to be in this section of send_message(): 912 def send_message(self, msg, from_addr=None, to_addrs=None, 913 mail_options=(), rcpt_options=()): 914 """Converts message to a bytestring and passes it to sendmail. ... 963 # Make a local copy so we can delete the bcc headers. 964 msg_copy = copy.copy(msg) ... 977 with io.BytesIO() as bytesmsg: 978 if international: 979 g = email.generator.BytesGenerator( 980 bytesmsg, policy=msg.policy.clone(utf8=True)) 981 mail_options = (*mail_options, 'SMTPUTF8', 'BODY=8BITMIME') 982 else: 983 g = email.generator.BytesGenerator(bytesmsg) 984 g.flatten(msg_copy, linesep='\r\n') 985 flatmsg = bytesmsg.getvalue() If 'international' is Frue, then the BytesGenerator uses msg.policy with utf8 added, and I don't get the bogus from mangling: the generator only does from mangling if the message policy has it enabled. If 'international' is False, then the generator always does from mangling regardless of the message's policy. According to https://docs.python.org/3/library/email.generator.html#email.generator.BytesGenerator the default from mangling behavior is _supposed_ to obey the message policy if (as seen at 983) no policy or mangle_from_ value was provided to the call to BytesGenerator(). In my tests, it doesn't actually seem to work that way. AFAICT, the default behavior when no policy or mangle_from_ value is passed to BytesGenerator() is to enable from mangling regardless of the message's policy. I belive that is a bug. This can be worked around by changing 983 g = email.generator.BytesGenerator(bytesmsg) to 983 g = email.generator.BytesGenerator(bytesmsg, policy=msg.policy) Or BytesGenerator() could be fixed... -- Grant From grant.b.edwards at gmail.com Mon Sep 27 00:36:28 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 21:36:28 -0700 (PDT) Subject: Bug in email.generator.BytesGenerator() [was: Why does SMTP.send_message() do from mangling?] References: <61513323.1c69fb81.78da5.a7ca@mx.google.com> <61514364.1c69fb81.e6837.218b@mx.google.com> Message-ID: <61514a4c.1c69fb81.a0c86.4177@mx.google.com> On 2021-09-27, Grant Edwards wrote: > According to > https://docs.python.org/3/library/email.generator.html#email.generator.BytesGenerator > the default from mangling behavior is _supposed_ to obey the message > policy if no policy or mangle_from_ value was > provided to the call to BytesGenerator(). Nope, I think both the author of smtplib.py and I were misled by the documentation for email.generator.BytesGenerator. After rereading it, it does say in one place that if no mangle_from_, value is passed to BytesGenerator(), it is supposed to default to the mangle_from_ setting in the policy **passed to BytesGenerator()**: If optional mangle_from_ is True, put a > character in front of any line in the body that starts with the exact string "From ", that is From followed by a space at the beginning of a line. mangle_from_ defaults to the value of the mangle_from_ setting of the policy. Where "the policy" refers to the one passed to BytesGenerator(). However, later on it also says If policy is None (the default), use the policy associated with the Message or EmailMessage object passed to flatten to control the message generation. That's misleading and only partially true. If you don't pass a policy to BytesGenerator(), only _some_ of the settings from the message's policy will be used. Some policy settings (e.g. mangle_from_) are obeyed when passed to BytesGenerator(), but ignored in the message's policy even if there was no policy passed to BytesGenerator(). I think that last sentence above needs to be changed, and smtplib.py needs to be fixed as shown in my previous post. -- Grant From grant.b.edwards at gmail.com Mon Sep 27 01:09:30 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 26 Sep 2021 22:09:30 -0700 (PDT) Subject: Why does SMTP.send_message() do from mangling? References: <61513323.1c69fb81.78da5.a7ca@mx.google.com> Message-ID: <6151520a.1c69fb81.b9e3.663f@mx.google.com> On 2021-09-27, Grant Edwards wrote: > Why does SMTP.send_message(msg) do from mangling even though msg's > policy has mangle_from_ set to False? I've concluded this is a bug in SMTP.send_message() https://bugs.python.org/issue45299 -- Grant From nad at python.org Mon Sep 27 01:39:18 2021 From: nad at python.org (Ned Deily) Date: Mon, 27 Sep 2021 01:39:18 -0400 Subject: Posts from gmane no longer allowed? In-Reply-To: <61511274.1c69fb81.7ebab.f590@mx.google.com> References: <61511274.1c69fb81.7ebab.f590@mx.google.com> Message-ID: <1bb7a7f9-6e13-5eec-e1c0-fd63a139812b@python.org> On 9/26/21 20:38, Grant Edwards wrote: > On 2021-09-26, Ethan Furman wrote: >> On 9/26/21 10:34 AM, Grant Edwards wrote: >>> On 2021-09-26, Ethan Furman wrote: >> >>>> I am unaware of a change in the newsgroup <--> mailing list policy, >>>> and other newsgroup posts were coming through last week (it's been a >>>> light weekend). >>> >>> We're not talking about the usenet<-->list gateway. >> >>> My first _guess_ would be that Mailman started refusing emailed posts >>> from the gmane server to python-list, and that triggered the gmane server >>> to stop accepting posts for python-list. >> >> I readily admit I may not understand, or know, all the usenet >> jargon, > > Once again, there is no Usenet involved. Gmane.io is an NNTP server, > and NNTP is _also_ used by Usenet servers, but Gmane isn't part of > Usenet and doesn't implement any of the Usenet peering "news" > protocols. > >> but looking at the Mailman server for Python list I see it is still >> fully configured to talk with News.FU-Berlin.DE, newsgroup >> comp.lang.python. > > This has nothing to do with Usenet servers or their connections to > python-list. > >>From the list's POV, gmane.io is a "normal" email subscriber who just > happens to archive all the articles it receives. I should never have > mentioned that gmane.io does NNTP -- it just seems to have confused > everybody. > > When a gmane.io user submits a post, the post arrives at python-list > by normal email channels having been sent via SMTP by the gmane.io > server "From:" the user (who, IIRC, has to be subscribed to the list). I have been in touch with the administrator of gmane. It appears that posting from python-list to gmane has been deliberately disabled, at least temporarily, with cause. I'll see if we can resolve the problem. From damienjenman at hotmail.com Mon Sep 27 02:36:51 2021 From: damienjenman at hotmail.com (damien jenman) Date: Mon, 27 Sep 2021 16:36:51 +1000 Subject: EuroPython Society: General Assembly 2021 In-Reply-To: <2c733060-d652-22e3-7492-191ac3acf7c6@europython.eu> Message-ID: Please stop e-mailing me On 26 Sep 2021 1:42 am, Marc-Andre Lemburg wrote: As last year, we are holding the General Assembly (GA) of the EuroPython Society (EPS) online for this year. General Assembly ---------------- In accordance with our bylaws, we are calling for the EuroPython Society General Assembly to be held on Sunday, October 10th 2020, from 19:00 - 21:00 CEST. We will use a Zoom meeting to hold the event and send around the URL closer to the event. All EPS members are welcome to join and vote at the meeting. Please be aware that we cannot allow non-EPS members to join, as we often do at the in-person GAs we hold at the conference, since we would then not be able to control access to the Zoom call. Board Nominations ----------------- As every year, we will vote in a new board. We have already sent out the list of board nominations in a separate blog post on 2021-09-23. Please see that post for details on the candidates and the nomination process. Motions by the Members ---------------------- EuroPython Society Members can propose motions to be put forward and voted on at the General Assembly. If you want to put forward a motion, please send this to board at europython.eu no later than Sunday, 2021-10-03, so that we can add them to the agenda. The bylaws require that any such motions be announced no later than 5 days before the GA and we will need time to clarify details and prepare the agenda. Agenda ------ We will publish the agenda with all motions put forward by the board and the members on Tuesday, 2020-10-05. The agenda will follow the template set out in our bylaws under section 8. https://www.europython-society.org/bylaws Reports ------- All reports for the GA will be published on Friday, 2020-10-08, to give the members enough time to read them and prepare questions. We'll then answer any questions at the GA. Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://www.europython-society.org/europython-society-general-assembly-2021/ Tweet: https://twitter.com/europythons/status/1441788705514065921 Thanks, -- EuroPython Society https://www.europython-society.org/ -- https://mail.python.org/mailman/listinfo/python-list From arj.python at gmail.com Mon Sep 27 03:15:05 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 27 Sep 2021 11:15:05 +0400 Subject: EuroPython Society: General Assembly 2021 In-Reply-To: References: <2c733060-d652-22e3-7492-191ac3acf7c6@europython.eu> Message-ID: Greetings Damien, The mailing list is public and i would have a hard time dealing with 1000s of people replying: `stop mailing me`, maybe a private message to mal would have sufficed. Kind Regards, Abdur-Rahmaan Janhangeer about | blog github Mauritius From mal at egenix.com Mon Sep 27 04:38:16 2021 From: mal at egenix.com (Marc-Andre Lemburg) Date: Mon, 27 Sep 2021 10:38:16 +0200 Subject: c extension finding the module in object initialization In-Reply-To: <05588d73-29c0-825e-82ae-7989d90cf31f@everest.reportlab.co.uk> References: <05588d73-29c0-825e-82ae-7989d90cf31f@everest.reportlab.co.uk> Message-ID: <330e5632-5c57-7818-06ba-e2515e17fb61@egenix.com> Hi Robin, seeing that no one replied to your question, I'd suggest to ask this on the Python C-API ML: https://mail.python.org/mailman3/lists/capi-sig.python.org/ That's where the experts are, including the ones who implemented the mutli-phase logic. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Sep 27 2021) >>> Python Projects, Coaching and Support ... https://www.egenix.com/ >>> Python Product Development ... https://consulting.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/ On 21.09.2021 14:08, Robin Becker wrote: > I have a c extension which is intended to implement a module the looks > structurally like this > > > ############ > a = 1 > b = 2 > > class T: > ??? def __init__(self): > ??????? self.a = a > ??????? self.b = b > ############ > > so when an object of type T is instantiated it can set up defaults based on the > current module values of a and b. > > In the past using old style single phase module creation the init function for > the type could look up the module by using the PyState_FindModule function. In > the new world where we can implement c extensions which might work with multiple > interpreters (multi-phase creation). The docs say PyState_FindModule won't work > for those and indeed it returns NULL so is useless. > > Is there a way I can set the current module onto the type definition during the > module's exec function? I thought it would be easy to add at the point in the > exec where I'm doing this to the module, m, > > TType.tp_base = &PyBaseObject_Type; > if(PyType_Ready(&TType)<0) goto fail; > if(PyModule_AddObject(m,"T", (PyObject *)&TType)<0) goto fail; > > but I don't see the place in the type where I can add these sorts of dynamic > attributes. The basic_size is for the created objects. > > The created type does have a __dict__ which is a mappingproxy. I wondered when > that actually gets instantiated. > > I can see that the type has a __module__ attribute after the module is imported > and I suppose if I can find the 'current' interpreter I might use the __module__ > to locate the module object via PyImport_GetModuleDict. > > Any expertise or advice gratefully received. > -- > Robin Becker > From robin at reportlab.com Mon Sep 27 05:16:08 2021 From: robin at reportlab.com (Robin Becker) Date: Mon, 27 Sep 2021 10:16:08 +0100 Subject: c extension finding the module in object initialization In-Reply-To: <330e5632-5c57-7818-06ba-e2515e17fb61@egenix.com> References: <05588d73-29c0-825e-82ae-7989d90cf31f@everest.reportlab.co.uk> <330e5632-5c57-7818-06ba-e2515e17fb61@egenix.com> Message-ID: <081f3e3b-84dd-e6a1-87c2-8a4aa4b7239d@everest.reportlab.co.uk> Hi Marc, Thanks for the suggestion, On 27/09/2021 09:38, Marc-Andre Lemburg wrote: > Hi Robin, > > seeing that no one replied to your question, I'd suggest to ask this > on the Python C-API ML: > > https://mail.python.org/mailman3/lists/capi-sig.python.org/ > > That's where the experts are, including the ones who implemented > the mutli-phase logic. > > Cheers, > I think I have this working using ob=PyImport_GetModuleDict() followed by PyDict_GetItemString(ob,"modulename"), but I will ask there to see if there's a more direct route. In Python >=3.7 there's PyImport_GetModule, but that seems more complex than is actually required for this simple case and has to wait until 3.6 dies for me :( -- Robin Becker From skip.montanaro at gmail.com Mon Sep 27 08:48:09 2021 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 27 Sep 2021 07:48:09 -0500 Subject: Package conflicts trying to install jpegdupes package Message-ID: I'd like to use the jpegdupes package but can't seem to get past compilation issued related to jpegtrans-cffi. Attempts to install any available versions on my XUbuntu system crap out with this C compiler error message: In file included from jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267: src/epeg_private.h:71:17: warning: ?_jpeg_init_source? declared ?static? but never defined [-Wunused-function] 71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo); | ^~~~~~~~~~~~~~~~~ src/epeg_private.h:72:20: warning: ?_jpeg_fill_input_buffer? declared ?static? but never defined [-Wunused-function] 72 | METHODDEF(boolean) _jpeg_fill_input_buffer(j_decompress_ptr cinfo); | ^~~~~~~~~~~~~~~~~~~~~~~ src/epeg_private.h:73:17: warning: ?_jpeg_skip_input_data? declared ?static? but never defined [-Wunused-function] 73 | METHODDEF(void) _jpeg_skip_input_data(j_decompress_ptr cinfo, long num_bytes); | ^~~~~~~~~~~~~~~~~~~~~ src/epeg_private.h:74:17: warning: ?_jpeg_term_source? declared ?static? but never defined [-Wunused-function] 74 | METHODDEF(void) _jpeg_term_source(j_decompress_ptr cinfo); | ^~~~~~~~~~~~~~~~~ src/epeg_private.h:76:17: warning: ?_jpeg_init_destination? declared ?static? but never defined [-Wunused-function] 76 | METHODDEF(void) _jpeg_init_destination(j_compress_ptr cinfo); | ^~~~~~~~~~~~~~~~~~~~~~ src/epeg_private.h:77:20: warning: ?_jpeg_empty_output_buffer? declared ?static? but never defined [-Wunused-function] 77 | METHODDEF(boolean) _jpeg_empty_output_buffer (j_compress_ptr cinfo); | ^~~~~~~~~~~~~~~~~~~~~~~~~ src/epeg_private.h:78:17: warning: ?_jpeg_term_destination? declared ?static? but never defined [-Wunused-function] 78 | METHODDEF(void) _jpeg_term_destination (j_compress_ptr cinfo); | ^~~~~~~~~~~~~~~~~~~~~~ ... This seems like a rather odd error message for a presumably sort-of-stable PyPI package. Looking at the version dependencies I see 2.6, 2.7, 3.3 or PyPy. I thought nmybe I needed to try with Python2... Create a Conda environment, attempt to install jpegdupes, only to discover it's Python 3 only. So, I'm kind of stuck. Maybe I need to install Python 3.3 and try that? Any other ideas? Thx, Skip From grant.b.edwards at gmail.com Mon Sep 27 09:39:54 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 27 Sep 2021 06:39:54 -0700 (PDT) Subject: Posts from gmane no longer allowed? References: <61511274.1c69fb81.7ebab.f590@mx.google.com> <1bb7a7f9-6e13-5eec-e1c0-fd63a139812b@python.org> Message-ID: <6151c9aa.1c69fb81.48cd2.a466@mx.google.com> On 2021-09-27, Ned Deily wrote: > I have been in touch with the administrator of gmane. It appears that > posting from python-list to gmane has been deliberately disabled, at > least temporarily, with cause. I'll see if we can resolve the problem. Thanks. Though I've figured out a solution that allows me to read the list via gmane using slrn and post "directly" via e-mail, I'm sure other gmane users will appreciate that. -- Grant From mal at europython.eu Mon Sep 27 10:23:25 2021 From: mal at europython.eu (Marc-Andre Lemburg) Date: Mon, 27 Sep 2021 16:23:25 +0200 Subject: EuroPython 2021: Edited videos of the first day available Message-ID: <9fd66eed-61fc-588b-a421-6921b08ebb18@europython.eu> We?re happy to release the first 42 cut videos of EuroPython 2021 covering the first day sessions of the conference. You can watch them on our YouTube channel: * EuroPython 2021 Playlist * https://www.youtube.com/playlist?list=PL8uoeex94UhHgMD9GOCbEHWku7pEPx9fW Over the next few weeks, we?ll continue releasing the videos for the second and third day of the conference. In total, we will have more than 115 videos with lots of valuable and interesting content for you, so please stop by and check the playlist for more videos, or subscribe to our YouTube channel. http://europython.tv/ Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://blog.europython.eu/europython-2021-edited-videos-of-the-first-day-available/ Tweet: https://twitter.com/europython/status/1442493825491025924 Enjoy, -- EuroPython 2021 Team https://ep2021.europython.eu/ https://www.europython-society.org/ From willcyr852000 at gmail.com Mon Sep 27 10:00:58 2021 From: willcyr852000 at gmail.com (Will) Date: Mon, 27 Sep 2021 07:00:58 -0700 Subject: Python added to PATH, cannot be directly accessed, cannot install pip Message-ID: Hello team at python.org, I've asked this question on a forum and tried to figure it out myself, but I'm at a dead end. I don't know if you guys answer questions like this but I have no where else to turn to. I am using a Lenovo Laptop using Windows. I'm trying to install get-pip.py, and when I enter "python get-pip.py" into the Command Prompt, it says "Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases." So for a week, I've been trying to add Python to PATH, so I can install get-pip.py to Python. I've added the exact file location for Python to Path in both User Variables and System Variables. I've executed Manage App Execution Aliases, turned off both App Installer python.exe and App Installer python3.exe. Still can't install pip, Python was still not found. I've tried different keywords ("py get-pip.py", python3 get-pip.py", etc.). Still doesn't work. Python is added to PATH and I can still run Python scripts, but I can't find Python directly through the Command Prompt, and I cannot install get-pip.py to Python. For reference, I have Python version 3.9.6 and I installed Python directly from the site (I did not use Anaconda). Can you guys help me with this? Or do I need to delete Python and reinstall it? Sincerely, Will From soyeomul at doraji.xyz Mon Sep 27 00:35:42 2021 From: soyeomul at doraji.xyz (=?utf-8?B?7Zmp67OR7Z2s?=) Date: Mon, 27 Sep 2021 13:35:42 +0900 Subject: Posts from gmane no longer allowed? References: Message-ID: Grant Edwards writes: > I've been reading (and posting to) this list for many years by > pointing an NNTP client > at news://gmane.comp.python.general. Sometime in the past few days posts started > being refused: > > You have tried posting to gmane.comp.python.general, which is a > unidirectional > mailing list. Gmane can therefore not send this message to that > mailing list. > > Was this a change made by the mailing list admins? > > If so, is it permanent? > > [Trying to send a plaintext e-mail via Gmail, but not sure if it's working.] Oh this is unhappy news. Because i also using Gmane with gmane.emacs.help and gmane.linux.debian.user. Only i use Usenet (eternal september) with comp.lang.python. Sincerely, Byung-Hee From Bischoop at vimart.net Mon Sep 27 04:56:56 2021 From: Bischoop at vimart.net (Bischoop) Date: Mon, 27 Sep 2021 08:56:56 -0000 (UTC) Subject: count consecutive elements References: Message-ID: I'd wish to have this challenge posted on my blog but because I suck in IT english, could you guys help me with decribing the task of what the code supposed to do? -- Thanks From michael.stemper at gmail.com Mon Sep 27 11:40:12 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 27 Sep 2021 10:40:12 -0500 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> Message-ID: On 25/09/2021 16.39, Avi Gross wrote: > Michael, > > I don't care what you choose. Whatever works is fine for an internal use. Maybe I should have taken the provoking article with a few more grains of salt. At this point, I'm not seeing any issues that are applicable to my use case. > But is the data scheme you share representative of your actual application? > >>From what I see below, unless the number of "point" variables is not always > exactly four, the application might be handled well by any format that > handles rectangular data, perhaps even CSV. > > You show a I mean anything like a data.frame can contain data columns like > p1,p2,p3,p4 and a categorical one like IHRcurve_name. > > Or do you have a need for more variability such as an undetermined number of > similar units in ways that might require more flexibility or be more > efficient done another way? As far as the number of points per IHR curve, the only requirement is that there must be at least two. It's hard to define a line segment with only one. The mock data that I have so far has curves ranging from two to five points. I didn't notice that the snippet that I posted had two curves with the same number of breakpoints, which was misleading. My former employer's systems had, IIRC, space for seven points per curve in the database structures. Of all the sizing changes made over a long career, I don't recall any customer ever requiring more than that. But, it's cleanest to use python lists (with no inherent sizing limitations) to represent the IHR (and incremental cost) curves. > MOST of the discussion I am seeing here seems peripheral to getting you what > you need for your situation and may require a learning curve to learn to use > properly. Are you planning on worrying about how to ship your data > encrypted, for example? Any file format you use for storage can presumably > be encrypted and send and decrypted if that matters. This work is intended to look at the feasability of relaxing some constraints normally required for the solution of Economic Dispatch. So all of my data are hypothetical. Once I have stuff up and running, I'll be making up data for lots of different generators. Being retired, I don't have access to any proprietary information about any specific generators, so all of the data is made up out of my head. I still need a way to get it into my programs, of course. > So, yes, from an abstract standpoint we can discuss the merits of various > approaches. If it matters that humans can deal with your data in a file or > that it be able to be imported into a program like EXCEL, those are > considerations. But if not, there are quite a few relatively binary formats > where your program can save a snapshot of the data into a file and read it > back in next time. Not needed here. I'm strictly interested in getting the models of (generic) generating fleets in. Output of significant results will probably be in CSV, which nicely replicates tabular displays that I used through most of my career. > Or, did I miss something and others have already produced the data using > other tools, in which case you have to read it in at least once/ Well, the "tool" is vi, but this is a good description of what I'm doing. -- Michael F. Stemper The FAQ for rec.arts.sf.written is at Please read it before posting. From michael.stemper at gmail.com Mon Sep 27 11:47:05 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 27 Sep 2021 10:47:05 -0500 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On 25/09/2021 16.48, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-25 at 15:20:19 -0500, > "Michael F. Stemper" wrote: > >> ... For instance, if >> I modeled a fuel like this: >> >> >> ton >> 21.96 >> 18.2 >> >> >> and a generating unit like this: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> why would the fact that I could have chosen, instead, to model >> the unit of measure as an attribute of the fuel, or its name >> as a sub-element matter? Once the modeling decision has been >> made, all of the decisions that might have been would seem to >> be irrelevant. > > Disclaimer: I am not a big XML fan, for a number of reasons > already stated in this thread. > > That said, please do include units in elements like heat_content, > whether or not it's Joules/kilogram/K, and price, even if is the > local currency in the only country to which your data applies. Since the units (dimensions) don't matter as long as they're consistent between heat_content and the IHR value (MBTU and MBTU/MWh or GJ and GJ/MWh), I was initially going to ignore this suggestion. However, it seems that if I added attributes for the unit of measure of heat, that would allow checking that the data provided are indeed consistent. Thanks for the suggestion. With respect to currency, I've had customers (back when I had to work for a living) use dollars, pesetas, Euros, and pounds. In of Wood and Wollenberg[1], the authors use \cancel{R} to represent a generic currency. But I might even add a currency attribute to the price element. > If there's a standard for your industry, or your company, or on > some other level, then at least document what it is and that > you're using it, so that the next person (which may be you a > year from now) doesn't have to guess. As far as power is concerned, this is utility-level generating fleets, so it's always going to be MW -- even in the US, where we still use BTUs for heat. [1] _Power Generation, Operation, and Control; Allen J. Wood and Bruce F. Wollenberg; (c) 1984, John Wiley & Sons. -- Michael F. Stemper The FAQ for rec.arts.sf.written is at Please read it before posting. From rosuav at gmail.com Mon Sep 27 12:41:30 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 02:41:30 +1000 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On Mon, Sep 27, 2021 at 10:49 PM Skip Montanaro wrote: > > I'd like to use the jpegdupes package but > can't seem to get past compilation issued related to jpegtrans-cffi. > Attempts to install any available versions on my XUbuntu system crap out > with this C compiler error message: > > In file included from > jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267: > src/epeg_private.h:71:17: warning: ?_jpeg_init_source? declared > ?static? but never defined [-Wunused-function] > 71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo); > | ^~~~~~~~~~~~~~~~~ > src/epeg_private.h:72:20: warning: ?_jpeg_fill_input_buffer? declared > ?static? but never defined [-Wunused-function] > 72 | METHODDEF(boolean) _jpeg_fill_input_buffer(j_decompress_ptr > cinfo); > | ^~~~~~~~~~~~~~~~~~~~~~~ > src/epeg_private.h:73:17: warning: ?_jpeg_skip_input_data? declared > ?static? but never defined [-Wunused-function] > 73 | METHODDEF(void) _jpeg_skip_input_data(j_decompress_ptr cinfo, > long num_bytes); > | ^~~~~~~~~~~~~~~~~~~~~ > src/epeg_private.h:74:17: warning: ?_jpeg_term_source? declared > ?static? but never defined [-Wunused-function] > 74 | METHODDEF(void) _jpeg_term_source(j_decompress_ptr cinfo); > | ^~~~~~~~~~~~~~~~~ > src/epeg_private.h:76:17: warning: ?_jpeg_init_destination? declared > ?static? but never defined [-Wunused-function] > 76 | METHODDEF(void) _jpeg_init_destination(j_compress_ptr cinfo); > | ^~~~~~~~~~~~~~~~~~~~~~ > src/epeg_private.h:77:20: warning: ?_jpeg_empty_output_buffer? declared > ?static? but never defined [-Wunused-function] > 77 | METHODDEF(boolean) _jpeg_empty_output_buffer (j_compress_ptr > cinfo); > | ^~~~~~~~~~~~~~~~~~~~~~~~~ > src/epeg_private.h:78:17: warning: ?_jpeg_term_destination? declared > ?static? but never defined [-Wunused-function] > 78 | METHODDEF(void) _jpeg_term_destination (j_compress_ptr cinfo); > | ^~~~~~~~~~~~~~~~~~~~~~ Those are all warnings. Are there any errors that follow them? > ... > This seems like a rather odd error message for a presumably sort-of-stable > PyPI package. Looking at the version dependencies I see 2.6, 2.7, 3.3 or > PyPy. I thought nmybe I needed to try with Python2... Create a Conda > environment, attempt to install jpegdupes, only to discover it's Python 3 > only. > > So, I'm kind of stuck. Maybe I need to install Python 3.3 and try that? Any > other ideas? > The package doesn't seem to have had a release since 2018, but the GitHub repository has had changes as recently as a month ago. You might have better luck cloning the repo and building that. ChrisA From rosuav at gmail.com Mon Sep 27 12:49:03 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 02:49:03 +1000 Subject: XML Considered Harmful In-Reply-To: References: Message-ID: On Tue, Sep 28, 2021 at 2:30 AM Michael F. Stemper wrote: > As far as power is concerned, this is utility-level generating fleets, > so it's always going to be MW -- even in the US, where we still use > BTUs for heat. > It's easy for *you* to know, and therefore assume, that it's always MW. But someone else coming along will appreciate some sort of indication that it's MW and not (say) KW or GW. I've spent a long time decoding other people's file formats, trying to figure out what unit something is in. "Huh. The date seems to be stored in........ hours?!?" ChrisA From eryksun at gmail.com Mon Sep 27 13:03:17 2021 From: eryksun at gmail.com (Eryk Sun) Date: Mon, 27 Sep 2021 12:03:17 -0500 Subject: Python added to PATH, cannot be directly accessed, cannot install pip In-Reply-To: References: Message-ID: On 9/27/21, Will wrote: > > I am using a Lenovo Laptop using Windows. I'm trying to install > get-pip.py The installer includes pip by default and has an option to update PATH for you, though the latter isn't enabled by default. Latest 64-bit release: https://www.python.org/ftp/python/3.9.7/python-3.9.7-amd64.exe From skip.montanaro at gmail.com Mon Sep 27 13:11:18 2021 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 27 Sep 2021 13:11:18 -0400 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: > > Those are all warnings. Are there any errors that follow them? > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try. The package doesn't seem to have had a release since 2018, but the > GitHub repository has had changes as recently as a month ago. You > might have better luck cloning the repo and building that. > Thanks, good suggestion. FWIW, I'm trying to whittle down at least 12,000 images to a more manageable number for the sister of a good friend who recently passed away. I've got a straightforward dedupe program, but need something which can compare just the data chunk of JPEGs, ignoring the metadata. This program apparently does that. Is like to avoid reinventing that wheel. Skip From rosuav at gmail.com Mon Sep 27 13:23:53 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 03:23:53 +1000 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro wrote: >> >> Those are all warnings. Are there any errors that follow them? > > > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try. Yeah, one of the annoying things of building large C packages is when the maintainers don't care about warnings, and then legit errors get lost in the spam. >> The package doesn't seem to have had a release since 2018, but the >> GitHub repository has had changes as recently as a month ago. You >> might have better luck cloning the repo and building that. > > > Thanks, good suggestion. > > FWIW, I'm trying to whittle down at least 12,000 images to a more manageable number for the sister of a good friend who recently passed away. I've got a straightforward dedupe program, but need something which can compare just the data chunk of JPEGs, ignoring the metadata. This program apparently does that. Is like to avoid reinventing that wheel. > Makes sense! Though if you're just doing a bitwise comparison of the data chunks, it shouldn't be TOO hard to reinvent. JFIF is a reasonably easy format to understand (although the mathematics of image encoding and decoding are a bit less so). ChrisA From dieter at handshake.de Mon Sep 27 13:50:10 2021 From: dieter at handshake.de (Dieter Maurer) Date: Mon, 27 Sep 2021 19:50:10 +0200 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: <24914.1106.451077.142723@ixdm.fritz.box> Skip Montanaro wrote at 2021-9-27 07:48 -0500: >I'd like to use the jpegdupes package but >can't seem to get past compilation issued related to jpegtrans-cffi. >Attempts to install any available versions on my XUbuntu system crap out >with this C compiler error message: > > In file included from >jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267: > src/epeg_private.h:71:17: warning: ?_jpeg_init_source? declared >?static? but never defined [-Wunused-function] > 71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo); > | ^~~~~~~~~~~~~~~~~ > ... You list only warnings (which usually can be ignored). If the compilation fails, you should see error messages, too. From grant.b.edwards at gmail.com Mon Sep 27 13:51:19 2021 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 27 Sep 2021 10:51:19 -0700 (PDT) Subject: Posts from gmane no longer allowed? References: Message-ID: <61520497.1c69fb81.3a367.b04c@mx.google.com> On 2021-09-27, ??? wrote: > Grant Edwards writes: > >> I've been reading (and posting to) this list for many years by >> pointing an NNTP client >> at news://gmane.comp.python.general. Sometime in the past few days posts started >> being refused: >> >> You have tried posting to gmane.comp.python.general, which is a >> unidirectional >> mailing list. Gmane can therefore not send this message to that >> mailing list. >> >> Was this a change made by the mailing list admins? >> >> If so, is it permanent? >> >> [Trying to send a plaintext e-mail via Gmail, but not sure if it's working.] > > Oh this is unhappy news. Because i also using Gmane with > gmane.emacs.help and gmane.linux.debian.user. Posting to other lists in gmane still works fine for me. It's only the python list that turned into read-only. > Only i use Usenet (eternal september) with comp.lang.python. I've switched back and forth between Usenet and gmane a couple times trying to figure out which one had the least breakage in references: headers. I acually wrote a Python program that acted as an NNTP client and counted thread breakage. The last time I ran that test gmane had a slight advantage in thread integrity and fewer posters in need of plonking. -- Grant From 2QdxY4RzWzUUiLuE at potatochowder.com Mon Sep 27 14:03:18 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Mon, 27 Sep 2021 11:03:18 -0700 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On 2021-09-28 at 03:23:53 +1000, Chris Angelico wrote: > On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro wrote: > >> > >> Those are all warnings. Are there any errors that follow them? > > > > > > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try. > > Yeah, one of the annoying things of building large C packages is when > the maintainers don't care about warnings, and then legit errors get > lost in the spam. It's also possible that the compiler is running in a "warnings are errors" mode. Hack the Makefile (or equivalent) at your own peril. One of the annoying things about warnings is that they're not standard, so squelching them under one compiler makes another compiler (or a different version of the same compiler) whine. IIRC, a compliant C compiler can issue a warning for anything it wants, including "this variable name contains too many vowels" or "this is a computer, and today may be Tuesday." In many ways, the engineering pieces of software engineering remain in their infancy. *sigh* From rosuav at gmail.com Mon Sep 27 14:16:58 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 04:16:58 +1000 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On Tue, Sep 28, 2021 at 4:04 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > > On 2021-09-28 at 03:23:53 +1000, > Chris Angelico wrote: > > > On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro wrote: > > >> > > >> Those are all warnings. Are there any errors that follow them? > > > > > > > > > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try. > > > > Yeah, one of the annoying things of building large C packages is when > > the maintainers don't care about warnings, and then legit errors get > > lost in the spam. > > It's also possible that the compiler is running in a "warnings are > errors" mode. Hack the Makefile (or equivalent) at your own peril. > > One of the annoying things about warnings is that they're not standard, > so squelching them under one compiler makes another compiler (or a > different version of the same compiler) whine. IIRC, a compliant C > compiler can issue a warning for anything it wants, including "this > variable name contains too many vowels" or "this is a computer, and > today may be Tuesday." > > In many ways, the engineering pieces of software engineering remain in > their infancy. *sigh* But that's kinda the point of having different compilers. Warnings that we consider fairly standard today didn't exist in the past, and someone had to be the first to add them. If the C language had to mandate every warning, it'd take forever to make changes. I don't think that warnings-are-errors mode is a good thing though. They should remain just as warnings. (And with the most popular compilers, using the compilation options as specified in their own makefile, maintainers should ideally try to ensure that there are as few warnings as possible.) ChrisA From 2QdxY4RzWzUUiLuE at potatochowder.com Mon Sep 27 14:44:45 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Mon, 27 Sep 2021 11:44:45 -0700 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On 2021-09-28 at 04:16:58 +1000, Chris Angelico wrote: > On Tue, Sep 28, 2021 at 4:04 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > > > > On 2021-09-28 at 03:23:53 +1000, > > Chris Angelico wrote: > > > > > On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro wrote: > > > >> > > > >> Those are all warnings. Are there any errors that follow them? > > > > > > > > > > > > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try. > > > > > > Yeah, one of the annoying things of building large C packages is when > > > the maintainers don't care about warnings, and then legit errors get > > > lost in the spam. > > > > It's also possible that the compiler is running in a "warnings are > > errors" mode. Hack the Makefile (or equivalent) at your own peril. > > > > One of the annoying things about warnings is that they're not standard, > > so squelching them under one compiler makes another compiler (or a > > different version of the same compiler) whine. IIRC, a compliant C > > compiler can issue a warning for anything it wants, including "this > > variable name contains too many vowels" or "this is a computer, and > > today may be Tuesday." > > > > In many ways, the engineering pieces of software engineering remain in > > their infancy. *sigh* > > But that's kinda the point of having different compilers. Warnings > that we consider fairly standard today didn't exist in the past, and > someone had to be the first to add them. If the C language had to > mandate every warning, it'd take forever to make changes. That is indeed one benefit of having different compilers and compilers that improve over time. It's also a nuisance when the compiler in your development/unit test environment produces different warnings than either cross compiler for your target environments (plural). In Python, it's more likely that MSVC, MinGW, LLVM, and gcc choose different warnings, and that no collection of settings supresses them all. Present C projects contain oodles of conditional compilation, often controlled by the preprocessor; achieving ideal results can be, well, "tricky." It's not a matter of not caring. At some point, it comes down to a question of ignoring a warning vs. cluttering up your code (preprocessor or otherwise) with extra conditionals. Or being stuck with a warning about unneeded defensive programming vs. being stuck with a warning about possible arithmetic overflow on your 32-bit target. I agree that the standard shouldn't regulate warnings. I'm also saying that sometimes, it's a pain that it doesn't, and that not every warning is the package maintainer's fault. :-) > I don't think that warnings-are-errors mode is a good thing though. > They should remain just as warnings. (And with the most popular > compilers, using the compilation options as specified in their own > makefile, maintainers should ideally try to ensure that there are as > few warnings as possible.) Ideally, yes. From rosuav at gmail.com Mon Sep 27 14:49:17 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 04:49:17 +1000 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On Tue, Sep 28, 2021 at 4:46 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > I agree that the standard shouldn't regulate warnings. I'm also saying > that sometimes, it's a pain that it doesn't, and that not every warning > is the package maintainer's fault. :-) Agreed. For the most part, a proper colorizing compiler can make the errors stand out among the spam of warnings, but it's hard when people copy and paste just the text. There's a skill to skimming compiler output to find what's relevant (for instance, just before an error, there might be a related warning, which actually is highlighting the real problem). ChrisA From mats at wichmann.us Mon Sep 27 15:43:34 2021 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 27 Sep 2021 13:43:34 -0600 Subject: Python added to PATH, cannot be directly accessed, cannot install pip In-Reply-To: References: Message-ID: <0f63f1cc-ed82-cbf9-89b7-e96c0341e64a@wichmann.us> On 9/27/21 08:00, Will wrote: > Hello team at python.org, > > > > I've asked this question on a forum and tried to figure it out myself, but > I'm at a dead end. I don't know if you guys answer questions like this but > I have no where else to turn to. > > > > I am using a Lenovo Laptop using Windows. I'm trying to install > get-pip.py, and when I enter "python get-pip.py" into the Command Prompt, > it says "Python was not found; run without arguments to install from the > Microsoft Store, or disable this shortcut from Settings > Manage App > Execution Aliases." > > > > So for a week, I've been trying to add Python to PATH, so I can install > get-pip.py to Python. > > > > I've added the exact file location for Python to Path in both User > Variables and System Variables. I've executed Manage App Execution > Aliases, turned off both App Installer python.exe and App Installer > python3.exe. Still can't install pip, Python was still not found. I've > tried different keywords ("py get-pip.py", python3 get-pip.py", etc.). > Still doesn't work. > > > > Python is added to PATH and I can still run Python scripts, but I can't > find Python directly through the Command Prompt, and I cannot install > get-pip.py to Python. > > > > For reference, I have Python version 3.9.6 and I installed Python directly > from the site (I did not use Anaconda). > > > > Can you guys help me with this? Or do I need to delete Python and > reinstall it? In addition to Eryk's comments... Try invoking Python from a command prompt by typing "py" (I think you knew this). For a python.org installation, as long as you didn't deselect the installation of the Python Launcher, that's the way to launch it. Meanwhile, you can also install from the Microsoft Store - and the message you're getting is from the little stub that tries to be helpful about telling you this when you try as "python" - that's not Python itself that's running, it's the Windows helper. pip, meanwhile, is not in the same directory as the python executable, so even "adding python to PATH" doesn't solve the problem of running pip. Invoke it like this instead: py -m pip install foo you _can_ add the scripts subdirectory of the python install location to PATH as well, but it's better to get in the habit of running pip as a module, because it works way better once you have multiple pythons installed (which happens a lot). From skip.montanaro at gmail.com Mon Sep 27 18:08:49 2021 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 27 Sep 2021 17:08:49 -0500 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: > > For the most part, a proper colorizing compiler can make the > errors stand out among the spam of warnings, but it's hard when people > copy and paste just the text. > In my defense, the entire traceback was red. :-) It's quite possible that GCC colorized its error/warning messages, but colorizing might have been suppressed by stderr being fed into a pipe, or by distutils tossing it out. Skip From rosuav at gmail.com Mon Sep 27 18:12:08 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 08:12:08 +1000 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On Tue, Sep 28, 2021 at 8:09 AM Skip Montanaro wrote: >> >> For the most part, a proper colorizing compiler can make the >> errors stand out among the spam of warnings, but it's hard when people >> copy and paste just the text. > > > In my defense, the entire traceback was red. :-) It's quite possible that GCC colorized its error/warning messages, but colorizing might have been suppressed by stderr being fed into a pipe, or by distutils tossing it out. > Ah, yeah, that's a nuisance. Honestly, most release systems should be configured with fewer warnings, but maybe people don't distinguish between dev and release. ChrisA From skip.montanaro at gmail.com Mon Sep 27 20:40:09 2021 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 27 Sep 2021 19:40:09 -0500 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h file. Thank goodness for the apt-file command. I was able to track that down to the libturbojpeg0-dev package, install that, and after a bit more fussing around now have jpegdupes installed. Thanks for the help, Skip From rosuav at gmail.com Mon Sep 27 20:44:02 2021 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Sep 2021 10:44:02 +1000 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On Tue, Sep 28, 2021 at 10:40 AM Skip Montanaro wrote: > > Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h > file. Thank goodness for the apt-file command. I was able to track > that down to the libturbojpeg0-dev package, install that, and after a > bit more fussing around now have jpegdupes installed. > > Thanks for the help, > Sweet! I like stories that have happy endings. (No compilers were harmed in the making of this.) ChrisA From avigross at verizon.net Mon Sep 27 21:01:04 2021 From: avigross at verizon.net (Avi Gross) Date: Mon, 27 Sep 2021 21:01:04 -0400 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> Message-ID: <016501d7b404$5006a2c0$f013e840$@verizon.net> Michael, Given your further explanation, indeed reading varying numbers of points in using a CSV is not valid, albeit someone might just make N columns (maybe a few more than 7) to handle a hopefully worst case. Definitely it makes more sense to read in a list or other data structure. You keep talking about generators, though. If the generators are outside of your program, then yes, you need to read in whatever they produce. But if your data generator is within your own program, that opens up other possibilities. I am not saying you necessarily would want to use the usual numpy/pandas modules and have some kind of data.frame. I do know other languages (like R) where I have used columns that are lists. My impression is you may not be using your set of data points for any other purposes except when ready to draw a spline. Again, in some languages this opens up many possibilities. A fairly trivial one is if you store your points as something like "1.2:3.86:12:83.2" meaning a character string with some divider. When ready to use that, it is fairly straightforward to convert it to a list to use for your purpose. Can I just ask if by a generator, you do NOT mean the more typical use of "generator" as used in python in which some code sort of runs as needed to keep generating the next item to work on. Do you mean something that creates realistic test cases to simulate a real-word scenario? These often can create everything at once and often based on random numbers. Again, if you have or build such code, it is not clear it needs to be written to disk and then read back. You may of course want to save it, perhaps as a log, to show what your program was working on. -----Original Message----- From: Python-list On Behalf Of Michael F. Stemper Sent: Monday, September 27, 2021 11:40 AM To: python-list at python.org Subject: Re: XML Considered Harmful On 25/09/2021 16.39, Avi Gross wrote: > Michael, > > I don't care what you choose. Whatever works is fine for an internal use. Maybe I should have taken the provoking article with a few more grains of salt. At this point, I'm not seeing any issues that are applicable to my use case. > But is the data scheme you share representative of your actual application? > >>From what I see below, unless the number of "point" variables is not >>always > exactly four, the application might be handled well by any format that > handles rectangular data, perhaps even CSV. > > You show a I mean anything like a data.frame can contain data columns > like > p1,p2,p3,p4 and a categorical one like IHRcurve_name. > > Or do you have a need for more variability such as an undetermined > number of similar units in ways that might require more flexibility or > be more efficient done another way? As far as the number of points per IHR curve, the only requirement is that there must be at least two. It's hard to define a line segment with only one. The mock data that I have so far has curves ranging from two to five points. I didn't notice that the snippet that I posted had two curves with the same number of breakpoints, which was misleading. My former employer's systems had, IIRC, space for seven points per curve in the database structures. Of all the sizing changes made over a long career, I don't recall any customer ever requiring more than that. But, it's cleanest to use python lists (with no inherent sizing limitations) to represent the IHR (and incremental cost) curves. > MOST of the discussion I am seeing here seems peripheral to getting > you what you need for your situation and may require a learning curve > to learn to use properly. Are you planning on worrying about how to > ship your data encrypted, for example? Any file format you use for > storage can presumably be encrypted and send and decrypted if that matters. This work is intended to look at the feasability of relaxing some constraints normally required for the solution of Economic Dispatch. So all of my data are hypothetical. Once I have stuff up and running, I'll be making up data for lots of different generators. Being retired, I don't have access to any proprietary information about any specific generators, so all of the data is made up out of my head. I still need a way to get it into my programs, of course. > So, yes, from an abstract standpoint we can discuss the merits of > various approaches. If it matters that humans can deal with your data > in a file or that it be able to be imported into a program like EXCEL, > those are considerations. But if not, there are quite a few relatively > binary formats where your program can save a snapshot of the data into > a file and read it back in next time. Not needed here. I'm strictly interested in getting the models of (generic) generating fleets in. Output of significant results will probably be in CSV, which nicely replicates tabular displays that I used through most of my career. > Or, did I miss something and others have already produced the data > using other tools, in which case you have to read it in at least once/ Well, the "tool" is vi, but this is a good description of what I'm doing. -- Michael F. Stemper The FAQ for rec.arts.sf.written is at Please read it before posting. -- https://mail.python.org/mailman/listinfo/python-list From 2QdxY4RzWzUUiLuE at potatochowder.com Mon Sep 27 21:19:53 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Mon, 27 Sep 2021 18:19:53 -0700 Subject: Package conflicts trying to install jpegdupes package In-Reply-To: References: Message-ID: On 2021-09-28 at 10:44:02 +1000, Chris Angelico wrote: > On Tue, Sep 28, 2021 at 10:40 AM Skip Montanaro > wrote: > > > > Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h > > file. Thank goodness for the apt-file command. I was able to track > > that down to the libturbojpeg0-dev package, install that, and after a > > bit more fussing around now have jpegdupes installed. > > > > Thanks for the help, > > > > Sweet! I like stories that have happy endings. > > (No compilers were harmed in the making of this.) Nor compiler authors/vendors/maintainers. From eryksun at gmail.com Tue Sep 28 02:55:28 2021 From: eryksun at gmail.com (Eryk Sun) Date: Tue, 28 Sep 2021 01:55:28 -0500 Subject: Python added to PATH, cannot be directly accessed, cannot install pip In-Reply-To: <0f63f1cc-ed82-cbf9-89b7-e96c0341e64a@wichmann.us> References: <0f63f1cc-ed82-cbf9-89b7-e96c0341e64a@wichmann.us> Message-ID: On 9/27/21, Mats Wichmann wrote: > > pip, meanwhile, is not in the same directory as the python executable, > so even "adding python to PATH" doesn't solve the problem of running > pip. Invoke it like this instead: The installer's option to add Python to PATH adds both the installation directory and the scripts directory. The latter has "pip.exe", assuming pip was installed. > py -m pip install foo If the `pip` command is on PATH, for an installation or active virtual environment, it's usually fine to use it instead of `py -m pip` or `python -m pip`. But there's one caveat. The `pip` command can't be used to upgrade pip itself. For that you have to run the module via `'py -m pip` or `python -m pip`. From hjp-python at hjp.at Tue Sep 28 03:25:34 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Tue, 28 Sep 2021 09:25:34 +0200 Subject: XML Considered Harmful In-Reply-To: <016501d7b404$5006a2c0$f013e840$@verizon.net> References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On 2021-09-27 21:01:04 -0400, Avi Gross via Python-list wrote: > You keep talking about generators, though. If the generators are outside of > your program, then yes, you need to read in whatever they produce. As I understood it, the "generators" don't generate the data, they are the subject of the data: Devices that generate electricity by burning fuel and he's modelling some aspect of their operation. Maybe efficiency or power output or something like that (I tried to search for "IHR curve", but couldn't find anything). hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From PythonList at DancesWithMice.info Tue Sep 28 03:27:44 2021 From: PythonList at DancesWithMice.info (dn) Date: Tue, 28 Sep 2021 20:27:44 +1300 Subject: XML Considered Harmful In-Reply-To: <1092b736-985e-6e1a-9b4c-534bbcd674de@DancesWithMice.info> References: <5717edf8-52df-eb99-5837-fccdd645dce9@DancesWithMice.info> <1092b736-985e-6e1a-9b4c-534bbcd674de@DancesWithMice.info> Message-ID: On 25/09/2021 11.26, David L Neil via Python-list wrote: > On 25/09/2021 11.00, Chris Angelico wrote: > >> Invented because there weren't enough markup languages, so we needed another? > > Anything You Can Do I Can Do Better > https://www.youtube.com/watch?v=_UB1YAsPD6U Article (rather brief) introducing YAML, of possible interest: https://opensource.com/article/21/9/intro-yaml -- Regards, =dn From michael.stemper at gmail.com Tue Sep 28 11:37:53 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 28 Sep 2021 10:37:53 -0500 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On 27/09/2021 20.01, Avi Gross wrote: > Michael, > > Given your further explanation, indeed reading varying numbers of points in > using a CSV is not valid, albeit someone might just make N columns (maybe a > few more than 7) to handle a hopefully worst case. Definitely it makes more > sense to read in a list or other data structure. > > You keep talking about generators, though. If the generators are outside of > your program, then yes, you need to read in whatever they produce. My original post (which is as the snows of yesteryear) made explicit the fact that when I refer to a generator, I'm talking about something made from tons of iron and copper that is oil-filled and rotates at 1800 rpm. (In most of the world other than North America, they rotate at 1500 rpm.) Nothing to do with the similarly-named python construct. Sorry for the ambiguity. > But if > your data generator is within your own program, The data is created in my mind, and approximates typical physical characteristics of real generators. > My impression is you may not be using your set of data points for any other > purposes except when ready to draw a spline. Nope, the points give a piecewise-linear curve, and values between two consecutive points are found by linear interpolation. It's industry standard practice. > Can I just ask if by a generator, you do NOT mean the more typical use of > "generator" as used in python Nope; I mean something that weighs 500 tons and rotates, producing electrical energy. > Do you mean something that creates > realistic test cases to simulate a real-word scenario? The thing that creates realistic test cases is my brain. > These often can > create everything at once and often based on random numbers. I have written such, but not in the last thirty years. At that time, I needed to make up data for fifty or one hundred generators, along with tie lines and loads. What I'm working on now only needs a handful of generators at a time; just enough to test my hypothesis. (Theoretically, I could get by with two, but that offends my engineering sensibilities.) > create everything at once and often based on random numbers. Again, if you > have or build such code, it is not clear it needs to be written to disk and > then read back. Well, I could continue to hard-code the data into one of the test programs, but that would mean that every time that I wanted to look at a different scenario, I'd need to modify a program. And when I discover anomalous behavior, I'd need to copy the hard-coded data into another program. Having the data in a separate file means that I can provide a function to read that file and return a list of generators (or fuels) to a program. Multiple test cases are then just multiple files, all of which are available to multiple programs. > You may of course want to save it, perhaps as a log, to show > what your program was working on. That's another benefit of having the data in external files. -- Michael F. Stemper A preposition is something you should never end a sentence with. From michael.stemper at gmail.com Tue Sep 28 11:45:17 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 28 Sep 2021 10:45:17 -0500 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On 28/09/2021 02.25, Peter J. Holzer wrote: > On 2021-09-27 21:01:04 -0400, Avi Gross via Python-list wrote: >> You keep talking about generators, though. If the generators are outside of >> your program, then yes, you need to read in whatever they produce. > > As I understood it, the "generators" don't generate the data, they are > the subject of the data: Devices that generate electricity by burning > fuel and he's modelling some aspect of their operation. Maybe efficiency > or power output or something like that (I tried to search for "IHR > curve", but couldn't find anything). If you expand "IHR curve" to "incremental heat rate curve", you'll get better results. When power engineers talk, we say the first, when we publish papers, we write the second. If you want to see the bigger picture, search on "Economic Dispatch". In fact, doing so points me to something written by a guy I worked with back in the 1980s: Slide 3 even shows a piecewise-linear curve. -- Michael F. Stemper A preposition is something you should never end a sentence with. From michael.stemper at gmail.com Tue Sep 28 13:53:49 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 28 Sep 2021 12:53:49 -0500 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On 28/09/2021 10.53, Stefan Ram wrote: > "Michael F. Stemper" writes: >> Well, I could continue to hard-code the data into one of the test >> programs > > One can employ a gradual path from a program with hardcoded > data to an entity sharable by different programs. > > When I am hurried to rush to a working program, I often > end up with code that contains configuration data spread > (interspersed) all over the code. For example: > 1st step: give a name to all the config data: > 2nd: move all config data to the top of the source code, > directly after all the import statements: > 3rd: move all config data to a separate "config.py" module: > > import ... > import config > ... > > ... > open( config.project_directory + "data.txt" ) > ... > >> but that would mean that every time that I wanted to look >> at a different scenario, I'd need to modify a program. > > Now you just have to modify "config.py" - clearly separated > from the (rest of the) "program". Well, that doesn't really address what format to store the data in. I was going to write a module that would read data from an XML file: import EDXML gens = EDXML.GeneratorsFromXML( "gendata1.xml" ) fuels = EDXML.FuelsFromXML( "fueldata3.xml" ) (Of course, I'd really get the file names from command-line arguments.) Then I read a web page that suggested use of XML was a poor idea, so I posted here asking for a clarification and alternate suggestions. One suggestion was that I use YAML, in which case, I'd write: import EDfromYAML gens = EDfromYAML( "gendata1.yaml" ) fuels = EDXML.FuelsFromYAML( "fueldata3.yaml" ) >> And when I discover anomalous behavior, I'd need to copy the >> hard-coded data into another program. > > Now you just have to import "config.py" from the other program. This sounds like a suggestion that I hard-code the data into a module. I suppose that I could have half-a-dozen modules with different data sets and ln them as required: $ rm GenData.py* FuelData.py* $ ln gendata1.py GenData.py $ ln fueldata3.py FuelData.py It seems to me that a more thorough separation of code and data might be useful. -- Michael F. Stemper The name of the story is "A Sound of Thunder". It was written by Ray Bradbury. You're welcome. From avigross at verizon.net Tue Sep 28 14:23:26 2021 From: avigross at verizon.net (Avi Gross) Date: Tue, 28 Sep 2021 14:23:26 -0400 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: <039f01d7b495$ede60300$c9b20900$@verizon.net> I replied to Michael privately but am intrigued by his words here: "The thing that creates realistic test cases is my brain." I consider extensions to my brain to include using a language like Python on my computer and in particular, to take a model I think of and instantiate it. Lots of people have shared modules that can be tweaked to do all kinds of simulations using a skeleton you provide that guides random number usage. Some will generate lots of those and stare at them and use their brain to further narrow it down to realistic ones. For example, in designing say a car with characteristics like miles per gallon should randomly range between 10 and 100 while engine size ranges from this to that and so on, it may turn out that large engines don't go well with large number for miles per gallon. I have worked on projects where a set of guides then created hundreds of thousands of fairly realistic scenarios using every combination of an assortment of categorical variables and the rest of the program sliced and diced the results and did all kinds of statistical calculations and then generated all kinds of graphs. There was no real data but there was a generator that was based on the kinds of distributions previously published in the field that helped guide parameters to be somewhat realistic. In your case, I understand you will decide how to do it and just note you used language with multiple meanings that misled a few of us into thinking you either had a python function in mind using one of several ways Python refers to as generators, such as one that efficiently yields the next prime number when asked. Clearly your explanation now shows you plan on making a handful of data sets by hand using an editor like vi. Fair enough. No need to write complex software if your mind is easily able to just make half a dozen variations in files. And, frankly, not sure why you need XML or much of anything. It obviously depends on how much you are working with and how variable. For simpler things, you can hard-code your data structure directly into your program, run an analysis, change the variables to your second simulation and repeat. I am afraid that I, like a few others here, assumed a more abstract and much more complex need to be addressed. Yours may be complex in other parts but may need nothing much for the part we are talking about. It sounds like you do want something easier to create while editing. -----Original Message----- From: Python-list On Behalf Of Michael F. Stemper Sent: Tuesday, September 28, 2021 11:38 AM To: python-list at python.org Subject: Re: XML Considered Harmful On 27/09/2021 20.01, Avi Gross wrote: > Michael, > > Given your further explanation, indeed reading varying numbers of > points in using a CSV is not valid, albeit someone might just make N > columns (maybe a few more than 7) to handle a hopefully worst case. > Definitely it makes more sense to read in a list or other data structure. > > You keep talking about generators, though. If the generators are > outside of your program, then yes, you need to read in whatever they produce. My original post (which is as the snows of yesteryear) made explicit the fact that when I refer to a generator, I'm talking about something made from tons of iron and copper that is oil-filled and rotates at 1800 rpm. (In most of the world other than North America, they rotate at 1500 rpm.) Nothing to do with the similarly-named python construct. Sorry for the ambiguity. > But if > your data generator is within your own program, The data is created in my mind, and approximates typical physical characteristics of real generators. > My impression is you may not be using your set of data points for any > other purposes except when ready to draw a spline. Nope, the points give a piecewise-linear curve, and values between two consecutive points are found by linear interpolation. It's industry standard practice. > Can I just ask if by a generator, you do NOT mean the more typical use > of "generator" as used in python Nope; I mean something that weighs 500 tons and rotates, producing electrical energy. > Do you mean something that creates > realistic test cases to simulate a real-word scenario? The thing that creates realistic test cases is my brain. > These often can > create everything at once and often based on random numbers. I have written such, but not in the last thirty years. At that time, I needed to make up data for fifty or one hundred generators, along with tie lines and loads. What I'm working on now only needs a handful of generators at a time; just enough to test my hypothesis. (Theoretically, I could get by with two, but that offends my engineering sensibilities.) > create everything at once and often based on random numbers. Again, if > you have or build such code, it is not clear it needs to be written to > disk and then read back. Well, I could continue to hard-code the data into one of the test programs, but that would mean that every time that I wanted to look at a different scenario, I'd need to modify a program. And when I discover anomalous behavior, I'd need to copy the hard-coded data into another program. Having the data in a separate file means that I can provide a function to read that file and return a list of generators (or fuels) to a program. Multiple test cases are then just multiple files, all of which are available to multiple programs. > You may of course want to save it, perhaps as a log, to show what > your program was working on. That's another benefit of having the data in external files. -- Michael F. Stemper A preposition is something you should never end a sentence with. -- https://mail.python.org/mailman/listinfo/python-list From avigross at verizon.net Tue Sep 28 14:27:35 2021 From: avigross at verizon.net (Avi Gross) Date: Tue, 28 Sep 2021 14:27:35 -0400 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: <03a801d7b496$8247da00$86d78e00$@verizon.net> Well, Michael, if you want to go back to the eighties, and people you worked with, I did my Thesis with a professor who later had an Erd?s number of 1! Too bad I never got around to publishing something with him or I could have been a 2! But that work, being so long ago, was not in Python but mainly in PASCAL. Ah the good old days. -----Original Message----- From: Python-list On Behalf Of Michael F. Stemper Sent: Tuesday, September 28, 2021 11:45 AM To: python-list at python.org Subject: Re: XML Considered Harmful On 28/09/2021 02.25, Peter J. Holzer wrote: > On 2021-09-27 21:01:04 -0400, Avi Gross via Python-list wrote: >> You keep talking about generators, though. If the generators are >> outside of your program, then yes, you need to read in whatever they produce. > > As I understood it, the "generators" don't generate the data, they are > the subject of the data: Devices that generate electricity by burning > fuel and he's modelling some aspect of their operation. Maybe > efficiency or power output or something like that (I tried to search > for "IHR curve", but couldn't find anything). If you expand "IHR curve" to "incremental heat rate curve", you'll get better results. When power engineers talk, we say the first, when we publish papers, we write the second. If you want to see the bigger picture, search on "Economic Dispatch". In fact, doing so points me to something written by a guy I worked with back in the 1980s: Slide 3 even shows a piecewise-linear curve. -- Michael F. Stemper A preposition is something you should never end a sentence with. -- https://mail.python.org/mailman/listinfo/python-list From Karsten.Hilbert at gmx.net Tue Sep 28 14:30:54 2021 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Tue, 28 Sep 2021 20:30:54 +0200 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: Am Tue, Sep 28, 2021 at 12:53:49PM -0500 schrieb Michael F. Stemper: > This sounds like a suggestion that I hard-code the data into a > module. I suppose that I could have half-a-dozen modules with > different data sets and ln them as required: > > $ rm GenData.py* FuelData.py* > $ ln gendata1.py GenData.py > $ ln fueldata3.py FuelData.py vi data.py generators = {} generators['name1'] = {'fuel': ..., ...} generators['name2'] = {...} ... vi simulation.py import sys import data generator = data.generators[sys.argv[1]] run_simulation(generator) or some such ? Your data "format" is ... Python code. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B From michael.stemper at gmail.com Tue Sep 28 14:41:04 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 28 Sep 2021 13:41:04 -0500 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> <03a801d7b496$8247da00$86d78e00$@verizon.net> Message-ID: On 28/09/2021 13.27, Avi Gross wrote: > Well, Michael, if you want to go back to the eighties, and people you worked > with, I did my Thesis with a professor who later had an Erd?s number of 1! > Too bad I never got around to publishing something with him or I could have > been a 2! Lucky you. If a paper that a friend of mine is submitting to various journals gets accepted by one of them, I'll end up with a 4 or 5 through him. However, as the months pass, it's looking more like mine will end up NaN. -- Michael F. Stemper Isaiah 58:6-7 From avigross at verizon.net Tue Sep 28 15:21:35 2021 From: avigross at verizon.net (Avi Gross) Date: Tue, 28 Sep 2021 15:21:35 -0400 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> <03a801d7b496$8247da00$86d78e00$@verizon.net> Message-ID: <042f01d7b49e$0e10e660$2a32b320$@verizon.net> Not lucky at all, Michael. The problem is he published a number of things with Paul Erd?s a few years after I got my degrees and went to Bell laboratories. I never met Erd?s but he was prolific and had 507 people publish with him as co-authors. I would have loved to as I also speak languages he spoke including Hungarian and Math. Well, time to get back to something remotely about Python. Is there any concept of a Rossum Number where anyone who worked directly with Guido Van Rossum is a 1 (or True or truthy) and ... Hey I just realized my Berners-Lee number might be 1 but it was so long ago we worked on what Hypertext should look like, ... -----Original Message----- From: Python-list On Behalf Of Michael F. Stemper Sent: Tuesday, September 28, 2021 2:41 PM To: python-list at python.org Subject: Re: XML Considered Harmful On 28/09/2021 13.27, Avi Gross wrote: > Well, Michael, if you want to go back to the eighties, and people you > worked with, I did my Thesis with a professor who later had an Erd?s number of 1! > Too bad I never got around to publishing something with him or I could > have been a 2! Lucky you. If a paper that a friend of mine is submitting to various journals gets accepted by one of them, I'll end up with a 4 or 5 through him. However, as the months pass, it's looking more like mine will end up NaN. -- Michael F. Stemper Isaiah 58:6-7 -- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Tue Sep 28 18:07:57 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 29 Sep 2021 08:07:57 +1000 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On Wed, Sep 29, 2021 at 8:00 AM Stefan Ram wrote: > JSON is a kind of a subset of JavaScript for JavaScript > programmers. In Python, we can use JSON too, or we can > use Python itself. > > When some external requirement to use a data exchange > notation like JSON should appear, one can still "translate" > such Python modules to JSON. This path is not blocked. JSON exists as a transport mechanism because it is restricted and can't contain malicious code. A Python equivalent would be ast.literal_eval - a strict subset of the language but restricted for safety. For trusted code, yes, straight code can be used. (And ast.literal_eval, unlike JSON, can handle comments.) ChrisA From PythonList at DancesWithMice.info Tue Sep 28 18:38:22 2021 From: PythonList at DancesWithMice.info (dn) Date: Wed, 29 Sep 2021 11:38:22 +1300 Subject: OT: AttributeError In-Reply-To: References: Message-ID: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> On 29/09/2021 10.50, Stefan Ram wrote: > (For Python programmers who have watched "Game of Thrones".) > > |>>> class girl: > |... pass > |... > |>>> girl = girl() > |>>> print( girl.name ) > |Traceback (most recent call last): > | File "", line 1, in > |AttributeError: A girl has no name. For oldies, who remember the musical "South Pacific" (to say nothing of calculating addresses on a backing-store): sing "on a clear disk, you can seek forever..." For C programmers who are fans of pirate stories: long John # Silver For those of us who remember/can compute in binary, octal, hex, or decimal as-needed: Why do programmers confuse All Hallows'/Halloween for Christmas Day? -- Regards, =dn From 2QdxY4RzWzUUiLuE at potatochowder.com Tue Sep 28 19:08:56 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Tue, 28 Sep 2021 16:08:56 -0700 Subject: OT: AttributeError In-Reply-To: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On 2021-09-29 at 11:38:22 +1300, dn via Python-list wrote: > For those of us who remember/can compute in binary, octal, hex, or > decimal as-needed: > Why do programmers confuse All Hallows'/Halloween for Christmas Day? That one is also very old. (Yes, I know the answer. No, I will not spoil it for those who might not.) What do I have to do to gain the insight necessary to have discovered that question and answer on my own? From rosuav at gmail.com Tue Sep 28 19:21:34 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 29 Sep 2021 09:21:34 +1000 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On Wed, Sep 29, 2021 at 9:10 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > > On 2021-09-29 at 11:38:22 +1300, > dn via Python-list wrote: > > > For those of us who remember/can compute in binary, octal, hex, or > > decimal as-needed: > > Why do programmers confuse All Hallows'/Halloween for Christmas Day? > > That one is also very old. (Yes, I know the answer. No, I will not > spoil it for those who might not.) What do I have to do to gain the > insight necessary to have discovered that question and answer on my own? You'd have to be highly familiar with numbers in different notations, to the extent that you automatically read 65 and 0x41 as the same number. Or, even better, to be able to read off a hex dump and see E8 03 and instantly read it as "1,000 little-endian". ChrisA From greg.ewing at canterbury.ac.nz Tue Sep 28 19:21:22 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Sep 2021 12:21:22 +1300 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On 29/09/21 4:37 am, Michael F. Stemper wrote: > I'm talking about something made > from tons of iron and copper that is oil-filled and rotates at 1800 rpm. To avoid confusion, we should rename them "electricity comprehensions". -- Greg From PythonList at DancesWithMice.info Tue Sep 28 19:49:05 2021 From: PythonList at DancesWithMice.info (dn) Date: Wed, 29 Sep 2021 12:49:05 +1300 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: <24e888de-dd3c-9a48-5e51-5f70b5ff88f7@DancesWithMice.info> On 29/09/2021 06.53, Michael F. Stemper wrote: > On 28/09/2021 10.53, Stefan Ram wrote: >> "Michael F. Stemper" writes: >>> Well, I could continue to hard-code the data into one of the test >>> programs >> >> ?? One can employ a gradual path from a program with hardcoded >> ?? data to an entity sharable by different programs. >> >> ?? When I am hurried to rush to a working program, I often >> ?? end up with code that contains configuration data spread >> ?? (interspersed) all over the code. For example: > >> ?? 1st step: give a name to all the config data: > >> ?? 2nd: move all config data to the top of the source code, >> ?? directly after all the import statements: > >> ?? 3rd: move all config data to a separate "config.py" module: >> >> import ... >> import config >> ... >> >> ... >> open( config.project_directory + "data.txt" ) >> ... >> >>> but that would mean that every time that I wanted to look >>> at a different scenario, I'd need to modify a program. >> >> ?? Now you just have to modify "config.py" - clearly separated >> ?? from the (rest of the) "program". > > Well, that doesn't really address what format to store the data > in. I was going to write a module that would read data from an > XML file: > > import EDXML > gens = EDXML.GeneratorsFromXML( "gendata1.xml" ) > fuels = EDXML.FuelsFromXML( "fueldata3.xml" ) > > (Of course, I'd really get the file names from command-line arguments.) > > Then I read a web page that suggested use of XML was a poor idea, > so I posted here asking for a clarification and alternate suggestions. > > One suggestion was that I use YAML, in which case, I'd write: > > import EDfromYAML > gens = EDfromYAML( "gendata1.yaml" ) > fuels = EDXML.FuelsFromYAML( "fueldata3.yaml" ) > >>> And when I discover anomalous behavior, I'd need to copy the >>> hard-coded data into another program. >> >> ?? Now you just have to import "config.py" from the other program. > > This sounds like a suggestion that I hard-code the data into a > module. I suppose that I could have half-a-dozen modules with > different data sets and ln them as required: > > $ rm GenData.py* FuelData.py* > $ ln gendata1.py GenData.py > $ ln fueldata3.py FuelData.py > > It seems to me that a more thorough separation of code and data > might be useful. Dear Michael, May I suggest that you are right - and that he is right! (which is a polite way of saying, also, that both are wrong. Oops!) (with any and all due apologies) There are likely cross-purposes here. I am interpreting various clues, from throughout the thread (from when the snowflakes were still falling!) that you and I were trained way-back: to first consider the problem, state the requirements ("hypothesis" in Scientific Method), and work our way to a solution on-paper. Only when we had a complete 'working solution', did we step up to the machine (quite possibly a Card Punch, cf a 'computer') and implement. Also, that we thought in terms of a clear distinction between "program[me]" and "data" - and the compiler and link[age]-editor software technology of the time maintained such. Whereas 'today', many follow the sequence of "Test-Driven Development" (er, um, often omitting the initial test) of attempting some idea as code, reviewing the result, and then "re-factoring" (improving), in a circular progression - until it not only works, but works well. This requires similar "stepwise decomposition" to what we learned, but differs when it comes to code-composition. This approach is more likely to accumulate a solution 'bottom-up' and component-wise, rather than creating an entire (and close-to-perfect) solution first and as an whole. Let's consider the Python REPL. Opening a terminal and starting the Python interpreter, gives us the opportunity to write short "snippets" of code and see the results immediately. This is VERY handy for ensuring that an idea is correct, or to learn exactly how a particular construct works. Thus, we can 'test' before we write any actual code (and can copy-paste the successful 'prototype' into our IDE/editor!). We didn't enjoy such luxury back in the good?bad old days. Young people today - they just don't know how lucky they are! (cue other 'grumpy old man' mutterings) Other points to consider: 'terminals' (cf mainframes), interpreted languages, and 'immediacy'. These have all brought "opportunities" and thus "change" to the way developers (can) work and think! (which is why I outlined what I think of as 'our training' and thus 'our thinking process' when it comes to software design, above) Another 'tectonic shift' is that in the old days 'computer time' was hugely expensive and thus had to be optimised. Whereas these days (even in retirement) programming-time has become the more expensive component as computers (or compute-time in cloud-speak) have become cheaper - and thus we reveal one of THE major attractive attributes of the Python programming language! Accordingly, (and now any apologies-due may be due to our colleague - who was amplifying/making a similar comment to my earlier contribution): if we decompose the wider-problem into (only) the aspects of collecting the data, we can assume/estimate/document/refer to that, as a Python function: def fetch_operating_parameters(): """Docstring!""" pass (yes, under TDD we would first write a test to call the function and test its results, but for brevity (hah!) I'll ignore that and stick with the dev.philosophy point) Decomposing further, we decide there's a need to pull-in characteristics of generators, fuel, etc. So, then we can similarly expect to need, and thus declare, a bunch more functions - with the expectation that they will probably be called from 'fetch_operating_parameters()'. (because that was our decomposition hierarchy) Now, let's return to the program[me] cf data contention. This can also be slapped-together 'now', and refined/improved 'later'. So, our first 'sub' input function could be: def fetch_generator_parameters() -> tuple[ dict ]: """Another docstring.""" skunk_creek_1 = { "IHRcurve_name" : "normal", "63" : "8.513", "105" : "8.907", etc } ... return skunk_creek_1, ... Accordingly, if we 'rinse-and-repeat' for each type of input parameter and flesh-out the coding of the overall input-construct (fetch_operating_parameters() ) we will be able to at least start meaningful work on the ensuing "process" and "output" decompositions of the whole. (indeed, reverting to the Input-Process-Output overview, if you prefer to stick with the way we were taught, there's no issue with starting at 'the far end' by writing an output routine and feeding it 'expected results' as arguments (which you have first calculated on-paper) to ensure it works, and continuing to work 'backwards' through 'Process' to 'Input'. Whatever 'works' for you!) Note that this is a Python-code solution to the original post about 'getting data in there'. It is undeniably 'quick-and-dirty', but it is working, and working 'now'! Secondly, because the total-system only 'sees' a function, you may come back 'later' and improve the code-within, eg by implementing a JSON-file interface, one for XML, one for YAML, or whatever your heart-desires - and that you can have the entire system up-and-running before you get to the stage of 'how can I make this [quick-and-dirty code] better?'. (with an alternate possible-conclusion) Here's where "skill" starts to 'count'. If sufficient forethought went into constructing the (sub-)function's "signature", changing the code within the function will not result in any 'ripple' of consequent-changes throughout the entire system! Thus, as long as 'whatever you decide to do' (initially, and during any improvements/changes) returns a tuple of dict-s (my example only), you can keep (learning, experimenting, and) improving the function without other-cost! (further reading: the Single Responsibility Principle) So, compared with our mouldy-old (initial) training, today's approach seems bumbling and to waste time on producing a first-attempt which (must) then require time to be improved (and watching folk work, I regularly have to 'bite my tongue' rather than say something that might generate philosophical conflict). However, when combined with TDD, whereby each sub-component is known to be working before it is incorporated into any larger component of the (and eventually the whole) solution, we actually find a practical and workable, alternate-approach to the business of coding! Yes, things are not as cut-and-dried as the attempted-description(s) here. It certainly pays to sit down and think about the problem first - but 'they' don't keep drilling down to 'our' level of detail, before making (some, unit) implementation. Indeed, as this thread shows, as long as we have an idea of the inputs required by Process, we don't need to detail the processes, we can attack the sub-problem of Input quite separately. Yes, it is a good idea to enact a 'design' step at each level of decomposition (rushing past which is too frequently a problem exhibited - at least by some of my colleagues). Having (some) working-code also enables learning - and in this case (but not at all), is a side-benefit. Once some 'learning' or implementation has been achieved, you may well feel it appropriate to improve the code - even to trial some other 'new' technique. At which point, another relevance arises (or should!): do I do it now, or do I make a ToDo note to come back to it later? (see also "Technical Debt", but please consider that the fleshing-out the rest of the solution (and 'learnings' from those steps) may (eventually) realise just as many, or even more, of the benefits of 'our approach' of producing a cohesive overall-design first! Possibly even more than the benefits we intended in 'our' approach(?). Unfortunately, it is a difficult adjustment to make (as related), and there are undoubtedly stories of how the 'fools rush in where angels fear to tread' approach is but a road to disaster and waste. The 'trick' is to "cherry pick" from today's affordances and modify our training/habits and experience to take the best advantages from both... Hope this helps to explain why you may have misunderstood some contributions 'here', or felt like arguing-back. Taking a step back or a 'wider' view, as has been attempted here, may show the implicit and intended value of (many) contributions. I'll leave you with a quote from Donald Knuth in The Art of Computer Programming, Volume 1: Fundamental Algorithms (which IIRC was first published in the late-60s/early-70s): ?Premature optimization is the root of all evil.? So, maybe early-coding/prototyping and later "optimisation" isn't all bad! -- Regards, =dn From greg.ewing at canterbury.ac.nz Tue Sep 28 19:53:17 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Sep 2021 12:53:17 +1300 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On 29/09/21 12:21 pm, Chris Angelico wrote: > to the extent that you automatically read 65 and 0x41 as the same > number. Am I too geeky for reading both of them as 'A'? -- Greg From rosuav at gmail.com Tue Sep 28 20:42:55 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 29 Sep 2021 10:42:55 +1000 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On Wed, Sep 29, 2021 at 10:06 AM Greg Ewing wrote: > > On 29/09/21 12:21 pm, Chris Angelico wrote: > > to the extent that you automatically read 65 and 0x41 as the same > > number. > > Am I too geeky for reading both of them as 'A'? > Not even slightly, and I did deliberately choose a printable ASCII value for that parallel. If E8 03 can be 1000, then 41 68 20 79 65 73 can be "Ah yes". ChrisA From PythonList at DancesWithMice.info Tue Sep 28 21:58:09 2021 From: PythonList at DancesWithMice.info (dn) Date: Wed, 29 Sep 2021 14:58:09 +1300 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: <74575601-0d6f-5678-e89f-705bef3d1e16@DancesWithMice.info> On 29/09/2021 12.21, Chris Angelico wrote: > On Wed, Sep 29, 2021 at 9:10 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: >> >> On 2021-09-29 at 11:38:22 +1300, >> dn via Python-list wrote: >> >>> For those of us who remember/can compute in binary, octal, hex, or >>> decimal as-needed: >>> Why do programmers confuse All Hallows'/Halloween for Christmas Day? >> >> That one is also very old. (Yes, I know the answer. No, I will not >> spoil it for those who might not.) What do I have to do to gain the >> insight necessary to have discovered that question and answer on my own? > > You'd have to be highly familiar with numbers in different notations, > to the extent that you automatically read 65 and 0x41 as the same > number. Or, even better, to be able to read off a hex dump and see E8 > 03 and instantly read it as "1,000 little-endian". Now, now, young-Chris, make way for those of us who are qualified to carry walking-sticks! ...or see "C1" and recognise that also to represent an "A"? To continue, to the bit-ter end: Sometimes I think you are a bit too much - said the 0 to the 1. If I hold up two fingers, am I insulting you, or asking for three of something? Programming in Java is the process of converting source code into core dumps. and, the Dijkstra quotation: "If debugging is the process of removing bugs, then programming must be the process of putting them in" -- Regards, =dn From rosuav at gmail.com Tue Sep 28 22:01:40 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 29 Sep 2021 12:01:40 +1000 Subject: OT: AttributeError In-Reply-To: <74575601-0d6f-5678-e89f-705bef3d1e16@DancesWithMice.info> References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> <74575601-0d6f-5678-e89f-705bef3d1e16@DancesWithMice.info> Message-ID: On Wed, Sep 29, 2021 at 11:59 AM dn via Python-list wrote: > If I hold up two fingers, am I insulting you, or asking for three of > something? > A Roman soldier walked into a bar holding up two fingers. "Five beers, please" ChrisA From 2QdxY4RzWzUUiLuE at potatochowder.com Tue Sep 28 22:03:47 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Tue, 28 Sep 2021 19:03:47 -0700 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On 2021-09-29 at 09:21:34 +1000, Chris Angelico wrote: > On Wed, Sep 29, 2021 at 9:10 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > > > > On 2021-09-29 at 11:38:22 +1300, > > dn via Python-list wrote: > > > > > For those of us who remember/can compute in binary, octal, hex, or > > > decimal as-needed: > > > Why do programmers confuse All Hallows'/Halloween for Christmas Day? > > > > That one is also very old. (Yes, I know the answer. No, I will not > > spoil it for those who might not.) What do I have to do to gain the > > insight necessary to have discovered that question and answer on my own? > > You'd have to be highly familiar with numbers in different notations, > to the extent that you automatically read 65 and 0x41 as the same > number ... I do that. And I have done that, with numbers that size, since the late 1970s (maybe the mid 1970s, for narrow definitions of "different"). There's at least one more [sideways, twisted] leap to the point that you even think of translating the names of those holidays into an arithmetic riddle. > ... Or, even better, to be able to read off a hex dump and see E8 03 > and instantly read it as "1,000 little-endian". 59535 big endian. Warningm flamebait ahead: Who thinks in little endian? (I was raised on 6502s and 680XX CPUs; 8080s and Z80s always did things backwards.) From 2QdxY4RzWzUUiLuE at potatochowder.com Tue Sep 28 22:25:05 2021 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Tue, 28 Sep 2021 19:25:05 -0700 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On 2021-09-29 at 09:21:34 +1000, Chris Angelico wrote: > ... read off a hex dump and see E8 > 03 and instantly read it as "1,000 little-endian". ITYM 000,1 little-endian. ;-) (Or possibly 000.1, depending on your locale.) From rosuav at gmail.com Tue Sep 28 22:48:19 2021 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 29 Sep 2021 12:48:19 +1000 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On Wed, Sep 29, 2021 at 12:06 PM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > > ... Or, even better, to be able to read off a hex dump and see E8 03 > > and instantly read it as "1,000 little-endian". > > 59535 big endian. Warningm flamebait ahead: Who thinks in little > endian? (I was raised on 6502s and 680XX CPUs; 8080s and Z80s always > did things backwards.) I do, because most of the file formats I've had to delve into have been little-endian. It seems to be far more common to have undocumented file formats created by Windows programs than, say, undocumented network packet formats (which would use network byte order, naturally). ChrisA From arj.python at gmail.com Wed Sep 29 01:02:58 2021 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 29 Sep 2021 09:02:58 +0400 Subject: FlaskCon's CFP is now open till Oct 24 Message-ID: Greetings folks, FlaskCon is around again this year, the very latest conf of the year. Bouncing in the first week of December, 1 2 3 4. CFP is open, submit -> https://flaskcon.com/ Any queries, let them fly to flaskcon at gmail.com Kind Regards, Abdur-Rahmaan Janhangeer about | blog github Mauritius From shishaozhong at gmail.com Wed Sep 29 08:49:39 2021 From: shishaozhong at gmail.com (Shaozhong SHI) Date: Wed, 29 Sep 2021 13:49:39 +0100 Subject: Automated data testing, checking, validation, reporting for data assurance Message-ID: There appear to be a few options for this. Has anyone tested and got experience with automated data testing, validation and reporting? Can anyone enlighten me? Regards, David From greg.ewing at canterbury.ac.nz Wed Sep 29 02:16:17 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Sep 2021 19:16:17 +1300 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On 29/09/21 3:03 pm, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > Who thinks in little > endian? (I was raised on 6502s and 680XX CPUs; 8080s and Z80s always > did things backwards.) The first CPU I wrote code for was a National SC/MP, which doesn't have an endianness, since it never deals with more than a byte at a time. The second was a 6800, which is big-endian. That's definitely more convenient when you're hand-assembling code! I can see the advantages of little-endian when you're implementing a CPU, though. -- Greg From hongyi.zhao at gmail.com Wed Sep 29 04:22:03 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 29 Sep 2021 01:22:03 -0700 (PDT) Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' Message-ID: I tried to convert a xls file into csv with the following command, but failed: $ in2csv --sheet 'Sheet1' 2021-2022-1.xls XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' The above testing file is located at here [1]. [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls Any hints for fixing this problem? Regards, HZ From user at example.net Wed Sep 29 05:40:38 2021 From: user at example.net (J.O. Aho) Date: Wed, 29 Sep 2021 11:40:38 +0200 Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: On 29/09/2021 10.22, hongy... at gmail.com wrote: > I tried to convert a xls file into csv with the following command, but failed: > > $ in2csv --sheet 'Sheet1' 2021-2022-1.xls > XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' > > The above testing file is located at here [1]. > > [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls > > Any hints for fixing this problem? You need to delete the 13 first lines in the file or you see to that your code does first trim the data before start xml parse it. -- //Aho From hongyi.zhao at gmail.com Wed Sep 29 07:10:02 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 29 Sep 2021 04:10:02 -0700 (PDT) Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: On Wednesday, September 29, 2021 at 5:40:58 PM UTC+8, J.O. Aho wrote: > On 29/09/2021 10.22, hongy... at gmail.com wrote: > > I tried to convert a xls file into csv with the following command, but failed: > > > > $ in2csv --sheet 'Sheet1' 2021-2022-1.xls > > XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' > > > > The above testing file is located at here [1]. > > > > [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls > > > > Any hints for fixing this problem? > You need to delete the 13 first lines in the file Yes. After deleting the top 3 lines, the problem has been fixed. > or you see to that your code does first trim the data before start xml parse it. Yes. I really want to do this trick programmatically, but how do I do it without manually editing the file? HZ From user at example.net Wed Sep 29 08:11:49 2021 From: user at example.net (J.O. Aho) Date: Wed, 29 Sep 2021 14:11:49 +0200 Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: On 29/09/2021 13.10, hongy... at gmail.com wrote: > On Wednesday, September 29, 2021 at 5:40:58 PM UTC+8, J.O. Aho wrote: >> On 29/09/2021 10.22, hongy... at gmail.com wrote: >>> I tried to convert a xls file into csv with the following command, but failed: >>> >>> $ in2csv --sheet 'Sheet1' 2021-2022-1.xls >>> XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' >>> >>> The above testing file is located at here [1]. >>> >>> [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls >>> >>> Any hints for fixing this problem? >> You need to delete the 13 first lines in the file > > Yes. After deleting the top 3 lines, the problem has been fixed. > >> or you see to that your code does first trim the data before start xml parse it. > > Yes. I really want to do this trick programmatically, but how do I do it without manually editing the file? You could do something like loading the XML into a string (myxmlstr) and then find the fist < in that string xmlstart = myxmlstr.find('<') xmlstr = myxmlstr[xmlstart:] then use the xmlstr in the xml parser, sure not as convenient as loading the file directly to the xml parser. I don't say this is the best way of doing it, I'm sure some python wiz here would have a smarter solution. -- //Aho From michael.stemper at gmail.com Wed Sep 29 09:04:27 2021 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Wed, 29 Sep 2021 08:04:27 -0500 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: On 28/09/2021 18.21, Greg Ewing wrote: > On 29/09/21 4:37 am, Michael F. Stemper wrote: >> I'm talking about something made >> from tons of iron and copper that is oil-filled and rotates at 1800 rpm. > > To avoid confusion, we should rename them "electricity comprehensions". Hah! -- Michael F. Stemper If you take cranberries and stew them like applesauce they taste much more like prunes than rhubarb does. From hongyi.zhao at gmail.com Wed Sep 29 09:22:27 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 29 Sep 2021 06:22:27 -0700 (PDT) Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: <25a560e8-5fa7-4302-a610-bb0a88bb7102n@googlegroups.com> On Wednesday, September 29, 2021 at 8:12:08 PM UTC+8, J.O. Aho wrote: > On 29/09/2021 13.10, hongy... at gmail.com wrote: > > On Wednesday, September 29, 2021 at 5:40:58 PM UTC+8, J.O. Aho wrote: > >> On 29/09/2021 10.22, hongy... at gmail.com wrote: > >>> I tried to convert a xls file into csv with the following command, but failed: > >>> > >>> $ in2csv --sheet 'Sheet1' 2021-2022-1.xls > >>> XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' > >>> > >>> The above testing file is located at here [1]. > >>> > >>> [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls > >>> > >>> Any hints for fixing this problem? > >> You need to delete the 13 first lines in the file > > > > Yes. After deleting the top 3 lines, the problem has been fixed. > > > >> or you see to that your code does first trim the data before start xml parse it. > > > > Yes. I really want to do this trick programmatically, but how do I do it without manually editing the file? > You could do something like loading the XML into a string (myxmlstr) How to do this operation? As you have seen, the file refused to be loaded at all. > and then find the fist < in that string > > xmlstart = myxmlstr.find('<') > > xmlstr = myxmlstr[xmlstart:] > > then use the xmlstr in the xml parser, sure not as convenient as loading > the file directly to the xml parser. > > I don't say this is the best way of doing it, I'm sure some python wiz > here would have a smarter solution. Another very strange thing: I trimmed the first 3 lines in the original file and saved it into a new one named as 2021-2022-1-trimmed-top-3-lines.xls. [1] Then I read the file with the following python script named as pandas-excel.py: ------ import pandas as pd excel_file='2021-2022-1-trimmed-top-3-lines.xls' #print(pd.ExcelFile(excel_file).sheet_names) newpd=pd.read_excel(excel_file, sheet_name='Sheet1') for i in newpd.index: if i >1: for j in newpd.columns: if int(j.split()[1]) > 2: if not pd.isnull(newpd.loc[i][j]): print(newpd.loc[i][j]) ------ $ python pandas-excel.py | sort -u ?????? [1-8]? 1-4? 38 ???413???????II ??1932? ?????????????? [1-12]? 1-4? 38 ???416?????????? ??1932? OTOH, I also tried to read the file with in2csv as follows: $ in2csv --sheet Sheet1 2021-2022-1-trimmed-top-3-lines.xls 2>/dev/null |tr ',' '\n' | \ sed -re '/^$/d' | sort -u | awk '{print length($0),$0}' | sort -k1n | tail -3 | cut -d ' ' -f2- ?????? [1-8]? 1-4? 38 ???413???????II ??1932? ???????? [1-8]? 6-9? 45 ???511????????? ??1931? ?????????????? [1-12]? 1-4? 38 ???416?????????? ??1932? As you can see, the above two methods give different results. I'm very puzzled by this phenomenon. Any hints/tips/comments will be greatly appreciated. [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1-trimmed-top-3-lines.xls Regards, HZ From jorge.conforte at inpe.br Wed Sep 29 12:16:26 2021 From: jorge.conforte at inpe.br (Jorge Conforte) Date: Wed, 29 Sep 2021 13:16:26 -0300 Subject: NUmpy Message-ID: <916308b1-f3ce-3c53-fab5-a421c3f8e531@inpe.br> Hi, I have a netcdf file "uwnd_850_1981.nc" and I'm using the commands to read it: from numpy import dtype ?fileu ='uwnd_850_1981.nc' ncu = Dataset(fileu,'r') uwnd=ncu.variables['uwnd'][:] and I had: :1: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations I didn't how I have this message. My numpy verison is 1.21.2. Please, how can I solve this. Thanks, Conrado From python at mrabarnett.plus.com Wed Sep 29 12:24:59 2021 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 29 Sep 2021 17:24:59 +0100 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: On 2021-09-29 03:03, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-29 at 09:21:34 +1000, > Chris Angelico wrote: > >> On Wed, Sep 29, 2021 at 9:10 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: >> > >> > On 2021-09-29 at 11:38:22 +1300, >> > dn via Python-list wrote: >> > >> > > For those of us who remember/can compute in binary, octal, hex, or >> > > decimal as-needed: >> > > Why do programmers confuse All Hallows'/Halloween for Christmas Day? >> > >> > That one is also very old. (Yes, I know the answer. No, I will not >> > spoil it for those who might not.) What do I have to do to gain the >> > insight necessary to have discovered that question and answer on my own? >> >> You'd have to be highly familiar with numbers in different notations, >> to the extent that you automatically read 65 and 0x41 as the same >> number ... > > I do that. And I have done that, with numbers that size, since the late > 1970s (maybe the mid 1970s, for narrow definitions of "different"). > > There's at least one more [sideways, twisted] leap to the point that you > even think of translating the names of those holidays into an arithmetic > riddle. > >> ... Or, even better, to be able to read off a hex dump and see E8 03 >> and instantly read it as "1,000 little-endian". > > 59535 big endian. Warningm flamebait ahead: Who thinks in little > endian? (I was raised on 6502s and 680XX CPUs; 8080s and Z80s always > did things backwards.) > 6502 is little-endian. From info at wingware.com Wed Sep 29 12:32:23 2021 From: info at wingware.com (Wingware) Date: Wed, 29 Sep 2021 12:32:23 -0400 Subject: ANN: Wing Python IDE 8.0.4 has been released Message-ID: Wing 8.0.4 adds Close Unmodified Others to the editor tab's context menu, documents using sitecustomize to automatically start debug, fixes the debugger on some Windows systems, improves icon rendering with some Windows scaling factors, and makes several other improvements. Details:????????? https://wingware.com/news/2021-09-28 Downloads:?? https://wingware.com/downloads == About Wing == Wing is a light-weight but full-featured Python IDE designed specifically for Python, with powerful editing, code inspection, testing, and debugging capabilities. Wing's deep code analysis provides auto-completion, auto-editing, and refactoring that speed up development. Its top notch debugger works with any Python code, locally or on a remote host, container, or cluster. Wing also supports test-driven development, version control, UI color and layout customization, and includes extensive documentation and support. Wing is available in three product levels:? Wing Pro is the full-featured Python IDE for professional developers, Wing Personal is a free Python IDE for students and hobbyists (omits some features), and Wing 101 is a very simplified free Python IDE for beginners (omits many features). Learn more at https://wingware.com/ From auriocus at gmx.de Wed Sep 29 13:27:32 2021 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 29 Sep 2021 19:27:32 +0200 Subject: NUmpy In-Reply-To: References: <916308b1-f3ce-3c53-fab5-a421c3f8e531@inpe.br> Message-ID: Am 29.09.21 um 18:16 schrieb Jorge Conforte: > > > Hi, > > I have a netcdf file "uwnd_850_1981.nc" and I'm using the commands to > read it: Your code is incomplete: > from numpy import dtype > ?fileu ='uwnd_850_1981.nc' > ncu = Dataset(fileu,'r') Where is "Dataset" defined? > uwnd=ncu.variables['uwnd'][:] > > > and I had: > > :1: DeprecationWarning: `np.bool` is a deprecated alias for the > builtin `bool`. To silence this warning, use `bool` by itself. Doing > this will not modify any behavior and is safe. If you specifically > wanted the numpy scalar type, use `np.bool_` here. > Deprecated in NumPy 1.20; for more details and guidance: > https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations > > I didn't how I have this message. My numpy verison is 1.21.2. > > > Please, how can I solve this. First, it is only a warning, therefore it should still work. Second, the problem is not in the code that you posted. POssibly in the definition of "Dataset". Maybe the netCDF-File contains boolean values and the package you use to read it should be updated? Christian From PythonList at DancesWithMice.info Wed Sep 29 14:28:15 2021 From: PythonList at DancesWithMice.info (dn) Date: Thu, 30 Sep 2021 07:28:15 +1300 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: <8c96cf28-f0a2-bcd0-78d9-9ab42d1f7e46@DancesWithMice.info> On 29/09/2021 19.16, Greg Ewing wrote: > On 29/09/21 3:03 pm, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: >> Who thinks in little >> endian?? (I was raised on 6502s and 680XX CPUs; 8080s and Z80s always >> did things backwards.) > > The first CPU I wrote code for was a National SC/MP, which doesn't > have an endianness, since it never deals with more than a byte at > a time. The second was a 6800, which is big-endian. That's definitely > more convenient when you're hand-assembling code! I can see the > advantages of little-endian when you're implementing a CPU, though. Oh yes! The D2 kit - I kept those books for years... https://www.electronixandmore.com/adam/temp/6800trainer/mek6800d2.html -- Regards, =dn From hjp-python at hjp.at Wed Sep 29 17:19:38 2021 From: hjp-python at hjp.at (Peter J. Holzer) Date: Wed, 29 Sep 2021 23:19:38 +0200 Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: On 2021-09-29 01:22:03 -0700, hongy... at gmail.com wrote: > I tried to convert a xls file into csv with the following command, but failed: > > $ in2csv --sheet 'Sheet1' 2021-2022-1.xls > XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' > > The above testing file is located at here [1]. > > [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls Why is that file name .xls when it's obviously an HTML file? hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From avigross at verizon.net Wed Sep 29 20:29:28 2021 From: avigross at verizon.net (Avi Gross) Date: Wed, 29 Sep 2021 20:29:28 -0400 Subject: XML Considered Harmful In-Reply-To: References: <00a401d7b255$d27d72c0$77785840$@verizon.net> <016501d7b404$5006a2c0$f013e840$@verizon.net> Message-ID: <021101d7b592$3ac439b0$b04cad10$@verizon.net> I think that to make electricity comprehend, you need a room temperature superconductor. The Cooper Pairs took a while to comprehend but now ... I think, seriously, we have established the problems with guessing that others are using the language in a way we assume. So how many comprehensions does Python have? [] - list comprehension {} - dictionary OR set comprehension () - generator expression Tuples are incomprehensible and I wonder if any other comprehensions might make sense to add, albeit we may need new symbols. -----Original Message----- From: Python-list On Behalf Of Michael F. Stemper Sent: Wednesday, September 29, 2021 9:04 AM To: python-list at python.org Subject: Re: XML Considered Harmful On 28/09/2021 18.21, Greg Ewing wrote: > On 29/09/21 4:37 am, Michael F. Stemper wrote: >> I'm talking about something made >> from tons of iron and copper that is oil-filled and rotates at 1800 rpm. > > To avoid confusion, we should rename them "electricity comprehensions". Hah! -- Michael F. Stemper If you take cranberries and stew them like applesauce they taste much more like prunes than rhubarb does. -- https://mail.python.org/mailman/listinfo/python-list From rob.cliffe at btinternet.com Wed Sep 29 20:04:05 2021 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 30 Sep 2021 01:04:05 +0100 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> Message-ID: <314f1b4e-caf2-98cf-0fb7-76904616796e@btinternet.com> Ah, Z80s (deep sigh).? Those were the days!? You could disassemble the entire CP/M operating system (including the BIOS), and still have many Kb to play with!? Real programmers don't need gigabytes! On 29/09/2021 03:03, 2QdxY4RzWzUUiLuE at potatochowder.com wrote: > On 2021-09-29 at 09:21:34 +1000, > Chris Angelico wrote: > >> On Wed, Sep 29, 2021 at 9:10 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: >>> On 2021-09-29 at 11:38:22 +1300, >>> dn via Python-list wrote: >>> >>>> For those of us who remember/can compute in binary, octal, hex, or >>>> decimal as-needed: >>>> Why do programmers confuse All Hallows'/Halloween for Christmas Day? >>> That one is also very old. (Yes, I know the answer. No, I will not >>> spoil it for those who might not.) What do I have to do to gain the >>> insight necessary to have discovered that question and answer on my own? >> You'd have to be highly familiar with numbers in different notations, >> to the extent that you automatically read 65 and 0x41 as the same >> number ... > I do that. And I have done that, with numbers that size, since the late > 1970s (maybe the mid 1970s, for narrow definitions of "different"). > > There's at least one more [sideways, twisted] leap to the point that you > even think of translating the names of those holidays into an arithmetic > riddle. > >> ... Or, even better, to be able to read off a hex dump and see E8 03 >> and instantly read it as "1,000 little-endian". > 59535 big endian. Warningm flamebait ahead: Who thinks in little > endian? (I was raised on 6502s and 680XX CPUs; 8080s and Z80s always > did things backwards.) From greg.ewing at canterbury.ac.nz Wed Sep 29 20:44:28 2021 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Sep 2021 13:44:28 +1300 Subject: OT: AttributeError In-Reply-To: References: <22e02e08-c091-ede1-d115-31310546da15@DancesWithMice.info> <8c96cf28-f0a2-bcd0-78d9-9ab42d1f7e46@DancesWithMice.info> Message-ID: On 30/09/21 7:28 am, dn wrote: > Oh yes! The D2 kit - I kept those books for years... > https://www.electronixandmore.com/adam/temp/6800trainer/mek6800d2.html My 6800 system was nowhere near as fancy as that! It was the result of replacing the CPU in my homebrew Miniscamp. -- Greg From hongyi.zhao at gmail.com Wed Sep 29 21:20:23 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 29 Sep 2021 18:20:23 -0700 (PDT) Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: On Thursday, September 30, 2021 at 5:20:04 AM UTC+8, Peter J. Holzer wrote: > On 2021-09-29 01:22:03 -0700, hongy... at gmail.com wrote: > > I tried to convert a xls file into csv with the following command, but failed: > > > > $ in2csv --sheet 'Sheet1' 2021-2022-1.xls > > XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' > > > > The above testing file is located at here [1]. > > > > [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls > Why is that file name .xls when it's obviously an HTML file? Good catch! Thank you for pointing this out. This file is automatically exported from my university's teaching management system, and it was assigned the .xls extension by default. HZ From shishaozhong at gmail.com Thu Sep 30 07:29:16 2021 From: shishaozhong at gmail.com (Shaozhong SHI) Date: Thu, 30 Sep 2021 12:29:16 +0100 Subject: Definitive guide for Regex Message-ID: Dear All, I am trying to look for a definitive guide for Regex in Python. Can anyone help? Regards, David From jitendrabeura001 at gmail.com Thu Sep 30 09:46:31 2021 From: jitendrabeura001 at gmail.com (jitendrabeura001) Date: Thu, 30 Sep 2021 19:16:31 +0530 Subject: python.exe - System Error Message-ID: Hello Sir/Madam, please help me to out from this difficulties, whenever I'm trying to install python idle it is showing me the error called**** "[1]python.exe - System Error"****** I have tried couple of ways to get rid out of this but failed every time, please please help me to find the exact solution for this problem. Thank You. References Visible links 1. http://python.exe/ From hongyi.zhao at gmail.com Wed Sep 29 23:53:44 2021 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 29 Sep 2021 20:53:44 -0700 (PDT) Subject: XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' In-Reply-To: References: Message-ID: On Thursday, September 30, 2021 at 9:20:37 AM UTC+8, hongy... at gmail.com wrote: > On Thursday, September 30, 2021 at 5:20:04 AM UTC+8, Peter J. Holzer wrote: > > On 2021-09-29 01:22:03 -0700, hongy... at gmail.com wrote: > > > I tried to convert a xls file into csv with the following command, but failed: > > > > > > $ in2csv --sheet 'Sheet1' 2021-2022-1.xls > > > XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\r\n\r\n\r\n\r\n' > > > > > > The above testing file is located at here [1]. > > > > > > [1] https://github.com/hongyi-zhao/temp/blob/master/2021-2022-1.xls > > Why is that file name .xls when it's obviously an HTML file? > Good catch! Thank you for pointing this out. This file is automatically exported from my university's teaching management system, and it was assigned the .xls extension by default. According to the above comment, after I change the extension to html, the following python code will do the trick: import sys import pandas as pd if len(sys.argv) != 2: print('Usage: ' + sys.argv[0] + ' input-file') exit(1) myhtml_pd = pd.read_html(sys.argv[1]) #In [25]: len(myhtml_pd) #Out[25]: 3 for i in myhtml_pd[2].index: if i > 0: for j in myhtml_pd[2].columns: if j >1 and not pd.isnull(myhtml_pd[2].loc[i][j]): print(myhtml_pd[2].loc[i][j]) HZ From anilanvesh at gmail.com Thu Sep 30 01:11:16 2021 From: anilanvesh at gmail.com (Anil Anvesh) Date: Wed, 29 Sep 2021 22:11:16 -0700 (PDT) Subject: How to pass a method as argument? Message-ID: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> I want to write a python calculator program that has different methods to add, subtract, multiply which takes 2 parameters. I need to have an execute method when passed with 3 parameters, should call respective method and perform the operation. How can I achieve that? class calc(): def __init__(self,a,b): self.a=a self.b=b def ex(self,fun): self.fun=fun if fun=="add": self.add() def add(self): return self.a+self.b def sub(self): return self.a-self.b def mul(self): return self.a*self.b def div (self): return self.a/self.b def execu( obj1=calc() obj1.execu("add",1,,2) From edmondo.giovannozzi at gmail.com Thu Sep 30 06:28:00 2021 From: edmondo.giovannozzi at gmail.com (Edmondo Giovannozzi) Date: Thu, 30 Sep 2021 03:28:00 -0700 (PDT) Subject: How to pass a method as argument? In-Reply-To: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> References: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> Message-ID: <32d7bbfe-3b66-4b08-880c-5b174df1b2dcn@googlegroups.com> Il giorno gioved? 30 settembre 2021 alle 07:11:28 UTC+2 anila... at gmail.com ha scritto: > I want to write a python calculator program that has different methods to add, subtract, multiply which takes 2 parameters. I need to have an execute method when passed with 3 parameters, should call respective method and perform the operation. How can I achieve that? > > > > class calc(): > def __init__(self,a,b): > self.a=a > self.b=b > > def ex(self,fun): > self.fun=fun > if fun=="add": > self.add() > > def add(self): > return self.a+self.b > def sub(self): > return self.a-self.b > def mul(self): > return self.a*self.b > def div (self): > return self.a/self.b > def execu( > obj1=calc() > obj1.execu("add",1,,2) For example: | def ex(self, fun): | getattr(self, fun)() From jitendrabeura001 at gmail.com Thu Sep 30 11:18:18 2021 From: jitendrabeura001 at gmail.com (jitendrabeura001) Date: Thu, 30 Sep 2021 20:48:18 +0530 Subject: How to pass a method as argument? References: <42fclh7limbdvlruheppt3vr.1633014988556@email.kingsoft.com> Message-ID: Please help me to out from this difficulties, whenever I'm trying to install python idle it is showing me the error called**** "[1]python.exe - System Error"****** I have tried couple of ways to get rid out of this but failed every time, please please help me to find the exact solution for this problem. Thank You Sent from OPPO Mail On Edmondo Giovannozzi , 30 Sep 2021 20:34 wrote: References Visible links 1. http://python.exe/ From barry at barrys-emacs.org Thu Sep 30 13:16:49 2021 From: barry at barrys-emacs.org (Barry Scott) Date: Thu, 30 Sep 2021 18:16:49 +0100 Subject: Definitive guide for Regex In-Reply-To: References: Message-ID: <7FE7CB5C-29E9-4C25-9C7A-9F1088A27959@barrys-emacs.org> > On 30 Sep 2021, at 12:29, Shaozhong SHI wrote: > > Dear All, > > I am trying to look for a definitive guide for Regex in Python. > Can anyone help? Have you read the python docs for the re module? Barry > > Regards, > > David > -- > https://mail.python.org/mailman/listinfo/python-list > From jitendrabeura001 at gmail.com Thu Sep 30 14:09:35 2021 From: jitendrabeura001 at gmail.com (jitendrabeura001) Date: Thu, 30 Sep 2021 23:39:35 +0530 Subject: python.exe - System Error Message-ID: please guide me to out from this difficulties, whenever I'm trying to install python idle it is showing me the error called**** [1]python.exe - System Error I have tried couple of ways to get rid out of this but failed every time, please please help me to find the exact solution for this problem. Thank You References Visible links 1. http://python.exe/ From PythonList at DancesWithMice.info Thu Sep 30 14:35:44 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 1 Oct 2021 07:35:44 +1300 Subject: Definitive guide for Regex In-Reply-To: <7FE7CB5C-29E9-4C25-9C7A-9F1088A27959@barrys-emacs.org> References: <7FE7CB5C-29E9-4C25-9C7A-9F1088A27959@barrys-emacs.org> Message-ID: <3cb68ea8-78cc-8e67-7432-4a2a1c092b17@DancesWithMice.info> On 01/10/2021 06.16, Barry Scott wrote: > > >> On 30 Sep 2021, at 12:29, Shaozhong SHI wrote: >> >> Dear All, >> >> I am trying to look for a definitive guide for Regex in Python. >> Can anyone help? > > Have you read the python docs for the re module? I learned from Jeffrey Friedl's book "Mastering Regular Expressions", but that was in a land far away, last century, and under a different language (and the original version - I see it's now up to its third edition). Despite their concise exercise of power (and the fact that in my Python-life I've never been put into a corner where I absolutely must use one), I'm no longer a fan... -- Regards, =dn From barry at barrys-emacs.org Thu Sep 30 16:59:08 2021 From: barry at barrys-emacs.org (Barry Scott) Date: Thu, 30 Sep 2021 21:59:08 +0100 Subject: Definitive guide for Regex In-Reply-To: <3cb68ea8-78cc-8e67-7432-4a2a1c092b17@DancesWithMice.info> References: <7FE7CB5C-29E9-4C25-9C7A-9F1088A27959@barrys-emacs.org> <3cb68ea8-78cc-8e67-7432-4a2a1c092b17@DancesWithMice.info> Message-ID: <11163B70-C9EA-49DA-9235-F44AD64F23CB@barrys-emacs.org> > On 30 Sep 2021, at 19:35, dn via Python-list wrote: > > On 01/10/2021 06.16, Barry Scott wrote: >> >> >>> On 30 Sep 2021, at 12:29, Shaozhong SHI wrote: >>> >>> Dear All, >>> >>> I am trying to look for a definitive guide for Regex in Python. >>> Can anyone help? >> >> Have you read the python docs for the re module? > > > I learned from Jeffrey Friedl's book "Mastering Regular Expressions", > but that was in a land far away, last century, and under a different > language (and the original version - I see it's now up to its third > edition). > > Despite their concise exercise of power (and the fact that in my > Python-life I've never been put into a corner where I absolutely must > use one), I'm no longer a fan... Agreed, regex is the last tool I reach for in python code. I find I use split() a lot to break up strings for processing. But there are cases where a regex is the best tool for a particular job and I then use the re module. But it costs in maintainability. I speak as the author of a regex engine and know how to write scary regex's when the need arises. Barry > -- > Regards, > =dn > -- > https://mail.python.org/mailman/listinfo/python-list > From avigross at verizon.net Thu Sep 30 17:11:21 2021 From: avigross at verizon.net (Avi Gross) Date: Thu, 30 Sep 2021 17:11:21 -0400 Subject: How to pass a method as argument? In-Reply-To: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> References: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> Message-ID: <025e01d7b63f$b8623e00$2926ba00$@verizon.net> Sounds like an excellent homework question. But your method of using an object is not what first comes to mind based on your cursory description. There is a python idiom using functional programming that looks like this: def doit(a, b, fun): return(fun(a,b)) So make up your own functions like: def addit(a,b): return(a+b) And make up others doing multiply and whatever and call something like this: doit(3,5, addit) Now if you want a user to put in text like "add" that is another story. If you want this in the context of an object, ditto. -----Original Message----- From: Python-list On Behalf Of Anil Anvesh Sent: Thursday, September 30, 2021 1:11 AM To: python-list at python.org Subject: How to pass a method as argument? I want to write a python calculator program that has different methods to add, subtract, multiply which takes 2 parameters. I need to have an execute method when passed with 3 parameters, should call respective method and perform the operation. How can I achieve that? class calc(): def __init__(self,a,b): self.a=a self.b=b def ex(self,fun): self.fun=fun if fun=="add": self.add() def add(self): return self.a+self.b def sub(self): return self.a-self.b def mul(self): return self.a*self.b def div (self): return self.a/self.b def execu( obj1=calc() obj1.execu("add",1,,2) -- https://mail.python.org/mailman/listinfo/python-list From PythonList at DancesWithMice.info Thu Sep 30 17:53:48 2021 From: PythonList at DancesWithMice.info (dn) Date: Fri, 1 Oct 2021 10:53:48 +1300 Subject: How to pass a method as argument? In-Reply-To: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> References: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> Message-ID: On 30/09/2021 18.11, Anil Anvesh wrote: > I want to write a python calculator program that has different methods to add, subtract, multiply which takes 2 parameters. I need to have an execute method when passed with 3 parameters, should call respective method and perform the operation. How can I achieve that? > > > > class calc(): > def __init__(self,a,b): > self.a=a > self.b=b > > def ex(self,fun): > self.fun=fun > if fun=="add": > self.add() > > def add(self): > return self.a+self.b > def sub(self): > return self.a-self.b > def mul(self): > return self.a*self.b > def div (self): > return self.a/self.b > def execu( > obj1=calc() > obj1.execu("add",1,,2) > This is a common course-assignment. NB the code-examples below are incomplete. You may need to research these techniques further, and thus learn how 'it' all fits-together (I set 'homework' rather than doing it!) There are several techniques which can be employed and/or combined here. 1 if-elif 'ladder' 2 dict[ionary] as select/case construct 3 functions as 'first-class objects' The basic problem is to take a 'command' which is formatted as a str[ing], and decide which of several options to choose: 1 if-elif is a simple and easy-to-read solution: if command == "add": do_this... elif command == "sub": do_that... ... else: # oops: "I'm afraid I can't do that, Dave" # don't forget to add a 'catch-all' to handle user-error! 2 using a dict is shorter and avoids some of the 'boiler-plate' repetition which causes many of us to rebel against using the above. We use the command received from the user as a key into the dict. 3 a function and/or its name is as much data as the 'variable' "command" or indeed the str-constant "add" - once it has been defined! Thus it is possible to treat functions like anything else: data_as_string = "my string" # can be likened to:- def my_function(): pass data_as_function = my_function Note that (like most else), the function must be defined before it can be 'used'! Returning to the stated-problem: def add( etc ): ... then using the if-elif 'ladder' above as a framework create a dict which 'links' the input command (as str) to the applicable function: calculator = { "add": add, "sub": sub, # indeed we can become rather more 'creative' "+" " add, ... } Thereafter, we can apply the dict to solve the problem: calculator.get( command, "Error message/advice, eg mentioning add, or + etc" ) NB it has been left to you to perfect the technique so that the value(s) to be calculated are properly communicated to the chosen function. PS you may find the Python-Tutor Discussion List helpful -- Regards, =dn From mats at wichmann.us Thu Sep 30 20:31:46 2021 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 30 Sep 2021 18:31:46 -0600 Subject: How to pass a method as argument? In-Reply-To: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> References: <3a55b010-6c93-49b5-8e27-baf9bdb8eb64n@googlegroups.com> Message-ID: <8d413a3e-6334-9519-f232-b1aae02cfee6@wichmann.us> On 9/29/21 23:11, Anil Anvesh wrote: > I want to write a python calculator program that has different methods to add, subtract, multiply which takes 2 parameters. I need to have an execute method when passed with 3 parameters, should call respective method and perform the operation. How can I achieve that? > let me add - this is probably not the place you are in your Python learning, so don't worry about this, but the operator module is designed for these kind of usages, when you want to pass an operator like + - etc. but of course can't pass the op itself to take actions because it's not a callable - the operator module has callables that can be used. From jfong at ms4.hinet.net Thu Sep 30 23:05:17 2021 From: jfong at ms4.hinet.net (Jach Feng) Date: Thu, 30 Sep 2021 20:05:17 -0700 (PDT) Subject: Definitive guide for Regex In-Reply-To: References: Message-ID: <7e646df5-f508-43a6-9a82-6196a70b2732n@googlegroups.com> Shaozhong SHI ? 2021?9?30? ?????7:29:47 [UTC+8] ?????? > Dear All, > > I am trying to look for a definitive guide for Regex in Python. > Can anyone help? > > Regards, > > David Try the rexegg.com which is a site dedicate to this subject and has many resources. --Jach