Hmm, Thanks Chris. I thought I was posting this to the correct place. I've never seen that "for line in open ..." after googling it many times! Why is this question so often asked then? Re:Indentation making end block markers not needed; well yes they aren't /needed/. However, they are useful for readability purposes. Perhaps if I use it some more I'll see that they aren't but I doubt it. Re:PEP249 & SQL, I thought I was proposing something like that but it can't be tacked on later I don't think - needs to be an inate part of Python to work as cleanly as 4gl languages. Re: your named tuple suggestion, wouldn't that mean that the naming is divorced from the result column names - that is part of what shouldn't be. Re:Everything being true of false. I don't see the value of that. Only boolean data should be valid in boolean contexts. I don't really see how that can be argued. On 09/01/17 21:31, python-ideas-request@python.org wrote:
Send Python-ideas mailing list submissions to python-ideas@python.org
To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/python-ideas or, via email, send a message with subject or body 'help' to python-ideas-request@python.org
You can reach the person managing the list at python-ideas-owner@python.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-ideas digest..."
Today's Topics:
1. Re: PEP 540: Add a new UTF-8 mode (INADA Naoki) 2. Python Reviewed (Simon Lovell) 3. Re: Python Reviewed (Chris Angelico)
----------------------------------------------------------------------
Message: 1 Date: Mon, 9 Jan 2017 11:21:41 +0900 From: INADA Naoki <songofacandy@gmail.com> To: "Stephen J. Turnbull" <turnbull.stephen.fw@u.tsukuba.ac.jp> Cc: Victor Stinner <victor.stinner@gmail.com>, python-ideas <python-ideas@python.org> Subject: Re: [Python-ideas] PEP 540: Add a new UTF-8 mode Message-ID: <CAEfz+TwaVHaKnyquXuUqBMk=AyUcwDMgyA8efuhU9=oMhaGZnQ@mail.gmail.com> Content-Type: text/plain; charset=UTF-8
On Sun, Jan 8, 2017 at 1:47 AM, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
INADA Naoki writes:
I want UTF-8 mode is enabled by default (opt-out option) even if locale is not POSIX, like `PYTHONLEGACYWINDOWSFSENCODING`.
Users depends on locale know what locale is and how to configure it. They can understand difference between locale mode and UTF-8 mode and they can opt-out UTF-8 mode. But many people lives in "UTF-8 everywhere" world, and don't know about locale.
I find all this very strange from someone with what looks like a Japanese name. I see mojibake and non-Unicode encodings around me all the time. Caveat: I teach at a University that prides itself on being the most international of Japanese national universities, so in my daily work I see Japanese in 4 different encodings (5 if you count the UTF-16 used internally by MS Office), Chinese in 3 different (claimed) encodings, and occasionally Russian in at least two encodings, ..., uh, I could go on but won't. In any case, the biggest problems are legacy email programs and busted websites in Japanese, plus email that is labeled "GB2312" but actually conforms to GBK (and this is a reply in Japanese to a Chinese applicant writing in Japanese encoded as GBK). Since I work on tech company, and use Linux for most only "server-side" program, I don't live such a situation.
But when I see non UTF-8 text, I don't change locale to read such text. (Actually speaking, locale doesn't solve mojibake because it doesn't change my terminal emulator's encoding). And I don't change my terminal emulator setting only for read such a text. What I do is convert it to UTF-8 through command like `view text-from-windows.txt ++enc=cp932`
So there are no problem when Python always use UTF-8 for fsencoding and stdio encoding.
I agree that people around me mostly know only two encodings: "works for me" and "mojibake", but they also use locales configured for them by technical staff. On top of that, international students (the most likely victims of "UTF-8 by default" because students are the biggest Python users) typically have non-Japanese locales set on their imported computers. Hmm, Which OS do they use? There are no problem in macOS and Windows. Do they use Linux with locale with encoding other than UTF-8, and their terminal emulator uses non-UTF-8 encoding?
As my feeling, UTF-8 start dominating from about 10 years ago, and ja_JP.EUC_JP (it was most common locale for Japanese befoer UTF-8) is complete legacy.
There is only one machine (which is in LAN, lives from 10+ years ago, /usr/bin/python is Python 1.5!), I can ssh which has ja_JP.eucjp locale.
------------------------------
Message: 2 Date: Mon, 9 Jan 2017 19:25:45 +0800 From: Simon Lovell <simon58500@bigpond.com> To: python-ideas@python.org Subject: [Python-ideas] Python Reviewed Message-ID: <69e3c5d4-d64b-063e-758e-2b0ac1720daa@bigpond.com> Content-Type: text/plain; charset=utf-8; format=flowed
Python Reviewed
Having used a lot of languages a little bit and not finding satisfactory answers to these in some cases often asked questions, I thought I'd join this group to make a post on the virtues and otherwise of python.
The Good: Syntactically significant new lines Syntactically significant indenting Different types of array like structures for different situations Mostly simple and clear structures Avoiding implicit structures like C++ references which add only negative value Avoiding overly complicated chaining expressions like "while(*d++=*s++);" Single syntax for block statements (well, sort of. I'm ignoring lines like "if a=b: c=d") Lack of a with statement which only obscures the code
The Bad: Colons at the end of if/while/for blocks. Most of the arguments in favour of this decision boil down to PEP 20.2 "Explicit is better than implicit". Well, no. if/while/for blocks are already explicit. Adding the colon makes it doubly explicit and therefore redundant. There is no reason I can see why this colon can't be made optional except for possibly PEP20.13 "There should be one-- and preferably only one --obvious way to do it". I don't agree that point is sufficient to require colons.
No end required for if/while/for blocks. This is particularly a problem when placing code into text without fixed width fonts. It also is a potential problem with tab expansion tricking the programmer. This could be done similarly to requiring declarations in Fortran, which if "implicit none" was added to the top of the program, declarations are required. So add a "Block Close Mandatory" (or similar) keyword to enforce this. In practice there is usually a blank line placed at the end of blocks to try to signal this to someone reading the code. Makes the code less readable and I would refer to PEP20.7 "Readability counts"
This code block doesn't compile, even given that function "process" takes one string parameter: f=open(file) endwhile="" while (line=f.readline())!=None: process(line) endwhile
I note that many solutions have been proposed to this. In C, it is the ability to write "while(line=fgets(f))" instead of "while((line=fgets(f))!=NULL)" which causes the confusion. No solutions have been accepted to the current method which is tacky: f=open(file) endwhile="" endif="" while True: line=f.readline if line = None: break endif process(line) endwhile
Inadequacy of PEP249 - Python Database Specification. This only supports dynamic SQL but SQL and particularly select statements should be easier to work with in the normal cases where you don't need such statements. e.g: endselect="" idList = select from identities where surname = 'JONES': idVar = id forenameVar = forename surnameVar = surname dobVar = dob endselect
endfor="" for id in idList: print id.forenameVar, id.dobVar endfor
as opposed to what is presently required in the select case which is: curs = connection.cursor() curs.execute("select id, forename, surname, dob from identities where surname = 'JONES'") idList=curs.fetchall()
endfor="" for id in idList: print id[1], id[3] endfor
I think the improvement in readibility for the first option should be plain to all even in the extremely simple case I've shown.
This is the sort of thing which should be possible in any language which works with a database but somehow the IT industry has lost it in the 1990s/2000s. Similarly an upgraded syntax for the insert/values statement which the SQL standard has mis-specified to make the value being inserted too far away from the column name. Should be more like: endinsert="" Insert into identities: id = 1 forename = 'John' surname = 'Smith' dob = '01-Jan-1970' endinsert
One of the major problems with the status quo is the lack of named result columns. The other is that the programmer is required to convert the where clause into a string. The functionality of dynamic where/from clauses can still be provided without needing to rely on numbered result columns like so: endselect="" idList = select from identities where :where_clause: id = id forename = forename surname = surname dob = dob endselect
Ideally, the bit after the equals sign would support all syntaxes allowed by the host database server which probably means it needs to be free text passed to the server. Where a string variable should be passed, the :variable syntax could be supported but this is not often required
Variables never set to anything do not error until they are used, at least in implementations of Python 2 I have tried. e.g. UnlikelyCondition = False endif="" if UnlikelyCondition: print x endif
The above code runs fine until UnlikelyCondition is set to True
No do-while construct
else keyword at the end of while loops is not obvious to those not familiar with it. Something more like whenFalse would be clearer
Changing print from a statement to a function in Python 3 adds no positive value that I can see
Upper delimiters being exclusive while lower delimiters are inclusive. This is very counter intuitive. e.g. range(1,4) returns [1,2,3]. Better to have the default base as one rather than zero IMO. Of course, the programmer should always be able to define the lower bound. This cannot be changed, of course.
Lack of a single character in a method to refer to an attribute instead of a local variable, similar to C's "*" for dereferencing a pointer
Inability to make simple chained assignments e.g. "a = b = 0"
Conditional expression (<true-value> if <condition> else <false-value>) in Python is less intuitive than in C (<condition> ? <true-value> : <false-value>). Ref PEP308. Why BDFL chose the syntax he did is not at all clear.
The Ugly: Persisting with the crapulence from C where a non zero integer is true and zero is false - only ever done because C lacked a boolean data type. This is a flagrant violation of PEP 20.2 "Explicit is better than implicit" and should be removed without providing backwards compatibility.
------------------------------
Message: 3 Date: Tue, 10 Jan 2017 00:31:42 +1100 From: Chris Angelico <rosuav@gmail.com> To: python-ideas <python-ideas@python.org> Subject: Re: [Python-ideas] Python Reviewed Message-ID: <CAPTjJmrTfYup+o7BYwnYB=yCbLof3Csq0CcNU36ke8J5ynb5QA@mail.gmail.com> Content-Type: text/plain; charset=UTF-8
On Mon, Jan 9, 2017 at 10:25 PM, Simon Lovell <simon58500@bigpond.com> wrote:
Python Reviewed
Having used a lot of languages a little bit and not finding satisfactory answers to these in some cases often asked questions, I thought I'd join this group to make a post on the virtues and otherwise of python. I think this thread belongs on python-list@python.org, where you'll find plenty of people happy to discuss why Python is and/or shouldn't be the way it is.
A couple of responses to just a couple of your points.
The Good: Syntactically significant new lines Syntactically significant indenting The Bad: No end required for if/while/for blocks. This is particularly a problem when placing code into text without fixed width fonts. It also is a potential problem with tab expansion tricking the programmer. If indentation and line endings are significant, you shouldn't need end markers. They don't buy you anything. In any case, I've never missed them; in fact, Python code follows the "header and lines" concept that I've worked with in many, MANY data files for decades (think of the sectioned config file format, for example).
This code block doesn't compile, even given that function "process" takes one string parameter: f=open(file) endwhile="" while (line=f.readline())!=None: process(line) endwhile
I note that many solutions have been proposed to this. In C, it is the ability to write "while(line=fgets(f))" instead of "while((line=fgets(f))!=NULL)" which causes the confusion. No solutions have been accepted to the current method which is tacky: f=open(file) endwhile="" endif="" while True: line=f.readline if line = None: break endif process(line) endwhile
Here's a better way:
for line in open(file): process(line)
If you translate C code to Python, sure, it'll sometimes come out even uglier than the C original. But there's often a Pythonic way to write things.
Inadequacy of PEP249 - Python Database Specification. This only supports dynamic SQL but SQL and particularly select statements should be easier to work with in the normal cases where you don't need such statements. e.g: endselect="" idList = select from identities where surname = 'JONES': idVar = id forenameVar = forename surnameVar = surname dobVar = dob endselect
endfor="" for id in idList: print id.forenameVar, id.dobVar endfor
You're welcome to propose something like this. I suspect you could build an SQL engine that uses a class to create those bindings - something like:
class people(identities): id, forename, surname, dob where="surname = 'JONES'"
for person in people: print(person.forename, person.dob)
Side point: "forename" and "surname" are inadvisable fields. http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-nam...
One of the major problems with the status quo is the lack of named result columns. The other is that the programmer is required to convert the where clause into a string. The functionality of dynamic where/from clauses can still be provided without needing to rely on numbered result columns like so: endselect="" idList = select from identities where :where_clause: id = id forename = forename surname = surname dob = dob endselect
That's easy enough to do with a namedtuple.
Variables never set to anything do not error until they are used, at least in implementations of Python 2 I have tried. e.g. UnlikelyCondition = False endif="" if UnlikelyCondition: print x endif
The above code runs fine until UnlikelyCondition is set to True
That's because globals and builtins could be created dynamically. It's a consequence of not having variable declarations. You'll find a lot of editors/linters will flag this, though.
Changing print from a statement to a function in Python 3 adds no positive value that I can see
Adds heaps of positive value to a lot of people. You simply haven't met the situations where it's better. It's sufficiently better that I often use __future__ to pull it in even in 2.7-only projects.
Lack of a single character in a method to refer to an attribute instead of a local variable, similar to C's "*" for dereferencing a pointer
Ehh. "self." isn't that long. Python isn't AWK.
Inability to make simple chained assignments e.g. "a = b = 0"
Really? Works fine. You can chain assignment like that.
Conditional expression (<true-value> if <condition> else <false-value>) in Python is less intuitive than in C (<condition> ? <true-value> : <false-value>). Ref PEP308. Why BDFL chose the syntax he did is not at all clear.
I agree with you on this one - specifically, because the order of evaluation is "middle then outside", instead of left-to-right.
The Ugly: Persisting with the crapulence from C where a non zero integer is true and zero is false - only ever done because C lacked a boolean data type. This is a flagrant violation of PEP 20.2 "Explicit is better than implicit" and should be removed without providing backwards compatibility. In Python, *everything* is either true or false. Anything that represents "something" is true, and anything that represents "nothing" is false. An empty list is false, but a list with items in it is true. This is incredibly helpful and most definitely not ugly; Python is not REXX.
ChrisA
------------------------------
Subject: Digest Footer
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas
------------------------------
End of Python-ideas Digest, Vol 122, Issue 20 *********************************************
On Mon, Jan 9, 2017 at 2:50 PM, Simon Lovell <simon58500@bigpond.com> wrote:
Hmm, Thanks Chris. I thought I was posting this to the correct place.
I've never seen that "for line in open ..." after googling it many times! Why is this question so often asked then?
The distinction and the explanation of this is the first result of the google search "Python process file line by line" http://stackoverflow.com/questions/11130312/line-by-line-file-processing-for... You probably haven't came across that because you think and search using C terms. Once you get used to context managers they are incredibly useful. I would advise to watch talks like "Beyond Pep 8" [1], comparing the same program in Python and Java.
Re:Indentation making end block markers not needed; well yes they aren't /needed/. However, they are useful for readability purposes. Perhaps if I use it some more I'll see that they aren't but I doubt it.
I use to like end markers. Though not having them make the code quite shorted. When you have 1 line condition, or context manager or loop, it literally adds 50% more line to your condition/loop/contextmanager. As the next line is anyway de-indented, you see your block anyway.
Re:Everything being true of false. I don't see the value of that. Only boolean data should be valid in boolean contexts. I don't really see how that can be argued.
Things are true-ish or false-ish, if you prefer. This allows idioms like if mycontainer: process_my_container(mycontainer) And you can process it only if you have items and it is not none. Your own object can raise if they get casted to bools, so if you really like your object to not behave like a bool, you can. It's not because they are truthy or falsy that they compare equal:
if [] == False: ... print('[] equals False') ... else: ... print('[] not equals...') ... [] not equals... if not []:print('[] is Falsy') ... [] is Falsy
Also Boolean in Python are singletons (like None) , so you will see comparison by identity `... is None` ( `... is False`, `... is True` rarely) if you really care about only being in boolean context you can. Also Python is "ducktyped", if it quack and walks like a duck , then it's a duck. If an object defines how to behave as a bool then that's great. You can then use your own object that are truthy-falsy and carry more information by still using other libraries. if not allowed: raise OhNoooe('You can't because', allowed.reason) -- M [1]:https://www.youtube.com/watch?v=wf-BqAjZb8M Sorry Simon for double Mail, I forgot to reply-all.
Simon Lovell writes:
Hmm, Thanks Chris. I thought I was posting this to the correct place.
Well, you didn't actually make any specific suggestions, and you describe it as a "review" rather than an RFE.
I've never seen that "for line in open ..." after googling it many times! Why is this question so often asked then?
Lot of C programmers out there, I guess. It's in all the textbooks and references I have, and in the tutorial: https://docs.python.org/3/tutorial/inputoutput.html#methods-of-file-objects
Re:PEP249 & SQL, I thought I was proposing something like that but it can't be tacked on later I don't think - needs to be an inate part of Python to work as cleanly as 4gl languages.
You should take a look at SQLAlchemy and other Python database managers. I don't know what your "untackable it" is so I can't be more specific. Note that PEP 249 is not intended to provide an API for ordinary Python programmers' use. It was expected that convenient management modules would be provided on top of the DBAPI. PEP 249 is intended to provide an API between the backend drivers and the database manager modules, so that any manager could easily interface with any driver.
Re:Everything being true of false. I don't see the value of that. Only boolean data should be valid in boolean contexts. I don't really see how that can be argued.
There's only a point in arguing it if you think that data types are fundamentally mutually exclusive. But they're not in object-oriented languages like Python. Something can be object, boolean, and str all at the same time. (The presence of a type called boolean is a red herring here. True and False are merely the representative boolean values, a convenience for programmers who want them.) In Python, all data is boolean, unambiguously being interpreted as "true" or "false" in boolean contexts. As far as boolean contexts are concerned, there are an infinite number of objects equivalent to True and another bunch (currently not infinite) equivalent to False. It could be argued that this leads to programmers making bugs, but I personally haven't found it so, in Python or Lisp, and I find the lack of it very annoying when I'm writing Scheme since it's so similar to Lisp.
The Bad: Colons at the end of if/while/for blocks. Most of the arguments in favour of this decision boil down to PEP 20.2 "Explicit is better than implicit".
I seem to recall that this has to do with an implementation requirement, that the syntax be parseable with an LL parser.
This could be done similarly to requiring declarations in Fortran, which if "implicit none" was added to the top of the program, declarations are required.
It could, but won't. That's a pragma, and Guido hates pragmas. It's certainly not worth a keyword.
"while((line=fgets(f))!=NULL)" which causes the confusion. No solutions have been accepted to the current method which is tacky: f=open(file) endwhile="" endif="" while True: line=f.readline if line = None: break endif process(line) endwhile
Aside: I really find those suite terminators gratuitous; if I need something "stronger" than a dedent, an empty line is much prettier than they are IMO. Actually the accepted loop-and-a-half idiom is f = open(file) line = f.readline() while line: process(line) line = f.readline() "for line in f" makes that unnecessary in this case, but there do remain cases where loop-and-a-half is needed because of the lack of an assignment expression.
else keyword at the end of while loops is not obvious to those not familiar with it. Something more like whenFalse would be clearer
Keywords are expensive in that every Python programmer needs to know all of them to avoid using one as an identifier. So it is a general design principle of Python to avoid adding new ones where possible. It turns out that for ... else is rarely used even by experts, and the main use case is extremely idiomatic, so probably no harm is done.
Changing print from a statement to a function in Python 3 adds no positive value that I can see
It eliminates a keyword, makes it possible to experiment with different implementations, and allows printing in the middle of expressions (although since print() always returns None, that's not as useful as it could be).
Upper delimiters being exclusive while lower delimiters are inclusive. This is very counter intuitive.
If you say so. But it is convenient because list == list[:n] + list[n:].
Conditional expression (<true-value> if <condition> else <false-value>) in Python is less intuitive than in C (<condition> ? <true-value> : <false-value>). Ref PEP308. Why BDFL chose the syntax he did is not at all clear.
I seem to recall that ?: was out because Guido at the time was adamantly against use of ? as syntax in Python, so we were kinda stuck with keywords. He didn't want to have non-unary expressions start with keywords (would have caused ugliness in the parser, I guess) and he did want to reuse keywords. "<condition> then <true-value> else <false-value>" was suggested but most who posted thought it less readable than the syntax chosen. YMMV, of course.
The Ugly: Persisting with the crapulence from C where a non zero integer is true and zero is false - only ever done because C lacked a boolean data type.
There were at least four reasons for this in C, in fact, and none have anything to do with the inability to add a Boolean type to a programming language: (0) that's the way the hardware works (1) the idiom "if (ptr)" (2) the idiom "while (i--)" (3) the idiom "while (*dest++ = *src++)" all of which compiled[1] nicely to efficient machine code without an optimization pass, and some take advantage of useful characteristics of the Unix OS. Today one might argue that these are an attractive nuisance and too cute to be allowed to live, but at that time not so much was known about the kind of mistakes programmers like to make. Footnotes: [1] On modern machines (3) is handled more efficiently by specialized instructions.
not even sure why Im engaging, but.... Note 1) Many of these issues have been widely discussed all over the internet -- I don't think I've seen anything new here. So it would have been nice to do some more research before posting. Now into the fray!
Re:Everything being true of false. I don't see the value of
that. Only boolean data should be valid in boolean contexts.
I actually agree with that -- but there are a lot of nice conveniences from Python's idea of "truthiness", too.
The Bad: Colons at the end of if/while/for blocks. Most of the arguments in favour of this decision boil down to PEP 20.2 "Explicit is better than implicit".
I seem to recall that this has to do with an implementation requirement, that the syntax be parseable with an LL parser.
I don't so -- but I DO think that this was a usability issue that was investigated early in PYthon's development. (Or maybe even ABC's development) -- in fact, I suspect is is one of the few programming language syntax decisions (in any language) that went through any kind of formal usability testing. I didn't bring it up -- so I'll leave the googling to others.
Actually the accepted loop-and-a-half idiom is
f = open(file) line = f.readline() while line: process(line) line = f.readline()
I used to write that style, but I've never liked it, so went with: f = open(file) while True: line = f.readline() if not f: break process(line) the while True and check for a break is kinda ugly, but I think less ugly than two separate calls to readline() and, of course, we now have: for line in f: process(line) which is cleaner an easier than anything I've seen in any other language -- so WHAT exactly was the original complaint???
else keyword at the end of while loops is not obvious to those
not familiar with it. Something more like whenFalse would be clearer
I've kinda wanted an "do-until" loop of some sort sometimes, but frankly, not that badly :-)
Changing print from a statement to a function in Python 3 adds no positive value that I can see
yeah, yeah yeah -- PLEASE go read an number of py3 justifications and rants! This is really a dead horse.
Upper delimiters being exclusive while lower delimiters are inclusive. This is very counter intuitive.
but SO much better than the alternative! This was also tested som, I think maybe by Dijkstra of C++ fame. but these identities are REALLY, REALLY useful: s[:n] + s[n:] == s len(s[:n]) == n len(s[:-n]) == n len(s[n:i]) == i - n (maybe a few others?) These prevent a HUGE number of off by one errors and ecta code adding and subtracting one all over the place -- this is almost as important to Pythons' usability as the significant indentation :-)
Conditional expression (<true-value> if <condition> else
<false-value>) in Python is less intuitive than in C (<condition> ? <true-value> : <false-value>).
Ha Ha Ha Ha!!! I could read and understand Python's version the first time I saw it -- I still have to look up the C version (not much of C programmer). maybe you think it's wordy -- but it's very readable. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Jan 9, 2017 at 8:19 PM, Chris Barker <chris.barker@noaa.gov> wrote:
I think maybe by Dijkstra of C++ fame.
Dijkstra is famous for many things, but C++ is another Dutchman's fault. Dijkstra's famous works include "GOTO Considered Harmful" [1] and "How do we tell truths that might hurt?" [2]. [1]: http://wiki.c2.com/?GotoConsideredHarmful [2]: http://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD498.html
participants (5)
-
Alexander Belopolsky -
Chris Barker -
Matthias Bussonnier -
Simon Lovell -
Stephen J. Turnbull