On 12 October 2016 at 23:58, Danilo J. S. Bellini wrote:

Decimal notation is hardly readable when we're dealing with stuff designed in base 2 (e.g. due to the visual separation of distinct bytes).

Hmm what keeps you from separateting the logical units to be represented each by a decimal number? like 001 023 255 ... Do you really think this is less readable than its hex equivalent? Then you are probably working with hex numbers only, but I doubt that.

I agree that mixing representations for the same abstraction (using decimal in some places, hexadecimal in other ones) can be a bad idea. "Can be"? It is indeed a horrible idea. Also not only for same abstraction but at all.

makes me believe "decimal unicode codepoint" shouldn't ever appear in string representations. I use this site to look the chars up: http://www.tamasoft.co.jp/en/general-info/unicode-decimal.html

PS: that is rather peculiar, three negative replies already but with no strong arguments why it would be bad to stick to decimal only, only some "others do it so" and "tradition" arguments. The "base 2" argument could work at some grade but if stick to this criteria why not speak about octal/quoternary/binary then? Please note, I am talking only about readability _of the character set_ actually. And it is not including your habit issues, but rather is an objective criteria for using this or that character set. And decimal is objectively way more readable than hex standard character set, regardless of how strong your habits are.

On 13 October 2016 at 01:50, Chris Angelico wrote:

On Thu, Oct 13, 2016 at 10:09 AM, Mikhail V wrote:

...

On 12 October 2016 at 23:58, Danilo J. S. Bellini wrote:

...

Decimal notation is hardly readable when we're dealing with stuff designed in base 2 (e.g. due to the visual separation of distinct bytes).

Hmm what keeps you from separateting the logical units to be represented each by a decimal number? like 001 023 255 ... Do you really think this is less readable than its hex equivalent? Then you are probably working with hex numbers only, but I doubt that.

Way WAY less readable, and I'm comfortable working in both hex and decimal.

Please don't mix the readability and personal habit, which previuos repliers seems to do as well. Those two things has nothing to do with each other. If you are comfortable with old roman numbering system this does not make it readable. And I am NOT comfortable with hex, as well as most people would be glad to use single notation. But some of them think that they are cool because they know several numbering notations ;) But I bet few can actually understand which is more readable.

You're the one who's non-standard here. Most of the world uses hex for Unicode codepoints. No I am not the one, many people find it silly to use different notations for same thing - index of the element, and they are very right about that. I am not silly, I refuse to use it and luckily I can. Also I know that decimal is more readable than hex so my choice is supportend by the understanding and not simply refusing.

...

PS: that is rather peculiar, three negative replies already but with no strong arguments why it would be bad to stick to decimal only, only some "others do it so" and "tradition" arguments.

"Others do it so" is actually a very strong argument. If all the rest of the world uses + to mean addition, and Python used + to mean subtraction, it doesn't matter how logical that is, it is *wrong*.

This actually supports my proposal perfectly, if everyone uses decimal why suddenly use hex for same thing - index of array. I don't see how your analogy contradicts with my proposal, it's rather supporting it.

quote; if you us 0x93, you are annoyingly wrong,

Please don't make personal assessments here, I can use whatever I want, moreover I find this notation as silly as using different measurement systems without any reason and within one activity, and in my eyes this is annoyingly wrong and stupid, but I don't call nobody here stupid. But I do want that you could abstract yourself from your habit for a while and talk about what would be better for the future usage.

everyone has to do the conversion from that to 201C.

Nobody need to do ANY conversions if use decimal, and as said everything is decimal: numbers, array indexes, ord() function returns decimal, you can imagine more examples so it is not only more readable but also more traditional.

How many decimal digits would you use to denote a single character?

for text, three decimal digits would be enough for me personally, and in long perspective when the world's alphabetical garbage will dissapear, two digits would be ok.

you have to pad everything to seven digits (\u0000034 for an ASCII quote)?

Depends on case, for input - some separator, or padding is also ok, I don't have problems with both. For printing obviously don't show leading zeros, but rather spaces. But as said I find this Unicode only some temporary happening, it will go to history in some future and be used only to study extinct glyphs. Mikhail

On 13 October 2016 at 04:18, Brendan Barnwell wrote:

On 2016-10-12 18:56, Mikhail V wrote:

...

Please don't mix the readability and personal habit, which previuos repliers seems to do as well. Those two things has nothing to do with each other.

You keep saying this, but it's quite incorrect. The usage of decimal notation is itself just a convention, and the only reason it's easy for you (and for many other people) is because you're used to it. If you had grown up using only hexadecimal or binary, you would find decimal awkward.

Exactly, but this is not called "readability" but rather "acquired ability to read" or simply habit, which does not reflect the "readability" of the character set itself.

There is nothing objectively better about base 10 than any other place-value numbering system.

Sorry to say, but here you are totally wrong. Not to treat you personally for your fallacy, that is quite common among those who are not familiar with the topic, but you should consider some important points: --- 1. Each taken character set has certain grade of readability which depends solely on the form of its units (aka glyphs). 2. Linear string representation is superior to anything else (spiral, arc, etc.) 3. There exist glyphs which provide maximal readability, those are particular glyphs with particular constant form, and this form is absolutely independent from the encoding subject. 4. According to my personal studies (which does not mean it must be accepted or blindly believed in, but I have solid experience in this area and acting quite successful in it) the amount of this glyphs is less then 10, namely I am by 8 glyphs now. 5. Main measured parameter which reflects the readability (somewhat indirect however) is the pair-wize optical collision of each character pair of the set. This refers somewhat to legibility, or differentiation ability of glyphs. --- Less technically, you can understand it better if you think of your own words "There is nothing objectively better about base 10 than any other place-value numbering system." If this could be ever true than you could read with characters that are very similar to each other or something messy as good as with characters which are easily identifyable, collision resistant and optically consistent. But that is absurd, sorry. For numbers obviously you don't need so many character as for speech encoding, so this means that only those glyphs or even a subset of it should be used. This means anything more than 8 characters is quite worthless for reading numbers. Note that I can't provide here the works currently so don't ask me for that. Some of them would be probably available in near future. Your analogy with speech and signs is not correct because speech is different but numbers are numbers. But also for different speech, same character set must be used namely the one with superior optical qualities, readability.

Saying we should dump hex notation because everyone understands decimal is like saying that all signs in Prague should only be printed in English

We should dump hex notation because currently decimal is simply superiour to hex, just like Mercedes is superior to Lada, aand secondly, because it is more common for ALL people, so it is 2:0 for not using such notation. With that said, I am not against base-16 itself in the first place, but rather against the character set which is simply visually inconsistent and not readable. Someone just took arabic digits and added first latin letters to it. It could be forgiven for a schoolboy's exercises in drawing but I fail to understand how it can be accepted as a working notation for medium supposed to be human readable. Practically all this notation does, it reduces the time before you as a programmer become visual and brain impairments.

Just look at the Wikipedia page for Unicode, which says: "Normally a Unicode code point is referred to by writing "U+" followed by its hexadecimal number." That's it.

Yeah that's it. And it sucks and migrated to coding standard, sucks twice. If a new syntax/standard is decided, there'll be only positive sides of using decimal vs hex. So nobody'll be hurt, this is only the question of remaking current implementation and is proposed only as a long-term theoretical improvement.

it's just a label that identifies the character.

Ok, but if I write a string filtering in Python for example then obviously I use decimal everywhere to compare index ranges, etc. so what is the use for me of that label? Just redundant conversions back and forth. Makes me sick actually.

Mikhail V wrote:

Ok, but if I write a string filtering in Python for example then obviously I use decimal everywhere to compare index ranges, etc. so what is the use for me of that label? Just redundant conversions back and forth.

I'm not sure what you mean by that. If by "index ranges" you're talking about the numbers you use to index into the string, they have nothing to do with character codes, so you can write them in whatever base is most convenient for you. If you have occasion to write a literal representing a character code, there's nothing to stop you writing it in hex to match the way it's shown in a repr(), or in published Unicode tables, etc. I don't see a need for any conversions back and forth. -- Greg

On Thu, Oct 13, 2016 at 1:46 AM, Mikhail V wrote:

Practically all this notation does, it reduces the time before you as a programmer become visual and brain impairments.

Even if you were right that your approach is somehow inherently easier, it is flat-out wrong that other approaches lead to "brain impairment". On the contrary, it is well-established that challenging the brain prevents or at least delays brain impairment. And it also makes no sense that it would cause visual impairment, either. Comparing glyphs is a higher-level task in the brain, it has little to do with your eyes. All your eyes detect are areas of changing contrast, any set of lines and curves, not even glyphs, is functionally identical at that level (and even at medium-level brain regions). The size of the glyphs can make a difference, but not the number of available ones. On the contrary, having more glyphs increases the information density of text, reducing the amount of reading you have to do to get the same information.

On Sun, Oct 16, 2016 at 3:26 PM, Mikhail V wrote:

One my supposition is that during the reading there is very intensive two-directional signalling between eye and brain. So generally you are correct, the eye is technically a camera attached to the brain and simply sends pictures at some frequency to the brain. But I would tend to think that it is not so simple actually. You probably have heard sometimes users who claim something like: "this text hurts my eyes" For example if you read non-antialiased text and with too high contrast, you'll notice that something is indeed going wrong with your eyes. This can happen probably because the brain starts to signal the eye control system "something is wrong, stop doing it" Since your eye cannot do anything with wrong contrast on your screen and you still need to continue reading, this happens again and again. This can cause indeed unwanted processes and overtiredness of muscles inside the eye.

The downards-projecting signals from the brain to the eye are heavily studied. In fact I have friends who specialize in studying those connections specifically. They simply don't behave the way you are describing. You are basing your claims on the superiority of certain sorts of glyphs on conjecture about how the brain works, conjecture that goes against what the evidence says about how the brain actually processes visual information. Yes, the quality of the glyphs can make a big difference. There is no indication, however, that the number of possible glyphs can.

And it also makes no sense that it would cause visual impairment, either.

...

Comparing glyphs is a higher-level task in the brain, it has little to do with your eyes.

You forget about that whith illegible font or wrong contrast for example you *do* need to do more concentrarion, This causes again your eye to try harder to adopt to the information you see, reread, which again affects your lens and eye movements.

I don't want to imply bad faith on your part, but you cut off an important part of what I said: "The size of the glyphs can make a difference, but not the number of available ones. On the contrary, having more glyphs increases the information density of text, reducing the amount of reading you have to do to get the same information." Badly-antialised text can be a problem from that standpoint too. But again, none of this has anything whatsoever to do with the number of glyphs, which is your complaint. Again, I don't want to imply bad faith, but the argument you are making now is completely different than the argument I was addressing. I don't disagree that bad text quality or too much reading can hurt your eyes. On the contrary, I said explicitly that it can. The claim of yours that I was addressing is that having too many glyphs can hurt your eyes or brain, which doesn't match with anything we know about how the eyes or brain work.

On Oct 12, 2016 9:25 PM, "Chris Angelico" wrote:

On Thu, Oct 13, 2016 at 12:56 PM, Mikhail V wrote:

...

But as said I find this Unicode only some temporary happening, it will go to history in some future and be used only to study extinct glyphs.

And what will we be using instead?

Emoji, of course! What else?

Morbid curiosity trumping a plonking, for the moment.

ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Ryan (ライアン) [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/

On 13 October 2016 at 04:49, Emanuel Barry wrote:

...

From: Mikhail V Sent: Wednesday, October 12, 2016 9:57 PM Subject: Re: [Python-ideas] Proposal for default character representation

Hello, and welcome to Python-ideas, where only a small portion of ideas go further, and where most newcomers that wish to improve the language get hit by the reality bat! I hope you enjoy your stay :) Hi, thanks! I enjoy the conversation indeed , never had so much interesting in a discussion actually!

...

On 13 October 2016 at 01:50, Chris Angelico wrote:

...

On Thu, Oct 13, 2016 at 10:09 AM, Mikhail V wrote:

Way WAY less readable, and I'm comfortable working in both hex and decimal.

Please don't mix the readability and personal habit, which previuos repliers seems to do as well. Those two things has nothing to do with each other. If you are comfortable with old roman numbering system this does not make it readable. And I am NOT comfortable with hex, as well as most people would be glad to use single notation. But some of them think that they are cool because they know several numbering notations ;) But I bet few can actually understand which is more readable.

I'll turn your argument around: Not being comfortable with hex does not make it unreadable; it's a matter of habit (as Brendan pointed out in his separate reply).

Matter of habit does not reflect the readability, see my last reply to Brandan. It is quite precise engeneering. And readability it is kind of serious stuff especially if you decide for programming carreer. Young people underestimate it and for oldies it is too late when they realize it :) And Python is all about readability and I like it. As for your other points, I'll need to read it with fresh head tomorrow, Of course I don't believe this would all suddenly happen with Python, or other programming language, it is just an idea anyway. And I do want to learn more actually. Especially want to see some example where it would be really beneficial to use hex, either technically (some low level binary related stuff?) or regarding comprehension, which is to my knowledge hardly possible.

- Indexing, and that's completely irrelevant to the topic at hand (also see above bullet point). Eee how would I find if the character lies in certain range? With index here I meant it's numeric value, I just called it index for some reason, I don't know why. So its a table - value and corresponding glyph. Just consieder analogy: I make an 3d array, first index is my value, and 2nd 3rd is image pixels, so simply image stack. Why on earth would I use for 1st index some other literals than decimal. Did you see much code written with hex literals? Some low level things probably ...

- ord() which returns an integer (which can be interpreted in any base!), Yes so my idea is to stick to other notations than hex. for low level bit manipulation obviously two-character notation should be used, so again I fail to see something...

Mikhail

On 13 Oct 2016, at 09:43, Greg Ewing wrote:

Mikhail V wrote:

...

Did you see much code written with hex literals?

From /usr/include/sys/fcntl.h:

Backing Greg up for a moment, hex literals are extremely common in any code that needs to work with binary data, such as network programming or fine data structure manipulation. For example, consider the frequent requirement to mask out certain bits of a given integer (e.g., keep the low 24 bits of a 32 bit integer). Here are a few ways to represent that: integer & 0x00FFFFFF # Hex integer & 16777215 # Decimal integer & 0o77777777 # Octal integer & 0b111111111111111111111111 # Binary Of those four, hexadecimal has the advantage of being both extremely concise and clear. The octal representation is infuriating because one octal digit refers to *three* bits, which means that there is a non-whole number of octal digits in a byte (that is, one byte with all bits set is represented by 0o377). This causes problems both with reading comprehension and with most other common tasks. For example, moving from 0xFF to 0xFFFF (or 255 to 65535, also known as setting the next most significant byte to all 1) is represented in octal by moving from 0o377 to 0o177777. This is not an obvious transition, and I doubt many programmers could do it from memory in any representation but hex or binary. Decimal is no clearer. Programmers know how to represent certain bit patterns from memory in decimal simply because they see them a lot: usually they can do the all 1s case, and often the 0 followed by all 1s case (255 and 128 for one byte, 65535 and 32767 for two bytes, and then increasingly few programmers know the next set). But trying to work out what mask to use for setting only bits 15 and 14 is tricky in decimal, while in hex it’s fairly easy (in hex it’s 0xC000, in decimal it’s 49152). Binary notation seems like the solution, but note the above case: the only way to work out how many bits are being masked out is to count them, and there can be quite a lot. IIRC there’s some new syntax coming for binary literals that would let us represent them as 0b1111_1111_1111_1111, which would help the readability case, but it’s still substantially less dense and loses clarity for many kinds of unusual bit patterns. Additionally, as the number of bits increases life gets really hard: masking out certain bits of a 64-bit number requires a literal that’s at least 66 characters long, not including the underscores that would add another 15 underscores for a literal that is 81 characters long (more than the PEP8 line width recommendation). That starts getting unwieldy fast, while the hex representation is still down at 18 characters. Hexadecimal has the clear advantage that each character wholly represents 4 bits, and the next 4 bits are independent of the previous bits. That’s not true of decimal or octal, and while it’s true of binary it costs a fourfold increase in the length of the representation. It’s definitely not as intuitive to the average human being, but that’s ok: it’s a specialised use case, and we aren’t requiring that all human beings learn this skill. This is a very long argument to suggest that your argument against hexadecimal literals (namely, that they use 16 glyphs as opposed to the 10 glyphs used in decimal) is an argument that is too simple to be correct. Different collections of glyphs are clearer in different contexts. For example, decimal numerals can be represented using 10 glyphs, while the english language requires 26 glyphs plus punctuation. But I don’t think you’re seriously proposing we should swap from writing English using the larger glyph set to writing it in decimal representation of ASCII bytes. Given this, I think the argument that says that the Unicode consortium said “write the number in hex” is good enough for me. Cory

On 13 October 2016 at 12:05, Cory Benfield wrote:

integer & 0x00FFFFFF # Hex integer & 16777215 # Decimal integer & 0o77777777 # Octal integer & 0b111111111111111111111111 # Binary

The octal representation is infuriating because one octal digit refers to *three* bits

Correct, makes it not so nice looking and 8-bit-paradigm friendly. Does not make it however bad option in general and according to my personal suppositions and works on glyph development the optimal set is exactly of 8 glyphs.

Decimal is no clearer.

In same alignment problematic context, yes, correct.

Binary notation seems like the solution, ...

Agree with you, see my last reply to Greg for more thoughts on bitstrings and quoternary approach.

IIRC there’s some new syntax coming for binary literals that would let us represent them as 0b1111_1111_1111_1111

Very good. Healthy attitude :)

less dense and loses clarity for many kinds of unusual bit patterns.

Not very clear for me what is exactly there with patterns.

Additionally, as the number of bits increases life gets really hard: masking out certain bits of a 64-bit number requires

Self the editing of such a BITmask in hex notation makes life hard. Editing it in binary notation makes life easier.

a literal that’s at least 66 characters long,

Length is a feature of binary, though it is not major issue, see my ideas on it in reply to Greg

Hexadecimal has the clear advantage that each character wholly represents 4 bits,

This advantage is brevity, but one need slightly less brevity to make it more readable. So what do you think about base 4 ?

This is a very long argument to suggest that your argument against hexadecimal literals (namely, that they use 16 glyphs as opposed to the 10 glyphs used in decimal) is an argument that is too simple to be correct.

I didn't understood this sorry :))) Youre welcome to ask more if youre intersted in this.

Different collections of glyphs are clearer in different contexts. How much different collections and how much different contexts?

while the english language requires 26 glyphs plus punctuation.

Does not *require*, but of course 8 glyphs would not suffice to effectively read the language, so one finds a way to extend the glyph set. Roughly speaking 20 letters is enough, but this is not exact science. And it is quite hard science.

But I don’t think you’re seriously proposing we should swap from writing English using the larger glyph set to writing it in decimal representation of ASCII bytes.

I didn't understand this sentence :) In general I think we agree on many points, thank you for the input! Cheers, Mikhail

On Thu, Oct 13, 2016 at 03:56:59AM +0200, Mikhail V wrote:

...

How many decimal digits would you use to denote a single character?

for text, three decimal digits would be enough for me personally,

Well, if it's enough for you, why would anyone need more?

and in long perspective when the world's alphabetical garbage will dissapear, two digits would be ok.

Are you serious? Talking about "alphabetical garbage" like that makes you seem to be an ASCII bigot: rude, ignorant, arrogant and rather foolish as well. Even 7-bit ASCII has more than 100 characters (128). -- Steve

On 13 October 2016 at 16:50, Chris Angelico wrote:

On Fri, Oct 14, 2016 at 1:25 AM, Steven D'Aprano wrote:

...

On Thu, Oct 13, 2016 at 03:56:59AM +0200, Mikhail V wrote:

...

and in long perspective when the world's alphabetical garbage will dissapear, two digits would be ok. Talking about "alphabetical garbage" like that makes you seem to be an ASCII bigot: rude, ignorant, arrogant and rather foolish as well. Even 7-bit ASCII has more than 100 characters (128).

This is sort of rude. Are you from unicode consortium?

Solution: Abolish most of the control characters. Let's define a brand new character encoding with no "alphabetical garbage". These characters will be sufficient for everyone:

* [2] Formatting characters: space, newline. Everything else can go. * [8] Digits: 01234567 * [26] Lower case Latin letters a-z * [2] Vital social media characters: # (now officially called "HASHTAG"), @ * [2] Can't-type-URLs-without-them: colon, slash (now called both "SLASH" and "BACKSLASH")

That's 40 characters that should cover all the important things anyone does - namely, Twitter, Facebook, and email. We don't need punctuation or capitalization, as they're dying arts and just make you look pretentious. I might have missed a few critical characters, but it should be possible to fit it all within 64, which you can then represent using two digits from our newly-restricted set; octal is better than decimal, as it needs less symbols. (Oh, sorry, so that's actually "50" characters, of which "32" are the letters. And we can use up to "100" and still fit within two digits.)

Is this the wrong approach, Mikhail?

This is sort of correct approach. We do need punctuation however. And one does not need of course to make it too tight. So 8-bit units for text is excellent and enough space left for experiments.

Perhaps we should go the other way, then, and be *inclusive* of people who speak other languages.

What keeps people from using same characters? I will tell you what - it is local law. If you go to school you *have* to write in what is prescribed by big daddy. If youre in europe or America, you are more lucky. And if you're in China you'll be punished if you want some freedom. So like it or not, learn hieroglyphs and become visually impaired in age of 18.

Thanks to Unicode's rich collection of characters, we can represent multiple languages in a single document;

Can do it without unicode in 8-bit boundaries with tagged text, just need fonts for your language, of course if your local charset is less than 256 letters. This is how it was before unicode I suppose. BTW I don't get it still what such revolutionary advantages has unicode compared to tagged text.

script, but have different characters. Alphabetical garbage, or accurate representations of sounds and words in those languages?

Accurate with some 50 characters is more than enough. Mikhail

On 14 Oct 2016, at 08:53, Mikhail V wrote:

What keeps people from using same characters? I will tell you what - it is local law. If you go to school you *have* to write in what is prescribed by big daddy. If youre in europe or America, you are more lucky. And if you're in China you'll be punished if you want some freedom. So like it or not, learn hieroglyphs and become visually impaired in age of 18.

So you know, for the future, I think this comment is going to be the one that causes most of the people who were left to disengage with this discussion. The many glyphs that exist for writing various human languages are not inefficiency to be optimised away. Further, I should note that most places to not legislate about what character sets are acceptable to transcribe their languages. Indeed, plenty of non-romance-language-speakers have found ways to transcribe their languages of choice into the limited 8-bit character sets that the Anglophone world propagated: take a look at Arabish for the best kind of example of this behaviour, where "الجو عامل ايه النهارده فى إسكندرية؟" will get rendered as "el gaw 3amel eh elnaharda f eskendereya?” But I think you’re in a tiny minority of people who believe that all languages should be rendered in the same script. I can think of only two reasons to argue for this: 1. Dealing with lots of scripts is technologically tricky and it would be better if we didn’t bother. This is the anti-Unicode argument, and it’s a weak argument, though it has the advantage of being internally consistent. 2. There is some genuine harm caused by learning non-ASCII scripts. Your paragraph suggest that you really believe that learning to write in Kanji (logographic system) as opposed to Katagana (alphabetic system with 48 non-punctuation characters) somehow leads to active harm (your phrase was “become visually impaired”). I’m afraid that you’re really going to need to provide one hell of a citation for that, because that’s quite an extraordinary claim. Otherwise, I’m afraid I have to say お先に失礼します. Cory

On 13.10.16 17:50, Chris Angelico wrote:

Solution: Abolish most of the control characters. Let's define a brand new character encoding with no "alphabetical garbage". These characters will be sufficient for everyone:

* [2] Formatting characters: space, newline. Everything else can go. * [8] Digits: 01234567 * [26] Lower case Latin letters a-z * [2] Vital social media characters: # (now officially called "HASHTAG"), @ * [2] Can't-type-URLs-without-them: colon, slash (now called both "SLASH" and "BACKSLASH")

That's 40 characters that should cover all the important things anyone does - namely, Twitter, Facebook, and email. We don't need punctuation or capitalization, as they're dying arts and just make you look pretentious.

https://en.wikipedia.org/wiki/DEC_Radix-50

On 2016-10-13 00:50, Chris Angelico wrote:

On Thu, Oct 13, 2016 at 10:09 AM, Mikhail V wrote:

...

On 12 October 2016 at 23:58, Danilo J. S. Bellini wrote:

...

Decimal notation is hardly readable when we're dealing with stuff designed in base 2 (e.g. due to the visual separation of distinct bytes).

Hmm what keeps you from separateting the logical units to be represented each by a decimal number? like 001 023 255 ... Do you really think this is less readable than its hex equivalent? Then you are probably working with hex numbers only, but I doubt that.

Way WAY less readable, and I'm comfortable working in both hex and decimal.

...

...

I agree that mixing representations for the same abstraction (using decimal in some places, hexadecimal in other ones) can be a bad idea. "Can be"? It is indeed a horrible idea. Also not only for same abstraction but at all.

...

makes me believe "decimal unicode codepoint" shouldn't ever appear in string representations. I use this site to look the chars up: http://www.tamasoft.co.jp/en/general-info/unicode-decimal.html

You're the one who's non-standard here. Most of the world uses hex for Unicode codepoints.

http://unicode.org/charts/

HTML entities permit either decimal or hex, but other than that, I can't think of any common system that uses decimal for Unicode codepoints in strings.

...

PS: that is rather peculiar, three negative replies already but with no strong arguments why it would be bad to stick to decimal only, only some "others do it so" and "tradition" arguments.

"Others do it so" is actually a very strong argument. If all the rest of the world uses + to mean addition, and Python used + to mean subtraction, it doesn't matter how logical that is, it is *wrong*. Most of the world uses U+201C or "\u201C" to represent a curly double quote; if you us 0x93, you are annoyingly wrong, and if you use 8220, everyone has to do the conversion from that to 201C. Yes, these are all differently-valid standards, but that doesn't make it any less annoying.

...

Please note, I am talking only about readability _of the character set_ actually. And it is not including your habit issues, but rather is an objective criteria for using this or that character set. And decimal is objectively way more readable than hex standard character set, regardless of how strong your habits are.

How many decimal digits would you use to denote a single character? Do you have to pad everything to seven digits (\u0000034 for an ASCII quote)? And if not, how do you mark the end? This is not "objectively more readable" if the only gain is "no A-F" and the loss is "unpredictable length".

Well, Perl doesn't have \u or \U; instead it has extended \x, so you can write, say, \x{201C}. Still in hex, though, as nature intended! :-)

Forgot to reply to all, duping my mesage... On 12 October 2016 at 23:48, M.-A. Lemburg wrote:

Hmm, in Python3, I get:

...

...

...

s = "абв.txt" s 'абв.txt'

I posted output with Python2 and Windows 7 BTW , In Windows 10 'print' won't work in cmd console at all by default with unicode but thats another story, let us not go into that. I think you get my idea right, it is not only about printing.

The hex notation for \uXXXX is a standard also used in many other programming languages, it's also easier to parse, so I don't think we should change this default.

In programming literature it is used often, but let me point out that decimal is THE standard and is much much better standard in sence of readability. And there is no solid reason to use 2 standards at the same time.

Take e.g.

...

...

...

s = "\u123456" s 'ሴ56'

With decimal notation, it's not clear where to end parsing the digit notation.

How it is not clear if the digit amount is fixed? Not very clear what did you mean.

On 13 October 2016 at 10:18, M.-A. Lemburg wrote:

I suppose you did not intend everyone to have to write \u0000010 just to get a newline code point to avoid the ambiguity.

Ok there are different usage cases. So in short without going into detail, for example if I need to type in a unicode string literal in ASCII editor I would find such notation replacement beneficial for me: u'\u0430\u0431\u0432.txt' --> u"{1072}{1073}{1074}.txt" Printing could be the same I suppose. I use Python 2.7. And printing so with numbers instead of non-ASCII would help me see where I have non-ASCII chars. But I think the print behavior must be easily configurable. Any critics on it? Besides not following the unicode consortium. Also I would not even mind fixed width 7-digit decimals actually. Ugly but still for me better than hex. Mikhail

FWIW, Python 3.6 should print this in the console just fine. Feel free to upgrade whenever you're ready. Cheers, Steve -----Original Message----- From: "Mikhail V" Sent: ‎10/‎12/‎2016 16:07 To: "M.-A. Lemburg" Cc: "python-ideas@python.org" Subject: Re: [Python-ideas] Proposal for default character representation Forgot to reply to all, duping my mesage... On 12 October 2016 at 23:48, M.-A. Lemburg wrote:

Hmm, in Python3, I get:

...

...

...

s = "абв.txt" s 'абв.txt'

I posted output with Python2 and Windows 7 BTW , In Windows 10 'print' won't work in cmd console at all by default with unicode but thats another story, let us not go into that. I think you get my idea right, it is not only about printing.

The hex notation for \uXXXX is a standard also used in many other programming languages, it's also easier to parse, so I don't think we should change this default.

In programming literature it is used often, but let me point out that decimal is THE standard and is much much better standard in sence of readability. And there is no solid reason to use 2 standards at the same time.

Take e.g.

...

...

...

s = "\u123456" s 'ሴ56'

With decimal notation, it's not clear where to end parsing the digit notation.

How it is not clear if the digit amount is fixed? Not very clear what did you mean. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 12 October 2016 at 23:50, Thomas Nyberg wrote:

Since when was decimal notation "standard"? Depends on what planet do you live. I live on planet Earth. And you?

opposite. For unicode representations, byte notation seems standard. How does this make it a good idea? Consider unicode table as an array with glyphs. Now the index of the array is suddenly represented in some obscure character set. How this index is other than index of any array or natural number? Think about it...

...

2. Mixing of two notations (hex and decimal) is a _very_ bad idea, I hope no need to explain why.

Still not sure which "mixing" you refer to.

Still not sure? These two words in brackets. Mixing those two systems.

On 13 October 2016 at 08:02, Greg Ewing wrote:

Mikhail V wrote:

...

Consider unicode table as an array with glyphs.

You mean like this one?

http://unicode-table.com/en/

Unless I've miscounted, that one has the characters arranged in rows of 16, so it would be *harder* to look up a decimal index in it.

-- Greg

Nice point finally, I admit, although quite minor. Where the data implies such pagings or alignment, the notation should be (probably) more binary-oriented. But: you claim to see bit patterns in hex numbers? Then I bet you will see them much better if you take binary notation (2 symbols) or quaternary notation (4 symbols), I guarantee. And if you take consistent glyph set for them also you'll see them twice better, also guarantee 100%. So not that the decimal is cool, but hex sucks (too big alphabet) and _the character set_ used for hex optically sucks. That is the point. On the other hand why would unicode glyph table which is to the biggest part a museum of glyphs would be necesserily paged in a binary-friendly manner and not in a decimal friendly manner? But I am not saying it should or not, its quite irrelevant for this particular case I think. Mikhail

Greg Ewing wrote:

#define O_RDONLY 0x0000 /* open for reading only */ #define O_WRONLY 0x0001 /* open for writing only */ #define O_RDWR 0x0002 /* open for reading and writing */ #define O_ACCMODE 0x0003 /* mask for above modes */

Good example. But it is not an average high level code of course. Example works again only if we for some reason follow binary segmentation which is bound to low level functionality, in this case 8 bit grouping.

If you have occasion to write a literal representing a character code, there's nothing to stop you writing it in hex to match the way it's shown in a repr(), or in published Unicode tables, etc.

...

...

c = "\u1235" if "\u1230" <= c <= "\u123f":

I don't see a need for any conversions back and forth.

I'll explain what I mean with an example. This is an example which I would make to support my proposal. Compare: if "\u1230" <= c <= "\u123f": and: o = ord (c) if 100 <= o <= 150: So yours is a valid code but for me its freaky, and surely I stick to the second variant. You said, I can better see in which unicode page I am by looking at hex ordinal, but I hardly need it, I just need to know one integer, namely where some range begins, that's it. Furthermore this is the code which would an average programmer better read and maintain. So it is the question of maintainability (+1). Secondly, for me it is the question of being able to type in and correct these decimal values: look, if I make a mistake, typo, or want to expand the range by some value I need to make summ and substract operation in my head to progress with my code effectively. Obviously nobody operates good with two notations in head simultanosly, so I will complete my code without extra effort. Is it clear now what I mean by conversions back and forth? This example alone actually explains my whole point very well, I feel however like being misunderstood or so.

...

I am not against base-16 itself in the first place, but rather against the character set which is simply visually inconsistent and not readable. Now you're talking about inventing new characters, or at least new glyphs for existing ones, and persuading everyone to use them. That's well beyond the scope of what Python can achieve!

Yes ideally one uses other glyphs for base-16 it does not however mean that one must use new invented glyphs. In standard ASCII there are enough glyphs that would work way better together, but it is too late anyway, should be better decided at the time of standard declaration. Got to love it.

The meaning of 0xC001 is much clearer to me than 1100000000000001, because I'd have to count the bits very carefully in the latter to distinguish it from, e.g. 6001 or 18001. The bits could be spaced out: 1100 0000 0000 0001 but that just takes up even more room to no good effect. I don't find it any faster to read -- if anything, it's

Greg, I feel somehow that you are an open minded person and I value this. You also can understand quite good how you read. What you refer to here is the brevity of the word Indeed there is some degrade of readability if the word is too big, or a font is set to big size, so you brake it, one step towards better. And now I'll explain you some further magic regarding the binary representation. If you find free time you can experiment a bit. So what is so peculiar about bitstring actually? Bitstring unlike higher bases can be treated as an abscence/presence of the signal, which is not possible for higher bases, literally binary string can be made almost "analphabetic" if one could say so. Consider such notation: instead of 1100 0000 0000 0001 you write ұұ-ұ ---- ---- ---ұ (NOTE: of course if you read this in non monospaced font you will not see it correct, I should make screenshots which I will do in a while) Note that I choose this letter not accidentally, this letter is similar to one of glyphs with peak readability. The unset value simply would be a stroke. So I take only one letter. ұұ-ұ ---- ---- ---ұ ---ұ ---ұ --ұ- -ұ-- --ұ- ---- ---- ---ұ ---- ---- --ұ- ---ұ -ұұ- ұұ-- ---- ---- ---- ---- ---- ---- --ұұ ---- ұ--- ---ұ -ұ-- --ұұ ---- ---ұ So the digits need not be equal-weighted as in higher bases. What does it bring? Simple: you can downscale the strings, so a 16-bit value would be ~60 pixels wide (for 96dpi display) without legibility loss, so it compensate the "too wide to scan" issue. And don't forget to make enough linespacing. Other benefits of binary string obviously: - nice editing feautures like bitshifting - very interesting cognitive features, (it becomes more noticable however if you train to work with it) ... So there is a whole bunch of good effects. Understand me right, I don't have reason not to believe you that you don't see any effect, but you should always remember that this can be simply caused by your habit. So if you are more than 40 years old (sorry for some familiarity) this can be really strong issue and unfortunately hardly changeable. It is not bad, it is natural thing, it is with everyone so.

When I say "instantly", I really do mean *instantly*. I fail to see how a different glyph set could reduce the recognition time to less than zero.

It is not about speed, it is about brain load. Chinese can read their hieroglyphs fast, but the cognition load on the brain is 100 times higher than current latin set. I know people who can read bash scripts fast, but would you claim that bash syntax can be any good compared to Python syntax?

Another point -- a string of hex digits is much easier for me to *remember* Could be, I personally can remember numbers in the above mentioned notation fotographically, opposed to decimal, where I also tend to speak it out to remember better, that is interesting, more of psychology however. Everyone is unique however in this sense.

Already noted, another good alternative for 8bit aligned data will be quoternary notation, it is 2x more compact and can be very legible due to few glyphs, it is also possible to emulate it with existing chars. Mikhail

On Fri, Oct 14, 2016 at 1:54 AM, Steven D'Aprano wrote:

...

and:

o = ord (c) if 100 <= o <= 150:

Which is clearly not the same thing, and better written as:

if "d" <= c <= "\x96": ...

Or, if you really want to use ord(), you can use hex literals: o = ord(c) if 0x64 <= o <= 0x96: ...

On Fri, Oct 14, 2016 at 10:56 PM, Random832 wrote:

On Fri, Oct 14, 2016, at 01:54, Steven D'Aprano wrote:

...

Good luck with that last one. Even if you could convince the Chinese and Japanese to swap to ASCII, I'd like to see you pry the emoji out of the young folk's phones.

This is actually probably the one part of this proposal that *is* feasible. While encoding emoji as a single character each makes sense for a culture that already uses thousands of characters; before they existed the English-speaking software industry already had several competing "standards" emerging for encoding them as sequences of ASCII characters.

:-) ChrisA

Steven D'Aprano wrote:

That's because some sequence of characters is being wrongly interpreted as an emoticon by the client software.

The only thing wrong here is that the client software is trying to interpret the emoticons. Emoticons are for *humans* to interpret, not software. Subtlety and cleverness is part of their charm. If you blatantly replace them with explicit images, you crush that. And don't even get me started on *animated* emoji... -- Greg

Mikhail V wrote:

if "\u1230" <= c <= "\u123f":

and:

o = ord (c) if 100 <= o <= 150:

Note that, if need be, you could also write that as if 0x64 <= o <= 0x96:

So yours is a valid code but for me its freaky, and surely I stick to the second variant.

The thing is, where did you get those numbers from in the first place? If you got them in some way that gives them to you in decimal, such as print(ord(c)), there is nothing to stop you from writing them as decimal constants in the code. But if you got them e.g. by looking up a character table that gives them to you in hex, you can equally well put them in as hex constants. So there is no particular advantage either way.

You said, I can better see in which unicode page I am by looking at hex ordinal, but I hardly need it, I just need to know one integer, namely where some range begins, that's it. Furthermore this is the code which would an average programmer better read and maintain.

To a maintainer who is familiar with the layout of the unicode code space, the hex representation of a character is likely to have some meaning, whereas the decimal representation will not. So for that person, using decimal would make the code *harder* to maintain. To a maintainer who doesn't have that familiarity, it makes no difference either way. So your proposal would result in a *decrease* of maintainability overall.

if I make a mistake, typo, or want to expand the range by some value I need to make summ and substract operation in my head to progress with my code effectively. Is it clear now what I mean by conversions back and forth?

Yes, but in my experience the number of times I've had to do that kind of arithmetic with character codes is very nearly zero. And when I do, I'm more likely to get the computer to do it for me than work out the numbers and then type them in as literals. I just don't see this as being anywhere near being a significant problem.

In standard ASCII there are enough glyphs that would work way better together,

Out of curiosity, what glyphs do you have in mind?

ұұ-ұ ---- ---- ---ұ

you can downscale the strings, so a 16-bit value would be ~60 pixels wide

Yes, you can make the characters narrow enough that you can take 4 of them in at once, almost as though they were a single glyph... at which point you've effectively just substituted one set of 16 glyphs for another. Then you'd have to analyse whether the *combined* 4-element glyphs were easier to disinguish from each other than the ones they replaced. Since the new ones are made up of repetitions of just two elements, whereas the old ones contain a much more varied set of elements, I'd be skeptical about that. BTW, your choice of ұ because of its "peak readibility" seems to be a case of taking something out of context. The readability of a glyph can only be judged in terms of how easy it is to distinguish from other glyphs. Here, the only thing that matters is distinguishing it from the other symbol, so something like "|" would perhaps be a better choice. ||-| ---- ---- ---|

So if you are more than 40 years old (sorry for some familiarity) this can be really strong issue and unfortunately hardly changeable.

Sure, being familiar with the current system means that it would take me some effort to become proficient with a new one. What I'm far from convinced of is that I would gain any benefit from making that effort, or that a fresh person would be noticeably better off if they learned your new system instead of the old one. At this point you're probably going to say "Greg, it's taken you 40 years to become that proficient in hex. Someone learning my system would do it much faster!" Well, no. When I was about 12 I built a computer whose only I/O devices worked in binary. From the time I first started toggling programs into it to the time I had the whole binary/hex conversion table burned into my neurons was maybe about 1 hour. And I wasn't even *trying* to memorise it, it just happened.

It is not about speed, it is about brain load. Chinese can read their hieroglyphs fast, but the cognition load on the brain is 100 times higher than current latin set.

Has that been measured? How? This one sets off my skepticism alarm too, because people that read Latin scripts don't read them a letter at a time -- they recognise whole *words* at once, or at least large chunks of them. The number of English words is about the same order of magnitude as the number of Chinese characters.

I know people who can read bash scripts fast, but would you claim that bash syntax can be any good compared to Python syntax?

For the things that bash was designed to be good for, yes, it can. Python wins for anything beyond very simple programming, but bash wasn't designed for that. (The fact that some people use it that way says more about their dogged persistence in the face of adversity than it does about bash.) I don't doubt that some sets of glyphs are easier to distinguish from each other than others. But the letters and digits that we currently use have already been pretty well optimised by scribes and typographers over the last few hundred years, and I'd be surprised if there's any *major* room left for improvement. Mixing up letters and digits is certainly jarring to many people, but I'm not sure that isn't largely just because we're so used to mentally categorising them into two distinct groups. Maybe there is some objective difference that can be measured, but I'd expect it to be quite small compared to the effect of these prior "habits" as you call them. -- Greg

On 14 October 2016 at 11:36, Greg Ewing wrote:

but bash wasn't designed for that. (The fact that some people use it that way says more about their dogged persistence in the face of adversity than it does about bash.)

I can not judge what bash is good for, since I never tried to learn it. But it *looks* indeed frightening. First feeling is OMG, I must close this and never see again. Also I can only hard imagine that special purpose of some language can ignore readability, even if it is assembler or whatever, it can be made readable without much effort. So I just look for some other solution for same task, let it be 10 times more code.

So for that person, using decimal would make the code *harder* to maintain. To a maintainer who doesn't have that familiarity, it makes no difference either way.

That is because that person from beginning (blindly) follows the convention. So my intention of course was not to find out if the majority does or not, but rather which one of two makes more sence *initially*, just trying to imagine that we can decide. To be more precise, if you were to choose between two options: 1. use hex for the glyph index and use hex for numbers (e.g. some arbitrary value like screen coordinates) 2. use decimal for both cases. I personally choose option 2. Probably nothing will convince me that option 1. will be better, all the more I don't believe that anything more than base-8 makes much sense for readable numbers. Just little bit dissapointed that others again and again speak of convention.

I just don't see this as being anywhere near being a significant problem.

I didn't mean that, it is just slightly annoys me.

...

In standard ASCII there are enough glyphs that would work way better together,

Out of curiosity, what glyphs do you have in mind?

If I were to decide, I would look into few options here: 1. Easy option which would raise less further questions is to take 16 first lowercase letters. 2. Better option would be to choose letters and possibly other glyphs to build up a more readable set. E.g. drop "c" letter and leave "e" due to their optical collision, drop some other weak glyphs, like "l" "h". That is of course would raise many further questions, like why you do you drop this glyph and not this and so on so it will surely end up in quarrel. Here lies another problem - non-constant width of letters, but this is more the problem of fonts and rendering, so adresses IDE and editors problematics. But as said I won't recommend base 16 at all.

...

ұұ-ұ ---- ---- ---ұ

you can downscale the strings, so a 16-bit value would be ~60 pixels wide

Yes, you can make the characters narrow enough that you can take 4 of them in at once, almost as though they were a single glyph... at which point you've effectively just substituted one set of 16 glyphs

No no. I didn't mean to shrink them till they melt together. The structure is still there, only that with such notation you don't need to keep the glyph so big as with many-glyph systems.

for another. Then you'd have to analyse whether the *combined* 4-element glyphs were easier to disinguish from each other than the ones they replaced. Since the new ones are made up of repetitions of just two elements, whereas the old ones contain a much more varied set of elements, I'd be skeptical about that.

I get your idea and this a very good point. Seems you have experience in such things? Currently I don't know for sure if such approach more effective or less than others and for what case. But I can bravely claim that it is better than *any* hex notation, it just follows from what I have here on paper on my table, namely that it is physically impossible to make up highly effective glyph system of more than 8 symbols. You want more only if really *need* more glyphs. And skepticism should always be present. One thing however especially interests me, here not only the differentiation of glyph comes in play, but also positional principle which helps to compare and it can be beneficaial for specific cases. So you can clearly see if one number is two times bigger than other for example. And of course, strictly speaking those bit groups are not glyphs, you can call them of course so, but this is just rhetorics. So one could call all english written words also glyphs but they are not really. But I get your analogy, this is how the tests should be made.

BTW, your choice of ұ because of its "peak readibility" seems to be a case of taking something out of context. The readability of a glyph can only be judged in terms of how easy it is to distinguish from other glyphs.

True and false. Each single taken glyph has a specific structure and put alone it has optical qualities. This is somewhat quite complicated and hardly describable by words, but anyway, only tests can tell what is better. In this case it is still 2 glyphs or better say one and a half glyph. And indeed you can distinguish them really good since they have different mass.

Here, the only thing that matters is distinguishing it from the other symbol, so something like "|" would perhaps be a better choice.

||-| ---- ---- ---|

I can get your idea, although not really correct statement, see above. A vertical stab is hardly a good glyph, actually quite a bad one. Such notation will cause quite uncomfortable effect on eyes, and there are many things here. Less technically, here is a rule: - a good glyph has a structure, and the boundary of the glyph is a proportional form (like a bulb) (not your case) - vertical gaps/sheers inside these boundaries are bad (your case). One can't always do without them, but vertical ones are much worse than horizontal. - too primitive glyph structure is bad (your case) So a stab is good only as some punctuation sign. For this exact reason such letters, as "l", "L", "i" are bad ones, especially their sans-serif variants. And *not* in the first place because they collide with other glyphs. This is somewhat non obvious. One should understand of course that I just took the standard symbols that only try to mimic the correct representation. So if sometime you will play around with bitstrings, here are the ASCII-only variants which are best working: -y-y ---y -yy- -y-- -o-o ---o -oo- -o-- -k-k ---k -kk- -k-- -s-s ---s -ss- -s-- No need to say that these will be way, way better than "01" notation which is used as standard. If you read a lot numbers you should have noticed how unpleasant is to scan through 010101

What I'm far from convinced of is that I would gain any benefit from making that effort, or that a fresh person would be noticeably better off if they learned your new system instead of the old one.

"far from convinced" sounds quite positive however :) it is never too late. I heard from Captain Crunch https://en.wikipedia.org/wiki/John_Draper That he was so tired of C syntax that he finally switched to Python for some projects. I can imagine how unwanted this can be in age. All depends on tasks that one often does. If say, imagine you'll read binaries for a long time, in one of notations I proposed above (like "-y-- --y-" for example) and after that try to switch back to "0100 0010" notation, I bet you will realize that better. Indeed learning new notation for numbers is quite easy, it is only some practice. And with base-2 you don't need learn at all, just can switch to other notation and use straight away.

...

It is not about speed, it is about brain load. Chinese can read their hieroglyphs fast, but the cognition load on the brain is 100 times higher than current latin set.

Has that been measured? How?

I don't think it is measurable at all. That is my opinion, and 100 just shows that I think it is very stressfull, also due to lot of meaning disambiguation that such system can cause. I also heard pesonal complains from chinese young students, they all had problems with vision already in early years, but I cannot support it oficially. So just imagine: if take for truth, max number of effective glyphs is 8. and hieroglyphs are *all* printed in same sized box! how would this provide efficient reading, and if you've seen chinese books, they all printed with quite small font. I am not very sentimental person but somehow feel sorry for people, one doesn't deserve it. You know, I become friends with one chinese girl, she loves to read and eager to learn and need to always carry pair of goglles with her everywhere. Somehow sad I become now writing it, she is so sweet young girl... And yes in this sence one can say that this cognition load can be measured. You go to universities in China and count those with vision problems.

I don't doubt that some sets of glyphs are easier to distinguish from each other than others. But the

That sounds good, this is not so often that one realizes that :) Most people would say "it's just matter of habit"

letters and digits that we currently use have already been pretty well optimised by scribes and typographers over the last few hundred years, and I'd be surprised if there's any *major* room left for improvement.

Here I would slightly disagree First, *Digits* are not optimised for anything, they are are just a heritage from ancient time. They have some minimal readability, namely "2" is not bad, others are quite poor. Second, *small latin letters* are indeed well fabricated. However don't have an illusion that someone cared much about their optimisation in last 1000 years. If you are skeptical about that, take a look at this http://daten.digitale-sammlungen.de/~db/bsb00003258/images/index.html?seite=... If believe (there are skeptics who do not believe) that this dates back end of 10th century, so we have an interesting picture here, You see that this is indeed very similar to what you read now, somewhat optimised of course, but without much improvements. Actually in some cases there is even some degradation: now we have "pbqd" letters, which are just rotation and reflection of each other, which is no good. Strictly speaking you can use only one of these 4 glyphs. And in last 500 hundred years there was zero modifications. How much improvent can be made is hard question. According to my results, indeed the peak readability forms are similar to certain small latin letters, But I would say quite significant improvement could be made. But this is not really measurable. Mikhail

On 15 October 2016 at 16:27, Chris Angelico wrote:

On Sun, Oct 16, 2016 at 12:06 AM, Mikhail V wrote:

...

But I can bravely claim that it is better than *any* hex notation, it just follows from what I have here on paper on my table, namely that it is physically impossible to make up highly effective glyph system of more than 8 symbols.

You should go and hang out with jmf. Both of you have made bold assertions that our current system is physically/mathematically

Who is jmf?

impossible, despite the fact that *it is working*. Neither of you can cite any actual scientific research to back your claims. No, please don't ask that. I have enough work in real life. And you tend to understand too literally my words here.

Mikhail

On 16 October 2016 at 02:58, Greg Ewing wrote:

...

even if it is assembler or whatever, it can be made readable without much effort.

You seem to be focused on a very narrow aspect of readability, i.e. fine details of individual character glyphs. That's not what we mean when we talk about readability of programs.

In this discussion yes, but layout aspects can be also improved and I suppose special purpose of language does not always dictate the layout of code, it is up to you who can define that also. And glyphs is not very narrow aspect, it is one of the fundamental aspects. Also it is much harder to develop than good layout, note that.

...

That is because that person from beginning (blindly) follows the convention.

What you seem to be missing is that there are *reasons* for those conventions. They were not arbitrary choices. Exactly, and in case of hex notation I fail to see how my proposal with using letters instead of what we have now, could be overseen at the time of decision. There must *very* solid reason for digits+letters against my variant, wonder what is it. Hope not that mono-width reason. And basic readability principles is somewhat that was clear for people 2000 years ago already.

So, if anything, *you're* the one who is "blindly following tradition" by wanting to use base 10.

Yes because when I was a child I learned it everywhere for everything, others too. As said I don't defend usage of base-10 as you can already note from my posts.

...

2. Better option would be to choose letters and

possibly other glyphs to build up a more readable set. E.g. drop "c" letter and leave "e" due to their optical collision, drop some other weak glyphs, like "l" "h". That is of course would raise many further questions, like why you do you drop this glyph and not this and so on so it will surely end up in quarrel.

Well, that's the thing. If there were large, objective, easily measurable differences between different possible sets of glyphs, then there would be no room for such arguments.

Those things cannot be easiliy measured, if at all, it requires a lot of tests and huge amount of time, you cannot plug measure device to the brain to precisely measure the load. In this case the only choice is to trust most experienced people who show the results which worked for them better and try self to implement and compare. Not saying you have special reason to trust me personally.

The fact that you anticipate such arguments suggests that any differences are likely to be small, hard to measure and/or subjective.

...

But I can bravely claim that it is better than *any* hex notation, it just follows from what I have here on paper on my table,

I think "on paper" is the important thing here. I suspect you are looking at the published results from some study or other and greatly overestimating the size of the effects compared to other considerations.

If you try to google that particular topic you'll see that there is zero related published material, there are tons of papers on readability, but zero concrete proposals or any attempts to develop something real. That is the thing. I would look in results if there was something. In my case I am looking at what I've achieved during years of my work on it and indeed there some interesting things there. Not that I am overestimating the role of it, but indeed it can really help in many cases, e.g, like in my example with bitstrings. Last but not the least, I am not a "paper ass" in any case, I try to keep only experimantal work where possible. Mikhail

On 16 October 2016 at 23:23, Greg Ewing wrote:

...

Those things cannot be easiliy measured, if at all,

If you can't measure something, you can't be sure it exists at all.

What do you mean I can't be sure? I am fully functional, mentally healthy man :)

Have you *measured* anything, though? Do you have any feel for how *big* the effects you're talking about are?

For what case, of course. So the difference for "0010 0011" and "--k- --kk" I can feel indeed big difference. Literally, I can read the latter clearly even I close my left eye and *fully* defocus my right eye. That is indeed a big difference and tells a lot. I suppose for disabled people this would be the only chance to see anything there. Currently I experiment myself and of course I plan to do it with experimental subjects. I plan one survey session in the end of November. But indeed this is very off-topic. So feel free to mail me, if anything. So back to hex notation, which is still not so off-topic I suppose.

...

There must *very* solid reason for digits+letters against my variant, wonder what is it.

The reasons only have to be *very* solid if there are *very* large advantages to the alternative you propose. My conjecture is that the advantages are

First ,I am the opinion that *initial* decision in such a case must be supported by solid reasons and not just like, "hey, John has already written them in such a manner, lets take it!". Second, I totally disagree that there always must be *very* large advantages for new standards, if we would follow such principle, we would still use cuneiform for writing or bash-like syntax, since everytime when someone proposes a slight improvement, there would be somebody who says : "but the new is not *that much* better than old!". Actually in many cases it is better when it is evolving - everybody is aware.

Here are some reasons in favour of the current system:

* At the point where most people learn to program, they are already intimately familiar with reading, writing and pronouncing letters and digits. * It makes sense to use 0-9 to represent the first ten digits, because they have the same numerical value.

So you mean they start to learn hex and see numbers and think like: ooo it looks like a number, not so scary. So less time to learn, yes, +1 (less pain now, more pain later) But if I am an adult intelligent man, I understand that there are only ten digits and I will despite need to extend the set and they *all* should be optically consequent and good readable. And what is a good readable set with >=16 glyphs? Small letters! Somewhat from the height of my current knowledge, since I know that digits anyway not very good readable.

* Using a consecutive sequence of letters makes sense because we're already familiar with their ordering.

I actually proposed consecutive, but that does not make much difference: being familiar with ordering of the alphabet will have next to zero influence on the reading of numbers encoded with letters, it is just an illusion that it will, since the letter is a symbol, if I see "z" I don't think of 26. More probably, the weight of the glyph could play some role, that means less the weight - less the number. Mikhail

On 17 October 2016 at 02:23, Steven D'Aprano wrote:

On Sun, Oct 16, 2016 at 05:02:49PM +0200, Mikhail V wrote:

...

In this discussion yes, but layout aspects can be also improved and I suppose special purpose of language does not always dictate the layout of code, it is up to you who can define that also. And glyphs is not very narrow aspect, it is one of the fundamental aspects. Also it is much harder to develop than good layout, note that.

This discussion is completely and utterly off-topic for this mailing list. If you want to discuss changing the world to use your own custom character set for all human communication, you should write a blog or a book. It is completely off-topic for Python: we're interested in improving the Python programming language, not yet another constructed language or artifical alphabet:

You're right, I was just answering the questions so it came to other thing somehow. BTW, among others we have discussed bitstring representation. So if you work with those, for example if you model cryptography algorithms or similar things in Python, this could help you for example to debug your programs and generally one could interpret it as, say, how is about adding an extra notation for this sake. And if you noticed this is not really about my glyphs, but lies in ASCII. So actually it is me who tried to turn it back to on-topic. Mikhail

On 10/12/2016 07:13 PM, Mikhail V wrote:

On 12 October 2016 at 23:50, Thomas Nyberg wrote:

...

Since when was decimal notation "standard"? Depends on what planet do you live. I live on planet Earth. And you?

If you mean that decimal notation is the standard used for _counting_ by people, then yes of course that is standard. But decimal notation certainly is not standard in this domain.

...

opposite. For unicode representations, byte notation seems standard. How does this make it a good idea? Consider unicode table as an array with glyphs. Now the index of the array is suddenly represented in some obscure character set. How this index is other than index of any array or natural number? Think about it...

Hexadecimal notation is hardly "obscure", but yes I understand that fewer people understand it than decimal notation. Regardless, byte notation seems standard for unicode and unless you can convince the unicode community at large to switch, I don't think it makes any sense for python to switch. Sometimes it's better to go with the flow even if you don't want to.

...

...

2. Mixing of two notations (hex and decimal) is a _very_ bad idea, I hope no need to explain why.

Still not sure which "mixing" you refer to.

Still not sure? These two words in brackets. Mixing those two systems.

There is not mixing for unicode in python; it displays as hexadecimal. Decimal is used in other places though. So if by "mixing" you mean python should not use the standard notations of subdomains when working with those domains, then I would totally disagree. The language used in different disciplines is and has always been variable. Until that's no longer true it's better to stick with convention than add inconsistency which will be much more confusing in the long-term than learning the idiosyncrasies of a specific domain (in this case the use of hexadecimal in the unicode world). Cheers, Thomas