After Glyph and Alex's email about their asks for assisting in writing
Python 2/3 code, it got me thinking about where in the toolchain various
warnings and such should go in order to help direct energy to help develop
whatever future toolchain to assist in porting.
There seems to be three places where issues are/can be caught once a
project has embarked down the road of 2/3 source compatibility:
1. -3 warnings
2. Some linter tool
3. Failing tests
-3 warnings are things that we know are flat-out wrong and do not cause
massive compatibility issues in the stdlib. For instance, warning that
buffer() is not in Python 3 is a py3k warning -- Glyph made a mistake when
he asked for it as a new warning -- is a perfect example of something that
isn't excessively noisy and won't cause issues when people run with it.
But what about warning about classic classes? The stdlib is full of them
and they were purposefully left alone for compatibility reasons. But there
is a subtle semantic difference between classic and new-style classes, and
so 2/3 code should consider switching (this is when people chime in saying
"this is why we want a 2.8 release!", but that still isn't happening). If
this were made a py3k warning in 2.7 then the stdlib itself would spew out
warnings which we can't change due to compatibility, so that makes it not
useful (http://bugs.python.org/issue21231). But as part of a lint tool
specific to Python 2.7 that kind of warning would not be an issue and is
easily managed and integrated into CI setups to make sure there are no
Lastly, there are things like string/unicode comparisons.
http://bugs.python.org/issue21401 has a patch from VIctor which warns when
comparing strings and unicode in Python 2.7. Much like the classic classes
example, the stdlib becomes rather noisy due to APIs that handle either/or,
etc. But unlike the classic classes example, you just can't systematically
verify that two variables are always going to be str vs. unicode in Python
2.7 if they aren't literals. If people want to implement type constraint
graphs for 2.7 code to help find them then that's great, but I personally
don't have that kind of time. In this instance it would seem like relying
on a project's unit tests to find this sort of problem is the best option.
With those three levels in mind, where do we draw the line between these
levels? Take for instance the print statement. Right now there is no
warning with -3. Do we add one and then update the 2.7 stdlib to prevent
warnings being generated by the stdlib? Or do we add it to some linter tool
to pick up when people accidentally leave one in their code?
The reason I ask is since this is clear I'm willing to spearhead the
tooling work we talked about at the language summit to make sure there's a
clear path for people wanting to port which is as easy as (reasonably)
possible, but I don't want to start on it until I have a clear indication
of what people are going to be okay with.
There is a discussion over at MicroPython about the internal
representation of Unicode strings. Micropython is aimed at embedded
devices, and so minimizing memory use is important, possibly even
more important than performance.
(I'm not speaking on their behalf, just commenting as an interested
At the moment, their Unicode support is patchy. They are talking about
* Having a build-time option to restrict all strings to ASCII-only.
(I think what they mean by that is that strings will be like Python 2
strings, ASCII-plus-arbitrary-bytes, not actually ASCII.)
* Implementing Unicode internally as UTF-8, and giving up O(1)
Would either of these trade-offs be acceptable while still claiming
"Python 3.4 compatibility"?
My own feeling is that O(1) string indexing operations are a quality of
implementation issue, not a deal breaker to call it a Python. I can't
see any requirement in the docs that str[n] must take O(1) time, but
perhaps I have missed something.
I added a new BaseEventLoop.is_closed() method to Tulip and Python 3.5
to fix an issue (see Tulip issue 169 for the detail). The problem is
that I don't want to add this method to Python 3.4 because usually we
don't add new methods in minor versions of Python (future version
3.4.2 in this case).
Guido just wrote in the issue: "Actually for asyncio we have special
dispensation to push new features to minor releases (until 3.5).
Please push to 3.4 so the source code is the same everywhere (except
selectors.py, which is not covered by the exception)."
I disagree with Guido. I would prefer to start to maintain a different
branch for Python 3.4, because I consider that only bugfixes should be
applied to Python 3.4.
It's not the first change that cannot be applied on Python 3.4 (only
in Tulip and Python 3.5): the selectors module now also supports
devpoll on Solaris. It's annoying because the Tulip script
"update_stdlib.sh" used to synchronize Tulip and Python wants to
replace Lib/selectors.py in Python 3.4. I have to revert the change each time.
I propose a new workflow: use Python default (future version 3.5) as
the new asyncio "upstream". Bugfixes would be applied as other Python
bugfixes: first in Python 3.4, than in Python 3.5. The
"update_stdlib.sh" script of Tulip should be modified to copy files
from Python default to Tulip (opposite of the current direction).
New feature: Python 3.5 => Tulip => Trollius
Bugfix: Python 3.4 => Python 3.5 => Tulip => Trollius
I don't think that Tulip should have minor release just for bugfixes,
it would be a pain to maintain. Tulip is a third party module, it
doesn't have the same constraints than Python stdlib.
What do you think?
Would it be possible to add a new "Asyncio" component on
bugs.python.org? If this component is selected, the default nosy list
for asyncio would be used (guido, yury and me, there is already such
list in the nosy list completion).
Full text search for "asyncio" returns too many results.
I think I know the answer to this, but I'm going to ask it anyway...
I know that there is a general policy of trying to write code in the
standard library that does not disadvantage other implementations. How
far does that go the other way? Should the standard library accept
slower code because it will be much faster in other implementations?
Briefly, I have a choice of algorithm for the median function in the
statistics module. If I target CPython, I will use a naive but simple
O(N log N) implementation based on sorting the list and returning the
middle item. (That's what the module currently does.) But if I target
PyPy, I will use an O(N) algorithm which knocks the socks off the naive
version even for smaller lists. In CPython that's typically 2-5 times
slower; in PyPy it's typically 3-8 times faster, and the bigger the data
set the more the advantage.
For the specific details, see http://bugs.python.org/issue21592
My feeling is that the CPython standard library should be written for
CPython, that is, it should stick to the current naive implementation of
median, and if PyPy wants to speed the function up, they can provide
their own version of the module. I should *not* complicate the
implementation by trying to detect which Python the code is running
under and changing algorithms accordingly. However, I should put a
comment in the module pointing at the tracker issue. Does this sound
right to others?
- micropython is designed to run on a machine with 192 kilobytes of
RAM and perhaps a megabyte of FLASH. The controller can execute
read-only code directly from FLASH. There is no dynamic linker in this
environment. (It also has a UNIX port).
- However it does include a full Python parser and REPL, so the board
can be programmed without a separate computer as opposed to, say,
having to upload bytecode compiled on a regular computer.
- It's definitely going to be a subset of Python. For example,
func.__name__ is not supported - to make it more micro?
- They have a C API. It is much different than the CPython C API.
- It mas more than one code emitter. A certain decorator causes a
function to be compiled to ARM Thumb code instead of bytecode.
- It even has an inline assembler than translates Python-syntax ARM
assembly (to re-use the same parser) into machine code.
Most information from
I'm helping out with the micropython project and am finding that one
of their tests fails on CPython 3.5 (fresh build from Mercurial this
morning). It comes down to this:
Python 3.4.1rc1 (default, May 5 2014, 14:28:34)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
Python 3.5.0a0 (default:88814d1f8c32, Jun 4 2014, 07:29:32)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: %x format: an integer is required, not float
Is this an intentional change? And if so, is it formally documented
somewhere? I don't recall seeing anything about it, but my
recollection doesn't mean much.
Stumbling over problems on AIX (Modules/python.exp not found) building libxml2 as python module
let me wonder about the intended use-cases for 'python-config' and 'pkg-config python'.
FWIW, I can see these distinct use cases here, and I'm kindly asking if I got them right:
* Build an application containing a python interpreter (like python$EXE itself):
+ link against libpython.so
+ re-export symbols from libpython.so for python-modules (platform-specific)
+ This is similar to build against any other library, thus
= 'python.pc' is installed (for 'pkg-config python').
* Build a python-module (like build/lib.<platform>-<pyver>/*.so):
+ no need to link against libpython.so, instead
+ expect symbols from libpython.so to be available at runtime, platform-specific either as
+ undefined symbols at build-time (Linux, others), or
+ a list of symbols to import from "the main executable" (AIX)
+ This is specific to python-modules, thus
= 'python-config' is installed.
I have two areas of questions about updating turtle.py. First the module
itself, then a turtle tracker issue versus code cleanup policies.
A. Unlike most stdlib modules, turtle is copyrighted and licensed by an
# turtle.py: a Tkinter based turtle graphics module for Python
# Version 1.1b - 4. 5. 2009
# Copyright (C) 2006 - 2010 Gregor Lingl
# email: glingl(a)aon.at
I am not sure what the copyright covers other than the exact text
contributed, with updates, by Gregor. It certainly does not cover the
API and whatever code he copied from the previous version (unless that
was also by him, and I have no idea how much he copied when
reimplementing). I don't think it should cover additions made by others
either. Should there be another line to cover these?
# Permission is granted to anyone to use this software for any purpose,
# including commercial applications, and to alter it and redistribute it
# freely, subject to the following restrictions:
# 1. The origin of this software must not be misrepresented; you must not
# claim that you wrote the original software. If you use this software
# in a product, an acknowledgment in the product documentation would be
# appreciated but is not required.
# 2. Altered source versions must be plainly marked as such, and must not be
# misrepresented as being the original software.
# 3. This notice may not be removed or altered from any source distribution.
As to point 2, the source has been altered a bit (by others) but it is
not marked as such. How should it be?
_ver = "turtle 1.1b- - for Python 3.1 - 4. 5. 2009"
Obsolete; delete or alter (how)?
When this replaced the previous turtle.py, it was considered 'owned' by
Gregor in that changes had to go through him. However, he became
inactive soon after and maintenance ceased. There has been only one
turtle-specific code change that I know of (by Ned Daily, a month ago,
for OSX. So is turtle.py unpatchable by anyone or fair game for anyone?
A particular example: Gregor added intermediate layers to isolate turtle
from tkinter. (He had a plan to add other backends, but I believe he
abandoned that.) If someone wanted to reduce the layering and make the
code easier to understand and maintain, while speeding execution, would
that be allowed now?
B. Lets assuming that turtle.py is, at least to some degree, fair game
for fixes and enhancements. PSF Python PyLadies (Jessica Keller, Lynn
Root) are participating in the 2014 GNOME Outreach Program for Women
(OPW) https://wiki.python.org/moin/OPW/2014 . One of the projects
(bottem of that page) is Graphical Python, in particular Turtle.
A few days ago, Jessica posted
http://bugs.python.org/issue21573 Clean up turtle.py code formatting
"Lib/turtle.py has some code formatting issues. Let's clean them up to
make the module easier to read as interns start working on it this
summer." She want to follow cleanup with code examination, fixes, and
Responding today, I cautioned that clean-up only patches, such as she
apparently would like to start with, are not in favor. But I cannot say
how the policy applies to her proposal. Does the 'promise' of later work
on the code make preliminary clean-up ok?
Since she only marked the issue for 3.5, I also cautioned that 3.5-only
cleanups would make fixing bugs in other issues harder. Is the code
clean-up policy the same for all branches?
Test? you ask?. There are apparently no unittests for turtle. On the
other hand, turtledemo tests the overall function of the module and of
many (most) of the top-level turtle functions.
Terry Jan Reedy