From ndbecker2 at gmail.com Tue Jun 9 14:00:31 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 09 Jun 2015 08:00:31 -0400
Subject: [Python-porting] overiding builtins breaks line_profiler
Message-ID: <ml6kgv$6f5$1@ger.gmane.org>
I am sad that future installs a 'builtins' package. This breaks
line_profiler.
I wish future installation could be more modular - to selectively install
modules as needed. Or somehow be more selective about overriding standard
packages.
From ndbecker2 at gmail.com Tue Jun 9 15:49:55 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 09 Jun 2015 09:49:55 -0400
Subject: [Python-porting] futurize - strange traceback
Message-ID: <ml6qu4$oml$1@ger.gmane.org>
futurize --stage2 -x fix_division --add-suffix=3 -n -w
test_unframed_multi_acic.py
Traceback (most recent call last):
File "/home/nbecker/.local/bin/futurize", line 9, in <module>
load_entry_point('future==0.14.3', 'console_scripts', 'futurize')()
File "/home/nbecker/.local/lib/python2.7/site-
packages/libfuturize/main.py", line 293, in main
options.processes)
File "/usr/lib64/python2.7/lib2to3/refactor.py", line 706, in refactor
items, write, doctests_only)
File "/usr/lib64/python2.7/lib2to3/refactor.py", line 301, in refactor
self.refactor_file(dir_or_file, write, doctests_only)
File "/usr/lib64/python2.7/lib2to3/refactor.py", line 747, in
refactor_file
*args, **kwargs)
File "/usr/lib64/python2.7/lib2to3/refactor.py", line 358, in
refactor_file
write=write, encoding=encoding)
File "/usr/lib64/python2.7/lib2to3/refactor.py", line 524, in
processed_file
self.write_file(new_text, filename, old_text, encoding)
File "/usr/lib64/python2.7/lib2to3/main.py", line 84, in write_file
os.makedirs(output_dir)
File "/usr/lib64/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 2] No such file or directory: ''
Also, -x fix_division isn't working. I want it to stop converting every a/b
into old_div (a, b)
From clay.gerrard at gmail.com Thu Jun 11 01:33:30 2015
From: clay.gerrard at gmail.com (Clay Gerrard)
Date: Wed, 10 Jun 2015 16:33:30 -0700
Subject: [Python-porting] python-future and six
Message-ID: <CA+_JKzqxUnXqWKPyWbKH0eC3u9rK3at92_BA_301+aVnMCfn-A@mail.gmail.com>
OpenStack [1] has been using six to handle python 2/3 compatibility in a
number of places and is making good progress! With eventlet now running a
python2/3 compatible code base (Segrey went with the just inline six
approach [2]) things are really starting to move quickly! Lots of patches
coming up with stuff imported from six.moves and isinstance
six.strings_type everywhere - not so much six.b and six.u though - as the
OpenStack projects already targets >=2.7 or >=3.3 (maybe even >=3.4?).
But, I'm finding myself wondering what we're going to do with all these
references to six in our code base once we drop python 2.7 support? Is it
worth it [3]?
Also I was looking at Brett Cannon's updates to the official porting guide
[4] which now references *both* six and python-future [5]? I sorta like
the idea that python-future might let me write more or less idiomatic
python3 code* and it will just work on python2?
* as idiomatic as writing python 1.6 code felt on python 2.3 anyway :)
So I've got at least these many choices:
* modernize + roll your own compat layer for stdlib imports
* inline six
* depend on six
* depend on python-future
OpenStack so far seems to have picked "depend on six" so it's probably
easiest to stick with the momentum - unless the momentum is heading us
towards a cliff?
Is there any hope the experts on the state of the art could tell me the
"one-- and preferably only one --obvious way to do it"? :)
-Clay
1. https://github.com/openstack
2.
https://bitbucket.org/eventlet/eventlet/src/80e4f11d037aaca81f9e0baf636282fd1e65236c/eventlet/support/six.py?at=default
3. http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/
4.
https://mail.python.org/pipermail/python-porting/2014-December/000477.html
5. http://python-future.org/overview.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-porting/attachments/20150610/327daeca/attachment.html>
From regebro at gmail.com Thu Jun 11 04:48:46 2015
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 11 Jun 2015 04:48:46 +0200
Subject: [Python-porting] python-future and six
In-Reply-To: <CA+_JKzqxUnXqWKPyWbKH0eC3u9rK3at92_BA_301+aVnMCfn-A@mail.gmail.com>
References: <CA+_JKzqxUnXqWKPyWbKH0eC3u9rK3at92_BA_301+aVnMCfn-A@mail.gmail.com>
Message-ID: <CAL0kPAXcAz_iK3oOgJTKt7pC23a7UbVeiRbkgcGZCWxHb2VC9g@mail.gmail.com>
Getting rid of the six dependency with for 90% of it's use just be a
question of doing search and replace, replaceing the six style with a
Python 3 style, so I don't think it's a problem.
Sure, python-future will, at least in theory be less work to drop, but
in any case I don't see it as being a lot of work. And since it can be
done module by module it can simply be done as a low priority cleanup
task, and could probably be handed over to people who want to
contribute but are new to the codebase, leaving less work for the core
devs. :-)
On Thu, Jun 11, 2015 at 1:33 AM, Clay Gerrard <clay.gerrard at gmail.com> wrote:
> OpenStack [1] has been using six to handle python 2/3 compatibility in a
> number of places and is making good progress! With eventlet now running a
> python2/3 compatible code base (Segrey went with the just inline six
> approach [2]) things are really starting to move quickly! Lots of patches
> coming up with stuff imported from six.moves and isinstance six.strings_type
> everywhere - not so much six.b and six.u though - as the OpenStack projects
> already targets >=2.7 or >=3.3 (maybe even >=3.4?).
>
> But, I'm finding myself wondering what we're going to do with all these
> references to six in our code base once we drop python 2.7 support? Is it
> worth it [3]?
>
> Also I was looking at Brett Cannon's updates to the official porting guide
> [4] which now references *both* six and python-future [5]? I sorta like the
> idea that python-future might let me write more or less idiomatic python3
> code* and it will just work on python2?
>
> * as idiomatic as writing python 1.6 code felt on python 2.3 anyway :)
>
> So I've got at least these many choices:
>
> * modernize + roll your own compat layer for stdlib imports
> * inline six
> * depend on six
> * depend on python-future
>
> OpenStack so far seems to have picked "depend on six" so it's probably
> easiest to stick with the momentum - unless the momentum is heading us
> towards a cliff?
>
> Is there any hope the experts on the state of the art could tell me the
> "one-- and preferably only one --obvious way to do it"? :)
>
> -Clay
>
> 1. https://github.com/openstack
> 2.
> https://bitbucket.org/eventlet/eventlet/src/80e4f11d037aaca81f9e0baf636282fd1e65236c/eventlet/support/six.py?at=default
> 3. http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/
> 4.
> https://mail.python.org/pipermail/python-porting/2014-December/000477.html
> 5. http://python-future.org/overview.html
>
> _______________________________________________
> Python-porting mailing list
> Python-porting at python.org
> https://mail.python.org/mailman/listinfo/python-porting
>
From mal at egenix.com Thu Jun 11 09:27:09 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 11 Jun 2015 09:27:09 +0200
Subject: [Python-porting] python-future and six
In-Reply-To: <CA+_JKzqxUnXqWKPyWbKH0eC3u9rK3at92_BA_301+aVnMCfn-A@mail.gmail.com>
References: <CA+_JKzqxUnXqWKPyWbKH0eC3u9rK3at92_BA_301+aVnMCfn-A@mail.gmail.com>
Message-ID: <5579384D.1000008@egenix.com>
On 11.06.2015 01:33, Clay Gerrard wrote:
> OpenStack [1] has been using six to handle python 2/3 compatibility in a
> number of places and is making good progress! With eventlet now running a
> python2/3 compatible code base (Segrey went with the just inline six
> approach [2]) things are really starting to move quickly! Lots of patches
> coming up with stuff imported from six.moves and isinstance
> six.strings_type everywhere - not so much six.b and six.u though - as the
> OpenStack projects already targets >=2.7 or >=3.3 (maybe even >=3.4?).
>
> But, I'm finding myself wondering what we're going to do with all these
> references to six in our code base once we drop python 2.7 support? Is it
> worth it [3]?
>
> Also I was looking at Brett Cannon's updates to the official porting guide
> [4] which now references *both* six and python-future [5]? I sorta like
> the idea that python-future might let me write more or less idiomatic
> python3 code* and it will just work on python2?
>
> * as idiomatic as writing python 1.6 code felt on python 2.3 anyway :)
>
> So I've got at least these many choices:
>
> * modernize + roll your own compat layer for stdlib imports
> * inline six
> * depend on six
> * depend on python-future
>
> OpenStack so far seems to have picked "depend on six" so it's probably
> easiest to stick with the momentum - unless the momentum is heading us
> towards a cliff?
>
> Is there any hope the experts on the state of the art could tell me the
> "one-- and preferably only one --obvious way to do it"? :)
As you can see from the development of available tools, there
is no obvious way to do it :-)
In the early days, people suggested doing the conversion to
Python 3 once and then forget about Python 2 or maintain two
branches. Didn't work out.
Then the idea was to continue writing Python 2 code
and use 2to3 during installation to create the Python 3
code. Resulted in very poor Python 3 code and poor Python 2
code due to the workarounds needed to make the automatic
conversion work. Way too fragile.
Next was six to help write new code which works in both Python 2
and 3; and later python-modernize and similar tools to help
convert Python 2 to runs-on-both-code.
Then people were concerned about a possible extra future effort
to drop Python 2 support in their code base. python-future
was born.
IMO, Armin Ronacher's approach works best if you have a
large code base and fans in both Python 2 and 3 camps.
Using various bits from six in your compat layer helps,
but be careful not to introduce too many indirections
to not slow down your code too much.
If you're writing Python 3 code mostly, you may want to
consider using python-future to also ship a Python 2
port.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jun 11 2015)
>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2015-07-20: EuroPython 2015, Bilbao, Spain ... 39 days to go
::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
From clay.gerrard at gmail.com Fri Jun 19 23:41:59 2015
From: clay.gerrard at gmail.com (Clay Gerrard)
Date: Fri, 19 Jun 2015 14:41:59 -0700
Subject: [Python-porting] json encoding
Message-ID: <CA+_JKzp1Z5Kqojb6dyTDz3VQdfokcEbugQ1ao8Ds5YKw_Oe2XA@mail.gmail.com>
Has anyone ported any projects that use json as a data interchange format -
or can think of any that probably had to port some json handling code?
I'm perplexed that python3's json.dumps returns a str. I mean I know it's
dump*s* - but that was from back when we only had binary strings ;)
My reading leads me to believe that the json format is not a "string" - its
a binary format - encoded in utf-8 [1]
Why would want to first create a unicode string - then encode it to utf-8?
What else are you doing with this this data exchange format if not using it
to exchange the representation of the data encoded as bytes to somewhere
else?
json_data = json.dumps(my_object)
if not isinstance(json_data, six.bytes_type):
json_data = json_data.encode('utf-8')
my_buffer = BytesIO(json_data)
^ Is that really the best I can do?
Why did json.dumps in python3 [3] loose the encoding kwarg from python2 [2].
Thanks for any guidance or suggestion - maybe I'm just thinking about it
wrong.
-Clay
1. https://tools.ietf.org/html/rfc7159#section-8.1
3. https://docs.python.org/3.2/library/json.html#json.dumps
2. https://docs.python.org/2/library/json.html#json.dumps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-porting/attachments/20150619/96b55332/attachment.html>
From rdmurray at bitdance.com Sat Jun 20 01:18:00 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 19 Jun 2015 19:18:00 -0400
Subject: [Python-porting] json encoding
In-Reply-To: <CA+_JKzp1Z5Kqojb6dyTDz3VQdfokcEbugQ1ao8Ds5YKw_Oe2XA@mail.gmail.com>
References: <CA+_JKzp1Z5Kqojb6dyTDz3VQdfokcEbugQ1ao8Ds5YKw_Oe2XA@mail.gmail.com>
Message-ID: <20150619231801.5C1D7B30003@webabinitio.net>
On Fri, 19 Jun 2015 14:41:59 -0700, Clay Gerrard <clay.gerrard at gmail.com> wrote:
> Has anyone ported any projects that use json as a data interchange format -
> or can think of any that probably had to port some json handling code?
>
> I'm perplexed that python3's json.dumps returns a str. I mean I know it's
> dump*s* - but that was from back when we only had binary strings ;)
>
> My reading leads me to believe that the json format is not a "string" - its
> a binary format - encoded in utf-8 [1]
Well, utf-8 is the wire encoding, but the *intent* (as I understand
it) is that json is a unicode string.
> Why would want to first create a unicode string - then encode it to utf-8?
> What else are you doing with this this data exchange format if not using it
> to exchange the representation of the data encoded as bytes to somewhere
> else?
>
> json_data = json.dumps(my_object)
> if not isinstance(json_data, six.bytes_type):
> json_data = json_data.encode('utf-8')
> my_buffer = BytesIO(json_data)
>
> ^ Is that really the best I can do?
>
> Why did json.dumps in python3 [3] loose the encoding kwarg from python2 [2].
Because json is a string. You encode it to utf-8 using the string
encode method when you put it on the wire. It's not json if its not
unicode or utf-8 (or -16 or -32), so it wouldn't make sense to have an
encoding keyword that can take an arbitrary codec.
> Thanks for any guidance or suggestion - maybe I'm just thinking about it
> wrong.
I haven't done any code that used json and needed to run on both python2
and python3, so I can't tell you if there is an easier way. But in general
you do have to jump through some hoops when supporting both dialects.
--David
From armin.ronacher at active-4.com Sat Jun 20 01:27:14 2015
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sat, 20 Jun 2015 01:27:14 +0200
Subject: [Python-porting] json encoding
In-Reply-To: <20150619231801.5C1D7B30003@webabinitio.net>
References: <CA+_JKzp1Z5Kqojb6dyTDz3VQdfokcEbugQ1ao8Ds5YKw_Oe2XA@mail.gmail.com>
<20150619231801.5C1D7B30003@webabinitio.net>
Message-ID: <5584A552.3000607@active-4.com>
Hi,
On 20/06/2015 01:18, R. David Murray wrote:
> I haven't done any code that used json and needed to run on both python2
> and python3, so I can't tell you if there is an easier way. But in general
> you do have to jump through some hoops when supporting both dialects.
There definitely is. Look at flask.json for a wrapper around the
stdlib's JSON module which supports bytes in more places. This also is
necessary to make code work with both json and simplejson which have
different behavior for unicode support.
For the curious:
https://github.com/mitsuhiko/flask/blob/master/flask/json.py
Regards,
Armin
From mal at egenix.com Sat Jun 20 10:24:24 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 20 Jun 2015 10:24:24 +0200
Subject: [Python-porting] json encoding
In-Reply-To: <20150619231801.5C1D7B30003@webabinitio.net>
References: <CA+_JKzp1Z5Kqojb6dyTDz3VQdfokcEbugQ1ao8Ds5YKw_Oe2XA@mail.gmail.com>
<20150619231801.5C1D7B30003@webabinitio.net>
Message-ID: <55852338.3030207@egenix.com>
On 20.06.2015 01:18, R. David Murray wrote:
> On Fri, 19 Jun 2015 14:41:59 -0700, Clay Gerrard <clay.gerrard at gmail.com> wrote:
>> Has anyone ported any projects that use json as a data interchange format -
>> or can think of any that probably had to port some json handling code?
>>
>> I'm perplexed that python3's json.dumps returns a str. I mean I know it's
>> dump*s* - but that was from back when we only had binary strings ;)
>>
>> My reading leads me to believe that the json format is not a "string" - its
>> a binary format - encoded in utf-8 [1]
>
> Well, utf-8 is the wire encoding, but the *intent* (as I understand
> it) is that json is a unicode string.
>
>> Why would want to first create a unicode string - then encode it to utf-8?
>> What else are you doing with this this data exchange format if not using it
>> to exchange the representation of the data encoded as bytes to somewhere
>> else?
>>
>> json_data = json.dumps(my_object)
>> if not isinstance(json_data, six.bytes_type):
>> json_data = json_data.encode('utf-8')
>> my_buffer = BytesIO(json_data)
>>
>> ^ Is that really the best I can do?
>>
>> Why did json.dumps in python3 [3] loose the encoding kwarg from python2 [2].
>
> Because json is a string. You encode it to utf-8 using the string
> encode method when you put it on the wire. It's not json if its not
> unicode or utf-8 (or -16 or -32), so it wouldn't make sense to have an
> encoding keyword that can take an arbitrary codec.
The encoding argument in Python 2 refers to the encoding to use
for binary strings in the object you pass to json.dumps():
https://docs.python.org/2/library/json.html#json.dump
"""
encoding is the character encoding for str instances, default is UTF-8.
"""
In Python 3, bytes are not supported by the json module, so you
don't need the encoding parameter anymore.
Python 2's json will convert all strings to Unicode on input
and then write out a UTF-8 encoded string or, when using
ensure_ascii, an ASCII string with escape sequence.
>> Thanks for any guidance or suggestion - maybe I'm just thinking about it
>> wrong.
>
> I haven't done any code that used json and needed to run on both python2
> and python3, so I can't tell you if there is an easier way. But in general
> you do have to jump through some hoops when supporting both dialects.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jun 20 2015)
>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78
2015-06-10: Released mxODBC Plone/Zope DA 2.2.2 http://egenix.com/go76
2015-07-20: EuroPython 2015, Bilbao, Spain ... 30 days to go
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
From tim_eichler at hotmail.com Thu Jun 18 13:16:46 2015
From: tim_eichler at hotmail.com (Tim eichler)
Date: Thu, 18 Jun 2015 21:16:46 +1000
Subject: [Python-porting] freezing when using future
Message-ID: <SNT147-W611311D484FD1F81E9145A9CA50@phx.gbl>
I am using cx freeze to bundle my python software.
running futurize my code changed from
import urllib2
to
from future import standard_library
standard_library.install_aliases()
import urllib.request, urllib.error, urllib.parse
when freezing on python2 its having problems finding the parent path of urllib.
I couldn't find much on the site in this regards so i'd like to know if there is any documentation in this area.
Otherwise i'll play around and see if i can find a nice workaround.
Kind Regards,
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-porting/attachments/20150618/ce090016/attachment.html>
From tim_eichler at hotmail.com Tue Jun 30 17:37:45 2015
From: tim_eichler at hotmail.com (Tim eichler)
Date: Wed, 1 Jul 2015 01:37:45 +1000
Subject: [Python-porting] freezing when using future
In-Reply-To: <SNT147-W611311D484FD1F81E9145A9CA50@phx.gbl>
References: <SNT147-W611311D484FD1F81E9145A9CA50@phx.gbl>
Message-ID: <SNT147-W862E8580B02383141239F49CA90@phx.gbl>
I developed some tests that show where the problem are.
The problem is present with cx freeze and py2exe but not with setuptools or distutils.
By including the future module and excluding around 50 sub modules from urllib and a handful from the collections module i was able to get freezing to work.
Is it possible for this to be fixed on the future end or should i see if i can get the guys at cx freeze or py2exe to include my patches?
Kind Regards,
Tim
From: tim_eichler at hotmail.com
To: python-porting at python.org
Date: Thu, 18 Jun 2015 21:16:46 +1000
Subject: [Python-porting] freezing when using future
I am using cx freeze to bundle my python software.
running futurize my code changed from
import urllib2
to
from future import standard_library
standard_library.install_aliases()
import urllib.request, urllib.error, urllib.parse
when freezing on python2 its having problems finding the parent path of urllib.
I couldn't find much on the site in this regards so i'd like to know if there is any documentation in this area.
Otherwise i'll play around and see if i can find a nice workaround.
Kind Regards,
Tim
_______________________________________________
Python-porting mailing list
Python-porting at python.org
https://mail.python.org/mailman/listinfo/python-porting
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-porting/attachments/20150701/856c76cf/attachment.html>