On 06/26/2013 02:23 PM, Donald Stufft wrote:

On Jun 26, 2013, at 4:21 PM, Erik Bray <erik.m.bray@gmail.com> wrote:

...

On Sun, Jun 16, 2013 at 3:13 AM, Ethan Furman <ethan@stoneleaf.us> wrote:

...

Here's my file layout:

<root> / |- setup.py | |- enum / |- __init__.py | |- py2_enum.py | |- py3_enum.py | |- test / |- test_enum.py | |- py2_test_enum.py | |- py3_test_enum.py

__init__ and test_enum are both smart enough to pull in the correct code when imported. The issue I am having is this:

--8<-------------------------------------------------------------- ethan@hydra:~$ sudo easy_install enum34 [sudo] password for ethan: Searching for enum34 Reading http://pypi.python.org/simple/enum34/ Best match: enum34 0.9 Downloading http://pypi.python.org/packages/source/e/enum34/enum34-0.9.zip#md5=4717b8c32... Processing enum34-0.9.zip Writing /tmp/easy_install-sB55B5/enum34-0.9/setup.cfg Running enum34-0.9/setup.py -q bdist_egg --dist-dir /tmp/easy_install-sB55B5/enum34-0.9/egg-dist-tmp-qUYAv5 SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/enum/py3_enum.py', 211, 43, ' def __call__(cls, value, names=None, *, module=None, type=None):\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/enum/test/py3_test_enum.py', 630, 47, ' class AutoNumberedEnum(Enum, metaclass=auto_enum):\n'))

zip_safe flag not set; analyzing archive contents... SyntaxError: ('invalid syntax', ('/usr/local/lib/python2.7/dist-packages/enum34-0.9-py2.7.egg/enum/py3_enum.py', 211, 43, ' def __call__(cls, value, names=None, *, module=None, type=None):\n'))

SyntaxError: ('invalid syntax', ('/usr/local/lib/python2.7/dist-packages/enum34-0.9-py2.7.egg/enum/test/py3_test_enum.py', 630, 47, ' class AutoNumberedEnum(Enum, metaclass=auto_enum):\n'))

Adding enum34 0.9 to easy-install.pth file

Installed /usr/local/lib/python2.7/dist-packages/enum34-0.9-py2.7.egg Processing dependencies for enum34 Finished processing dependencies for enum34 --8<--------------------------------------------------------------

distutils is trying to load the py3 versions, which of course fails on a py2 install. The package installs successfully anyway, but if I were a user I would be wondering if the install was trustworthy.

It seems to me that I need to either have distutils only install the version appropriate files, or to not try to scan the version inappropriate files, but at this point I do not know how to do either.

Any pointers would be greatly appreciated.

That's odd. I work on a package that ships Python 2 and Python 3 versions of some modules and I have never seen this problem before. Perhaps you could post your setup.py?

If I recall this is because it's trying to compile byte code.

That's correct. And you'll note that despite the scary-looking syntax errors from byte-compilation, the install actually succeeded. Carl

On 06/26/2013 01:23 PM, Donald Stufft wrote:

If I recall this is because it's trying to compile byte code.

Is there an option to tell distutils not to compile byte code? -- ~Ethan~

On 06/26/2013 01:21 PM, Erik Bray wrote:

On Sun, Jun 16, 2013 at 3:13 AM, Ethan Furman <ethan@stoneleaf.us> wrote:

...

Here's my file layout:

<root> / |- setup.py | |- enum / |- __init__.py | |- py2_enum.py | |- py3_enum.py | |- test / |- test_enum.py | |- py2_test_enum.py | |- py3_test_enum.py

That's odd. I work on a package that ships Python 2 and Python 3 versions of some modules and I have never seen this problem before. Perhaps you could post your setup.py?

--8< -- setup.py ----------------------------------------------------------------- from distutils.core import setup long_desc = open('enum/enum.rst').read() setup( name='enum34', version='0.9', url='https://pypi.python.org/pypi/enum34', packages=['enum'], package_data={ 'enum':[ 'enum.rst', 'enum.pdf', 'test/test_enum.py', 'test/py2_test_enum.py', 'test/py3_test_enum.py', ], }, license='BSD License', description='Python 3.4 Enum backported to 3.3, 3.2, 3.1, 2.7, 2.6, 2.5, and 2.4', long_description=long_desc, provides=['enum'], author='Ethan Furman', author_email='ethan@stoneleaf.us', classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Programming Language :: Python', 'Topic :: Software Development' ], ) --8< -- setup.py ----------------------------------------------------------------- -- ~Ethan~

On 06/27/2013 10:19 AM, Erik Bray wrote:

On Wed, Jun 26, 2013 at 4:21 PM, Erik Bray <erik.m.bray@gmail.com> wrote:

...

On Sun, Jun 16, 2013 at 3:13 AM, Ethan Furman <ethan@stoneleaf.us> wrote:

...

Here's my file layout:

<root> / |- setup.py | |- enum / |- __init__.py | |- py2_enum.py | |- py3_enum.py | |- test / |- test_enum.py | |- py2_test_enum.py | |- py3_test_enum.py

__init__ and test_enum are both smart enough to pull in the correct code when imported. The issue I am having is this:

--8<-------------------------------------------------------------- ethan@hydra:~$ sudo easy_install enum34 [sudo] password for ethan: Searching for enum34 Reading http://pypi.python.org/simple/enum34/ Best match: enum34 0.9 Downloading http://pypi.python.org/packages/source/e/enum34/enum34-0.9.zip#md5=4717b8c32... Processing enum34-0.9.zip Writing /tmp/easy_install-sB55B5/enum34-0.9/setup.cfg Running enum34-0.9/setup.py -q bdist_egg --dist-dir /tmp/easy_install-sB55B5/enum34-0.9/egg-dist-tmp-qUYAv5 SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/enum/py3_enum.py', 211, 43, ' def __call__(cls, value, names=None, *, module=None, type=None):\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/enum/test/py3_test_enum.py', 630, 47, ' class AutoNumberedEnum(Enum, metaclass=auto_enum):\n'))

zip_safe flag not set; analyzing archive contents... SyntaxError: ('invalid syntax', ('/usr/local/lib/python2.7/dist-packages/enum34-0.9-py2.7.egg/enum/py3_enum.py', 211, 43, ' def __call__(cls, value, names=None, *, module=None, type=None):\n'))

SyntaxError: ('invalid syntax', ('/usr/local/lib/python2.7/dist-packages/enum34-0.9-py2.7.egg/enum/test/py3_test_enum.py', 630, 47, ' class AutoNumberedEnum(Enum, metaclass=auto_enum):\n'))

Adding enum34 0.9 to easy-install.pth file

Installed /usr/local/lib/python2.7/dist-packages/enum34-0.9-py2.7.egg Processing dependencies for enum34 Finished processing dependencies for enum34 --8<--------------------------------------------------------------

distutils is trying to load the py3 versions, which of course fails on a py2 install. The package installs successfully anyway, but if I were a user I would be wondering if the install was trustworthy.

It seems to me that I need to either have distutils only install the version appropriate files, or to not try to scan the version inappropriate files, but at this point I do not know how to do either.

Any pointers would be greatly appreciated.

That's odd. I work on a package that ships Python 2 and Python 3 versions of some modules and I have never seen this problem before. Perhaps you could post your setup.py?

It occur to me now that the reason I don't see this kind of error on my project is because in the setup.py I am just excluding the version-specific files based on what Python version the user has. Perhaps you should do the same--only install the Python 2 tests on Python 2 and so on. Just make sure that the MANIFEST.in is set up to include both versions in the source distribution. As long as you don't install the Py2 files in Py3 or vice versa it shoudn't try to compile the bytecode for those files.

I would be willing to do that, but I don't know how, and so far my searching hasn't yielded anything useful besides this mailing list. -- ~Ethan~

On 06/27/2013 11:55 AM, Oscar Benjamin wrote:

On 27 June 2013 18:31, Ethan Furman <ethan@stoneleaf.us> wrote:

...

...

It occur to me now that the reason I don't see this kind of error on my project is because in the setup.py I am just excluding the version-specific files based on what Python version the user has. Perhaps you should do the same--only install the Python 2 tests on Python 2 and so on. Just make sure that the MANIFEST.in is set up to include both versions in the source distribution. As long as you don't install the Py2 files in Py3 or vice versa it shoudn't try to compile the bytecode for those files.

I would be willing to do that, but I don't know how, and so far my searching hasn't yielded anything useful besides this mailing list.

Do this do what you want:

#setup.py import sys from distutils.core import setup

package_data = {'enum': [...]}

if sys.version_info >= (3,0): package_data['enum'].append( 'test/py3_test_enum.py') else: package_data['enum'].append( 'test/py2_test_enum.py')

setup(package_data = package_data, ...)

Cool! Is there a way to have files renamed? Ideally the installed package would just have enum.py, test_enum.py, and not py2_enum.py and py2_test_enum.py -- ~Ethan~

On 29 June 2013 10:09, Ethan Furman <ethan@stoneleaf.us> wrote:

Am I making this too complicated? Can I just do my renaming in the setup.py script before calling the setup function?

You can't easily create a wheel that way. What would actually be better is if you could avoid the need for any Python 3 specific syntax in the first place. Are you sure you can't tweak the code to use types.new_class or an inner function call to avoid the syntax problems? For example, Python 2 can do keyword only arguments like this: def _unpack_args(module=None, type=None): return module, type class CallableWithKeywordOnlyArgs def __call__(cls, value, names=None, **kwargs): module, type = _unpack_kwargs(**kwargs) The introspection support and error messages aren't as good as those for true Python 3 keyword-only arguments, but they're not *that* bad. There's a reason shared syntax compatible 2/3 source has become the most popular approach for straddling the 2/3 boundary - the alternatives are all lousy by comparison. Conditional distribution of version specific source files is far more painful than jumping through a few syntactic hoops to stay within the common 2/3 subset of the language. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 30 June 2013 05:08, Ethan Furman <ethan@stoneleaf.us> wrote:

I was hoping to provide good examples of Python 3 code (as opposed to good examples of 2/3 boundary straddling), but yeah, it's danged difficult!

Writing idiomatic Python 3 code and supporting both Python 2 & 3 aren't really compatible goals. I suggest pointing to the stdlib implementation for the idiomatic Python 3 only code, and just targeting the common 2/3 subset for the PyPI version - a lot of people have been down this road before you and it really is the easiest way to do it :)

Is there a way to have both a Py2 distribution and a Py3 distribution available on PyPI?

Register two different names (which also makes for a fairly poor end user experience) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia