Hi,

After installing pypy 4.0.1 on FreeBSD (from ports), I get the above exception when trying to install setuptools, or when installing anything with pip (after pypy -m ensurepip).

This all worked with the previous version (2.6).

I could find this problem elsewhere: http://stackoverflow.com/questions/34566676/failed-to-install-pip-for-pypy-o... so it doesn't seem to be related to FreeBSD.

An example full trace: # pypy -m pip install --upgrade pip You are using pip version 6.1.1, however version 8.0.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting pip Exception: Traceback (most recent call last): File "/usr/local/pypy-4.0/site-packages/pip/basecommand.py", line 246, in main status = self.run(options, args) File "/usr/local/pypy-4.0/site-packages/pip/commands/install.py", line 342, in run requirement_set.prepare_files(finder) File "/usr/local/pypy-4.0/site-packages/pip/req/req_set.py", line 345, in prepare_files functools.partial(self._prepare_file, finder)) File "/usr/local/pypy-4.0/site-packages/pip/req/req_set.py", line 290, in _walk_req_to_install more_reqs = handler(req_to_install) File "/usr/local/pypy-4.0/lib_pypy/_functools.py", line 42, in __call__ return self._func(*(self._args + fargs), **fkeywords) File "/usr/local/pypy-4.0/site-packages/pip/req/req_set.py", line 487, in _prepare_file download_dir, do_download, session=self.session, File "/usr/local/pypy-4.0/site-packages/pip/download.py", line 827, in unpack_url session, File "/usr/local/pypy-4.0/site-packages/pip/download.py", line 673, in unpack_http_url from_path, content_type = _download_http_url(link, session, temp_dir) File "/usr/local/pypy-4.0/site-packages/pip/download.py", line 887, in _download_http_url with open(file_path, 'wb') as content_file: TypeError: expected string, got NoneType object

And the one for setuptools: # pypy setup.py install running install Checking .pth file support in /usr/local/pypy-4.0/site-packages/ /usr/local/bin/pypy -E -c pass TEST PASSED: /usr/local/pypy-4.0/site-packages/ appears to support .pth files running bdist_egg running egg_info writing setuptools.egg-info/PKG-INFO writing dependency_links to setuptools.egg-info/dependency_links.txt writing entry points to setuptools.egg-info/entry_points.txt writing requirements to setuptools.egg-info/requires.txt writing top-level names to setuptools.egg-info/top_level.txt Traceback (most recent call last): File "setup.py", line 169, in <module> dist = setuptools.setup(**setup_params) File "/usr/local/pypy-4.0/lib-python/2.7/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/local/pypy-4.0/lib-python/2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/usr/local/pypy-4.0/lib-python/2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/tmp/setuptools-19.4/setuptools/command/install.py", line 67, in run self.do_egg_install() File "/tmp/setuptools-19.4/setuptools/command/install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "/usr/local/pypy-4.0/lib-python/2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/usr/local/pypy-4.0/lib-python/2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/tmp/setuptools-19.4/setuptools/command/bdist_egg.py", line 152, in run self.run_command("egg_info") File "/usr/local/pypy-4.0/lib-python/2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/usr/local/pypy-4.0/lib-python/2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/tmp/setuptools-19.4/setuptools/command/egg_info.py", line 186, in run self.find_sources() File "/tmp/setuptools-19.4/setuptools/command/egg_info.py", line 209, in find_sources mm.run() File "/tmp/setuptools-19.4/setuptools/command/egg_info.py", line 293, in run self.add_defaults() File "/tmp/setuptools-19.4/setuptools/command/egg_info.py", line 322, in add_defaults sdist.add_defaults(self) File "/tmp/setuptools-19.4/setuptools/command/sdist.py", line 100, in add_defaults self.filelist.append(fn) File "/tmp/setuptools-19.4/setuptools/command/egg_info.py", line 236, in append if self._safe_path(path): File "/tmp/setuptools-19.4/setuptools/command/egg_info.py", line 256, in _safe_path u_path = unicode_utils.filesys_decode(path) File "/tmp/setuptools-19.4/setuptools/unicode_utils.py", line 31, in filesys_decode return path.decode(enc) TypeError: expected string, got NoneType object

In this case the context is: def filesys_decode(path): """ Ensure that the given path is decoded, NONE when no expected encoding works """

fs_enc = sys.getfilesystemencoding() if isinstance(path, six.text_type): return path

for enc in (fs_enc, "utf-8"): try: return path.decode(enc) except UnicodeDecodeError: continue

On python 2.7, fs_enc here is: Python 2.7.11 (default, Dec 20 2015, 01:15:21) [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10 Type "help", "copyright", "credits" or "license" for more information.

...

...

import sys repr(sys.getfilesystemencoding())

"'US-ASCII'"

While on pypy: Python 2.7.10 (5f8302b8bf9f53056e40426f10c72151564e5b19, Jan 16 2016, 01:16:36) [PyPy 4.0.1 with GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10 Type "help", "copyright", "credits" or "license" for more information.

...

...

...

import sys repr(sys.getfilesystemencoding())

'None'

Returning None here is fine according to the docs, but the above code snippet doesn't handle the TypeError, which it gets when doing path.decode(None). How this could work?

On pypy 2.6 on the same machine: Python 2.7.9 (295ee98b69288471b0fcf2e0ede82ce5209eb90b, Jun 12 2015, 19:25:58) [PyPy 2.6.0] on freebsd10 Type "help", "copyright", "credits" or "license" for more information.

...

...

...

'test'.decode(None)

u'test' No exception!

On pypy 4.0.1: Python 2.7.10 (5f8302b8bf9f53056e40426f10c72151564e5b19, Dec 10 2015, 01:17:03) [PyPy 4.0.1 with GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10 Type "help", "copyright", "credits" or "license" for more information.

...

...

...

'test'.decode(None)

Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: expected string, got NoneType object

Unhandled TypeError exception!

python 2.7 works the same way:

...

...

'test'.decode(None)

Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: decode() argument 1 must be string, not None

Which installs setuptools just fine (although it doesn't get to the above snippet, because it doesn't return None to sys.getfilesystemencoding()). If I set fs_enc to None in the above snippet, even python 2.7 fails.

So the key here (if I'm not completely lost) seems to be not returning None to the getfilesystemencoding to fix broken(?) software.

What do you think about this?

Thanks,