[Distutils] distribute's sandboxing doesn't preserve working_set; leads to setup_requires problems

[Erik Bray]

On Thu, May 19, 2011 at 4:10 PM, Erik Bray <erik.m.bray at gmail.com> wrote:
> Hello all,
> I've got a tricky problem I'm trying to deal with.  Here's the scenario:
>
> I'm trying to build a package that has two requirements in
> setup_requires; let's say `setup_requires = ['package_A',
> 'package_B']`.
> The problem is that "package_B" also contains "package_A" in its own
> setup_requires.
>
> What happens when I do an install/build is that package_B is
> downloaded and installed--in the process it also downloads and
> installs package_A in its sandbox.  This wouldn't be a problem except
> that the sandboxed temp install of package_A is left in
> pkg_resources.working_set.entries, and so it's assumed package_A is
> already installed, and the main install doesn't try to fetch it.
> However, when it goes to import from package_A, I get an import error
> (because the sandbox it was installed in has since been deleted).
>
> If I reverse the order it doesn't work either for a different, but
> related problem.  package_A gets installed into the cwd and is added
> to pkg_resources.working_set.  When package_B is installed it uses the
> existing package_A installation, so that's fine.  But then, due to
> some complicated interplay, package_A never gets added to sys.path.
>
> I'd consider this a defect, though I'm wondering if there's something
> I could do differently to avoid this situation.

Just to confirm my theory about this, I modified
setuptools.sandbox.run_setup to also save off and restore
pkg_resources.working_set.{entries,entry_keys,by_key}.  This solves
the problem, and the build works regardless of how my setup_requires
are ordered.  Is there any reason this shouldn't be done?  It seems
reasonable to me...

It would also still be nice to find a workaround until and unless this
gets patched.  All I can think of at the moment is monkey-patching :/

Thanks,
Erik