[pytest-dev] How to sandbox test execution with pytest

[Alessandro Amici]

Hi,

while researching the use of pytest-xdist for my CI-like use case and I
realised that having a "disposable" environment is as important as having a
isolated one, so that every test or every session runs from the same
initial state and it is not influenced by what older sessions might have
left on folders writable by the *testuser* (/home/testuser, /tmp, /var/tmp,
etc).

This made me go back searching the web again for a way to use pytest with
docker and I'm currently evaluating the following projects:

https://github.com/search?q=pytest+docker

Does anybody have experience with pytest/docker integration or with any of
these projects?

I'll report back what I find.

Thanks,
Alessandro

On Thu, 11 Feb 2016 at 11:07 Alessandro Amici <alexamici at gmail.com> wrote:

> Ionel and Holger,
>
> I had considered the --box option, but it only implements process-level
> isolation which is not enough for my use case as I want to protect also
> from unrestricted filesystem read/write access. Sorry, I should have
> mentioned it in the question.
>
> Holger's suggestion instead looks almost good enough as user-level
> isolation is strong and highly customizable. The only problem is that you
> need administration access to the machine to create and configure a new
> user. I'll look into it as I've never used the --tx option.
>
> Thanks,
> Alessandro
>
>
> On Thu, 11 Feb 2016 at 10:44 holger krekel <holger at merlinux.eu> wrote:
>
>> On Thu, Feb 11, 2016 at 11:39 +0200, Ionel Cristian Mărieș wrote:
>> > There's the `--boxed` option (from
>> https://pypi.python.org/pypi/pytest-xdist)
>> > but you can easily do your own "insulator" by using fork syscall.
>>
>> Also one can use plain "xdist" which provides process-level isolation.
>> It shouldn't be too hard to also use another unix user via e.g.:
>>
>>     py.test --tx ssh=OTHERUSER at localhost --dist=each
>>
>> but you need to have setup rsyncing, see
>> https://pypi.python.org/pypi/pytest-xdist
>> for some more info.  The current "remote" test running is a bit limited
>> and we plan to improve xdist distribution during the upcoming pytest
>> sprint,
>> see
>> https://www.indiegogo.com/projects/python-testing-sprint-mid-2016/x/4034848#/
>> creating an issue for "user-level separation of running tests" or
>> something
>> similar and adding a little donation is one way to help it :)
>>
>> best,
>> holger
>>
>>
>> >
>> > Thanks,
>> > -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>> >
>> > On Thu, Feb 11, 2016 at 10:33 AM, Alessandro Amici <alexamici at gmail.com
>> >
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I asked the following question on StackOverflow first
>> > > http://stackoverflow.com/questions/35322452, but someone quickly (and
>> > > wrongly in my opinion) marked that as duplicate of the general
>> "sandboxing
>> > > in python", so I can't get answers there anymore :(
>> > >
>> > > Here is the question:
>> > >
>> > > I'm interested in executing potentially untrusted tests with pytest in
>> > > some kind of sandbox, similarly to what continuous integration
>> services do.
>> > >
>> > > I understand that to properly sandbox a python process you need
>> OS-level
>> > > isolation, like running the tests in a disposable chroot/container,
>> but in
>> > > my use case I don't need to protect against intentionally malicious
>> code,
>> > > only from dangerous behaviour of pairing "randomly" functions with
>> > > arguments. So lesser strict sandboxing may still be acceptable. But I
>> > > didn't find any plugin that enables any form of sandboxing.
>> > >
>> > > What is the best way to sandbox tests execution in pytest?
>> > >
>> > > **Update**: This question is not about [python sandboxing in general](
>> > >
>> http://stackoverflow.com/questions/3068139/how-can-i-sandbox-python-in-pure-python
>> )
>> > > as the tests' code is run by pytest and I can't change the way it is
>> > > executed to use `exec` or `ast` or whatever. Also using pypy-sandbox
>> is not
>> > > an option unfortunately as it is "a prototype only" as per the [PyPy
>> > > feature page](http://pypy.org/features.html).
>> > >
>> > > End of question.
>> > >
>> > > For context I'm writing a pytest plugin that lets you search installed
>> > > modules for functions that pass the given feature-specification tests
>> > > (still alpha):
>> > >
>> > > https://github.com/alexamici/pytest-wish
>> > >
>> > > If you find the question useful and not answered already on
>> stackoverflow,
>> > > the best course of action would be to reopen it and answer there, but
>> I'll
>> > > take also an answer here :)
>> > >
>> > > Alessandro
>> > >
>> > >
>> > > _______________________________________________
>> > > pytest-dev mailing list
>> > > pytest-dev at python.org
>> > > https://mail.python.org/mailman/listinfo/pytest-dev
>> > >
>> > >
>>
>> > _______________________________________________
>> > pytest-dev mailing list
>> > pytest-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pytest-dev
>>
>>
>> --
>> about me:    http://holgerkrekel.net/about-me/
>> contracting: http://merlinux.eu
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20160212/e85fdebf/attachment.html>