[Chicago] Testing Submitted Python Scripts

[Carl Karsten]

On Mon, Dec 6, 2010 at 10:27 PM, Walter Askew <waltaskew at gmail.com> wrote:
> I'm going to be teaching an Intro to Programming class at Pumping Station
> One sometime after the holiday stuff clears up, and I'll be using Python to
> impart the art of computer programming.  I'll probably post again once I'm
> about to fire up the class to try get the word out and grab some more
> students.
> I'm working on a website for the class, and I'd like to have a way to
> distribute homework to the students and then validate the programs they
> write.  I'm trying to think of a clever way to do this, but I can't quite
> come up with something I'm happy with.
> I think the best thing to do would be to distribute a tester program, let
> the students run it, and on a successful check of the code they've written
> pop out a random-looking code they can enter in the website to prove they've
> solved the problem.  I'd like this because I'd be able to avoid the whole
> 'execution of foreign code on my server' thing.
> That'd be great, but I can't really come up with a clever way to generate a
> secret code without exposing the secret code itself in the tester program
> I'd be distributing.  I don't really want to get into distributing
> non-source tester programs, because the source of the tester could be useful
> to the students and I'd have to get involved with distributing non-source
> python programs.
> The other alternative is to let students upload their programs and do the
> validation on the server.  If I do this I'll need to get involved with a
> bunch of security precautions -- I'm thinking a chroot with nothing but the
> read-only binaries for the python interpreter -- but I'm appropriately
> concerned with executing arbitrary code on my server.
> What would you guys do in this situation?  Is there a clever solution I'm
> overlooking?  Any advice in implementing the solutions I've put forward?
>  I'm doing the site in Django, so bonus points for any Django snippets.
> Thanks folks.

I would look into running the code on the server.   the security
problem has been solved. Here is one example:

http://codepad.org/UngHomhU

I am sure the build bot things are secure.  like Hudson, which I know
nothing about.

If needed I am up for helping you setup a VM that gets the .py file,
runs it and drops output.txt somewhere.  If the VM stays alive for
more than.. um.. 15 seconds, it gets killed.  Pretty sure this is
overkill, and will make sure it is really needed if it comes to it.

OTOH, is this level of validation really needed?  I would think "click
here to see the answer... did you get it right?"  would be good
enough.    You may want to expand on your goal if not.

-- 
Carl K