[Edu-sig] DocTest Quiz

[Ian Bicking]

Michael Tobis wrote:
> It is possible to do this entirely through HTTP requests on the
> client. The student would be doing conventional doctests during
> development, but would be running a validator that hits a server with
> HTTP requests ("validate my code for exercise 42") gets HTTP responses
> ('send me the output from "foo(42)" with this ticket T ') and
> resubmits ('here's ticket T, foo(42) returns "don't talk to me about
> life") and the server responds ("OK, you are validated in the
> database, and here is ticket U which will allow you to request
> exercise 43).

Is it necessary to do this, instead of just having the client-hosted 
program report the student's progress?  The only advantage is if you 
don't trust the client hosted program to report accurately.  But then, 
it's not that hard to get the client-hosted program to lie about the 
output of the program either.  Doing a chatty doctest across hosts seems 
like more trouble than its worth.

> You DON'T want the client code to be a real XML-RPC server, because it
> has to be an HTTP client, since these kids will be running in an
> arbitrarily locked-down requirement, except for outbound HTTP
> requests. But, except for being the client at the lowest level,  it
> looks a lot like an XML-RPC server, interpreting known requests

XML-RPC is just specially-formatted HTTP requests, so it wouldn't effect 
any port security.

>> If there were student-contributed doctests this
>> seems like a potential concern.
> 
> Yes, this is the problem with my approach. I don't handle that, and
> that is why a sandbox solution is still a good idea.

If everything is purely run on the client side, it's probably not *that* 
big of a deal, if you only accept code from 'trusted' students, i.e., 
students actually in the class, or doctests vetted by some trusted group 
(e.g., teachers moderating a wiki).  Then the students can only mess up 
other people's computers to same degree they can mess up their 
computers, which if you are in a lab isn't a big deal (and you probably 
need a restoration process for other reasons anyway).


-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org