[Edu-sig] DocTest Quiz

[Andre Roberge]

On 4/23/06, Michael Tobis <mtobis at gmail.com> wrote:
[snip]

> Ultimately to get to the big one-room python schoolhouse we all seem
> to want, it really has to be a peer relationship with peers picking
> which puzzles are interesting.
>
> However, to test kids' passing microtests in a school curriculum, you
> don;t need that.
> All you need is a way for the kid to pass the doctest and ask a server
> for a credential.
>
> Specifically:
[... more below...]

When I taught, I used to tell students: "if you don't understand, please
ask your questions even if you feel that you're probably the only one
who doesn't understand; chances are that *many* other students feel
the same way."

So, here goes ;-)

Here's how I understand the description below:
Doctests are on the server.
Student's code is on the client.
"Running" the code includes three steps:
1. a request to the server to send the doctest to the client
2. the client executes the student's code together with the doctest.
3. the result is sent back to the server for validation.

This (step 2) means that the Python interpreter is located on the
client's computer.
Security worries are the client's alone.

I thought the whole idea was to not require the client to have Python
installed...

Here's how I understood the desired situation.
Doctests are on the server.
Student's code is on the client.
"Running" the code includes the following three steps.
1. The code is sent to the server
2. The server executes the student's code together with the doctest.
3. The result can be checked by the server for validation, with some
feedback sent
to the client.

Step 2 in this case means security concerns on the server ... hence,
the need for a Python
sandbox.

Where did I go wrong?

André

>
> It is possible to do this entirely through HTTP requests on the
> client. The student would be doing conventional doctests during
> development, but would be running a validator that hits a server with
> HTTP requests ("validate my code for exercise 42") gets HTTP responses
> ('send me the output from "foo(42)" with this ticket T ') and
> resubmits ('here's ticket T, foo(42) returns "don't talk to me about
> life") and the server responds ("OK, you are validated in the
> database, and here is ticket U which will allow you to request
> exercise 43).
>
> You DON'T want the client code to be a real XML-RPC server, because it
> has to be an HTTP client, since these kids will be running in an
> arbitrarily locked-down requirement, except for outbound HTTP
> requests. But, except for being the client at the lowest level,  it
> looks a lot like an XML-RPC server, interpreting known requests
>
> Security advantages abound. The validation server address will be
> hardwired into the code that the client runs, and will be a trusted
> machine on the part of the school system that delivers those codes to
> the users.. That code can, like a vanilla XML-RPC server, only run
> registered procedures, if you are worried about executing arbitrary
> code from a hacked server. ALL TESTS RUN ON THE CLIENT. You need not
> fear for any horror that the kid the kid's evil older cousin comes up
> with, because they will never get to run any scripts server side.
>
> Probably you will want the server code to be directly munging doctests
> to make its requests. That way, the teacher can specify conventional
> doctests, and it will be the server's job to parse them into requests.
>
> Again, this is to say nothing against solving the sandbox problem, as
> even bigger things become possible with true sandboxes. My intent is
> to knock this thing out in a few days, though, so we can spend the
> summer producing curriculum for Jeff to try out. The first version
> would probably just wrap XML-RPC on the client and doctest on the
> server. Unfortunately, those few days haven't materialized yet, and
> won't for a couple of weeks.
>
> This was the inspiration that knowcked me over the head at PyCon. I
> hope there's still room to go with it.
>
> > If there were student-contributed doctests this
> > seems like a potential concern.
>
> Yes, this is the problem with my approach. I don't handle that, and
> that is why a sandbox solution is still a good idea.
>
> But my approach solves Jeff's immediate functional requirements very
> simply without any security problems at all.
>
> mt
> _______________________________________________
> Edu-sig mailing list
> Edu-sig at python.org
> http://mail.python.org/mailman/listinfo/edu-sig
>