[Edu-sig] automatically grading small programming assignments

[Jeff Rush]

Brian Blais wrote:
> 
> I envision a number of possible solutions.  In one solution, I provide a function 
> template with a docstring, and they have to fill it in to past a doctest.  Is there a 
> good (and safe) way to do that online?  Something like having a student post code, 
> and the doctest returns.  I'd love to allow them to submit until they get it, logging 
> each attempt.

Crunchy Frog (now called Crunchy, as I understand).  I just researched and 
presented on it this past weekend and was impressed with its abilities, 
including that of the instructor providing a doctest and the student working 
to write code that lets it pass.  Very cool.

There is not however currently any logging of progress, counts to get it 
right, etc.  But I would imagine that would not be hard to add to the Crunchy 
Frog backend.  It is just an HTTP proxy with some template expansion to get a 
Python interpreter inside the browser window.

Safety -- ehh.  Each Python interpreter is running inside that HTTP proxy with 
full access to the underlying system, as whatever user it is running as.  The 
design is to have each student run it locally, so they can only trash their 
own system.  However, I could imagine you could set it up to run on a private 
classroom server, where the attempt records would be kept, and still be safe.

   http://crunchy.sourceforge.net/index.html

For another solution, I wonder whether you could make use of the new Abstract 
Syntax Tree (AST) in Python 2.5, where you convert the source of an attempt 
into an abstract data structure, anonymize the method/variable/class names and 
compare the tree against a correct solution.  It would let you quickly handle 
those students who solved it in a conformist way, and then you'd need to 
manually review the rest for creatively solving it another way. ;-)

But I think Crunchy is the most classroom-friendly way to quickly solve this. 
  A weekend's work at most.

-Jeff