on perhaps unloading modules?

[Hope Rouselle]

Hope Rouselle <hrouselle at jevedi.com> writes:

> Chris Angelico <rosuav at gmail.com> writes:
>
>> On Tue, Aug 17, 2021 at 4:02 AM Greg Ewing
>> <greg.ewing at canterbury.ac.nz> wrote:
>>> The second best way would be to not use import_module, but to
>>> exec() the student's code. That way you don't create an entry in
>>> sys.modules and don't have to worry about somehow unloading the
>>> module.
>>
>> I would agree with this. If you need to mess around with modules and
>> you don't want them to be cached, avoid the normal "import" mechanism,
>> and just exec yourself a module's worth of code.
>
> Sounds like a plan.  Busy, haven't been able to try it out.  But I will.
> Soon.  Thank you!

Just to close off this thread, let me share a bit of what I wrote.  The
result is a lot better.  Thanks for all the help!

I exec the student's code into a dictionary g.

--8<---------------cut here---------------start------------->8---
def fs_read(fname):
  with open(fname, "r") as f:
    return f.read()

def get_student_module_exec(fname):
  g = {}
  try:
    student_code = fs_read(fname)
    student = exec(student_code, g)
  except Exception as e:
    return False, str(e)
  return True, g

def get_student_module(fname):
  return get_student_module_exec(fname)
--8<---------------cut here---------------end--------------->8---

And now write the test's key as if I were a student and named my test as
"test_key.py".

--8<---------------cut here---------------start------------->8---
def get_key():
  okay, k = get_student_module("test_key.py")
  if not okay:
    # Stop everything.
    ...
  return g
--8<---------------cut here---------------end--------------->8---

The procedure for grading a question consumes the student's code as a
dictionary /s/, grabs the key as /k/ and checks whether the procedures
are the same.  So, suppose I want to check whether a certain function
/fn/ written in the student's dictionary-code /s/ matches the key's.
Then I invoke check_student_procedure(k, s, fn).

--8<---------------cut here---------------start------------->8---
def check_student_procedure(k, s, fn, args = [], wrap = identity):
  return check_functions_equal(g[fn], s.get(fn, None), args, wrap)
--8<---------------cut here---------------end--------------->8---

For completeness, here's check_functions_equal.

--8<---------------cut here---------------start------------->8---
def check_functions_equal(fn_original, fn_candidate, args = [], wrap = identity):
  flag, e = is_function_executable(fn_candidate, args)
  if not flag:
    return False, "runtime", e
  # run original and student's code, then compare them
  answer_correct = fn_original(*args)
  answer_student = wrap(fn_candidate(*args))
  if answer_correct != answer_student:
    return False, None, str(answer_student)
  return True, None, None

def identity(x):
  return x
--8<---------------cut here---------------end--------------->8---

To explain my extra complication there: sometimes I'm permissive with
student's answers.  Suppose a question requires a float as an answer but
in some cases the answer is a whole number --- such as 1.0.  If the
student ends up producing an int, the student gets that case right: I
wrap the student's answer in a float() and the check turns out
successful.

I probably don't need to check whether a procedure is executable first,
but I decided to break the procedure into two such steps.

--8<---------------cut here---------------start------------->8---
def is_function_executable(f, args = []):
  try:
    f(*args)
  except Exception as e:
    return False, str(e)
  return True, None
--8<---------------cut here---------------end--------------->8---