[Tutor] crashed with Runtime Error: NZEC (non-zero exit code)

[bob gailer]

This thread got started due to the DIV15 problem on the Sphere Online Judge.

This is almost a "black box" test environment. Submit a program. If it 
generates the expected output you get "accepted". The other results are 
" compilation error", "time limit exceeded", " runtime error (followed 
by (SIGSEGV), (SIGABRT), (other), (NZEC))", or "wrong answer". If the 
program did not exceed the time limit you also get the execution time. 
(Some problems give problem-specific results but that does not apply to 
the DIV15 problem)

In looking over the status list of submitted programs I noted cases 
where the results were "100", "102". I wonder what that means and how it 
was generated

So there is a (limited) way to get information out of a program, and 
that is to use time.sleep() to control the execution time. But that 
costs one program submission for each thing one might want to test. 
That's how I determined there was a bad character in the input.

So now I imagine creating an automated program submitter, so I could 
create a large number of test cases and an automated way to read the 
execution times off the results page. Eventually one might garner enough 
information.

Of course that might get the attention of the site operators and lead to 
a change in how things are run to discourage automated testing.

-- 
Bob Gailer
919-636-4239 Chapel Hill, NC