[C++-SIG] Reimplementing Lib/codeop.py in C fails...

[Alex Farber]

Dear colleagues,

I am trying to create a custom "readline" function in C. I have looked 
at Modules/readline.c, Python/pythonrun.c and Parser/myreadline.c. I've 
also read the http://www.python.org/doc/current/api/api.html and many 
postings in Deja.com... I would very appreciate any hints.

Here is a simple C code reimplementing Lib/codeop.py for test purposes:

  int main(int argc, char* argv[])
  {
    char* line;
    PyObject *globals, *source;

    Py_Initialize();
    globals = PyDict_New();
    PyDict_SetItemString(globals, "__builtins__", PyEval_GetBuiltins());

  /*
    while (1) {
      line = readline(">>> ");
  */
      printf("\nCompiling <for i in [1,2,3]:>\n");
      source = Py_CompileString("for i in [1,2,3]:", "<stdin>", Py_file_input);
      if (source == NULL) PyErr_Print();

      printf("\nCompiling <for i in [1,2,3]:\\n>\n");
      source = Py_CompileString("for i in [1,2,3]:\n", "<stdin>", Py_file_input);
      if (source == NULL) PyErr_Print();

      printf("\nCompiling <for i in [1,2,3]:\\n\\n>\n");
      source = Py_CompileString("for i in [1,2,3]:\n\n", "<stdin>", Py_file_input);
      if (source == NULL) PyErr_Print();

  /*
      free(line);
    }
  */

    Py_DECREF(globals);
    Py_Finalize();
    exit(0);
  }

Which produces the following output:

  Compiling <for i in [1,2,3]:>
    File "<string>", line 1
      for i in [1,2,3]:
                     ^
  SyntaxError: unexpected EOF while parsing

  Compiling <for i in [1,2,3]:\n>
    File "<string>", line 1
      for i in [1,2,3]:
                      ^
  SyntaxError: unexpected EOF while parsing

  Compiling <for i in [1,2,3]:\n\n>
   File "<string>", line 2
     
     ^
  SyntaxError: invalid syntax

However the comment in Lib/codeop.py suggests, that the errors in 
the first and the second case should be different. How can I see 
this difference in my C code? Shouldn't the second string compile?

Regards
Alex

PS: The background is that I am programming a 3D-viewer in C++
    which uses some finite elements library (written in my university;
    not thread-safe), Qt (not thread-safe) and which embeds Python.

    Because of the not-thread-safeness I have to run FE, Qt and Python 
    in the same thread. But then I can not run PyRun_InteractiveLoop() 
    or PyRun_InteractiveOne() - the Qt would "freeze".

    So I am running a readline in another thread and send received
    strings to a UNIX-pipe. Those strings should be read by the first
    thread and executed with PyRun_SimpleString().

    Now I get the problem with the strings like "for i in [1,2,3]:" - 
    the Python needs more input. To detect such cases I was hoping to 
    mimic the Lib/codeop.py and then either save the received string 
    (if more input is needed) or run it with PyEval_EvalCode().