[py-dev] issues with stdout redirection, ctypes and py.test

[holger krekel]

On Sat, Aug 08, 2009 at 14:43 +1000, Gordon Wrigley wrote:
> Since I have no clear concept of what is supposed to happen when a C
> function called via ctypes does a printf, I have instead gone through
> the C library and arranged to have all the printing routed through a
> function pointer which I now setup as a python callback that does a
> sys.stdout.write(). Problem solved.

if you can try to ensure open the file (look for "fdopen(2)" or so) 
in unbuffered mode that should work.  Would be good to know. 
I have played on the command line and it appeared to me that
this buffering mode is very likely the culprit. 

cheers,
holger


> On Fri, Aug 7, 2009 at 8:31 PM, Gordon Wrigley<py-dev at tolomea.com> wrote:
> > I need to have a think about what you've said and look at producing a
> > minimal reproduction. However I did forget to mention that the prints
> > from the python code are fine, it's only the ones from C that are
> > affected, in the good case the two sets are interleaved in the py.test
> > capture, in the bad case the python prints are in the py.test capture
> > and the C prints come out of py.test's stdout.
> > Now that I think about it some more I'm inclined to wonder from where
> > (and when) the C library acquires it's stdout handle.
> >
> > On Fri, Aug 7, 2009 at 8:02 PM, holger krekel<holger at merlinux.eu> wrote:
> >> Hi Gordon,
> >>
> >> On Fri, Aug 07, 2009 at 18:53 +1000, Gordon Wrigley wrote:
> >>> My python code talks to a C library via ctypes and it in turn prints
> >>> debug to stdout.
> >>> And we test this with py.test, which by itself works just fine.
> >>
> >> by this you mean that it looks roughly like this:
> >>
> >>    http://paste.pocoo.org/show/132982/
> >>
> >> right?
> >>
> >>> But if at the bash prompt we then redirect the output of py.test to a
> >>> file suddenly the C debug starts propagating out of py.test.
> >>
> >> hum, redirecting the above example with "py.test x.py >log" works fine.
> >>
> >> The following may be interesting to observe (i just did):
> >>
> >>    def test_hello():
> >>        myfile = os.fdopen(1, "w")
> >>        myfile.write("hello\n")
> >>        myfile.flush()
> >>        assert 0
> >>
> >> This works fine but if i leave out the flush() the write
> >> will *not* be captured appropriately.  I guess that it is because
> >> of the missing buffering ("os.fdopen(1, 'w', 1)" and no flush works)
> >>
> >> FYI the pytest_capture plugin resumes capturing for calling
> >> into test methods, setup/teardown code and collection
> >> respectively and otherwise suspends capturing.  Even if this
> >> was modified (i.e. capturing would always happen) i imagine
> >> that the missing flush/close could associate the actual write
> >> into a different test.
> >>
> >> Maybe your behaviour is still a different issue or even a bug, though.
> >>
> >>> We generally run py.test with -v and -x, but removing those didn't
> >>> change the behavior at all.
> >>
> >> that shouldn't change anything related to capturing, indeed.
> >>
> >>> I'm at a bit of loss as to where to begin with this one, any suggestions?
> >>
> >> maybe my above considerations help a bit and you can maybe
> >> come up with a breaking example or fix things on your side?
> >>
> >> I want to do a 1.0.1 soon with some capturing fixes (related
> >> to unicode handling, though) and i'd like to include any other
> >> related improvements or add to the docs.
> >>
> >> best,
> >> holger
> >>
> >>> gordonw at gohma:~$ uname -a
> >>> Linux gohma 2.6.28-14-generic #47-Ubuntu SMP Sat Jul 25 00:28:35 UTC
> >>> 2009 i686 GNU/Linux
> >>> gordonw at gohma:~$ python
> >>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
> >>> [GCC 4.3.3] on linux2
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >>> >>> import py
> >>> >>> py.version
> >>> '1.0.0'
> >>>
> >>> Regards,
> >>> Gordon
> >>> _______________________________________________
> >>> py-dev mailing list
> >>> py-dev at codespeak.net
> >>> http://codespeak.net/mailman/listinfo/py-dev
> >>>
> >>
> >> --
> >> Metaprogramming, Python, Testing: http://tetamap.wordpress.com
> >> Python, PyPy, pytest contracting: http://merlinux.eu
> >>
> >
> 

-- 
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu