Understanding exception raised in tests

Hi, I've recently started seeing this in some of my tests: xception ignored in: <generator object WorksheetWriter.get_stream at 0x11aef6138> Traceback (most recent call last): File "/Users/charlieclark/Projects/openpyxl/openpyxl/worksheet/_writer.py", line 296, in get_stream pass File "src/lxml/serializer.pxi", line 1400, in lxml.etree._FileWriterElement.__exit__ File "src/lxml/serializer.pxi", line 1136, in lxml.etree._IncrementalFileWriter._write_end_element lxml.etree.LxmlSyntaxError: inconsistent exit action in context manager Exception ignored in: <generator object WriteOnlyWorksheet._write_rows at 0x11aef61b0> Traceback (most recent call last): File "/Users/charlieclark/Projects/openpyxl/openpyxl/worksheet/write_only.py", line 115, in _write_rows pass File "src/lxml/serializer.pxi", line 1400, in lxml.etree._FileWriterElement.__exit__ File "src/lxml/serializer.pxi", line 1134, in lxml.etree._IncrementalFileWriter._write_end_element lxml.etree.LxmlSyntaxError: not in an element The relevant method looks like this: def _write_rows(self): """ Send rows to the writer's stream """ try: xf = self._writer.xf.send(True) except StopIteration: self._already_saved() with xf.element("sheetData"): row_idx = 1 try: while True: row = (yield) row = self._values_to_row(row, row_idx) self._writer.write_row(xf, row, row_idx) row_idx += 1 except GeneratorExit: pass self._writer.xf.send(None) I think the exception is coming from some cleanup when the test has run but the context manager hasn't exited. But I'm not sure of the details. Outside tests the context manager will always be closed so I don't think this is critical but I would like to fix it. Going by the exception I guess I should handle GeneratorExit differently but I'm afraid I don't understand the Python internals here so well! Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Kronenstr. 27a Düsseldorf D- 40217 Tel: +49-211-600-3657 Mobile: +49-178-782-6226

Hi Charlie! Charlie Clark schrieb am 06.12.18 um 19:20:
I've recently started seeing this in some of my tests:
You mentioned that you started compiling your Python code with Cython. That's the most likely reason for the "exception ignored" messages, and the rest might just result from it. This does not seem related to lxml.
xception ignored in: <generator object WorksheetWriter.get_stream at 0x11aef6138>
My guess is that you changed the compiled signature of this method to return a C value instead of a Python object. That's ok, but it prevents exceptions from propagating. You must explicitly define an exception return value if exceptions can occur. Cython also issues a compile time warning when it finds such a case. It's a common pitfall and we are thinking about changing the default behaviour in the next Cython major release. http://docs.cython.org/en/latest/src/userguide/language_basics.html#error-re... Stefan

Am .12.2018, 09:06 Uhr, schrieb Stefan Behnel <stefan_ml@behnel.de>:
Hi Charlie!
Hi Stefan,
Not quite I thought I might investigate it, but the path I was looking at (reading Excel worksheets) is already optimised by using cElementTree – we've discussed the relative performance of cElementTree and lxml on this particular task in the past.
Thanks for the note but I suspect it's something simpler: I've "dissociated" the xmlfile context manager in some recent refactoring which allows for better code reuse and testability but I think that there is a race condition if garbage collection cleans up the coroutines rather than closing them explicitly. For the particular test I was able to avoid the warning by closing explicitly. Hopefully as openpyxl 2.6 gets more of its tyres kicked someone with a better understanding of coroutines will point out any glaring idiocy! :-D Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Kronenstr. 27a Düsseldorf D- 40217 Tel: +49-211-600-3657 Mobile: +49-178-782-6226

Hi Charlie! Charlie Clark schrieb am 06.12.18 um 19:20:
I've recently started seeing this in some of my tests:
You mentioned that you started compiling your Python code with Cython. That's the most likely reason for the "exception ignored" messages, and the rest might just result from it. This does not seem related to lxml.
xception ignored in: <generator object WorksheetWriter.get_stream at 0x11aef6138>
My guess is that you changed the compiled signature of this method to return a C value instead of a Python object. That's ok, but it prevents exceptions from propagating. You must explicitly define an exception return value if exceptions can occur. Cython also issues a compile time warning when it finds such a case. It's a common pitfall and we are thinking about changing the default behaviour in the next Cython major release. http://docs.cython.org/en/latest/src/userguide/language_basics.html#error-re... Stefan

Am .12.2018, 09:06 Uhr, schrieb Stefan Behnel <stefan_ml@behnel.de>:
Hi Charlie!
Hi Stefan,
Not quite I thought I might investigate it, but the path I was looking at (reading Excel worksheets) is already optimised by using cElementTree – we've discussed the relative performance of cElementTree and lxml on this particular task in the past.
Thanks for the note but I suspect it's something simpler: I've "dissociated" the xmlfile context manager in some recent refactoring which allows for better code reuse and testability but I think that there is a race condition if garbage collection cleans up the coroutines rather than closing them explicitly. For the particular test I was able to avoid the warning by closing explicitly. Hopefully as openpyxl 2.6 gets more of its tyres kicked someone with a better understanding of coroutines will point out any glaring idiocy! :-D Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Kronenstr. 27a Düsseldorf D- 40217 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
participants (2)
-
Charlie Clark
-
Stefan Behnel