Hello, I'm experimenting with cython and lxml. I would like to "cimport" lxml in my own project (parsing XBRL files), but when I build my extension I get the following error: ---------------------------------------------------------------------------------------------------- cyxbrl\cy_xbrl.pyx:5:0: 'lxml.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... # encoding: utf-8 import os from lxml cimport etree ^ ------------------------------------------------------------ cyxbrl\cy_xbrl.pyx:5:0: 'lxml\etree.pxd' not found Traceback (most recent call last): File ".\cysetup.py", line 24, in <module> setup(ext_modules=cythonize("cyxbrl/*.pyx")) File "C:\Users\Wietse\projects\pyenv35\cython\lib\site-packages\Cython\Build\Dependencies.py", line 877, in cythonize cythonize_one(*args) File "C:\Users\Wietse\projects\pyenv35\cython\lib\site-packages\Cython\Build\Dependencies.py", line 997, in cythonize_ one raise CompileError(None, pyx_file) Cython.Compiler.Errors.CompileError: cyxbrl\cy_xbrl.pyx ---------------------------------------------------------------------------------------------------- I've been able to build lxml on windows from source - with some hacks in the setup.py and setupinfo.py scripts - with the following command: python .\setup.py build_ext -i --with-cython --static (I've disabled the automatic download of libxml and libxslt because I wanted to use my own 64 bit build). I've set PYTHONPATH to the lxml.src directory. Everything works fine if I use a normal python import. (This is on Windows 10 with Visual Studio 2015 Community Edition and python 3.5 - 64 bit). Is there a way to fix this? Thanks, Wietse
Wietse Jacobs schrieb am 21.01.2016 um 12:18:
I'm experimenting with cython and lxml. I would like to "cimport" lxml in my own project (parsing XBRL files), but when I build my extension I get the following error:
---------------------------------------------------------------------------------------------------- cyxbrl\cy_xbrl.pyx:5:0: 'lxml.pxd' not found
Error compiling Cython file: ------------------------------------------------------------ ... # encoding: utf-8
import os from lxml cimport etree ^ ------------------------------------------------------------
cyxbrl\cy_xbrl.pyx:5:0: 'lxml\etree.pxd' not found Traceback (most recent call last): File ".\cysetup.py", line 24, in <module> setup(ext_modules=cythonize("cyxbrl/*.pyx"))
Did you read this? http://lxml.de/capi.html Admittedly, the C-API of lxml predates some Cython features that would enable your usage above, but it might need some work to change that (and I'm not even sure if I really want to go down that path). Stefan
On Fri, Jan 22, 2016 at 10:41 AM Stefan Behnel <stefan_ml@behnel.de> wrote:
Did you read this?
Duh, I feel silly now. Sorry, I didn't yet, thanks for the pointer. Admittedly, the C-API of lxml predates some Cython features that would
enable your usage above, but it might need some work to change that (and I'm not even sure if I really want to go down that path).
Could you elaborate on that last remark? I'm not a C programmer at all, so I don't fully understand what it would entail to support "cimport" between Cython packages. (Probably I should just start with that document you pointed to and come back when I know more). Thanks, Wietse
Wietse Jacobs schrieb am 22.01.2016 um 12:05:
On Fri, Jan 22, 2016 at 10:41 AM Stefan Behnel wrote: Admittedly, the C-API of lxml predates some Cython features that would
enable your usage above, but it might need some work to change that (and I'm not even sure if I really want to go down that path).
Could you elaborate on that last remark? I'm not a C programmer at all, so I don't fully understand what it would entail to support "cimport" between Cython packages.
It just means that there is a module.pxd file next to a module.pyx file which exports certain types, functions, etc., and that the declarations in the .pxd file must match exactly those in the .pyx file. Then, Cython can use the .pxd file in order to understand what is exported in what way and generate code in other modules that talks to the implementation natively. The problem is that existing lxml C-API code uses what's there and I had originally kept the API as tight as possible, especially the exported extension type declarations (Cython is rather forgiving about C details it doesn't know about and just leaves them to the C compiler). If I let Cython do the job, it will require exporting the complete (and correct) object structs, and I'd like to keep some things private. For example, the _Document type keeps an internal reference to the XML parser object that built it, and I'd like to avoid making the parser internals public just because there is a reference to the type somewhere. Once C internals are out in the wild, it becomes difficult to take them back and change them later. I'm sure I'll reconsider the whole thing at some point, but that point is not now. Stefan
On Fri, Jan 22, 2016 at 1:53 PM Stefan Behnel <stefan_ml@behnel.de> wrote:
Wietse Jacobs schrieb am 22.01.2016 um 12:05:
On Fri, Jan 22, 2016 at 10:41 AM Stefan Behnel wrote: Admittedly, the C-API of lxml predates some Cython features that would
enable your usage above, but it might need some work to change that (and I'm not even sure if I really want to go down that path).
Could you elaborate on that last remark? I'm not a C programmer at all, so I don't fully understand what it would entail to support "cimport" between Cython packages.
It just means that there is a module.pxd file next to a module.pyx file which exports certain types, functions, etc., and that the declarations in the .pxd file must match exactly those in the .pyx file. Then, Cython can use the .pxd file in order to understand what is exported in what way and generate code in other modules that talks to the implementation natively.
The problem is that existing lxml C-API code uses what's there and I had originally kept the API as tight as possible, especially the exported extension type declarations (Cython is rather forgiving about C details it doesn't know about and just leaves them to the C compiler). If I let Cython do the job, it will require exporting the complete (and correct) object structs, and I'd like to keep some things private. For example, the _Document type keeps an internal reference to the XML parser object that built it, and I'd like to avoid making the parser internals public just because there is a reference to the type somewhere. Once C internals are out in the wild, it becomes difficult to take them back and change them later.
I think I get the picture, thanks!
I'm sure I'll reconsider the whole thing at some point, but that point is not now.
I've been coding python for my bread (and butter!) these last 10 years, and lxml has played an important part in that code all this time. I'm using it since version 1.3. Allow me to thank you for a great tool! Kind regards, Wietse
participants (2)
-
Stefan Behnel -
Wietse Jacobs