[lxml-dev] some comments about the file shuffling

Hi there, Just took a glance at the reshuffling branch. I'm glad you're doing this work. Some comments about it though: it's now lxml.etree.etree and lxml.etree.dom in the old situation, and intended, it was just lxml.etree and lxml.dom i.e. there's no need for this separate etree package, we can just put the modules in the lxml container package. dom has nothing to do with etree; it's a completely different way to layer over libxml2. Where did the testrunner test.py go? I see you moved all tests into a separate top-level test directory, but I've been following the pattern in use in Zope (2 and 3), Schooltool, Pypy and Twisted, to name just a few Python projects, where there's a 'tests' or 'test' directory in or next to the package/module being tested. Since these different subpackages can be developed more or less in isolation, this seems to be like a good thing to do, instead of putting them all in the same place. They're unit tests, and belong with the unit they're testing. I see you what looks like an empty __init__.py in the etree package. While that works fine on unixy systems, Windows tools like winzip sometimes have issues, not extracting empty files. I don't use Windows but want my software to be portable to it if possible, so I've therefore made it a habit to place a marker comment "# this is a package" in any __init__.py that I create. Regards, Martijn

On 25-Nov-04, at 12:35 PM, Martijn Faassen wrote:
I actually haven't had a chance to touch the etree and dom packages yet as I couldn't get them to compile until today. On OSX we have the Fink project which is for all intents and purposes - Debian on OSX. I've got libxml2 2.6.11 and the etree and dom packages build properly from trunk now. Building against Fink On a side note - I find it incredibly amusing that Windows is a second class citizen next to Linux and OSX for this project. :) I'll move the etree and dom packages back up to lxml.etree and lxml.dom. I was primarily concerned with getting the unit tests to run and have a clean compile working.
Shoot! You know I never noticed that? I couldn't figure out what was going on with the location of test files before. It does seem a little weird to me though since we still end up compiling lxml first and then we can run the tests. I'll reshuffle the files again.
I'll keep that in mind - I also don't have a Windows machine to test on so stuff like that would've slipped by - thanks. Ack.. Too much to do at work right now. I've got a friend who has used lxml a little on Windows now so he's willing to help out with the packaging on that platform once I get the new file structure sorted out. See ya, vic

Hey Victor, You and vlibxml2 got a mention on xml.com today. A little bit critical, but I'm working on Uche (see also my comment). :) I think he'll be happy to find out we've already solved one of the things he's so critical about by merging our projects. http://www.xml.com/pub/a/2004/11/24/py-xml.html?page=2 Victor Ng wrote:
I'm just happily commenting away anyway from the sidelines. :)
Great -- I'm very happy to hear that you can make things work with Fink!
On a side note - I find it incredibly amusing that Windows is a second class citizen next to Linux and OSX for this project. :)
How come? It's currently a second-class citizen as nobody is developing in Windows yet. This will change if someone comes along who needs to deploy on Windows. It may even happen to myself, but not yet. I'm definitely hoping someone else will come along and help here.
Understood.
True, but the idea is you can at least *look* at these packages in isolation, from a developer's perspective. Having the tests near the code also facilitates distributing just part of the stuff. We could for instance say "oh, well, etree isn't finished yet, so let's just not include it (and its tests) yet, but just ship vlibxml2".
I'll reshuffle the files again.
Thanks! I was really happy with that test runner I took from schooltool (with permission from the author, of course). Marius (the author) gave a presentation on it at EuroPython this year.
Yeah, it's just a thing I learned through the pain of not doing it in the past. :)
That's awesome! We need to figure out how to get stuff compiled on Windows then; this may affect the setup.py. Once we got the file structure worked out I am looking forward to working on vlibxml2 as a base for etree.pyx. I'd still like to try using Pyrex for it instead of a Python-level wrapping, as this gives the most control, which I suspect we might need to do some of the tricky bits. We'll gain a bit more performance to boot, and Pyrex code isn't much harder to write than Python code, so I'm pretty happy with this approach. Regards, Martijn

On 25-Nov-04, at 12:35 PM, Martijn Faassen wrote:
I actually haven't had a chance to touch the etree and dom packages yet as I couldn't get them to compile until today. On OSX we have the Fink project which is for all intents and purposes - Debian on OSX. I've got libxml2 2.6.11 and the etree and dom packages build properly from trunk now. Building against Fink On a side note - I find it incredibly amusing that Windows is a second class citizen next to Linux and OSX for this project. :) I'll move the etree and dom packages back up to lxml.etree and lxml.dom. I was primarily concerned with getting the unit tests to run and have a clean compile working.
Shoot! You know I never noticed that? I couldn't figure out what was going on with the location of test files before. It does seem a little weird to me though since we still end up compiling lxml first and then we can run the tests. I'll reshuffle the files again.
I'll keep that in mind - I also don't have a Windows machine to test on so stuff like that would've slipped by - thanks. Ack.. Too much to do at work right now. I've got a friend who has used lxml a little on Windows now so he's willing to help out with the packaging on that platform once I get the new file structure sorted out. See ya, vic

Hey Victor, You and vlibxml2 got a mention on xml.com today. A little bit critical, but I'm working on Uche (see also my comment). :) I think he'll be happy to find out we've already solved one of the things he's so critical about by merging our projects. http://www.xml.com/pub/a/2004/11/24/py-xml.html?page=2 Victor Ng wrote:
I'm just happily commenting away anyway from the sidelines. :)
Great -- I'm very happy to hear that you can make things work with Fink!
On a side note - I find it incredibly amusing that Windows is a second class citizen next to Linux and OSX for this project. :)
How come? It's currently a second-class citizen as nobody is developing in Windows yet. This will change if someone comes along who needs to deploy on Windows. It may even happen to myself, but not yet. I'm definitely hoping someone else will come along and help here.
Understood.
True, but the idea is you can at least *look* at these packages in isolation, from a developer's perspective. Having the tests near the code also facilitates distributing just part of the stuff. We could for instance say "oh, well, etree isn't finished yet, so let's just not include it (and its tests) yet, but just ship vlibxml2".
I'll reshuffle the files again.
Thanks! I was really happy with that test runner I took from schooltool (with permission from the author, of course). Marius (the author) gave a presentation on it at EuroPython this year.
Yeah, it's just a thing I learned through the pain of not doing it in the past. :)
That's awesome! We need to figure out how to get stuff compiled on Windows then; this may affect the setup.py. Once we got the file structure worked out I am looking forward to working on vlibxml2 as a base for etree.pyx. I'd still like to try using Pyrex for it instead of a Python-level wrapping, as this gives the most control, which I suspect we might need to do some of the tricky bits. We'll gain a bit more performance to boot, and Pyrex code isn't much harder to write than Python code, so I'm pretty happy with this approach. Regards, Martijn
participants (2)
-
Martijn Faassen
-
Victor Ng