Giovanni Bajo wrote:

Thomas Heller wrote:

...

Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?)

I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time.

The goal of this work would exactly fit your need: make the creation of extension modules trivially easy (just as easy as adding the modules to the main python.dll). My personal goal, in fact, is to move many of those builtin extension modules from python.dll out into their own .pyd files where they'd belong (were not for this technical annoyance of being forced to use the settings dialog in MSVC).

Ideally this would be integrated with distutils because the setup-script has most of the information that's needed. OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough. Oh, IIRC the pywin32 setup script has code that is able to parse MSVC6 project files and create a distutils Extension instance from them - maybe this can help you. Thomas

Giovanni Bajo wrote:

Thomas Heller wrote:

...

...

I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time. Ideally this would be integrated with distutils because the setup-script has most of the information that's needed.

OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough.

I fear this is an orthogonal change. Alas, using distutils to build core extensions is not something I'm ready to tackle at the moment.

I was just thiking of something much more naive like using a free tool to build .vcproj/.sln (see www.cmake.org for instance) from a script description. With such a tool, it would be very easy to build a .pyd file around a self-contained .c file (like _heapqmodule.c, etc.), it would mostly be a couple of line changes in the script file, and then re-run the tool executable to regenerate vcproj/sln. OTOH, both the tool executable (cmake.exe in this example) and the current version of the generated vcproj/sln files would be committed in SVN under PCbuild, so to have a minimal impact on developer habits.

While I don't understand *why* you would want to do this (building Python interpreters with a customized list of builtin modules could be a reason for this, I've had thought about something like that for py2exe a long time ago), I've clicked myself through the MSVC dialog and I'm happy now because the two .vcproj files are ready. I'll commit them and let others (Martin, probably) extend them for 64-bit Windows versions ;-). Thomas

[Thomas Heller]

... And I never had tried it before on a sparc machine - all the intel and ppc processors seem to have no problems with it.

Pentiums don't enforce "natural" alignment restrictions, but run much slower on unaligned access (varying by specific chip model, and generally more heavily penalized as time goes on). In the good old days, Pentium was one of dozens of competing architectures, and was the oddball in catering to unaligned access. Now it's eternal "backward compatibility" with an early implementation accident. Most other architectures never catered to unaligned access, or did so only at the cost of generating an interrupt so that kernel-mode software could fake unaligned access. Bottom line is that unaligned access isn't portable and never was, and even on architectures where "it works" it can be extremely expensive to use it.

Tim Peters wrote:

[Thomas Heller]

...

... And I never had tried it before on a sparc machine - all the intel and ppc processors seem to have no problems with it.

Pentiums don't enforce "natural" alignment restrictions, but run much slower on unaligned access (varying by specific chip model, and generally more heavily penalized as time goes on). In the good old days, Pentium was one of dozens of competing architectures, and was the oddball in catering to unaligned access. Now it's eternal "backward compatibility" with an early implementation accident. Most other architectures never catered to unaligned access, or did so only at the cost of generating an interrupt so that kernel-mode software could fake unaligned access. Bottom line is that unaligned access isn't portable and never was, and even on architectures where "it works" it can be extremely expensive to use it.

I was astonished to find out that also the ARM processor on Windows CE doesn't support unaligned accesses. When one thinks about it, it probably makes sense on small devices. Thomas

Thomas Wouters wrote: (about ctypes and Py_ssize_t)

...

you want to do that yourself, or do you want me to submit a patch? (Or I

...

could just check it in ;) You can do it faster then me, I assume - so go ahead and check it in. I'll backport it to the upstream ctypes CVS repository.

I'm not sure if I'll be faster. It's quite a lot of work to make it completely Py_ssize_t-aware (but worth it, IMHO; ints just don't cut it on 64-bit platforms. :) I have a simple patch to silence the warnings, but it doesn't support indexing beyond int-size and such. I'm also going through all of the code and changing most appropriate things into Py_ssize_t's, and that would be much nicer. I'll probably have some XXX markers left when I'm done, though. Also, how stable should the C API be? Does the C code have any direct users besides Python?

Apart from a few functions that are used as foreign functions from ctypes itself (memmove and memset currently), the C api is not used from the outside. The important thing when changing something in the ctypes sources is to run the unittests (*) on as many platforms and Python versions as possible - currently I do this on all the sourceforge compile farm machines that have either Python >= 2.3, or where I have been able to compile Python myself. Plus WindowsXP 32 and OSX 10.4. This is for the ctypes upstream version, of course. If your patch is going to change more than a few int -> Py_ssize_t replacements it would probably be less work to do it in ctypes CVS instead of Python SVN. Maintaining code in multiple repositories is no fun ;) (*) ctypes unittests are run by 'python ctypes/test/runtests.py', for example. Thomas

Tim Peters wrote:

[Thomas Heller]

...

... I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?)

Under VC 6 I generally just renamed a copy of "a similar" existing project file and used a text editor to change the paths. I haven't tried that under the 7.1 MSDev.

That's what I did when I created _ctypes_test.vcproj from _ctypes.vcproj. I had to insert a new guid though.