Status of MS Windows CE port
Greetings! I'm currently trying to assess the effort required for a CE port. I'm already aware of the project at http://pythonce.sourceforge.net, but I find it a waste of time to have two separate projects which then require syncing changes back and forth. Questions: - What are the feelings towards a port in the mainline? I'm aware that you are trying to get 2.6 and 3.0 out of the door and are probably not willing to risk such changes to the release code right now, but in general? - Is anyone already working on this? I know that Brad Clements made an attempt some years ago but didn't get it integrated, are there any others? Maybe even any ongoing work? - I see that you are using Subversion and Bazaar, would it be possible to get a feature branch for the CE port? I'm thinking about the effort going on at pythonce.sf.net and some further porting that I need for my specific platform. IMHO both would be better off in the same repository as your upstream sources, since merging changes back and forth is much easier that way. Thanks! Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
On Tue, Sep 23, 2008 at 1:52 AM, Ulrich Eckhardt <eckhardt@satorlaser.com> wrote:
In general, we try to ensure that there is an active maintainer, preferably more than one. As long as it doesn't make the code that much harder to maintain, it is desirable to support more platforms.
I'm not sure if anyone here knows. I haven't seen anything on this list, but that doesn't mean there isn't another list either on python.org or elsewhere that does have the information. Try searching if you haven't.
It's possible. The best way to get the ball rolling is to produce some patches against the mainline. Demonstrate what you can do and privileges will follow. It's a similar process to becoming a python-committer. Perhaps the best way to start is using Bazaar or some distributed version control system for your patches. Once you've demonstrated your ability and intent, we can make a branch. Cheers, n
On Wednesday 24 September 2008, Neal Norwitz wrote:
Actually, looking at the sources, there are several places where even existing win32 ports would benefit. There are four main differences between CE and the desktop variants: 1. TCHAR is always WCHAR 2. some APIs are missing 3. the vendor-supplied C or C++ stdlibs are rather minimal 4. you always cross-compile The first point is interesting to the desktop win32 variants because everything since NT4 (or even 3.x) also used UCS2 or UTF-16 internally. Only win9x and ME(?) use 8-bit chars, and since support for them is being dropped, this would save a few conversions while allowing better Unicode support. I think getting Python to compile with _UNICODE #defined will be my first step. The second point is mainly about features that don't exist like a current directory for a process or environment variables. Some other APIs don't support all parameters or similar differences. This should be rather easy to do. The third means that functions like e.g. system() are simply not available in the supplied C API. If the functionality itself exists, it can be built on top of the win32 API. There might be a few cases where the desktop variants would benefit from one less adaption layer, too. The last one only affects the build system and possibly the integration of unit testing. pythonce.sf.net already have one based on SCons, which is nice and easily extended. Being able to build 64 bit variants on 32 bit systems and vice versa might be another benefit.
It's possible. The best way to get the ball rolling is to produce some patches against the mainline.
Hmmm, which mainline? 2.5, 2.6 or 3.0? Up to now I have only looked at 2.5, because that one is stable. However, quite a few of above changes are not suitable for a stable version though, so I'll either have to restrict them to only CE or rather apply them to version 3. We can discuss that when I have actual patches.
I'll be digging into Bazaar, looks like an interesting topic by itself anyway. You'll hear from me! cheers Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
1. TCHAR is always WCHAR
Python shouldn't be using TCHAR at all; that getpathp uses it is misguided (IMO).
Please don't. Whether or not _UNICODE is defined should have no effect.
The third means that functions like e.g. system() are simply not available in the supplied C API.
In this case, not defining HAVE_SYSTEM would be already sufficient, no?
Again, in the specific case of system(), the replacement is already there, in form of the subprocess module· There might be other places also, but apart from stdio (which is mostly gone in 3.0) and memory management, I can't think of any.
Wouldn't it be easier to extend the VS projects? VS targets CE fairly well, no?
The mainline is 2.6 (i.e. trunk). 2.5 and 3.0 are branches.
I guess none of your changes are suitable for 2.5 (as they probably constitute new features). You should really target trunk (i.e. 2.7) when you produce patches. Regards, Martin
On Thursday 25 September 2008, Martin v. Löwis wrote:
It already does, see below...
Well, currently it does make a difference. Simple example: CreateFile(). One of the parameters is actually documented at LPCTSTR, which is a TCHAR string. That means that under CE, CreateFile(), which is actually a macro, resolves to CreateFileW and the existing code that passes a char-string doesn't work.
I'll take a look, I actually don't know.
VS2005 works really good with CE5 and later. My problem here is that I can't provide a projectfile that suits everyone, because different devices have different SDKs and each configuration/SDK/CPU is another entry in that file. Further, the SDKs or devices differ in what they support, the platform CE as such doesn't exist. Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
Please don't. Whether or not _UNICODE is defined should have no effect.
Well, currently it does make a difference. Simple example: CreateFile().
It's not so simple: Python doesn't actually call CreateFile, AFAICT (well, it does, in _msi.c, but I hope you aren't worried about that module on CE).
True, but irrelevant: Python doesn't actually use CreateFile. Instead, it calls both CreateFileA and CreateFileW (in posixmodule.c).
Hmm. And you *can* provide an SCons file that supports all the SDKs? Regards, Martin
On Friday 26 September 2008, Martin v. Löwis wrote:
Martin, CreateFile() was just used as an example. You can substitute it with LoadString() or CreateProcess() if you like, the problem remains the same. [about using SCons for building]
And you *can* provide an SCons file that supports all the SDKs?
No, but I can provide one that allows parametrisation. ;) Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
Ulrich Eckhardt wrote:
However, the solution should be different from the one you propose. I don't know what call of CreateProcess you are referring to specifically, but I think they should all be changed to call CreateProcessW. Again, whether or not _UNICODE is defined should have no effect. If it does, it's a bug, and the solution is not to sprinkle TCHAR all over the place.
And, with proper parametrization, then supports all SDKs? Regards, Martin
On Tuesday 30 September 2008, Martin v. Löwis wrote:
I think we're misunderstanding each other, because that is exactly the solution I'm targetting. I'm aware that TCHAR is just a hack to ease transition between obsolete MS Windows versions and NT and later. However, current state is that Python uses CreateProcessA() and changing that is not always trivial. Therefore, my first step in porting is also to remove the dependency on TCHAR, i.e. replace things like CreateProcess() with preferably CreateProcessW() or alternatively CreateProcessA(). Just #defining _UNICODE in the build already turns up around 50 places in pythoncore that need work. I'll send patches soon.
Hopefully, yes, but I'm not going to make any claims which I'm not sure about. SCons is just convenient because the PythonCE project already uses it but I'm not adamant on that matter. The approach of Jack Jansen also sounds good, generating the projectfiles (which are XML, btw) from a Python script also sounds nice. cheers Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
Martin v. Löwis wrote:
I used to follow the Python-CE list. I believe that Luke Dunstan (who has done the most recent work on a port) discussed a previous attempt to get the patch-set accepted into trunk but it was rejected because it required too *many* changes. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/
Perhaps. But I also recall committing patches that suggested a number of changes necessary for CE (e.g. r46819, from #1495999 by Lukas Dunstan, and r46064, from #1492356, by the same author). I don't recall rejecting a patch from him, but I may have forgotten, or somebody else may have rejected such a patch. I do recall a certain unhappiness with the patches. If you look at http://bugs.python.org/issue1492356, it starts with "This patch contains part of the changes". What would be the rationale for including part of the changes, but not all of them? This appears to be a widespread view of port maintainers: let's try to get some changes in. Apparently, some believe that suggesting fewer changes might be more successful. I feel uncomfortable with that, because I then don't know how many more times they will approach me with another collection of a few changes, and whether learning what the later changes are might have impact on the earlier changes (which I had already accepted). Regards, Martin
On Tue, Sep 23, 2008 at 1:52 AM, Ulrich Eckhardt <eckhardt@satorlaser.com> wrote:
In general, we try to ensure that there is an active maintainer, preferably more than one. As long as it doesn't make the code that much harder to maintain, it is desirable to support more platforms.
I'm not sure if anyone here knows. I haven't seen anything on this list, but that doesn't mean there isn't another list either on python.org or elsewhere that does have the information. Try searching if you haven't.
It's possible. The best way to get the ball rolling is to produce some patches against the mainline. Demonstrate what you can do and privileges will follow. It's a similar process to becoming a python-committer. Perhaps the best way to start is using Bazaar or some distributed version control system for your patches. Once you've demonstrated your ability and intent, we can make a branch. Cheers, n
On Wednesday 24 September 2008, Neal Norwitz wrote:
Actually, looking at the sources, there are several places where even existing win32 ports would benefit. There are four main differences between CE and the desktop variants: 1. TCHAR is always WCHAR 2. some APIs are missing 3. the vendor-supplied C or C++ stdlibs are rather minimal 4. you always cross-compile The first point is interesting to the desktop win32 variants because everything since NT4 (or even 3.x) also used UCS2 or UTF-16 internally. Only win9x and ME(?) use 8-bit chars, and since support for them is being dropped, this would save a few conversions while allowing better Unicode support. I think getting Python to compile with _UNICODE #defined will be my first step. The second point is mainly about features that don't exist like a current directory for a process or environment variables. Some other APIs don't support all parameters or similar differences. This should be rather easy to do. The third means that functions like e.g. system() are simply not available in the supplied C API. If the functionality itself exists, it can be built on top of the win32 API. There might be a few cases where the desktop variants would benefit from one less adaption layer, too. The last one only affects the build system and possibly the integration of unit testing. pythonce.sf.net already have one based on SCons, which is nice and easily extended. Being able to build 64 bit variants on 32 bit systems and vice versa might be another benefit.
It's possible. The best way to get the ball rolling is to produce some patches against the mainline.
Hmmm, which mainline? 2.5, 2.6 or 3.0? Up to now I have only looked at 2.5, because that one is stable. However, quite a few of above changes are not suitable for a stable version though, so I'll either have to restrict them to only CE or rather apply them to version 3. We can discuss that when I have actual patches.
I'll be digging into Bazaar, looks like an interesting topic by itself anyway. You'll hear from me! cheers Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
1. TCHAR is always WCHAR
Python shouldn't be using TCHAR at all; that getpathp uses it is misguided (IMO).
Please don't. Whether or not _UNICODE is defined should have no effect.
The third means that functions like e.g. system() are simply not available in the supplied C API.
In this case, not defining HAVE_SYSTEM would be already sufficient, no?
Again, in the specific case of system(), the replacement is already there, in form of the subprocess module· There might be other places also, but apart from stdio (which is mostly gone in 3.0) and memory management, I can't think of any.
Wouldn't it be easier to extend the VS projects? VS targets CE fairly well, no?
The mainline is 2.6 (i.e. trunk). 2.5 and 3.0 are branches.
I guess none of your changes are suitable for 2.5 (as they probably constitute new features). You should really target trunk (i.e. 2.7) when you produce patches. Regards, Martin
On Thursday 25 September 2008, Martin v. Löwis wrote:
It already does, see below...
Well, currently it does make a difference. Simple example: CreateFile(). One of the parameters is actually documented at LPCTSTR, which is a TCHAR string. That means that under CE, CreateFile(), which is actually a macro, resolves to CreateFileW and the existing code that passes a char-string doesn't work.
I'll take a look, I actually don't know.
VS2005 works really good with CE5 and later. My problem here is that I can't provide a projectfile that suits everyone, because different devices have different SDKs and each configuration/SDK/CPU is another entry in that file. Further, the SDKs or devices differ in what they support, the platform CE as such doesn't exist. Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
Please don't. Whether or not _UNICODE is defined should have no effect.
Well, currently it does make a difference. Simple example: CreateFile().
It's not so simple: Python doesn't actually call CreateFile, AFAICT (well, it does, in _msi.c, but I hope you aren't worried about that module on CE).
True, but irrelevant: Python doesn't actually use CreateFile. Instead, it calls both CreateFileA and CreateFileW (in posixmodule.c).
Hmm. And you *can* provide an SCons file that supports all the SDKs? Regards, Martin
On Friday 26 September 2008, Martin v. Löwis wrote:
Martin, CreateFile() was just used as an example. You can substitute it with LoadString() or CreateProcess() if you like, the problem remains the same. [about using SCons for building]
And you *can* provide an SCons file that supports all the SDKs?
No, but I can provide one that allows parametrisation. ;) Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
Ulrich Eckhardt wrote:
However, the solution should be different from the one you propose. I don't know what call of CreateProcess you are referring to specifically, but I think they should all be changed to call CreateProcessW. Again, whether or not _UNICODE is defined should have no effect. If it does, it's a bug, and the solution is not to sprinkle TCHAR all over the place.
And, with proper parametrization, then supports all SDKs? Regards, Martin
On Tuesday 30 September 2008, Martin v. Löwis wrote:
I think we're misunderstanding each other, because that is exactly the solution I'm targetting. I'm aware that TCHAR is just a hack to ease transition between obsolete MS Windows versions and NT and later. However, current state is that Python uses CreateProcessA() and changing that is not always trivial. Therefore, my first step in porting is also to remove the dependency on TCHAR, i.e. replace things like CreateProcess() with preferably CreateProcessW() or alternatively CreateProcessA(). Just #defining _UNICODE in the build already turns up around 50 places in pythoncore that need work. I'll send patches soon.
Hopefully, yes, but I'm not going to make any claims which I'm not sure about. SCons is just convenient because the PythonCE project already uses it but I'm not adamant on that matter. The approach of Jack Jansen also sounds good, generating the projectfiles (which are XML, btw) from a Python script also sounds nice. cheers Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************
Martin v. Löwis wrote:
I used to follow the Python-CE list. I believe that Luke Dunstan (who has done the most recent work on a port) discussed a previous attempt to get the patch-set accepted into trunk but it was rejected because it required too *many* changes. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/
Perhaps. But I also recall committing patches that suggested a number of changes necessary for CE (e.g. r46819, from #1495999 by Lukas Dunstan, and r46064, from #1492356, by the same author). I don't recall rejecting a patch from him, but I may have forgotten, or somebody else may have rejected such a patch. I do recall a certain unhappiness with the patches. If you look at http://bugs.python.org/issue1492356, it starts with "This patch contains part of the changes". What would be the rationale for including part of the changes, but not all of them? This appears to be a widespread view of port maintainers: let's try to get some changes in. Apparently, some believe that suggesting fewer changes might be more successful. I feel uncomfortable with that, because I then don't know how many more times they will approach me with another collection of a few changes, and whether learning what the later changes are might have impact on the earlier changes (which I had already accepted). Regards, Martin
participants (4)
-
"Martin v. Löwis" -
Michael Foord -
Neal Norwitz -
Ulrich Eckhardt