On 2015-05-25 21:09, Ryan Gonzalez wrote:

Try building the module with -m32. The error message basically means:

"../libpython35.a is 32-bit, but what you're building is 64-bit." Gotta love ld!

Unless I've missing something, I'm already passing it to gcc.

All of the other versions build without a problem: Python 2.5-2.7 and Python 3.1-3.4, both 32-bit and 64-bit, and now Python 3.5 64-bit.

That's 15 building and 1 failing!

On May 25, 2015 3:06:01 PM CDT, MRAB python@mrabarnett.plus.com wrote:

As the subject says, I've been unable to build the regex module 

against

Python 3.5b1 for 32-bit. MingGW says:

     skipping incompatible .../libpython35.a when searching for 

-lpython35

It builds without a problem against Python 3.5 for 64-bit.

Any ideas? Should I just wait until beta 2?

On 2015-05-25 22:59, Paul Moore wrote:

On 25 May 2015 at 21:06, MRAB python@mrabarnett.plus.com wrote:

...

As the subject says, I've been unable to build the regex module against Python 3.5b1 for 32-bit. MingGW says:

skipping incompatible .../libpython35.a when searching for 

-lpython35

...

It builds without a problem against Python 3.5 for 64-bit.

Any ideas? Should I just wait until beta 2?

MinGW is (and always has been) only marginally supported, unfortunately. I'd rather it didn't break totally for 3.5, but I am anticipating some difficulties (there have been a lot of compiler-related changes with 3.5).

Could you raise a bug, including details of precisely how you tried to build the module (presumably https://pypi.python.org/pypi/regex/2015.05.10) and assign it to me? I'll try to take a look and reproduce the issue. With luck, it may be as simple as the wrong version of libpython35.a being picked up somewhere.

Just to check the obvious - you *are* using 32-bit Python 3.5b1 and a 32-bit Mingw to build the 32-bit version, and 64-bit Python 3.5b1 and a 64-bit Mingw to build the 64-bit one? (I.e., two installations of Python and two of Mingw)

I'm not sure what happened, but I'm now getting this for Python 3.5 (32-bit):

C:\Python35(32-bit)\libs/libpython35.a(dsxbs01290.o):(.idata$7+0x0): undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a' C:\Python35(32-bit)\libs/libpython35.a(dsxbs00283.o):(.idata$7+0x0): undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a' C:\Python35(32-bit)\libs/libpython35.a(dsxbs00291.o):(.idata$7+0x0): undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a' C:\Python35(32-bit)\libs/libpython35.a(dsxbs00273.o):(.idata$7+0x0): undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a' C:\Python35(32-bit)\libs/libpython35.a(dsxbs00255.o):(.idata$7+0x0): undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a' C:\Python35(32-bit)\libs/libpython35.a(dsxbs01280.o):(.idata$7+0x0): more undefined references to `_head_C__build_cpython_PCBuild_win32_libpython35_a' follow collect2: ld returned 1 exit status

All other builds, from Python 2.5 to Python 3.4, both 32-bit and 64-bit, and also Python 3.5 (64-bit), work.

The 32-bit Python says it's 32-bit and the 64-bit Python says it's 64-bit.

---8<---

C:

rem Compile for Python 3.5 (64-bit) [works] cd C:\MinGW64\bin "C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35\include" -c "D:\projects\mrab-regex\regex_3\regex_regex_unicode.c" -o "D:\projects\mrab-regex\regex_3\Release(3.5)_regex_unicode.o" "C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35\include" -c "D:\projects\mrab-regex\regex_3\regex_regex.c" -o "D:\projects\mrab-regex\regex_3\Release(3.5)_regex.o" "C:\MinGW64\bin\gcc.exe" -m64 -shared -s "D:\projects\mrab-regex\regex_3\Release(3.5)_regex_unicode.o" "D:\projects\mrab-regex\regex_3\Release(3.5)_regex.o" -L"C:\Python35\libs" -lpython35 -o "D:\projects\mrab-regex\regex_3\Release(3.5)_regex.pyd"

rem Compile for Python 3.5 (32-bit) [fails] cd C:\MinGW\bin "C:\MinGW\bin\gcc.exe" -mdll -m32 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35(32-bit)\include" -c "D:\projects\mrab-regex\regex_3\regex_regex_unicode.c" -o "D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)_regex_unicode.o" "C:\MinGW\bin\gcc.exe" -mdll -m32 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35(32-bit)\include" -c "D:\projects\mrab-regex\regex_3\regex_regex.c" -o "D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)_regex.o" "C:\MinGW\bin\gcc.exe" -m32 -shared -s "D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)_regex_unicode.o" "D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)_regex.o" -L"C:\Python35(32-bit)\libs" -lpython35 -o "D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)_regex.pyd"

---8<---

On 26 May 2015 at 07:49, Paul Moore p.f.moore@gmail.com wrote:

Do you get the same failure when using distutils to build the extension?

Hmm, I just checked and it seems that only Python 3.5 ships libpythonXY.a by default - 3.4 and earlier (at least on my machine) don't have it. Presumably you generated it yourself for the previous versions?

So I wonder if this is a difference between how you built your version previously and how the shipped version is now built. The code to build the shipped version is in Tools\msi\dev\dev.wixproj. It runs

gendef - "$(BuildPath)$(PyDllName).dll" > "$(IntermediateOutputPath)mingwlib.def" dlltool --dllname $(PyDllName).dll --def "$(IntermediateOutputPath)mingwlib.def" --output-lib "$(BuildPath)lib$(PyDllName).a" -m $(_GenDefPlatform)

which looks OK to me (_GenDefPlatform is i386 on 32-bit and i386:x86-64 on 64-bit).

Paul

On 26 May 2015 at 13:55, Steve Dower Steve.Dower@microsoft.com wrote:

The builds I am responsible for include it because someone reported an issue and was persistent and helpful enough that I fixed it for them.

That said, until MinGW agrees on a stable branch/version/fork, there seems to be a good chance that the shipped lib won't work for some people. If this is what's happened here, I see it as a good enough reason to stop shipping the lib and to add instructions on generating it instead (the gendef/dlltool dance).

Agreed. If shipping it helps, then great. If it's going to cause bug reports, let's go back to the status quo of not having it. The instructions for generating it were in the old distutils docs, now removed in the cleanup / redirection to packaging.python.org. I'm inclined to just leave it undocumented - the people who need it know how to do it or can find it, whereas documenting the process implies a level of support that we're not yet really able to provide.

Let's wait to see what the OP comes back with before making a final decision. I see a few questions:

1. Does using distutils work (as opposed to the quoted manual compile steps)? 2. Does using whatever process he used in the past to generate libpythonXY.a result in a working version?

https://docs.python.org/2.7/install/index.html#gnu-c-cygwin-mingw suggests using pexports rather than gendef. I don't know if that could produce a different result, but I can't see how... It also implicitly assumes you're using dlltool from the toolchain you'll be building with rather than using -m. Again, I can't see why that would affect the results.

Paul.

On 26 May 2015 at 18:23, MRAB python@mrabarnett.plus.com wrote:

I made libpython35.a like I did for the others and it's all working now. All the tests pass. :-)

Cool. How did you make libpython35.a? Sounds like there is some difference in the process being used for the shipped version.

Paul

On 27 May 2015 at 03:02, MRAB python@mrabarnett.plus.com wrote:

When I tried:

C:\MinGW64\bin\dlltool.exe

or:

C:\MinGW64\x86_64-w64-mingw32\bin\dlltool.exe

for the 32-bit builds, they wouldn't link.

Was that with "-m i386"? If so, then I suspect that's the issue. Steve, did you use 64-bit mingw to build the .a files? Assuming so, I guess we either stop shipping libpythonXY.a, or the instructions for building the Windows release need to clearly state that you want a 32-bit mingw on PATH for the 32-bit builds, and a 64-bit mingw on PATH for the 64-bit builds (which sounds messy and error-prone :-()

I'd be inclined to call this a mingw bug. However, I don't have the first clue where to report it.

Paul

On 27 May 2015 at 09:10, Nick Coghlan ncoghlan@gmail.com wrote:

The old distutils docs aren't gone, the top level links just moved to the distutils package docs: https://docs.python.org/3/library/distutils.html

I kept them (with the same deep link URLs) because I know there's stuff in there that isn't currently documented anywhere else. I moved them to a more obscure location because there's also stuff in there that's thoroughly outdated, and it's a non-trivial task to figure out which is which and move the still useful stuff to a more appropriate home :)

Thanks.

Your plan worked perfectly, because I never knew they were there :-)

https://docs.python.org/3/install/index.html#older-versions-of-python-and-mi... implies that the libpythonXY.a files are only needed in older versions of Python/mingw. I don't know how true that is, although I do know that mingw should be able to link directly to a DLL without needing a lib file.

It would be interesting to know if MRAB's build process can use the DLL, rather than requiring a lib file (or for that matter if distutils works without the lib file!) Paul

On 2015-06-05 01:37, MRAB wrote:

On 2015-05-27 09:25, Paul Moore wrote:

...

On 27 May 2015 at 09:10, Nick Coghlan ncoghlan@gmail.com wrote:

...

The old distutils docs aren't gone, the top level links just moved to the distutils package docs: https://docs.python.org/3/library/distutils.html

I kept them (with the same deep link URLs) because I know there's stuff in there that isn't currently documented anywhere else. I moved them to a more obscure location because there's also stuff in there that's thoroughly outdated, and it's a non-trivial task to figure out which is which and move the still useful stuff to a more appropriate home :)

Thanks.

Your plan worked perfectly, because I never knew they were there :-)

https://docs.python.org/3/install/index.html#older-versions-of-python-and-mi... implies that the libpythonXY.a files are only needed in older versions of Python/mingw. I don't know how true that is, although I do know that mingw should be able to link directly to a DLL without needing a lib file.

It would be interesting to know if MRAB's build process can use the DLL, rather than requiring a lib file (or for that matter if distutils works without the lib file!)

Steve Dower's post has prompted me to look again at building the regex module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem to want libpython??.a.

For reference, here's how I can build the regex module on Windows 8.1, 64-bit, using only MinGW64.

For Python 3.5, I can link against "python35.dll", but for earlier versions, including Python 2.7, I need "libpython.a".

I have built regex module for all of the 16 supported versions of Python (2.5-2.7, 3.1-3.5, 64-bit and 32-bit) and they have all passed the tests.

rem For Python 3.5, 64-bit. rem Can link against the Python DLL.

rem Compile "C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35-64\include" -c "D:\mrab-regex\source_regex_unicode.c" -o "D:\mrab-regex\release\3.5-64_regex_unicode.o"

"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35-64\include" -c "D:\mrab-regex\source_regex.c" -o "D:\mrab-regex\release\3.5-64_regex.o"

rem Link "C:\MinGW64\bin\gcc.exe" -m64 -shared -s "D:\mrab-regex\release\3.5-64_regex_unicode.o" "D:\mrab-regex\release\3.5-64_regex.o" -L"C:\Python35" -lpython35 -o "D:\mrab-regex\release\3.5-64_regex.pyd"

rem For Python 3.5, 32-bit. rem Can link against the Python DLL.

rem Compile "C:\MinGW64\bin\gcc.exe" -mdll -m32 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35-32\include" -c "D:\mrab-regex\source_regex_unicode.c" -o "D:\mrab-regex\release\3.5-32_regex_unicode.o"

"C:\MinGW64\bin\gcc.exe" -mdll -m32 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python35-32\include" -c "D:\mrab-regex\source_regex.c" -o "D:\mrab-regex\release\3.5-32_regex.o"

rem Link "C:\MinGW64\bin\gcc.exe" -m32 -shared -s "D:\mrab-regex\release\3.5-32_regex_unicode.o" "D:\mrab-regex\release\3.5-32_regex.o" -L"C:\Python35-32" -lpython35 -o "D:\mrab-regex\release\3.5-32_regex.pyd"

rem For Python 3.4, 64-bit. rem Need to link against the Python .a file.

rem Make libpython34.a "C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe" - "C:\Windows\System32\python34.dll" >"C:\Python34-64\libs\libpython34.def"

"C:\MinGW64\bin\dlltool.exe" --dllname python34.dll --def "C:\Python34-64\libs\libpython34.def" --output-lib "C:\Python34-64\libs\libpython34.a"

rem Compile "C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python34-64\include" -c "D:\mrab-regex\source_regex_unicode.c" -o "D:\mrab-regex\release\3.4-64_regex_unicode.o"

rem Link "C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python34-64\include" -c "D:\mrab-regex\source_regex.c" -o "D:\mrab-regex\release\3.4-64_regex.o"

"C:\MinGW64\bin\gcc.exe" -m64 -shared -s "D:\mrab-regex\release\3.4-64_regex_unicode.o" "D:\mrab-regex\release\3.4-64_regex.o" -L"C:\Python34-64\libs" -lpython34 -o "D:\mrab-regex\release\3.4-64_regex.pyd"

rem For Python 3.4, 32-bit. rem Need to link against the Python .a file.

rem Make libpython34.a "C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe" - "C:\Windows\SysWOW64\python34.dll" >"C:\Python34-32\libs\libpython34.def"

"C:\MinGW64\x86_64-w64-mingw32\bin\dlltool.exe" --as-flags=--32 -m i386 --dllname python34.dll --def "C:\Python34-32\libs\libpython34.def" --output-lib "C:\Python34-32\libs\libpython34.a"

rem Compile "C:\MinGW64\bin\gcc.exe" -mdll -m32 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python34-32\include" -c "D:\mrab-regex\source_regex_unicode.c" -o "D:\mrab-regex\release\3.4-32_regex_unicode.o"

"C:\MinGW64\bin\gcc.exe" -mdll -m32 -O2 -Wall -Wsign-compare -Wconversion -I"C:\Python34-32\include" -c "D:\mrab-regex\source_regex.c" -o "D:\mrab-regex\release\3.4-32_regex.o"

rem Link "C:\MinGW64\bin\gcc.exe" -m32 -shared -s "D:\mrab-regex\release\3.4-32_regex_unicode.o" "D:\mrab-regex\release\3.4-32_regex.o" -L"C:\Python34-32\libs" -lpython34 -o "D:\mrab-regex\release\3.4-32_regex.pyd"

rem For earlier versions of Python, follow the pattern of Python 3.4.

On 05/06/2015 01:37, MRAB wrote:

Steve Dower's post has prompted me to look again at building the regex module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem to want libpython??.a.

This http://bugs.python.org/issue24385 should interest you.