Re: [pypy-dev] win32 buildbot
Hi Matti, I've just updated the win32 slave, hopefully nightly builds will work. It seems that the instructions you linked are not correct anymore though as the paths <base_dir>\tcltk\* no longer exist. Another note: I'm still building pypy with pypy 2.5.1 (which is probably bad, because new versions are probably a lot faster, and are not getting regular testing on the buildbot) since later versions used to abort with mysterious OOM errors. Have you been able to build pypy on win32 with latest pypy and/or reproduce the errors I'm getting? -- Sincerely yours, Yury V. Zaytsev On Fri, 21 Apr 2017, Matti Picus wrote:
I had to update the zip, the sha256 is now different. Use the one on the link
http://doc.pypy.org/en/latest/windows.html#abridged-method-for-ojit-builds-u...
On 21/04/17 16:06, Matti Picus wrote:
Hi. The buildbot seems stuck, could you restart it?
I have updated the external build dependencies, available at https://bitbucket.org/pypy/pypy/downloads/local_5.8.zip
Could you also update the buildbot with these? The checksum is
cc375a677aabcb46a50b96f23bdebd0b526cbb02202b85a7148845e2849ffd6d
which I calculated with hashlib.sha256(open(r'local_5.8.zip').read()).digesthex()
On 06/05/17 23:45, Yury V. Zaytsev wrote:
Hi Matti,
I've just updated the win32 slave, hopefully nightly builds will work. It seems that the instructions you linked are not correct anymore though as the paths <base_dir>\tcltk\* no longer exist.
Thanks, that should get us one step closer to a successful win32 packaging run on the py3.5 branch. I hope those files are no longer needed, we will see tonight
Another note: I'm still building pypy with pypy 2.5.1 (which is probably bad, because new versions are probably a lot faster, and are not getting regular testing on the buildbot) since later versions used to abort with mysterious OOM errors. Have you been able to build pypy on win32 with latest pypy and/or reproduce the errors I'm getting?
Yes, it is worth updating to 5.7.1. How much memory does the buildslave have? The buildslave uses the Microsoft Python Tools for Python 2.7 compiler bundle. Perhaps it needs more memory than the Visual Studio 9.0 compiler I am using, and so requires more than 8GB? Matti
On Sun, 7 May 2017, Matti Picus wrote:
Yes, it is worth updating to 5.7.1. How much memory does the buildslave have? The buildslave uses the Microsoft Python Tools for Python 2.7 compiler bundle. Perhaps it needs more memory than the Visual Studio 9.0 compiler I am using, and so requires more than 8GB?
It currently has 16G RAM, and I'm sorry, but I don't think I can add more at the moment. In as far as compiler is concerned, the Python Tools compiler should be exactly the same as MSVC9 in terms of resource consumption (the difference is in features / headers / ..., i.e. it's closer to the SDK compilers rather than to MSVC itself): https://mail.python.org/pipermail/python-dev/2014-September/136499.html Let's see if the current build runs through, and then I can try it ;-) -- Sincerely yours, Yury V. Zaytsev
On Sun, 7 May 2017, Yury V. Zaytsev wrote:
Let's see if the current build runs through, and then I can try it ;-)
Well, good news is that the compilation has worked, but bad news is that as I mentioned, the directory structure (and contents) has changed, and it now fails to find tcl/tk runtime where it expects it to be, because it picks the libraries from my old pypy installation instead of local_5.8: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3092/steps/sh... I guess the problem is that you're shipping tcl85t.dll & tk85t.dll in local_5.8, whereas it is looking for tcl85.dll & tk85.dll as they were shipped in local_2.4, and so not finding them picks the ones from pypy, consequently failing to find the runtime, so either pypy packaging script, or local package should be adjusted. I will see now if I can update pypy when the build finishes completely... -- Sincerely yours, Yury V. Zaytsev
On Sun, 7 May 2017, Yury V. Zaytsev wrote:
I will see now if I can update pypy when the build finishes completely...
So, the good news is that apparently with PyPy 5.7.1 the translation & compilation works again, so I'll stick with it, yay! The not so good news is that the TCL/TK packaging problem didn't magically solve itself, so you'd have to look into it... -- Sincerely yours, Yury V. Zaytsev
On Mon, 8 May 2017, Matti Picus wrote:
I rebuilt the local_5.8.zip, and fixed the instructions. Please try the newer package.
Hi Matti, I've just updated the build slave, let's see if this works tomorrow morning... keeping the fingers crossed ;-) -- Sincerely yours, Yury V. Zaytsev
On Mon, 8 May 2017, Yury V. Zaytsev wrote:
On Mon, 8 May 2017, Matti Picus wrote:
I rebuilt the local_5.8.zip, and fixed the instructions. Please try the newer package.
I've just updated the build slave, let's see if this works tomorrow morning... keeping the fingers crossed ;-)
So, PyPy builds again, high five! :-) Only PyPy3 bails out while compiling the TCL/TK extension, but the problem is a different one now. -- Sincerely yours, Yury V. Zaytsev
On Wed, 10 May 2017, Matti Picus wrote:
It seems own tests are taking a very long time on this machine. I have explored using jom.exe (https://github.com/qt-labs/jom) instead of nmake to allow parallel compilation, which reduces my run of rpython's translator\c\test\test_typed.py from ~700 secs to ~600 secs, where on the same machine it is under 200 secs with linux. Note that the buildbot requires over 2800 secs for the same test (see http://buildbot.pypy.org/builders/own-win-x86-32/builds/1395/steps/shell_6/l...)
It is simply so that the machine kindly provided by the folks at Dartmouth College where I migrated the Windows VM to is way slower than my old box, and has only 2 physical cores in total... Also take a note, that the same machine also does build & test runs for git-annex, which take about an hour each, so that also doesn't speedup PyPy builds. Anyways, in the mean time, it looks like I've found colo in Germany for my old machine, but I still need to: * Get it shipped to the new datacenter * Partly replace old hardware * Install it & set it up anew No ETAs, but something is moving in some direction at least. Getting hardware is for now an unsolved problem though. I need 2 x 2/3 TB SATA enterprise HDDs and 2 x enterprise SSDs + would be good to replace unreliable DDR3 RAM sticks... Stuff that I have is 5 years old, and HDDs tend to fail when they are old :-/ -- Sincerely yours, Yury V. Zaytsev
Hi Yuri. While I do not have powers to make a decision, I think it's very reasonable if pypy pays for such parts. If you can provide cost estimates, I'll try to go through the appropriate channels to have this approved, if you're interested. On Wed, May 10, 2017 at 9:53 PM, Yury V. Zaytsev <yury@shurup.com> wrote:
On Wed, 10 May 2017, Matti Picus wrote:
It seems own tests are taking a very long time on this machine. I have explored using jom.exe (https://github.com/qt-labs/jom) instead of nmake to allow parallel compilation, which reduces my run of rpython's translator\c\test\test_typed.py from ~700 secs to ~600 secs, where on the same machine it is under 200 secs with linux. Note that the buildbot requires over 2800 secs for the same test (see http://buildbot.pypy.org/builders/own-win-x86-32/builds/1395/steps/shell_6/l...)
It is simply so that the machine kindly provided by the folks at Dartmouth College where I migrated the Windows VM to is way slower than my old box, and has only 2 physical cores in total...
Also take a note, that the same machine also does build & test runs for git-annex, which take about an hour each, so that also doesn't speedup PyPy builds.
Anyways, in the mean time, it looks like I've found colo in Germany for my old machine, but I still need to:
* Get it shipped to the new datacenter * Partly replace old hardware * Install it & set it up anew
No ETAs, but something is moving in some direction at least. Getting hardware is for now an unsolved problem though. I need 2 x 2/3 TB SATA enterprise HDDs and 2 x enterprise SSDs + would be good to replace unreliable DDR3 RAM sticks... Stuff that I have is 5 years old, and HDDs tend to fail when they are old :-/
-- Sincerely yours, Yury V. Zaytsev _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
participants (3)
-
Maciej Fijalkowski -
Matti Picus -
Yury V. Zaytsev