Hello, I am curious about the status of unit tests of numpy for pypy. Where can I get actual information about it? I have installed pypy 4.0.1, numpypy for it (tag 4.0.1) and launched numpy tests using numpy.test() Looks like there is a plenty of failed tests:
Ran 4157 tests in 15.592s FAILED (KNOWNFAIL=5, SKIP=116, errors=648, failures=149)
Is it the actual state or may be I did something wrong? -- Sergey
This is the actual state On Fri, Jan 1, 2016 at 11:43 PM, Sergey Matyunin <sergeymatyunin@gmail.com> wrote:
Hello,
I am curious about the status of unit tests of numpy for pypy. Where can I get actual information about it?
I have installed pypy 4.0.1, numpypy for it (tag 4.0.1) and launched numpy tests using numpy.test() Looks like there is a plenty of failed tests:
Ran 4157 tests in 15.592s FAILED (KNOWNFAIL=5, SKIP=116, errors=648, failures=149)
Is it the actual state or may be I did something wrong?
-- Sergey _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Dear Maciej, Thanks for the answer! Does it make sense to fix failing tests? Or may be they were left as is because they are irrelevant? Do numpypy developers use these tests or some other tests? Does any guidelines exist for numpypy developers? Do I need to create an issue for each PR? Is this maillist the right place for asking that questions? On Fri, Jan 1, 2016 at 10:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
This is the actual state
On Fri, Jan 1, 2016 at 11:43 PM, Sergey Matyunin <sergeymatyunin@gmail.com> wrote:
Hello,
I am curious about the status of unit tests of numpy for pypy. Where can I get actual information about it?
I have installed pypy 4.0.1, numpypy for it (tag 4.0.1) and launched numpy tests using numpy.test() Looks like there is a plenty of failed tests:
Ran 4157 tests in 15.592s FAILED (KNOWNFAIL=5, SKIP=116, errors=648, failures=149)
Is it the actual state or may be I did something wrong?
-- Sergey _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Sergey
yes, it makes sense to fix failing tests, they're real problems. You don't need to create the issue and you can just do the PR. This is the appropriate mailing list, but we work a lot through IRC On Sat, Jan 2, 2016 at 1:44 AM, Sergey Matyunin <sergeymatyunin@gmail.com> wrote:
Dear Maciej,
Thanks for the answer! Does it make sense to fix failing tests? Or may be they were left as is because they are irrelevant?
Do numpypy developers use these tests or some other tests? Does any guidelines exist for numpypy developers? Do I need to create an issue for each PR?
Is this maillist the right place for asking that questions?
On Fri, Jan 1, 2016 at 10:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
This is the actual state
On Fri, Jan 1, 2016 at 11:43 PM, Sergey Matyunin <sergeymatyunin@gmail.com> wrote:
Hello,
I am curious about the status of unit tests of numpy for pypy. Where can I get actual information about it?
I have installed pypy 4.0.1, numpypy for it (tag 4.0.1) and launched numpy tests using numpy.test() Looks like there is a plenty of failed tests:
Ran 4157 tests in 15.592s FAILED (KNOWNFAIL=5, SKIP=116, errors=648, failures=149)
Is it the actual state or may be I did something wrong?
-- Sergey _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Sergey
On 02/01/16 01:44, Sergey Matyunin wrote:
Dear Maciej,
Thanks for the answer! Does it make sense to fix failing tests? Or may be they were left as is because they are irrelevant?
Do numpypy developers use these tests or some other tests? Does any guidelines exist for numpypy developers? Do I need to create an issue for each PR?
Is this maillist the right place for asking that questions?
The test suite in the pypy/numpy repo is forked directly from the upstream numpy repo. We so far consider our inability to run them failures in the pypy/pypy micronumpy module. You should not fix these tests or the pypy/numpy repo itself for most of these issues, rather the essence of the failing test should be rewritten in pypy/pypy/module/micronumpy/tests/test*.py and fixed there. Some of the failures may actually require modification of the pypy/numpy repo, but that would be only after we exhaust options to make our micronumpy implementation 100% compatible. And just to point out one such exception to the rule of modifying only micronumpy: One of the most prevalent test failures (as can be seen here https://gist.github.com/mattip/2e6f05f1900eb6a9fd99 ) is the lack of numpy.ndarray.partition. I have a plan to reimplement this in the pypy/numpy repo in cffi (see here for an outline https://gist.github.com/mattip/ab34268b049b859554ad ) rather than rewriting it in rpython, but have not been able to get around to actually doing it. Feel free to ask here or on IRC for more help Matti
Thanks Maciej, Matti! That was really helpful! Could you please point me to the place where I could find overview of numpypy architecture? (if it exists) I got some vision after looking at the code. I would be grateful if someone says if it’s correct. -Implementation of numpy for pypy is splitted into two parts: micronumpy and numpypy -micronumpy is implementation of basic concepts of numpy inside pypy. Source code is in https://bitbucket.org/pypy/pypy/ ->pypy/module/micronumpy. --micronumpy is compiled together with pypy itself and cannot be compiled separately. --micronumpy uses timsort instead of quicksort in original numpy -numpypy implements the rest of the library (not everything is implemented yet). Source code is in https://bitbucket.org/pypy/numpypy. This module must be installed manually. --numpypy uses basic entities from micronumpy --Source code of numpypy contains source codes of numpy with a bunch of fixes. Code is partially unused because versions from micronumpy is used instead. Looking at the code is the only way to find out which part of the code is used. Also I have a couple of naive questions about pypy in general. I investigated very little here. Feel free to skip. -Compiling pypy for linux on takes about 1 hour, right? -Does faster build mode exist? Probably with less optimizations (without -O3 etc.) -Is there way to rebuild only micronumpy? -How do developers usually debug modules such as micronumpy? I am curious about tools and techniques. On Sat, Jan 2, 2016 at 6:17 PM, Matti Picus <matti.picus@gmail.com> wrote:
On 02/01/16 01:44, Sergey Matyunin wrote:
Dear Maciej,
Thanks for the answer! Does it make sense to fix failing tests? Or may be they were left as is because they are irrelevant?
Do numpypy developers use these tests or some other tests? Does any guidelines exist for numpypy developers? Do I need to create an issue for each PR?
Is this maillist the right place for asking that questions?
The test suite in the pypy/numpy repo is forked directly from the upstream numpy repo. We so far consider our inability to run them failures in the pypy/pypy micronumpy module. You should not fix these tests or the pypy/numpy repo itself for most of these issues, rather the essence of the failing test should be rewritten in pypy/pypy/module/micronumpy/tests/test*.py and fixed there. Some of the failures may actually require modification of the pypy/numpy repo, but that would be only after we exhaust options to make our micronumpy implementation 100% compatible.
And just to point out one such exception to the rule of modifying only micronumpy: One of the most prevalent test failures (as can be seen here https://gist.github.com/mattip/2e6f05f1900eb6a9fd99 ) is the lack of numpy.ndarray.partition. I have a plan to reimplement this in the pypy/numpy repo in cffi (see here for an outline https://gist.github.com/mattip/ab34268b049b859554ad ) rather than rewriting it in rpython, but have not been able to get around to actually doing it.
Feel free to ask here or on IRC for more help
Matti
-- Сергей
On Sun, 2016-01-03 at 02:07 +0100, Sergey Matyunin wrote:
-Compiling pypy for linux on takes about 1 hour, right?
Something in this range, depending on your hardware.
-Does faster build mode exist? Probably with less optimizations (without -O3 etc.)
As you might have already noticed, most of the time goes into *translation* and not compilation itself. Unfortunately, translation is still single-threaded, and therefore doesn't benefit from having more cores. One thing that you definitively should do is to translate PyPy with the latest version of PyPy instead of CPython, this makes a huge difference. I think it is also still possible to disable gc to speed the translation up a bit, but I'm not sure if this makes much sense (see below).
-Is there way to rebuild only micronumpy? -How do developers usually debug modules such as micronumpy? I am curious about tools and techniques.
PyPy has a interpreted mode, in which the interpreter is interpreted, rather than translated and compiled. This mode is way too slow for normal usage, but it's good enough to run most of the tests. The developers usually write tests for the functionality they want to implement and make sure they fail, then implement it and make sure they pass in the interpreted mode (without doing full translations between the iterations). Only then they run a full translation or wait for a nightly, and hopefully the tests still pass for the translated version as well. P.S. As a side note, I found it curious and amusing that lots of people are talking TDD, but apparently for a project of complexity and scale of PyPy, there is simply no other practical way to do development, irrespectively of whether you like it or not :-) P.P.S. Feel free to re-use my email for a FAQ and such, if something along these lines isn't already in there... -- Sincerely yours, Yury V. Zaytsev
Thank you, Yury. Looks like your lines are already in FAQ. At least I couldn't find reasonable way to update any chapter. Thing look clear in theory. However I cannot go on in practice. How to launch any test for micronumpy in interactive mode? I suppose it should be possible to import some modules from micronumpy using interactive mode of pypy. I check out branch release-4.0.x, then ~/work/pypy/pypy_src$ python pypy/bin/pyinteractive.py --allworkingmodules -c "import pypy.module.micronumpy.MultiArrayModule" I complains about signal module and fails. The same happens when I use python 2.7 and pypy 4.0.1 for launching pyinteractive. The same for module micronumpy.ctor. Whole output is here: https://gist.github.com/serge-m/d3f9f9863e15fc5c6af2 What am I doing wrong? In general I want to run a test for micronumpy. Then make it debuggable to see how micronumpy works. I also tried to use pytest and test. I extracted a single test from pypy/pypy_src/pypy/module/micronumpy/test/test_selection.py into test_selection_2.py to make things faster. Output is here: https://gist.github.com/serge-m/3c51f35c702cc57b00c2 On Sun, Jan 3, 2016 at 8:19 AM, Yury V. Zaytsev <yury@shurup.com> wrote:
On Sun, 2016-01-03 at 02:07 +0100, Sergey Matyunin wrote:
-Compiling pypy for linux on takes about 1 hour, right?
Something in this range, depending on your hardware.
-Does faster build mode exist? Probably with less optimizations (without -O3 etc.)
As you might have already noticed, most of the time goes into *translation* and not compilation itself. Unfortunately, translation is still single-threaded, and therefore doesn't benefit from having more cores. One thing that you definitively should do is to translate PyPy with the latest version of PyPy instead of CPython, this makes a huge difference. I think it is also still possible to disable gc to speed the translation up a bit, but I'm not sure if this makes much sense (see below).
-Is there way to rebuild only micronumpy? -How do developers usually debug modules such as micronumpy? I am curious about tools and techniques.
PyPy has a interpreted mode, in which the interpreter is interpreted, rather than translated and compiled. This mode is way too slow for normal usage, but it's good enough to run most of the tests.
The developers usually write tests for the functionality they want to implement and make sure they fail, then implement it and make sure they pass in the interpreted mode (without doing full translations between the iterations). Only then they run a full translation or wait for a nightly, and hopefully the tests still pass for the translated version as well.
P.S. As a side note, I found it curious and amusing that lots of people are talking TDD, but apparently for a project of complexity and scale of PyPy, there is simply no other practical way to do development, irrespectively of whether you like it or not :-)
P.P.S. Feel free to re-use my email for a FAQ and such, if something along these lines isn't already in there...
-- Sincerely yours, Yury V. Zaytsev
-- Сергей
Hello, to launch a single test you can do it that way, assuming you're in pypy's top level and that py.test is installed on your system: py.test pypy/module/micronumpy/test/test_ndarray.py -k test_array_indexing_bool On Sun, Jan 3, 2016 at 9:53 PM, Sergey Matyunin <sergeymatyunin@gmail.com> wrote:
Thank you, Yury. Looks like your lines are already in FAQ. At least I couldn't find reasonable way to update any chapter.
Thing look clear in theory. However I cannot go on in practice. How to launch any test for micronumpy in interactive mode?
I suppose it should be possible to import some modules from micronumpy using interactive mode of pypy. I check out branch release-4.0.x, then ~/work/pypy/pypy_src$ python pypy/bin/pyinteractive.py --allworkingmodules -c "import pypy.module.micronumpy.MultiArrayModule" I complains about signal module and fails. The same happens when I use python 2.7 and pypy 4.0.1 for launching pyinteractive. The same for module micronumpy.ctor. Whole output is here: https://gist.github.com/serge-m/d3f9f9863e15fc5c6af2
What am I doing wrong?
In general I want to run a test for micronumpy. Then make it debuggable to see how micronumpy works.
I also tried to use pytest and test. I extracted a single test from pypy/pypy_src/pypy/module/micronumpy/test/test_selection.py into test_selection_2.py to make things faster. Output is here: https://gist.github.com/serge-m/3c51f35c702cc57b00c2
On Sun, Jan 3, 2016 at 8:19 AM, Yury V. Zaytsev <yury@shurup.com> wrote:
On Sun, 2016-01-03 at 02:07 +0100, Sergey Matyunin wrote:
-Compiling pypy for linux on takes about 1 hour, right?
Something in this range, depending on your hardware.
-Does faster build mode exist? Probably with less optimizations (without -O3 etc.)
As you might have already noticed, most of the time goes into *translation* and not compilation itself. Unfortunately, translation is still single-threaded, and therefore doesn't benefit from having more cores. One thing that you definitively should do is to translate PyPy with the latest version of PyPy instead of CPython, this makes a huge difference. I think it is also still possible to disable gc to speed the translation up a bit, but I'm not sure if this makes much sense (see below).
-Is there way to rebuild only micronumpy? -How do developers usually debug modules such as micronumpy? I am curious about tools and techniques.
PyPy has a interpreted mode, in which the interpreter is interpreted, rather than translated and compiled. This mode is way too slow for normal usage, but it's good enough to run most of the tests.
The developers usually write tests for the functionality they want to implement and make sure they fail, then implement it and make sure they pass in the interpreted mode (without doing full translations between the iterations). Only then they run a full translation or wait for a nightly, and hopefully the tests still pass for the translated version as well.
P.S. As a side note, I found it curious and amusing that lots of people are talking TDD, but apparently for a project of complexity and scale of PyPy, there is simply no other practical way to do development, irrespectively of whether you like it or not :-)
P.P.S. Feel free to re-use my email for a FAQ and such, if something along these lines isn't already in there...
-- Sincerely yours, Yury V. Zaytsev
-- Сергей _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Vincent Legoll
participants (5)
-
Maciej Fijalkowski -
Matti Picus -
Sergey Matyunin -
Vincent Legoll -
Yury V. Zaytsev