Mailman 3 March 2017 - pypy-dev

Remaining test_importlib failures
by Manuel Jacob March 16, 2017

March 16, 2017

Hi, I'm currently trying to fix the remaining test_importlib failures (http://buildbot.pypy.org/summary/longrepr?testname=%3Aunmodified&builder=py…), of which some are a bit obscure. Most (or all) tests from test_importlib.frozen are failing because PyPy doesn't really have frozen modules (in the sense of CPython, where these are a special kind of modules besides normal Python modules and extension modules). Does it sound reasonable to skip these tests completely? Another class of … [View More]

2 1

What is RuntimeTypeInfo?
by John Zhang March 16, 2017

March 16, 2017

Hi all, Can anyone does the favour for me by explaining the story of RuntimeTypeInfo in RPython? I couldn’t quite understand the description on the online documentation (http://rpython.readthedocs.io/en/latest/rtyper.html#opaque-types). From looking at the source code, and inspecting the generated C source, RTTI doesn’t seem to be used at all at runtime. The generated C source seems to just return an uninitialised function pointer on the stack when compiling the following code: from rpython.… [View More]

4 6

Programming Language Implementation Summer School (PLISS)
by Carl Friedrich Bolz March 15, 2017

March 15, 2017

============================================================================ Programming Language Implementation Summer School (PLISS) May 20-27, 2017, Bertinoro Italy https://pliss2017.github.io/ ============================================================================ Programming languages are our interface to the myriad of computer systems we interact with on a daily basis. They allow us to craft complex sequences of operations at … [View More]

1 0

Freeze of pypy2 and pypy3 for upcoming release
by Matti Picus March 15, 2017

March 15, 2017

1 0

jitted region start address and size
by Shubha Ramani March 10, 2017

March 10, 2017

1) in assemble_loop:Is this correct and if not, what is the correct answer ?starting address of jitted region: looppos + rawstartsize of jitted region:: size_excluding_failure_stuff - looppos 2) in assemble_bridge, Is this correct and if not, what is the correct answer ?starting address of jitted region: startpos + rawstartsize of jitted region: codeendpos - startpos Thanks, Shubha

1 0

looptoken.number of bridge
by Shubha Ramani March 10, 2017

March 10, 2017

1) Is it a true statement to say that the looptoken.number between an original loop and a bridge stays the same ? 2) Is it a true statement to say "loopname" as passed into assemble_loop will apply to a bridge if indeed statement 1)above is true - if a loop and bridge share the same original looptoken.number, do they also share the same loopname(as passed in from get_printable_location) ? Thanks, Shubha

1 0

Re: [pypy-dev] Speeds of various utf8 operations
by Richard Plangger March 8, 2017

March 8, 2017

Yes ;)… At some point, now I'm still experimenting with the operations we think we need. Cheers, Richard On Mar 8, 2017 6:50 PM, "David Edelsohn" <dje.gcc(a)gmail.com> wrote: > And POWER VSX and Z VX? ;-) > > - David > > > On Wed, Mar 8, 2017 at 12:17 PM, Richard Plangger <planrichi(a)gmail.com> > wrote: > > Hi, > > > > as we discussed on the sprint I have now experimented with an SSE/AVX > > implementation to 'len(utf8 string)' (this … [View More]includes a check if it is > > valid utf8). Since this is related to this mailing list thread I'll just > > add it here! > > > > I ran some small measurements on it: > > > > Here some explanation of the names: > > > > pypy-seq-.*: sequential implementation in C, nothing fancy just a > baseline > > pypy-vec-sse4-.*: implementation using sse4 (128 bit registers) > > pypy-vec-avx2-.*: implementation using avx2 (256 bit registers) > > libunistring-.*: benchmarking the function u8_check in that gnu library, > > NO length is calculated > > mystrlenutf8-.*: some guy doing length calculation (no validity check) > > only using 64bit words instead of per byte iteration. (see here [1]) > > > > .*-news-de: html of a german website (has quite a lot of 2 byte code > > points), ~ 1MB > > .*-news-cn: worldjournarl.com -> mandarin (html website with lots of 4 > > byte code points) ~ 700 KB > > .*-tipitaka-thai: xml page of some religious text with lots of 3 byte > > code points (~4.5 MB) copied many times (original file was 300KB) > > > > Why is u8u16 missing? Well, as far as I can tell there is no function in > > u8u16 that returns the length of an utf8 string and checks if it is > > valid at the same time, without rewriting it. u8u16 is really just for > > transforming utf8 to utf16. > > > > The benchmark runs read the content from a file (e.g. .*-news-de, a > > german html news website), and in a loop iterates 10 times the > > utf-8-get-length-and-check function written in C and sums up the time > > for each run (using clock_t clock(void) in C, man 3 clock). > > > > ..................... > > pypy-seq-news-de: Median +- std dev: 76.0 us +- 1.4 us > > ..................... > > pypy-sse4-vec-news-de: Median +- std dev: 5.16 us +- 0.14 us > > ..................... > > pypy-avx2-vec-news-de: Median +- std dev: 384 ns +- 11 ns > > ..................... > > libunistring-news-de: Median +- std dev: 33.0 us +- 0.4 us > > ..................... > > mystrlenutf8-news-de: Median +- std dev: 9.25 us +- 0.22 us > > ..................... > > pypy-seq-news-cn: Median +- std dev: 59.8 us +- 1.2 us > > ..................... > > pypy-sse4-vec-news-cn: Median +- std dev: 7.70 us +- 0.12 us > > ..................... > > pypy-avx2-vec-news-cn: Median +- std dev: 23.3 ns +- 0.4 ns > > ..................... > > libunistring-news-cn: Median +- std dev: 30.5 us +- 0.4 us > > ..................... > > mystrlenutf8-news-cn: Median +- std dev: 6.54 us +- 0.20 us > > ..................... > > pypy-seq-tipitaka-thai: Median +- std dev: 939 us +- 39 us > > ..................... > > pypy-sse4-vec-tipitaka-thai: Median +- std dev: 425 us +- 7 us > > ..................... > > pypy-avx2-vec-tipitaka-thai: Median +- std dev: 19.9 ns +- 0.3 ns > > ..................... > > libunistring-tipitaka-thai: Median +- std dev: 615 us +- 28 us > > ..................... > > WARNING: the benchmark seems unstable, the standard deviation is high > > (stdev/median: 17%) > > Try to rerun the benchmark with more runs, samples and/or loops > > > > mystrlenutf8-tipitaka-thai: Median +- std dev: 45.1 us +- 7.9 us > > > > What do you think? > > > > I think it would even be a good idea to take a look at AVX512 (which > > gives you a crazy amount of 512 bits (or 64 bytes) in your vector > register). > > > > The AVX implementation is a bit fishy (compare avx2-vec-tipitaka-thai > > and pypy-avx2-vec-news-cn). I need to recheck that, it would not make > > sense to process 10x 4.5 MB in 20ns and 10x 700KB in 23ns. > > > > As soon as I have ironed out the issue I'll start to think about > indexing... > > > > Cheers, > > Richard > > > > [1] http://www.daemonology.net/blog/2008-06-05-faster-utf8-strlen.html > > > > On 03/04/2017 07:01 PM, Maciej Fijalkowski wrote: > >> Hello everyone > >> > >> I've been experimenting a bit with faster utf8 operations (and > >> conversion that does not do much). I'm writing down the results so > >> they don't get forgotten, as well as trying to put them in rpython > >> comments. > >> > >> As far as non-SSE algorithms go, for things like splitlines, split > >> etc. is important to walk the utf8 string quickly and check properties > >> of characters. > >> > >> So far the current finding has been that lookup table, for example: > >> > >> def next_codepoint_pos(code, pos): > >> chr1 = ord(code[pos]) > >> if chr1 < 0x80: > >> return pos + 1 > >> return pos + ord(runicode._utf8_code_length[chr1 - 0x80]) > >> > >> is significantly slower than following code (both don't do error > checking): > >> > >> def next_codepoint_pos(code, pos): > >> chr1 = ord(code[pos]) > >> if chr1 < 0x80: > >> return pos + 1 > >> if 0xC2 >= chr1 <= 0xDF: > >> return pos + 2 > >> if chr >= 0xE0 and chr <= 0xEF: > >> return pos + 3 > >> return pos + 4 > >> > >> The exact difference depends on how much multi-byte characters are > >> there and how big the strings are. It's up to 40%, but as a general > >> rule, the more ascii characters are, the less of an impact it has, as > >> well as the larger they are, the more impact memory/L2/L3 cache has. > >> > >> PS. SSE will be faster still, but we might not want SSE for just > splitlines > >> > >> Cheers, > >> fijal > >> _______________________________________________ > >> pypy-dev mailing list > >> pypy-dev(a)python.org > >> https://mail.python.org/mailman/listinfo/pypy-dev > >> > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev(a)python.org > > https://mail.python.org/mailman/listinfo/pypy-dev > [View Less]

1 0

Speeds of various utf8 operations
by Maciej Fijalkowski March 8, 2017

March 8, 2017

Hello everyone I've been experimenting a bit with faster utf8 operations (and conversion that does not do much). I'm writing down the results so they don't get forgotten, as well as trying to put them in rpython comments. As far as non-SSE algorithms go, for things like splitlines, split etc. is important to walk the utf8 string quickly and check properties of characters. So far the current finding has been that lookup table, for example: def next_codepoint_pos(code, pos): chr1 = ord(… [View More]

4 8

PyPy as a Sub-org
by Richard Plangger March 7, 2017

March 7, 2017

Hi, as we discussed during this years PyPy sprint, PyPy again wants to participate as a Sub-org in this years Google Summer of Code. We are already on the wiki ideas page, but I think we did not formally apply by writing this email. Regards, Richard

3 2

revisit web assembly?
by Maciej Fijalkowski March 5, 2017

March 5, 2017

I just found that: https://sourceware.org/ml/binutils/2017-03/msg00044.html It might be cool to see, maybe we can relatively easily compile to the web platform. Even interpreter-only version could be quite interesting. Cheers, fijal

3 2