[pypy-issue] Issue #2709: filter() way slower than inlining the check (pypy/pypy)

[Andrew Szeto]

New issue 2709: filter() way slower than inlining the check
https://bitbucket.org/pypy/pypy/issues/2709/filter-way-slower-than-inlining-the-check

Andrew Szeto:

Inspired by a small programming puzzle found online ([Advent of Code Day #15](http://adventofcode.com/2017/day/15)), I found the following disparity in two different approaches to writing the same program:

**pypy3 --version:**
```
#!shell

Python 3.5.3 (d72f9800a42b46a8056951b1da2426d2c2d8d502, Oct 07 2017, 08:21:37)
[PyPy 5.9.0-beta0 with GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]
```

**inline.py:**
```
#!python

#! /usr/bin/env pypy3

def linear_congruential_generator(seed, filter_fn):
    state = seed
    while True:
        state *= 16807
        state %= 2**31-1
        if filter_fn(state):
            yield state

g = linear_congruential_generator(116, lambda x: x % 4 == 0)
accumulator = 0
for _ in range(5000000):
    accumulator += next(g)
print(accumulator)
```
**withfilter.py:**

```
#!python

#! /usr/bin/env pypy3

def linear_congruential_generator(seed):
    state = seed
    while True:
        state *= 16807
        state %= 2**31-1
        yield state

g = linear_congruential_generator(116)
g = filter(lambda x: x % 4 == 0, g)
accumulator = 0
for _ in range(5000000):
    accumulator += next(g)
print(accumulator)
```

**timing results:**
```
#!shell

Andrews-MacBook-Pro:tmp andrew$ time pypy3 inline.py 
5369485192468068

real	0m0.394s
user	0m0.368s
sys	0m0.018s

Andrews-MacBook-Pro:tmp andrew$ time pypy3 withfilter.py 
5369485192468068

real	0m8.853s
user	0m8.644s
sys	0m0.055s
```

I understand that "Abuse of itertools" under "Unrelated things that we know PyPy to be slow at (note that we're probably working on it)" on the [performance page](https://pypy.org/performance.html) may be related, but I was wondering if anyone had more insight into this ongoing work for it.