Mailman 3 January 2018 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

Re: [pypy-dev] Mysterious IndexError in service running with PyPy
by William ML Leslie 22 May '18

22 May '18

Where is the code that changes the size of self.heap? How do we know that size(self.heap) is constant? My guess is that some thread changes this; but l is not recomputed. On 18 Dec 2017 6:59 PM, "hubo" <hubo(a)jiedaibao.com> wrote: > I'm reporting this issue in this mail group, though I don't know if it is > related with PyPy, because it is really strange, and is not able to > reproduce stably. But I hope someone may know the reason or have some > points. > > I'm running some SDN services written in Python with PyPy 5.3.0. The > related code is here: > > https://github.com/hubo1016/vlcp/blob/8022e3a3c67cf4305af503d507640a730ca39… > > The full code is also in the repo, but may be too complex to describe. But > this related piece is quite simple: > > def _siftdown(self, pos): > temp = self.heap[pos] > l = len(self.heap) > while pos * 2 + 1 < l: > cindex = pos * 2 + 1 > pt = self.heap[cindex] > if cindex + 1 < l and self.heap[cindex+1][0] < pt[0]: > cindex = cindex + 1 > pt = self.heap[cindex] > if pt[0] < temp[0]: > self.heap[pos] = pt > self.index[pt[1]] = pos > else: > break > pos = cindex > self.heap[pos] = temp > self.index[temp[1]] = pos > It is a simple heap operation. The service uses a heap to process timers. > When the service is not busy, it usually runs this piece of code several > times per minute. > > I have 32 servers running this service. They are quite stable in about > three months, but one day one of the services crashes on line 100 reporting > IndexError: > pt = self.heap[cindex] > > As you can see, cindex = pos * 2 + 1, which is tested by the while > pre-conditon just two lines before. And there is not any multi-threading > issues here because this piece of code always runs in the same thread. So > it is not possible in theory for this to happen. > > Only the following facts are known about this issue: > > 1. It reproduces - through quite rarely. I've met this kind of > crashes 4 times, each with different machine, so it should not be related > to hardware issuess. Since I've got 32 servers, it might take more than one > year to reproduce with a single server. > 2. It seems not to be related to pressures. All of the crashes happens > at night, when there are little requests. Only some cleanup tasks are > running in fixed interval. > > The services are running with PyPy 5.3.0. I've upgraded a few of them to > 5.9, but it will take a long time to validate whether this still happens. > And It is not validated on CPython too. I'm also trying to collect more > debugging information for this issue, but it is very hard since it rarely > reproduces. > > It is not a serious issue. It could be workarounded with a auto-restart, > but I'm searching the cause. > > > 2017-12-18 > ------------------------------ > hubo > > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > https://mail.python.org/mailman/listinfo/pypy-dev > >

2 3

PhD Positions in programming language runtimes (Python)
by David Bremner 02 May '18

02 May '18

1 Position I: Python on OMR =========================== The candidate will explore the benefits and challenges of using OMR [1] components (e.g. JIT, GC) in Python [2,3] runtimes. This may involve extending OMR in novel ways to better support Python, or vice-versa. If questions like how to balance fast startup time for scripting applications against good performance for compute-intensive workloads sound interesting, this might an opportunity for you. 2 Position II: Python and PGAS ============================== The candidate will implement support for parallel programming with Python, using the PGAS [4] (parallel global address space) model. The PGAS model can significantly simplify the implementation of parallel algorithms on distributed and shared memory computers. Challenges include enabling parallelism while dealing with Python's Global Interpreter Lock (GIL), and implementing distributed garbage collection efficiently. 3 Qualifications ================ Normally applicants to the Computer Science PhD program at UNB require a Master's degree in computer science or a closely related area. This project will require substantial hands-on and experimental work, and being comfortable with Python and C++. Familiarity with Linux and Open Source Software development would be a plus. For position I, exposure to programming language concepts like GC, JIT, and bytecode would be helpful. For position II, experience with parallel programming frameworks and/or languages such as MPI, OpenMP, X10 is an asset. 4 Environment ============= This project is supported by the IBM Centre for Advanced Studies (Atlantic) at the University of New Brunswick [5]. We have a lively group of students, postdocs, and research assistants working on related problems. Students work closely with both UNB researchers and technical experts from IBM, gaining valuable perspective and experience. 5 Support ========= These positions are "fully funded"; the exact funding package is subject to finalization but will be consistent with other fully funded students in Computer Science at UNB. You can find some information about expenses on the UNB web site [6]. 6 Contact ========= Send your resumé and unofficial transcripts by email to Stephen MacKay <smackay3(a)unb.ca> Please specify which position(s) you are interested in. [1]: [http://www.eclipse.org/omr/] [2]: [http://www.python.org] [3]: [http://www.pypy.org] [4]: [http://www.pgas.org/] [5]: [http://www.unb.ca/research/casatlantic/] [6]: [http://www.unb.ca/fredericton/studentservices/international/new_internation…]

1 1

Joining the Leysin sprint
by Floris Bruynooghe 02 Feb '18

02 Feb '18

Hi, I would be interested in joining the Leysin sprint and possibly contribute to either the cffi work and/or some of the testing work. I won''t be able to attend the entire sprint however am tentatively thinking of the 1st half, so arriving on Sat 17 leaving on Wed 21 March. Is this something which is possible? I could probably be flexible as well and attend the 2nd half of the week instead, starting from Wed, if this is easier logistically. Kind Regards, Floris

2 3

adding a pypy runner for speed.python.org
by Matti Picus 31 Jan '18

31 Jan '18

I am cross-posting to both speed and pypy-dev to ask what needs to be done to get pypy2 and pypy3 benchmark runners onto speed.python.org I am willing to be the contact person from the PyPy side, who do we need to talk to on the speed maintainers' side? Instead of spamming the lists again, we could discuss this off-line, and report back with a plan, required resources, and a timeline. Thanks, Matti Picus

3 2

Re: [pypy-dev] [pypy-commit] pypy cpyext-faster-arg-passing: document branch
by Antonio Cuni 31 Jan '18

31 Jan '18

Hi Carl, wow, this looks awesome. Did you run benchmarks to measure the speedup? If yes, should we add them to my repo? https://github.com/antocuni/cpyext-benchmarks ciao, Anto On Tue, Jan 30, 2018 at 1:31 PM, cfbolz <pypy.commits(a)gmail.com> wrote: > Author: Carl Friedrich Bolz-Tereick <cfbolz(a)gmx.de> > Branch: cpyext-faster-arg-passing > Changeset: r93724:627a1425607c > Date: 2018-01-30 14:30 +0100 > http://bitbucket.org/pypy/pypy/changeset/627a1425607c/ > > Log: document branch > > diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst > --- a/pypy/doc/whatsnew-head.rst > +++ b/pypy/doc/whatsnew-head.rst > @@ -23,3 +23,11 @@ > added, then the performance using mapdict is linear in the number of > attributes. This is now fixed (by switching to a regular dict after 80 > attributes). > + > + > +.. branch: cpyext-faster-arg-passing > + > +When using cpyext, improve the speed of passing certain objects from PyPy > to C > +code, most notably None, True, False, types, all instances of C-defined > types. > +Before, a dict lookup was needed every time such an object crossed over, > now it > +is just a field read. > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > https://mail.python.org/mailman/listinfo/pypy-commit >

2 2

Safety of replacing the instance dict
by Tin Tvrtković 30 Jan '18

30 Jan '18

Hello, this question is not directly related to PyPy but to general Python semantics. I'm posting it here since I know there are very knowledgeable people here and honestly this is the friendliest list for questions like this. I'm working on optimizing __init__s that the attrs library generates. My question is, is there anything fundamentally wrong with having an __init__ like this: def __init__(self, a, b, c): self.__dict__ = {'a': b, 'b': b, 'c': c} (basically throwing away the existing instance dict and replacing it with a fresh one). As far as I can see, this is actually faster on CPython if there is more than one attribute, and about the same speed on PyPy (benchmarking on PyPy is kinda hard, the difference is 0.2 ns vs 0.22 ns, not sure 0.02 ns is a significant difference). And this approach works for frozen classes (an attrs feature) too. It's just that doing it this way is unconventional and a little scary. Would we be violating a Python rule somewhere and making stuff blow up later if we went this way?

3 4

How to implement a JIT compiler for Django ?
by Etienne Robillard 28 Jan '18

28 Jan '18

Hi, Is it possible to use the PyPy JIT compiler embedded into a CPython (Django) application? I would like to translate a Django app into C code then compile the binary with clang to optimize the code with a JIT engine. What do you think? Etienne -- Etienne Robillard tkadm30(a)yandex.com https://www.isotopesoftware.ca/

3 7

Re: [pypy-dev] How to install pip for pypy along with the pip for python?
by Peng Yu 19 Jan '18

19 Jan '18

Yes. It works. Thanks. On Fri, Jan 19, 2018 at 2:54 AM, Matti Picus <matti.picus(a)gmail.com> wrote: > Does "pypy -mpip install ... " work? > Matti > > > On 19/01/18 00:55, Peng Yu wrote: > > Both pypy and python are installed with homebrew on Mac OS X. > > On Thu, Jan 18, 2018 at 3:16 PM, Matti Picus <matti.picus(a)gmail.com> wrote: > > How did you install pypy? > Matti > > On 18/01/18 20:28, Peng Yu wrote: > > Hi, I got this. Is there anything wrong? Also, how to install a python > module with pypy pip? Thanks. > > $ pypy -m ensurepip > /var/folders/r7/bvmh1vvx41d63snvgbdz7bl40000gr/T/tmpelZs6W/pip-6.1.1-py2.py3-none-any.whl/pip/_vendor/__init__.py:72: > UserWarning: Module pip was already imported from > /var/folders/r7/bvmh1vvx41d63snvgbdz7bl40000gr/T/tmpelZs6W/pip-6.1.1-py2.py3-none-any.whl/pip/__init__.py, > but /usr/local/lib/pypy/site-packages/pip-8.1.2-py2.7.egg is being > added to sys.path > Ignoring indexes: https://pypi.python.org/simple > Requirement already satisfied (use --upgrade to upgrade): setuptools > in /usr/local/lib/pypy/site-packages/setuptools-26.1.1-py2.7.egg > Requirement already satisfied (use --upgrade to upgrade): pip in > /usr/local/lib/pypy/site-packages/pip-8.1.2-py2.7.egg > > On Thu, Jan 18, 2018 at 12:02 PM, Matti Picus <matti.picus(a)gmail.com> wrote: > > On 18/01/18 19:59, Peng Yu wrote: > > Hi, > > I installed pip with python without using virtual environment. > > https://pip.pypa.io/en/stable/installing/ > > I also need to install pip with pypy without using virtual > environment. Is it possible? Could anybody show me how to do it? > Thanks. > > pypy -m ensurepip > > Matti > > > > > > -- Regards, Peng

1 0

How to install pip for pypy along with the pip for python?
by Peng Yu 18 Jan '18

18 Jan '18

Hi, I installed pip with python without using virtual environment. https://pip.pypa.io/en/stable/installing/ I also need to install pip with pypy without using virtual environment. Is it possible? Could anybody show me how to do it? Thanks. -- Regards, Peng

2 2