[pypy-dev] Questions on the pypy+numpy project

[Gary Robinson]

> By the way, did you ever considered the possibility of running pypy and cpython side-by-side?
> You do your pure-python computation on pypy, then you pipe them (e.g. by using execnet) to a cpython process which does the processing using scipy. Depending on how big the data is, the overhead of passing the data around should not be too high
.
Absolutely -- I've thought about that general approach though this is the first time I recall hearing about execnet. Of course I'm concerned that the overhead would be too much in some cases, such as huge numbers of calls to scipy.stats.stats.chisqprob. Such overhead seems like it might cancel all the benefit of PyPy, depending on the script. But maybe it's not as much overhead as I fear. For example I see that execnet does not do pickling. Hm.

-- 

Gary Robinson
CTO
Emergent Discovery, LLC
personal email: garyrob at me.com
work email: grobinson at emergentdiscovery.com
Company: http://www.emergentdiscovery.com
Blog:    http://www.garyrobinson.net




On Oct 19, 2011, at 10:27 AM, Antonio Cuni wrote:

> Hello Gary,
> 
> On 19/10/11 15:38, Gary Robinson wrote:
>>>> You would like pypy+numpy+scipy so that you could write fast
>>>> python-only algorithms and still use the existing libraries.  I
>>>> suppose this is a perfectly reasonable usecase, and indeed
>>>> the current plan does not focus on this.
>>> 
>> 
>> Yes. That is exactly what I want.
> [cut]
> 
> thank you for the input: indeed, I agree that for your usecase the current plan is not the best. OTOH, there is probably someone else for which the current plan is better than others, we cannot make everyone happy at the same time, although we might do it eventually :-).
> 
> By the way, did you ever considered the possibility of running pypy and cpython side-by-side?
> You do your pure-python computation on pypy, then you pipe them (e.g. by using execnet) to a cpython process which does the processing using scipy. Depending on how big the data is, the overhead of passing the data around should not be too high
> 
> It's not ideal, but it might be worth of being tried.
> 
> ciao,
> Anto