[pypy-dev] Success histories needed

Mon Oct 31 23:47:38 CET 2011

Well... that took a bit long-ish because I was preparing a small conference we 
are having here in Tampere. Anyway, as promised. You are free to do whatever 
you like with the stuff (cut/remove stuff if it is too big) as long as there 
is my name somewhere nearby.

PYPY + SWIG as a basis for simulation platform

System-level simulations of communications networks are very resource-hungry, 
mostly due to complex statistical relations that have to be uncovered. For 
example, it is not enough to observe just one or two packets in the network to 
conclude if it is stable or not, the actual observation should last a lot 
longer then any feedback process in the network. If the buffer sizes are 
large, this can mean hundreds of thousands of packets. Wireless networks add 
one extra problem - the channel model is typically an $N^2$ complexity 
algorithm, and it has to be run for each and every symbol transmitted over the 
radio interface.

Unfortunately, most of the existing simulation platforms for wireless networks 
do not allow for multiple sources to coexist and cooperate, and therefore are 
unsuitable for my research. A new simulation environment was to be devised. 
Originally, my simulator was implemented as a time-advance model with Matlab. 
Unfortunately, complex logic of the network in question made it so slow, that 
it was practically useless. The second implementation in C was scalable enough 
for a single wireless cell, but it was clearly unsuitable for implementation 
of complex scheduling algorithms.

Generally, scheduling algorithms operate with such terms as sets, maps and 
dynamic priority queues. The relay selection algorithms, which are the prime 
topic, are also operating with sets and mappings. Unfortunately, such high-
level data structures do not fit well into the paradigm of C/C++. Therefore, 
it was decided to split the model into two parts: the link-level model 
implemented with C and the control level implemented with Python. Obviously, 
the bulk of calculations was to be performed at the link-level, and the 
complex control logic was to be implemented with Python. 

Unfortunately, the performance of Python interpreter soon appeared to be a 
bottleneck, but redesigning the model again did not seem like a right thing to 
do. And here, following the Zen of Python, the one obvious solution popped out 
- PyPy. PyPy is essentially a JIT-compiler for Python, and it allows to run 
Python programs with speeds close to those approachable with pure C++. At the 
same time it retains the flexibility of the scripting language. 

As for the C part, the SWIG wrapper generator allows to maintain the linking 
between the python top-level and binary backend in a clean, efficient and 
cross-platform manner. It also allows for a nice trick - the wrappers can be 
also generated for Octave, which allows to verify the channel model parts 
against their analytical models in Matlab-like environment. Unfortunately, 
SWIG can not generate interfaces for Matlab yet, but in most cases I have 
actually found Octave better and faster for small prototyping and testing 
projects.

Currently, the PyPy 1.6 runs the model about 200\% faster then standard 
CPython 2.7.2. Also it provides the unique opportunities in configuration, as 
most of the model scenario can be constructed with object constructors and for 
loops, rather then some freaky config files or infinite bulks of XML. Overall,
I am very statisfied with PyPy experience, and I plan on trying it in even 
more demanding environment - as an experimental engine for a P2P protocol 
tracker.

Best regards,
Alexander Pyattaev,
Researcher,
Department of Communications Engineering,
Tampere University of Technology, Finland

On Friday 21 October 2011 16:36:10 Bea During wrote:
> Hi there
> 
> Alexander Pyattaev skrev 2011-10-21 15:53:
> > I think i will finalize that thing on this weekend. Of course you will
> > get permissions to do whatever you wish with it, I am not a microsoft
> > fan=)
> Thanks! We look forward to reading it - and publishing it ;-)
> 
> Cheers
> 
> Bea
> 
> > On Wednesday 19 October 2011 14:36:15 Bea During wrote:
> >> Hi there
> >> 
> >> Maciej Fijalkowski skrev 2011-10-17 10:30:
> >>> On Mon, Oct 17, 2011 at 10:17 AM, Alex
> >>> Pyattaev<alex.pyattaev at gmail.com>> 
> > wrote:
> >>>> I have a fully-functional wireless network simulation tool written
> >>>> in
> >>>> pypy+swig. Is that nice? Have couple papers to refer to as well.
> >>>> If
> >>>> you want I could write a small abstract on how it was so great to
> >>>> use
> >>>> pypy (which, in fact, was really great)?
> >>> 
> >>> Please :)
> >> 
> >> As Maciej says - please do ;-)
> >> 
> >> And I think we may want to be able to publish your abstract on our
> >> blog
> >> if that would be possible?
> >> 
> >> Cheers
> >> 
> >> Bea
> >> 
> >>> _______________________________________________
> >>> pypy-dev mailing list
> >>> pypy-dev at python.org
> >>> http://mail.python.org/mailman/listinfo/pypy-dev
> >>> 
> >>> 
> >>> 
> >>> --
> >>> 
> >>> CronLab scanned this message. We don't think it was spam. If it was,
> >>> please report by copying this link into your browser:
> >>> http://cronlab01.terratel.se/mail/index.php?id=A74652E2162.6D318-&le
> >>> arn
> >>> =spam&host=212.91.134.155
> >> 
> >> _______________________________________________
> >> pypy-dev mailing list
> >> pypy-dev at python.org
> >> http://mail.python.org/mailman/listinfo/pypy-dev
> > 
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> > 
> > 
> > 
> > --
> > 
> > CronLab scanned this message. We don't think it was spam. If it was,
> > please report by copying this link into your browser:
> > http://cronlab01.terratel.se/mail/index.php?id=EBB6A2E21D4.94541-&learn
> > =spam&host=212.91.134.155
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev