[pypy-dev] Great experience with PyPy

[Gelin Yan]

On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar <phyo.arkarlwin at gmail.com>wrote:

> Pypy should have a page for "Success Stories!"
>
> Now with this and Quora proving Power of PyPy , i am beginning to start
> converting my projects into PyPy soon!
> I am only withholding right now because my projects uses a lot of C
> Libraries and Numpy/Matplotlib/scilit-learn.
>
> Thanks
>
> Phyo.
>
> On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
>
>> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic <mtasic85 at gmail.com> wrote:
>> > Hi,
>> >
>> > I would like to share short story with you and share what we have
>> > accomplished with PyPy and its friends so far.
>> >
>> > Company that I have worked for last 7 months (intentionally unnamed)
>> > gave me absolute permission to pick up technologies on which we based
>> > our solution. What we do is: crawl for PDFs and newspapers articles,
>> > download, translate them if needed, OCR if needed, do extensive
>> > analysis of downloaded PDFs and articles, store them in more organized
>> > structures for faster querying, search for them and generate bunch of
>> > complex reports.
>> >
>> > From very beginning I decided to go with PyPy no matter what. What we
>> > picked is following:
>> > * Flask for web framework, and few of its extensions such as
>> > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
>> > * Cassandra as database because of its features and great experience
>> > with it. PyCassa is used as client to talk to Cassandra server.
>> > * ElasticSearch as distributed search engine, and its client library
>> pyes.
>> > * Whoosh as search engine, but with some modifications to support
>> > Cassandra as storage and distributed locking.
>> > * Redis, and its client library redis-py, for caching and to speed up
>> > common auto-completion patterns.
>> > * ZooKeeper, and its client library Kazoo, for distributed locking
>> > which plays essential role in system for transaction-like behavior
>> > over many services at once.
>> > * Celery in conjunction with RabbitMQ for task distribution.
>> > * Sentry for error logging.
>> >
>> > What we have developed on our own are wrappers and clients for:
>> > * Moses which is language translator
>> > * Tesseract which is OCR engine
>> > * Cassandra store for Whoosh
>> > * wkhtmltopdf and wkhtmltoimage which are used for conversion of HTML
>> > to PDF/Image
>> > * etc
>> >
>> > Now when product is finished and in final testing phase, I can say
>> > that we did not regret because we used PyPy and stack around it.
>> > Typical speed improvement is 2x-3x over CPython in our case, but
>> > anyway we are mostly IO and memory bound, expect for Celery workers
>> > where we do analysis which are again many small CPU intensive tasks
>> > that are exchanged via RabbitMQ. Another reason why we don't see
>> > speedup us is that we are dependent on external software (servers)
>> > written in Erlang and Java.
>> >
>> > I'm already planing to do Cassandra (distributed key/value only
>> > database without index features), ZooKeeper, Redis and ElasticSearch
>> > ports in Python for next projects, and hopefully opensource them.
>> >
>> > Regards,
>> > Marko Tasic
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev at python.org
>> > http://mail.python.org/mailman/listinfo/pypy-dev
>>
>> Awesome!
>>
>> I'm glad people can make pypy work for non-trivial tasks which require
>> a lot of dependencies. We're trying to lower the bar, however it takes
>> time.
>>
>> Cheers,
>> fijal
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>

Hi, It might be off topic. I want to know whether pypy support postgres.
The last time I noticed ctypes based psycopg2 was still beta. I mainly use
twisted & postgres. pypy supports twisted well but not good for psycopg2.

Regards

gelin yan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130207/331ee843/attachment-0001.html>