<a href="http://speed.pypy.org/">http://speed.pypy.org/</a><br clear="all"><div><br></div><div><br></div><span style="border-collapse:collapse;font-family:arial, sans-serif;font-size:13px">Arigatou gozaimasu</span>,<div>(Thank you very much)<br>
Brian Herman<br><br><a href="http://brianjherman.com" target="_blank">brianjherman.com</a><br><a href="mailto:brianherman@acm.org" target="_blank">brianherman@acm.org</a><br><br><br><br><br><br></div><br>
<br><br><div class="gmail_quote">On Sun, Jul 24, 2011 at 10:46 PM, Tal Liron <span dir="ltr"><<a href="mailto:tal.liron@threecrickets.com">tal.liron@threecrickets.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
For the people recommending PyPy right now, a serious question:<br>
<br>
<br>
Who would you recommend PyPy to? Assume a user or dev who does not care about speed benchmarks.<br>
<br>
<br>
On 07/24/2011 10:13 PM, Brian Herman wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+1 for PYPY<br>
<br>
<br>
Arigatou gozaimasu,<br>
(Thank you very much)<br>
Brian Herman<br>
<br>
<a href="http://brianjherman.com" target="_blank">brianjherman.com</a> <<a href="http://brianjherman.com" target="_blank">http://brianjherman.com</a>><br>
<a href="mailto:brianherman@acm.org" target="_blank">brianherman@acm.org</a> <mailto:<a href="mailto:brianherman@acm.org" target="_blank">brianherman@acm.org</a>><br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
On Sun, Jul 24, 2011 at 7:57 PM, Alex Gaynor <<a href="mailto:alex.gaynor@gmail.com" target="_blank">alex.gaynor@gmail.com</a> <mailto:<a href="mailto:alex.gaynor@gmail.com" target="_blank">alex.gaynor@gmail.com</a>><u></u>> wrote:<br>
<br>
<br>
<br>
On Sun, Jul 24, 2011 at 5:38 PM, Tal Liron<br>
<<a href="mailto:tal.liron@threecrickets.com" target="_blank">tal.liron@threecrickets.com</a> <mailto:<a href="mailto:tal.liron@threecrickets.com" target="_blank">tal.liron@<u></u>threecrickets.com</a>>><br>
wrote:<br>
<br>
JVM 7 will have some neat features, but they haven't been<br>
stabilized yet, and at this point it's mostly experimentation.<br>
Fact is, even though JVM 6 has been out for a few years<br>
already, many deployments still stick to JVM 5. It does the<br>
job, and "upgrades" have their costs, money and otherwise. I<br>
choose JVM for my project not because of speed, but because of<br>
the maturity of the platform, which includes administration<br>
tools, monitoring, security, and several best-in-class 3rd<br>
party libraries. It's nice to know that performance is very<br>
high up there if I really need it (at which case I just "drop<br>
down" to Java, rather than use a dynamic JVM language).<br>
<br>
<br>
The whole Jython codebase could use some help... it's even<br>
messier than CPython's, if that's possible. There's a lot of<br>
room for optimization, even before igniting JVM 7 shortcuts,<br>
though it will surely be at the cost of regressions and<br>
stability.Luckily, there's a decent test suite, which makes it<br>
easy to experiment for. The Jython community would LOVE help,<br>
and it doesn't have to be just in terms of coding. Their<br>
recent big project was to move the whole codebase from<br>
Subversion to Mercurial. Another big item on the todo list is<br>
to get up to date with Python 3. (Jython = Python 2.5<br>
formally, though it has quite a few 2.6 additions.)<br>
<br>
<br>
Jython also has some nice collaboration with JRuby, including<br>
people who work on both projects. But, what I would make me<br>
happier is if there was real code sharing, allowing for a<br>
dynamic core that would work well for both projects.<br>
<br>
<br>
Anyway. I guess I'm always confused by what people mean by<br>
"faster." What are you trying to code for, exactly? Where is<br>
your bottleneck? What is your funding? It's more likely that<br>
(although not necessarily) what you really are looking for is<br>
"scalability," for which shear computational performance is<br>
likely not the real issue. If money is coming, getting more<br>
expensive, faster machines may do the trick better than any<br>
JVM 7 optimization.<br>
<br>
<br>
If you just want a command line tool that starts fast, JVM is<br>
*not* where you want to go. It has notoriously slow startup,<br>
for exactly those mechanisms that make it perform so well as<br>
it runs.<br>
<br>
<br>
Another way to look at "faster" is as a way to save money.<br>
Weird, huh? But consider Facebook's HipHop project. (Sorry<br>
that all of my examples are from the web arena; it's where I<br>
mostly work these days.) The issue was not that PHP was<br>
"slow," it was that when you have 1,000 machines running at<br>
90% CPU, a faster PHP runtime means that you can use 800<br>
machines, instead, for the same workload. A few orders of<br>
magnitude forward, and savings can be enormous.<br>
<br>
<br>
If you have a project with 1,000 machines running at 90% CPU,<br>
please hire me! It may be very worthwhile for you to create a<br>
more performant Python runtime (JVM-based or not), and I'd<br>
love to be paid to do that. :) And it would also make a lot of<br>
irrational Python speed freaks happy.<br>
<br>
<br>
-Tal<br>
<br>
<br>
<minor derail><br>
No offense, but if you want a more performant Python runtime, it's<br>
here today: <a href="http://speed.pypy.org/" target="_blank">http://speed.pypy.org/</a>, no need to start from scratch.<br>
</minor derail><br>
<br>
Alex<br>
<br>
<br>
<br>
On 07/24/2011 06:18 PM, John Stoner wrote:<br>
<br>
Jython's not bad. I've used it a lot, and it plays well<br>
with lots of Java APIs. Pretty slick, actually. I hear<br>
Java 1.7 has some new dynamic features at the JVM level. I<br>
always imagined Jython would run a lot faster if it took<br>
advantage of them. Tal, do you know if there's any work on<br>
that? Googling around a bit I'm not seeing much.<br>
<br>
On Sun, Jul 24, 2011 at 4:32 PM, Joshua Herman<br>
<<a href="mailto:zitterbewegung@gmail.com" target="_blank">zitterbewegung@gmail.com</a><br>
<mailto:<a href="mailto:zitterbewegung@gmail.com" target="_blank">zitterbewegung@gmail.<u></u>com</a>><br>
<mailto:<a href="mailto:zitterbewegung@gmail.com" target="_blank">zitterbewegung@gmail.<u></u>com</a><br>
<mailto:<a href="mailto:zitterbewegung@gmail.com" target="_blank">zitterbewegung@gmail.<u></u>com</a>>>> wrote:<br>
<br>
At least erlang works for the use cases. I wasn't aware<br>
that Jython<br>
was that powerful I will have to play with it.<br>
<br>
On Sun, Jul 24, 2011 at 3:46 PM, Tal Liron<br>
<<a href="mailto:tal.liron@threecrickets.com" target="_blank">tal.liron@threecrickets.com</a><br>
<mailto:<a href="mailto:tal.liron@threecrickets.com" target="_blank">tal.liron@<u></u>threecrickets.com</a>><br>
<mailto:<a href="mailto:tal.liron@threecrickets.com" target="_blank">tal.liron@<u></u>threecrickets.com</a><br>
<mailto:<a href="mailto:tal.liron@threecrickets.com" target="_blank">tal.liron@<u></u>threecrickets.com</a>>>><br>
<br>
wrote:<br>
> There is an alternative: Jython, which is Python on the<br>
JVM, and<br>
has no GIL.<br>
> It's real, it works, and has a very open community. If<br>
you want<br>
to do<br>
> high-concurrency in Python, it's the way to go. (And it has<br>
other advantages<br>
> and disadvantages, of course.)<br>
><br>
><br>
> I am always a bit frightened by community attempts to<br>
create new<br>
virtual<br>
> machines for favorite languages in order to solve problem X.<br>
This shows a<br>
> huge under-estimation of what it means to create a<br>
robust, reliable,<br>
> performative generic platform. Consider how many really<br>
reliable<br>
versions of<br>
> the C standard library out there -- and how many decades<br>
they<br>
took to<br>
> mature, even with thousands of expert eyes poring over<br>
the code<br>
and testing<br>
> it. And this is without duck typing (or ANY typing), data<br>
integrity, scoping<br>
> (+call/cc), tail recursion, or any other of the other<br>
huge (and<br>
exciting)<br>
> challenges required to run a dynamic language like Python.<br>
><br>
><br>
> So, it's almost amusing to see projects like Rubinius or<br>
Parrot<br>
come to be.<br>
> Really? This is the best use of our time and effort? I'm<br>
equally<br>
impressed<br>
> by the ballsiness of Erlang to create a new virtual<br>
machine from<br>
scratch.<br>
><br>
><br>
> But those are rather unique histories. CPython has it's own<br>
unique history.<br>
> Not many people realize this, but Python is about 6<br>
years older<br>
than Java,<br>
> and the JVM would take another decade before reaching<br>
prominence. JavaScript<br>
> engines (running in web browsers only) at the time were<br>
terrible, and Perl<br>
> was entirely interpreted (no VM). So, in fact, CPython was<br>
written where<br>
> there was no really good platform for dynamic languages. It<br>
wasn't a matter<br>
> of hubris ("not invented here") to build a VM from scratch;<br>
there was simply<br>
> no choice.<br>
><br>
><br>
> Right now, though, there are many good choices. People<br>
like Rich<br>
Hickey<br>
> (Clojure) and Martin Odersky (Scala) have it right in<br>
targeting<br>
the JVM,<br>
> although both projects are also exploring .NET/Mono. If<br>
Python<br>
were invented<br>
> today, I imagine it also would start with "Jython,"<br>
instead of<br>
trying to<br>
> reinvent the wheel (well, reinvent a whole damn car fleet,<br>
really, in terms<br>
> of the work required).<br>
><br>
><br>
> One caveat: I think there is room for "meta-VM" projects<br>
like<br>
PyPy and LLVM.<br>
> These signify a real progress in architecture, whereas "yet<br>
another dynamic<br>
> VM" does not.<br>
><br>
><br>
> -Tal<br>
><br>
><br>
> On 07/24/2011 02:56 PM, Jason Rexilius wrote:<br>
><br>
>> I also have to quote:<br>
>><br>
>> "rather that, for problems for which shared-memory<br>
concurrency is<br>
>> appropriate (read: the valid cases to complain about<br>
the GIL),<br>
message<br>
>> passing will not be, because of the marshal/unmarshal<br>
overhead<br>
(plus data<br>
>> size/locality ones)."<br>
>><br>
>><br>
>> I have to say this is some of the best discussion in<br>
quite a<br>
while. Dave's<br>
>> passionate response is great as well as others. I think the<br>
rudeness, or<br>
>> not, is kinda besides the point.<br>
>><br>
>> There is a valid point to be made about marshal/unmarshal<br>
overhead in<br>
>> situations where data-manipulation-concurrency AND _user<br>
expectation_ or<br>
>> environmental constraints apply. I think that's why people<br>
have some<br>
>> grounds to be unhappy with the GIL concept (for me its a<br>
concept) in certain<br>
>> circumstances. Tal is dead on in that "scalability" means<br>
different things.<br>
>><br>
>> Oddly, I'm more engaged in this as an abstract comp sci<br>
question than a<br>
>> specific python question. The problem set applies across<br>
languages.<br>
>><br>
>> The question I would raise is if, given that an engineer<br>
understands the<br>
>> problem he is facing, are there both tools in the<br>
toolbox? Is<br>
there an<br>
>> alternative to GIL for the use-cases where it is not<br>
the ideal<br>
solution?<br>
>><br>
>> BTW, I will stand up for IPC as one of the tools in the<br>
toolbox<br>
to deal<br>
>> with scale/volume/speed/concurrency problems.<br>
>><br>
>><br>
>> On 7/24/11 1:58 PM, Tal Liron wrote:<br>
>>><br>
>>> I would say that there's truth in both approaches.<br>
"Scalability" means<br>
>>> different things at different levels of scale. A web<br>
example: the<br>
>>> architecture of Twitter or Facebook is nothing like the<br>
architecture of<br>
>>> even a large Django site. It's not even the same<br>
problem field.<br>
>>><br>
>>><br>
>>> A good threading model can be extremely efficient at<br>
certain<br>
scales. For<br>
>>> data structures that are mostly read, not written,<br>
synchronization is<br>
>>> not a performance issue, and you get the best throughput<br>
possible in<br>
>>> multicore situations. The truly best scalability would be<br>
achieved by a<br>
>>> combined approach: threading on a single node, message<br>
passing<br>
between<br>
>>> nodes. Programming for that, though, is a nightmare<br>
(unless<br>
you had a<br>
>>> programming language that makes both approaches<br>
transparent)<br>
and so<br>
>>> usually at the large scale the latter approach is<br>
chosen. One<br>
>>> significant challenge is to make sure that operations that<br>
MIGHT use the<br>
>>> same data structures are actually performed on the<br>
same node,<br>
so that<br>
>>> threading would be put to use.<br>
>>><br>
>>><br>
>>> So, what Dave said applies very well to threading,<br>
too: "you<br>
still need<br>
>>> to know what you're doing and how to decompose your<br>
application to use<br>
>>> it."<br>
>>><br>
>>><br>
>>> Doing concurrency right is hard. Doing message passing<br>
right<br>
is hard.<br>
>>> Functional (persistent data structure) languages are hard,<br>
too. Good<br>
>>> thing we're all such awesome geniuses, bursting with<br>
experience and a<br>
>>> desire to learn.<br>
>>><br>
>>><br>
>>> -Tal<br>
>>><br>
>>><br>
>>> On 07/23/2011 01:40 PM, David Beazley wrote:<br>
>>><br>
>>>>> "high performance just create multi processes that<br>
message" very<br>
>>>>> rarely have<br>
>>>>> I heard IPC and high performance in the same sentence.<br>
>>>>><br>
>>>>> Alex<br>
>>>>><br>
>>>> Your youth and inexperience is the only reason would<br>
make a<br>
statement<br>
>>>> that ignorant. Go hang out with some people doing<br>
Python and<br>
>>>> supercomputing for awhile and report back---you will find<br>
that almost<br>
>>>> significant application is based on message passing<br>
(e.g.,<br>
MPI). This<br>
>>>> is because message passing has proven itself to be<br>
about the<br>
only sane<br>
>>>> way of scaling applications up to run across<br>
thousands to tens of<br>
>>>> thousands of CPU cores.<br>
>>>><br>
>>>> I speak from some experience as I was writing such<br>
software<br>
for large<br>
>>>> Crays, Connection Machines, and other systems when I<br>
first<br>
discovered<br>
>>>> Python back in 1996. As early as 1995, our group had done<br>
performance<br>
>>>> experiments comparing threads vs. message passing on some<br>
>>>> multiprocessor SMP systems and found that threads<br>
just didn't<br>
scale or<br>
>>>> perform as well as message passing even on machines<br>
with as<br>
few as 4<br>
>>>> CPUs. This was all highly optimized C code for<br>
numerics (i.e., no<br>
>>>> Python or GIL).<br>
>>>><br>
>>>> That said, in order to code with message passing, you<br>
still<br>
need to<br>
>>>> know what you're doing and how to decompose your<br>
application<br>
to use it.<br>
>>>><br>
>>>> Cheers,<br>
>>>> Dave<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> ______________________________<u></u>_________________<br>
>>>> Chicago mailing list<br>
>>>> <a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>>><br>
<br>
>>>> <a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
>>><br>
>>> ______________________________<u></u>_________________<br>
>>> Chicago mailing list<br>
>>> <a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>>><br>
<br>
>>> <a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
>><br>
>> ______________________________<u></u>_________________<br>
>> Chicago mailing list<br>
>> <a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>>><br>
<br>
>> <a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
><br>
> ______________________________<u></u>_________________<br>
> Chicago mailing list<br>
> <a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>>><br>
<br>
> <a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
><br>
______________________________<u></u>_________________<br>
Chicago mailing list<br>
<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>>><br>
<br>
<a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
<br>
<br>
<br>
<br>
-- blogs:<br>
<a href="http://johnstoner.wordpress.com/" target="_blank">http://johnstoner.wordpress.<u></u>com/</a><br>
'In knowledge is power; in wisdom, humility.'<br>
<br>
<br>
______________________________<u></u>_________________<br>
Chicago mailing list<br>
<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
<br>
<br>
______________________________<u></u>_________________<br>
Chicago mailing list<br>
<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
<br>
<br>
<br>
<br>
-- "I disapprove of what you say, but I will defend to the death your<br>
right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)<br>
"The people's good is the highest law." -- Cicero<br>
<br>
<br>
______________________________<u></u>_________________<br>
Chicago mailing list<br>
<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a> <mailto:<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a>><br>
<a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
<br>
<br>
<br>
<br>
______________________________<u></u>_________________<br>
Chicago mailing list<br>
<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
</blockquote>
<br>
______________________________<u></u>_________________<br>
Chicago mailing list<br>
<a href="mailto:Chicago@python.org" target="_blank">Chicago@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/chicago" target="_blank">http://mail.python.org/<u></u>mailman/listinfo/chicago</a><br>
</blockquote></div><br>