On Sun, Feb 24, 2019 at 10:04 PM Eric Snow <ericsnowcurrently@gmail.com> wrote:

I'll take a look tonight.

I made 2 successive runs of the script (on my laptop) for a commit from early Saturday, and 2 runs from a commit this afternoon (close to master). The output is below, with the earlier commit first. That one is a little faster in places and a little slower in others. However, I also saw quite a bit of variability in the results for the same commit. So I'm not sure what to make of it. I'll look into it in more depth tomorrow. FWIW, I have a few commits in the range you described, so I want to make sure I didn't slow things down for us. :) -eric * commit 175421b58cc97a2555e474f479f30a6c5d2250b0 (HEAD) | Author: Pablo Galindo <Pablogsal@gmail.com> | Date: Sat Feb 23 03:02:06 2019 +0000 | | bpo-36016: Add generation option to gc.getobjects() (GH-11909) $ ./python Tools/scripts/var_access_benchmark.py Variable and attribute read access: 18.1 ns read_local 19.4 ns read_nonlocal 48.3 ns read_global 52.4 ns read_builtin 55.7 ns read_classvar_from_class 56.1 ns read_classvar_from_instance 78.6 ns read_instancevar 67.6 ns read_instancevar_slots 65.9 ns read_namedtuple 106.1 ns read_boundmethod Variable and attribute write access: 25.1 ns write_local 26.9 ns write_nonlocal ^[[A 78.0 ns write_global 154.1 ns write_classvar 132.0 ns write_instancevar 88.2 ns write_instancevar_slots Data structure read access: 69.6 ns read_list 69.0 ns read_deque 68.4 ns read_dict Data structure write access: 73.2 ns write_list 79.0 ns write_deque 103.5 ns write_dict Stack (or queue) operations: 348.3 ns list_append_pop 169.0 ns deque_append_pop 170.8 ns deque_append_popleft Timing loop overhead: 1.3 ns loop_overhead $ ./python Tools/scripts/var_access_benchmark.py Variable and attribute read access: 17.7 ns read_local 19.2 ns read_nonlocal 39.9 ns read_global 50.3 ns read_builtin 54.4 ns read_classvar_from_class 55.8 ns read_classvar_from_instance 80.3 ns read_instancevar 70.7 ns read_instancevar_slots 66.1 ns read_namedtuple 108.9 ns read_boundmethod Variable and attribute write access: 25.1 ns write_local 25.6 ns write_nonlocal 70.0 ns write_global 151.5 ns write_classvar 133.9 ns write_instancevar 90.7 ns write_instancevar_slots Data structure read access: 140.7 ns read_list 89.6 ns read_deque 86.6 ns read_dict Data structure write access: 97.9 ns write_list 100.5 ns write_deque 120.0 ns write_dict Stack (or queue) operations: 375.9 ns list_append_pop 179.3 ns deque_append_pop 179.4 ns deque_append_popleft Timing loop overhead: 1.5 ns loop_overhead * commit 3b0abb019662e42070f1d6f7e74440afb1808f03 (HEAD) | Author: Giampaolo Rodola <g.rodola@gmail.com> | Date: Sun Feb 24 15:46:40 2019 -0800 | | bpo-33671: allow setting shutil.copyfile() bufsize globally (GH-12016) $ ./python Tools/scripts/var_access_benchmark.py Variable and attribute read access: 20.2 ns read_local 20.0 ns read_nonlocal 41.9 ns read_global 52.9 ns read_builtin 56.3 ns read_classvar_from_class 56.9 ns read_classvar_from_instance 80.2 ns read_instancevar 70.6 ns read_instancevar_slots 69.5 ns read_namedtuple 114.5 ns read_boundmethod Variable and attribute write access: 23.4 ns write_local 25.0 ns write_nonlocal 74.5 ns write_global 152.0 ns write_classvar 131.7 ns write_instancevar 90.1 ns write_instancevar_slots Data structure read access: 69.9 ns read_list 73.4 ns read_deque 77.8 ns read_dict Data structure write access: 83.3 ns write_list 94.9 ns write_deque 120.6 ns write_dict Stack (or queue) operations: 383.4 ns list_append_pop 187.1 ns deque_append_pop 182.2 ns deque_append_popleft Timing loop overhead: 1.4 ns loop_overhead $ ./python Tools/scripts/var_access_benchmark.py Variable and attribute read access: 19.1 ns read_local 20.9 ns read_nonlocal 43.8 ns read_global 57.8 ns read_builtin 58.4 ns read_classvar_from_class 61.3 ns read_classvar_from_instance 84.7 ns read_instancevar 72.9 ns read_instancevar_slots 69.7 ns read_namedtuple 109.9 ns read_boundmethod Variable and attribute write access: 23.1 ns write_local 23.7 ns write_nonlocal 72.8 ns write_global 149.9 ns write_classvar 133.3 ns write_instancevar 89.4 ns write_instancevar_slots Data structure read access: 69.0 ns read_list 69.6 ns read_deque 69.1 ns read_dict Data structure write access: 74.5 ns write_list 80.9 ns write_deque 105.4 ns write_dict Stack (or queue) operations: 338.2 ns list_append_pop 165.6 ns deque_append_pop 164.7 ns deque_append_popleft Timing loop overhead: 1.3 ns loop_overhead

On Feb 25, 2019, at 2:54 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Have you tried bisecting to find out the offending changeset, if there any?

I got it down to two checkins before running out of time: Between git checkout 463572c8beb59fd9d6850440af48a5c5f4c0c0c9 And: git checkout 3b0abb019662e42070f1d6f7e74440afb1808f03 So the subinterpreter patch was likely the trigger. I can reproduce it over and over again on Clang, but not for a GCC-8 build, so it is compiler specific (and possibly macOS specific). Will look at it more after work this evening. I posted here to try to solicit independent confirmation. Raymond

On Mon, Feb 25, 2019 at 10:42 AM Eric Snow <ericsnowcurrently@gmail.com> wrote:

I'll look into it around then too. See https://bugs.python.org/issue33608.

I ran the "performance" suite (https://github.com/python/performance), which has 57 different benchmarks. In the results, 9 were marked as "significantly" different between the two commits.. 2 of the benchmarks showed a marginal slowdown and 7 showed a marginal speedup: +-------------------------+--------------+-------------+--------------+-----------------------+ | Benchmark | speed.before | speed.after | Change | Significance | +=========================+==============+=============+==============+=======================+ | django_template | 177 ms | 172 ms | 1.03x faster | Significant (t=3.66) | +-------------------------+--------------+-------------+--------------+-----------------------+ | html5lib | 126 ms | 122 ms | 1.03x faster | Significant (t=3.46) | +-------------------------+--------------+-------------+--------------+-----------------------+ | json_dumps | 17.6 ms | 17.2 ms | 1.02x faster | Significant (t=2.65) | +-------------------------+--------------+-------------+--------------+-----------------------+ | nbody | 157 ms | 161 ms | 1.03x slower | Significant (t=-3.85) | +-------------------------+--------------+-------------+--------------+-----------------------+ | pickle_dict | 29.5 us | 30.5 us | 1.03x slower | Significant (t=-6.37) | +-------------------------+--------------+-------------+--------------+-----------------------+ | scimark_monte_carlo | 144 ms | 139 ms | 1.04x faster | Significant (t=3.61) | +-------------------------+--------------+-------------+--------------+-----------------------+ | scimark_sparse_mat_mult | 5.41 ms | 5.25 ms | 1.03x faster | Significant (t=4.26) | +-------------------------+--------------+-------------+--------------+-----------------------+ | sqlite_synth | 3.99 us | 3.91 us | 1.02x faster | Significant (t=2.49) | +-------------------------+--------------+-------------+--------------+-----------------------+ | unpickle_pure_python | 497 us | 481 us | 1.03x faster | Significant (t=5.04) | +-------------------------+--------------+-------------+--------------+-----------------------+ (Issue #33608 has more detail.) So it looks like commit ef4ac967 is not responsible for a performance regression. -eric

On 2019-02-25, Eric Snow wrote:

So it looks like commit ef4ac967 is not responsible for a performance regression.

I did a bit of exploration myself and that was my conclusion as well. Perhaps others would be interested in how to use "perf" so I did a little write up: https://discuss.python.org/t/profiling-cpython-with-perf/940 To me, it looks like using a register based VM could produce a pretty decent speedup. Research project for someone. ;-) Regards, Neil

On 2019-02-26, Victor Stinner wrote:

I made an attempt once and it was faster: https://faster-cpython.readthedocs.io/registervm.html

Interesting. I don't think I have seen that before. Were you aware of "Rattlesnake" before you started on that? It seems your approach is similar. Probably not because I don't think it is easy to find. I uploaded a tarfile I had on my PC to my web site: http://python.ca/nas/python/rattlesnake20010813/ It seems his name doesn't appear in the readme or source but I think Rattlesnake was Skip Montanaro's project. I suppose my idea of unifying the local variables and the registers could have came from Rattlesnake. Very little new in the world. ;-P Cheers, Neil

No, I wasn't aware of this project. My starting point was: http://static.usenix.org/events/vee05/full_papers/p153-yunhe.pdf Yunhe Shi, David Gregg, Andrew Beatty, M. Anton Ertl, 2005 See also my email to python-dev that I sent in 2012: https://mail.python.org/pipermail/python-dev/2012-November/122777.html Ah, my main issue was my implementation is that I started without taking care of clearing registers when the stack-based bytecode implicitly cleared a reference (decref), like "POP_TOP" operation. I added "CLEAR_REG" late in the development and it caused me troubles, and the "correct" register-based bytecode was less efficient than bytecode without CLEAR_REG. But my optimizer was very limited, too limited. Another implementation issue that I had was to understand some "implicit usage" of the stack like try/except which do black magic, whereas I wanted to make everything explicit for registers. I'm talking about things like "POP_BLOCK" and "SETUP_EXCEPT". In my implementation, I kept support for stack-based bytecode, and so I had some inefficient code and some corner cases. My approach was to convert stack-based bytecode to register-based bytecode on the fly. Having both in the same code allowed to me run some benchmarks. Maybe it wasn't the best approach, but I didn't feel able to write a real compiler (AST => bytecode). Victor Le mar. 26 févr. 2019 à 21:58, Neil Schemenauer <nas-python@python.ca> a écrit :

On 2019-02-26, Victor Stinner wrote:

...

I made an attempt once and it was faster: https://faster-cpython.readthedocs.io/registervm.html

Interesting. I don't think I have seen that before. Were you aware of "Rattlesnake" before you started on that? It seems your approach is similar. Probably not because I don't think it is easy to find. I uploaded a tarfile I had on my PC to my web site:

http://python.ca/nas/python/rattlesnake20010813/

It seems his name doesn't appear in the readme or source but I think Rattlesnake was Skip Montanaro's project. I suppose my idea of unifying the local variables and the registers could have came from Rattlesnake. Very little new in the world. ;-P

Cheers,

Neil

-- Night gathers, and now my watch begins. It shall not end until my death.

Jeroen Demeyer wrote:

Let me just say that the code for METH_FASTCALL function/method calls is optimized for a stack layout: a piece of the stack is used directly for calling METH_FASTCALL functions

We might be able to get some ideas for dealing with this kind of thing from register-window architectures such as the SPARC, where the registers containing the locals of a calling function become the input parameters to a called function. More generally, it's common to have a calling convention where the first N parameters are assumed to reside in a specific range of registers. If the compiler is smart enough, it can often arrange the evaluation of the parameter expressions so that the results end up in the right registers for making the call. -- Greg

METH_FASTCALL passing arguments on the stack doesn't necessarily mean it will be slow. In x86 there are calling conventions that read all the arguments from the stack, but the rest of the machine is register based. Python could also look at ABI calling conventions for inspiration, like x86-64 where some arguments up to a fixed amount are passed on the stack and the rest are passed on the stack. One thing that I am wondering is would Python want to use a global set of registers and a global data stack, or continue to have a new data stack (and now registers) per call stack. If Python switched to a global stack and global registers we may be able to eliminate a lot of instructions that just shuffle data from the caller's stack to the callee's stack. On Tue, Feb 26, 2019 at 4:55 PM Victor Stinner <vstinner@redhat.com> wrote:

Hum, I read again my old REGISTERVM.txt that I wrote a few years ago.

A little bit more context. In my "registervm" fork I also tried to implement further optimizations like moving invariants out of the loop. Some optimizations could change the Python semantics, like remove "duplicated" LOAD_GLOBAL whereas the global might be modified in the middle. I wanted to experiment such optimizations. Maybe it was a bad idea to convert stack-based bytecode to register-based bytecode and experiment these optimizations at the same time.

Victor

Le mar. 26 févr. 2019 à 22:42, Victor Stinner <vstinner@redhat.com> a écrit :

...

No, I wasn't aware of this project. My starting point was:

http://static.usenix.org/events/vee05/full_papers/p153-yunhe.pdf Yunhe Shi, David Gregg, Andrew Beatty, M. Anton Ertl, 2005

See also my email to python-dev that I sent in 2012: https://mail.python.org/pipermail/python-dev/2012-November/122777.html

Ah, my main issue was my implementation is that I started without taking care of clearing registers when the stack-based bytecode implicitly cleared a reference (decref), like "POP_TOP" operation.

I added "CLEAR_REG" late in the development and it caused me troubles, and the "correct" register-based bytecode was less efficient than bytecode without CLEAR_REG. But my optimizer was very limited, too limited.

Another implementation issue that I had was to understand some "implicit usage" of the stack like try/except which do black magic, whereas I wanted to make everything explicit for registers. I'm talking about things like "POP_BLOCK" and "SETUP_EXCEPT". In my implementation, I kept support for stack-based bytecode, and so I had some inefficient code and some corner cases.

My approach was to convert stack-based bytecode to register-based bytecode on the fly. Having both in the same code allowed to me run some benchmarks. Maybe it wasn't the best approach, but I didn't feel able to write a real compiler (AST => bytecode).

Victor

Le mar. 26 févr. 2019 à 21:58, Neil Schemenauer <nas-python@python.ca>

a écrit :

...

...

On 2019-02-26, Victor Stinner wrote:

...

I made an attempt once and it was faster: https://faster-cpython.readthedocs.io/registervm.html

Interesting. I don't think I have seen that before. Were you aware of "Rattlesnake" before you started on that? It seems your approach is similar. Probably not because I don't think it is easy to find. I uploaded a tarfile I had on my PC to my web site:

http://python.ca/nas/python/rattlesnake20010813/

It seems his name doesn't appear in the readme or source but I think Rattlesnake was Skip Montanaro's project. I suppose my idea of unifying the local variables and the registers could have came from Rattlesnake. Very little new in the world. ;-P

Cheers,

Neil

-- Night gathers, and now my watch begins. It shall not end until my death.

-- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com

On 2019-02-27, Greg Ewing wrote:

Joe Jevnik via Python-Dev wrote:

...

If Python switched to a global stack and global registers we may be able to eliminate a lot of instructions that just shuffle data from the caller's stack to the callee's stack.

That would make implementing generators more complicated.

Right. I wonder though, could we avoid allocating the Python frame object until we actually need it? Two situations when you need a heap allocated frame come to mind immediately: generators that are suspended and frames as part of a traceback. I guess sys._getframe() is another. Any more? I'm thinking that perhaps for regular Python functions and regular calls, you could defer creating the full PyFrame object and put the locals, stack, etc on the C stack. That would make calling Python functions a lot similar to the machine calling convention and presumably could be much faster. If you do need the frame object, copy over the data from the C stack into the frame structure. I'm sure there are all kinds of reasons why this idea is not easy to implement or not possible. It seems somewhat possible though. I wonder how IronPython works in this respect? Apparently it doesn't support sys._getframe(). Regards, Neil

Le mar. 26 févr. 2019 à 21:58, Neil Schemenauer <nas-python@python.ca> a écrit :

It seems his name doesn't appear in the readme or source but I think Rattlesnake was Skip Montanaro's project. I suppose my idea of unifying the local variables and the registers could have came from Rattlesnake. Very little new in the world. ;-P

In my implementation, constants, local variables and registers live all in the same array: frame.f_localsplus. Technically, there isn't much difference between a constant, local variable or a register. It's just the disassembler which has to worry to display "R3" or "x" depending on the register index ;-) There was a LOAD_CONST_REG instruction in my implementation, but it was more to keep a smooth transition from existing LOAD_CONST instruction. LOAD_CONST_REG could be avoided to pass directly the constant (ex: as a function argument). For example, I compiled "range(2, n)" as: LOAD_CONST_REG R0, 2 (const#2) LOAD_GLOBAL_REG R1, 'range' (name#0) CALL_FUNCTION_REG 4, R1, R1, R0, 'n' Whereas it could be just: LOAD_GLOBAL_REG R1, 'range' (name#0) CALL_FUNCTION_REG 4, R1, R1, <const #2>, 'n' Compare it to stack-based bytecode: LOAD_GLOBAL 0 (range) LOAD_CONST 2 (const#2) LOAD_FAST 'n' CALL_FUNCTION 2 (2 positional, 0 keyword pair) Victor -- Night gathers, and now my watch begins. It shall not end until my death.

Le mar. 26 févr. 2019 à 23:40, Greg Ewing <greg.ewing@canterbury.ac.nz> a écrit :

Victor Stinner wrote:

...

LOAD_CONST_REG R0, 2 (const#2) LOAD_GLOBAL_REG R1, 'range' (name#0) CALL_FUNCTION_REG 4, R1, R1, R0, 'n'

Out of curiosity, why is the function being passed twice here?

Ah, I should have explained that :-) The first argument of CALL_FUNCTION_REG is the name of the register used to store the result. The compiler begins with using static single assignment form (SSA) but then uses a register allocator to reduce the number of used registers. Usually, at the end you have less than 5 registers for a whole function. Since R1 was only used to store the function before the call and isn't used after, the R1 register can be re-used. Using a different register may require an explicit "CLEAR_REG R1" (decref the reference to the builtin range function) which is less efficient. Note: The CALL_FUNCTION instruction using the stack implicitly put the result into the stack (and "pop" function arguments from the stack). Victor -- Night gathers, and now my watch begins. It shall not end until my death.

On 2019-02-27, Victor Stinner wrote:

The compiler begins with using static single assignment form (SSA) but then uses a register allocator to reduce the number of used registers. Usually, at the end you have less than 5 registers for a whole function.

In case anyone is interested on working on this, I dug up some discussion from years ago. Advice from Tim Peters: [Python-Dev] Rattlesnake progress https://mail.python.org/pipermail/python-dev/2002-February/020172.html https://mail.python.org/pipermail/python-dev/2002-February/020182.html https://mail.python.org/pipermail/python-dev/2002-February/020146.html Doing a prototype register-based compiler in Python seems like a good idea. Using the 'compiler' package would give you a good start. I think this is the most recent version of that package: https://github.com/pfalcon/python-compiler Based on a little poking around, I think it has not been updated for the 16-bit word code. Shouldn't be too hard to make it work though. I was thinking about the code format on the weekend. Using three-register opcodes seems a good idea. We could could retain the 16-bit word code format. For opcodes that use three registers, use a second word for the last two registers. I.e. <8 bit opcode><8 bit register #> <8 bit register #><8 bit register #> Limit the number of registers to 256. If you run out, just push and pop from stack. You want to keep the instruction decode path in the evaluation loop simple and not confuse the CPU branch predictor. Regards, Neil

On Mon, 11 Mar 2019 18:03:26 -0400 David Mertz <mertz@gnosis.cx> wrote:

Parrot got rather further along than rattlesnake as a register based VM. I don't think it every really beat CPython in speed though.

http://parrot.org/

But Parrot also had a "generic" design that was supposed to cater for all dynamic programming languages. AFAIU, it was heavily over-engineered and under-staffed, and the design documents were difficult to understand. Regards Antoine.

On Feb 25, 2019, at 8:23 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:

So it looks like commit ef4ac967 is not responsible for a performance regression.

I did narrow it down to that commit and I can consistently reproduce the timing differences. That said, I'm only observing the effect when building with the Mac default Clang (Apple LLVM version 10.0.0 (clang-1000.11.45.5). When building GCC 8.3.0, there is no change in performance. I conclude this is only an issue for Mac builds.

I ran the "performance" suite (https://github.com/python/performance), which has 57 different benchmarks.

Many of those benchmarks don't measure eval-loop performance. Instead, they exercise json, pickle, sqlite etc. So, I would expect no change in many of those because they weren't touched. Victor said he generally doesn't care about 5% regressions. That makes sense for odd corners of Python. The reason I was concerned about this one is that it hits the eval-loop and seems to effect every single op code. The regression applies somewhat broadly (increasing the cost of reading and writing local variables by about 20%). The effect is somewhat broad based. That said, it seems to be compiler specific and only affects the Mac builds, so maybe we can decide that we don't care. Raymond

On 2019-02-26, Raymond Hettinger wrote:

That said, I'm only observing the effect when building with the Mac default Clang (Apple LLVM version 10.0.0 (clang-1000.11.45.5). When building GCC 8.3.0, there is no change in performance.

My guess is that the code in _PyEval_EvalFrameDefault() got changed enough that Clang started emitting a bit different machine code. If the conditional jumps are a bit different, I understand that could have a significant difference on performance. Are you compiling with --enable-optimizations (i.e. PGO)? In my experience, that is needed to get meaningful results. Victor also mentions that on his "how-to-get-stable-benchmarks" page. Building with PGO is really (really) slow so I supect you are not doing it when bisecting. You can speed it up greatly by using a simpler command for PROFILE_TASK in Makefile.pre.in. E.g. PROFILE_TASK=$(srcdir)/my_benchmark.py Now that you have narrowed it down to a single commit, it would be worth doing the comparison with PGO builds (assuming Clang supports that).

That said, it seems to be compiler specific and only affects the Mac builds, so maybe we can decide that we don't care.

I think the key question is if the ceval loop got a bit slower due to logic changes or if Clang just happened to generate a bit worse code due to source code details. A PGO build could help answer that. I suppose trying to compare machine code is going to produce too large of a diff. Could you try hoisting the eval_breaker expression, as suggested by Antoine: https://discuss.python.org/t/profiling-cpython-with-perf/940/2 If you think a slowdown affects most opcodes, I think the DISPATCH change looks like the only cause. Maybe I missed something though. Also, maybe there would be some value in marking key branches as likely/unlikely if it helps Clang generate better machine code. Then, even if you compile without PGO (as many people do), you still get the better machine code. Regards, Neil

Le mer. 27 févr. 2019 à 00:17, Victor Stinner <vstinner@redhat.com> a écrit :

My sad story with code placement: https://vstinner.github.io/analysis-python-performance-issue.html

tl; dr Use PGO.

Hum wait, this article isn't complete. You have to see the follow-up: https://bugs.python.org/issue28618#msg286662 """ Victor: "FYI I wrote an article about this issue: https://haypo.github.io/analysis-python-performance-issue.html Sadly, it seems like I was just lucky when adding __attribute__((hot)) fixed the issue, because call_method is slow again!" I upgraded speed-python server (running benchmarks) to Ubuntu 16.04 LTS to support PGO compilation. I removed all old benchmark results and ran again benchmarks with LTO+PGO. It seems like benchmark results are much better now. I'm not sure anymore that _Py_HOT_FUNCTION is really useful to get stable benchmarks, but it may help code placement a little bit. I don't think that it hurts, so I suggest to keep it. Since benchmarks were still unstable with _Py_HOT_FUNCTION, I'm not interested to continue to tag more functions with _Py_HOT_FUNCTION. I will now focus on LTO+PGO for stable benchmarks, and ignore small performance difference when PGO is not used. I close this issue now. """ Now I recall that I tried hard to avoid PGO: the server used by speed.python.org to run benchmarks didn't support PGO. I fixed the issue by upgrading Ubuntu :-) Now speed.python.org uses PGO. I stopped to stop to manually help the compiler with code placement. Victor

Raymond Hettinger writes:

We're trying to keep performant the ones that people actually use. For the Mac, I think there are only four that matter:

1) The one we distribute on the python.org website at https://www.python.org/ftp/python/3.8.0/python-3.8.0a2-macosx10.9.pkg

2) The one installed by homebrew

3) The way folks typically roll their own: $ ./configure && make (or some variant of make install)

4) The one shipped by Apple and put in /usr/bin

I don't see the relevance of (4) since we're talking about the bleeding edge AFAICT. Not clear about Homebrew -- since I've been experimenting with it recently I use the bottled versions, which aren't bleeding edge. If prebuilt packages matter, I would add MacPorts (or substitute it for (4) since nothing seems to get Apple's attention) and Anaconda (which is what I recommend to my students). But I haven't looked at MacPorts' recent download stats, and maybe I'm just the odd one out. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN