Thanks for the help.
What I am actually doing is computing a gradient to a least squares objective. That is,
X.T.dot(X.dot(beta) - Y)
If X is such that X.dot(beta) is fast (i.e. matvec is fast) then am I missing a "simple" optimization here at the cost of a copy? Alternatively, if X is such that vecmat is fast, then what is the best way to do this? A copy seems easiest, and possibly applying the previous "simple" optimization.
Based on my understanding of the other replies, I would guess that if X2=X.copy(), then the fastest way would be
(X2.dot(beta) - Y).dot(X)
This doesn't pan out in my example, the winner is
X2.T.dot(X2.dot(beta) - Y)
which is about the same as
(X2.dot(beta)-Y).dot(X2)
One difference I see between a numpy array with the same strides and the array loaded from a MAT file is the ALIGNED flag.