<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 2, 2017 at 12:23 PM, Ryan May <span dir="ltr"><<a href="mailto:rmay31@gmail.com" target="_blank">rmay31@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span class="">On Thu, Nov 2, 2017 at 6:46 AM, <span dir="ltr"><<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>></span> wrote:<br></span><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="m_7149487869349513819gmail-">On Wed, Nov 1, 2017 at 6:55 PM, Nathan Goldbaum <span dir="ltr"><<a href="mailto:nathan12343@gmail.com" target="_blank">nathan12343@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I think the biggest issues could be resolved if __array_concatenate__ were finished. Unfortunately I don't feel like I can take that on right now.<br><br></div>See Ryan May's talk at scipy about using an ndarray subclass for units and the issues he's run into:<br><br><a href="https://www.youtube.com/watch?v=qCo9bkT9sow" target="_blank">https://www.youtube.com/watch?<wbr>v=qCo9bkT9sow</a></div></blockquote><div><br></div><div><br></div></span><div>Interesting talk, but I don't see how general library code should know what units the output has.</div><div>for example if units are some flows per unit of time and we average, sum or integrate over time, then what are the new units? (e.g. pandas time aggregation)</div><div></div></div></div></div></blockquote><div><br></div></span><div>A general library doesn't have to do anything--just not do annoying things like isinstance() checks and calling np.asarray() everywhere. Honestly one of those is the core of most of the problems I run into. It's currently more complicated when doing things in compiled land, but that's implementation details, not any kind of fundamental incompatibility.</div><div><br></div><div>For basic mathematical operations, units have perfectly well defined semantics that many of us encountered in an introductory physics or chemistry class:</div><div>- Want to add or subtract two things? They need to have the same units; a units library can handle conversion provided they have the same dimensionality (e.g. length, time)</div><div>- Multiplication/Divison: combine and cancel units ( m/s * s -> m)</div><div><br></div><div>Everything else we do on a computer with data in some way boils down to: add, subtract, multiply, divide.</div><div><br></div><div>Average keeps the same units -- it's just a sum and division by a unit-less constant</div><div>Integration (in 1-D) involves *two* variables, your data as well as the time/space coordinates (or dx or dt); fundamentally it's a multiplication by dx and a summation. The units results then are e.g. data.units * dx.units. This works just like it does in Physics 101 where you integrate velocity (i.e. m/s) over time (e.g. s) and get displacement (e.g. m)</div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>What are units of covariance or correlation between two variables with the same units, and what are they between variables with different units?</div></div></div></div></blockquote><div><br></div></span><div>Well, covariance is subtracting the mean from each variable and multiplying the residuals; therefore the units for cov(x, y):</div><div><br></div><div>(x.units - x.units) * (y.units - y.units) -> x.units * y.units</div><div><br></div><div>Correlation takes covariance and divides by the product of the standard deviations, so that's:<br></div><div><br></div><div>(x.units * y.units) / (x.units * y.units) -> dimensionless</div><div><br></div><div>Which is what I'd expect for a correlation.</div><span class=""><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>How do you concatenate and operate arrays with different units?</div></div></div></div></blockquote><div><br></div></span><div>If all arrays have compatible dimensionality (say meters, inches, miles), you convert to one (say the first) and concatenate like normal. If they're not compatible, you error out.</div><span class=""><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>interpolation or prediction would work with using the existing units.</div></div></div></div></blockquote><div><br></div></span><div>I'm sure you wrote that thinking units didn't play a role, but the math behind those operations works perfectly fine with units, with things cancelling out properly to give the same units out as in.</div></div></div></div></blockquote><div><br></div><div>Some of it is in my reply to Marten.</div><div><br></div><div>regression and polyfit requires an X matrix with different units and then some linear algebra like solve, pinv or svd.</div><div><br></div><div>So, while the predicted values have well defined units, the computation involves some messier operations, unless you want to forgo linear algebra in all intermediate step and reduce it to sum, division and inverse.</div><div><br></div><div>Josef</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="HOEnZb"><font color="#888888"><div><br></div><div>Ryan</div><div><br></div></font></span></div><span class="HOEnZb"><font color="#888888">-- <br><div class="m_7149487869349513819gmail_signature"><div dir="ltr"><div>Ryan May<br><br></div></div></div>
</font></span></div></div>
<br>______________________________<wbr>_________________<br>
NumPy-Discussion mailing list<br>
<a href="mailto:NumPy-Discussion@python.org">NumPy-Discussion@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/numpy-discussion" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/numpy-<wbr>discussion</a><br>
<br></blockquote></div><br></div></div>