[Numpy-discussion] low level optimization in NumPy and minivect
markflorisson88 at gmail.com
Thu Jun 27 07:36:46 EDT 2013
On 27 June 2013 01:48, Frédéric Bastien <nouiz at nouiz.org> wrote:
> On Wed, Jun 26, 2013 at 7:30 AM, mark florisson <markflorisson88 at gmail.com>
>> On 26 June 2013 09:05, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no>
>> > On 06/25/2013 04:21 PM, Frédéric Bastien wrote:
>> >> Hi,
>> >> I wasn't able to attend this year Scipy Conference. My tutorial
>> >> proposal
>> >> was rejected and other deadline intefered with this conference date.
>> >> Will the presentation be recorded? If not, can you make the slide
>> >> available?
>> >> What is your opinion on this question:
>> >> - Should other lib like NumPy/Theano/Cython/Numba base their elemwise
>> >> implemention (or part of it) on dynd or minivect? I know cython and
>> >> Numba do it, but it was before dynd and I don't know where dynd fit in
>> >> the big picture. Do dynd reuse minivect itself?
>> > Actually, I think the Cython branch with minivect support was in the end
>> > not
>> > merged, due to lack of interest/manpower to maintain support for
>> > vectorization in the long term (so it was better to not add the feature
>> > than
>> > have a badly supported feature).
>> > My understanding is that Numba is based on minivect and not on dynd, so
>> > it's
>> > more of a competitor.
>> > Perhaps Mark Florisson will be able to comment.
>> > Dag Sverre
>> Hey Dag,
>> Indeed, numba uses it for its array expression support, but it will
>> likely remove the minivect dependency and generate a simple loop nest
>> for now. I'm working on pykit now
>> (https://github.com/ContinuumIO/pykit) which similarly to minivect
>> defines its own intermediate representation, with array expressions in
>> the form of map/reduce/scan/etc functions. The project has a broader
>> scope than minivect, to be used by projects like numba, what but a
>> "minivect baked in".
>> As such, minivect isn't really maintained any longer, and I wouldn't
>> recommend anyone using the code at this point (just maybe some of the
>> ideas :)).
> thanks for the information. I checked the repo rapidly and didn't found
> information on how to use it the way I expect to use it. I would like to be
> able to take a small Theano graph (like just elemwise operation) and make a
> graph in it to have it generate the c code. Do you have some tests/tests/doc
> that demonstrate something in that direction?
> Ideally I would like to be able to implement something like this simple
> (x ** 2).sum(1) or (x ** 2).sum()
> Is pykit or Numba IR ready for that?
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
It's in no way ready for public use, it doesn't really actually do
anything yet :) Numba doesn't really optimize reductions yet, so I
don't think it addresses any of your needs - but the input would be a
Python function (compiled from generated source code, or an AST).
I don't know how much further pykit would go beyond simple fusion and
perhaps tiling - I imagine it will defer to libraries like dynd to
perform actual work. This is offtopic for numpy itself, but it may be
useful to Theano in the future, I'll be sure to keep you in the loop
and bounce ideas of for feedback and collaboration.
More information about the NumPy-Discussion