great news with regards to OpenCV support...
At least I think it is great! I've managed (with the help of the community) to get ctypes function pointers to play nice with cython. I've also re-implented the IplImage struct in cython and figured out how to populate the struct with numpy stride information and the numpy data pointer. This all means that we now have the ability to to call OpenCV functions with plain numpy arrays as arguments, and the array gets modified in place!. And since i'm using ctypes to grab the function handle, OpenCV is not required to be on the users machine during build time, but if and when they install the opencv lib, they automagically work! I've a got a working (albeit ugly) example attached. You'll obviously need opencv installed if you want to test it. Just use PIL to open any image on your system then dump it into a numpy array then call testnumpy.test(arr) on that array. Dump the array back into a PIL image and call show() on it. Voila! your image has been guassianed blurred courtesy of OpenCV. So, now that I have the logistics of this figured out and I know it actually works, looks like I'll have plenty to implement during the upcoming sprint. Cheers! Chris
I just realized a fantastic byproduct of this: In OpenCV there is a concept of Region-of-Interest, where, if defined, the function will only operate on that section of the image. But because of numpy handles views, there is no need to provide support for that, you just pass in the view to the function as if it were a normal image. Cheers! Chris On Thu, Oct 8, 2009 at 9:37 PM, Chris Colbert <sccolbert@gmail.com> wrote:
At least I think it is great!
I've managed (with the help of the community) to get ctypes function pointers to play nice with cython.
I've also re-implented the IplImage struct in cython and figured out how to populate the struct with numpy stride information and the numpy data pointer.
This all means that we now have the ability to to call OpenCV functions with plain numpy arrays as arguments, and the array gets modified in place!. And since i'm using ctypes to grab the function handle, OpenCV is not required to be on the users machine during build time, but if and when they install the opencv lib, they automagically work!
I've a got a working (albeit ugly) example attached. You'll obviously need opencv installed if you want to test it.
Just use PIL to open any image on your system then dump it into a numpy array then call testnumpy.test(arr) on that array.
Dump the array back into a PIL image and call show() on it. Voila! your image has been guassianed blurred courtesy of OpenCV.
So, now that I have the logistics of this figured out and I know it actually works, looks like I'll have plenty to implement during the upcoming sprint.
Cheers!
Chris
Fantastic! This is great news. On Oct 8, 2009, at 3:37 PM, Chris Colbert wrote:
At least I think it is great!
I've managed (with the help of the community) to get ctypes function pointers to play nice with cython.
I've also re-implented the IplImage struct in cython and figured out how to populate the struct with numpy stride information and the numpy data pointer.
This all means that we now have the ability to to call OpenCV functions with plain numpy arrays as arguments, and the array gets modified in place!. And since i'm using ctypes to grab the function handle, OpenCV is not required to be on the users machine during build time, but if and when they install the opencv lib, they automagically work!
I've a got a working (albeit ugly) example attached. You'll obviously need opencv installed if you want to test it.
Just use PIL to open any image on your system then dump it into a numpy array then call testnumpy.test(arr) on that array.
Dump the array back into a PIL image and call show() on it. Voila! your image has been guassianed blurred courtesy of OpenCV.
So, now that I have the logistics of this figured out and I know it actually works, looks like I'll have plenty to implement during the upcoming sprint.
Cheers!
Chris <testnumpy.pyx>
So my next question is: how much hand holding should I do on my end for the user? There are several things I would like to address here: - opencv makes extensive use of *out arguments, should we require the user to preallocate their out array or should we make it for them and return it. The latter option is more pythonic, but comes with a small overhead for determining a proper return dtype - how much checking should I do on the input array. OpenCV images can accept 6 different dtypes. If the user passes an incorrect dtype, should I raise an exception or just let it fail with a KeyError during the dtype lookup? How much checking should I do on the array dimensions. We can use 2D or 3D arrays with the third dimension equal to 2, 3, or 4. Should I check that the passed arrays conform to this, or just let everything fail? Again, how much validation overhead should we allow On the technical side, I'm wondering if I should INCREF the numpy array when I pass it to OpenCV. If Python somehow gc'ed the array while OpenCV is working on it, that could be nasty. The only way I see this happening is if I start releasing the GIL on opencv calls. This brings the advantage of performance during threading but will not at all be thread safe since I'm "borrowing" the numpy data pointer. Cheers! Chris On Thu, Oct 8, 2009 at 10:59 PM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
Fantastic! This is great news.
On Oct 8, 2009, at 3:37 PM, Chris Colbert wrote:
At least I think it is great!
I've managed (with the help of the community) to get ctypes function pointers to play nice with cython.
I've also re-implented the IplImage struct in cython and figured out how to populate the struct with numpy stride information and the numpy data pointer.
This all means that we now have the ability to to call OpenCV functions with plain numpy arrays as arguments, and the array gets modified in place!. And since i'm using ctypes to grab the function handle, OpenCV is not required to be on the users machine during build time, but if and when they install the opencv lib, they automagically work!
I've a got a working (albeit ugly) example attached. You'll obviously need opencv installed if you want to test it.
Just use PIL to open any image on your system then dump it into a numpy array then call testnumpy.test(arr) on that array.
Dump the array back into a PIL image and call show() on it. Voila! your image has been guassianed blurred courtesy of OpenCV.
So, now that I have the logistics of this figured out and I know it actually works, looks like I'll have plenty to implement during the upcoming sprint.
Cheers!
Chris <testnumpy.pyx>
On Thu, Oct 8, 2009 at 2:18 PM, Chris Colbert <sccolbert@gmail.com> wrote:
So my next question is: how much hand holding should I do on my end for the user?
There are several things I would like to address here:
- opencv makes extensive use of *out arguments, should we require the user to preallocate their out array or should we make it for them and return it. The latter option is more pythonic, but comes with a small overhead for determining a proper return dtype
I think having out be None by default is best.
- how much checking should I do on the input array. OpenCV images can accept 6 different dtypes. If the user passes an incorrect dtype, should I raise an exception or just let it fail with a KeyError during the dtype lookup?
An exception with a meaningful error message like "type unsupported" is more useful to the user than a KeyError in an undocumented, local variable data structure.
How much checking should I do on the array dimensions. We can use 2D or 3D arrays with the third dimension equal to 2, 3, or 4. Should I check that the passed arrays conform to this, or just let everything fail? Again, how much validation overhead should we allow
It's not clear to me how the best way to write to a 4D array. It's probably best to throw an exception unless you can come up with a clear use case for high dimensional arrays.
On the technical side, I'm wondering if I should INCREF the numpy array when I pass it to OpenCV. If Python somehow gc'ed the array while OpenCV is working on it, that could be nasty. The only way I see this happening is if I start releasing the GIL on opencv calls. This brings the advantage of performance during threading but will not at all be thread safe since I'm "borrowing" the numpy data pointer.
Will users call the OpenCV functions directly or do you have Python wrappers? If your Python wrapper function keeps a reference to the array throughout execution in C-space, you can probably avoid the INCREF. Damian
On Thu, Oct 8, 2009 at 10:59 PM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
Fantastic! This is great news.
On Oct 8, 2009, at 3:37 PM, Chris Colbert wrote:
At least I think it is great!
I've managed (with the help of the community) to get ctypes function pointers to play nice with cython.
I've also re-implented the IplImage struct in cython and figured out how to populate the struct with numpy stride information and the numpy data pointer.
This all means that we now have the ability to to call OpenCV functions with plain numpy arrays as arguments, and the array gets modified in place!. And since i'm using ctypes to grab the function handle, OpenCV is not required to be on the users machine during build time, but if and when they install the opencv lib, they automagically work!
I've a got a working (albeit ugly) example attached. You'll obviously need opencv installed if you want to test it.
Just use PIL to open any image on your system then dump it into a numpy array then call testnumpy.test(arr) on that array.
Dump the array back into a PIL image and call show() on it. Voila! your image has been guassianed blurred courtesy of OpenCV.
So, now that I have the logistics of this figured out and I know it actually works, looks like I'll have plenty to implement during the upcoming sprint.
Cheers!
Chris <testnumpy.pyx>
-- ----------------------------------------------------- Damian Eads Ph.D. Candidate University of California Computer Science 1156 High Street Machine Learning Lab, E2-489 Santa Cruz, CA 95064 http://www.soe.ucsc.edu/~eads
On Thu, Oct 8, 2009 at 11:38 PM, Damian Eads <eads@soe.ucsc.edu> wrote:
On Thu, Oct 8, 2009 at 2:18 PM, Chris Colbert <sccolbert@gmail.com> wrote:
So my next question is: how much hand holding should I do on my end for the user?
There are several things I would like to address here:
- opencv makes extensive use of *out arguments, should we require the user to preallocate their out array or should we make it for them and return it. The latter option is more pythonic, but comes with a small overhead for determining a proper return dtype
I think having out be None by default is best.
That's how i've been going about it so far, but that obviously incurs an overhead of determining, exactly if and what to return.
- how much checking should I do on the input array. OpenCV images can accept 6 different dtypes. If the user passes an incorrect dtype, should I raise an exception or just let it fail with a KeyError during the dtype lookup?
An exception with a meaningful error message like "type unsupported" is more useful to the user than a KeyError in an undocumented, local variable data structure.
this is leading me to think it would be easiest to just have a general validator that ensures each array conforms to a common set of requirements
How much checking should I do on the array dimensions. We can use 2D or 3D arrays with the third dimension equal to 2, 3, or 4. Should I check that the passed arrays conform to this, or just let everything fail? Again, how much validation overhead should we allow
It's not clear to me how the best way to write to a 4D array. It's probably best to throw an exception unless you can come up with a clear use case for high dimensional arrays.
OpenCV cant handle 4D arrays. But it can handle 3D arrays with 4 channels (i.e. RGBA) These errors can be caught in a validator.
On the technical side, I'm wondering if I should INCREF the numpy array when I pass it to OpenCV. If Python somehow gc'ed the array while OpenCV is working on it, that could be nasty. The only way I see this happening is if I start releasing the GIL on opencv calls. This brings the advantage of performance during threading but will not at all be thread safe since I'm "borrowing" the numpy data pointer.
Will users call the OpenCV functions directly or do you have Python wrappers? If your Python wrapper function keeps a reference to the array throughout execution in C-space, you can probably avoid the INCREF.
the opencv calls are being made in a wrapper, and the function has a reference to the array until it returns.
Damian
On Thu, Oct 8, 2009 at 10:59 PM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
Fantastic! This is great news.
On Oct 8, 2009, at 3:37 PM, Chris Colbert wrote:
At least I think it is great!
I've managed (with the help of the community) to get ctypes function pointers to play nice with cython.
I've also re-implented the IplImage struct in cython and figured out how to populate the struct with numpy stride information and the numpy data pointer.
This all means that we now have the ability to to call OpenCV functions with plain numpy arrays as arguments, and the array gets modified in place!. And since i'm using ctypes to grab the function handle, OpenCV is not required to be on the users machine during build time, but if and when they install the opencv lib, they automagically work!
I've a got a working (albeit ugly) example attached. You'll obviously need opencv installed if you want to test it.
Just use PIL to open any image on your system then dump it into a numpy array then call testnumpy.test(arr) on that array.
Dump the array back into a PIL image and call show() on it. Voila! your image has been guassianed blurred courtesy of OpenCV.
So, now that I have the logistics of this figured out and I know it actually works, looks like I'll have plenty to implement during the upcoming sprint.
Cheers!
Chris <testnumpy.pyx>
-- ----------------------------------------------------- Damian Eads Ph.D. Candidate University of California Computer Science 1156 High Street Machine Learning Lab, E2-489 Santa Cruz, CA 95064 http://www.soe.ucsc.edu/~eads
On Thu, Oct 8, 2009 at 11:42 PM, Chris Colbert <sccolbert@gmail.com> wrote:
On Thu, Oct 8, 2009 at 11:38 PM, Damian Eads <eads@soe.ucsc.edu> wrote:
On Thu, Oct 8, 2009 at 2:18 PM, Chris Colbert <sccolbert@gmail.com>
wrote:
So my next question is: how much hand holding should I do on my end for the user?
There are several things I would like to address here:
- opencv makes extensive use of *out arguments, should we require the
user to
preallocate their out array or should we make it for them and return it. The latter option is more pythonic, but comes with a small overhead for determining a proper return dtype
I think having out be None by default is best.
That's how i've been going about it so far, but that obviously incurs an overhead of determining, exactly if and what to return.
Doing it the same as for ufuncs would make sense I think. Is that what you mean by "having out None"? The overhead should be small compared to most operations on images. Ufuncs return a view on `out` if given by the user. scipy.ndimage returns None in that case. I think returning a view is more convenient. Cheers, Ralf
On Fri, Oct 9, 2009 at 12:45 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Oct 8, 2009 at 11:42 PM, Chris Colbert <sccolbert@gmail.com> wrote:
On Thu, Oct 8, 2009 at 11:38 PM, Damian Eads <eads@soe.ucsc.edu> wrote:
On Thu, Oct 8, 2009 at 2:18 PM, Chris Colbert <sccolbert@gmail.com> wrote:
So my next question is: how much hand holding should I do on my end for the user?
There are several things I would like to address here:
- opencv makes extensive use of *out arguments, should we require the user to preallocate their out array or should we make it for them and return it. The latter option is more pythonic, but comes with a small overhead for determining a proper return dtype
I think having out be None by default is best.
That's how i've been going about it so far, but that obviously incurs an overhead of determining, exactly if and what to return.
Doing it the same as for ufuncs would make sense I think. Is that what you mean by "having out None"? The overhead should be small compared to most operations on images.
Ufuncs return a view on `out` if given by the user. scipy.ndimage returns None in that case. I think returning a view is more convenient.
Cheers, Ralf
So far, I've been doing it like this: If the user passes an 'out' array, I verify that its compatible for the operation. If the user does not pass an out array, I create an appropriate one for them. The final case is operations that can performed in place by opencv. In these cases, the function takes an extra boolean kwarg 'in_place'. If true, the operation is performed in place. This is done by assiging 'src' to 'out' and passing both to opencv. In all these cases, I return the 'out' array. So if the user supplied it or requested inplace, they can just ignore the return value. Cheers! Chris
On Thu, Oct 8, 2009 at 2:42 PM, Chris Colbert <sccolbert@gmail.com> wrote:
On Thu, Oct 8, 2009 at 11:38 PM, Damian Eads <eads@soe.ucsc.edu> wrote:
On Thu, Oct 8, 2009 at 2:18 PM, Chris Colbert <sccolbert@gmail.com> wrote:
So my next question is: how much hand holding should I do on my end for the user?
There are several things I would like to address here:
- opencv makes extensive use of *out arguments, should we require the user to preallocate their out array or should we make it for them and return it. The latter option is more pythonic, but comes with a small overhead for determining a proper return dtype
I think having out be None by default is best.
That's how i've been going about it so far, but that obviously incurs an overhead of determining, exactly if and what to return.
True but I think the overhead of checking for a None is minimal compared to the overhead of performing most image processing operations.
- how much checking should I do on the input array. OpenCV images can accept 6 different dtypes. If the user passes an incorrect dtype, should I raise an exception or just let it fail with a KeyError during the dtype lookup?
An exception with a meaningful error message like "type unsupported" is more useful to the user than a KeyError in an undocumented, local variable data structure.
this is leading me to think it would be easiest to just have a general validator that ensures each array conforms to a common set of requirements
Yes, the cluster package has this to check for validity of data structures. The first part of each function could read something like: check_valid_image(I) # throws exception if something goes wrong if out is None: out = np.zeros(I.shape, dtype=I.dtype) else: check_image_compatibility(I, X) (call OpenCV via ctypes)
How much checking should I do on the array dimensions. We can use 2D or 3D arrays with the third dimension equal to 2, 3, or 4. Should I check that the passed arrays conform to this, or just let everything fail? Again, how much validation overhead should we allow
It's not clear to me how the best way to write to a 4D array. It's probably best to throw an exception unless you can come up with a clear use case for high dimensional arrays.
OpenCV cant handle 4D arrays. But it can handle 3D arrays with 4 channels (i.e. RGBA) These errors can be caught in a validator.
On the technical side, I'm wondering if I should INCREF the numpy array when I pass it to OpenCV. If Python somehow gc'ed the array while OpenCV is working on it, that could be nasty. The only way I see this happening is if I start releasing the GIL on opencv calls. This brings the advantage of performance during threading but will not at all be thread safe since I'm "borrowing" the numpy data pointer.
Will users call the OpenCV functions directly or do you have Python wrappers? If your Python wrapper function keeps a reference to the array throughout execution in C-space, you can probably avoid the INCREF.
the opencv calls are being made in a wrapper, and the function has a reference to the array until it returns.
Great, it's always nice to avoid reference counting. Damian
Hey all, To Chris, congratulations -- this is a very encouraging result! I really like the design. To the rest of you, thanks for answering all his questions so well. As I read the thread this morning, I formulated a bunch of answers but by the end of the thread all were given! Chris, could you make your development available as a branch on github? I'd like to have a look at the code, even if it is a roughly hacked version. Happy coding, Stéfan
I'd be more than happy to put what I have on GIT (as soon as I figure out how to do that :)) I've cleaned up quite a bit from what was posted earlier, and am working on the getting "validation" machinery in place. Chris 2009/10/9 Stéfan van der Walt <stefan@sun.ac.za>:
Hey all,
To Chris, congratulations -- this is a very encouraging result! I really like the design.
To the rest of you, thanks for answering all his questions so well. As I read the thread this morning, I formulated a bunch of answers but by the end of the thread all were given!
Chris, could you make your development available as a branch on github? I'd like to have a look at the code, even if it is a roughly hacked version.
Happy coding, Stéfan
participants (5)
-
Chris Colbert -
Damian Eads -
Ralf Gommers -
Stéfan van der Walt -
Zachary Pincus